inkdroid

At $work recently many late nights were spent hackety-hacking on a prototype that got written up in the New York Times today. Apart from some promotional materials, not much is available to the public just yet. I just got pulled in near the end to do some search stuff. Over the past few months I’ve seen dchud in top form managing complicated data/organizational workflows while making technical decisions. A nice outgrowth of working with smarties is ending up with a fun and productive technology stack: python, django, postgres, jquery, solr, tilecache, ubuntu, trac, subversion, vmware. Given the press and the commitment to UNESCO I think the code is going to start being a bit more than a prototype pretty soon :-)

If you haven’t tried Exhibit out yet the simile folks have created a truly wonderful data publishing framework which runs entirely in your browser with a bit of javascript, html and css.

The remarkable part is that it requires no backend database, but simply operates on a stream of json. If you have a couple minutes take a look at their Getting Started Tutorial which shows you how to create a exhibit of MIT related nobel laureates with a tiny bit of HTML, CSS and JavaScript.

Just as an experiment I tried pointing it at my delicious json feed for metadata. It turns out that exhibit wants json data to be a hash with a key ‘items’ that points to a list of items. In addition it also wants each item to have a ‘label’ key. I quickly reformatted the delicious json with simplejson, and got this.

A few minutes later I prodded the simile folks to see if there is a way of filtering json data on the way into exhibit so that it can be normalized…time passes (like maybe an hour) and then I hear from Johan Sundström that the latest/greatest exhibit code has this sort of filtering built in!

Tangential to the exhibit code, there has been an interesting discussion recently about how to expose exhibit content to indexing services like google. Since exhibit content is generated with pure javascript, and google (as far as we know) primarily indexes html content–the exhibit content is rendered invisible. This is a problem that digital library applications and repositories have to deal with as well, so it may be of interest.

So I’ve been dabbling with that four letter word at $work to create a hierarchical journal/volume/issue/article browser. Le rails and scriptaculous make it pretty easy indeed.

I figured I’d be a good developer and try to understand what’s actually going on behind the scenes, so I picked up a copy of Ajax in Action [Illustrated] and am working through it.

There is so much hype surrounding Ajax that I had pretty low expectations–but the book is actually very well written and a joy to read. I noticed before diving in that there was an appendix on object-oriented JavaScript. I’ve been around the block enough times to know that JavaScript is actually quite a nice functional language; but apart from DHTML I haven’t really had the opportunity to dabble in it much. This appendix really made it clear how JavaScript is really quite elegant, and for someone who has done object-oriented-programming in Perl the idioms for doing OOP in JavaScript didn’t seem quite that bad.

Anyhow, I quickly wanted to start fiddling around with the language with a JavaScript interpreter so I downloaded Rhino and discovered that you can:

frizz:~/Projects/rhino1_6R4 edsu$ java -jar js.jar
Rhino 1.6 release 4 2006 09 09
js> print("hello world");
hello world
js>

Pretty sweet :-)

Dan, Jeremy, Peter, Michael, Mike, Ross and I wrote an article in the latest Ariadne introducing the lightweight web protocol unAPI. Essentially unAPI is an easy way to include references to digital objects in your HTML which can then be predictably retrieved by a machine…yes ‘machine’ includes JavaScript running in a browser :-) Dan and a really nice cross section of developers around the world have been working on this spec for over a year now and I think it could be poised to play an important role in the emerging open data movement.

Imagine you have a citation database which is searchable via the web. The search results include hits. Wouldn’t it be nice to align your human viewable results with machine readable representations so that people could write browser hacks and the like to remix your application data?

As far as I can tell there are a few options available to help you do this (apart from doing something ad-hoc).

1. use a citation microformat and mark up your HTML predictably so that it can be recognized and parsed
2. use GRDDL to map your HTML to RDF via an XLST profile.
3. embed RDF in your HTML essentially using an RDF microformat.
4. OpenURL and/or COinS to link in page IDs to OpenURL servers.
5. use unAPI and include a unapi server url (familiar autodiscovery like RSS/Atom), and identifiers (simple element attributes) and write a simple server side script that emits xml for a given identifier.

I like microformats a lot and I think a citation format will eventually get done. But it’s been a long time coming and there’s no indication it’s going to get done any time soon. What’s more unAPI is bigger than just citation data–and it allows you to publish all kinds of rich data objects without waiting for a community to ratify a particular representation in HTML.

Options 2 and 3 use RDF which I actually like quite a bit as well. GRDDL implies a GRDDL aware browser which would be cool but is a bit heavy weight. XSLT will require clean XHTML–or pipelines to clean it. Embedding RDF in HTML using microformat techniques is compelling because you can theoretically process the RDF data similarly–whereas unAPI doesn’t require any particular kind of machine readable format (apart from HTML). Actually there’s nothing stopping you from using unAPI to link human viewable objects with RDF representations. The advantage unAPI has here is you can learn RDF if you want to, but you don’t have to learn RDF to get going with unAPI today.

Option 4 leverages work done in the library community on citation linking. OpenURL routers are widely deployed in libraries around the world and COinS is a quasi-microformat for putting OpenURL context objects into your HTML so that they can be extracted and fired off at an OpenURL server. OpenURL is a relatively complex and subtle standard which can do a lot more than just citation linking. Compared to OpenURL/COinS unAPI allows for ease of implementation in languages like JavaScript and provides a simple introspection mechanism for discovering what formats a particular resource is available in. AFAIK this can’t be done simply using OpenURL/COinS. If I’m wrong, comments should be open. I would argue that the sheer power and flexibility of OpenURL paradoxically make it hard to understand…and that unAPI in Dan’s adherence to a one-page-spec is more limited and simple. Less is more…

So if this piques your interest read the article. It does a much better job of describing the origins of the work, where it’s headed, has examples and links out to sites/tools that use unAPI today. I must admit I wrote very little of the article, and mostly contributed text snippets and screenshots of the unAPI validator I wrote, which uses my unapi ruby gem.

I just noticed over on the del.icio.us blog that my data is available as JavaScript Object Notation (JSON) by following a simple URL like: http://del.icio.us/feeds/json/inkdroid.

Essentially you just load the URL as javascript source in your HTML:


<script type="text/javascript" src="http://del.icio.us/feeds/json/inkdroid?count=20"></script>


and voila you’ve magically got a new javascript array variable Delicious.posts, each element of which is a hash describing your link on delicious. It’s a very elegant (and simple) technique…much more elegant than that taken in the XML::RSS::JavaScript module which I helped create. It’s so elegant in fact that I got it working off to the side of this page in 2 minutes. I downloaded the python and ruby extensions for working with JSON just to take a look. The python version is a pleasant read, especially the unittests! The ruby version is a lesson in minimalism:

jsonobj = eval(json.gsub(/(["'])\s*:\s*(['"0-9tfn\[{])/){"#{$1}=>#{$2}"})

Now, if I were to use this I’d probably put a wrapper around it :-) Although it’s less minimalistic I think I prefer the explicitness of the python code. I’ve been digging into Ruby a bit more lately as I work on ruby-marc, and while I’m really enjoying the language I tend to shy away from one line regex hacks like this…which more often than not turn out to be a pain to extend and maintain.

I first heard of JSON from Mike Rylander at the open-ils project who are using JSON heavily in the opensource library catalog that they are developing for the state of Georgia. It is nice to see library technologists leading the curve.

Adrian pointed out at the last chipy meeting that a formal API for GoogleMaps was in the works…but I had no idea it was this close.

After you’ve got an authentication key for your site directory, all you need to do to embed a map in your page is include a javascript library source URL directly from google, create a <div> tag with an id (say “map”) and add some javascript to your page.

    var map = new GMap( document.getElementById("map") );
    map.addControl( new GSmallMapControl() );
    map.centerAndZoom( new GPoint(-88.316385,42.247090), 4);

This took literally 2 minutes to do, if that. It’s a bit tedious that the token is only good on a per-directory basis, but I guess this is because of partitioned blogging sites where different users have different directories with the same hostname.

update: I guess I’m not the only one who finds the per-directory limit to be kind of a hassle.

RFC 2397 has been around since August 1998 and I’m just learning about the data URL scheme today. Perhaps browser support for it is new? Basically data URLs allow you to embed data, like images directly in an HTML page. Data URLs remind me of Fred Lindberg’s old idea (circa 2001) of “mailing the web” by freezing web pages as email with MIME attachments.

It’s fun to be learning new things about HTML/HTTP: technologies that I thought I was familiar with already. Perhaps I’ve been out of web development for long enough to fall behind. The other day I learned about iframes from my friend and sometime coworker Jason and was similarly blown away by something new under the sun. iframes are esentially the same things as regular frames but for the browser user they don’t see separate panes. Useful for scrolling panels inside of pages and other things I’m sure.

I guess this is all part of the web renaissance that is going on now, spurred on by Google’s forays and investment in javascript and xml. It’s really interesting to see how a big player like Google can redefine what is acceptable technology to rely on in web applications. For years I’ve avoided doing too much in javascript since it was a headache to get it working across different browsers, at least for this programmer. Now javascript is on my list of things to learn more about.