So for example this newspaper page on Chronicling America:

Is also available on Flickr:

I haven’t written about it here yet, but Chronicling America is just a regular vanilla Django/MySQL/Apache webapp which exposes machine readable metadata for the newspaper content using the Linked Data pattern. It just so happens that Dave was able to use these linked data views to determine the metadata to use when uploading content to Flickr. For example if a curator wants to have this newspaper page uploaded to Flickr, Dave’s flickr uploading program is able to use the associated metadata (referenced in a link element inthe HTML) to get the newspaper title, issue, date, and other related metadata. The beauty of this was Dave was able to do the technical work on his own, and it didn’t require any formal project coordination.

A few weeks ago, on learning of the Flickr / Chronicling America integration Rob Sanderson asked if we could possibly reference the Flickr content in our own data views. Rob became interested in Chronicling America because of its use of the Open Archives Initiative Object Reuse and Exchange Vocabulary in the linked data views. Rob has written a rather nice oai-ore Greasemonkey visualization tool called foresite-explorer, which can visualize oai-ore links to Flickr. It also makes sense from a curatorial perspective to want to capture these bi-directional links between Chronicling America and Flickr in the Chronicling America database.

After agreeing with Rob I’ve had it on my plate to get the list of Flickr URLs and their associated Chronicling America URLs from Dave, for loading into Chronicling America so that the links can be served up in the data views, and perhaps maybe in the HTML. But yesterday morning I had the realization that I didn’t really need to ask (and keep asking every month) Dave for the list. Since Dave created a Flickr set for these pages, and has embedded the URI for the Chronicling America page as a machine tag I can get it right from Flickr. So I hacked together a quick script, and now I have the list too.

ed@rorty:~$ ./flickr_chronam.py
(u'http://www.flickr.com/photos/library_of_congress/3608399458', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-01-03/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3608400834', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-01-10/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3608402104', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-01-17/ed-1/seq-16/')
(u'http://www.flickr.com/photos/library_of_congress/3608403362', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-01-24/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3607588861', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-01-31/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3608405718', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-02-07/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3608407068', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-02-14/ed-1/seq-16/')
(u'http://www.flickr.com/photos/library_of_congress/3608408274', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-02-21/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3607593693', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-02-28/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3608410606', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-03-07/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3607596267', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-03-14/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3607597927', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-03-21/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3608414374', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-03-28/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3608415708', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-04-04/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3607601559', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-04-11/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3608418042', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-04-18/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3608419060', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-04-25/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3607604705', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-05-02/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3608421240', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-05-09/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3608422694', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-05-16/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3607608459', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-05-23/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3608425436', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-05-30/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3607611709', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-06-06/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3607637819', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-06-13/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3607638897', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-06-20/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3608455948', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-06-27/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3607641409', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-07-04/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3607642551', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-07-11/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3607889205', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-07-18/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3608709982', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-07-25/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3607894517', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-08-01/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3607896027', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-08-08/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3608713826', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-08-15/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3608715804', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-08-22/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3607900561', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-08-29/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3608718394', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-09-05/ed-1/seq-15/')
(u'http://www.flickr.com/photos/library_of_congress/3608719874', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-09-12/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3608721302', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-09-19/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3607906387', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-09-26/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3608724542', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-10-03/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3607909093', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-10-10/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3607910739', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-10-17/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3608728408', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-10-24/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3607913989', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-10-31/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3607915481', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-11-07/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3607916497', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-11-14/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3608734354', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-11-21/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3607919583', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-11-28/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3608737124', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-12-05/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3608738658', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-12-12/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3608740242', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-12-19/ed-1/seq-17/')
(u'http://www.flickr.com/photos/library_of_congress/3608741612', u'http://chroniclingamerica.loc.gov/lccn/sn83030214/1909-12-26/ed-1/seq-13/')

The point of this blog post is that Dave, Rob and I were able to independently work on tools for pushing the same repository content around without having to talk much at all. The World Wide Web, and URIs in particular, enables this sort of integration. I guess some people would argue that this is a Web2.0 Mashup, and I guess they would be right in a way. But really it is using the Web the way it was designed to be used–as a decentralized information space. Flickr can tell me about the Chronicling America content that’s on Flickr; and Chronicling America can tell other people about the repository objects themselves.

Now I just need to make those links available for Rob in the data views :-)

Some folks at LC and CDL are trying to kick-start a new public discussion list for talking about digital curation in its many guises: repositories, tools, standards, techniques, practices, etc. The intuition being that there is a social component to the problems of digital preservation and repository interoperability.

Of course NDIIPP (the arena for the CDL/LC collaboration) has always been about building and strengthening a network of partners. But as Priscilla Caplan points out in her survey of the digital preservation landscape Ten Years After, organizations in Europe like the JISC and NESTOR seem to have understood that there is an educational component to digital preservation as well. Yet even the JISC and NESTOR have tended to focus more on the preservation of scholarly output, whereas digital preservation really extends beyond that realm of materials.

The continual need to share good ideas and hard-won-knowledge about digital curation, and to build a network of colleagues and experts that extends out past the normal project/institution specific boundaries is just as important as building the collections and the technologies themselves.

So I guess this is a rather highfalutin goal … here’s some text stolen from the digital-curation home page to give you more of a flavor:

The digital preservation and repositories domain is fortunate to have a diverse set of institutional and consortial efforts, software projects, and standardization initiatives. Many discussion lists have been created for these individual efforts. The digital-curation discussion list is intended to be a public forum that encourages cross-pollination across these project and institutional boundaries in order to foster wider awareness of project- and institution-specific work and encourage further collaboration.

Topic of conversation can include (but is not limited to)

• digital repository software (Fedora, DSpace, EPrints, etc.)
• management of digital formats (JHOVE, djatoka, etc.)
• use and development of standards (OAIS, OAI-PMH/ORE, MPEG21, METS, BagIt, etc.)
• issues related to identifiers, packaging, and data transfer
• best practices and case studies around curation and preservation of digital content
• repository interoperability
• conference, workshop, tutorial announcements
• recent papers
• job announcements
• general chit chat about problems, solutions, itches to be scratched
• humor and fun

We’ll see how it goes. If you are at all interested please sign up.

I finally got to meet Ben O’Steen after bantering with him on #code4lib and #talis … and also got to chat with Jim Downing (Cambridge Univ) about SWORD stuff, and Stephan Drescher (Los Alamos National Lab) about validating OAI-ORE.

Stephan and I had a varied and wide ranging discussion about the web in general, which was a lot of fun. I really dug his metaphor of the web as an aquatic ecosystem, with interdependent organisms and shared environments. It reminded me a bit of how shocked I was to discover how rich and varied the ecosystem is around a “simple” service like twitter. If I ever return to school it will be to study something along the lines of web science.

It was also interesting to hear that other people saw a parallel between OAI-ORE Resource Maps and BagIt’s fetch.txt. The parallel being that both resource maps and bags are aggregations of web resources. Of course bags can also just be files on disk, it’s when the fetch.txt is present in the bag that the package is made up of web resources. It would be interesting to see what vocabularies are available for expressing fixity information (md5 checksums and the like), and if they could be layered into the resource map atom serialization. Perhaps PREMIS v2.0? It might be fun to code up what a simple OAI-ORE resource map harvester would look like, that checked fixity values — using LC’s existing BagIt parallelretriever.py as a starting point. God I wish I could just hyperlink to that :-(

At any rate, I now need to investigate OAuth because Jim thinks it fits really nicely with AtomPub and SWORD in particular. And if it’s good enough for Google it’s probably worth checking out. Jim also said that there is a possibility that the SWORD 2.0 might take shape as an IETF RFC, which would be good to see.

Thanks to all that made it happen, and for all of you that traveled long distances to join us at the Library of Congress.