Category Archives: xml

bad xml smells

Leave a reply

I’m used to refactoring code smells, but sometimes you can catch a bad whiff in XML too.

Before:

< ?xml version="1.0" encoding="UTF-8"?>
<mets TYPE="urn:library-of-congress:ndnp:mets:encyclopedia:encyclopediaEntry" PROFILE="urn:library-of-congress:mets:profiles:ndnp:encyclopediaEntry:v1.1" LABEL="The National Forum Scope Note" 
     xmlns:mods="http://www.loc.gov/mods/v3" xmlns="http://www.loc.gov/METS/" xmlns:dsig="http://www.w3.org/2000/09/xmldsig#" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
 
     <!--METS HEADER-->
     <metshdr CREATEDATE="2007-01-10T09:00:00" ><!--CREATEDATE should be populated with creation date of the record.  RECORDSTATUS should only be set by Validation Application.-->
        <agent ROLE="CREATOR" TYPE="ORGANIZATION">
            <name><!--Awardee, if the awardee created the METS record.  If LC created the METS record, should be "Library of Congress"-->Library of Congress</name>
        </agent>
    </metshdr>
 
    <!--Descriptive metadata for encyclopedia entry-->
    <dmdsec ID="encyclopediaMods">                
        <mdwrap MDTYPE="MODS" LABEL="Encyclopedia entry metadata">
            <xmldata>
                <mods:mods>
                    <mods:name type="corporate">
                            <mods:displayform><!--Awardee-->Library of Congress</mods:displayform>
                    </mods:name>
                    <mods:relateditem>
                        <mods:identifier type="lccn"><!--LCCN-->sn82015056</mods:identifier>
                        <!-- mods:identifier elements are repeatable -->
                    </mods:relateditem>
                </mods:mods>
            </xmldata>
        </mdwrap>
    </dmdsec>
 
 
 
    <!--FILE SECTION-->
    <filesec>
        <filegrp>
            <file ID="encyclopediaEntryText">
                <fcontent>
                    <xmldata>
                        <xhtml:html xmlns:xhtml="http://www.w3.org/1999/xhtml"  lang="en">
                           <xhtml:head>
                              <xhtml:title><!--Newspaper title-->The National Forum (Washington, DC), 1910-19??</xhtml:title>
                           </xhtml:head>
 
                           <xhtml:body>
                              <xhtml:p>
The first issue of the <xhtml:cite>National Forum</xhtml:cite> was likely released on April 30, 1910 
and the newspaper ran through at least November 12 of that year. The four-page African-American 
weekly covered such local events as Howard University graduations and Baptist church activities, but its 
pages also included national news, sports, home maintenance, women's news, science, editorial 
cartoons, and reprinted stories from national newspapers. Its primary focus was on how the news 
affected the city's black community. A unique feature was its coverage of Elks Club meetings and 
activities.  Business manager John H. Wills contributed the community-centered "Vanity Fair" column that
 usually appeared on the front page of each issue. The publisher and editor was Ralph W. White, who 
went on to publish another African-American newspaper, the <xhtml:a href="info:lccn/sn86092050">
<xhtml:cite>McDowell Times</xhtml:cite></xhtml:a> of Keystone, West Virginia. Originally located at 
609 F St., NW, the newspaper's offices moved in August to 1022 U Street, N.W. to be closer to the 
African-American community it served.  No extant first issue of the <xhtml:cite>National
 Forum</xhtml:cite> exists.
                              </xhtml:p>
                           </xhtml:body>
                        </xhtml:html>
                    </xmldata>
                </fcontent>
            </file>
        </filegrp>
    </filesec>
 
    <!--STRUCTURAL MAP-->    
    <structmap xmlns:np="urn:library-of-congress:ndnp:mets:newspaper">
        <div TYPE="np:encyclopediaEntry" DMDID="encyclopediaMods">
            <fptr FILEID="encyclopediaEntryText"/>
        </div>
    </structmap>    
</mets>

After:

< ?xml version="1.0" encoding="utf-8"?>
< !DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:dcterms="http://purl.org/dc/terms/">
  <head>
    <title>The National Forum (Washington, DC), 1910-19??</title>
    <meta content="The National Forum (Washington, DC), 1910-19??" property="dcterms:title"/>
    <meta content="2007-01-10T09:00:00" property="dcterms:created"/>
    <meta content="Library of Congress" property="dcterms:creator"/>
    <meta content="http://chroniclingamerica.loc.gov/lccn/sn82015056#title" property="dcterms:subject"/>
  </head>
  <body rel="dcterms:description">
    <p>The first issue of the <cite>National Forum</cite> was likely released on April 30, 1910 and the 
newspaper ran through at least November 12 of that year. The four-page African-American weekly 
covered such local events as Howard University graduations and Baptist church activities, but its pages 
also included national news, sports, home maintenance, women's news, science, editorial cartoons, and 
reprinted stories from national newspapers. Its primary focus was on how the news affected the city's black community. A unique feature was its coverage of Elks Club meetings and activities.  Business 
manager John H. Wills contributed the community-centered "Vanity Fair" column that usually appeared 
on the front page of each issue. The publisher and editor was Ralph W. White, who went on to publish 
another African-American newspaper, the <a href="http://chroniclingamerica.loc.gov/lccn/sn86092050">
<cite>McDowell Times</cite></a> of Keystone, West Virginia. Originally located at 609 F St., NW, the 
newspaper's offices moved in August to 1022 U Street, N.W. to be closer to the African-American 
community it served.  No extant first issue of the <cite>National Forum</cite> exists.</p>
  </body>
</html>

I basically took a complicated METS wrapper around some XHTML, which was really just expressing metadata about the HTML, and refactored it as XHTML. Not that METS is a bad XML smell generally, but in this particular case it was overkill. If you look closely you’ll see I’m using RDFa, similar to what Facebook are doing with their OpenGraph Protocol. There’s less to get wrong, what’s there should look more familiar to web developers who aren’t versed in arcane library standards, and I can now read the metadata from the XHTML with an RDFa aware parser, like Python’s rdflib:

>>> import rdflib
>>> g = rdflib.Graph()
>>> g.parse('essays/1.html', format='rdfa')
>>> for triple in g: print triple
... 
(rdflib.term.URIRef('file:///home/ed/Projects/essays/essays/1.html'), rdflib.term.URIRef('http://purl.org/dc/terms/creator'), rdflib.term.Literal(u'Library of Congress'))
(rdflib.term.URIRef('file:///home/ed/Projects/essays/essays/1.html'), rdflib.term.URIRef('http://purl.org/dc/terms/title'), rdflib.term.Literal(u'The National Forum (Washington, DC), 1910-19??'))
(rdflib.term.URIRef('file:///home/ed/Projects/essays/essays/1.html'), rdflib.term.URIRef('http://purl.org/dc/terms/description'), rdflib.term.Literal(u'\n    <p xmlns="http://www.w3.org/1999/xhtml">\nThe first issue of the <cite>National Forum</cite> was likely released on April 30, 1910 and the newspaper ran through at least November 12 of that year. The four-page African-American weekly covered such local events as Howard University graduations and Baptist church activities, but its pages also included national news, sports, home maintenance, women\'s news, science, editorial cartoons, and reprinted stories from national newspapers. Its primary focus was on how the news affected the city\'s black community. A unique feature was its coverage of Elks Club meetings and activities.  Business manager John H. Wills contributed the community-centered &quot;Vanity Fair&quot; column that usually appeared on the front page of each issue. The publisher and editor was Ralph W. White, who went on to publish another African-American newspaper, the <a href="http://chroniclingamerica.loc.gov/lccn/sn86092050"><cite>McDowell Times</cite></a> of Keystone, West Virginia. Originally located at 609 F St., NW, the newspaper\'s offices moved in August to 1022 U Street, N.W. to be closer to the African-American community it served.  No extant first issue of the <cite>National Forum</cite> exists.\n</p>\n  ', datatype=rdflib.term.URIRef('http://www.w3.org/1999/02/22-rdf-syntax-ns#XMLLiteral')))
(rdflib.term.URIRef('file:///home/ed/Projects/essays/essays/1.html'), rdflib.term.URIRef('http://purl.org/dc/terms/subject'), rdflib.term.Literal(u'http://chroniclingamerica.loc.gov/lccn/sn82015056#title'))
(rdflib.term.URIRef('file:///home/ed/Projects/essays/essays/1.html'), rdflib.term.URIRef('http://purl.org/dc/terms/created'), rdflib.term.Literal(u'2007-01-10T09:00:00'))

archival context on the web

3 Replies

On Mark’s advice I attended the Moving Forward With Authority Society of American Archivists pre-conference, which focused on the role of authority control in archival descriptions. There were lots of presentations/discussions over the day, but by and large most of them revolved around the recent release of the Encoded Archival Context: Corporate Bodies, Persons and Families (EAC-CPF) XML schema. Work on the EAC-CPF began in 2001 with the Toronto Tenets, which articulated the need for encoding not only the contents of archival finding aids (w/ Encoded Archival Description), but also the context (people, families, organizations) that the finding aid referenced:

Archival context information consists of information describing the circumstances under which records (defined broadly here to include personal papers and records of organizations) have been created and used. This context includes the identification and characteristics of the persons, organizations, and families who have been the creators, users, or subjects of records, as well as the relationships amongst them.
Toronto Tenets

I’ll admit I’m jaded. So much XML has gone under the bridge, that it’s hard for me to get terribly excited (anymore) about yet-another-xml-schema. Yes, structured data is good. Yes, encouraging people make their data available similarly is important. But for me to get hooked I need a story for how this structured data is going to live, and be used on the Web. This is the main reason I found Daniel Pitti’s talk about the Social Networks of Archival Context (SNAC) to be so exciting.

SNAC is an NEH (and possibly IMLS soon) funded project of the University of Virginia, California Digital Library, and the Berkeley School for Information. The project’s general goal is to:

… unlock descriptions of people from descriptions of their records and link them together in exciting new ways.

Where “descriptions of their records” are EAD XML documents, and “linking them together” means exposing the entities buried in finding aids (people, organizations, etc), assigning identifiers to them, and linking them together on the web. I guess I might be reading what I want a bit into the goal based on the presentation, and my interest in Linked Data. If you are interested, more accurate information about the project can be found in the NEH Proposal that this quote came from.

Even though SNAC was only very recently funded, Daniel was already able to demonstrate a prototype application that he and Brian Tingle worked on. If I’m remembering this right, Daniel basically got a hold of the EAD finding aids from the Library of Congress, extracted contextual information (people, families, corporate names) from relevant EAD elements, and serialized these facts as EAC-CPF documents. Brian then imported the documents using an extension to the eXtensible Text Framwork, which allowed XTF to be EAC-CPF aware.

The end result is a web application that lets you view distinct web pages for individuals mentioned in the archival material. For example here’s one for John von Neumann

All those names in the list on the right are themselves hyperlinks which take you to that person’s page. If you were able to scroll down (the prototype hasn’t been formally launched yet) you could see links to corporate names like Los Alamos Scientific Laboratory, The Institute for Advanced Study, US Atomic Energy Commission, etc. You would also see a Resources section that lists related finding aids and books, such as the John von Neumann Papers finding aid at the Library of Congress.

I don’t think I was the only one in the audience to immediately see the utility of this. In fact it is territory well trodden by OCLC and the other libraries involved in the VIAF project which essentially creates web pages for authority records for people like John Von Neumann who have written books. It’s also similar to what People Australia, BibApp and VIVO are doing to establish richly linked public pages for people. As Daniel pointed out: archives, libraries and museums do a lot of things differently; but ultimately they all have a deep and abiding interest in the intellectual output, and artifacts created by people. So maybe this is an area where we can see more collaboration across the cultural divides between cultural heritage institutions. The activity of putting EAD documents, and their transformed HTML cousins on the web is important. But for them to be more useful they need to be contextualized in the web itself using applications like this SNAC prototype.

I immediately found myself wondering if the URL say for this SNAC view of John von Neumann could be considered an identifier for the EAC-CPF record. And what if the HTML contained the structured EAC-CPF data using xml+xslt, a microformat, rdfa or a link rel pointing at an external XML document? Notice I said an instead of the identifier. If something like EAC-CPF is going to catch on lots of archives would need to start generating (and publishing) them. Inevitably there would be duplication: e.g. multiple institutions with their own notion of John von Neumann. I think this would be a good problem to have, and that having web resolvable identifiers for these records would allow them to be knitted together. It would also allow hubs of EAC-CPF to bubble up, rather than requiring some single institution to serve as *the* master database (as in the case of VIAF).

A few things that would be nice to see for EAC-CPF would be:

  • Instructions on how to link EAD documents to EAC-CPF documents.
  • Recommendations on how to make EAC-CPF data available on the Web.
  • A wikipage or some low-cost page for letting people share where they are publishing EAC-CPF.

In addition I think it would be cool to see:

Anyhow I just wanted to take a moment to say how exciting it is to see the stuff hiding in finding aids making it out onto the Web, with URLs for resources like People, Corporations and Families. I hope to see more as the SNAC project integrates more with existing name authority files (work that Ray Larson at Berkeley is going to be doing), and importing finding aids from more institutions with different EAD encoding practices.

skos as atom

3 Replies

I’ll be the first to admit the tone and content of my last post was a bit off kilter. I guess it was pretty clear immediately from the title of the post. Chalk it up to a second night of insomnia; and also to my unrealistic and probably unnecessary goal of bringing the Atom/REST camp in closer alignment with the RDF/LinkedData camp … at least in my own brain if not on the web.

So, ever the pragmatist, Ian Davis called my bluff a bit on some of the crazier stuff I said:

I know Peter Keane took a stab at this over the summer. But I couldn’t find sample output lying around on the web, so I marked up one by hand to serve as a strawman. So here’s the turtle for the LCSH concept “World Wide Web”:

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix dcterms: <http://purl.org/dc/terms/> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix skos: <http://www.w3.org/2004/02/skos/core#> .

<http://id.loc.gov/authorities/sh95000541#concept>
    a skos:Concept ;
    skos:prefLabel "World Wide Web"@en ;
    dcterms:modified "2001-10-01T09:56:06-04:00"^^<http://www.w3.org/2001/XMLSchema#dateTime> ;
    skos:altLabel "W3 (World Wide Web)"@en, "WWW (World Wide Web)"@en, "Web (World Wide Web)"@en, "World Wide Web (Information retrieval system)"@en ;
    skos:broader <http://id.loc.gov/authorities/sh88002671#concept>, <http://id.loc.gov/authorities/sh92002381#concept> ;
    skos:narrower <http://id.loc.gov/authorities/sh2002000569#concept>, <http://id.loc.gov/authorities/sh2003001415#concept>, <http://id.loc.gov/authorities/sh2007008317#concept>, <http://id.loc.gov/authorities/sh2007008319#concept>, <http://id.loc.gov/authorities/sh2008009697#concept>, <http://id.loc.gov/authorities/sh97003254#concept> ;
    skos:related <http://id.loc.gov/authorities/sh92002816#concept> ;
    skos:closeMatch <http://stitch.cs.vu.nl/vocabularies/rameau/ark:/12148/cb13319953j> .

And here’s the “corresponding” atom:

<?xml version="1.0" encoding="utf-8"?>
<entry xmlns="http://www.w3.org/2005/Atom" xmlns:skos="http://www.w3.org/2004/02/skos/core#">
    <id>http://id.loc.gov/authorities/sh95000541#concept</id>
    <title>LCSH: World Wide Web</title>
    <author><name>Library of Congress</name></author>
    <updated>2001-10-01T09:56:06Z</updated>
    <skos:prefLabel>World Wide Web</skos:prefLabel>
    <skos:altLabel>W3 (World Wide Web)</skos:altLabel>
    <skos:altLabel>Web (World Wide Web)</skos:altLabel>
    <skos:altLabel>World Wide Web (Information retrieval system)</skos:altLabel>
    <skos:altLabel>WWW (World Wide Web)</skos:altLabel>
    <link rel="http://www.w3.org/2004/02/skos/core#broader" href="http://id.loc.gov/authorities/sh88002671#concept" title="Hypertext systems" />
    <link rel="http://www.w3.org/2004/02/skos/core#broader" href="http://id.loc.gov/authorities/sh92002381#concept" title="Multimedia systems" />
    <link rel="http://www.w3.org/2004/02/skos/core#narrower" href="http://id.loc.gov/authorities/sh2008009697#concept" title="Invisible web"/>
    <link rel="http://www.w3.org/2004/02/skos/core#narrower" href="http://id.loc.gov/authorities/sh2007008317#concept" title="Mashups (World Wide Web)" />
    <link rel="http://www.w3.org/2004/02/skos/core#narrower" href="http://id.loc.gov/authorities/sh2002000569#concept" title="Semantic Web" />
    <link rel="http://www.w3.org/2004/02/skos/core#narrower" href="http://id.loc.gov/authorities/sh2007008319#concept" title="Web 2.0" />
    <link rel="http://www.w3.org/2004/02/skos/core#narrower" href="http://id.loc.gov/authorities/sh97003254#concept" title="WebDAV (Standard)" />
    <link rel="http://www.w3.org/2004/02/skos/core#narrower" href="http://id.loc.gov/authorities/sh97003254#concept" title="WebTV (Trademark)" />
    <link rel="http://www.w3.org/2004/02/skos/core#related" href="http://id.loc.gov/authorities/sh92002816#concept" title="Internet" />
    <link rel="http://www.w3.org/2004/02/skos/core#closeMatch" href="http://stitch.cs.vu.nl/vocabularies/rameau/ark:/12148/cb13319953j" title="Web" />
    <link rel="alternate" href="http://id.loc.gov/authorities/sh95000541" type="text/html" />
    <link rel="alternate" href="http://id.loc.gov/authorities/sh95000541.json" type="application/json" />
</entry>

Maybe I botched something? It could use a GRDDL stylesheet I suppose. At least the Atom validates. I really am a bit conflicted posting any of this here because there is so much about the Linked Data community that I like, and want to be a part of. But I’m finding it increasingly difficult to see a Linked Data future where RDF/XML is deployed all over. Instead I bleakly expect we’ll see more fragmentation, and dueling idioms/cultures … and I’m trying to see if perhaps things aren’t as bleak as they seem by grasping at what the groups have in common. Maybe John Cowan’s idea (in the comments) of coming up with an RDF serialization that is valid Atom wasn’t so bad after all? My apologies to any Linked Data folks who have helped me in the past who may have been rubbed the wrong way by my last blog post.

Update: Sean Palmer clued me in to some earlier work he has done in the area of Atom and RDF, the Atom Extensibility Framework. And Niklas Lindström let me know of some thinking he’s done on the topic that is grounded in some work he has been doing for legal information systems in Sweden.