The *Child of 10* standard

While Pelagios has been largely about building an alliance of leading ancient world research groups with the aim of linking their data in an open and transparent way, the 'front end' of our product has never been far from our minds. After all, many of the partners are also users of the data that they gather, or, if not the actual users, they have their own user groups to think about and appeal to. As a classicist myself - that is, as someone who spends most of the time reading and analysing ancient Greek texts - I want to be able to access sources easily and trust the data that I get: in other words, I want to be able to turn on the tap and find that the water runs (either hot or cold, depending on what I'm doing); I'm not interested in the plumbing that brings the water to me.

So it is with timely fashion that JISC brought to our attention a fellow jiscGEO project, called G3. In an earlier post, they had talked about a useful benchmark in user interface design being the Child of 10 standard, meaning that a child of 10 should be able to learn to do something useful with the system within 10 minutes. This indicates whether a system is “easy to use” or not.

Will our tool, the Pelagios Graph Explorer, fit the bill, I wonder? While our natural target audience are university researchers (lecturers and undergrads), given the seemingly never-ending appeal of Classics in popular culture, we would be mad not to take seriously the point that a 10 year old should be able to use our tool to find out interesting stuff about the ancient world. Indeed, the technical skills of the average Classicist researcher - not least this one - makes it imperative that we address this question. At the time of writing, then, we are currently engaging in user testing of the Graph Explorer with a sample representative audience, the results from which we will help inform our delivery of the product at the end of October (though it's already clear that this will be a work-in-progress...). All next week Mia Ridge, who has been conducting the user testing, will blog about it, setting out the methods (why we chose them, what prep is done), what actually happens in a session, and then some initial results.

But I can give a sneak preview here of the answer to that question, does the Pelagios Graph Explorer pass the *child of 10* test. On current performance, that would be a 'no'. Which is not to say that things haven't gone well! On the contrary, the very fact that issues are being raised with what you can do now that stuff is linked shows how successful we've been in linking our data: when we started out, it simply wasn't possible to imagine an ancient world of linked data, let alone think seriously about traversing it. But now that we have linked stuff together, the bar has been raised and people - rightly - want to do more with it. This presents a challenge to all the Pelagios partners to provide as much detail as possible in their metadata, in order to allow the kind of free play that a 10 year old - or a classicist - might want.

Perhaps we could start with the name: the Pelagios Graph Explorer isn't very sexy. Suggestions on the back of a postcard, or, ideally, on this blog, welcome.

(Re-)Using the Graph Explorer Pt. 3: Getting Your Data Inside

With a little delay, I'd like to conclude our 3-part introduction to the PELAGIOS Graph Explorer (see here for part 1 and part 2). This time we're looking at data importing.

Data Preparation - the Basics

Getting our initial batch of data from the PELAGIOS partners into the Graph Explorer was both easy and a bit of a challenge at the same time. As for the easy part: the two 'PELAGIOS principles' of...

• aligning place references with PLEIADES and
• using the OAC vocabulary to express them in RDF

make the 'baseline' import almost effortless. We can simply parse the RDF, pick out the OAC annotations, verify whether they point to a valid Pleiades URI - job done. Therefore, if you want to make your own data PELAGIOS-ready, complying with these two principles is really all you need to do. We've included some RDF samples into our code repository, which you can use as a reference regarding the exact RDF syntax. Furthermore, we are also working on a (yet unfinished) online application that generates maps from properly-formatted data dumps, thereby providing online validation for your data.

Structuring the Data

Now on to the advanced part... Once you have produced your OAC-formatted list of PLEIADES URIs, it's really just that: a long, flat list of places. Already that's useful for building basic visualizations - such as maps showing a dataset's geographic extent, or Google-Map-mashups where pushpin-markers link to source texts. But for the Graph Explorer, we wanted to show a more fine-grained picture of the connections within the data.

Usually, a dataset will have some sort of internal hierarchy: a subdivision of an archaeological collection into different sub-collections perhaps; or a structuring of a text corpus into books, subdivided into volumes, chapters, paragraphs and so on. Speaking in terms of the Graph Explorer, this means that when we search for, say, Memphis and Delos, it can tell us that both are mentioned on Herodotus, page 125, rather than giving us the (somewhat less useful) information that both are referenced in GAP's Google Books dataset.

Unfortunately, the 'PELAGIOS principles' don't define an explicit mechanism for expressing such structural information at the moment. Nonetheless our partners' datasets often reflect hierarchy in the design of their resources' URIs: for example, GAP's Google Book URIs carry book IDs and page numbers; annotations provided by Perseus include subdivisions into individual chapters, sections, poems, etc.

To make the Graph Explorer's output more useful, I therefore exploited this implicit information to build the hierarchy in the import script. The import scripts also generate human-readable labels for the hierarchical dataset units, based either on consultation with partners (e.g. we simply agreed on how we would name SPQR's sub-collections and coded that into the import script), or additional metadata in the data dumps (e.g. GAP has rdfs:labels included in the data dump to define the labels).

In hindsight, it may have made sense to think about such an additional 'principle' to cover this (e.g. by including an RDF vocabulary like VoID). But then again, at the start of the project the discussion was very much revolving around the groundwork of getting datasets aligned at all, and the Graph Explorer was still yet a vague idea. (Not to mention that the sheer diversity of the datasets would make the development of a consistent, reasonably fine-grained description scheme a project in its own...)

Importing your Data

The bottom line of all this is: getting a hierarchical, custom-labeled dataset into the Graph Explorer will still require some manual tweaking (read 'coding effort') at the moment. With a little bit of Java development skills, the process should be fairly viable, however: the essential importer classes are fairly well documented, and there are a number of code examples in the repository.

By the way: we've implemented most of the importers in the Java-based scripting language Groovy, which worked really well for us and helped us trim down the size of the import scripts by some lines of code, compared to plain old Java. In particular I'd recommend taking a look at the GAP and Perseus importer source code to get started.

P.S.: The online demo of the PELAGIOS Graph Explorer is available here. Screencasts explaining the basic usage are in this blogpost: The PELAGIOS Graph Explorer: A First Look

An Update for the Arachne-Pleiades annotations

After some bug fixing and general fiddling, we are happy to announce a 'new and improved' version of the 'Arachne to Pleiades matching'. The earlier blogpost explains the main efforts behind this work: here I will mention what has been improved since the first delivery of annotations.

The major new features are:
* Human readable contents behind the Arachne URIs (places, topographical units, objects)
* Improved matching labels using regular expressions
* Annotation of Arachne objects

1. Human readable content behind an URI

This means that the URIs in the annotations can now be used to check what lies behind them and deliver the results in a human friendly form. Normaly an URI is "just" an Identifier, but If you copy the URI of Arachne Dataset to your adressbar the Brower redirects you to the identified Arachne Entity in the Arachne Database. The URI can also be used as an URL.

Examples:
Object: http://arachne.uni-koeln.de/entity/1075708
Place: http://arachne.uni-koeln.de/entity/1206008
Topographical Unit: http://arachne.uni-koeln.de/entity/5152

2. The matching of the labels from Pleiades has been enhanced in the following way:

The old matching like described in step 2 has been enhanced by using regular expressions in the SQL-queries. Regular Expression are a standard, for the expression of text-search patters. These expressions .The Pleiades labels have been cleaned of any regular expression characters like . ? * [ ] and (), which would interfere with the matching process. In addition labels smaller than three characters are ignored because with the regular expressions they could create too many meaningless matches. The new matching, for example, handles that the labels of the Pleiades data set are correctly matched to fields that contain enumerations of Place names.

For Example a field with alternative Placenames contains "athen,atenes,athens" Then it wont match exactly to the string "athen" (the german Version). Its a substring of the String "athen,atehes,athens".
On the otherhand we don't want to match "athen" with "rathenow". Rathenow is not even near Athens. This Problem can only be solved with an Text search pattern like a Regular Expressions.

Also some bugs have been fixed that unintentionally lead to skipping some Pleiades labels in the matching process.


3. Annotations of Arachne Objects:

The data has also been extended, so now the annotations contain links to objects, which form the largest part of the Arachne Database. For every matched place in Arachne the objects that refer to these places are collected.
for example:
A Pleiades Places Links an Arachne Place which Links some Arachne Objects.
Now the script links the Arachne Objects directly to the Pleiades Place, skipping the Arachne Place.



This is done in context of Step 4.

These changes sound quiet small but they have an imense impact. The bugfix and some other changes have been more than doubling the ammount of Arachne Place to Pleiades Place annotations. The Annotations of Objects nearly exploded. As seen in the Example pictures each Place Link creates several Annotations linking Arachne Objects to Pleiades Places.
Connections Links these tend to explode. If the average Arachne Place is connected to 5 Arachne Objects there are 5 Times more Pleiades Place to Arachne Objects Annotations than there are Pleiades Place to Arachne Place annotations!