Given the need in Pelagios 1 to get a demonstration up and running quickly, the Graph Explorer ended up being rather a bit of a monolith. Key goal of the re-write has been, therefore, to introduce a better modularization of the codebase, and to consolidate the core functionality into one software library that can be more easily re-used elsewhere. We're still working on finalizing and testing this library as partners deliver updates to their data, but the essentials are finished. There's a convenient programming API to work with Pelagios's core model primitives - Datasets, GeoAnnotations and Places - in your own software. Bindings to store Pelagios data in a graph database are included, but without the hard-wired dependency that existed in the Graph Explorer. In this regard the Tinkerpop graph database abstraction framework has greatly helped to achieve good decoupling between data model and implementation classes, reduce code size by eliminating the need for much of the boilerplate code, and keep things generic: i.e. the bindings should be re-usable for a variety of graph database brands now (although some of the more advanced I/O and query functionality remains specific to Neo4j - our DB of choice for Pelagios).
Less Speed, More Memory ConsumptionOr was this the other way round? Regarding our toolset to read Pelagios data into the system, we switched our underlying RDF parser from Jena to the OpenRDF Rio parser framework. This allows us to more directly hook into the RDF parsing lifecycle, and avoids the need to construct full RDF graphs in memory before we can actually work with the data. As a result, parsing is now faster and less memory intensive. (Credit goes to Arachne for letting us learn the hard way that datasets can be... LARGE.)
Getting our Feet Wet with ScalaAs with Pelagios 1, the technological basis for our server-side components is still the Java Virtual Machine. This time, however, we chose to go with Scala:
- Scala's syntax is, in general, more compact than that of Java.
- Scala's functional aspects and comprehensive features for dealing with collections and lists are a very good fit with the things we frequently do when handling Pelagios data. Scala almost always eliminates the need for iterations and loops in those cases, and often achieves the same result with a single line of code.
- Pattern matching has been another nice feature to make our parser classes (in particular) much more concise.
- Last but not least: someone once suggested that as a developer, you should learn at least one new programming language every year. Although I find that advice a little fierce, new languages definitely encourage you to think about the same problems in different ways!