Ontopia JDO

It is with great pleasure that we announce the next step in the Ontopia development, the Ontopia JDO project. 

Rationale
As we worked toward a release of Ontopia 5.3.1, we encountered the massive block of code that is Ontopia RDBMS backend. This huge block of extremely important code was developed as part of the OKS, before it was open sourced. As such, most of this code is now at least 6 years old, and the core code probably even older. We have come to understand the limitations of this code throughout the years, to name a few important ones:

• Supported databases are limited
• Optimization requires extensive knowledge of the code
• Optimization is database dependent
• Tracing and debugging is complicated
• Full text search is complicated, and database dependent
• TOLOG RDBMS is incomplete or broken
• Adding use case oriented optimized SQL queries requires complex code hacking

When faced with these issues during real world projects, we set out to improve upon Ontopia. Soon thereafter we came to the conclusion that improving or changing the code would be a massive undertaking. So instead we choose to research new, yet proven, technologies that offer features that Ontopia requires.

JDO
JDO, which stands for Java Data Objects, is a specification of Java object persistence. It allows a domain model, represented by Pojo’s, to be mapped to a persistence store. JDO was initially designed in JSR 12 in 2002 and the last version (3.0) was released in 2010. JDO was not the only ORM technology we had a look at, but best suited the needs of Ontopia.

DataNucleus
Because JDO is a JSR, several implementations exist. We choose to work with Datanucleus initially, as it is the reference implementation for the latest JDO specification and supports the largest number of data stores.

Benefits
Here are some of the benefits we should be able to achieve with this project:

• Many datastores: RDBMS (all?), Graph based (Neo4j!) , Document based (Mongo, …), Object based, web based.
• External optimization: optimization is (mostly) part of the JDO abstraction layer, which means we won’t have to program it.
• Use of open source community: JDO and Datanucleus are maintained by a large open source community, which means we get improvements on each new version.
• Better integration: extending Ontopia’s datamodel with your own JDO persisted Pojo’s should now be possible.
• Basic full-text searching for every datastore: The project provides a very basic full text search over JDO.
• TOLOG RDBMS remake possible: the inner workings of tolog-rdbms creates JDO queries that are converted to SQL. This could now directly leverage JDO features.

Downside
Sadly, there is a downside. The RDBMS schema has changed. Although the schema closely resembles the Ontopia 5.x schema, it was impossible to fully reuse it. We plan to create a tool that can migrate from an existing RDBMS backend to a new JDO backend as optimized as possible to mitigate this issue.

Project status
The code committed to GitHub at the time of this post has been in development for about a year. It has been tested within the scope of Ontopia code, meaning all the backend tests in net.ontopia.topicmaps.core. Beyond these basic tests, Morpheus has tested the project in combination with existing frameworks and projects based on Ontopia. All these tests are now successful, which means the project is ready to be beta tested.

Roadmap
The project goes into beta testing with this post. We ask you, the Ontopia community, to test it in your projects and frameworks. We especially would like to see all the different datastores tested before we officially claim that Ontopia can be used with all stores. Do not hesitate to ask questions, report issues, or even better: create pull requests. In the coming days we will add known issues and todo’s to the issue tracker. See the README on GitHub to get started with ontopia-jdo.

The Ontopia committers.