Introducing Ontopia REST

It is with great joy that we announce a new addition to the Ontopia framework: Ontopia REST. This new module aims to provide a complete REST implementation for core Ontopia functionality. The module is loosely based on the work David Damen did in the Tropics sandbox project. As such it also uses the well known and often used Restlet Java framework.

Goal

The goal of this new module is two-fold:

  • To provide developers new ways to use the power of Ontopia, without the need for in-depth knowledge of the full Ontopia code
  • To work towards replacing deprecated and antiquated modules such as
    • Webeditor framework
    • TMRAP
    • Vizigator
    • Navigator framework
    • Omnigator

Where the first goal requires only an Ontopia release to realize, the second goal will be an undertaking of its own.

Functionality

A short summary of functionality included:

  • Getting, adding, changing and removing all Topic Map constructs
  • Communication mainly focussed on JSON, by leveraging Jackson
  • XTM, CTM, LTM and TMXML exposing of full Topic Map
  • XTM and TMXML exposing of topic fragments
  • Paging on collection requests
  • Exposing methods in indexes such as ClassInstanceIndexIF and SearcherIF

Status

The module that was added to the Ontopia source contains a big part of the functionality envisioned. A large test suite has been set up to cover all the implemented functionality, containing close to 600 tests. However, there is still work to be done:

  • Documentation
  • TOLOG integration
  • Extensibility
  • Dedicated clients (javascript, java, …)

For this last point we ask you for help. If you are an Ontopia enthusiast that has experience in creating REST clients and that wants to help us out, please let us know!

How to get it

Ontopia REST will be added to the next major release of Ontopia. You can build it from source if you would like to use or test it before the release. You can find the source in the ontopia-rest directory in the master branch.

 

Open source perks

With the move to GitHub, several perks of being an open source project came to light:

Travis-CI

travis-mascot-200pxGitHub has a nice integration with Travis-CI, which offers free continuous integration for open source projects. Every push to a branch or pull-request branch can lead to a build and test of the project. The configuration of the build process is contained in the repository so that each branch may determine it’s own testing parameters.

We’ve enabled the Travis-CI functionality for the main Ontopia repository, and the results are publically available:

Screen Shot 2016-07-07 at 14.45.54

The automated building and testing will assist the developers with determining if branches or pull requests can be merged into the master branch or more work should be done first.

Codacy

SyZuYA39_400x400Codacy offers code analysis and measurements in a cloud service model. These measures can uncover possible improvements of the project. Improvements such as coding style, performance and security threats.

As with Travis, use of Codacy is free for open source projects. The analysis and measurements of Ontopia are publically available.

Some of the issues Codacy reports are a good practice exercise for people that would like to contribute to the project without needed a full in-depth understanding of all the code. Feel free to open merge requests referencing the issues you resolve.

Ontopia has moved to GitHub

It might be old news to the members of the Ontopia mailinglist, but we kinda forgot to update the rest of the world. So here it is.

As Google announced to shut down GoogleCode back in March 2015, we set out to find a new home for the Ontopia code base. GoogleCode would go into read-only mode in August, so we had to find a new home before this happened or we would loose the ability to commit new changes to Ontopia. Continue reading Ontopia has moved to GitHub

Ontopia JDO

It is with great pleasure that we announce the next step in the Ontopia development, the Ontopia JDO project.

Rationale
As we worked toward a release of Ontopia 5.3.1, we encountered the massive block of code that is Ontopia RDBMS backend. This huge block of extremely important code was developed as part of the OKS, before it was open sourced. As such, most of this code is now at least 6 years old, and the core code probably even older. We have come to understand the limitations of this code throughout the years, to name a few important ones:

  • Supported databases are limited
  • Optimization requires extensive knowledge of the code
  • Optimization is database dependent
  • Tracing and debugging is complicated
  • Full text search is complicated, and database dependent
  • TOLOG RDBMS is incomplete or broken
  • Adding use case oriented optimized SQL queries requires complex code hacking

When faced with these issues during real world projects, we set out to improve upon Ontopia. Soon thereafter we came to the conclusion that improving or changing the code would be a massive undertaking. So instead we choose to research new, yet proven, technologies that offer features that Ontopia requires.

JDO
JDO, which stands for Java Data Objects, is a specification of Java object persistence. It allows a domain model, represented by Pojo’s, to be mapped to a persistence store. JDO was initially designed in JSR 12 in 2002 and the last version (3.0) was released in 2010. JDO was not the only ORM technology we had a look at, but best suited the needs of Ontopia.

DataNucleus
Because JDO is a JSR, several implementations exist. We choose to work with Datanucleus initially, as it is the reference implementation for the latest JDO specification and supports the largest number of data stores.

Benefits
Here are some of the benefits we should be able to achieve with this project:

  • Many datastores: RDBMS (all?), Graph based (Neo4j!) , Document based (Mongo, …), Object based, web based.
  • External optimization: optimization is (mostly) part of the JDO abstraction layer, which means we won’t have to program it.
  • Use of open source community: JDO and Datanucleus are maintained by a large open source community, which means we get improvements on each new version.
  • Better integration: extending Ontopia’s datamodel with your own JDO persisted Pojo’s should now be possible.
  • Basic full-text searching for every datastore: The project provides a very basic full text search over JDO.
  • TOLOG RDBMS remake possible: the inner workings of tolog-rdbms creates JDO queries that are converted to SQL. This could now directly leverage JDO features.

Downside
Sadly, there is a downside. The RDBMS schema has changed. Although the schema closely resembles the Ontopia 5.x schema, it was impossible to fully reuse it. We plan to create a tool that can migrate from an existing RDBMS backend to a new JDO backend as optimized as possible to mitigate this issue.

Project status
The code committed to GitHub at the time of this post has been in development for about a year. It has been tested within the scope of Ontopia code, meaning all the backend tests in net.ontopia.topicmaps.core. Beyond these basic tests, Morpheus has tested the project in combination with existing frameworks and projects based on Ontopia. All these tests are now successful, which means the project is ready to be beta tested.

Roadmap
The project goes into beta testing with this post. We ask you, the Ontopia community, to test it in your projects and frameworks. We especially would like to see all the different datastores tested before we officially claim that Ontopia can be used with all stores. Do not hesitate to ask questions, report issues, or even better: create pull requests. In the coming days we will add known issues and todo’s to the issue tracker. See the README on GitHub to get started with ontopia-jdo.

The Ontopia committers.