Annotator Dataset Workflow Howto

Chapter 1: Synchronizing Data with BioPortal

• The synchronization with BioPortal data should be performed regularly (currently scheduled bi-weekly or on-demand).
• The synchronization (Incremental Update) should be done in separate environment from Staging & PROD.

Environment for OBS Data Population
Instance of tomcat : ncbodev-obs
Instance of DB : ncbodev-obsdb1.sunet

• The steps are as following:

1. Remove out-dated ontologies from OBS Database (e.g. older version of ontologies that does not in BioPortal anymore). By invoking this Restlet, it will remove all the outdated ontology data and the associated entities such as concepts, terms, relations, semantic types and hierarchy information.

   Query: Get List of ontologies to be removed (old): http://ncbodev-obs:8080/obs_hibernate/admin/ontologies/list/old
Run: http://ncbodev-obs:8080/obs_hibernate/admin/ontologies/remove

2. Add new ontologies from BioPortal to OBS. By invoking this Restlet, it will add all the new ontology data and the associated entities such as concepts, terms, relations, semantic types and hierarchy information.

   Query: Get List of ontologies to be added (new): http://ncbodev-obs:8080/obs_hibernate/admin/ontologies/list/new
Run: http://ncbodev-obs:8080/obs_hibernate/admin/ontologies/add

3. Populate Concepts (For details, please refer to Chapter 2.1)

   http://ncbodev-obs:8080/obs_hibernate/loaderBigConcepts/all

4. Populate Hierarchy (For details, please refer to Chapter 2.2)

   http://ncbodev-obs:8080/obs_hibernate/loaderBigPaths/all

   To monitor the progress and error, refer to:

   1. The "status" field in the table obs_ontology in OBS DB. (ncbo-dev-obsdb1.sunet)
   2. Check the log in tomcat. (ncbodev-obs: /usr/local/tomcat5/logs)
5. Create Dictionary: To run this, this step has to be complete: "3. Populate Concepts" (For details, please refer to Chapter 4)
6. Create Mapping Data (For details, please refer to Chapter 6)

• When the update is complete, the snapshot of DB should be copied (or replicated) to Staging/PROD.