Annotator Dataset Workflow Howto - Revision history

Palexand: Replaced content with "As of Virtual Appliance v2.2, populating the Annotator Dataset is done automatically when an ontology is processed."

2015-01-15T00:06:53Z

Replaced content with "As of Virtual Appliance v2.2, populating the Annotator Dataset is done automatically when an ontology is processed."

Show changes

Palexand: /* Create Dictionary File */

2013-04-03T19:12:27Z

Create Dictionary File

Surajsingh Thakur: /* Number of paths to roots for a specific Ontology */

2011-11-10T13:21:58Z

Number of paths to roots for a specific Ontology

Surajsingh Thakur at 13:15, 10 November 2011

2011-11-10T13:15:55Z

Surajsingh Thakur at 13:09, 10 November 2011

2011-11-10T13:09:20Z

Surajsingh Thakur: /* Number of concepts for a specific Ontology */

2011-11-10T13:00:34Z

Number of concepts for a specific Ontology

Surajsingh Thakur: /* Number of concepts for a specific Ontology */

2011-11-10T12:58:52Z

Number of concepts for a specific Ontology

Surajsingh Thakur: /* REST calls available: */

2011-11-10T12:56:00Z

REST calls available:

Surajsingh Thakur: /* REST calls available: */

2011-11-10T12:54:33Z

REST calls available:

Surajsingh Thakur: /* REST calls available */

2011-11-10T12:53:21Z

REST calls available

@@ Line 44: / Line 44: @@
 <pre>
 # Dictionary File path
-obs.dictionary.path=/ncbo/bioportal_resources/annotator/dictionary/
+obs.dictionary.path=/srv/ncbo/bioportal_resources/annotator/dictionary/dict
 </pre>
 <li>The dictionary file is generated in several chunks to avoid memory problems. There is a 1,000,000 term limit per file.</li>
@@ Line 50: / Line 50: @@
 <pre>
 http://example/obs/createDictionary/0?apikey=ObsApiKey
 </pre>
 <li>After the files are generated, you can run the command <code>ncborestart</code> to concatenate the resulting dictionary files and to restart the MGREP server.</li>

@@ Line 217: / Line 217: @@
 == Number of paths to roots for a specific Ontology ==
 <pre>
-select OO.local_ontology_id, OO.name, count(OPR.id) as path_to_roots, OO.status from obs_ontology
+select OO.local_ontology_id, OO.name,
-OO, obs_concept OC, obs_path_to_root OPR
+count(OPR.id) as path_to_roots, OO.status
 where OO.id=OC.ontology_id
 and OC.id = OPR.concept_id
-and OO.local_ontology_id ="46116";
+and OO.local_ontology_id ="40133";
 </pre>

@@ Line 153: / Line 153: @@
 delete a.* from obs_relation a, obs_concept b, obs_ontology c
-	where a.level = 1 and a.concept_id = b.id and b.ontology_id = c.id and c.local_ontology_id = @var_ontology_version_id;
+where a.level = 1 and a.concept_id = b.id and b.ontology_id = c.id
 delete a.* from obs_term a, obs_concept b, obs_ontology c
-	where a.concept_id = b.id and b.ontology_id = c.id and c.local_ontology_id = @var_ontology_version_id;
+where a.concept_id = b.id and b.ontology_id = c.id
-delete a.* from obs_semantic_type a, obs_concept b,  obs_ontology c
+delete a.* from obs_semantic_type a, obs_concept b, obs_ontology c
-	where a.concept_id = b.id and b.ontology_id = c.id and c.local_ontology_id = @var_ontology_version_id;
+where a.concept_id = b.id and b.ontology_id = c.id
 delete a.* from obs_concept a, obs_ontology b
-	where a.ontology_id = b.id and b.local_ontology_id = @var_ontology_version_id;
+where a.ontology_id = b.id
-UPDATE obs_ontology set status = 3 where local_ontology_id = @var_ontology_version_id;
+UPDATE obs_ontology set status = 3
 /*
@@ Line 175: / Line 180: @@
 delete a.* from obs_relation a, obs_concept b, obs_ontology c
-	where a.concept_id = b.id and b.ontology_id = c.id and c.local_ontology_id = @var_ontology_version_id and a.level > 1;
+where a.concept_id = b.id and b.ontology_id = c.id
 delete a.* from obs_path_to_root a, obs_concept b, obs_ontology c
-	where a.concept_id = b.id and b.ontology_id = c.id and c.local_ontology_id = @var_ontology_version_id;
+where a.concept_id = b.id and b.ontology_id = c.id
 delete a.* from obs_path_to_leaf a, obs_concept b, obs_ontology c
-	where a.concept_id = b.id and b.ontology_id = c.id and c.local_ontology_id = @var_ontology_version_id;
+where a.concept_id = b.id and b.ontology_id = c.id
-UPDATE obs_ontology set status = 14 where local_ontology_id = @var_ontology_version_id;
+UPDATE obs_ontology set status = 14
 </pre>
@@ Line 218: / Line 227: @@
 <pre>
 select count(*) from obs_relation a, obs_concept b, obs_ontology c
-  where a.concept_id = b.id and b.ontology_id = c.id and c.local_ontology_id = '40133' and a.level > 1;
+where a.concept_id = b.id and b.ontology_id = c.id
-select a.*, b.local_concept_id from obs_path_to_root a, obs_concept b, obs_ontology c
+select a.*, b.local_concept_id from obs_path_to_root a,
-	where a.concept_id = b.id and b.ontology_id = c.id and c.local_ontology_id = '40483';
+obs_concept b, obs_ontology c
 </pre>

@@ Line 49: / Line 49: @@
 <li>Generate the dictionary file:</li>
 <pre>
-http://example/obs/createDictionary/0
+http://example/obs/createDictionary/0?apikey=ObsApiKey
 </pre>
 <li>After the files are generated, you can run the command <code>ncborestart</code> to concatenate the resulting dictionary files and to restart the MGREP server.</li>
@@ Line 187: / Line 188: @@
 === Useful SQL Queries for Monitoring and Validation ===
 The source for several queries that can be used for monitoring and data validation is located in <b>/ncbo/sources/annotator/2004/db/sql/obs_db_cleanup.sql</b> (In the same file as the Data Clean-Up)
 == Number of concepts for a specific Ontology ==
@@ Line 195: / Line 204: @@
 WHERE OT.local_ontology_id in ('40133')
 GROUP BY OT.local_ontology_id;
 </pre>

@@ Line 190: / Line 190: @@
 == Number of concepts for a specific Ontology ==
 <pre>
-select OT.name, OT.local_ontology_id AS local_ontology_id, count(CT.id) AS concepts, OT.status
+select OT.name, OT.local_ontology_id AS local_ontology_id,
-from obs_ontology OT left join obs_concept CT ON CT.ontology_id=OT.id
+count(CT.id) AS concepts, OT.status  from obs_ontology OT
 WHERE OT.local_ontology_id in ('40133')
 GROUP BY OT.local_ontology_id;

@@ Line 102: / Line 102: @@
 <li>For a specific ontology: populate the given ontology if "status = 14". Otherwise the data population process will complain that the status is invalid in the Tomcat log and exit.<br/>
 <code>http://example/obs/loaderBigPaths/{ontology_version_id}?apikey=ObsApiKey<br/>
-e.g. http://example/obs/loaderBigPaths/40671</code></li>
+e.g. http://example/obs/loaderBigPaths/40671?apikey=ObsApiKey</code></li>
 </ol>

@@ Line 99: / Line 99: @@
 <ol>
 <li>For all ontologies: populate all ontologies with "status = 14"<br/>
-<code>http://example/obs/loaderBigPaths/all</code></li>
+<code>http://example/obs/loaderBigPaths/all?apikey=ObsApiKey</code></li>
 <li>For a specific ontology: populate the given ontology if "status = 14". Otherwise the data population process will complain that the status is invalid in the Tomcat log and exit.<br/>
-<code>http://example/obs/loaderBigPaths/{ontology_version_id}<br/>
+<code>http://example/obs/loaderBigPaths/{ontology_version_id}?apikey=ObsApiKey<br/>
 e.g. http://example/obs/loaderBigPaths/40671</code></li>
 </ol>

@@ Line 71: / Line 71: @@
 <ol>
 <li>For all ontologies: populate all ontologies with valid status ("status = 3") currently stored in Hierarchy DB (ontologies are added during the synchronization process).<br/>
-<code>http://example/obs/loaderBigConcepts/all</code></li>
+<code>http://example/obs/loaderBigConcepts/all?apikey=ObsApiKey
 <li>For a specific ontology: populate the given ontology if "status = 3". Otherwise the data population process will complain that the status is invalid in the Tomcat log and exit.<br/>
-<code>http://example/obs/loaderBigConcepts/{ontology_versoin_id}</code></li>
+<code>http://example/obs/loaderBigConcepts/{ontology_versoin_id}?apikey=ObsApiKey
 </ol>