Difference between revisions of "Processing OBR Resources"

From NCBO Wiki
Jump to navigation Jump to search
Line 150: Line 150:
 
| valign="top" width="10%" |Palani
 
| valign="top" width="10%" |Palani
 
| valign="top" |Ontology update + OBR execution with new delta dictionary of all the resources
 
| valign="top" |Ontology update + OBR execution with new delta dictionary of all the resources
−
 
−
|}
 
−
 
−
 
−
 
−
* Oct 05, 2009 [Palani]:
 
−
 
−
{| class="wikitable" border="1" cellspacing="0" cellpadding="1"
 
−
|-
 
−
!  RAT
 
−
!  Date first processed
 
−
!  Date last processed
 
−
!  Database/schema
 
−
!  Type of work flow execution
 
 
|-
 
|-
 
| valign="top" width="5%" | PCM
 
| valign="top" width="5%" | PCM
Line 169: Line 155:
 
| valign="top" width="25%" | Monday, Oct 05, 2009 06:11 PM
 
| valign="top" width="25%" | Monday, Oct 05, 2009 06:11 PM
 
| valign="top" width="10%" | obsdb1.obs_stage
 
| valign="top" width="10%" | obsdb1.obs_stage
 +
| valign="top" width="10%" |Palani
 
| valign="top" | Resource update + OBR execution with full dictionary of the new delta of resources
 
| valign="top" | Resource update + OBR execution with full dictionary of the new delta of resources
−
|}
 
−
 
−
 
−
* Aug 24, 2009 [Palani]:
 
−
 
−
{| class="wikitable" border="1" cellspacing="0" cellpadding="1"
 
−
|-
 
−
!  RAT
 
−
!  Date first processed
 
−
!  Date last processed
 
−
!  Database/schema
 
−
!  Type of work flow execution
 
 
|-
 
|-
 
|  valign="top" width="5%" |GAP
 
|  valign="top" width="5%" |GAP
Line 187: Line 162:
 
|  valign="top" width="25%" | Monday, Aug 24, 2009 04:40 AM
 
|  valign="top" width="25%" | Monday, Aug 24, 2009 04:40 AM
 
|  valign="top" width="10%" |obsdb1.obs_stage
 
|  valign="top" width="10%" |obsdb1.obs_stage
 +
| valign="top" width="10%" |Palani
 
|  valign="top" |Resource update + OBR execution with full dictionary of the new delta of resources
 
|  valign="top" |Resource update + OBR execution with full dictionary of the new delta of resources
−
|}
 
−
 
−
       
 
−
* Aug 04, 2009 [Palani]:
 
−
 
−
{| class="wikitable" border="1" cellspacing="0" cellpadding="1"
 
−
|-
 
−
!  RAT
 
−
!  Date first processed
 
−
!  Date last processed
 
−
!  Database/schema
 
−
!  Type of work flow execution
 
 
|-
 
|-
 
|  valign="top" width="5%" |SMD, DBK
 
|  valign="top" width="5%" |SMD, DBK
Line 205: Line 169:
 
|  valign="top" width="25%" | Tuesday, Aug 04, 2009 02:12 PM
 
|  valign="top" width="25%" | Tuesday, Aug 04, 2009 02:12 PM
 
|  valign="top" width="10%" | obsdb1.obs_stage
 
|  valign="top" width="10%" | obsdb1.obs_stage
 +
| valign="top" width="10%" |Palani
 
|  valign="top" |Resource update + OBR execution with full dictionary of the new delta of resources
 
|  valign="top" |Resource update + OBR execution with full dictionary of the new delta of resources
−
|}
 
−
       
 
−
 
−
* July 29, 2009 [Palani]:
 
−
 
−
{| class="wikitable" border="1" cellspacing="0" cellpadding="1"
 
−
|-
 
−
!  RAT
 
−
!  Date first processed
 
−
!  Date last processed
 
−
!  Database/schema
 
−
!  Type of work flow execution
 
 
|-
 
|-
 
|  valign="top" width="5%" | AE
 
|  valign="top" width="5%" | AE
Line 223: Line 176:
 
|  valign="top" width="25%" | Wednesday, July 29, 2009 02:42 AM
 
|  valign="top" width="25%" | Wednesday, July 29, 2009 02:42 AM
 
|  valign="top" width="10%" | obsdb1.obs_stage
 
|  valign="top" width="10%" | obsdb1.obs_stage
 +
| valign="top" width="10%" |Palani
 
|  valign="top" |Resource update + OBR execution with full dictionary of the new delta of resources
 
|  valign="top" |Resource update + OBR execution with full dictionary of the new delta of resources
−
|}
 
−
 
−
       
 
−
* July 21, 2009 [Kuladip/Palani]:
 
−
 
−
{| class="wikitable" border="1" cellspacing="0" cellpadding="1"
 
−
|-
 
−
!  RAT
 
−
!  Date first processed
 
−
!  Date last processed
 
−
!  Database/schema
 
−
!  Type of work flow execution
 
 
|-
 
|-
 
|  valign="top" width="5%" | (All)
 
|  valign="top" width="5%" | (All)
Line 241: Line 183:
 
|  valign="top" width="25%" | Tuesday, July 21, 2009 03:46 AM
 
|  valign="top" width="25%" | Tuesday, July 21, 2009 03:46 AM
 
|  valign="top" width="10%" | obsdb1.obs_stage
 
|  valign="top" width="10%" | obsdb1.obs_stage
 +
| valign="top" width="10%" |Kuladip/Palani
 
|  valign="top" |Resource update + OBR execution with full dictionary of the new delta of resources
 
|  valign="top" |Resource update + OBR execution with full dictionary of the new delta of resources
−
|}
 
−
   
 
−
   
 
−
* June 17, 2009 [Kuladip]:
 
−
 
−
{| class="wikitable" border="1" cellspacing="0" cellpadding="1"
 
−
|-
 
−
!  RAT
 
−
!  Date first processed
 
−
!  Date last processed
 
−
!  Database/schema
 
−
!  Type of work flow execution
 
 
|-
 
|-
 
|  valign="top" width="5%" | AE
 
|  valign="top" width="5%" | AE
Line 259: Line 190:
 
|  valign="top" width="25%" | Wednesday, June 17, 2009 10:04 AM
 
|  valign="top" width="25%" | Wednesday, June 17, 2009 10:04 AM
 
|  valign="top"  width="10%" | obsdb1.obs_stage
 
|  valign="top"  width="10%" | obsdb1.obs_stage
−
|  valign="top" |Contexts Populated : name, description, species  
+
| valign="top" width="10%" |Kuladip
 +
|  valign="top" |Contexts Populated : name, description, species
 +
|-
 +
|  valign="top" width="5%" | (All)
 +
|  valign="top" width="25%" |
 +
|  valign="top" width="25%" | June 8, 2009
 +
|  valign="top"  width="10%" | obsdb1.obs_stage
 +
| valign="top" width="10%" |Clement
 +
|  valign="top" |End of execution of the OBR workflow & switch of OBR API to the obsdb1.obs_stage schema
 +
|-
 +
|  valign="top" width="5%" | (All)
 +
|  valign="top" width="25%" |
 +
|  valign="top" width="25%" | May 22, 2009
 +
|  valign="top"  width="10%" | obsdb1.obs_stage
 +
| valign="top" width="10%" |Clement
 +
|  valign="top" |OBR workflow run (from scratch) on the new schema obsdb1.obs_stage for all resources
 +
|-
 +
|  valign="top" width="5%" | RXRD
 +
|  valign="top" width="25%" |
 +
|  valign="top" width="25%" | May 2009
 +
|  valign="top"  width="10%" | obsdb1.obs_stage
 +
| valign="top" width="10%" |Adrien
 +
|  valign="top" |Execution of the OBR index on RXRD independently
 +
|-
 +
|  valign="top" width="5%" | (All)
 +
|  valign="top" width="25%" |
 +
|  valign="top" width="25%" | March 2009
 +
|  valign="top"  width="10%" |
 +
| valign="top" width="10%" |Clement
 +
|  valign="top" |DB schema use right now: ncbodb2.obs
 
|}
 
|}
−
       
 
−
 
−
* June 8, 2009 [Clement]: End of execution of the OBR workflow & switch of OBR API to the obsdb1.obs_stage schema
 
−
* May 22, 2009 [Clement]: OBR workflow run (from scratch) on the new schema obsdb1.obs_stage for all resources
 
−
* May 2009 [Adrien]: Execution of the OBR index on RXRD independently (on obsdb1.obs_stage schema)
 
−
* March 2009 [Clement]: DB schema use right now: ncbodb2.obs
 
  
 
== Resources logs ==
 
== Resources logs ==

Revision as of 05:57, 14 October 2009

This wiki page describes the status of the OBR index.

Execution of the OBR worklow & maintenance of the OBR index

Technical documentation

Error creating thumbnail: /srv/www/vhosts/www.bioontology.org/app/mediawiki-1.35.9/includes/shell/limit.sh: line 61: ulimit: cpu time: cannot modify limit: Permission denied /srv/www/vhosts/www.bioontology.org/app/mediawiki-1.35.9/includes/shell/limit.sh: line 84: ulimit: virtual memory: cannot modify limit: Permission denied /srv/www/vhosts/www.bioontology.org/app/mediawiki-1.35.9/includes/shell/limit.sh: line 90: ulimit: file size: cannot modify limit: Permission denied /bin/bash: /usr/bin/convert: No such file or directory Error code: 127
Population of the OBR tables
  • Population And Maintenance Of The OBR Index

Population of OBR and OBS tables is described in document Population_And_Maintenance_Of_The_OBR_Index

  • Technical Instructions for Configuring OBR Workflow

Population of OBR and OBS tables using properties files and shell script is described in document Technical_Instructions_for_Configuring_OBR_Workflow

RAT Start Date End Date Database Type of workflow execution PCM Sunday, Oct 04, 2009 10:47 PM Monday, Oct 05, 2009 06:11 PM Obsdb1.obs_stage Resource update + OBR execution with full dictionary of the new delta of resources


Headline text

OBR workflow execution history

  • Oct 13, 2009 [Palani]:
RAT Date first processed Date last processed Database/schema Executed by Type of work flow execution
OMIM Monday Oct 05, 2009 10:32 PM Monday Oct 05, 2009 10:45 PM obsdb1.obs_stage Palani Ontology update + OBR execution with new delta dictionary of all the resources
PGDR Monday Oct 05, 2009 10:45 PM Monday Oct 05, 2009 10:47 PM obsdb1.obs_stage Palani Ontology update + OBR execution with new delta dictionary of all the resources
CDD Monday Oct 05, 2009 10:47 PM Monday Oct 05, 2009 11:27 PM obsdb1.obs_stage Palani Ontology update + OBR execution with new delta dictionary of all the resources
UPKB Monday Oct 05, 2009 11:27 PM Monday Oct 05, 2009 11:44 PM obsdb1.obs_stage Palani Ontology update + OBR execution with new delta dictionary of all the resources
PGDI Monday Oct 05, 2009 11:44 PM Monday Oct 05, 2009 11:46 PM obsdb1.obs_stage Palani Ontology update + OBR execution with new delta dictionary of all the resources
PGGE Monday Oct 05, 2009 11:46 PM Monday Oct 05, 2009 11:47 PM obsdb1.obs_stage Palani Ontology update + OBR execution with new delta dictionary of all the resources
GEO Monday Oct 05, 2009 11:47 PM Tuesday Oct 06, 2009 00:53 AM obsdb1.obs_stage Palani Ontology update + OBR execution with new delta dictionary of all the resources
WP Tuesday Oct 06, 2009 00:53 AM Tuesday Oct 06, 2009 00:54 AM obsdb1.obs_stage Palani Ontology update + OBR execution with new delta dictionary of all the resources
GAP Tuesday Oct 06, 2009 00:54 AM Tuesday Oct 06, 2009 00:55 AM obsdb1.obs_stage Palani Ontology update + OBR execution with new delta dictionary of all the resources
AE Tuesday Oct 06, 2009 00:55 AM Tuesday Oct 06, 2009 04:37 AM obsdb1.obs_stage Palani Ontology update + OBR execution with new delta dictionary of all the resources
BSM Tuesday Oct 06, 2009 04:37 AM Tuesday Oct 06, 2009 04:39 AM obsdb1.obs_stage Palani Ontology update + OBR execution with new delta dictionary of all the resources
CT Tuesday Oct 06, 2009 04:39 AM Tuesday Oct 06, 2009 05:29 PM obsdb1.obs_stage Palani Ontology update + OBR execution with new delta dictionary of all the resources
DBK Tuesday Oct 06, 2009 05:29 PM Tuesday Oct 06, 2009 05:52 PM obsdb1.obs_stage Palani Ontology update + OBR execution with new delta dictionary of all the resources
RXRD Tuesday Oct 06, 2009 05:52 PM Monday Oct 12, 2009 02:11 AM obsdb1.obs_stage Palani Ontology update + OBR execution with new delta dictionary of all the resources
REAC Monday Oct 12, 2009 02:11 AM Monday Oct 12, 2009 02:23 AM obsdb1.obs_stage Palani Ontology update + OBR execution with new delta dictionary of all the resources
GM Monday Oct 12, 2009 02:23 AM Monday Oct 12, 2009 02:31 AM obsdb1.obs_stage Palani Ontology update + OBR execution with new delta dictionary of all the resources
SMD Monday Oct 12, 2009 02:31 AM Tuesday Oct 13, 2009 08:06 AM obsdb1.obs_stage Palani Ontology update + OBR execution with new delta dictionary of all the resources
PCM Sunday, Oct 04, 2009 10:47 PM Monday, Oct 05, 2009 06:11 PM obsdb1.obs_stage Palani Resource update + OBR execution with full dictionary of the new delta of resources
GAP Sunday, Aug 23, 2009 10:44 PM Monday, Aug 24, 2009 04:40 AM obsdb1.obs_stage Palani Resource update + OBR execution with full dictionary of the new delta of resources
SMD, DBK Monday, Aug 03, 2009 11:28 PM Tuesday, Aug 04, 2009 02:12 PM obsdb1.obs_stage Palani Resource update + OBR execution with full dictionary of the new delta of resources
AE Monday, July 27, 2009 11:19 PM Wednesday, July 29, 2009 02:42 AM obsdb1.obs_stage Palani Resource update + OBR execution with full dictionary of the new delta of resources
(All) Monday, July 13, 2009 02:45 AM Tuesday, July 21, 2009 03:46 AM obsdb1.obs_stage Kuladip/Palani Resource update + OBR execution with full dictionary of the new delta of resources
AE Wednesday, June 17, 2009 3:45 AM Wednesday, June 17, 2009 10:04 AM obsdb1.obs_stage Kuladip Contexts Populated : name, description, species
(All) June 8, 2009 obsdb1.obs_stage Clement End of execution of the OBR workflow & switch of OBR API to the obsdb1.obs_stage schema
(All) May 22, 2009 obsdb1.obs_stage Clement OBR workflow run (from scratch) on the new schema obsdb1.obs_stage for all resources
RXRD May 2009 obsdb1.obs_stage Adrien Execution of the OBR index on RXRD independently
(All) March 2009 Clement DB schema use right now: ncbodb2.obs

Resources logs

  • Execution of the OBR workflow log files

Log files are available here : /project/obs/docman/logs

Resources Access Tools (RATs) development and update

This page is for keeping track of Resources Access Tools (RATs) developement.

How to write Resources Access Tools

How to write a resource access tool is described in this tutorial by Adrien Coulet.

Available resources

Resources fully functional and available in the OBR index are available here: /obr/resources

On development resources

  • PubMed (PM)
  • Stanford Microarray Database (SMD)
  • Pathway Commons
  • CaNanoLab

Others in queue

  • ChemSpider
  • Human Gene Mutation Database

Development history

1. Resource: Clinicaltrials.gov

  • Person responsible: Kuladip Yadav(Optra),Sanjay Jadhav(Optra).
  • Notes: Fixed issue of authentication, fixed other xml related issues.
  • Status:
       Changes done as per suggestions : 
       1. Decrease request delay from 1000 ms to 800ms.
       2. Implement log4j logger mechanism.

2. Resource: GEO

  • Person responsible: Kuladip Yadav(Optra), Sanjay Jadhav(Optra).
  • Notes:Modified GEO resource access tool to get data from both GSE and GDS database.
  • Status:
       Changes done as per suggestions :
       1. Implement log4j logger mechanism.

3. Resource: Pubmed

  • Person responsible: Kuladip Yadav(Optra), Sanjay Jadhav(Optra).
  • Notes:Built a new resource access tool from existing PubMedAccessTool to populate data from eutils and pubmed xml files .
  • Status:
      Changes done as per suggestions :
      1. Remove direct database call from PubMed RAT.
      2. Implement mapStringsToLocalConceptIDs method into ObsOntologiesAccessTool and TermTable.       
      3. Implement log4j logger in related classes.
      4. Code changes pushed in SVN.

Migration to a production architecture & framework

No yet started

Retrieved from "https://www.bioontology.org//mediawiki/index.php?title=Processing_OBR_Resources&oldid=9279"