Virtual Appliance FAQ

NCBO VIRTUAL APPLIANCE v2.0 FAQ

This FAQ now covers the NCBO Virtual Appliance v2.0. The FAQ for v1.0 is available in the archive

Ontology Management

How do I add or change categories or groups?

There is currently no UI administrator interface (though this will likely be added in future versions). Categories and groups can be added using a console after logging into the Appliance as the root user.

# from the bash shell:
cd /srv/ncbo/ncbo_cron
bin/ncbo_cron --console
# once in the ruby console:
category = LinkedData::Models::Category.new
category.name = "My Category"
category.acronym = "MY_CAT"
category.save
group = LinkedData::Models::Group.new
group.name = "My Group"
group.acronym = "MY_GRP"
group.save

How can I migrate ontologies from BioPortal or previous NCBO Virtual Appliance versions into a new Appliance?

Programmatic migrations are currently unsupported. Ontologies can be manually downloaded and added using the Web UI.

Ontology Parsing

When are new ontologies parsed?

The ncbo_cron project uses a scheduler to run a process that collects new ontology submissions and parses them, adds them to the search index, calculates metrics, and processes them for use with the annotator. You can also parse ontologies manually.

How do I manually parse an ontology?

To manually parse an ontology, you will need to interact with the code using the console:

# from the bash shell:
cd /srv/ncbo/ncbo_cron
bin/ncbo_cron --console
# once in the ruby console:
ontology = LinkedData::Models::Ontology.find("MY_ACRONYM").first
submission = ontology.latest_submission(status: :any)
logger = Logger.new(STDOUT)
submission.process_submission(logger)
# make available in annotator
annotator = Annotator::Models::NcboAnnotator.new
annotator.create_cache_for_submission(logger, submission)
annotator.generate_dictionary_file()

How can I process a UMLS ontology?

The BioPortal UI currently lacks the ability to select UMLS as a format. To get around this, add the ontology as OWL, then do the following from the command line after logging into the Appliance:

# from the bash shell:
cd /srv/ncbo/ncbo_cron
bin/ncbo_cron --console
# once in the ruby console:
ontology = LinkedData::Models::Ontology.find("MY_ACRONYM").first
submission = ontology.latest_submission(status: :any)
submission.bring_remaining
umls = LinkedData::Models::OntologyFormat.find("UMLS").first
submission.hasOntologyLanguage = umls
submission.save
logger = Logger.new(STDOUT)
submission.process_submission(logger)
# make available in annotator
annotator = Annotator::Models::NcboAnnotator.new
annotator.create_cache_for_submission(logger, submission)
annotator.generate_dictionary_file()

How do I know if an ontology has parsed?

The BioPortal Web UI will cache information about ontologies for 60 seconds. After parsing is complete, just refresh the ontology summary page to see the status for the most recent submission listed under the "Submissions" table.

In addition, you can look at the REST service directly, which will always give you the most updated information. To do this, visit the following URL:

• http://example:8080/ontologies/{ontology_acronym}/latest_submission?include=all
• You can look for the submissionStatus attribute to get the status

Is there a log file for parsing?

Parsing progress is logged in the ontology submission repository folder: /srv/ncbo/repository/{ontology acronym}/{submission id}

Web User Interface

How can I clear the memcached-based UI cache?

• If you are logged in as the admin user, simply visit http://example/admin and click the "Flush Memcache" button. There should be a response indicating success or failure.

How can I use widgets with my Virtual Appliance?

In addition to the existing instructions, you must define an additional Javascript variable in order to have the widgets communicate with your instance of the Virtual Appliance.

 var BP_SEARCH_SERVER = "http://{your_appliance_ip_or_domain_name}";

Replace the '{your_appliance_ip_or_domain_name}' text with the IP address or domain name that's assigned to your Virtual Appliance.

Virtualization Environments

How can I use the OVF image with my virtualization software?
(VMware, VirtualBox, KVM, Xen, etc)

VMware

You can use VMware's ovftool to convert the appliance to work with your VMware product. For example, to convert the appliance for use in VMware Player or Workstation, you would run the command:

ovftool ncbo-appliance.ovf ncbo-appliance.vmx

VirtualBox

VirtualBox supports importing OVF images directly. Simply start your VirtualBox software, then select File->Import Appliance and select the OVF file included in the NCBO Virtual Appliance download.

KVM

First, convert the OVF to VMX format as mentioned in the VMware section above.

Next, ensure that the kvm-qemu-img RPM (or qemu-kvm DEB) is installed. Then, convert the [new] VMDKs (from the VMX conversion step) to raw disk images via the following command:

for disk in `ls -1 *.vmdk`; do diskbase=`basename $disk .vmdk`; qemu-img convert -O raw ${diskbase}.vmdk ${diskbase}.img; done

Create /etc/libvirt/qemu/ncbo-appliance.xml with the following contents:

<domain type='kvm'>
  <name>ncbo-appliance</name>
  <memory>4194304</memory>
  <vcpu>2</vcpu>
  <os>
    <type arch='x86_64' machine='rhel5.4.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'>
    <timer name='pit' tickpolicy='delay'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/var/lib/libvirt/images/ncbo-appliance/ncbo-appliance-disk1.img'/>
      <target dev='hda' bus='ide'/>
      <address type='drive' controller='0' bus='0' unit='0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/var/lib/libvirt/images/ncbo-appliance/ncbo-appliance-disk2.img'/>
      <target dev='hdb' bus='ide'/>
      <address type='drive' controller='0' bus='0' unit='1'/>
    </disk>
    <controller type='ide' index='0'/>
    <interface type='network'>
      <source network='default'/>
      <model type='virtio'/>
    </interface>
    <serial type='pty'>
      <target port='0'/>
    </serial>
    <console type='pty'>
      <target port='0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
    </video>
  </devices>
</domain>

Finally, make any necessary edits to the above file, and run:

virsh start ncbo-appliance

Xen

First, convert the VMDKs to raw disk images as mentioned in the KVM section above.

Create /etc/xen/ncbo-appliance.cfg with the following contents:

name = "ncbo-appliance"
memory = 4096
vcpus = 2
builder = "hvm"
kernel = "/usr/lib/xen/boot/hvmloader"
boot = "c"
pae = 1
acpi = 1
apic = 1
localtime = 0
on_poweroff = "destroy"
on_reboot = "destroy"
on_crash = "destroy"
device_model = "/usr/lib64/xen/bin/qemu-dm"
sdl = 0
vnc = 1
vncunused = 1
keymap = "en-us"
disk = [ "file:/var/lib/xen/images/ncbo-appliance/ncbo-appliance-disk1.img,hda,w", "file:/var/lib/xen/images/ncbo-appliance/ncbo-appliance-disk2.img,hdb,w" ]
vif = [ "bridge=xenbr0,script=vif-bridge,vifname=vif41.0" ]
parallel = "none"
serial = "pty"

Finally, make any necessary edits to the above file, and run:

xm create ncbo-appliance

How can I use the Appliance on Amazon EC2?

• Login to AWS Console, go to EC2 and select US West (Oregon) region from the top right corner: https://console.aws.amazon.com/ec2/home?region=us-west-2

• Click on Launch Instances which will bring up Launch Instance Wizard with multiple tabs.

• 1. Choose AMI:
  • Select Community AMIs and search of “NCBO Appliance” and choose the latest version
• 2. Choose Instance Type Tab
  • Instance Type = General Purpose m1.xlarge (or any other type with at least 4 vCPUs and ~8 GB of RAM)
• 3. Configure Instance Tab
  • Availability Zone = No preference
  • Termination Protection = Prevention against accidental termination
  • Shutdown Behavior = Stop
• 5. Instance Tags:
  • Name = Name this instance something meaningful
• 6. Configure Security Group
  • Choose (you may need to create one first) a Security Group that has ports 22, 80, and 8082 open. It is recommended that you only allow the networks you need. We accept no responsibility/liability for machines getting compromised.

Click "Launch" You will have to create or choose an existing Key Pairs or Create a Key Pair.

• Click on Instances (left hand side of the screen):
  • Once the instance State has changed from Pending to Running and Status Checks is “2/2 checks passed”, login to the instance via the public hostname provided.
  • Now you can SSH into the machine using your key-pair and ec2-user as the user name:
    • ssh -i yourkey_pair ec2-user@{amazon public domain name}
  • The domain name can be retrieved by clicking on the instance and looking at the details that appear at the bottom of the screen. It should look similar to this but with a different set of numbers in the subdomain: ec2-10-0-0-1.us-west-1.compute.amazonaws.com