Difference between revisions of "Virtual Appliance FAQ"
(removed predictions about UIs coming soon) |
|||
(23 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
[[Category:NCBO Virtual Appliance]] | [[Category:NCBO Virtual Appliance]] | ||
− | = | + | = ONTOPORTAL VIRTUAL APPLIANCE v2.5 FAQ = |
− | This FAQ now covers the | + | This FAQ now covers the Ontoportal Virtual Appliance v2.5 The FAQ for v1.0 is available in the [http://www.bioontology.org/wiki/index.php?title=Virtual_Appliance_FAQ&oldid=12621 archive] |
+ | = General = | ||
+ | == How do I access Virtual Appliance? == | ||
+ | |||
+ | Virtual Appliance Web UI can be accessed by going to http://{your_appliance_ip_or_domain_name} on your web browser.<br> | ||
+ | API can be accessed at http://{your_appliance_ip_or_domain_name}:8080 | ||
+ | |||
+ | |||
+ | You can get IP address of the Appliance with the following methods: | ||
+ | * Local IP Address from terminal: | ||
+ | <code>ip address show eth0 | awk '/inet / {print $2}' | cut -d/ -f1'</code> | ||
+ | * External IP address from the terminal: | ||
+ | <code>curl http://ipecho.net/plain; echo</code> | ||
+ | * Vmware Appliance: | ||
+ | Check 'IP Addresses' on the summary page of the appliance. Please note that Virtual Appliance does not come with vmware tools installed so IP Address would not be displayed until vmware tools are installed. | ||
+ | * Amazon AWS: | ||
+ | use Public IP Addresses or Public DNS listed in your EC2 management console. | ||
+ | |||
+ | == How can I enable emails for lost passwords, notes, and ontology processing? == | ||
+ | |||
+ | Emails are sent via the ontologies_api project on the Appliance. You need to provide a valid mail server (smtp) configuration. The configuration should be provided in the /srv/ncbo/ontologies_api/current/config/environments/production.rb file. | ||
+ | |||
+ | Here are the available settings: | ||
+ | <code> | ||
+ | config.enable_notifications = true # Set to 'true' to send emails | ||
+ | config.email_sender = "admin@example.org" # Default sender for emails | ||
+ | config.email_disable_override = true # If this is set to 'false', all emails will be sent to the email configured in the 'email_override' setting | ||
+ | config.email_override = "admin@example.org" | ||
+ | config.smtp_host = "smtp.example.org" | ||
+ | config.smtp_port = 25 | ||
+ | config.smtp_auth_type = :none # :none, :plain, :login, :cram_md5 | ||
+ | config.smtp_user = "username" # only used if auth_type is not :none | ||
+ | config.smtp_password = "password" # only used if auth_type is not :none | ||
+ | config.smtp_domain = "example.org" | ||
+ | </code> | ||
+ | |||
+ | Once you have changed your settings, you will need to restart the server by running the command <code>/sbin/service unicorn restart</code> | ||
+ | |||
+ | == Can I use the Annotator and Recommender on their own? == | ||
+ | |||
+ | Because services like the Annotator and Recommender depend on the Ontologies API in BioPortal, you can not usefully install those services in a stand-alone way without also installing the core BioPortal software and ontologies. | ||
= Ontology Management = | = Ontology Management = | ||
== How do I add or change categories or groups? == | == How do I add or change categories or groups? == | ||
− | There is currently no UI administrator interface | + | There is currently no UI administrator interface. Categories and groups can be added using a console after logging into the Appliance as the ontoportal user. |
<pre> | <pre> | ||
# from the bash shell: | # from the bash shell: | ||
Line 22: | Line 62: | ||
group.acronym = "MY_GRP" | group.acronym = "MY_GRP" | ||
group.save | group.save | ||
+ | </pre> | ||
+ | |||
+ | == How do I add or change slices? == | ||
+ | There is currently no UI administrator interface. Slices can be added using a console after logging into the Appliance as the ontoportal user. | ||
+ | <pre> | ||
+ | # from the bash shell: | ||
+ | cd /srv/ncbo/ncbo_cron | ||
+ | bin/ncbo_cron --console | ||
+ | # once in the ruby console: | ||
+ | ont1 = LinkedData::Models::Ontology.find("ONT1").first | ||
+ | ont2 = LinkedData::Models::Ontology.find("ONT2").first | ||
+ | slice = LinkedData::Models::Slice.new | ||
+ | slice.name = "My Slice", | ||
+ | slice.description = "This is my custom slice", | ||
+ | slice.acronym = "my_slice", | ||
+ | slice.ontologies = [ont1, ont2] | ||
+ | slice.save | ||
</pre> | </pre> | ||
== How do I delete an ontology? == | == How do I delete an ontology? == | ||
− | Deleting can be done using a console after logging into the Appliance as the | + | Deleting can be done using a console after logging into the Appliance as the ontoportal user. |
<pre> | <pre> | ||
# from the bash shell: | # from the bash shell: | ||
Line 36: | Line 93: | ||
== How can I migrate ontologies from BioPortal or previous NCBO Virtual Appliance versions into a new Appliance? == | == How can I migrate ontologies from BioPortal or previous NCBO Virtual Appliance versions into a new Appliance? == | ||
− | Programmatic migrations are | + | |
+ | Programmatic migrations are not officially supported ("Ontologies can be manually downloaded and added using the Web UI") but we included a script for importing ontologies. | ||
+ | |||
+ | /srv/ncbo/virtual_appliance/deployment/utils/bioportal_ontologies_import.rb, you will need to modify SOURCE_API, SOURCE_APIKEY and ONTOLOGIES_TO_IMPORT variables. | ||
+ | |||
+ | == How can I restrict ontology downloads via the UI? == | ||
+ | |||
+ | This line of code should be modified to include the acronyms of the ontologies for which you want to restrict downloads via the UI: | ||
+ | |||
+ | https://github.com/ncbo/bioportal_web_ui/blob/master/app/views/ontologies/_submissions.html.haml#L32 | ||
+ | |||
+ | This will prevent the “Downloads” column in the Submissions table from appearing on ontology summary pages. | ||
+ | |||
+ | If you want to restrict download via the REST endpoint, you need to modify your environment configuration files (e.g., production.rb, staging.rb, etc.) in your ontologies_api project. The production.rb file should contain a “config.restrict_download” property -- there’s an example of what it should look like in the sample configuration file in GitHub: | ||
+ | |||
+ | https://github.com/ncbo/ontologies_api/blob/master/config/environments/config.rb.sample#L38 | ||
= Ontology Parsing = | = Ontology Parsing = | ||
Line 43: | Line 115: | ||
== How do I manually parse an ontology? == | == How do I manually parse an ontology? == | ||
− | To manually parse an ontology, you will need to interact with the code using the console: | + | To manually parse an ontology, you will need to interact with the code using the console after switching to ontoportal user: |
<pre> | <pre> | ||
# from the bash shell: | # from the bash shell: | ||
Line 55: | Line 127: | ||
# make available in annotator | # make available in annotator | ||
annotator = Annotator::Models::NcboAnnotator.new | annotator = Annotator::Models::NcboAnnotator.new | ||
− | annotator. | + | annotator.create_term_cache_for_submission(logger, submission) |
annotator.generate_dictionary_file() | annotator.generate_dictionary_file() | ||
</pre> | </pre> | ||
Line 61: | Line 133: | ||
== How can I process a UMLS ontology? == | == How can I process a UMLS ontology? == | ||
− | + | UMLS ontologies can be processed to work with our system by converting them to RDF. There is no automated way to do this and you must have your own UMLS MySQL installation and an OSX/Linux/Unix machine with 8GB+ of RAM in order for the conversion process to work. The scripts to convert UMLS to RDF are available on [https://github.com/ncbo/umls2rdf Github]. | |
− | + | ||
− | + | Once you have converted UMLS to RDF, you will get Turtle (.ttl) files that can be uploaded using the BioPortal Web UI. Please select UMLS as the format for these ontologies. | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
== How do I know if an ontology has parsed? == | == How do I know if an ontology has parsed? == | ||
Line 85: | Line 141: | ||
In addition, you can look at the REST service directly, which will always give you the most updated information. To do this, visit the following URL: | In addition, you can look at the REST service directly, which will always give you the most updated information. To do this, visit the following URL: | ||
− | * http:// | + | * http://{your_appliance_ip_or_domain_name}:8080/ontologies/{ontology_acronym}/latest_submission?include=all |
* You can look for the submissionStatus attribute to get the status | * You can look for the submissionStatus attribute to get the status | ||
Line 94: | Line 150: | ||
= Web User Interface = | = Web User Interface = | ||
== How can I clear the memcached-based UI cache? == | == How can I clear the memcached-based UI cache? == | ||
− | * If you are logged in as the admin user, simply visit http:// | + | * If you are logged in as the admin user, simply visit http://{your_appliance_ip_or_domain_name}/admin and click the "Flush Memcache" button. There should be a response indicating success or failure. |
== How can I use widgets with my Virtual Appliance? == | == How can I use widgets with my Virtual Appliance? == | ||
Line 224: | Line 280: | ||
== How can I use the Appliance on Amazon EC2? == | == How can I use the Appliance on Amazon EC2? == | ||
− | + | Appliance is available on [https://aws.amazon.com/marketplace/pp/B00MX5YKVU Amazon Marketplace] | |
− | |||
− | |||
− | + | === What is the admin password for the AWS Appliance? === | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | + | The default application administrator is 'admin' and the initial password is the Instance ID (i-xxxxxxxx). Please change this as soon as you install the virtual appliance. | |
− | |||
− | |||
− | |||
− |
Revision as of 16:06, 4 June 2019
ONTOPORTAL VIRTUAL APPLIANCE v2.5 FAQ
This FAQ now covers the Ontoportal Virtual Appliance v2.5 The FAQ for v1.0 is available in the archive
General
How do I access Virtual Appliance?
Virtual Appliance Web UI can be accessed by going to http://{your_appliance_ip_or_domain_name} on your web browser.
API can be accessed at http://{your_appliance_ip_or_domain_name}:8080
You can get IP address of the Appliance with the following methods:
- Local IP Address from terminal:
ip address show eth0 | awk '/inet / {print $2}' | cut -d/ -f1'
- External IP address from the terminal:
curl http://ipecho.net/plain; echo
- Vmware Appliance:
Check 'IP Addresses' on the summary page of the appliance. Please note that Virtual Appliance does not come with vmware tools installed so IP Address would not be displayed until vmware tools are installed.
- Amazon AWS:
use Public IP Addresses or Public DNS listed in your EC2 management console.
How can I enable emails for lost passwords, notes, and ontology processing?
Emails are sent via the ontologies_api project on the Appliance. You need to provide a valid mail server (smtp) configuration. The configuration should be provided in the /srv/ncbo/ontologies_api/current/config/environments/production.rb file.
Here are the available settings:
config.enable_notifications = true # Set to 'true' to send emails
config.email_sender = "admin@example.org" # Default sender for emails
config.email_disable_override = true # If this is set to 'false', all emails will be sent to the email configured in the 'email_override' setting
config.email_override = "admin@example.org"
config.smtp_host = "smtp.example.org"
config.smtp_port = 25
config.smtp_auth_type = :none # :none, :plain, :login, :cram_md5
config.smtp_user = "username" # only used if auth_type is not :none
config.smtp_password = "password" # only used if auth_type is not :none
config.smtp_domain = "example.org"
Once you have changed your settings, you will need to restart the server by running the command /sbin/service unicorn restart
Can I use the Annotator and Recommender on their own?
Because services like the Annotator and Recommender depend on the Ontologies API in BioPortal, you can not usefully install those services in a stand-alone way without also installing the core BioPortal software and ontologies.
Ontology Management
How do I add or change categories or groups?
There is currently no UI administrator interface. Categories and groups can be added using a console after logging into the Appliance as the ontoportal user.
# from the bash shell: cd /srv/ncbo/ncbo_cron bin/ncbo_cron --console # once in the ruby console: category = LinkedData::Models::Category.new category.name = "My Category" category.acronym = "MY_CAT" category.save group = LinkedData::Models::Group.new group.name = "My Group" group.acronym = "MY_GRP" group.save
How do I add or change slices?
There is currently no UI administrator interface. Slices can be added using a console after logging into the Appliance as the ontoportal user.
# from the bash shell: cd /srv/ncbo/ncbo_cron bin/ncbo_cron --console # once in the ruby console: ont1 = LinkedData::Models::Ontology.find("ONT1").first ont2 = LinkedData::Models::Ontology.find("ONT2").first slice = LinkedData::Models::Slice.new slice.name = "My Slice", slice.description = "This is my custom slice", slice.acronym = "my_slice", slice.ontologies = [ont1, ont2] slice.save
How do I delete an ontology?
Deleting can be done using a console after logging into the Appliance as the ontoportal user.
# from the bash shell: cd /srv/ncbo/ncbo_cron bin/ncbo_cron --console # once in the ruby console: ontology = LinkedData::Models::Ontology.find("MY_ONTOLOGY_ACRONYM").first ontology.delete
How can I migrate ontologies from BioPortal or previous NCBO Virtual Appliance versions into a new Appliance?
Programmatic migrations are not officially supported ("Ontologies can be manually downloaded and added using the Web UI") but we included a script for importing ontologies.
/srv/ncbo/virtual_appliance/deployment/utils/bioportal_ontologies_import.rb, you will need to modify SOURCE_API, SOURCE_APIKEY and ONTOLOGIES_TO_IMPORT variables.
How can I restrict ontology downloads via the UI?
This line of code should be modified to include the acronyms of the ontologies for which you want to restrict downloads via the UI:
https://github.com/ncbo/bioportal_web_ui/blob/master/app/views/ontologies/_submissions.html.haml#L32
This will prevent the “Downloads” column in the Submissions table from appearing on ontology summary pages.
If you want to restrict download via the REST endpoint, you need to modify your environment configuration files (e.g., production.rb, staging.rb, etc.) in your ontologies_api project. The production.rb file should contain a “config.restrict_download” property -- there’s an example of what it should look like in the sample configuration file in GitHub:
https://github.com/ncbo/ontologies_api/blob/master/config/environments/config.rb.sample#L38
Ontology Parsing
When are new ontologies parsed?
The ncbo_cron project uses a scheduler to run a process that collects new ontology submissions and parses them, adds them to the search index, calculates metrics, and processes them for use with the annotator. You can also parse ontologies manually.
How do I manually parse an ontology?
To manually parse an ontology, you will need to interact with the code using the console after switching to ontoportal user:
# from the bash shell: cd /srv/ncbo/ncbo_cron bin/ncbo_cron --console # once in the ruby console: ontology = LinkedData::Models::Ontology.find("MY_ACRONYM").first submission = ontology.latest_submission(status: :any) logger = Logger.new(STDOUT) submission.process_submission(logger) # make available in annotator annotator = Annotator::Models::NcboAnnotator.new annotator.create_term_cache_for_submission(logger, submission) annotator.generate_dictionary_file()
How can I process a UMLS ontology?
UMLS ontologies can be processed to work with our system by converting them to RDF. There is no automated way to do this and you must have your own UMLS MySQL installation and an OSX/Linux/Unix machine with 8GB+ of RAM in order for the conversion process to work. The scripts to convert UMLS to RDF are available on Github.
Once you have converted UMLS to RDF, you will get Turtle (.ttl) files that can be uploaded using the BioPortal Web UI. Please select UMLS as the format for these ontologies.
How do I know if an ontology has parsed?
The BioPortal Web UI will cache information about ontologies for 60 seconds. After parsing is complete, just refresh the ontology summary page to see the status for the most recent submission listed under the "Submissions" table.
In addition, you can look at the REST service directly, which will always give you the most updated information. To do this, visit the following URL:
- http://{your_appliance_ip_or_domain_name}:8080/ontologies/{ontology_acronym}/latest_submission?include=all
- You can look for the submissionStatus attribute to get the status
Is there a log file for parsing?
Parsing progress is logged in the ontology submission repository folder: /srv/ncbo/repository/{ontology acronym}/{submission id}
Web User Interface
How can I clear the memcached-based UI cache?
- If you are logged in as the admin user, simply visit http://{your_appliance_ip_or_domain_name}/admin and click the "Flush Memcache" button. There should be a response indicating success or failure.
How can I use widgets with my Virtual Appliance?
In addition to the existing instructions, you must define an additional Javascript variable in order to have the widgets communicate with your instance of the Virtual Appliance.
var BP_SEARCH_SERVER = "http://{your_appliance_ip_or_domain_name}";
Replace the '{your_appliance_ip_or_domain_name}' text with the IP address or domain name that's assigned to your Virtual Appliance.
Virtualization Environments
How can I use the OVF image with my virtualization software?
(VMware, VirtualBox, KVM, Xen, etc)
VMware
You can use VMware's ovftool to convert the appliance to work with your VMware product. For example, to convert the appliance for use in VMware Player or Workstation, you would run the command:
ovftool ncbo-appliance.ovf ncbo-appliance.vmx
VirtualBox
VirtualBox supports importing OVF images directly. Simply start your VirtualBox software, then select File->Import Appliance and select the OVF file included in the NCBO Virtual Appliance download.
KVM
First, convert the OVF to VMX format as mentioned in the VMware section above.
Next, ensure that the kvm-qemu-img
RPM (or qemu-kvm
DEB) is installed. Then, convert the [new] VMDKs (from the VMX conversion step) to raw disk images via the following command:
for disk in `ls -1 *.vmdk`; do diskbase=`basename $disk .vmdk`; qemu-img convert -O raw ${diskbase}.vmdk ${diskbase}.img; done
Create /etc/libvirt/qemu/ncbo-appliance.xml
with the following contents:
<domain type='kvm'> <name>ncbo-appliance</name> <memory>4194304</memory> <vcpu>2</vcpu> <os> <type arch='x86_64' machine='rhel5.4.0'>hvm</type> <boot dev='hd'/> </os> <features> <acpi/> <apic/> <pae/> </features> <clock offset='utc'> <timer name='pit' tickpolicy='delay'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/libexec/qemu-kvm</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/ncbo-appliance/ncbo-appliance-disk1.img'/> <target dev='hda' bus='ide'/> <address type='drive' controller='0' bus='0' unit='0'/> </disk> <disk type='file' device='disk'> <driver name='qemu' type='raw'/> <source file='/var/lib/libvirt/images/ncbo-appliance/ncbo-appliance-disk2.img'/> <target dev='hdb' bus='ide'/> <address type='drive' controller='0' bus='0' unit='1'/> </disk> <controller type='ide' index='0'/> <interface type='network'> <source network='default'/> <model type='virtio'/> </interface> <serial type='pty'> <target port='0'/> </serial> <console type='pty'> <target port='0'/> </console> <input type='mouse' bus='ps2'/> <graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/> <video> <model type='cirrus' vram='9216' heads='1'/> </video> </devices> </domain>
Finally, make any necessary edits to the above file, and run:
virsh start ncbo-appliance
Xen
First, convert the VMDKs to raw disk images as mentioned in the KVM section above.
Create /etc/xen/ncbo-appliance.cfg
with the following contents:
name = "ncbo-appliance" memory = 4096 vcpus = 2 builder = "hvm" kernel = "/usr/lib/xen/boot/hvmloader" boot = "c" pae = 1 acpi = 1 apic = 1 localtime = 0 on_poweroff = "destroy" on_reboot = "destroy" on_crash = "destroy" device_model = "/usr/lib64/xen/bin/qemu-dm" sdl = 0 vnc = 1 vncunused = 1 keymap = "en-us" disk = [ "file:/var/lib/xen/images/ncbo-appliance/ncbo-appliance-disk1.img,hda,w", "file:/var/lib/xen/images/ncbo-appliance/ncbo-appliance-disk2.img,hdb,w" ] vif = [ "bridge=xenbr0,script=vif-bridge,vifname=vif41.0" ] parallel = "none" serial = "pty"
Finally, make any necessary edits to the above file, and run:
xm create ncbo-appliance
How can I use the Appliance on Amazon EC2?
Appliance is available on Amazon Marketplace
What is the admin password for the AWS Appliance?
The default application administrator is 'admin' and the initial password is the Instance ID (i-xxxxxxxx). Please change this as soon as you install the virtual appliance.