This workflow (formerly known as BioAID_watskeburt_RDF_MR20_demo) adds the results of a workflow that returns diseases related to an enzyme discovered through text mining to the AIDA RDF repository. Diseases and the reference to this workflow are added to a template ontology or proto-ontology that contains the classes and manually added instances ('myModel'): a form of ontology enrichment. Notes: * you can change the enzyme to any other enzyme or a user input. Technically you can change it to any string, but non-sense results are likely to be produced when it is not a single enzyme. A boolean query is not expected. * in case you increase 'maxHits' in BioAID_DiseaseDiscovery, scaling issues may arise * our demo repository is not a safe place to keep your data. This is preliminary work. For web services inside BioAID_DiseaseDiscovery (formerly BioAID_watskeburt_MR5). RDF repository web services by Willem van Hage (AID/TNO). EZH2 The workflow URI should be 'escaped', i.e. characters such as ':' should be replaced by hexadecimal character codes. http%3A//rdf.adaptivedisclosure.org/~marco/BioAID/Preliminary/Workflows/BioAID_watskeburt/BioAID_DiseaseDiscovery.xml http://rdf.adaptivedisclosure.org/owl/BioAID/myModel/Enriched-ontology/DiscoveredDiseases.owl org.embl.ebi.escience.scuflworkers.java.FlattenList BioAID_enzyme-to-disease_workflow Workflow that links an enzyme to diseases via proteins discovered from relevant medline documents and OMIM net.sourceforge.taverna.scuflworkers.io.TextFileWriter http://ws.adaptivedisclosure.org/axis/services/RepositoryWS?wsdl extractRdf http://aida.science.uva.nl:8888/axis/services/RepositoryWS?wsdl clear The role of this 'workflow' is to provide defaults for the AIDA rdf repository, especially for bio and food application in the VL-e project. Add this workflow to your workflow as a nested workflow. Open it and the beanshell in it to switch defaults (e.g. from bio to food). YOUR DATA IS NOT SAFE IN OUR AIDA REPOSITORY! In principle, we can delete all data without notice, but let us know if you would like to be informed of any changes to the repository. We advocate installing your own Sesame server, for which these defaults provide examples. To download sesame go to http://www.openrdf.org Meer voorbeeld van Willem: username: testuser password: opensesame rdf_format: rdfxml turtle n3 query_language: serql rql subject: (eerste positie in een triple, altijd een URI) http://adaptivedisclosure.org/2007/03/watskeburt#Melanoma predicate: (tweede positie in een triple, altijd een URI) http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://www.w3.org/2000/01/rdf-schema#label object: (derde positie in een triple, een URI, of een literal, soms met type of taal) http://adaptivedisclosure.org/2007/03/watskeburt#Disease Melanoma "Melanoma"@en "10"^^<http://www.w3.org/2001/XMLSchema#integer> context: (gebruik deze nog maar niet, ik schrijf later wel documentatie) data_uri: (de URL van het file waar de RDF uit komt) file:///home/roos/bla/bla/disease_and_enzymes.rdf data: (afhankelijk van het rdf_format RDF in XML of Turtle, o.i.d.) <rdf:RDF ...> ... </rdf:RDF> query: (afhankelijk van de query_language een SeRQL query, o.i.d.) select distinct S from {S} rdfs:label {O} where O like "simsala*" read_write: (of je alle repositories, of alleen die waar je read en/of write permissies op hebt wilt zien, je kunt ook niets meegeven) r rw w The role of this 'workflow' is to provide defaults for the AIDA rdf repository, especially for bio and food application in the VL-e project. Add this workflow to your workflow as a nested workflow. Open it and the beanshell in it to switch defaults (e.g. from bio to food). YOUR DATA IS NOT SAFE IN OUR AIDA REPOSITORY! In principle, we can delete all data without notice, but let us know if you would like to be informed of any changes to the repository. We advocate installing your own Sesame server, for which these defaults provide examples. To download sesame go to http://www.openrdf.org Meer voorbeeld van Willem: username: testuser password: opensesame rdf_format: rdfxml turtle n3 query_language: serql rql subject: (eerste positie in een triple, altijd een URI) http://adaptivedisclosure.org/2007/03/watskeburt#Melanoma predicate: (tweede positie in een triple, altijd een URI) http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://www.w3.org/2000/01/rdf-schema#label object: (derde positie in een triple, een URI, of een literal, soms met type of taal) http://adaptivedisclosure.org/2007/03/watskeburt#Disease Melanoma "Melanoma"@en "10"^^<http://www.w3.org/2001/XMLSchema#integer> context: (gebruik deze nog maar niet, ik schrijf later wel documentatie) data_uri: (de URL van het file waar de RDF uit komt) file:///home/roos/bla/bla/disease_and_enzymes.rdf data: (afhankelijk van het rdf_format RDF in XML of Turtle, o.i.d.) <rdf:RDF ...> ... </rdf:RDF> query: (afhankelijk van de query_language een SeRQL query, o.i.d.) select distinct S from {S} rdfs:label {O} where O like "simsala*" read_write: (of je alle repositories, of alleen die waar je read en/of write permissies op hebt wilt zien, je kunt ook niets meegeven) r rw w 'rdfxml' or 'turtle' or 'n3' rdfxml 'serql' or 'rql' serql bioaid_demo aidademo mem-rdf-db-bio-demo http://rdf.adaptivedisclosure.org/sesame 'rw' or 'r' or 'w' rw http://rdf.adaptivedisclosure.org/~marco/BioAID/Public/Workflows/UtilityWorkflows/AddReceivingOntologyToRdfRepository_MR1.xml Previously known as BioAID_watskeburt_MR5 EZH2 MedLine content import java.util.regex.*; Pattern pattern = Pattern.compile("</?[\\w\\d-]+>"); Matcher matcher = pattern.matcher(tagged_term); String term= matcher.replaceAll(""); tagged_term term 10 org.embl.ebi.escience.scuflworkers.java.FlattenList org.embl.ebi.escience.scuflworkers.java.StringStripDuplicates This workflow retrieves relevant documents, based on a query optimized by adding a string to the original query that will rank the search output according to the most recent years. The added string adds years with priorities (most recent is highest); it starts at 2007. This workflow retrieves relevant documents, based on a query optimized by adding a string to the original query that will rank the search output according to the most recent years. The added string adds years with priorities (most recent is highest); it starts at 2007. This workflow does four things: 1. it retrieves documents relevant for the query string 2. it discovers entities in those documents, these are considered relevant entities 3. it filters proteins from those entities (on the tag protein_molecule) 4. it removes all terms from the list produced by 3 (query terms temporarily considered proteins) ToDo * Replace step 4 by the following procedure: 1. remove the query terms from the output of NER (probably by a regexp matching on what is inside the tag, possibly case-insensitive) 2. remove tag_as_protein_molecule (obsolete) * Add synonym service/workflow Note that Remove_inputquery has an alternative iteration strategy (dot product instead of cross product). Idem for 'Join' in 'SplitQuery'. This workflow does four things: 1. it retrieves documents relevant for the query string 2. it discovers entities in those documents, these are considered relevant entities 3. it filters proteins from those entities (on the tag protein_molecule) 4. it removes all terms from the list produced by 3 (query terms temporarily considered proteins) ToDo * Replace step 4 by the following procedure: 1. remove the query terms from the output of NER (probably by a regexp matching on what is inside the tag, possibly case-insensitive) 2. remove tag_as_protein_molecule (obsolete) * Add synonym service/workflow Note that Remove_inputquery has an alternative iteration strategy (dot product instead of cross product). Idem for 'Join' in 'SplitQuery'. year:(2007^10 2006^9 2005^8 2004^7 2004^6 2003^5 2002^4 2001^3 2000^2 1999^1) StringBuffer temp=new StringBuffer(); temp.append("+("); temp.append(query_string); temp.append(") +"); temp.append(priority_string); String lucene_query = temp.toString(); query_string priority_string lucene_query Lucene query string This workflow applies the search web service from the AIDA toolbox. Comments: This search service is based on lucene defaults; it may be necessary to optimize the querystring to adopt the behaviour to what is most relevant in a particular domain (e.g. for medline prioritizing based on publication date is useful). Lucene favours shorter sentences, which may be bad for subsequent information extraction. This workflow applies the search web service from the AIDA toolbox. Comments: This search service is based on lucene defaults; it may be necessary to optimize the querystring to adopt the behaviour to what is most relevant in a particular domain (e.g. for medline prioritizing based on publication date is useful). Lucene favours shorter sentences, which may be bad for subsequent information extraction. http://ws.adaptivedisclosure.org/axis/services/SearcherWS?wsdl search text/xml text/xml This workflow applies the discovery workflow built around the AIDA 'Named Entity Recognize' web service by Sophia Katrenko. It uses the pre-learned genomics model, named 'MedLine', to find genomics concepts in a set of documents in lucene output format. This workflow applies the discovery workflow built around the AIDA 'Named Entity Recognize' web service by Sophia Katrenko. It uses the pre-learned genomics model, named 'MedLine', to find genomics concepts in a set of documents in lucene output format. MedLine This workflow contains the 'Named Entity Recognize' web service from the AIDA toolbox, created by Sophia Katrenko. It can be used to discover entities of a certain type (determined by 'learned_model') in documents provided in a lucene output format. This workflow contains the 'Named Entity Recognize' web service from the AIDA toolbox, created by Sophia Katrenko. It can be used to discover entities of a certain type (determined by 'learned_model') in documents provided in a lucene output format. Known issues: The output of NErecognize contains concepts with / characters, breaking the xml. For post-processing its results it is better to use string manipulation than xml manipulations. The output is per document, which means entities will be redundant if they occur in more than one document. NElist lucene http://ws.adaptivedisclosure.org/axis/services/NERecognizerService?wsdl NErecognize Model to discover a set of specific concepts; e.g. the prelearned model named 'MedLine' will make the service discover genomics concepts. text/rdf text/xml Entities discoverd in documents provided in lucene output format. This workflow filters protein_molecule-labeled terms from an input string(list). The result is a tagged list of proteins (disregarding false positives in the input). Internal information: This workflow is a copy of 'filter_protein_molecule_MR3' used for the NBIC poster (now in Archive). This workflow filters protein_molecule-labeled terms from an input string(list). The result is a tagged list of proteins (disregarding false positives in the input). Internal information: This workflow is a copy of 'filter_protein_molecule_MR3' used for the NBIC poster (now in Archive). org.embl.ebi.escience.scuflworkers.java.StringStripDuplicates <protein_molecule>\w*</protein_molecule> org.embl.ebi.escience.scuflworkers.java.SplitByRegex (?=<protein_molecule>)|(?<=</protein_molecule>) org.embl.ebi.escience.scuflworkers.java.FilterStringList text/xml text/rdf text/xml org.embl.ebi.escience.scuflworkers.java.FlattenList (#\d+ .+)|(%\d+ .+) org.embl.ebi.escience.scuflworkers.java.SplitByRegex StringBuffer temp= new StringBuffer(); temp.append("<OMIM_disease_label>"); temp.append(OMIM_disease_string); temp.append("</OMIM_disease_label>"); String OMIM_disease_label = temp.toString(); OMIM_disease_string OMIM_disease_label org.embl.ebi.escience.scuflworkers.java.FilterStringList org.embl.ebi.escience.scuflworkers.java.StringStripDuplicates \n get Keyword http://xml.nig.ac.jp/wsdl/OMIM.wsdl search text/xml Query for retrieving document from an indexed corpus. It is assumed the query will be used for a search service based on Lucene. In short that means the query should be string of terms with logical operators or +/- signs to denote if terms are wanted or unwanted. Documents that comply with this query will be used to discover entities in. text/rdf text/xml This workflow creates a query string from the query term (without quotes!), using Martijn Schuemie's synonym service. Known issues: The synonym services may fail instead of returning an empty list when it can not return a result. This workflow creates a query string from the query term (without quotes!), using Martijn Schuemie's synonym service. Known issues: The synonym services may fail instead of returning an empty list when it can not return a result. org.embl.ebi.escience.scuflworkers.java.FlattenList org.embl.ebi.escience.scuflworkers.java.FlattenList import java.util.*; String synstring="\"" + query_term + "\""; String syn; Iterator iterator = synonymlist.iterator(); while ( iterator.hasNext() ) { synstring = synstring + " OR "; syn = ((String) iterator.next()); synstring = synstring + "\"" + syn + "\""; } new_query = synstring; synonymlist query_term new_query http://aida.science.uva.nl:8888/axis/SynsetServer.jws?wsdl getSynsets http://rdf.adaptivedisclosure.org/~marco/BioAID/Public/Workflows/UtilityWorkflows/Split_query_string_MR3.xml Query term without quotes. This workflow creates an RDF document (RDF statements in N-triples format) and saves it in the ntriplesFileDirectory under a name determined by the workflowURI (must be unique). This workflow creates an RDF document (RDF statements in N-triples format) and saves it in the ntriplesFileDirectory under a name determined by the workflowURI (must be unique). // http://rdf.adaptivedisclosure.org/~marco/BioAID/Preliminary/Workflows/Beanshell_code/Beanshell_workflowRDFXML_070713.txt // D://Marco/adaptivedisclosure.org/public_html/BioAID/Preliminary/Workflows/Beanshell_code/Beanshell_workflowRDFXML_070713.txt // Comment: a lot of URIs (namespaces of ontology elements) are hard-coded here; I would like to find ways to make it less so // Notation String RDFformat = "rdfxml"; //data or base URI data_URI = "http://rdf.adaptivedisclosure.org/owl/BioAID/myModel/Enriched-ontology/DiscoveredDiseases.owl"; //Concepts String tminewfCon = "http://rdf.adaptivedisclosure.org/owl/BioAID/myModel/Enriched-ontology/DiscoveredEntities.owl#TextMiningDiscoveryWorkflow"; //Properties //Individuals String wfInd = "http://rdf.adaptivedisclosure.org/owl/BioAID/myModel/Enriched-ontology/DiscoveredDiseases.owl#" + workflowURI; //Relations (triples): String rdf_doc; String oftypestring = "rdf:datatype=\"http://www.w3.org/2001/XMLSchema#string\""; //header rdf_doc = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"; rdf_doc = rdf_doc + "<rdf:RDF\n xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\"\n xmlns:rdfs=\"http://www.w3.org/2000/01/rdf-schema#\">\n"; //body rdf_doc = rdf_doc + "<rdf:Description rdf:about=\"" + wfInd +"\">\n"; rdf_doc = rdf_doc + " <rdf:type rdf:resource= \"" + tminewfCon + "\"/>\n"; if (workflowLabel.length()>0) { rdf_doc = rdf_doc + " <rdfs:label " + oftypestring + ">" + workflowLabel + "</rdfs:label>\n"; } else { rdf_doc = rdf_doc + " <rdfs:label " + oftypestring + ">" + workflowURI + "</rdfs:label>\n"; } if (workflowComment.length()>0) { rdf_doc = rdf_doc + " <rdfs:comment " + oftypestring + ">" + workflowComment + "</rdfs:comment>\n"; } //footer rdf_doc = rdf_doc + "</rdf:Description>\n</rdf:RDF>\n"; rdf_document = rdf_doc; workflowURI workflowLabel workflowComment RDFformat rdf_document data_URI http://ws.adaptivedisclosure.org/axis/services/RepositoryWS?wsdl addRdf http://rdf.adaptivedisclosure.org/~marco/BioAID/Public/Workflows/UtilityWorkflows/AIDA_rdf_bio_repository_MR1_demo.xml Think especially carefully about the iteration strategies. If the inputs for 'CreateDiscoveredRDF' should be seen as pairs (equal amounts of values that can be viewed as rows in a table) use a dot product. If the diseases come as a list, while other parameters such as ontology URIs are singular you may need to group the singular values together in a dot product, and the whole as a cross product. Think especially carefully about the iteration strategies. If the inputs for 'CreateDiscoveredRDF' should be seen as pairs (equal amounts of values that can be viewed as rows in a table) use a dot product. If the diseases come as a list, while other parameters such as ontology URIs are singular you may need to group the singular values together in a dot product, and the whole as a cross product. import java.util.regex.*; String abbreviation = ""; Pattern p = Pattern.compile("([%#]\\d+) (.+[,;]) (.+)<.*"); Matcher m = p.matcher(OMIMdisease); if (m.find()) abbreviation = m.group(3); OMIMdisease abbreviation OMIM_uri = ontology_uri + "#" + OMIM_id; OMIM_id ontology_uri OMIM_uri alt_empty = ""; OMIM_disease alt_empty OMIM_ org.embl.ebi.escience.scuflworkers.java.StringConcat 1 org.embl.ebi.escience.scuflworkers.java.RegularExpressionStringList [%#](\d+) (.+)<.* import java.util.regex.*; String shortname = ""; Pattern p = Pattern.compile("([%#]\\d+) (.+[,;]) (.+)<.*"); Matcher m = p.matcher(OMIMdisease); if (m.find()) { shortname = m.group(3); } else { p = Pattern.compile("([%#]\\d+) (.*)<.*"); m = p.matcher(OMIMdisease); if (m.find()) shortname = m.group(2); }; OMIMdisease shortname 2 org.embl.ebi.escience.scuflworkers.java.RegularExpressionStringList Carefully set the iteration strategy taking into account how this workflow receives its input from workflows it may be used in. In the default case we expect the input of the workflow of this subworkflow to have been tabular (a cross product was set for iteration). // http://rdf.adaptivedisclosure.org/~marco/BioAID/Preliminary/Workflows/Beanshell_code/Beanshell_diseaseTriples_070713.txt // D://Marco/adaptivedisclosure.org/public_html/BioAID/Preliminary/Workflows/Beanshell_code/Beanshell_diseaseTriples_070713.txt // Comment: a lot of URIs (namespaces of ontology elements) are hard-coded here; I would like to find ways to make it less so // perhaps by asking a user to select the right elements from a list which may have been compiled from a search in the ontology on keyword or label value // Notation String RDFformat = "rdfxml"; //data or base URI data_URI = "http://rdf.adaptivedisclosure.org/owl/BioAID/myModel/Enriched-ontology/DiscoveredDiseases.owl"; //Concepts String discdisCon = "http://rdf.adaptivedisclosure.org/owl/BioAID/myModel/Enriched-ontology/DiscoveredDiseases.owl#DiscoveredDisease"; //Properties String hasDiscStatProp = "disc:discoveredThroughProcedure"; String assocenzymeProp = "proto:associatedWithEnzyme"; String abbrevProp = "proto:abbreviation"; String fullnameProp = "proto:full_name"; String shortnameProp = "proto:short_name"; //Individuals String diseaseInd = "http://rdf.adaptivedisclosure.org/owl/BioAID/myModel/Enriched-ontology/DiscoveredDiseases.owl#" + disease_id; String discoveryInd = "http://rdf.adaptivedisclosure.org/owl/BioAID/myModel/Enriched-ontology/DiscoveredDiseases.owl#" + workflowURI; String enzymeInd = "http://rdf.adaptivedisclosure.org/owl/BioAID/myModel/Proto-ontology/EnzymeDisease.owl#" + associatedEnzyme; //Relations (rdfxml): String rdf_doc; String oftypestring = " rdf:datatype=\"http://www.w3.org/2001/XMLSchema#string\""; //header rdf_doc = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"; rdf_doc = rdf_doc + "<rdf:RDF\n xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\"\n xmlns:proto=\"http://rdf.adaptivedisclosure.org/owl/BioAID/myModel/Proto-ontology/EnzymeDisease.owl#\"\n xmlns:disc=\"http://rdf.adaptivedisclosure.org/owl/BioAID/myModel/Enriched-ontology/DiscoveredEntities.owl#\"\n xmlns:rdfs=\"http://www.w3.org/2000/01/rdf-schema#\">\n"; //body rdf_doc = rdf_doc + "<rdf:Description rdf:about=\"" + diseaseInd + "\">\n"; rdf_doc = rdf_doc + " <rdf:type rdf:resource=\"" + discdisCon + "\"/>\n"; rdf_doc = rdf_doc + " <rdfs:label " + oftypestring + ">" + disease_id + "</rdfs:label>\n"; rdf_doc = rdf_doc + " <" + hasDiscStatProp + " rdf:resource=\"" + discoveryInd + "\"/>\n"; rdf_doc = rdf_doc + " <" + assocenzymeProp + " rdf:resource=\"" + enzymeInd + "\"/>\n"; if (abbreviation.length()>0) { rdf_doc = rdf_doc + " <" + abbrevProp + oftypestring + ">" + abbreviation + "</" + abbrevProp + ">\n"; } if (full_name.length()>0) { rdf_doc = rdf_doc + " <" + fullnameProp + oftypestring + ">" + full_name + "</" + fullnameProp + ">\n"; } if (short_name.length()>0) { rdf_doc = rdf_doc + " <" + shortnameProp + oftypestring + ">" + short_name + "</" + shortnameProp + ">\n"; } if (alt_abbreviation.length()>0) { rdf_doc = rdf_doc + " <" + abbrevProp + oftypestring + ">" + alt_abbreviation + "</" + abbrevProp + ">\n"; } if (alt_full_name.length()>0) { rdf_doc = rdf_doc + " <" + fullnameProp + oftypestring + ">" + alt_full_name + "</" + fullnameProp + ">\n"; } if (alt_short_name.length()>0) { rdf_doc = rdf_doc + " <" + shortnameProp + oftypestring + ">" + alt_short_name + "</" + shortnameProp + ">\n"; } //footer rdf_doc = rdf_doc + "</rdf:Description>\n</rdf:RDF>\n"; rdf_document = rdf_doc; workflowURI associatedEnzyme disease_id full_name abbreviation short_name alt_full_name alt_abbreviation alt_short_name RDFformat rdf_document data_URI http://ws.adaptivedisclosure.org/axis/services/RepositoryWS?wsdl addRdf http://rdf.adaptivedisclosure.org/~marco/BioAID/Public/Workflows/UtilityWorkflows/AIDA_rdf_bio_repository_MR1_demo.xml Example: APNEA Example: APNEA, OBSTRUCTIVE SLEEP Example: APNEA If none exists define this input with nothing (zero length string) If none exists define this input with nothing (zero length string) If none exists define this input with nothing (zero length string) Example: http://ws.adaptivedisclosure.org/workflows/BioAID/BioAID_EnrichOntology_MR1.xml Example: EZH2 For instance an OMIM id or the abbreviation of the disease. Complete filepath (filepath+filename) of ontology file to store. E.g. C:\TavernaOutput\EZH2Diseases.owl Completed clear OntologyToRepository Scheduled Running Completed OntologyToRepository WorkflowRefToRDF Scheduled Running Completed WorkflowRefToRDF DiscoveredOMIMDiseasesToRDF Scheduled Running Completed DiscoveredOMIMDiseasesToRDF extractRdf Scheduled Running