Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

0ntologies diagnostics (January 2022) - Ontologies with RDF data, without SOLR data #198

Closed
Tracked by #192
syphax-bouazzouni opened this issue Feb 4, 2022 · 10 comments
Closed
Tracked by #192

0ntologies diagnostics (January 2022) - Ontologies with RDF data, without SOLR data #198

syphax-bouazzouni opened this issue Feb 4, 2022 · 10 comments
Assignees
@jonquet @syphax-bouazzouni
Labels
bug content Issues related to the content of AgroPortal

Comments

@syphax-bouazzouni
Copy link
Contributor

syphax-bouazzouni commented Feb 4, 2022 •
edited
Loading

Ontologies with RDF data, without SOLR data:


http://bioportal.bioontology.org/ontologies/ATOL
        flat=
        submissionId=154
        hasOntologyLanguage=OWL
        metrics=averageChildCount:3,classes:2352,classesWithMoreThan25Children:2,classesWithNoDefinition:2352,classesWithOneChild:136,individuals:0,maxChildCount:35,maxDepth:12,properties:0
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_320
        flat=
        submissionId=63
        hasOntologyLanguage=OWL
        metrics=averageChildCount:10,classes:2558,classesWithMoreThan25Children:10,classesWithNoDefinition:1935,classesWithOneChild:1,individuals:0,maxChildCount:399,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_321
        flat=
        submissionId=63
        hasOntologyLanguage=OWL
        metrics=averageChildCount:22,classes:1821,classesWithMoreThan25Children:13,classesWithNoDefinition:1135,classesWithOneChild:2,individuals:0,maxChildCount:465,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_322
        flat=
        submissionId=62
        hasOntologyLanguage=OWL
        metrics=averageChildCount:18,classes:1078,classesWithMoreThan25Children:10,classesWithNoDefinition:630,classesWithOneChild:3,individuals:0,maxChildCount:344,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_324
        flat=
        submissionId=60
        hasOntologyLanguage=OWL
        metrics=averageChildCount:16,classes:1133,classesWithMoreThan25Children:9,classesWithNoDefinition:657,classesWithOneChild:1,individuals:0,maxChildCount:324,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_325
        flat=
        submissionId=61
        hasOntologyLanguage=OWL
        metrics=averageChildCount:14,classes:1427,classesWithMoreThan25Children:10,classesWithNoDefinition:832,classesWithOneChild:3,individuals:0,maxChildCount:355,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_327
        flat=
        submissionId=60
        hasOntologyLanguage=OWL
        metrics=averageChildCount:8,classes:238,classesWithMoreThan25Children:2,classesWithNoDefinition:154,classesWithOneChild:1,individuals:0,maxChildCount:43,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_330
        flat=
        submissionId=57
        hasOntologyLanguage=OWL
        metrics=averageChildCount:10,classes:1480,classesWithMoreThan25Children:10,classesWithNoDefinition:1043,classesWithOneChild:0,individuals:0,maxChildCount:261,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_331
        flat=
        submissionId=61
        hasOntologyLanguage=OWL
        metrics=averageChildCount:10,classes:1921,classesWithMoreThan25Children:9,classesWithNoDefinition:1361,classesWithOneChild:0,individuals:0,maxChildCount:307,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_334
        flat=
        submissionId=60
        hasOntologyLanguage=OWL
        metrics=averageChildCount:10,classes:1566,classesWithMoreThan25Children:12,classesWithNoDefinition:1122,classesWithOneChild:1,individuals:0,maxChildCount:265,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_337
        flat=
        submissionId=63
        hasOntologyLanguage=OWL
        metrics=averageChildCount:11,classes:765,classesWithMoreThan25Children:8,classesWithNoDefinition:503,classesWithOneChild:2,individuals:0,maxChildCount:164,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_338
        flat=
        submissionId=63
        hasOntologyLanguage=OWL
        metrics=averageChildCount:10,classes:440,classesWithMoreThan25Children:4,classesWithNoDefinition:282,classesWithOneChild:0,individuals:0,maxChildCount:80,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_339
        flat=
        submissionId=63
        hasOntologyLanguage=OWL
        metrics=averageChildCount:9,classes:403,classesWithMoreThan25Children:3,classesWithNoDefinition:273,classesWithOneChild:1,individuals:0,maxChildCount:66,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_340
        flat=
        submissionId=60
        hasOntologyLanguage=OWL
        metrics=averageChildCount:11,classes:1078,classesWithMoreThan25Children:10,classesWithNoDefinition:759,classesWithOneChild:0,individuals:0,maxChildCount:173,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_341
        flat=
        submissionId=62
        hasOntologyLanguage=OWL
        metrics=averageChildCount:9,classes:321,classesWithMoreThan25Children:2,classesWithNoDefinition:196,classesWithOneChild:1,individuals:0,maxChildCount:68,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_343
        flat=
        submissionId=56
        hasOntologyLanguage=OWL
        metrics=averageChildCount:12,classes:810,classesWithMoreThan25Children:7,classesWithNoDefinition:431,classesWithOneChild:3,individuals:0,maxChildCount:206,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_345
        flat=
        submissionId=57
        hasOntologyLanguage=OWL
        metrics=METRICS_MISSING
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_MISSING_STATUS,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_346
        flat=
        submissionId=59
        hasOntologyLanguage=OWL
        metrics=averageChildCount:8,classes:461,classesWithMoreThan25Children:3,classesWithNoDefinition:310,classesWithOneChild:3,individuals:0,maxChildCount:89,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_348
        flat=
        submissionId=58
        hasOntologyLanguage=OWL
        metrics=averageChildCount:17,classes:480,classesWithMoreThan25Children:4,classesWithNoDefinition:251,classesWithOneChild:3,individuals:0,maxChildCount:151,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_350
        flat=
        submissionId=57
        hasOntologyLanguage=OWL
        metrics=averageChildCount:30,classes:997,classesWithMoreThan25Children:9,classesWithNoDefinition:460,classesWithOneChild:0,individuals:0,maxChildCount:328,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_358
        flat=
        submissionId=53
        hasOntologyLanguage=OWL
        metrics=averageChildCount:15,classes:1100,classesWithMoreThan25Children:10,classesWithNoDefinition:650,classesWithOneChild:1,individuals:0,maxChildCount:279,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_360
        flat=
        submissionId=50
        hasOntologyLanguage=OWL
        metrics=averageChildCount:10,classes:206,classesWithMoreThan25Children:3,classesWithNoDefinition:102,classesWithOneChild:4,individuals:0,maxChildCount:71,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/CO_365
        flat=
        submissionId=51
        hasOntologyLanguage=OWL
        metrics=averageChildCount:11,classes:443,classesWithMoreThan25Children:3,classesWithNoDefinition:268,classesWithOneChild:2,individuals:0,maxChildCount:94,maxDepth:3,properties:3
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_MISSING_STATUS,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/DEMETER-AIM
        flat=
        submissionId=5
        hasOntologyLanguage=OWL
        metrics=averageChildCount:7,classes:180,classesWithMoreThan25Children:3,classesWithNoDefinition:87,classesWithOneChild:14,individuals:137,maxChildCount:45,maxDepth:6,properties:286
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ERROR_DIFF,ANNOTATOR,INDEXED,INDEX_ERROR:submissionId:4,ANNOTATIONS_AVAILABLE

http://bioportal.bioontology.org/ontologies/GAO
        flat=
        submissionId=4
        hasOntologyLanguage=OWL
        metrics=averageChildCount:5,classes:65,classesWithMoreThan25Children:0,classesWithNoDefinition:48,classesWithOneChild:0,individuals:0,maxChildCount:5,maxDepth:1,properties:0
        submissionStatus=RDF,OBSOLETE,INDEXED_PROPERTIES,METRICS,RDF_LABELS,UPLOADED,ANNOTATOR,DIFF,INDEXED,INDEX_ERROR:missing,ANNOTATIONS_AVAILABLE
@syphax-bouazzouni syphax-bouazzouni mentioned this issue Feb 4, 2022
Closed
7 tasks
@syphax-bouazzouni syphax-bouazzouni added the content Issues related to the content of AgroPortal label Feb 4, 2022
@jonquet
Copy link
Contributor

jonquet commented Feb 8, 2022 •
edited
Loading

Friday Fev 4th :
bin/ncbo_ontology_process -o THVIANDE,THESAE,GECO,GAO,DSW,CO_360,CO_358, CO_350
=> It seems a bug has generated "ERROR_RDF" issues for almost all these ontologies (except THVIANDE) (maybe the issue was cause by not running the script with ontoportal user)

Monday Fev 7th :
bin/ncbo_ontology_process -o THESAE
bin/ncbo_ontology_process -o GECO,GAO,DSW,CO_360,CO_358,CO_350,CO_324,CO_327,CO_330,CO_337,CO_338,CO_339,CO_340
=> Last command failed after CO_358

@jonquet
Copy link
Contributor

jonquet commented Feb 8, 2022 •
edited by syphax-bouazzouni
Loading

List of ontologies to reprocess

(while monitoring the server)

  • ATOL
  • CO_320 => not reprocessed but works with search
  • CO_321 => not reprocessed but works with search
  • CO_322 => not reprocessed but works with search
  • CO_324
  • CO_325 => not reprocessed but works with search
  • CO_327
  • CO_330
  • CO_331 => not reprocessed but works with search
  • CO_334 => not reprocessed but works with search
  • CO_337
  • CO_338
  • CO_339
  • CO_340 => not reprocessed but works with search
  • CO_341
  • CO_343
  • CO_345 => not reprocessed but works with search
  • CO_346
  • CO_348
  • CO_350
  • CO_358
  • CO_360
  • CO_365 => not reprocessed but works with search
  • DEMETER-AIM => DEMETER-AIM can't parse #211
  • GAO => done but another issue show up GAO has parsed but can't display  #205

@jonquet
Copy link
Contributor

jonquet commented Feb 11, 2022

Feb 11th:
bin/ncbo_ontology_process -o CO_324,CO_327,CO_330,CO_337,CO_338,CO_339

@syphax-bouazzouni
Copy link
Contributor Author

syphax-bouazzouni commented Feb 14, 2022 •
edited
Loading

Feb 14th:

  • There is no more problems in the following ontologies : CO_320, CO_321, CO_322, CO_325, CO_331, CO_334, CO_345 CO_365
  • Reprocessing of CO_340 done
  • DEMETER-AIM can't parse #211

@jonquet
Copy link
Contributor

jonquet commented Feb 14, 2022

Feb 14th:
bin/ncbo_ontology_process -o CO_341,CO_343,CO_346,CO_348,CO_350

@syphax-bouazzouni
Copy link
Contributor Author

syphax-bouazzouni commented Feb 14, 2022 •
edited
Loading

New ones from Admin UI:

  • DSW => @jonquet to repair the ontology (some classes do not have labels)
    => Adding missing rdfs:label statements darwin-sw/dsw#7
  • THVIANDE => To be removed soon and changed by MEAT-T
  • THESAE
  • CROPUSAGE
  • ASCOPAIN-T
  • GECO
    => labels are in French (expliclity tagged) and are not indexed
    => Uploaded to StagePortal (which is setup with both eng and fr) to test indexing.
  • CO_347 => Fixed by running ncbo_ontology_index

@jonquet
Copy link
Contributor

jonquet commented Feb 17, 2022 •
edited
Loading

I confirm that parsing GECO in StagePortal fix the indexing issue.

image

However the problem still appear for the Annotator that certainly adopts the same behaviour (ignoring labels in French) when building its dictionary independently of the language parameters of the portal:

image

However, for the Annotator, this is a desired behaviour. With AgroPortal not fully multulingual, we have defined the Annotator to handle only labels in english or without language tags.

@jonquet

This comment was marked as outdated.

Sign in to view
@jonquet jonquet mentioned this issue Feb 17, 2022
Closed
@jonquet jonquet added the bug label Feb 18, 2022
@jonquet

This comment was marked as outdated.

Sign in to view
@syphax-bouazzouni syphax-bouazzouni mentioned this issue Mar 2, 2022
Closed
3 tasks
@syphax-bouazzouni
Copy link
Contributor Author

Our problems with indexing were related to our handling of multilingual values, where for the case of Agroportal we take only attributes values that are tagged with the English language or that have no language tag.

So for the case of resources (ontologies) that are only in French (or any other language), we don't get any values from the triple store.

Our solution was to update this behavior like detailed here: ontoportal-lirmm/goo#6 and ontoportal-lirmm/goo#8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jonquet jonquet

@syphax-bouazzouni syphax-bouazzouni

Labels
bug content Issues related to the content of AgroPortal
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jonquet @syphax-bouazzouni