Fixes for the BC5CDR evaluation #226

caufieldjh · 2023-09-29T17:59:36Z

No description provided.

caufieldjh · 2023-10-03T23:19:35Z

This will still need some checks to ensure the results are getting compared to the test set as expected.
Example output for one test object:

  - test_object:
      publication:
        id: '254'
        title: Hepatitis and renal tubular acidosis after anesthesia with methoxyflurane.
        abstract: A 69-year-old man operated for acute cholecystitis under methoxyflurane
          anesthesia developed postoperatively a hepatic insufficiency syndrome and
          renal tubular acidosis. Massive bleeding appeared during surgery which lasted
          for six hours. Postoperative evolution under supportive therapy was favourable.
          Complete recovery was confirmed by repeated controls performed over a period
          of one year after surgery.
        combined_text: N/A
        full_text: N/A
      triples:
        - subject: MESH:D008733
          predicate: induces
          object: MESH:D000141
          qualifier:
          subject_qualifier:
          object_qualifier:
        - subject: MESH:D008733
          predicate: induces
          object: MESH:D056486
          qualifier:
          subject_qualifier:
          object_qualifier:
    true_positives:
      -   - MESH:D008733 Methoxyflurane
          - MESH:D000141 Acidosis, Renal Tubular
      -   - MESH:D008733 Methoxyflurane
          - MESH:D056486 Chemical and Drug Induced Liver Injury
    num_true_positives: 2
    false_positives:
      -   - MESH:D008733 Methoxyflurane
          - MESH:D017093 Liver Failure
    num_false_positives: 1
    false_negatives: []
    num_false_negatives: 0
    scores:
      similarity:
        jaccard: 0.75
        false_positives:
          - MESH:D017093 Liver Failure
        false_negatives: []
        common:
          - MESH:D000141 Acidosis, Renal Tubular
          - MESH:D008733 Methoxyflurane
          - MESH:D056486 Chemical and Drug Induced Liver Injury
    predicted_object:
      publication:
      triples:
        - subject: MESH:D008733
          predicate: INDUCES
          object: MESH:D056486
          qualifier:
          subject_qualifier:
          object_qualifier:
        - subject: MESH:D008733
          predicate: INDUCES
          object: MESH:D000141
          qualifier:
          subject_qualifier:
          object_qualifier:
        - subject: MESH:D008733
          predicate: INDUCES
          object: MESH:D056486
          qualifier:
          subject_qualifier:
          object_qualifier:
        - subject: MESH:D008733
          predicate: INDUCES
          object: MESH:D000141
          qualifier:
          subject_qualifier:
          object_qualifier:
        - subject: MESH:D008733
          predicate: INDUCES
          object: MESH:D056486
          qualifier:
          subject_qualifier:
          object_qualifier:
        - subject: MESH:D008733
          predicate: INDUCES
          object: MESH:D000141
          qualifier:
          subject_qualifier:
          object_qualifier:
        - subject: MESH:D008733
          predicate: INDUCES
          object: MESH:D017093
          qualifier:
          subject_qualifier:
          object_qualifier:
        - subject: MESH:D008733
          predicate: INDUCES
          object: MESH:D000141
          qualifier:
          subject_qualifier:
          object_qualifier:
    named_entities:
      - id: MESH:D008733
        label: Methoxyflurane
      - id: MESH:D056486
        label: Hepatitis
      - id: MESH:D000141
        label: renal tubular acidosis
      - id: MESH:D008733
        label: Methoxyflurane
      - id: MESH:D056486
        label: Hepatitis
      - id: MESH:D000141
        label: renal tubular acidosis
      - id: MESH:D008733
        label: Methoxyflurane
      - id: MESH:D056486
        label: Hepatitis
      - id: MESH:D000141
        label: renal tubular acidosis
      - id: MESH:D008733
        label: Methoxyflurane
      - id: MESH:D056486
        label: Hepatitis
      - id: MESH:D000141
        label: renal tubular acidosis
      - id: MESH:D017093
        label: hepatic insufficiency syndrome

cmungall · 2023-10-04T18:09:06Z

I see a list of lists under TPs and FPs. That is very suspect. We didn't have that schema before. It makes me thing there is an `add` happening in place of `extend`

…

On Tue, Oct 3, 2023 at 4:19 PM Harry Caufield ***@***.***> wrote: This will still need some checks to ensure the results are getting compared to the test set as expected. Example output for one test object: - test_object: publication: id: '254' title: Hepatitis and renal tubular acidosis after anesthesia with methoxyflurane. abstract: A 69-year-old man operated for acute cholecystitis under methoxyflurane anesthesia developed postoperatively a hepatic insufficiency syndrome and renal tubular acidosis. Massive bleeding appeared during surgery which lasted for six hours. Postoperative evolution under supportive therapy was favourable. Complete recovery was confirmed by repeated controls performed over a period of one year after surgery. combined_text: N/A full_text: N/A triples: - subject: MESH:D008733 predicate: induces object: MESH:D000141 qualifier: subject_qualifier: object_qualifier: - subject: MESH:D008733 predicate: induces object: MESH:D056486 qualifier: subject_qualifier: object_qualifier: true_positives: - - MESH:D008733 Methoxyflurane - MESH:D000141 Acidosis, Renal Tubular - - MESH:D008733 Methoxyflurane - MESH:D056486 Chemical and Drug Induced Liver Injury num_true_positives: 2 false_positives: - - MESH:D008733 Methoxyflurane - MESH:D017093 Liver Failure num_false_positives: 1 false_negatives: [] num_false_negatives: 0 scores: similarity: jaccard: 0.75 false_positives: - MESH:D017093 Liver Failure false_negatives: [] common: - MESH:D000141 Acidosis, Renal Tubular - MESH:D008733 Methoxyflurane - MESH:D056486 Chemical and Drug Induced Liver Injury predicted_object: publication: triples: - subject: MESH:D008733 predicate: INDUCES object: MESH:D056486 qualifier: subject_qualifier: object_qualifier: - subject: MESH:D008733 predicate: INDUCES object: MESH:D000141 qualifier: subject_qualifier: object_qualifier: - subject: MESH:D008733 predicate: INDUCES object: MESH:D056486 qualifier: subject_qualifier: object_qualifier: - subject: MESH:D008733 predicate: INDUCES object: MESH:D000141 qualifier: subject_qualifier: object_qualifier: - subject: MESH:D008733 predicate: INDUCES object: MESH:D056486 qualifier: subject_qualifier: object_qualifier: - subject: MESH:D008733 predicate: INDUCES object: MESH:D000141 qualifier: subject_qualifier: object_qualifier: - subject: MESH:D008733 predicate: INDUCES object: MESH:D017093 qualifier: subject_qualifier: object_qualifier: - subject: MESH:D008733 predicate: INDUCES object: MESH:D000141 qualifier: subject_qualifier: object_qualifier: named_entities: - id: MESH:D008733 label: Methoxyflurane - id: MESH:D056486 label: Hepatitis - id: MESH:D000141 label: renal tubular acidosis - id: MESH:D008733 label: Methoxyflurane - id: MESH:D056486 label: Hepatitis - id: MESH:D000141 label: renal tubular acidosis - id: MESH:D008733 label: Methoxyflurane - id: MESH:D056486 label: Hepatitis - id: MESH:D000141 label: renal tubular acidosis - id: MESH:D008733 label: Methoxyflurane - id: MESH:D056486 label: Hepatitis - id: MESH:D000141 label: renal tubular acidosis - id: MESH:D017093 label: hepatic insufficiency syndrome — Reply to this email directly, view it on GitHub <#226 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAMMONLJ4QUEZ4V4ZY5MC3X5SMRHAVCNFSM6AAAAAA5MZZ4XGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBVHA3TINRTGQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

caufieldjh · 2023-10-04T20:32:14Z

I see a list of lists under TPs and FPs. That is very suspect. We didn't
have that schema before. It makes me thing there is an add happening in
place of extend

I think that's because they're lists of tuples, and in the ctd-run-2023-01-01 output they looked like this:

  true_positives:
  - !!python/tuple
    - MESH:D019386 Alendronate
    - MESH:D010146 Pain
  num_true_positives: 1
  false_positives:
  - !!python/tuple
    - MESH:D019386 Alendronate
    - MESH:D059352 Musculoskeletal Pain

Still looks strange, though it looks like the pairs are getting compared to test appropriately.

cmungall · 2023-10-04T20:48:37Z

ah that makes sense. Everything looks in order then

caufieldjh added 2 commits September 29, 2023 13:59

Use model_dump instead of dict

fc8aefd

use get_adapter with OAK

e991a65

caufieldjh linked an issue Sep 29, 2023 that may be closed by this pull request

Pydantic ValidationError in Chemical To Disease evaluation #211

Closed

caufieldjh added 4 commits October 3, 2023 15:50

Fix validation error for Publication

7d84e89

Fix validation error on Triple vs ChemicalToDiseaseRelationship

f9ef356

Restore original logger output

37e2d6c

Repair negation

7b38c39

caufieldjh added 2 commits October 4, 2023 13:46

Remove extra fields from Publication

ec3a151

Cleanup and linting

de98bc5

Name a param

96aa7e5

caufieldjh marked this pull request as ready for review October 5, 2023 15:30

caufieldjh merged commit 7838cce into main Oct 5, 2023
2 checks passed

caufieldjh deleted the update_bc5cdr_eval branch October 5, 2023 15:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixes for the BC5CDR evaluation #226

Fixes for the BC5CDR evaluation #226

caufieldjh commented Sep 29, 2023

caufieldjh commented Oct 3, 2023

cmungall commented Oct 4, 2023 via email

caufieldjh commented Oct 4, 2023 •

edited

Loading

cmungall commented Oct 4, 2023

Fixes for the BC5CDR evaluation #226

Fixes for the BC5CDR evaluation #226

Conversation

caufieldjh commented Sep 29, 2023

caufieldjh commented Oct 3, 2023

cmungall commented Oct 4, 2023 via email

caufieldjh commented Oct 4, 2023 • edited Loading

cmungall commented Oct 4, 2023

caufieldjh commented Oct 4, 2023 •

edited

Loading