Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support property-based hierarchies from CodeSystemSupport #3448

Closed
lmsurpre opened this issue Mar 9, 2022 · 6 comments
Closed

support property-based hierarchies from CodeSystemSupport #3448

lmsurpre opened this issue Mar 9, 2022 · 6 comments
Assignees
Labels
enhancement New feature or request P2 Priority 2 - Should Have

Comments

@lmsurpre
Copy link
Member

lmsurpre commented Mar 9, 2022

Is your feature request related to a problem? Please describe.
In the hl7 codesystem artifacts that ship with FHIR R4, hiearchy is expressed by nesting the concepts and the "hiearchyMeaning" field is set to indicate the semantics of that nesting.

For at least some of the CodeSystems moved to https://terminology.hl7.org/ these have been updated to express hierarchy through properties. For the v3 CodeSystems, that seems to be indicated by a custom "subsumedBy" property and these are the affected CodeSystems as of 3.1:
image

I assume this was done so that they can properly reflect a "polyhierarchy" (where a single concept belongs "under" two different parent concepts).

Unfortunately, the current support in CodeSystemSupport for IS-A filters will only work for nested hierarchies at the moment:

        public IsAFilter(CodeSystem codeSystem, Concept concept) {
            this.codeSystem = codeSystem;
            descendantsAndSelf = getConcepts(concept, getCodeValueFunction(codeSystem));
        }
    /**
     * Get a set containing {@link R} instances mapped from concepts where all structural
     * hierarchies have been flattened.
     *
     * @param <R>
     *     the element type of the result set
     * @param concept
     *     the root of the tree containing the Concept instances to be flattened
     * @param function
     *     the function to apply to each element of the result set
     * @return
     *     flattened set of {@link R} instances mapped from concepts for the given tree
     */
    public static <R> Set<R> getConcepts(Concept concept, Function<Concept, ? extends R> function) {
        if (concept == null) {
            return Collections.emptySet();
        }
        Set<R> result = new LinkedHashSet<>();
        result.add(function.apply(concept));
        for (Concept c : concept.getConcept()) {
            result.addAll(getConcepts(c, function));
        }
        return result;
    }

How are we supposed to know that the subsumedBy property is the one we're supposed to follow in this case!?

Describe the solution you'd like

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Acceptance Criteria

  1. GIVEN [a precondition]
    AND [another precondition]
    WHEN [test step]
    AND [test step]
    THEN [verification step]
    AND [verification step]

Additional context
See the below for some general guidance on the nested hierarchy vs property-based hierarchy:
https://chat.fhir.org/#narrow/stream/179166-implementers/topic/hierarchical.20CodeSystems.20and.20ValueSets/near/234918418

I know we have support for these property-based hierarchies in the fhir-term-graph stuff, but I was hoping for something a little simpler that can be simply implemented on top of the RegistryTermServiceProvider.

@lmsurpre lmsurpre added the enhancement New feature or request label Mar 9, 2022
@lmsurpre lmsurpre changed the title support property-based hierarchies from the support property-based hierarchies from CodeSystemSupport Mar 9, 2022
@lmsurpre
Copy link
Member Author

lmsurpre commented Mar 10, 2022

How are we supposed to know that the subsumedBy property is the one we're supposed to follow in this case!?

I think I managed to work out the linkage: the CodeSystem resource has a top-level property field where you're supposed to define the properties used by the codesystem: https://www.hl7.org/fhir/codesystem-definitions.html#CodeSystem.property

One of the fields under CodeSystem.property is uri: Reference to the formal meaning of the property. One possible source of meaning is the Concept Properties code system.

In all of the v3 CodeSystem resources I list above (or at least the ones I checked), the CodeSystem.property.uri for the "subsumedBy" property is set to http://hl7.org/fhir/concept-properties#parent.
Finally, if you dereference that link, you get to http://hl7.org/fhir/R4/codesystem-concept-properties.html where we have the following definition for "parent":

The concept identified in this property is a parent of the concept on which it is a property. The property type will be 'code'. The meaning of 'parent' is defined by the hierarchyMeaning attribute

So there is this optional field thats of type uri which can optionally point at one of these under-defined URIs and that is supposed to use the definition from this related codesystem somehow which tells you to look back at the hiearchyMeaning of the CodeSystem for the actual meaning... that feels awful complicated / loose / non-computable and I'd be nervous about baking that into our algorithm at this point.

Instead, I think we should just see about either:
A. packaging up all the pre-expanded valuesets in our fhir-hl7-terminology module; and/or
B. grab just the ones that have "Preferred", "Extensible", or "Required" bindings from the core spec and put their expanded flavor in with our fhir-core-r4b registry provider

@lmsurpre
Copy link
Member Author

lmsurpre commented Mar 10, 2022

these are the non-example bindings to terminology.hl7.org valuesets in R4B as of 2022-03-09:

http://terminology.hl7.org/ValueSet/v2-0116
http://terminology.hl7.org/ValueSet/v2-0276
http://terminology.hl7.org/ValueSet/v2-0493
http://terminology.hl7.org/ValueSet/v2-0916
http://terminology.hl7.org/ValueSet/v3-ActConsentDirective
http://terminology.hl7.org/ValueSet/v3-ActEncounterCode
http://terminology.hl7.org/ValueSet/v3-ActIncidentCode
http://terminology.hl7.org/ValueSet/v3-Confidentiality|2.0.0|2.0.0 [sic]
http://terminology.hl7.org/ValueSet/v3-PurposeOfUse
http://terminology.hl7.org/ValueSet/v3-ServiceDeliveryLocationRoleType

and the Confidentiality one is already covered by the model (because its a required binding on a code element).

obtained via the following:

java -jar fhir-path-*-cli.jar --path "entry.resource.differential.element.where(binding.valueSet.value.startsWith('http://terminology.hl7.org') and binding.strength != 'example')" --file profiles-resources.json

@lmsurpre
Copy link
Member Author

currently we look up the CodeSystem to see if its case-sensitive or not.
additionally, we need to be able to walk the hiearchy of concepts in these codesystems in order to support the :above and :below modifiers for searches.
see related discussion at
https://chat.fhir.org/#narrow/stream/179166-implementers/topic/R4B.20questions.20.2F.20issues/near/274827413

lmsurpre added a commit that referenced this issue Mar 14, 2022
https://build.fhir.org/branches/R4B/extension-family-member-history-genetics-parent.json.html
has a required binding to
http://hl7.org/fhir/ValueSet/parent-relationship-codes

The examples are actually correct but we need to either package an
expanded version of that valueset or get to
#3448 in order to properly validate
these.

Signed-off-by: Lee Surprenant <[email protected]>
lmsurpre added a commit that referenced this issue Mar 14, 2022
https://build.fhir.org/branches/R4B/extension-family-member-history-genetics-parent.json.html
has a required binding to
http://hl7.org/fhir/ValueSet/parent-relationship-codes

The examples are actually correct but we need to either package an
expanded version of that valueset or get to
#3448 in order to properly validate
these.

Signed-off-by: Lee Surprenant <[email protected]>
lmsurpre added a commit that referenced this issue Mar 14, 2022
https://build.fhir.org/branches/R4B/extension-family-member-history-genetics-parent.json.html
has a required binding to
http://hl7.org/fhir/ValueSet/parent-relationship-codes

The examples are actually correct but we need to either package an
expanded version of that valueset or get to
#3448 in order to properly validate
these.

Signed-off-by: Lee Surprenant <[email protected]>
lmsurpre added a commit that referenced this issue Mar 14, 2022
https://build.fhir.org/branches/R4B/extension-family-member-history-genetics-parent.json.html
has a required binding to
http://hl7.org/fhir/ValueSet/parent-relationship-codes

The examples are actually correct but we need to either package an
expanded version of that valueset or get to
#3448 in order to properly validate
these.

Signed-off-by: Lee Surprenant <[email protected]>
lmsurpre added a commit that referenced this issue Apr 8, 2022
https://build.fhir.org/branches/R4B/extension-family-member-history-genetics-parent.json.html
has a required binding to
http://hl7.org/fhir/ValueSet/parent-relationship-codes

The examples are actually correct but we need to either package an
expanded version of that valueset or get to
#3448 in order to properly validate
these.

Signed-off-by: Lee Surprenant <[email protected]>
lmsurpre added a commit that referenced this issue Apr 22, 2022
https://build.fhir.org/branches/R4B/extension-family-member-history-genetics-parent.json.html
has a required binding to
http://hl7.org/fhir/ValueSet/parent-relationship-codes

The examples are actually correct but we need to either package an
expanded version of that valueset or get to
#3448 in order to properly validate
these.

Signed-off-by: Lee Surprenant <[email protected]>
lmsurpre added a commit that referenced this issue May 12, 2022
https://build.fhir.org/branches/R4B/extension-family-member-history-genetics-parent.json.html
has a required binding to
http://hl7.org/fhir/ValueSet/parent-relationship-codes

The examples are actually correct but we need to either package an
expanded version of that valueset or get to
#3448 in order to properly validate
these.

Signed-off-by: Lee Surprenant <[email protected]>
@lmsurpre
Copy link
Member Author

proposal: look for a property named "subsumedBy"

optional: instead confirm its URI is "http://hl7.org/fhir/concept-properties#parent" and the hierarchyMeaning is "IS-A"? what if we see other hierarchyMeanding instead?

decision: only handle this in the RegistryTermServiceProvider for now

@lmsurpre lmsurpre added the P2 Priority 2 - Should Have label Jun 20, 2022
@lmsurpre lmsurpre self-assigned this Jun 20, 2022
lmsurpre added a commit that referenced this issue Jun 22, 2022
1. added `hasPropertyHierarchy` and `convertToSimpleCodeSystem`
utilities to CodeSystemSupport. the conversion utility uses JGraphT to
simplify the creation of an acyclic directed graph and then traverses
that structure to build an updated CodeSystem resource with nested
concepts. this allows the rest of our CodeSystemSupport to stay the
same.

2. call `convertToSimpleCodeSystem` from
RegistryTermServiceProvider.getConcepts to convert property-based
hierarchy into the more-common nested concept flavor so that we can use
our existing CodeSystemSupport filter approach for CodeSystems in the
registry

3. added hand-crafted CodeSystem resources that test parent-to-child and
child-to-parent edges with the conversion logic

4. added tests to verify our updated expansions match the expansions
from tx.fhir.org for select ValueSets that draw from polyhierarchical
CodeSystems like v3-ActCode and v3-RoleCode

Signed-off-by: Lee Surprenant <[email protected]>
lmsurpre added a commit that referenced this issue Jun 29, 2022
#3730)

* issue #3448 - convert property-based concept hiearchies in getConcepts

1. added `hasPropertyHierarchy` and `convertToSimpleCodeSystem`
utilities to CodeSystemSupport. the conversion utility uses JGraphT to
simplify the creation of an acyclic directed graph and then traverses
that structure to build an updated CodeSystem resource with nested
concepts. this allows the rest of our CodeSystemSupport to stay the
same.

2. call `convertToSimpleCodeSystem` from
RegistryTermServiceProvider.getConcepts to convert property-based
hierarchy into the more-common nested concept flavor so that we can use
our existing CodeSystemSupport filter approach for CodeSystems in the
registry

3. added hand-crafted CodeSystem resources that test parent-to-child and
child-to-parent edges with the conversion logic

4. added tests to verify our updated expansions match the expansions
from tx.fhir.org for select ValueSets that draw from polyhierarchical
CodeSystems like v3-ActCode and v3-RoleCode

Signed-off-by: Lee Surprenant <[email protected]>

* Add fhir-hl7-terminology to fhir-parent pom

So that it gets built with the rest of the project.

I also moved the JGraphT version to the fhir-parent pom's
dependencyManagement section.

Signed-off-by: Lee Surprenant <[email protected]>

* address PR feedback

removed commented out code, added implNote, formatted test resources,
and fixed comment typos

Signed-off-by: Lee Surprenant <[email protected]>

* Apply suggestions from code review

Signed-off-by: Lee Surprenant <[email protected]>
@lmsurpre
Copy link
Member Author

lmsurpre commented Aug 4, 2022

QA:

  1. identify a valueset that has concepts from a codesystem that uses a property-based hierarchy (probably the v3 ones mentioned above like http://terminology.hl7.org/ValueSet/v3-ActConsentDirective etc)
  2. test valueset expansion on this using [base]/ValueSet/$expand and passing the canonical url of that valueset in the url parameter

@punktilious
Copy link
Collaborator

punktilious commented Aug 5, 2022

GET [base]/ValueSet/$expand?url=http://terminology.hl7.org/ValueSet/v3-ActConsentDirective'

produces the following response:

{
  "resourceType": "ValueSet",
  "id": "v3-ActConsentDirective",
  "language": "en",
  "url": "http://terminology.hl7.org/ValueSet/v3-ActConsentDirective",
  "identifier": [
    {
      "system": "urn:ietf:rfc:3986",
      "value": "urn:oid:2.16.840.1.113883.1.11.20425"
    }
  ],
  "version": "2.0.0",
  "name": "ActConsentDirective",
  "title": "ActConsentDirective",
  "status": "active",
  "date": "2014-03-26",
  "description": "ActConsentDirective codes are used to specify the type of Consent Directive to which a Consent Directive Act conforms.",
  "compose": {
    "include": [
      {
        "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
        "filter": [
          {
            "property": "concept",
            "op": "is-a",
            "value": "_ActConsentDirective"
          }
        ]
      }
    ]
  },
  "expansion": {
    "identifier": "urn:uuid:81c7242f-7317-4803-8a65-f76ebfff3d04",
    "timestamp": "2022-05-28T12:49:24+10:00",
    "total": 11,
    "parameter": [
      {
        "name": "excludeNested",
        "valueBoolean": true
      },
      {
        "name": "includeDesignations",
        "valueBoolean": true
      },
      {
        "name": "version",
        "valueUri": "http://terminology.hl7.org/CodeSystem/v3-ActCode|6.1.0"
      }
    ],
    "contains": [
      {
        "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
        "abstract": true,
        "code": "_ActConsentDirective",
        "display": "ActConsentDirective"
      },
      {
        "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
        "code": "EMRGONLY",
        "display": "emergency only"
      },
      {
        "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
        "code": "GRANTORCHOICE",
        "display": "grantor choice"
      },
      {
        "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
        "code": "IMPLIED",
        "display": "implied consent"
      },
      {
        "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
        "code": "IMPLIEDD",
        "display": "implied consent with opportunity to dissent"
      },
      {
        "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
        "code": "NOCONSENT",
        "display": "no consent"
      },
      {
        "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
        "code": "NOPP",
        "display": "notice of privacy practices"
      },
      {
        "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
        "code": "OPTIN",
        "display": "opt-in"
      },
      {
        "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
        "code": "OPTINR",
        "display": "opt-in with restrictions"
      },
      {
        "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
        "code": "OPTOUT",
        "display": "op-out"
      },
      {
        "system": "http://terminology.hl7.org/CodeSystem/v3-ActCode",
        "code": "OPTOUTE",
        "display": "opt-out with exceptions"
      }
    ]
  }
}

We see that ValueSet-v3-ActConsentDirective is composed of http://terminology.hl7.org/CodeSystem/v3-ActCode. If we look at CodeSystem-v3-ActCode.json and scan for codes with a subsumedBy property valueCode of _ActConsentDirective we get the following list:

EMRGONLY, GRANTORCHOICE, IMPLIED, IMPLIEDD, NOCONSENT, NOPP, OPTIN, OPTINR, OPTOUT, OPTOUTE

which matches the above expansion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request P2 Priority 2 - Should Have
Projects
None yet
Development

No branches or pull requests

2 participants