feat(ingestion): business-glossary - Add values and relatedTerms supp…

…ort (datahub-project#6148)

* WIP

* updated snapshot

* merge

* while creating BusinessGlossaryConfig object, the version field always get set to None

* WIP

* lint fix

* remove field

* doc update

* rename the fields

* review comments

* lintFix

* Fix the business glossary example link

* small fixes to examples and docs

* some small tweaks to the model annotations

Co-authored-by: MohdSiddique Bagwan <[email protected]>
Co-authored-by: Shirshanka Das <[email protected]>
Co-authored-by: Gabe Lyons <[email protected]>

metadata-ingestion/docs/sources/business-glossary/datahub-business-glossary.md

@@ -16,6 +16,7 @@ The business glossary source file should be a `.yml` file with the following top
 **GlossaryNode**: a container of **GlossaryNode** and **GlossaryTerm** objects
 - **name**: name of the node
 - **description**: description of the node
+- **id**: (optional) identifier of the node (normally inferred from the name, see `enable_auto_id` config. Use this if you need a stable identifier)
 - **owners**: (optional) owners contains two nested fields
   - **users**: (optional) a list of user ids
   - **groups**: (optional) a list of group ids
@@ -25,6 +26,7 @@ The business glossary source file should be a `.yml` file with the following top
 **GlossaryTerm**: a term in your business glossary
 - **name**: name of the term
 - **description**: description of the term
+- **id**: (optional) identifier of the term (normally inferred from the name, see `enable_auto_id` config. Use this if you need a stable identifier)
 - **owners**: (optional) owners contains two nested fields
   - **users**: (optional) a list of user ids
   - **groups**: (optional) a list of group ids
@@ -35,7 +37,7 @@ The business glossary source file should be a `.yml` file with the following top
 - **contains**: (optional) List of **GlossaryTerm** that this term contains
 - **custom_properties**: A map of key/value pairs of arbitrary custom properties
 
-You can also view an example business glossary file checked in [here](../examples/bootstrap_data/business_glossary.yml)
+You can also view an example business glossary file checked in [here](../../../examples/bootstrap_data/business_glossary.yml)
 
 ## Compatibility
 

metadata-ingestion/docs/sources/business-glossary/datahub-business-glossary_recipe.yml

@@ -3,6 +3,6 @@ source:
   config:
     # Coordinates
     file: /path/to/business_glossary_yaml
+    enable_auto_id: true # recommended to set to true so datahub will auto-generate guids from your term names
 
-sink:
-  # sink configs
+# sink configs if needed

metadata-ingestion/examples/bootstrap_data/business_glossary.yml

@@ -27,6 +27,8 @@ nodes:
         - mjames
     terms:
       - name: Email
+        ## An example of using an id to pin a term to a specific guid
+        # id: "urn:li:glossaryTerm:41516e310acbfd9076fffc2c98d2d1a3"
         description: An individual's email address
         inherits:
           - Classification.Confidential
@@ -59,3 +61,52 @@ nodes:
         term_source: "EXTERNAL"
         source_ref: FIBO
         source_url: "https://spec.edmcouncil.org/fibo/ontology/FBC/ProductsAndServices/ClientsAndAccounts/Balance"
+  - name: Housing
+    description: Provides terms related to the housing domain
+    owners:
+      groups:
+        - interior
+    nodes:
+      - name: Colors
+        description: "Colors that are used in Housing construction"
+        terms:
+          - name: Red
+            description: "red color"
+            term_source: "EXTERNAL"
+            source_ref: FIBO
+            source_url: "https://spec.edmcouncil.org/fibo/ontology/FBC/ProductsAndServices/ClientsAndAccounts/Account"
+
+          - name: Green
+            description: "green color"
+            term_source: "EXTERNAL"
+            source_ref: FIBO
+            source_url: "https://spec.edmcouncil.org/fibo/ontology/FBC/ProductsAndServices/ClientsAndAccounts/Account"
+
+          - name: Pink
+            description: pink color
+            term_source: "EXTERNAL"
+            source_ref: FIBO
+            source_url: "https://spec.edmcouncil.org/fibo/ontology/FBC/ProductsAndServices/ClientsAndAccounts/Account"
+    terms:
+      - name: WindowColor
+        description: Supported window colors
+        term_source: "EXTERNAL"
+        source_ref: FIBO
+        source_url: "https://spec.edmcouncil.org/fibo/ontology/FBC/ProductsAndServices/ClientsAndAccounts/Account"
+        values:
+          - House.Colors.Red
+          - House.Colors.Pink
+
+      - name: Kitchen
+        description: a room or area where food is prepared and cooked.
+        term_source: "EXTERNAL"
+        source_ref: FIBO
+        source_url: "https://spec.edmcouncil.org/fibo/ontology/FBC/ProductsAndServices/ClientsAndAccounts/Account"
+
+      - name: Spoon
+        description: an implement consisting of a small, shallow oval or round bowl on a long handle, used for eating, stirring, and serving food.
+        term_source: "EXTERNAL"
+        source_ref: FIBO
+        source_url: "https://spec.edmcouncil.org/fibo/ontology/FBC/ProductsAndServices/ClientsAndAccounts/Account"
+        related_terms:
+          - House.Kitchen

metadata-ingestion/examples/recipes/business_glossary_to_datahub.dhub.yaml

@@ -2,10 +2,9 @@ source:
   type: datahub-business-glossary
   config:
     file: ./examples/bootstrap_data/business_glossary.yml
-
+    enable_auto_id: True
 
 sink:
-  type: datahub-rest 
+  type: datahub-rest
   config:
     server: http://localhost:8080
-