Feature/show all metrics and table cells (#798)

* filling in the empty metrics Signed-off-by: Tynan DeBold <[email protected]> * cleanup; add comments; update one run for testing Signed-off-by: Tynan DeBold <[email protected]> * fix failing test; update caniuse Signed-off-by: Tynan DeBold <[email protected]> * formatting; update release notes Signed-off-by: Tynan DeBold <[email protected]> * PR review additions Signed-off-by: Tynan DeBold <[email protected]> * add test for showDiff and diverging metrics Signed-off-by: Tynan DeBold <[email protected]> * true to false on a test Signed-off-by: Tynan DeBold <[email protected]> * make the test more robust Signed-off-by: Tynan DeBold <[email protected]> * update release notes PR number Signed-off-by: Tynan DeBold <[email protected]>
kedro-org · Mar 30, 2022 · 7645039 · 7645039
1 parent 96001ac
commit 7645039
Show file tree

Hide file tree

Showing 10 changed files with 144 additions and 55 deletions.
diff --git a/README.md b/README.md
@@ -25,7 +25,7 @@ Live Demo: <a href="https://demo.kedro.org/" target="_blank">https://demo.kedro.
 
 ## Introduction
 
-Kedro-Viz is an interactive development tool for building data science pipelines with [Kedro](https://github.com/kedro-org/kedro). Kedro-Viz also allows users to view and compare different runs in the Kedro project. 
+Kedro-Viz is an interactive development tool for building data science pipelines with [Kedro](https://github.com/kedro-org/kedro). Kedro-Viz also allows users to view and compare different runs in the Kedro project.
 
 ## Features
 
@@ -39,26 +39,26 @@ Kedro-Viz is an interactive development tool for building data science pipelines
 - 🧪 Supports tracking and comparing runs in a Kedro project
 - 🎩 Many more to come
 
-
 ## Installation
 
 There are two ways you can use Kedro-Viz:
 
-* As a [Kedro plugin](https://kedro.readthedocs.io/en/stable/07_extend_kedro/04_plugins.html) (the most common way).
+- As a [Kedro plugin](https://kedro.readthedocs.io/en/stable/07_extend_kedro/04_plugins.html) (the most common way).
+
+  To install Kedro-Viz as a Kedro plugin:
 
-    To install Kedro-Viz as a Kedro plugin:
+  ```bash
+  pip install kedro-viz
+  ```
 
-    ```bash
-    pip install kedro-viz
-    ```
+- As a standalone React component (for embedding Kedro-Viz in your web application).
 
-* As a standalone React component (for embedding Kedro-Viz in your web application).
+  To install the standalone React component:
 
-   To install the standalone React component:
+  ```bash
+  npm install @quantumblack/kedro-viz
+  ```
 
-    ```bash
-    npm install @quantumblack/kedro-viz
-    ```
 ## Usage
 
 ### CLI Usage
@@ -107,7 +107,7 @@ Options:
 
 To enable [experiment tracking](https://kedro.readthedocs.io/en/stable/08_logging/02_experiment_tracking.html) in Kedro-Viz, you need to add the Kedro-Viz `SQLiteStore` to your Kedro project.
 
-This can be done by adding the below code to `settings.py` in the `src` folder of your Kedro project. 
+This can be done by adding the below code to `settings.py` in the `src` folder of your Kedro project.
 
 ```python
 from kedro_viz.integrations.kedro.sqlite_store import SQLiteStore
@@ -120,8 +120,8 @@ Once the above set-up is complete, tracking datasets can be used to track releva
 
 **Notes:**
 
-* Experiment Tracking is only available for Kedro-Viz >= 4.0.2 and Kedro >= 0.17.5
-* Prior to Kedro 0.17.6, when using tracking datasets, you will have to explicitly mark the datasets as `versioned` for it to show up properly in Kedro-Viz experiment tracking tab. From Kedro >= 0.17.6, this is done automatically:
+- Experiment Tracking is only available for Kedro-Viz >= 4.0.2 and Kedro >= 0.17.5
+- Prior to Kedro 0.17.6, when using tracking datasets, you will have to explicitly mark the datasets as `versioned` for it to show up properly in Kedro-Viz experiment tracking tab. From Kedro >= 0.17.6, this is done automatically:
 
 ```yaml
 train_evaluation.r2_score_linear_regression:
@@ -138,12 +138,10 @@ To use Kedro-Viz as a standalone React component, import the component and suppl
 import KedroViz from '@quantumblack/kedro-viz';
 
 const MyApp = () => (
-  <div style={{ height: "100vh" }}>
-    <KedroViz
-      data={json}
-    />
+  <div style={{ height: '100vh' }}>
+    <KedroViz data={json} />
   </div>
-)
+);
 ```
 
 The JSON can be obtained by running:
@@ -158,8 +156,8 @@ We also recommend wrapping the `Kedro-Viz` component with a parent HTML/JSX elem
 
 Kedro-Viz uses features flags to roll out some experimental features. The following flags are currently in use:
 
-| Flag | Description |
-|------| ------------|
+| Flag        | Description                                                                             |
+| ----------- | --------------------------------------------------------------------------------------- |
 | sizewarning | From release v3.9.1. Show a warning before rendering very large graphs (default `true`) |
 | expandAllPipelines | From release v4.3.2. Expand all modular pipelines on first load (default `false`) |
 

diff --git a/RELEASE.md b/RELEASE.md
@@ -9,6 +9,7 @@ Please follow the established format:
 -->
 
 ## Major features and improvements
+
 - Set up of a pop-up reminder to nudge users to upgrade Kedro-Viz when a new version is released. (#746)
 - Set up the 'export run' button to allow exporting of selected run data into a csv file for download. (#757)
 - Set up new display props to standalone React component. (#786)
@@ -22,6 +23,8 @@ Please follow the established format:
 - Create a `version` GraphQL query to get versions of Kedro-Viz. (#727)
 - Fix Kedro-Viz to work with projects that have no `__default__` registered pipeline. This also fixes the `--pipeline` CLI option. (#729)
 - Fix lazy pipelines loading causes `get_current_session` to throw an error. (#726, #727)
+- Fix experiment tracking not showing all metrics. (#798)
+- Fix experiment tracking not display the correct empty table cells. (#798)
 
 # Release 4.3.1
 

diff --git a/demo-project/data/09_tracking/rf_params.json/2021-12-16T09.45.55.269Z/rf_params.json b/demo-project/data/09_tracking/rf_params.json/2021-12-16T09.45.55.269Z/rf_params.json
@@ -11,5 +11,6 @@
   "oob_score": false,
   "verbose": 0,
   "warm_start": false,
-  "ccp_alpha": 0
-}
+  "ccp_alpha": 0,
+  "model_author": "richard feynman"
+}
diff --git a/package-lock.json b/package-lock.json
diff --git a/package.json b/package.json
@@ -39,6 +39,7 @@
   },
   "dependencies": {
     "@apollo/client": "^3.5.6",
+    "@faker-js/faker": "^6.0.0-beta.0",
     "@graphql-tools/schema": "7.1.5",
     "@material-ui/core": "^4.11.4",
     "@material-ui/icons": "^4.11.2",
@@ -55,7 +56,6 @@
     "d3-zoom": "^2.0.0",
     "dayjs": "^1.10.7",
     "deepmerge": "^4.2.2",
-    "@faker-js/faker": "^6.0.0-beta.0",
     "fetch-mock": "^9.11.0",
     "fishery": "^1.4.0",
     "graphql": "^15.8.0",

diff --git a/src/components/experiment-tracking/details/details.js b/src/components/experiment-tracking/details/details.js
@@ -12,15 +12,15 @@ const Details = ({
   metadataError,
   onRunSelection,
   pinnedRun,
+  runMetadata,
+  runTrackingData,
   selectedRunIds,
   setPinnedRun,
   setShowRunDetailsModal,
   showRunDetailsModal,
   sidebarVisible,
   theme,
   trackingDataError,
-  runMetadata,
-  runTrackingData,
 }) => {
   const [runMetadataToEdit, setRunMetadataToEdit] = useState(null);
 
@@ -66,6 +66,7 @@ const Details = ({
           enableShowChanges={enableShowChanges}
           isSingleRun={isSingleRun}
           pinnedRun={pinnedRun}
+          selectedRunIds={selectedRunIds}
           trackingData={runTrackingData}
         />
       </div>

diff --git a/src/components/experiment-tracking/run-dataset/run-dataset.js b/src/components/experiment-tracking/run-dataset/run-dataset.js
@@ -33,14 +33,18 @@ const resolveRunDataWithPin = (runData, pinnedRun) => {
 
 /**
  * Display the dataset of the experiment tracking run.
- * @param {array} props.isSingleRun Whether or not this is a single run.
+ * @param {boolean} props.enableShowChanges Are changes enabled or not.
+ * @param {boolean} props.isSingleRun Indication to display a single run.
+ * @param {string} props.pinnedRun ID of the pinned run.
+ * @param {array} props.selectedRunIds Array of strings of runIds.
  * @param {array} props.trackingData The experiment tracking run data.
  */
 const RunDataset = ({
+  enableShowChanges,
   isSingleRun,
-  trackingData = [],
   pinnedRun,
-  enableShowChanges,
+  selectedRunIds,
+  trackingData = [],
 }) => {
   return (
     <div
@@ -61,15 +65,18 @@ const RunDataset = ({
             size="large"
           >
             {Object.keys(data)
-              .sort()
+              .sort((a, b) => {
+                return a.localeCompare(b);
+              })
               .map((key, rowIndex) => {
                 return buildDatasetDataMarkup(
                   key,
                   dataset.data[key],
                   rowIndex,
                   isSingleRun,
                   pinnedRun,
-                  enableShowChanges
+                  enableShowChanges,
+                  selectedRunIds
                 );
               })}
           </Accordion>
@@ -85,17 +92,21 @@ const RunDataset = ({
  * @param {array} datasetValues A single dataset array from a run.
  * @param {number} rowIndex The array index of the dataset data.
  * @param {boolean} isSingleRun Whether or not this is a single run.
+ * @param {string} pinnedRun ID of the pinned run.
+ * @param {boolean} enableShowChanges Are changes enabled or not.
+ * @param {array} selectedRunIds Array of strings of runIds.
  */
 function buildDatasetDataMarkup(
   datasetKey,
   datasetValues,
   rowIndex,
   isSingleRun,
   pinnedRun,
-  enableShowChanges
+  enableShowChanges,
+  selectedRunIds
 ) {
-  // function to return new set of runData with appropriate pin from datasetValues and pinnedRun
-  const runDataWithPin = resolveRunDataWithPin(datasetValues, pinnedRun);
+  const updatedDatasetValues = fillEmptyMetrics(datasetValues, selectedRunIds);
+  const runDataWithPin = resolveRunDataWithPin(updatedDatasetValues, pinnedRun);
 
   return (
     <React.Fragment key={datasetKey + rowIndex}>
@@ -144,4 +155,34 @@ function buildDatasetDataMarkup(
   );
 }
 
+/**
+ * Fill in missing run metrics if they don't match the number of runIds.
+ * @param {array} datasetValues Array of objects for a metric, e.g. r2_score.
+ * @param {array} selectedRunIds Array of strings of runIds.
+ * @returns Array of objects, the length of which matches the length
+ * of the selectedRunIds.
+ */
+function fillEmptyMetrics(datasetValues, selectedRunIds) {
+  if (datasetValues.length === selectedRunIds.length) {
+    return datasetValues;
+  }
+
+  const metrics = [];
+
+  selectedRunIds.forEach((id) => {
+    const foundIdIndex = datasetValues.findIndex((item) => {
+      return item.runId === id;
+    });
+
+    // We didn't find a metric with this runId, so add a placeholder.
+    if (foundIdIndex === -1) {
+      metrics.push({ runId: id, value: null });
+    } else {
+      metrics.push(datasetValues[foundIdIndex]);
+    }
+  });
+
+  return metrics;
+}
+
 export default RunDataset;
diff --git a/src/components/experiment-tracking/run-dataset/run-dataset.test.js b/src/components/experiment-tracking/run-dataset/run-dataset.test.js
@@ -12,7 +12,7 @@ const booleanTrackingData = [
   },
 ];
 
-const ObjectTrackingData = [
+const objectTrackingData = [
   {
     datasetName: 'Data Analysis',
     data: {
@@ -21,7 +21,7 @@ const ObjectTrackingData = [
   },
 ];
 
-const ComparisonTrackingData = [
+const comparisonTrackingData = [
   {
     datasetName: 'Data Analysis',
     data: {
@@ -33,11 +33,25 @@ const ComparisonTrackingData = [
   },
 ];
 
+const showDiffTrackingData = [
+  {
+    datasetName: 'Data Analysis',
+    data: {
+      classWeight: [
+        { runId: 'My Favorite Sprint', value: 12 },
+        { runId: 'My second Favorite Sprint', value: 13 },
+      ],
+      r2Score: [{ runId: 'My second Favorite Sprint', value: 0.2342356 }],
+    },
+  },
+];
+
 describe('RunDataset', () => {
   it('renders without crashing', () => {
     const wrapper = shallow(
       <RunDataset
         isSingleRun={runs.length === 1 ? true : false}
+        selectedRunIds={['abc']}
         trackingData={trackingData}
       />
     );
@@ -48,15 +62,23 @@ describe('RunDataset', () => {
 
   it('renders a boolean value as a string', () => {
     const wrapper = mount(
-      <RunDataset isSingleRun={true} trackingData={booleanTrackingData} />
+      <RunDataset
+        isSingleRun={true}
+        selectedRunIds={['abc']}
+        trackingData={booleanTrackingData}
+      />
     );
 
     expect(wrapper.find('.details-dataset__value').text()).toBe('false');
   });
 
   it('renders a boolean value as a string', () => {
     const wrapper = mount(
-      <RunDataset isSingleRun={true} trackingData={ObjectTrackingData} />
+      <RunDataset
+        isSingleRun={true}
+        selectedRunIds={['abc']}
+        trackingData={objectTrackingData}
+      />
     );
 
     const datasetValue = wrapper.find('.details-dataset__value').text();
@@ -67,13 +89,28 @@ describe('RunDataset', () => {
   it('renders the comparison arrow when showChanges is on', () => {
     const wrapper = mount(
       <RunDataset
-        isSingleRun={false}
-        trackingData={ComparisonTrackingData}
         enableShowChanges={true}
+        isSingleRun={false}
         pinnedRun={'My Favorite Sprint'}
+        selectedRunIds={['abc', 'def']}
+        trackingData={comparisonTrackingData}
       />
     );
 
     expect(wrapper.find('.dataset-arrow-icon').length).toBe(1);
   });
+
+  it('for runs with different metrics, it renders a cell with a - value', () => {
+    const wrapper = mount(
+      <RunDataset
+        isSingleRun={false}
+        selectedRunIds={['My Favorite Sprint', 'My second Favorite Sprint']}
+        trackingData={showDiffTrackingData}
+      />
+    );
+
+    console.log(wrapper.debug());
+
+    expect(wrapper.find('.details-dataset__value').at(2).text()).toBe('-');
+  });
 });