From 7645039ed77fed2ae774203a2fa1c498c00e1a08 Mon Sep 17 00:00:00 2001 From: Tynan DeBold Date: Wed, 30 Mar 2022 17:30:09 +0100 Subject: [PATCH] Feature/show all metrics and table cells (#798) * filling in the empty metrics Signed-off-by: Tynan DeBold * cleanup; add comments; update one run for testing Signed-off-by: Tynan DeBold * fix failing test; update caniuse Signed-off-by: Tynan DeBold * formatting; update release notes Signed-off-by: Tynan DeBold * PR review additions Signed-off-by: Tynan DeBold * add test for showDiff and diverging metrics Signed-off-by: Tynan DeBold * true to false on a test Signed-off-by: Tynan DeBold * make the test more robust Signed-off-by: Tynan DeBold * update release notes PR number Signed-off-by: Tynan DeBold --- README.md | 42 +++++++------- RELEASE.md | 3 + .../2021-12-16T09.45.55.269Z/rf_params.json | 5 +- package-lock.json | 9 +-- package.json | 2 +- .../experiment-tracking/details/details.js | 5 +- .../run-dataset/run-dataset.js | 57 ++++++++++++++++--- .../run-dataset/run-dataset.test.js | 49 ++++++++++++++-- .../experiment-wrapper/experiment-wrapper.js | 16 +++--- src/components/ui/button/button.scss | 11 +++- 10 files changed, 144 insertions(+), 55 deletions(-) diff --git a/README.md b/README.md index 7884e3af8c..ef0bf16bbd 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ Live Demo: https://demo.kedro. ## Introduction -Kedro-Viz is an interactive development tool for building data science pipelines with [Kedro](https://github.com/kedro-org/kedro). Kedro-Viz also allows users to view and compare different runs in the Kedro project. +Kedro-Viz is an interactive development tool for building data science pipelines with [Kedro](https://github.com/kedro-org/kedro). Kedro-Viz also allows users to view and compare different runs in the Kedro project. ## Features @@ -39,26 +39,26 @@ Kedro-Viz is an interactive development tool for building data science pipelines - 🧪 Supports tracking and comparing runs in a Kedro project - 🎩 Many more to come - ## Installation There are two ways you can use Kedro-Viz: -* As a [Kedro plugin](https://kedro.readthedocs.io/en/stable/07_extend_kedro/04_plugins.html) (the most common way). +- As a [Kedro plugin](https://kedro.readthedocs.io/en/stable/07_extend_kedro/04_plugins.html) (the most common way). + + To install Kedro-Viz as a Kedro plugin: - To install Kedro-Viz as a Kedro plugin: + ```bash + pip install kedro-viz + ``` - ```bash - pip install kedro-viz - ``` +- As a standalone React component (for embedding Kedro-Viz in your web application). -* As a standalone React component (for embedding Kedro-Viz in your web application). + To install the standalone React component: - To install the standalone React component: + ```bash + npm install @quantumblack/kedro-viz + ``` - ```bash - npm install @quantumblack/kedro-viz - ``` ## Usage ### CLI Usage @@ -107,7 +107,7 @@ Options: To enable [experiment tracking](https://kedro.readthedocs.io/en/stable/08_logging/02_experiment_tracking.html) in Kedro-Viz, you need to add the Kedro-Viz `SQLiteStore` to your Kedro project. -This can be done by adding the below code to `settings.py` in the `src` folder of your Kedro project. +This can be done by adding the below code to `settings.py` in the `src` folder of your Kedro project. ```python from kedro_viz.integrations.kedro.sqlite_store import SQLiteStore @@ -120,8 +120,8 @@ Once the above set-up is complete, tracking datasets can be used to track releva **Notes:** -* Experiment Tracking is only available for Kedro-Viz >= 4.0.2 and Kedro >= 0.17.5 -* Prior to Kedro 0.17.6, when using tracking datasets, you will have to explicitly mark the datasets as `versioned` for it to show up properly in Kedro-Viz experiment tracking tab. From Kedro >= 0.17.6, this is done automatically: +- Experiment Tracking is only available for Kedro-Viz >= 4.0.2 and Kedro >= 0.17.5 +- Prior to Kedro 0.17.6, when using tracking datasets, you will have to explicitly mark the datasets as `versioned` for it to show up properly in Kedro-Viz experiment tracking tab. From Kedro >= 0.17.6, this is done automatically: ```yaml train_evaluation.r2_score_linear_regression: @@ -138,12 +138,10 @@ To use Kedro-Viz as a standalone React component, import the component and suppl import KedroViz from '@quantumblack/kedro-viz'; const MyApp = () => ( -
- +
+
-) +); ``` The JSON can be obtained by running: @@ -158,8 +156,8 @@ We also recommend wrapping the `Kedro-Viz` component with a parent HTML/JSX elem Kedro-Viz uses features flags to roll out some experimental features. The following flags are currently in use: -| Flag | Description | -|------| ------------| +| Flag | Description | +| ----------- | --------------------------------------------------------------------------------------- | | sizewarning | From release v3.9.1. Show a warning before rendering very large graphs (default `true`) | | expandAllPipelines | From release v4.3.2. Expand all modular pipelines on first load (default `false`) | diff --git a/RELEASE.md b/RELEASE.md index 871ff594c8..9119d13d4f 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -9,6 +9,7 @@ Please follow the established format: --> ## Major features and improvements + - Set up of a pop-up reminder to nudge users to upgrade Kedro-Viz when a new version is released. (#746) - Set up the 'export run' button to allow exporting of selected run data into a csv file for download. (#757) - Set up new display props to standalone React component. (#786) @@ -22,6 +23,8 @@ Please follow the established format: - Create a `version` GraphQL query to get versions of Kedro-Viz. (#727) - Fix Kedro-Viz to work with projects that have no `__default__` registered pipeline. This also fixes the `--pipeline` CLI option. (#729) - Fix lazy pipelines loading causes `get_current_session` to throw an error. (#726, #727) +- Fix experiment tracking not showing all metrics. (#798) +- Fix experiment tracking not display the correct empty table cells. (#798) # Release 4.3.1 diff --git a/demo-project/data/09_tracking/rf_params.json/2021-12-16T09.45.55.269Z/rf_params.json b/demo-project/data/09_tracking/rf_params.json/2021-12-16T09.45.55.269Z/rf_params.json index 3453d7f039..ad0bc6efb5 100644 --- a/demo-project/data/09_tracking/rf_params.json/2021-12-16T09.45.55.269Z/rf_params.json +++ b/demo-project/data/09_tracking/rf_params.json/2021-12-16T09.45.55.269Z/rf_params.json @@ -11,5 +11,6 @@ "oob_score": false, "verbose": 0, "warm_start": false, - "ccp_alpha": 0 -} \ No newline at end of file + "ccp_alpha": 0, + "model_author": "richard feynman" +} diff --git a/package-lock.json b/package-lock.json index 2e795a37e1..4db36ea142 100644 --- a/package-lock.json +++ b/package-lock.json @@ -5561,12 +5561,9 @@ } }, "caniuse-lite": { - "version": "1.0.30001260", - "resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001260.tgz", - "integrity": "sha512-Fhjc/k8725ItmrvW5QomzxLeojewxvqiYCKeFcfFEhut28IVLdpHU19dneOmltZQIE5HNbawj1HYD+1f2bM1Dg==", - "requires": { - "nanocolors": "^0.1.0" - } + "version": "1.0.30001322", + "resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001322.tgz", + "integrity": "sha512-neRmrmIrCGuMnxGSoh+x7zYtQFFgnSY2jaomjU56sCkTA6JINqQrxutF459JpWcWRajvoyn95sOXq4Pqrnyjew==" }, "canvas": { "version": "2.8.0", diff --git a/package.json b/package.json index 40d037aa7a..d55311002b 100644 --- a/package.json +++ b/package.json @@ -39,6 +39,7 @@ }, "dependencies": { "@apollo/client": "^3.5.6", + "@faker-js/faker": "^6.0.0-beta.0", "@graphql-tools/schema": "7.1.5", "@material-ui/core": "^4.11.4", "@material-ui/icons": "^4.11.2", @@ -55,7 +56,6 @@ "d3-zoom": "^2.0.0", "dayjs": "^1.10.7", "deepmerge": "^4.2.2", - "@faker-js/faker": "^6.0.0-beta.0", "fetch-mock": "^9.11.0", "fishery": "^1.4.0", "graphql": "^15.8.0", diff --git a/src/components/experiment-tracking/details/details.js b/src/components/experiment-tracking/details/details.js index fb4e96dbf8..6508078ee8 100644 --- a/src/components/experiment-tracking/details/details.js +++ b/src/components/experiment-tracking/details/details.js @@ -12,6 +12,8 @@ const Details = ({ metadataError, onRunSelection, pinnedRun, + runMetadata, + runTrackingData, selectedRunIds, setPinnedRun, setShowRunDetailsModal, @@ -19,8 +21,6 @@ const Details = ({ sidebarVisible, theme, trackingDataError, - runMetadata, - runTrackingData, }) => { const [runMetadataToEdit, setRunMetadataToEdit] = useState(null); @@ -66,6 +66,7 @@ const Details = ({ enableShowChanges={enableShowChanges} isSingleRun={isSingleRun} pinnedRun={pinnedRun} + selectedRunIds={selectedRunIds} trackingData={runTrackingData} />
diff --git a/src/components/experiment-tracking/run-dataset/run-dataset.js b/src/components/experiment-tracking/run-dataset/run-dataset.js index 84f41a5357..2f9bd701e8 100644 --- a/src/components/experiment-tracking/run-dataset/run-dataset.js +++ b/src/components/experiment-tracking/run-dataset/run-dataset.js @@ -33,14 +33,18 @@ const resolveRunDataWithPin = (runData, pinnedRun) => { /** * Display the dataset of the experiment tracking run. - * @param {array} props.isSingleRun Whether or not this is a single run. + * @param {boolean} props.enableShowChanges Are changes enabled or not. + * @param {boolean} props.isSingleRun Indication to display a single run. + * @param {string} props.pinnedRun ID of the pinned run. + * @param {array} props.selectedRunIds Array of strings of runIds. * @param {array} props.trackingData The experiment tracking run data. */ const RunDataset = ({ + enableShowChanges, isSingleRun, - trackingData = [], pinnedRun, - enableShowChanges, + selectedRunIds, + trackingData = [], }) => { return (
{Object.keys(data) - .sort() + .sort((a, b) => { + return a.localeCompare(b); + }) .map((key, rowIndex) => { return buildDatasetDataMarkup( key, @@ -69,7 +75,8 @@ const RunDataset = ({ rowIndex, isSingleRun, pinnedRun, - enableShowChanges + enableShowChanges, + selectedRunIds ); })} @@ -85,6 +92,9 @@ const RunDataset = ({ * @param {array} datasetValues A single dataset array from a run. * @param {number} rowIndex The array index of the dataset data. * @param {boolean} isSingleRun Whether or not this is a single run. + * @param {string} pinnedRun ID of the pinned run. + * @param {boolean} enableShowChanges Are changes enabled or not. + * @param {array} selectedRunIds Array of strings of runIds. */ function buildDatasetDataMarkup( datasetKey, @@ -92,10 +102,11 @@ function buildDatasetDataMarkup( rowIndex, isSingleRun, pinnedRun, - enableShowChanges + enableShowChanges, + selectedRunIds ) { - // function to return new set of runData with appropriate pin from datasetValues and pinnedRun - const runDataWithPin = resolveRunDataWithPin(datasetValues, pinnedRun); + const updatedDatasetValues = fillEmptyMetrics(datasetValues, selectedRunIds); + const runDataWithPin = resolveRunDataWithPin(updatedDatasetValues, pinnedRun); return ( @@ -144,4 +155,34 @@ function buildDatasetDataMarkup( ); } +/** + * Fill in missing run metrics if they don't match the number of runIds. + * @param {array} datasetValues Array of objects for a metric, e.g. r2_score. + * @param {array} selectedRunIds Array of strings of runIds. + * @returns Array of objects, the length of which matches the length + * of the selectedRunIds. + */ +function fillEmptyMetrics(datasetValues, selectedRunIds) { + if (datasetValues.length === selectedRunIds.length) { + return datasetValues; + } + + const metrics = []; + + selectedRunIds.forEach((id) => { + const foundIdIndex = datasetValues.findIndex((item) => { + return item.runId === id; + }); + + // We didn't find a metric with this runId, so add a placeholder. + if (foundIdIndex === -1) { + metrics.push({ runId: id, value: null }); + } else { + metrics.push(datasetValues[foundIdIndex]); + } + }); + + return metrics; +} + export default RunDataset; diff --git a/src/components/experiment-tracking/run-dataset/run-dataset.test.js b/src/components/experiment-tracking/run-dataset/run-dataset.test.js index 6c4fdf2b4a..4fa142c457 100644 --- a/src/components/experiment-tracking/run-dataset/run-dataset.test.js +++ b/src/components/experiment-tracking/run-dataset/run-dataset.test.js @@ -12,7 +12,7 @@ const booleanTrackingData = [ }, ]; -const ObjectTrackingData = [ +const objectTrackingData = [ { datasetName: 'Data Analysis', data: { @@ -21,7 +21,7 @@ const ObjectTrackingData = [ }, ]; -const ComparisonTrackingData = [ +const comparisonTrackingData = [ { datasetName: 'Data Analysis', data: { @@ -33,11 +33,25 @@ const ComparisonTrackingData = [ }, ]; +const showDiffTrackingData = [ + { + datasetName: 'Data Analysis', + data: { + classWeight: [ + { runId: 'My Favorite Sprint', value: 12 }, + { runId: 'My second Favorite Sprint', value: 13 }, + ], + r2Score: [{ runId: 'My second Favorite Sprint', value: 0.2342356 }], + }, + }, +]; + describe('RunDataset', () => { it('renders without crashing', () => { const wrapper = shallow( ); @@ -48,7 +62,11 @@ describe('RunDataset', () => { it('renders a boolean value as a string', () => { const wrapper = mount( - + ); expect(wrapper.find('.details-dataset__value').text()).toBe('false'); @@ -56,7 +74,11 @@ describe('RunDataset', () => { it('renders a boolean value as a string', () => { const wrapper = mount( - + ); const datasetValue = wrapper.find('.details-dataset__value').text(); @@ -67,13 +89,28 @@ describe('RunDataset', () => { it('renders the comparison arrow when showChanges is on', () => { const wrapper = mount( ); expect(wrapper.find('.dataset-arrow-icon').length).toBe(1); }); + + it('for runs with different metrics, it renders a cell with a - value', () => { + const wrapper = mount( + + ); + + console.log(wrapper.debug()); + + expect(wrapper.find('.details-dataset__value').at(2).text()).toBe('-'); + }); }); diff --git a/src/components/experiment-wrapper/experiment-wrapper.js b/src/components/experiment-wrapper/experiment-wrapper.js index 6361cfdec9..96a3dba283 100644 --- a/src/components/experiment-wrapper/experiment-wrapper.js +++ b/src/components/experiment-wrapper/experiment-wrapper.js @@ -14,7 +14,7 @@ import Sidebar from '../sidebar'; import './experiment-wrapper.css'; -const MAX_NUMBER_COMPARISONS = 2; // 0-based, so three +const MAX_NUMBER_COMPARISONS = 2; // 0-based, so three. const ExperimentWrapper = ({ theme }) => { const [disableRunSelection, setDisableRunSelection] = useState(false); @@ -26,9 +26,10 @@ const ExperimentWrapper = ({ theme }) => { const [selectedRunData, setSelectedRunData] = useState(null); const [showRunDetailsModal, setShowRunDetailsModal] = useState(false); + // Fetch all runs. const { subscribeToMore, data, loading } = useApolloQuery(GET_RUNS); - // Fetch all metadata and tracking data from selected runs + // Fetch all metadata for selected runs. const { data: { runMetadata } = [], metadataError } = useApolloQuery( GET_RUN_METADATA, { @@ -37,10 +38,11 @@ const ExperimentWrapper = ({ theme }) => { } ); + // Fetch all tracking data for selected runs. const { data: { runTrackingData } = [], error: trackingDataError } = useApolloQuery(GET_RUN_TRACKING_DATA, { skip: selectedRunIds.length === 0, - variables: { runIds: selectedRunIds, showDiff: false }, + variables: { runIds: selectedRunIds, showDiff: true }, }); const onRunSelection = (id) => { @@ -163,8 +165,8 @@ const ExperimentWrapper = ({ theme }) => { isExperimentView onRunSelection={onRunSelection} onToggleComparisonView={onToggleComparisonView} - runsListData={data.runsList} runMetadata={runMetadata} + runsListData={data.runsList} runTrackingData={runTrackingData} selectedRunData={selectedRunData} selectedRunIds={selectedRunIds} @@ -177,9 +179,11 @@ const ExperimentWrapper = ({ theme }) => {
1} - onRunSelection={onRunSelection} metadataError={metadataError} + onRunSelection={onRunSelection} pinnedRun={pinnedRun} + runMetadata={runMetadata} + runTrackingData={runTrackingData} selectedRunIds={selectedRunIds} setPinnedRun={setPinnedRun} setShowRunDetailsModal={setShowRunDetailsModal} @@ -187,8 +191,6 @@ const ExperimentWrapper = ({ theme }) => { sidebarVisible={isSidebarVisible} theme={theme} trackingDataError={trackingDataError} - runMetadata={runMetadata} - runTrackingData={runTrackingData} /> ) : null} diff --git a/src/components/ui/button/button.scss b/src/components/ui/button/button.scss index 9e94bd03ba..bc8f01be3e 100644 --- a/src/components/ui/button/button.scss +++ b/src/components/ui/button/button.scss @@ -31,7 +31,7 @@ $secondary-underline-offset-hover: 4px; .button__btn { font-size: 1.6em; - line-height: 1em; + line-height: 1.3; font-weight: 600; letter-spacing: 0.1px; color: var(--color-button__text); @@ -53,6 +53,7 @@ $secondary-underline-offset-hover: 4px; .button__btn--small { font-size: 1.4em; letter-spacing: 0.1px; + line-height: 1.4; padding: 0.6em 0.85em 0.7em; } @@ -64,6 +65,10 @@ $secondary-underline-offset-hover: 4px; background: var(--color-button--active); } + &:focus { + box-shadow: 0 0 0 7px $color-accent-blue; + } + &:hover { color: var(--color-button__text--hover); background: var(--color-button--outline); @@ -92,6 +97,10 @@ $secondary-underline-offset-hover: 4px; background: none; } + &:focus { + outline: 4px solid $color-accent-blue; + } + &:hover::after { background: var(--color-button--outline); transform: translateY($secondary-underline-offset-hover);