diff --git a/docs/site/content/en/docs/Concepts/core-concepts/index.md b/docs/site/content/en/docs/Concepts/core-concepts/index.md index c9bb78846..bf3c3aa8e 100644 --- a/docs/site/content/en/docs/Concepts/core-concepts/index.md +++ b/docs/site/content/en/docs/Concepts/core-concepts/index.md @@ -90,4 +90,36 @@ A `Fingerprint` is combination of [`Filtering labels`](#label) that unique ident ## Datasource -A `Datasource` is a **required** top-level organizational construct that defines the source of the data to be stored or retrieved by Horreum. Currently, Horreum supports 2 types of `Datasource`: Postgres and Elasticsearch \ No newline at end of file +A `Datasource` is a **required** top-level organizational construct that defines the source of the data to be stored or retrieved by Horreum. Currently, Horreum supports 2 types of `Datasource`: Postgres and Elasticsearch + +## Baseline + +The *initial* sample for an `Experiment` comparison. Configured in an `Experiment Profile`. + +## Change Detection Variable + +Change detection tracks `Schema` `Label`s that have been configured as a Change Detection Variable. + +## Experiment + +This enables running a comparison between `Runs` for a particular `Test`. There can be multiple `Profile`s configured for an `Experiment` to check for a variety of desired conditions. The outcome status for each `Profile` condition will be one of the following: +- SAME +- WORSE +- BETTER + +## Experiment Profile + +A `Profile` consists of: +- `Experiment` selector +- `Baseline` +- 0, 1 or many Comparison conditions + +## JSON validation schema +An *optional* schema added to a `Test` to validate uploaded `Run` JSON data. + + + + + + + diff --git a/docs/site/content/en/docs/Tasks/experiment-evaluation/dataset_list.png b/docs/site/content/en/docs/Tasks/experiment-evaluation/dataset_list.png new file mode 100644 index 000000000..3793e3bb5 Binary files /dev/null and b/docs/site/content/en/docs/Tasks/experiment-evaluation/dataset_list.png differ diff --git a/docs/site/content/en/docs/Tasks/experiment-evaluation/dataset_selection.png b/docs/site/content/en/docs/Tasks/experiment-evaluation/dataset_selection.png new file mode 100644 index 000000000..061664c13 Binary files /dev/null and b/docs/site/content/en/docs/Tasks/experiment-evaluation/dataset_selection.png differ diff --git a/docs/site/content/en/docs/Tasks/experiment-evaluation/index.md b/docs/site/content/en/docs/Tasks/experiment-evaluation/index.md new file mode 100644 index 000000000..44781d544 --- /dev/null +++ b/docs/site/content/en/docs/Tasks/experiment-evaluation/index.md @@ -0,0 +1,42 @@ +--- +title: Dataset Experiment Evaluation +description: Document explaining the use of Experiment Evaluation function in Horreum UI +date: 2024-01-18 +weight: 5 +--- + +## Using the Dataset Experiment Evaluation View + +Using the Experiment evaluation window you can quickly see the values and the relative difference of a Run. +Start by initially loading the Test. Then click the `Dataset list` button. + +{{% imgproc dataset_list Fit "1200x900" %}} Dataset List {{% /imgproc %}} + +Then select the individual Test Run you want to compare with it's Baseline. + +By navigating to the uploaded run Dataset view page you will see a button “Evaluate experiment”. Clicking this button opens a window revealing the comparison result for the current Run and the Baseline Run.. + +{{% imgproc individual_evaluation Fit "1200x900" %}} Individual Evaluation {{% /imgproc %}} + +Results show the values then the percentage difference. + + +## Datasets Comparison View + +Horreum provides multiple Run comparisons in the Dataset Comparison View. We can filter based on the Filter labels defined in the Schema. + +Start by initially loading the Dataset list. Then click the button “Select for comparison”. + +{{% imgproc select_for_comparison Fit "1200x900" %}} Select for comparison {{% /imgproc %}} + +Next the Comparison view is displayed. This is where filters are set. + +{{% imgproc dataset_selection Fit "1200x900" %}} Dataset selection {{% /imgproc %}} + +Select a number of Runs to be compared using the “Add to comparison” button. Then click the “Compare labels”. Displayed next is the Labels Comparison view. + +{{% imgproc multiple_dataset_comparison Fit "1200x900" %}} Multiple Dataset comparison {{% /imgproc %}} + + Displayed here are multiple Datasets. Schema Labels can be expanded to display a bar graph representing each Run. + + In this guide two things were shown. How to compare an individual Dataset. Followed by comparing multiple Datasets. \ No newline at end of file diff --git a/docs/site/content/en/docs/Tasks/experiment-evaluation/individual_evaluation.png b/docs/site/content/en/docs/Tasks/experiment-evaluation/individual_evaluation.png new file mode 100644 index 000000000..3b414362a Binary files /dev/null and b/docs/site/content/en/docs/Tasks/experiment-evaluation/individual_evaluation.png differ diff --git a/docs/site/content/en/docs/Tasks/experiment-evaluation/multiple_dataset_comparison.png b/docs/site/content/en/docs/Tasks/experiment-evaluation/multiple_dataset_comparison.png new file mode 100644 index 000000000..36271ebcb Binary files /dev/null and b/docs/site/content/en/docs/Tasks/experiment-evaluation/multiple_dataset_comparison.png differ diff --git a/docs/site/content/en/docs/Tasks/experiment-evaluation/select_for_comparison.png b/docs/site/content/en/docs/Tasks/experiment-evaluation/select_for_comparison.png new file mode 100644 index 000000000..2716f6336 Binary files /dev/null and b/docs/site/content/en/docs/Tasks/experiment-evaluation/select_for_comparison.png differ