Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(data-integrity): add checks-selector #348

Merged
merged 29 commits into from
Mar 16, 2022
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
172cc7d
feat(data-integrity): add checks-selector
Birkbjo Feb 25, 2022
9343cf2
fix: cleanup
Birkbjo Feb 25, 2022
18dfc25
chore(translations): update translations
Birkbjo Feb 25, 2022
48fa4b8
fix: cleanup
Birkbjo Feb 25, 2022
8f81118
fix: use Label instead of custom TransferOption
Birkbjo Feb 28, 2022
45e0d92
fix: cleanup react warnings
Birkbjo Feb 28, 2022
e23c4c1
fix: cleanup styles for transfer-option
Birkbjo Feb 28, 2022
dc5f213
refactor: cleanup
Birkbjo Mar 1, 2022
4f51dae
fix: add min-width, remove paramter-width change
Birkbjo Mar 1, 2022
a4d8d68
test: update test-fixtures, use 38
Birkbjo Mar 1, 2022
30921ad
style: run prettier
Birkbjo Mar 1, 2022
d9078b4
style: fix lint
Birkbjo Mar 1, 2022
0e04e4d
test: update more fixtures
Birkbjo Mar 1, 2022
931385d
style: run prettier on cypress
Birkbjo Mar 1, 2022
4e4e066
chore: update dhis2ApiVersion for tests
Birkbjo Mar 1, 2022
77ddb80
style: css lint
Birkbjo Mar 1, 2022
12b149f
test: fix fixtures
Birkbjo Mar 1, 2022
05fa734
chore: upgrade cypress dependencies and config
HendrikThePendric Mar 3, 2022
a699792
chore: produce new network fixtures
HendrikThePendric Mar 3, 2022
38df2b4
chore: change dhsi2 base url to localhost
HendrikThePendric Mar 3, 2022
41d5f23
feat(data-integrity): add support for report-type field
Birkbjo Mar 9, 2022
d7349fa
fix: actual add reportTypeField file
Birkbjo Mar 9, 2022
f2608ad
fix: cleanup
Birkbjo Mar 9, 2022
6915b18
fix: don't remove parameterName
Birkbjo Mar 9, 2022
559bebd
chore: update cypress dhis2ApiVersion
Birkbjo Mar 10, 2022
f41a728
fix: minor review
Birkbjo Mar 10, 2022
0671703
style: run prettier
Birkbjo Mar 10, 2022
27a80e2
fix(parameter-fields): spread parameterprops instead of new object
Birkbjo Mar 11, 2022
63105e7
fix(data-integrity): fix validation for empty checks
Birkbjo Mar 11, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 147 additions & 2 deletions i18n/en.pot
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ msgstr ""
"Content-Type: text/plain; charset=utf-8\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n != 1)\n"
"POT-Creation-Date: 2021-11-18T16:21:37.799Z\n"
"PO-Revision-Date: 2021-11-18T16:21:37.799Z\n"
"POT-Creation-Date: 2022-02-25T17:25:37.537Z\n"
"PO-Revision-Date: 2022-02-25T17:25:37.537Z\n"

msgid "Not authorized"
msgstr "Not authorized"
Expand All @@ -30,6 +30,24 @@ msgstr "Something went wrong whilst creating your job"
msgid "CRON Expression"
msgstr "CRON Expression"

msgid "Please select checks to run."
msgstr "Please select checks to run."

msgid "Checks to run"
msgstr "Checks to run"

msgid "Run all available checks"
msgstr "Run all available checks"

msgid "Only run selected checks"
msgstr "Only run selected checks"

msgid "Severity"
msgstr "Severity"

msgid "Select checks to run."
msgstr "Select checks to run."

msgid "Delay"
msgstr "Delay"

Expand Down Expand Up @@ -201,6 +219,133 @@ msgstr "Enrollment"
msgid "Validation result"
msgstr "Validation result"

msgid "Program indicators with invalid expressions"
msgstr "Program indicators with invalid expressions"

msgid "Data elements without groups"
msgstr "Data elements without groups"

msgid "Indicators with invalid numerator"
msgstr "Indicators with invalid numerator"

msgid "Program rule actions without notification"
msgstr "Program rule actions without notification"

msgid "Only one \"default\" category combo should exist"
msgstr "Only one \"default\" category combo should exist"

msgid "Data elements without data sets"
msgstr "Data elements without data sets"

msgid "Category combos being invalid"
msgstr "Category combos being invalid"

msgid "Indicators with identical formulas"
msgstr "Indicators with identical formulas"

msgid "Indicators without groups"
msgstr "Indicators without groups"

msgid "Data elements in data set not in form"
msgstr "Data elements in data set not in form"

msgid "Program rules without priority"
msgstr "Program rules without priority"

msgid "Validation rules without groups"
msgstr "Validation rules without groups"

msgid "Program indicators with invalid filters"
msgstr "Program indicators with invalid filters"

msgid "Categories with no category options"
msgstr "Categories with no category options"

msgid "Program rules without condition"
msgstr "Program rules without condition"

msgid "Program rule actions without stage"
msgstr "Program rule actions without stage"

msgid "Only one \"default\" category should exist"
msgstr "Only one \"default\" category should exist"

msgid ""
"Lists category combos that share a combination of categories with at least "
"one other category combo"
msgstr ""
"Lists category combos that share a combination of categories with at least "
"one other category combo"

msgid "Org units with cyclic references"
msgstr "Org units with cyclic references"

msgid "Program rule variables without data element"
msgstr "Program rule variables without data element"

msgid "Validation rules with invalid right side expression"
msgstr "Validation rules with invalid right side expression"

msgid "Data sets not assigned to org units"
msgstr "Data sets not assigned to org units"

msgid "Data elements violating exclusive group sets"
msgstr "Data elements violating exclusive group sets"

msgid "Org unit groups without group sets"
msgstr "Org unit groups without group sets"

msgid "Program rule variables without attribute"
msgstr "Program rule variables without attribute"

msgid "Org units violating exclusive group sets"
msgstr "Org units violating exclusive group sets"

msgid "Program rule actions without data object"
msgstr "Program rule actions without data object"

msgid "Only one \"default\" category option should exist"
msgstr "Only one \"default\" category option should exist"

msgid "Only one \"default\" category option combo should exist"
msgstr "Only one \"default\" category option combo should exist"

msgid "Org units without groups"
msgstr "Org units without groups"

msgid "Program indicators without expression"
msgstr "Program indicators without expression"

msgid "Indicators violating exclusive group sets"
msgstr "Indicators violating exclusive group sets"

msgid "Periods duplicates"
msgstr "Periods duplicates"

msgid "Data elements assigned to data sets with different period types"
msgstr "Data elements assigned to data sets with different period types"

msgid "Program rules without action"
msgstr "Program rules without action"

msgid "Org units being orphaned"
msgstr "Org units being orphaned"

msgid "Validation rules with invalid left side expression"
msgstr "Validation rules with invalid left side expression"

msgid "Indicators with invalid denominator"
msgstr "Indicators with invalid denominator"

msgid "Program rule actions without section"
msgstr "Program rule actions without section"

msgid "Warning"
msgstr "Warning"

msgid "Severe"
msgstr "Severe"

msgid "Completed"
msgstr "Completed"

Expand Down
158 changes: 158 additions & 0 deletions src/components/FormFields/DataIntegrityChecksField.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
import React, { useCallback, useRef, useState } from 'react'
import { PropTypes } from '@dhis2/prop-types'
import i18n from '@dhis2/d2-i18n'
import {
FieldGroup,
Radio,
Transfer,
TransferOption,
ReactFinalForm,
InputFieldFF,
Help,
} from '@dhis2/ui'
import cx from 'classnames'
import { hooks } from '../Store'
import {
getCheckName,
severityMap,
} from '../../services/server-translations/dataIntegrityChecks'
import styles from './DataIntegrityChecksField.module.css'

const { Field, useField } = ReactFinalForm

const VALIDATOR = value =>
value && value.length < 1
? i18n.t('Please select checks to run.')
: undefined

const DataIntegrityChecksField = ({ label, name }) => {
const options = hooks.useParameterOptions('dataIntegrityChecks')
const {
input: { value, onChange },
} = useField(name)

const hasValue = !!value && value.length > 0
const [runSelected, setRunSelected] = useState(hasValue)

const translatedOptions = options
.map(option => ({
...option,
value: option.name,
label: getCheckName(option.name),
severity: severityMap[option.severity],
}))
.sort((a, b) => a.label.localeCompare(b.label))

const toggle = ({value}) => {
const checked = value === 'true'

if (!checked) {
// clear checks when "Run all" is selected
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have destroyOnUnregister enabled, see here, which should take care of removing the value if the field is unregistered. But I see you linked an open issue about conditional rendering causing issues. We're rendering the parameter fields conditionally, could what you encountered be solved in a similar manner?

Copy link
Member Author

@Birkbjo Birkbjo Mar 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I got errors that crashed the app, so I added the hidden-prop to fix this. It wasn't just because of conditional rendering, but conditional rendering in combination with updating the state of the component.

It does not seem like the value is cleared when conditionally rendering the field either. As you can toggle back and forth, and the selected checks will be there. We also need to mark the form as dirty when toggling back and forth (which onChange([]) does) - since we need to be able to switch from "some selected checks" to "run all", and be able to save the form (which is disabled when it's not dirty).

Copy link

@ghost ghost Mar 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. It seems like you're initializing the state for runSelected based on the previous value of the field. Which will be there if the user toggles back and forth between "run all" and "run only". It'll be lost anyway as soon as the user switches to another job type. If you lose that bit of logic and initialize runSelected to false there's no longer a need to clear the onChange (provided you conditionally render the transfer field in the field group), or use useField. That means you shouldn't run into the linked bug either (at least I didn't during testing, but let me know if I missed it).

I'd go with that. We don't save state for parameter settings between jobs anyway, and if not saving it for this section doesn't require workarounds and simplifies things I would personally go in that direction.

Copy link
Member Author

@Birkbjo Birkbjo Mar 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you lose that bit of logic and initialize runSelected to false

I'm not sure I follow, this would make it so that when you edit a Data integrity check with "selected checks", it would change it to Run all? That does not work.
The reason for checking for previous value of the field is so that you select the correct "toggle" based on the value of checks. Simple use case is to add a new check for a previous job; always defaulting to false, would make the user need to select the run selected checks?

I'd go with that. We don't save state for parameter settings between jobs anyway, and if not saving it for this section doesn't require workarounds and simplifies things I would personally go in that direction.

This is not the reason for the logic; see above. It's for when editing a job.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haha, oh yeah of course you're right. Minor oversight on my part 😅. Wait, let me check one thing that might still satisfy that requirement.

Copy link
Member Author

@Birkbjo Birkbjo Mar 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking into this! Actually makes sense for useField to prevent unregistering of the field. However, I don't really think this is much better than the current solution? And may have unintended side effect - see below. This is a bug in react-final-form anyway, and it's referenced in the code - so once it's fixed we could move the "hidden"-prop and conditionally render instead.

It might be cleaner to unmount the field, however we still need some logic to actually clear the data, if you edit an already existing job, and toggle from "selected checks" to "run all" in your example above, you won't be able to save the form. I think this is because the field is not mounted, and thus does not count as dirty? So the form would then be "pristine".

Other than that, your solution would potentially fix an annoying situation by not unmounting the field; we need to "change" the validation-function when Run all is selected, since we don't want to validate the checks in that case. Not sure if the current "complexity" is worth it, or if we should just drop validation entirely, as it's not actually that bad to not select any checks.

I do wonder if our usage of initialValues in the app is a little flawed. They'll be lost as soon as final form unregisters a field and clears the value, like when a user switches jobTypes for an existing job.

Yeah, haven't looked too much into that, but seems like we would need to change the destroyOnUnmount or something if we would like to keep the initialValues when unmounting fields?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than that, your solution would potentially fix an annoying situation by not unmounting the field; we need to "change" the validation-function when Run all is selected, since we don't want to validate the checks in that case. Not sure if the current "complexity" is worth it, or if we should just drop validation entirely, as it's not actually that bad to not select any checks.

Regarding this, I decided to fix this by having empty array meaning "run selected only", and null or undefined, meaning "run all". Should be a simple enough fix for the situation above.

Copy link

@ghost ghost Mar 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, haven't looked too much into that, but seems like we would need to change the destroyOnUnmount or something if we would like to keep the initialValues when unmounting fields?

Yeah, yeah the reason that that's there is so that if you switch a jobType, the existing parameters don't persist. Otherwise those parameters will be sent to the backend if the user submits, even if they don't match the current job. I suppose the backend would probably drop unrecognized job parameters, but it seemed more correct to me at the time.

That's also why I was thinking of a way to automatically remove the transfer values if it unregisters. My assumption was that selecting run all would mean you don't need the checks job parameter at all. But of course, run all is identical to selecting all checks (I assume?). Which is obvious of course, but I'd not even stopped to consider that.

So, first off, I'll approve the PR since I don't want to block you. Since I rewrote a lot of this app I tend to get a little product-managery about it, but that's not actually my position.

That being said. I can't help myself thinking that the design could be simplified. Instead of a "run all" radio button I'd just render the transfer at all times and forego the radios. After all, the transfer already has a convenient "select all" control. That would simplify our code and the UI if you ask me. Do you know if there's a technical reason why we didn't go with that approach?

(Also, let me know if I'm misunderstanding anything here. We're getting a little into the weeds technically, so maybe I'm overlooking sth here)

Copy link
Member Author

@Birkbjo Birkbjo Mar 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, first off, I'll approve the PR since I don't want to block you. Since I rewrote a lot of this app I tend to get a little product-managery about it, but that's not actually my position.

I absolutely understand this, no worries! There's a lot of decision that's taken here, and I don't know the full background. I've just tried to implement this feature without too drastic changes.

That being said. I can't help myself thinking that the design could be simplified. Instead of a "run all" radio button I'd just render the transfer at all times and forego the radios. After all, the transfer already has a convenient "select all" control. That would simplify our code and the UI if you ask me. Do you know if there's a technical reason why we didn't go with that approach?

I do agree with this point, and this was my first implementation, with a "If no checks are selected, all checks will be run"-message in the "SelectedEmptyComponent". But I talked to Joe, and he proposed the implemented solution. The reason being that it might be a bit confusing that actually selecting nothing selects all, and the reason below:
Manually having to select all would also probably not be the best (even with the "select all"-button), since if new checks are added in the backend, you would have to manually select them again. In the current solution, all checks will run regardless of user interaction. You might argue that the user have more "control" of not adding new checks - but this is how it always has been and we don't want this change to be too disruptive. It would potentially be confusing to users why some checks are not running.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Since we already talked a bit on slack, I'll just add here what I said there: seems tricky to really simplify the UI then on our end. The requirements kind of dictate the current solution. Would be nice if eventually we could get rid of the two types of "all checks", because now effectively we have "all current and future checks" and "all explicitly selected checks". To me that wasn't really clear from the UI. Anyway, something for the future I guess. Thanks for explaining!

onChange([])
}
setRunSelected(checked)
}

return (
<FieldGroup label={i18n.t('Checks to run')}>
<Radio
name={'checksToRun'}
value={String(false)}
label={i18n.t('Run all available checks')}
checked={!runSelected}
onChange={toggle}
/>
<Radio
name={'checksToRun'}
value={String(true)}
label={i18n.t('Only run selected checks')}
checked={runSelected}
onChange={toggle}
/>
<Field
name={name}
component={ChecksTransfer}
options={translatedOptions}
label={label}
validate={VALIDATOR}
// conditional rendering of FinalForm-fields cause some issues,
// see https://github.com/final-form/react-final-form/issues/809
hidden={!runSelected}
/>
</FieldGroup>
)
}

const LabelComponent = ({ label, severity, highlighted, disabled }) => (
<div
className={cx(styles.transferOption, {
[styles.highlighted]: highlighted,
[styles.disabled]: disabled,
})}
>
<div className={styles.optionName}>{label}</div>
<div className={cx(styles.optionSeverity, {
[styles.highlighted]: highlighted
})}>{`${i18n.t(
'Severity'
)}: ${severity}`}</div>
</div>
)

LabelComponent.propTypes = TransferOption.propTypes

const renderOption = option => (
<TransferOption {...option} label={<LabelComponent {...option} />} />
)

const ChecksTransfer = ({ input, meta, options = [], hidden }) => {
const { onChange } = input

const handleChange = useCallback(
({ selected }) => {
onChange(selected)
},
[onChange]
)

if (hidden) {
return null
}

const isErr = meta.touched && meta.invalid

return (
<>
<Transfer
options={options}
onChange={handleChange}
selected={input.value || []}
renderOption={renderOption}
maxSelections={Infinity}
enableOrderChange={true}
filterable={true}
height={'450px'}
selectedEmptyComponent={<SelectedEmptyComponent />}
className={styles.transfer}
/>
{isErr && <Help error={isErr}>{meta.error}</Help>}
</>
)
}

ChecksTransfer.propTypes = InputFieldFF.propTypes

const SelectedEmptyComponent = () => (
<p className={styles.selectedEmptyComponent}>
{i18n.t('Select checks to run.')}
</p>
)

const { string } = PropTypes

DataIntegrityChecksField.propTypes = {
label: string.isRequired,
name: string.isRequired,
}

export default DataIntegrityChecksField
27 changes: 27 additions & 0 deletions src/components/FormFields/DataIntegrityChecksField.module.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
.transfer {
margin-top: 8px;
}

.transferOption {
font-size: 14px;
padding: 4px 0px;
}

.optionName {
font-weight: 500;
margin-bottom: 4px;
}

.optionSeverity {
font-weight: 400;
font-size: 13px;
color: var(--colors-grey600);
}

.transferOption.highlighted .optionSeverity {
color: var(--colors-grey300);
}

.selectedEmptyComponent {
text-align: center;
}
17 changes: 14 additions & 3 deletions src/components/FormFields/ParameterFields.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,23 @@ import { hooks } from '../Store'
import { formatToString } from './formatters'
import SkipTableTypesField from './SkipTableTypesField'
import LabeledOptionsField from './LabeledOptionsField'
import DataIntegrityChecksField from './DataIntegrityChecksField'
import styles from './ParameterFields.module.css'

const { Field } = ReactFinalForm

// The key under which the parameters will be sent to the backend
const FIELD_NAME = 'jobParameters'

const getCustomComponent = (jobType, parameterName) => {
if (jobType === 'DATA_INTEGRITY' && parameterName === 'checks') {
return DataIntegrityChecksField
} else if (parameterName === 'skipTableTypes') {
return SkipTableTypesField
}
return null
}

// Renders all parameters for a given jobtype
const ParameterFields = ({ jobType }) => {
const parameters = hooks.useJobTypeParameters(jobType)
Expand All @@ -29,11 +39,12 @@ const ParameterFields = ({ jobType }) => {
}
let parameterComponent = null

// Specific case, as the options here need specific translations
if (name === 'skipTableTypes') {
const CustomParameterComponent = getCustomComponent(jobType, name)

if (CustomParameterComponent) {
return (
<Box marginTop="16px" key={name}>
<SkipTableTypesField
<CustomParameterComponent
{...defaultProps}
parameterName={name}
/>
Expand Down
Loading