Fix omitExtraData bugs for nested empties and nonspecified objects #1419

Tonexus · 2019-08-19T07:38:21Z

Reasons for making this change

Fixes #1388 and fixes #1396. Alternative to #1418.

Modifies behavior of toPathSchema in utils.js (afaik the function is only used in omitting the extra data, let me know if this breaks anything else) to create $name key for arrays as well to be consistent with behavior for objects. Modifies behavior of getFieldNames in Form.js to also add a path to the array of paths if that path points to something empty. Refactors the surrounding code to be more concise and, hopefully, readable. Adds tests for omitExtraData and liveOmit.

Checklist

I'm updating documentation
- I've checked the rendering of the Markdown text I've added
I'm adding or updating code
- I've added and/or updated tests
- I've updated docs if needed
I'm adding a new feature
- I've updated the playground with an example use of the feature

epicfaace · 2019-08-19T15:54:17Z

src/utils.js

@@ -873,43 +873,34 @@ export function toIdSchema(

 export function toPathSchema(schema, name = "", definitions, formData = {}) {
  const pathSchema = {
-    $name: name,
+    $name: name.replace(/\.$/, ""),


Why is this needed?

As a result of refactoring and because the initial call starts with name as an empty string, I ended up with a period prepended to all of the $names, so I swapped it such that the period is appended, then removed when the $name value is set. e.g. here

https://github.com/Tonexus/react-jsonschema-form/blob/ff66b13dd0e74a36394f3e45e546137b024d709e/src/utils.js#L890

changes `${name}.${i}` to `${name}${i}.`.

Now that I think about it, it was honestly was kind of an arbitrary change. If you think the old style is more clear, I can change it back.

I think you can change it to the old style, unless there are any additional advantages to this new style.

src/utils.js

epicfaace · 2019-08-19T15:56:19Z

Modifies behavior of getFieldNames in Form.js to also add a path to the array of paths if that path points to something empty.

My PR #1418 also does this, although the way it does so isn't very good -- it relies on looking for the first "." in the string (which could be an issue with specially-named keys later). Your method seems better!

https://github.com/mozilla-services/react-jsonschema-form/pull/1418/files#diff-07a63ede6fe55fca89398d42da737369R170

…onschema-form into feature/omitExtraData-array-of-empties

Tonexus · 2019-08-19T20:49:15Z

@epicfaace I added your tests, and if I understand correctly, describeRepeated essentially parameterizes the tests and runs the 3 relevant combinations of omitExtraData and liveOmit for the enclosed tests. In using the function in Form_test.js, you repeat all of the current tests for the 3 combinations, right? So then there are no tests to make sure that omitExtraData actually omits the correct data yet, right?

epicfaace · 2019-08-20T01:10:50Z

@epicfaace I added your tests, and if I understand correctly, describeRepeated essentially parameterizes the tests and runs the 3 relevant combinations of omitExtraData and liveOmit for the enclosed tests. In using the function in Form_test.js, you repeat all of the current tests for the 3 combinations, right?

Yes.

So then there are no tests to make sure that omitExtraData actually omits the correct data yet, right?

There are tests at the end of Form_test.js that are not within the describeRepeated block; I believe they include a test that tests that behavior of omitExtraData.

epicfaace · 2019-08-20T01:20:53Z

By the way, the test that is failing is due to issues with form defaults when omitExtraData and/or liveOmit are true. I fixed them in the other PR with these changes: https://github.com/mozilla-services/react-jsonschema-form/pull/1418/files#diff-07a63ede6fe55fca89398d42da737369R249-R253

Tonexus · 2019-08-20T01:57:34Z

Yeah I see what you are doing, but I think that that logic can be moved to onChange instead. Right now, you have to recompute pathSchema on submit to fix the bug, but it should really be computed whenever the form data changes, as the paths don't actually change on submit.

epicfaace · 2019-08-20T04:45:45Z

@Tonexus ok, but if omitExtraData is true and liveOmit is false, then don't we only need to compute the pathSchema on submit? Computing it during onChange is unnecessary and might slow things down.

Tonexus · 2019-08-20T14:53:29Z

@epicfaace That's a good point, if liveOmit is false, pathSchema probably doesn't need to be computed. However, it looks like pathSchema is already being computed on component creation and on props change without checking for liveOmit, and I wanted to be consistent with those cases. Should I change those cases to check for omitExtraData and liveOmit as well?

Also, right now it looks like pathSchema is not propagated to the state in onChange even if both omitExtraData and liveOmit are true, so I'll go ahead and change it so that pathSchema is propagated there as well.

epicfaace · 2019-08-20T16:23:14Z

@epicfaace That's a good point, if liveOmit is false, pathSchema probably doesn't need to be computed. However, it looks like pathSchema is already being computed on component creation and on props change without checking for liveOmit, and I wanted to be consistent with those cases. Should I change those cases to check for omitExtraData and liveOmit as well?

Yes, please do.

Also, right now it looks like pathSchema is not propagated to the state in onChange even if both omitExtraData and liveOmit are true, so I'll go ahead and change it so that pathSchema is propagated there as well.

Sounds good, thanks!

Tonexus · 2019-08-21T03:11:39Z

@epicfaace Actually, should pathSchema even be part of the state? It is only computed on submit or, with liveOmit true, whenever formData changes. And then it is only applied directly after those two cases, so it doesn't really need to persist.

On a side note, a lot of the logic in onChange and onSubmit seem redundant, maybe some of it can be refactored in another PR...

Tonexus · 2019-08-21T06:19:37Z

@epicfaace I'm now having issues with the very last test case in the file.

https://github.com/Tonexus/react-jsonschema-form/blob/d23c6480175cf5eb1c041bdb1b7df67eabd0a515/test/Form_test.js#L2663-L2687

Can you explain the reasoning for that one? It looks like it is correctly applying the new schema to the new formData. Is it supposed to use the new formData with the old schema that then removes the baz property?

epicfaace · 2019-08-21T16:23:51Z

@epicfaace Actually, should pathSchema even be part of the state?

Yeah, you're right.

For the very last test case, you're right, I think the test case is wrong. I based it off an original test that tested based on an external (props-based) formData change, which used comp.componentWillReceiveProps. I meant to change the test to be based on an internal (user interaction-based) formData change, using Simulate.change. However, this test (as well as the one before it) mixes up both types of props changes in the same test case. Essentially, I meant to remove the comp.componentWillReceiveProps call, but neglected to.

It may be good to split this up; for each test in the final "Schema and formData updates" describe block, make one test that uses comp.componentWillReceiveProps to test an external formData change and another test that tests Simulate.change to test the internal formData change.

Tonexus · 2019-08-21T17:55:49Z

@epicfaace Ok, made those changes. There are now 4 new tests for each combo of omitExtraData and liveOmit, and only one actually omits data on form change. Should be ready for review/merge now.

epicfaace · 2019-08-22T04:55:09Z

Oh, I remember why pathSchema was initially part of the state. The idea was that it could be passed on to widgets/fields eventually, so that they would be able to know the "path" of the formData that they refer to. However, given that pathSchema is only computed live when liveOmit is true, there would be no point keeping it in the state for now. We can re-add pathSchema into the state later on (later PR) when also giving access to it for widgets/fields.

epicfaace

Looks good! Just a few more things.

src/utils.js

src/components/Form.js

src/utils.js

epicfaace · 2019-08-22T04:52:58Z

src/utils.js

+  if (
+    schema.hasOwnProperty("items") &&
+    Array.isArray(formData) &&
+    formData.length > 0


This line is probably not necessary (line 885)

Tonexus · 2019-08-22T14:59:23Z

@epicfaace Added most of the requested changes. See review comment.

epicfaace

I think we're good! Thanks for all your work on this.

MatinF · 2019-08-22T17:42:26Z

Happy to do some tests of this to see if it resolves #1396 and #1425 - is the playground link updated?

epicfaace · 2019-08-22T18:40:55Z

You can use the playground link for the deploy preview of this PR: https://deploy-preview-1419--rjsf.netlify.com/

Tony Lau added 3 commits August 18, 2019 23:43

Fixed disappearing empty objects and arrays

9500a11

Fixed some tests

47b8845

Fixed bugs

ff66b13

Tonexus changed the title ~~Feature/omit extra data array of empties~~ Fix omitExtraData bugs for nested empties and nonspecified objects Aug 19, 2019

Tonexus mentioned this pull request Aug 19, 2019

Live omit fixes #1418

Closed

7 tasks

epicfaace reviewed Aug 19, 2019

View reviewed changes

Tony Lau added 3 commits August 19, 2019 12:17

Updated toPathSchema tests

8e4f549

Merge branch 'master' of https://github.com/mozilla-services/react-js…

a77af9e

…onschema-form into feature/omitExtraData-array-of-empties

Added tests from #1418

80e52bd

Uncommented test cases

6d6a23c

epicfaace mentioned this pull request Aug 20, 2019

#226 - Restrict accepted mimetypes in file widget #1246

Merged

8 tasks

Tony Lau added 2 commits August 20, 2019 23:11

Remove unecessary or

fe3203b

Update usage of toPathSchema

d23c648

Revert back to old period separation style in toPathSchema

8804bf4

Tony Lau added 2 commits August 21, 2019 10:09

Converted change props with liveOmit test to only test for liveOmit

340a3cb

Revert one other case of period positioning

b992fe2

Tonexus marked this pull request as ready for review August 21, 2019 17:53

epicfaace reviewed Aug 22, 2019

View reviewed changes

Changed according to review suggestions

3f3a8b9

epicfaace approved these changes Aug 22, 2019

View reviewed changes

epicfaace merged commit 4251a2c into rjsf-team:master Aug 22, 2019

Tonexus mentioned this pull request Aug 22, 2019

omitExtraData trims out required property groups #1425

Closed

3 tasks

snyk-bot mentioned this pull request Jan 30, 2020

[Snyk] Upgrade react-jsonschema-form from 1.2.0 to 1.8.1 preset-io/superset#11

Closed

achendrick mentioned this pull request May 13, 2020

Additional properties are not submitted when nested and omit data enabled #1497

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix omitExtraData bugs for nested empties and nonspecified objects #1419

Fix omitExtraData bugs for nested empties and nonspecified objects #1419

Tonexus commented Aug 19, 2019 •

edited

Loading

epicfaace Aug 19, 2019

Tonexus Aug 19, 2019

epicfaace Aug 19, 2019

epicfaace commented Aug 19, 2019

Tonexus commented Aug 19, 2019

epicfaace commented Aug 20, 2019

epicfaace commented Aug 20, 2019

Tonexus commented Aug 20, 2019

epicfaace commented Aug 20, 2019

Tonexus commented Aug 20, 2019

epicfaace commented Aug 20, 2019

Tonexus commented Aug 21, 2019

Tonexus commented Aug 21, 2019

epicfaace commented Aug 21, 2019

Tonexus commented Aug 21, 2019

epicfaace commented Aug 22, 2019

epicfaace left a comment

epicfaace Aug 22, 2019 •

edited

Loading

Tonexus commented Aug 22, 2019

epicfaace left a comment

MatinF commented Aug 22, 2019

epicfaace commented Aug 22, 2019

Fix omitExtraData bugs for nested empties and nonspecified objects #1419

Fix omitExtraData bugs for nested empties and nonspecified objects #1419

Conversation

Tonexus commented Aug 19, 2019 • edited Loading

Reasons for making this change

Checklist

epicfaace Aug 19, 2019

Choose a reason for hiding this comment

Tonexus Aug 19, 2019

Choose a reason for hiding this comment

epicfaace Aug 19, 2019

Choose a reason for hiding this comment

epicfaace commented Aug 19, 2019

Tonexus commented Aug 19, 2019

epicfaace commented Aug 20, 2019

epicfaace commented Aug 20, 2019

Tonexus commented Aug 20, 2019

epicfaace commented Aug 20, 2019

Tonexus commented Aug 20, 2019

epicfaace commented Aug 20, 2019

Tonexus commented Aug 21, 2019

Tonexus commented Aug 21, 2019

epicfaace commented Aug 21, 2019

Tonexus commented Aug 21, 2019

epicfaace commented Aug 22, 2019

epicfaace left a comment

Choose a reason for hiding this comment

epicfaace Aug 22, 2019 • edited Loading

Choose a reason for hiding this comment

Tonexus commented Aug 22, 2019

epicfaace left a comment

Choose a reason for hiding this comment

MatinF commented Aug 22, 2019

epicfaace commented Aug 22, 2019

Tonexus commented Aug 19, 2019 •

edited

Loading

epicfaace Aug 22, 2019 •

edited

Loading