Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a common Int Validator and use it in ingest_pipelines and Index_lifecycle_management #196527

Merged
merged 5 commits into from
Oct 24, 2024

Conversation

SoniaSanzV
Copy link
Contributor

@SoniaSanzV SoniaSanzV commented Oct 16, 2024

Closes #110417

Summary

In the Ingest Node Pipelines section, when the users created a new pipeline selecting de Community ID processor the users could set a non-integer number in this field. Then, they received a server side error when tried to create a pipeline. For fixing this, a validation must be added in the client.

We didn't have a reusable validation for this case, but we did have a custom validation for integer values in the Index lifecycle management plugin. We also had the necessary translation in that plugin. So I went forward with:

  • I created a new integer validator in the es_ui_shared package as it is a fairly common validation and we could take advantage of it in the future. Also added a couple of unit test there for this validator.

  • I reused in the ingest_pipelines plugin the strings that already existed in index_lifecycle_management.

  • I added the new validation in the Community ID form in the ingest_pipelines plugin. Also added some test verifying that the processor doesn't create when the seeds validation fails.

  • Changed the method in the index_lifecycle_management validator so now it uses the reusable one.

Now the Ingest pipeline forms shows the validation when the number is not an integer:
Screenshot 2024-10-16 at 12 16 47

And the index_lifecycle_management still shows the validations as expected:

Screenshot 2024-10-16 at 11 49 53

@SoniaSanzV SoniaSanzV self-assigned this Oct 16, 2024
@SoniaSanzV SoniaSanzV requested a review from a team as a code owner October 16, 2024 10:41
@SoniaSanzV SoniaSanzV added Feature:Index Management Index and index templates UI Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more release_note:skip Skip the PR/issue when compiling release notes v9.0.0 Feature:Ingest Node Pipelines Ingest node pipelines management backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) labels Oct 16, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-management (Team:Kibana Management)

@SoniaSanzV SoniaSanzV force-pushed the #110417_CommunityId_intValidation branch from 427762d to 431f535 Compare October 17, 2024 09:40
@SoniaSanzV SoniaSanzV removed the Feature:Index Management Index and index templates UI label Oct 17, 2024
@ElenaStoeva ElenaStoeva self-requested a review October 17, 2024 15:07
Copy link
Contributor

@ElenaStoeva ElenaStoeva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing job @SoniaSanzV! Many thanks for adding unit tests and jest tests for the processor! Tested locally and it works well. I only have a suggestion on the validation logic.


const numValue = typeof value === 'string' ? Number(value) : value;

return Number.isInteger(numValue) ? undefined : { message, code: 'ERR_NOT_INT_NUMBER' };
Copy link
Contributor

@ElenaStoeva ElenaStoeva Oct 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am thinking of a case where this might not work properly - if value is true or false, Number(value) would evaluate to a number 1 or 0, which is an integer.

From what I understand, we only return undefined in these three cases:

  • If value is an empty string
  • If value is of type number and it is an integer
  • If value is of type string and represents an integer
    Otherwise returning the error message.

Following the above rules, I think the logic can be simplified to something like this:

  if (
    value === '' ||
    (typeof value === 'number' && Number.isInteger(value)) || 
    (typeof value === 'string' && Number.isInteger(Number(value)))
  ) {
    return undefined;
  }

  return { message, code: 'ERR_NOT_INT_NUMBER' };

wdyt?

Copy link
Contributor Author

@SoniaSanzV SoniaSanzV Oct 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank for your comments, Elena!

I don't think it would be a problem in the case of a boolean. If value is a boolean it won't do Number(value) because it only do so for typeof string. If type is a boolean the evaluation will be Number.isInteger(false) and then will return { message, code: 'ERR_NOT_INT_NUMBER' } (I now realize that the name of the const numValue is not good because it's not always a number, like this boolean case)

In any case, probably your option is more precise so I can change it. And I'll add the boolean cases in the UTs too, I think they are a good case to be tested.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right about the booleans, I missed that it will directly evaluate to Number.isInteger(false)!

I guess my suggestion was mainly around simplifying the logic so that the cases where we return undefined are more explicit and readable.

And I'll add the boolean cases in the UTs too, I think they are a good case to be tested.

Yes, that would be great, thanks!

@SoniaSanzV SoniaSanzV force-pushed the #110417_CommunityId_intValidation branch from 431f535 to 81a7eda Compare October 18, 2024 08:36
Copy link
Contributor

@ElenaStoeva ElenaStoeva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @SoniaSanzV, sorry for the delay with my review! Latest changes lgtm, thanks for addressing my feedback!

Before we merge this, I know one more place which would benefit from this new validator! 😄 We just added integer validation for the data stream retention period modal (#196524) - could you please replace the custom integer validation here with the new es_ui_shared validation?

@SoniaSanzV SoniaSanzV force-pushed the #110417_CommunityId_intValidation branch from 2b9bea2 to c43b0a7 Compare October 24, 2024 13:57
@SoniaSanzV
Copy link
Contributor Author

As @ElenaStoeva requested, the validator has also been used in the EditDataRetentionModal.

validatorDataRetention

Copy link
Contributor

@ElenaStoeva ElenaStoeva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job @SoniaSanzV! Tested locally again and works as expected 🎉

@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
esUiShared 220 221 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
indexLifecycleManagement 165.9KB 166.0KB +120.0B
indexManagement 688.4KB 688.3KB -49.0B
ingestPipelines 404.6KB 404.8KB +202.0B
total +273.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
esUiShared 106.7KB 106.9KB +235.0B

History

  • 💚 Build #243739 succeeded 81a7eda3f4970a19ce1da5c0cd39c147754bbb3d
  • 💚 Build #243400 succeeded 431f53542adcbf6534769a38079cf468436550d6
  • 💔 Build #243031 failed 427762d7a68f648b6bbd126c05dba77f3442c58c

cc @SoniaSanzV

@SoniaSanzV SoniaSanzV merged commit 5a42e58 into elastic:main Oct 24, 2024
24 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11503207094

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 24, 2024
…x_lifecycle_management (elastic#196527)

Closes [elastic#110417 ](elastic#110417)
## Summary
In the Ingest Node Pipelines section, when the users created a new
pipeline selecting de Community ID processor the users could set a
non-integer number in this field. Then, they received a server side
error when tried to create a pipeline. For fixing this, a validation
must be added in the client.

We didn't have a reusable validation for this case, but we did have a
custom validation for integer values in the Index lifecycle management
plugin. We also had the necessary translation in that plugin. So I went
forward with:

* I created a new integer validator in the `es_ui_shared` package as it
is a fairly common validation and we could take advantage of it in the
future. Also added a couple of unit test there for this validator.

* I reused in the `ingest_pipelines` plugin the strings that already
existed in `index_lifecycle_management`.

* I added the new validation in the Community ID form in the
`ingest_pipelines` plugin. Also added some test verifying that the
processor doesn't create when the seeds validation fails.

* Changed the method in the `index_lifecycle_management` validator so
now it uses the reusable one.

Now the Ingest pipeline forms shows the validation when the number is
not an integer:
![Screenshot 2024-10-16 at 12 16
47](https://github.com/user-attachments/assets/1db9ad22-b144-44a5-9012-d3ebd5a19b6f)

And the `index_lifecycle_management` still shows the validations as
expected:

<img width="756" alt="Screenshot 2024-10-16 at 11 49 53"
src="https://github.com/user-attachments/assets/680886a5-355e-4637-9da9-4b93b396e751">

(cherry picked from commit 5a42e58)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Oct 24, 2024
…d Index_lifecycle_management (#196527) (#197685)

# Backport

This will backport the following commits from `main` to `8.x`:
- [Create a common Int Validator and use it in ingest_pipelines and
Index_lifecycle_management
(#196527)](#196527)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Sonia Sanz
Vivas","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-10-24T16:09:38Z","message":"Create
a common Int Validator and use it in ingest_pipelines and
Index_lifecycle_management (#196527)\n\nCloses [#110417
](https://github.com/elastic/kibana/issues/110417)\r\n## Summary\r\nIn
the Ingest Node Pipelines section, when the users created a
new\r\npipeline selecting de Community ID processor the users could set
a\r\nnon-integer number in this field. Then, they received a server
side\r\nerror when tried to create a pipeline. For fixing this, a
validation\r\nmust be added in the client.\r\n\r\nWe didn't have a
reusable validation for this case, but we did have a\r\ncustom
validation for integer values in the Index lifecycle
management\r\nplugin. We also had the necessary translation in that
plugin. So I went\r\nforward with:\r\n\r\n* I created a new integer
validator in the `es_ui_shared` package as it\r\nis a fairly common
validation and we could take advantage of it in the\r\nfuture. Also
added a couple of unit test there for this validator.\r\n\r\n* I reused
in the `ingest_pipelines` plugin the strings that already\r\nexisted in
`index_lifecycle_management`.\r\n\r\n* I added the new validation in the
Community ID form in the\r\n`ingest_pipelines` plugin. Also added some
test verifying that the\r\nprocessor doesn't create when the seeds
validation fails.\r\n\r\n* Changed the method in the
`index_lifecycle_management` validator so\r\nnow it uses the reusable
one.\r\n\r\nNow the Ingest pipeline forms shows the validation when the
number is\r\nnot an integer:\r\n![Screenshot 2024-10-16 at 12
16\r\n47](https://github.com/user-attachments/assets/1db9ad22-b144-44a5-9012-d3ebd5a19b6f)\r\n\r\nAnd
the `index_lifecycle_management` still shows the validations
as\r\nexpected:\r\n\r\n<img width=\"756\" alt=\"Screenshot 2024-10-16 at
11 49
53\"\r\nsrc=\"https://github.com/user-attachments/assets/680886a5-355e-4637-9da9-4b93b396e751\">","sha":"5a42e5861dc53911e10a3140d8cb5366fea98ff7","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Kibana
Management","release_note:skip","v9.0.0","Feature:Ingest Node
Pipelines","backport:prev-minor","v8.17.0"],"title":"Create a common Int
Validator and use it in ingest_pipelines and
Index_lifecycle_management","number":196527,"url":"https://github.com/elastic/kibana/pull/196527","mergeCommit":{"message":"Create
a common Int Validator and use it in ingest_pipelines and
Index_lifecycle_management (#196527)\n\nCloses [#110417
](https://github.com/elastic/kibana/issues/110417)\r\n## Summary\r\nIn
the Ingest Node Pipelines section, when the users created a
new\r\npipeline selecting de Community ID processor the users could set
a\r\nnon-integer number in this field. Then, they received a server
side\r\nerror when tried to create a pipeline. For fixing this, a
validation\r\nmust be added in the client.\r\n\r\nWe didn't have a
reusable validation for this case, but we did have a\r\ncustom
validation for integer values in the Index lifecycle
management\r\nplugin. We also had the necessary translation in that
plugin. So I went\r\nforward with:\r\n\r\n* I created a new integer
validator in the `es_ui_shared` package as it\r\nis a fairly common
validation and we could take advantage of it in the\r\nfuture. Also
added a couple of unit test there for this validator.\r\n\r\n* I reused
in the `ingest_pipelines` plugin the strings that already\r\nexisted in
`index_lifecycle_management`.\r\n\r\n* I added the new validation in the
Community ID form in the\r\n`ingest_pipelines` plugin. Also added some
test verifying that the\r\nprocessor doesn't create when the seeds
validation fails.\r\n\r\n* Changed the method in the
`index_lifecycle_management` validator so\r\nnow it uses the reusable
one.\r\n\r\nNow the Ingest pipeline forms shows the validation when the
number is\r\nnot an integer:\r\n![Screenshot 2024-10-16 at 12
16\r\n47](https://github.com/user-attachments/assets/1db9ad22-b144-44a5-9012-d3ebd5a19b6f)\r\n\r\nAnd
the `index_lifecycle_management` still shows the validations
as\r\nexpected:\r\n\r\n<img width=\"756\" alt=\"Screenshot 2024-10-16 at
11 49
53\"\r\nsrc=\"https://github.com/user-attachments/assets/680886a5-355e-4637-9da9-4b93b396e751\">","sha":"5a42e5861dc53911e10a3140d8cb5366fea98ff7"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/196527","number":196527,"mergeCommit":{"message":"Create
a common Int Validator and use it in ingest_pipelines and
Index_lifecycle_management (#196527)\n\nCloses [#110417
](https://github.com/elastic/kibana/issues/110417)\r\n## Summary\r\nIn
the Ingest Node Pipelines section, when the users created a
new\r\npipeline selecting de Community ID processor the users could set
a\r\nnon-integer number in this field. Then, they received a server
side\r\nerror when tried to create a pipeline. For fixing this, a
validation\r\nmust be added in the client.\r\n\r\nWe didn't have a
reusable validation for this case, but we did have a\r\ncustom
validation for integer values in the Index lifecycle
management\r\nplugin. We also had the necessary translation in that
plugin. So I went\r\nforward with:\r\n\r\n* I created a new integer
validator in the `es_ui_shared` package as it\r\nis a fairly common
validation and we could take advantage of it in the\r\nfuture. Also
added a couple of unit test there for this validator.\r\n\r\n* I reused
in the `ingest_pipelines` plugin the strings that already\r\nexisted in
`index_lifecycle_management`.\r\n\r\n* I added the new validation in the
Community ID form in the\r\n`ingest_pipelines` plugin. Also added some
test verifying that the\r\nprocessor doesn't create when the seeds
validation fails.\r\n\r\n* Changed the method in the
`index_lifecycle_management` validator so\r\nnow it uses the reusable
one.\r\n\r\nNow the Ingest pipeline forms shows the validation when the
number is\r\nnot an integer:\r\n![Screenshot 2024-10-16 at 12
16\r\n47](https://github.com/user-attachments/assets/1db9ad22-b144-44a5-9012-d3ebd5a19b6f)\r\n\r\nAnd
the `index_lifecycle_management` still shows the validations
as\r\nexpected:\r\n\r\n<img width=\"756\" alt=\"Screenshot 2024-10-16 at
11 49
53\"\r\nsrc=\"https://github.com/user-attachments/assets/680886a5-355e-4637-9da9-4b93b396e751\">","sha":"5a42e5861dc53911e10a3140d8cb5366fea98ff7"}},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Sonia Sanz Vivas <[email protected]>
@SoniaSanzV SoniaSanzV deleted the #110417_CommunityId_intValidation branch October 25, 2024 13:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) Feature:Ingest Node Pipelines Ingest node pipelines management release_note:skip Skip the PR/issue when compiling release notes Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more v8.17.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Ingest Node Pipelines] Community ID Form Does Not Validate Against Decimal Values for Seed Parameter
4 participants