Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLEET] Increase asset size limit for installing ML model packages #115890

Conversation

alvarezmelissa87
Copy link
Contributor

@alvarezmelissa87 alvarezmelissa87 commented Oct 21, 2021

Summary

Increases asset size limit for ML model packages.
This is a potentially temporary change to accommodate DGA model (which is about 45mb) installation - the 4mb limit is kept for all other asset types as per elastic/elastic-agent-client#17

This could be improved on in the future if necessary by making the limit an option to be passed in or other options discussed in the above issue.

Checklist

Delete any items that are not applicable to this PR.

For maintainers

@alvarezmelissa87 alvarezmelissa87 self-assigned this Oct 21, 2021
@ruflin ruflin requested a review from joshdover October 21, 2021 07:02
Comment on lines 32 to 35
// could be anything, picked this from https://github.com/elastic/elastic-agent-client/issues/17
const MAX_ES_ASSET_BYTES = 4 * 1024 * 1024;
// Updated to accomodate larger package size in some ML model packages
const ML_MAX_ES_ASSET_BYTES = 50 * 1024 * 1024;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: could we use a ONE_BYTE constant to make this more clear:

Suggested change
// could be anything, picked this from https://github.com/elastic/elastic-agent-client/issues/17
const MAX_ES_ASSET_BYTES = 4 * 1024 * 1024;
// Updated to accomodate larger package size in some ML model packages
const ML_MAX_ES_ASSET_BYTES = 50 * 1024 * 1024;
const ONE_BYTE = 1024 * 1024;
// could be anything, picked this from https://github.com/elastic/elastic-agent-client/issues/17
const MAX_ES_ASSET_BYTES = 4 * ONE_BYTE;
// Updated to accommodate larger package size in some ML model packages
const ML_MAX_ES_ASSET_BYTES = 50 * ONE_BYTE;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking a look! 😄 Updated in 2706fd0

@alvarezmelissa87 alvarezmelissa87 force-pushed the ml-models-in-fleet-increase-size-limit branch from 7c8e11a to d4d56ad Compare October 21, 2021 15:00
@alvarezmelissa87 alvarezmelissa87 marked this pull request as ready for review October 21, 2021 16:39
@alvarezmelissa87 alvarezmelissa87 requested a review from a team as a code owner October 21, 2021 16:39
@alvarezmelissa87 alvarezmelissa87 added :ml Team:Fleet Team label for Observability Data Collection Fleet team v8.0.0 labels Oct 21, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

@alvarezmelissa87 alvarezmelissa87 added release_note:enhancement release_note:skip Skip the PR/issue when compiling release notes and removed release_note:enhancement labels Oct 21, 2021
Copy link
Member

@kpollich kpollich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once CI is green 🚀

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

  • 💔 Build #786 failed 7c8e11ae60c1c5c604c8f1687a9454ba6bc8f756

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @alvarezmelissa87

@alvarezmelissa87 alvarezmelissa87 merged commit 29e5a3a into elastic:master Oct 21, 2021
@alvarezmelissa87 alvarezmelissa87 deleted the ml-models-in-fleet-increase-size-limit branch October 21, 2021 20:19
shivindera pushed a commit to shivindera/kibana that referenced this pull request Oct 25, 2021
…lastic#115890)

* update size limit for assets to 50mb

* use different size limit for ml model

* add byte constant for clarity
@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Oct 25, 2021
@kibanamachine
Copy link
Contributor

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create backports run node scripts/backport --pr 115890 or prevent reminders by adding the backport:skip label.

@peteharverson peteharverson added backport:skip This commit does not require backporting and removed backport missing Added to PRs automatically when the are determined to be missing a backport. labels Oct 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting :ml release_note:skip Skip the PR/issue when compiling release notes Team:Fleet Team label for Observability Data Collection Fleet team v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants