Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove rollup functionality #112690

Closed
martijnvg opened this issue Sep 10, 2024 · 7 comments
Closed

Remove rollup functionality #112690

martijnvg opened this issue Sep 10, 2024 · 7 comments
Labels
>breaking :StorageEngine/Rollup Turn fine-grained time-based data into coarser-grained data Team:StorageEngine

Comments

@martijnvg
Copy link
Member

Rollup has been deprecated since version 8.11.0 (almost a year ago) en since version 8.15.0 it is no longer possible to create new jobs in cluster without rollup usage. We plan to remove rollup functionality as part of the 9.0 release.
@martijnvg martijnvg added :StorageEngine/Rollup Turn fine-grained time-based data into coarser-grained data >breaking labels Sep 10, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

@kertal kertal mentioned this issue Sep 10, 2024
Open
3 tasks
@thomasneirynck
Copy link
Contributor

@martijnvg

ES-Clusters in 8.x which have rollup indices, would they be able to upgrade to 9.0, or would this upgrade fail?

@martijnvg
Copy link
Member Author

martijnvg commented Sep 10, 2024 •
edited
Loading

ES-Clusters in 8.x which have rollup indices, would they be able to upgrade to 9.0, or would this upgrade fail?

@thomasneirynck No, these clusters would need to migrate to downsampling before upgrading to 9.0.

@lukeelmers
Copy link
Member

these clusters would need to migrate to downsampling before upgrading to 9.0.

@martijnvg Are you saying the existing rollup indices would be deleted, or just that the rollup jobs would fail to run but existing indices would still be accessible? Is ES logging a deprecation when it sees clusters with rollup jobs configured?

@martijnvg
Copy link
Member Author

Are you saying the existing rollup indices would be deleted, or just that the rollup jobs would fail to run but existing indices would still be accessible?

I think we still need to determine this. If we chose to keep the ability to read rollup indices we need to at least keep some of the rollup functionality, but it could make migrating to 9.0 easier.

Is ES logging a deprecation when it sees clusters with rollup jobs configured?

Currently there non deprecation warnings in Elasticsearch. It is deprecated in the docs and from 8.15.0 clusters with no rollup usage can't define new rollup jobs (I believe rollup ui in kibana also was updated to show meaningful warnings/errors).

@thomasneirynck
Copy link
Contributor

@martijnvg

wrt

If we chose to keep the ability to read rollup indices we need to at least keep some of the rollup functionality, but it could make migrating to 9.0 easier

do we know where we can track this discussion?

martijnvg added a commit to martijnvg/elasticsearch that referenced this issue Sep 12, 2024
@martijnvg
Emit deprecation warning when executing one of the rollup APIs
fdcc3fe 
Relates to elastic#112690
@martijnvg martijnvg mentioned this issue Sep 12, 2024
Merged
@kertal kertal mentioned this issue Sep 13, 2024
Closed
martijnvg added a commit that referenced this issue Sep 17, 2024
@martijnvg
Emit deprecation warning when executing one of the rollup APIs (#112807)
3d6e9a2 
Currently none of the rollup APIs deprecation warning. Only the reference docs indicate that rollup has been deprecated.

Relates to #112690
martijnvg added a commit to martijnvg/elasticsearch that referenced this issue Sep 17, 2024
@martijnvg
Emit deprecation warning when executing one of the rollup APIs (elast…
fb9261d 
…ic#112807)

Currently none of the rollup APIs deprecation warning. Only the reference docs indicate that rollup has been deprecated.

Relates to elastic#112690
martijnvg added a commit to martijnvg/elasticsearch that referenced this issue Sep 17, 2024
@martijnvg
[8.15] Emit deprecation warning when executing one of the rollup APIs
30da624 
Backporting elastic#112807 to 8.15 branch.
Currently none of the rollup APIs deprecation warning. Only the reference docs indicate that rollup has been deprecated.

Relates to elastic#112690
@martijnvg martijnvg mentioned this issue Sep 17, 2024
Merged
elasticsearchmachine pushed a commit that referenced this issue Sep 17, 2024
@martijnvg
[8.15] Emit deprecation warning when executing one of the rollup APIs (…
b80b018 
…#113042)

* [8.15] Emit deprecation warning when executing one of the rollup APIs

Backporting #112807 to 8.15 branch.
Currently none of the rollup APIs deprecation warning. Only the reference docs indicate that rollup has been deprecated.

Relates to #112690

* Fixed compile errors after cherry-picking from main branch.

* fix issue after merging

* added some more allowed warnings
javanna pushed a commit to javanna/elasticsearch that referenced this issue Sep 18, 2024
@martijnvg @javanna
Emit deprecation warning when executing one of the rollup APIs (elast…
1a33e55 
…ic#112807)

Currently none of the rollup APIs deprecation warning. Only the reference docs indicate that rollup has been deprecated.

Relates to elastic#112690
martijnvg added a commit to martijnvg/elasticsearch that referenced this issue Sep 18, 2024
@martijnvg
Emit deprecation warning when executing one of the rollup APIs
d931c0e 
Attempt 2.

Relates to elastic#112690
@martijnvg martijnvg mentioned this issue Sep 18, 2024
Open
@martijnvg
Copy link
Member Author

Closing this issue. Rollup functionality will not be removed as part of 9.0 release, but will stay around in deprecated form. The next moment that we will decide to remove rollup will be around the 10.0 release timeline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
>breaking :StorageEngine/Rollup Turn fine-grained time-based data into coarser-grained data Team:StorageEngine
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@martijnvg @lukeelmers @thomasneirynck @elasticsearchmachine