Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: remove old marker logic #2730

Merged
merged 8 commits into from
Sep 10, 2024
Merged

perf: remove old marker logic #2730

merged 8 commits into from
Sep 10, 2024

Conversation

roggervalf
Copy link
Collaborator

@roggervalf roggervalf commented Aug 24, 2024

No need to support old marker logic removal as markers are living in a different key now in recent versions
Also we should validate if a jobId contains : as this separator could break flow logic

const [prefix, queueName, id] = key.split(':');

@roggervalf roggervalf changed the base branch from master to v6 August 24, 2024 13:40
@manast
Copy link
Contributor

manast commented Aug 26, 2024

What would be the proper way to migrate from a version older than v5 to v6? for instance, if we upgrade from v4, then to v5 and then to v6, there could still be markers left, and break the queue.

@roggervalf
Copy link
Collaborator Author

What would be the proper way to migrate from a version older than v5 to v6? for instance, if we upgrade from v4, then to v5 and then to v6, there could still be markers left, and break the queue.

I added a method to remove legacy markers in case someone come from an older version

@roggervalf roggervalf changed the base branch from v6 to master August 27, 2024 04:21
@manast
Copy link
Contributor

manast commented Aug 27, 2024

I added a method to remove legacy markers in case someone come from an older version

I think we also need a code example on how to do the migration. For instance, when should this method be called, and also is there a risk that the queue gets stuck before calling it? maybe we need to do a pause/remove markers/resume cycle or something like this.

@roggervalf
Copy link
Collaborator Author

I added a method to remove legacy markers in case someone come from an older version

I think we also need a code example on how to do the migration. For instance, when should this method be called, and also is there a risk that the queue gets stuck before calling it? maybe we need to do a pause/remove markers/resume cycle or something like this.

code example could be added in migration guide similar to what was done in previous breaking changes like https://bullmq.io/news/062123/faster-priority-jobs/ where removeDeprecatedPriorityKey was mentioned. This method is similar as it's also intended to be used one time. I also added a test case to demonstrate that queues won't get stuck and having queues in a paused state will be the best option to not process that legacy marker as a regular job.

@roggervalf roggervalf changed the base branch from master to v6 August 31, 2024 14:38
Copy link
Contributor

@manast manast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@roggervalf roggervalf merged commit dbef80d into v6 Sep 10, 2024
10 checks passed
@roggervalf roggervalf deleted the remove-old-marker branch September 10, 2024 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants