Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[5.8] Fixed queue jobs using SerializesModels losing order of passed in collections #29136

Merged

Conversation

itsgoingd
Copy link
Contributor

SerializesModels is a trait for queue jobs that converts passed in models and collections to keys on serialization and restores them back from database when unserialized.

The current implementation loses the intended order for passed in collections.

Eg. let's say we have a queue job exporting users sorted by email address.

User::create([ 'email' => '[email protected]' ]);
User::create([ 'email' => '[email protected]' ]);

$users = User::orderBy('name')->get();
// SELECT * FROM users ORDER BY name
// [ [ 'id' => 2, 'email' => '[email protected] ], [ 'id' => 1, 'email' => '[email protected]' ] ]

$serialized = serialize(new Export($users));

Now on unserialization, Laravel doesn't know about the order clause and runs a simple WHERE IN query, which returns the records in the original order as inserted into db.

$unserialized = unserialize($serialized);
// SELECT * FROM users WHERE id IN (2, 1)
// [ [ 'id' => 1, 'email' => '[email protected]' ], [ 'id' => 2, 'email' => '[email protected] ] ]

The proposed fix adds an extra step where the restored collection is re-mapped based on the serialized collection of keys which is in the correct order, thus restoring the intended order.

Included an integration test demonstrating the issue.

@devcircus
Copy link
Contributor

Are there performance implications on large collections with this? I'd rather default to the current implementation and order, if necessary, in the job itself.

@devcircus
Copy link
Contributor

Serialization tests are failing.

@itsgoingd
Copy link
Contributor Author

Fixed the failing tests by skipping the order recovery step for pivot records.

Kinda share the performance concern, but with collections of tens or hundreds of thousands of items you'd run into memory limitations before cpu imo. The memory footprint shouldn't be too bad since both original and sorted collections are referencing the same objects. But I'm just guessing. 🙃

The current fix makes for some awkward code, maybe someone else has a better idea?

Alternatively we could document this as a known limitation as it's definitively an unexpected behaviour atm.

@driesvints
Copy link
Member

StyleCI is failing.

@itsgoingd
Copy link
Contributor Author

Sorry, thought these get merged automatically, fixed now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants