Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(db-mongodb): improve performance of all operations, up to 50% faster #9594

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

r1tsuu
Copy link
Member

@r1tsuu r1tsuu commented Nov 28, 2024

This PR improves speed and memory efficiency across all operations with the Mongoose adapter.

How?

  • Removes Mongoose layer from all database calls, instead uses MongoDB directly.
  • Replaces deep copying of read results using JSON.parse(JSON.stringify(data)) with the transform operation: 'read' function which converts Date's, ObjectID's in relationships / joins to strings. As before, it also handles transformations for write operations.
  • Faster hasNearConstraint for potentially large where's
  • traverseFields now can accept flattenedFields which we use in transform. Less recursive calls with tabs/rows/collapsible

Additional fixes

  • Uses current transaction for querying nested relationships properties in buildQuery, previously it wasn't used which could've led to wrong results
  • Allows to clear not required point fields with passing null from the Local API. Previously it didn't work in both, MongoDB and Postgres

Benchmarks using this file https://github.com/payloadcms/payload/blob/chore/db-benchmark/test/_community/int.spec.ts

Small Dataset Performance

Metric Before Optimization After Optimization Improvement (%)
Average FULL (ms) 1170 844 27.86%
payload.db.create (ms) 1413 691 51.12%
payload.db.find (ms) 2856 2204 22.83%
payload.db.deleteMany (ms) 15206 8439 44.53%
payload.db.updateOne (ms) 21444 12162 43.30%
payload.db.findOne (ms) 159 112 29.56%
payload.db.deleteOne (ms) 3729 2578 30.89%
DB small FULL (ms) 64473 46451 27.93%

Medium Dataset Performance

Metric Before Optimization After Optimization Improvement (%)
Average FULL (ms) 9407 6210 33.99%
payload.db.create (ms) 10270 4321 57.93%
payload.db.find (ms) 20814 16036 22.93%
payload.db.deleteMany (ms) 126351 61789 51.11%
payload.db.updateOne (ms) 201782 99943 50.49%
payload.db.findOne (ms) 1081 817 24.43%
payload.db.deleteOne (ms) 28534 23363 18.12%
DB medium FULL (ms) 519518 342194 34.13%

Large Dataset Performance

Metric Before Optimization After Optimization Improvement (%)
Average FULL (ms) 26575 17509 34.14%
payload.db.create (ms) 29085 12196 58.08%
payload.db.find (ms) 58497 43838 25.04%
payload.db.deleteMany (ms) 372195 173218 53.47%
payload.db.updateOne (ms) 544089 288350 47.00%
payload.db.findOne (ms) 3058 2197 28.14%
payload.db.deleteOne (ms) 82444 64730 21.49%
DB large FULL (ms) 1461097 969714 33.62%

@r1tsuu r1tsuu changed the title perf(db-mongdb): improve performance of all operations, up to 50% faster perf(db-mongodb): improve performance of all operations, up to 50% faster Nov 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant