Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use orjson for our json renderer #11675

Merged
merged 1 commit into from
Jun 26, 2022
Merged

Use orjson for our json renderer #11675

merged 1 commit into from
Jun 26, 2022

Conversation

dstufft
Copy link
Member

@dstufft dstufft commented Jun 26, 2022

This switches our pyramid renderer from using the stdlib json module, to using orjson instead.

I ran benchmarks testing the speed of dumps on the JSON version of /simple/ from live PyPI, and I got these results:

--------------------------------------------------------------------------- benchmark 'pypi.json serialization': 2 tests ---------------------------------------------------------------------------
Name (time in ms)                     Min                 Max                Mean            StdDev              Median               IQR            Outliers      OPS            Rounds  Iterations
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_dumps[pypi.json-orjson]      19.4031 (1.0)       20.7105 (1.0)       19.8266 (1.0)      0.5764 (1.0)       19.4697 (1.0)      0.8415 (1.0)           1;0  50.4373 (1.0)           5         100
test_dumps[pypi.json-json]       120.4422 (6.21)     122.5178 (5.92)     121.7277 (6.14)     0.8201 (1.42)     121.8010 (6.26)     1.1179 (1.33)          1;0   8.2151 (0.16)          5          10
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

This should make us something like 6x faster on serializing our json endpoints according to the above benchmarks.

The one gotcha with using orjson that makes it not a drop in replacement for json is that instead of returning a str it returns bytes. This isn't a big deal for us as Pyramid's render system supports both, and will just encode str results anyways into bytes.

@dstufft dstufft merged commit 9f9f3d9 into pypi:main Jun 26, 2022
@dstufft dstufft deleted the orjson branch June 26, 2022 17:29
dstufft added a commit that referenced this pull request Jun 27, 2022
ewdurbin pushed a commit that referenced this pull request Jun 27, 2022
dstufft added a commit to dstufft/warehouse that referenced this pull request Jul 8, 2022
ewdurbin pushed a commit that referenced this pull request Jul 8, 2022
* Revert "Revert "Use orjson for our json renderer (#11675)" (#11697)"

This reverts commit 6d39d8d.

* Upgrade orjson
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant