Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a JsonNode roundtrip benchmark #2714

Merged
merged 1 commit into from
Nov 11, 2022

Conversation

eiriktsarpalis
Copy link
Member

We don't have any benchmarks tracking JsonNode performance, adding one here. Should help us catch performance regressions like the one reported in dotnet/runtime#78089.

.NET 7

Method IsDataIndented TestCase Mean Error StdDev Median Min Max Gen 0 Gen 1 Gen 2 Allocated
ParseThenWrite False HelloWorld 1.629 us 0.1034 us 0.1191 us 1.631 us 1.445 us 1.861 us 0.1127 0.0059 0.0059 1.21 KB
ParseThenWrite False DeepTree 71.814 us 5.7978 us 6.6767 us 71.981 us 61.361 us 83.650 us 3.9261 0.4619 0.2309 41.39 KB
ParseThenWrite False BroadTree 143.157 us 9.4310 us 10.8607 us 144.141 us 124.357 us 161.210 us 7.6923 1.0989 0.5495 85.58 KB
ParseThenWrite False LotsOfNumbers 33.528 us 3.2085 us 3.4331 us 33.050 us 28.688 us 40.852 us 1.8508 - - 20.65 KB
ParseThenWrite False LotsOfStrings 30.900 us 2.3209 us 2.6727 us 30.905 us 25.774 us 36.619 us 1.6572 0.1184 0.1184 17.37 KB
ParseThenWrite False Json400B 13.306 us 1.0877 us 1.2526 us 13.080 us 11.222 us 15.969 us 0.7189 0.0479 0.0479 7.54 KB
ParseThenWrite False Json4KB 98.384 us 5.3244 us 5.9181 us 97.605 us 87.727 us 109.771 us 5.3248 0.7100 0.3550 55.93 KB
ParseThenWrite False Json400KB 11,597.244 us 1,219.7139 us 1,305.0802 us 11,788.595 us 9,782.182 us 14,438.336 us 545.4545 454.5455 90.9091 5419.64 KB
ParseThenWrite True HelloWorld 1.652 us 0.0918 us 0.1058 us 1.640 us 1.475 us 1.908 us 0.1112 0.0062 0.0062 1.2 KB
ParseThenWrite True DeepTree 79.871 us 5.4731 us 6.3029 us 79.029 us 69.537 us 91.046 us 4.1667 0.5556 0.2778 44.28 KB
ParseThenWrite True BroadTree 127.201 us 8.7583 us 9.3713 us 127.193 us 114.161 us 147.240 us 8.2237 1.6447 0.5482 87.66 KB
ParseThenWrite True LotsOfNumbers 33.675 us 2.3780 us 2.7385 us 34.034 us 27.549 us 39.641 us 1.9960 0.1248 0.1248 20.93 KB
ParseThenWrite True LotsOfStrings 30.747 us 1.8929 us 2.1799 us 30.955 us 25.725 us 34.167 us 1.6681 - - 17.75 KB
ParseThenWrite True Json400B 13.257 us 0.9355 us 1.0774 us 13.175 us 11.739 us 15.118 us 0.6960 0.0464 0.0464 7.65 KB
ParseThenWrite True Json4KB 96.272 us 7.9884 us 9.1994 us 93.730 us 78.500 us 111.757 us 5.4054 0.7207 0.3604 56.99 KB
ParseThenWrite True Json400KB 10,162.729 us 732.9558 us 814.6784 us 10,055.850 us 8,930.960 us 11,691.760 us 500.0000 400.0000 100.0000 5543.76 KB

.NET 8 (after fix)

Method IsDataIndented TestCase Mean Error StdDev Median Min Max Gen 0 Gen 1 Gen 2 Allocated
ParseThenWrite False HelloWorld 611.4 ns 19.22 ns 20.56 ns 609.1 ns 577.1 ns 647.6 ns 0.0702 - - 712 B
ParseThenWrite False DeepTree 30,617.2 ns 1,239.33 ns 1,326.07 ns 30,430.6 ns 27,339.7 ns 33,301.3 ns 2.1149 - - 22192 B
ParseThenWrite False BroadTree 52,627.7 ns 2,671.54 ns 3,076.55 ns 52,192.6 ns 47,220.4 ns 58,579.6 ns 4.2017 0.4202 - 43216 B
ParseThenWrite False LotsOfNumbers 13,226.7 ns 734.68 ns 846.05 ns 13,009.4 ns 11,974.5 ns 15,061.0 ns 1.0400 - - 10784 B
ParseThenWrite False LotsOfStrings 7,290.9 ns 317.29 ns 365.39 ns 7,300.4 ns 6,686.2 ns 8,056.4 ns 0.5378 - - 5480 B
ParseThenWrite False Json400B 5,196.0 ns 252.67 ns 280.85 ns 5,206.3 ns 4,710.1 ns 5,748.4 ns 0.4109 - - 4240 B
ParseThenWrite False Json4KB 36,775.5 ns 1,880.67 ns 2,012.30 ns 37,072.4 ns 32,836.0 ns 40,909.0 ns 2.7498 0.1528 - 28504 B
ParseThenWrite False Json400KB 6,355,027.6 ns 597,473.16 ns 664,089.84 ns 6,320,892.1 ns 5,047,955.3 ns 7,476,928.9 ns 315.7895 263.1579 105.2632 2803212 B
ParseThenWrite True HelloWorld 655.1 ns 25.97 ns 28.86 ns 658.6 ns 608.7 ns 709.1 ns 0.0699 - - 712 B
ParseThenWrite True DeepTree 39,053.1 ns 2,295.58 ns 2,643.59 ns 38,891.6 ns 34,736.8 ns 44,070.4 ns 2.3718 0.1482 - 24992 B
ParseThenWrite True BroadTree 56,388.3 ns 1,863.04 ns 2,070.77 ns 56,406.7 ns 51,892.5 ns 61,454.6 ns 4.3270 0.4555 - 45352 B
ParseThenWrite True LotsOfNumbers 13,440.2 ns 471.28 ns 504.26 ns 13,381.9 ns 12,777.5 ns 14,620.3 ns 1.0434 - - 11072 B
ParseThenWrite True LotsOfStrings 8,390.4 ns 358.16 ns 412.46 ns 8,255.7 ns 7,722.6 ns 9,254.7 ns 0.5560 - - 5792 B
ParseThenWrite True Json400B 5,224.6 ns 178.13 ns 197.99 ns 5,224.6 ns 4,984.9 ns 5,672.0 ns 0.4225 - - 4336 B
ParseThenWrite True Json4KB 38,281.5 ns 1,744.40 ns 2,008.85 ns 37,902.4 ns 35,839.9 ns 41,678.9 ns 2.7941 0.1471 - 29536 B
ParseThenWrite True Json400KB 5,878,425.8 ns 1,287,095.94 ns 1,482,221.60 ns 5,264,561.1 ns 4,362,961.1 ns 8,828,844.4 ns 222.2222 111.1111 55.5556 2906385 B

Comment on lines +40 to +47
if (!IsDataIndented)
{
_dataUtf8 = DocumentHelpers.RemoveFormatting(jsonString);
}
else
{
_dataUtf8 = Encoding.UTF8.GetBytes(jsonString);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (!IsDataIndented)
{
_dataUtf8 = DocumentHelpers.RemoveFormatting(jsonString);
}
else
{
_dataUtf8 = Encoding.UTF8.GetBytes(jsonString);
}
_dataUtf8 = IsDataIndented ?
Encoding.UTF8.GetBytes(jsonString) :
DocumentHelpers.RemoveFormatting(jsonString);

?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initialization code is copypasta'd from the equivalent JsonDocument benchmark (also called ParseThenWrite.cs).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initialization code is copypasta'd from the equivalent

FWIW Setup and cleanup code can be always refactored and improved (duplicated code moved to helper type), as changing it typically does not affect the benchmark itself

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants