Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding/json: Add Custom JSON Package with Build Tag Support for Sonic #1623

Open
wants to merge 28 commits into
base: master
Choose a base branch
from

Conversation

shazbert
Copy link
Collaborator

PR Description

This pull request introduces a custom JSON package located in encoding/json/ to facilitate swapping between the default Go encoding/json package and the sonic library based on build tags.

Adds:

  • json.go (default implementation using encoding/json).
  • sonic.go (alternative implementation using sonic library).

Import Updates:

All imports to the standard encoding/json package have been updated to the new custom package github.com/thrasher-corp/gocryptotrader/encoding/json.

Added basic benchmarks for both the default encoding/json and the sonic implementation to evaluate hot-swap performance.

Type of change

Please delete options that are not relevant and add an x in [] as item is complete.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How has this been tested

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration and
also consider improving test coverage whilst working on a certain feature or package.

  • go test ./... -race
  • golangci-lint run
  • Test X

Checklist

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation and regenerated documentation via the documentation tool
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally and on Github Actions with my changes
  • Any dependent changes have been merged and published in downstream modules

@shazbert shazbert self-assigned this Aug 20, 2024
Copy link

codecov bot commented Aug 20, 2024

Codecov Report

Attention: Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Project coverage is 37.07%. Comparing base (72a2a16) to head (135e2b6).

Files with missing lines Patch % Lines
cmd/exchange_wrapper_issues/main.go 0.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #1623   +/-   ##
=======================================
  Coverage   37.07%   37.07%           
=======================================
  Files         414      414           
  Lines      180274   180274           
=======================================
+ Hits        66835    66845   +10     
- Misses     105574   105577    +3     
+ Partials     7865     7852   -13     
Files with missing lines Coverage Δ
backtester/btcli/main.go 0.00% <ø> (ø)
backtester/config/backtesterconfig.go 88.46% <ø> (ø)
backtester/config/strategyconfig.go 83.51% <ø> (ø)
backtester/config/strategyconfigbuilder/main.go 0.00% <ø> (ø)
backtester/eventhandlers/statistics/statistics.go 74.63% <ø> (ø)
backtester/main.go 0.00% <ø> (ø)
cmd/apichecker/apicheck.go 38.20% <ø> (ø)
cmd/config/config.go 8.62% <ø> (ø)
cmd/config_builder/builder.go 0.00% <ø> (ø)
cmd/documentation/documentation.go 0.00% <ø> (ø)
... and 89 more

... and 11 files with indirect coverage changes

@shazbert shazbert added review me This pull request is ready for review low priority This enhancement or update will be implemented at a later date. and removed blocked labels Aug 20, 2024
@shazbert shazbert added medium priority and removed low priority This enhancement or update will be implemented at a later date. labels Oct 10, 2024
@shazbert
Copy link
Collaborator Author

Flagged this as medium because this is beast. Sorry for those that cant use it. 🙏

Copy link
Collaborator

@gloriousCode gloriousCode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work on finding this. I like the way you've implemented it and makes it easy to experiment with out parsers in future with your groundwork

orig:

// BenchmarkUnmarshal-10    	  949106	      1065 ns/op	     816 B/op	      24 allocs/op

sonic:

// BenchmarkUnmarshal-10    	 1818868	       643.8 ns/op	     951 B/op	      19 allocs/op

encoding/json/sonic_benchmark_test.go Outdated Show resolved Hide resolved
- os: windows-latest
goarch: amd64
psql: true
skip_wrapper_tests: true
sonic: false
- os: ubuntu-latest-sonic
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should stick to the same style as the existing builds names with the false, false,false,true, would also just be fine with including sonic somewhere in the build name. Maybe its just not showing up below because its not merged?

Also, do we need two of each build? Would one extra one with sonic enabled suffice?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should stick to the same style as the existing builds names with the false, false,false,true, would also just be fine with including sonic somewhere in the build name. Maybe its just not showing up below because its not merged?

That should have been reverted. I think.

Also, do we need two of each build? Would one extra one with sonic enabled suffice?

@thrasher- Confirming entire suite test or can just have one?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can target amd64 linux for now since that's our main deployment OS in use ATM

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

encoding/json/sonic.go Outdated Show resolved Hide resolved
encoding/json/sonic_benchmark_test.go Outdated Show resolved Hide resolved
@@ -13,22 +13,52 @@ jobs:
goarch: amd64
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated to this line, would you consider a log on GCT start to say which one is enabled? Yes, you have build tags, but then it puts it in the logs of the application

Could add to the build tag files to call like json.Version() where its fmt.Println("hello my name is Sonic and I gotta go fast, okay?") as an example

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shazbert shazbert requested a review from gloriousCode January 10, 2025 02:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
medium priority review me This pull request is ready for review
Projects
Status: In review
Development

Successfully merging this pull request may close these issues.

3 participants