Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(internal/json): add arrow_json_stdlib build tag #199

Merged
merged 1 commit into from
Dec 4, 2024

Conversation

toddtreece
Copy link
Contributor

@toddtreece toddtreece commented Nov 22, 2024

Rationale for this change

Grafana and Grafana plugins both use arrow-go, but do not use arrow's internal/json. A decent amount of used heap is initialized by https://github.com/goccy/go-json, so it would be useful to be able to prevent the initialization.

The example below shows that 60% of this heap profile in a Grafana plugin is used by go-json's encoder & decoder packages.

image

I have submitted a PR to go-json to switch to lazy initialization (additional details in the PR), but I am having a hard time getting a response from the maintainer, so am attempting a different approach to the problem with this PR.

What changes are included in this PR?

Adds arrow_json_stdlib build tag so that it's possible to switch to encoding/json and avoid the overhead of https://github.com/goccy/go-json.

Are these changes tested?

I tested this locally, but I can add coverage if this seems like an acceptable approach.

Are there any user-facing changes?

@toddtreece toddtreece force-pushed the toddtreece/json-stdlib branch from 8de74c2 to c812146 Compare November 22, 2024 21:22
@zeroshade
Copy link
Member

We mostly use goccy/go-json for performance reasons, but this seems to be a perfectly reasonable way to handle the situation. I definitely prefer fixing it upstream in go-json via your PR if possible. But this is a good stop-gap.

@toddtreece
Copy link
Contributor Author

@zeroshade sounds good, thanks! i will continue trying to get the go-json PR merged, and if i do, i'll open another PR to update the dependency here.

@toddtreece toddtreece force-pushed the toddtreece/json-stdlib branch 2 times, most recently from 608feff to 0171ae5 Compare November 25, 2024 14:41
@toddtreece toddtreece force-pushed the toddtreece/json-stdlib branch from 0171ae5 to 4291143 Compare November 25, 2024 14:46
@toddtreece
Copy link
Contributor Author

@zeroshade it looks like maybe you pushed a fix for the failing mac OS checks, so i rebased this against main

@zeroshade zeroshade merged commit 74ebd3a into apache:main Dec 4, 2024
24 checks passed
zeroshade pushed a commit that referenced this pull request Dec 12, 2024
### Rationale for this change

A follow up to
#199 (comment)

> i will continue trying to get the go-json PR merged, and if i do, i'll
open another PR to update the dependency here.

### What changes are included in this PR?

Bumps go-json dependency to include changes from
goccy/go-json#490

### Are these changes tested?

I am assuming CI tests should cover this.

### Are there any user-facing changes?

No
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants