[Console] Add support for JSON with long numerals

[AMoo-Miki]

Add support for JSON with long numerals in Dev Tools.

Also:

• Add support for parsing and stringifying JSON with long numerals into @osd/std
• Upgrade @opensearch/[email protected] which supports long numerals
• Add support for long numerals to http/fetch

Description

• When long numerals were used in the editor, they lost precision when
  • requests were run
  • text was auto-indented
• When a result contained long numerals, it lost precision when displayed in console.

Issues Resolved

Fixes #4536

Screenshot

Before

before-long-numerals-fix.mp4

After

after-long-numerals-fix.mp4

Check List

• All tests pass
  • yarn test:jest
  • yarn test:jest_integration
  • yarn test:ftr
• New functionality includes testing.
• New functionality has been documented.
• Update CHANGELOG.md
• Commits are signed per the DCO using --signoff

[codecov]

Codecov Report

Merging #4562 (40b4783) into main (9d7f576) will increase coverage by 0.04%.
Report is 1 commits behind head on main.
The diff coverage is 92.03%.

@@            Coverage Diff             @@
##             main    #4562      +/-   ##
==========================================
+ Coverage   66.39%   66.43%   +0.04%     
==========================================
  Files        3397     3398       +1     
  Lines       64804    64904     +100     
  Branches    10360    10385      +25     
==========================================
+ Hits        43026    43120      +94     
- Misses      19218    19226       +8     
+ Partials     2560     2558       -2     

Flag	Coverage Δ
Linux_1	34.80% <15.46%> (-0.05%)	⬇️
Linux_2	55.30% <92.00%> (+0.15%)	⬆️
Linux_3	44.46% <66.37%> (+0.23%)	⬆️
Linux_4	34.89% <26.47%> (-0.03%)	⬇️
Windows_1	34.82% <15.46%> (-0.05%)	⬇️
Windows_2	?
Windows_3	44.46% <66.37%> (+0.22%)	⬆️
Windows_4	34.89% <26.47%> (-0.03%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
src/core/server/opensearch/client/mocks.ts	98.21% <ø> (ø)
...lugins/console/public/lib/opensearch/opensearch.ts	75.00% <ø> (ø)
src/plugins/console/public/services/storage.ts	0.00% <0.00%> (ø)
.../server/routes/api/console/proxy/create_handler.ts	67.39% <40.00%> (-2.38%)	⬇️
packages/osd-std/src/json.ts	95.74% <95.74%> (ø)
packages/osd-std/src/index.ts	100.00% <100.00%> (ø)
src/core/public/http/fetch.ts	96.05% <100.00%> (ø)
...rc/core/server/opensearch/client/cluster_client.ts	85.71% <100.00%> (+1.50%)	⬆️
...equest_to_opensearch/send_request_to_opensearch.ts	70.49% <100.00%> (ø)
src/plugins/console/public/lib/utils/index.ts	86.04% <100.00%> (+27.90%)	⬆️
... and 1 more

... and 8 files with indirect coverage changes

[ashwin-pc]

Nice addition, can you add documentation for the change in the package @AMoo-Miki ?

[zhongnansu]

re-running failed CIs

[zhongnansu]

@AMoo-Miki Hi Miki, Would you check the CI failures? Is it because we haven't release the js client with the fix?

[AMoo-Miki]

Thanks. This needs to be bumped to use the 2.3.1 client.

[zhongnansu]

Thanks. This needs to be bumped to use the 2.3.1 client.

js client 2.3.1 is out. Would you update the PR? Thx! @AMoo-Miki

[AMoo-Miki]

I am stuck with making a decision for how this should be implemented as we don't want to add it all over OSD. Will continue thinking.

[ashwin-pc]

@AMoo-Miki should i remove the 2.10 tag for this then?

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch-Dashboards/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch-Dashboards/backport-2.x
# Create a new branch
git switch --create backport/backport-4562-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 0709333c641f1e2bcaf116eb7d5504b33afcd71d
# Push it to GitHub
git push --set-upstream origin backport/backport-4562-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch-Dashboards/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-4562-to-2.x.

[ashwin-pc]

Just one concern about how we are exposing the custom client. The rest are just minor nits or opinions, feel free to use them if you think it makes sense :)

[ashwin-pc]

While this is verbose, cant we make it more generic? This is the first customization that we are introducing but it doubt it will be the last. And since this client will become a part of the API it will be annoying to change this in the future.

interface ExtendedClient extends OpenSearchClient {
    _custom : {
        [key: string]: OpenSearchClient,
    }
}

Suggested change

	public readonly asCurrentUser: OpenSearchClient,
	public readonly asInternalUserWithLongNumeralsSupport: OpenSearchClient,
	public readonly asCurrentUserWithLongNumeralsSupport: OpenSearchClient
	public readonly asCurrentUser: ExtendedClient,

And add the custom long numeral client like:

internalClient.customClient = {
    longNumeral: client,
}

[ashwin-pc]

nit: Cant we make this a lot simpler by just converting anything greater than a generic high number? e.g. 9000000000000000. What you are doing should work, but this reduces the matchers from 12 to 2

[AMoo-Miki]

I explained why not in the comments in the code. Before parsing the long string, I wouldn't know what's in it. Assuming 8192 is max int, if i take everything greater than 8000 and turn them into BigInt, that would be unexpected to everyone who don't use numbers bigger than 8192. Hence, doing this one time activity of making a marginally more complex regex.

[ashwin-pc]

I didnt understand this comment

[AMoo-Miki]

The code could be

loop {
  condition ? doA : doB
}

but then condition is evaluated repeatedly. I decided to do

condition ? loop { doA } : loop { doB }

this results in a little less maintainable code.

[ashwin-pc]

nit: cant we use UUID here? That way instead of a marker system, we can keep a dictionary of uuid's and values that can be substituted when parsing fails.

[AMoo-Miki]

What if a value uses UUIDs? also, the longer the UUID, the more expensive my regex will be. Some more details here.

[kavilla]

@AMoo-Miki I have reverted this on 2.10 and tagged with 2.11. The reason being I'm not positive if the JS client cherry pick commit opensearch-project/opensearch-js#550 is intended to make the client 2.x version to be non-breaking for major version bump. So that we can utilize the declaration of opensearch instead of using opensearch-next

[kavilla]

@AMoo-Miki will address the backport