View call RPC is returning byte arrays instead of base64

[ilblackdragon]

Steps to reproduce:

1. Run any view call, for example query call/evm.test.near/view_call_contract
2. Observe that output has byte array in result.

{"jsonrpc":"2.0","result":{"result":[34,82,101,118,101,114,116,101,100,32,48,56,99,51,55,57,97,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,50,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,49,50,52,53,53,50,53,50,53,102,52,101,52,102,53,52,53,102,52,51,52,102,52,101,53,52,53,50,52,102,52,99,52,99,52,53,53,50,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,48,34],"logs":[],"block_height":31287,"block_hash":"GvE1UnAoiTKWZAcD6tNCJGvnmKnTmkz8RnwVJ2Uk1jhu"},"id":280}

Expectation: result is base64 encoded.

[MaksymZavershynskyi]

Assigning to @khorolets , CC-ing @frol

[frol]

We should not break the existing API, so I see two ways to handle this:

1. Respond with two fields (result and result_base64)
2. Extend the query API (we have the extended version of query API where we can now easily add new parameters), so user can request the encoding (bytes or base64 string), this way the deprecated API (query call/...) will not be touched in any way.

UPD: I am re-assigning to myself to cast the decision first

[alexauroradev]

Check with Illia whether this is connected with old EVM implementation @artob

[frol]

@djsatok How exactly is this related to EVM? This is JSON RPC representation, and we cannot really change it in a backward-compatible way, so the only option is to return both old bytes array and a new base64 representation. We will also need to update the logs field for Events support to be also old (utf-8 strings) and new (base64-encoded strings allowing binary logs) style

[artob]

@frol OK, understood, and assigning this back to you.

[ilblackdragon]

It's a general issue for sure. The EVM is affected by it because it uses binary encoding to return values and the responses ending up huge due to pure JSON encoding.

…

On Mon, Dec 7, 2020 at 6:08 AM Arto Bendiken ***@***.***> wrote: @frol <https://github.com/frol> OK, understood, and assigning this back to you. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3179 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABK27QBJNU3KQUIW45R2CTSTTOWDANCNFSM4QAYDIPA> .

-- Best regards, Illia Polosukhin

[frol]

@ilblackdragon can you elaborate more on the EVM use case? Do we need a v2 endpoint with base64 encoding instead? Should it be completely different? Should it be something that is not JSON in the first place?

[ilblackdragon]

I mean it's not specific to EVM, just non consistent and takes more bytes over the network. Hm, the response from view calls is not JSON - it's arbitrary bytes. EVM sends a byte array encoded with their RLP encoding. But so does the bridge contracts for example, they are returning Borsh encoded information for proofs.

…

On Tue, Dec 8, 2020 at 12:00 PM Vlad Frolov ***@***.***> wrote: @ilblackdragon <https://github.com/ilblackdragon> can you elaborate more on the EVM use case? Do we need a v2 endpoint with base64 encoding instead? Should it be completely different? Should it be something that is not JSON in the first place? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3179 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABK27S6NJIL6HBZLAHO5GDST2AXDANCNFSM4QAYDIPA> .

-- Best regards, Illia Polosukhin

[frol]

@ilblackdragon I meant that if the RPC response in JSON bytes encoding is too big, we may as well consider having the RPC responses in non-JSON format in the first place (instead of using base64 inside JSON RPC response).

[ilblackdragon]

Well it's still needs to be encoded to something, so base64 is a reasonable encoding. I don't really understand what it would be instead of JSON RPC response here in your suggestion. Generally responses can be pretty big, like when fetching all accounts staked to given pool. This is where indexing and alternative to RPC querying comes in.

…

On Wed, Dec 9, 2020, 6:09 PM Vlad Frolov ***@***.***> wrote: @ilblackdragon <https://github.com/ilblackdragon> I meant that if the RPC response in JSON bytes encoding is too big, we may as well consider having the RPC responses in non-JSON format in the first place (instead of using base64 inside JSON RPC response). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3179 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABK27WB26QAKR6OT2F424LST5EF3ANCNFSM4QAYDIPA> .

[stale]

This issue has been automatically marked as stale because it has not had recent activity in the last 2 months.
It will be closed in 7 days if no further activity occurs.
Thank you for your contributions.

[bowenwang1996]

@frol what is the status of this issue?

[stale]

This issue has been automatically marked as stale because it has not had recent activity in the last 2 months.
It will be closed in 7 days if no further activity occurs.
Thank you for your contributions.

[bowenwang1996]

@frol ?

[frol]

We cannot handle it in a backward-compatible way and it is not important implementation detail if you deal with it through some near-api-* library