Incorrect Hex Encoding for Quantities

[fabioberger]

Hi there,

As per the Ethereum wiki JSON-rpc section on hex encoding correct encoding of quantities (numbers e.g nonces, etc...) should be the most compact hex representation:

• 0x41 (65 in decimal)
• 0x400 (1024 in decimal)
• WRONG: 0x (should always have at least one digit - zero is "0x0")
• WRONG: 0x0400 (no leading zeroes allowed)
• WRONG: ff (must be prefixed 0x)

The ethereumjs-tx library quantities are padded to an even number of digits, i.e. 0x01, instead of 0x1 as expected by the specification. This has already caused issues for dependent projects such as testrpc (trufflesuite/ganache-cli-archive#220) and introduces an incompatibility with the Geth client and the parity client (their wiki entry on encoding).

Happy to help implement this fix, but since it's a quite pervasive issue, thought I'd get your input on how to approach it first.

[fanatid]

0x01 was made instead of 0x1 for node Buffer
Buffer.from('1', 'hex') thorws TypeError: Invalid hex string

Patch which will fix this issue need major version bump.

[wanderer]

This is b/c ethereum TX was written to parse raw TX from the network directly. Nodes buffer API expects hex number to also be an even number. This lib should probably be wrapped by something higher level for the RPC. But I'm open to suggestions!

…

On 26 Feb 2017 12:53 p.m., "Fabio Berger" ***@***.***> wrote: Hi there, As per the Ethereum wiki JSON-rpc section on hex encoding <https://github.com/ethereum/wiki/wiki/JSON-RPC#hex-value-encoding> correct encoding of quantities (numbers e.g nonces, etc...) should be the most compact hex representation: - 0x41 (65 in decimal) - 0x400 (1024 in decimal) - WRONG: 0x (should always have at least one digit - zero is "0x0") - WRONG: 0x0400 (no leading zeroes allowed) - WRONG: ff (must be prefixed 0x) The ethereumjs-tx library quantities are padded to an even number of digits, i.e. 0x01, instead of 0x1 as expected by the specification. This has already caused issues for dependent projects such as testrpc <https://github.com/ethereumjs/testrpc> (trufflesuite/ganache-cli-archive#220 <trufflesuite/ganache-cli-archive#220>) and introduces an incompatibility with the Geth client and the parity client (their wiki entry on encoding <https://github.com/ethcore/parity/wiki/JSONRPC>). Happy to help implement this fix, but since it's a quite pervasive issue, thought I'd get your input on how to approach it first. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#51>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAJqA8jkeXLu3qSbQcTuXKxmwEUxGUjoks5rgWergaJpZM4MMW67> .

[fabioberger]

After further investigation, it seems this issue is related to the Buffer classes conversion of bytes to hex. Since a byte is 8 bits, values below 16 will result in the first 4 bits being represented by a zero.

Since the Ethereum spec expects a more compact representation, one solution would be to wrap the Buffer implementation and have the caller specify whether the value being stored in the buffer is a quantity or unformatted data. The wrapper will then hex encode the value using the spec rules accordingly. e.g:

module.exports = class EthBuffer extends Buffer {
    constructor(value, encoding = 'utf8', type = 'unformatted') {
        if (arguments.length === 2 && typeof value === 'string') {
                super(value, encoding)
        } else {
            super(value)
        }

        if (arguments.length === 2 && typeof value !== 'string') {
            type = encoding
        }
        this._type = type
    }
    toString() {
        const args = Array.prototype.slice.call(arguments)
        const val = super.toString.apply(this, args)
        if (args[0] === 'hex' && this._type === 'quantity' && val[0] === '0') {
            return val.substring(1)
        } else {
            return val
        }
    }
}

And then replacing instantiations of Buffer with EthBuffer. The issue I ran into is that instantiating an EthBuffer returns a Buffer instance for some reason... This might be a bug in Node :(

[fabioberger]

For posterity, Buffer does not yet support inheritance. The debate on how to refactor the API is ongoing: nodejs/node#9531

[seishun]

@fabioberger You're welcome to chime in in that thread. It would certainly be helpful to have an example of a real-world use case for extending Buffer.

[AlTremor]

Hello,
I am trying to implement a voting system in nodejs. My server needs to be able to create transactions and send them to the ethereum blockchain when provided with the user private key. I have tried doing so using different modules however I seem to always run into the same issue.

• The connection with the RPC works fine
• The transaction seems to be created just right
• Transaction is registered as sent to the network by my local node but will never appear in the blokchain.

I have managed to have transactions registered while using web3 but I can't do it the same way if I want to be able to sign transactions myself. Tests were done using both a local or remote ( infura ) node on the testnet ( ropsten and kovan ). I may have a hint of the problem :

• When creating a transaction (succesful) with web3 my local node sees the transaction hash as '0x456515...' and the address as '0x28452215...'
• When doing the same with ethereumjs-tx to sign the transaction, the node sees it as '456515...' and '&28452215..'

Could it be that this issue is related to this padding problem ? I have yet to manage to send a transaction that I have signed myself in nodejs.
Thank You

[MicahZoltu]

@AlTremor I don't believe this issue is the one you are running into. It looks like you are simply missing the 0x prefix on your hex values, and the second one has an extraneous & in it. I'm not particularly familiar with ethereumjs-tx so I can't help troubleshoot much, but you may want to try either the gitter.im channel or opening a new issue.

[kumavis]

ethereumjs-tx only exposes buffers, not strings. coercing buffers to hex strings is out of scope.
(happy to reopen if i missed something)