SerializationError('Asked to read 0x%x bytes; MAX_SIZE exceeded' % n)

[herrmoltke]

I am running v9.55.4 of counterparty-lib, v1.1.3 of `counterparty server and v0.13.2 of BTCDrak's addrindex patched Bitcoin Core.

I was building the Counterparty database from scratch, and ran into the following error:

[2018-03-08 17:42:22][INFO] Resuming parsing.
[2018-03-08 17:42:22][ERROR] Unhandled Exception
Traceback (most recent call last):
  File "/usr/local/bin/counterparty-server", line 9, in <module>
    load_entry_point('counterparty-cli==1.1.3', 'console_scripts', 'counterparty-server')()
  File "/home//counterparty-cli-master/counterpartycli/__init__.py", line 16, in server_main
    server.main()
  File "/home//counterparty-cli-master/counterpartycli/server.py", line 157, in main
    server.start_all(db)
  File "/home//counterparty-lib/counterpartylib/server.py", line 408, in start_all
    blocks.follow(db)
  File "/home//counterparty-lib/counterpartylib/lib/blocks.py", line 1150, in follow
    current_cblock = backend.getblock(current_hash)
  File "/home//counterparty-lib/counterpartylib/lib/backend/__init__.py", line 50, in getblock
    return CBlock.deserialize(util.unhexlify(block_hex))
  File "/usr/local/lib/python3.5/dist-packages/bitcoin/core/serialize.py", line 115, in deserialize
    r = cls.stream_deserialize(fd)
  File "/usr/local/lib/python3.5/dist-packages/bitcoin/core/__init__.py", line 506, in stream_deserialize
    vtx = VectorSerializer.stream_deserialize(CTransaction, f)
  File "/usr/local/lib/python3.5/dist-packages/bitcoin/core/serialize.py", line 250, in stream_deserialize
    r.append(inner_cls.stream_deserialize(f))
  File "/usr/local/lib/python3.5/dist-packages/bitcoin/core/__init__.py", line 329, in stream_deserialize
    vin = VectorSerializer.stream_deserialize(CTxIn, f)
  File "/usr/local/lib/python3.5/dist-packages/bitcoin/core/serialize.py", line 250, in stream_deserialize
    r.append(inner_cls.stream_deserialize(f))
  File "/usr/local/lib/python3.5/dist-packages/bitcoin/core/__init__.py", line 199, in stream_deserialize
    scriptSig = script.CScript(BytesSerializer.stream_deserialize(f))
  File "/usr/local/lib/python3.5/dist-packages/bitcoin/core/serialize.py", line 234, in stream_deserialize
    return ser_read(f, l)
  File "/usr/local/lib/python3.5/dist-packages/bitcoin/core/serialize.py", line 78, in ser_read
    raise SerializationError('Asked to read 0x%x bytes; MAX_SIZE exceeded' % n)
bitcoin.core.serialize.SerializationError: Asked to read 0x7816cb1234a565ff bytes; MAX_SIZE exceeded

I added a print(block_hash) statement to getblock() and ran counterparty-server start several more times, and found that block 0000000000000000001c8018d9cb3b742ef25114f27563e3fc4a1902167f9893 is always the offending block.

I also found that the same error is thrown by api_status_poller.run() (which server.start_all() calls before, blocks.follow(), from which the above error is derived). The difference here is that the error is always returned for the most recently mined block.

I then ran counterparty-server bootstrap and ran into the same problem, but this time the offending block was 0000000000000000003214b15321a9198a3ff583ea140d5dcc6ff01e41b6c4cc.

[jdogresorg]

Possibly related to https://github.com/CounterpartyXCP/counterparty-lib/issues/950

[herrmoltke]

Could this have something to do with SegWit? According to this article, SegWit was activated on the very block my instance of Counterparty chokes on (i.e. 481,824). I know Counterparty doesn't yet support SegWit, is using an addrindex-patched Bitcoin Core without SegWit[EDIT: v0.13.0 of Bitcoin Core enables SegWit], and uses a release of python-bitcoinlib without SegWit support.

[herrmoltke]

FYI I just bumped my python-bitcoinlib to v0.9.0 and counterparty-server has successfully resumed parsing. I don't know however whether there are any secondary, negative consequences of doing this, so I don't want to make a PR with this change. You can feel free to close this, though.

[herrmoltke]

Closing this. The real problem is Counterblock. See here: CounterpartyXCP/counterblock#164