Fix wallet load error on incomplete gettransaction (Failed to load utxos, IndexError: list index out of range) #2467

leon-costa · 2024-08-01T08:44:44Z

The error happens when the wallet has some locked outputs in listlockunspent and the corresponding gettransaction doesn't contain the details of this particular output.

It may be related to bitcoin/bitcoin#28555. It is at least the same symptom.

The locked outputs were locked by creating a transaction in Specter. This has happened to me multiple times.

The UI says it failed to load the wallet with the following error: Failed to load utxos, IndexError: list index out of range

The raw transaction does contain the missing output so we use that instead.

Here's an example that generates the error:

listlockunspent:
[
  {
    "txid": "<some txid>",
    "vout": 0
  }
]

gettransaction <some txid>:
{
  "amount": <redacted>,
  "fee": <redacted>,
  "confirmations": <redacted>,
  "blockhash": "<redacted>",
  "blockheight": <redacted>,
  "blockindex": <redacted>,
  "blocktime": <redacted>,
  "txid": "<some tx id>",
  "wtxid": "<redacted>",
  "walletconflicts": [
  ],
  "time": <redacted>,
  "timereceived": <redacted>,
  "bip125-replaceable": "no",
  "details": [
    {
      "address": "<redacted>",
      "category": "send",
      "amount": <redacted>,
      "label": "<redacted>",
      "vout": 1,
      "fee": <redacted>,
      "abandoned": false
    }
  ],
  "hex": "<some raw tx hex>",
  "lastprocessedblock": {
    "hash": "<redacted>",
    "height": <redacted>
  }
}

decoderawtransaction <some raw tx hex>:
{
  "txid": "<some txid>",
  "hash": "<redacted>",
  "version": 2,
  "size": <redacted>,
  "vsize": <redacted>,
  "weight": <redacted>,
  "locktime": <redacted>,
  "vin": [
    {
      "txid": "<some other txid>",
      "vout": 0,
      "scriptSig": {
        "asm": "",
        "hex": ""
      },
      "txinwitness": [
        "<redacted>",
        "<redacted>"
      ],
      "sequence": <redacted>
    }
  ],
  "vout": [
    {
      "value": <redacted>,
      "n": 0,
      "scriptPubKey": {
        "asm": "<redacted>",
        "desc": "<redacted>",
        "hex": "<redacted>",
        "address": "<redacted>",
        "type": "<redacted>"
      }
    },
    {
      "value": <redacted>,
      "n": 1,
      "scriptPubKey": {
        "asm": "<redacted>",
        "desc": "<redacted>",
        "hex": "<redacted>",
        "address": "<redacted>",
        "type": "<redacted>"
      }
    }
  ]
}

Note: I'm connecting to a Bitcoin Core version v27.1.0.

The error happens when the wallet has some locked outputs in `listlockunspent` and the corresponding `gettransaction` doesn't contain the details of this particular output. It may be related to bitcoin/bitcoin#28555. It is at least the same symptom. The locked outputs were locked by creating a transaction in Specter. This has happened to me multiple times. The UI says it failed to load the wallet with the following error: Failed to load utxos, IndexError: list index out of range The raw transaction does contain the missing output so we use that instead. Here's an example that generates the error: listlockunspent: [ { "txid": "<some txid>", "vout": 0 } ] gettransaction <some txid>: { "amount": <redacted>, "fee": <redacted>, "confirmations": <redacted>, "blockhash": "<redacted>", "blockheight": <redacted>, "blockindex": <redacted>, "blocktime": <redacted>, "txid": "<some tx id>", "wtxid": "<redacted>", "walletconflicts": [ ], "time": <redacted>, "timereceived": <redacted>, "bip125-replaceable": "no", "details": [ { "address": "<redacted>", "category": "send", "amount": <redacted>, "label": "<redacted>", "vout": 1, "fee": <redacted>, "abandoned": false } ], "hex": "<some raw tx hex>", "lastprocessedblock": { "hash": "<redacted>", "height": <redacted> } } decoderawtransaction <some raw tx hex>: { "txid": "<some txid>", "hash": "<redacted>", "version": 2, "size": <redacted>, "vsize": <redacted>, "weight": <redacted>, "locktime": <redacted>, "vin": [ { "txid": "<some other txid>", "vout": 0, "scriptSig": { "asm": "", "hex": "" }, "txinwitness": [ "<redacted>", "<redacted>" ], "sequence": <redacted> } ], "vout": [ { "value": <redacted>, "n": 0, "scriptPubKey": { "asm": "<redacted>", "desc": "<redacted>", "hex": "<redacted>", "address": "<redacted>", "type": "<redacted>" } }, { "value": <redacted>, "n": 1, "scriptPubKey": { "asm": "<redacted>", "desc": "<redacted>", "hex": "<redacted>", "address": "<redacted>", "type": "<redacted>" } } ] }

netlify · 2024-08-01T08:45:05Z

✅ Deploy Preview for specter-desktop-docs canceled.

Name	Link
🔨 Latest commit	`2efd56c`
🔍 Latest deploy log	https://app.netlify.com/sites/specter-desktop-docs/deploys/66c76cf240f1f3000860cd4c

moneymanolis · 2024-08-20T16:28:43Z

First, thanks for your time and effort looking into this and making a PR.

I just tested this real quick because I had the same error while testing (again) and one "issue" to mention is that decoderawtransaction is not implemented in Spectrum. I haven't looked at the rest of the code yet to be fair, but if it is needed, ideally, this would be added as well to Spectrum. From a quick look, it might be similar to how getrawtransaction was implemented,, i.e. using an ElectrumX API call.

Traceback (most recent call last):
  File "/Users/manolis/coding/repos/specter-desktop/src/cryptoadvance/specter/wallet/wallet.py", line 735, in check_utxo
    raw_transaction = self.rpc.decoderawtransaction(
  File "/Users/manolis/coding/repos/specter-desktop/src/cryptoadvance/specter/rpc.py", line 507, in fn
    r = self.multi([(method, *args)], **kwargs)[0]
  File "/Users/manolis/coding/repos/specter-desktop/src/cryptoadvance/specterext/spectrum/bridge_rpc.py", line 115, in multi
    raise SpecterRpcError(f"Request error: {rpce.message}", mock_response)
cryptoadvance.specter.rpc.RpcError: Request error: Method not found (decoderawtransaction)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/manolis/coding/repos/specter-desktop/src/cryptoadvance/specter/managers/wallet_manager.py", line 226, in _update
    loaded_wallet = self.WalletClass.from_json(
  File "/Users/manolis/coding/repos/specter-desktop/src/cryptoadvance/specter/wallet/wallet.py", line 589, in from_json
    return cls(
  File "/Users/manolis/coding/repos/specter-desktop/src/cryptoadvance/specter/wallet/wallet.py", line 175, in __init__
    self.update()
  File "/Users/manolis/coding/repos/specter-desktop/src/cryptoadvance/specter/wallet/wallet.py", line 497, in update
    self.getdata()
  File "/Users/manolis/coding/repos/specter-desktop/src/cryptoadvance/specter/wallet/wallet.py", line 763, in getdata
    self.check_utxo()
  File "/Users/manolis/coding/repos/specter-desktop/src/cryptoadvance/specter/wallet/wallet.py", line 759, in check_utxo
    raise SpecterError(f"Failed to load utxos, {type(e).__name__}: {e}")
cryptoadvance.specter.specter_error.SpecterError: Failed to load utxos, RpcError: Request error: Method not found (decoderawtransaction)

Just for reference this is a sample PR to add methods to Spectrum, if there is no ElectrumX API call that can chucked in.

leon-costa · 2024-08-22T09:12:54Z

ElectrumX doesn't seem to have such an API. blockchain.transaction.get with verbose=true seems pretty close, but it takes a transaction hash as an input, not the transaction hex. It could work here because that's what we start with anyway, but I don't know how that could be done with all the intermediaries in your setup.

I see in the spectrum PR you mentioned that embit is used, which seems to be able to decode raw transactions. So it should not be difficult to implement a minimal version of decoderawtransaction that only returns the fields used by this PR, i.e. the vout array, and inside it only n, value and scriptPubKey / address. The first example in the embit page I linked seems to provide an example of all the necessary code.

Actually, specter-desktop also uses embit, so maybe this should be done directly here instead of calling decoderawtransaction.

moneymanolis · 2024-08-22T13:28:47Z

Actually, specter-desktop also uses embit, so maybe this should be done directly here instead of calling decoderawtransaction.

Might be the easiest solution? :-)

Because the decoderawtransaction RPC is not available in Spectrum and the tools to decode the raw transaction are already there.

leon-costa · 2024-08-22T16:54:23Z

I made the change to use embit in this fix.

oren-z0 · 2024-11-05T21:24:09Z

@leon-costa Could it be related to #2475 ?
After sending funds from a wallet to itself, Instead of "list index out of range" I'm getting "Failed to load utxos, KeyError: '51845...'".

oren-z0 · 2024-11-06T12:19:53Z

@leon-costa Please see #2476 . I wonder if it fixes your problem too.

Apply black formatting

ec5e1b2

Use embit to decode the raw transaction

2efd56c

Because the decoderawtransaction RPC is not available in Spectrum and the tools to decode the raw transaction are already there.

leon-costa force-pushed the fix-list-index-out-of-range branch from 301fc08 to 2efd56c Compare August 22, 2024 16:53

leon-costa mentioned this pull request Sep 6, 2024

Failed to load utxos, IndexError: list index out of range #2468

Open

leon-costa mentioned this pull request Oct 12, 2024

Can't spend funds #2459

Open

oren-z0 mentioned this pull request Nov 5, 2024

Wallet becomes unusable after trying to send funds from the wallet to itself. #2475

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix wallet load error on incomplete gettransaction (Failed to load utxos, IndexError: list index out of range) #2467

Fix wallet load error on incomplete gettransaction (Failed to load utxos, IndexError: list index out of range) #2467

leon-costa commented Aug 1, 2024

netlify bot commented Aug 1, 2024 •

edited

Loading

moneymanolis commented Aug 20, 2024 •

edited

Loading

leon-costa commented Aug 22, 2024

moneymanolis commented Aug 22, 2024

leon-costa commented Aug 22, 2024

oren-z0 commented Nov 5, 2024 •

edited

Loading

oren-z0 commented Nov 6, 2024

Fix wallet load error on incomplete gettransaction (Failed to load utxos, IndexError: list index out of range) #2467

Are you sure you want to change the base?

Fix wallet load error on incomplete gettransaction (Failed to load utxos, IndexError: list index out of range) #2467

Conversation

leon-costa commented Aug 1, 2024

netlify bot commented Aug 1, 2024 • edited Loading

✅ Deploy Preview for specter-desktop-docs canceled.

moneymanolis commented Aug 20, 2024 • edited Loading

leon-costa commented Aug 22, 2024

moneymanolis commented Aug 22, 2024

leon-costa commented Aug 22, 2024

oren-z0 commented Nov 5, 2024 • edited Loading

oren-z0 commented Nov 6, 2024

netlify bot commented Aug 1, 2024 •

edited

Loading

moneymanolis commented Aug 20, 2024 •

edited

Loading

oren-z0 commented Nov 5, 2024 •

edited

Loading