Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ipfs init with weird datastore fails...but still works? #9155

Open
3 tasks done
Ericson2314 opened this issue Jul 29, 2022 · 3 comments
Open
3 tasks done

ipfs init with weird datastore fails...but still works? #9155

Ericson2314 opened this issue Jul 29, 2022 · 3 comments
Labels
kind/bug A bug in existing code (including security flaws) P4 Very low priority

Comments

@Ericson2314
Copy link

Checklist

Installation method

built from source

Version

ipfs version 0.12.2

Config

{
  "API": {
    "HTTPHeaders": {}
  },
  "Addresses": {
    "API": "/ip4/127.0.0.1/tcp/5002",
    "Announce": [],
    "AppendAnnounce": [],
    "Gateway": "/ip4/127.0.0.1/tcp/8081",
    "NoAnnounce": [],
    "Swarm": [
      "/ip4/0.0.0.0/tcp/4002",
      "/ip6/::/tcp/4002",
      "/ip4/0.0.0.0/udp/4002/quic",
      "/ip6/::/udp/4002/quic"
    ]
  },
  "AutoNAT": {},
  "Bootstrap": [
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt",
    "/ip4/104.131.131.82/tcp/4002/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
    "/ip4/104.131.131.82/udp/4002/quic/p2p/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb"
  ],
  "DNS": {
    "Resolvers": {}
  },
  "Datastore": {
    "BloomFilterSize": 0,
    "GCPeriod": "1h",
    "HashOnRead": false,
    "Spec": {
      "mounts": [
        {
          "child": {
            "auth-token": "<CENSORED>",
            "type": "swhbridge"
          },
          "mountpoint": "/blocks",
          "prefix": "swhbridge.datastore",
          "type": "measure"
        },
        {
          "child": {
            "compression": "none",
            "path": "datastore",
            "type": "levelds"
          },
          "mountpoint": "/",
          "prefix": "leveldb.datastore",
          "type": "measure"
        }
      ],
      "type": "mount"
    },
    "StorageGCWatermark": 90,
    "StorageMax": "10GB"
  },
  "Discovery": {
    "MDNS": {
      "Enabled": true,
      "Interval": 10
    }
  },
  "Experimental": {
    "AcceleratedDHTClient": false,
    "FilestoreEnabled": false,
    "GraphsyncEnabled": false,
    "Libp2pStreamMounting": false,
    "P2pHttpProxy": false,
    "StrategicProviding": false,
    "UrlstoreEnabled": false
  },
  "Gateway": {
    "APICommands": [],
    "HTTPHeaders": {
      "Access-Control-Allow-Headers": [
        "X-Requested-With",
        "Range",
        "User-Agent"
      ],
      "Access-Control-Allow-Methods": [
        "GET"
      ],
      "Access-Control-Allow-Origin": [
        "*"
      ]
    },
    "NoDNSLink": false,
    "NoFetch": false,
    "PathPrefixes": [],
    "PublicGateways": null,
    "RootRedirect": "",
    "Writable": false
  },
  "Identity": {
    "PeerID": "12D3KooWMf22rjoqjixa3AHh6Qi7Kkonv8wKRD1sTfmZogC8qroe"
  },
  "Internal": {},
  "Ipns": {
    "RecordLifetime": "",
    "RepublishPeriod": "",
    "ResolveCacheSize": 128
  },
  "Migration": {
    "DownloadSources": [],
    "Keep": ""
  },
  "Mounts": {
    "FuseAllowOther": false,
    "IPFS": "/ipfs",
    "IPNS": "/ipns"
  },
  "Peering": {
    "Peers": null
  },
  "Pinning": {
    "RemoteServices": {}
  },
  "Plugins": {
    "Plugins": null
  },
  "Provider": {
    "Strategy": ""
  },
  "Pubsub": {
    "DisableSigning": false,
    "Router": ""
  },
  "Reprovider": {
    "Interval": "12h",
    "Strategy": "all"
  },
  "Routing": {
    "Type": "dht"
  },
  "Swarm": {
    "AddrFilters": null,
    "ConnMgr": {
      "GracePeriod": "20s",
      "HighWater": 900,
      "LowWater": 600,
      "Type": "basic"
    },
    "DisableBandwidthMetrics": false,
    "DisableNatPortMap": false,
    "RelayClient": {},
    "RelayService": {},
    "Transports": {
      "Multiplexers": {},
      "Network": {},
      "Security": {}
    }
  }
}

Description

Per my config, I am just using my datastore plugin. My datastore plugin is intentionally read-only, since it is a bridge to another interface that allows content-addressed reads, and only of sha-1 hashes.

$ GOLOG_LOG_LEVEL="swh-bridge=debug" IPFS_PATH=(mktemp -d) ./result/bin/ipfs init -e -p swhbridge
generating ED25519 keypair...done
peer identity: 12D3KooWHxWwyzX1aZB433nTyEGdbqFWsyr4ZSahvuye7YwsxoBy
initializing IPFS node at /tmp/tmp.zQqqP2vNm7
2022-07-28T21:02:29.611-0400	DEBUG	swh-bridge	bridge/datastore.go:160	Requested key bridge can't get: Data was a multihash encoded with 18, but expected 17 (SHA1)
Error: merkledag: not found

As one can see my datastore rejects a get request as impossible to translate. And then the command fails with Error: merkledag: not found. However (1), the local data seems initialized perfectly fine after all, at least I have used it without issue. So whatever failed here seems...not important?

I am normally a "fail fast" sort of person, but if I can in fact initialize a local node without doing any datastore read/writes (which seems like a reasonable thing to be able to do, all things equal), could this work more gracefully? Maybe with a --no-use-datastore flag to opt into non trying to the main version can stay fail fast?

Note I am already using -e / --empty-repo so it should not failing trying to download the read me. Of course I would expect it to fail (and that would be good) if it were! :)

@Ericson2314 Ericson2314 added kind/bug A bug in existing code (including security flaws) need/triage Needs initial labeling and prioritization labels Jul 29, 2022
@Jorropo
Copy link
Contributor

Jorropo commented Jul 29, 2022

@Ericson2314 we think it's the empty MFS root,
and the MFS is incompatible over sha1-readonly datastore anyway.

So it's an incompatible feature that report it self poorly, it's not preventing other features from working.
Imo the only good simple solution here is making the log message better,
but that P4 since it require a pretty obscure setup to begin with, if you want to send a PR I'm gonna take a look at it.

@aschmahmann aschmahmann added P4 Very low priority and removed need/triage Needs initial labeling and prioritization labels Jul 29, 2022
@Ericson2314
Copy link
Author

@Jorropo Thanks for the info. Is it possible to disable MFS from the config?

Agreed if I can do and skip whatever is causing the error, making the error itself nicer is low priority.

@Ericson2314
Copy link
Author

FWIW I updated the project to Kubo 0.14, and the error is now a bit better:

 $ GOLOG_LOG_LEVEL="swh-bridge=debug" IPFS_PATH=~/.ipfs-bridge ./result/bin/ipfs init -e -p swhbridge
generating ED25519 keypair...done
peer identity: 12D3KooWEacGn4bQGRCTov1guErppmisAvJrfTkaYirHArUWwYYF
initializing IPFS node at /home/jcericson/.ipfs-bridge
2022-07-30T11:32:37.880-0400	DEBUG	swh-bridge	bridge/datastore.go:175	Requested key bridge can't get: Data was a multihash encoded with 18, but expected 17 (SHA1)
Error: ipld: could not find QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn

QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn is an empty directory, which would seem to match the the MFS hypothesis?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug A bug in existing code (including security flaws) P4 Very low priority
Projects
None yet
Development

No branches or pull requests

3 participants