Support a DHT-only proxy mode exposed over HTTP

[willscott]

In thinking about what a joint deployment of indexer & dht might look like, we likely want to run instances of accelerated DHT clients that can fill in DHT records in delegated routing responses.

We probably want those responses to take advantage of the same caching as current indexing responses.

The potential difference in latency means that for uncached responses we'll likely want to make sure we split the response into multiple response packets as results return, and parse incrementally on the client side.

[lidel]

If we want DHT-only proxy mode that allows us shutting down DHT client on gateways, and delegating it to someguy instance deployed in or next to cid.contact, we need:

1. Expose /routing/v1 HTTP APIs for proxying DHT+IPNI #12
2. Support DHT-based Peer routing over HTTP #14
3. Support DHT-based IPNS routing over HTTP #15

[willscott]

We're part way through the direct cascade for find-providers via https://github.com/ipni/caskadht

The IPNS part would be, i think, on the top level ipfs.tech gateways, rather than something needed in the L1s in my current understanding of architecture. does that match with your understanding?

[masih]

@lidel would it make sense to move the three issues above to caskadht repo? That's the service we plan to run on cid.contact.

[lidel]

@willscott depends on how Block/CAR requests are handled. current diagram assumes client will make direct request to L1s (immediatelly, or after getting HTTP 302 from ipfs.io)

• If CAR request for /ipns/{id} is sent directly to L1, then L1 needs to be able to resolve it to a CID.
• if there is no HTTP API for IPNS delegation, then ipfs.io gateway still needs to run DHT client to resolve IPNS (which defeats the plan to remove DHT cost and move it to saturn/ipni)

@masih will caskadht be exposing streaming API compatible with IPIP-337 (GET /routing/v1/providers/{CID})? If caskadht can do that, then we no longer need someguy 👍

[masih]

will caskadht be exposing streaming API compatible with IPIP-337

Yes. That code already implements ndjosn response. I'll add HTTP delegated routing APIs next.

[lidel]

Hm.. we need a binary that has only one API, the IPIP-337 one. Would that be acceptable?
Otherwise, we will be leaking internal IPNI APIs to IPFS ecosystem, increasing surface of things we need to support.
(We have too many APIs doing the same thing, it kills UX/DX)

[willscott]

if there is no HTTP API for IPNS delegation, then ipfs.io gateway still needs to run DHT client to resolve IPNS (which defeats the plan to remove DHT cost and move it to saturn/ipni)

Can we use logs to get rough sense of how many queries involve IPNS vs find-providers? my sense is that there are relatively few IPNS lookups, so that even with IPNS resolution remaining gateways, we would still mostly remove gateway costs.

[willscott]

Hm.. we need a binary that has only one API, the IPIP-337 one.

Do you mean as what we run on cid.contact? I'm confused about this requirement - indexers have compatibility back to reframe and previous delegated routing APIs, and will retire those as we see usage drop off.
Making a separate container here adds operational complexity and doesn't seem like it changes that.

[lidel]

my sense is that there are relatively few IPNS lookups, so that even with IPNS resolution remaining gateways, we would still mostly remove gateway costs.

Agree, it is relatively small %. We can do it, just flagging that adding DHT server to new ipfs.io adds to the scope on our end,
and still does not remove IPNS resolution requirement from L1s.

Do you mean as what we run on cid.contact?

No, this is for IPFS Ecosystem. What people can run themselves. Reusable DHT proxy that speaks IPIP-337 + ndjson.

Whatever IPFS Stewards build, users are able to self-host, and remove their dependency on infrastructure run by a third-party such as Protocol Labs. In this case, DHT proxy compatible with IPIP-337 is a very useful component which we want to provide for download at https://dist.ipfs.tech (just like we do with libp2p-relay-daemon).

[willscott]

Reusable DHT proxy that speaks IPIP-337 + ndjson.

this is probably best served by something like someguy rather than caskadht.

I'm happy to defer to @masih - but if the expectation of an ecosystem delegation is that it can't include non-ipip behavior, that will likely create tensions with service deployment goals.

[masih]

Reusable DHT proxy that speaks IPIP-337 + ndjson.

I am easy. I am happy to make caskadht be that, and or move the repo wherever you see fit. Also happy for someguy to be it. Considering caskadht implements ndjson, probably using that is the path of less resistance.

[lidel]

Ack, I think you focusing on caskadht is fine for the immediate task at hand (especially if it is being hidden behind cid.contact, and that is the only IPNI in Kubo).

Long term, for delegate.routing.ipfs.tech (https://github.com/protocol/bifrost-infra/issues/2142) we will need productized something like the old someguy, something that can be put in front of multiple IPNIs like cid.contact, and also run the own DHT, so the lookups in self-hosted setups and our browser nodes (helia, js-ipfs) are not tied to well-being of a specific IPNI.

[willscott]

I would hope long term we can use ambient discover as the way to not depend on well-being of a specific IPNI. delegate.routing.ipfs.tech will have similar single-point-of-failure characteristics as a single IPNI.