cc @aschmahmann as it aims to address concerns from #496
cc @willscott @masih for visibility and feedback from IPNI
cc @vasco-santos @ribasushi @alanshaw from Storacha side of things

Context: this IPIP explores idea of only authorized PeerIDs being able to announce /tls/http endpoint on Amino DHT (DHT servers would be ignoring multiaddrs that dont pass this validation), but could also act as blueprint for other routing systems that don't want to be used for amplification attacks.

Why we need this and how this relates to Storacha if it does not use DHT?

• HTTP retrieval is still wip, and remains opt-in for now
  • We are already running HTTP-only retrieval from Storacha at ipfs.io/dweb.link/trustless-gateway.link, however Storacha still has to run Bitswap for the rest of the swarm, because HTTP retrieval is not enabled by default everywhere
  • To be able to enable HTTP retrieval client by default across the stack (GO/JS), we need to resolve Spec for HTTPS-only providers on Amino DHT #496 first.

How does this relate to IPNI or other routing systems?

• The way IPFS clients query routing systems does not care about details of specific routing system. Query for CID produces a list of providers as PeerID + multiaddrs.
• IPNI itself could benefit from similar validation of multiaddrs and HTTP providers (afaik none exists today, addrs are accepted blindly, same amplification attack concerns as Amino DHT)
• IPNI+DHT+FOO hybrid announcement during the initial data onboarding
  • The onboarding latency between announcing new CIDs to IPNI, IPNI pulling that info, and CID being resolvable via https://cid.contact could be mitigated by doing a one-time announcement to Amino DHT. DHT acting as a "hot storage" for routing info, before it gets propagated to systems like IPNI.
  • This would be done by storage providers that onboard data, and if they, like Storacha, move towards being HTTP-only, we need to agree how to handle /tls/http maddrs)

cc @aschmahmann as it aims to address concerns from #496

This approach does not address my first issue from #496 (comment) if this is supposed to be "enough" to do trustless-gateway based retrieval.

How to handle more than 1 HTTP-based protocol

More info (background, some options, what IPNI does, potential solutions, etc.) is in the linked comment. If we're going to flat out ignore the issue then we should at least document the ramifications / implied spec changes that come from not choosing an explicit mechanism for handling more than 1 HTTP-based protocol

Good point!

The focus of this IPIP is to define the verification mechanism for HTTP Trustless Gateways advertisements in the Amino DHT, to prevent HTTP-only clients from being used in a DDoS (reflection) attack.

In parallel, we should have another IPIP defining the handling of multiple transfer protocols in the DHT. The current IPIP (#501) will probably depend on the future IPIP, so it will block on that.

cc @aschmahmann as it aims to address concerns from #496

This approach does not address my second issue from #496 (comment).

Inconsistency regarding attack vectors for HTTPS vs libp2p-based protocols like tcp + tls + yamux + bitswap

A particularly easy example of this is that IIUC just about all the work this proposal tries to prevent from happening with /tls/http addresses happens anyway with /tls/ws addresses.

To illustrate:

• Assume a malicious user, Malory, wants to attack foo.tld which exposes an HTTPS server, but knows nothing about IPFS
• The unwitting clients who will be dialing foo.tld are expecting to speak the IPFS HTTP Trustless Gateway protocol to them
• Malory now can't easily ask DHT servers to host provider records for /dns/foo.tld/tls/http/p2p/<peerID> (where the peerID is implied since we don't have delegated providing)
• However, she can ask the DHT servers to host provider records for /dns/foo.tld/tls/ws/p2p/<peerID> which should have the same result
• The result is those target HTTP servers have to handle the TLS handshake and will then fail on the first HTTP request

I gave some options in the linked comment, but I suppose another one if you wanted to push the extra burden onto the routing system (rather than to the clients/servers) is to do more checking across addresses and/or to label which ones have been checked vs not. This does mean even more work for DHT servers though (e.g. you're now exposing them to attack vectors where Malory does a little bit of work and the DHT server ends up doing a bunch of work).

The goal here is not to make libp2p records/maddrs more secure against DDoS (reflection) attacks, nor preventing libp2p nodes from being fooled into participating to such an attack.

The goal is to avoid adding another reflection attack vector to the Amino DHT, by preventing HTTP-only clients from being used in such attacks. This is important since we want/hope for a larger number of HTTP-only clients than we have currently of libp2p clients.

HTTP-only clients are trustless gateway users in this context.

So if this IPIP is adopted, HTTP addresses will have better security guarantees in the Amino DHT than pure libp2p addresses. It is fine to have this inconsistency because 1) the check for HTTP addresses is cheap 2) we expect HTTP-only clients to outgrow libp2p clients 3) verifying libp2p addresses is probably not cost effective (for now).

A particularly easy example of this is that IIUC just about all the work this proposal tries to prevent from happening with /tls/http addresses happens anyway with /tls/ws addresses.

IIUC trustless gateways don't use tls/ws addresses anyway, so their clients cannot be fooled into dialing /tls/ws addresses, which would require them libp2p compatibility.

This approach does not address my second issue from #496 (comment)

I gave some options in the linked comment

We built on one the possible suggested solutions.

Adding additional checks to the HTTP-based approach to bring it roughly equal to the libp2p one. There are options like:

• Use some other binding to the HTTP peer to make assertions that the endpoint actually wanted to be requested (e.g. DNS, HTTP, etc. based as happens with ACME)