[Tracking issue] http delegated routing used in production

[BigLep]

Done Criteria

1. HTTP delegated routing is enabled on the ipfs.io gateway, hitting cid.contact in parallel to querying the DHT to determine providers for a CID.
2. ipfs.io gateway operators are able to assess delegated routing calls health with metrics.

Notes

This corresponds with "stage 1" in https://www.notion.so/pl-strflt/Indexer-IPFS-Reframe-Q3-Plan-77d0f3fa193b438784d2d85096e315e7

It doesn't matter if STI gives empty results or if the results are unreachable because they're to Filecoin Storage Providers that don't allow data transfer with Bitswap. Enabling retrieval over Bitswap for Filecoin SP's is a separate effort.

Reframe shipped in Kubo 0.14 as part of #8775. That said, it isn't used in production that we know of. The purpose of this tracking issue is to track followups that we have already identified as needing to be done or that come up as a result of attempting production deployment.

• Design desired comprehensive routing config #9188
• Expose Kubo metrics for reframe calls so a Kubo client has visibility into reframe activity (success/failure, latency).
• Convert to using the updated HTTP Delegated Routing API: Updated/new HTTP API Migration go-delegated-routing#63
• Enable on ipfs.io gateway (production usecase):
  • Old: https://github.com/protocol/bifrost-infra/issues/2120
  • New: https://github.com/protocol/bifrost-infra/issues/2183

---

Expanded scope for items observed while in production:

• Split provide messages (message size limits) go-delegated-routing#55

Other followup issues that should be done, but aren't directly tied to the done criteria.

• Add cacheable GET endpoint for findProviders go-delegated-routing#27
• Method proto.DelegatedRouting_Client.GetIPNS is never used. go-delegated-routing#30
• [Blog Post] Introducing Reframe ipfs-blog#446
• https://github.com/protocol/bifrost-infra/issues/2142

[BigLep]

2022-08-19 conversation:

Tasks involved in finishing this off:

1. Polish off the design

• Answer last remaining questions.
• We didn't estimate this, but it should be low/quick.

2. New routers (3 dev days)

• New routers
  • Sequential router
  • Parallel router
  • Router to filter by method
• May include refractoring
• Includes unit tests
• Code will live in https://github.com/libp2p/go-libp2p-routing-helpers
• Going to use existing routers if possible so we have less code to maintain but if this turns into a headache with Lotus, then may need new version.

3. Make this work in Kubo (5 dev days)

• Config parsing
• Generate Routing instances from config (simple approach)
• Using FX

4. Documentation (1 dev days)

• User-friendly config documentation
• Move design documentation around
• We can have a doc like "design decisions for routing routers".
• Idea: you could create kubo/docs/exploration-reposts dir, and put it there (ipld folks did that in past: https://github.com/ipld/specs/tree/master/design/history/exploration-reports, and then moved content to the proper website doics/specs )

5. Sharness smoke test (1 dev day)

• This will be adjusting the existing Sharness test we have.

6. Operational preparation (1 dev days)

• Need to answer how much time each of the routers are taking
• Edelweiss: define metrics we want, RPC metrics are already there

7. Testing on gateways (2 dev days)

Total estimate: 13 ideal dev days

Assuming a velocity of .7 ideal dev day to actual day, that puts us at 19 calendar days assuming just one person working on this.

Next steps here:

• @BigLep communicate worst-case calendar date in #reframe
• Clean up the existing issues
• Create new issues that PRs can be linked against

[BigLep]

@ajnavarro : I moved over the estimates from #9188 (comment) to here.

Can you help with the issue cleanup here? Specifically:

1. I think we need a new issue in go-libp2p-routing-helpers for the new routers.
2. I assume we'll update Routing: conform DHT to common Router interface #9079 to account for the design changes. This is "Make this work in Kubo" above right or are you going to create another issue?
3. I assume Routing: priority and mandatory Router Params #9083 and Delegated Routing: Allow to configure methods used by router.  #9157 can be closed in favor of the work happening in # 2 above, is that right?

[ajnavarro]

• New issue for parallel and sequential routers New Routing helpers: parallel and sequential. libp2p/go-libp2p-routing-helpers#56
• Updated issue Routing: conform DHT to common Router interface #9079
• What issue are you referring to? I think it is a typo.

[BigLep]

@ajnavarro : thanks. I updated "# 3" above. I believe the confusion stemmed from the fact that I wrote "#2" which got translated to issue "#2" in the repo.

[ajnavarro]

@BigLep yep, we can close #9083

[BigLep]

As communicated in Slack:

Total estimate is 13 ideal dev days
Assuming a velocity of .7 ideal dev day to actual day, that puts us at 19 calendar days assuming just one person working on this.
That feels high to me, but I think Bedrock should plan around that for now given we'll likely experience a jolt when Adin heads out, there is ongoing Kubo maintenance efforts that the team gets pulled into, and I don't think we'll be able to have an extra engineer on this beyond helping with code reviews.

[BigLep]

@ajnavarro : should #9157 combine into #9079 or will the config work be a separate PR. If it will be separate then I agree we should have two issues.

[ajnavarro]

@BigLep I think we can do them separately.