Feat: end to end payments

[hopeyen]

Problem statement

Finishing #31

Expectation proposal

Working payments from client depositing to server redeeming; validate TAP Escrow receipt flow and make necessary changes

[hopeyen]

Notes from testing; pausing the effort for now as issues get fixed

started with a fresh db
ran indexer-agent migration

as the data provider, created an allocation from file-exchange wallet CLI

• allocation provides an unique id for the escrow account, which include (sender, receiver, allocation id). data provider is the receiver.
• on arbitrum sepolia, there are occasionally issues between epoch manager and network subgraph's current epoch
• indexer-common's TAP query might requires a fix; use lowercase addresses for indexer filter; currently using hardcoded address

as the data consumer, approved and deposit Escrow with file-exchange wallet CLI (this is currently automated for the downloader which uses the consumer's receipt signer wallet instead of sender, need to fix)

as the data consumer, set an additional authorized signer on Escrow contract. This is a different wallet than the sender for security

• local network, gustavo/m1 branch, tap-contract-calls contains a script for generating proof and setting authorized signers; make sure to update the environmental variables as well as chain_id which is hard-coded in

as the data consumer, using the signer wallet, ran downloader command with wallet payment method

Scalar receipts were sent from the consumer directly to the provider,

• we need to run a tap-agent that sends aggregation requests - this is normal for indexer-service
• need to run an aggerator for the receipt signer (gateway, but since we are gateway-less, we need the authorized signer to run this piece of software)

producer can then use tap-agent to send receipts to the aggregator, that returns RAVs and be used to redeem on the indexer-agent's side)

Reported issues with DB migration in indexer-agent

More specifically on P2P TAP requiring aggregators. They are supposed to be ran by a gateway that provides payment and routing abstractions. Providers can consider running their own aggregator but requires trust from the consumer to set the aggregator as their authorized signer (not good!), or client runs their own aggregator which requires providers to add the aggregator addresses and endpoints to the indexer TAP agent (also not good!), or something more dynamic and on-chain; need a more detailed session as we only had a few minutes

[hopeyen]

On who should run the aggregator

Aggregator properties
The aggregator can aggregate any receipt from any signer by including those signer public keys. These keys are published as "Authorized Signers" for that specific sender. The indexer will only accept receipts from those signers. A sender can revoke any signer anytime.
The Aggregator verifies the signature for all receipts and previous rav if the address is the signer or contains on that public_keys list, such that a RAV returned by one aggregator can be used to aggregate more receipts by another aggregator.

Downloader runs aggregator

• Heavier client
• Must run even after downloading the files; if they shut down aggregator before all receipts gets aggregated into a RAV, provider risks the value of unaggregated receipts
  • There's no clear time to shut down the aggregator, since it depends on when providers finish aggregating all their receipts; the providers can configure their aggregation triggers locally and the configurations are not visible to the downloaders' aggregator
• Aggregator endpoint must be shared with the providers, this requires an additional exchange between the providers and downloaders. Such a request shouldn't be paid and comes with DOS risk for the providers.

Provider runs aggregator

• Server is slightly heavier. The additional weight of aggregator should be negligible for the existing server.
• Server must maintain a list of receipt signers (this can come from the request's receipts; a future iteration of the aggregator could read from TAP subgraph to learn all authorised signers)
• No trust required from providers to a particular receipt as they can aggregate receipts whenever, once those receipts are validated and stored upon service request.
• Risk of adding receipt signers is when adding one address with leaked private key, then anyone could forge receipts.

My assessment is to recommend providers to run an aggregator instance to guarantee all validated receipt values. This keeps the downloading client as light and predictable as possible, while providers gets receipt value guaranteed and self-sufficient once a receipt gets validated.

• A provider is not forced to run an aggregator, they can always rely on an external one, risking the aggregator rejecting some authorised signers.

Current stage is running file server, downloader, and aggregator. The next steps is value collection, which involves running an indexer tap agent for aggregating receipts, and an indexer agent for redeeming RAVs. This is currently blocked by successfully running indexer tap agent, seeking support from the TAP team

Also, make a feature request to TAP aggregator to manage public keys without restarts