Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: usage/record capability definition #1562

Merged
merged 4 commits into from
Oct 24, 2024
Merged

feat: usage/record capability definition #1562

merged 4 commits into from
Oct 24, 2024

Conversation

fforbeck
Copy link
Member

@fforbeck fforbeck commented Oct 11, 2024 •
edited
Loading

Add usage/record Capability definition for Egress Traffic Tracking

Summary:

  • Introduced a new capability usage/record definition to track egress traffic metrics.
  • The w3upInfra upload-api will be updated to implement this new capability by leveraging Stripe's Usage Records API for accurate tracking and billing.

RFC: storacha/RFC#37

@fforbeck fforbeck changed the title feat: usage/record capability feat: usage/record capability definition Oct 11, 2024
@fforbeck fforbeck force-pushed the feat/egress-tracking branch 4 times, most recently from 845af69 to b853d98 Compare October 11, 2024 17:19
alanshaw
alanshaw requested changes Oct 11, 2024
View reviewed changes
packages/capabilities/src/types.ts Outdated Show resolved Hide resolved
packages/capabilities/src/usage.js Outdated Show resolved Hide resolved
packages/upload-api/test/storage/usage-storage.js Outdated Show resolved Hide resolved
packages/w3up-client/src/capability/usage.js Outdated Show resolved Hide resolved
@fforbeck fforbeck mentioned this pull request Oct 18, 2024
Merged
travis
travis previously requested changes Oct 21, 2024
View reviewed changes
Copy link
Member

@travis travis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One question about the customer account DID that needs to be resolved - I think this PR probably needs to do the space DID -> account DID resolution OR needs to assume the underlying UsageStore implementation does that

packages/capabilities/src/usage.js Outdated Show resolved Hide resolved
packages/upload-api/src/usage/record.js Show resolved Hide resolved
packages/w3up-client/src/capability/usage.js Show resolved Hide resolved
@fforbeck fforbeck requested a review from travis October 22, 2024 13:43
alanshaw
alanshaw requested changes Oct 23, 2024
View reviewed changes
packages/upload-api/src/usage/record.js Outdated Show resolved Hide resolved
packages/upload-api/src/usage/record.js Outdated Show resolved Hide resolved
packages/upload-api/src/types/usage.ts Outdated Show resolved Hide resolved
packages/upload-api/src/types/usage.ts Show resolved Hide resolved
packages/upload-api/src/types/usage.ts Show resolved Hide resolved
packages/w3up-client/src/capability/usage.js Outdated Show resolved Hide resolved
packages/capabilities/src/types.ts Outdated Show resolved Hide resolved
packages/capabilities/src/usage.js Outdated Show resolved Hide resolved
packages/upload-api/src/types/usage.ts Show resolved Hide resolved
packages/w3up-client/test/capability/usage.test.js Outdated Show resolved Hide resolved
@fforbeck
Copy link
Member Author

@alanshaw I've implemented the suggested changes and added an additional note to the resource field explaining that it is not a CID of a blob/shard. It is ready for another review when you get a chance.

@fforbeck
Copy link
Member Author

@travis I've implemented the suggested changes. It is ready for another review.

alanshaw
alanshaw approved these changes Oct 23, 2024
View reviewed changes
packages/w3up-client/src/capability/usage.js
* @param {API.SpaceDID} egressData.space
* @param {API.UnknownLink} egressData.resource
* @param {number} egressData.bytes
* @param {string} egressData.servedAt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could make this optional and default to now.

@fforbeck fforbeck dismissed travis’s stale review October 24, 2024 13:30

changes implemented.

@fforbeck fforbeck merged commit 98c8a87 into main Oct 24, 2024
8 checks passed
@fforbeck fforbeck deleted the feat/egress-tracking branch October 24, 2024 13:31
This was referenced Oct 24, 2024
Merged
Merged
This was referenced Oct 24, 2024
Merged
Merged
fforbeck pushed a commit that referenced this pull request Oct 24, 2024
@github-actions
chore(main): release w3up-client 16.3.0 (#1568)
89b4962 
🤖 I have created a release *beep* *boop*
---


##
[16.3.0](w3up-client-v16.2.0...w3up-client-v16.3.0)
(2024-10-24)


### Features

* usage/record capability definition
([#1562](#1562))
([98c8a87](98c8a87))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
fforbeck pushed a commit that referenced this pull request Oct 24, 2024
@github-actions
chore(main): release capabilities 17.4.0 (#1559)
feaea7a 
🤖 I have created a release *beep* *boop*
---


##
[17.4.0](capabilities-v17.3.0...capabilities-v17.4.0)
(2024-10-24)


### Features

* Generate Space proofs on the fly, on `access/claim`
([#1555](#1555))
([9e2b1d4](9e2b1d4))
* usage/record capability definition
([#1562](#1562))
([98c8a87](98c8a87))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
fforbeck added a commit that referenced this pull request Oct 24, 2024
@github-actions @fforbeck
chore(main): release upload-api 18.1.0 (#1543)
e93190d 
🤖 I have created a release *beep* *boop*
---


##
[18.1.0](upload-api-v18.0.3...upload-api-v18.1.0)
(2024-10-24)


### Features

* Generate Space proofs on the fly, on `access/claim`
([#1555](#1555))
([9e2b1d4](9e2b1d4))
* usage/record capability definition
([#1562](#1562))
([98c8a87](98c8a87))


### Fixes

* Error should refer to Resource, not Issuer
([#1558](#1558))
([25e35e3](25e35e3))
* repo URLs ([#1550](#1550))
([e02ddf3](e02ddf3))
* **test:** await promise and check error
([#1563](#1563))
([86e7a46](86e7a46))


### Other Changes

* Add `pnpm dev` to watch-build all packages
([#1533](#1533))
([07970ef](07970ef))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Felipe Forbeck <fforbeck@gmail.com>
fforbeck added a commit that referenced this pull request Oct 24, 2024
@github-actions @fforbeck
chore(main): release upload-client 17.1.0 (#1542)
1a778a3 
🤖 I have created a release *beep* *boop*
---


##
[17.1.0](upload-client-v17.0.1...upload-client-v17.1.0)
(2024-10-24)


### Features

* usage/record capability definition
([#1562](#1562))
([98c8a87](98c8a87))


### Fixes

* repo URLs ([#1550](#1550))
([e02ddf3](e02ddf3))


### Other Changes

* Add `pnpm dev` to watch-build all packages
([#1533](#1533))
([07970ef](07970ef))
* Warn if mocks aren't running
([#1534](#1534))
([fb2fd66](fb2fd66))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Felipe Forbeck <fforbeck@gmail.com>
alanshaw pushed a commit to storacha/upload-service that referenced this pull request Nov 5, 2024
@github-actions
chore(main): release capabilities 1.0.0 (#16)
1c0902d 
🤖 I have created a release *beep* *boop*
---


## 1.0.0 (2024-11-05)


### ⚠ BREAKING CHANGES

* add `index/add` handler
([storacha#1421](https://github.com/storacha/upload-service/issues/1421))
* restrict store API to CARs
([storacha#1415](https://github.com/storacha/upload-service/issues/1415))
* **capabilities:** `BlobMultihash` type in `@web3-storage/capabilities`
renamed to `Multihash`.
* allocations storage interface now requires remove to be implemented
* return allocated bytes in `store/add` receipt
([storacha#1213](https://github.com/storacha/upload-service/issues/1213))
* coupon
([storacha#1136](https://github.com/storacha/upload-service/issues/1136))
* see latest specs
https://github.com/web3-storage/specs/blob/cbdb706f18567900c5c24d7fb16ccbaf93d0d023/w3-filecoin.md
* filecoin client to use new capabilities
* filecoin capabilities

### refactor

* filecoin api services events and tests
([storacha#974](https://github.com/storacha/upload-service/issues/974))
([953537b](953537b))
* filecoin capabilities
([c0b97bf](c0b97bf))
* filecoin client to use new capabilities
([b0d9c3f](b0d9c3f))


### Features

* add "plan/create-admin-session" capability
([storacha#1411](https://github.com/storacha/upload-service/issues/1411))
([50eeeb5](50eeeb5))
* add `index/add` handler
([storacha#1421](https://github.com/storacha/upload-service/issues/1421))
([cbe9524](cbe9524))
* add `initialize` method to `PlansStorage`
([storacha#1278](https://github.com/storacha/upload-service/issues/1278))
([6792126](6792126))
* add `store/get` and `upload/get` capabilities
([storacha#942](https://github.com/storacha/upload-service/issues/942))
([40c79eb](40c79eb))
* add `subscription/list` capability
([storacha#1088](https://github.com/storacha/upload-service/issues/1088))
([471d7e5](471d7e5))
* add a function to verify and return Abilities.
([storacha#1252](https://github.com/storacha/upload-service/issues/1252))
([2f026a2](2f026a2))
* add blob list and remove
([storacha#1385](https://github.com/storacha/upload-service/issues/1385))
([2f69946](2f69946))
* add blob protocol to upload-client
([storacha#1425](https://github.com/storacha/upload-service/issues/1425))
([49aef56](49aef56))
* add blob/get
([storacha#1484](https://github.com/storacha/upload-service/issues/1484))
([328039d](328039d))
* add revocation to access-client and w3up-client
([storacha#975](https://github.com/storacha/upload-service/issues/975))
([6c877aa](6c877aa))
* add usage/report capability
([storacha#1079](https://github.com/storacha/upload-service/issues/1079))
([6418b4b](6418b4b))
* blob, web3.storage and ucan conclude capabilities together with api
handlers
([storacha#1342](https://github.com/storacha/upload-service/issues/1342))
([00735a8](00735a8))
* **capabilities:** add `index/add` capability
([storacha#1410](https://github.com/storacha/upload-service/issues/1410))
([1b71b89](1b71b89))
* change `plan/update` to `plan/set` and use existing `PlansStorage#set`
to implement an invocation handler
([storacha#1258](https://github.com/storacha/upload-service/issues/1258))
([1ccbfe9](1ccbfe9))
* coupon
([storacha#1136](https://github.com/storacha/upload-service/issues/1136))
([1b94f2d](1b94f2d))
* filecoin info
([storacha#1091](https://github.com/storacha/upload-service/issues/1091))
([adb2442](adb2442))
* Generate Space proofs on the fly, on `access/claim`
([storacha#1555](https://github.com/storacha/upload-service/issues/1555))
([9e2b1d4](9e2b1d4))
* implement `plan/get` capability
([storacha#1005](https://github.com/storacha/upload-service/issues/1005))
([f0456d2](f0456d2))
* introduce capability for changing billing plan
([storacha#1253](https://github.com/storacha/upload-service/issues/1253))
([d33b3a9](d33b3a9))
* move aggregate information out of deals in filecoin/info
([storacha#1192](https://github.com/storacha/upload-service/issues/1192))
([18dc590](18dc590))
* publish index claim
([storacha#1487](https://github.com/storacha/upload-service/issues/1487))
([237b0c6](237b0c6))
* restrict store API to CARs
([storacha#1415](https://github.com/storacha/upload-service/issues/1415))
([e53aa87](e53aa87))
* return allocated bytes in `store/add` receipt
([storacha#1213](https://github.com/storacha/upload-service/issues/1213))
([5d52e44](5d52e44))
* router ([#11](#11))
([c810735](c810735))
* upgrade ucanto/transport to 9.1.0 in all packages to get more verbose
errors from HTTP transport on non-ok response
([storacha#1312](https://github.com/storacha/upload-service/issues/1312))
([d6978d7](d6978d7))
* usage/record capability definition
([storacha#1562](https://github.com/storacha/upload-service/issues/1562))
([98c8a87](98c8a87))
* wip router
([ffcd9c7](ffcd9c7))


### Fixes

* add missing ContentNotFound definition for filecoin offer failure
([c0b97bf](c0b97bf))
* add missing filecoin submit success and failure types
([c0b97bf](c0b97bf))
* capabilities should export blob caps
([storacha#1376](https://github.com/storacha/upload-service/issues/1376))
([460729e](460729e))
* client tests
([b0d9c3f](b0d9c3f))
* fix arethetypesworking errors in all packages
([storacha#1004](https://github.com/storacha/upload-service/issues/1004))
([2e2936a](2e2936a))
* issue where typedoc docs would only show full docs for w3up-client
([storacha#1141](https://github.com/storacha/upload-service/issues/1141))
([0b8d3f3](0b8d3f3))
* migrate repo
([storacha#1389](https://github.com/storacha/upload-service/issues/1389))
([475a287](475a287))
* one more tweak to the `PlanStorage` interface
([storacha#1280](https://github.com/storacha/upload-service/issues/1280))
([5a44565](5a44565))
* package metadata
([storacha#1161](https://github.com/storacha/upload-service/issues/1161))
([b8a1cc2](b8a1cc2))
* put access.session back
([storacha#1100](https://github.com/storacha/upload-service/issues/1100))
([10a1a4b](10a1a4b))
* rename blob and index client capabilities
([storacha#1478](https://github.com/storacha/upload-service/issues/1478))
([17e3a31](17e3a31))
* repo URLs
([storacha#1550](https://github.com/storacha/upload-service/issues/1550))
([e02ddf3](e02ddf3))
* tests
([b179910](b179910))
* trigger capabilities release
([storacha#1399](https://github.com/storacha/upload-service/issues/1399))
([7d9ab35](7d9ab35))
* type errors
([c0b97bf](c0b97bf))
* update data-segment dep
([228ff79](228ff79))
* upgrade @ucanto/validator with bugfix
([storacha#1151](https://github.com/storacha/upload-service/issues/1151))
([d4e961b](d4e961b))
* upgrade ucanto core
([storacha#1127](https://github.com/storacha/upload-service/issues/1127))
([5ce4d22](5ce4d22))
* upgrade ucanto in filecoin api
([c95fb54](c95fb54))
* upgrade ucanto libs and format filecoin api
([storacha#1359](https://github.com/storacha/upload-service/issues/1359))
([87ca098](87ca098))
* upload API test fixes
([6b0d72d](6b0d72d))


### Other Changes

* Add `pnpm dev` to watch-build all packages
([storacha#1533](https://github.com/storacha/upload-service/issues/1533))
([07970ef](07970ef))
* **main:** release capabilities 10.1.0
([storacha#979](https://github.com/storacha/upload-service/issues/979))
([bdd3970](bdd3970))
* **main:** release capabilities 10.2.0
([storacha#983](https://github.com/storacha/upload-service/issues/983))
([a906488](a906488))
* **main:** release capabilities 11.0.0
([storacha#994](https://github.com/storacha/upload-service/issues/994))
([76b0489](76b0489))
* **main:** release capabilities 11.0.1
([storacha#1008](https://github.com/storacha/upload-service/issues/1008))
([37cdc5a](37cdc5a))
* **main:** release capabilities 11.1.0
([storacha#1026](https://github.com/storacha/upload-service/issues/1026))
([7fdb541](7fdb541))
* **main:** release capabilities 11.2.0
([storacha#1084](https://github.com/storacha/upload-service/issues/1084))
([0e7b6dc](0e7b6dc))
* **main:** release capabilities 11.3.0
([storacha#1098](https://github.com/storacha/upload-service/issues/1098))
([7d671bd](7d671bd))
* **main:** release capabilities 11.3.1
([storacha#1101](https://github.com/storacha/upload-service/issues/1101))
([20b5b35](20b5b35))
* **main:** release capabilities 11.4.0
([storacha#1105](https://github.com/storacha/upload-service/issues/1105))
([1b6210f](1b6210f))
* **main:** release capabilities 11.4.1
([storacha#1131](https://github.com/storacha/upload-service/issues/1131))
([a5b7154](a5b7154))
* **main:** release capabilities 12.0.0
([storacha#1137](https://github.com/storacha/upload-service/issues/1137))
([bb23e9f](bb23e9f))
* **main:** release capabilities 12.0.1
([storacha#1147](https://github.com/storacha/upload-service/issues/1147))
([d1a9c78](d1a9c78))
* **main:** release capabilities 12.0.2
([storacha#1152](https://github.com/storacha/upload-service/issues/1152))
([b9d7ff5](b9d7ff5))
* **main:** release capabilities 12.0.3
([storacha#1163](https://github.com/storacha/upload-service/issues/1163))
([ec5c385](ec5c385))
* **main:** release capabilities 12.1.0
([storacha#1195](https://github.com/storacha/upload-service/issues/1195))
([a21c1a5](a21c1a5))
* **main:** release capabilities 13.0.0
([storacha#1230](https://github.com/storacha/upload-service/issues/1230))
([3d5b3ef](3d5b3ef))
* **main:** release capabilities 13.1.0
([storacha#1257](https://github.com/storacha/upload-service/issues/1257))
([85adc9a](85adc9a))
* **main:** release capabilities 13.1.1
([storacha#1283](https://github.com/storacha/upload-service/issues/1283))
([31c38e9](31c38e9))
* **main:** release capabilities 13.2.0
([storacha#1315](https://github.com/storacha/upload-service/issues/1315))
([0505458](0505458))
* **main:** release capabilities 13.2.1
([storacha#1362](https://github.com/storacha/upload-service/issues/1362))
([26b5751](26b5751))
* **main:** release capabilities 13.3.0
([storacha#1366](https://github.com/storacha/upload-service/issues/1366))
([d6fbc4a](d6fbc4a))
* **main:** release capabilities 13.3.1
([storacha#1377](https://github.com/storacha/upload-service/issues/1377))
([149f592](149f592))
* **main:** release capabilities 14.0.0
([storacha#1386](https://github.com/storacha/upload-service/issues/1386))
([69bfc08](69bfc08))
* **main:** release capabilities 14.0.1
([storacha#1395](https://github.com/storacha/upload-service/issues/1395))
([a76c970](a76c970))
* **main:** release capabilities 14.0.2
([storacha#1400](https://github.com/storacha/upload-service/issues/1400))
([7b46852](7b46852))
* **main:** release capabilities 15.0.0
([storacha#1412](https://github.com/storacha/upload-service/issues/1412))
([ec90b81](ec90b81))
* **main:** release capabilities 16.0.0
([storacha#1419](https://github.com/storacha/upload-service/issues/1419))
([50e3934](50e3934))
* **main:** release capabilities 17.0.0
([storacha#1428](https://github.com/storacha/upload-service/issues/1428))
([6ee21fd](6ee21fd))
* **main:** release capabilities 17.1.0
([storacha#1447](https://github.com/storacha/upload-service/issues/1447))
([8ee1fd9](8ee1fd9))
* **main:** release capabilities 17.1.1
([storacha#1483](https://github.com/storacha/upload-service/issues/1483))
([5394ed5](5394ed5))
* **main:** release capabilities 17.2.0
([storacha#1494](https://github.com/storacha/upload-service/issues/1494))
([99876a5](99876a5))
* **main:** release capabilities 17.3.0
([storacha#1503](https://github.com/storacha/upload-service/issues/1503))
([891c2a5](891c2a5))
* **main:** release capabilities 17.4.0
([storacha#1559](https://github.com/storacha/upload-service/issues/1559))
([feaea7a](feaea7a))
* no longer depends on hd-scripts, packages use/configure eslint
directly, fixes warnings from npm lint script
([storacha#1058](https://github.com/storacha/upload-service/issues/1058))
([ebdb99b](ebdb99b))
* package renames
([0f797ed](0f797ed))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alanshaw alanshaw alanshaw approved these changes

@travis travis Awaiting requested review from travis

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@fforbeck @travis @alanshaw