Skip to content

Commit

Permalink
feat: added script to generate ucan for w3up space and docs for how t…
Browse files Browse the repository at this point in the history
…o access the space using console.web3.storage (#2554)

Context
* nft storage can now upload things to w3up spaces using w3up-client
* i wanted to be able to visualize the things that were being added now
that we have that running a bit on staging.nft.storage
* console.web3.storage is good for visualizing w3up spaces
* but it isn't trivial to authorize a console.web3.storage session to be
able to access the space, so I wanted to figure it out and document it
for others

What?
* this adds a command to the nft.storage api cli script that prompts the
user for some stuff that is in our configuration secrets vault, asks for
a DID of their console.web3.storage session, then builds a UCAN
delegation authorizing their console.web3.storage session to access the
space and saves it as a CAR file. The UCAN CAR file can then be imported
at https://console.web3.storage/space/import to access the space.
* adds docs to README

How to Test
* follow docs in README to try to authorize your console.web3.storage
session to be able to browse data added to the w3up space that
staging.nft.storage is configured to use
https://console.web3.storage/space/did:key:z6MkmRQnfDq1fpjnZJoXAKU6ScgirKSg97ZGRYT549jBnhXV
  • Loading branch information
gobengo authored Apr 3, 2024
1 parent 10fd471 commit ad33faf
Show file tree
Hide file tree
Showing 5 changed files with 373 additions and 5 deletions.
52 changes: 52 additions & 0 deletions packages/api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,3 +136,55 @@ We use [pickup](https://github.com/web3-storage/pickup) to fetch DAGs from IPFS
- `PICKUP_BASIC_AUTH_TOKEN` must be set as a secret in the env.

For local dev, we use a local ipfs-cluster container for the same service.

## w3up

Some uploads sent to nft.storage/api will be sent to up.web3.storage (aka 'w3up') for storage, serving on IPFS, and persistence to filecoin.

All uploads sent to w3up will be stored in the same web3.storage space configured by env var `W3_NFTSTORAGE_SPACE`.

### using console.web3.storage to browse uploads to w3up

You can use console.web3.storage to browse uploads in the W3_NFTSTORAGE_SPACE.

The DID used by your console.web3.storage session will need to be authorized to access the space.

The credentials used in staging/production are in the usual vault of secrets under 'w3up credentials'.

Run the cli command `w3up console ucan generate`

```shell
(
cd packages/api
node scripts/cli.js w3up console ucan generate
)
```

If you see a prompt like "ID of subject that should be authorized":

- you need to enter the ID of your console.web3.storage session. To get this, visit https://console.web3.storage/space/import. Look for "Send your DID to your friend". After that is a URI starting with `did:`. Copy that and enter it in the prompt.

If you see a prompt like "space recovery key mnemonic":

- look in the secrets vault for the mnemonic phrase labeled "Space Recovery Key". Copy that value into the prompt.

If you see a prompt like "What name do you want to appear in console.web3.storage when this space is
imported?":

- enter whatever name you want that will help you distinguish this space from other spaces listed in console.web3.storage.

If you see a prompt like "output ucan car to file /tmp/nftstorage-w3up-1712101390513.ucan.car?"

- hit enter or type 'Y' to confirm

Now a UCAN delegation has been written to a CAR file at a path like `/tmp/nftstorage-w3up-1712101390513.ucan.car`.

To add this delegation to console.web3.storage:

1. use a web browser to access https://console.web3.storage/space/import
2. Click 'Import UCAN'. This should open a file picker
3. Select the file generated from the last script, e.g. `/tmp/nftstorage-w3up-1712101390513.ucan.car`

You should see 'Added' and a list containing the space with the name you chose when generating the UCAN CAR. Click 'View' to view the contents of the Space.

After importing the space, it will also be listed in the space listing at https://console.web3.storage/.
2 changes: 2 additions & 0 deletions packages/api/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
"dev": "miniflare dist/worker.js --watch --debug --env ../../.env",
"dev:persist": "PERSIST_VOLUMES=true npm run dev",
"build": "scripts/cli.js build",
"w3up:console:ucan:generate": "scripts/cli.js w3up console ucan generate",
"pretest": "tsc",
"test": "./docker/run-with-dependencies.sh ./scripts/run-test.sh",
"db-types": "./scripts/cli.js db-types"
Expand Down Expand Up @@ -50,6 +51,7 @@
},
"devDependencies": {
"@cloudflare/workers-types": "^3.17.0",
"@inquirer/prompts": "^4.3.1",
"@miniflare/core": "^2.10.0",
"@sentry/cli": "^1.71.0",
"@sentry/webpack-plugin": "^1.16.0",
Expand Down
7 changes: 7 additions & 0 deletions packages/api/scripts/cli.js
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import {
import { dbSqlCmd } from './cmds/db-sql.js'
import { dbTypesCmd } from './cmds/db-types.js'
import { minioBucketCreateCmd, minioBucketRemoveCmd } from './cmds/minio.js'
import { requestW3upConsoleUcan } from './cmds/w3up-console-ucan-request.js'

const __dirname = path.dirname(fileURLToPath(import.meta.url))
const require = createRequire(__dirname)
Expand Down Expand Up @@ -136,4 +137,10 @@ prog
.describe('Remove a bucket, automatically removing all contents')
.action(minioBucketRemoveCmd)

.command('w3up console ucan generate')
.describe(
'request info to build and output a UCAN that can be imported into console.web3.storage to browse data nft.storage stores with w3up'
)
.action(requestW3upConsoleUcan)

prog.parse(process.argv)
150 changes: 150 additions & 0 deletions packages/api/scripts/cmds/w3up-console-ucan-request.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
import { input, confirm } from '@inquirer/prompts'
import { DID } from '@ucanto/core/schema'
import * as Space from '@web3-storage/access/space'
import * as fsp from 'node:fs/promises'
import { CarWriter } from '@ipld/car/writer'

export async function requestW3upConsoleUcan() {
console.log('requestW3upConsoleUcan')

/*
We're going to get everything we need to build a UCAN
that authorizes a session (e.g. a console.web3.storage agent) to
do w3up stuff in some space.
*/

// subject of the authorization,
// e.g. a DID of a console.web3.storage session
/** @type {`did:${string}:${string}` | undefined} */
let subjectId
{
// if env var W3UP_UCAN_SUBJECT is set,
// parse as DID and use as subjectId
let subjectIdFromEnv
if ((subjectIdFromEnv = process.env.W3UP_UCAN_SUBJECT)) {
const did = DID.read(subjectIdFromEnv).ok
if (!did)
throw new Error(
`failed to parse W3UP_UCAN_SUBJECT env var as DID: ${subjectIdFromEnv}`
)
if (
await confirm({
message: `use W3UP_UCAN_SUBJECT env var value as authorization subject? ${did}`,
})
) {
subjectId = did
} else {
console.warn('will not use value of W3UP_UCAN_SUBJECT env var')
}
} else {
const subjectIdFromPrompt = await input({
message:
'ID of subject that should be authorized. e.g. you can copy the DID of your console session from https://console.web3.storage/space/import',
})
const subjectIdDid = DID.read(subjectIdFromPrompt).ok
if (subjectIdDid) {
subjectId = subjectIdDid
} else {
throw new Error(`Failed to parse as DID: ${subjectIdFromPrompt}`)
}
}
}
// now we should have a subjectId
const subjectPrincipal = {
did() {
if (!subjectId) throw new Error(`failed to determine subject of ucan`)
return subjectId
},
}

/**
* Issuer of the authorization.
* e.g. ultimately the issuer must have authority rooted in the space itself,
* so this issuer may be the space ID.
* @type {Space.OwnedSpace | undefined}
*/
let space

// if W3UP_SPACE_RECOVERY is set and user confirms, build issuer of UCANs from it
let spaceRecoveryMnemonic
if (process.env.W3UP_SPACE_RECOVERY) {
if (
await confirm({
message: `issue ucan using space from env var W3UP_SPACE_RECOVERY?`,
})
) {
spaceRecoveryMnemonic = process.env.W3UP_SPACE_RECOVERY
} else {
console.warn(`will not use env var W3UP_SPACE_RECOVERY`)
}
}
// if no spaceRecoveryKey from env var, prompt for a recovery key
if (!spaceRecoveryMnemonic) {
spaceRecoveryMnemonic = await input({
message: 'space recovery key mnemonic',
})
}

/**
* name that should appear for this space in console.web3.storage.
* @type {string}
*/
let nameForSpaceInConsole = await input({
message: `What name do you want to appear in console.web3.storage when this space is imported? (e.g. 'staging.nft.storage NFTs')`,
default: `nftstorage-${Date.now()}`,
})

space = await Space.fromMnemonic(spaceRecoveryMnemonic, {
name: nameForSpaceInConsole,
})

// We now have a space object
if (!space)
throw new Error(`unable to build a space object from inputs: ${space}`)
console.warn('space', space.did())

// now let's have the space sign a UCAN that authorizes the subject
// to access the space
const authorizationForSubjectToAccessSpace = await space.createAuthorization(
subjectPrincipal
)

const exportedUcan = await toCarBlob(authorizationForSubjectToAccessSpace)

// we want to save to a file
const outputPath = `/tmp/nftstorage-w3up-${Date.now()}.ucan.car`
if (await confirm({ message: `output ucan car to file ${outputPath}?` })) {
await fsp.writeFile(outputPath, exportedUcan.stream())
console.warn(`wrote`, outputPath)
console.warn(
`When this delegation is imported into console.web3.storage, the space will be shown with the name "${nameForSpaceInConsole}"`
)
} else {
console.warn(`did not output ucan to file because no confirmation`)
}
}

/**
* given a UCAN delegation, return a Blob of the serialized delegation.
* It's serialized to a CAR file in a way where console.web3.storage import will accept it
* when imported via https://github.com/web3-storage/console/blob/main/src/share.tsx#L138
* @param {import('@ucanto/interface').Delegation} delegation
* @returns {Promise<Blob>}
*/
export async function toCarBlob(delegation) {
const { writer, out } = CarWriter.create()
for (const block of delegation.export()) {
// @ts-expect-error slight Block type mismatch
void writer.put(block)
}
void writer.close()

const carParts = []
for await (const chunk of out) {
carParts.push(chunk)
}
const car = new Blob(carParts, {
type: 'application/vnd.ipld.car',
})
return car
}
Loading

0 comments on commit ad33faf

Please sign in to comment.