feat: w3 delegation create --base64 & w3 space add <base64> (storacha#158)

**export** a delegation as base64 encoded identity CID with `w3
delegation create --base64`

```shell
$ w3 delegation create did:key:z6MkviAsUfBwegmB57byQ7SZTFtX4jNjo315EvgurjWYoTRX --can 'store/add' --can 'upload/add' --base64
mAYIEAO0OEaJlcm9vdHOAZ3ZlcnNpb24BvQUBcRIgL7w/mAWPOV8Wt/B0ygdaOI+20/ZG8pX+5YzZ5X9U4y6oYXNYRO2hA0CbPDJlxyrorHHdNAUnRUDA4xU7KHgHHstkM8tBxq+6KaQP5xLCknOh9TjkR0S0yuK/fiFxKwRDUHfECFEWQn4DYXZlMC45LjFjYXR0hqJjY2FuZ3NwYWNlLypkd2l0aHg4ZGlkOmtleTp6Nk1raGZ6VHdaSjI4YVJvYkNwNzZ1WFJxenNqSDZHTnUxOFdGd011bWtBRjVvaleiY2NhbmdzdG9yZS8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5odXBsb2FkLypkd2l0aHg4ZGlkOmtleTp6Nk1raGZ6VHdaSjI4YVJvYkNwNzZ1WFJxenNqSDZHTnUxOFdGd011bWtBRjVvaleiY2NhbmhhY2Nlc3MvKmR3aXRoeDhkaWQ6a2V5Ono2TWtoZnpUd1pKMjhhUm9iQ3A3NnVYUnF6c2pINkdOdTE4V0Z3TXVta0FGNW9qV6JjY2FuamZpbGVjb2luLypkd2l0aHg4ZGlkOmtleTp6Nk1raGZ6VHdaSjI4YVJvYkNwNzZ1WFJxenNqSDZHTnUxOFdGd011bWtBRjVvaleiY2Nhbmd1c2FnZS8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXY2F1ZFgi7QHob+19JDMUBs+u1e646vN2MLovQUXA7xJeFs2THUcb+mNleHAaZzctGmNmY3SBoWVzcGFjZaFkbmFtZWV0b290c2Npc3NYIu0BL9X+p4Uyz05zSH0ol8TYPXpwU9EljNRo1O18uYbWlvljcHJmgL0FAXESIC+8P5gFjzlfFrfwdMoHWjiPttP2RvKV/uWM2eV/VOMuqGFzWETtoQNAmzwyZccq6Kxx3TQFJ0VAwOMVOyh4Bx7LZDPLQcavuimkD+cSwpJzofU45EdEtMriv34hcSsEQ1B3xAhRFkJ+A2F2ZTAuOS4xY2F0dIaiY2NhbmdzcGFjZS8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5nc3RvcmUvKmR3aXRoeDhkaWQ6a2V5Ono2TWtoZnpUd1pKMjhhUm9iQ3A3NnVYUnF6c2pINkdOdTE4V0Z3TXVta0FGNW9qV6JjY2FuaHVwbG9hZC8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5oYWNjZXNzLypkd2l0aHg4ZGlkOmtleTp6Nk1raGZ6VHdaSjI4YVJvYkNwNzZ1WFJxenNqSDZHTnUxOFdGd011bWtBRjVvaleiY2NhbmpmaWxlY29pbi8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5ndXNhZ2UvKmR3aXRoeDhkaWQ6a2V5Ono2TWtoZnpUd1pKMjhhUm9iQ3A3NnVYUnF6c2pINkdOdTE4V0Z3TXVta0FGNW9qV2NhdWRYIu0B6G/tfSQzFAbPrtXuuOrzdjC6L0FFwO8SXhbNkx1HG/pjZXhwGmc3LRpjZmN0gaFlc3BhY2WhZG5hbWVldG9vdHNjaXNzWCLtAS/V/qeFMs9Oc0h9KJfE2D16cFPRJYzUaNTtfLmG1pb5Y3ByZoDbAwFxEiB9iHpD1ttdKEQCvBZ8jJBD7Wqw1abOtYwNCKAKYALXMqhhc1hE7aEDQJ7U8I+a4Au/eb10r9T89weG/Nl2jccEUXHs8wq+i2tU0Iaik8KaKvovDqqE57JU8ZoY0JAzOBW7cMLGcV6/UwthdmUwLjkuMWNhdHSComNjYW5pc3RvcmUvYWRkZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5qdXBsb2FkL2FkZGR3aXRoeDhkaWQ6a2V5Ono2TWtoZnpUd1pKMjhhUm9iQ3A3NnVYUnF6c2pINkdOdTE4V0Z3TXVta0FGNW9qV2NhdWRYIu0B8YzaLs8NDe7oZt6rlpsW6iMh8XsoXZLvkHPqtftYRXZjZXhw9mNmY3SBoWVzcGFjZaFkbmFtZWV0b290c2Npc3NYIu0B6G/tfSQzFAbPrtXuuOrzdjC6L0FFwO8SXhbNkx1HG/pjcHJmgtgqWCUAAXESIC+8P5gFjzlfFrfwdMoHWjiPttP2RvKV/uWM2eV/VOMu2CpYJQABcRIgL7w/mAWPOV8Wt/B0ygdaOI+20/ZG8pX+5YzZ5X9U4y4
```

...yes, what if we put the CAR _in_ the CID! That way we can detect when
the input has been truncted, and the screed is self describing... you
can paste it into
[cid.ipfs.tech](https://cid.ipfs.tech/#mAYIEAO0OEaJlcm9vdHOAZ3ZlcnNpb24BvQUBcRIgL7w/mAWPOV8Wt/B0ygdaOI+20/ZG8pX+5YzZ5X9U4y6oYXNYRO2hA0CbPDJlxyrorHHdNAUnRUDA4xU7KHgHHstkM8tBxq+6KaQP5xLCknOh9TjkR0S0yuK/fiFxKwRDUHfECFEWQn4DYXZlMC45LjFjYXR0hqJjY2FuZ3NwYWNlLypkd2l0aHg4ZGlkOmtleTp6Nk1raGZ6VHdaSjI4YVJvYkNwNzZ1WFJxenNqSDZHTnUxOFdGd011bWtBRjVvaleiY2NhbmdzdG9yZS8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5odXBsb2FkLypkd2l0aHg4ZGlkOmtleTp6Nk1raGZ6VHdaSjI4YVJvYkNwNzZ1WFJxenNqSDZHTnUxOFdGd011bWtBRjVvaleiY2NhbmhhY2Nlc3MvKmR3aXRoeDhkaWQ6a2V5Ono2TWtoZnpUd1pKMjhhUm9iQ3A3NnVYUnF6c2pINkdOdTE4V0Z3TXVta0FGNW9qV6JjY2FuamZpbGVjb2luLypkd2l0aHg4ZGlkOmtleTp6Nk1raGZ6VHdaSjI4YVJvYkNwNzZ1WFJxenNqSDZHTnUxOFdGd011bWtBRjVvaleiY2Nhbmd1c2FnZS8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXY2F1ZFgi7QHob+19JDMUBs+u1e646vN2MLovQUXA7xJeFs2THUcb+mNleHAaZzctGmNmY3SBoWVzcGFjZaFkbmFtZWV0b290c2Npc3NYIu0BL9X+p4Uyz05zSH0ol8TYPXpwU9EljNRo1O18uYbWlvljcHJmgL0FAXESIC+8P5gFjzlfFrfwdMoHWjiPttP2RvKV/uWM2eV/VOMuqGFzWETtoQNAmzwyZccq6Kxx3TQFJ0VAwOMVOyh4Bx7LZDPLQcavuimkD+cSwpJzofU45EdEtMriv34hcSsEQ1B3xAhRFkJ+A2F2ZTAuOS4xY2F0dIaiY2NhbmdzcGFjZS8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5nc3RvcmUvKmR3aXRoeDhkaWQ6a2V5Ono2TWtoZnpUd1pKMjhhUm9iQ3A3NnVYUnF6c2pINkdOdTE4V0Z3TXVta0FGNW9qV6JjY2FuaHVwbG9hZC8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5oYWNjZXNzLypkd2l0aHg4ZGlkOmtleTp6Nk1raGZ6VHdaSjI4YVJvYkNwNzZ1WFJxenNqSDZHTnUxOFdGd011bWtBRjVvaleiY2NhbmpmaWxlY29pbi8qZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5ndXNhZ2UvKmR3aXRoeDhkaWQ6a2V5Ono2TWtoZnpUd1pKMjhhUm9iQ3A3NnVYUnF6c2pINkdOdTE4V0Z3TXVta0FGNW9qV2NhdWRYIu0B6G/tfSQzFAbPrtXuuOrzdjC6L0FFwO8SXhbNkx1HG/pjZXhwGmc3LRpjZmN0gaFlc3BhY2WhZG5hbWVldG9vdHNjaXNzWCLtAS/V/qeFMs9Oc0h9KJfE2D16cFPRJYzUaNTtfLmG1pb5Y3ByZoDbAwFxEiB9iHpD1ttdKEQCvBZ8jJBD7Wqw1abOtYwNCKAKYALXMqhhc1hE7aEDQJ7U8I+a4Au/eb10r9T89weG/Nl2jccEUXHs8wq+i2tU0Iaik8KaKvovDqqE57JU8ZoY0JAzOBW7cMLGcV6/UwthdmUwLjkuMWNhdHSComNjYW5pc3RvcmUvYWRkZHdpdGh4OGRpZDprZXk6ejZNa2hmelR3WkoyOGFSb2JDcDc2dVhScXpzakg2R051MThXRndNdW1rQUY1b2pXomNjYW5qdXBsb2FkL2FkZGR3aXRoeDhkaWQ6a2V5Ono2TWtoZnpUd1pKMjhhUm9iQ3A3NnVYUnF6c2pINkdOdTE4V0Z3TXVta0FGNW9qV2NhdWRYIu0B8YzaLs8NDe7oZt6rlpsW6iMh8XsoXZLvkHPqtftYRXZjZXhw9mNmY3SBoWVzcGFjZaFkbmFtZWV0b290c2Npc3NYIu0B6G/tfSQzFAbPrtXuuOrzdjC6L0FFwO8SXhbNkx1HG/pjcHJmgtgqWCUAAXESIC+8P5gFjzlfFrfwdMoHWjiPttP2RvKV/uWM2eV/VOMu2CpYJQABcRIgL7w/mAWPOV8Wt/B0ygdaOI+20/ZG8pX+5YzZ5X9U4y4)
and it'll tell you it's CAR flavour identity hashed bytes!

**import** a space from a stringified proof

```shell
$ w3 space add $PROOF
did:key:z6MkhfzTwZJ28aRobCp76uXRqzsjH6GNu18WFwMumkAF5ojW
```

see:
storacha#154 (comment)

## Usage

**on your machine**

- with `@web3-storage/w3cli` installed.
- Set the space you want CI to upload to as the current space

```shell
# create a key for ci
$ w3 key create --json > ci.json

# create a proof for that key. copy paste it to env var in CI
$ w3 delegation create $(jq -r .did ci.json) -c 'store/add' -c 'upload/add' --base64
mA...
```

**in CI**

- install `@web3-storage/w3cli`. No need to login, we pass it the proof
to use.
- copy paste the key we generated above `(jq -r .key ci.json)` as a
secret in your ci workflow. _KEEP IT SECRET!_
- copy paste the proof we generated above as a variable or secret in
your ci workflow. _it's ok to share this, only the holder of the secret
key can use it._

```shell
# set key (jq -r .key ci.json) in env so w3cli uses it instead of generating one
$ W3_PRINCIPAL=${{ secrets.w3key }}

# import the space from the stringified proof
$ w3 space add ${{ vars.w3proof }}

# upload yer stuff
$ w3 upload ./my/cool/stuff
```


License: MIT

---------

Signed-off-by: Oli Evans <oli@protocol.ai>

Author: olizilla

README.md

@@ -112,9 +112,11 @@ w3 open bafybeidluj5ub7okodgg5v6l4x3nytpivvcouuxgzuioa6vodg3xt2uqle/olizilla.png
 
 Print information about the current agent.
 
-### `w3 space add <proof.ucan>`
+### `w3 space add <proof>`
 
-Add a space to the agent. The proof is a CAR encoded delegation to _this_ agent.
+Add a space to the agent. The proof is a CAR encoded UCAN delegating capabilities over a space to _this_ agent.
+
+`proof` is a filesystem path to a CAR encoded UCAN, as generated by `w3 delegation create` _or_ a base64 identity CID string as created by `w3 delegation create --base64`.
 
 ### `w3 space create [name]`
 
@@ -144,17 +146,14 @@ Create a delegation to the passed audience for the given abilities with the _cur
 - `--name` Human readable name for the audience receiving the delegation.
 - `--type` Type of the audience receiving the delegation, one of: device, app, service.
 - `--output` Path of file to write the exported delegation data to.
+- `--base64` Format as base64 identity CID string. Useful when saving it as an environment variable.
 
 ```bash
-# delegate space/info to did:key:z6MkrwtRceSo2bE6vAY4gi8xPNfNszSpvf8MpAHnxVfMYreN
-w3 delegation create did:key:z6MkrwtRceSo2bE6vAY4gi8xPNfNszSpvf8MpAHnxVfMYreN --can space/info
-
-# delegate store/* and upload/* to did:key:z6MkrwtRceSo2bE6vAY4gi8xPNfNszSpvf8MpAHnxVfMYreN
-w3 delegation create did:key:z6MkrwtRceSo2bE6vAY4gi8xPNfNszSpvf8MpAHnxVfMYreN --can 'store/*' --can 'upload/*'
+# delegate space/info to did:key:z6M..., output as a CAR
+w3 delegation create did:key:z6M... --can space/info --output ./info.ucan
 
-# delegate all capabilities to did:key:z6MkrwtRceSo2bE6vAY4gi8xPNfNszSpvf8MpAHnxVfMYreN
-# WARNING - this is bad practice and should generally only be done in testing and development
-w3 delegation create did:key:z6MkrwtRceSo2bE6vAY4gi8xPNfNszSpvf8MpAHnxVfMYreN --can '*'
+# delegate store/* and upload/* to did:key:z6M..., output as a string
+w3 delegation create did:key:z6M... --can 'store/*' --can 'upload/*' --base64
 ```
 
 ### `w3 delegation ls`

bin.js

@@ -142,7 +142,7 @@ cli
 cli
   .command('space add <proof>')
   .describe(
-    'Add a space to the agent. The proof is a CAR encoded delegation to _this_ agent.'
+    'Import a space from a proof: a CAR encoded UCAN delegating capabilities to this agent. proof is a filesystem path, or a base64 encoded cid string.'
   )
   .action(addSpace)
 
@@ -181,7 +181,7 @@ cli
 cli
   .command('delegation create <audience-did>')
   .describe(
-    'Create a delegation to the passed audience for the given abilities with the _current_ space as the resource.'
+    'Output a CAR encoded UCAN that delegates capabilities to the audience for the current space.'
   )
   .option('-c, --can', 'One or more abilities to delegate.')
   .option(
@@ -201,6 +201,10 @@ cli
     '-o, --output',
     'Path of file to write the exported delegation data to.'
   )
+  .option(
+    '--base64',
+    'Format as base64 identity CID string. Useful when saving it as an environment variable.'
+  )
   .action(createDelegation)
 
 cli

index.js

@@ -1,7 +1,9 @@
 import fs from 'fs'
 import ora, { oraPromise } from 'ora'
-import { Readable } from 'stream'
+import { pipeline } from 'node:stream/promises'
 import { CID } from 'multiformats/cid'
+import { base64 } from 'multiformats/bases/base64'
+import { identity } from 'multiformats/hashes/identity'
 import * as DID from '@ipld/dag-ucan/did'
 import * as dagJSON from '@ipld/dag-json'
 import { CarWriter } from '@ipld/car'
@@ -15,6 +17,7 @@ import {
   filesize,
   filesizeMB,
   readProof,
+  readProofFromBytes,
   uploadListResponseToString,
   startOfLastMonth,
 } from './lib.js'
@@ -253,11 +256,33 @@ export async function createSpace(name) {
 }
 
 /**
- * @param {string} proofPath
+ * @param {string} proofPathOrCid
  */
-export async function addSpace(proofPath) {
+export async function addSpace(proofPathOrCid) {
   const client = await getClient()
-  const delegation = await readProof(proofPath)
+
+  let cid
+  try {
+    cid = CID.parse(proofPathOrCid, base64)
+  } catch (/** @type {any} */ err) {
+    if (err?.message?.includes('Unexpected end of data')) {
+      console.error(`Error: failed to read proof. The string has been truncated.`)
+      process.exit(1)
+    }
+    /* otherwise, try as path */
+  }
+
+  let delegation
+  if (cid) {
+    if (cid.multihash.code !== identity.code) {
+      console.error(`Error: failed to read proof. Must be identity CID. Fetching of remote proof CARs not supported by this command yet`)
+      process.exit(1)
+    }
+    delegation = await readProofFromBytes(cid.multihash.digest)
+  } else {
+    delegation = await readProof(proofPathOrCid)
+  }
+
   const space = await client.addSpace(delegation)
   console.log(space.did())
 }
@@ -339,6 +364,7 @@ Providers: ${providers || chalk.dim('none')}`)
  * @param {number} [opts.expiration]
  * @param {string} [opts.output]
  * @param {string} [opts.with]
+ * @param {boolean} [opts.base64]
  */
 export async function createDelegation(audienceDID, opts) {
   const client = await getClient()
@@ -367,7 +393,25 @@ export async function createDelegation(audienceDID, opts) {
   const { writer, out } = CarWriter.create()
   const dest = opts.output ? fs.createWriteStream(opts.output) : process.stdout
 
-  Readable.from(out).pipe(dest)
+  pipeline(
+    out,
+    async function* maybeBaseEncode(src) {
+      const chunks = []
+      for await (const chunk of src) {
+        if (!opts.base64) {
+          yield chunk
+        } else {
+          chunks.push(chunk)
+        }
+      }
+      if (!opts.base64) return
+      const blob = new Blob(chunks)
+      const bytes = new Uint8Array(await blob.arrayBuffer())
+      const idCid = CID.createV1(ucanto.CAR.code, identity.digest(bytes))
+      yield idCid.toString(base64)
+    },
+    dest
+  )
 
   for (const block of delegation.export()) {
     // @ts-expect-error

lib.js

@@ -139,24 +139,31 @@ export function getClient() {
  * @param {string} path Path to the proof file.
  */
 export async function readProof(path) {
+  let bytes
   try {
-    await fs.promises.access(path, fs.constants.R_OK)
+    const buff = await fs.promises.readFile(path)
+    bytes = new Uint8Array(buff.buffer)
   } catch (/** @type {any} */ err) {
     console.error(`Error: failed to read proof: ${err.message}`)
     process.exit(1)
   }
+  return readProofFromBytes(bytes)
+}
 
+/**
+ * @param {Uint8Array} bytes Path to the proof file.
+ */
+export async function readProofFromBytes(bytes) {
   const blocks = []
   try {
-    const reader = await CarReader.fromIterable(fs.createReadStream(path))
+    const reader = await CarReader.fromBytes(bytes)
     for await (const block of reader.blocks()) {
       blocks.push(block)
     }
   } catch (/** @type {any} */ err) {
     console.error(`Error: failed to parse proof: ${err.message}`)
     process.exit(1)
   }
-
   try {
     // @ts-expect-error
     return importDAG(blocks)

test/bin.spec.js

@@ -18,6 +18,7 @@ import { UCAN, Provider } from '@web3-storage/capabilities'
 import * as ED25519 from '@ucanto/principal/ed25519'
 import { sha256, delegate } from '@ucanto/core'
 import * as Result from '@web3-storage/w3up-client/result'
+import { base64 } from 'multiformats/bases/base64'
 
 const w3 = Command.create('./bin.js')
 
@@ -349,6 +350,34 @@ export const testSpace = {
     assert.ok(listSome.output.includes(spaceDID))
   }),
 
+  'w3 space add `base64 proof car`': test(async (assert, context) => {
+    const { env } = context
+    const spaceDID = await loginAndCreateSpace(context, { env: env.alice })
+    const whosBob = await w3.args(['whoami']).env(env.bob).join()
+    const bobDID = SpaceDID.from(whosBob.output.trim())
+    const res = await w3
+      .args([
+        'delegation',
+        'create',
+        bobDID,
+        '-c',
+        'store/*',
+        'upload/*',
+        '--base64'
+      ])
+      .env(env.alice)
+      .join()
+
+    const listNone = await w3.args(['space', 'ls']).env(env.bob).join()
+    assert.ok(!listNone.output.includes(spaceDID))
+
+    const add = await w3.args(['space', 'add', res.output]).env(env.bob).join()
+    assert.equal(add.output.trim(), spaceDID)
+
+    const listSome = await w3.args(['space', 'ls']).env(env.bob).join()
+    assert.ok(listSome.output.includes(spaceDID))
+  }),
+
   'w3 space add invalid/path': test(async (assert, context) => {
     const fail = await w3
       .args(['space', 'add', 'djcvbii'])
@@ -784,6 +813,44 @@ export const testDelegation = {
     assert.equal(delegate.status.success(), true)
   }),
 
+  'w3 delegation create -c store/add -c upload/add --base64': test(
+    async (assert, context) => {
+      const env = context.env.alice
+      const { bob } = Test
+      const spaceDID = await loginAndCreateSpace(context)
+      const res = await w3
+        .args([
+          'delegation',
+          'create',
+          bob.did(),
+          '-c',
+          'store/add',
+          '-c',
+          'upload/add',
+          '--base64'
+        ])
+        .env(env)
+        .join()
+
+      assert.equal(res.status.success(), true)
+
+      const identityCid = parseLink(res.output, base64)
+      const reader = await CarReader.fromBytes(identityCid.multihash.digest)
+      const blocks = []
+      for await (const block of reader.blocks()) {
+        blocks.push(block)
+      }
+
+      // @ts-expect-error
+      const delegation = importDAG(blocks)
+      assert.equal(delegation.audience.did(), bob.did())
+      assert.equal(delegation.capabilities[0].can, 'store/add')
+      assert.equal(delegation.capabilities[0].with, spaceDID)
+      assert.equal(delegation.capabilities[1].can, 'upload/add')
+      assert.equal(delegation.capabilities[1].with, spaceDID)
+    }
+  ),
+
   'w3 delegation ls --json': test(async (assert, context) => {
     const { mallory } = Test