Skip to content

Commit

Permalink
ipip-402: final test fixtures section
Browse files Browse the repository at this point in the history 
This backports CIDs and CARs from gateway conformance to the IPIP-402,
and provides some basic hints on how each fixture should be used.
  • Loading branch information
@lidel
lidel committed Jul 4, 2023
1 parent f5f33ff commit 4ebbb96
Showing 1 changed file with 73 additions and 54 deletions.
127 changes: 73 additions & 54 deletions src/ipips/ipip-0402.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -138,11 +138,6 @@ Gateways ignore unknown URL parameters. A client sending them to a
gateway that does not implement this IPIP will get all blocks for the requested
DAG.

<!-- TODO
mention feature detection via OPTIONS -- a separate IPIP?
OR suggest executing test request and client-side detection the first time a gateway is used.
-->

#### CAR roots and determinism

As of 2023-06-20, the behavior of the `roots` CAR field remains an [unresolved item within the CARv1 specification](https://web.archive.org/web/20230328013837/https://ipld.io/specs/transport/car/carv1/#unresolved-items):
Expand Down Expand Up @@ -235,9 +230,14 @@ single request.

## Test fixtures

<!-- TODO: describe what and how should be tested
- [ ] add relevant tests to https://github.com/ipfs/gateway-conformance → and reference fixtures here
-->
Relevant tests were added to
[gateway-conformance](https://github.com/ipfs/gateway-conformance) test suite
in [#56](https://github.com/ipfs/gateway-conformance/pull/56) and
[#85](https://github.com/ipfs/gateway-conformance/issues/85).
Detailed list of compliance checks for `dag-scope` and `entity-bytes` can be found in
[`v0.2.0/trustless_gateway_car_test.go`](https://github.com/ipfs/gateway-conformance/blob/v0.2.0/tests/trustless_gateway_car_test.go) or later.

Below are CIDs, CARs, and short summary of each fixture.

### Testing pathing

Expand All @@ -249,15 +249,24 @@ returned CAR MUST include both the block with the file data and all blocks
necessary for traversing from the root CID to the terminating element (all
parents, root CID and a subdirectory below it).

Fixtures:

:::example

- TODO(gateway-conformance): `/ipfs/dag-pb-cid/parent/file?format=car` (UnixFS file in a subdirectory)
Sample fixtures:

- TODO(gateway-conformance): `/ipfs/dag-pb-cid/hamt-parent1/file?format=car` (UnixFS file on a path within HAMT-sharded parent directory)
- `bafybeietjm63oynimmv5yyqay33nui4y4wx6u3peezwetxgiwvfmelutzu`
from [`subdir-with-two-single-block-files.car`](https://github.com/ipfs/gateway-conformance/raw/v0.2.0/fixtures/trustless_gateway_car/subdir-with-two-single-block-files.car)
for testing `/ipfs/dag-pb-cid/subdir/ascii.txt?format=car`
(UnixFS file in a subdirectory)

- TODO(gateway-conformance): `/ipfs/dag-cbor-cid/file?format=car` (UnixFS file on a path with DAG-CBOR root CID)
- `bafybeidbclfqleg2uojchspzd4bob56dqetqjsj27gy2cq3klkkgxtpn4i`
from [`single-layer-hamt-with-multi-block-files.car`](https://github.com/ipfs/gateway-conformance/raw/v0.2.0/fixtures/trustless_gateway_car/single-layer-hamt-with-multi-block-files.car)
for testing `/ipfs/dag-pb-hamt-cid/686.txt?format=car`
(UnixFS file on a path within HAMT-sharded parent directory)

- `bafybeia264q44a3kmfc2otctzu4egp2k235o3t7mslz2yjraymp4nv6asi`
from [`dir-with-dag-cbor-with-links.car`](https://github.com/ipfs/gateway-conformance/raw/v0.2.0/fixtures/trustless_gateway_car/dir-with-dag-cbor-with-links.car)
for testing `/ipfs/dag-cbor-cid/document?format=car`
(UnixFS file on a path with DAG-CBOR root CID)

:::

Expand All @@ -271,13 +280,23 @@ To test real world use, request UnixFS `file` or a `directory` from a sub-path.
The returned CAR MUST include blocks required for path traversal and ONLY the
root block of the terminating entity.

Fixtures:

:::example

- TODO(gateway-conformance): `/ipfs/cid/parent/directory?format=car&dag-scope=block` (UnixFS directory on a path)
Sample fixtures:

- `bafybeietjm63oynimmv5yyqay33nui4y4wx6u3peezwetxgiwvfmelutzu`
from [`subdir-with-two-single-block-files.car`](https://github.com/ipfs/gateway-conformance/raw/v0.2.0/fixtures/trustless_gateway_car/subdir-with-two-single-block-files.car)
for testing
- `/ipfs/dag-pb-cid/subdir/ascii.txt?format=car&dag-scope=block` (UnixFS file in a subdirectory)
- `/ipfs/dag-pb-cid?format=car&dag-scope=block` (UnixFS directory)

- TODO(gateway-conformance): `/ipfs/cid/parent1/parent2/file?format=car&dag-scope=block` (UnixFS file on a path)
- `bafybeidbclfqleg2uojchspzd4bob56dqetqjsj27gy2cq3klkkgxtpn4i`
from [`single-layer-hamt-with-multi-block-files.car`](https://github.com/ipfs/gateway-conformance/raw/v0.2.0/fixtures/trustless_gateway_car/single-layer-hamt-with-multi-block-files.car)
for testing:
- `/ipfs/dag-pb-hamt-cid/1.txt?format=car&dag-scope=block`
(UnixFS multi-block file on a path within HAMT-sharded parent directory)
- `/ipfs/dag-pb-hamt-cid?format=car&dag-scope=block`
(UnixFS HAMT-sharded directory)

:::

Expand All @@ -296,48 +315,45 @@ IPLD entity. Currently, the most popular entity types are:
- `raw` / `dag-cbor`
(block with raw data or DAG-CBOR document, potentially linking to other CIDs)

Fixtures:

:::example

- TODO(gateway-conformance): `/ipfs/cid/chunked-dag-pb-file?format=car&dag-scope=entity`
- Request a `chunked-dag-pb-file` (UnixFS file encoded with `dag-pb` with
more than one chunk). Returned blocks MUST be enough to deserialize the file.
Sample fixtures:

- TODO(gateway-conformance): `/ipfs/cid/dag-cbor-with-link?format=car&dag-scope=entity`
- Request a `dag-cbor-with-link` (DAG-CBOR document with CBOR Tag 42 pointing
at a third-party CID). The response MUST include the terminating entity (DAG-CBOR)
and MUST NOT include the CID from the Tag 42 (IPLD Link).
- `bafybeidh6k2vzukelqtrjsmd4p52cpmltd2ufqrdtdg6yigi73in672fwu`
from [`subdir-with-mixed-block-files.car`](https://github.com/ipfs/gateway-conformance/raw/v0.2.0/fixtures/trustless_gateway_car/subdir-with-mixed-block-files.car)
for testing:
- `/ipfs/dag-pb-cid/subdir/multiblock.txt?format=car&dag-scope=entity` (UnixFS multi-block file in a subdirectory)
- `/ipfs/dag-pb-cid/subdir?format=car&dag-scope=entity` (UnixFS directory)

- TODO(gateway-conformance): `/ipfs/cid/flat-directory/file?format=car&dag-scope=entity`
- Request UnixFS `flat-directory`. The response MUST include the minimal set of
blocks required for enumeration of directory contents, and no blocks that
belong to child entities.
- `bafybeidbclfqleg2uojchspzd4bob56dqetqjsj27gy2cq3klkkgxtpn4i`
from [`single-layer-hamt-with-multi-block-files.car`](https://github.com/ipfs/gateway-conformance/raw/v0.2.0/fixtures/trustless_gateway_car/single-layer-hamt-with-multi-block-files.car)
for testing:
- `/ipfs/dag-pb-hamt-cid/1.txt?format=car&dag-scope=entity`
(UnixFS multi-block file on a path within HAMT-sharded parent directory, returned blocks MUST be enough to deserialize the file)
- `/ipfs/dag-pb-hamt-cid?format=car&dag-scope=entity`
(UnixFS HAMT-sharded directory, response MUST include the minimal set of blocks required for enumeration of directory contents, and no blocks that belong to child entities)

- TODO(gateway-conformance): `/ipfs/cid/hamt-directory/file?format=car&dag-scope=entity`
- Request UnixFS `hamt-directory`. The response MUST include the minimal set of
blocks required for enumeration of directory contents, and no blocks that
belong to child entities.
- `bafybeia264q44a3kmfc2otctzu4egp2k235o3t7mslz2yjraymp4nv6asi`
from [`dir-with-dag-cbor-with-links.car`](https://github.com/ipfs/gateway-conformance/raw/v0.2.0/fixtures/trustless_gateway_car/dir-with-dag-cbor-with-links.car)
for testing `/ipfs/dag-cbor-cid/document?format=car&dag-scope=entity`
(DAG-CBOR document with IPLD Links must return all necessary blocks to verify the path, the document itself, but not the content behind any of the child entity IPLD Links)

:::

### Testing `dag-scope=all`

This is the implicit default used when `dag-scope` is not present,
and explicitly used in the context of proxy gateway supporting :cite[ipip-0288].

Fixtures:
and used in the context of deserialized UnixFS TAR responses from :cite[ipip-0288].

:::example

- TODO(gateway-conformance): `/ipfs/cid-of-a-directory?format=car&dag-scope=all`
- Request a CID of UnixFS `directory` which contains two files. The response MUST
contain all blocks that can be accessed by recursively traversing all IPLD
Links from the root CID.
Sample fixtures:

- TODO(gateway-conformance): `/ipfs/cid/chunked-dag-pb-file?format=car&dag-scope=all`
- Request a CID of UnixFS `file` encoded with `dag-pb` codec and more than
one chunk. The response MUST contain blocks for all `file` chunks.
- `bafybeidh6k2vzukelqtrjsmd4p52cpmltd2ufqrdtdg6yigi73in672fwu`
from [`subdir-with-mixed-block-files.car`](https://github.com/ipfs/gateway-conformance/raw/v0.2.0/fixtures/trustless_gateway_car/subdir-with-mixed-block-files.car)
for testing:
- `/ipfs/dag-pb-cid/subdir?format=car&dag-scope=all` (path parents and the entire UnixFS subdirectory, returned recursively)
- `/ipfs/dag-pb-cid/subdir/multiblock.txt?format=car&dag-scope=all` (path parents and the entire UnixFS multi-block file)

:::

Expand All @@ -358,16 +374,19 @@ Use of the below fixture is highly recommended:

:::example

- TODO(gateway-conformance): `/ipfs/dag-pb-file?format=car&entity-bytes=40000000000-40000000002`

- Request a byte range from the middle of a big UnixFS `file`. The response MUST
contain only the minimal set of blocks necessary for fullfilling the range
request.

- TODO(gateway-conformance): `/ipfs/10-bytes-cid?format=car&entity-bytes=4:-2`

- Request a byte range from the middle of a small file, to -2 bytes from the end.
- (TODO confirm we want keep this -- added since it was explicitly stated as a supported thing in path-gateway.md)
- `bafybeidh6k2vzukelqtrjsmd4p52cpmltd2ufqrdtdg6yigi73in672fwu`
from [`subdir-with-mixed-block-files.car`](https://github.com/ipfs/gateway-conformance/raw/v0.2.0/fixtures/trustless_gateway_car/subdir-with-mixed-block-files.car)
for testing:
- `/ipfs/dag-pb-cid/subdir/multiblock.txt?format=car&dag-scope=entity&entity-bytes=0:*` (path blocks and all the blocks for the multi-block UnixFS file)
- `multiblock.txt?format=car&dag-scope=entity&entity-bytes=512:1023` (path blocks and all the blocks for the the range request within multi-block UnixFS file)
- `multiblock.txt?format=car&dag-scope=entity&entity-bytes=512:-256` (path blocks and all the blocks for the the range request within multi-block UnixFS file)
- `/ipfs/dag-pb-cid/subdir?format=car&dag-scope=entity&entity-bytes=0:*` (path blocks and all the blocks to enumerate UnixFS directory)

- `QmYhmPjhFjYFyaoiuNzYv8WGavpSRDwdHWe5B4M5du5Rtk`
from [`file-3k-and-3-blocks-missing-block.car`](https://github.com/ipfs/gateway-conformance/raw/v0.2.0/fixtures/trustless_gateway_car/file-3k-and-3-blocks-missing-block.car)
for testing:
- `/ipfs/dag-pb-cid?format=car&dag-scope=entity&entity-bytes=0:1000` (only the blocks needed to fullfill the request, MUST succeed despite the fact that a block after the range is not retrievable)
- `/ipfs/dag-pb-cid?format=car&dag-scope=entity&entity-bytes=2200:*` (only the blocks needed to fullfill the request, MUST succeed despite the fact that a block before the range is not retrievable)

:::

Expand Down

0 comments on commit 4ebbb96

Please sign in to comment.