Skip to content

Commit

Permalink
feat: improved subdomain-url handling (#211)
Browse files Browse the repository at this point in the history
* fix: subdomain-url handling

- removed implicit tests based on SubdomainLocalhostGatewayURL
- subdomain tests are now run only against origins defined via
  `--subdomain-url`
- DNSLink gateway tests are no longer tied to origin passed via
  `--subdomain-url`

* fix(wip): avoid mangling Host header handling

- splitting into subdomain and dnslink gateway variants
- towards avoiding running everything three times
- keep custom Host header when defined

* refactor: remove subdomains from dnslink fixtures

these concepts should not mix, and dnslinks should be testable
independent of --subdomain-url

i also added more useful export format for them, compatible with
IPNS_NS_MAP used by Boxo/Kubo/Rainbow

* refactor: handling Host header vs proxying

towards fixing underlying issue

* feat: make start-kubo-docker

all-in-one command for running local kubo with all fixtures in

* fix: correctly pass subdomain-url to go test

* docs: skip and run examples

* refactor: UnwrapSubdomainTests → proxy-gateway

This replaces magic helpers.UnwrapSubdomainTests with explicit
spec preset named proxy-gateway, which can be enabled/disabled
the same way as other presets.

While it was nice to run every test in both proxy modes,
it was extremely noisy and hard to debug once things went wrong.

By having explicit tests in *_proxy_test.go it is easy to skip them
where proxy functionality is not relevant.

Other:

- Send meaningful User-Agent
- Sent detached path in HTTP Connect proxy tunnel tests (instead of full
  URL – in tunnel mode we pretend we talk to remote server on the other
  end of tunnel, and not send full URL like it is done in plain proxy
  mode)

* docs: fix/improve docker examples

* refactor: require explicit gateway URLs

This is important UX change. We no longer ship with default URLs.

User has to provide explicit one, or the test suite will refuse to run.
This ensures misconfigurations and testing different gateway endpoint
than desired do not happen.

Explicit is better than implicit.
  • Loading branch information
lidel authored Jun 10, 2024
1 parent c0dafe1 commit 3e23e98
Show file tree
Hide file tree
Showing 28 changed files with 1,368 additions and 1,055 deletions.
8 changes: 2 additions & 6 deletions .github/workflows/test-kubo-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,11 +46,7 @@ jobs:
- name: Provision Kubo Gateway
run: |
# Import car files
cars=$(find ./fixtures -name '*.car')
for car in $cars
do
ipfs dag import --pin-roots=false --stats "$car"
done
find ./fixtures -name '*.car' -exec ipfs dag import --pin-roots=false {} \;
# Import ipns records
records=$(find ./fixtures -name '*.ipns-record')
Expand All @@ -63,7 +59,7 @@ jobs:
uses: ./gateway-conformance/.github/actions/test
with:
gateway-url: http://127.0.0.1:8080
subdomain-url: http://example.com
subdomain-url: http://example.com:8080
json: output.json
xml: output.xml
html: output.html
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
/gateway-conformance
/dnslinks.json
/*.ipns-record
extracted-fixtures

# Logs
logs
Expand Down Expand Up @@ -142,3 +143,4 @@ dist

# reports
reports
report.json
30 changes: 30 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,36 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.6.0] - 2024-05-27 ### Changed
- Gateway URL
- `--gateway-url` is no longer defaulting to predefined URL. User has to
provide it via CLI or `GATEWAY_URL` environment variable or the test suite
will refuse to start.
- This aims to ensure no confusion about which gateway endpoint is being
tested.
- Docs and examples use `--gateway-url http://127.0.0.1:8080` to ensure no
confusion with `localhost:8080` subdomain gateway feature in IPFS
implementations like Kubo.
- Subdomain URL and UX related to subdomain tests
- The `--subdomain-url` is no longer set by default.
- User has to provide the origin of the subdomain gateway via CLI or
`SUBDOMAIN_GATEWAY_URL` to be used during subdomain tests. This aims to
ensure no confusion about which domain name is being tested.
- Simplified the way `--subdomain-url` works. We no longer run implicit tests
against `http://localhost` in addition to the URL passed via
`--subdomain-url`. To test more than one domain, run test multiple times.
- DNSLink test fixtures changed
- DNSLink fixtures no longer depend on `--subdomain-url` and use unrelated
`*.example.org` domains instead.
- `gateway-conformance extract-fixtures` creates `dnslinks.IPFS_NS_MAP` with
content that can be directly set as `IPNS_NS_MAP` environment variable
supported by various implementations, incl.
[Kubo](https://github.com/ipfs/kubo/blob/master/docs/environment-variables.md#ipfs_ns_map)
and
[Rainbow](https://github.com/ipfs/rainbow/blob/main/docs/environment-variables.md#ipfs_ns_map).
- Docker
- The image can now be run under non-root user

## [0.5.2] - 2024-05-20
### Changed
- Fixed: relaxed dag-cbor error check ([#205](https://github.com/ipfs/gateway-conformance/pull/205))
Expand Down
6 changes: 5 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
FROM golang:1.20-alpine
WORKDIR /app
ENV GATEWAY_CONFORMANCE_HOME=/app
ENV GATEWAY_CONFORMANCE_HOME=/app \
GOCACHE=/go/cache

COPY ./go.mod ./go.sum ./
RUN go mod download
Expand All @@ -9,4 +10,7 @@ COPY . .
ARG VERSION=dev
RUN go build -ldflags="-X github.com/ipfs/gateway-conformance/tooling.Version=${VERSION}" -o ./gateway-conformance ./cmd/gateway-conformance

# Relaxed perms for cache dir to allow running under regular user
RUN mkdir -p $GOCACHE && chmod -R 777 $GOCACHE

ENTRYPOINT ["/app/gateway-conformance"]
23 changes: 20 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,15 +1,18 @@
GIT_COMMIT := $(shell git rev-parse --short HEAD)
DIRTY_SUFFIX := $(shell test -n "`git status --porcelain`" && echo "-dirty" || true)
CLI_VERSION := dev-$(GIT_COMMIT)$(DIRTY_SUFFIX)
KUBO_VERSION ?= latest
KUBO_DOCKER_NAME ?= kubo-$(KUBO_VERSION)-gateway-conformance

all: gateway-conformance

clean: clean-docker
rm -f ./gateway-conformance
rm -f *.ipns-record
rm -f fixtures.car
rm -f dnslinks.json
rm -f ./reports/*
rm -f dnslinks.*
rm -f dnslink*.yml
rm -rf ./reports/*

test-cargateway: provision-cargateway fixtures.car gateway-conformance
./gateway-conformance test --json reports/output.json --gateway-url http://127.0.0.1:8040 --specs -subdomain-gateway
Expand All @@ -22,13 +25,27 @@ test-kubo: provision-kubo gateway-conformance
./gateway-conformance test --json reports/output.json --gateway-url http://127.0.0.1:8080 --specs -subdomain-gateway

provision-cargateway: ./fixtures.car
# cd go-libipfs/examples/car && go install
car -c ./fixtures.car &

provision-kubo:
find ./fixtures -name '*.car' -exec ipfs dag import --stats --pin-roots=false {} \;
find ./fixtures -name '*.ipns-record' -exec sh -c 'ipfs routing put --allow-offline /ipns/$$(basename -s .ipns-record "{}" | cut -d'_' -f1) "{}"' \;

#start-kubo-docker: stop-kubo-docker gateway-conformance
# ./gateway-conformance extract-fixtures --dir=.temp/fixtures
# docker pull ipfs/kubo:$(KUBO_VERSION)
# docker run -d --rm --net=host --name $(KUBO_DOCKER_NAME) -v "$(shell realpath .temp/fixtures)":/fixtures -v kubo-config.example.sh:/container-init.d/001-config.sh ipfs/kubo:$(KUBO_VERSION) daemon --init --offline
# @until docker exec $(KUBO_DOCKER_NAME) ipfs --api=/ip4/127.0.0.1/tcp/5001 dag stat /ipfs/QmUNLLsPACCz1vLxQVkXqqLX5R1X345qqfHbsf67hvA3Nn >/dev/null 2>&1; do sleep 0.1; done
# docker exec $(KUBO_DOCKER_NAME) find /fixtures -name '*.car' -exec ipfs dag import --stats --pin-roots=false {} \;
# docker exec $(KUBO_DOCKER_NAME) find /fixtures -name '*.ipns-record' -exec sh -c 'ipfs routing put --allow-offline /ipns/$$(basename -s .ipns-record "{}" | cut -d'_' -f1) "{}"' \;
# TODO: provision Kubo config at Gateway.PublicGateways to have subdomain gateway on example.com and also enable inlining on localhost
# See: https://github.com/ipfs/kubo/blob/a07852a3f0294974b802923fb136885ad077384e/.github/workflows/gateway-conformance.yml#L22-L34
# (this is not as trivial as it sounds because Kubo does not apply config inrealtime, and a restart is required.)

stop-kubo-docker: clean
docker stop $(KUBO_DOCKER_NAME) || true
docker rm -f $(KUBO_DOCKER_NAME) || true

# tools
fixtures.car: gateway-conformance
./gateway-conformance extract-fixtures --merged=true --dir=.
Expand Down
59 changes: 33 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,12 @@
- [CLI](#cli)
- [Docker](#docker)
- [Github Action](#github-action)
- [Web Dashboard](#web-dashboard)
- [Commands](#commands)
- [Examples](#examples)
- [Releases](#releases)
- [Development](#development)
- [Test DSL Syntax](#test-dsl-syntax)
- [Web Dashboard](#web-dashboard)
- [License](#license)

<!-- /TOC -->
Expand Down Expand Up @@ -67,25 +67,32 @@ Two high level [commands](/docs/commands.md) exist:
```console
$ # Install the gateway-conformance binary
$ go install github.com/ipfs/gateway-conformance/cmd/gateway-conformance@latest
$ # run subdomain-gateway tests against endpoint at http://localhost:8080 output as JSON
$ gateway-conformance test --gateway-url http://localhost:8080 --json report.json --specs +subdomain-gateway,-path-gateway -- -timeout 30m

$ # skip path gateway tests, and run subdomain-gateway tests against endpoint at http://127.0.0.1:8080 and use *.ipfs.example.com subdomains, output as JSON
$ gateway-conformance test --gateway-url http://127.0.0.1:8080 --subdomain-url http://example.com:8080 --json report.json --specs +subdomain-gateway,-path-gateway -- -timeout 5m
```

If you are looking for TLDR, see [examples](/docs/examples.md).
> [!TIP]
> If want skip individual tests, or only run specific ones based on a regex, see [`/docs/examples`](/docs/examples.md).
### Docker

Prebuilt image at `ghcr.io/ipfs/gateway-conformance` can be used for both `test` and `extract-fixtures` commands:
The `gateway-conformace` requires golang runtime to be present to facilitate `go test`.
If you want to run it on a box without having to instal golang runtime, prebuilt image at `ghcr.io/ipfs/gateway-conformance` is provided.

It can be used for both `test` and `extract-fixtures` commands:

```console
$ # extract fixtures to ./fixtures directory
$ docker run -v "${PWD}:/workspace" -w "/workspace" ghcr.io/ipfs/gateway-conformance:vA.B.C extract-fixtures --directory fixtures --merged false
$ # extract fixtures to ./extracted-fixtures directory under the current user's permissions
$ docker run -u "$(id -u):$(id -g)" -v "$(pwd):/workspace" -w "/workspace" ghcr.io/ipfs/gateway-conformance:latest extract-fixtures --directory extracted-fixtures

$ # run subdomain-gateway tests against endpoint at http://localhost:8080
$ docker run --network host -v "${PWD}:/workspace" -w "/workspace" ghcr.io/ipfs/gateway-conformance:vA.B.C test --gateway-url http://localhost:8080 --json report.json --specs +subdomain-gateway,-path-gateway -- -timeout 30m
$ # skip path gateway tests, and run subdomain-gateway tests against endpoint at http://127.0.0.1:8080 and use *.ipfs.example.com subdomains, output as JSON
$ docker run --net=host -u "$(id -u):$(id -g)" -v "$(pwd):/workspace" -w "/workspace" ghcr.io/ipfs/gateway-conformance:latest test --gateway-url http://127.0.0.1:8080 --subdomain-url http://example.com:8080 --json report.json --specs +subdomain-gateway,-path-gateway -- -timeout 5m
```

**NOTE:** replace `vA.B.C` with a [semantic version](https://github.com/ipfs/gateway-conformance/releases) version you want to test against
> [!IMPORTANT]
> - for stable CI/CD, replace `latest` with a [semantic version](https://github.com/ipfs/gateway-conformance/releases) version you want to test against
> - `-u` ensures extracted fixtures and created report files can be read by your local user, make sure to adjust it to suit your use case
### Github Action

Expand All @@ -94,22 +101,9 @@ Common operations are possible via reusable GitHub actions:
- [`ipfs/gateway-conformance/.github/actions/extract-fixtures`](https://github.com/ipfs/gateway-conformance/blob/main/.github/actions/extract-fixtures/action.yml)

To learn how to integrate them in the CI of your project, see real world examples in:
- [`kubo/../gateway-conformance.yml`](https://github.com/ipfs/kubo/blob/master/.github/workflows/gateway-conformance.yml) (fixtures imported into tested node)
- [`boxo/../gateway-conformance.yml`](https://github.com/ipfs/boxo/blob/main/.github/workflows/gateway-conformance.yml) (fixtures imported into a sidecar kubo node that is peered with tested library)
- [`rainbow/../gateway-conformance.yml`](https://github.com/ipfs/rainbow/blob/main/.github/workflows/gateway-conformance.yml) (fixtures imported into a kubo node that acts as a delegated block backend)

### Web Dashboard

Conformance test suite output can be plain text or JSON, which in turn can be
represented as a web dashboard which aggregates results from many test runs and
renders them on a static website.

The Implementation Dashboard instance at
[conformance.ipfs.tech](https://conformance.ipfs.tech/) is a view that
showcases some of well known and complete implementations of IPFS Gateways
in the ecosystem.

Learn more at [`/docs/web-dashboard.md`](/docs/web-dashboard.md)
- [`kubo/../gateway-conformance.yml`](https://github.com/ipfs/kubo/blob/master/.github/workflows/gateway-conformance.yml) (fixtures imported into tested kubo node that exposes HTTP gateway feature)
- [`boxo/../gateway-conformance.yml`](https://github.com/ipfs/boxo/blob/main/.github/workflows/gateway-conformance.yml) (fixtures imported into a sidecar kubo node that is peered with small HTTP server used for testing `boxo/gateway` library)
- [`rainbow/../gateway-conformance.yml`](https://github.com/ipfs/rainbow/blob/main/.github/workflows/gateway-conformance.yml) (fixtures imported into a kubo node that acts as a remote block provider, than tested against different `boxo/gateway` backends)

## Commands

Expand Down Expand Up @@ -142,6 +136,19 @@ Interested in write a new test case?
Test cases are written in Domain Specific Language (DLS) based on Golang.
More details at [`/docs/test-dsl-syntax.md`](/docs/test-dsl-syntax.md)

### Web Dashboard

Conformance test suite output can be plain text or JSON, which in turn can be
represented as a web dashboard which aggregates results from many test runs and
renders them on a static website.

Experimental Implementation Dashboard instance at
[conformance.ipfs.tech](https://conformance.ipfs.tech/) is a view that
showcases some of well known and complete implementations of IPFS Gateways
in the ecosystem.

Learn more at [`/docs/web-dashboard.md`](/docs/web-dashboard.md)

## License

This project is dual-licensed under Apache 2.0 and MIT terms:
Expand Down
Loading

0 comments on commit 3e23e98

Please sign in to comment.