fix(ci): replace the retired MinIO image with SeaweedFS in the objectstore step - #943
Merged
Merged
Conversation
…store step MinIO retired its open-source distribution (the successor, AIStor, is commercial), and quay.io/minio/minio now fails to pull with "unauthorized", which breaks every CI run at the Start MinIO step. The objectstore contract tests are store-agnostic — Arc-side key rejections plus ordinary S3 write/read/list against a real server — so SeaweedFS (chrislusf/seaweedfs:4.47, pinned) stands in, serving S3 on the same host port with the same credentials via an identities file. The config file travels by docker create + docker cp instead of a bind mount, which a macOS /tmp symlink turns into an empty directory in the container. The bucket is created with the runner's preinstalled AWS CLI, retiring the quay.io/minio/mc image as well. One store behavior differs and one test encoded it: SeaweedFS, like Azure, treats a backslash as a path separator, so a raw PUT of "ba\d.parquet" lands under a folded key and the literal spelling names nothing — MinIO preserved it, and TestS3ListUnusablePartitionsTheBucket required it reported as unusable. The test now asserts the actual invariant: a spelling the store's own raw listing preserves must be reported unusable, and a spelling the store cannot hold leaves nothing unaddressable behind. HEAD cannot witness preservation (a folding store folds lookups too), so the witness is a raw signed ListObjectsV2. Verified locally against chrislusf/seaweedfs:4.47: the full objectstore suite passes, with the folding branch exercised by the backslash key and the reporting branch by the preserved .part key.
5 tasks done
xe-nvdk
added a commit
that referenced
this pull request
Sep 25, 2026
…ose and docs (#944) MinIO retired its open-source distribution (its images no longer pull; the successor, AIStor, is commercial). CI already moved to SeaweedFS in #943; this migrates everything else that deployed MinIO. Arc itself is untouched: it speaks S3, external MinIO keeps working, and storage.backend="minio" remains an accepted alias of s3. Helm (arc-enterprise) — breaking values change: - The minio block becomes seaweedfs (chrislusf/seaweedfs:4.47, pinned); credentials are accessKey/secretKey, rendered into the S3 identities file at container startup by the shell (not kubelet $(VAR) expansion, which would land secrets in argv), so existingSecret works without template-time secret reads. - The chart-managed Secret uses the same access-key/secret-key keys as an external-S3 secret, deleting the useMinioCredKeys special case. - values.schema.json rejects any leftover minio.* key outright, including keys merged back by helm upgrade --reuse-values, so the rename cannot be picked up silently. - SeaweedFS credentials guard only the S3 port while the single process opens master/volume/filer planes; the chart disables the HTTP data planes (-disableHttp, S3 path verified live) and ships a default-on NetworkPolicy pinning ingress to 8333 from the release's own pods. - Buckets are created on first authenticated write, so no init job: Arc's first flush creates the bucket. - Validation reordered so an IRSA-with-bundled misconfiguration gets the pointed message instead of a generic missing-credentials one. Compose (oss-s3, enterprise-shared): - SeaweedFS service with startup-rendered identities, /status healthcheck gating Arc, -disableHttp, only the authenticated S3 port published plus the master status port bound to loopback; the one-shot mc bucket-init container is gone. Verified live: write/flush/query round-trip with the parquet landing in the auto-created bucket. - smoke.sh stray-container guard and prose follow the service rename. Docs and scripts: root README, all four compose READMEs (including stale folder links), chart README and presets, NOTES, bug-report dropdown (SeaweedFS added, MinIO kept for external deployments), release-build marketing text, test_cluster_raft_env.sh (all three CI invocations verified green), debug-s3-structure.sh moved from mc to the AWS CLI. Release notes carry the change and the breaking upgrade note: existing bundled-MinIO data is not migrated — sync the bucket or stay external.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
quay.io/minio/miniopulls returnunauthorized— MinIO retired its open-source distribution (its successor, AIStor, is commercial). First hit was feat(compaction): per-database exclusion list (compaction.exclude_databases) #942; every run since Sep 21 would fail the same way.chrislusf/seaweedfs:4.47, pinned), serving S3 on the same host port 9000 with the sameminioadmincredentials via an identities file, soARC_TEST_S3_ENDPOINTand the tests' defaults are unchanged.docker create+docker cpinstead of a bind mount (a macOS/tmpsymlink turns the mount into an empty in-container directory; cp has no filesystem-sharing dependency on any runner).quay.io/minio/mcimage too. No MinIO-hosted image remains in CI.TestS3ListUnusablePartitionsTheBucketencoded a MinIO-specific behavior: MinIO preserved a raw-PUT backslash key, so the test demanded it be reported unusable. SeaweedFS, like Azure, folds\into a path separator (on lookup as well as PUT), so the literal spelling names nothing and nothing unaddressable exists. The test now asserts the real invariant — a spelling the store's own raw listing preserves must be reported unusable — using a raw signed ListObjectsV2 as the witness (HEAD cannot be one: folding stores fold lookups too). The preserved.partkey keeps the reporting branch enforced.Test plan
go test -tags="duckdb_arrow objectstore" ./internal/storage/green locally againstchrislusf/seaweedfs:4.47started exactly as the workflow does (create/cp/start, identities file, AWS-CLI bucket).partkey preserved and reported)gofmt/go vetclean