Tags: apple/swift-container-plugin
Tags
ContainerRegistry: Reject invalid image tags and digests (#142) Motivation ---------- Cherry pick of #140. `ImageReference` does not check for illegal characters in parsed image digests and tags. This means that `containertool` will send illegal image names to the registry. The registry will reject them, but the error message might not explain why, so a generic error message will be printed. Runtimes reject illegal image references immediately, without sending them to the registry. Some desktop runtimes accept local image names which the registry will reject; other runtimes reject these names even for local names. `containertool` now also rejects them. Modifications ------------- * Check validity of tags and digests when parsing image names * Change the low-level API functions to accept `Digest` or `Reference` instead of `String`. Result ------ It is impossible to create a `Repository` object containing a malformed tag or digest, because the constructor checks the string value. It is impossible to send a malformed name to the registry because the API wrappers only accept `Digest` or `Reference (Digest | Tag)` objects. Fixes #139 Test Plan --------- Existing tests continue to pass. New tests exercise additional checks which were previously missing. Removed tests which checked tags which seemed to be accepted by some desktop runtimes, but which were not accepted by registries.
containertool: Correct help text for --username and --password Motivation ---------- The help text for `--username` and `--password` states that these options override `.netrc`. In fact they are used as defaults if no suitable entry is found in `.netrc`. Modifications ------------- Correct the option descriptions in the manual page. Result ------ Documentation accurately describes what the options do. Test Plan --------- All existing tests continue to pass. No functional change.
docs: Add reference documentation for plugin (#109) Motivation ---------- Reference documentation, such as manual pages, complements getting started guides and tutorials by explaining a tool's interface and options in one place. The `build-container-image` command plugin is the main user-facing tool in the project, so users will benefit from having reference documentation for it, available from the main documentation site. At present, `containertool` will mostly be used through the plugin, which is a thin wrapper which calls `containertool` as a helper. Documenting `containertool` as a separate tool might be confusing. It has command line help for those who need to use it directly, and as the plugin is largely a wrapper its reference documentation also explains most of `containertool`'s options. Modifications ------------- * Rewrite the `containertool` manual page for the `build-container-image` plugin. * Add a Reference section to the curation, linking to the plugin manual page. Result ------ Users have access to reference documentation which explains the interface and options of the `build-container-image` command plugin. Test Plan --------- All existing tests continue to pass.
Examples: Adopt structure used in the Vapor template repository (#85) Motivation ---------- The [Vapor template repository](https://github.com/vapor/template-bare) splits the server into a main entry point file and a separate route installation function. Adopting the same structure makes this example more familiar to users who are used to the official template. Fixes #61 Modifications ------------- * Adopt the Vapor template structure for `HelloWorldVapor` * Use the `-warnings-as-errors` flag when building end to end examples. Result ------ The example works in the same way as before but has a more familiar structure for Vapor users. Test Plan --------- All tests continue to pass.
plugin: Add resource bundles defined in Package.swift to container im… …ages (#78) Motivation ---------- A target in `Package.swift` can include [resources](https://developer.apple.com/documentation/packagedescription/target/resources). When the target is built, the resources are copied into a bundle directory and a [`Bundle`](https://developer.apple.com/documentation/foundation/bundle) class is generated, giving the executable a convenient way to retrieve them at run time. A target with resources should be able to access them when it is packaged in a container image. For example, a web server might bundle resources such as images or fonts. Fixes #48 Modifications ------------- * Add a new `--resource` parameter to `containertool`. This specifies a resource bundle directory which is added to the container image using the directory archiving support added to the `Tar` module in #74. When `containertool` is used alone, multiple `--resource` flags can be specified and will be added to the image in the order in which they appear on the command line. A target in `Package.swift` can only define one resource bundle, so when used through the plugin only one bundle will be added. * Update the plugin to pass the `--resource` flag to `containertool` when an executable target has resources. Result ------ If a target includes resources, the resource bundle will be copied into the container image at a path where the generated `Bundle` class can find it. If a target does not include resources, the container image will only include the executable. Test Plan --------- * Existing tests continue to pass. * New integration tests verify that all expected resources are present.
plugin: handle chunks of output correctly (#77) Motivation ---------- `containertool` produces its normal output in single writes of less than PIPE_BUF bytes, which should be delivered atomically through the pipe to the plugin: > Reading or writing pipe data is atomic if the size of data written is not greater than PIPE_BUF. This means that the data transfer seems to be an instantaneous unit, in that nothing else in the system can observe a state in which it is partially complete. Atomic I/O may not begin right away (it may need to wait for buffer space or for data), but once it does begin it finishes immediately. https://www.gnu.org/software/libc/manual/html_node/Pipe-Atomicity.html Progress messages written in this way are displayed correctly. Unfortunately error output from `swift-argument-parser` is delivered in smaller chunks which are often broken across several lines by the plugin: ``` % swift package --allow-network-connections all build-container-image Building for debugging... [1/1] Write swift-version-2380AA06D3543E1B.txt Build of product 'containertool' complete! (1.74s) Building for debugging... [0/3] Write swift-version-2380AA06D3543E1B.txt Build of product 'hello-world' complete! (2.04s) [ContainerImageBuilder] itory <repository>' Help: --repository < [ContainerImageBuilder] repository> Repository path [ContainerImageBuilder] Usage: containertoo [ContainerImageBuilder] l [<optio [ContainerImageBuilder] ns>] --re [ContainerImageBuilder] posito [ContainerImageBuilder] ry <rep error: Missing expected argument '--repos [ContainerImageBuilder] ository [ContainerImageBuilder] > <ex [ContainerImageBuilder] ecutable> Se [ContainerImageBuilder] e 'containertool --help' for more inf [ContainerImageBuilder] ormation [ContainerImageBuilder] . ``` At one point the plugin reassembled output from the pipe and re-split it on newlines, but the code triggered strict concurrency warnings with the Swift 6 language mode so it was removed in favour of `PIPE_BUF` write atomicity. This commit restores the pipe output reassembly. Modifications ------------- * Plugin reassembles output from `containertool` and splits it on newlines * All output after an error is now printed using `Diagnostics.error`. Otherwise error and progress output would be interleaved. Result ------ Error output from `containertool` will be presented correctly by the plugin. Test Plan --------- Existing tests continue to pass. A new integration test checks that error output is not broken into multiple lines.
containertool: Add basic ELF file type detection (#63) Motivation ---------- The container's architecture metadata field should match the architecture of the executable it contains. The plugin can't currently tell which architecture the SDK targets, but `containertool` can infer the architecture by reading the ELF headers. Modifications ------------- * Adds basic ELF header reader. Reading the whole ELF header is not required - the necessary information is provided by the first few fields of the header. * Changes `containertool` to use the detected architecture unless overridden by command line flags or environment variables. Result ------ Adding ELF detection reduces the risk of building a mismatched container image, where the architecture of the packaged binary does not match the architecture of the underlying Linux distribution. Test Plan --------- * New unit tests exercise ELF header parsing * A new integration test checks that `containertool` selects the correct base image architecture for different cross-compiled binaries * All previous tests continue to pass Fixes #49
container: Add application layer to the correct end of the layer stack ( #51) Motivation ---------- containertool currently adds the app layer to the beginning of the layer stack array in the manifest. This results in the app layer being the first to be unpacked, with the others stacked on top. We can show this by adding a plain text file as the executable. If we stack another layer on top with a file of the same name, it should replace the underlying one but it does not: echo first > bar swift run containertool --repository localhost:5555/bar bar podman run --pull=always -it --rm --entrypoint=cat localhost:5555/bar:latest bar # prints: first echo second > bar swift run containertool --repository localhost:5555/bar bar --from localhost:5555/bar:latest podman run --pull=always -it --rm --entrypoint=cat localhost:5555/bar:latest bar # prints: first # should print: second Currently containertool is only used to add the application binary to the application layer. This bug will only cause a problem if the base layer adds a binary at the same path, because this will override the application. This bug probably arose because the specification for the rootfs.diff_ids field of the image configuration defines the layers as being "in order from first to last", which could be read ambiguously: https://github.com/opencontainers/image-spec/blob/main/config.md?plain=1#L220-L222 The specification for the manifest.layers field is much more explicit about the ordering: https://github.com/opencontainers/image-spec/blob/fbb4662eb53b80bd38f7597406cf1211317768f0/manifest.md?plain=1#L70-L71 Modifications ------------- Append the application layer to layer stacks in the manifest and configuration blobs, instead of prepending. Result ------ This with this change, the second build and container run in the example above prints "second" as expected. Test Plan --------- This PR adds a new integration test which uses `containertool` to build two layers and check that they override each other correctly. All existing tests continue to pass. Fixes #57
ContainerRegistry: The registry may return a relative blob upload URL (… …#44) Motivation ---------- In the 'Post then Put' blob upload method (https://github.com/opencontainers/distribution-spec/blob/main/spec.md#post-then-put) the client starts by making a POST request asking the registry to start an upload session. The registry responds with a URL to which the client should PUT the blob. The upload location might not be provided by the registry server, allowing the registry to offload storage to a different service. Until now all registries we have encountered have returned absolute upload URLs, however GHCR returns a relative URL causing uploads to fail as reported in #43. Modifications ------------- If the registry returns a relative URL, startBlobUpload() rewrites it to be an absolute URL referring to the registry. Result ------ Images can be pushed to GHCR and other registries which return relative upload locations. Test Plan --------- Automated tests continue to pass. Tested manually with GHCR and other known registries. Fixes: #43
containertool: Use epoch date in image metadata (#38) Motivation ---------- The registry is a content-addressable store in which objects are identified by their hashes. Many objects, such as image manifests, contain timestamp fields. A difference in a timestamp field will cause otherwise identical objects to have different hashes, causing a variety of problems: * an image cannot be verified by rebuilding it from the same original parts and comparing the result * the registry cannot completely deduplicate two images with almost identical contents because their hashes do not match, wasting storage and network bandwidth (some block-level deduplication may still be possible, but clients which already have one image will have to pull the other in its entirety) These problems can be avoided by setting timestamp fields to fixed values, often the Unix epoch: https://reproducible-builds.org/docs/source-date-epoch/ Modifications ------------- Set the overall image manifest timestamp and the timestamp used in the image history log to the Unix epoch. `containertool` already sets file modification times to the epoch when creating image layers. Result ------ An image packaging the same executable will have the same hash when built at different times on the same machine or different machines. Test Plan --------- * All existing tests, including end to end tests, continue to pass. * Manually verified that repeated builds produces images with identical hashes.
PreviousNext