Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libcnb-test: Clean up Docker volumes after each test #741

Merged
merged 2 commits into from
Nov 17, 2023

Conversation

edmorley
Copy link
Member

When using pack build, by default Pack will create two Docker volumes for caching - one for the build layer cache, and the other for the launch layer cache. Pack bases the names for these volumes on the app image name, so that the next time a build is performed with the same app image name, the associated cache is used for that rebuild.

When libcnb-test runs tests, it intentionally uses a random Docker image name for each new test, and then tears down the image and any associated containers after the tests.

However, until now the Docker volumes automatically created by Pack were not being removed, which can cause issues when running the tests locally, since the Docker VM can run out of disk space - as seen in #570. The leftover volumes can also be seen in the tests log output added in #740.

Pack doesn't currently offer a "clean up the volumes associated with this app name" feature, so we have to remove these volumes manually. To do so we need the volume name - which we achieve by generating a name ourselves, and passing it to pack build using the --cache options, so Pack uses our volume name instead of generating its own automatic volume name.

Since we're now cleaning up not only images but volumes too, I've refactored the way resources are cleaned up, to reduce the amount of duplication (since otherwise we'd need to handle the "still clean up for expected failures" case for volumes too). This new implementation also solves #735 and #736.

Fixes #570.
Fixes #735.
Fixes #736.
GUS-W-13195631.
GUS-W-14503186.
GUS-W-14503192.

@edmorley edmorley added bug Something isn't working enhancement New feature or request libcnb-test labels Nov 16, 2023
@edmorley edmorley self-assigned this Nov 16, 2023
@edmorley edmorley marked this pull request as ready for review November 16, 2023 14:03
@edmorley edmorley requested a review from a team as a code owner November 16, 2023 14:03
@edmorley edmorley removed the enhancement New feature or request label Nov 16, 2023
Copy link
Member

@Malax Malax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left one comment, let's discuss there - otherwise this has my approval. :)

libcnb-test/src/test_runner.rs Outdated Show resolved Hide resolved
When using `pack build`, by default Pack will create two Docker
volumes for caching - one for the build layer cache, and the other
for the launch layer cache. Pack bases the names for these volumes
on the app image name, so that the next time a build is performed
with the same app image name, the associated cache is used for
that rebuild.

When libcnb-test runs tests, it intentionally uses a random Docker
image name for each new test, and then tears down the image and
any associated containers after the tests.

However, until now the Docker volumes automatically created by
Pack were not being removed, which can cause issues when
running the tests locally, since the Docker VM can run out of disk
space - as seen in #570. The leftover volumes can also be seen
in the tests log output added in #740.

Pack doesn't currently offer a "clean up the volumes associated
with this app name" feature, so we have to remove these volumes
manually. To do so we need the volume name - which we achieve
by generating a name ourselves, and passing it to `pack build`
using the `--cache` options, so Pack uses our volume name instead
of generating its own automatic volume name.

Since we're now cleaning up not only images but volumes too, I've
refactored the way resources are cleaned up, to reduce the amount
of duplication (since otherwise we'd need to handle the "still clean up
for expected failures" case for volumes too).

This new implementation also solves #735 and #736.

Fixes #570.
Fixes #735.
Fixes #736.
GUS-W-13195631.
GUS-W-14503186.
GUS-W-14503192.
@edmorley edmorley force-pushed the edmorley/cache-volume-cleanup branch from 9794db0 to 851242e Compare November 16, 2023 17:46
@edmorley edmorley merged commit 50b7a96 into main Nov 17, 2023
4 checks passed
@edmorley edmorley deleted the edmorley/cache-volume-cleanup branch November 17, 2023 10:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working libcnb-test
Projects
None yet
3 participants