Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run apt-get in docker builds without using any cache. #7011

Conversation

daniellehrner
Copy link
Contributor

PR description

The docker build step can fail if apt-get gets it's dependencies from a transparent cache (e.g. from an ISP) and this cache serves corrupted or outdated packages. This PR clears all the apt-get cache first and afterwards executes the next steps while making sure that no cache whatsoever is used.

I have further split the the RUN command into several steps to make debugging easier, because like that the exact command that is failing while be shown. Right now we have several commands chained together and it's not always clear which of the several commands was failing.

Fixed Issue(s)

fixes #7009

Thanks for sending a pull request! Have you done the following?

  • Checked out our contribution guidelines?
  • Considered documentation and added the doc-change-required label to this PR if updates are required.
  • Considered the changelog and included an update if required.
  • For database changes (e.g. KeyValueSegmentIdentifier) considered compatibility and performed forwards and backwards compatibility tests

Locally, you can run these tests to catch failures early:

  • unit tests: ./gradlew build
  • acceptance tests: ./gradlew acceptanceTest
  • integration tests: ./gradlew integrationTest
  • reference tests: ./gradlew ethereum:referenceTests:referenceTests

Copy link
Contributor

@fab-10 fab-10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which is the impact, of splitting the RUN command, in term of bigger image size?

docker/Dockerfile Outdated Show resolved Hide resolved
docker/Dockerfile Outdated Show resolved Hide resolved
docker/Dockerfile Outdated Show resolved Hide resolved
…individual RUN commands

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
…l RUN commands

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
@daniellehrner daniellehrner force-pushed the fix/issue-7009/non-caching-apt-get branch from a2fd185 to 0008660 Compare May 6, 2024 14:39
@daniellehrner
Copy link
Contributor Author

Which is the impact, of splitting the RUN command, in term of bigger image size?

@fab-10 I misunderstood how docker images are built and did not realize the impact in size of several run commands. I've reverted the change and use only one run command to save disk size.

@daniellehrner daniellehrner requested a review from fab-10 May 6, 2024 14:43
@daniellehrner daniellehrner enabled auto-merge (squash) May 8, 2024 08:17
@daniellehrner daniellehrner merged commit 033d4d6 into hyperledger:main May 8, 2024
42 checks passed
@daniellehrner daniellehrner deleted the fix/issue-7009/non-caching-apt-get branch May 8, 2024 10:37
jflo pushed a commit to jflo/besu that referenced this pull request May 28, 2024
* Run apt-get in docker builds without using any cache. Split steps in individual RUN commands

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* use env variable for apt non-cache settings, revert split into several RUN commands

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* clean apt-get cache after installation to decrease docker image size

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

---------

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Signed-off-by: Justin Florentine <justin+github@florentine.us>
matthew1001 pushed a commit to kaleido-io/besu that referenced this pull request Jun 7, 2024
* Run apt-get in docker builds without using any cache. Split steps in individual RUN commands

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* use env variable for apt non-cache settings, revert split into several RUN commands

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* clean apt-get cache after installation to decrease docker image size

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

---------

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
jflo pushed a commit to jflo/besu that referenced this pull request Jun 10, 2024
* Run apt-get in docker builds without using any cache. Split steps in individual RUN commands

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* use env variable for apt non-cache settings, revert split into several RUN commands

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

* clean apt-get cache after installation to decrease docker image size

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>

---------

Signed-off-by: Daniel Lehrner <daniel.lehrner@consensys.net>
Signed-off-by: Justin Florentine <justin+github@florentine.us>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Docker build fails because of transparent caching problems
2 participants