Discard cache for rules affected by --experimental_docker_image

[allada]

Description of the problem / feature request:

--experimental_docker_image should discard cache on actions that run inside docker.

We are using this feature because our builds are not fully hermetic and efforts to do so keep increasing the complexity of our workspace/build rules, so we decided to rely on docker (for now) to do it for us.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

1. Setup docker + bazel (like in this link: https://docs.bazel.build/versions/master/remote-execution-sandbox.html)
2. Run a build bazel build //...
3. Clean build bazel clean
4. Make small change to docker image so you get a new container id and setup bazel to use it
5. Run a build bazel build //...

At this point it will reuse the same cache from the old container, but it should discard cache.

What operating system are you running Bazel on?

Any

What's the output of bazel info release?

0.26.1 (but according to documentation happens on latest)

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

Cannot, private repo.

Any other information, logs, or outputs that you want to share?

The current work around is to wrap the bazel command in a custom python script (we already do this to inject --experimental_docker_image={docker_image_id}, so it always grabs the docker image for the checkedout build), and append --platform_suffix={my_image_hash} to the build on behalf of the user.

The down side of doing this is that we update our image about once a week, and our bazel-out is on the order of 100G, resulting in many hundreds of gigs of stale cache. We fix this by having our clean up the stale/old folders in bazel-out.

[buchgr]

Hi there, do I understand correctly the issue is that changing the value of --experimental_docker_image does not invalidate the cache? That's a known problem across all execution strategies. I believe @aragos is working on a solution.

[allada]

@buchgr, Yes that is correct. As stated before, there is a workaround so severity is not terribly high.

[katre]

@aragos, please confirm if this is something your work on execution strategies will affect.

[katre]

Commenting: yes, this will be affected by the work on execution strategies and platforms.

I'm closing this now and redirecting you to #11432 as the tracking bug for that effort.