[Task] Fix the docker image build flow

[Skarlso]

Right now, the build flow creates a new image by tagging the Dockerbuild file with a new sha. This has the problem that if there is a change that needs to be rebuilt or there was a change that failed and we need to redo it, it will create the same layers and same image. There is no way to force a complete rebuild or at least part of the layers.

A simple solution is to introduce a version file into the top build process and increase the version when you want to nuke the layer.

Another option is to use --no-cache during the build process. But that still will create the same image id and upstream (github) might aggressively cache that. The point is the generate a new ID.

[Skarlso]

Okay, so I tried doing a simple build by changing a tiny thing in the requirements.
The run the container build process a we did in the past:

make -f Makefile.docker update-build-image-tag
make -f Makefile.docker commit-new-image-tag
make -f Makefile.docker push-build-image

And push and run a build here:
https://github.com/weaveworks/eksctl/actions/runs/2494618827

Same error.

Prepare all required actions
Run ./.github/actions/setup-identity
Run git config user.name "weaveworksbot"
fatal: not in a git directory
Error: Process completed with exit code 1[2](https://github.com/weaveworks/eksctl/runs/6878704574?check_suite_focus=true#step:5:2)8.

[Skarlso]

FINALLY! Found the problem.

So I added this by hand:

      - name: Configure git to trust the workspace despite the different owner
        run:
          git config --global --add safe.directory "$GITHUB_WORKSPACE"

which adds /__w/eksctl/eksctl as a safe folder.

BUT! Checkout does that already. :O

Adding repository directory to the temporary git global config as a safe directory
/usr/bin/git config --global --add safe.directory /__w/eksctl/eksctl

It seems to loose this after checkout.... I have no idea why.

[Skarlso]

Oh. That's because github workspace is NOT /__w/eksctl/eksctl.

[Skarlso]

This bags the question if this would work at all or where it would be staying at when it runs an actual workflow.

[Skarlso]

Also, this still doesn't explain why an older container is workiiinggggg.

[Skarlso]

    steps:
      - name: Checkout
        uses: actions/checkout@v3
        with:
          token: ${{ secrets.WEAVEWORKSBOT_TOKEN }}
          fetch-depth: 0
          set-safe-directory: '$GITHUB_WORKSPACE'

Looks like set-safe-directory has to be set now... but that's super weird.

[Skarlso]

Apparently. A combination with v3 and some linux updates, ( or a git update maybe ) causes this issue. Once the container was updated, it no longer worked because we also updated some other packages and pulled a more later go1.18 image. Either because of docker or because of a new git version. In any case. The fix is to add set-safe-directory to all of our checkouts.

[Skarlso]

Note that this only applies to actions which use the eksctl build image. All others use latest ubuntu and that fixes the problem too.