pre-commit hooks to delete the containers they create

[rawwar]

Description

For every single pre-commit run, it seems we use docker to run several checks. But, these containers are not removed by default. Hence, I can see something as below in my docker desktop:

Use case/motivation

Would be nice if we can just use --rm when running the container or some other possible mechanism so that they are removed right after they are stopped.

Related issues

Could not find anything related.

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[potiuk]

1. Are you on latest main ?

2. Can you look at the containers and see what commands they used / logs ?

3. Is it likely that you simply Ctrl-C'd those pre-commits while running ?

We do remove all containers (and that has been improved some 2 months ago where I switched all pre-commits to use docker-compose under the hood and remove all of the remaining containers withdocker-compose down after they do the job (one of the reasons to unify that was precisely to not forget about removing the containers).

But maybe there is some remaining one in some specific circumstances?

Can you post some details on what you did and about those containers that are remaining ?

[potiuk]

BTW. I just run pre-commit run --all-files and I see no containers remaining after that.

[rawwar]

I also have pre-commits' for pre-push enabled. I think containers are created when running them. I will re-run and confirm.

Are you on latest main ?

I am using main branch.

Can you look at the containers and see what commands they used / logs ?

I will share them in sometime.

[rawwar]

Update:

While I was working on pinecone provider update, I tried to add few work in progress files and commit. But, they were failing pre-commit checks. That's when 4 containers were left behind.

Command used for all of the left-behind containers is "/opt/airflow/scripts/in_container/run_fix_ownership.py"

[potiuk]

Ah cool. It's not pre-commit.

It's breeze trying to make sure that the files created are owned by host user. Only happens on Linux when docker is not rootless - because then files are created with root owner and we want to make sure that you can still own them as host user.

Fix in #37277