Skaffold after sync hook to target specific containers

[RossKohler]

Let me just start by saying that I've really been enjoying using Skaffold in one of my projects. I would like to make a feature request, or maybe someone can give me some idea of how I can achieve this using the existing skaffold.yaml schema https://skaffold.dev/docs/references/yaml/.

I'm running the sync functionality for the local development of an Airflow instance within a Kind Kubernetes cluster. Basically, every time I make changes to files in my local directory they are synced with the Airflow deployment and I'm able to dev in a complete environment that accurately simulates production. This works well!

The problem is that Airflow has intervals in which it scans the file structure for changes. So between syncs, there could at most be 30 seconds to wait before Airflow picks up the changes. But if I could run airflow dags reserialize inside one of the pods after every sync, I could force Airflow to rescan immediately. This is almost completely possible with the current structure of the skaffold.yaml schema, I could just use the after sync hook and run the command in the containers. But what happens, is that because I cant target a specific pod, like I can in other hooks, this command runs 4 or 5 times (depending on how many pods are running).

Would it make sense to allow that after-sync hook to target specific pods? like the other hooks seem to allow you to do?

A workaround is that instead of using the container parameter, I just write a script that get's triggered after every sync that basically does a kubectl exec on one of the pods. I'm not sure if there are any other suggestions on this?

[ericzzzzzzz]

Hi @RossKohler , Thank you for creating this issue. Just to make sure I understand this correctly. The problem is that you have some containers use the same image, and you want to only invoke hook on only one of those?

[RossKohler]

Correct. When you're deploying AIrflow there's multiple pods, airflow-triggerer, airflow-scheduler, airflow-webserver and I would like to interact with the CLI just once on the airflow-webserver pod. I would like to run the airflow dags reserialize command. All these pods use the same image, so what will happen is that the command will be run on each pod thus executing 3 times. The command fails on some pods so it ends up just being a bit of a messy solution. I would like to be able to target which pod I want this after sync hook command to run.

[ericzzzzzzz]

@RossKohler Thank you for the explanation, using a image for different purposes sounds a valid use case to me. Marking this as p2 at the moment.