-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Archive Logs for init and wait container #12640
Comments
I think it'd be enough if the docs (https://argo-workflows.readthedocs.io/en/latest/configure-archive-logs/) on top of suggesting:
Had a link to the relevant section in workflow-controller-configmap.yaml for suggesting a better alternative. |
Hello. I believe that we can simply modify this section slightly to make it feasible. Wouldn't this be a argo-workflows/workflow/executor/executor.go Lines 597 to 622 in d5a4f7e
|
Yes, But you would also need to consider
|
Thanks for confirming.
I'm thinking this could be recorded without awareness if |
📝 Simply adding the container name doesn't seem to work. argo-workflows/workflow/executor/executor.go Line 607 in 5cd8415
ChangescontainerNames := append(we.Template.GetMainContainerNames(), common.InitContainerName, common.WaitContainerName) Logsvscode ➜ ~/go/src/github.com/argoproj/argo-workflows (main) $ kubectl logs arguments-parameters-from-configmap-14 -c wait
time="2024-05-22T06:48:56.008Z" level=info msg="Starting Workflow Executor" version=latest+d4b9327.dirty
time="2024-05-22T06:48:56.108Z" level=info msg="Using executor retry strategy" Duration=1s Factor=1.6 Jitter=0.5 Steps=5
time="2024-05-22T06:48:56.110Z" level=info msg="Executor initialized" deadline="2024-05-22 06:53:49 +0000 UTC" includeScriptOutput=false namespace=argo podName=arguments-parameters-from-configmap-14 templateName=whalesay version="&Version{Version:latest+d4b9327.dirty,BuildDate:2024-05-22T06:46:22Z,GitCommit:d4b9327b93511164d4f4401df6e867cd08faa2f7,GitTag:untagged,GitTreeState:dirty,GoVersion:go1.21.10,Compiler:gc,Platform:linux/amd64,}"
time="2024-05-22T06:48:56.211Z" level=debug msg="Create workflowtaskresults 403"
time="2024-05-22T06:48:56.274Z" level=warning msg="failed to patch task result, falling back to legacy/insecure pod patch, see https://argo-workflows.readthedocs.io/en/latest/workflow-rbac/" error="workflowtaskresults.argoproj.io is forbidden: User \"system:serviceaccount:argo:argo\" cannot create resource \"workflowtaskresults\" in API group \"argoproj.io\" in the namespace \"argo\""
time="2024-05-22T06:48:56.282Z" level=debug msg="Patch pods 200"
time="2024-05-22T06:48:56.298Z" level=info msg="+++++++++++++++++Starting deadline monitor+++++++++++++++++"
time="2024-05-22T06:48:57.303Z" level=info msg="Main container completed" error="<nil>"
time="2024-05-22T06:48:57.305Z" level=info msg="No Script output reference in workflow. Capturing script output ignored"
time="2024-05-22T06:48:57.305Z" level=info msg="No output parameters"
time="2024-05-22T06:48:57.306Z" level=info msg="No output artifacts"
time="2024-05-22T06:48:57.316Z" level=info msg="S3 Save path: /tmp/argo/outputs/logs/main.log, key: arguments-parameters-from-configmap-14/arguments-parameters-from-configmap-14/main.log"
time="2024-05-22T06:48:57.496Z" level=info msg="Creating minio client using static credentials" endpoint=s3.amazonaws.com
time="2024-05-22T06:48:57.497Z" level=info msg="Saving file to s3" bucket=panicboat-sandbox-723535945756 endpoint=s3.amazonaws.com key=arguments-parameters-from-configmap-14/arguments-parameters-from-configmap-14/main.log path=/tmp/argo/outputs/logs/main.log
time="2024-05-22T06:48:57.973Z" level=info msg="Save artifact" artifactName=main-logs duration=656.608042ms error="<nil>" key=arguments-parameters-from-configmap-14/arguments-parameters-from-configmap-14/main.log
time="2024-05-22T06:48:57.973Z" level=info msg="not deleting local artifact" localArtPath=/tmp/argo/outputs/logs/main.log
time="2024-05-22T06:48:57.973Z" level=info msg="Successfully saved file: /tmp/argo/outputs/logs/main.log"
time="2024-05-22T06:48:57.974Z" level=error msg="executor error: open /var/run/argo/ctr/init/combined: no such file or directory"
time="2024-05-22T06:48:57.974Z" level=error msg="executor error: open /var/run/argo/ctr/wait/combined: no such file or directory"
time="2024-05-22T06:48:57.978Z" level=debug msg="Create workflowtaskresults 403"
time="2024-05-22T06:48:57.979Z" level=warning msg="failed to patch task result, falling back to legacy/insecure pod patch, see https://argo-workflows.readthedocs.io/en/latest/workflow-rbac/" error="workflowtaskresults.argoproj.io is forbidden: User \"system:serviceaccount:argo:argo\" cannot create resource \"workflowtaskresults\" in API group \"argoproj.io\" in the namespace \"argo\""
time="2024-05-22T06:48:57.991Z" level=debug msg="Patch pods 200"
time="2024-05-22T06:48:57.999Z" level=info msg="Alloc=8996 TotalAlloc=16888 Sys=24677 NumGC=6 Goroutines=10"
time="2024-05-22T06:48:58.001Z" level=debug msg="Patch workflowtaskresults 403"
time="2024-05-22T06:48:58.001Z" level=warning msg="failed to patch task result, falling back to legacy/insecure pod patch, see https://argo-workflows.readthedocs.io/en/latest/workflow-rbac/" error="workflowtaskresults.argoproj.io \"arguments-parameters-from-configmap-14\" is forbidden: User \"system:serviceaccount:argo:argo\" cannot patch resource \"workflowtaskresults\" in API group \"argoproj.io\" in the namespace \"argo\""
time="2024-05-22T06:48:58.007Z" level=debug msg="Patch pods 200"
time="2024-05-22T06:48:58.011Z" level=fatal msg="open /var/run/argo/ctr/init/combined: no such file or directory" |
I'd prefer it if this was separately controllable. |
I would like to work on this assignment, although the modifications are going to be more extensive than I originally thought. I believe this issue requires capturing the logs of the init/wait container as well as the main container, but I don't understand how to implement this. |
dupe of #8902 |
Marked it as superseded since there are more implementation details here and more upvotes |
Summary
Currently argo
ArchiveLogs
only supports logs frommain
container.From time to time user is asking if it supports
init
andwait
container log.I'm aware the archivelog docs recommend a proper logging facility for logging,
Maybe this is still something we could consider if there's enough 👍
Use Cases
When would you use this?
Archive
init
andwait
container log for debugging purposeMessage from the maintainers:
Love this enhancement proposal? Give it a 👍. We prioritize the proposals with the most 👍.
The text was updated successfully, but these errors were encountered: