Robustness tests etcd logs should go to artifacts and not spam output

[serathius]

What would you like to be added?

In Linearizability action we use EXPECT_DEBUG=true to get logs to stdout. Because of continious failpoint injection their volume is overwhelming. Better approach would be to redirect logs to file and upload them to artifacts on failed test.

Why is this needed?

Should improve clarity and performance of "Linearizability" test action.

[anushkamittal20]

Hey @serathius, I would like to start contributing to etcd with this issue.
I see that we have set EXPECT_DEBUG=true in linearizability-template.yaml, to redirect them to a separate file we could use the braces '{ ... } 1> logs.txt' to provide a unit of I/O redirection. But we want to eliminate an stdout completely and redirect to a separate file. For the same, I was trying to look to read up on the EXPECT_DEBUG ( parameter but couldn't find much on it. Could you please direct me to some documentation? ) I was thinking that in this part of the code we could change the code to send to a file and tar it instead of printing it directly using fmt.
It would be great to hear your feedback on this!
Thank you very much!

[serathius]

This issue is about changing https://github.com/etcd-io/etcd/blob/ee566c492bb2e0962068a531666c68e1c39d3723/pkg/expect/expect.go package to support writing to files.

[pchan]

Is anyone looking at this ? If not, I can attempt to add the capability.

[jmhbnz]

Hi @pchan there are no work in progress pull requests linked so feel free to work on it, I have assigned the issue to you, thanks! :)

[serathius]

To give better idea on how it should work. I think we should add support for an environment variable EXPECT_LOG_DIR that will change behavior of pkg/expect to also write logs to file. This could be compatible with EXPECT_DEBUG (write to both), however it would be ok if the expect just failed if both are configured.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 21 days if no further activity occurs. Thank you for your contributions.

[chaochn47]

Hi @pchan, is there any progress on this?

Feel free to flush the local changes as a drafted PR even if it is not yet fully ready for review.

Handle inactive PRs
Poke PR owner if review comments are not addressed in 15 days. If PR owner does not reply in 90 days, update the PR with a new commit if possible. If not, inactive PR should be closed after 180 days.

https://etcd.io/docs/v3.5/triage/prs/#handle-inactive-prs

[serathius]

Note, with migration to Prow this might not be needed. Prow is much much better in handling logs. #18136

[jmhbnz]

Note, with migration to Prow this might not be needed. Prow is much much better in handling logs. #18136

Agreed - Prow handles large volumes of log output gracefully. Let's close this and #16664 for now as not being a priority currently. We can always re-open if priorities change.