Skip to content

Exit mc mirror when errors occur if retry is disabled and it's a watch operation #5163

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

jadolg
Copy link

@jadolg jadolg commented Mar 11, 2025

Community Contribution License

All community contributions in this pull request are licensed to the project maintainers
under the terms of the Apache 2 license.
By creating this pull request I represent that I have the right to license the
contributions to the project maintainers under the Apache 2 license.

Description

When running mc mirror --watch the cli exits with a non-zero exit code instead of just printing the error.

Motivation and Context

I'm running mc mirror --watch in a systemd unit to constantly mirror my minio buckets to AWS S3. In the current state, it's not possible to monitor mc effectively since the process will just continue running.

How to test this PR?

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Optimization (provides speedup with no functional changes)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • Fixes a regression (If yes, please add commit-id or PR # here)
  • Unit tests added/updated
  • Internal documentation updated
  • Create a documentation update request here

@jadolg
Copy link
Author

jadolg commented Mar 11, 2025

Solves #5162

@jadolg jadolg marked this pull request as ready for review March 11, 2025 12:10
Copy link
Contributor

@klauspost klauspost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I may be missing something, but this seems very likely to break all kind of stuff already running?

@jadolg
Copy link
Author

jadolg commented May 6, 2025

@klauspost it is a breaking change but I think it makes sense.
mc mirror --watch is a long running operation. Without directly looking at the logs, there's currently no way to know that the application is failing to do it's job.
In my case, I was trying to mirror a bucket into an unexisting bucket in another system. I'm running mc mirror --watch in an "unatended" way (systemd unit) so I'm not looking at the logs unless something goes wrong. mc mirror --watch will fail and I have no idea my bucket is not being mirrored.
I could technically monitor the logs and trigger some alert if errors occur, but then I'd need to account for all possible errors and error format making this not really bulletproof.
My point is: I think it should break if an error occurs so it is obvious the operation can't be completed.
If you are concerned about breaking other people's already existing workflows, we could also hide this behavior behind a flag (Eg: --fail-on-error)
wdyt?

@klauspost
Copy link
Contributor

If you are concerned about breaking other people's already existing workflows, we could also hide this behavior behind a flag (Eg: --fail-on-error)

Yeah. I think that is the only reasonable approach. mc is running in a lot of places, so we need to be reasonable about back-compat.

@jadolg
Copy link
Author

jadolg commented May 6, 2025

New flag added and removed the need for the --watch parameter since the new flag should make it explicit for all mirror operations.

@klauspost
Copy link
Contributor

ok, so my only remaining reservation is that it may not exit in all cases where people would expect it, but I can live with that.

@jadolg
Copy link
Author

jadolg commented May 7, 2025

ok, so my only remaining reservation is that it may not exit in all cases where people would expect it, but I can live with that.

It's tricky to get it to fail everywhere where it might fail because everything happens separately but I think monitorMirrorStatus should cover most cases since the errors were only logged there anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants