Remove upgrade marker if rolling back to versions older than 9.2.0

[swiatekm]

What does this PR do?

Conditionally remove upgrade marker when an elastic-agent upgrade is rolled back to a version that does not contain #8407
This PR will delete the upgrade marker by default (in order to maintain backward compatibility).

Why is it important?

There's a race condition if the rolled back agent starts slowly for the old (i.e. rolled back) watcher to pick up the .upgrade-marker file when the watcher that triggered the rollback has already terminated.
The old watcher code will interpret the presence of the upgrade marker as an ongoing upgrade for which it will have perform the watching of the agent for the grace period without checking agent version or upgrade state.

This may lead to elastic-agent trying to delete itself after the rollback has already been performed.

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• [ ] I have added an entry in ./changelog/fragments using the changelog tool
• [ ] I have added an integration test or an E2E test

Disruptive User Impact

How to test this PR locally

See the additional unit tests in watch_test.go and rollback_test.go.
Manual testing requires an agent that is slow to restart in case of rollback, may need some custom code modifications.

Related issues

• Closes https://github.com/elastic/ingest-dev/issues/6250
• Relates Preserve upgrade marker when rolling back and add rollback reason to upgrade details #8407

Questions to ask yourself

• How are we going to support this in production?
• How are we going to measure its adoption?
• How are we going to debug this?
• What are the metrics I should take care of?
• ...

[mergify]

This pull request does not have a backport label. Could you fix it @swiatekm? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-./d./d is the label that automatically backports to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: c7a5ac7

Failed CI Steps

• x86_64:sudo: default

cc @swiatekm

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)