Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[chore]: fix linux packaging tests #30202

Merged

Conversation

cwegener
Copy link
Contributor

Fixes #16450

The original packaging tests for the Linux packages (deb, rpm) that were
created in #405 used a bit of gnarly work around in order to allow very
limited Systemd functionality to be used in Docker/Containerd

At the time when those packaging tests were written, podman was already
able to run Systemd, but podman was not included in Ubuntu 20.04 yet ...

Now that Podman is available in Ubuntu 22.04, running SystemD in Podman
just works.

Signed-off-by: Christoph Wegener cwegener@users.noreply.github.com

Fixes open-telemetry#16450

The original packaging tests for the Linux packages (deb, rpm) that were
created in open-telemetry#405 used a bit of gnarly work around in order to allow very
limited Systemd functionality to be used in Docker/Containerd

At the time when those packaging tests were written, podman was already
able to run Systemd, but podman was not included in Ubuntu 20.04 yet ...

Now that Podman is available in Ubuntu 22.04, running SystemD in Podman
just works.

Signed-off-by: Christoph Wegener <cwegener@users.noreply.github.com>

# test install
echo
$docker_run
podman run --name $container_name -d $image_name
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In your comment here, you said there should be a check to ensure that the run does not silently fail. Will podman fail here if the container fails to start or should we still add an additional check?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, podman-run will return a non-zero exit code ala chroot, 125 - podman command itself failed, 126 - the contained command failed to run, 127 - the contained command could not be found. (These are the same as docker-run should return)

However, the original issue in #16450 is actually the result of the initial docker-run to succeeding, but then the container immediately shuts off again due to unknown reasons (apart from the reason that Docker and SystemD have been at loggerheads for years and expecting the two to run smoothly together is not an option at this point in time)

So, going back to my original comment in your question. Yes, there should be a health-check command running inside the otelcolcontrib-*-test images that waits until SystemD has finished starting up all services before returning a 'healthy' state. Then the test.sh bash script could wait for the container to transition into a healthy state after podman-run and before podman-cp, which would cover for that period of uncertainty where the container started, but might have not fully initialized as expected.

I think this should be part of a separate patch though. I'll start working on adding that health check function.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough answer. If this PR gets merged before the health check function PR is submitted please create a new tracking issue for it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created a tracking issue for the health check issue. #30239


# test install
echo
$docker_run
podman run --name $container_name -d $image_name
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough answer. If this PR gets merged before the health check function PR is submitted please create a new tracking issue for it.

@cwegener
Copy link
Contributor Author

cwegener commented Jan 4, 2024

@codeboten Pinging you since you currently are assigned to the issue that this patch fixes.

Copy link
Member

@TylerHelmuth TylerHelmuth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm no ubuntu or podman expert, but the replacements make sense to me and the CI is passing. I'd like a few more @open-telemetry/collector-contrib-approvers to review before merging.

Copy link
Member

@djaglowski djaglowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@cwegener
Copy link
Contributor Author

cwegener commented Jan 8, 2024

I'm no ubuntu or podman expert, but the replacements make sense to me and the CI is passing. I'd like a few more @open-telemetry/collector-contrib-approvers to review before merging.

This is a good intro to the Docker vs. SystemD feud. https://lwn.net/Articles/676831/

Edit: As to the question of why the original hack of making Systemd successfully initialize inside of docker/containerd instance stops working when upgrade from Ubuntu 20.04 to 22.04, I also have no idea. But I thought it would be wasted time to investigate it, given the historically antagonistic nature of Systemd and Containerd(docker) - as per LWN article.

And given that alternative container runtimes to docker/containerd have been available and ready to use, I think it makes perfect sense to switch to a container runtime that works seamlessly (and has a more "unixy" architecture than docker with a fork/exec model instead of client/server)

Copy link
Contributor

@codeboten codeboten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this on @cwegener!

@codeboten codeboten merged commit 7917bb2 into open-telemetry:main Jan 8, 2024
86 checks passed
@github-actions github-actions bot added this to the next release milestone Jan 8, 2024
cparkins pushed a commit to AmadeusITGroup/opentelemetry-collector-contrib that referenced this pull request Jan 10, 2024
Fixes open-telemetry#16450

The original packaging tests for the Linux packages (deb, rpm) that were
created in open-telemetry#405 used a bit of gnarly work around in order to allow very
limited Systemd functionality to be used in Docker/Containerd

At the time when those packaging tests were written, podman was already
able to run Systemd, but podman was not included in Ubuntu 20.04 yet ...

Now that Podman is available in Ubuntu 22.04, running SystemD in Podman
just works.

Signed-off-by: Christoph Wegener <cwegener@users.noreply.github.com>
@cwegener cwegener deleted the refactor/linux-package-tests branch January 12, 2024 22:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[gh-actions] build-package jobs are failing on the ubuntu-22.04 runners
5 participants