ci(itest): print container health and latest logs on each readiness check

[andrewazores]

Welcome to Cryostat! 👋

Before contributing, make sure you have:

• Read the contributing guidelines
• Linked a relevant issue which this PR resolves
• Linked any other relevant issues, PR's, or documentation, if any
• Resolved all conflicts, if any
• Rebased your branch PR on top of the latest upstream main branch
• Attached at least one of the following labels to the PR: [chore, ci, docs, feat, fix, test]
• Signed the last commit: git commit --amend --signoff

---

Related to #1543

How to test:
ex. bash repeated-integration-tests.bash 1 HealthIT

[github-actions]

Test image available:

$ CRYOSTAT_IMAGE=ghcr.io/cryostatio/cryostat:pr-1544-283c369ef6501a88507a1fb46fe3ff743e3fd2f1 sh smoketest.sh

[andrewazores]

Okay, I think I'm done messing with this PR now. If everything works as intended this should allow the CI runners to push PR preview images to ghcr.io even if the itests fail, which they do for ARM images because 1. they don't seem to respond to the container health checks, and 2. they don't have a working JDP client.

This should also allow the on-push-to-main CI runner to publish the multiarch manifest image to quay.io even if integration tests fail.

Hopefully with one or both of those tools in hand we can then retrieve a CI-built ARM image and see if there is some other build environment or setup issue causing these test failures.

[andrewazores]

Oh, what? The integration test container successfully started this time:

https://github.com/cryostatio/cryostat/actions/runs/5271647223/jobs/9532967857?pr=1544

as expected, the tests are still failing all over the place because discovery is broken (no JDP).

[andrewazores]

Here's the relevant JDP failure log snippet:

INFO: JDP Discovery started
Jun 15, 2023 1:51:37 PM io.cryostat.core.log.Logger warn
WARNING: Exception thrown
java.net.SocketException: Protocol not available
	at java.base/sun.nio.ch.Net.getInterface6(Native Method)
	at java.base/sun.nio.ch.DatagramChannelImpl.getOption(DatagramChannelImpl.java:427)
	at java.base/sun.nio.ch.DatagramSocketAdaptor.getOption(DatagramSocketAdaptor.java:426)
	at java.base/sun.nio.ch.DatagramSocketAdaptor.outgoingNetworkInterface(DatagramSocketAdaptor.java:657)
	at java.base/sun.nio.ch.DatagramSocketAdaptor.defaultNetworkInterface(DatagramSocketAdaptor.java:672)
	at java.base/sun.nio.ch.DatagramSocketAdaptor.joinGroup(DatagramSocketAdaptor.java:522)
	at java.base/sun.nio.ch.DatagramSocketAdaptor.joinGroup(DatagramSocketAdaptor.java:481)
	at java.base/java.net.MulticastSocket.joinGroup(MulticastSocket.java:325)
	at org.openjdk.jmc.jdp.client.JDPClient.start(JDPClient.java:87)
	at io.cryostat.core.net.discovery.JvmDiscoveryClient.start(JvmDiscoveryClient.java:105)
	at io.cryostat.platform.internal.DefaultPlatformClient.start(DefaultPlatformClient.java:83)
	at io.cryostat.discovery.BuiltInDiscovery.lambda$start$5(BuiltInDiscovery.java:134)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.DistinctOps$1$2.accept(DistinctOps.java:174)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1707)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at io.cryostat.discovery.BuiltInDiscovery.start(BuiltInDiscovery.java:97)
	at io.vertx.core.impl.DeploymentManager.lambda$doDeploy$5(DeploymentManager.java:196)
	at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:264)
	at io.vertx.core.impl.WorkerContext.lambda$run$3(WorkerContext.java:106)
	at io.vertx.core.impl.WorkerContext.lambda$null$1(WorkerContext.java:92)
	at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:833)

[andrewazores]

Not especially helpful: https://github.com/openjdk/jdk17u-dev/blob/f9a60759b13f242c45bef8506e25ffea3e972247/src/java.base/share/classes/sun/nio/ch/Net.java#L783