Beam gcp test modules

[RustedBones]

Moving test implementation in test modules.

Follow-up from #25713

See #25806

[RustedBones]

By setting workerHarnessContainerImage to an empty string, we instruct to pick up the non-versioned image as explained in #6694. It looks like this image ignores the java version, and runs with java 8.

CI steps Java_Examples_Dataflow_Java11 and Java_Examples_Dataflow_Java17 are failing because the test lib target respectively java 11 and java 17.

Someone should ensure that the test image runs with proper java version

[RustedBones]

@Abacn since you followed the original PR. Will leave this as draft until CI is fixed.

[github-actions]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions.

[RustedBones]

@Abacn sorry to tag you again. Can someone at dataflow/google check why the image used with workerHarnessContainerImage= does not respect the java version ?

[Abacn]

Hi @RustedBones thanks for being persistent working on this. I did not understand why "testRuntimeMigration" and the java version issue either. It unfortunately becomes a tech debt when we updated gradle major version. Since everything working fine currently I just did not dig into further.

[RustedBones]

No worries, I'll try to summarize my findings:

• moving test implementation in the test scope, so test classes and dependencies are not packaged in the main artifact
• updating java example to depend on sdks:java:io:google-cloud-platform with testRuntimeMigration configuration
• test configuration do not target java 8, but the java version used during compilation
• in the Java_Examples_Dataflow_Java[11|17], we compile and run tests with the associated java version.

Most test work except those including --workerHarnessContainerImage=. This option looks to be used to take a base image without a worker jar. (taking the one compiled locally).
This image does not take in account the local java version used, and always runs with java 8. This causes issue as the classpath now contains compiled test classes targeting java 11/17 bytecode.

When removing the --workerHarnessContainerImage= option (running with the default dataflow image, packaging the worker), the tests pass, (default image respects the java version used during job submission).

IMHO, the problem simply lies in this specific dataflow image used for testing which needs to be updated to respect java version used during job submission.

[github-actions]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions.

[github-actions]

This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.