Issue: #3708 add ParameterizedTest#argumentCountValidation

[JonasJebing]

Overview

This allows parameterized tests to fail
when there are more arguments provided than declared by the test method. This is done in a backwards compatible way
by only enabling that validation when the new
junit.jupiter.params.argumentCountValidation is set to strict or ParameterizedTest#argumentCountValidation is set to ArgumentCountValidationMode.STRICT.

Open Questions

• Should these additions be declared as experimental or stable?
• Should this feature be documented in the User Guide and Release Notes, given that it is declared as experimental?
• Should the new precondition be documented on org.junit.jupiter.params.ParameterizedTestExtension#provideTestTemplateInvocationContexts, even though it is just an interface override and none of the existing preconditions are documented there?

---

I hereby agree to the terms of the JUnit Contributor License Agreement.

---

Definition of Done

• There are no TODOs left in the code
• Method preconditions are checked and documented in the method's Javadoc
• Coding conventions (e.g. for logging) have been followed
• Change is covered by automated tests including corner cases, errors, and exception handling
• Public API has Javadoc and @API annotations
• Change is documented in the User Guide and Release Notes

[marcphilipp]

The error message should be the same as in EnumConfigurationParameterConverter. Unfortunately, we can't (easily) reuse that here.

[JonasJebing]

Is there some way that we could make it more easily reusable across the project? Maybe I could add another PR to address that after merging this one.
For now, I basically copied the messages from EnumConfigurationParameterConverter.

[marcphilipp]

• Should these additions be declared as experimental or stable?

Our guideline is to introduce new features as "experimental".

• Should this feature be documented in the User Guide and Release Notes, given that it is declared as experimental?

I think we should add a small sample to the User Guide otherwise the chance to get feedback is slim.

• Should the new precondition be documented on org.junit.jupiter.params.ParameterizedTestExtension#provideTestTemplateInvocationContexts, even though it is just an interface override and none of the existing preconditions are documented there?

Documenting preconditions is mainly interesting for public API so we can omit it here since ParameterizedTestExtension is package-private.

[JonasJebing]

Thank you for all the pointers and comments on this PR @marcphilipp. I've addressed all your comments. Could you have another look?

[marcphilipp]

Thanks for making those changes! I think we're very close now. Please also add an entry to the 5.12.0-M1 release notes.

[JonasJebing]

@marcphilipp could you review again? I applied all suggestions and added a release note.

[marcphilipp]

Please move this snippet to ParameterizedTestDemo (and annotate it with @ExpectToFail; see other usages on how to avoid that annotation from being included in the user guide). We include them from there to ensure they compile and pass or fail as expected.

[marcphilipp]

Looks like @ExpectToFail doesn't work as I had expected. 😅 I'll take a closer look.

[marcphilipp]

Playing around with this made me realize that we need to defer this validation until the invocation is being executed. Otherwise, all invocations will fail rather than just the offending ones. For example, annotating the added test in ParameterizedTestDemo with @CsvSource({ "42, -666", "1", "2", "3" }) should result in the first invocation being reported as failed due to the new validation but all others should pass.

I think the cleanest way to do this is to return another extension from ParameterizedTestInvocationContext#getAdditionalExtensions, say ArgumentCountValidator, that implements InvocationInterceptor and overrides interceptTestTemplateMethod with the validation before calling invocation.proceed().

Please let me know whether that's enough information for you to proceed. Alternatively, I'd also be willing to take over if you'd prefer that.

[JonasJebing]

Yes, I also thought something wasn't quite right after playing around with ParameterizedTestDemo. Thanks for the pointers with ParameterizedTestInvocationContext#getAdditionalExtensions. That definitely seems doable to me.