Make NormalizationService pluggable?

[globbestael]

Should AuthorVariantsExperimentsTest and PublicationExperiment be refactored?

Reason

• AuthorVariantsExperimentsTest tests the performance of several alternative normalizations of Author input fields
  • uses a PublicationExperiment (extends Publication) which has the alternative normalization methods. Publication has been refactored to a POJO and the normalization has been moved to IOService and NormalizationService. PublicationExperiment should follow the same pattern / will be removed once the methods are moved (?)

The original normalization of Publication (e.g. for Authors) has been spread to

• IOService::addNormalizedAuthor
• NormalizationService::normalizeInputAuthors
  There is some logic in the IOService::addNormalized... methods which cannot be moved to the NormalizationService: the fields of Publication are only set in IOService, NormalizationService is not aware of Publication.

Old solution

But the alternatives to be tested belong to the NormalizationService. This would entail for this test that:

• an alternative IOService can be injected into the DeduplicationService (and there should be a IOService interface). Or there should be a getter and setter for this IOService field of the DeduplicationService (?).
• there should be a NormalizationService interface
• the alternative NormalizationService implementations can be injected into this IOService.
• @DirtiesContext should applied to this test class?

Old Evaluation

• GOOD: production code and test code use same structure / principles
• GOOD: the AuthorComparatorService already uses this format
• GOOD: extending this to testing alternative implementations of normalization of journal, pages and title will be easier
• VERY GOOD: the current AuthorVariantsExperimentsTest only allows seeing the performance of the comparison for this field only. In the refactored version a full comparison for (1) all fields for (2) real test files would be possible.
• BAD: a lot of refactoring work

Update

In the meantime there has been serious refactoring of the code and the tests, a.o. NormalizationService as a class with static functions.
Making the NormalizationService pluggable will need a new analysis and probably a big refactoring.

2026-04-25: rename title of this issue

Questions

The current tests already have 2 types of tests / experiments / alternatives (but not for many fields):

Both types are learning test types? Is this a third type of tests next to unit and integration tests, or is this an aspect of the 2 existing types?

Previous refactorings of the tests has made a distinction between unit tests (JWSImilarity and SimilarityTests) and integration tests. The current tests in edu.dedupendnote.unit.services.AuthorVariantsExperimentsTest are a new subtype of the unit tests, and currently the assertions are dummy assertions (1 == 1), and we are more intersted in the side effects (log and system.err output) where the similarity of the production version is compared with the similarity of the alternative. Should this be removed from the test part of the proejct and get its own ...(?)

1. Alternative normalizations

e.g. record only the first initial, record only the first 5 authors, ...

Currently: edu.dedupendnote.unit.services.AuthorVariantsExperimentsTest

NormalizationService should be pluggable, and the functions should not be static functions. If we want to test a lot of alternatives, this one NormalizationService should probably be split into a number of services

2. Alternative comparisons

Currently: edu.dedupendnote.integration.services.AuthorExperimentsTests

But the current test with AuthorVariantsExperimentsTest is a minimal case, more a proof of concept: use different thresholds for author comparisons

First steps

It looks as if this would be a big refactoring. It would be useful if there are plans to test alternative normalizations and comparisons.

In the meantime, renaming some files and functions could make the intent of the present setup clearer:

• edu.dedupendnote.integration.services.AuthorExperimentsTests
  • promote the inner ExperimentalAuthorsComparator to its own class (and rename it (AuthorsComparatorWithHigherThresholds? AuthorsComparisonServiceWithHigherThresholds? ComparisonServiceAuthorsWithHigherThresholds?) See the NormalizationService[Author|Doi|...]Tests fot the last suggestion
  • rename AuthorExperimentsTests to AuthorsComparisonServiceExperimentsTests?
• edu.dedupendnote.unit.services.AuthorVariantsExperimentsTest rename to NormalizationServiceAuthorExperimentsTests?
• PublicationExperiment.addAuthor... rename to normalizeInputAuthors...?