CompareTemplateOutputHandler.compareText returns incorrect match token

[douglasclarke]

When reviewing the difference messages it appears in failures to match tokens that the message returned is for the token after the failed match.

Spdx-Java-Library/src/main/java/org/spdx/utility/compare/CompareTemplateOutputHandler.java

Line 794 in 0906908

String nextMatchToken = LicenseTextHelper.getTokenAt(matchTokens, matchTokenCounter++);

String nextMatchToken = LicenseTextHelper.getTokenAt(matchTokens, matchTokenCounter++);

This line gets the next token to match but also increments the counter so that on line 851 when the negative value of this is returned the result message identifies the token after the failed match.

alphabet = "a b c d e f g h i j k l m n o p q r s t u v w x y z" template = "a b c d e f g h i j k l m n o p q r s t u v w x y zebra was here"

When comparing these two the DifferenceDescription message appears as:
Normal text of license does not match starting at line #1 column #58 "was" when comparing to template text "a b c d e f g h i j k l m n o p q r s t u v w x y z"
If line 851 returned the index of the tokens that did not match taking into account any of the skipping I believe this would resolve. If time allows I will work on a PR with supporting test cases but wanted to raise the issue first.

[goneall]

Thanks @douglasclarke for reporting this. If you could create a PR, that would be much appreciated.

[douglasclarke]

Still working through test scenarios but for my initial tests its just

diff --git a/src/main/java/org/spdx/utility/compare/CompareTemplateOutputHandler.java b/src/main/java/org/spdx/utility/compare/CompareTemplateOutputHandler.java
index 1b4637f4..1bd0ac53 100644
--- a/src/main/java/org/spdx/utility/compare/CompareTemplateOutputHandler.java
+++ b/src/main/java/org/spdx/utility/compare/CompareTemplateOutputHandler.java
@@ -217,7 +217,7 @@ public class CompareTemplateOutputHandler implements
                                        }
                                        nextToken = compareText(textTokens, matchTokens, nextToken, this);
                                        if (nextToken < 0) {
-                                               int errorLocation = -nextToken;
+                                               int errorLocation = -nextToken - 1;
                                                differences.addDifference(tokenToLocation.get(errorLocation), LicenseTextHelper.getTokenAt(matchTokens, errorLocation), 
                                                                                "Normal text of license does not match", text, null, getLastOptionalDifference());
                                        }

[goneall]

Thanks @douglasclarke - That should work

[douglasclarke]

I am getting some mixed results as I change versions. Will firm up some test cases and provide PR or withdraw the bug

[goneall]

@douglasclarke - I'm going to do a release over the next couple of days - just checking in on progress. No pressure - we can always include fixes in a following release.

[douglasclarke]

ok. I attempted a simple fix with 2 test cases to validate

#310

If this works for the release that is great. If you believe the fix is more involved its probably fine to delay for next.

[goneall]

Fixed with #310