Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to Apache Lucene 10 for 3.0.0 #16366

Merged
merged 117 commits into from
Jan 22, 2025
Merged
Changes from 1 commit
Commits
Show all changes
117 commits
Select commit Hold shift + click to select a range
fb9623b
Update to Apache Lucene 10 for 3.0.0
reta Oct 17, 2024
1afb95a
More Lucene 10 build fixes
msfroh Nov 12, 2024
e6ad633
Merge remote-tracking branch 'origin/main' into issue-11415
reta Nov 12, 2024
b284053
Fix more compilation failures
reta Nov 12, 2024
213f9c0
Fix more compilation failures
reta Nov 13, 2024
560116c
Merge remote-tracking branch 'origin/main' into issue-11415
reta Nov 13, 2024
06e90f0
Fix more compilation failures
reta Nov 13, 2024
1ab1a5f
Fix more compilation failures
andrross Nov 13, 2024
0fafd24
Replace DocValuesFieldExistsQuery with FieldExistsQuery
andrross Nov 13, 2024
ce3af73
Fix signature changes in o.a.l.search.BooleanQuery
andrross Nov 13, 2024
77dbc74
Fix more compilation failures, update shadow Gradle plugin and fix lo…
reta Nov 14, 2024
39749bf
Fix more compilation failures
reta Nov 18, 2024
52fddbb
Fix analysis-common compilation issues
reta Nov 18, 2024
965a4ad
Fix more compilation failures
reta Nov 18, 2024
90dbe1a
Merge remote-tracking branch 'upstream/main' into issue-11415
andrross Nov 19, 2024
7af891d
Fix more compilation failures
reta Nov 19, 2024
111029d
More compilation fixes
andrross Nov 19, 2024
8064d06
More compilation fixes
andrross Nov 20, 2024
d09cb90
Fix assemble tasks
reta Nov 20, 2024
cfba336
Fix more compilation failures
reta Nov 20, 2024
e167a0d
Fix more compilation failures, add CHANGELOG.md
reta Nov 20, 2024
d263774
Fix more compilation failures
reta Nov 20, 2024
19c2956
Fixed Lucene latest codec name
reta Nov 20, 2024
ae12a81
Refactor preload tests in FsDirectoryFactoryTests
andrross Nov 20, 2024
3bb74c1
Fix more compilation failures
andrross Nov 20, 2024
b0a89a4
Fix precommit failures
andrross Nov 20, 2024
f971515
Fix analysis test cases
reta Nov 21, 2024
66c0616
Fix FieldData & ParentJoinAggregator ordinal traversal
reta Nov 21, 2024
5b9e939
Fix IpFieldMapper & CollapsingDocValuesSource ordinal traversal
reta Nov 21, 2024
28a18b2
Fix MultiValueMode, GlobalOrdinalValuesSource, BinaryRangeAggregator,…
reta Nov 21, 2024
f9ac2bb
Fix MultiValueModeTests, BinaryRangeAggregatorTests, IncludeExcludeTe…
reta Nov 21, 2024
9fea439
Fix ICUCollationKeywordFieldMapper & VectorFieldTests tests
reta Nov 21, 2024
9cc86b1
Merge remote-tracking branch 'origin/main' into issue-11415
reta Nov 26, 2024
d0a4727
Merge remote-tracking branch 'origin/main' into issue-11415
reta Nov 29, 2024
a265dc0
Fix org.opensearch.common.xcontent.support.XContentMapValuesTests tes…
reta Dec 2, 2024
2bf22a1
Fix FunctionScoreQuery::CustomBoostFactorWeight::scorerSupplier that …
reta Dec 2, 2024
8b51a8d
Fix CodecTests and non-concurrent ContextIndexSearcher::search implem…
reta Dec 3, 2024
8ef5ce1
Fixed GlobalOrdinalMapping ordinals iteration and SortedWiderNumericS…
reta Dec 3, 2024
9074f26
Merge remote-tracking branch 'origin/main' into issue-11415
reta Dec 3, 2024
d6a9ea3
Fix FieldData and SortedSetDocValuesWriterWrapperTests
reta Dec 3, 2024
6550715
Fix org.opensearch.repositories.azure.AzureBlobStoreRepositoryTests t…
reta Dec 3, 2024
e14df71
Fix org.opensearch.search.aggregations.bucket.composite.CompositeAggr…
reta Dec 4, 2024
61ee1b4
Fix MultiValueMode ordinal traversal
reta Dec 4, 2024
d44b0d8
Fix GlobalOrdinalMapping and GlobalOrdinalsStringTermsAggregator ordi…
reta Dec 4, 2024
84818f8
Fix QueryStringQueryBuilderTests and QueryStringQueryBuilder fuzzy qu…
reta Dec 5, 2024
e26e7db
Fix StarTree*DocValueFormatTests
msfroh Dec 13, 2024
040e121
Fix various MultiTermQuery-related tests
msfroh Dec 13, 2024
ea509df
Fix Spotless
msfroh Dec 13, 2024
0fb9e04
Fix RegEx automaton processing using Operations.determinize(...)
reta Dec 16, 2024
660a3ff
Merge remote-tracking branch 'origin/main' into issue-11415
reta Dec 16, 2024
1c25393
Fix compilation issues (post merge)
reta Dec 16, 2024
6c154d5
Fix UnsignedLongComparator::sortableBytesToLong encoding
reta Dec 16, 2024
2500fce
Merge remote-tracking branch 'origin/main' into issue-11415
reta Dec 17, 2024
510f765
Fix HalfFloatComparator::sortableBytesToLong and HalfFloatComparator:…
reta Dec 17, 2024
67d17c1
Fix infinite recursion in CollapsingTopDocsCollectorTests
msfroh Dec 17, 2024
15dd0e8
Fix SingleDimensionValuesSourceTests
msfroh Dec 17, 2024
92e2e54
Fix a few more MultiTermQuery-related test failures
msfroh Dec 17, 2024
bae0338
Merge branch 'main' into issue-11415
reta Dec 18, 2024
b633132
Fixed QueryPhaseTests#minScoreOptimisation by not delegating setWeigh…
expani Dec 18, 2024
b2ac498
Merge pull request #201 from expani/issue-11415-clone
reta Dec 18, 2024
59e7d03
Fix DerivedFieldMapperQueryTests
msfroh Dec 18, 2024
8861111
Fix SortedNumericDocValuesWriterWrapperTests
msfroh Dec 18, 2024
7ca821a
Fix IpRangeAggregatorTests
msfroh Dec 19, 2024
df31192
Fix IpFieldMapperTests
msfroh Dec 19, 2024
b780ac5
Fix MissingValues ordinal traversal and MultiOrdinalsTests tests
reta Dec 19, 2024
fd14e80
Prevent cascading test failures in SearchServiceTests
msfroh Dec 19, 2024
ea0549b
Fix DerivedFieldTypeTests
msfroh Dec 19, 2024
3d0b52d
Fix TransferManagerRemoteDirectoryReaderTests
msfroh Dec 19, 2024
fda70e4
Add missing javadoc
reta Dec 19, 2024
17ecd4c
Merge remote-tracking branch 'origin/main' into issue-11415
reta Dec 19, 2024
2ae505b
Uploaded Index generated from Lucene 8.x ( OpenSearch 1.3.0 ) to be u…
expani Dec 19, 2024
737ab3a
Moved path to a constant
expani Dec 19, 2024
0874389
Merge pull request #202 from expani/issue-11415-2
reta Dec 20, 2024
3549c2d
Update Apache Lucene to 10.1.0
reta Dec 20, 2024
60b781b
Fixed unit tests for lucene 10 upgrade
expani Dec 21, 2024
5871134
Merge pull request #203 from expani/issue-11415-3
reta Dec 21, 2024
2c35bfc
Fix posting format for CompletionFieldMapper
reta Dec 23, 2024
72cff6d
Fix org.opensearch.search.SearchServiceTests test cases
reta Dec 23, 2024
e3973eb
Removed extra iteration of DISI and added stored field option to bypa…
expani Dec 23, 2024
9e13d1e
Changed to use advance and init to use docId() instead of nextDoc()
expani Dec 23, 2024
a74e65e
Merge pull request #204 from expani/issue-11415-4
reta Dec 23, 2024
b5ae870
Increased number of indexed docs to match segment count
expani Dec 24, 2024
285dd26
Reduced segment count to match assertion instead of increasing docs
expani Dec 24, 2024
2ce5cfb
Merge pull request #205 from expani/issue-11415-5
reta Dec 24, 2024
a292351
Revert to make 90_search_after integ pass as hits are not shown
expani Dec 26, 2024
1d66ec7
Making Half Float failures work with it's conversions
expani Dec 27, 2024
2a2c955
Changing max long to signed from unsigned
expani Dec 27, 2024
d50e56e
Addressed PR Comments
expani Dec 27, 2024
7bc5cbf
Trying another approach for gracefully handling unsigned long convers…
expani Dec 27, 2024
54ac23e
Simplified unsigned to signed long conversion
expani Dec 27, 2024
598b9af
Reverted changes as need to think more on this
expani Dec 27, 2024
eb4cb71
Merge pull request #206 from expani/issue-11415-6
reta Dec 27, 2024
2bd2256
Merge remote-tracking branch 'origin/main' into issue-11415
reta Dec 27, 2024
a0e043b
Fix azure-fixture & s3-fixture Docker images
reta Dec 27, 2024
dfb3465
Fix 'search/90_search_after/numeric skipping logic with competitive m…
reta Dec 27, 2024
9bd7512
Fix gcs-fixture Docker images
reta Dec 27, 2024
bce495b
Updated to use modern Romanian Unicode
expani Dec 30, 2024
6b871a0
Merge pull request #207 from expani/issue-11415-7
reta Dec 30, 2024
15803bd
Fix org.opensearch.wildfly.WildflyIT test cases
reta Dec 31, 2024
9c78a53
Fix org.opensearch.search.aggregations.bucket.MinDocCountIT test cases
reta Jan 2, 2025
3ab4fad
Merge remote-tracking branch 'origin/main' into issue-11415
reta Jan 2, 2025
391a6b9
Fix CardinalityAggregator.OrdinalsCollector ordinals traversal
reta Jan 2, 2025
6613391
Fix org.opensearch.search.query.ScriptScoreQueryIT test cases
reta Jan 2, 2025
7ff9330
Fix org.opensearch.search.scriptfilter.ScriptQuerySearchIT test cases
reta Jan 2, 2025
1be79a4
Merge remote-tracking branch 'origin/main' into issue-11415
reta Jan 3, 2025
4d275b1
Fix UnsignedLongComparator top/bottom as comparable long conversions
reta Jan 6, 2025
a51b30e
Merge remote-tracking branch 'origin/main' into issue-11415
reta Jan 6, 2025
4592e49
Fix post-merge (with main) compilation issues
reta Jan 6, 2025
837fb05
Fix UnsignedLongComparator with respect to sortable bytes to long con…
reta Jan 7, 2025
6437317
Merge remote-tracking branch 'origin/main' into issue-11415
reta Jan 7, 2025
c6f7b90
Merge remote-tracking branch 'origin/main' into issue-11415
reta Jan 10, 2025
ad8b3dc
Merge remote-tracking branch 'origin/main' into issue-11415
reta Jan 10, 2025
1d225f9
Merge remote-tracking branch 'origin/main' into issue-11415
reta Jan 15, 2025
a8d43d2
Merge remote-tracking branch 'origin/main' into issue-11415
reta Jan 15, 2025
72496b0
Merge remote-tracking branch 'origin/main' into issue-11415
reta Jan 21, 2025
4c0b91e
Merge remote-tracking branch 'origin/main' into issue-11415
reta Jan 22, 2025
e5516fc
Fix FlatObjectFieldTypeTests test cases
reta Jan 22, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix IpFieldMapperTests
I broke these tests as a side-effect for my fix to
DerivedFieldMapperQueryTests.

Signed-off-by: Michael Froh <froh@amazon.com>
  • Loading branch information
msfroh committed Dec 19, 2024
commit df31192c18ba7bc12b4a0915bdfcf9acc11f9739
Original file line number Diff line number Diff line change
Expand Up @@ -84,16 +84,13 @@ public void testDefaults() throws Exception {
ParsedDocument doc = mapper.parse(source(b -> b.field("field", "::1")));

IndexableField[] fields = doc.rootDoc().getFields("field");
assertEquals(2, fields.length);
IndexableField pointField = fields[0];
assertEquals(1, pointField.fieldType().pointIndexDimensionCount());
assertEquals(16, pointField.fieldType().pointNumBytes());
assertFalse(pointField.fieldType().stored());
assertEquals(new BytesRef(InetAddressPoint.encode(InetAddresses.forString("::1"))), pointField.binaryValue());
IndexableField dvField = fields[1];
assertEquals(DocValuesType.SORTED_SET, dvField.fieldType().docValuesType());
assertEquals(new BytesRef(InetAddressPoint.encode(InetAddresses.forString("::1"))), dvField.binaryValue());
assertFalse(dvField.fieldType().stored());
assertEquals(1, fields.length);
IndexableField pointFieldAndDVField = fields[0];
assertEquals(1, pointFieldAndDVField.fieldType().pointIndexDimensionCount());
assertEquals(16, pointFieldAndDVField.fieldType().pointNumBytes());
assertFalse(pointFieldAndDVField.fieldType().stored());
assertEquals(new BytesRef(InetAddressPoint.encode(InetAddresses.forString("::1"))), pointFieldAndDVField.binaryValue());
assertEquals(DocValuesType.SORTED_SET, pointFieldAndDVField.fieldType().docValuesType());
}

public void testNotIndexed() throws Exception {
Expand Down Expand Up @@ -145,12 +142,11 @@ public void testStore() throws Exception {
ParsedDocument doc = mapper.parse(source(b -> b.field("field", "::1")));

IndexableField[] fields = doc.rootDoc().getFields("field");
assertEquals(3, fields.length);
IndexableField pointField = fields[0];
assertEquals(1, pointField.fieldType().pointIndexDimensionCount());
IndexableField dvField = fields[1];
assertEquals(DocValuesType.SORTED_SET, dvField.fieldType().docValuesType());
IndexableField storedField = fields[2];
assertEquals(2, fields.length);
IndexableField pointFieldAndDVField = fields[0];
assertEquals(1, pointFieldAndDVField.fieldType().pointIndexDimensionCount());
assertEquals(DocValuesType.SORTED_SET, pointFieldAndDVField.fieldType().docValuesType());
IndexableField storedField = fields[1];
assertTrue(storedField.fieldType().stored());
assertEquals(new BytesRef(InetAddressPoint.encode(InetAddress.getByName("::1"))), storedField.binaryValue());
}
Expand Down Expand Up @@ -190,16 +186,13 @@ public void testNullValue() throws IOException {
doc = mapper.parse(source(b -> b.nullField("field")));

IndexableField[] fields = doc.rootDoc().getFields("field");
assertEquals(2, fields.length);
IndexableField pointField = fields[0];
assertEquals(1, pointField.fieldType().pointIndexDimensionCount());
assertEquals(16, pointField.fieldType().pointNumBytes());
assertFalse(pointField.fieldType().stored());
assertEquals(new BytesRef(InetAddressPoint.encode(InetAddresses.forString("::1"))), pointField.binaryValue());
IndexableField dvField = fields[1];
assertEquals(DocValuesType.SORTED_SET, dvField.fieldType().docValuesType());
assertEquals(new BytesRef(InetAddressPoint.encode(InetAddresses.forString("::1"))), dvField.binaryValue());
assertFalse(dvField.fieldType().stored());
assertEquals(1, fields.length);
IndexableField pointFieldAndDVField = fields[0];
assertEquals(1, pointFieldAndDVField.fieldType().pointIndexDimensionCount());
assertEquals(16, pointFieldAndDVField.fieldType().pointNumBytes());
assertFalse(pointFieldAndDVField.fieldType().stored());
assertEquals(new BytesRef(InetAddressPoint.encode(InetAddresses.forString("::1"))), pointFieldAndDVField.binaryValue());
assertEquals(DocValuesType.SORTED_SET, pointFieldAndDVField.fieldType().docValuesType());

mapper = createDocumentMapper(fieldMapping(b -> {
b.field("type", "ip");
Expand Down
Loading