Set default index mode for TimeSeries to `null` #98586

kkrik-es · 2023-08-17T12:22:50Z

If the default index mode matches the specified, we skip printing the synthetic source info in mappings. This leads to confusion as it's not immediately visible (or well known) that time series indices use synthetic source by default.

Leaving the default index mode to null does the trick here. We do pass the right value for time series indexes while building the mapping so there's no other functional impact.

Fixes #97429

If the default index mode matches the specified, we skip printing the synthetic source info in the mappings printing. This leads to confusion as it's not immediately visible (or well known) that time series indices use synthetic source by default. Leaving the default index mode to null does the trick here. We do pass the right value for time series indexes while building the mapping so there's no functional impact here. Fixes elastic#97429

elasticsearchmachine · 2023-08-17T12:23:37Z

Hi @kkrik-es, I've created a changelog YAML for you.

elasticsearchmachine · 2023-08-17T14:19:52Z

Pinging @elastic/es-analytics-geo (Team:Analytics)

kkrik-es · 2023-08-21T08:39:10Z

@elasticsearchmachine run elasticsearch-ci/bwc

server/src/main/java/org/elasticsearch/index/mapper/SourceFieldMapper.java

martijnvg

Left one comment about the assertion. Otherwise looks good.

server/src/main/java/org/elasticsearch/index/mapper/DocumentMapper.java

martijnvg · 2023-08-23T03:43:29Z

server/src/main/java/org/elasticsearch/index/mapper/DocumentMapper.java

+             * This got restored in v.8.9 (and patched in v.8.8) to avoid confusion. The change is only restricted to
+             * mapping printout, it has no functional effect as the synthetic source already applies.
+             */
+            this.mappingSource = mapping.toCompressedXContent();


By doing this.mappingSource = mapping.toCompressedXContent(); we are changing the mapping source to contain the missing _source: mode: synthetic right?

If we would do this.mappingSource = source; here, then the that would remain to be missing?

Correct. I was thinking of storing a source that matches this output to avoid surprises further down the processing pipeline, but up to you.

martijnvg

LGTM

* Default index mode null for TimeSeries If the default index mode matches the specified, we skip printing the synthetic source info in the mappings printing. This leads to confusion as it's not immediately visible (or well known) that time series indices use synthetic source by default. Leaving the default index mode to null does the trick here. We do pass the right value for time series indexes while building the mapping so there's no functional impact here. Fixes elastic#97429 * Update docs/changelog/98586.yaml * Restore other error messages. * Update source in DocumentMapper to include synthetic source. * Add version check for skipping assert

elasticsearchmachine · 2023-08-23T09:32:00Z

💚 Backport successful

Status	Branch	Result
✅	8.10

* Default index mode null for TimeSeries If the default index mode matches the specified, we skip printing the synthetic source info in the mappings printing. This leads to confusion as it's not immediately visible (or well known) that time series indices use synthetic source by default. Leaving the default index mode to null does the trick here. We do pass the right value for time series indexes while building the mapping so there's no functional impact here. Fixes #97429 * Update docs/changelog/98586.yaml * Restore other error messages. * Update source in DocumentMapper to include synthetic source. * Add version check for skipping assert

This reverts commit 56abb86.

* Skip segment for MatchNoDocsQuery filters. When a query of a filter gets rewritten to MatchNoDocsQuery, segments will not produce any results. We can therefore skip processing such segments. This applies to FilterByFilterAggregator and FiltersAggregator, as well as to TermsAggregator when it uses StringTermsAggregatorFromFilters internally; the latter is an adapter aggregator to FilterByFilterAggregator. Fixes #94637 * Update docs/changelog/98295.yaml * Check all filters for `MatchNoDocsQuery`. * Skip optimization when 'other' bucket is requested. * Revert "Set default index mode for TimeSeries to `null` (#98586)" This reverts commit 56abb86.

* Skip segment for MatchNoDocsQuery filters. When a query of a filter gets rewritten to MatchNoDocsQuery, segments will not produce any results. We can therefore skip processing such segments. This applies to FilterByFilterAggregator and FiltersAggregator, as well as to TermsAggregator when it uses StringTermsAggregatorFromFilters internally; the latter is an adapter aggregator to FilterByFilterAggregator. Fixes elastic#94637 * Update docs/changelog/98295.yaml * Check all filters for `MatchNoDocsQuery`. * Skip optimization when 'other' bucket is requested. * Revert "Set default index mode for TimeSeries to `null` (elastic#98586)" This reverts commit 56abb86.

* Skip segment for MatchNoDocsQuery filters. When a query of a filter gets rewritten to MatchNoDocsQuery, segments will not produce any results. We can therefore skip processing such segments. This applies to FilterByFilterAggregator and FiltersAggregator, as well as to TermsAggregator when it uses StringTermsAggregatorFromFilters internally; the latter is an adapter aggregator to FilterByFilterAggregator. Fixes #94637 * Update docs/changelog/98295.yaml * Check all filters for `MatchNoDocsQuery`. * Skip optimization when 'other' bucket is requested. * Revert "Set default index mode for TimeSeries to `null` (#98586)" This reverts commit 56abb86.

This reverts commit e370194.

* Skip segment for MatchNoDocsQuery filters. When a query of a filter gets rewritten to MatchNoDocsQuery, segments will not produce any results. We can therefore skip processing such segments. This applies to FilterByFilterAggregator and FiltersAggregator, as well as to TermsAggregator when it uses StringTermsAggregatorFromFilters internally; the latter is an adapter aggregator to FilterByFilterAggregator. Fixes #94637 * Update docs/changelog/98295.yaml * Check all filters for `MatchNoDocsQuery`. * Skip optimization when 'other' bucket is requested. * Revert "Set default index mode for TimeSeries to `null` (#98586)" This reverts commit 56abb86. * Revert "Rollback of #98586 (#98805)" This reverts commit e370194. * Skip updating source when missing synthetic mode * Update docs/changelog/98808.yaml * Skip matching assert in MapperService too * Refine the assert * Extend versions before 8.6, when TS had no synthetic source * Add source field mapping for non-synthetic TSDB * Delete 98586.yaml Duplicate changelog * Add comment to TSDB_NO_SYNTHETIC mapping * Spotless fix * Add yaml test * Fix version skip in yaml test

* Skip segment for MatchNoDocsQuery filters. When a query of a filter gets rewritten to MatchNoDocsQuery, segments will not produce any results. We can therefore skip processing such segments. This applies to FilterByFilterAggregator and FiltersAggregator, as well as to TermsAggregator when it uses StringTermsAggregatorFromFilters internally; the latter is an adapter aggregator to FilterByFilterAggregator. Fixes elastic#94637 * Update docs/changelog/98295.yaml * Check all filters for `MatchNoDocsQuery`. * Skip optimization when 'other' bucket is requested. * Revert "Set default index mode for TimeSeries to `null` (elastic#98586)" This reverts commit 56abb86. * Revert "Rollback of elastic#98586 (elastic#98805)" This reverts commit e370194. * Skip updating source when missing synthetic mode * Update docs/changelog/98808.yaml * Skip matching assert in MapperService too * Refine the assert * Extend versions before 8.6, when TS had no synthetic source * Add source field mapping for non-synthetic TSDB * Delete 98586.yaml Duplicate changelog * Add comment to TSDB_NO_SYNTHETIC mapping * Spotless fix * Add yaml test * Fix version skip in yaml test

* Skip segment for MatchNoDocsQuery filters. When a query of a filter gets rewritten to MatchNoDocsQuery, segments will not produce any results. We can therefore skip processing such segments. This applies to FilterByFilterAggregator and FiltersAggregator, as well as to TermsAggregator when it uses StringTermsAggregatorFromFilters internally; the latter is an adapter aggregator to FilterByFilterAggregator. Fixes #94637 * Update docs/changelog/98295.yaml * Check all filters for `MatchNoDocsQuery`. * Skip optimization when 'other' bucket is requested. * Revert "Set default index mode for TimeSeries to `null` (#98586)" This reverts commit 56abb86. * Revert "Rollback of #98586 (#98805)" This reverts commit e370194. * Skip updating source when missing synthetic mode * Update docs/changelog/98808.yaml * Skip matching assert in MapperService too * Refine the assert * Extend versions before 8.6, when TS had no synthetic source * Add source field mapping for non-synthetic TSDB * Delete 98586.yaml Duplicate changelog * Add comment to TSDB_NO_SYNTHETIC mapping * Spotless fix * Add yaml test * Fix version skip in yaml test

* Skip segment for MatchNoDocsQuery filters. When a query of a filter gets rewritten to MatchNoDocsQuery, segments will not produce any results. We can therefore skip processing such segments. This applies to FilterByFilterAggregator and FiltersAggregator, as well as to TermsAggregator when it uses StringTermsAggregatorFromFilters internally; the latter is an adapter aggregator to FilterByFilterAggregator. Fixes elastic#94637 * Update docs/changelog/98295.yaml * Check all filters for `MatchNoDocsQuery`. * Skip optimization when 'other' bucket is requested. * Revert "Set default index mode for TimeSeries to `null` (elastic#98586)" This reverts commit 56abb86. * Revert "Rollback of elastic#98586 (elastic#98805)" This reverts commit e370194. * Skip updating source when missing synthetic mode * Update docs/changelog/98808.yaml * Skip matching assert in MapperService too * Refine the assert * Extend versions before 8.6, when TS had no synthetic source * Add source field mapping for non-synthetic TSDB * Delete 98586.yaml Duplicate changelog * Add comment to TSDB_NO_SYNTHETIC mapping * Spotless fix * Add yaml test * Fix version skip in yaml test

kkrik-es added >bug :Analytics/Aggregations Aggregations Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) labels Aug 17, 2023

kkrik-es self-assigned this Aug 17, 2023

elasticsearchmachine added the v8.11.0 label Aug 17, 2023

Update docs/changelog/98586.yaml

d5f478a

kkrik-es added 3 commits August 17, 2023 16:24

Merge branch 'main' into fix/97429

c8bb5fb

Restore other error messages.

3b76e7c

Merge remote-tracking branch 'upstream/fix/97429' into fix/97429

c661a32

kkrik-es marked this pull request as ready for review August 17, 2023 14:19

kkrik-es requested a review from martijnvg August 17, 2023 14:19

Merge branch 'main' into fix/97429

570928d

Update source in DocumentMapper to include synthetic source.

f5b8948

kkrik-es added v8.9.0 v8.8.3 v8.9.2 auto-backport Automatically create backport pull requests when merged and removed v8.9.0 labels Aug 21, 2023

Merge branch 'main' into fix/97429

ecbb4f2

martijnvg reviewed Aug 22, 2023

View reviewed changes

server/src/main/java/org/elasticsearch/index/mapper/SourceFieldMapper.java Show resolved Hide resolved

martijnvg reviewed Aug 22, 2023

View reviewed changes

server/src/main/java/org/elasticsearch/index/mapper/DocumentMapper.java Outdated Show resolved Hide resolved

kkrik-es added v8.10.1 and removed v8.8.3 v8.9.2 labels Aug 22, 2023

Add version check for skipping assert

dfcef4f

martijnvg reviewed Aug 23, 2023

View reviewed changes

martijnvg approved these changes Aug 23, 2023

View reviewed changes

kkrik-es merged commit 56abb86 into elastic:main Aug 23, 2023

kkrik-es mentioned this pull request Aug 23, 2023

[8.10] Set default index mode for TimeSeries to null (#98586) #98780

Merged

kkrik-es deleted the fix/97429 branch August 23, 2023 10:44

thecoop mentioned this pull request Aug 23, 2023

[CI] MixedClusterClientYamlTestSuiteIT test {p0=tsdb/15_timestamp_mapping/explicitly enable timestamp meta field} failing #98803

Closed

kkrik-es restored the fix/97429 branch August 23, 2023 14:09

kkrik-es deleted the fix/97429 branch August 23, 2023 14:11

kkrik-es restored the fix/97429 branch August 23, 2023 14:16

kkrik-es added a commit to kkrik-es/elasticsearch that referenced this pull request Aug 23, 2023

Revert "Set default index mode for TimeSeries to null (elastic#98586)"

d22c37f

This reverts commit 56abb86.

kkrik-es mentioned this pull request Aug 23, 2023

[8.10] Rollback of #98586 (#98805) #98806

Merged

kkrik-es added a commit to kkrik-es/elasticsearch that referenced this pull request Aug 23, 2023

Revert "Rollback of elastic#98586 (elastic#98805)"

1b0eaf3

This reverts commit e370194.

kkrik-es mentioned this pull request Aug 23, 2023

Set default index mode for TimeSeries to null #98808

Merged

JVerwolf added v8.10.0 and removed v8.10.1 labels Aug 31, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Set default index mode for TimeSeries to `null` #98586

Set default index mode for TimeSeries to `null` #98586

Uh oh!

kkrik-es commented Aug 17, 2023 •

edited

Loading

Uh oh!

elasticsearchmachine commented Aug 17, 2023

Uh oh!

elasticsearchmachine commented Aug 17, 2023

Uh oh!

kkrik-es commented Aug 21, 2023

Uh oh!

Uh oh!

martijnvg left a comment

Uh oh!

Uh oh!

martijnvg Aug 23, 2023

Uh oh!

kkrik-es Aug 23, 2023

Uh oh!

martijnvg left a comment

Uh oh!

elasticsearchmachine commented Aug 23, 2023

Uh oh!

Uh oh!

Set default index mode for TimeSeries to null #98586

Set default index mode for TimeSeries to null #98586

Uh oh!

Conversation

kkrik-es commented Aug 17, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticsearchmachine commented Aug 17, 2023

Uh oh!

elasticsearchmachine commented Aug 17, 2023

Uh oh!

kkrik-es commented Aug 21, 2023

Uh oh!

Uh oh!

martijnvg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

martijnvg Aug 23, 2023

Choose a reason for hiding this comment

Uh oh!

kkrik-es Aug 23, 2023

Choose a reason for hiding this comment

Uh oh!

martijnvg left a comment

Choose a reason for hiding this comment

Uh oh!

elasticsearchmachine commented Aug 23, 2023

💚 Backport successful

Uh oh!

Uh oh!

Set default index mode for TimeSeries to `null` #98586

Set default index mode for TimeSeries to `null` #98586

kkrik-es commented Aug 17, 2023 •

edited

Loading