add a feature that flattens custom result index when enabled #1401

jackiehanyang · 2025-01-15T02:46:11Z

Description

wip - sending implementation out first.

Related Issues

Resolves #[Issue number to be closed when this PR is merged]

Check List

New functionality includes testing.
New functionality has been documented.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Jackie Han <jkhanjob@gmail.com>

Signed-off-by: Jackie Han <hnyng@amazon.com>

jackiehanyang · 2025-01-24T00:47:09Z

resolved
~~non related IT test case failed:~~
REPRODUCE WITH: ./gradlew ':integTest' --tests "org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testSearchTopAnomalyResultsWithCustomResultIndex" -Dtests.seed=764AE145CE6907EA -Dtests.security.manager=false -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=es-419 -Dtests.timezone=NZ -Druntime.java=21
org.opensearch.ad.rest.AnomalyDetectorRestApiIT > testSearchTopAnomalyResultsWithCustomResultIndex FAILED
org.opensearch.client.ResponseException: method [POST], host [http://127.0.0.1:44973/], URI [/_plugins/_anomaly_detection/detectors], status line [HTTP/1.1 500 Internal Server Error]
{"error":{"root_cause":[{"type":"status_exception","reason":"Fail to create detector"}],"type":"status_exception","reason":"Fail to create detector"},"status":500}
at __randomizedtesting.SeedInfo.seed([764AE145CE6907EA:A9D4336811E01C9A]:0)
at app//org.opensearch.client.RestClient.convertResponse(RestClient.java:501)
at app//org.opensearch.client.RestClient.performRequest(RestClient.java:384)
at app//org.opensearch.client.RestClient.performRequest(RestClient.java:359)
at app//org.opensearch.timeseries.TestHelpers.makeRequest(TestHelpers.java:231)
at app//org.opensearch.timeseries.TestHelpers.makeRequest(TestHelpers.java:204)
at app//org.opensearch.ad.AnomalyDetectorRestTestCase.createAnomalyDetector(AnomalyDetectorRestTestCase.java:133)
at app//org.opensearch.ad.rest.AnomalyDetectorRestApiIT.testSearchTopAnomalyResultsWithCustomResultIndex(AnomalyDetectorRestApiIT.java:2069)

src/main/java/org/opensearch/ad/settings/AnomalyDetectorSettings.java

src/main/java/org/opensearch/timeseries/indices/IndexManagement.java

src/main/java/org/opensearch/ad/transport/ADResultBulkTransportAction.java

src/main/java/org/opensearch/timeseries/rest/handler/AbstractTimeSeriesActionHandler.java

src/main/java/org/opensearch/timeseries/indices/IndexManagement.java

src/main/java/org/opensearch/timeseries/rest/handler/AbstractTimeSeriesActionHandler.java

src/test/java/org/opensearch/ad/rest/AnomalyDetectorRestApiIT.java

Signed-off-by: Jackie Han <hnyng@amazon.com>

kaituo · 2025-01-28T18:23:06Z

bwc test failure is similar to opensearch-project/OpenSearch#15234

=== Standard error of node `node{::adBwcCluster#twoThirdsUpgradedClusterTask-0}` ===
?  	at org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:393) [opensearch-2.19.0.jar:2.19.0]
?  	... 41 more
?  Caused by: java.lang.IllegalStateException: unexpected byte [0xac]
?  	at org.opensearch.core.common.io.stream.StreamInput.readBoolean(StreamInput.java:596) ~[opensearch-core-2.19.0.jar:2.19.0]
?  	at org.opensearch.core.common.io.stream.StreamInput.readOptionalBoolean(StreamInput.java:606) ~[opensearch-core-2.19.0.jar:2.19.0]
?  	at org.opensearch.search.internal.ShardSearchRequest.<init>(ShardSearchRequest.java:255) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.RequestHandlerRegistry.newRequest(RequestHandlerRegistry.java:87) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.newRequest(NativeMessageHandler.java:316) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.handleRequest(NativeMessageHandler.java:271) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.handleMessage(NativeMessageHandler.java:146) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.messageReceived(NativeMessageHandler.java:126) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundHandler.messageReceivedFromPipeline(InboundHandler.java:120) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundHandler.inboundMessage(InboundHandler.java:112) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.TcpTransport.inboundMessage(TcpTransport.java:768) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundBytesHandler.forwardFragments(InboundBytesHandler.java:137) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundBytesHandler.doHandleBytes(InboundBytesHandler.java:77) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:124) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:113) ~[opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:95) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
?  	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
?  	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:107) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
?  	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[?:?]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
?  	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868) ~[?:?]
?  	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[?:?]
?  	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[?:?]
?  	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:689) ~[?:?]
?  	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:652) ~[?:?]
?  	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[?:?]
?  	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[?:?]
?  	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
?  	at java.lang.Thread.run(Thread.java:1583) ~[?:?]
? WARN ][r.suppressed             ] [adBwcCluster0-2] path: /_opendistro/_anomaly_detection/detectors/yHNUrJQBrgHv3CV3UwAC/_start, params: {detectorID=yHNUrJQBrgHv3CV3UwAC}
?  org.opensearch.OpenSearchStatusException: Fail to start detector
?  	at org.opensearch.timeseries.util.RestHandlerUtils.lambda$wrapRestActionListener$2(RestHandlerUtils.java:243) [opensearch-anomaly-detection-2.19.0.0.jar:2.19.0.0]
?  	at org.opensearch.core.action.ActionListener$1.onFailure(ActionListener.java:90) [opensearch-core-2.19.0.jar:2.19.0]
?  	at org.opensearch.core.action.ActionListener$1.onFailure(ActionListener.java:90) [opensearch-core-2.19.0.jar:2.19.0]
?  	at org.opensearch.timeseries.rest.handler.IndexJobActionHandler.lambda$onGetJobForWrite$10(IndexJobActionHandler.java:303) [opensearch-anomaly-detection-2.19.0.0.jar:2.19.0.0]
?  	at org.opensearch.core.action.ActionListener$1.onFailure(ActionListener.java:90) [opensearch-core-2.19.0.jar:2.19.0]
?  	at org.opensearch.timeseries.task.TaskManager.lambda$getAndExecuteOnLatestTasks$17(TaskManager.java:605) [opensearch-anomaly-detection-2.19.0.0.jar:2.19.0.0]
?  	at org.opensearch.core.action.ActionListener$1.onFailure(ActionListener.java:90) [opensearch-core-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.support.TransportAction$1.onFailure(TransportAction.java:124) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.core.action.ActionListener$1.onFailure(ActionListener.java:90) [opensearch-core-2.19.0.jar:2.19.0]
?  	at org.opensearch.core.action.ActionListener$5.onFailure(ActionListener.java:277) [opensearch-core-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.search.AbstractSearchAsyncAction.raisePhaseFailure(AbstractSearchAsyncAction.java:802) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:775) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:395) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:815) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:548) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:316) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.search.SearchExecutionStatsCollector.onFailure(SearchExecutionStatsCollector.java:104) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:75) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:766) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.TransportService$9.handleException(TransportService.java:1741) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1527) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.lambda$handleException$5(NativeMessageHandler.java:454) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.common.util.concurrent.OpenSearchExecutors$DirectExecutorService.execute(OpenSearchExecutors.java:343) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.handleException(NativeMessageHandler.java:452) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.handlerResponseError(NativeMessageHandler.java:444) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.handleMessage(NativeMessageHandler.java:172) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.NativeMessageHandler.messageReceived(NativeMessageHandler.java:126) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundHandler.messageReceivedFromPipeline(InboundHandler.java:120) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundHandler.inboundMessage(InboundHandler.java:112) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.TcpTransport.inboundMessage(TcpTransport.java:768) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundBytesHandler.forwardFragments(InboundBytesHandler.java:137) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundBytesHandler.doHandleBytes(InboundBytesHandler.java:77) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:124) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:113) [opensearch-2.19.0.jar:2.19.0]
?  	at org.opensearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:95) [transport-netty4-client-2.19.0.jar:2.19.0]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280) [netty-handler-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1357) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:868) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:689) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:652) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) [netty-transport-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) [netty-common-4.1.117.Final.jar:4.1.117.Final]
?  	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.117.Final.jar:4.1.117.Final]
?  	at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
?   ? last 40 non error or warning messages from /__w/anomaly-detection/anomaly-detection/build/testclusters/adBwcCluster0-2/logs/opensearch.stdout.log ?
? [2025-01-28T09:53:13,046][DEBUG][o.o.c.c.C.CoordinatorPublication] [adBwcCluster0-2] publishing version 63 to [PublicationTarget{discoveryNode={adBwcCluster0-0}{yxG8heptSiK2eK9S9Ks46A}{mFsI26s-QTuJOWcGDLRAjQ}{127.0.0.1}{127.0.0.1:37589}{dimr}{upgraded=true, testattr=test, shard_indexing_pressure_enabled=true}, state=NOT_STARTED, ackIsPending=true}, PublicationTarget{discoveryNode={adBwcCluster0-1}{th_t3o4ySIqKWHrFfQg35g}{IePje0osQvilETYvk6-fFA}{127.0.0.1}{127.0.0.1:45277}{dimr}{upgraded=true, testattr=test, shard_indexing_pressure_enabled=true}, state=NOT_STARTED, ackIsPending=true}, PublicationTarget{discoveryNode={adBwcCluster0-2}{jpEavNb_SKyKsfA6FVKT0A}{SalIDjFNQGC9PjW8hGFlLQ}{127.0.0.1}{127.0.0.1:33951}{dimr}{testattr=test, shard_indexing_pressure_enabled=true}, state=NOT_STARTED, ackIsPending=true}]
? [2025-01-28T09:53:13,046][DEBUG][o.o.c.c.PublicationTransportHandler] [adBwcCluster0-2] received diff cluster state version [63] with uuid [_1E3aVONQ_6Q0Oj7pl9LVg], diff size [589]

src/main/java/org/opensearch/timeseries/indices/IndexManagement.java

src/main/java/org/opensearch/timeseries/rest/handler/AbstractTimeSeriesActionHandler.java

Signed-off-by: Jackie Han <hnyng@amazon.com>

src/test/java/org/opensearch/ad/rest/AnomalyDetectorRestApiIT.java

src/main/java/org/opensearch/timeseries/rest/handler/AbstractTimeSeriesActionHandler.java

Signed-off-by: Jackie Han <hnyng@amazon.com>

src/main/java/org/opensearch/timeseries/rest/handler/AbstractTimeSeriesActionHandler.java

src/test/java/org/opensearch/ad/rest/AnomalyDetectorRestApiIT.java

…ult index Signed-off-by: Jackie Han <hnyng@amazon.com>

src/main/java/org/opensearch/timeseries/indices/IndexManagement.java

src/main/java/org/opensearch/ad/transport/ADResultBulkTransportAction.java

Signed-off-by: Jackie Han <hnyng@amazon.com>

codecov · 2025-01-29T10:32:01Z

Codecov Report

Attention: Patch coverage is 78.46154% with 42 lines in your changes missing coverage. Please review.

Project coverage is 81.54%. Comparing base (926b2e1) to head (8a9429d).
Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
...opensearch/timeseries/indices/IndexManagement.java	60.00%	11 Missing and 3 partials ⚠️
.../rest/handler/AbstractTimeSeriesActionHandler.java	86.27%	11 Missing and 3 partials ⚠️
...g/opensearch/timeseries/util/RestHandlerUtils.java	36.36%	6 Missing and 1 partial ⚠️
...transport/ADSingleStreamResultTransportAction.java	0.00%	2 Missing ⚠️
...ort/ForecastSingleStreamResultTransportAction.java	0.00%	2 Missing ⚠️
...n/java/org/opensearch/timeseries/model/Config.java	75.00%	1 Missing and 1 partial ⚠️
.../forecast/ratelimit/ForecastResultWriteWorker.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #1401      +/-   ##
============================================
- Coverage     81.58%   81.54%   -0.05%     
- Complexity     5826     5857      +31     
============================================
  Files           533      533              
  Lines         23522    23675     +153     
  Branches       2367     2375       +8     
============================================
+ Hits          19191    19305     +114     
- Misses         3175     3204      +29     
- Partials       1156     1166      +10

Flag	Coverage Δ
plugin	`81.54% <78.46%> (-0.05%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...a/org/opensearch/ad/indices/ADIndexManagement.java	`84.61% <100.00%> (+1.28%)`	⬆️
.../java/org/opensearch/ad/model/AnomalyDetector.java	`88.85% <ø> (ø)`
.../opensearch/ad/ratelimit/ADResultWriteRequest.java	`100.00% <100.00%> (ø)`
...g/opensearch/ad/ratelimit/ADResultWriteWorker.java	`100.00% <100.00%> (ø)`
.../opensearch/ad/ratelimit/ADSaveResultStrategy.java	`96.66% <100.00%> (+0.11%)`	⬆️
...arch/ad/transport/ADResultBulkTransportAction.java	`94.11% <100.00%> (+2.68%)`	⬆️
...java/org/opensearch/forecast/model/Forecaster.java	`80.81% <ø> (+1.74%)`	⬆️
...forecast/ratelimit/ForecastResultWriteRequest.java	`50.00% <100.00%> (ø)`
...forecast/ratelimit/ForecastSaveResultStrategy.java	`96.55% <100.00%> (+0.12%)`	⬆️
...opensearch/timeseries/constant/CommonMessages.java	`97.50% <ø> (ø)`
... and 11 more

... and 10 files with indirect coverage changes

src/main/java/org/opensearch/timeseries/indices/IndexManagement.java

src/main/java/org/opensearch/timeseries/rest/handler/AbstractTimeSeriesActionHandler.java

Signed-off-by: Jackie Han <hnyng@amazon.com>

opensearch-trigger-bot · 2025-01-29T19:47:11Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/anomaly-detection/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/anomaly-detection/backport-2.x
# Create a new branch
git switch --create backport/backport-1401-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 e33ff46cd1e6d8c0607708f139b1424a612360c7
# Push it to GitHub
git push --set-upstream origin backport/backport-1401-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/anomaly-detection/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-1401-to-2.x.

…rch-project#1401) * add a feature that flattens custom result index when enabled Signed-off-by: Jackie Han <jkhanjob@gmail.com> * clean up Signed-off-by: Jackie Han <jkhanjob@gmail.com> * add IT Signed-off-by: Jackie Han <hnyng@amazon.com> * cleanup Signed-off-by: Jackie Han <hnyng@amazon.com> * address comments Signed-off-by: Jackie Han <hnyng@amazon.com> * update IT Signed-off-by: Jackie Han <hnyng@amazon.com> * update IT Signed-off-by: Jackie Han <hnyng@amazon.com> * clean up Signed-off-by: Jackie Han <hnyng@amazon.com> * add more IT Signed-off-by: Jackie Han <hnyng@amazon.com> * add more IT Signed-off-by: Jackie Han <hnyng@amazon.com> * add more IT Signed-off-by: Jackie Han <hnyng@amazon.com> * address comments * address comments Signed-off-by: Jackie Han <hnyng@amazon.com> * address comments Signed-off-by: Jackie Han <hnyng@amazon.com> * utlizing a node state manager when writing results into flattened result index Signed-off-by: Jackie Han <hnyng@amazon.com> * build flatten resuilt index enabled into the ResultWriteRequest Signed-off-by: Jackie Han <hnyng@amazon.com> * unbind ingest pipeline with flattened result index when it's disabled Signed-off-by: Jackie Han <hnyng@amazon.com> * test * address comments * cleanup Signed-off-by: Jackie Han <hnyng@amazon.com> * make flatten result index use detector name Signed-off-by: Jackie Han <hnyng@amazon.com> --------- Signed-off-by: Jackie Han <jkhanjob@gmail.com> Signed-off-by: Jackie Han <hnyng@amazon.com>

…nabled (#1401) (#1405) * add a feature that flattens custom result index when enabled (#1401) * add a feature that flattens custom result index when enabled Signed-off-by: Jackie Han <jkhanjob@gmail.com> * clean up Signed-off-by: Jackie Han <jkhanjob@gmail.com> * add IT Signed-off-by: Jackie Han <hnyng@amazon.com> * cleanup Signed-off-by: Jackie Han <hnyng@amazon.com> * address comments Signed-off-by: Jackie Han <hnyng@amazon.com> * update IT Signed-off-by: Jackie Han <hnyng@amazon.com> * update IT Signed-off-by: Jackie Han <hnyng@amazon.com> * clean up Signed-off-by: Jackie Han <hnyng@amazon.com> * add more IT Signed-off-by: Jackie Han <hnyng@amazon.com> * add more IT Signed-off-by: Jackie Han <hnyng@amazon.com> * add more IT Signed-off-by: Jackie Han <hnyng@amazon.com> * address comments * address comments Signed-off-by: Jackie Han <hnyng@amazon.com> * address comments Signed-off-by: Jackie Han <hnyng@amazon.com> * utlizing a node state manager when writing results into flattened result index Signed-off-by: Jackie Han <hnyng@amazon.com> * build flatten resuilt index enabled into the ResultWriteRequest Signed-off-by: Jackie Han <hnyng@amazon.com> * unbind ingest pipeline with flattened result index when it's disabled Signed-off-by: Jackie Han <hnyng@amazon.com> * test * address comments * cleanup Signed-off-by: Jackie Han <hnyng@amazon.com> * make flatten result index use detector name Signed-off-by: Jackie Han <hnyng@amazon.com> --------- Signed-off-by: Jackie Han <jkhanjob@gmail.com> Signed-off-by: Jackie Han <hnyng@amazon.com> * spotless apply Signed-off-by: Jackie Han <hnyng@amazon.com> * update recency_emphasis to be greater than 1 in test cases Signed-off-by: Jackie Han <hnyng@amazon.com> --------- Signed-off-by: Jackie Han <jkhanjob@gmail.com> Signed-off-by: Jackie Han <hnyng@amazon.com>

add a feature that flattens custom result index when enabled

41d5949

Signed-off-by: Jackie Han <jkhanjob@gmail.com>

jackiehanyang requested review from VijayanB, amitgalitz, dbwiddis, jmazanec15, jngz-es, joshpalis, kaituo, ohltyler, owaiskazi19, saratvemulapalli, sean-zheng-amazon, vamshin and ylwu-amzn as code owners January 15, 2025 02:46

opensearch-trigger-bot bot added the backport 2.x label Jan 15, 2025

jackiehanyang force-pushed the ingestPipelineFlatten branch 3 times, most recently from dc1c604 to 9f0dcf6 Compare January 15, 2025 03:10

clean up

352075b

Signed-off-by: Jackie Han <jkhanjob@gmail.com>

jackiehanyang force-pushed the ingestPipelineFlatten branch from d0d60f0 to 352075b Compare January 15, 2025 21:19

add IT

b131f9a

Signed-off-by: Jackie Han <hnyng@amazon.com>

jackiehanyang force-pushed the ingestPipelineFlatten branch from c947793 to bf552b5 Compare January 23, 2025 23:50

cleanup

9fced72

Signed-off-by: Jackie Han <hnyng@amazon.com>

jackiehanyang force-pushed the ingestPipelineFlatten branch from bf552b5 to 9fced72 Compare January 23, 2025 23:52

kaituo reviewed Jan 24, 2025

View reviewed changes

jackiehanyang force-pushed the ingestPipelineFlatten branch 2 times, most recently from 2fd6bc2 to 8b12fd7 Compare January 27, 2025 07:22

address comments

3ce0734

Signed-off-by: Jackie Han <hnyng@amazon.com>

jackiehanyang force-pushed the ingestPipelineFlatten branch from 8b12fd7 to 3ce0734 Compare January 27, 2025 08:07

jackiehanyang force-pushed the ingestPipelineFlatten branch from ad3d99c to c844922 Compare January 28, 2025 09:44

kaituo reviewed Jan 28, 2025

View reviewed changes

jackiehanyang added 2 commits January 28, 2025 11:58

address comments

d4b9262

address comments

781d2da

Signed-off-by: Jackie Han <hnyng@amazon.com>

kaituo reviewed Jan 28, 2025

View reviewed changes

src/test/java/org/opensearch/ad/rest/AnomalyDetectorRestApiIT.java Show resolved Hide resolved

src/main/java/org/opensearch/timeseries/rest/handler/AbstractTimeSeriesActionHandler.java Outdated Show resolved Hide resolved

address comments

a623fcf

Signed-off-by: Jackie Han <hnyng@amazon.com>

kaituo reviewed Jan 28, 2025

View reviewed changes

jackiehanyang mentioned this pull request Jan 29, 2025

[DOC] Flatten Custom Result Index in Anomaly Detection opensearch-project/documentation-website#9132

Closed

4 tasks

utlizing a node state manager when writing results into flattened res…

cf8bed4

…ult index Signed-off-by: Jackie Han <hnyng@amazon.com>

kaituo reviewed Jan 29, 2025

View reviewed changes

src/main/java/org/opensearch/timeseries/indices/IndexManagement.java Outdated Show resolved Hide resolved

src/main/java/org/opensearch/ad/transport/ADResultBulkTransportAction.java Outdated Show resolved Hide resolved

build flatten resuilt index enabled into the ResultWriteRequest

d557c50

Signed-off-by: Jackie Han <hnyng@amazon.com>

jackiehanyang force-pushed the ingestPipelineFlatten branch from 590fb5a to d557c50 Compare January 29, 2025 06:18

jackiehanyang added 4 commits January 28, 2025 22:54

unbind ingest pipeline with flattened result index when it's disabled

b5240df

Signed-off-by: Jackie Han <hnyng@amazon.com>

test

ffd7d94

address comments

05f56a3

cleanup

64cc3bc

Signed-off-by: Jackie Han <hnyng@amazon.com>

jackiehanyang mentioned this pull request Jan 29, 2025

add ingest pipeline and indices related permissions for anomaly_full_access role opensearch-project/security#5065

Merged

5 tasks

kaituo reviewed Jan 29, 2025

View reviewed changes

src/main/java/org/opensearch/timeseries/indices/IndexManagement.java Outdated Show resolved Hide resolved

src/main/java/org/opensearch/timeseries/rest/handler/AbstractTimeSeriesActionHandler.java Outdated Show resolved Hide resolved

make flatten result index use detector name

8a9429d

Signed-off-by: Jackie Han <hnyng@amazon.com>

kaituo approved these changes Jan 29, 2025

View reviewed changes

jackiehanyang merged commit e33ff46 into opensearch-project:main Jan 29, 2025
19 of 20 checks passed

opensearch-trigger-bot bot added the backport-failed label Jan 29, 2025

jackiehanyang mentioned this pull request Jan 29, 2025

[Backport 2.x] add a feature that flattens custom result index when enabled (#1401) #1405

Merged

This was referenced Jan 30, 2025

add documentation for AD flatten result index feature opensearch-project/documentation-website#9140

Merged

Add release notes for 2.19.0 #1412

Merged

add a feature that flattens custom result index when enabled #1401

add a feature that flattens custom result index when enabled #1401

Uh oh!

Conversation

jackiehanyang commented Jan 15, 2025

Description

Related Issues

Check List

Uh oh!

jackiehanyang commented Jan 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kaituo commented Jan 28, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Jan 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

opensearch-trigger-bot bot commented Jan 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jackiehanyang commented Jan 24, 2025 •

edited

Loading

codecov bot commented Jan 29, 2025 •

edited

Loading