Skip to content

Commit e5997e8

Browse files
opensearch-trigger-bot[bot]github-actions[bot]tanqiuliubowenlan-amznlezzago
authored
Added minimum timeout for transforms search of 10 minutes (#1033) (#1057)
* Added minimum timeout for transforms search of 10 minutes * Extracted cancel minimum code to function * Fixed transform code to use cluster setting * Removed log statements * Changed timeout logic * Switched to basing off seconds * [Feature] Support Transform as an ISM action (#760) * Initial impl * fix style * end to end functional * ISM transform unit tests & integ tests * Fix after core #8157 (#857) * Upgrade the backport workflow (#862) * Added 2.9 release notes. (#851) * Added 2.9 release notes. * Added 2.9 release notes. * Added 2.9 release notes. --------- * Handle NPE in isRollupIndex (#855) * Handle NPE in isRollupIndex `metadata.index()` can return `null`, so handle that case by returning `false`. * unit test --------- * Fix core XcontentType refactor (#873) * fix for max & min aggregations when no metric property exist (#870) * core refactor change (#884) * update backport branch name (#885) * core refactor change (#887) * Fix breaking change by core refactor (#888) * fix core breaking (#906) * Support copy alias in rollover (#907) * Support copy alias in rollover * 2.10 --------- * Set preference to _primary when searching control-center index (#911) * Set preference to _primary when searching control-center index * Use _primary_first instead --------- * Add primary first preference to all search requests (#912) * fix intelliJ IDEA gradle sync error (#916) * make control center index as system index (#919) * Updates demo certs used in integ tests (#921) * Added 2.10 release notes (#925) * Bump bwc version (#930) * fix integ tests; upgrade mappings versions * Fix DCO * Addressed pr comments; Add integ test case for re-execute the same transform action * Addressed detekt error * Added ISMTransform writeable test * Addressed comments; Moved updateTransformStartTime to IndexManagementRestTestCase --------- * [Test] increase the wait time after transform job triggered (#999) * Drafted 2.11 release notes. (#1004) * Drafted 2.11 release notes. * Drafted 2.11 release notes. --------- * Refactor change policy API and the policy in managed index to be non-null (#967) * Refactor the policy to be non null in managed index config * Update * fix bug --------- * Add more error notification at fail points (#1000) * Add more error notification at fail points * Handle exception gracefully * small fix --------- * fix the race condition in test reset action start time (#1007) * Bump bwc version after 2.11 release (#1015) * added type check for pipeline aggregator types in Transform initialization (#1014) * Improve security plugin enabling check (#1017) * Onboard jenkins prod docker images to github actions (#1025) * Onboard jenkins prod docker images to github actions * Add more * Add more --------- * Support switch aliases in shrink action. (#987) * Transform pipeline aggr test (#1027) * tester code: pipeline aggr. transform job * made test case for pipeline aggregator in transform job * removed unnec. test lines * re-added method call on Transform obj --------- * Added unit test for switchAliases method. (#1035) * Added unit test for switchAliases method. * Added unit test for switchAliases method checking the flow when switchAliases is disabled. --------- * Interval schedule should take start time from the request, should not set it to the current time of request execution. (#1036) * Added minimum for search.cancel_after_time_interval setting for rollups (#1026) * Added minimum for search.cancel_after_time_interval setting for rollups * Added constant for cancel_after_time_interval for rollup search * Handled case of default value for cancel interval * Added comment explanation for default rollup cancel after time interval * Fixed github workflow checks --------- * Update 2.11.1 release note (#1042) * Interval schedule should take start time from the request, should not… (#1040) * Interval schedule should take start time from the request, should not set it to the current time of request execution. * Changed the "delayed continuous execution test" to be more expressive about what it should test. * fixed the NPE if schedule.startTime is NULL * fixed the NPE if schedule.startTime is NULL * fixed styling * - removed null checks from RollUp and Transforms - fixed comments in the "delayed execution" test --------- * Removed unused imports --------- (cherry picked from commit 60a8513) Signed-off-by: Joshua Au <joshuahyau@gmail.com> Signed-off-by: Tanqiu Liu <liutanqiu@gmail.com> Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Signed-off-by: Ashish Agrawal <ashisagr@amazon.com> Signed-off-by: AWSHurneyt <hurneyt@amazon.com> Signed-off-by: Bryce Lampe <brycelampe@gmail.com> Signed-off-by: Hailong Cui <ihailong@amazon.com> Signed-off-by: Subhobrata Dey <sbcd90@gmail.com> Signed-off-by: gaobinlong <gbinlong@amazon.com> Signed-off-by: Darshit Chanpura <dchanp@amazon.com> Signed-off-by: n-dohrmann <noah@dohrmann.org> Signed-off-by: Peter Zhu <zhujiaxi@amazon.com> Signed-off-by: Oleg Kravchuk <oleg.ikibo@gmail.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Tanqiu Liu <liutanqiu@gmail.com> Co-authored-by: bowenlan-amzn <bowenlan23@gmail.com> Co-authored-by: Ashish Agrawal <ashisagr@amazon.com> Co-authored-by: AWSHurneyt <hurneyt@amazon.com> Co-authored-by: Bryce Lampe <brycelampe@gmail.com> Co-authored-by: Hailong Cui <ihailong@amazon.com> Co-authored-by: Subhobrata Dey <sbcd90@gmail.com> Co-authored-by: gaobinlong <gbl_long@163.com> Co-authored-by: Darshit Chanpura <35282393+DarshitChanpura@users.noreply.github.com> Co-authored-by: n-dohrmann <87952011+n-dohrmann@users.noreply.github.com> Co-authored-by: Joanne Wang <109310487+jowg-amazon@users.noreply.github.com> Co-authored-by: Peter Zhu <zhujiaxi@amazon.com> Co-authored-by: Oleg Kravchuk <oleg.ikibo@gmail.com> Co-authored-by: ikibo <oleg.kravchuk@logz.io>
1 parent 55e689a commit e5997e8

File tree

2 files changed

+24
-1
lines changed

2 files changed

+24
-1
lines changed

‎src/main/kotlin/org/opensearch/indexmanagement/transform/TransformSearchService.kt‎

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ import org.opensearch.action.bulk.BackoffPolicy
1616
import org.opensearch.action.index.IndexRequest
1717
import org.opensearch.action.search.SearchRequest
1818
import org.opensearch.action.search.SearchResponse
19+
import org.opensearch.action.search.TransportSearchAction.SEARCH_CANCEL_AFTER_TIME_INTERVAL_SETTING
1920
import org.opensearch.client.Client
2021
import org.opensearch.cluster.metadata.IndexMetadata
2122
import org.opensearch.cluster.service.ClusterService
@@ -47,6 +48,7 @@ import org.opensearch.indexmanagement.util.IndexUtils.Companion.LUCENE_MAX_CLAUS
4748
import org.opensearch.indexmanagement.util.IndexUtils.Companion.ODFE_MAGIC_NULL
4849
import org.opensearch.indexmanagement.util.IndexUtils.Companion.hashToFixedSize
4950
import org.opensearch.core.rest.RestStatus
51+
import org.opensearch.indexmanagement.transform.settings.TransformSettings.Companion.MINIMUM_CANCEL_AFTER_TIME_INTERVAL_SECONDS
5052
import org.opensearch.search.aggregations.Aggregation
5153
import org.opensearch.search.aggregations.bucket.composite.CompositeAggregation
5254
import org.opensearch.search.aggregations.bucket.composite.CompositeAggregationBuilder
@@ -78,11 +80,17 @@ class TransformSearchService(
7880
@Volatile private var backoffPolicy =
7981
BackoffPolicy.constantBackoff(TRANSFORM_JOB_SEARCH_BACKOFF_MILLIS.get(settings), TRANSFORM_JOB_SEARCH_BACKOFF_COUNT.get(settings))
8082

83+
@Volatile private var cancelAfterTimeInterval = SEARCH_CANCEL_AFTER_TIME_INTERVAL_SETTING.get(settings)
84+
8185
init {
8286
clusterService.clusterSettings.addSettingsUpdateConsumer(TRANSFORM_JOB_SEARCH_BACKOFF_MILLIS, TRANSFORM_JOB_SEARCH_BACKOFF_COUNT) {
8387
millis, count ->
8488
backoffPolicy = BackoffPolicy.constantBackoff(millis, count)
8589
}
90+
91+
clusterService.clusterSettings.addSettingsUpdateConsumer(SEARCH_CANCEL_AFTER_TIME_INTERVAL_SETTING) {
92+
cancelAfterTimeInterval = it
93+
}
8694
}
8795

8896
@Suppress("RethrowCaughtException")
@@ -187,7 +195,11 @@ class TransformSearchService(
187195
val searchStart = Instant.now().epochSecond
188196
val searchResponse = backoffPolicy.retryTransformSearch(logger, transformContext.transformLockManager) {
189197
val pageSizeDecay = 2f.pow(retryAttempt++)
190-
val searchRequestTimeoutInSeconds = transformContext.getMaxRequestTimeoutInSeconds()
198+
199+
var searchRequestTimeoutInSeconds = transformContext.getMaxRequestTimeoutInSeconds()
200+
if (searchRequestTimeoutInSeconds == null) {
201+
searchRequestTimeoutInSeconds = getCancelAfterTimeIntervalSeconds(cancelAfterTimeInterval.seconds)
202+
}
191203

192204
client.suspendUntil { listener: ActionListener<SearchResponse> ->
193205
// If the previous request of the current transform job execution was successful, take the page size of previous request.
@@ -224,6 +236,16 @@ class TransformSearchService(
224236
}
225237
}
226238

239+
private fun getCancelAfterTimeIntervalSeconds(givenIntervalSeconds: Long): Long {
240+
// The default value for the cancelAfterTimeInterval is -1 and so, in this case
241+
// we should ignore processing on the value
242+
if (givenIntervalSeconds == -1L) {
243+
return -1
244+
}
245+
246+
return max(givenIntervalSeconds, MINIMUM_CANCEL_AFTER_TIME_INTERVAL_SECONDS)
247+
}
248+
227249
companion object {
228250
const val failedSearchErrorMessage = "Failed to search data in source indices"
229251
const val modifiedBucketsErrorMessage = "Failed to get the modified buckets in source indices"

‎src/main/kotlin/org/opensearch/indexmanagement/transform/settings/TransformSettings.kt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ class TransformSettings {
1414
companion object {
1515
const val DEFAULT_RENEW_LOCK_RETRY_COUNT = 3
1616
const val DEFAULT_RENEW_LOCK_RETRY_DELAY = 1000L
17+
const val MINIMUM_CANCEL_AFTER_TIME_INTERVAL_SECONDS = 600L
1718

1819
val TRANSFORM_JOB_SEARCH_BACKOFF_COUNT: Setting<Int> = Setting.intSetting(
1920
"plugins.transform.internal.search.backoff_count",

0 commit comments

Comments
 (0)