Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[history] more cautious in deciding domain state to make decisions on… #5164

Merged
merged 1 commit into from
Mar 17, 2023
Merged

[history] more cautious in deciding domain state to make decisions on… #5164

merged 1 commit into from
Mar 17, 2023

Conversation

shijiesheng
Copy link
Contributor

@shijiesheng shijiesheng commented Mar 16, 2023 •
edited
Loading

What changed?

When domain cache returned entity not found error, don't drop queued tasks to be more conservative.

Why?

In cases when the cache is dubious, we shouldn't drop the queued tasks.

@coveralls
Copy link

Pull Request Test Coverage Report for Build 0186ec6f-f681-41f3-a94d-3710932d9573

  • 0 of 1 (0.0%) changed or added relevant line in 1 file are covered.
  • 136 unchanged lines in 17 files lost coverage.
  • Overall coverage decreased (-0.05%) to 57.072%
Changes Missing Coverage Covered Lines Changed/Added Lines %
service/history/queue/task_allocator.go 0 1 0.0%
Files with Coverage Reduction New Missed Lines %
common/task/weightedRoundRobinTaskScheduler.go 1 89.64%
service/history/queue/task_allocator.go 1 53.33%
service/history/queue/timer_queue_processor_base.go 1 77.26%
common/peerprovider/ringpopprovider/config.go 2 81.58%
common/task/fifoTaskScheduler.go 2 84.54%
service/frontend/workflowHandler.go 2 59.86%
service/history/handler.go 2 47.15%
service/history/task/transfer_standby_task_executor.go 2 87.66%
service/matching/matcher.go 2 91.46%
common/persistence/statsComputer.go 3 94.64%
Totals Coverage Status
Change from base Build 0186ec4a-0629-4049-8935-b8ff08697b97: -0.05%
Covered Lines: 85207
Relevant Lines: 149297
💛 - Coveralls

davidporter-id-au
davidporter-id-au approved these changes Mar 16, 2023
View reviewed changes
Copy link
Contributor

@davidporter-id-au davidporter-id-au left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this might be unit-test worthy

@Groxx
Copy link
Contributor

Groxx commented Mar 16, 2023

Given the contents of #5140, I think the error(s) in this func won't be logged... tbh we probably do want to log and/or metric issues here, since they'd be unexpected. Currently they're created and lost forever.

But yea, 👍 for this change. Accidentally-merged clusters could also be dangerous otherwise, with not-founds.

@shijiesheng shijiesheng merged commit ff4eab2 into uber:master Mar 17, 2023
@shijiesheng
Copy link
Contributor Author

this might be unit-test worthy

I'll add unit test (along with the log, metrics) in the follow-up

Groxx pushed a commit that referenced this pull request Mar 17, 2023
@shijiesheng @Groxx
[history] more cautious in deciding domain state to make decisions on…
8e81044 
… dropping queued tasks (#5164)

What changed?

When domain cache returned entity not found error, don't drop queued tasks to be more conservative.

Why?

In cases when the cache is dubious, we shouldn't drop the queued tasks.
davidporter-id-au added a commit that referenced this pull request Mar 18, 2023
@davidporter-id-au
Squashed commit of the following:
a31ccba 
commit f1e2476
Author: sonpham96 <sonpham1996@gmail.com>
Date:   Sat Mar 18 05:32:01 2023 +0700

    Upgrade Golang base image to 1.18 to remediate CVEs (#5035)

    Co-authored-by: David Porter <david.porter@uber.com>

commit 1519ace
Author: charlese-instaclustr <76502507+charlese-instaclustr@users.noreply.github.com>
Date:   Fri Mar 17 22:11:27 2023 +0000

    Fix type validation in configstore DC client value updating (#5110)

    * Remove misleading type check, Add more detailed log message

    * removing debugging logging

    * Handle nil update edge case

    ---------

    Co-authored-by: allenchen2244 <102192478+allenchen2244@users.noreply.github.com>
    Co-authored-by: Zijian <Shaddoll@users.noreply.github.com>

commit a3e2774
Author: charlese-instaclustr <76502507+charlese-instaclustr@users.noreply.github.com>
Date:   Fri Mar 17 19:02:40 2023 +0000

    Add Canary TLS support (#5086)

    * add support for TLS connections by Canary, add development config for Canary with TLS

    * update README to include new config option

    * remove testing config

    ---------

    Co-authored-by: David Porter <david.porter@uber.com>
    Co-authored-by: Shijie Sheng <shengs@uber.com>
    Co-authored-by: Zijian <Shaddoll@users.noreply.github.com>

commit ff4eab2
Author: Shijie Sheng <shengs@uber.com>
Date:   Thu Mar 16 20:10:54 2023 -0700

    [history] more cautious in deciding domain state to make decisions on dropping queued tasks (#5164)

    What changed?

    When domain cache returned entity not found error, don't drop queued tasks to be more conservative.

    Why?

    In cases when the cache is dubious, we shouldn't drop the queued tasks.

commit 55a8d93
Author: neil-xie <104041627+neil-xie@users.noreply.github.com>
Date:   Thu Mar 16 14:18:35 2023 -0700

    Add Pinot docker files, table config and schema (#5163)

    * Initial checkin for pinot config files

commit 1304570
Author: Mantas Šidlauskas <mantass@netapp.com>
Date:   Thu Mar 16 15:20:29 2023 +0200

    Set poll interval for filebased dynamic config if not set (#5160)

    * Set poll interval for filebased dynamic config if not set

    * update unit test

commit 42a14b1
Author: Mantas Šidlauskas <mantass@netapp.com>
Date:   Thu Mar 16 10:49:21 2023 +0200

    Elasticsearch: reduce code duplication (#5137)

    * Elasticsearch: reduce code duplication

    * address comments

    ---------

    Co-authored-by: Zijian <Shaddoll@users.noreply.github.com>

commit cbf0d14
Author: bowen xiao <xbowen@uber.com>
Date:   Wed Mar 15 10:19:34 2023 -0700

    fix samples documentation (#5088)

commit ba19a29
Author: Mantas Šidlauskas <mantass@netapp.com>
Date:   Wed Mar 15 12:52:29 2023 +0200

    Add ShardID to valid attributes (#5161)

commit a25cba8
Author: Mantas Šidlauskas <mantass@netapp.com>
Date:   Wed Mar 15 10:56:50 2023 +0200

    ES: single interface for different ES/OpenSearch versions (#5158)

    * ES: single interface for different ES/OpenSearch versions

    * make fmt

commit e3ac246
Author: Ketsia <115650494+ketsiambaku@users.noreply.github.com>
Date:   Tue Mar 14 12:47:40 2023 -0700

    added logging with workflow/domain tags (#5159)

commit 9581488
Author: Ketsia <115650494+ketsiambaku@users.noreply.github.com>
Date:   Mon Mar 13 16:56:45 2023 -0700

    Consistent query pershard metric (#5143)

    * added and update consistent query per shard metric

    * testing pershard metric

    * move sample logger into persistence metric client for cleaness

    * fix test

    * fix lint

    * fix test again

    * fix lint

    * sample logging with workflowid tag

    * added domain tag to logger

    * metric completed

    * addressing comments

    * fix lint

    * Revert "fix lint"

    This reverts commit 1e96944.

    * fix lint second attempt

    ---------

    Co-authored-by: Allen Chen <allenchen2244@uber.com>
davidporter-id-au added a commit that referenced this pull request Mar 30, 2023
@davidporter-id-au
Squashed commit of the following:
9f8a8d7 
commit 9d01035
Author: allenchen2244 <102192478+allenchen2244@users.noreply.github.com>
Date:   Wed Mar 29 20:50:38 2023 -0700

    large workflow hot shard detection (#5166)

    Metrics for large workflows

commit dd51c53
Author: David Porter <david.porter@uber.com>
Date:   Wed Mar 29 18:30:06 2023 -0700

    fix build (#5180)

commit 7b281c2
Author: David Porter <david.porter@uber.com>
Date:   Mon Mar 27 10:38:37 2023 -0700

    Adds a small test to catch issues with deadlocks (#5171)

    * Adds a small test to catch issues with deadlocks

commit f1e2476
Author: sonpham96 <sonpham1996@gmail.com>
Date:   Sat Mar 18 05:32:01 2023 +0700

    Upgrade Golang base image to 1.18 to remediate CVEs (#5035)

    Co-authored-by: David Porter <david.porter@uber.com>

commit 1519ace
Author: charlese-instaclustr <76502507+charlese-instaclustr@users.noreply.github.com>
Date:   Fri Mar 17 22:11:27 2023 +0000

    Fix type validation in configstore DC client value updating (#5110)

    * Remove misleading type check, Add more detailed log message

    * removing debugging logging

    * Handle nil update edge case

    ---------

    Co-authored-by: allenchen2244 <102192478+allenchen2244@users.noreply.github.com>
    Co-authored-by: Zijian <Shaddoll@users.noreply.github.com>

commit a3e2774
Author: charlese-instaclustr <76502507+charlese-instaclustr@users.noreply.github.com>
Date:   Fri Mar 17 19:02:40 2023 +0000

    Add Canary TLS support (#5086)

    * add support for TLS connections by Canary, add development config for Canary with TLS

    * update README to include new config option

    * remove testing config

    ---------

    Co-authored-by: David Porter <david.porter@uber.com>
    Co-authored-by: Shijie Sheng <shengs@uber.com>
    Co-authored-by: Zijian <Shaddoll@users.noreply.github.com>

commit ff4eab2
Author: Shijie Sheng <shengs@uber.com>
Date:   Thu Mar 16 20:10:54 2023 -0700

    [history] more cautious in deciding domain state to make decisions on dropping queued tasks (#5164)

    What changed?

    When domain cache returned entity not found error, don't drop queued tasks to be more conservative.

    Why?

    In cases when the cache is dubious, we shouldn't drop the queued tasks.

commit 55a8d93
Author: neil-xie <104041627+neil-xie@users.noreply.github.com>
Date:   Thu Mar 16 14:18:35 2023 -0700

    Add Pinot docker files, table config and schema (#5163)

    * Initial checkin for pinot config files

commit 1304570
Author: Mantas Šidlauskas <mantass@netapp.com>
Date:   Thu Mar 16 15:20:29 2023 +0200

    Set poll interval for filebased dynamic config if not set (#5160)

    * Set poll interval for filebased dynamic config if not set

    * update unit test

commit 42a14b1
Author: Mantas Šidlauskas <mantass@netapp.com>
Date:   Thu Mar 16 10:49:21 2023 +0200

    Elasticsearch: reduce code duplication (#5137)

    * Elasticsearch: reduce code duplication

    * address comments

    ---------

    Co-authored-by: Zijian <Shaddoll@users.noreply.github.com>

commit cbf0d14
Author: bowen xiao <xbowen@uber.com>
Date:   Wed Mar 15 10:19:34 2023 -0700

    fix samples documentation (#5088)

commit ba19a29
Author: Mantas Šidlauskas <mantass@netapp.com>
Date:   Wed Mar 15 12:52:29 2023 +0200

    Add ShardID to valid attributes (#5161)

commit a25cba8
Author: Mantas Šidlauskas <mantass@netapp.com>
Date:   Wed Mar 15 10:56:50 2023 +0200

    ES: single interface for different ES/OpenSearch versions (#5158)

    * ES: single interface for different ES/OpenSearch versions

    * make fmt

commit e3ac246
Author: Ketsia <115650494+ketsiambaku@users.noreply.github.com>
Date:   Tue Mar 14 12:47:40 2023 -0700

    added logging with workflow/domain tags (#5159)

commit 9581488
Author: Ketsia <115650494+ketsiambaku@users.noreply.github.com>
Date:   Mon Mar 13 16:56:45 2023 -0700

    Consistent query pershard metric (#5143)

    * added and update consistent query per shard metric

    * testing pershard metric

    * move sample logger into persistence metric client for cleaness

    * fix test

    * fix lint

    * fix test again

    * fix lint

    * sample logging with workflowid tag

    * added domain tag to logger

    * metric completed

    * addressing comments

    * fix lint

    * Revert "fix lint"

    This reverts commit 1e96944.

    * fix lint second attempt

    ---------

    Co-authored-by: Allen Chen <allenchen2244@uber.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@davidporter-id-au davidporter-id-au davidporter-id-au approved these changes

@Groxx Groxx Groxx approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@shijiesheng @coveralls @Groxx @davidporter-id-au