Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix consistent query metric #5170

Merged
merged 8 commits into from
Apr 12, 2023

Conversation

ketsiambaku
Copy link
Contributor

What changed?
Add shardid tag to log
Add shardid tag to shardMetricScope
Increment counter per shard

Why?
Hot shard detection

How did you test it?
tested in staging

Potential risks

Release notes

Documentation Changes


consistentQueryEnabled := e.config.EnableConsistentQuery() && e.config.EnableConsistentQueryByDomain(request.GetRequest().GetDomain())
if request.GetRequest().GetQueryConsistencyLevel() == types.QueryConsistencyLevelStrong {
if !consistentQueryEnabled {
return nil, workflow.ErrConsistentQueryNotEnabled
}
scope.IncCounter(metrics.ConsistentQueryPerShard)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this have been removed? kinda seems like this is to intentionally dual-emit for more efficient total metrics.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this should be removed. I was thinking of making it consistent with something else but it's not needed for this particular operation since it's technically not a persistence operation.

Copy link
Contributor

@allenchen2244 allenchen2244 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you've tested and it looks good feel free to merge.

@coveralls
Copy link

coveralls commented Apr 6, 2023

Pull Request Test Coverage Report for Build 01877779-f9c1-451b-a93a-825615385bb4

  • 2 of 2 (100.0%) changed or added relevant lines in 1 file are covered.
  • 76 unchanged lines in 10 files lost coverage.
  • Overall coverage increased (+0.006%) to 57.07%

Files with Coverage Reduction New Missed Lines %
common/task/parallelTaskProcessor.go 2 92.75%
service/history/execution/mutable_state_builder.go 2 68.82%
service/history/execution/mutable_state_util.go 2 36.04%
service/history/task/transfer_standby_task_executor.go 2 86.4%
common/persistence/statsComputer.go 3 93.57%
service/history/queue/timer_queue_processor_base.go 3 77.26%
common/persistence/nosql/nosqlplugin/cassandra/workflow.go 6 59.1%
service/frontend/workflowHandler.go 7 59.97%
service/history/execution/mutable_state_task_refresher.go 15 64.56%
common/persistence/nosql/nosqlplugin/cassandra/workflowParsingUtils.go 34 79.09%
Totals Coverage Status
Change from base Build 0187773f-663d-4b6f-aa3b-df281da9578e: 0.006%
Covered Lines: 85234
Relevant Lines: 149350

💛 - Coveralls

@ketsiambaku ketsiambaku enabled auto-merge (squash) April 12, 2023 21:59
@ketsiambaku ketsiambaku merged commit c5678dd into uber:master Apr 12, 2023
davidporter-id-au added a commit that referenced this pull request Apr 19, 2023
commit b18be27
Author: Mantas Šidlauskas <mantass@netapp.com>
Date:   Tue Apr 18 14:12:09 2023 +0300

    Add generic ES query building utilities (#5168)

commit 824f0ac
Author: agautam478 <72432016+agautam478@users.noreply.github.com>
Date:   Fri Apr 14 13:37:29 2023 -0700

    Fixed the nil pointer issues in the InactiveDomain Invariant (#5213)

commit c5678dd
Author: Ketsia <115650494+ketsiambaku@users.noreply.github.com>
Date:   Wed Apr 12 15:21:10 2023 -0700

    Fix consistent query metric (#5170)

    * add shardid tag to log

    * remove counter for overall scope

    * fix lint

commit eade936
Author: David Porter <david.porter@uber.com>
Date:   Wed Apr 12 13:54:13 2023 -0700

    Corrects the config-store handling for not-found errors (#5203)

commit 9fc4485
Author: lancezhao-ins <99238165+lancezhao-ins@users.noreply.github.com>
Date:   Tue Apr 11 04:21:15 2023 +1000

    Allow registering search attributes without Advance Visibility enabled (#5185)

    * remove validation & test for add search attribute with no advanced config

    - Remove validation for Advance Visibility Store
    - Add Advance Visibility Config check before update ElasticSearch/OpenSearch mapping
    - Remove co-related test for 'no advanced config'

    * Update CHANGELOG.md

    Update CHANGELOG.md

    * Add warn level message if skip updating OpenSearch/ElasticSearch mapping

    * Add warn level message and add validSearchAttributes in development.yaml

    ---------

    Co-authored-by: Quanzheng Long <prclqz@gmail.com>

commit d165c7b
Author: neil-xie <104041627+neil-xie@users.noreply.github.com>
Date:   Mon Apr 10 10:46:04 2023 -0700

    Update idls version (#5200)

commit b2bc8bf
Author: Mantas Šidlauskas <mantass@netapp.com>
Date:   Mon Apr 10 20:12:53 2023 +0300

    Add thin ES clients (#5162)

    * Add thin ES clients
@ketsiambaku ketsiambaku deleted the fix-consistent-query-metric branch May 30, 2023 23:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants