Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Response Ops][Task Manager] Onboard 12.5% of ECH clusters to use mget task claiming #196317

Merged
merged 3 commits into from
Oct 16, 2024
Merged

[Response Ops][Task Manager] Onboard 12.5% of ECH clusters to use mget task claiming #196317

merged 3 commits into from
Oct 16, 2024
+294 −19

Conversation

ymao1
Copy link
Contributor

@ymao1 ymao1 commented Oct 15, 2024 •
edited by kibanamachine
Loading

Resolves https://github.com/elastic/response-ops-team/issues/239

Summary

Deployed to cloud: deployment ID was ab4e88d139f93d43024837d96144e7d4. Since the deployment ID starts with an a, this should start with mget and I can see in the logs with the latest push that this is true

Screenshot 2024-10-15 at 2 59 20 PM

Deployed to serverless: project ID was d33d22a94ce246d091220eace2c4e4bb. See in the logs: Using claim strategy mget as configured for deployment d33d22a94ce246d091220eace2c4e4bb

@ymao1 ymao1 added ci:cloud-deploy Create or update a Cloud deployment ci:project-deploy-elasticsearch Create an Elasticsearch Serverless project labels Oct 15, 2024
mikecote
mikecote reviewed Oct 15, 2024
View reviewed changes
Copy link
Contributor

@mikecote mikecote left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR looks good so far!

x-pack/plugins/task_manager/server/lib/set_claim_strategy.ts Outdated Show resolved Hide resolved
@ymao1 ymao1 changed the title wip [Response Ops][Task Manager] Onboard 12.5% of ECH clusters to use mget task claiming Oct 15, 2024
mikecote
mikecote approved these changes Oct 15, 2024
View reviewed changes
Copy link
Contributor

@mikecote mikecote left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM, tested locally and verified on ECH and serverless.

@ymao1
Copy link
Contributor Author

ymao1 commented Oct 15, 2024

@elasticmachine merge upstream

@ymao1 ymao1 marked this pull request as ready for review October 15, 2024 22:22
@ymao1 ymao1 requested a review from a team as a code owner October 15, 2024 22:22
@ymao1 ymao1 self-assigned this Oct 15, 2024
@ymao1 ymao1 added release_note:skip Skip the PR/issue when compiling release notes Feature:Task Manager Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v9.0.0 backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) v8.16.0 labels Oct 15, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

@ymao1
Copy link
Contributor Author

ymao1 commented Oct 15, 2024

@elasticmachine merge upstream

@elasticmachine
Copy link
Contributor

elasticmachine commented Oct 15, 2024 •
edited
Loading

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #91 / Rules Management - Rule Bulk Action API @ess perform_bulk_action - ESS specific logic should disable rules and migrate actions
  • [job] [logs] Jest Tests #10 / threatMatchRowRenderer #renderRow rendered when indicator matches are more than MAX rendered

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
taskManager 64 65 +1
Unknown metric groups

API count

id before after diff
taskManager 108 109 +1

History

cc @ymao1

@ymao1 ymao1 merged commit 267efdf into elastic:main Oct 16, 2024
44 checks passed
@ymao1 ymao1 deleted the mget-ech branch October 16, 2024 00:24
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11356536561

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 16, 2024
@ymao1
[Response Ops][Task Manager] Onboard 12.5% of ECH clusters to use `mg…
bd76a73 
…et` task claiming (elastic#196317)

Resolves elastic/response-ops-team#239

## Summary

Deployed to cloud: deployment ID was `ab4e88d139f93d43024837d96144e7d4`.
Since the deployment ID starts with an `a`, this should start with
`mget` and I can see in the logs with the latest push that this is true

<img width="2190" alt="Screenshot 2024-10-15 at 2 59 20 PM"
src="https://github.com/user-attachments/assets/079bc4d8-365e-4ba6-b7a9-59fe506283d9">

Deployed to serverless: project ID was
`d33d22a94ce246d091220eace2c4e4bb`. See in the logs: `Using claim
strategy mget as configured for deployment
d33d22a94ce246d091220eace2c4e4bb`

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
(cherry picked from commit 267efdf)
@kibanamachine kibanamachine mentioned this pull request Oct 16, 2024
Merged
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
✅ 8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Oct 16, 2024
@kibanamachine @ymao1
[8.x] [Response Ops][Task Manager] Onboard 12.5% of ECH clusters to u…
0c13333 
…se &#x60;mget&#x60; task claiming (#196317) (#196460)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Response Ops][Task Manager] Onboard 12.5% of ECH clusters to use
&#x60;mget&#x60; task claiming
(#196317)](#196317)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Ying
Mao","email":"ying.mao@elastic.co"},"sourceCommit":{"committedDate":"2024-10-16T00:24:52Z","message":"[Response
Ops][Task Manager] Onboard 12.5% of ECH clusters to use `mget` task
claiming (#196317)\n\nResolves
https://github.com/elastic/response-ops-team/issues/239\r\n\r\n##
Summary\r\n\r\nDeployed to cloud: deployment ID was
`ab4e88d139f93d43024837d96144e7d4`.\r\nSince the deployment ID starts
with an `a`, this should start with\r\n`mget` and I can see in the logs
with the latest push that this is true\r\n\r\n<img width=\"2190\"
alt=\"Screenshot 2024-10-15 at 2 59
20 PM\"\r\nsrc=\"https://github.com/user-attachments/assets/079bc4d8-365e-4ba6-b7a9-59fe506283d9\">\r\n\r\n\r\nDeployed
to serverless: project ID was\r\n`d33d22a94ce246d091220eace2c4e4bb`. See
in the logs: `Using claim\r\nstrategy mget as configured for
deployment\r\nd33d22a94ce246d091220eace2c4e4bb`\r\n\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"267efdf31fe9ae314b0bed99bc23db5452a2aaa3","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Feature:Task
Manager","Team:ResponseOps","v9.0.0","backport:prev-minor","ci:cloud-deploy","ci:project-deploy-elasticsearch","v8.16.0"],"title":"[Response
Ops][Task Manager] Onboard 12.5% of ECH clusters to use `mget` task
claiming","number":196317,"url":"https://github.com/elastic/kibana/pull/196317","mergeCommit":{"message":"[Response
Ops][Task Manager] Onboard 12.5% of ECH clusters to use `mget` task
claiming (#196317)\n\nResolves
https://github.com/elastic/response-ops-team/issues/239\r\n\r\n##
Summary\r\n\r\nDeployed to cloud: deployment ID was
`ab4e88d139f93d43024837d96144e7d4`.\r\nSince the deployment ID starts
with an `a`, this should start with\r\n`mget` and I can see in the logs
with the latest push that this is true\r\n\r\n<img width=\"2190\"
alt=\"Screenshot 2024-10-15 at 2 59
20 PM\"\r\nsrc=\"https://github.com/user-attachments/assets/079bc4d8-365e-4ba6-b7a9-59fe506283d9\">\r\n\r\n\r\nDeployed
to serverless: project ID was\r\n`d33d22a94ce246d091220eace2c4e4bb`. See
in the logs: `Using claim\r\nstrategy mget as configured for
deployment\r\nd33d22a94ce246d091220eace2c4e4bb`\r\n\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"267efdf31fe9ae314b0bed99bc23db5452a2aaa3"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/196317","number":196317,"mergeCommit":{"message":"[Response
Ops][Task Manager] Onboard 12.5% of ECH clusters to use `mget` task
claiming (#196317)\n\nResolves
https://github.com/elastic/response-ops-team/issues/239\r\n\r\n##
Summary\r\n\r\nDeployed to cloud: deployment ID was
`ab4e88d139f93d43024837d96144e7d4`.\r\nSince the deployment ID starts
with an `a`, this should start with\r\n`mget` and I can see in the logs
with the latest push that this is true\r\n\r\n<img width=\"2190\"
alt=\"Screenshot 2024-10-15 at 2 59
20 PM\"\r\nsrc=\"https://github.com/user-attachments/assets/079bc4d8-365e-4ba6-b7a9-59fe506283d9\">\r\n\r\n\r\nDeployed
to serverless: project ID was\r\n`d33d22a94ce246d091220eace2c4e4bb`. See
in the logs: `Using claim\r\nstrategy mget as configured for
deployment\r\nd33d22a94ce246d091220eace2c4e4bb`\r\n\r\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>","sha":"267efdf31fe9ae314b0bed99bc23db5452a2aaa3"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Ying Mao <ying.mao@elastic.co>
@mikecote mikecote mentioned this pull request Oct 17, 2024
Merged
mikecote added a commit that referenced this pull request Oct 18, 2024
@mikecote
Onboard elastic owned ECH clusters to use mget task claiming (#196757)
97f2a90 
Similar to #196317

In this PR, I'm flipping the mget feature flag to on for all elastic
owned ECH clusters. Elastic owned clusters are determined by looking at
`plugins.cloud?.isElasticStaffOwned`.

## To verify
Observe the PR deployment which doesn't start with `a` or `b` yet is
using the mget claim strategy by logging `Using claim strategy mget` on
startup.
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 18, 2024
@mikecote
Onboard elastic owned ECH clusters to use mget task claiming (elast…
b5b8cc1 
…ic#196757)

Similar to elastic#196317

In this PR, I'm flipping the mget feature flag to on for all elastic
owned ECH clusters. Elastic owned clusters are determined by looking at
`plugins.cloud?.isElasticStaffOwned`.

## To verify
Observe the PR deployment which doesn't start with `a` or `b` yet is
using the mget claim strategy by logging `Using claim strategy mget` on
startup.

(cherry picked from commit 97f2a90)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Oct 18, 2024
@mikecote
Onboard elastic owned ECH clusters to use mget task claiming (elast…
62a9e06 
…ic#196757)

Similar to elastic#196317

In this PR, I'm flipping the mget feature flag to on for all elastic
owned ECH clusters. Elastic owned clusters are determined by looking at
`plugins.cloud?.isElasticStaffOwned`.

## To verify
Observe the PR deployment which doesn't start with `a` or `b` yet is
using the mget claim strategy by logging `Using claim strategy mget` on
startup.

(cherry picked from commit 97f2a90)
mikecote added a commit to mikecote/kibana that referenced this pull request Oct 21, 2024
@mikecote
Revert "[Response Ops][Task Manager] Onboard 12.5% of ECH clusters to…
9e6737f 
… use `mget` task claiming (elastic#196317)"

This reverts commit 267efdf.
@mikecote mikecote mentioned this pull request Oct 21, 2024
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mikecote mikecote mikecote approved these changes

Assignees

@ymao1 ymao1

Labels
backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) ci:cloud-deploy Create or update a Cloud deployment ci:project-deploy-elasticsearch Create an Elasticsearch Serverless project Feature:Task Manager release_note:skip Skip the PR/issue when compiling release notes Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v8.16.0 v9.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ymao1 @elasticmachine @kibanamachine @mikecote