Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New elasticsearch command to update/rewrite data #310

Merged
merged 2 commits into from
Oct 12, 2020
Merged

New elasticsearch command to update/rewrite data #310

merged 2 commits into from
Oct 12, 2020

Conversation

dguendisch
Copy link
Member

What this PR does / why we need it:
Adds a new command elasticsearch to update es data.
For now has only one subcommand precompute, which allows to rerun the computation of the precompute fields (stored in the teststep es documents in .pre) and update the data in elasticsearch.
Useful for modifying or introducing new pre fields and updating all existing data.

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Release note:

A new command `elasticsearch precompute` was introduced to allow to update all teststep `.pre` fields in elasticsearch to the current/most up-to-date precompute values.

@dguendisch dguendisch requested a review from schrodit October 10, 2020 15:48
@dguendisch dguendisch requested a review from a team as a code owner October 10, 2020 15:48
@gardener-robot gardener-robot added needs/review Needs review size/xl Size of pull request is huge (see gardener-robot robot/bots/size.py) needs/second-opinion Needs second review by someone else labels Oct 10, 2020
@gardener-robot-ci-1 gardener-robot-ci-1 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Oct 10, 2020
schrodit
schrodit suggested changes Oct 12, 2020
View reviewed changes
Copy link

@schrodit schrodit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just some minor observations

cmd/elasticsearch/cmd/precompute/precompute.go Outdated Show resolved Hide resolved
cmd/elasticsearch/cmd/precompute/precompute.go Outdated Show resolved Hide resolved
cmd/elasticsearch/cmd/precompute/precompute.go Outdated Show resolved Hide resolved
pkg/util/elasticsearch/elasticsearch.go Outdated Show resolved Hide resolved
cmd/elasticsearch/cmd/precompute/precompute.go Outdated Show resolved Hide resolved
pkg/testmachinery/collector/precompute.go Show resolved Hide resolved
@gardener-robot gardener-robot added needs/changes Needs (more) changes and removed needs/review Needs review labels Oct 12, 2020
@gardener-robot-ci-3 gardener-robot-ci-3 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Oct 12, 2020
@gardener-robot-ci-2 gardener-robot-ci-2 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Oct 12, 2020
@dguendisch
Copy link
Member Author

Addressed your comments, thanks for them! Also added some better error handling/reporting in case bucket updates contain failures.
I now also added the first new precomputed field: providerEnhanced.
PTAL :)

schrodit
schrodit approved these changes Oct 12, 2020
View reviewed changes
Copy link

@schrodit schrodit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@gardener-robot gardener-robot added reviewed/lgtm Has approval for merging needs/changes Needs (more) changes and removed needs/changes Needs (more) changes needs/second-opinion Needs second review by someone else reviewed/lgtm Has approval for merging labels Oct 12, 2020
@dguendisch dguendisch merged commit 8a01839 into master Oct 12, 2020
@dguendisch dguendisch deleted the precompute-applier branch October 12, 2020 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@schrodit schrodit schrodit approved these changes

Assignees
No one assigned
Labels
needs/changes Needs (more) changes needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) size/xl Size of pull request is huge (see gardener-robot robot/bots/size.py)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@dguendisch @schrodit @gardener-robot-ci-1 @gardener-robot-ci-2 @gardener-robot-ci-3 @gardener-robot