Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vulndash: Add page size when listing vulnerabilities more logs and build v0.3.0 image #1667

Merged
merged 2 commits into from
Nov 1, 2020

Conversation

cpanato
Copy link
Member

@cpanato cpanato commented Oct 31, 2020

What type of PR is this?

/kind feature
/kind design

What this PR does / why we need it:

The current execution log just sends start and finish messages, sometimes is missing more information where the service is. Adding more logs to make it easier to see what is going on and troubleshoot if need.

Also, add an option to set the page size when listing the vulnerabilities, the default value for the API call is 20, with that it takes too much time to get all info needed.

Add the flag to default to 200, some local tests:

With the page size 20 (default for the API call if you not set) and the new logs took 1:27 minutes

$ time vulndash --project=k8s-staging-artifact-promoter --bucket=test-carlos-dash --dashboard-file-path=cmd/vulndash/ --log-level debug --page-size 20
DEBU Setting commands globally into verbose mode
DEBU Using log level "debug"                       file="cmd/root.go:117"
INFO Updating the vulnerability dashboard...       file="cmd/root.go:138"
INFO opening cmd/vulndash/dashboard.html           file="adapter/adapter.go:175"
INFO parsing cmd/vulndash/dashboard.html           file="adapter/adapter.go:181"
INFO uploading cmd/vulndash/dashboard.html to gcs  file="adapter/adapter.go:187"
INFO uploading updated dashboard.js to gcs         file="adapter/adapter.go:194"
INFO checking all vulnerabilities for k8s-staging-artifact-promoter  file="adapter/adapter.go:201"
INFO listing the vulnerabilities...                file="adapter/adapter.go:90"
INFO done listing the vulnerabilities              file="adapter/adapter.go:103"
INFO parsing the vulnerabilities for k8s-staging-artifact-promoter  file="adapter/adapter.go:207"
INFO writing the vulnerabilities for k8s-staging-artifact-promoter in the file cmd/vulndash/dashboard.json  file="adapter/adapter.go:215"
INFO uploading updated cmd/vulndash/dashboard.json to gcs  file="adapter/adapter.go:222"
INFO Finished vulnerability dashboard updates      file="cmd/root.go:150"
vulndash --project=k8s-staging-artifact-promoter --bucket=test-carlos-dash     2.13s user 1.55s system 4% cpu 1:27.00 total

With 200 as the page size, it took 19.231 seconds

$ time vulndash --project=k8s-staging-artifact-promoter --bucket=test-carlos-dash --dashboard-file-path=cmd/vulndash/ --log-level debug
DEBU Setting commands globally into verbose mode
DEBU Using log level "debug"                       file="cmd/root.go:117"
INFO Updating the vulnerability dashboard...       file="cmd/root.go:138"
INFO opening cmd/vulndash/dashboard.html           file="adapter/adapter.go:177"
INFO parsing cmd/vulndash/dashboard.html           file="adapter/adapter.go:183"
INFO uploading cmd/vulndash/dashboard.html to gcs  file="adapter/adapter.go:189"
INFO uploading updated dashboard.js to gcs         file="adapter/adapter.go:196"
INFO checking all vulnerabilities for k8s-staging-artifact-promoter  file="adapter/adapter.go:203"
INFO listing the vulnerabilities, will take a while...  file="adapter/adapter.go:90"
INFO done listing the vulnerabilities              file="adapter/adapter.go:105"
INFO parsing the vulnerabilities for k8s-staging-artifact-promoter  file="adapter/adapter.go:209"
INFO writing the vulnerabilities for k8s-staging-artifact-promoter in the file cmd/vulndash/dashboard.json  file="adapter/adapter.go:217"
INFO uploading updated cmd/vulndash/dashboard.json to gcs  file="adapter/adapter.go:224"
INFO Finished vulnerability dashboard updates      file="cmd/root.go:150"
vulndash --project=k8s-staging-artifact-promoter --bucket=test-carlos-dash     1.02s user 0.71s system 8% cpu 19.231 total

did not add more because looks like it consume more memory and cpu

Which issue(s) this PR fixes:

None

/assing @justaugustus

Special notes for your reviewer:

Does this PR introduce a user-facing change?

vulndash: Add page size when listing vulnerabilities more logs and build v0.3.0 image

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. kind/design Categorizes issue or PR as related to design. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-priority size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Oct 31, 2020
@k8s-ci-robot k8s-ci-robot added sig/release Categorizes an issue or PR as relevant to SIG Release. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Oct 31, 2020
Copy link
Member

@justaugustus justaugustus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cpanato -- You beat me to filing an issue for this. Thank you!

I left one comment for follow-up.
/lgtm
/approve

cmd/vulndash/cmd/root.go Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 1, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cpanato, justaugustus

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [cpanato,justaugustus]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit f12c7b4 into kubernetes:master Nov 1, 2020
@k8s-ci-robot k8s-ci-robot added this to the v1.20 milestone Nov 1, 2020
@cpanato cpanato deleted the vulndash-logs branch November 1, 2020 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/release-eng Issues or PRs related to the Release Engineering subproject cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/design Categorizes issue or PR as related to design. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/release Categorizes an issue or PR as relevant to SIG Release. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants