Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Add kicbase download progress to JSON output #15685

Merged
merged 1 commit into from
Mar 13, 2023
Merged

feat: Add kicbase download progress to JSON output #15685

merged 1 commit into from
Mar 13, 2023

Conversation

ComradeProgrammer
Copy link
Member

@ComradeProgrammer ComradeProgrammer commented Jan 23, 2023 •
edited
Loading

feat: Add kicbase download progress to JSON output

FIXES #15481

Before:
Before this PR, when we execute minikube start -o json, the output looks like this:

{"specversion":"1.0","id":"43c8e879-3385-44b0-83b6-127a6e19d18b","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"0","message":"minikube v1.28.0 on Ubuntu 22.04 (amd64)","name":"Initial Minikube Setup","totalsteps":"19"}}
{"specversion":"1.0","id":"4bb57aa3-c344-4235-bccc-0e29bb1461b2","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"1","message":"Automatically selected the docker driver","name":"Selecting Driver","totalsteps":"19"}}
{"specversion":"1.0","id":"20590f8a-5be6-47ab-8ab8-f625a9044921","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"Using image repository registry.cn-hangzhou.aliyuncs.com/google_containers"}}
{"specversion":"1.0","id":"364a2ec2-8cf7-44a5-a42b-8e9b7f059c8d","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.info","datacontenttype":"application/json","data":{"message":"Using Docker driver with root privileges"}}
{"specversion":"1.0","id":"9f721f95-1a70-4e46-b812-7eaf4b7e9165","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"3","message":"Starting control plane node minikube in cluster minikube","name":"Starting Node","totalsteps":"19"}}
{"specversion":"1.0","id":"eff70d70-6c03-4634-b6dc-39bd010411be","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"5","message":"Pulling base image ...","name":"Pulling Base Image","totalsteps":"19"}}
{"specversion":"1.0","id":"a66b877b-0aa2-4a01-9ce3-a3ed85b49220","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.step","datacontenttype":"application/json","data":{"currentstep":"8","message":"Creating docker container (CPUs=2, Memory=2200MB) ...","name":"Creating Container","totalsteps":"19"}}
......

No output for kicbase download progress in json output
After
After this PR, when we execute minikube start -o json, we can find this part

......(similar logs at the beginning are omitted) 

{"specversion":"1.0","id":"ba07f967-db4b-4684-8d6b-6d613dbbfb0e","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.download.progress","datacontenttype":"application/json","data":{"artifact":"docker.io/kicbase/build:v0.0.36-1673540226-15630@sha256:03c9592728381094cbd0ff9603f75ae6b485dd7a390c3e35f02ae5ec10f2f3ad","currentstep":"5","progress":"0","totalsteps":"19"}}
{"specversion":"1.0","id":"a1f4296c-47f7-4983-bed2-3d2b2842bd85","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.download.progress","datacontenttype":"application/json","data":{"artifact":"docker.io/kicbase/build:v0.0.36-1673540226-15630@sha256:03c9592728381094cbd0ff9603f75ae6b485dd7a390c3e35f02ae5ec10f2f3ad","currentstep":"5","progress":"0.000013","totalsteps":"19"}}

......(lines of logs omitted)

{"specversion":"1.0","id":"f32754ae-0733-4cd8-9772-0dc9bf354e89","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.download.progress","datacontenttype":"application/json","data":{"artifact":"docker.io/kicbase/build:v0.0.36-1673540226-15630@sha256:03c9592728381094cbd0ff9603f75ae6b485dd7a390c3e35f02ae5ec10f2f3ad","currentstep":"5","progress":"0.999978","totalsteps":"19"}}
{"specversion":"1.0","id":"cbcd8e85-d102-4a4f-9c88-91b1f1ab13a4","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.download.progress","datacontenttype":"application/json","data":{"artifact":"docker.io/kicbase/build:v0.0.36-1673540226-15630@sha256:03c9592728381094cbd0ff9603f75ae6b485dd7a390c3e35f02ae5ec10f2f3ad","currentstep":"5","progress":"0.999981","totalsteps":"19"}}
{"specversion":"1.0","id":"8be461b8-1d3c-4919-920f-24a88f0030d9","source":"https://minikube.sigs.k8s.io/","type":"io.k8s.sigs.minikube.download.progress","datacontenttype":"application/json","data":{"artifact":"docker.io/kicbase/build:v0.0.36-1673540226-15630@sha256:03c9592728381094cbd0ff9603f75ae6b485dd7a390c3e35f02ae5ec10f2f3ad","currentstep":"5","progress":"1","totalsteps":"19"}}
......(similar logs at the end are omitted)

Similar to downloading progress for preloads, the output is genearted per second.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jan 23, 2023
@k8s-ci-robot
Copy link
Contributor

Hi @ComradeProgrammer. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jan 23, 2023
@minikube-bot
Copy link
Collaborator

Can one of the admins verify this patch?

@ComradeProgrammer
Copy link
Member Author

ComradeProgrammer commented Jan 23, 2023 •
edited
Loading

@spowelljr Could you please have a look at this?

medyagh
medyagh reviewed Jan 24, 2023
View reviewed changes
Copy link
Member

@medyagh medyagh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am curious how many new lines does this add to the json file ? maybe the json users wouldnt need that level of details

@ComradeProgrammer
Copy link
Member Author

I am curious how many new lines does this add to the json file ? maybe the json users wouldnt need that level of details

Well, for normal users with a good network connection(10MB/S), there would be about 48 lines?. Kicbase image is 480MB.

If that is too much, I can make it generate less logs.

spowelljr
spowelljr requested changes Feb 21, 2023
View reviewed changes
Copy link
Member

@spowelljr spowelljr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this outputs a JSON log every second, for people with slow internet the download can take a long time and the JSON output can get quite large. I'd recommend this outputs say every 5 seconds instead to lessen the logs while still giving consistent output that the process is still running.

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ComradeProgrammer
Once this PR has been reviewed and has the lgtm label, please assign medyagh for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ComradeProgrammer
Copy link
Member Author

I believe this outputs a JSON log every second, for people with slow internet the download can take a long time and the JSON output can get quite large. I'd recommend this outputs say every 5 seconds instead to lessen the logs while still giving consistent output that the process is still running.

(Well the current code is exactly logging every 5 seconds

spowelljr
spowelljr requested changes Feb 27, 2023
View reviewed changes
pkg/minikube/download/image.go Outdated Show resolved Hide resolved
@medyagh medyagh merged commit 282c4ef into kubernetes:master Mar 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@medyagh medyagh medyagh left review comments

@spowelljr spowelljr spowelljr requested changes

@prezha prezha Awaiting requested review from prezha

@sharifelgamal sharifelgamal Awaiting requested review from sharifelgamal

Assignees
No one assigned
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add kicbase download progress to JSON output
5 participants
@ComradeProgrammer @k8s-ci-robot @minikube-bot @medyagh @spowelljr