boot-utils: Rewrite in Python #67
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Over the years, this script has gotten quite complex with the logic of
the QEMU options and user input. To make it easier to extend this in the
future, rewrite it in Python, which allows us to use much higher
built-in functions to parse input and make decisions based on that.
The performance difference between the shell and Python implementation
is not much. It is most noticeable when parsing the Linux kernel version
from an aarch64 kernel but the overall performance is still reasonable.
On an aarch64 host with a SoC with 16 Cortex-A72 cores:
arm32_v7:
Benchmark 1: boot-qemu.sh
Time (mean ± σ): 14.265 s ± 0.021 s [User: 15.572 s, System: 1.062 s]
Range (min … max): 14.246 s … 14.305 s 10 runs
Benchmark 2: boot-qemu.py
Time (mean ± σ): 14.319 s ± 0.025 s [User: 15.811 s, System: 0.925 s]
Range (min … max): 14.282 s … 14.361 s 10 runs
Summary
'boot-qemu.sh' ran
1.00 ± 0.00 times faster than 'boot-qemu.py'
arm64:
Benchmark 1: boot-qemu.sh
Time (mean ± σ): 3.949 s ± 0.024 s [User: 1.145 s, System: 1.165 s]
Range (min … max): 3.916 s … 3.992 s 10 runs
Benchmark 2: boot-qemu.py
Time (mean ± σ): 4.009 s ± 0.032 s [User: 1.426 s, System: 1.149 s]
Range (min … max): 3.967 s … 4.049 s 10 runs
Summary
'boot-qemu.sh' ran
1.02 ± 0.01 times faster than 'boot-qemu.py'
arm64be:
Benchmark 1: boot-qemu.sh
Time (mean ± σ): 3.942 s ± 0.035 s [User: 1.332 s, System: 1.114 s]
Range (min … max): 3.910 s … 4.016 s 10 runs
Benchmark 2: boot-qemu.py
Time (mean ± σ): 3.985 s ± 0.028 s [User: 1.196 s, System: 1.143 s]
Range (min … max): 3.954 s … 4.034 s 10 runs
Summary
'boot-qemu.sh' ran
1.01 ± 0.01 times faster than 'boot-qemu.py'
x86:
Benchmark 1: boot-qemu.sh
Time (mean ± σ): 14.993 s ± 0.059 s [User: 12.278 s, System: 1.094 s]
Range (min … max): 14.926 s … 15.095 s 10 runs
Benchmark 2: boot-qemu.py
Time (mean ± σ): 14.985 s ± 0.045 s [User: 12.455 s, System: 0.898 s]
Range (min … max): 14.907 s … 15.039 s 10 runs
Summary
'boot-qemu.py' ran
1.00 ± 0.00 times faster than 'boot-qemu.sh'
x86_64:
Benchmark 1: boot-qemu.sh
Time (mean ± σ): 16.663 s ± 0.049 s [User: 14.539 s, System: 0.544 s]
Range (min … max): 16.601 s … 16.732 s 10 runs
Benchmark 2: boot-qemu.py
Time (mean ± σ): 16.692 s ± 0.059 s [User: 14.044 s, System: 1.057 s]
Range (min … max): 16.607 s … 16.819 s 10 runs
Summary
'boot-qemu.sh' ran
1.00 ± 0.00 times faster than 'boot-qemu.py'
On an x86_64 host with an AMD Threadripper 3990X:
arm32_v7:
Benchmark 1: boot-qemu.sh
Time (mean ± σ): 4.773 s ± 0.014 s [User: 2.716 s, System: 0.065 s]
Range (min … max): 4.750 s … 4.796 s 10 runs
Benchmark 2: boot-qemu.py
Time (mean ± σ): 4.807 s ± 0.021 s [User: 2.749 s, System: 0.065 s]
Range (min … max): 4.777 s … 4.841 s 10 runs
Summary
'boot-qemu.sh' ran
1.01 ± 0.01 times faster than 'boot-qemu.py'
arm64:
Benchmark 1: boot-qemu.sh
Time (mean ± σ): 5.089 s ± 0.022 s [User: 3.196 s, System: 0.105 s]
Range (min … max): 5.057 s … 5.122 s 10 runs
Benchmark 2: boot-qemu.py
Time (mean ± σ): 5.311 s ± 0.016 s [User: 3.263 s, System: 0.141 s]
Range (min … max): 5.297 s … 5.340 s 10 runs
Summary
'boot-qemu.sh' ran
1.04 ± 0.01 times faster than 'boot-qemu.py'
arm64be:
Benchmark 1: boot-qemu.sh
Time (mean ± σ): 5.011 s ± 0.021 s [User: 3.124 s, System: 0.093 s]
Range (min … max): 4.986 s … 5.049 s 10 runs
Benchmark 2: boot-qemu.py
Time (mean ± σ): 5.249 s ± 0.017 s [User: 3.181 s, System: 0.155 s]
Range (min … max): 5.223 s … 5.273 s 10 runs
Summary
'boot-qemu.sh' ran
1.05 ± 0.01 times faster than 'boot-qemu.py'
x86:
Benchmark 1: boot-qemu.sh
Time (mean ± σ): 2.869 s ± 0.022 s [User: 0.532 s, System: 0.236 s]
Range (min … max): 2.850 s … 2.906 s 10 runs
Benchmark 2: boot-qemu.py
Time (mean ± σ): 2.910 s ± 0.029 s [User: 0.551 s, System: 0.236 s]
Range (min … max): 2.856 s … 2.950 s 10 runs
Summary
'boot-qemu.sh' ran
1.01 ± 0.01 times faster than 'boot-qemu.py'
x86_64:
Benchmark 1: boot-qemu.sh
Time (mean ± σ): 3.242 s ± 0.043 s [User: 1.390 s, System: 0.636 s]
Range (min … max): 3.134 s … 3.277 s 10 runs
Benchmark 2: boot-qemu.py
Time (mean ± σ): 3.251 s ± 0.052 s [User: 1.434 s, System: 0.602 s]
Range (min … max): 3.130 s … 3.317 s 10 runs
Summary
'boot-qemu.sh' ran
1.00 ± 0.02 times faster than 'boot-qemu.py'
This is designed to be a one-to-one rewrite, with the exception of
dropping support for '--debian' for simplicity's sake. It was always a
little janky at times and using other options such as libvirt is more
reliable. I may eventually contribute my other QEMU Python script to
this repository, which aims to drive full distributions.
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Now that there is Python in this repository, we should make sure that it stays consistently formatted like the shell scripts are. Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Now that boot-qemu.sh has been rewritten in Python, this does not serve any real purpose, as Python's argparse automatically shows the options and help text. Rewrite the README to give a brief overview of the repository. Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Support for this was not forward ported during the Python rewrite, as it did not always work that well. There are other alternatives such as libvirt for running full distributions with QEMU. A future contribution might add a Python script for roughly achieving what this set out to accomplish. Signed-off-by: Nathan Chancellor <nathan@kernel.org>
nathanchance
requested review from
msfjarvis,
nickdesaulniers and
tpimh
as code owners
nickdesaulniers
approved these changes
Aug 2, 2022
Member
nickdesaulniers
left a comment
nickdesaulniers
left a comment
There was a problem hiding this comment.
cool; this was overdue. Thanks for the rewrite. Untested on my end though. 🐗 🪵
Can you tell I use vim? :) Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
The version code is not printed anywhere so the function name is not as accurate as it should be. Signed-off-by: Nathan Chancellor <nathan@kernel.org>
This will be better in case we add anything after the 'gdb' if block. Signed-off-by: Nathan Chancellor <nathan@kernel.org>
…_qemu() It is not necessary. Signed-off-by: Nathan Chancellor <nathan@kernel.org>
Member
Author
|
If there is any additional feedback from others, I am happy to address it in follow up pull requests. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See all of the individual commits for the details. I tested this with all architectures on x86_64 and 32-bit and 64-bit ARM and 32-bit and 64-bit x86 on aarch64.