Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set consistent host name in Go benchmarks in CI #33726

Closed
alistaire47 opened this issue Jan 17, 2023 · 0 comments · Fixed by #33728
Closed

Set consistent host name in Go benchmarks in CI #33726

alistaire47 opened this issue Jan 17, 2023 · 0 comments · Fixed by #33728

Comments

@alistaire47
Copy link
Contributor

Describe the bug, including details regarding any error messages, version, and platform.

Currently Go benchmarks are running in CI, but because the particular runner nodes vary and we do not set a consistent hostname, we can't see the history of the benchmarks because the hostnames keep changing, e.g.

The solution for this is to set an environment variable called CONBENCH_MACHINE_INFO_NAME to a consistent value so this code will pick it up and use it instead. We're running on two types of runners, so we'll need to insert the env var in https://github.com/apache/arrow/blob/master/.github/workflows/go.yml in both the envs on L94-98 and L264-268. We can hardcode values in those two locations with names for the types of runners they are, e.g. something like amd64-debian-11 and amd64-macos-11, respectively. Some of our other host names, for reference:

  • ec2-m5-4xlarge-us-east-2
  • arm64-t4g-linux-compute
  • ursa-i9-9960x

Long-term, we plan to move these benchmarks out of Arrow's CI and together with the rest in voltrondata-labs/arrow-benchmarks-ci, but there's work to do before we're ready for that, and in the mean time, cleaning up our naming will let us see the history we're generating for Go benchmarks.

Component(s)

Continuous Integration, Go

alistaire47 added a commit to alistaire47/arrow that referenced this issue Jan 17, 2023
Closes apache#33726, setting host name in Go benchmarks via environment variable so benchmark history will be available on Conbench.
assignUser pushed a commit that referenced this issue Jan 17, 2023
Closes #33726; see issue for comprehensive explanation. cc @ zeroshade 

### Rationale for this change

Go benchmarks are sending host names from runners that vary slightly, preventing us from seeing the full history of the benchmarks on Conbench.

### What changes are included in this PR?

Sets host name in Go benchmarks via environment variable so benchmark history will be available on Conbench.

### Are these changes tested?

Yes, for the benchadapt code that will pick it up: https://github.com/conbench/conbench/blob/main/benchadapt/python/tests/test_result.py#L107-L114

### Are there any user-facing changes?

No
* Closes: #33726

Authored-by: Edward Visel <1693477+alistaire47@users.noreply.github.com>
Signed-off-by: Jacob Wujciak-Jens <jacob@wujciak.de>
@assignUser assignUser added this to the 12.0.0 milestone Jan 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants