Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Execlog parser should either prune or set a default for all the timing information. #24071

Open
fzakaria opened this issue Oct 23, 2024 · 0 comments
Labels
help wanted Someone outside the Bazel team could own this P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Performance Issues for Performance teams type: bug

Comments

@fzakaria
Copy link
Contributor

fzakaria commented Oct 23, 2024

Description of the bug:

Using something like:

> bazel-bin/src/tools/execlog/parser \
  --log_path=/Users/fzakaria/Documents/exec_protoc1.log \
  --log_path=/Users/fzakaria/Documents/exec_protoc2.log \
  --output_path=/tmp/exec1.log.txt \
  --output_path=/tmp/exec2.log.txt

I have included two compact exec logs to see how they are similar except for PATH and timings.
Using diff tools produces many differences on account of the timings.

> diff /tmp/exec1.log.txt /tmp/exec2.log.txt | head -n 30
67c67
<   value: "/Users/fzakaria/Library/Caches/bazelisk/downloads/sha256/7dc88df4842a5b49830c55b174551445587ae60f027a67d0f847dbb35f1f40c6/bin:/Users/fzakaria/.pyenv/shims:/Users/fzakaria/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/opt/homebrew/bin:/Applications/iTerm.app/Contents/Resources/utilities:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/access-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/cloud-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/connect-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/k8saas-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/devprod-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/traffic-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/ksql-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/orchestrator-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/kafka-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/servicefoundations-beta:/Users/fzakaria/.zsh/plugins/powerlevel10k"
---
>   value: "/Users/fzakaria/Library/Caches/bazelisk/downloads/sha256/7dc88df4842a5b49830c55b174551445587ae60f027a67d0f847dbb35f1f40c6/bin:/Users/fzakaria/.pyenv/shims:/Users/fzakaria/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/opt/homebrew/bin:/Applications/iTerm.app/Contents/Resources/utilities:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/access-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/cloud-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/connect-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/k8saas-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/devprod-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/traffic-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/ksql-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/orchestrator-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/kafka-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/servicefoundations-beta:/Users/fzakaria/.zsh/plugins/powerlevel10k:/bin2/"
568,569c568
< runner: "disk cache hit"
< cache_hit: true
---
> runner: "darwin-sandbox"
573c572
<   hash: "630a6822f70ec0fbb920b1ecfe78137657d6a46d989df99034b4f163913c7684"
---
>   hash: "c36a3b77ddd78a42d736c42c8c2ebb341548049db36a3e6591d440d8a004fc6b"
579c578
<     nanos: 8000000
---
>     nanos: 350000000
581,582c580,581
<   fetch_time {
<     nanos: 2000000
---
>   execution_wall_time {
>     nanos: 350000000
584,585d582
<   input_bytes: 1812815
<   input_files: 59
587,588c584,585
<     seconds: 1729717622
<     nanos: 557000000
---

exec_protoc2.log
exec_protoc1.log

My vote would be for this to be the default, but I could imagine it needing to be behind a flag to retain original timings for whatever other workflows exist.

@github-actions github-actions bot added the team-Performance Issues for Performance teams label Oct 23, 2024
@jin jin added P3 We're not considering working on this, but happy to review a PR. (No assignee) help wanted Someone outside the Bazel team could own this and removed untriaged labels Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Someone outside the Bazel team could own this P3 We're not considering working on this, but happy to review a PR. (No assignee) team-Performance Issues for Performance teams type: bug
Projects
None yet
Development

No branches or pull requests

5 participants