Skip to content

docs: add HashJoinExec metrics to user guide - #24055

Open
fyrsta7 wants to merge 2 commits into
apache:mainfrom
fyrsta7:codex/hashjoin-metrics-docs
Open

docs: add HashJoinExec metrics to user guide#24055
fyrsta7 wants to merge 2 commits into
apache:mainfrom
fyrsta7:codex/hashjoin-metrics-docs

Conversation

@fyrsta7

@fyrsta7 fyrsta7 commented Aug 2, 2026

Copy link
Copy Markdown

Which issue does this PR close?

Rationale for this change

HashJoinExec exposes several operator-specific metrics, but the metrics user guide did not document them yet. This PR follows the metrics documentation tracking issue by adding the HashJoinExec section.

What changes are included in this PR?

  • Adds HashJoinExec operator-specific metrics to docs/source/user-guide/metrics.md.
  • Adds a note near BuildProbeJoinMetrics to keep metric descriptions in sync with the user guide.
  • Clarifies that join_time covers join processing after build-side collection.
  • Clarifies that probe_hit_rate and avg_fanout measure join-key matches before applying any join filter.

Are these changes tested?

This is a documentation/comment-only change with no runtime behavior change.

Checked locally:

  • git diff --check
  • cargo fmt --check
  • npx prettier@2.7.1 --check docs/source/user-guide/metrics.md
  • ./dev/rust_lint.sh
  • cd docs && uv run --package datafusion-docs ./build.sh
  • DataFusion CLI EXPLAIN ANALYZE queries with and without a join filter

Are there any user-facing changes?

Yes. The metrics user guide now documents the existing HashJoinExec metrics. There are no runtime behavior or public API changes.

AI-assisted contribution note

This change is limited to documentation and comments. I verified the metric names and semantics against BuildProbeJoinMetrics, the HashJoinExec execution path, and local EXPLAIN ANALYZE output. I do not have unknowns or assumptions to call out.

@github-actions github-actions Bot added documentation Improvements or additions to documentation physical-plan Changes to the physical-plan crate labels Aug 2, 2026
@fyrsta7
fyrsta7 marked this pull request as ready for review August 2, 2026 17:15
@fyrsta7
fyrsta7 marked this pull request as draft August 2, 2026 17:44
@fyrsta7
fyrsta7 marked this pull request as ready for review August 3, 2026 06:27
@fyrsta7

fyrsta7 commented Aug 12, 2026

Copy link
Copy Markdown
Author

Hi maintainers, just a gentle follow-up on this documentation PR. I have rechecked the metric semantics against the implementation and local EXPLAIN ANALYZE output, and the formatting, lint, and documentation checks pass locally. The full GitHub Actions workflows are currently awaiting approval for a first-time contributor. When you have a chance, could someone please approve the workflows and take a look? Thank you!

@kosiew kosiew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fyrsta7,

Thanks for adding the HashJoinExec metrics documentation and keeping the BuildProbeJoinMetrics comments in sync. This looks good overall. I just have one small wording suggestion to make the memory metric a bit more precise.

| build_time | Total time spent collecting and building the build side of the join. |
| build_input_batches | Number of input batches consumed from the build side. |
| build_input_rows | Number of input rows consumed from the build side. |
| build_mem_used | Peak memory used by the build side, in bytes. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we say "Peak tracked memory used by the build side" instead of "Peak memory used" here? build_mem_used is based on accounting from RecordBatch sizes, ArrayMap and hash-table estimates, and the visited bitmap. That makes it useful for the operator's reservation and accounting, but it is not an allocator-measured process-memory peak.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Include HashJoinExec in the metrics user-guide

2 participants