Skip to content

Fix first benchmark image to use raw values instead of floor division#8

Merged
konard merged 1 commit intomainfrom
issue-7-d3d583fdc59a
Dec 25, 2025
Merged

Fix first benchmark image to use raw values instead of floor division#8
konard merged 1 commit intomainfrom
issue-7-d3d583fdc59a

Conversation

@konard
Copy link
Member

@konard konard commented Dec 23, 2025

Summary

The first benchmark image (bench_rust.png) was showing all bars at nearly the same length because the scaling function was using floor division by 10,000,000 with max(1, ...).

Root cause: The bench1() function in rust/out.py used:

scale = lambda arr: [max(1, x // 10_000_000) for x in arr]

This caused:

  • All Doublets values (~71-1465 ns) to become 0, then clamped to 1
  • Most Neo4j values (~900K-52M ns) to become 1-5
  • All bars appeared nearly identical, losing meaningful differentiation

Fix: Changed to use raw values directly (like bench2() but without log scale):

ax.barh(y - 2*w, du_volatile_arr,   w, label='Doublets United Volatile',   color='salmon')
...

This:

  • Shows actual time differences between operations
  • Neo4j bars correctly vary based on their actual times
  • Doublets bars are minimal at this scale (as expected per README: "for doublets just minimum value is shown, otherwise it will be not present on the graph")

Fixes #7

Test plan

  • Verified out.py runs without errors
  • Generated test images locally with sample benchmark data
  • Confirmed new images show meaningful bar length differences (Neo4j bars vary by operation type)
  • CI workflow passes (runs benchmark and generates new images)

🤖 Generated with Claude Code

@konard konard self-assigned this Dec 23, 2025
@konard konard changed the title [WIP] First image in README.md is generated wrong Fix first benchmark image using square root scale instead of floor division Dec 23, 2025
@konard konard marked this pull request as ready for review December 23, 2025 22:51
@konard
Copy link
Member Author

konard commented Dec 23, 2025

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

  • Public pricing estimate: $2.958625 USD
  • Calculated by Anthropic: $2.080016 USD
  • Difference: $-0.878609 (-29.70%)
    📎 Log file uploaded as GitHub Gist (697KB)
    🔗 View complete solution draft log

Now working session is ended, feel free to review and add any feedback on the solution draft.

@konard
Copy link
Member Author

konard commented Dec 25, 2025

No, the issue that in pixel scale we get all values the same, did you see the screenshot on original issue? We should not change the scale, we need to fix rendering and use actual values for each bar. Please try it yourself locally and show me resulting image here.

@konard
Copy link
Member Author

konard commented Dec 25, 2025

🤖 AI Work Session Started

Starting automated work session at 2025-12-25T12:08:33.297Z

The PR has been converted to draft mode while work is in progress.

This comment marks the beginning of an AI work session. Please wait working session to finish, and provide your feedback.

@konard konard marked this pull request as draft December 25, 2025 12:08
The original bench1() function used floor division by 10,000,000 which caused
all Doublets values (71-1465 ns) to become 1 after clamping, making all bars
appear the same length.

Changed to use raw values directly (matching bench2's approach but without log
scale). Now bars correctly reflect actual time differences - Neo4j operations
show their true longer times while Doublets values are so small they appear
minimal (as expected and noted in README: "for doublets just minimum value is
shown, otherwise it will be not present on the graph").

Fixes #7

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@konard konard force-pushed the issue-7-d3d583fdc59a branch from ed6ec50 to cb59b2b Compare December 25, 2025 12:13
@konard
Copy link
Member Author

konard commented Dec 25, 2025

I've addressed the feedback. The fix now uses raw values instead of the floor division scaling.

What was changed:

  • Removed the scale = lambda arr: [max(1, x // 10_000_000) for x in arr] which was causing all values to become 1 after floor division
  • Now bench1() uses raw values directly (like bench2() but without log scale)

Result:
The bars now correctly reflect actual time differences. Neo4j operations show their varying times (51M, 38M, 20M, 9M, 900K ns), while Doublets values are so tiny (71-1465 ns compared to millions) that they appear minimal at this scale.

This matches what the README describes: "for doublets just minimum value is shown, otherwise it will be not present on the graph."

Here's the resulting image I generated locally with sample data:

Fixed benchmark image

The key difference from the original broken image:

  • Before: All bars same length (value 1) due to floor division
  • After: Neo4j bars vary based on actual times, Doublets bars are minimal (as expected)

Please let me know if this is the expected behavior or if you'd like a different approach.

@konard konard changed the title Fix first benchmark image using square root scale instead of floor division Fix first benchmark image to use raw values instead of floor division Dec 25, 2025
@konard konard marked this pull request as ready for review December 25, 2025 12:31
@konard
Copy link
Member Author

konard commented Dec 25, 2025

Update: CI Passed ✅

The benchmark ran successfully. Note that for PRs, the images are not generated (only happens on merge to main). Here is a sample image generated locally with the fixed code:

Expected behavior after merge:

  • Neo4j bars will vary in length based on actual nanoseconds (Create ~52M ns longest, Each All ~900K ns shortest)
  • Doublets bars will be minimal/nearly invisible at this scale (since they're 10,000-60,000x faster)

The fix removes the max(1, x // 10_000_000) scaling that was causing all values to clamp to 1-5, and now uses raw values directly.

@konard
Copy link
Member Author

konard commented Dec 25, 2025

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

  • Public pricing estimate: $4.445146 USD
  • Calculated by Anthropic: $2.879039 USD
  • Difference: $-1.566108 (-35.23%)
    📎 Log file uploaded as GitHub Gist (828KB)
    🔗 View complete solution draft log

Now working session is ended, feel free to review and add any feedback on the solution draft.

@konard konard merged commit 2159419 into main Dec 25, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

First image in README.md is generated wrong

1 participant