Skip to content

ci(codspeed): add memory, walltime and Python benchmark jobs#88

Merged
bug-ops merged 2 commits intomainfrom
feature/codspeed-memory-walltime
Feb 2, 2026
Merged

ci(codspeed): add memory, walltime and Python benchmark jobs#88
bug-ops merged 2 commits intomainfrom
feature/codspeed-memory-walltime

Conversation

@bug-ops
Copy link
Owner

@bug-ops bug-ops commented Feb 2, 2026

Summary

Add comprehensive CodSpeed benchmark coverage with multiple instruments:

  • Simulation (existing): CPU instruction counting for deterministic results
  • Memory: Heap allocation tracking for memory profiling
  • Walltime: Real execution time measurement
  • Python: pytest-codspeed integration for Python benchmarks

Changes

  • Add memory profiling job with mode: instrumentation and instruments: memory
  • Add walltime job with mode: walltime for real-time measurements
  • Add Python benchmark job using pytest-codspeed
  • Add pytest-codspeed to benchmark dependencies in pyproject.toml

Documentation references

Test plan

  • All CodSpeed jobs run successfully
  • Benchmark results appear in CodSpeed dashboard

- Add memory profiling job with heap allocation tracking
- Add walltime job for real execution time measurement
- Add Python benchmark job using pytest-codspeed
- Add pytest-codspeed to benchmark dependencies
@github-actions github-actions bot added dependencies Pull requests that update a dependency file python ci size/M labels Feb 2, 2026
@codecov-commenter
Copy link

codecov-commenter commented Feb 2, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

Impacted file tree graph

@@           Coverage Diff           @@
##             main      #88   +/-   ##
=======================================
  Coverage   86.64%   86.64%           
=======================================
  Files          63       63           
  Lines       13349    13349           
=======================================
  Hits        11566    11566           
  Misses       1783     1783           
Flag Coverage Δ
hdbconnect-arrow 86.59% <ø> (ø)
hdbconnect-mcp 86.59% <ø> (ø)
python 100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- Use mode: memory instead of mode: instrumentation with instruments
- Add -m walltime flag for walltime build
- Add -m memory flag for memory build
@codspeed-hq
Copy link

codspeed-hq bot commented Feb 2, 2026

CodSpeed Performance Report

Merging this PR will not alter performance

Comparing feature/codspeed-memory-walltime (036fbe3) with main (7d02d61)

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

Summary

✅ 137 untouched benchmarks
🆕 324 new benchmarks

Performance Changes

Mode Benchmark BASE HEAD Efficiency
🆕 WallTime get_no_ttl_check N/A 166 ns N/A
🆕 WallTime direct_inmemory_get N/A 166 ns N/A
🆕 WallTime key_creation_query_result N/A 143 ns N/A
🆕 WallTime factory_single_builder_large_binary N/A 113 ns N/A
🆕 WallTime size_bytes[1000] N/A 3.8 µs N/A
🆕 WallTime size_bytes[10000] N/A 33.5 µs N/A
🆕 WallTime size_bytes[1000000] N/A 15.8 ms N/A
🆕 WallTime traced_cache_get N/A 188 ns N/A
🆕 WallTime columns[50] N/A 3.6 ms N/A
🆕 WallTime concurrent_mixed[1] N/A 54.4 µs N/A
🆕 WallTime factory_single_builder_time64 N/A 70 ns N/A
🆕 WallTime read_heavy[2] N/A 93.9 µs N/A
🆕 WallTime compute_stats[10] N/A 47 ns N/A
🆕 WallTime batch_sizes[1024] N/A 725.9 µs N/A
🆕 WallTime time64_nulls N/A 85.4 µs N/A
🆕 WallTime list_tables_memory_miss N/A 11.5 µs N/A
🆕 WallTime int64_conversion[100000] N/A 658.7 µs N/A
🆕 WallTime analytics_1m_rows N/A 69.3 ms N/A
🆕 WallTime len[100] N/A 112.2 µs N/A
🆕 WallTime entries[100] N/A 38.1 µs N/A
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

@bug-ops bug-ops merged commit e66dda0 into main Feb 2, 2026
40 checks passed
@bug-ops bug-ops deleted the feature/codspeed-memory-walltime branch February 2, 2026 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci dependencies Pull requests that update a dependency file python size/M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants