Dk/buffer pool

[danking]

Summary

Closes: #000

Testing

[codspeed-hq]

Merging this PR will degrade performance by 65.22%

⚡ 129 improved benchmarks
❌ 50 regressed benchmarks
✅ 774 untouched benchmarks
⏩ 1459 skipped benchmarks¹

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
⚡	Simulation	chunked_bool_canonical_into[(1000, 10)]	66.4 µs	44 µs	+50.82%
⚡	Simulation	encode_varbin[(1000, 2)]	225.5 µs	199.7 µs	+12.93%
⚡	Simulation	encode_varbin[(1000, 32)]	238.8 µs	209.9 µs	+13.74%
⚡	Simulation	encode_varbin[(1000, 4)]	227.4 µs	200.7 µs	+13.31%
⚡	Simulation	encode_varbin[(1000, 8)]	228.8 µs	201.3 µs	+13.66%
⚡	Simulation	take_fsl_random[64, 100]	162.6 µs	147.7 µs	+10.03%
⚡	Simulation	take_fsl_random[256, 1000]	4.9 ms	4.5 ms	+10.13%
⚡	Simulation	take_fsl_random[16, 100]	70.5 µs	63.5 µs	+11.07%
⚡	Simulation	take_fsl_random[4096, 100]	8.2 ms	7.4 ms	+10.09%
⚡	Simulation	take_fsl_random[64, 1000]	1.3 ms	1.2 ms	+11.65%
⚡	Simulation	take_fsl_random[16, 1000]	377.4 µs	322.9 µs	+16.89%
❌	Simulation	varbinview_rebuild	9.3 ms	14.3 ms	-34.66%
⚡	Simulation	fsst_compress_string	171.9 ms	151 ms	+13.85%
❌	Simulation	decompress[alp_for_bp_f64]	30.3 ms	50 ms	-39.55%
❌	Simulation	decompress[datetime_for_bp]	41.7 ms	60.4 ms	-30.93%
⚡	Simulation	runend_decompress_u32	40 ms	30.6 ms	+30.7%
❌	Simulation	alp_rd_decompress_f64	27.4 ms	35.7 ms	-23.18%
⚡	Simulation	runend_compress_u32	71.3 ms	48.4 ms	+47.25%
⚡	Simulation	compress_fsst[(10000, 4, 4)]	1.8 ms	1.6 ms	+14.45%
⚡	Simulation	compress_fsst[(1000, 4, 8)]	230.1 µs	208.3 µs	+10.45%
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

---

_{Comparing dk/buffer-pool (6670532) with develop (c89a92f)}

Footnotes

1. 1459 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[AdamGS]

I know this is still WIP, but should it be optionally extendable? Arrow-cpp has a MemoryPool extension point, and some logic for picking a default one (in the rust case - probably fallback to the default global allocator one?)