feat[buffer]: use BufferHandle in more places

[joseph-isaacs]

Use BufferHandle in segment source

[codecov]

Codecov Report

❌ Patch coverage is 40.35088% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.48%. Comparing base (d8cab4c) to head (9abd386).
⚠️ Report is 4 commits behind head on develop.

Files with missing lines	Patch %	Lines
vortex-array/src/serde.rs	8.33%	33 Missing ⚠️
vortex-layout/src/layouts/flat/reader.rs	80.00%	1 Missing ⚠️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.