Skip to content

Conversation

@connortsui20
Copy link
Contributor

@connortsui20 connortsui20 commented Nov 25, 2025

Ports over the take implementation on primitive arrays to vortex-compute for take on generic slices.

@codspeed-hq
Copy link

codspeed-hq bot commented Nov 25, 2025

CodSpeed Performance Report

Merging #5537 will improve performances by 17.18%

Comparing ct/take-slice (8b07ace) with develop (7e1d377)

Summary

⚡ 5 improvements
✅ 1473 untouched
⏩ 235 skipped1

Benchmarks breakdown

Benchmark BASE HEAD Change
slice_arrow_buffer[1024] 397.8 ns 339.4 ns +17.18%
slice_arrow_buffer[128] 397.8 ns 339.4 ns +17.18%
slice_arrow_buffer[16384] 397.8 ns 339.4 ns +17.18%
slice_arrow_buffer[2048] 397.8 ns 339.4 ns +17.18%
slice_arrow_buffer[65536] 397.8 ns 339.4 ns +17.18%

Footnotes

  1. 235 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@codecov
Copy link

codecov bot commented Nov 25, 2025

Codecov Report

❌ Patch coverage is 88.61789% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.43%. Comparing base (7e1d377) to head (8b07ace).
⚠️ Report is 1 commits behind head on develop.

Files with missing lines Patch % Lines
vortex-compute/src/take/slice/mod.rs 33.33% 6 Missing ⚠️
vortex-compute/src/take/buffer.rs 0.00% 3 Missing ⚠️
vortex-compute/src/take/slice/avx2.rs 96.62% 3 Missing ⚠️
...ex-array/src/arrays/primitive/compute/take/avx2.rs 90.90% 1 Missing ⚠️
vortex-buffer/src/buffer.rs 90.90% 1 Missing ⚠️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@connortsui20 connortsui20 force-pushed the ct/take-slice branch 2 times, most recently from e228da9 to 3bc2fdc Compare November 25, 2025 21:59
@a10y
Copy link
Contributor

a10y commented Nov 25, 2025

Are we going to handle nullability or is that handled a level up?

@connortsui20
Copy link
Contributor Author

@a10y yeah I think that it doesnt really make sense to handle nullable indices for a plain buffer, once we add take on things like vectors then we can take a primitive array or pvector as indices

@joseph-isaacs
Copy link
Contributor

Can we name this PR appropriately?

@connortsui20
Copy link
Contributor Author

@joseph-isaacs why is it not named appropriately? Do you mean it should be named a migration instead?

@connortsui20
Copy link
Contributor Author

connortsui20 commented Nov 26, 2025

ok I'm just going to port it over now to make things cleaner

cc @a10y since I believe you were the one who implemented the AVX2 impl? Everything is identical except that I pulled out the validity (which was just getting passed in so that it could construct a PrimitiveArray) and I changed a buffer transmute to use a new cast_into method I added on Buffer.

@connortsui20 connortsui20 force-pushed the ct/take-slice branch 8 times, most recently from fc55e05 to e7e0ef2 Compare November 26, 2025 17:12
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
@cloudflare-workers-and-pages
Copy link

cloudflare-workers-and-pages bot commented Nov 26, 2025

Deploying vortex-bench with  Cloudflare Pages  Cloudflare Pages

Latest commit: 8b07ace
Status: ✅  Deploy successful!
Preview URL: https://cedce7ce.vortex-93b.pages.dev
Branch Preview URL: https://ct-take-slice.vortex-93b.pages.dev

View logs

Copy link
Contributor

@a10y a10y left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, avx2 tests are running in CI and still passing

@connortsui20 connortsui20 merged commit cd80330 into develop Nov 26, 2025
47 checks passed
@connortsui20 connortsui20 deleted the ct/take-slice branch November 26, 2025 18:33
connortsui20 added a commit that referenced this pull request Dec 9, 2025
Adds `take` implementations and tests for:

- `BitBuffer`
- `Mask`
- `BoolVector`
- `PrimitiveVector`

For `BitBuffer`, I don't think it makes sense to implement take with
nullable indices since I would argue that there isn't a clear behavior
to have (because it might not be obvious that `true` means not null and
`false` means null). But for `Mask` it feels a bit different, we are
using it for validity in all of our vectors, so I think it makes a bit
more sense (and it will make the implementation of `take` for the rest
of the vectors easier).

We represent indices in 3 different ways here (a generic indices slice
of unsigned integers, generic unsigned pvector, and primitive vector). I
feel quite strongly that we should not allow signed indices, as that
opens the door to too many bugs.

This is also based on top of
#5537, so that needs to merge
first.

---------

Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants