Use tokio::sync::oneshot to prevent FuturesUnordered reentrant drop crash

[sgrebnov]

🗣 Description

Fixes SIGSEGV and future still here when dropping panics when using buffer_unordered with vortex I/O futures under high concurrency.

Root Cause

The oneshot crate drops stored wakers inside Receiver::drop(). When the waker is an Arc<FuturesUnordered::Task>, this causes reentrant drops that trigger either:

• Safety panic: "future still here when dropping"
• SIGSEGV from use-after-free

The reentrant Task::drop() is undefined behavior because:

• Partially destroyed state: The outer Task::drop() may have already freed some fields. The inner drop accesses deallocated memory.
• Double-free: Both drops might try to deallocate the same memory, corrupting the allocator and causing crashes

PR switches to tokio::sync::oneshot which drops wakers outside the destructor chain, preventing this behavior.

oneshot crate drop chain (CRASHES):
  FuturesUnordered::drop()
    → release_task() starts dropping future
      → ReadFuture::drop()
        → oneshot::Receiver::drop()
          → Channel::drop() releases stored waker   ← waker dropped INSIDE destructor
            → Arc<FU::Task> refcount → 0
              → FU::Task::drop() REENTRANT! 💥
                → "future still here when dropping" panic OR SIGSEGV

tokio::sync::oneshot drop chain (SAFE):
  FuturesUnordered::drop()
    → release_task() starts dropping future
      → ReadFuture::drop()
        → tokio::oneshot::Receiver::drop()
          → AtomicWaker notifies, no Arc ref held   ← waker dropped OUTSIDE destructor
            → Clean drop, no reentrant issue ✓

🔨 Related Issues

Fixes

• Cayenne acceleration panic in FuturesUnordered::release_task spiceai#8830

🤔 Concerns

[Copilot]

Pull request overview

This PR addresses a critical crash issue (SIGSEGV and "future still here when dropping" panics) in high-concurrency scenarios by replacing the oneshot crate with tokio::sync::oneshot when the tokio feature is enabled. The root cause is that the oneshot crate drops stored wakers inside the destructor chain, which can trigger reentrant drops of Arc<FuturesUnordered::Task>, leading to undefined behavior. The tokio::sync::oneshot implementation avoids this by dropping wakers outside the destructor chain.

Changes:

• Added conditional compilation to use tokio::sync::oneshot when tokio feature is enabled, falling back to oneshot crate otherwise
• Updated error handling in ReadRequest::resolve to account for API differences between the two oneshot implementations
• Applied changes consistently across all files that use oneshot channels

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
vortex-io/src/runtime/single.rs	Added conditional imports for oneshot based on tokio feature flag
vortex-io/src/runtime/handle.rs	Added conditional imports for oneshot based on tokio feature flag
vortex-io/src/file/read/request.rs	Added conditional imports and updated error handling in resolve method to handle API differences
vortex-io/src/file/read/mod.rs	Added conditional imports for oneshot based on tokio feature flag
vortex-io/src/file/driver.rs	Added conditional imports for oneshot in test code based on tokio feature flag

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.