Skip to content

ipc stream reader RecordBatchReader::schema ignores column projection #10676

Closed

Description

@codeAnqiang-ma

Describe the bug

Same as #10382 (csv) and #10613 (ipc file reader), but for the ipc stream reader.

When a projection is set, StreamReader::schema() and its RecordBatchReader::schema() impl return the full stream schema, while the batches the reader yields have the projected schema. #10627 fixed FileReader, StreamReader has the same problem.

StreamReader::try_new already computes the projected schema and stores it in the private projection field, but that value is never read back.

To Reproduce

// arrow-ipc/src/reader.rs (mirrors test_file_reader_projected_schema_matches_batch_schema)

#[test]
fn test_stream_reader_projected_schema_matches_batch_schema() {
    let schema = create_test_projection_schema();
    let batch = create_test_projection_batch_data(&schema);

    let mut buf = Vec::new();
    {
        let mut writer = crate::writer::StreamWriter::try_new(&mut buf, &schema).unwrap();
        writer.write(&batch).unwrap();
        writer.finish().unwrap();
    }

    let projection = vec![3, 2, 1];
    let mut reader = StreamReader::try_new(Cursor::new(buf), Some(projection)).unwrap();

    let reader_schema = RecordBatchReader::schema(&reader);
    let read_batch = reader.next().unwrap().unwrap();

    assert_eq!(reader_schema, read_batch.schema());
}

On eaf21ed this fails with a 14 column schema on the left (the full stream schema) and a 3 column schema on the right (the projected batch schema):

assertion `left == right` failed
  left: Schema { fields: [f0 UInt32, f1 Utf8, f2 Boolean, f3 Union(..), f4 Null, f5 Float64,
        f6 List, f7 FixedSizeBinary(3), f8 FixedSizeList, f9 Struct, f10 RunEndEncoded,
        f11 Boolean, f12 Dictionary(Int8, Utf8), f13 Utf8], metadata: {} }
 right: Schema { fields: [f3 Union(..), f2 Boolean, f1 Utf8], metadata: {} }

The same projection against FileReader passes, so the two IPC readers currently disagree.

Expected behavior

RecordBatchReader::schema should match the schema of the output batches

/// Trait for types that can read `RecordBatch`'s.
///
/// To create from an iterator, see [RecordBatchIterator].
pub trait RecordBatchReader: Iterator<Item = Result<RecordBatch, ArrowError>> {
/// Returns the schema of this `RecordBatchReader`.
///
/// Implementation of this trait should guarantee that all `RecordBatch`'s returned by this
/// reader should have the same schema as returned from this method.
fn schema(&self) -> SchemaRef;
}

Additional context

Found by auditing the remaining RecordBatchReader impls after #10613 was fixed. This report was prepared with AI assistance; I ran the reproduction locally and reviewed the result.

Activity

  1. added a commit that references this issue on Aug 15, 2026
  2. added a commit that references this issue on Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions