Skip to content

Commit b540248

Browse files
alambmbrobbel
andauthored
Refactor: Move parquet metadata parsing code into its own module (#8436)
# Which issue does this PR close? - Part of #8000 - Prep PR for #8340, to make it easier to review Note while this is a large (in line count) code change, it should be relatively easy to review as it is just moving code around # Rationale for this change In #8340 I am trying to split the "IO" from the "where is the metadata in the file" from the "decode thrift into Rust structures" logic. The first part of this is simply to move the code that handles the "decode thrift into Rust structures" into its own module. # What changes are included in this PR? 1. Move most of the "parse thrift bytes into rust structure" code from `parquet/src/file/metadata/mod.rs ` to `parquet/src/file/metadata/parser.rs` # Are these changes tested? yes, by CI # Are there any user-facing changes? No, this is entirely internal reorganization --------- Co-authored-by: Matthijs Brobbel <m1brobbel@gmail.com>
1 parent 28c7c52 commit b540248

File tree

3 files changed

+535
-455
lines changed

3 files changed

+535
-455
lines changed

parquet/src/file/metadata/mod.rs

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@
4040
//! metadata into parquet files. To work with metadata directly,
4141
//! the following APIs are available:
4242
//!
43-
//! * [`ParquetMetaDataReader`] for reading from a reader for I/O
43+
//! * [`ParquetMetaDataReader`] for reading metadata from an I/O source (sync and async)
4444
//! * [`ParquetMetaDataPushDecoder`] for decoding from bytes without I/O
4545
//! * [`ParquetMetaDataWriter`] for writing.
4646
//!
@@ -91,6 +91,7 @@
9191
//! * Same name, different struct
9292
//! ```
9393
mod memory;
94+
mod parser;
9495
mod push_decoder;
9596
pub(crate) mod reader;
9697
mod writer;
@@ -195,10 +196,10 @@ impl ParquetMetaData {
195196
ParquetMetaData {
196197
file_metadata,
197198
row_groups,
198-
#[cfg(feature = "encryption")]
199-
file_decryptor: None,
200199
column_index: None,
201200
offset_index: None,
201+
#[cfg(feature = "encryption")]
202+
file_decryptor: None,
202203
}
203204
}
204205

0 commit comments

Comments
 (0)