Patch/pq memory opt#141

Merged

singjc merged 16 commits intoPyProphet:masterfrom

singjc:patch/pq_memory_opt

May 18, 2025

Commits on May 16, 2025

feat: Add functionality to extract and export memory usage information in convert_osw_to_parquet
singjc
committed
feat: move format_bytes function to data handling
singjc
committed
feat: Improve coalesce handling and data processing in convert_osw_to_parquet
singjc
committed

Commits on May 17, 2025

feat: Remove batch_size option and optimize memory usage in parquet export
singjc
committed
feat: Include FEATURE_ID in feature transition columns for improved data extraction in convert_osw_to_parquet
singjc
committed
feat: Include RUN_ID in feature transition data extraction for improved data processing in convert_osw_to_parquet
singjc
committed