Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Move parquet async functionality behind feature flag #1586

Merged
merged 1 commit into from
Oct 28, 2023

Conversation

MostlyAmiable
Copy link
Contributor

Addresses #1296 with a new io_parquet_async feature flag.

Since this is a breaking change for anyone using the async functions through the io_parquet flag, let me know if you want me to bump the version or just add io_parquet_async as a dependency of io_parquet (are circular feature dependencies allowed?).

cargo check output:

$ cargo clean
$ cargo check -F io_parquet
   Compiling proc-macro2 v1.0.69
   Compiling unicode-ident v1.0.12
   Compiling libc v0.2.149
   Compiling semver v1.0.20
   Compiling version_check v0.9.4
   Compiling autocfg v1.1.0
    Checking cfg-if v1.0.0
   Compiling serde v1.0.189
    Checking zerocopy v0.7.15
    Checking array-init-cursor v0.2.0
    Checking fallible-streaming-iterator v0.1.9
    Checking once_cell v1.18.0
    Checking streaming-decompression v0.1.2
   Compiling ahash v0.8.6
    Checking planus v0.3.1
    Checking parquet-format-safe v0.2.4
   Compiling num-traits v0.2.17
   Compiling seq-macro v0.3.5
    Checking base64 v0.21.5
    Checking simdutf8 v0.1.4
    Checking foreign_vec v0.1.0
    Checking streaming-iterator v0.1.9
    Checking either v1.9.0
    Checking hash_hasher v2.0.3
   Compiling quote v1.0.33
    Checking parquet2 v0.17.2
    Checking dyn-clone v1.0.14
   Compiling rustc_version v0.4.0
    Checking ethnum v1.4.0
   Compiling syn v2.0.38
    Checking getrandom v0.2.10
   Compiling arrow2 v0.17.4 (/Users/blevin/dev/lib/arrow2)
    Checking hashbrown v0.14.2
    Checking chrono v0.4.31
   Compiling serde_derive v1.0.189
   Compiling bytemuck_derive v1.5.0
    Checking bytemuck v1.14.0
    Checking arrow-format v0.8.1
    Finished dev [unoptimized + debuginfo] target(s) in 7.99s
$ cargo check -F io_parquet_async
   Compiling futures-core v0.3.28
   Compiling futures-channel v0.3.28
   Compiling futures-task v0.3.28
    Checking pin-project-lite v0.2.13
    Checking futures-sink v0.3.28
   Compiling slab v0.4.9
   Compiling futures-util v0.3.28
   Compiling syn v2.0.38
    Checking pin-utils v0.1.0
    Checking futures-io v0.3.28
    Checking memchr v2.6.4
   Compiling async-trait v0.1.74
   Compiling arrow2 v0.17.4 (/Users/blevin/dev/lib/arrow2)
   Compiling futures-macro v0.3.28
   Compiling async-stream-impl v0.3.5
   Compiling serde_derive v1.0.189
   Compiling bytemuck_derive v1.5.0
    Checking async-stream v0.3.5
    Checking bytemuck v1.14.0
    Checking serde v1.0.189
    Checking futures-executor v0.3.28
    Checking futures v0.3.28
    Checking parquet-format-safe v0.2.4
    Checking arrow-format v0.8.1
    Checking parquet2 v0.17.2
    Finished dev [unoptimized + debuginfo] target(s) in 8.41s

@codecov
Copy link

codecov bot commented Oct 26, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (b073454) 83.39% compared to head (7b82982) 83.40%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1586      +/-   ##
==========================================
+ Coverage   83.39%   83.40%   +0.01%     
==========================================
  Files         391      391              
  Lines       43008    43008              
==========================================
+ Hits        35866    35871       +5     
+ Misses       7142     7137       -5     
Files Coverage Δ
src/io/parquet/read/mod.rs 100.00% <ø> (ø)
src/io/parquet/read/row_group.rs 96.53% <ø> (ø)
src/io/parquet/write/mod.rs 90.09% <ø> (ø)

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@MostlyAmiable
Copy link
Contributor Author

Looks like the Miri check has been getting cancelled for the past month? 🤔

@sundy-li
Copy link
Collaborator

LGTM

@sundy-li sundy-li merged commit 3ddc6a1 into jorgecarleitao:main Oct 28, 2023
24 of 25 checks passed
@MostlyAmiable MostlyAmiable deleted the io-parquet-async branch October 28, 2023 19:12
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants