Provide an `async` ParquetReader for arrow

[alamb]

Note: migrated from original JIRA: https://issues.apache.org/jira/browse/ARROW-10307

The aim of this issue is to discuss and try to implement async in the Parquet crate for read traits.

It focuses on the read part to limit the complexity and impact of the changes. The design choices should also make sense for the write part.

Related issues:
ARROW-9275 is a more generic and abstract discussion about async. This issue focuses on Parquet read

ARROW-9464 focuses on threading in datafusion but overlaps with this issue when datafusion reads from parquet

 

[alamb]

There is a PR in arrow2 for such functionality: jorgecarleitao/arrow2#260 which may serve as an inspiration

[alamb]

The approach that @jorgecarleitao took in jorgecarleitao/arrow2#260 is quite clever. Rather than a single struct that can read parquet files synchronously and asynchronously, I think he effectively added a second API for reading the required portions of the files into memory buffers and then uses shared encoding/decoding logic with the serialized reader.

Thus, one idea for adding async support to the parquet crate might be to follow this example and create a new reader like AsyncFileReader (alongside the existing SerializedFileReader) that handles the I/O to fetch the required parts (e.g. fetching the bytes that contain metadata, or encoded pages), and then calls into the existing encoder/decoder logic

Something like

               ┌────────────────────────────┐                
               │ Existing common encoding + │                
               │decoding logic that operates│                
               │     on bytes in memory     │                
               └────────────────────────────┘                
                              ▲                              
                 ┌────────────┴──────────┐                   
                 │                       │                   
                 │                       │                   
            .─────────.             .─────────.              
         ,─'           '─.       ,─'           '─.           
        ;    Logic to     :     ;  new logic to   :          
        :   read bytes    ;     :   read bytes    ;          
         ╲ synchronously ╱       ╲asynchronously ╱           
          '─.         ,─'         '─.         ,─'            
             `───────'               `───────'               
                 ▲                       ▲                   
            ┌────┘                       └──────┐            
            │                                   │            
            │                                   │            
┌───────────────────────┐           ┌───────────────────────┐
│ SerializedFileReader  │           │    AsyncFileReader    │
└───────────────────────┘           └───────────────────────┘
                                                             
    existing parquet                           new           
          crate                       entrypoint for async   
                                             reader          

Here is the current read API:
https://docs.rs/parquet/5.3.0/parquet/file/reader/index.html

cc @yjshen

[alamb]

FYI @yjshen @neverchanje , @tustvold has created a proof of concept of a async parquet reader #1154