Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More trivial implementation of Box<dyn AsyncArrowWriter> and Box<dyn AsyncArrowReader> #6748

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ethe
Copy link
Contributor

@ethe ethe commented Nov 18, 2024

In the current,

impl AsyncFileReader for Box<dyn AsyncFileReader> {
    ...
}

equals to

impl AsyncFileReader for Box<dyn AsyncFileReader + 'static> {
    ...
}

which 'static bound is not necessary, for all 'a, Box<dyn AsyncFileReader + 'a> should be an AsyncFileReader. I removed this bound to make it more trivial

@github-actions github-actions bot added the parquet Changes to the parquet crate label Nov 18, 2024
@ethe ethe changed the title Wilder implementor of Box<dyn AsyncArrowWriter> and Box<dyn AsyncArrowReader> in parquet More trivial implementation of Box<dyn Async Arrow Writer> and Box<dyn AsyncArrowReader> Nov 18, 2024
@ethe ethe changed the title More trivial implementation of Box<dyn Async Arrow Writer> and Box<dyn AsyncArrowReader> More trivial implementation of Box<dyn AsyncArrowWriter> and Box<dyn AsyncArrowReader> Nov 18, 2024
@jayzhan211
Copy link
Contributor

I think adding read and write is not the same as static

@ethe
Copy link
Contributor Author

ethe commented Nov 19, 2024

It is not the same, but a superset of it. 'read and 'write are able to be 'static, and also make non-static for<'a>: Box<dyn AsyncArrowWriter + 'a> be an AsyncArrowWriter

@jayzhan211
Copy link
Contributor

jayzhan211 commented Nov 19, 2024

I'm still not convinced why we need to add lifetime for it. 👀
In my understanding, we should start from the simple one without lifetime until we want to implement with reference that compiler starts to complain about it.

It would be clear if there is an example that compilation failed without lifetime

@ethe
Copy link
Contributor Author

ethe commented Nov 19, 2024

Considering this case:

pub struct ArrowReader<'reader, T> {
    reader: &'reader T,
}

impl<T: AsyncFileReader + Send + Sync> AsyncFileReader for ArrowReader<'_, T> {
    fn get_bytes(
        &mut self,
        range: std::ops::Range<usize>,
    ) -> std::pin::Pin<
        Box<dyn std::future::Future<Output = parquet::errors::Result<bytes::Bytes>> + Send + '_>,
    > {
        todo!()
    }

    fn get_metadata(
        &mut self,
    ) -> std::pin::Pin<
        Box<
            dyn std::future::Future<
                    Output = parquet::errors::Result<Arc<parquet::file::metadata::ParquetMetaData>>,
                > + Send
                + '_,
        >,
    > {
        todo!()
    }
}

fn test<'r, T: AsyncFileReader + Send + Sync>(
    file: ArrowReader<'r, T>,
) -> impl AsyncFileReader + 'r {
    Box::new(file) as Box<dyn AsyncFileReader>
}

test function would not be passed using current implementers, after merging this pr, it will

@ethe
Copy link
Contributor Author

ethe commented Nov 19, 2024

In my understanding, we should start from the simple one without lifetime

Actually it does have one lifetime in the current: 'static, lifetime is just elided rather than non-existent.

If you are still confused, there is a basic explanation about it:
https://quinedot.github.io/rust-learning/dyn-elision-basic.html#impl-headers

Consider using implementations like the following if possible, as they are more general:

// Implemented for all lifetimes
impl Two for Box<dyn Trait + '_> {}

// Implemented for all lifetimes such that the inner lifetime is
// at least as long as the outer lifetime
impl Two for &(dyn Trait + '_) {}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
parquet Changes to the parquet crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants