Skip to content

Conversation

@tilowiklundSensmetry
Copy link
Member

@tilowiklundSensmetry tilowiklundSensmetry commented Oct 15, 2025

This introduces new async traits ProjectReadAsync, ReadEnvironmentAsync, and ResolveReadAsync; mirroring ProjectRead, ReadEnvironment, and ResolveRead.

All reqwest-based implementations have been changed to async, using reqwest_middleware::ClientWithMiddleware instead of reqwest::blocking::Client.

Utility classes such as SequentialResolver should now work for both synchronous and asynchoronous use, i.e. SequentialResolver<T> is ResolveReadAsync if T: ResolveReadAsync and is ResolveRead if T: ResolveRead. To the extent possible, they try to intelligently join futures to allow concurrency.

Utility classes AsAsync[Project|Environment|Resolver]<T> and AsSync[Project|Environment|Resolver]Tokio<T> allow treating a synchronous implementation as a (trivially) asynchronous one, and to treat an asynchronous one as if it were synchronous (if provided with a tokio runtime). Traits functions to_async and to_sync_tokio have been added to all the relvant traits.

The CLI utilises the new async version of HTTP implementations, at the cost of having to pass around an Arc<tokio::runtime::Runtime>. This is a bit messy, but, I think, good enough for the moment.

NOTE: I have (for now) removed support for range requests when accessing kpar archives. In practice this was (at the moment) slower, and I couldn't find a readily available, simple, async way of implementing it.

@tilowiklundSensmetry
Copy link
Member Author

@vytautas-astrauskas-sensmetry @victor-linroth-sensmetry @andrius-puksta-sensmetry I will clarify later today what would be good to review.

@tilowiklundSensmetry
Copy link
Member Author

@vytautas-astrauskas-sensmetry @victor-linroth-sensmetry @andrius-puksta-sensmetry I will clarify later today what would be good to review.

My main concerns at the moment are the async versions of the traits. I did a very naive translation from synchronous constructs to futures constructs, simply replacing Iterators by Streams, made all functions async, and replaced Readers by futures::AsyncReaders. The traits are here:

There are lots of things to improve about these traits in general (for example splitting off Ext-traits and improving naming), for now I'm only concerned with the async aspects in particular.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skimmed through. I would need to start using async more extensively to see whether the design works since I do not have much experience with it. I would suggest merging and if needed changing things later.

The only thing that I did not get is why there is much commented out code. Is all of it related to range requests?

@tilowiklundSensmetry
Copy link
Member Author

Skimmed through. I would need to start using async more extensively to see whether the design works since I do not have much experience with it. I would suggest merging and if needed changing things later.

The only thing that I did not get is why there is much commented out code. Is all of it related to range requests?

This is still effectively a draft merge request.

@tilowiklundSensmetry
Copy link
Member Author

Ok, this appears to be mostly working now, I will try to fix the bindings and address comments.

Signed-off-by: Tilo Wiklund <tilo.wiklund@sensmetry.com>
tilowiklundSensmetry and others added 5 commits November 4, 2025 10:52
Signed-off-by: Tilo Wiklund <tilo.wiklund@sensmetry.com>
Signed-off-by: Tilo Wiklund <tilo.wiklund@sensmetry.com>
Signed-off-by: Tilo Wiklund <tilo.wiklund@sensmetry.com>
Co-authored-by: Andrius Pukšta <andrius.puksta@sensmetry.com>
Signed-off-by: Tilo Wiklund <75035892+tilowiklundSensmetry@users.noreply.github.com>
Signed-off-by: Tilo Wiklund <tilo.wiklund@sensmetry.com>
@tilowiklundSensmetry tilowiklundSensmetry enabled auto-merge (squash) November 5, 2025 11:42
@tilowiklundSensmetry tilowiklundSensmetry merged commit 0ed876a into main Nov 5, 2025
27 of 31 checks passed
@tilowiklundSensmetry tilowiklundSensmetry deleted the tw/async branch November 5, 2025 11:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants