-
Notifications
You must be signed in to change notification settings - Fork 8.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HADOOP-18379 rebase feature/HADOOP-18028-s3a-prefetch to trunk #4654
Closed
steveloughran
wants to merge
1
commit into
apache:trunk
from
steveloughran:s3/feature-HADOOP-18028-s3a-prefetch
Closed
HADOOP-18379 rebase feature/HADOOP-18028-s3a-prefetch to trunk #4654
steveloughran
wants to merge
1
commit into
apache:trunk
from
steveloughran:s3/feature-HADOOP-18028-s3a-prefetch
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
steveloughran
changed the title
HADOOP-18028-s3a-prefetch
HADOOP-18379 rebase feature/HADOOP-18028-s3a-prefetch to trunk
Jul 28, 2022
test failure due to more than one stream read. implies that prefetching is still on by default
|
💔 -1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
This is the the a rollup patch of the HADOOP-18028 S3A performance input stream feature branch. Contains HADOOP-18028. High performance S3A input stream (apache#4109) This is the the merge of the HADOOP-18028 S3A performance input stream. This patch on its own is incomplete and must be accompanied by all other commits with HADOOP-18028 in their git commit message. Consult the JIRA for that list Contributed by Bhalchandra Pandit. HADOOP-18180. Replace use of twitter util-core with java futures in S3A prefetching stream (apache#4115) Contributed by PJ Fanning. HADOOP-18177. Document prefetching architecture. (apache#4205) Contributed by Ahmar Suhail HADOOP-18175. fix test failures with prefetching s3a input stream (apache#4212) Contributed by Monthon Klongklaew HADOOP-18231. S3A prefetching: fix failing tests & drain stream async. (apache#4386) * adds in new test for prefetching input stream * creates streamStats before opening stream * updates numBlocks calculation method * fixes ITestS3AOpenCost.testOpenFileLongerLength * drains stream async * fixes failing unit test Contributed by Ahmar Suhail HADOOP-18254. Disable S3A prefetching by default. (apache#4469) Contributed by Ahmar Suhail HADOOP-18190. Collect IOStatistics during S3A prefetching (apache#4458) This adds iOStatisticsConnection to the S3PrefetchingInputStream class, with new statistic names in StreamStatistics. This stream is not (yet) IOStatisticsContext aware. Contributed by Ahmar Suhail. Change-Id: I48f217086531c12d6e2f0f91e39f17054a74d20f
steveloughran
force-pushed
the
s3/feature-HADOOP-18028-s3a-prefetch
branch
from
August 2, 2022 10:59
e23f70a
to
b542d91
Compare
cancelling this and doing it slightly differently
|
💔 -1 overall
This message was automatically generated. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of PR
This is the feature branch feature/HADOOP-18028-s3a-prefetch rebased on trunk to see how yetus likes it.
If all is good I will force push it to the asf branch, so things are up to date.
rebase merge issues
some compilation errors to fix
How was this patch tested?
For code changes:
LICENSE
,LICENSE-binary
,NOTICE-binary
files?