HDDS-13660. Ozone client support readVectored. #9402
Conversation
|
Thanks @ashishkumar50 for the patch. I have ported the corresponding contract test from Hadoop, which shows that some input validation is missing (negative length and offset, overlapping ranges, same ranges). Please feel free to pick it from adoroszlai@ee11cf3 and include in this PR. https://github.com/adoroszlai/ozone/actions/runs/19905480757/job/57061761264#step:13:5599 |
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/MultipartInputStream.java
Outdated
Show resolved
Hide resolved
hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/OzoneInputStream.java
Outdated
Show resolved
Hide resolved
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/OzoneFSInputStream.java
Outdated
Show resolved
Hide resolved
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/OzoneFSInputStream.java
Outdated
Show resolved
Hide resolved
hadoop-ozone/ozonefs-common/src/main/java/org/apache/hadoop/fs/ozone/OzoneFSInputStream.java
Outdated
Show resolved
Hide resolved
(cherry picked from commit ee11cf3)
|
@adoroszlai Thanks for the review, handled comments. |
| (offset, buffer) -> readRangeData(offset, buffer, initialPosition)); | ||
| } finally { | ||
| // Restore position | ||
| synchronized (this) { |
There was a problem hiding this comment.
what would happen in case of concurrent readVectored() calls?
There was a problem hiding this comment.
Synchronized now to avoid race condition in same concurrent stream reads.
hadoop-hdds/client/src/main/java/org/apache/hadoop/hdds/scm/storage/MultipartInputStream.java
Outdated
Show resolved
Hide resolved
hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/utils/VectoredReadUtils.java
Show resolved
Hide resolved
| }); | ||
| } finally { | ||
| // Restore position | ||
| synchronized (this) { |
There was a problem hiding this comment.
what would happen in case of concurrent readVectored() calls?
There was a problem hiding this comment.
Synchronized now to avoid race condition in same concurrent stream reads.
5 participants
What changes were proposed in this pull request?
Ozone client support readVectored which allows reading multiple file ranges in parallel.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-13660
How was this patch tested?
Tests are added