Commits on Aug 2, 2022

1. HADOOP-18028. High performance S3A input stream. …

   This is the the a rollup patch of the HADOOP-18028 S3A performance input stream
   feature branch.
   
   Contains
   
   HADOOP-18028. High performance S3A input stream (apache#4109)
   
   This is the the merge of the HADOOP-18028 S3A performance input stream.
   This patch on its own is incomplete and must be accompanied by all other commits
   with HADOOP-18028 in their git commit message. Consult the JIRA for that list
   
   Contributed by Bhalchandra Pandit.
   
   HADOOP-18180. Replace use of twitter util-core with java futures in S3A prefetching stream (apache#4115)
   
   Contributed by PJ Fanning.
   
   HADOOP-18177. Document prefetching architecture. (apache#4205)
   
   Contributed by Ahmar Suhail
   
   HADOOP-18175. fix test failures with prefetching s3a input stream (apache#4212)
   
   Contributed by Monthon Klongklaew
   
   HADOOP-18231.  S3A prefetching: fix failing tests & drain stream async.  (apache#4386)
   
   * adds in new test for prefetching input stream
   * creates streamStats before opening stream
   * updates numBlocks calculation method
   * fixes ITestS3AOpenCost.testOpenFileLongerLength
   * drains stream async
   * fixes failing unit test
   
   Contributed by Ahmar Suhail
   
   HADOOP-18254. Disable S3A prefetching by default. (apache#4469)
   
   Contributed by Ahmar Suhail
   
   HADOOP-18190. Collect IOStatistics during S3A prefetching (apache#4458)
   
   This adds iOStatisticsConnection to the S3PrefetchingInputStream class, with
   new statistic names in StreamStatistics.
   
   This stream is not (yet) IOStatisticsContext aware.
   
   Contributed by Ahmar Suhail.
   
   Change-Id: I48f217086531c12d6e2f0f91e39f17054a74d20f

   

   steveloughran committed Aug 2, 2022
   Configuration menu

   • View commit details
   • Copy full SHA for b542d91
   • Browse repository at this point

   Copy the full SHA

   b542d91 View commit details

   Browse the repository at this point in the history