Commits

Breadcrumbs

History for

onmain

User selector

All users

Datepicker

All time

Commit History

Commits on Nov 27, 2024

• [CELEBORN-1700] Flink supports fallback to vanilla Flink built-in shuffle implementation

  Show description for 9cd6d96
  

  SteNicholas

  authored and

  RexXiong

  committed
  9cd6d96

  Copy full SHA for 9cd6d96

Commits on Nov 20, 2024

• [CELEBORN-1719] Introduce celeborn.client.spark.stageRerun.enabled with alternative celeborn.client.spark.fetch.throwsFetchFailure to enable spark stage rerun

  Show description for 1d0032b
  SteNicholas

  committed
  1d0032b

  Copy full SHA for 1d0032b

Commits on Nov 12, 2024

• [CELEBORN-1685][FOLLOWUP] CelebornShuffleFallbackPolicyRunner should compute shuffleFallbackCounts via class name of ShuffleFallbackPolicy implementation

  Show description for 14d9cd1
  

  SteNicholas

  authored and

  RexXiong

  committed
  14d9cd1

  Copy full SHA for 14d9cd1

• [CELEBORN-1071] Support stage rerun for shuffle data lost

  Show description for 42d5d42
  

  FMX

  authored and

  RexXiong

  committed
  42d5d42

  Copy full SHA for 42d5d42

Commits on Nov 11, 2024

• [CELEBORN-1685] ShuffleFallbackPolicy supports ShuffleFallbackCount metric

  Show description for 169b6f6
  

  SteNicholas

  authored and

  RexXiong

  committed
  169b6f6

  Copy full SHA for 169b6f6

• [CELEBORN-1686] Avoid return the same pushTaskQueue

  Show description for 8f34d15
  

  cxzl25

  authored and

  RexXiong

  committed
  8f34d15

  Copy full SHA for 8f34d15

Commits on Nov 7, 2024

• [CELEBORN-1680] Introduce ShuffleFallbackCount metrics

  Show description for f1bda46
  

  turboFei

  authored and

  FMX

  committed
  f1bda46

  Copy full SHA for f1bda46

Commits on Nov 6, 2024

• [CELEBORN-1671] CelebornShuffleReader will try replica if create client failed

  Show description for 7dcd259
  

  FMX

  authored and

  RexXiong

  committed
  7dcd259

  Copy full SHA for 7dcd259

Commits on Oct 30, 2024

• [CELEBORN-1577][PHASE1] Storage quota should support interrupt shuffle

  Show description for 6ad02f1
  

  leixm

  authored and

  RexXiong

  committed
  6ad02f1

  Copy full SHA for 6ad02f1

Commits on Oct 22, 2024

• [CELEBORN-1663][FOLLOWUP] Only register appShuffleDeterminate if stage using celeborn for shuffle

  Show description for 1e77f01
  

  s0nskar

  authored and

  SteNicholas

  committed
  1e77f01

  Copy full SHA for 1e77f01

• [CELEBORN-1663] Only register appShuffleDeterminate if stage using celeborn for shuffle

  Show description for 813b45f
  

  s0nskar

  authored and

  RexXiong

  committed
  813b45f

  Copy full SHA for 813b45f

Commits on Oct 17, 2024

• [CELEBORN-1648] Refine AppUniqueId with UUID suffix

  Show description for eb49ed7
  

  chenkovsky

  authored and

  RexXiong

  committed
  eb49ed7

  Copy full SHA for eb49ed7

Commits on Oct 15, 2024

• [CELEBORN-1645] Introduce ShuffleFallbackPolicy to support custom implementation of shuffle fallback policy for CelebornShuffleFallbackPolicyRunner

  Show description for 298eadb
  

  SteNicholas

  authored and

  RexXiong

  committed
  298eadb

  Copy full SHA for 298eadb

Commits on Oct 9, 2024

• [CELEBORN-474][FOLLOWUP] Use JavaUtils#newConcurrentHashMap to speed up ConcurrentHashMap#computeIfAbsent

  Show description for b3bf68e
  

  SteNicholas

  authored and

  FMX

  committed
  b3bf68e

  Copy full SHA for b3bf68e

Commits on Sep 27, 2024

• [CELEBORN-1616] Shade com.google.thirdparty to prevent dependency conflicts

  Show description for 3b6dc5b
  

  akpatnam25

  authored and

  FMX

  committed
  3b6dc5b

  Copy full SHA for 3b6dc5b

Commits on Sep 6, 2024

• [CELEBORN-1587] Change to debug logging on client side for SortBasedPusher trigger push

  Show description for 9ec345b
  

  s0nskar

  authored and

  FMX

  committed
  9ec345b

  Copy full SHA for 9ec345b

Commits on Sep 5, 2024

• [CELEBORN-912][FOLLOWUP] Support columnar shuffle for Spark 3.5

  Show description for 60dbb3e
  

  SteNicholas

  authored and

  kerwin-zk

  committed
  60dbb3e

  Copy full SHA for 60dbb3e

• [CELEBORN-912][FOLLOWUP] Support columnar shuffle for Spark 3.5

  Show description for cd91604
  

  SteNicholas

  authored and

  kerwin-zk

  committed
  cd91604

  Copy full SHA for cd91604

Commits on Aug 30, 2024

• [CELEBORN-1496] Differentiate map results with only different stageAttemptId

  Show description for 3853075
  

  3 peoplecommitted
  3853075

  Copy full SHA for 3853075

Commits on Aug 29, 2024

• [CELEBORN-1565] Introduce warn-unused-import in Scala

  Show description for 3ee672e
  cxzl25

  committed
  3ee672e

  Copy full SHA for 3ee672e

Commits on Aug 21, 2024

• [CELEBORN-1544][FOLLOWUP] ShuffleWriter needs to catch exception and call abort to avoid memory leaks

  Show description for bc3bd46
  

  cxzl25

  authored and

  waitinfuture

  committed
  bc3bd46

  Copy full SHA for bc3bd46

Commits on Aug 12, 2024

• [CELEBORN-1518] Add support for Apache Spark barrier stages

  Show description for 3234bef
  

  mridulm

  authored and

  waitinfuture

  committed
  3234bef

  Copy full SHA for 3234bef

Commits on Aug 2, 2024

• [CELEBORN-1544] ShuffleWriter needs to call close finally to avoid memory leaks

  Show description for 48ce9b9
  

  cxzl25

  authored and

  waitinfuture

  committed
  48ce9b9

  Copy full SHA for 48ce9b9

Commits on Jul 25, 2024

• [CELEBORN-1190][FOLLOWUP] Fix WARNING of error prone

  Show description for c0ca952
  

  SteNicholas

  authored and

  Mridul Muralidharan

  committed
  c0ca952

  Copy full SHA for c0ca952

Commits on Jul 18, 2024

• [CELEBORN-1446] Enable chunk prefetch when initialize CelebornInputStream

  Show description for d692e49
  

  waitinfuture

  authored and

  RexXiong

  committed
  d692e49

  Copy full SHA for d692e49

• [CELEBORN-1515] SparkShuffleManager should set lifecycleManager to null after stopping lifecycleManager in Spark 2

  Show description for 0dc84a5
  

  SteNicholas

  authored and

  pan3793

  committed
  0dc84a5

  Copy full SHA for 0dc84a5

Commits on Jul 11, 2024

• [CELEBORN-1500] Filter out empty InputStreams

  Show description for 1ed9245
  

  waitinfuture

  authored and

  FMX

  committed
  1ed9245

  Copy full SHA for 1ed9245

• [CELEBORN-1495] CelebornColumnDictionary supports dictionary of float and double column type

  Show description for 70e3b24
  

  SteNicholas

  authored and

  FMX

  committed
  70e3b24

  Copy full SHA for 70e3b24

Commits on Jun 13, 2024

• [CELEBORN-775][FOLLOWUP] Fix executorCores calculation in SparkShuffleManager for Spark2 local mode

  Show description for e177a20
  

  cxzl25

  authored and

  SteNicholas

  committed
  e177a20

  Copy full SHA for e177a20

Commits on May 17, 2024

• [CELEBORN-1402] SparkShuffleManager print warning log for spark.executor.userClassPathFirst=true with ShuffleManager defined in user jar

  Show description for 9908035
  SteNicholas

  committed
  9908035

  Copy full SHA for 9908035

Commits on May 16, 2024

• [CELEBORN-1422] Remove tmpRecords array when collecting written count metrics

  Show description for 90db37b
  

  onebox-li

  authored and

  waitinfuture

  committed
  90db37b

  Copy full SHA for 90db37b

• [CELEBORN-1424] Fix getChunk NPE when enable local read

  Show description for cbaef74
  

  onebox-li

  authored and

  waitinfuture

  committed
  cbaef74

  Copy full SHA for cbaef74

Commits on May 15, 2024

• [CELEBORN-1369][FOLLOWUP] Improve docs for shuffle fallback policy

  Show description for e66d509
  

  pan3793

  and

  cfmcgrady

  committed
  e66d509

  Copy full SHA for e66d509

Commits on May 14, 2024

• [CELEBORN-1419] Avoid adding shuffle id repeatedly

  Show description for 0591959
  

  huangxiaopingRD

  authored and

  SteNicholas

  committed
  0591959

  Copy full SHA for 0591959

Commits on May 3, 2024

• [CELEBORN-1369] Support for disable fallback to Spark's default shuffle

  Show description for 7b1645f
  

  xinyuwang1

  and

  waitinfuture

  committed
  7b1645f

  Copy full SHA for 7b1645f

Pagination

Previous

Next