Commits

Branch selector

 master

User selector

anishshri-db

Datepicker

All time

Commit History

Commits on Feb 1, 2025

• [SPARK-51047][SS][TESTS] Add tests to verify scan ordering for non-zero start ordinals as well as non-ascending ordinals

  Show description for 5cd25fc
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  5cd25fc

  Copy full SHA for 5cd25fc

• [SPARK-51044][SS][TESTS] Add ordering related tests for list state

  Show description for 23704d5
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  23704d5

  Copy full SHA for 23704d5

• [SPARK-50967][SS] Add option to skip emitting initial state keys within the FMGWS operator

  Show description for cfc3f1f
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  cfc3f1f

  Copy full SHA for cfc3f1f

Commits on Dec 14, 2024

• [SPARK-50526][SS] Add store encoding format conf into offset log and block non supported stateful operators from using avro

  Show description for 3db46bf
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  3db46bf

  Copy full SHA for 3db46bf

Commits on Dec 10, 2024

• [SPARK-50513][SS][SQL] Split EncoderImplicits from SQLImplicits and provide helper object within StatefulProcessor to access underlying SQL Encoder related implicit functions

  Show description for 03c5799
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  03c5799

  Copy full SHA for 03c5799

Commits on Dec 7, 2024

• [SPARK-50516][SS][MINOR] Fix the init state related test to use StreamManualClock

  Show description for fff6793
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  fff6793

  Copy full SHA for fff6793

Commits on Nov 26, 2024

• [SPARK-50387][SS] Update condition for timer expiry and relevant test

  Show description for 69d433b
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  69d433b

  Copy full SHA for 69d433b

Commits on Nov 22, 2024

• [SPARK-50378][SS] Add custom metric for tracking spent for proc initial state in transformWithState

  Show description for d3119fa
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  d3119fa

  Copy full SHA for d3119fa

Commits on Nov 9, 2024

• [SPARK-50273][SS] Improve logging for RocksDB lock acquire/release cases

  Show description for 202400a
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  202400a

  Copy full SHA for 202400a

Commits on Nov 6, 2024

• [SPARK-50162][SS][TESTS] Add tests for loading snapshot with given version for transformWithState operator state and state data source reader

  Show description for 52df0cd
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  52df0cd

  Copy full SHA for 52df0cd

Commits on Nov 5, 2024

• [SPARK-50128][SS] Add stateful processor handle APIs using implicit encoders in Scala

  Show description for 47063a6
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  47063a6

  Copy full SHA for 47063a6

Commits on Nov 1, 2024

• [SPARK-50193][SS] Fix exception handling for validating time modes

  Show description for c027270
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  c027270

  Copy full SHA for c027270

Commits on Oct 21, 2024

• [SPARK-50035][SS] Add support for explicit handleExpiredTimer function part of the stateful processor

  Show description for 738dfa3
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  738dfa3

  Copy full SHA for 738dfa3

Commits on Oct 18, 2024

• [SPARK-49846][SS] Add numUpdatedStateRows and numRemovedStateRows metrics for use with transformWithState operator

  Show description for b1d1f10
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  b1d1f10

  Copy full SHA for b1d1f10

• [SPARK-49802][SS] Add support for read change feed for map and list types used in stateful processors

  Show description for 8280249
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  8280249

  Copy full SHA for 8280249

Commits on Oct 12, 2024

• [SPARK-49930][SS] Ensure that socket updates are flushed on exception from the python worker

  Show description for 3ecfe8e
  

  anishshri-db

  authored and

  HyukjinKwon

  committed
  3ecfe8e

  Copy full SHA for 3ecfe8e

Commits on Sep 30, 2024

• [SPARK-49823][SS] Avoid flush during shutdown in rocksdb close path

  Show description for 885c3fa
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  885c3fa

  Copy full SHA for 885c3fa

Commits on Sep 26, 2024

• [SPARK-49656][SS] Add support for state variables with value state collection types and read change feed options

  Show description for fe1cf32
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  fe1cf32

  Copy full SHA for fe1cf32

Commits on Sep 25, 2024

• [SPARK-49745][SS] Add change to read registered timers through state data source reader

  Show description for d230232
  

  anishshri-db

  and

  HeartSaVioR

  committed
  d230232

  Copy full SHA for d230232

Commits on Sep 24, 2024

• [SPARK-49630][SS] Add flatten option to process collection types with state data source reader

  Show description for 73d6bd7
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  73d6bd7

  Copy full SHA for 73d6bd7

Commits on Sep 20, 2024

• [SPARK-49677][SS] Ensure that changelog files are written on commit and forceSnapshot flag is also reset

  Show description for d4665fa
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  d4665fa

  Copy full SHA for d4665fa

Commits on Sep 11, 2024

• [SPARK-49593][SS] Throw RocksDB exception to the caller on DB close if an error is seen

  Show description for 6d8b842
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  6d8b842

  Copy full SHA for 6d8b842

Commits on Sep 10, 2024

• [SPARK-49575][SS] Add logging for lock release only if acquiredThreadInfo is not null

  Show description for 726999f
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  726999f

  Copy full SHA for 726999f

• [SPARK-49539][SS] Update internal col families start identifier to a different one

  Show description for 3633cfb
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  3633cfb

  Copy full SHA for 3633cfb

Commits on Sep 6, 2024

• [SPARK-49467][SS] Add support for state data source reader and list state

  Show description for 5d1d44f
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  5d1d44f

  Copy full SHA for 5d1d44f

Commits on Aug 29, 2024

• [SPARK-49021][SS] Add support for reading transformWithState value state variables with state data source reader

  Show description for c7a9c1e
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  c7a9c1e

  Copy full SHA for c7a9c1e

Commits on Aug 26, 2024

• [SPARK-49372][SS] Ensure that latestSnapshot is set to none on close to avoid subsequent use

  Show description for fb13344
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  fb13344

  Copy full SHA for fb13344

Commits on Aug 7, 2024

• [SPARK-49115][DOCS] Add docs for state metadata source for operators using schema format v2 such as transformWithState

  Show description for 435a01a
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  435a01a

  Copy full SHA for 435a01a

Commits on Aug 6, 2024

• [SPARK-49048][SS] Add support for reading relevant operator metadata at given batch id

  Show description for 2a75210
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  2a75210

  Copy full SHA for 2a75210

Commits on Jul 30, 2024

• [SPARK-49032][SS] Add schema path in metadata table entry, verify expected version and add operator metadata related test for operator metadata format v2

  Show description for 76bbcc5
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  76bbcc5

  Copy full SHA for 76bbcc5

Commits on Jul 23, 2024

• [SPARK-48957][SS] Return sub-classified error class on state store load for hdfs and rocksdb provider

  Show description for 285489b
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  285489b

  Copy full SHA for 285489b

Commits on Jul 21, 2024

• [SPARK-48891][SS] Refactor StateSchemaCompatibilityChecker to unify all state schema formats

  Show description for 2fb3c35
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  2fb3c35

  Copy full SHA for 2fb3c35

Commits on Jul 17, 2024

• [SPARK-48903][SS] Set the RocksDB last snapshot version correctly on remote load

  Show description for 2ce5d72
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  2ce5d72

  Copy full SHA for 2ce5d72

Commits on Jul 15, 2024

• [SPARK-48886][SS] Add version info to changelog v2 to allow for easier evolution

  Show description for 5ff6a52
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  5ff6a52

  Copy full SHA for 5ff6a52

• [SPARK-48888][SS] Remove snapshot creation based on changelog ops size

  Show description for 23b2849
  

  anishshri-db

  authored and

  HeartSaVioR

  committed
  23b2849

  Copy full SHA for 23b2849

Pagination

Previous

Next