Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(datastore): data lost when save table with error #1938

Merged

Conversation

goldenxinxing
Copy link
Contributor

@goldenxinxing goldenxinxing commented Mar 9, 2023

Description

#1916

  1. [Client] add ut for python sdk(dataset and datastore with data of SwObject)
  2. [Server] fix lost data bug when save table with error
  3. [Server] fix load data exception when data is null at the step of insertRecords
  4. [Server] add retry logic when load is invalid

Modules

  • UI
  • Controller
  • Agent
  • Client
  • Python-SDK
  • Others

Checklist

  • run code format and lint check
  • add unit test
  • add necessary doc

@goldenxinxing goldenxinxing added bug 🐛 Something isn't working e2e labels Mar 9, 2023
@goldenxinxing goldenxinxing self-assigned this Mar 9, 2023
@goldenxinxing goldenxinxing changed the title fix(datastore): data lost when table save occur error fix(datastore): data lost when save table with error Mar 9, 2023
@codecov
Copy link

codecov bot commented Mar 9, 2023

Codecov Report

Merging #1938 (a587b24) into main (a24c902) will increase coverage by 0.05%.
The diff coverage is 83.95%.

@@             Coverage Diff              @@
##               main    #1938      +/-   ##
============================================
+ Coverage     82.36%   82.41%   +0.05%     
- Complexity     2063     2068       +5     
============================================
  Files           374      377       +3     
  Lines         20344    20466     +122     
  Branches       1121     1122       +1     
============================================
+ Hits          16757    16868     +111     
- Misses         3076     3087      +11     
  Partials        511      511              
Flag Coverage Δ
controller 73.56% <83.95%> (+0.03%) ⬆️
standalone 90.92% <ø> (+0.01%) ⬆️
unittests 90.92% <ø> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
.../java/ai/starwhale/mlops/datastore/ColumnType.java 89.41% <ø> (+1.03%) ⬆️
...n/java/ai/starwhale/mlops/datastore/DataStore.java 85.32% <ø> (ø)
...ai/starwhale/mlops/datastore/parquet/SwWriter.java 77.77% <77.77%> (ø)
...tarwhale/mlops/datastore/impl/MemoryTableImpl.java 88.98% <81.13%> (-1.08%) ⬇️
...rwhale/mlops/datastore/parquet/SwWriteSupport.java 92.85% <91.66%> (+1.19%) ⬆️
...lops/datastore/parquet/SwParquetWriterBuilder.java 53.57% <100.00%> (+5.57%) ⬆️
...arwhale/mlops/datastore/parquet/SwReadSupport.java 73.33% <100.00%> (+2.96%) ⬆️

... and 16 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@tianweidut
Copy link
Member

LGTM

Copy link
Contributor

@jialeicui jialeicui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jialeicui jialeicui merged commit 3c945a4 into star-whale:main Mar 13, 2023
@goldenxinxing goldenxinxing deleted the fix/data_lost_at_datastore_error branch April 27, 2023 10:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🐛 Something isn't working e2e
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants