-
Notifications
You must be signed in to change notification settings - Fork 2.7k
feat: DIA-2062: Batch writes for cloud storage import #7372
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
🔍 Existing Issues For ReviewYour pull request is modifying functions with the following pre-existing issues: 📄 File: label_studio/io_storages/base_models.py
Did you find this useful? React with a 👍 or 👎 |
✅ Deploy Preview for label-studio-storybook ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
✅ Deploy Preview for label-studio-docs-new-theme ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
✅ Deploy Preview for heartex-docs ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #7372 +/- ##
===========================================
- Coverage 77.27% 77.22% -0.06%
===========================================
Files 190 190
Lines 14697 14721 +24
===========================================
+ Hits 11357 11368 +11
- Misses 3340 3353 +13
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
/fm sync |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe we should add FF and keep the old implementation in place until we are completely sure that the new one is safe
Changes how cloud storage import works so that we are bulk_create-ing Tasks, Annotations, and Predictions
Before we were calling add_task for every single new key (path) from cloud storage. Now we will create tasks in batches (setting controlled to 50 currently)