Skip to content

[ENG-7922] chunky delete_pretrove_data (and --chunksize param)#865

Merged
aaxelb merged 1 commit intoCenterForOpenScience:release/25.2.0from
aaxelb:fix/chunky-delete
May 2, 2025
Merged

[ENG-7922] chunky delete_pretrove_data (and --chunksize param)#865
aaxelb merged 1 commit intoCenterForOpenScience:release/25.2.0from
aaxelb:fix/chunky-delete

Conversation

@aaxelb
Copy link
Contributor

@aaxelb aaxelb commented May 2, 2025

update delete_pretrove_data management command to accept --chunksize param (default 1024) and delete data in small chunks (and from one sourceconfig at a time) instead of deleting all-at-once

local (contrived) example:

root@eac74ef154f9:/code# python manage.py delete_pretrove_data
pre-trove source-configs with deletable rawdata:
	system.v2_push
really DELETE ALL raw metadata records belonging to these source-configs? (y/n)y
system.v2_push: deleting all rawdata...
system.v2_push: deleted 1024
system.v2_push: deleted 1024
system.v2_push: deleted 1024
system.v2_push: deleted 1024
system.v2_push: deleted 904
system.v2_push: done; deleted 5000
deleted 5000 items
root@eac74ef154f9:/code# python manage.py delete_pretrove_data
nothing to delete

ENG-7922

@aaxelb aaxelb marked this pull request as ready for review May 2, 2025 17:24
@coveralls
Copy link

Coverage Status

coverage: 79.141%. remained the same
when pulling 3f158c6 on aaxelb:fix/chunky-delete
into 9bfea80 on CenterForOpenScience:release/25.2.0.

@aaxelb aaxelb changed the title chunky delete_pretrove_data (and --chunksize param) [ENG-7922] chunky delete_pretrove_data (and --chunksize param) May 2, 2025
@aaxelb aaxelb requested a review from mfraezz May 2, 2025 17:32
@aaxelb aaxelb merged commit 1a07924 into CenterForOpenScience:release/25.2.0 May 2, 2025
3 checks passed
@aaxelb aaxelb deleted the fix/chunky-delete branch May 2, 2025 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants