Fix PostgreSQL NUL byte errors and ONNX GPU memory allocation failures #244
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Two issues causing analysis instability: (1) PostgreSQL rejecting artist names containing NULL bytes from corrupted metadata, (2) ONNX Runtime BFCArena allocation failures from GPU memory fragmentation accumulating across track analyses.
Changes
New utility module
tasks/memory_utils.py:sanitize_string_for_db()- strips NULL bytes and control characters before database writescleanup_cuda_memory()- forces CUDA cache clearing and garbage collectioncleanup_onnx_session()- explicit session disposal with immediate GChandle_onnx_memory_error()- detects allocation errors, triggers cleanup, enables retrySessionRecycler- recreates sessions every N tracks to prevent cumulative leaksDatabase string sanitization:
app_helper_artist.pybefore all artist name insertions"Tyler\x00YoungBoy\x00Ty Dolla"→"TylerYoungBoyTy Dolla"Memory management in analyzers:
tasks/clap_analyzer.py,tasks/mulan_analyzer.py- error detection with single retry after cleanuptasks/analysis.py- session recycling every 20 tracks, CUDA cleanup after albumsUsage:
Root cause: GPU memory fragmenting from incomplete cleanup between inferences. BFCArena allocator fails to find contiguous blocks despite sufficient total memory. Python GC timing doesn't align with CUDA memory lifecycle. Solution: explicit cleanup at strategic points + periodic session recreation.
Documentation: See
docs/MEMORY_FIXES.mdfor implementation details and monitoring guidance.Original prompt
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.