Skip to content
This repository was archived by the owner on Sep 11, 2023. It is now read-only.

Conversation

BirkhoffG
Copy link
Owner

Before, even if sample_frac is set, we use the entire data for building train_dataset, val_dataset, test_dataset.

When data is very large, this approach could unwantedly blow the memory because building datasets ( train_dataset etc.) assign new memory.

Now, I only load data with first sample_frac of data.

@BirkhoffG BirkhoffG added the bug Something isn't working label Jan 19, 2023
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant