Skip to content

Issues: Lightning-AI/litdata

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
23 Open 30 Closed
23 Open 30 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Feat/add support for numpy datatypes in tokensloader enhancement New feature or request
#401 opened Oct 27, 2024 by bhimrazy • Draft
4 tasks
1
@bhimrazy
2
Feature: Add support for numpy datatypes in TokensLoader enhancement New feature or request
#400 opened Oct 27, 2024 by bhimrazy
1
Combine Small StreamingDatasets into 1 Large StreamingDataset enhancement New feature or request
#396 opened Oct 11, 2024 by schopra8
4
Improve CombinedStreamingDataset to handle multiple subdatasets efficiently enhancement New feature or request
#386 opened Oct 2, 2024 by bhimrazy
How can I shut down automatically distributing data when using StreamingDataset? enhancement New feature or request question Further information is requested
#368 opened Sep 12, 2024 by ygtxr1997
3
Lazyload subsamples if subsample=1.0 enhancement New feature or request question Further information is requested
#339 opened Aug 21, 2024 by deependujha
Use different batch sizes in CombinedStreamingDataset enhancement New feature or request help wanted Extra attention is needed
#327 opened Aug 10, 2024 by schopra8
1
Add support for multi sample item in optimize and yielding from the _getitem_ of the StreamingDataset enhancement New feature or request help wanted Extra attention is needed
#317 opened Aug 8, 2024 by tchaton
Explore about integrating homomorphic encryption enhancement New feature or request help wanted Extra attention is needed
#313 opened Aug 7, 2024 by bhimrazy
Investigate keeping the content of the downloaded chunks in RAM instead of writing it to file. enhancement New feature or request help wanted Extra attention is needed
#291 opened Aug 1, 2024 by tchaton
Add training mode compression for zstd enhancement New feature or request help wanted Extra attention is needed
#283 opened Jul 31, 2024 by tchaton
Add support for sample windowing enhancement New feature or request help wanted Extra attention is needed
#282 opened Jul 31, 2024 by tchaton
Integration with DAG framework Prefect enhancement New feature or request help wanted Extra attention is needed
#226 opened Jul 12, 2024 by tchaton
@deependujha
1
Add support for the reduce operator enhancement New feature or request help wanted Extra attention is needed
#225 opened Jul 12, 2024 by tchaton
1
Add support for parquet files for storing the chunks enhancement New feature or request help wanted Extra attention is needed
#191 opened Jun 27, 2024 by tchaton
3
LitData doesn't support s3 bucket connection outside server enhancement New feature or request help wanted Extra attention is needed
#183 opened Jun 25, 2024 by sanyalsunny111
11
Using fsspec to download files enhancement New feature or request help wanted Extra attention is needed
#181 opened Jun 23, 2024 by samsja
1
5
Stream selected channels enhancement New feature or request help wanted Extra attention is needed
#128 opened May 13, 2024 by robmarkcole
TPU support enhancement New feature or request
#79 opened Mar 26, 2024 by miguelalba96
5
Allow a StreamingDataset to wrap around when running in a CombinedStreamingDataset enhancement New feature or request
#74 opened Mar 14, 2024 by lantiga
2
map device management enhancement New feature or request
#15 opened Feb 23, 2024 by ethanwharris
2
Fast random access for StreamingDataset enhancement New feature or request
#14 opened Feb 23, 2024 by ethanwharris
2
Support StreamingDataLoader passed to map enhancement New feature or request
#13 opened Feb 23, 2024 by ethanwharris
1
ProTip! Find all open issues with in progress development work with linked:pr.