You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Reading these datasets into a Daft dataframe, and providing an easy API for loading any Pytorch dataset into Daft, would be very useful for performing pre-processing using a DataFrame API
Daft Dataframes .to_pytorch_dataloader()
Daft Dataframes can provide a Pytorch Dataloader API for performing distributed training or inference. This should work well for both local and distributed training. This needs to be scoped out more!
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Summary
Pytorch is used for distributed training, model inference etc. The most common abstraction used for data ingestion is a Pytorch Dataloader
Machine learning is an integral part of the Daft ecosystem, and having integrations with Pytorch will be extremely useful for Daft's users.
Read from Pytorch Dataset
Pytorch provides some datasets out-of-the-box:
Reading these datasets into a Daft dataframe, and providing an easy API for loading any Pytorch dataset into Daft, would be very useful for performing pre-processing using a DataFrame API
Daft Dataframes .to_pytorch_dataloader()
Daft Dataframes can provide a Pytorch Dataloader API for performing distributed training or inference. This should work well for both local and distributed training. This needs to be scoped out more!
Beta Was this translation helpful? Give feedback.
All reactions