Skip to content
This repository was archived by the owner on May 6, 2025. It is now read-only.
This repository was archived by the owner on May 6, 2025. It is now read-only.

Loading from TFDS/Parquet without copying files results in files at two locations #74

@coufon

Description

@coufon

The features of loading a TFDS datasets (append_array_record) and Parquet files (append_parquet) don't copy/rewrite the source files. As the consequence, a Space's dataset will be split across two locations: the original files and the new Space storage directory.

To support an option that first copies or moves the source files to the Space storage directory. Note that such copy should be still faster than writing files in the normal append methods.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions