Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow os.PathLike objects in read_* and scan_* functions #17828

Open
edavisau opened this issue Jul 24, 2024 · 1 comment
Open

Allow os.PathLike objects in read_* and scan_* functions #17828

edavisau opened this issue Jul 24, 2024 · 1 comment
Labels
A-input-parsing Area: parsing input arguments A-io Area: reading and writing data accepted Ready for implementation enhancement New feature or an improvement of an existing feature

Comments

@edavisau
Copy link
Contributor

Description

This basically already works: because current normalize_filepath implementation already works with os.PathLike (just needs updated type hints).

The only place where change is needed is here

if isinstance(source, (str, Path)):   #   ->   (str, os.PathLike) instead
    source = normalize_filepath(source, check_not_directory=False)
else:
    source = [
        normalize_filepath(source, check_not_directory=False) for source in source
    ]

The rest of the change would be type hints

Example

import tempfile
import polars as pl

tmp_file = tempfile.NamedTemporaryFile()
pl.DataFrame(dict(a=[1,2,3])).write_parquet(tmp_file.name)

class Foo:
    def __fspath__(self):
        return tmp_file.name

isinstance(Foo(), os.PathLike)
# True

pl.read_parquet(Foo())
# TypeError: 'Foo' object is not iterable

pl.read_parquet([Foo()])
# shape: (3, 1)
# ┌─────┐
# │ a   │
# │ --- │
# │ i64 │
# ╞═════╡
# │ 1   │
# │ 2   │
# │ 3   │
# └─────┘
@edavisau edavisau added the enhancement New feature or an improvement of an existing feature label Jul 24, 2024
@stinodego stinodego added the accepted Ready for implementation label Jul 26, 2024
@stinodego
Copy link
Member

We should be able to support os.PathLike inputs.

A PR is welcome if it includes extensive testing for various pathlike inputs other than pathlib.Path.

@stinodego stinodego added A-io Area: reading and writing data A-input-parsing Area: parsing input arguments labels Jul 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-input-parsing Area: parsing input arguments A-io Area: reading and writing data accepted Ready for implementation enhancement New feature or an improvement of an existing feature
Projects
Status: Ready
Development

No branches or pull requests

2 participants