Skip to content

When read_csv is called on a json file modin rises an exception #1386

@gshimansky

Description

@gshimansky

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04):

Ubuntu 19.10

  • Modin version (modin.__version__):

0.7.2+66.g272ac5d.dirty

  • Python version:

Python 3.7.5

  • Code we can use to reproduce:
df1 = pd.read_csv("categories.json", dtype={"one": "int64", "two": "category"})

Describe the problem

Pandas apparently is able to load the following file as a CSV :)

{"one":{"0":111,"1":222,"2":333},"two":{"0":"AAA","1":"BBB","2":"CCC"}}

while on Modin I get an exception like this:

Traceback (most recent call last):
  File "cat_test.py", line 4, in <module>
    df1 = pd.read_csv("categories.json", dtype={"one": "int64", "two": "category"})
  File "/localdisk/gashiman/modin/modin/pandas/io.py", line 111, in parser_func
    return _read(**kwargs)
  File "/localdisk/gashiman/modin/modin/pandas/io.py", line 124, in _read
    pd_obj = BaseFactory.read_csv(**kwargs)
  File "/localdisk/gashiman/modin/modin/data_management/factories.py", line 69, in read_csv
    return cls._determine_engine()._read_csv(**kwargs)
  File "/localdisk/gashiman/modin/modin/data_management/factories.py", line 73, in _read_csv
    return cls.io_cls.read_csv(**kwargs)
  File "/localdisk/gashiman/modin/modin/engines/base/io/file_reader.py", line 29, in read
    query_compiler = cls._read(*args, **kwargs)
  File "/localdisk/gashiman/modin/modin/engines/base/io/text/csv_reader.py", line 199, in _read
    dtypes = cls.get_dtypes(dtypes_ids)
  File "/localdisk/gashiman/modin/modin/backends/pandas/parsers.py", line 61, in get_dtypes
    pandas.concat(cls.materialize(dtypes_ids), axis=1)
  File "/nfs/site/home/gashiman/.local/lib/python3.7/site-packages/pandas/core/reshape/concat.py", line 281, in concat
    sort=sort,
  File "/nfs/site/home/gashiman/.local/lib/python3.7/site-packages/pandas/core/reshape/concat.py", line 329, in __init__
    raise ValueError("No objects to concatenate")
ValueError: No objects to concatenate

Source code / logs

Metadata

Metadata

Assignees

Labels

bug 🦗Something isn't working

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions