Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When read_csv is called on a json file modin rises an exception #1386

Closed
gshimansky opened this issue Apr 22, 2020 · 0 comments · Fixed by #2076
Closed

When read_csv is called on a json file modin rises an exception #1386

gshimansky opened this issue Apr 22, 2020 · 0 comments · Fixed by #2076
Assignees
Labels
bug 🦗 Something isn't working
Milestone

Comments

@gshimansky
Copy link
Collaborator

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04):

Ubuntu 19.10

  • Modin version (modin.__version__):

0.7.2+66.g272ac5d.dirty

  • Python version:

Python 3.7.5

  • Code we can use to reproduce:
df1 = pd.read_csv("categories.json", dtype={"one": "int64", "two": "category"})

Describe the problem

Pandas apparently is able to load the following file as a CSV :)

{"one":{"0":111,"1":222,"2":333},"two":{"0":"AAA","1":"BBB","2":"CCC"}}

while on Modin I get an exception like this:

Traceback (most recent call last):
  File "cat_test.py", line 4, in <module>
    df1 = pd.read_csv("categories.json", dtype={"one": "int64", "two": "category"})
  File "/localdisk/gashiman/modin/modin/pandas/io.py", line 111, in parser_func
    return _read(**kwargs)
  File "/localdisk/gashiman/modin/modin/pandas/io.py", line 124, in _read
    pd_obj = BaseFactory.read_csv(**kwargs)
  File "/localdisk/gashiman/modin/modin/data_management/factories.py", line 69, in read_csv
    return cls._determine_engine()._read_csv(**kwargs)
  File "/localdisk/gashiman/modin/modin/data_management/factories.py", line 73, in _read_csv
    return cls.io_cls.read_csv(**kwargs)
  File "/localdisk/gashiman/modin/modin/engines/base/io/file_reader.py", line 29, in read
    query_compiler = cls._read(*args, **kwargs)
  File "/localdisk/gashiman/modin/modin/engines/base/io/text/csv_reader.py", line 199, in _read
    dtypes = cls.get_dtypes(dtypes_ids)
  File "/localdisk/gashiman/modin/modin/backends/pandas/parsers.py", line 61, in get_dtypes
    pandas.concat(cls.materialize(dtypes_ids), axis=1)
  File "/nfs/site/home/gashiman/.local/lib/python3.7/site-packages/pandas/core/reshape/concat.py", line 281, in concat
    sort=sort,
  File "/nfs/site/home/gashiman/.local/lib/python3.7/site-packages/pandas/core/reshape/concat.py", line 329, in __init__
    raise ValueError("No objects to concatenate")
ValueError: No objects to concatenate

Source code / logs

@gshimansky gshimansky added the bug 🦗 Something isn't working label Apr 22, 2020
@devin-petersohn devin-petersohn added this to the 0.7.4 milestone Apr 22, 2020
@devin-petersohn devin-petersohn modified the milestones: 0.8.0, 0.8.1 Jul 29, 2020
@prutskov prutskov self-assigned this Sep 10, 2020
prutskov added a commit to prutskov/modin that referenced this issue Sep 15, 2020
anmyachev pushed a commit that referenced this issue Sep 15, 2020
aregm pushed a commit to aregm/modin that referenced this issue Sep 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🦗 Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants