Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TST: sql add more tests (NaN handling, ..) #7100

Merged
merged 1 commit into from
May 13, 2014

Conversation

jorisvandenbossche
Copy link
Member

This PR just adds some tests, not fixes to get them all running.

Summary:

  • NaN writing not working for MySQL (or pymysql), but does work for sqlite and postgresql
  • NaN reading does work for float columns. But not for string columns (converted to None or u'NaN') or for sqlite full float columns
  • NaT in datetime columns only works for sqlite for writing. Writing postgresql gives an error on inserting the date 0001-255-255T00:00:00.

@jreback
Copy link
Contributor

jreback commented May 12, 2014

ok....maybe an issue specifically to track the fixing of these (and sub-link to the issues with checkboxes?)

and if you want to push some to 0.14.1 ok too

@jorisvandenbossche
Copy link
Member Author

Added it to the issue #2754

jorisvandenbossche added a commit that referenced this pull request May 13, 2014
TST: sql add more tests (NaN handling, ..)
@jorisvandenbossche jorisvandenbossche merged commit cd422c2 into pandas-dev:master May 13, 2014
@jorisvandenbossche jorisvandenbossche deleted the sql-tests branch May 13, 2014 10:29
@jreback
Copy link
Contributor

jreback commented May 13, 2014

these are currently breaking on windows because the sqlite is not skipping

should it be?

its having a parse error on the NaT

@jreback
Copy link
Contributor

jreback commented May 13, 2014

========================================================================
ERROR: test_datetime_NaT (pandas.io.tests.test_sql.TestSQLiteAlchemy)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Jeff Reback\Documents\GitHub\pandas\build\lib.win32-3.4\pandas\io\tests\test_sql.py", line 912, in test_datetime_NaT
    result = sql.read_sql_table('test_datetime', self.conn)
  File "C:\Users\Jeff Reback\Documents\GitHub\pandas\build\lib.win32-3.4\pandas\io\sql.py", line 270, in read_sql_table
    parse_dates=parse_dates, columns=columns)
  File "C:\Users\Jeff Reback\Documents\GitHub\pandas\build\lib.win32-3.4\pandas\io\sql.py", line 797, in read_table
    parse_dates=parse_dates, columns=columns)
  File "C:\Users\Jeff Reback\Documents\GitHub\pandas\build\lib.win32-3.4\pandas\io\sql.py", line 609, in read
    data, columns=column_names, coerce_float=coerce_float)
  File "C:\Users\Jeff Reback\Documents\GitHub\pandas\build\lib.win32-3.4\pandas\core\frame.py", line 798, in from_records
    coerce_float=coerce_float)
  File "C:\Users\Jeff Reback\Documents\GitHub\pandas\build\lib.win32-3.4\pandas\core\frame.py", line 4684, in _to_arrays
    data = lmap(tuple, data)
  File "C:\Users\Jeff Reback\Documents\GitHub\pandas\build\lib.win32-3.4\pandas\compat\__init__.py", line 87, in lmap
    return list(map(*args, **kwargs))
  File "C:\Python34-32\lib\site-packages\sqlalchemy-0.9.2-py3.4.egg\sqlalchemy\engine\result.py", line 63, in __iter__
    yield processor(value)
  File "C:\Python34-32\lib\site-packages\sqlalchemy-0.9.2-py3.4.egg\sqlalchemy\processors.py", line 39, in process
    "'%s'" % (type_.__name__, value))
ValueError: Couldn't parse datetime string: '-001--1--1 -1:-1:-1.-00001'

----------------------------------------------------------------------
Ran 23 tests in 3.896s

FAILED (errors=1)

@jorisvandenbossche
Copy link
Member Author

Strange, I tested this on Windows (python 2.7) and it works. Something with the python version?

@jreback
Copy link
Contributor

jreback commented May 14, 2014

its happenin on several version 26-64,33-32,33-64 (where I have things installed)

@jorisvandenbossche
Copy link
Member Author

I can't reproduce this on windows, python 2.7, 64bit, sqlalchemy 0.8.2, so difficult to debug. But I suppose something is different in the datetime parsing between different versions, leading to an error or to NaN.
But maybe just skip this test for now? As it is not really supported? (and it is already skipped for postgresql and mysql)

@jreback
Copy link
Contributor

jreback commented May 15, 2014

yes
just skip for now (and in any event NaT support issue should remain open)
but can deal with in later releases

jorisvandenbossche added a commit that referenced this pull request May 15, 2014
- tests were failing on some combination of platform/version, see #7100, so skip for now until real solution is implemented
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
IO SQL to_sql, read_sql, read_sql_query Testing pandas testing functions or related to the test suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants