Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: io.sql.write_frame(if_exists='replace') not working as expected (with fix) #2971

Closed
davidstackio opened this issue Mar 5, 2013 · 5 comments
Labels
Blocker Blocking issue or pull request for an upcoming release Bug IO Data IO issues that don't fit into a more specific label IO SQL to_sql, read_sql, read_sql_query
Milestone

Comments

@davidstackio
Copy link

If if_exists='replace' and the table already exists, then the following error occurs:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\site-packages\pandas\io\sql.py", line 202, in write_frame
    cur.execute(create)
sqlite3.OperationalError: no such table: aggregatedData

This is the code I ran with no issue:
psql.write_frame(outDf, 'aggregatedData', conn)

This is the code I ran to get the above error:
psql.write_frame(outDf, 'aggregatedData', conn, if_exists='replace')

If the same code is re-run after the error it will work as expected because the table is not there anymore. Running the code a third time will cause the error; the error happens every other run.

Here's the fix I implemented in pandas.io.sql.py (~line 191):

    # create or drop-recreate if necessary
    create = None
    if exists and if_exists == 'replace':
        create = "DROP TABLE %s" % name
        cur = con.cursor()
        cur.execute(create)
        cur.close()
        create = get_schema(frame, name, flavor)

Basically, I just dropped the existing table and set it to be recreated later in the code.

@iamlemec
Copy link
Contributor

I can confirm this. The proposed fix worked well for me.

@jreback
Copy link
Contributor

jreback commented Jul 20, 2013

related to #4163.

@hayd hayd mentioned this issue Jul 20, 2013
20 tasks
davidshinn added a commit to davidshinn/pandas that referenced this issue Jul 28, 2013
@hayd
Copy link
Contributor

hayd commented Aug 8, 2013

@ghost
Copy link

ghost commented Jan 29, 2014

We've got to get this fixed in 0.14 or sooner. You can't just drop people's tables
and not fix it for 11 months. That's crazy.

hayd pushed a commit to hayd/pandas that referenced this issue Jan 29, 2014
The if_exists argument in io.sql.write_frame needed data validation
because the logic of the function implicitly used 'append' if the argument
value was any string that was not either 'fail' or 'replace'.
I added a new unit test to support the requirement.

BUG: Fix if_exists='replace' functionality in io.sql.write_frame

This should resolve issues pandas-dev#2971 and pandas-dev#4110

CLN: Refactor in between test clean ups to be more DRY

TST: Complete test coverage for if_exists uses in io.sql.write_frame

CLN: Refactor to make interaction between exists and if_exists clearer

This refactor results in the function logic being clearer, since if_exists is
only relevant when exists is True, the program flow is better served to
have if_exists control flow only when exists is True

BUG: Fix regression introduced by c28f11a0041a9f3b25f33b0539e42fa802b1d8d4

sqlite3 convenience function executescript not available
in other database flavors.

TST: Adding if_exist test for mysql flavor
jreback pushed a commit to jreback/pandas that referenced this issue Feb 4, 2014
The if_exists argument in io.sql.write_frame needed data validation
because the logic of the function implicitly used 'append' if the argument
value was any string that was not either 'fail' or 'replace'.
I added a new unit test to support the requirement.

BUG: Fix if_exists='replace' functionality in io.sql.write_frame

This should resolve issues pandas-dev#2971 and pandas-dev#4110

CLN: Refactor in between test clean ups to be more DRY

TST: Complete test coverage for if_exists uses in io.sql.write_frame

CLN: Refactor to make interaction between exists and if_exists clearer

This refactor results in the function logic being clearer, since if_exists is
only relevant when exists is True, the program flow is better served to
have if_exists control flow only when exists is True

BUG: Fix regression introduced by c28f11a0041a9f3b25f33b0539e42fa802b1d8d4

sqlite3 convenience function executescript not available
in other database flavors.

TST: Adding if_exist test for mysql flavor
@jreback
Copy link
Contributor

jreback commented Feb 14, 2014

doesn't this #6164 close the issue?

@hayd hayd closed this as completed Feb 14, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Blocker Blocking issue or pull request for an upcoming release Bug IO Data IO issues that don't fit into a more specific label IO SQL to_sql, read_sql, read_sql_query
Projects
None yet
4 participants