-
Notifications
You must be signed in to change notification settings - Fork 162
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Initial IMPORT FOREIGN SCHEMA
support
#62
Initial IMPORT FOREIGN SCHEMA
support
#62
Conversation
Thanks. I plan to test this feature next week.
|
scan_relid, | ||
params_list, | ||
fdw_private); | ||
return make_foreignscan(tlist |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Moving the commas to the start of the line is not common to the PostgreSQL code base and is unrelated to the feature.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I switched the formatting around to allow for capturing the change (within the #if
) as locally as possible while maintaining consistent formatting across the items in the argument list of make_foreignscan
.
I'm also fine with only using a start of line comma on the line within the #if
(maintaining previous formatting/trailing commas elsewhere); or encapsulating the whole return
in a #if
/#else
macro block (possibly cleaner looking, but more duplicated lines).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am getting the error below when trying to compile the mysql_fdw with PG
9.5.
Ibrar is going to commit a fix for this later today.
make USE_PGXS=1
gcc -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard
-O2 -fpic -I/home/edb/mysql-5.6.19-linux-glibc2.5-x86_64/include -D
_MYSQL_LIBNAME="libmysqlclient.so" -I. -I./
-I/home/edb/Projects/PG_servers/postgres/binary/include/server
-I/home/edb/Projects/PG_servers/postgres/binary/include/internal
-D_GNU_SOURCE -c -o connection.o connection.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard
-O2 -fpic -I/home/edb/mysql-5.6.19-linux-glibc2.5-x86_64/include -D
_MYSQL_LIBNAME="libmysqlclient.so" -I. -I./
-I/home/edb/Projects/PG_servers/postgres/binary/include/server
-I/home/edb/Projects/PG_servers/postgres/binary/include/internal
-D_GNU_SOURCE -c -o option.o option.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard
-O2 -fpic -I/home/edb/mysql-5.6.19-linux-glibc2.5-x86_64/include -D
_MYSQL_LIBNAME="libmysqlclient.so" -I. -I./
-I/home/edb/Projects/PG_servers/postgres/binary/include/server
-I/home/edb/Projects/PG_servers/postgres/binary/include/internal
-D_GNU_SOURCE -c -o deparse.o deparse.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard
-O2 -fpic -I/home/edb/mysql-5.6.19-linux-glibc2.5-x86_64/include -D
_MYSQL_LIBNAME="libmysqlclient.so" -I. -I./
-I/home/edb/Projects/PG_servers/postgres/binary/include/server
-I/home/edb/Projects/PG_servers/postgres/binary/include/internal
-D_GNU_SOURCE -c -o mysql_query.o mysql_query.c
gcc -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard
-O2 -fpic -I/home/edb/mysql-5.6.19-linux-glibc2.5-x86_64/include -D
_MYSQL_LIBNAME="libmysqlclient.so" -I. -I./
-I/home/edb/Projects/PG_servers/postgres/binary/include/server
-I/home/edb/Projects/PG_servers/postgres/binary/include/internal
-D_GNU_SOURCE -c -o mysql_fdw.o mysql_fdw.c
mysql_fdw.c: In function ‘mysqlGetForeignPlan’:
mysql_fdw.c:949:8: error: too few arguments to function ‘make_foreignscan’
In file included from mysql_fdw.c:41:0:
/home/edb/Projects/PG_servers/postgres/binary/include/server/optimizer/planmain.h:46:21:
note: declared here
mysql_fdw.c: In function ‘mysqlPlanForeignModify’:
mysql_fdw.c:1073:35: error: ‘RangeTblEntry’ has no member named
‘modifiedCols’
mysql_fdw.c: In function ‘mysqlGetForeignPlan’:
mysql_fdw.c:950:1: warning: control reaches end of non-void function
[-Wreturn-type]
make: *** [mysql_fdw.o] Error 1
On Sat, Jun 27, 2015 at 4:18 AM, jjthiessen notifications@github.com
wrote:
In mysql_fdw.c
#62 (comment):@@ -942,11 +950,15 @@ mysqlGetForeignPlan(PlannerInfo *root,RelOptInfo *baserel, Oid foreigntableid, F
* field of the finished plan node; we can't keep them in private state
* because then they wouldn't be subject to later planner processing.
*/
- return make_foreignscan(tlist,
local_exprs,
scan_relid,
params_list,
fdw_private);
- return make_foreignscan(tlist
I switched the formatting around to allow for capturing the change (within
the #if) as locally as possible while maintaining consistent formatting
across the items in the argument list of make_foreignscan.I'm also fine with only using a start of line comma on the line within the
#if (maintaining previous formatting/trailing commas elsewhere); or
encapsulating the whole return in a #if/#else macro block (possibly
cleaner looking, but more duplicated lines).—
Reply to this email directly or view it on GitHub
https://github.com/EnterpriseDB/mysql_fdw/pull/62/files#r33404303.
Ahsan Hadi
Snr Director Product Development
EnterpriseDB Corporation
The Enterprise Postgres Company
Phone: +92-51-8358874
Mobile: +92-333-5162114
Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb
This e-mail message (and any attachment) is intended for the use of the
individual or entity to whom it is addressed. This message contains
information from EnterpriseDB Corporation that may be privileged,
confidential, or exempt from disclosure under applicable law. If you are
not the intended recipient or authorized to receive this for the intended
recipient, any use, dissemination, distribution, retention, archiving, or
copying of this communication is strictly prohibited. If you have received
this e-mail in error, please notify the sender immediately by reply e-mail
and delete this message.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks.
I was able to apply your patch and get the import foreign schema working
successfully. However I am running into crash when i try to select from any
of the mysql foreign table.
It seem to be crashing in the external library. Is anyone else seeing this
issue?
postgres=# select * from test.emp;
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
(gdb) bt
#0 0x00007f8243a7c037 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007f8243a7f698 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00000000007d7327 in ExceptionalCondition ()
#3 0x000000000061ad47 in list_delete ()
#4 0x00007f82399c31d4 in mysql_stmt_close (stmt=0x1d07de0)
at
/export/home/pb2/build/sb_0-12116699-1399374319.34/mysql-5.6.19/libmysql/libmysql.c:4705
#5 0x00007f8239f5d38a in mysqlEndForeignScan () from
/home/edb/Projects/PG_servers/postgres/binary/lib/mysql_fdw.so
#6 0x00000000005ffa5e in ExecEndForeignScan ()
#7 0x00000000005db9d6 in standard_ExecutorEnd ()
#8 0x00000000005a5203 in PortalCleanup ()
#9 0x00000000007fbd13 in PortalDrop ()
#10 0x00000000006e895c in PostgresMain ()
#11 0x00000000004654b3 in ServerLoop ()
#12 0x000000000068a6de in PostmasterMain ()
#13 0x00000000004660c6 in main ()
(gdb)
-- Ahsan
On Thu, Jul 2, 2015 at 9:55 PM, jjthiessen notifications@github.com wrote:
In mysql_fdw.c
#62 (comment):@@ -942,11 +950,15 @@ mysqlGetForeignPlan(PlannerInfo *root,RelOptInfo *baserel, Oid foreigntableid, F
* field of the finished plan node; we can't keep them in private state
* because then they wouldn't be subject to later planner processing.
*/
- return make_foreignscan(tlist,
local_exprs,
scan_relid,
params_list,
fdw_private);
- return make_foreignscan(tlist
This branch is based on my PG_95 branch which should have fixed the
compile time errors that you listed (please see commit 5f8e57c
5f8e57c).
What branch are you on, and is 5f8e57c
5f8e57c
in the commit log?—
Reply to this email directly or view it on GitHub
https://github.com/EnterpriseDB/mysql_fdw/pull/62/files#r33798674.
Ahsan Hadi
Snr Director Product Development
EnterpriseDB Corporation
The Enterprise Postgres Company
Phone: +92-51-8358874
Mobile: +92-333-5162114
Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb
This e-mail message (and any attachment) is intended for the use of the
individual or entity to whom it is addressed. This message contains
information from EnterpriseDB Corporation that may be privileged,
confidential, or exempt from disclosure under applicable law. If you are
not the intended recipient or authorized to receive this for the intended
recipient, any use, dissemination, distribution, retention, archiving, or
copying of this communication is strictly prohibited. If you have received
this e-mail in error, please notify the sender immediately by reply e-mail
and delete this message.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tested without the patch, the crash is not reproducible.
Can the author of the patch please look into this issue...I will do further
testing on import foreign schema once this issue is fixed...
On Sat, Jul 4, 2015 at 1:28 PM, Ahsan Hadi ahsan.hadi@enterprisedb.com
wrote:
Thanks.
I was able to apply your patch and get the import foreign schema working
successfully. However I am running into crash when i try to select from any
of the mysql foreign table.It seem to be crashing in the external library. Is anyone else seeing this
issue?postgres=# select * from test.emp;
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.(gdb) bt
#0 0x00007f8243a7c037 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007f8243a7f698 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00000000007d7327 in ExceptionalCondition ()
#3 0x000000000061ad47 in list_delete ()
#4 0x00007f82399c31d4 in mysql_stmt_close (stmt=0x1d07de0)
at
/export/home/pb2/build/sb_0-12116699-1399374319.34/mysql-5.6.19/libmysql/libmysql.c:4705
#5 0x00007f8239f5d38a in mysqlEndForeignScan () from
/home/edb/Projects/PG_servers/postgres/binary/lib/mysql_fdw.so
#6 0x00000000005ffa5e in ExecEndForeignScan ()
#7 0x00000000005db9d6 in standard_ExecutorEnd ()
#8 0x00000000005a5203 in PortalCleanup ()
#9 0x00000000007fbd13 in PortalDrop ()
#10 0x00000000006e895c in PostgresMain ()
#11 0x00000000004654b3 in ServerLoop ()
#12 0x000000000068a6de in PostmasterMain ()
#13 0x00000000004660c6 in main ()
(gdb)-- Ahsan
On Thu, Jul 2, 2015 at 9:55 PM, jjthiessen notifications@github.com
wrote:In mysql_fdw.c
#62 (comment):@@ -942,11 +950,15 @@ mysqlGetForeignPlan(PlannerInfo *root,RelOptInfo *baserel, Oid foreigntableid, F
* field of the finished plan node; we can't keep them in private state
* because then they wouldn't be subject to later planner processing.
*/
- return make_foreignscan(tlist,
local_exprs,
scan_relid,
params_list,
fdw_private);
- return make_foreignscan(tlist
This branch is based on my PG_95 branch which should have fixed the
compile time errors that you listed (please see commit 5f8e57c
5f8e57c).
What branch are you on, and is 5f8e57c
5f8e57c
in the commit log?—
Reply to this email directly or view it on GitHub
https://github.com/EnterpriseDB/mysql_fdw/pull/62/files#r33798674.Ahsan Hadi
Snr Director Product Development
EnterpriseDB Corporation
The Enterprise Postgres CompanyPhone: +92-51-8358874
Mobile: +92-333-5162114Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedbThis e-mail message (and any attachment) is intended for the use of the
individual or entity to whom it is addressed. This message contains
information from EnterpriseDB Corporation that may be privileged,
confidential, or exempt from disclosure under applicable law. If you are
not the intended recipient or authorized to receive this for the intended
recipient, any use, dissemination, distribution, retention, archiving, or
copying of this communication is strictly prohibited. If you have received
this e-mail in error, please notify the sender immediately by reply e-mail
and delete this message.
Ahsan Hadi
Snr Director Product Development
EnterpriseDB Corporation
The Enterprise Postgres Company
Phone: +92-51-8358874
Mobile: +92-333-5162114
Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb
This e-mail message (and any attachment) is intended for the use of the
individual or entity to whom it is addressed. This message contains
information from EnterpriseDB Corporation that may be privileged,
confidential, or exempt from disclosure under applicable law. If you are
not the intended recipient or authorized to receive this for the intended
recipient, any use, dissemination, distribution, retention, archiving, or
copying of this communication is strictly prohibited. If you have received
this e-mail in error, please notify the sender immediately by reply e-mail
and delete this message.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have yet to be able to reproduce the crash that you're reporting (I've admittedly done very limited testing). Could you please provide the SQL that you used when you encountered the crash (on both the MySQL and PostgreSQL sides)? Could you also please provide the PostgreSQL commit that you were on? Any other details about your build environment would be appreciated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On Tue, Jul 7, 2015 at 9:33 PM, jjthiessen notifications@github.com wrote:
In mysql_fdw.c
#62 (comment):@@ -942,11 +950,15 @@ mysqlGetForeignPlan(PlannerInfo *root,RelOptInfo *baserel, Oid foreigntableid, F
* field of the finished plan node; we can't keep them in private state
* because then they wouldn't be subject to later planner processing.
*/
- return make_foreignscan(tlist,
local_exprs,
scan_relid,
params_list,
fdw_private);
- return make_foreignscan(tlist
I have yet to be able to reproduce the crash that you're reporting (I've
admittedly done very limited testing). Could you please provide the SQL
that you used when you encountered the crash (on both the MySQL and
PostgreSQL sides)? Could you also please provide the PostgreSQL commit that
you were on? Any other details about your build environment would be
appreciated.
I can reproduce the crash consistently using the following steps :
1- Compiled and installed mysql fdw by pulling the latest code from github
2- Applied the attached patch for import foreign schema functionality and
the import schema was done fine
3- Tried to create another mysql fdw foreign table and did select from it
and it crashed.
4- Crash is not reproducible without the patch
I am using PG 9.5 alpha and mysql 5.6.19.
—
Reply to this email directly or view it on GitHub
https://github.com/EnterpriseDB/mysql_fdw/pull/62/files#r34060458.
Ahsan Hadi
Snr Director Product Development
EnterpriseDB Corporation
The Enterprise Postgres Company
Phone: +92-51-8358874
Mobile: +92-333-5162114
Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb
This e-mail message (and any attachment) is intended for the use of the
individual or entity to whom it is addressed. This message contains
information from EnterpriseDB Corporation that may be privileged,
confidential, or exempt from disclosure under applicable law. If you are
not the intended recipient or authorized to receive this for the intended
recipient, any use, dissemination, distribution, retention, archiving, or
copying of this communication is strictly prohibited. If you have received
this e-mail in error, please notify the sender immediately by reply e-mail
and delete this message.
e2fad16
to
e25d2b8
Compare
Amended commit to use ANSI quoting of string literals. |
@ahsanhadi I tried to replicate the crash, but I've been unable to do so thus far. However, there are potentially significant differences between our environments:
I did encounter an issue with how I was quoting string literals (that would cause imports to fail (without crashing) depending on execution order). This particular quoting bug should be fixed in my most recent push. While you'll have to amend the SQL test code and login parameters to match your server, I've used something like the following to bootstrap, build, and test (in reality, I generally use separate scripts for each component and I don't do clean builds each time I test, but I have run the following in my environment (with different SQL matching my MySQL setup), and it executes without the crash that you mention):
Are you able to run (something like) the above (with the SQL adapted appropriately)? Are you able to give the actual SQL script that you were using for testing (with any sensitive details redacted)? Do you have a scripted build and test process that I could replicate on my end? |
e25d2b8
to
7bcc63f
Compare
Ibrar has been able to reproduce the crash after applying your patch. The
"mysql_stmt_bind_param"); Please see the comments for function mysql_load_library in mysql_fdw.c, it Ibrar will commit a fix for this after applying your patch. On Thu, Jul 9, 2015 at 12:52 AM, jjthiessen notifications@github.com
Ahsan Hadi Phone: +92-51-8358874 Website: www.enterprisedb.com This e-mail message (and any attachment) is intended for the use of the |
jjthiessen: Please rebase your patch, so I can merge the pull request. On Thu, Jul 23, 2015 at 8:03 PM, Ahsan Hadi notifications@github.com
Ibrar Ahmed |
7bcc63f
to
4685f39
Compare
I've amended and rebased my branch accordingly. |
Can you please generate a pull request for this? On Fri, Jul 24, 2015 at 5:17 AM, jjthiessen notifications@github.com
Ahsan Hadi Phone: +92-51-8358874 Website: www.enterprisedb.com This e-mail message (and any attachment) is intended for the use of the |
This (#62) is a PR for merging jjthiessen/mysql_fdw:import-foreign-schema into EnterpriseDB/mysql_fdw:master. Were you wanting a separate PR to a different branch? |
It looks good to commit now... On Fri, Jul 24, 2015 at 10:10 PM, jjthiessen notifications@github.com
Ahsan Hadi Phone: +92-51-8358874 Website: www.enterprisedb.com This e-mail message (and any attachment) is intended for the use of the |
Initial `IMPORT FOREIGN SCHEMA` support
This provides basic/initial support for
IMPORT FOREIGN SCHEMA
in PostgreSQL 9.5+