Skeleton of bottom-up FDW #530
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Added quick sketch of bottom-up FDW java and C. It's not remotely functional - but provides a quick look at what a bottom-up approach may require on the pl/java side. I assume the C backend will use the 'simple_fdw' I'm developing elsewhere so the main thing we need on the backend is a way to actually provide a few functions to the backend and then have the backend make the necessary JNI calls. One complication is that a wrapper may have multiple servers, each server can have multiple tables, and each table can have multiple (possibly concurrent) queries. It would be nice if we could reuse any common wrappers and servers... but for the first efforts it's probably fine to treat it as one-to-one-to-one as long as we don't close any doors.
Updated FDW - I had started but got distracted. This is definitely hand-wavy but it should be good enough to get a good feel on whether this is a viable approach. The initial java implementation should be something simple, e.g, a class with a static List with a few values like (1, 'red'), (2, 'greem'), etc. That means the only think we need to track is the current position within the array and to provide the current row in an acceptable format.
Author
|
I noted elsewhere that I borrowed heavily from the existing UDT but there's a huge difference since a UDT must expose its methods while a FDW should hide them. (It will still need to expose two - but they're only called when the FOREIGN TABLE is created.) That means that all of the That said... DURING DEVELOPMENT it might be useful to expose these methods (plus state) in order to test the implementation outside of the FDW framework. However this would be a stopgap measure and only used until there's no doubt that the 'glue' is working as expected. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Skeleton of bottom-up FDW design.
The idea is to fully implement the FDW in the C backend but provide an SPI that can be implemented in java and called via JNI. The java should be completely ignorant of the fact that it's called by a PostgreSQL FDW.
This is the opposite of a top-down design where you try to implement everything in java until you have no choice but to call the backend. This approach has some benefits - but comes at the very high expense of requiring the java code to duplicate a ton of functionality that's already implemented by the back end. That's a lot of extra effort and begging for inconsistent behavior.
The final solution may be somewhere in the middle but if so I hope we can use annotations and automated code generation instead of requiring individual FDW developers to learn backend details.