Revert commits #443
Closed
Revert commits #443
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Start * Add tests for delta write, modify macros * Update delta file * Fix tests * Define delta extra * Add support for installing extensions from custom repositories, modify init db to create secrets prior to attaching catalogs, add UC client to delta plugin to dynamically create schemas/tables if they don't exist * fix secret creation to use the __default_ prefix when no name is specified * fix tests, add date as supported type to convert * run linter and run add unitycatalog to dev reqs * fix tests * fix delta write test * clean up tests * fixed all tests * rollback some accidently edited files * clean up * added unitycatalog as delta-plugin option, rewrite delta plugin * fix credential tests * Add TODOs * add correct type hinting for schema conversion * update type mapping * remove redundant mock * create new unity plugin, separate unity from delta * refactoring * Add tenacity to add retries for executing target config compiled code * Add tenacity to add retries for executing target config compiled code * Overwrite ref macro to support unity table reads * fix ref to fix failing tests * refactor ref macro * wait incrementally, get catalog from profile.yml * remove None check for uc client * add tests for the unity plugin * fix Mock unity catalog server step in pipeline * fix Mock unity catalog server step in pipeline * fix Mock unity catalog server step in pipeline * fix Mock unity catalog server step in pipeline * fix Mock unity catalog server step in pipeline * change access for test-unity.sh * update tox.ini * fix tests * move decorator to utils, make handle more robust for concurrent operations by a retry decorator * make handle more robust for concurrent operations by a retry decorator * remove secrets creation for each cursor init * fix test * remove threads from unity test * clean up tests * add unity catalog load support * clean up * remove catalog from full_table_name * clean up unity plugin * fix the catalog base url to add the http path to the host_and_port * add link to git issue * update doc --------- Co-authored-by: Frank Mbonu <fcm073@gmail.com>
…te (#2) * add support for dynamically loading table credentials to write delta * add AWS_SESSION_TOKEN * handle non-existing tables * remove support for loading delta tables directly as unity catalog tables
This reverts commit b1121e6.
This reverts commit 2a7a7ea.
|
@dan1elt0m this looks interesting, but I must be missing some context on what you all are cooking here (or possibly this was meant for a fork of this repo for now?) |
|
Hey Josh, First of all, thanks for creating this great library! My apologies for creating the PR here. Indeed ment for a fork. I'm experimenting with running dbt-duckdb against the OSS Unity Catalog using a unity plugin. Would like to come back later to create an actual PR if you're interested. Probably after delta plugin is merged, because it depends heavily on that. |
|
No worries man, thank you for the kind words! I'm interested in making dbt-duckdb work with all kinds of catalogs, so when you have something that works, let me know! |
|
That's great. I'll surely do that. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.