Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle empty string values for datetime, date, and time types #27895

Open
cynthiaxyin opened this issue Jun 30, 2023 · 3 comments
Open

Handle empty string values for datetime, date, and time types #27895

cynthiaxyin opened this issue Jun 30, 2023 · 3 comments
Labels
1-stream~1-table change the format of the final table and any airbyte required tables destinations-v2-bugs frozen Not being actively worked on good first issue team/destinations Destinations team's backlog

Comments

@cynthiaxyin
Copy link
Contributor

cynthiaxyin commented Jun 30, 2023

In the dbt type parsing logic for old normalization, empty string values for these 3 types are always converted to null for all destination types (L534, L558, L583).

In the 1s1t world, how should we handle this?

  • Each destination's try-cast logic decides if "" should be a typing error or null. ✅
  • Or, different destinations should handle this consistently (when technically possible). The protocol should define whether "" is valid, and enforce that on sources (fail CAT if they produce "") and destinations (fail DAT if they choke on "").
@cynthiaxyin cynthiaxyin added 1-stream~1-table change the format of the final table and any airbyte required tables destinations-v2-bugs team/destinations Destinations team's backlog labels Jun 30, 2023
@evantahler
Copy link
Contributor

Grooming:

  • we think this already works with our try/catch logic!
  • this story is to write a test that confirms updated_at: "" becomes "null" in the column and and error is stored within airbyte_meta

@harsha-mangena
Copy link

harsha-mangena commented Nov 20, 2023

@evantahler, @cynthiaxyin Can I pick this up?

@cynthiaxyin
Copy link
Contributor Author

@harsha-mangena Sure! Contributions are always welcome :)

@bleonard bleonard added the frozen Not being actively worked on label Mar 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
1-stream~1-table change the format of the final table and any airbyte required tables destinations-v2-bugs frozen Not being actively worked on good first issue team/destinations Destinations team's backlog
Projects
No open projects
Development

No branches or pull requests

5 participants