feat: Developers can now more easily override the mapping from SQL column type to JSON schema

[edgarrmondragon]

Related:

• bug: JSON serialization of date types causes schema validation to fail in target-postgres #1831
• Support maxLength in JSON Schema TicketSwap/target-redshift#105
• Add maxLength to discovered schema MeltanoLabs/tap-postgres#477
• feat: define a maxLength limit for varchar at the connector level #1812
• feat: ✨ add maxLength to schema MeltanoLabs/tap-postgres#485
• https://github.com/thkwag/target-mysql/blob/c84fe01850b0470d39e4d0c56fd884e2019174c4/target_mysql/sinks.py#L195-L196

New docs links:

• Custom type mapping guide: https://meltano-sdk--2618.org.readthedocs.build/en/2618/guides/sql-tap.html#custom-type-mapping
• Default type mapping reference docs: https://meltano-sdk--2618.org.readthedocs.build/en/2618/classes/singer_sdk.connectors.sql.SQLToJSONSchema.html#singer_sdk.connectors.sql.SQLToJSONSchema

---

📚 Documentation preview 📚: https://meltano-sdk--2618.org.readthedocs.build/en/2618/

[codspeed-hq]

CodSpeed Performance Report

Merging #2618 will not alter performance

_{Comparing edgarrmondragon/feat/sqltype-to-json-schema-singledispatch (c57e23e) with main (efdc735)}

Summary

✅ 6 untouched benchmarks

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.82%. Comparing base (efdc735) to head (c57e23e).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2618      +/-   ##
==========================================
+ Coverage   89.61%   89.82%   +0.21%     
==========================================
  Files          58       58              
  Lines        4861     4884      +23     
  Branches      955      959       +4     
==========================================
+ Hits         4356     4387      +31     
+ Misses        352      346       -6     
+ Partials      153      151       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[kgpayne]

Hey @edgarrmondragon 👋 This is great! Are there plans to replace singer_sdk.typing.to_sql_type() with a JSONSchemaToSQL class too? I am writing a target for Databricks and typing has been a real headache 🤕

[edgarrmondragon]

Hey @kgpayne, it's nice to see you around these parts!

Hey @edgarrmondragon 👋 This is great! Are there plans to replace singer_sdk.typing.to_sql_type() with a JSONSchemaToSQL class too? I am writing a target for Databricks and typing has been a real headache 🤕

That's the plan. I'll give it a try and see what I come up with tomorrow.

[edgarrmondragon]

#2732

[edgarrmondragon]

I've made a reference implementation in MeltanoLabs/target-postgres#469 if anyone's interested in looking.