Describe the bug
On T-SQL adapters (dbt-sqlserver / dbt-fabric / dbt-synapse), varbinary/binary columns are serialized into result_row (in test_result_rows, and result_rows in elementary_test_results) as a Python bytes repr string rather than a usable value.
Example stored value for a 16-byte binary column (e.g. a Data Vault hash key):
b'\x1a_,:\xed\x08\x9dAL\xdadM\x83`\x8e\xad'
This is unusable downstream: it isn't the original binary, isn't hex, and can't be reliably reconstructed in SQL because the bytes repr mixes \xNN escapes with literal printable ASCII bytes.
Root cause
macros/utils/run_queries/agate_to_dicts.sql:
{% macro agate_val_serialize(val) %}
{% if val.year is defined %} {% do return(val.isoformat()) %} {% endif %}
{% if elementary.edr_is_decimal(val) %}
{% do return(elementary.edr_serialize_decimal(val)) %}
{% endif %}
{% do return(val) %}
{% endmacro %}
bytes values fall through to return(val), and the subsequent tojson renders them via their Python repr (b'...').
Expected behavior
Binary values should be serialized to a stable, adapter-neutral string — hex is the natural choice (matches T-SQL CONVERT(varchar, col, 1) → 0x...).
Suggested fix
Handle bytes/bytearray explicitly in agate_val_serialize:
{% macro agate_val_serialize(val) %}
{# binary (e.g. hash keys) -> 0x hex, instead of a Python bytes repr #}
{% if val is not none and val.hex is defined and val.decode is defined %}
{% do return('0x' ~ (val.hex() | upper)) %}
{% endif %}
{% if val.year is defined %} {% do return(val.isoformat()) %} {% endif %}
{% if elementary.edr_is_decimal(val) %}
{% do return(elementary.edr_serialize_decimal(val)) %}
{% endif %}
{% do return(val) %}
{% endmacro %}
The val.hex is defined and val.decode is defined guard matches only bytes/bytearray (avoids float, which also has .hex(); str has no .decode in py3).
Can't be worked around by consumers
agate_val_serialize is called everywhere as elementary.agate_val_serialize(...) (package-qualified) and is not adapter.dispatch-ed, so it cannot be overridden from a downstream project. The only current workaround is editing the package file directly (lost on dbt deps). Either applying the fix above, or routing this through adapter.dispatch so downstream projects can override it, would resolve this.
Environment
- elementary package version:
0.25.0
- Adapter: T-SQL family (
dbt-sqlserver / dbt-fabric / dbt-synapse)
Describe the bug
On T-SQL adapters (
dbt-sqlserver/dbt-fabric/dbt-synapse),varbinary/binarycolumns are serialized intoresult_row(intest_result_rows, andresult_rowsinelementary_test_results) as a Pythonbytesrepr string rather than a usable value.Example stored value for a 16-byte binary column (e.g. a Data Vault hash key):
This is unusable downstream: it isn't the original binary, isn't hex, and can't be reliably reconstructed in SQL because the
bytesrepr mixes\xNNescapes with literal printable ASCII bytes.Root cause
macros/utils/run_queries/agate_to_dicts.sql:bytesvalues fall through toreturn(val), and the subsequenttojsonrenders them via their Python repr (b'...').Expected behavior
Binary values should be serialized to a stable, adapter-neutral string — hex is the natural choice (matches T-SQL
CONVERT(varchar, col, 1)→0x...).Suggested fix
Handle
bytes/bytearrayexplicitly inagate_val_serialize:The
val.hex is defined and val.decode is definedguard matches onlybytes/bytearray(avoidsfloat, which also has.hex();strhas no.decodein py3).Can't be worked around by consumers
agate_val_serializeis called everywhere aselementary.agate_val_serialize(...)(package-qualified) and is notadapter.dispatch-ed, so it cannot be overridden from a downstream project. The only current workaround is editing the package file directly (lost ondbt deps). Either applying the fix above, or routing this throughadapter.dispatchso downstream projects can override it, would resolve this.Environment
0.25.0dbt-sqlserver/dbt-fabric/dbt-synapse)