Skip to content

[client-v2] BigDecimal in a Dynamic column: high-scale values truncated, values with an integer part overflow on insert #2966

Description

@polyglotAI-bot

Describe the bug

When a BigDecimal is written into a Dynamic column, client-v2 infers the target Decimal type incorrectly. SerializerUtils.valueToColumnForDynamicType(Object) picks the width solely from BigDecimal.precision() and then forces the scale to that width's maximum (Decimal32→9, Decimal64→18, Decimal128→38, Decimal256→76).

Because DecimalN(S) is Decimal(P, S) with P fixed to the width precision (9/18/38/76) and 0 ≤ S ≤ P, forcing S = P leaves no room for an integer part and cannot represent a scale larger than P. This produces two failure modes:

  1. Silent truncation — a low-precision value whose scale exceeds the width maximum is rounded on write with no error to the caller.
  2. Insert failure — a value with an integer part overflows the inferred Decimal(P, P) and throws.

Only sub‑1 values whose scale fits the width (e.g. 0.5) currently round‑trip.

How to reproduce

  • Client version: client-v2 (current main, 0.11.0-rc1)
  • ClickHouse server: 26.5 (any version supporting Dynamic, 24.8+)

Insert each BigDecimal into a Dynamic column via the client and read it back:

Input precision / scale Read back Result
0.0123456789 9 / 10 0.012345678 lossy (last digit dropped)
0.00012345678901234567 17 / 20 0.000123456789012345 lossy (digits dropped)
19.99 4 / 2 throws IllegalArgumentException: BigDecimal(19990000000.00) should be between -1000000000 and 1000000000
0.5 1 / 1 0.500000000 ok

Expected behaviour

Inserting a BigDecimal into a Dynamic column should round‑trip without silent data loss and without an insert-time overflow for ordinary values that carry an integer part (e.g. a price like 19.99).

Root cause

valueToColumnForDynamicType sizes the width off precision() alone and sets the scale to the width maximum. A correct inference must size the width to hold both the integer digits and the value's scale, and choose a scale that does not steal room from the integer part. A second, coupled site — writeDynamicTypeTag — writes dataType.getMaxScale() into the Dynamic type tag instead of the actual column scale, so the tag and the serialized data must be kept consistent.

A fix (non-breaking; preserves the existing representation for values that already round-trip) is on the way.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions