Skip to content

Standardization fails if default value is not defined for a non-nullable array #86

@yruslan

Description

@yruslan

Describe the bug

The error is:

Caused by: java.lang.IllegalStateException: No default value defined for data type array
	at za.co.absa.standardization.types.CommonTypeDefaults.getDataTypeDefaultValue(CommonTypeDefaults.scala:63) ~[pramen-components_2.12_3.5-0.3.5-SNAPSHOT.jar:0.3.5-SNAPSHOT]
	at za.co.absa.standardization.types.CommonTypeDefaults.$anonfun$getDataTypeDefaultValueWithNull$1(CommonTypeDefaults.scala:71) ~[pramen-components_2.12_3.5-0.3.5-SNAPSHOT.jar:0.3.5-SNAPSHOT]
	at scala.util.Try$.apply(Try.scala:213) ~[scala-library-2.12.18.jar:?]
	at za.co.absa.standardization.types.CommonTypeDefaults.getDataTypeDefaultValueWithNull(CommonTypeDefaults.scala:71) ~[pramen-components_2.12_3.5-0.3.5-SNAPSHOT.jar:0.3.5-SNAPSHOT]

But the natural default value for an array is the empty array.

To Reproduce

  1. Create a schema with a non-nullable array column.
  2. Try to standardize data with this schema while not specifying a default value for the array column.
  3. See the error.

Expected behavior

Standardization should use empty array as the default value for array type automatically.

Business Value

--

Screenshots

--

Additional context

--

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions