[Bug]: Data Grid + Metadata Injection - input line is not split, exception is thrown

[dave-csc]

Apache Hop version?

2.10.0

Java version?

17.0.2

Operating system

Linux

What happened?

Steps to reproduce:

1. Create a "child" pipeline with a Data grid transform
2. Create a "parent" pipeline that injects to the Data grid data lines separated by ; and the according meta name specification
3. Enable the output pipeline in the ETL Metadata injection transform (to view the result)

Result: The data grid structure is created but only the first column is populated, with all the values unseparated. The log reports an ArrayIndexOutOfBoundsException, when trying to access a "index 1" element on an array with a single element.

Workaround:

1. Add a Split fields transform in the child pipeline, after the Data grid
2. Inject the Data grid with the data line, but use constant values in the meta section (e.g. Meta -> Name = data_line and Meta -> Type = String
3. Inject the meta structure in the Split fields transform, and specify the same name data_line in The name of the field to split. Plus: you can specify both the separator and the enclosure of columns (Data grid requires the separator only)

Issue Priority

Priority: 3

Issue Component

Component: Transforms

[hansva]

Hi @dave-csc, Did you also specify the separator? The default is "," not ";".

Could you take a look at our integration test and see where your configuration differs?
https://github.com/apache/hop/tree/main/integration-tests/mdi

• 0011-data-grid-parent.hpl
• 0011-data-grid-child.hpl

I just read our documentation again and that might not be completely clear

[dave-csc]

Hi @hansva, I did specify the separator, both as a constant and as an added field (with constant value ;).

I tried the integration test above and it works, without any change. But if you substitute all the , with the ; (in both dataLines and splitfield columns, of course), you'll get the error described above: perhaps the specified separator is actually ignored?

[hansva]

Thanks for testing.
Then this bug is valid and we will look into it!

[hansva]

.take-issue

[hansva]

I found the issue. It was not smart of me to build the test with the default value.