Newlines in values are replaced with a custom line separator when writing out

[HyukjinKwon]

It looks setting a custom line separator as below:

import java.io.StringWriter;

import com.univocity.parsers.csv.CsvWriter;
import com.univocity.parsers.csv.CsvWriterSettings;

class Main {
    public static void main(String[] args) {
        CsvWriterSettings settings = new CsvWriterSettings();
        settings.getFormat().setLineSeparator("!");
        StringWriter writer = new StringWriter();
        CsvWriter gen = new CsvWriter(writer, settings);

        gen.writeRow(new String[]{"a\n\na", "\nb"});
        gen.writeRow(new String[]{"a\n\na", "\nb"});

        System.out.println(writer.toString());
    }
}

replaces the newlines in values as below:

"a!!a",b!"a!!a",b!

I expected the newlines remain in the value.

I am not very sure if it is maybe expected or a bug but I hope this report is helpful.

[jbax]

It happens because \n is still configured as the normalized line ending character.

Add this line:

settings.getFormat().setNormalizedNewline('!');

And it should work.

Now that you mentioned it, it makes sense to set that normalizedNewLine property to the character you gave if the line separator has 1 character only. I'll implement this adjustment.

[HyukjinKwon]

Thank you @jbax. This saved a lot of my time!

[HyukjinKwon]

One following question is, though, can we do this with, for example, \r\n?

[jbax]

The normalized newline is a single special character that represents a line break formed by one or two characters.

Suppose you do this:

    settings.getFormat().setLineSeparator("\r\n");

When the input is read and \r\n is found on the input, the sequence will be converted into the normalizedNewLine character.

If you define your normalizedNewLine to be !:

   value 1\r\nvalue2

You will get two rows: the first with value 1 and the second with value 2 - notice it doesn't really matter most of the time.

However if your input comes with:

    wow! A value\r\nvalue2

You will have issues because the ! in the input collides with the normalizedNewLine, and you will get:

wow
A value
value2

All you need is to make sure the normalizedNewLine is not a character that will not occur in the input, or it will occur only as part of actual line endings.

Notice that when parsing quoted values, you will get the normalizedNewLine character instead of the original line break, unless you set normalizeLineEndingsWithinQuotes = false

All of this works in reverse when writing. If your normalizedNewLine = ! , then writing 1,"Wow! A value",2 will generate a quoted value with:

1,"Wow\r\n A value",2

Hope this helps.

[HyukjinKwon]

Yea, these should be corner cases. This helps me a lot. Will open an issue more carefully next time. Thank you @jbax for clarifying this.