Skip to content

Multiline string normalization on windows is inconsistent #11988

Closed
@c42f

Description

The following code results in inconsistent line endings and first line normalization on windows:

open("test.txt", "w") do f
    # Write a header
    write(f, """
    This is the first few lines of a text file
    created using a multiline string.  A header, say.
    """
    )
    # Now write some data
    for i=1:5
        write(f, "line $i\n")
    end
end

Result (explicitly writing all nonprintable characters):

\r\n
This is the first few lines of a text file\r\n
created using a multiline string.  A header, say.\r\n
line 1\n
line 2\n
line 3\n
line 4\n
line 5\n

The fact that the first blank line isn't removed as it would be on linux is inconsistent. The underlying issue seems to be that a multiline string gets whatever line endings happen to exist in the source file. Unfortunately this makes multiline strings a bit useless for writing consistent text files on windows, and can lead to some of the more pedantic file parsers rejecting files outright (As recently encountered in the wild by @visr - see discussion on c42f/displaz#40).

Of course, if the julia code above is distributed to windows users with unix line endings the problem won't occur, but windows programs have an annoying habit of "helpfully" converting things (eg, see git.autocrlf). Given this, I think the only sensible option is to normalize multiline strings to the One True Line Ending of "\n" during parsing.

See also #3946, which was the only other place I found this mentioned.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions