Marshalling long, single-line strings results in (correct) line breaks

[tyangliu]

When long, single-line strings (i.e. does not contain "\n") are marshalled, the yaml emitter seems to force line breaks at 80 characters, but does so without prefixing the string with the multi-line notation (e.g. ">" or "|"). This results in unusable yaml output like in the example below.

Example:

data := yaml.MapSlice{{"test", "abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd"}}

res, _ := yaml.Marshal(data)
fmt.Println(string(res))

Output:

test: abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd
  abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd

As a workaround, simply appending "\n" to the end of a string that's known to be lengthy works fine, although not ideal since the yaml output now exceeds the 80 char width. Mutiline ">" prefixed outputs would be preferable.

Output with trailing \n:

test: |
  abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd aabdasfadsfdasfadsfasd abdasfadsfdasfadsfasd abdasfadsfdasfadsfasd

Appreciate if someone could confirm whether this is an issue or just incorrect usage. If a fix is needed, I'd be happy to work on a PR. Thanks!

[deitch]

Been banging my head on this for the last while. Is there any way to force it not to break? It should be perfectly legitimate to say, "give me a really long line".

[wayt]

I'm facing the same issue. A tag would be very nice.

[rogpeppe]

@tyangliu

This results in unusable yaml output like in the example below.

What do you mean by "unusable yaml output" there? By my reading of the specification, that YAML looks valid. See section 8.1.3 in http://yaml.org/spec/1.2/spec.html, which says "Folding allows long lines to be broken anywhere a single space character separates two non-space characters.". That means that the example is perfectly valid YAML and the map value holds exactly the string you produced.

The Go YAML marshaler uses heuristics to try to produce decent looking output for any possible string. There are so many possible output style choices in YAML, that I'm not sure it's a good idea to start down the slippery slope of allowing string style choices to be specified with struct tags.

That said, suggestions are welcome - any such struct tag would need to be applicable to all possible strings.

[wayt]

I understand this is valid according to the specification, but not compatible in all implementations (Perl for instance).
A fallback option would be appreciated.

[rogpeppe]

I understand this is valid according to the specification, but not compatible in all implementations

How hard should go-yaml try to produce YAML that is compatible with other broken YAML implementations? As the YAML spec is insanely complex, I wouldn't be surprised if every single implementation was broken in some way.

If we decide that we want to be compatible with perl's broken implementation, the right answer I think would be to avoid breaking lines like this at all. Providing a tag so that the behaviour can be changed in individual cases doesn't seem like the right fix (what if some other field happens to have a long value with spaces in?).

tbh, if you're marshalling YAML for data interchange, I'd suggest not using YAML but using JSON instead - it's simple enough that most implrmentations get it right, and it's also compatible (mostly) with YAML.

[alexandrevicenzi]

My guess to solve this problem is to expose YAML Encoder and Decoder class. Go encoding/json exposes its encoder and decoder, this allows setting custom parameters.

In apic.go there's a function called yaml_emitter_set_width with allows to disable long line breaks if set to -1, but this is an internal function that is never called.

Something like this code would be nice:

e := yaml.NewEncoder(wr)
e.SetLineWidth(-1)
err := e.Encode(myObj)

Using a tag would be hard I think, there's no easy way to pass the tag info to the yaml_emitter_t.

[rogpeppe]

@alexandrevicenzi Now that we have an Encoder type, I think having a SetLineWidth method on it seems very reasonable. I think I'd treat any line width <= 0 as disabling line breaks. Happy to accept a PR for that.

[alexandrevicenzi]

@rogpeppe this Encoder type is only available in devel branch? If so, I'll send a PR.

[vivook]

Is it any closer to fix? This is breaking my parsing yaml -> go template by marshal/unmarshal yaml because it puts in new lines breaking go template