WebVTT/SRT: Line breaks wrapped in italics get removed

[franontanaya]

For subtitles downloaded in VTT format (either API or from the site), any caption that on the editor looks like this:

(tape machine whirring)*
*We have literally software.

becomes:

(tape machine whirring)<i></i>We have literally software.

with no line break.

[franontanaya]

This affects SRTs too, so the bug is probably in their common generator.

[jdragojevic]

Curiously - why would you have the * in the given example?

[franontanaya]

I was testing a variety of human errors and catching them with regex.

Both VTT and SRT generators have this:

MAPPINGS = dict(linebreaks="\n", 

then:

self.line_delimiter = '\r\n' 

And on the WebVTT one:

output = ['WEBVTT\n']

I haven't looked at the other end of the generator, but maybe the line break within the substitution pattern isn't being recognized because it's a "\n" instead of the "\r\n" indicated as line_delimiter.

Another possibility is that the substitution pattern for the asterisks needed to work in single-line mode (by starting with (?s)) to capture the line break -- although if that was the case it would just not match at all.

http://www.regular-expressions.info/modifiers.html

This is related to this issue:

https://github.com/pculture/unisubs/issues/1733

[franontanaya]

This affects a more common case.

If a word is made regular within a caption that is in italics, the spaces around the asterisks are removed.

E.g.

*If you're not doing what you love,
then I think *you're* crazy.*

becomes:

<i>If you're not doing what you love,
then I think</i>you're<i>crazy.</i>

[franontanaya]

This is still ongoing. It only happens upon downloading from the site or via API, and I assume also for any integration that sends WebVTT. The spaces are still on the editor when a WebVTT file is uploaded.

[bendk]

Pretty sure that last change fixes things. Can you test and close if it does?

[franontanaya]

My test track and this track still break:

http://www.amara.org/en/videos/Sgpox8QPQang/en/1004546/

In the editor we have:

*to finish his first Super Enduro season
with *another* victory.*

But in the revision view above, and on download the spaces around another are gone. It only happens when there's more than one italic span, and regardless of the span or line break positions.

This would be where they are lost:

def get_contents(el):
    """Get the contents of the given element as a string of XML.
    """
    return "".join([x for x in el.itertext()]).strip()

Basically it's taking *to finish his first Super Enduro season with * and stripping the trailing space.

For style marks, whenever a mark is found with a white space on both sides:

• Odd marks should lose the space after.
• Even marks should lose the space before.

Other than that no white space trimming would happen.

As an aside, some beginning/end stripping should happen on the raw caption content, before retrieving the nodes. In regex something like:

s/^ | $| \n|\n //

Those extra spaces throw off character counts and get displayed on some media.

[jdragojevic]

@franontanaya - let me redeploy staging and then you should be able to test it there.

[franontanaya]

Sorry -- I thought it was included in a production deployment. All cases seem to work correctly in staging.