[FEATURE] FreeType-based text renderer (-out=spupng with teletext/EIA608) #877

harrynull · 2018-01-05T11:26:24Z

Please prefix your pull request with one of the following: [FEATURE] [FIX] [IMPROVEMENT].

In raising this pull request, I confirm the following (please check boxes):

I have read and understood the contributors guide.
I have checked that another pull request for this purpose does not exist.
I have considered, and confirmed that this submission will be valuable to others.
I accept that this submission may not be used, and the pull request closed at the will of the maintainer.
I give this submission freely, and claim no ownership to its content.

My familiarity with the project is as follows (check one):

I have never used CCExtractor.
I have used CCExtractor just a couple of times.
I absolutely love CCExtractor, but have not contributed previously.
I am an active contributor to CCExtractor.

ABSTRACT
The pull request implements a FreeType-based text renderer, which can be used to generate the bitmaps for spupng in the case of teletext/EIA608.

CHANGES

Remove "ccx_encoders_spupng.h" (The old implementation of it) and "spupng_encoder.h" (encoders don't usually have header files)
Rename "spupng_encoder.c" to "ccx_encoders_spupng.c" (For consistency).
Add new implementation based on FreeType to "ccx_encoders_spupng.c".
Add a lot comments/document to the code.

KNOWN ISSUES
Note that the issues will also affect EIA-608 which may not have these issues before (But the new implementation is generally much cleaner, easier to maintain and more extensible and customizable than the old one).

~~Font needs to be manually set.~~ FIXED (New option -font added to specify font)
No color: tag is rendered as-is.
XML position may be not correct.
~~Characters like "$" will be cut.~~ FIXED
~~Accented letters don't render. (could be encoding problem)~~ FIXED

NOTE
Please merge this PR after #876 , or it won't compile.

EXAMPLES
Note that the preview is not latest. Please refer to comments for the latest preview.

Example of a good output.

Example of tag rendered as-is.

![sub0000](https://user-images.githubusercontent.com/7413706/34639001-3e7d4ae2-f312-11e7-806b-b92c4d739b7b.png) (original generated png file) ![sub0000](https://user-images.githubusercontent.com/7413706/34639115-71ab680c-f314-11e7-8c70-5f75801d7dbe.jpg) (transparency is replaced by black) Example of "$" being cut.

FIXED

saurabhshri · 2018-01-06T20:19:51Z

This is looking good. 🙂 Okay, so couple of questions and points :

Could you please also attach the generate XML file?
Does the program generate the png with transparent background or black?

This question is important because these are directly displayed as it is by video players at the position defined by the XML file. For example :

So, if they are of transparent bg, they might not be visible properly, and if the complete rectangular frame is black : they might cover full regions.

This is how usual SPUPNG png looks :

Notice how only the text has filled bg?

The rendering of tags, is it the limitation of library or it can be implemented later? Because they can definitely have style in them.

I did not understand "Example of "$" being cut.". I can see the $ symbol, what does being cut mean?
"Note that the issues will also affect EIA-608 which may not have these issues before" could you please point to specifics about what might get broken if this is added. It'll be an important criteria as we'd not like avoid breaking stuff as much as possible.

harrynull · 2018-01-07T04:02:30Z

Could you please also attach the generate XML file?

I just noticed that the generated xml has incorrect offset values. I will upload them as soon as I fix them.

Does the program generate the png with transparent background or black?

The program generates png with transparent background. But it won't be hard to change it to black.
I can also clip the png so that it only contains the text.
So which is the one that we want?
transparent background + fixed width
Like
OR
text with black background + fixed width + transparent background where no text?
Like
(The old implementation)

The rendering of tags, is it the limitation of library or it can be implemented later? Because they can definitely have style in them.

FreeType can support colors, styles (bold/italic, by using different .tts), as long as you can parse these styles from tags.

I did not understand "Example of "$" being cut.". I can see the $ symbol, what does being cut mean?

Although it may be not that obvious, "$" is actually cut a little bit, depending on what font you use.
I've changed a new font which can demonstrate the issue more clearly.

"Note that the issues will also affect EIA-608 which may not have these issues before" could you please point to specifics about what might get broken if this is added. It'll be an important criteria as we'd not like avoid breaking stuff as much as possible.

All the issues that I mentioned.

Font needs to be manually set.

This one is reasonable I think, since we don't know what font the user wants to use, and we can't specify a default font since we don't know if the user has it.

Characters like "$" will be cut.

Not really hard to fix. I will see if I can fix it.

Accented letters don't render. (could be encoding problem)

Same.

No color: tags are rendered as-is.

It needs parse of tags. I will see how the old code did it.

I'd open a new branch for it and merge it when everything works if we don't want to break anything.

saurabhshri · 2018-01-07T09:11:20Z

I just noticed that the generated xml has incorrect offset values. I will upload them as soon as I fix them.

Sure!

So which is the one that we want?

I think the PNGs like the following would be perfect. They can then be placed anywhere on the screen (with the help of respective XML file).

FreeType can support colors, styles (bold/italic, by using different .tts), as long as you can parse these styles from tags.

If you're not including those changes in this PR, do remember to raise an issue once this is merged so that we don't forget it later on. 😉

harrynull · 2018-01-07T13:44:35Z

A sample of generated xml is attached.
Note that the positioning may be not correct. The offset values are constant now. To get it work, we maybe need to know the size of the video.
I cannot make a video with SPUPNG subtitles because spumux somewhy didn't work for me so I can't test it.
COMEDY.zip

saurabhshri · 2018-01-08T09:31:52Z

Please use relative path in XML instead of full path :

/mnt/d/Tests/EIA608/COMEDY.d/sub0000.png --> COMEDY.d/sub0000.png

Also, could you also show how the new pngs look? Attach them here.

harrynull · 2018-01-08T10:12:23Z

The images in the PR has updated to the new pngs.
Relative path issue fixed (though it's not me who introduced the issue)

New xml file sample: teletext.zip

saurabhshri · 2018-01-08T11:35:22Z

src/CMakeLists.txt

@@ -52,6 +52,7 @@ include_directories ("${PROJECT_SOURCE_DIR}/extractors/")
 include_directories ("${PROJECT_SOURCE_DIR}/wrappers/")
 include_directories ("${PROJECT_SOURCE_DIR}/libpng/")
 include_directories ("${PROJECT_SOURCE_DIR}/zlib/")
+include_directories ("${PROJECT_SOURCE_DIR}/freetype/include")


Please add trailing slash. It doesn't include all files without it.

saurabhshri · 2018-01-08T11:36:26Z

src/lib_ccx/ccx_encoders_spupng.c

+#define FONT_PATH "C:\\Windows\\Fonts\\calibri.ttf"
+#else
+#define FONT_PATH "/usr/share/fonts/truetype/noto/NotoSans-Regular.ttf"
+#endif


Missing font for macOS. Results in

Failed to init freetype when trying to init face, error code: 1 Error: Cannot write teletext2.d/sub0000.png: No such file or directory

The font problem is actually one of the issue. I didn't fix it because I think it's part of customization. But yeah, I will fix it now, by adding an option.

saurabhshri

Please allow a parameter to pass the path of font in case the default one is missing, changing path in code is not desirable.

I tried couple of fonts and for some reason the pngs generated were quite pixelated - is it normal?

Looks like text justification is not really making things neat. It leads to words breaking in newlines which doesn't contribute much to readability.

Let the text be as it is, do not justify it.

saurabhshri · 2018-01-08T11:47:10Z

@harrynull Also it'd would have been better if you commented instead of changing description because, first, it tampers the history and comments seem senseless, and second it's easier to follow things. From next time always comment.

harrynull · 2018-01-08T13:03:18Z

@saurabhshri
Text justification does not break words in newlines. Words are break simply because the given space is not enough to contain them so I break them to a new line so that no subtitles will be lost.
To solve it, you can either make the font smaller, or make the canvas bigger, by changing the value of FONT_SIZE and CANVAS_WIDTH.
The pixelated output issue is not expected. Could you tell me what font and FONT_SIZE and CANVAS_WIDTH values you are using?

harrynull · 2018-01-08T14:10:24Z

Nevermind, it's a bug related to my black background algorithm. I've fixed it.

harrynull · 2018-01-08T14:24:56Z

The latest preview.

saurabhshri · 2018-01-09T17:56:54Z

src/lib_ccx/params.c

 	mprint ("                       comments in the XML file.\n");
+	mprint ("                -font: Specify the font that is used when generating png files\n");


Change it to Specify the full path of the font that is to be used when generating SPUPNG files.

saurabhshri · 2018-01-09T17:58:18Z

@cfsmp3 Once that parameter description is edited, it's good for merge, thanks @harrynull - nice job! :)

Implementation of text renderer

8037a09

harrynull mentioned this pull request Jan 5, 2018

[FEATURE] Add FreeType2 #876

Merged

9 tasks

harrynull added 4 commits January 7, 2018 19:27

Fix some characters being cut

0a3dd03

Fix encoding and other bugs

1a1f7d1

Add black background & fix bugs

e4739b0

Fix more bugs

03c99b9

Change to relative path

159a6f4

saurabhshri reviewed Jan 8, 2018

View reviewed changes

saurabhshri suggested changes Jan 8, 2018

View reviewed changes

Add a font option & Default font for MacOS & Fix anti-aliasing

8848a55

Document -font & enlarge default canvas

79d1355

saurabhshri reviewed Jan 9, 2018

View reviewed changes

cfsmp3 merged commit 930ca71 into CCExtractor:master Jan 9, 2018

This was referenced Jan 9, 2018

[BUG] -out=spupng with EIA608/teletext: tags are rendered as-is. #892

Closed

[BUG] -out=spupng with EIA608/teletext: offset values in XML may be not correct #893

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FEATURE] FreeType-based text renderer (-out=spupng with teletext/EIA608) #877

[FEATURE] FreeType-based text renderer (-out=spupng with teletext/EIA608) #877

harrynull commented Jan 5, 2018 •

edited

Loading

saurabhshri commented Jan 6, 2018

harrynull commented Jan 7, 2018 •

edited

Loading

saurabhshri commented Jan 7, 2018

harrynull commented Jan 7, 2018

saurabhshri commented Jan 8, 2018

harrynull commented Jan 8, 2018 •

edited

Loading

saurabhshri Jan 8, 2018

saurabhshri Jan 8, 2018 •

edited

Loading

harrynull Jan 8, 2018

saurabhshri left a comment •

edited

Loading

saurabhshri commented Jan 8, 2018 •

edited

Loading

harrynull commented Jan 8, 2018

harrynull commented Jan 8, 2018

harrynull commented Jan 8, 2018

saurabhshri Jan 9, 2018

saurabhshri commented Jan 9, 2018

		mprint (" comments in the XML file.\n");
		mprint (" -font: Specify the font that is used when generating png files\n");

[FEATURE] FreeType-based text renderer (-out=spupng with teletext/EIA608) #877

[FEATURE] FreeType-based text renderer (-out=spupng with teletext/EIA608) #877

Conversation

harrynull commented Jan 5, 2018 • edited Loading

saurabhshri commented Jan 6, 2018

harrynull commented Jan 7, 2018 • edited Loading

saurabhshri commented Jan 7, 2018

harrynull commented Jan 7, 2018

saurabhshri commented Jan 8, 2018

harrynull commented Jan 8, 2018 • edited Loading

saurabhshri Jan 8, 2018

Choose a reason for hiding this comment

saurabhshri Jan 8, 2018 • edited Loading

Choose a reason for hiding this comment

harrynull Jan 8, 2018

Choose a reason for hiding this comment

saurabhshri left a comment • edited Loading

Choose a reason for hiding this comment

saurabhshri commented Jan 8, 2018 • edited Loading

harrynull commented Jan 8, 2018

harrynull commented Jan 8, 2018

harrynull commented Jan 8, 2018

saurabhshri Jan 9, 2018

Choose a reason for hiding this comment

saurabhshri commented Jan 9, 2018

harrynull commented Jan 5, 2018 •

edited

Loading

harrynull commented Jan 7, 2018 •

edited

Loading

harrynull commented Jan 8, 2018 •

edited

Loading

saurabhshri Jan 8, 2018 •

edited

Loading

saurabhshri left a comment •

edited

Loading

saurabhshri commented Jan 8, 2018 •

edited

Loading