Method WordFontAttributes does not work

[zikcheng]

Environment

• Tesseract Version: tesseract 4.00.00alpha
• Commit Number: 8e55e52
• Platform: Ubuntu 16.04.1

Current Behavior:

Method WordFontAttributes returns null if using tesseract 4.00.00alpha with 4.00 tessdata, but it returns font name if using tesseract 4.00.00alpha with 3.04.00 tessdata. The test image link is eurotext.tif
I first met this problem when I use tesserocr [tesserocr#68] .(sirfz/tesserocr#68)

Expected Behavior:

With method WordFontAttributes we can get correct font attributes of recognized words.

[amitdo]

The new LSTM engine does not support this feature and probably won't support it any time soon.

[phildrip]

Is there an alternative way to get font sizing etc? Do you mean that just this method won't be supported, or the feature in general?

[amitdo]

Is there an alternative way to get font sizing etc?

You can still use --oem 0 with traineddata from here: https://github.com/tesseract-ocr/tessdata.
Note that the traineddata in the 'best' folder won't work with --oem 0.

[amitdo]

Do you mean that just this method won't be supported, or the feature in general?

I have reasons to believe that the new LSTM engine is unlikely to have a feature that includes font identification (name and properties like is_bold) in the near future.

Important note: I'm a contributer from the community, and the main developer not always shares all his plans for upcoming release(s) with the community.

[phildrip]

Thanks for the reply! It looks like the old ocr engine is going to be removed, though (issue #707)... And does using OcrEngineMode 0 mean the behaviour is the same as v3?

What I'm getting to is:

1. I need to be able to extract font size information (font names aren't so useful) - is there any way at all of doing so with LSTM/v4?
2. If I use OcrEngineMode 0 to be able to get this info, will that be removed from v4 at a later date?
3. Is there any advantage to using v4 with OcrEngineMode 0 vs v3.05?

Thanks again for the help!

[amitdo]

It looks like the old ocr engine is going to be removed, though (issue #707)...

It's not known when exactly it will be removed. Until then you can still use it.

And does using OcrEngineMode 0 mean the behaviour is the same as v3?

It's basically the same as 3.05.01.

I need to be able to extract font size information (font names aren't so useful) - is there any way at all of doing so with LSTM/v4?

There is no method in the API to get font sizes for the lstm engine.

If I use OcrEngineMode 0 to be able to get this info, will that be removed from v4 at a later date?

Probably yes.

Is there any advantage to using v4 with OcrEngineMode 0 vs v3.05?

The accuracy should be the same.

[amitdo]

The relative font size for a textline can be estimated by calculating the xheight of the line and compare it
to the median xheight of the other textlines in the page.

[phildrip]

Ok, thanks for the info 👍

[amitdo]

@phildrip,

I looked at the relevant code again, and I think the font size functionality (but not font name and properties like is_bold) can be restored when using the lstm engine.

I will provide further details (and probably send a PR) in the upcoming days.

[phildrip]

That's great news, thanks!

[amitdo]

// Returns the font attributes of the current word. If iterating at a higher
// level object than words, eg textlines, then this will return the
// attributes of the first word in that textline.
// The actual return value is a string representing a font name. It points
// to an internal table and SHOULD NOT BE DELETED. Lifespan is the same as
// the iterator itself, ie rendered invalid by various members of
// TessBaseAPI, including Init, SetImage, End or deleting the TessBaseAPI.
// Pointsize is returned in printers points (1/72 inch.)
const char* LTRResultIterator::WordFontAttributes(bool* is_bold,
                                                  bool* is_italic,
                                                  bool* is_underlined,
                                                  bool* is_monospace,
                                                  bool* is_serif,
                                                  bool* is_smallcaps,
                                                  int* pointsize,
                                                  int* font_id) const {
  if (it_->word() == NULL) return NULL;  // Already at the end!
  if (it_->word()->fontinfo == NULL) {
    *font_id = -1;
    return NULL;  // No font information.
  }
  const FontInfo& font_info = *it_->word()->fontinfo;
  *font_id = font_info.universal_id;
  *is_bold = font_info.is_bold();
  *is_italic = font_info.is_italic();
  *is_underlined = false;  // TODO(rays) fix this!
  *is_monospace = font_info.is_fixed_pitch();
  *is_serif = font_info.is_serif();
  *is_smallcaps = it_->word()->small_caps;
  float row_height = it_->row()->row->x_height() +
      it_->row()->row->ascenders() - it_->row()->row->descenders();
  // Convert from pixels to printers points.
  *pointsize = scaled_yres_ > 0
      ? static_cast<int>(row_height * kPointsPerInch / scaled_yres_ + 0.5)
      : 0;

  return font_info.name;
}

The problem:

if (it_->word()->fontinfo == NULL) {
    *font_id = -1;
    return NULL;  // No font information.
}

With the LSTM engine the it_->word()->fontinfo will always be NULL.
So pointsize has no chance to be calculated.

pointsize is calculated based on row (=line) height. pointsize is the font size in points of the line, so it should not be in WordFontAttributes().

There is another function where you can get row height.

void LTRResultIterator::RowAttributes(float* row_height, float* descenders,
                                      float* ascenders) const {
  *row_height = it_->row()->row->x_height() + it_->row()->row->ascenders() -
                it_->row()->row->descenders();
  *descenders = it_->row()->row->descenders();
  *ascenders = it_->row()->row->ascenders();
}

I think pointsize calculation should be moved into this function.

[amitdo]

@zdenop, @stweil
Do you have any comment?

[zdenop]

At the moment I have a limited internet access. If you make a pull request I can merge it ;.-)

[stweil]

Although my current main focus is getting the text from images, there are also important use cases where text attributes are important as well. As I understand your comments, currently the new LSTM recognizer does not support the method WordFontAttributes, so it is not possible to get text attributes with that recognizer. Adding support for the font size recognition with LSTM seems to be feasible, but other text attributes like for example bold or italic are desirable, too.