Building Source Han Serif fails when building HVAR: 'H' format requires 0 <= number <= 65535

[NightFurySL2001]

I did the following steps:

1. download https://github.com/adobe-fonts/source-han-serif/blob/master/Masters/ExtraLight/OTC/VF/cidfont.VF.K.unhinted and then pass it through tx -ufo cidfont.VF.K.unhinted el.ufo
2. download https://github.com/adobe-fonts/source-han-serif/blob/master/Masters/Heavy/OTC/VF/cidfont.VF.K.unhinted and then pass it through tx -ufo cidfont.VF.K.unhinted h.ufo
3. download https://github.com/adobe-fonts/source-han-serif/blob/master/Masters/designspaces/SourceHanSerifK-VF.designspace and change source filename to el.ufo and h.ufo (referencing the UFOs from above step)
4. run fontmake -o variable-cff2 -m SourceHanSerifK-VF.designspace --verbose DEBUG. Observed the following error.

INFO:fontTools.varLib:Internal master locations:
[{'weight': 0.0}, {'weight': 1000.0}]
INFO:fontTools.varLib:Internal axis supports:
{'weight': [0.0, 0.0, 1000.0]}
INFO:fontTools.varLib:Normalized master locations:
[{'weight': 0.0}, {'weight': 1.0}]
INFO:fontTools.varLib:Index of base master: 0
INFO:fontTools.varLib:Building variable font
INFO:fontTools.varLib:Loading master fonts
INFO:fontTools.varLib:Generating fvar
INFO:fontTools.varLib:Building variations tables
INFO:fontTools.varLib:Generating avar
INFO:fontTools.varLib:Generating MVAR
INFO:fontTools.varLib:Generating HVAR
ERROR:root:In 'sourcehanserif.designspace': Generating fonts from Designspace failed: 'H' format requires 0 <= number <= 65535
Traceback (most recent call last):
  File "C:\Python311\Lib\site-packages\fontmake\font_project.py", line 1208, in run_from_designspace
    self._run_from_designspace_interpolatable(
  File "C:\Python311\Lib\site-packages\fontmake\font_project.py", line 1326, in _run_from_designspace_interpolatable
    self.build_variable_fonts(
  File "C:\Python311\Lib\site-packages\fontmake\font_project.py", line 470, in build_variable_fonts
    fonts = ufo2ft.compileVariableCFF2s(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\ufo2ft\__init__.py", line 295, in compileVariableCFF2s
    return VariableCFF2sCompiler(**kwargs).compile_variable(designSpaceDoc)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\ufo2ft\_compilers\baseCompiler.py", line 420, in compile_variable
    vfNameToTTFont = self._merge(designSpaceDoc, excludeVariationTables)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\ufo2ft\_compilers\interpolatableOTFCompiler.py", line 32, in _merge
    return varLib.build_many(
           ^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\fontTools\varLib\__init__.py", line 1133, in build_many
    vf = build(
         ^^^^^^
  File "C:\Python311\Lib\site-packages\fontTools\varLib\__init__.py", line 1218, in build
    _add_HVAR(vf, model, master_fonts, axisTags)
  File "C:\Python311\Lib\site-packages\fontTools\varLib\__init__.py", line 518, in _add_HVAR
    _add_VHVAR(font, axisTags, HVAR_FIELDS, getAdvanceMetrics)
  File "C:\Python311\Lib\site-packages\fontTools\varLib\__init__.py", line 589, in _add_VHVAR
    directSize = len(writer.getAllData())
                     ^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\fontTools\ttLib\tables\otBase.py", line 680, in getAllData
    self._doneWriting(internedTables)
  File "C:\Python311\Lib\site-packages\fontTools\ttLib\tables\otBase.py", line 507, in _doneWriting
    item.subWriter._doneWriting(
  File "C:\Python311\Lib\site-packages\fontTools\ttLib\tables\otBase.py", line 505, in _doneWriting
    items[i] = item.getCountData()
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\fontTools\ttLib\tables\otBase.py", line 860, in getCountData
    return {1: packUInt8, 2: packUShort, 4: packULong}[self.size](v)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\fontTools\ttLib\tables\otBase.py", line 868, in packUShort
    return struct.pack(">H", value)
           ^^^^^^^^^^^^^^^^^^^^^^^^
struct.error: 'H' format requires 0 <= number <= 65535

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Python311\Lib\site-packages\fontmake\__main__.py", line 728, in main
    project.run_from_designspace(inputs.designspace_path, **args)
  File "C:\Python311\Lib\site-packages\fontmake\font_project.py", line 1224, in run_from_designspace
    raise FontmakeError(
fontmake.errors.FontmakeError: In 'sourcehanserif.designspace': Generating fonts from Designspace failed: 'H' format requires 0 <= number <= 65535

[punchcutter]

Isn't this the same as #939 ? We don't use fontmake to build Source Han. Related discussion: googlefonts/ufo2ft#462

[NightFurySL2001]

This seem to be on ufo2ft; #939 was only about TT VF.

[punchcutter]

TTF or CFF2 VF is really the same thing in this case and either one needs ufo2ft to set things right unless you override it all like we do for Source Han.

[ChiuMing-Neko]

#939 According to this issue, a glyph count exceeding 65527 may break the fontmake building pipeline. This was observed when building Source Han Serif with its full glyph sets, which consistently failed. However, fontmake was able to output the fonts after some glyphs were removed from Source Han Serif.

This issue affects not only variable-cff2 but also the building of other instances. Even after attempting to set a custom attribute within the UFO file itself to hope it able to build over on format 3, fontmake still cannot produce the font and consistently breaks due to the excessively large glyph count.