HH\FormatString Typing Errors in sprintf() and invariant()

[justinwray]

Within the Facebook CTF project, we just upgraded from HHVM 3.14.5 to 3.18.1 in our latest PR facebookarchive/fbctf#535.

Now that we are on a newer version of HHVM, we are getting typing errors for a few sprintf() and invariant() calls. The errors indicate the argument is incompatible with HH\FormatString.

Our understanding is the HH\FormatString definition looks like such:

namespace HH {
  newtype FormatString<T> = string;
}

We can confirm that the various [relevant] values passed into sprintf() and invariant() are of type string. Even adding in strval() or additional is_string() checks made no difference to the typing errors.

Unfortunately, there doesn't seem to be any documentation available on HH\FormatString usage or the expectations for it's typing in functions like sprintf(), as issue #209 is still outstanding. We were hoping someone might have some insight on how to properly use HH\FormatString or correct this typing error.

Here is an example typing error, from hh_client:

src/Utils.php:12:5,52: Invalid argument (Typing[4110])
  /tmp/hh_server/hhi_217a2dc9/printf.hhi:92:30,45: This is an object of type HH\FormatString
  /tmp/hh_server/hhi_217a2dc9/printf.hhi:87:70,75: It is incompatible with a string
src/Utils.php:12:5,52: This argument must be a literal string (Typing[4027])

Here is the code-block from the above example error:

function must_have_idx<Tk, Tv>(?KeyedContainer<Tk, Tv> $arr, Tk $idx): Tv {
  invariant($arr !== null, 'Container is null');
  $result = idx($arr, $idx);
  invariant(
    $result !== null,
    sprintf('Index %s not found in container', $idx),
  );
  return $result;
}

[fredemmott]

Which specific parameter is it complaining about? None line up with the column numbers in the error.

Anyway, that error is telling you that the parameter must be literal strings, i.e. not a string-typed variable.

[fredemmott]

$pattern = '%s';
sprintf($pattern, 'foo');

^^^ invalid, not a literal string

sprintf('%s', 'foo');

^ valid, literal string

HH\FormatString is special-cased in the typechecker, using a template interface (PlainSprintf is the relevant one here), so it understands invalid/incorrect format strings + parameters, eg:

sprintf('%s'); // typechecker error
sprintf('%d', 'foo'); // typechecker error

[justinwray]

@fredemmott Thank you for the assistance and response.

Which specific parameter is it complaining about? None line up with the column numbers in the error.

From the example error and example code, line 12 is the following line:

    sprintf('Index %s not found in container', $idx),

Anyway, that error is telling you that the parameter must be literal strings, i.e. not a string-typed variable.

Fully understood that the format string passed to sprintf() cannot be a variable and must be a string, but that is the case here.

In fact, switching the above code to:

    sprintf('%s', 'foo');

Results in the same error:

src/Utils.php:12:5,24: Invalid argument (Typing[4110])
  /tmp/hh_server/hhi_343a78b7/printf.hhi:92:30,45: This is an object of type HH\FormatString
  /tmp/hh_server/hhi_343a78b7/printf.hhi:87:70,75: It is incompatible with a string

[justinwray]

After further testing, it looks like we found the culprit. The original code is passing an sprintf() call into an invariant() call. As seen here:

function must_have_idx<Tk, Tv>(?KeyedContainer<Tk, Tv> $arr, Tk $idx): Tv {
  invariant($arr !== null, 'Container is null');
  $result = idx($arr, $idx);
  invariant(
    $result !== null,
    sprintf('Index %s not found in container', $idx),
  );
  return $result;
}

This worked in HHVM 3.14.15 without warning as the sprintf() and family functions did not have the literal string type enforcement.

We can see the invariant() definition here facebook/hhvm/blob/master/hphp/hack/hhi/func_pointers.hhi:

function invariant(
  $condition, // e.g. is_int($x) or ($y instanceof SomeClass)
  \HH\FormatString<PlainSprintf> $f, ...$f_args
): void;

The second parameter of invariant() is expected to be of \HH\FormatString type.

Changing the above code to the following works and no longer results in any typechecking errors:

function must_have_idx<Tk, Tv>(?KeyedContainer<Tk, Tv> $arr, Tk $idx): Tv {
  invariant($arr !== null, 'Container is null');
  $result = idx($arr, $idx);
  invariant(
    $result !== null,
    'Index %s not found in container',
    $idx,
  );
  return $result;
}

@fredemmott Thank you again for your help!

[fredemmott]

Closing; for future reference, https://www.facebook.com/groups/hhvm.general/ or #hhvm on Freenode are better places for support.

Partially dupe of #209