[lua] Port _hx_bit_clamp to haxe code

[tobil4sk]

We can now rely on normal dce so that the code for this function isn't generated unless required. It also provides more control over what code is generated, which has allowed adding checks like lua_ver >= 5.3 to avoid redundant compatibility implementations.

This also allows compile time abstractions that aren't supported on lua, like inline function calls, which reduce duplicate code.

The generic implementation checks what is supported to decide which implementation to use, then patches the method to avoid having to rerun the feature detection every time. This can't go into static init because bitwise operations are needed in regex constructors, which are called in some static initialisers. I think this way is also fairly self contained which is nice.

Currently a test is failing because __lua_Boot.clampInt32 is nil, I think my check @:ifFeature("use._bitop") isn't sufficient.

Below is the generated lua code, which could be improved, but I'm surprised at how close it is to the original!

Expand to view

__lua_Boot.clampInt32 = function(v) 
  local _hx_1_result_status, _hx_1_result_value = _G.pcall(_G.load, [[return function(v) 
    if ((v <= 2147483647) and (v >= -2147483648)) then 
      if (v > 0) then 
        do return _G.math.floor(v) end;
      else
        do return _G.math.ceil(v) end;
      end;
    end;
    if ((v ~= v) or not ((v > -_G.math.huge) and (v < _G.math.huge))) then 
      do return nil end;
    end;
    local v = v;
    if (v > 2251798999999999) then 
      v = v * 2;
    end;
    do return (v & 0x7FFFFFFF) - (v & 0x80000000) end;
  end]]);
  local nativeOperators;
  if (_hx_1_result_status and (_hx_1_result_value ~= nil)) then 
    local fn = _hx_1_result_value;
    nativeOperators = fn();
  else
    nativeOperators = nil;
  end;
  local clampImpl = (function() 
    local _hx_2
    if (nativeOperators ~= nil) then 
    _hx_2 = nativeOperators; elseif (_hx_bit_raw == nil) then 
    _hx_2 = function(v) 
      if ((v <= 2147483647) and (v >= -2147483648)) then 
        if (v > 0) then 
          do return _G.math.floor(v) end;
        else
          do return _G.math.ceil(v) end;
        end;
      end;
      if ((v ~= v) or not ((v > -_G.math.huge) and (v < _G.math.huge))) then 
        do return nil end;
      end;
      local v = v;
      v = _G.math.fmod(v, 4294967296);
      if (v >= 2147483648) then 
        v = v - 4294967296;
      end;
      do return v end;
    end; else 
    _hx_2 = function(v) 
      if ((v <= 2147483647) and (v >= -2147483648)) then 
        if (v > 0) then 
          do return _G.math.floor(v) end;
        else
          do return _G.math.ceil(v) end;
        end;
      end;
      if ((v ~= v) or not ((v > -_G.math.huge) and (v < _G.math.huge))) then 
        do return nil end;
      end;
      local v = v;
      if (v > 2251798999999999) then 
        v = v * 2;
      end;
      local band = _hx_bit_raw.band;
      do return band(v, 2147483647) - _G.math.abs(band(v, -2147483648)) end;
    end; end
    return _hx_2
  end )();
  __lua_Boot.clampInt32 = clampImpl;
  do return clampImpl(v) end;
end

Related to #8852.

[tobil4sk]

This is something that needs to be addressed, otherwise using --dce=no generates clampNativeOperator code outside testFunctionSupport which breaks older versions.

[tobil4sk]

@Simn Would you have any suggestions here? I tried turning clampWrapper into a macro, but that gives:

[ERROR] /.../haxe/std/lua/Boot.hx:283: characters 48-81

 283 |    final nativeOperators = testFunctionSupport(clampWrapper(clampNativeOperator));
     |                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     | You cannot access the lua package while in a macro (for lua.Boot)

Even if everything in that file is wrapped with #if !macro

[jdonaldson]

I looked into why @:ifFeature("use._bitop") doesn't work here. The root cause is that use._bitop has two separate systems that don't connect:

Code generation time (after DCE): genlua.ml's gen_bitop calls add_feature ctx "use._bitop" when compiling a & b → _hx_bit.band(a, b). This tells genlua.ml to include _hx_bit.lua, but DCE has already finished by this point.

DCE time: @:ifFeature("use._bitop") is only triggered when DCE encounters __define_feature__("use._bitop", ...). The only place that exists is Bit.__init__() — which only runs if someone explicitly uses the lua.Bit class. Normal Haxe bitwise operators (a & b, a | b) are compiled directly by genlua.ml and never touch lua.Bit, so DCE never sees the feature being defined.

Fix: Remove @:ifFeature from clampInt32. Normal DCE tracing should keep it alive through its callers:

• Std.int() → Boot.clampInt32()
• Boot.__instanceof() → clampInt32()

If nothing calls clampInt32, DCE removes it naturally — which is correct.

Also worth noting: _hx_bit.lua (raw Lua) calls the global _hx_bit_clamp directly, not __lua_Boot.clampInt32. So if _hx_bit_clamp.lua is removed in favor of the Haxe port, the bit32 fallback path in _hx_bit.lua would need updating too (either assign _hx_bit_clamp = __lua_Boot.clampInt32, or port _hx_bit.lua to Haxe as well).

[tobil4sk]

Code generation time (after DCE): genlua.ml's gen_bitop calls add_feature ctx "use._bitop" when compiling a & b → _hx_bit.band(a, b). This tells genlua.ml to include _hx_bit.lua, but DCE has already finished by this point.

This analysis seems correct, but claude was a bit too confident about the solution.

I did some digging and it seems like python already deals with the same problem: #11427. I lifted the solution from there, and the test seems to pass now.