OverflowError: long int not supported in this build

[WilliamZh99]

Auto-reload is on. Simply save files over USB to run them or enter REPL to disable.
code.py output:
Traceback (most recent call last):
File "code.py", line 14, in
OverflowError: long int not supported in this build

Code done running.

Press any key to enter the REPL. Use CTRL-D to reload.

Adafruit CircuitPython 7.2.4 on 2022-03-31; Adafruit QT Py M0 with samd21e18

[caternuson]

hmmm....may not be able to support the small board builds.

There are several large values needed internally for computing timing values as well as masking.

Adafruit CircuitPython 7.2.5 on 2022-04-06; Adafruit QT Py M0 with samd21e18
>>> macro_period_us = 0x40000000
OverflowError: long int not supported in this build
>>> mask = 0xFFFFFF00
OverflowError: long int not supported in this build
>>> 

These were taken from the STM Arduino driver and the specifics are not documented elsewhere. So would need to figure out how to re-write this code to accommodate the small builds:

Adafruit_CircuitPython_VL53L4CD/adafruit_vl53l4cd.py

Lines 252 to 271 in a401767

	macro_period_us = int(2304 * (0x40000000 / osc_freq)) >> 6
	if inter_meas == 0:
	# continuous mode
	timing_budget_us -= 2500
	else:
	# autonomous mode
	timing_budget_us -= 4300
	timing_budget_us //= 2
	# VL53L4CD_RANGE_CONFIG_A register
	ms_byte = 0
	timing_budget_us <<= 12
	tmp = macro_period_us * 16
	ls_byte = int(((timing_budget_us + ((tmp >> 6) >> 1)) / (tmp >> 6)) - 1)
	while ls_byte & 0xFFFFFF00 > 0:
	ls_byte >>= 1
	ms_byte += 1
	ms_byte = (ms_byte << 8) + (ls_byte & 0xFF)
	self._write_register(_VL53L4CD_RANGE_CONFIG_A, struct.pack(">H", ms_byte))

[tekktrik]

Is it worth using a guard statement when importing to check for large integer support, and raise a custom exception explaining incompatibility instead?

[caternuson]

Maybe. The other option here, as suggested by @dhalbert, is to use reduced precision in the math. Just haven't worked out the specifics since there's a bit going on with the computations. But that might be too much of a hack just to accommodate the small int boards?

[dhalbert]

0x40000000 can be expressed exactly as a float. It's 1073741824, which is 2**30. So you could change

macro_period_us = int(2304 * (0x40000000 / osc_freq)) >> 6
to
macro_period_us = int(2304 * (1073741824.0 / osc_freq)) >> 6
since the division is already a float operation anyway. So this wouldn't lose any precision whether or not you have long ints.

[dhalbert]

For the 0xFFFFFF00 mask, I think you could shift 8 bits right before you mask, unless ls_byte is initially computed to be a longint.

[tekktrik]

I can try some of these changes and check it out.

[caternuson]

@tekktrik cool. thanks. i think it was some of the lines further down that gave me pause also. and didnt help that these were just translated from other source. i don't think i found any other documentation describing what is being done, which could have also helped to potentially just reimplement in some other way.

[caternuson]

neat. this got fixed. @FoamyGuy thanks!