Occasional Integration test failure

[pitrou]

On a PR I got what seemed to be an occasional unrelated error in the integration tests between Javascript and C++:
https://github.com/apache/arrow/actions/runs/15113644177/job/42478772147#step:11:20082

If you expand the failure, you get the following logs:

  ################# FAILURES #################
  FAILED TEST: interval_mdn C++ producing,  JS consuming
  <class 'RuntimeError'>: Command failed: /build/js/bin/integration.ts -a /tmp/tmp9z_g9id9/4b71e8dd_generated_interval_mdn.json_as_file -j /tmp/arrow-integration-da8j13w3/generated_interval_mdn.json --mode VALIDATE
  With output:
  --------------
  
   json: /tmp/arrow-integration-da8j13w3/generated_interval_mdn.json
  arrow: /tmp/tmp9z_g9id9/4b71e8dd_generated_interval_mdn.json_as_file
  Error: json and arrow record batches report the same values: fail 
   Error: 4: -1109815901,1503025502,1803493504,261723424 !== -1109815901,1503025502,1803493496,261723424
      at file:///build/js/bin/integration.ts:216:23
      at compareVectors (file:///build/js/bin/integration.ts:219:7)
      at file:///build/js/bin/integration.ts:191:13
      at compareTableIsh (file:///build/js/bin/integration.ts:193:7)
      at validateReaderIntegration (file:///build/js/bin/integration.ts:147:13)
      at validate (file:///build/js/bin/integration.ts:134:5)
      at async file:///build/js/bin/integration.ts:59:21
      at validateReaderIntegration (file:///build/js/bin/integration.ts:150:15)
      at validate (file:///build/js/bin/integration.ts:134:5)
      at async file:///build/js/bin/integration.ts:59:21
  
  --------------
  
  
  FAILED TEST: interval_mdn JS producing,  C++ consuming
  <class 'RuntimeError'>: Command failed: /build/cpp/debug/arrow-json-integration-test --integration --arrow=/tmp/tmp9z_g9id9/847e76a4_generated_interval_mdn.json_as_file --json=/tmp/arrow-integration-da8j13w3/generated_interval_mdn.json --mode=VALIDATE
  With output:
  --------------
  Error message: Invalid: Record batch 1 did not match
  JSON:
  f1:   [
      null,
      null,
      -1233922984M1832700302d3945364585489654316ns,
      -865764767M-1868167704d5995137276854357748ns,
      -1109815901M1503025502d1124093548480635000ns,
      null,
      -1853878757M-470410382d-3482109426864569063ns,
      -274343993M150680692d1713410614578950687ns,
      null,
      1204985559M1793434370d37200152077710886ns
    ]
  
  Arrow:
  f1:   [
      null,
      null,
      -1233922984M1832700302d3945364585489654316ns,
      -865764767M-1868167704d5995137276854357748ns,
      -1109815901M1503025502d1124093548480635008ns,
      null,
      -1853878757M-470410382d-3482109426864569063ns,
      -274343993M150680692d1713410614578950687ns,
      null,
      1204985559M1793434370d37200152077710886ns
    ]

And the explanation seems to be limited floating-point precision when the value -1109815901M1503025502d1124093548480635000ns is read from the JSON file. The nanoseconds components is 1124093548480635000 in the JSON, but Javascript only knows floating-point numbers and they don't have enough precision for this large integer. The nanoseconds component is therefore rounded to 1124093548480635008, and that fails comparing to the data received over Arrow IPC.

[pitrou]

cc @sumimakito @trxcllnt @domoritz

[trxcllnt]

These two lines look fishy:

arrow-js/src/util/interval.ts

Lines 49 to 50 in ea5593a

	data[ai++] = Number(BigInt(nanoseconds) & BigInt(0xFFFFFFFF));
	data[ai++] = Number(BigInt(nanoseconds) >> BigInt(32));