refactor: use usize for Array, ArrayBuffer, TypedArray, DataView

[zhuzhu81998]

Closes #4766 .

Many lines of changes are just deleting as usize really.

[github-actions]

Test262 conformance changes

Test result	main count	PR count	difference
Total	52,963	52,963	0
Passed	50,073	50,073	0
Ignored	2,072	2,072	0
Failed	818	818	0
Panics	0	0	0
Conformance	94.54%	94.54%	0.00%

Tested main commit: 0698e7063866a1b40be043a1089be43d3224db4b
Tested PR commit: 3ca31ead50be325a8dbbc2bbc82f3a3a63e29907
Compare commits: 0698e70...3ca31ea

[codecov]

Codecov Report

❌ Patch coverage is 56.13208% with 93 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.25%. Comparing base (6ddc2b4) to head (3ca31ea).
⚠️ Report is 917 commits behind head on main.

Files with missing lines	Patch %	Lines
core/engine/src/builtins/typed_array/builtin.rs	24.39%	31 Missing ⚠️
core/engine/src/builtins/array/mod.rs	61.42%	27 Missing ⚠️
core/engine/src/builtins/regexp/mod.rs	52.94%	8 Missing ⚠️
core/engine/src/builtins/array_buffer/mod.rs	40.00%	6 Missing ⚠️
core/engine/src/object/builtins/jsdataview.rs	0.00%	6 Missing ⚠️
cli/src/debug/limits.rs	0.00%	2 Missing ⚠️
core/engine/src/builtins/array_buffer/shared.rs	66.66%	2 Missing ⚠️
core/engine/src/builtins/string/mod.rs	71.42%	2 Missing ⚠️
core/engine/src/builtins/array/from_async.rs	0.00%	1 Missing ⚠️
core/engine/src/builtins/array_buffer/utils.rs	0.00%	1 Missing ⚠️
... and 7 more

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #4787       +/-   ##
===========================================
+ Coverage   47.24%   59.25%   +12.00%     
===========================================
  Files         476      563       +87     
  Lines       46892    62790    +15898     
===========================================
+ Hits        22154    37204    +15050     
- Misses      24738    25586      +848     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[hansl]

Just a nit, that's not a useful conversion. You can use MAX_SAFE_INTEGER_I64.

[zhuzhu81998]

just saw #4083 😅 (@jedel1043 ig the "spec-related checks" reasons are rediscovered)
however the theoretically possible case of the length of e.g. a string being over usize will never happen in practice. both firefox and chrome has the length limit well below a usize on 32-bit: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/length .

[jedel1043]

I think we just need to ensure we evaluate spec-related checks before rejecting indices bigger than usize (probably)

[jedel1043]

For good measure we should still test this on a 32-bit system

[zhuzhu81998]

I think we just need to ensure we evaluate spec-related checks before rejecting indices bigger than usize (probably)

well if we dont reject indices bigger than usize at the latest the OS would anyway?

[zhuzhu81998]

For good measure we should still test this on a 32-bit system

hmm dont have one. dont think github action runner has that either

[zhuzhu81998]

hmm dont have one. dont think github action runner has that either

cross-compiling does not work out of the box because of openssl-sys 🙄 .
But even without this PR, tests on 32-bit target fail:

test test::weak::miri::eph_self_referential_chain ... FAILED
test test::weak::miri::eph_self_referential ... FAILED
test test::allocation::miri::gc_recursion ... ok

failures:

---- test::weak::miri::eph_self_referential_chain stdout ----

thread '<unnamed>' (373283) panicked at core/gc/src/test/mod.rs:42:13:
assertion `left == right` failed
  left: 108
 right: 168
stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/std/src/panicking.rs:689:5
   1: core::panicking::panic_fmt
             at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/core/src/panicking.rs:80:14
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed
             at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/core/src/panicking.rs:394:5
   4: boa_gc::test::Harness::assert_exact_bytes_allocated::{{closure}}
             at ./src/test/mod.rs:42:13
   5: std::thread::local::LocalKey<T>::try_with
             at /usr/local/rustup/toolchains/1.93.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:513:12
   6: std::thread::local::LocalKey<T>::with
             at /usr/local/rustup/toolchains/1.93.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:477:20
   7: boa_gc::test::Harness::assert_exact_bytes_allocated
             at ./src/test/mod.rs:40:16
   8: boa_gc::test::weak::miri::eph_self_referential_chain::{{closure}}
             at ./src/test/weak.rs:215:17
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

thread 'test::weak::miri::eph_self_referential_chain' (373277) panicked at core/gc/src/test/weak.rs:186:9:
called `Result::unwrap()` on an `Err` value: Any { .. }
stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/std/src/panicking.rs:689:5
   1: core::panicking::panic_fmt
             at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/core/src/panicking.rs:80:14
   2: core::result::unwrap_failed
             at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/core/src/result.rs:1867:5
   3: core::result::Result<T,E>::unwrap
             at /usr/local/rustup/toolchains/1.93.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/result.rs:1233:23
   4: boa_gc::test::run_test
             at ./src/test/mod.rs:50:19
   5: boa_gc::test::weak::miri::eph_self_referential_chain
             at ./src/test/weak.rs:186:9
   6: boa_gc::test::weak::miri::eph_self_referential_chain::{{closure}}
             at ./src/test/weak.rs:181:36
   7: core::ops::function::FnOnce::call_once
             at /usr/local/rustup/toolchains/1.93.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
   8: core::ops::function::FnOnce::call_once
             at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

---- test::weak::miri::eph_self_referential stdout ----

thread '<unnamed>' (373275) panicked at core/gc/src/test/mod.rs:42:13:
assertion `left == right` failed
  left: 36
 right: 56
stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/std/src/panicking.rs:689:5
   1: core::panicking::panic_fmt
             at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/core/src/panicking.rs:80:14
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed
             at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/core/src/panicking.rs:394:5
   4: boa_gc::test::Harness::assert_exact_bytes_allocated::{{closure}}
             at ./src/test/mod.rs:42:13
   5: std::thread::local::LocalKey<T>::try_with
             at /usr/local/rustup/toolchains/1.93.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:513:12
   6: std::thread::local::LocalKey<T>::with
             at /usr/local/rustup/toolchains/1.93.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/thread/local.rs:477:20
   7: boa_gc::test::Harness::assert_exact_bytes_allocated
             at ./src/test/mod.rs:40:16
   8: boa_gc::test::weak::miri::eph_self_referential::{{closure}}
             at ./src/test/weak.rs:169:17
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

thread 'test::weak::miri::eph_self_referential' (373274) panicked at core/gc/src/test/weak.rs:153:9:
called `Result::unwrap()` on an `Err` value: Any { .. }
stack backtrace:
   0: __rustc::rust_begin_unwind
             at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/std/src/panicking.rs:689:5
   1: core::panicking::panic_fmt
             at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/core/src/panicking.rs:80:14
   2: core::result::unwrap_failed
             at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/core/src/result.rs:1867:5
   3: core::result::Result<T,E>::unwrap
             at /usr/local/rustup/toolchains/1.93.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/result.rs:1233:23
   4: boa_gc::test::run_test
             at ./src/test/mod.rs:50:19
   5: boa_gc::test::weak::miri::eph_self_referential
             at ./src/test/weak.rs:153:9
   6: boa_gc::test::weak::miri::eph_self_referential::{{closure}}
             at ./src/test/weak.rs:144:30
   7: core::ops::function::FnOnce::call_once
             at /usr/local/rustup/toolchains/1.93.0-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:250:5
   8: core::ops::function::FnOnce::call_once
             at /rustc/254b59607d4417e9dffbc307138ae5c86280fe4c/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.


failures:
    test::weak::miri::eph_self_referential
    test::weak::miri::eph_self_referential_chain

test result: FAILED. 19 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.16s

error: test failed, to rerun pass `-p boa_gc --lib`

[jedel1043]

Oof, let me see if I can fix those

[zhuzhu81998]

Oof, let me see if I can fix those

I think the problem is that on 32-bit targets (or anything that is not 64-bit), the exact bytes allocated will be different, because of the different pointer sizes, so this will fail 😂 :

boa/core/gc/src/test/weak.rs

Line 215 in 94360bf

Harness::assert_exact_bytes_allocated(168);

[jedel1043]

#5090 added CI for 32-bit targets, so it should be much easier to catch bugs on this now

[zhuzhu81998]

#5090 added CI for 32-bit targets, so it should be much easier to catch bugs on this now

very nice.

[hansl]

I can also test this on my ARMv7, or you can use QEMU to spin up an emulated one.

[jedel1043]

Ran the test262 suite using cross on the i686-unknown-linux-gnu target and I got this list of failing tests:

Broken tests (55):
test/RegExp/prototype/Symbol.replace/coerce-lastindex.js (previously Passed)
test/RegExp/prototype/exec/failure-lastindex-set.js (previously Passed)
test/ArrayBuffer/data-allocation-after-object-creation.js (previously Passed)
test/Array/prototype/includes/length-boundaries.js (previously Passed)
test/Array/prototype/unshift/length-near-integer-limit.js (previously Passed)
test/Array/prototype/unshift/clamps-to-integer-limit.js (previously Passed)
test/Array/prototype/unshift/throws-if-integer-limit-exceeded.js (previously Passed)
test/Array/prototype/copyWithin/length-near-integer-limit.js (previously Passed)
test/Array/prototype/push/S15.4.4.7_A4_T2.js (previously Passed)
test/Array/prototype/push/S15.4.4.7_A4_T1.js (previously Passed)
test/Array/prototype/push/length-near-integer-limit.js (previously Passed)
test/Array/prototype/push/S15.4.4.7_A3.js (previously Passed)
test/Array/prototype/push/S15.4.4.7_A2_T2.js (previously Passed)
test/Array/prototype/push/clamps-to-integer-limit.js (previously Passed)
test/Array/prototype/push/throws-if-integer-limit-exceeded.js (previously Passed)
test/Array/prototype/push/length-near-integer-limit-set-failure.js (previously Passed)
test/Array/prototype/fill/length-near-integer-limit.js (previously Passed)
test/Array/prototype/concat/arg-length-exceeding-integer-limit.js (previously Passed)
test/Array/prototype/concat/arg-length-near-integer-limit.js (previously Passed)
test/Array/prototype/indexOf/15.4.4.14-3-28.js (previously Passed)
test/Array/prototype/indexOf/length-near-integer-limit.js (previously Passed)
test/Array/prototype/indexOf/15.4.4.14-3-8.js (previously Passed)
test/Array/prototype/indexOf/15.4.4.14-3-14.js (previously Passed)
test/Array/prototype/indexOf/15.4.4.14-3-29.js (previously Passed)
test/Array/prototype/splice/length-exceeding-integer-limit-shrink-array.js (previously Passed)
test/Array/prototype/splice/length-and-deleteCount-exceeding-integer-limit.js (previously Passed)
test/Array/prototype/splice/create-species-length-exceeding-integer-limit.js (previously Passed)
test/Array/prototype/splice/length-near-integer-limit-grow-array.js (previously Passed)
test/Array/prototype/splice/clamps-length-to-integer-limit.js (previously Passed)
test/Array/prototype/splice/throws-if-integer-limit-exceeded.js (previously Passed)
test/Array/prototype/splice/S15.4.4.12_A3_T1.js (previously Passed)
test/Array/prototype/reduceRight/length-near-integer-limit.js (previously Passed)
test/Array/prototype/findLast/maximum-index.js (previously Passed)
test/Array/prototype/every/15.4.4.16-3-14.js (previously Passed)
test/Array/prototype/every/15.4.4.16-3-29.js (previously Passed)
test/Array/prototype/every/15.4.4.16-3-8.js (previously Passed)
test/Array/prototype/findLastIndex/maximum-index.js (previously Passed)
test/Array/prototype/pop/S15.4.4.6_A3_T2.js (previously Passed)
test/Array/prototype/pop/length-near-integer-limit.js (previously Passed)
test/Array/prototype/pop/S15.4.4.6_A2_T2.js (previously Passed)
test/Array/prototype/pop/S15.4.4.6_A3_T1.js (previously Passed)
test/Array/prototype/pop/clamps-to-integer-limit.js (previously Passed)
test/Array/prototype/reverse/length-exceeding-integer-limit-with-proxy.js (previously Passed)
test/Array/prototype/reverse/length-exceeding-integer-limit-with-object.js (previously Passed)
test/Array/prototype/some/15.4.4.17-3-8.js (previously Passed)
test/Array/prototype/some/15.4.4.17-3-29.js (previously Passed)
test/Array/prototype/some/15.4.4.17-3-14.js (previously Passed)
test/Array/prototype/some/15.4.4.17-3-28.js (previously Passed)
test/Array/prototype/lastIndexOf/length-near-integer-limit.js (previously Passed)
test/Array/prototype/lastIndexOf/15.4.4.15-3-28.js (previously Passed)
test/Array/prototype/toSpliced/length-clamped-to-2pow53minus1.js (previously Passed)
test/Array/prototype/toSpliced/length-exceeding-array-length-limit.js (previously Passed)
test/Array/prototype/slice/length-exceeding-integer-limit-proxied-array.js (previously Passed)
test/Array/prototype/slice/length-exceeding-integer-limit.js (previously Passed)
test/SharedArrayBuffer/data-allocation-after-object-creation.js (previously Passed)

[zhuzhu81998]

thanks, will have a look later.

[jedel1043]

Btw if you install cross it is kinda easy to run individual tests to see if they work on 32-bit machines:

cross run --target i686-unknown-linux-gnu --bin boa_tester -- run -s test/Array/prototype/some/15.4.4.17-3-28.js

Just don't run the full suite because it consistently OOM'd for me, too much memory usage from all the thousands of tests running haha

[zhuzhu81998]

`/workspaces/boa/test262/test/built-ins/Array/prototype/concat/arg-length-exceeding-integer-limit.js`: starting
`/workspaces/boa/test262/test/built-ins/Array/prototype/concat/arg-length-exceeding-integer-limit.js`: Failed
`/workspaces/boa/test262/test/built-ins/Array/prototype/concat/arg-length-exceeding-integer-limit.js`: result text
Uncaught Test262Error {
    message: "[1].concat(spreadableLengthOutOfRange) throws a TypeError exception Expected a TypeError but got a RangeError"
}
    at <anonymous> (/workspaces/boa/test262/harness/assert.js:95:13)
    at <main> (/workspaces/boa/test262/test/built-ins/Array/prototype/concat/arg-length-exceeding-integer-limit.js:26:14)

sigh....

[zhuzhu81998]

I had to take another look at the spec because of these failing tests.

The problem seems to be that since "array" can also just be a normal object (in which case index is technically speaking not index, but rather a normal PropertyKey which could be an index if it is inside u32 or it would be a string instead), thus it is expected that an engine accepts something as an "index" even though the platform can not technically handle it....

I wish I never touched this... But it seems that the main branch implementation is, spec-wise, strictly speaking, largely correct with the using u64 everywhere and "unsafely" converting it to usize. The test suite does not test for the platform-specific edge cases and the spec says nothing about implementation details anyway.

There are 1 or 2 small issues with e.g. byte_offset which returns a usize though. We should probably just use u64 as much as possible. Type casting like might not be free, but should be efficient enough such that it does not matter.

Btw, technically spec says "suitable", but does not consider platform at all (not even a little bit).

So unless there is something else, I think we can close this PR, and I will open another one which adjusts fewer things to ensure the u64 principle.