items, pairs and friends now use unCheckedInc

[ringabout]

{.push overflowChecks: off.} works in backends. Though it could be implemented as a magic function.

By inspecting the generated C code, the overflow check is eliminated in the debug or release mode.

Likewise, the index checking is probably not needed.

[arnetheduck]

we've had problems with push spilling over to unintended scopes due to codegen bugs - I'm not sure if the bugs are reported and fixed but it would be good to test that overflow checking works in surrounding code before, after and in generic contexts, as well as inside the expanded for loop when using items, ie in the body of the calling code..

[ringabout]

Well, I have created the https://github.com/nim-lang/Nim/blob/devel/compiler/backendpragmas.nim module to handle push options in the backend. I suppose it improves a lot than before. Better safe than sorry. I will open an alternative PR, which would implement the feature as a magic/codegen function.

[arnetheduck]

. I will open an alternative PR, which implements the feature as a magic/codegen function.

TBH what I'd really love to see in terms of overflow support etc would be that it was all implemented in nim instead of living in the backends / ast.. basically:

proc `+`(a, b: int) = checkedAdd(a, b)

proc checkedAdd(a, b: int) =
  when overflowEnabled:
    if addWithOverflow(a, b, result):
      raise overflowerror...
  else:
    addWithoutOverflow(a, b, result)

Notably, in such a world, all defects raised would be part of the nim code in the std lib with without the C/other backends doing any of the work.

This would have the advantage that the low-level emitted functions would be available for libraries to reuse and it would be possible to "emulate" the compile-time options in libraries with perfect compatibility - consider https://github.com/status-im/nim-stint/ - it gives inxXX support and should behave 1:1 like the native builtin integers, including its respect of compile-time flags and overflow checks etc - with "normal" intXX being just a thin front for low-level implementations.

As is, we have to write these utilities in libraries (status-im/nim-stew#187) but it's difficult to maintain 1:1 compatibility because of the special treatment the compiler gives.

[arnetheduck]

The above would of course require some more work on the way when is evaluated (when enabledInScope(overflowchecks)) but over-all, the less of this stuff that lives in codegen, the better imo

[Varriount]

@ringabout Love the additional organization. 👍

[metagn]

work on the way when is evaluated (when enabledInScope(overflowchecks))

We would need to generate different procs in codegen based on each when.

The case in this PR is just an optimization, we shouldn't need something more rigorous than push/pop.

[arnetheduck]

btw, have you looked at the assembly code of the optimised iteration code? Even though this PR looks nice from a C generation perspective, it might be one of the things that the C compiler can remove (or not - I don't remember).

My recollection when investigating this was that iteration performance is bad for a number reasons:

• length in range check is compared against the length in the seq which might change during iteration, preventing hoisting the check outside of the loop and / or removing it
• unclear pointer aliasing exacerbating the above point
• exception raising in general prevents vectorization since the exception must be raised at the correct iteration, so we can't for example copy 4 bytes at a time

In all this, I do not remember which checks the compiler is able to remove - ie some of them can be removed by giving it a hint about the values involved via an explicit assertion before iteration begins which removes the data dependency on the loop variable for exception raising.

The situation also changes when a constant length is used vs dynamic.

Relevant issue: #20232

[Araq]

btw, have you looked at the assembly code of the optimised iteration code?

Does it matter? The C compiler might regress as integer overflow ops are not commonly used in C.

As for your other points: We should address these issues somehow. Hopefully without having to resort to pointer arithmetic.

[github-actions]

Thanks for your hard work on this PR!
The lines below are statistics of the Nim compiler built from d82bc0a

Hint: mm: orc; opt: speed; options: -d:release
169924 lines; 9.042s; 620.496MiB peakmem