Skip to content

fix: invmod returning a value outside the range of a negative modulus - #3710

Open
kevin9327 wants to merge 1 commit into
josdejong:developfrom
kevin9327:fix-invmod-negative-modulus
Open

kevin9327 wants to merge 1 commit into
josdejong:developfrom
kevin9327:fix-invmod-negative-modulus

Conversation

@kevin9327

Copy link
Copy Markdown

invmod gives a wrong-range result when the modulus is negative:

math.invmod(3, -11) // -18, expected -7
math.invmod(2, -7)  // -10, expected -3

mod() already returns a value with the sign of b, so after inv = mod(inv, b) the result lies between 0 and b. The extra if (inv < 0) inv = add(inv, b) step can only fire for a negative b, and then it moves the value out of that range. (-18 is still congruent, but it's not the reduced inverse, and it doesn't match what mod returns.) For a positive b the branch never runs, so removing it changes nothing there.

The new results match Python's pow(a, -1, m), e.g. pow(3, -1, -11) == -7.

This PR removes that step and the smaller/add dependencies that are no longer used, and adds tests for a negative modulus with numbers and BigNumbers. npx mocha test/unit-tests/function/arithmetic/ passes and eslint is clean.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant