Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fast modular inverse - 9.4x acceleration #83

Merged
merged 2 commits into from
Oct 19, 2023
Merged

Fast modular inverse - 9.4x acceleration #83

merged 2 commits into from
Oct 19, 2023

Conversation

mratsim
Copy link
Contributor

@mratsim mratsim commented Aug 29, 2023 •
edited
Loading

Upstreaming on behalf of @AlekseiVambol - original: taikoxyz#2

This closes privacy-scaling-explorations/halo2#28 and implement modular inversion according to Bernstein-Yang paper, as detailed in (#28 (comment))

It also addresses:

Algorithm

Bernstein-Yang inversion:

The explainer from Bitcoin's team is foundational to understand the implementation.

Note: there is another recent implementation as well in MCL (2 weeks ago): herumi/mcl@cf16bf3

The original PR for internal review: taikoxyz#2

Performance

Performance is about 9.4x faster than previous.

image

Future direction

We are planning to accelerate Legendre symbol by the same order of magnitude as this is apparently a bottleneck for hash-to-curve:

@han0110 han0110 self-requested a review August 29, 2023 07:23
@davidnevadoc davidnevadoc self-requested a review August 29, 2023 14:34
@mratsim mratsim mentioned this pull request Aug 30, 2023
Closed
han0110
han0110 approved these changes Sep 5, 2023
View reviewed changes
Copy link
Contributor

@han0110 han0110 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! The CInt abstraction looks nice and all algorithms are clear and well documented.

Small note to self and perhaps other reviewers: I did the review with https://github.com/bitcoin-core/secp256k1/blob/0775283/doc/safegcd_implementation.md as reference (pretty clear explaination), and the only difference are the bits canceled per step in jump is 5 without using table, and using delta instead of eta.

@mratsim mratsim mentioned this pull request Oct 13, 2023
Merged
@mratsim
Copy link
Contributor Author

mratsim commented Oct 13, 2023

Rebased against latest master.

Also bernsteinyang.rs has been renamed ff_inverse.rs so it's easier to understand what it does (instead of how). Similarly in privacy-scaling-explorations/halo2#95, the algorithm will be in a file ff_jacobi.

davidnevadoc
davidnevadoc approved these changes Oct 18, 2023
View reviewed changes
Copy link
Contributor

@davidnevadoc davidnevadoc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for all the clear explanations and extra references :)

@han0110 han0110 added this pull request to the merge queue Oct 19, 2023
Merged via the queue into privacy-scaling-explorations:main with commit a835096 Oct 19, 2023
@mratsim mratsim deleted the pse-pr-invmod branch October 19, 2023 10:06
@huitseeker huitseeker mentioned this pull request Oct 23, 2023
Closed
mratsim added a commit to taikoxyz/halo2curves that referenced this pull request Oct 25, 2023
@mratsim @AlekseiVambol
Fast modular inverse - 9.4x acceleration (privacy-scaling-exploration…
93f4f3b 
…s#83)

* Bernstein yang modular multiplicative inverter (#2)

* rename similar to privacy-scaling-explorations#95

---------

Co-authored-by: Aleksei Vambol <77882392+AlekseiVambol@users.noreply.github.com>
@mratsim mratsim mentioned this pull request Oct 25, 2023
Closed
3 tasks
jonathanpwang added a commit to axiom-crypto/halo2curves that referenced this pull request Nov 13, 2023
@jonathanpwang @davidnevadoc
@han0110 @huitseeker @CPerezz @einar-taiko @mratsim @AlekseiVambol
Sync with PSE upstream (#9)
cf6dfa4 
* Add field conversion to/from `[u64;4]` (privacy-scaling-explorations#80)

* feat: add field conversion to/from `[u64;4]`

* Added conversion tests
* Added `montgomery_reduce_short` for no-asm
* For bn256, uses assembly conversion when asm feature is on

* fix: remove conflict for asm

* chore: bump rust-toolchain to 1.67.0

* Compute Legendre symbol for `hash_to_curve` (privacy-scaling-explorations#77)

* Add `Legendre` trait and macro

 - Add Legendre macro with norm and legendre symbol computation
 - Add macro for automatic implementation in prime fields

* Add legendre macro call for prime fields

* Remove unused imports

* Remove leftover

* Add `is_quadratic_non_residue` for hash_to_curve

* Add `legendre` function

* Compute modulus separately

* Substitute division for shift

* Update modulus computation

* Add quadratic residue check func

* Add quadratic residue tests

* Add hash_to_curve bench

* Implement Legendre trait for all curves

* Move misplaced comment

* Add all curves to hash bench

* fix: add suggestion for legendre_exp

* fix: imports after rebase

* Add simplified SWU method (privacy-scaling-explorations#81)

* Fix broken link

* Add simple SWU algorithm

* Add simplified SWU hash_to_curve for secp256r1

* add: sswu z reference

* update MAP_ID identifier

Co-authored-by: Han <tinghan0110@gmail.com>

---------

Co-authored-by: Han <tinghan0110@gmail.com>

* Bring back curve algorithms for `a = 0` (privacy-scaling-explorations#82)

* refactor: bring back curve algorithms for `a = 0`

* fix: clippy warning

* fix: Improve serialization for prime fields (privacy-scaling-explorations#85)

* fix: Improve serialization for prime fields

Summary: 256-bit field serialization is currently 4x u64, ie. the native format. This implements the standard of byte-serialization (corresponding to the PrimeField::{to,from}_repr), and an hex-encoded variant of
that for (de)serializers that are human-readable (concretely, json).

- Added a new macro `serialize_deserialize_32_byte_primefield!` for custom serialization and deserialization of 32-byte prime field in different struct (Fq, Fp, Fr) across the secp256r, bn256, and derive libraries.
- Implemented the new macro for serialization and deserialization in various structs, replacing the previous `serde::{Deserialize, Serialize}` direct use.
- Enhanced error checking in the custom serialization methods to ensure valid field elements.
- Updated the test function in the tests/field.rs file to include JSON serialization and deserialization tests for object integrity checking.

* fixup! fix: Improve serialization for prime fields

---------

Co-authored-by: Carlos Pérez <37264926+CPerezz@users.noreply.github.com>

* refactor: (De)Serialization of points using `GroupEncoding` (privacy-scaling-explorations#88)

* refactor: implement (De)Serialization of points using the `GroupEncoding` trait

- Updated curve point (de)serialization logic from the internal representation to the
  representation offered by the implementation of the `GroupEncoding` trait.

* fix: add explicit json serde tests

* Insert MSM and FFT code and their benchmarks. (privacy-scaling-explorations#86)

* Insert MSM and FFT code and their benchmarks.

Resolves taikoxyz/zkevm-circuits#150.

* feedback

* Add instructions

* feeback

* Implement feedback:  Actually supply the correct arguments to `best_multiexp`.

Split into `singlecore` and `multicore` benchmarks so Criterion's result
caching and comparison over multiple runs makes sense.

Rewrite point and scalar generation.

* Use slicing and parallelism to to decrease running time.

Laptop measurements:
k=22: 109 sec
k=16:   1 sec

* Refactor msm

* Refactor fft

* Update module comments

* Fix formatting

* Implement suggestion for fixing CI

* Re-export also mod `pairing` and remove flag `reexport` to alwasy re-export (privacy-scaling-explorations#93)

fix: re-export also mod `pairing` and remove flag `reexport` to alwasy re-export

* fix regression in privacy-scaling-explorations#93 reexport field benches aren't run (privacy-scaling-explorations#94)

fix regression in privacy-scaling-explorations#93, field benches aren't run

* Fast modular inverse - 9.4x acceleration (privacy-scaling-explorations#83)

* Bernstein yang modular multiplicative inverter (#2)

* rename similar to privacy-scaling-explorations#95

---------

Co-authored-by: Aleksei Vambol <77882392+AlekseiVambol@users.noreply.github.com>

* Fast isSquare / Legendre symbol / Jacobi symbol - 16.8x acceleration (privacy-scaling-explorations#95)

* Derivatives of the Pornin's method (taikoxyz#3)

* renaming file

* make cargo fmt happy

* clarifications from privacy-scaling-explorations#95 (comment) [skip ci]

* Formatting and slightly changing a comment

---------

Co-authored-by: Aleksei Vambol <77882392+AlekseiVambol@users.noreply.github.com>

* chore: delete bernsteinyang module (replaced by ff_inverse)

* Bump version to 0.4.1

---------

Co-authored-by: David Nevado <davidnevadoc@users.noreply.github.com>
Co-authored-by: Han <tinghan0110@gmail.com>
Co-authored-by: François Garillot <4142+huitseeker@users.noreply.github.com>
Co-authored-by: Carlos Pérez <37264926+CPerezz@users.noreply.github.com>
Co-authored-by: einar-taiko <126954546+einar-taiko@users.noreply.github.com>
Co-authored-by: Mamy Ratsimbazafy <mamy_github@numforge.co>
Co-authored-by: Aleksei Vambol <77882392+AlekseiVambol@users.noreply.github.com>
tarcieri added a commit to RustCrypto/crypto-bigint that referenced this pull request Dec 2, 2023
@tarcieri
[WIP] Bernstein-Yang modular inversion algorithm
ff2bb22 
Imported from: privacy-scaling-explorations/halo2curves#83

Original code is Apache 2.0+MIT. Attribution has been added to the top
of the module.
@tarcieri tarcieri mentioned this pull request Dec 2, 2023
Merged
tarcieri added a commit to RustCrypto/crypto-bigint that referenced this pull request Dec 3, 2023
@tarcieri
Bernstein-Yang modular inversion algorithm (#372)
7f93018 
Adapted from: privacy-scaling-explorations/halo2curves#83

Original code is Apache 2.0+MIT. Attribution has been added to the top
of the module.
@CPerezz CPerezz mentioned this pull request Dec 18, 2023
Merged
Velaciela pushed a commit to scroll-tech/halo2curves that referenced this pull request Aug 14, 2024
@mratsim @AlekseiVambol @Velaciela
Fast modular inverse - 9.4x acceleration (privacy-scaling-exploration…
6ae1ddc 
…s#83)

* Bernstein yang modular multiplicative inverter (#2)

* rename similar to privacy-scaling-explorations#95

---------

Co-authored-by: Aleksei Vambol <77882392+AlekseiVambol@users.noreply.github.com>
@Velaciela Velaciela mentioned this pull request Aug 14, 2024
Merged
sebastiencs added a commit to openmina/algebra that referenced this pull request Jan 14, 2025
@sebastiencs
Make field inversion faster: Use Bernstein-Yang algorithm
aea157a 
Based on:
- privacy-scaling-explorations/halo2curves#83
- https://github.com/RustCrypto/crypto-bigint/blob/682f17a979c3a1886fde5426b26400dfc3f4775b/src/modular/safegcd.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@han0110 han0110 han0110 approved these changes

@davidnevadoc davidnevadoc davidnevadoc approved these changes

@huitseeker huitseeker huitseeker approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@mratsim @huitseeker @davidnevadoc @han0110 @AlekseiVambol