ElligatorSwift + integrated x-only DH

[sipa]

Builds on top of #979, #1118. Replaces #982.

This implements encoding of curve points using the ElligatorSwift algorithm, using 4 new API calls:

• secp256k1_ellswift_encode, which converts a public key to a 64-byte pseudorandom encoding.
• secp256k1_ellswift_decode, the reverse operation to convert back to normal public keys.
• secp256k1_ellswift_create, which can be seen as a combination of secp256k1_ec_pubkey_create + secp256k1_ellswift_encode, but is somewhat safer.
• secp256k1_ellswift_xdh, which implements x-only Diffie-Hellman directly on top of 64-byte encoded public keys, and more efficiently than decoding + invoking normal ECDH.

The scheme matches that of the SwiftEC paper (https://eprint.iacr.org/2022/759), with two changes (remapping undefined inputs, and encoding the Y parity in the u/t values themselves rather than in a separate bit). To decode an ElligatorSwift 64-byte encoded public key:

• Interpret the encoding as two field elements $(u, t)$ in big-endian 32-byte encoding each, reducing modulo $p$.
• If $u=0$, let $u=1$ instead.
• If $t=0$, let $t=1$ instead.
• If $u^3 + 7 + t^2 = 0$, let $t = 2t$ instead.
• Let $X = \dfrac{u^3 + 7 - t^2}{2t}$.
• Let $Y = \dfrac{X + t}{\sqrt{-3}u}$.
• Let $x$ be the first of $(u + 4Y^2, \dfrac{-X}{2Y} - \dfrac{u}{2}, \dfrac{X}{2Y} - \dfrac{u}{2})$ which is on the curve (at least one will be).
• Let $y$ be the corresponding Y coordinate, with the same parity as (the original) $t$.
• Return $(x,y)$.

This is significantly faster than the Elligator Squared code in #982.

Relevant benchmark (AMD Ryzen 5950X, GCC 12.2.0, default config options; frequency fixed at 2.80 GHz):

Benchmark                     ,    Min(us)    ,    Avg(us)    ,    Max(us)    

ec_keygen                     ,    27.9       ,    28.0       ,    28.0    
ecdh                          ,    53.9       ,    53.9       ,    54.0    
ellswift_encode               ,    24.5       ,    24.5       ,    24.6    
ellswift_decode               ,    11.1       ,    11.1       ,    11.1    
ellswift_keygen               ,    52.5       ,    52.5       ,    52.6    
ellswift_ecdh                 ,    57.9       ,    58.0       ,    58.0    

[real-or-random]

(just skimming)

Do you think there's a nice way to avoid code duplication between divsteps and posdivsteps?

[sipa]

@real-or-random Re posdivsteps see #979 which this PR is based on.

[paulmillr]

Looks awesome! Wanted to clarify this bit, for implementors of EllSwift in other languages / for standardization purpose.

Inputs (u,t) where u=0, t=0, or u^2+t^3+7=0, are remapped to other finite points, rather than outputting infinity.

To which points exactly? It's not immediately obvious from the code.

[sipa]

@paulmillr They are remapped as follows:

• If $t=0$, set $t=1$ instead
• If $u=0$, set $u=1$ instead
• If $u^2+t^3+B=0$, set $t=2t$ instead
• Run the normal algorithm

(for applicable odd-ordered curves with A=0, this covers all otherwise unmapped points, but this remapping doesn't work for every ellswift-compatible curve).

[sipa]

Added test vectors.

[sipa]

I've made a number of improvements:

• Addressed comments so far
• Added test vectors that are verified against the paper author's Sage code (https://github.com/Jchavezsaab/SwiftEC).
• Documented the algorithms line-by-line
• Renamed functions to better match the paper's names.
• Split up the commits
• A few small performance optimizations
• Factored out helper functions for checking if an X coordinate is on the curve (directly, and when given as a fraction).

I think it's ready for more review.

[sipa]

Rebased on updated #979.

[sipa]

Added an explanation of the algorithm and its relation to the paper, in doc/ellswift.md. I've also changed some of the code to better match the algorithm as described in that document (I found some simplifications while writing it).

[Davidson-Souza]

tACK 90e360a. Full testing backlog:

• Iterated through the implementation, checking all the math (the verbose comments helped a lot!)
• Indepently generated all constants using python/sage (Python 3.10.10 and SageMath version 9.8, Release Date: 2023-02-11)
• A "manual mutation test" for the valgrind ctime tests, adding some dummy, secret dependent branching, it indeed causes memcheck to complain about uninitialized values. Using VALGRIND_MAKE_MEM_DEFINED silences those warnings, as expected.
• Executed all tests, including the memcheck ones (OS and hardware below). Compilation using both gcc and clang, configure script bellow.
• The documentation is sound and very helpful

Test environments:

Config script: [CC=clang] ./configure --with-valgrind=yes --enable-module-ecdh --enable-module-ellswift --enable-experimental --no-recursion

Machine 1: Arch Linux on a AMD Ryze 5 (Linux 6.1.21-hardened1-1-hardened #1 SMP PREEMPT_DYNAMIC x86_64 GNU/Linux )
gcc version 12.2.1 20230201
clang version 14.0.7
GNU libc stable release version 2.37.
valgrind-3.20.0

Machine 2: Raspbian on a Raspbery Pi 4b (Linux raspberrypi 6.1.21-v8+ #1642 SMP PREEMPT aarch64 GNU/Linux)
gcc version 10.2.1 20210110 (Debian 10.2.1-6)
Debian clang version 11.0.1-2
valgrind-3.16.1
(Debian GLIBC 2.31-13+rpt2+rpi1+deb11u5) stable release version 2.31

Benchmarks (only for the new module)

Machine 1, clang

ellswift_encode               ,    23.8       ,    24.1       ,    24.9    
ellswift_decode               ,    10.9       ,    11.4       ,    11.6    
ellswift_keygen               ,    55.3       ,    58.8       ,    61.7    
ellswift_ecdh                 ,    63.3       ,    64.1       ,    65.3 

Machine 1, gcc

ellswift_encode               ,    22.6       ,    22.7       ,    22.9    
ellswift_decode               ,    10.7       ,    10.8       ,    11.0    
ellswift_keygen               ,    54.3       ,    54.7       ,    55.9    
ellswift_ecdh                 ,    62.7       ,    63.2       ,    64.1

Machine 2, clang

ellswift_encode               ,    63.7       ,    63.7       ,    63.8    
ellswift_decode               ,    33.7       ,    33.8       ,    34.0    
ellswift_keygen               ,   177.0       ,   177.0       ,   178.0    
ellswift_ecdh                 ,   233.0       ,   234.0       ,   234.0

Machine 2, gcc

ellswift_encode               ,    66.4       ,    66.4       ,    66.5    
ellswift_decode               ,    34.9       ,    34.9       ,    34.9    
ellswift_keygen               ,   182.0       ,   182.0       ,   183.0    
ellswift_ecdh                 ,   230.0       ,   231.0       ,   231.0  

[real-or-random]

ACK 90e360a

[jonasnick]

ACK 90e360a

[jonasnick]

This has been merged as it received adequate reviews and enough ACKs (as discussed in IRC). Post-merge ACKs are still welcome. If any issues still arise, they can be addressed in follow-up PRs.

[stratospher]

Suggested change

	* Returns: 1: shared secret was succesfully computed
	* Returns: 1: shared secret was successfully computed

[stratospher]

swiftEC paper shows this:
$\dfrac{g(u) + h(u)(Y_0(u) - X_0(u)t)^2}{X_0(u)(1 + h(u)t^2)}$

[stratospher]

tried using WolframAlpha(my first time) for this condition - link.

EDIT: sorry for the spam! got it when i plugged in the right equation.

[stratospher]

(from L332 actually) how does this happen? (when Y = 0 in x1, x2 computation or something else?)

EDIT: L332 needs to be updated to $w(u+2v) = 2X_0(u)$. (this happens when X = X0)

[stratospher]

why not $sign(w)$?

[stratospher]

post-merge ACK 90e360a.
rereviewed the code and skimmed through the swiftEC paper.