ORE Operators #83

tobyhede · 2025-01-22T22:57:12Z

Add ORE operators for the cs_encrypted_v1 type
Also generalises equality operators to support both unique and ore indexes.

Addresses:
https://linear.app/cipherstash/issue/CIP-1148/implement-ore-operators-for-cs-encrypted-v1
https://linear.app/cipherstash/issue/CIP-1149/equality-operator-needs-to-handle-both-unique-and-ore-index-types

coderdan · 2025-01-23T22:54:37Z

sql/015-operators-eq.sql

+    o boolean;
+  BEGIN
+    BEGIN
+      u := (SELECT cs_unique_v1(a) = cs_unique_v1(b));


I'm not 100% what the threat model might be but its worth noting that this comparison won't run in constant time. It might leave the possibility for an attacker to learn the value of the plaintext in certain circumstances.

There's a constant time comparison function in the EQL repo somewhere I think.

Something like this:

encrypt-query-language/sql/001-ore-cllw.sql

Line 21 in 2ee50af

CREATE FUNCTION __bytea_ct_eq(a bytea, b bytea) RETURNS boolean AS $$

cs_unique_v1 is the existing function, and simply extracts the index value from the eql::ciphertext

Unique indexes are cast as text, so this ends up using the default text eq operators.

Yeah that's what I figured. The standard equality operator will bail as soon as a character differs which can lead to timing side-channel attacks. A constant time comparison should always take the same amount of time regardless of which character index differs first.

Oh right. We should fix that for unique in general then

coderdan · 2025-01-23T22:54:45Z

sql/015-operators-eq.sql

+    END;
+
+    BEGIN
+      o := (SELECT cs_ore_64_8_v1(a) = cs_ore_64_8_v1(b));


Same for this one.

cs_ore_64_8_v1 is same as unique, pulls the field from the json
The returned data is cast as ore_64_8_v1 and then the underlying ore operators are used for the comparison

coderdan · 2025-01-23T23:02:42Z

sql/666-drop-operators.sql

+DROP OPERATOR IF EXISTS @> (cs_encrypted_v1, cs_encrypted_v1);
+
+DROP OPERATOR IF EXISTS @> (cs_encrypted_v1, cs_match_index_v1);
+DROP OPERATOR IF EXISTS @> (cs_match_index_v1, cs_encrypted_v1);
+
+DROP OPERATOR IF EXISTS <@ (cs_encrypted_v1, cs_encrypted_v1);
+
+DROP OPERATOR IF EXISTS <@ (cs_encrypted_v1, cs_match_index_v1);
+DROP OPERATOR IF EXISTS <@ (cs_match_index_v1, cs_encrypted_v1);
+
+
+DROP OPERATOR IF EXISTS <= (ore_64_8_v1, cs_encrypted_v1);
+DROP OPERATOR IF EXISTS <= (cs_encrypted_v1, ore_64_8_v1);
+DROP OPERATOR IF EXISTS <= (jsonb, cs_encrypted_v1);
+DROP OPERATOR IF EXISTS <= (cs_encrypted_v1, jsonb);
+DROP OPERATOR IF EXISTS <= (cs_encrypted_v1, cs_encrypted_v1);
+
+DROP OPERATOR IF EXISTS >=(ore_64_8_v1, cs_encrypted_v1);
+DROP OPERATOR IF EXISTS >= (jsonb, cs_encrypted_v1);
+DROP OPERATOR IF EXISTS >= (cs_encrypted_v1, ore_64_8_v1);
+DROP OPERATOR IF EXISTS >= (cs_encrypted_v1, jsonb);
+DROP OPERATOR IF EXISTS >=(cs_encrypted_v1, cs_encrypted_v1);
+
+DROP OPERATOR IF EXISTS <(ore_64_8_v1, cs_encrypted_v1);
+DROP OPERATOR IF EXISTS <(cs_encrypted_v1, ore_64_8_v1);
+DROP OPERATOR IF EXISTS < (jsonb, cs_encrypted_v1);
+DROP OPERATOR IF EXISTS < (cs_encrypted_v1, jsonb);
+DROP OPERATOR IF EXISTS < (cs_encrypted_v1, cs_encrypted_v1);
+
+DROP OPERATOR IF EXISTS > (ore_64_8_v1, cs_encrypted_v1);
+DROP OPERATOR IF EXISTS > (jsonb, cs_encrypted_v1);
+DROP OPERATOR IF EXISTS > (cs_encrypted_v1, ore_64_8_v1);
+DROP OPERATOR IF EXISTS > (cs_encrypted_v1, jsonb);
+DROP OPERATOR IF EXISTS > (cs_encrypted_v1, cs_encrypted_v1);
+
+
+DROP OPERATOR IF EXISTS = (cs_encrypted_v1, cs_encrypted_v1);
+
+DROP OPERATOR IF EXISTS = (cs_encrypted_v1, jsonb);
+DROP OPERATOR IF EXISTS = (jsonb, cs_encrypted_v1);
+
+DROP OPERATOR IF EXISTS = (cs_encrypted_v1, cs_unique_index_v1);
+DROP OPERATOR IF EXISTS = (cs_unique_index_v1, cs_encrypted_v1);
+
+DROP OPERATOR IF EXISTS = (cs_encrypted_v1, ore_64_8_v1);
+DROP OPERATOR IF EXISTS = (ore_64_8_v1, cs_encrypted_v1);
+
+DROP OPERATOR IF EXISTS <> (cs_encrypted_v1, cs_encrypted_v1);
+
+DROP OPERATOR IF EXISTS <> (cs_encrypted_v1, jsonb);
+DROP OPERATOR IF EXISTS <> (jsonb, cs_encrypted_v1);
+
+DROP OPERATOR IF EXISTS <> (cs_encrypted_v1, cs_unique_index_v1);
+DROP OPERATOR IF EXISTS <> (cs_unique_index_v1, cs_encrypted_v1);
+
+DROP OPERATOR IF EXISTS <> (ore_64_8_v1, cs_encrypted_v1);
+DROP OPERATOR IF EXISTS <> (cs_encrypted_v1, ore_64_8_v1);


Note: I haven't checked that you've captured all the operators here. I'm assuming running the script will show if that's not the case!

coderdan · 2025-01-23T23:05:52Z

tests/operators-match.sql

@@ -0,0 +1,90 @@
+\set ON_ERROR_STOP on


coderdan · 2025-01-23T23:07:36Z

tests/operators-match.sql

+-- MATCH <@ OPERATORS
+DO $$
+  BEGIN
+    -- SANITY CHECK FOR UNIQUE payloads


I don't understand the comment. Is that a copy-paste error or am I missing something about what this test does?

coderdan · 2025-01-23T23:10:27Z

tests/operators-ore.sql

+    ASSERT (SELECT EXISTS (
+      SELECT id FROM users WHERE name_encrypted < ore_cs_encrypted::jsonb
+    ));


These tests are fine but in my experience ORE tests with hard-coded values can give false confidence. Because an incorrect implementation will still "work" ~50% of the time. Not for this PR but we may want to think about how to generate random ORE encrypted values for every test (like a property test) to validate correctness more robustly.

coderdan

EPIC PR. Nice work.

A couple of suggestions for future improvements.

tobyhede added 2 commits January 22, 2025 20:44

ore operators

1bcce1f

Genealise equality operators

2e5aa78

tobyhede changed the title ~~I don't want to talk about it~~ ORE Operators Jan 23, 2025

tobyhede force-pushed the CIP-1148/i-dont-want-to-talk-about-it branch 2 times, most recently from da620ec to bfa139f Compare January 23, 2025 05:59

eq operators with tests

7442ae8

tobyhede force-pushed the CIP-1148/i-dont-want-to-talk-about-it branch from bfa139f to 7442ae8 Compare January 23, 2025 06:04

coderdan reviewed Jan 23, 2025

View reviewed changes

tests/operators-match.sql

@@ -0,0 +1,90 @@

\set ON_ERROR_STOP on

Copy link

Contributor

coderdan Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL.

coderdan reviewed Jan 23, 2025

View reviewed changes

cleanup comments

deab454

coderdan reviewed Jan 23, 2025

View reviewed changes

coderdan approved these changes Jan 23, 2025

View reviewed changes

tobyhede merged commit f919fb3 into main Jan 24, 2025
4 checks passed

tobyhede deleted the CIP-1148/i-dont-want-to-talk-about-it branch January 24, 2025 05:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ORE Operators #83

ORE Operators #83

Uh oh!

tobyhede commented Jan 22, 2025 •

edited

Loading

Uh oh!

coderdan Jan 23, 2025

Uh oh!

coderdan Jan 23, 2025

Uh oh!

tobyhede Jan 23, 2025

Uh oh!

tobyhede Jan 23, 2025

Uh oh!

coderdan Jan 23, 2025

Uh oh!

tobyhede Jan 23, 2025

Uh oh!

coderdan Jan 23, 2025

Uh oh!

tobyhede Jan 23, 2025

Uh oh!

coderdan Jan 23, 2025

Uh oh!

coderdan Jan 23, 2025

Uh oh!

coderdan Jan 23, 2025

Uh oh!

coderdan Jan 23, 2025

Uh oh!

coderdan left a comment

Uh oh!

Uh oh!

Uh oh!

ORE Operators #83

ORE Operators #83

Uh oh!

Conversation

tobyhede commented Jan 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

coderdan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tobyhede commented Jan 22, 2025 •

edited

Loading