Skip to content

ORE Operators and Sorting #86

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Feb 19, 2025
Merged

ORE Operators and Sorting #86

merged 15 commits into from
Feb 19, 2025

Conversation

tobyhede
Copy link
Contributor

@tobyhede tobyhede commented Jan 30, 2025

Make ORDER by work for cs_encrypted_v1 by encoding in the EQL json as an array rather than the PostgreSQL Ore Types.

Example ORE encoding

'{"o": ["121212121212594be28282d03415e7714fccd69eb7eb476c70743e485e20331f59cbc1c848dcdeda716f351eb20588c406a7df5fb8917ebf816739aa1414ac3b8498e493bf0badea5c9fdb3cc34da8b152b995957591880c523beb1d3f12487c38d18f62dd26209a727674e5a5fe3a3e3037860839afd80132248f0640e89761a123fad8155748d764f347a29e059758575a770618ab6f82d06bad973c3fb62505d9749f4f8483c8d607c61bae7c75ef09add6d91b728449726534e65379f7b3442d2a4aa2b8c3cdb90311b53dc333bbf6b213949a8990b4300473985f60c09c6a91ac963c802e319c28bafc2be66eceb3f1924081724e44d173de2091251d1ea69ec827b94ea5ab63436f0701dd2bf299e1a66a22c4b44b32b88620949736e088bc3ec6e7974426e4b392ecece0e88a7acaf510322d1726da6bc9580dad3c8717619051c220d8654a35eb7fa0a6de4be0456522054f124bbb0bdda4bc177b35a6ca20bd996f3a3499ffd00c93d4705cc4bc05f428541c3adcc36f0b9b9aebc61a88cd4bad8f034dd4a483de9bd3291e4bee06449083c83e"]}';

Requires a corresponding change in the proxy to encode the ore index value as an array:
cipherstash/proxy#110

Current status: ORE works here, but does not work via proxy and I am not sure why.

@tobyhede tobyhede changed the title WIP WIP ORE Operators and Sorting Feb 7, 2025
Copy link
Contributor

@coderdan coderdan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the simplicity of just using bytea (especially if it actually works!).

These changes introduce a potential side channel vulnerability though (timing attack) because the comparison now bails early.

@tobyhede tobyhede changed the title WIP ORE Operators and Sorting ORE Operators and Sorting Feb 14, 2025
Copy link
Contributor

@coderdan coderdan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work. I bet you're glad to get this one over the line!

@tobyhede tobyhede merged commit 8564572 into main Feb 19, 2025
4 checks passed
@tobyhede tobyhede deleted the operator-please branch February 19, 2025 05:29
CDThomas added a commit that referenced this pull request Feb 25, 2025
This change reverts this repo back to 01dcc24.

The changes reverted include commits from:
- #86
- #87

We mostly want to revert the changes in #86 (since they aren't
working as intended with Proxy), but #87 is also included since it's
more recent (and also includes some ORE-related changes that
would be tedious to untangle).

Since there aren't many changes after #86, the most pragmatic
option is to revert to the last known-good state and redo the
install/uninstall changes by hand on top of that.

Commands used to revert:
```
git reset --hard 01dcc24
git reset --soft ed460fc
```

This change doesn't use `git revert` because there were > 20 commits
to revert and merge commits also don't play well with `git revert`.
CDThomas added a commit that referenced this pull request Feb 25, 2025
This change updates `cs_ore_64_8_v1` to parse ORE indexes (the
`'o'` field) as JSON arrays of hex-encoded strings (instead of casting
from the Postgres text format).

The corresponding change for encoding ORE indexes as JSON arrays
of hex-encoded stings has already been merged in Proxy.

This is similar to some of the changes in
#86, but
we're parsing into the composite types for ORE indexes instead of
into a plain `bytea[]`. Parsing into the composite type allows for
ordering with an operator class on the output from `cs_ore_64_8_v1`.
CDThomas added a commit that referenced this pull request Feb 25, 2025
This change updates `cs_ore_64_8_v1` to parse ORE indexes (the
`'o'` field) as JSON arrays of hex-encoded strings (instead of casting
from the Postgres text format).

The corresponding change for encoding ORE indexes as JSON arrays
of hex-encoded stings has already been merged in Proxy.

This is similar to some of the changes in
#86, but
we're parsing into the composite types for ORE indexes instead of
into a plain `bytea[]`. Parsing into the composite type allows for
ordering with an operator class on the output from `cs_ore_64_8_v1`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants