Client Encryption: Adds fix to allow partition key path and id to be part of client encryption policy.

[kr-santosh]

Description

The PR brings in the following changes.

• Allow support to include partition key path and id during encryption policy creation (only deterministic encryption type) and bumps up the format version.

Type of change

Please delete options that are not relevant.

• [] New feature (non-breaking change which adds functionality)
• [] This change requires a documentation update

Closing issues

To automatically close an issue: closes #IssueNumber

[github-actions]

Please follow the required format: "[Internal] Category: (Adds|Fixes|Refactors|Removes) Description"

Internal should be used for PRs that have no customer impact. This flag is used to help generate the changelog to know which PRs should be included. Examples:
Diagnostics: Adds GetElapsedClientLatency to CosmosDiagnostics
PartitionKey: Fixes null reference when using default(PartitionKey)
[v4] Client Encryption: Refactors code to external project
[Internal] Query: Adds code generator for CosmosNumbers for easy additions in the future.

[github-actions]

Please follow the required format: "[Internal] Category: (Adds|Fixes|Refactors|Removes) Description"

Internal should be used for PRs that have no customer impact. This flag is used to help generate the changelog to know which PRs should be included. Examples:
Diagnostics: Adds GetElapsedClientLatency to CosmosDiagnostics
PartitionKey: Fixes null reference when using default(PartitionKey)
[v4] Client Encryption: Refactors code to external project
[Internal] Query: Adds code generator for CosmosNumbers for easy additions in the future.

[FabianMeiswinkel]

My 2 cents: Don't even try to support non-strings as partition keys. numbers in json are evil (neither range nor precision are clearly standardized - so supported ranges/precisions depend on the json parsers/libraries used - and this can cause hard to diagnose and mitigate issues). The good thing for end-to-end encryption is that all workloads are green-field - no scenario where a customer has TBs of data already. I think we should simply only support strings, instead of hacking something together that might work in most scenarios but doesn't solve the underlying problem that json doesn't have a clear definition for numbers - that solves the encryption-specific challenge of having to know the type and prevents customers form running into the other issues with numbers as PK later.
To be honest only supporting strings as PK is what we should have done across the board - too late now, because it is a breaking change. But with encryption we have the benefit of a "fresh start" for every workload - so let's do it right this time.

[kr-santosh]

Closed by mistake :)

[kr-santosh]

My 2 cents: Don't even try to support non-strings as partition keys. numbers in json are evil (neither range nor precision are clearly standardized - so supported ranges/precisions depend on the json parsers/libraries used - and this can cause hard to diagnose and mitigate issues). The good thing for end-to-end encryption is that all workloads are green-field - no scenario where a customer has TBs of data already. I think we should simply only support strings, instead of hacking something together that might work in most scenarios but doesn't solve the underlying problem that json doesn't have a clear definition for numbers - that solves the encryption-specific challenge of having to know the type and prevents customers form running into the other issues with numbers as PK later. To be honest only supporting strings as PK is what we should have done across the board - too late now, because it is a breaking change. But with encryption we have the benefit of a "fresh start" for every workload - so let's do it right this time.

Fixed.

[abhijitpai]

    public IEnumerable<ClientEncryptionIncludedPath> IncludedPaths

Please add a comment on this class, something like: 'The <see cref=ClientEncryptionPolicy should be initialized with policyFormatVersion 2 if "id" or properties that are part of the partition key need to be encrypted'. Also we need to mention that all PK property values need to be JSON strings. Goal is that this documentation should suffice to let customers know how to use the functionality. Ideally add some examples too where possible.

---

Refers to: Microsoft.Azure.Cosmos/src/Resource/Settings/ClientEncryptionPolicy.cs:40 in cba1c4d. [](commit_id = cba1c4d, deletion_comment = False)

[abhijitpai]

[ealsur]

@kirankumarkolli Please review public API change

Bot not working

[FabianMeiswinkel]

Thanks - looks good to me now.