feat(sdk): custom assertion provider

.gitignore

@@ -42,11 +42,11 @@ coverage.lcov
 sdk/nanotest1.ntdf
 
 *.zip
-sensitive.txt.tdf
 keys/
-/examples/sensitive.txt.ntdf
-sensitive.txt.ntdf
 traces/
 
 # Cucumber / BDD log files
 *.log
+/examples/examples-cli
+/examples/*.tdf
+/examples/*.ntdf

adr/decisions/2025-01-04-custom-assertion-providers.md

@@ -0,0 +1,217 @@
+# Custom Assertion Providers for OpenTDF SDK
+
+## Status
+Implemented
+
+## Context
+
+The OpenTDF SDK previously only supported assertion signing and validation using predefined symmetric or asymmetric keys. This limitation prevented integration with external signing mechanisms such as:
+
+- Hardware security modules (HSMs)
+- Smart cards (CAC/PIV)
+- Cloud-based key management services
+- X.509 certificate-based signing
+
+Additionally, the otdfctl tool (on a feature branch) had diverged from the SDK by implementing a different assertion binding approach, creating interoperability issues.
+
+## Decision
+
+We have implemented **custom assertion provider interfaces** that allow developers to supply their own signing and validation implementations while maintaining full backward compatibility with existing code.
+
+### Key Design Principles
+
+1. **Pluggable Architecture**: Developers can pass custom providers to the SDK
+2. **Backward Compatible**: When no custom provider is supplied, SDK uses existing DEK-based logic
+3. **Single Binding Style**: All tools use the same assertion binding approach (`assertionHash` + `assertionSig`)
+4. **Flexible Verification**: Support both DEK-based and X.509-based verification methods
+
+### Architecture
+
+```
+                 Assertion Providers
+                         │
+            ┌────────────┴────────────┐
+            │                         │
+     Signing Provider          Validation Provider
+            │                         │
+    ┌───────┴───────┐        ┌───────┴───────┐
+    │               │        │               │
+ Default         X.509    Default         X.509
+ (DEK)     (Certificate)   (DEK)     (Certificate)
+    │               │        │               │
+ Built-in      External   Built-in      External
+```
+
+### Interfaces
+
+#### AssertionSigningProvider
+
+```go
+type AssertionSigningProvider interface {
+    // Sign creates a JWS signature for the given assertion
+    Sign(ctx context.Context, assertion *Assertion, 
+         assertionHash, assertionSig string) (signature string, err error)
+
+    // GetSigningKeyReference returns a reference for the signing key
+    GetSigningKeyReference() string
+
+    // GetAlgorithm returns the signing algorithm (e.g., "RS256", "ES256")
+    GetAlgorithm() string
+}
+```
+
+#### AssertionValidationProvider
+
+```go
+type AssertionValidationProvider interface {
+    // Validate verifies the assertion signature
+    Validate(ctx context.Context, assertion Assertion) (hash, sig string, err error)
+
+    // IsTrusted checks if the signing entity is trusted
+    IsTrusted(ctx context.Context, assertion Assertion) error
+
+    // GetTrustedAuthorities returns list of trusted authorities
+    GetTrustedAuthorities() []string
+}
+```
+
+### Usage
+
+#### Creating TDFs with Custom Signing
+
+```go
+// X.509 certificate signing (e.g., for PIV/CAC cards)
+provider := sdk.NewX509SigningProvider(privateKey, certChain)
+client.CreateTDF(output, input, 
+    sdk.WithAssertionSigningProvider(provider),
+    sdk.WithAssertions(assertionConfigs))
+
+// Default behavior (unchanged)
+client.CreateTDF(output, input, sdk.WithAssertions(assertionConfigs))
+```
+
+#### Reading TDFs with Custom Validation
+
+```go
+// X.509 certificate validation
+provider := sdk.NewX509ValidationProvider(options)
+client.LoadTDF(file,
+    sdk.WithReaderAssertionValidationProvider(provider))
+
+// Default behavior (unchanged)
+client.LoadTDF(file) // Uses DEK-based validation
+```
+
+## Implementation
+
+### Completed Work
+
+1. **Core Interfaces** (`sdk/assertion_provider.go`)
+   - `AssertionSigningProvider` interface
+   - `AssertionValidationProvider` interface
+   - Supporting types and options
+
+2. **Built-in Implementations**
+   - `DefaultSigningProvider/DefaultValidationProvider` - Existing DEK-based behavior
+   - `X509SigningProvider/X509ValidationProvider` - X.509 certificate support
+   - `PKCS11Provider` - Template for hardware token integration
+
+3. **SDK Integration**
+   - `WithAssertionSigningProvider()` - Configure custom signing for TDF creation
+   - `WithReaderAssertionValidationProvider()` - Configure custom validation for TDF reading
+   - Automatic fallback to default providers when none specified
+
+4. **Unified Binding Style**
+   - All implementations use `assertionHash` and `assertionSig` claims
+   - otdfctl updated to match SDK approach (completed 2025-01-04)
+   - Full interoperability between all tools
+
+### Example: Command Line Usage
+
+```bash
+# Decrypt TDF with X.509 assertion validation (e.g., from otdfctl)
+./examples-cli decrypt --x509-verify signed.tdf
+
+# Standard decryption (DEK-based validation)
+./examples-cli decrypt signed.tdf
+```
+
+## Consequences
+
+### Positive
+
+- ✅ **Extensibility**: Support for any signing mechanism (HSM, cloud KMS, hardware tokens)
+- ✅ **Backward Compatibility**: Existing code continues to work without changes
+- ✅ **Interoperability**: All OpenTDF tools create compatible TDFs
+- ✅ **Security**: X.509 support enables certificate-based identity and non-repudiation
+- ✅ **Simplicity**: Single binding style reduces complexity
+
+### Negative
+
+- ❌ **Migration Required**: Developers using custom keys must implement providers
+- ❌ **API Surface**: New interfaces to learn and maintain
+
+### Neutral
+
+- ↔️ **Performance**: Negligible impact - provider pattern adds minimal overhead
+- ↔️ **Documentation**: Requires examples for common integration scenarios
+
+## Security Considerations
+
+1. **Key Management**: Custom providers are responsible for secure key handling
+2. **Certificate Validation**: X.509 providers should verify certificate chains and revocation
+3. **Trust Models**: Different providers may implement different trust policies
+4. **Audit**: Providers should log signing operations for compliance
+
+## Acceptance Criteria Verification
+
+✅ **SDK exposes interface for custom signing implementation**
+- `AssertionSigningProvider` interface defined and integrated
+
+✅ **SDK exposes interface for custom assertion validation**
+- `AssertionValidationProvider` interface defined and integrated
+
+✅ **Defaults to existing logic when no custom providers supplied**
+- SDK uses `DefaultSigningProvider/DefaultValidationProvider` automatically
+
+✅ **Documentation explains implementation and usage**
+- This ADR plus `CUSTOM_ASSERTION_PROVIDERS.md` guide
+
+## Migration Guide
+
+### For Existing Applications
+No changes required. The SDK maintains full backward compatibility.
+
+### For Hardware Token Integration
+1. Implement `AssertionSigningProvider` using your PKCS#11 library
+2. Pass provider via `WithAssertionSigningProvider()`
+3. For validation, use `X509ValidationProvider` or implement custom
+
+### For Cloud KMS Integration
+1. Implement `AssertionSigningProvider` using your KMS SDK
+2. Handle key references and permissions in your provider
+3. Pass provider to SDK when creating TDFs
+
+## Future Work
+
+1. **Provider Registry**: Allow registration of providers by name/type
+2. **Caching**: Add caching support for validation providers
+3. **Batch Operations**: Optimize providers for bulk signing/validation
+4. **Standard Providers**: Build providers for common KMS services (AWS, Azure, GCP)
+
+## Decision Record
+
+- **Date**: 2025-01-04  
+- **Authors**: Platform SDK Team
+- **Stakeholders**: Security Team, otdfctl Team, Enterprise Customers
+- **Supersedes**: Initial assertion binding divergence
+- **Related PRs**: 
+  - SDK custom providers implementation
+  - otdfctl alignment to SDK binding style
+
+## References
+
+- [OpenTDF Specification](https://github.com/opentdf/spec)
+- [PKCS#11 Specification](http://docs.oasis-open.org/pkcs11/pkcs11-base/v2.40/os/pkcs11-base-v2.40-os.html)
+- [X.509 Certificate Standard](https://www.itu.int/rec/T-REC-X.509)
+- [PIV Card Specification](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-73-4.pdf)

examples/README.md

@@ -5,17 +5,111 @@ Examples demonstrating the usage of the OpenTDF Platform Services.
 These examples can be launched using an example CLI.
 [See the autogenerated cobra docs](./docs/examples.md).
 
-Example: Build and run the attributes example:
+## Building the Examples CLI
 
 ```shell
-go build
+go build -o examples-cli
 ```
 
+## Quick Start
+
+```shell
+# List attributes
+./examples-cli attributes ls
+
+# Run encryption example
+./examples-cli encrypt sensitive.txt -o sensitive.txt.tdf
+```
+
+## Available Examples
+
+### Core Services
+- [Attribute Service](./cmd/attributes.go) - Manage and query data attributes
+- [Authorization Service](./cmd/authorization.go) - Authorization and access control
+
+### Assertion
+- [Assertion](./cmd/assertion.go) - Examples of custom signing and validation providers.
+
+### Encryption & Decryption
+- [Encrypt](./cmd/encrypt.go) - Create encrypted TDF files.
+- [Decrypt](./cmd/decrypt.go) - Decrypt TDF files. Now includes a "Magic Word" example for simple custom validation.
+- [IsValid](./cmd/isvalid.go) - Validate TDF files
+
+### Key Access Service (KAS)
+- [KAS Operations](./cmd/kas.go) - Key access service operations
+
+### Benchmarking
+- [Benchmark](./cmd/benchmark.go) - Performance benchmarking
+- [Benchmark Bulk](./cmd/benchmark_bulk.go) - Bulk operation benchmarks
+- [Decision Benchmarks](./cmd/benchmark_decision.go) - Decision service benchmarks
+
+## Assertion Provider Examples
+
+The `assertion` command demonstrates how to use the `ProviderFactory` to implement custom signing and validation for TDF assertions.
+
+### Adding an Assertion to an Existing TDF
+
+The `assertion add` subcommand demonstrates how to add a new assertion to an existing TDF. To ensure the integrity of the TDF is maintained, this is accomplished by performing a full decrypt and re-encrypt workflow. This process guarantees that the TDF's root signature is correctly recalculated to include the new assertion.
+
+**To use this example:**
+
+```shell
+# Add a new assertion to a TDF, creating a new output file
+./examples-cli assertion add --in sensitive.tdf --out sensitive-plus-assertion.tdf --magic-word swordfish
+```
+
+### Implementation Details
+
+The provider interfaces allow developers to:
+- Implement custom signing logic without modifying the core SDK.
+- Support hardware tokens that never expose private keys.
+- Validate assertions using custom trust models.
+- Maintain backward compatibility while adding new capabilities.
+
+For full implementation details, see:
+- [Provider Factory](../sdk/assertion_provider_factory.go)
+- [Provider Interfaces](../sdk/assertion_provider.go)
+
+## Configuration Options
+
+The examples CLI supports various configuration options:
+
 ```shell
-./examples attributes ls
+# Platform connection options
+./examples-cli --platformEndpoint https://your-platform.example.com \
+              --creds client-id:client-secret \
+              --tokenEndpoint https://auth.example.com/token
+
+# Security options
+./examples-cli --insecureSkipVerify    # Skip TLS verification (dev only)
+./examples-cli --insecurePlaintextConn # Use plaintext connection (dev only)
+
+# Example with full configuration
+./examples-cli assertion add --in sensitive.tdf --out sensitive-plus-assertion.tdf --magic-word swordfish \
+              --platformEndpoint https://platform.example.com 
+              --creds myapp:mysecret
 ```
 
-Examples:
+## Environment Setup
+
+For local development and testing:
 
-- [Attribute Service](./cmd/attributes.go)
-- [Authorization Service](./cmd/authorization.go)
+```shell
+./examples-cli --platformEndpoint "http://localhost:8080" --creds "opentdf:secret" \
+attributes ls
+```
+
+```shell
+./examples-cli --platformEndpoint "http://localhost:8080" --creds "opentdf:secret" \
+encrypt --autoconfigure=false README.md -o sensitive.txt.tdf
+```
+
+```shell
+./examples-cli --platformEndpoint "http://localhost:8080" --creds "opentdf:secret" \
+assertion add --in sensitive.txt.tdf --out sensitive-plus-assertion.tdf --magic-word swordfish
+```
+
+```shell
+./examples-cli --platformEndpoint "http://localhost:8080" --creds "opentdf:secret" \
+decrypt sensitive-plus-assertion.tdf --magic-word swordfish
+```