Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .cargo/audit.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,6 @@
[advisories]
# List of advisory IDs to ignore (extracted from deny.toml)
ignore = [
"RUSTSEC-2023-0071", # rsa
"RUSTSEC-2024-0436", # unmaintained paste
"RUSTSEC-2023-0071", # rsa
"RUSTSEC-2024-0436", # unmaintained paste
]
25 changes: 21 additions & 4 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,23 +7,27 @@ Always reference these instructions first and fallback to search or bash command
## Working Effectively

### Initial Repository Setup

- Initialize git submodules (REQUIRED for test data): `git submodule update --init --recursive`
- The repository requires the nightly Rust toolchain: `nightly-2025-03-31` (configured in `rust-toolchain.toml`)
- Verify Rust toolchain: `rustc --version && cargo --version`

### Building the Project

- Debug build: `cargo build` -- takes 6-7 minutes on first build. NEVER CANCEL. Set timeout to 15+ minutes.
- Release build: `cargo build --release` -- takes 3-4 minutes. NEVER CANCEL. Set timeout to 10+ minutes.
- Build specific crates: `cargo build -p cosmian_findex_server` or `cargo build -p cosmian_findex_cli`
- The main server binary: `target/debug/cosmian_findex_server` or `target/release/cosmian_findex_server`

### Testing the Project

- **CRITICAL**: Tests require Redis database running. Use: `docker compose up -d` before running tests.
- Library tests only: `cargo test --lib --package cosmian_findex_server` -- takes 30-45 seconds. Set timeout to 2+ minutes.
- Integration tests: `export FINDEX_TEST_DB="redis-findex" && cargo test --package cosmian_findex_cli --lib` -- takes 3+ minutes but may fail without KMS server. Set timeout to 10+ minutes.
- **DO NOT** run full integration tests without proper infrastructure setup (requires both Redis and KMS servers).

### Code Quality and Formatting

- Install clippy: `rustup component add clippy`
- Format check: `cargo fmt --check` -- takes <1 second
- Format code: `cargo fmt`
Expand All @@ -33,30 +37,35 @@ Always reference these instructions first and fallback to search or bash command
### Running the Server

#### With Docker (Recommended for Quick Testing)

- Start Redis only: `docker compose up -d`
- Run local server: `./target/debug/cosmian_findex_server`
- Test server: `curl http://localhost:6668/version`
- Default configuration: HTTP on port 6668, Redis on localhost:6379

#### From Source (Development)

- **PREREQUISITE**: Start Redis: `docker compose up -d`
- Run server: `cargo run --bin cosmian_findex_server`
- Or use built binary: `./target/debug/cosmian_findex_server`
- **IMPORTANT**: Server will fail to start without Redis running

#### Docker Quick Start (May Have Issues)

- Full stack: `docker compose -f docker-compose-quick-start.yml up -d`
- **NOTE**: The Docker quick start may have connectivity issues in some environments

### Server Configuration

- Default config: HTTP server on 0.0.0.0:6668, Redis on localhost:6379
- Configuration via environment variables (prefix: `FINDEX_SERVER_`)
- Configuration via TOML file (see `documentation/docs/configuration.md`)
- Help: `./target/debug/cosmian_findex_server --help`

## Validation

### Always validate changes by:
### Always validate changes by

1. Building successfully: `cargo build` (NEVER CANCEL - 6+ minutes)
2. Running clippy: `cargo clippy --workspace --all-targets -- -D warnings` (NEVER CANCEL - 2+ minutes)
3. Formatting: `cargo fmt --check`
Expand All @@ -65,6 +74,7 @@ Always reference these instructions first and fallback to search or bash command
6. Running lib tests: `cargo test --lib --package cosmian_findex_server`

### Expected Success Outputs

- **Build success**: "Finished `dev` profile [unoptimized + debuginfo] target(s) in Xm Ys"
- **Test success**: "test result: ok. 19 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in X.Ys"
- **Server startup**: Log messages including "starting 4 workers" and "listening on: 0.0.0.0:6668"
Expand All @@ -76,23 +86,26 @@ Always reference these instructions first and fallback to search or bash command
- **Server startup sequence**: Look for log messages including "starting 4 workers" and "Starting the HTTP Findex server..."

### CRITICAL Timeout Guidelines

- **NEVER CANCEL builds or long-running commands**
- Debug build: 15+ minute timeout
- Release build: 10+ minute timeout
- Release build: 10+ minute timeout
- Clippy: 5+ minute timeout
- Library tests: 2+ minute timeout
- Integration tests: 10+ minute timeout (but expect failures without full infrastructure)

## Key Projects and Structure

### Workspace Crates

- `crate/server` - Main Findex server binary (`cosmian_findex_server`)
- `crate/cli` - Command-line interface library (`cosmian_findex_cli`)
- `crate/findex_client` - Client library for interacting with server
- `crate/structs` - Shared data structures
- `crate/test_findex_server` - Test utilities

### Important Files

- `Cargo.toml` - Workspace configuration
- `rust-toolchain.toml` - Specifies required nightly toolchain
- `docker-compose.yml` - Redis for development
Expand All @@ -101,23 +114,27 @@ Always reference these instructions first and fallback to search or bash command
- `documentation/` - MkDocs-based documentation

### Dependencies

- **Redis**: Required for all server operations and tests
- **Docker**: Used for Redis and integration testing
- **Git submodules**: `test_data` and `.github/reusable_scripts` contain required test certificates and scripts

## Common Issues

### Build Issues

- **Network timeouts during initial build**: Retry `cargo build` - dependency downloads can be slow
- **Missing git submodules**: Run `git submodule update --init --recursive`
- **Wrong Rust toolchain**: Verify `rust-toolchain.toml` is respected

### Runtime Issues
### Runtime Issues

- **Server won't start**: Ensure Redis is running with `docker compose up -d`
- **Test failures**: Most integration tests require both Redis and KMS servers
- **Port conflicts**: Default ports are 6668 (Findex) and 6379 (Redis)

### Development Workflow

- Always start with: `git submodule update --init --recursive`
- Always ensure Redis is running before testing: `docker compose up -d`
- Always run the full validation sequence after making changes
Expand All @@ -130,4 +147,4 @@ Always reference these instructions first and fallback to search or bash command
- **Memory**: 4GB+ recommended for builds
- **Disk**: 2GB+ for full build artifacts
- **Network**: Required for initial dependency downloads and Docker image pulls
- **Docker**: Required for Redis database and integration testing
- **Docker**: Required for Redis database and integration testing
5 changes: 3 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ repos:
args: [--skip-string-normalization]

- repo: https://github.com/Cosmian/git-hooks.git
rev: v1.0.36
rev: v1.0.37
hooks:
- id: cargo-format
- id: dprint-toml-fix
Expand All @@ -146,7 +146,8 @@ repos:
- id: docker-compose-down

- repo: https://github.com/EmbarkStudios/cargo-deny
rev: 0.18.2 # choose your preferred tag
rev: 0.18.4
hooks:
- id: cargo-deny
args: [--all-features, check]
stages: [manual]
1 change: 0 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ incremental = false
opt-level = 0

[workspace.dependencies]
actix-server = { version = "2.5", default-features = false }
actix-web = { version = "4.9", default-features = false }
actix-server = { version = "2.6", default-features = false }
actix-web = { version = "4.11", default-features = false }
async-sqlite = { version = "=0.4" }
base64 = "0.22"
clap = { version = "4.5", default-features = false }
Expand All @@ -64,7 +64,7 @@ reqwest = { version = "=0.11", default-features = false }
serde = "1.0"
serde_json = "1.0"
strum = { version = "0.27", default-features = false }
tempfile = "3.17"
tempfile = "3.20"
thiserror = "2.0"
tokio = { version = "1.47", default-features = false }
tracing = "0.1"
Expand Down
1 change: 0 additions & 1 deletion crate/findex_client/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ doctest = false
non-fips = ["cosmian_kms_cli/non-fips", "test_kms_server/non-fips"]

[dependencies]
base64 = { workspace = true }
cosmian_findex = { workspace = true }
cosmian_findex_structs = { path = "../structs", version = "0.4.1" }
cosmian_http_client = { workspace = true }
Expand Down
6 changes: 2 additions & 4 deletions crate/findex_client/src/findex_rest_client.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
use base64::{Engine as _, engine::general_purpose};
use cosmian_findex_structs::{Addresses, Bindings, Guard, OptionalWords};
use cosmian_findex_structs::{Addresses, Bindings, Guard, OptionalWords, Word};
use cosmian_sse_memories::{ADDRESS_LENGTH, Address, MemoryADT};
use tracing::{debug, trace, warn};
use uuid::Uuid;
Expand Down Expand Up @@ -126,9 +125,8 @@ impl<const WORD_LENGTH: usize> MemoryADT for FindexRestClient<WORD_LENGTH> {
trace!(
"guarded_write successful on server url {}. guard: {}",
server_url,
guard.map_or("None".to_owned(), |g| general_purpose::STANDARD.encode(g))
guard.map_or_else(|| "None".to_owned(), |g| format!("{}", Word::from(g)))
);

Ok(guard)
}
}
2 changes: 1 addition & 1 deletion crate/findex_client/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
clippy::renamed_function_params,
clippy::verbose_file_reads,
clippy::str_to_string,
clippy::string_to_string,
clippy::implicit_clone,
clippy::unreachable,
clippy::as_conversions,
clippy::print_stdout,
Expand Down
2 changes: 1 addition & 1 deletion crate/server/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ uuid = { workspace = true, features = ["v4"] }
cosmian_findex = { workspace = true, features = ["test-utils"] }
cosmian_sse_memories = { workspace = true, features = ["test-utils"] }
tempfile = { workspace = true }
variant_count = "1.1"
variant_count = "1.2"

# ------------------------------------------------------------------------------
# START DEBIAN PACKAGING
Expand Down
2 changes: 1 addition & 1 deletion crate/server/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
clippy::renamed_function_params,
clippy::verbose_file_reads,
clippy::str_to_string,
clippy::string_to_string,
clippy::implicit_clone,
clippy::unreachable,
clippy::as_conversions,
clippy::print_stdout,
Expand Down
2 changes: 1 addition & 1 deletion crate/structs/src/findex/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@ pub use guard::Guard;
pub use keywords::{Keyword, KeywordToDataSetsMap, Keywords};
pub use search_results::SearchResults;
pub use value::Value;
pub use words::OptionalWords;
pub use words::{OptionalWords, Word};
84 changes: 79 additions & 5 deletions crate/structs/src/findex/tests.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,20 @@ mod findex_tests {
CsRng, Sampling,
reexport::rand_core::{RngCore, SeedableRng},
};
use cosmian_findex::WORD_LENGTH;
use cosmian_sse_memories::{ADDRESS_LENGTH, Address};

use crate::findex::{
addresses::Addresses, bindings::Bindings, guard::Guard, words::OptionalWords,
use crate::{
CUSTOM_WORD_LENGTH,
findex::{
addresses::Addresses,
bindings::Bindings,
guard::Guard,
words::{OptionalWords, Word},
},
};

const SEED: [u8; 32] = [1_u8; 32]; // arbitrary seed for the RNG
const WORD_LENGTH: usize = CUSTOM_WORD_LENGTH;

#[test]
fn test_ser_deser_addresses() {
Expand Down Expand Up @@ -54,7 +60,7 @@ mod findex_tests {
let mut word = [0_u8; WORD_LENGTH];
rng.fill_bytes(&mut word[..]);

let guard_some: Guard<WORD_LENGTH> = Guard(address1, Some(word));
let guard_some: Guard<{ WORD_LENGTH }> = Guard(address1, Some(word));
let serialized_some = guard_some.serialize().expect("Serialization failed");
let deserialized_some =
Guard::deserialize(&serialized_some).expect("Deserialization failed");
Expand All @@ -65,7 +71,7 @@ mod findex_tests {
);

let address2: Address<ADDRESS_LENGTH> = Address::random(&mut rng);
let guard_none: Guard<WORD_LENGTH> = Guard(address2, None);
let guard_none: Guard<{ WORD_LENGTH }> = Guard(address2, None);

let serialized_none = guard_none.serialize().expect("Serialization failed");
let deserialized_none =
Expand Down Expand Up @@ -95,4 +101,72 @@ mod findex_tests {

assert_eq!(bindings, deserialized, "Bindings do not match");
}

#[test]
fn test_word_display() {
// Test with a known byte array to verify base64 encoding
let word_bytes = [1, 2, 3, 4, 5, 6, 7, 8];
let word: Word<8> = Word::new(word_bytes);

// Expected base64 encoding of [1, 2, 3, 4, 5, 6, 7, 8]
let expected = "AQIDBAUGBwg=";
assert_eq!(format!("{word}"), expected);
}

#[test]
fn test_word_conversions() {
let word_bytes = [1, 2, 3, 4, 5, 6, 7, 8];

// Test From<[u8; N]>
let word: Word<8> = Word::from(word_bytes);
assert_eq!(word.as_bytes(), &word_bytes);

// Test Into<[u8; N]>
let back_to_bytes: [u8; 8] = word.into();
assert_eq!(back_to_bytes, word_bytes);

// Test AsRef<[u8; N]>
let word2: Word<8> = Word::new(word_bytes);
let as_ref_array: &[u8; 8] = word2.as_ref();
assert_eq!(as_ref_array, &word_bytes);

// Test AsRef<[u8]>
let as_ref_slice: &[u8] = word2.as_ref();
assert_eq!(as_ref_slice, &word_bytes);

// Test Deref
assert_eq!(*word2, word_bytes);
}

#[test]
fn test_word_with_word_length() {
let mut rng = CsRng::from_seed(SEED);
let mut word_bytes = [0_u8; WORD_LENGTH];
rng.fill_bytes(&mut word_bytes[..]);

let word: Word<{ WORD_LENGTH }> = Word::new(word_bytes);

// Test that display produces valid base64
let display_str = format!("{word}");
assert!(!display_str.is_empty());

// Test round-trip conversion
let back_to_bytes: [u8; WORD_LENGTH] = word.into_inner();
assert_eq!(back_to_bytes, word_bytes);
}

#[test]
fn test_word_real_example() {
// Test with "Hello World!" which should produce "SGVsbG8gV29ybGQh" in base64
let hello_bytes = [72, 101, 108, 108, 111, 32, 87, 111, 114, 108, 100, 33]; // "Hello World!"
let word: Word<12> = Word::new(hello_bytes);

let display_str = format!("{word}");
let expected = "SGVsbG8gV29ybGQh"; // Base64 of "Hello World!"
assert_eq!(display_str, expected);

// Test round-trip
let back_bytes: [u8; 12] = word.into();
assert_eq!(back_bytes, hello_bytes);
}
}
Loading
Loading