Use IMemoryCache for process-local tool embeddings

AGENTS.md

@@ -192,6 +192,8 @@ If no new rule is detected -> do not update the file.
 - Keep runtime services DI-native from their public/internal constructors; types such as `McpGatewayRegistry` must be creatable through `IOptions<McpGatewayOptions>` and other DI-managed dependencies rather than ad-hoc state-only constructors, because the package design requires services to live fully inside the container.
 - When emitting package identity to external protocols such as MCP client info, never hardcode a fake version string; use the actual assembly/build version so runtime metadata stays aligned with the package being shipped.
 - For search-quality improvements, prefer mathematical or statistical ranking changes over hardcoded phrase lists or ad-hoc query text hacks, because the user explicitly wants tokenizer search to improve through general scoring behavior rather than manual exceptions.
+- Prefer framework-provided in-memory caching primitives such as `IMemoryCache` over custom process-local storage implementations when they cover the lifecycle and lookup needs, because self-rolled memory stores age poorly and make scaling/concurrency behavior harder to trust.
+- Never keep legacy compatibility shims, obsolete paths, or lingering documentation references to removed implementations when a replacement is accepted, because this repository should converge on the current design instead of carrying dead historical baggage.
 
 ### Critical (NEVER violate)
 
@@ -230,10 +232,14 @@ If no new rule is detected -> do not update the file.
 - Search + execute flows covered by automated tests
 - Clean root packaging and CI setup
 - Direct fixes over preserving legacy compatibility paths when cleanup or review-driven corrections are requested
+- Framework-provided caching primitives over self-rolled in-memory stores when the package only needs process-local cache semantics
+- Removing replaced code paths completely instead of keeping legacy mentions or compatibility leftovers
 
 ### Dislikes
 
 - Agentic Framework dependency creep in this repository
 - App-specific logic leaking into the shared gateway package
 - Duplicate metadata and versions across multiple files
 - Shipping behavior without tests
+- Self-rolled in-memory storage when standard .NET caching abstractions already fit the scenario
+- Legacy/obsolete compatibility leftovers after a replacement is accepted

Directory.Packages.props

@@ -7,6 +7,7 @@
     <PackageVersion Include="DotNet.ReproducibleBuilds" Version="2.0.2" />
     <PackageVersion Include="Microsoft.Agents.AI" Version="1.0.0-rc3" />
     <PackageVersion Include="Microsoft.Extensions.AI" Version="10.3.0" />
+    <PackageVersion Include="Microsoft.Extensions.Caching.Memory" Version="10.0.3" />
     <PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="10.0.3" />
     <PackageVersion Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="10.0.3" />
     <PackageVersion Include="Microsoft.Extensions.Hosting.Abstractions" Version="10.0.3" />
@@ -20,4 +21,4 @@
     <PackageVersion Include="ModelContextProtocol" Version="1.1.0" />
     <PackageVersion Include="TUnit" Version="1.19.0" />
   </ItemGroup>
-</Project>
+</Project>

README.md

@@ -427,17 +427,21 @@ services.AddManagedCodeMcpGateway(options =>
 
 If the keyed chat client is missing or normalization fails, search continues normally.
 
-## Optional Persistent Tool Embeddings
+## Optional Tool Embedding Stores
 
-For process-local caching, use the built-in in-memory store:
+For process-local caching, use the built-in `IMemoryCache`-backed store:
 
 ```csharp
 services.AddKeyedSingleton<IEmbeddingGenerator<string, Embedding<float>>, MyEmbeddingGenerator>(
     McpGatewayServiceKeys.EmbeddingGenerator);
-services.AddSingleton<IMcpGatewayToolEmbeddingStore, McpGatewayInMemoryToolEmbeddingStore>();
+services.AddMcpGatewayInMemoryToolEmbeddingStore();
 ```
 
-For durable caching, register your own `IMcpGatewayToolEmbeddingStore` implementation:
+This built-in store reuses the application's shared `IMemoryCache` and only caches embeddings inside the current process. It is useful for local reuse, but it is not durable and does not synchronize across replicas.
+
+.NET also provides `IDistributedCache` for out-of-process cache storage and `HybridCache` for a combined local + distributed cache model. `ManagedCode.MCPGateway` does not hardcode either dependency into the gateway runtime. If you need shared cache state across instances, implement `IMcpGatewayToolEmbeddingStore` over the cache technology your host already uses.
+
+For multi-instance or durable caching, register your own `IMcpGatewayToolEmbeddingStore` implementation:
 
 ```csharp
 services.AddKeyedSingleton<IEmbeddingGenerator<string, Embedding<float>>, MyEmbeddingGenerator>(

docs/ADR/ADR-0004-process-local-embedding-store-uses-imemorycache.md

@@ -0,0 +1,135 @@
+# ADR-0004: Process-Local Embedding Store Uses IMemoryCache
+
+## Context
+
+`ManagedCode.MCPGateway` exposes `IMcpGatewayToolEmbeddingStore` so hosts can reuse tool embeddings between index builds.
+
+The package still needs a built-in process-local option for hosts that only want cheap embedding reuse inside one app instance, but horizontal scale, durability, and cross-replica cache coherence remain separate concerns.
+
+## Decision
+
+The built-in `McpGatewayInMemoryToolEmbeddingStore` will use `IMemoryCache` for process-local embedding reuse, and the package will expose a dedicated `AddMcpGatewayInMemoryToolEmbeddingStore()` registration path that wires the store to the host's shared cache services.
+
+Durable or distributed embedding reuse will remain the responsibility of host-provided `IMcpGatewayToolEmbeddingStore` implementations.
+
+## Diagram
+
+```mermaid
+flowchart LR
+    Host["Host application"] --> DI["AddManagedCodeMcpGateway(...)"]
+    Host --> CacheRegistration["AddMcpGatewayInMemoryToolEmbeddingStore()"]
+    CacheRegistration --> MemoryCache["IMemoryCache"]
+    CacheRegistration --> Store["IMcpGatewayToolEmbeddingStore"]
+    Store --> Runtime["McpGatewayRuntime"]
+    Runtime --> Search["Index build / vector reuse"]
+    Host --> CustomStore["Custom durable store (optional)"]
+    CustomStore --> Runtime
+```
+
+## Alternatives
+
+### Alternative 1: Ship no built-in process-local embedding store
+
+Pros:
+
+- smallest core package surface
+- no opinionated local cache behavior
+
+Cons:
+
+- worse onboarding for hosts that only need local embedding reuse
+- forces boilerplate for a very common optional scenario
+
+### Alternative 2: Make the built-in store durable or distributed
+
+Pros:
+
+- stronger scaling story out of the box
+- state can survive process restarts
+
+Cons:
+
+- introduces infrastructure and configuration assumptions into the core package
+- forces dependency choices that belong to the host
+- conflicts with the package goal of keeping embedding persistence optional
+
+### Alternative 3: Depend directly on distributed cache abstractions in runtime code
+
+Pros:
+
+- shared cache behavior is explicit in the package internals
+- less host code for cache-backed multi-instance deployments
+
+Cons:
+
+- couples gateway runtime code to a specific cache family
+- weakens the current abstraction boundary around `IMcpGatewayToolEmbeddingStore`
+- makes single-instance local reuse heavier than necessary
+
+## Consequences
+
+Positive:
+
+- the built-in store relies on a standard .NET caching primitive
+- hosts can register the process-local store with one DI call and reuse the shared `IMemoryCache`
+- process-local cache behavior stays explicitly separate from durable/distributed storage concerns
+- fingerprint-agnostic lookups become deterministic by reusing the latest cached embedding for the same tool document
+
+Trade-offs:
+
+- the package takes a new runtime dependency on `Microsoft.Extensions.Caching.Memory`
+- the built-in store is still process-local only and does not solve multi-instance cache sharing
+- direct construction without DI now owns a private `MemoryCache` instance and must be disposed like any other cache owner
+
+Mitigations:
+
+- keep `IMcpGatewayToolEmbeddingStore` as the only abstraction consumed by runtime code
+- document clearly that `AddMcpGatewayInMemoryToolEmbeddingStore()` is for process-local reuse only
+- keep durable/distributed examples based on host-provided store implementations
+
+## Invariants
+
+- `McpGatewayRuntime` MUST continue to depend only on `IMcpGatewayToolEmbeddingStore`, not on `IMemoryCache` directly.
+- `McpGatewayInMemoryToolEmbeddingStore` MUST remain optional and MUST NOT become a mandatory dependency for gateway usage.
+- `AddMcpGatewayInMemoryToolEmbeddingStore()` MUST register the built-in store through the host `IServiceCollection` and MUST provision `IMemoryCache`.
+- Hosts that need cross-instance persistence or replication MUST continue to provide their own `IMcpGatewayToolEmbeddingStore`.
+- The built-in store MUST clone vectors on read/write boundaries so callers cannot mutate cached embedding buffers in place.
+
+## Rollout And Rollback
+
+Rollout:
+
+1. Add the `Microsoft.Extensions.Caching.Memory` dependency to the package.
+2. Implement `McpGatewayInMemoryToolEmbeddingStore` with `IMemoryCache`.
+3. Expose `AddMcpGatewayInMemoryToolEmbeddingStore()` for host DI registration.
+4. Update README and architecture docs to distinguish process-local cache reuse from durable storage.
+
+Rollback:
+
+1. Remove `AddMcpGatewayInMemoryToolEmbeddingStore()` only if the package intentionally stops shipping a built-in process-local embedding store.
+2. Keep `IMcpGatewayToolEmbeddingStore` as the only runtime dependency boundary unless the package intentionally adopts a different cache abstraction.
+
+## Verification
+
+- `dotnet restore ManagedCode.MCPGateway.slnx`
+- `dotnet build ManagedCode.MCPGateway.slnx -c Release --no-restore`
+- `dotnet build ManagedCode.MCPGateway.slnx -c Release --no-restore -p:RunAnalyzers=true`
+- `dotnet test --solution ManagedCode.MCPGateway.slnx -c Release --no-build`
+- `roslynator analyze src/ManagedCode.MCPGateway/ManagedCode.MCPGateway.csproj -p Configuration=Release --severity-level warning`
+- `roslynator analyze tests/ManagedCode.MCPGateway.Tests/ManagedCode.MCPGateway.Tests.csproj -p Configuration=Release --severity-level warning`
+- `cloc --include-lang=C# src tests`
+
+## Implementation Plan (step-by-step)
+
+1. Register `Microsoft.Extensions.Caching.Memory` as a package dependency.
+2. Implement the built-in store with `IMemoryCache` lookups keyed by tool id, document hash, and embedding fingerprint.
+3. Keep the runtime abstraction unchanged by continuing to consume `IMcpGatewayToolEmbeddingStore`.
+4. Add tests for the cache-backed store, including deterministic fingerprint-agnostic fallback behavior.
+5. Update README and architecture docs so process-local cache reuse is not described as durable persistence.
+
+## Stakeholder Notes
+
+- Product: the package still offers a zero-infrastructure local cache option, but durable storage remains opt-in.
+- Dev: use `AddMcpGatewayInMemoryToolEmbeddingStore()` when the host only needs process-local embedding reuse.
+- QA: verify vector reuse, clone safety, and fallback-to-latest behavior through the embedding-store tests.
+- DevOps: multi-instance deployments still need a host-provided durable/distributed `IMcpGatewayToolEmbeddingStore`.

docs/Architecture/Overview.md

@@ -94,7 +94,7 @@ flowchart LR
     RegistrationCollection --> SourceRegistrations["McpGatewayToolSourceRegistration*"]
     RuntimeSearch --> Json["McpGatewayJsonSerializer"]
     Warmup["McpGatewayIndexWarmupService"] --> McpGateway
-    InMemoryStore["McpGatewayInMemoryToolEmbeddingStore"] --> StoreIndex["McpGatewayToolEmbeddingStoreIndex"]
+    InMemoryStore["McpGatewayInMemoryToolEmbeddingStore"] --> MemoryCache["IMemoryCache"]
 ```
 
 ## Module Index
@@ -103,45 +103,47 @@ flowchart LR
 - Public abstractions: [`src/ManagedCode.MCPGateway/Abstractions/`](../../src/ManagedCode.MCPGateway/Abstractions/) defines the stable interfaces consumers resolve from DI.
 - Public configuration: [`src/ManagedCode.MCPGateway/Configuration/`](../../src/ManagedCode.MCPGateway/Configuration/) contains options and service keys that shape host integration.
 - Public models: [`src/ManagedCode.MCPGateway/Models/`](../../src/ManagedCode.MCPGateway/Models/) contains request/result contracts and enums grouped by search, invocation, catalog, and embeddings behavior.
-- Public embeddings: [`src/ManagedCode.MCPGateway/Embeddings/`](../../src/ManagedCode.MCPGateway/Embeddings/) provides optional embedding-store implementations.
+- Public embeddings: [`src/ManagedCode.MCPGateway/Embeddings/`](../../src/ManagedCode.MCPGateway/Embeddings/) provides optional embedding-store implementations, including the built-in `IMemoryCache`-backed process-local store.
 - Public meta-tools: [`src/ManagedCode.MCPGateway/McpGatewayToolSet.cs`](../../src/ManagedCode.MCPGateway/McpGatewayToolSet.cs) exposes the gateway as reusable `AITool` instances for model-driven search and invoke flows.
 - Public chat-options bridge: [`src/ManagedCode.MCPGateway/Registration/McpGatewayChatOptionsExtensions.cs`](../../src/ManagedCode.MCPGateway/Registration/McpGatewayChatOptionsExtensions.cs) attaches the gateway meta-tools to `ChatOptions` without replacing existing tools.
 - Public auto-discovery wrapper: [`src/ManagedCode.MCPGateway/McpGatewayAutoDiscoveryChatClient.cs`](../../src/ManagedCode.MCPGateway/McpGatewayAutoDiscoveryChatClient.cs) stages model-visible tools as `2 meta-tools -> latest discovered proxies -> replace on next search`.
 - Public chat-client extensions: [`src/ManagedCode.MCPGateway/Registration/McpGatewayChatClientExtensions.cs`](../../src/ManagedCode.MCPGateway/Registration/McpGatewayChatClientExtensions.cs) wraps any `IChatClient` with the recommended staged auto-discovery flow.
 - Internal catalog module: [`src/ManagedCode.MCPGateway/Internal/Catalog/`](../../src/ManagedCode.MCPGateway/Internal/Catalog/) owns mutable tool-source registration state and read-only snapshots for indexing.
 - Internal catalog sources: [`src/ManagedCode.MCPGateway/Internal/Catalog/Sources/`](../../src/ManagedCode.MCPGateway/Internal/Catalog/Sources/) owns transport-specific source registrations and MCP client creation.
 - Internal runtime module: [`src/ManagedCode.MCPGateway/Internal/Runtime/`](../../src/ManagedCode.MCPGateway/Internal/Runtime/) owns orchestration and is split by core, catalog, search, invocation, and embeddings concerns.
-- Internal embedding helpers: [`src/ManagedCode.MCPGateway/Internal/Embeddings/`](../../src/ManagedCode.MCPGateway/Internal/Embeddings/) contains non-public embedding indexing helpers.
 - Internal serialization: [`src/ManagedCode.MCPGateway/Internal/Serialization/`](../../src/ManagedCode.MCPGateway/Internal/Serialization/) contains the canonical JSON materialization path used by runtime features.
 - Warmup hooks: [`src/ManagedCode.MCPGateway/Registration/McpGatewayServiceProviderExtensions.cs`](../../src/ManagedCode.MCPGateway/Registration/McpGatewayServiceProviderExtensions.cs) and [`src/ManagedCode.MCPGateway/Internal/Warmup/McpGatewayIndexWarmupService.cs`](../../src/ManagedCode.MCPGateway/Internal/Warmup/McpGatewayIndexWarmupService.cs) provide optional eager index-building integration.
-- DI registration: [`src/ManagedCode.MCPGateway/Registration/McpGatewayServiceCollectionExtensions.cs`](../../src/ManagedCode.MCPGateway/Registration/McpGatewayServiceCollectionExtensions.cs) wires facade, registry, meta-tools, and warmup support into the container.
+- DI registration: [`src/ManagedCode.MCPGateway/Registration/McpGatewayServiceCollectionExtensions.cs`](../../src/ManagedCode.MCPGateway/Registration/McpGatewayServiceCollectionExtensions.cs) wires facade, registry, meta-tools, warmup support, and the optional `IMemoryCache`-backed embedding store into the container.
 
 ## Dependency Rules
 
 - Public code may depend on `Models`, `Configuration`, and `Abstractions`, but internal modules must not depend on tests or docs.
 - `McpGateway` is a thin facade only. It may delegate to `McpGatewayRuntime`, but it must not own registry mutation logic.
 - `Internal/Catalog` owns mutable source registration state. `Internal/Runtime` may read snapshots from it, but must not mutate registrations directly.
 - `Internal/Catalog/Sources` owns MCP transport-specific creation and caching. Transport setup must not leak into `Internal/Runtime`, `Models`, or `Configuration`.
-- `Internal/Runtime` may depend on `Internal/Catalog`, `Internal/Embeddings`, `Embeddings`, `Models`, `Configuration`, and `Abstractions`.
+- `Internal/Runtime` may depend on `Internal/Catalog`, `Embeddings`, `Models`, `Configuration`, and `Abstractions`.
 - Optional AI services such as embedding generators and query-normalization chat clients must stay outside the package core and be resolved through DI service keys rather than hardwired provider code.
 - Chat-client and agent integrations must stay `AITool`-centric in the core package. Host-specific frameworks may consume those tools, but the base package should not take a hard dependency on a specific agent host unless that becomes an explicit product decision.
 - `McpGatewayAutoDiscoveryChatClient` may orchestrate tool visibility for host chat loops, but it must stay generic over `IChatClient` and must not take a dependency on Microsoft Agent Framework.
 - The recommended staged host flow is: advertise only the two gateway meta-tools first, then project only the latest search matches as direct proxy tools, then replace that discovered set on the next search result.
 - `Models` should stay contract-first. Internal transport, registry, or lifecycle helpers do not belong there.
 - Embedding support must stay optional and isolated behind `IMcpGatewayToolEmbeddingStore` and embedding-generator abstractions.
+- The built-in process-local embedding store may depend on `IMemoryCache`, but cross-instance persistence and cache replication must stay behind host-provided `IMcpGatewayToolEmbeddingStore` implementations.
 - Warmup remains optional. The package must work correctly with lazy indexing and must not require manual initialization for every host.
 
 ## Key Decisions (ADRs)
 
 - [`docs/ADR/ADR-0001-runtime-boundaries-and-index-lifecycle.md`](../ADR/ADR-0001-runtime-boundaries-and-index-lifecycle.md): documents the public/runtime/catalog split, DI boundaries, lazy indexing, cancellation-aware single-flight builds, and optional warmup hooks.
 - [`docs/ADR/ADR-0002-search-ranking-and-query-normalization.md`](../ADR/ADR-0002-search-ranking-and-query-normalization.md): documents the default `Auto` search behavior, tokenizer-backed fallback, optional English query normalization, and mathematical ranking strategy.
 - [`docs/ADR/ADR-0003-reusable-chat-client-and-agent-tool-modules.md`](../ADR/ADR-0003-reusable-chat-client-and-agent-tool-modules.md): documents why chat-client and agent integrations stay generic around reusable `AITool` modules instead of adding a hard Agent Framework dependency to the core package.
+- [`docs/ADR/ADR-0004-process-local-embedding-store-uses-imemorycache.md`](../ADR/ADR-0004-process-local-embedding-store-uses-imemorycache.md): documents why the built-in process-local embedding cache uses `IMemoryCache` and why durable/distributed caching remains a host responsibility.
 
 ## Related Docs
 
 - [`README.md`](../../README.md)
 - [`docs/ADR/ADR-0001-runtime-boundaries-and-index-lifecycle.md`](../ADR/ADR-0001-runtime-boundaries-and-index-lifecycle.md)
 - [`docs/ADR/ADR-0002-search-ranking-and-query-normalization.md`](../ADR/ADR-0002-search-ranking-and-query-normalization.md)
 - [`docs/ADR/ADR-0003-reusable-chat-client-and-agent-tool-modules.md`](../ADR/ADR-0003-reusable-chat-client-and-agent-tool-modules.md)
+- [`docs/ADR/ADR-0004-process-local-embedding-store-uses-imemorycache.md`](../ADR/ADR-0004-process-local-embedding-store-uses-imemorycache.md)
 - [`docs/Features/SearchQueryNormalizationAndRanking.md`](../Features/SearchQueryNormalizationAndRanking.md)
 - [`AGENTS.md`](../../AGENTS.md)