Pixelbadger.Toolkit.Rag

An ASP.NET Core service for retrieval-augmented generation (RAG). Upload text, image and audio files over HTTP, and the service embeds them and answers queries with vector search over SQL Server (cosine similarity). It can be queried over REST and through an MCP server (Streamable HTTP) for AI assistants.

• Embeddings run locally. EmbeddingGemma 2 (ONNX, q8 quantised, 256-d Matryoshka) via ONNX Runtime. No API key, no data leaves your machine, and the service never downloads a model.
• Storage is SQL Server 2025 / Azure SQL. Documents, chunks (text and metadata), vector(256) embeddings and the ingest job queue live in SQL, which is also the only search index.
• Multimodal. Text (.txt, .md), images and audio are each ingested as their own documents and searched together.
• Document-centric REST. POST /api/documents uploads one or more files and returns 202 with a server-assigned document id per file; each file is queued as a job in SQL and processed in the background (unfinished jobs survive restarts) and you poll GET /api/documents/{id}. Re-ingest a document with POST /api/documents/{id}, delete it with DELETE /api/documents/{id}.
• MCP server. /mcp exposes a single Search tool over Streamable HTTP. Uploading and deleting are REST-only.
• Browser UI. The service serves a small React app at / (same origin as the API): a live jobs dashboard (/), a query page that shows the ranked chunks with links back to each source file (/query), and an upload page (/ingest). It is an operational UI over the REST API, not a chat interface.

Version 6.0 changes how ingest runs. Upgrading from 5.x: upload any file that was still queued or being processed again. Ingest is now event-driven: every job status change is published through a SlimMessageBus outbox on the service's own SQL database, and in-process consumers run the jobs. The polling worker and the startup reset are gone. The 6.0 migration (EventDrivenIngest) deletes the jobs that were Queued or Processing (and documents that were never indexed, which existed only for those jobs); everything else is kept, and documents keep their indexed chunks. It also turns on read-committed snapshot isolation where it is off (Azure SQL's default). The bus creates its BusMessages* and BusOutbox* tables the first time it is used. The Rag:Ingest:LeaseSeconds and PollIntervalSeconds settings keep their names but now mean the bus message lock and the bus poll interval.

Version 5.0 is a breaking change from 4.x: it deletes all your documents. The embedding model moves from the fp32 to the q8 (8-bit quantised) ONNX graphs of EmbeddingGemma 2. Vectors from the two are not compatible and there is no conversion, so the 5.0 migration (QuantizedEmbeddingsQ8) deletes every document, chunk and ingest job (including job history) and replaces dbo.Chunks_EG2_256 with dbo.Chunks_EG2Q8_256. Before starting 5.0, download the q8 model files (see the model section); on Azure, re-run scripts/upload-model.sh (or azd up, whose postprovision hook does it). Then upload your content again.

Version 4.0 is a breaking change from 3.x. BM25 keyword search (Lucene), hybrid fusion and the RRF reranker are removed: search is now pure vector search over SQL Server. In hybrid mode image and audio chunks could only score from the vector list, so they were systematically under-ranked against text; ranking by one metric treats all modalities alike. In responses, keywordRank and vectorRank are gone and score is now the cosine similarity (1 - cosine distance; higher is more similar) instead of a fused RRF score. Rag:IndexPath / PBRAG_INDEX_PATH are removed (ignored if still set; the old index directory can be deleted). No re-upload is needed: the SQL data is unchanged.

Version 3.0 is a breaking change from 2.x. The pbrag CLI, the stdio MCP transport and the dotnet-tool package are gone: the application is now a web service shipped as a container image (on GHCR). Ingest is upload-only (the server never reads caller-named paths from its own disk, so --content-path, folder ingestion, --allow-symlinks and --max-files no longer exist), pbrag query became POST /api/query, and pbrag serve became /mcp. Configuration moved from command-line options to the Rag configuration section (the PBRAG_* variables still work as fallbacks). Documents are no longer identified by their path: each upload creates a document with a server-assigned GUID id, the path is only metadata (the same path uploaded twice is two documents), and the search filter sourceIds became documentIds (GUIDs). 2.0 was itself a breaking change from 1.x (OpenAI embeddings and SQLite-vec removed, hybrid-only search).

Upgrading from 2.x requires re-uploading your content. Document ids cannot be converted, so the 3.0 migration drops and recreates the Documents, chunk (then Chunks_EG2_256) and ingest job tables (all existing data in them is lost). Upload your files again after starting 3.0.

Table of Contents

• Prerequisites
• Running the service
• Deploying to Azure
• Configuration
• HTTP API
• MCP server
• Supported content
• How it works
• Development and testing

Prerequisites

1. SQL Server 2025 or Azure SQL

The vector type needs SQL Server 2025, Azure SQL Database, Azure SQL Managed Instance or SQL database in Fabric. Locally, the Developer edition in Docker is the easiest option:

docker run -d --name sql2025 -p 1433:1433 \
  -e ACCEPT_EULA=Y -e MSSQL_PID=Developer \
  -e 'MSSQL_SA_PASSWORD=Pbrag_Dev_Pass1!' \
  mcr.microsoft.com/mssql/server:2025-latest

Connection string for that container:

Server=localhost,1433;User Id=sa;Password=Pbrag_Dev_Pass1!;TrustServerCertificate=True;Encrypt=False

On startup the service creates the database/tables for you by applying EF Core migrations (Rag:ApplyMigrationsOnStartup, on by default); the login needs permission to create them. The message bus creates its own tables (BusMessages*, BusOutbox*) the first time it is used.

Approximate vector index and PREVIEW_FEATURES. Once a chunk table holds at least 100 rows, ingestion creates a DiskANN vector index so queries use approximate VECTOR_SEARCH. That index is generally available in Azure SQL Database but is a preview feature on SQL Server 2025, where the database needs ALTER DATABASE SCOPED CONFIGURATION SET PREVIEW_FEATURES = ON; (the tool sets this where needed). The local preview build can lag Azure's, so if you hit problems set Rag:ExactVectorSearch to true to always use exact VECTOR_DISTANCE search. Results are the same up to approximation; exact search is simply slower on very large corpora.

2. The EmbeddingGemma 2 model (local copy)

The service loads the model from a local directory given by Rag:ModelPath (or PBRAG_MODEL_PATH). Download the onnx-community/embeddinggemma-2-ONNX files once:

pip install -U "huggingface_hub[cli]"

huggingface-cli download onnx-community/embeddinggemma-2-ONNX \
  --local-dir ~/models/embeddinggemma-2-onnx \
  --include "tokenizer.json" "tokenizer_config.json" "config.json" "processor_config.json" \
            "onnx/model_quantized.onnx*" \
            "onnx/vision_encoder_quantized.onnx*" \
            "onnx/audio_encoder_quantized.onnx*"

export PBRAG_MODEL_PATH=~/models/embeddinggemma-2-onnx

Notes:

• Use the q8 (8-bit quantised) graphs: onnx/model_quantized.onnx (text) plus onnx/vision_encoder_quantized.onnx and onnx/audio_encoder_quantized.onnx. The fp32 (model.onnx etc.) and fp16 variants are not used. The trailing * in the --include patterns also fetches a .onnx_data sidecar when a graph has one; sidecars are optional.
• A .onnx_data file, when present, must sit beside its .onnx with its original name: the external-data reference is by filename.
• Disk use is roughly a quarter of the fp32 snapshot. Queries only load the text model; the vision and audio encoders load lazily when an image or audio file is ingested.

3. ffmpeg (audio only)

Audio ingestion decodes files with ffmpeg (16 kHz mono). Install it and make sure it is on PATH (apt install ffmpeg, brew install ffmpeg, winget install ffmpeg); the container image already includes it. Without it, audio files fail with a clear per-file error; text and images are unaffected.

Running the service

Docker

Every merge to master publishes the image to the GitHub Container Registry as ghcr.io/pixelbadger/pixelbadger.toolkit.rag, tagged with the version (6.0.0), latest and sha-<short commit>. The image contains the service and ffmpeg, but not the model: mount your local copy of the EmbeddingGemma 2 snapshot read-only, and give it a SQL Server connection string.

docker pull ghcr.io/pixelbadger/pixelbadger.toolkit.rag:latest

docker run -d --name pbrag -p 8080:8080 \
  -e Rag__ConnectionString='Server=host.docker.internal,1433;User Id=sa;Password=Pbrag_Dev_Pass1!;TrustServerCertificate=True;Encrypt=False' \
  -v ~/models/embeddinggemma-2-onnx:/models/embeddinggemma-2-onnx:ro \
  ghcr.io/pixelbadger/pixelbadger.toolkit.rag:latest

curl http://localhost:8080/health

Open http://localhost:8080/ for the browser UI (/ jobs, /query, /ingest).

To build the image yourself instead: docker build -t pixelbadger-rag ..

The package is created private the first time the workflow pushes it. If anonymous docker pull is denied, a repository admin has to set the package's visibility to public once (GitHub: Packages > pixelbadger.toolkit.rag > Package settings > Change visibility).

The image defaults Rag__ModelPath=/models/embeddinggemma-2-onnx.

With .NET Aspire (local orchestration)

Pixelbadger.Toolkit.Rag.AppHost is an Aspire AppHost that runs the whole stack locally: SQL Server 2025 in a persistent container (data volume, generated sa password kept in the AppHost's user secrets) and the service as a .NET project (debuggable), with the Aspire dashboard for logs, traces and metrics. It needs Docker, the .NET 10 SDK, the local model (above) and ffmpeg on PATH for audio.

# Once: where your local model snapshot is
dotnet user-secrets set Parameters:model-path ~/models/embeddinggemma-2-onnx --project Pixelbadger.Toolkit.Rag.AppHost

dotnet run --project Pixelbadger.Toolkit.Rag.AppHost     # or: aspire run

The dashboard URL is printed at startup; the service listens on http://localhost:8080 as before. Without the secret, the dashboard asks for model-path. appsettings.Development.json forces exact vector search locally. The connection string is injected as Rag__ConnectionString.

From source (.NET 10 SDK)

export PBRAG_CONNECTION_STRING='Server=localhost,1433;User Id=sa;Password=Pbrag_Dev_Pass1!;TrustServerCertificate=True;Encrypt=False'
export PBRAG_MODEL_PATH=~/models/embeddinggemma-2-onnx

dotnet run --project Pixelbadger.Toolkit.Rag --urls http://localhost:8080

Running from source serves the API only, unless Pixelbadger.Toolkit.Rag/wwwroot holds a built UI; for UI work run the Vite dev server alongside it (see Development and testing).

The service refuses to start (with a clear message and exit code 1) when the connection string or model path is missing, or the model directory does not exist. Validation never loads the model or connects to SQL.

One instance per database. Ingest processes one job at a time and the code assumes it is the only instance: a delete cancels the running job through an in-process registry. A job left Processing by a crashed or restarted process is delivered again when its bus message lock expires (Rag:Ingest:LeaseSeconds). Run exactly one instance per database; multiple instances are not supported.

Deploying to Azure

The same AppHost describes the Azure deployment. azd up (or aspire deploy) provisions:

Resource	Details
Azure Container Apps environment	Consumption workload profile, Log Analytics, Aspire dashboard, Azure Container Registry.
Container App rag	The repository's Dockerfile image (includes ffmpeg). 4 vCPU / 8 GiB (the Consumption profile maximum). Scales to zero, at most one replica (one instance runs the ingest consumers). A request wakes it; it stays up while it serves requests or ingest has work (see below), then scales to zero. External HTTPS ingress on port 8080.
Storage queue ingest-active	The ingest keep-alive marker, watched by the app's queue scale rule. The app gets Storage Queue Data Contributor only.
Azure Files share mounted at /data	Holds the model (/data/models/embeddinggemma-2-onnx). Mounted as the container's app user.
Azure SQL Database ragdb	The free offer (General Purpose serverless, auto-pauses when idle), set to bill overage rather than pause for the rest of the month once the free allowance is used. Entra ID only: the app connects with its user-assigned managed identity (Authentication="Active Directory Default"), which the deployment adds to the database as db_owner so migrations can run at startup.

az login && azd auth login
export PBRAG_MODEL_PATH=~/models/embeddinggemma-2-onnx   # lets the postprovision hook upload the model
azd up                                                   # pick a subscription, location and environment name

The model. The service never downloads models, so the model snapshot (q8 files) has to be uploaded to the file share once. azd up does that in its postprovision hook when PBRAG_MODEL_PATH is set (it skips the upload when the q8 model is already there; the marker is onnx/audio_encoder_quantized.onnx, so the first run after upgrading to 5.0 uploads the q8 files next to the old fp32 ones, which are no longer used and can be deleted from the share). Otherwise, or with aspire deploy, run it yourself after provisioning (needs the Azure CLI, signed in):

AZURE_RESOURCE_GROUP=rg-<environment> scripts/upload-model.sh ~/models/embeddinggemma-2-onnx   # --force re-uploads

Until the model is there the app exits at startup ("Model directory ... not found") and Container Apps keeps restarting it; it comes up on its own once the upload finishes (or restart the revision, as the script prints).

Things to know:

• There is no authentication. The ingress is public, so anyone with the URL can upload, delete and search. Restrict it before putting real data there (for example Container Apps IP restrictions or built-in authentication, or set the ingress to internal).
• Scale to zero and ingest. Ingest runs inside the app. While a job is queued or running the app keeps a marker message in the ingest-active queue, and a queue-length scale rule keeps the replica running until no job is active and the marker is cleared, so a long ingest is not cut off when HTTP traffic stops. If the marker cannot be written (storage unreachable) the app may scale down mid-ingest: nothing is lost, the job is delivered again on the next start once its bus message lock has expired (Rag:Ingest:LeaseSeconds, 10 minutes by default). A marker left by a crashed process expires after 2 hours, or is cleared as soon as the app next goes idle.
• Cold starts. The first request after an idle period starts the container, resumes the database (it auto-pauses when nothing uses it) and loads the model from the file share: expect it to take a while, and MCP clients may time out on that first call.
• Cost. Idle costs nothing for compute; you pay per second while the app is awake (after the Consumption plan's monthly free grant), plus the database beyond its free allowance, the file share and the container registry (Basic tier, a small fixed monthly cost).
• Deployments briefly overlap. Container Apps starts a new revision before it stops the old one, so for a short time two instances share the queue. Avoid deploying while ingest jobs are running (the app stays up while the queue has work, so this means after the queue drains); a job the old instance was processing is delivered again by the new one once the job's bus message lock expires.

Configuration

Configuration is read from the Rag section: appsettings.json, environment variables (Rag__ConnectionString, Rag__Ingest__MaxFilesPerRequest, ...), or command-line arguments (--Rag:ModelPath=...).

Key	Fallback env var	Default	Description
Rag:ConnectionString	PBRAG_CONNECTION_STRING		Required. SQL Server 2025 / Azure SQL connection string.
Rag:ModelPath	PBRAG_MODEL_PATH		Required. Local EmbeddingGemma 2 ONNX snapshot directory (must exist).
Rag:ExactVectorSearch		false	Always use exact VECTOR_DISTANCE instead of the approximate vector index.
Rag:ApplyMigrationsOnStartup		true	Apply EF Core migrations when the host starts, before the message bus starts.
Rag:Ingest:MaxFileSizeBytes		10 MiB	Largest accepted single file. Also drives the multipart and request-body limits (MaxFilesPerRequest x MaxFileSizeBytes plus slack).
Rag:Ingest:MaxFilesPerRequest		100	Most files in one POST /api/documents request (each becomes its own document and job).
Rag:Ingest:MaxChunkCharacters		20000	Largest text chunk. A request may lower it, never raise it.
Rag:Ingest:MaxAttempts		3	How many times a job may be started before it is marked Failed: a retry after an infrastructure error or a redelivery after a crash each count (a problem with the file fails it at once).
Rag:Ingest:LeaseSeconds		600	Bus message lock duration: a job whose process died (crash, restart) is delivered again once the lock expires. The lock is not renewed, so this must exceed the longest ingest.
Rag:Ingest:PollIntervalSeconds		2	How often the bus consumers poll the message table when idle (a new upload's event is forwarded from the outbox at once, then picked up on the next poll). Also the outbox sender's idle sleep and the base backoff after an infrastructure error.
Rag:Ingest:CancelTimeoutSeconds		30	How long DELETE /api/documents/{id} waits for the document's running job to stop before answering 409.
ConnectionStrings:ingest-active			Optional. Azure Storage queue for the ingest keep-alive marker (scale-to-zero hosting). Set by the Aspire AppHost; without it the keep-alive does nothing.

Search data lives only in the SQL database. Rag:IndexPath and PBRAG_INDEX_PATH are no longer read; delete any old index directory.

HTTP API

Errors are RFC 9457 problem details (application/problem+json). Unexpected failures return a generic 500; the details are only in the server log. There is no authentication: put the service behind a reverse proxy or network boundary of your own.

Method	Route	Description
POST	/api/documents	Upload one or more files (multipart); creates a document and queues a job for each.
GET	/api/documents/{id}	A document with the status of its latest job.
POST	/api/documents/{id}	Re-ingest a document with a new version of its file.
DELETE	/api/documents/{id}	Cancel the document's active job and delete the document.
GET	/api/documents/{id}/content	The document's source file (the version that is currently indexed).
GET	/api/jobs	Ingest jobs, newest first, paginated and filterable by status.
POST	/api/query	Vector search.
any	/mcp	MCP Streamable HTTP (Search tool).
GET	/health	Liveness (touches neither SQL nor the model).
GET	/, /query, /ingest	Browser UI (any other extension-less path outside /api, /mcp and /health also returns the app; unknown /api/* routes stay 404).

Documents

Each document has a series of ingest jobs; one job is one file for one document. A document is addressed by its server-assigned GUID, returned when it is created. The DocumentDto returned by the endpoints:

{
  "documentId": "0f8fad5b-d9cb-469f-a165-70867728950e",
  "path": "docs/guide.md",
  "title": "guide.md",
  "modality": "Text",
  "indexStatus": "Queued",
  "chunkCount": 0,
  "updatedAtUtc": "2026-10-06T12:00:00Z",
  "latestJob": {
    "jobId": "3c8fad5b-d9cb-469f-a165-70867728950e",
    "status": "Queued",
    "attempts": 0,
    "createdAtUtc": "2026-10-06T12:00:00Z",
    "startedAtUtc": null,
    "completedAtUtc": null,
    "error": null
  }
}

• indexStatus: Queued, Processing, Indexed or Failed; it follows the latest job. chunkCount is the number of searchable chunks right now (the previous version stays searchable while a re-ingest is queued).
• latestJob.status: Queued, Processing, Succeeded, Skipped (the file had no content, so no chunks), or Failed (see error; a problem with the file is recorded once, an infrastructure failure is retried up to MaxAttempts times).
• Uploaded bytes are stored on the job until it finishes and are then discarded from it; the job history is kept. When a job succeeds (or is Skipped), its bytes become the document's canonical source, in the same SQL transaction that replaces the document's chunks, so the downloadable file and the searchable chunks are always the same version. path, title and modality also change only then. A failed job leaves the previous source, metadata and chunks as they were.
• A crash or restart is safe: the job is delivered again once its bus message lock expires (Rag:Ingest:LeaseSeconds) and processed again.

Create documents (batch)

Send one or more files parts. Each part's filename is the logical path of the new document. It is metadata (it becomes the document's path and title), so give it a meaningful relative path such as docs/guide.md: no .., no leading / or drive letter, no empty segments. Paths are not unique: uploading the same path twice creates two documents. An optional maxChunkCharacters field lowers the chunk limit.

curl -i -X POST http://localhost:8080/api/documents \
  -F "files=@./docs/guide.md;filename=docs/guide.md" \
  -F "files=@./photos/mars.jpg;filename=photos/mars.jpg" \
  -F "files=@./talks/intro.mp3;filename=talks/intro.mp3"

The whole request is validated before anything is stored: at least one file, at most MaxFilesPerRequest, each at most MaxFileSizeBytes, a supported extension and a safe path. Any violation returns 400 and creates nothing. On success you get 202 Accepted and { "documents": [ DocumentDto, ... ] } in upload order (for a single file the response also has Location: /api/documents/{id}). Poll GET /api/documents/{id} until latestJob.status is terminal:

curl http://localhost:8080/api/documents/0f8fad5b-d9cb-469f-a165-70867728950e

Re-ingest a document

POST /api/documents/{id} with exactly one files part (400 otherwise) uploads a new version. It is a POST, not a PUT, because it queues a job rather than replacing the document immediately. The file's name becomes the document's new path once the new version has been indexed; until then the document (its path, chunks and source file) stays the previous version.

curl -i -X POST http://localhost:8080/api/documents/0f8fad5b-d9cb-469f-a165-70867728950e \
  -F "files=@./docs/guide-v2.md;filename=docs/guide.md"

• 202 with the DocumentDto (and Location) when the new version is queued. If the document already had a queued job, that job's content is replaced in place instead of queueing a second one.
• 409 while the document's job is in progress: wait for it to finish, then upload again.
• 404 for an unknown id.

Delete a document

curl -i -X DELETE http://localhost:8080/api/documents/0f8fad5b-d9cb-469f-a165-70867728950e

Cancels the document's active job (queued or running; a running one is stopped and awaited), then deletes the document: its SQL rows (chunks and jobs). 204 on success, 404 for an unknown id, 409 if the running job did not stop within Rag:Ingest:CancelTimeoutSeconds (nothing is deleted; retry).

Download a document's source

curl -OJ http://localhost:8080/api/documents/0f8fad5b-d9cb-469f-a165-70867728950e/content

Returns the bytes of the version that is currently indexed, read from SQL, with its media type (text/markdown, image/png, audio/mpeg, ...) and Content-Disposition: inline; filename="guide.md" (the last segment of the document's path). ?download=true makes it attachment. Range requests are supported (audio seeking). 404 when the document does not exist or has never been ingested successfully, including documents uploaded before 4.1 (re-upload them to make their source available).

Jobs

curl 'http://localhost:8080/api/jobs?page=1&pageSize=25&status=Failed'

Parameter	Default	Description
page	1	1-based page number.
pageSize	25	1 to 100.
status		Optional filter: Queued, Processing, Succeeded, Skipped or Failed (case-insensitive).

Jobs are ordered newest first (createdAtUtc, then job id). Invalid parameters return a 400 validation problem.

{
  "jobs": [
    {
      "jobId": "019a0c1e-...",
      "documentId": "019a0c1e-...",
      "path": "docs/guide.md",
      "status": "Succeeded",
      "attempts": 1,
      "maxChunkCharacters": 20000,
      "sizeBytes": 48321,
      "chunkCount": 12,
      "createdAtUtc": "2026-10-07T00:00:00Z",
      "startedAtUtc": "2026-10-07T00:00:01Z",
      "completedAtUtc": "2026-10-07T00:00:04Z",
      "error": null
    }
  ],
  "page": 1,
  "pageSize": 25,
  "totalCount": 137,
  "totalPages": 6
}

path and chunkCount are the job's own (its uploaded path and the chunks it produced), not the document's current values. File contents are never returned.

Query

curl -X POST http://localhost:8080/api/query \
  -H 'Content-Type: application/json' \
  -d '{ "query": "why is Mars red", "maxResults": 5, "documentIds": ["0f8fad5b-d9cb-469f-a165-70867728950e"] }'

Field	Default	Description
query (required)		Query text (up to 4096 characters).
maxResults	10	1 to 100 documents (one result per document).
documentIds		Restrict results to these documents (GUIDs, up to 100). A value that is not a GUID returns 400.

A blank query or invalid argument returns 400. The response is { "results": [...] }; each result has score (cosine similarity, 1 - cosine distance; higher is more similar), chunkId, documentId, sourcePath (the document's path), sourceFile, ordinal, modality (Text, Image or Audio), locatorStart / locatorEnd (character offsets for text, milliseconds for audio, null for images), content (text chunks only).

MCP server

/mcp is a Streamable HTTP MCP endpoint (stateless: no session affinity needed) exposing the Search tool only.

Client configuration

Claude Code:

claude mcp add --transport http rag-search http://localhost:8080/mcp

Other clients that take a JSON config:

{
  "mcpServers": {
    "rag-search": {
      "type": "http",
      "url": "http://localhost:8080/mcp"
    }
  }
}

Tool: Search

Parameter	Type	Description
query	string, required	The search query.
maxResults	int, default 5	Maximum number of results (1 to 100), one per document.
documentIds	string[], optional	Restrict results to specific documents (GUIDs). A value that is not a GUID is returned as a tool error.

There is no search-mode parameter: every search is a vector search. Results carry the chunk id, document id, source, modality, locator and content. All returned content is framed as untrusted document text: clients must treat it as data, not instructions. Invalid arguments return their message as a tool error; unexpected failures return a generic error and are logged.

Supported content

Kind	Extensions	How it is indexed
Text	.txt, .md	Paragraph chunks (.txt) or header sections (.md); one chunk per row, with character-offset locators. Embedded with the EmbeddingGemma document prompt.
Image	.png .jpg .jpeg .webp .gif .bmp .tif .tiff	One chunk per image, embedded by the vision encoder (not OCR'd). Matched by text queries; results show [image].
Audio	.wav .mp3 .m4a .flac .ogg .opus .aac	About 30-second windows, one chunk each, with millisecond locators. Requires ffmpeg. Results show [audio mm:ss–mm:ss].

Video and images embedded inside documents are not supported. Text, image and audio chunks are ranked together by vector similarity.

How it works

Ingest:  POST /api/documents (multipart) -> validate -> SQL: Document (Queued) + job with the file bytes, one transaction -> 202 + ids
         job consumer: begin job -> temp file -> modality routing (extension)
              text : reader -> chunker -> EmbeddingGemma 2 (text)
              image: decode -> patchify -> vision encoder -> EmbeddingGemma 2
              audio: ffmpeg -> log-mel windows -> audio encoder -> EmbeddingGemma 2
          -> SQL Server (Chunk + vector(256), replacing the document's chunks; the same transaction makes the job's bytes the document's canonical source)
         -> job done (bytes discarded from the job) -> vector index built once the queue is idle
Delete:  DELETE /api/documents/{id} -> cancel the running job and wait -> delete SQL rows (cascade)

Search:  query -> embed -> SQL vector top-N (cosine) -> hydrate from SQL -> results

Domain model. A Document is one uploaded file; it has an integer primary key and a unique Guid global id assigned by the server when it is created (a version 7 GUID; the path plays no part in it). A Document has many Chunks and many ingest jobs; a chunk has an integer clustered primary key, a unique Guid global id (shown to users as the chunk id), the document foreign key, its ordinal, modality, locator range, text (text chunks only) and the 256-d embedding. Chunks are stored in dbo.Chunks_EG2Q8_256. The EG2Q8_256 suffix names the model (EmbeddingGemma 2, q8) and dimension; changing either means a new table and a full re-embed.

Vector search. The query is embedded and the n nearest documents by cosine distance are fetched from SQL, each represented by its nearest chunk, so one long document cannot fill the results with its own chunks (approximate DiskANN VECTOR_SEARCH over-fetches candidate chunks when the index exists and falls back to exact VECTOR_DISTANCE when they cover fewer than n documents; otherwise exact). Results are hydrated from SQL and keep that order; score is 1 - distance.

Embeddings. Model id embeddinggemma-2-q8@256: the q8 ONNX graphs, 256 dimensions (the first 256 of the model's Matryoshka output, re-normalised to unit length). Queries and documents use EmbeddingGemma's asymmetric prompts.

Job queue and events. dbo.IngestJobs holds one row per job, linked to its document (a document delete cascades to its jobs). Every status change is a JobStatusChanged event written to the SlimMessageBus outbox in the same SQL transaction as the change, and delivered over the SQL transport (BusMessages* tables) to two in-process subscriptions: ingest runs a job when it becomes Queued, and vector-index runs when a job finishes and no job is active (it builds the vector index if it is due, purges delivered messages older than a day and lets a scale-to-zero host go idle). Each subscription has one consumer working one message at a time. The bus starts after migrations, once SQL accepts connections. Deleting a document cancels its running job through an in-process registry and waits for it to stop before removing anything; a job whose document disappears meanwhile stops quietly.

Browser UI. Pixelbadger.Toolkit.Rag/ClientApp is a React + TypeScript app (Vite, Tailwind, shadcn/ui components, TanStack Query). The container build compiles it in a Node stage and copies it into the image's wwwroot; ASP.NET serves it from the same origin as the API, so there is no CORS and no API URL to configure.

Privacy. Chunk text, document paths and the uploaded source files are stored in SQL Server in plaintext. Protect the database and avoid ingesting secrets unless that is acceptable.

Development and testing

dotnet build
dotnet test

Browser UI (Node 22):

cd Pixelbadger.Toolkit.Rag/ClientApp
npm ci
npm run dev            # Vite dev server; proxies /api, /mcp and /health to http://localhost:8080 (run the service there)
npm run lint
npm run test -- --run  # Vitest + Testing Library + MSW
npm run build          # type-check and build into dist/

• Tests use xUnit, FluentAssertions and Moq. SQL-backed tests (store, pipeline, ingest queue, message bus) run against a SQL Server 2025 container started by Testcontainers (Docker required; shared via the SqlServer collection fixture).
• To reuse an existing SQL Server instead of starting a container, set PBRAG_TEST_SQL_CONNECTION_STRING to its connection string. The tests do not use the Aspire AppHost: they start their own container, so the only local requirement is Docker, as for Aspire.
• API and MCP tests host the real application in-process with WebApplicationFactory<Program> (mocked search service, queue and store, no hosted services, no SQL, no model).
• Embedding-dependent tests use a deterministic mock embedding service. Golden tests that compare against the real ONNX model run only when PBRAG_MODEL_PATH points at a local model; otherwise they are skipped. Reference vectors are generated with the transformers.js scripts in tools/golden, against the same q8 graphs.
• CI (.github/workflows) runs on .NET 10 and Node 22. Pull requests lint, test and build the UI, build and test the .NET solution, build the container image without pushing it and, when they change Pixelbadger.Toolkit.Rag/, require <Version> to be higher than the latest v* git tag. A merge to master builds, tests and pushes the image to GHCR (tags <Version>, latest, sha-<short>), then creates the v<Version> tag; the push is refused if that tag already exists, so bump <Version> for every release. The registry login uses the built-in GITHUB_TOKEN.

Evaluations

EVALS.md and docs/bm25-vs-hybrid-analysis.md are historical (pre-2.0, OpenAI embeddings, the old CLI, and an evaluation of BM25 and hybrid search that predates the vector-only search of 4.0) and kept for context.

License

MIT