Rawblob

Client-side forensic file carving & reconstruction.

Rawblob scans raw byte streams and documents for files hidden inside them — whether embedded directly in a binary blob or encoded as Base64 inside a text document — and reconstructs them for inspection, right in your browser. Nothing is uploaded or stored server-side: parsing, carving, entropy analysis, and decoding all run locally via Web Workers.

What it does

• Carves embedded files out of raw binary blobs. Rather than only classifying what a whole file is, Rawblob scans every offset of a buffer for known file signatures, so a file concatenated or hidden inside another file is found and extracted independently — not just the outermost one.
• Validates both header and footer before trusting a match. A short header alone (2–4 bytes) can coincidentally occur inside high-entropy compressed data, especially in containers like PDFs that are full of Flate/DCT-compressed streams. Rawblob corroborates structurally where a format allows it (JPEG requires a real marker code immediately after the SOI bytes, not just any byte; SVG requires the byte after <svg to plausibly end a tag name, not just any element that starts with those four characters) and always reports whether a standard end-of-file marker was actually located — see Signature confidence below. When no footer is found, the carve is capped at the next detected file's start offset rather than the rest of the buffer, so an unbounded format can never silently claim bytes that belong to a file found right after it.
• Corroborates PDF matches by scanning their interior structure, not just their boundary bytes. %PDF/%%EOF alone only proves the header and footer bytes exist somewhere — it says nothing about what's between them. Rawblob scans the carved range for the ASCII tokens intrinsic to PDF's object model (obj/endobj, stream/endstream, xref, trailer, /Root) and reports a confidence tier based on what it actually finds: a %PDF/%%EOF pair with no obj/endobj structure inside it is a strong signal of a coincidental match rather than a real PDF, the same class of problem the JPEG marker check above catches for images. This is adapted from published forensic research on content- based carving as a defense against file-signature obfuscation, where header/footer-only tools were shown to recover 0% of deliberately obfuscated PDFs while a content-aware approach recovered over 93%.
• Detects SVG images via an XML tag sniff, not a fixed magic number — SVG has none. Bounded by either a matching </svg> closing tag or a self-closing <svg .../> root element, and always rendered inside a script-disabled sandbox since SVG can carry active markup.
• Detects and decodes Base64-encoded payloads embedded in document text, filtering out incidental alphanumeric noise (hashes, tokens, UUIDs) so you see genuine embedded files and plaintext, not everything that happens to be base64-shaped. Plain-text formats (TXT/MD/CSV/LOG/JSON) are scanned directly; PDF and DOCX are scanned via extracted text — a PDF's visible text is stored in structured/font-encoded form in the raw bytes, not as plain ASCII, so a Base64 block pasted into a PDF or Word document's body text wouldn't be found by raw byte carving alone. Text extraction runs as a second, independent pass alongside raw-buffer carving (which is unaffected either way) via pdfjs-dist for PDF and mammoth for DOCX. The extracted text itself is also shown directly in a collapsible Extracted Document Text panel — regardless of whether any Base64 payload was found in it — so extraction has something visible to check even for the common case of a document with no embedded blobs.
• Scores every payload with Shannon entropy to flag likely encryption or compression, with confidence tiers so small samples aren't given false authority.
• Manual Signature Search. Search the loaded buffer directly for a byte pattern in hex, ASCII, or decimal — useful for verifying a specific signature, chasing a hunch, or finding something outside the built-in format database entirely.
• Never executes or live-renders anything unsafe. Executables are hex-dumped and offered only as a guarded, non-executable download. Markup formats (SVG/HTML) that could carry active code are rendered only inside a script-disabled sandboxed frame.

Directory structure

rawblob/
├── app/
│   ├── layout.tsx                # Root layout, metadata, global styles
│   ├── page.tsx                  # Home page — renders <Dashboard />
│   └── globals.css               # Design tokens, fonts, focus/motion rules
├── components/
│   ├── Dashboard.tsx             # Top-level composition + session state
│   ├── DropZone.tsx              # Drag-and-drop ingest, format/size validation
│   ├── TelemetryMatrix.tsx       # Real-time results table
│   ├── ReconstructionCanvas.tsx  # Split-screen preview + hex inspector +
│   │                             #   matched header/footer bytes
│   ├── PatternSearchPanel.tsx    # Manual hex/ASCII/decimal signature search
│   ├── ExtractedTextPanel.tsx    # Shows the actual text pulled from a
│   │                             #   PDF/DOCX/plain-text file, independent
│   │                             #   of whether any payload was found in it
│   ├── ByteRuler.tsx             # Offset ruler (full + mini position-bar)
│   └── StatusBadges.tsx          # Entropy / signature / footer-confidence /
│                                 #   render-mode badges
├── lib/
│   ├── workers/
│   │   ├── signatures.ts         # File signature DB (magic numbers, offsets,
│   │   │                         #   structural checks, end markers,
│   │   │                         #   hasStandardFooter per format)
│   │   ├── carving.ts            # Sliding-window carving engine + entropy +
│   │   │                         #   header/footer hex capture
│   │   ├── contentValidation.ts  # Content-based corroboration (PDF interior
│   │   │                         #   marker scan) — catches coincidental
│   │   │                         #   header/footer matches with no real
│   │   │                         #   structure inside
│   │   ├── base64scanner.ts      # Confidence-scored Base64 payload detector
│   │   ├── textExtraction.ts     # PDF (pdfjs-dist) / DOCX (mammoth) text
│   │   │                         #   extraction, feeding into base64scanner
│   │   ├── patternSearch.ts      # Hex/ASCII/decimal query parsing + buffer search
│   │   └── rawblob.worker.ts     # Worker entry point — ties it all together,
│   │                             #   retains the analyzed buffer for search
│   └── hooks/
│       └── useRawblobWorker.ts   # Worker lifecycle, Object URL tracking/cleanup,
│                                 #   safe render-mode classification, search calls
├── assets/
│   └── rawblob-banner.svg        # Animated README banner (SMIL, GitHub-safe)
├── postcss.config.js             # Tailwind v4 PostCSS plugin wiring
├── tsconfig.json
├── package.json
└── README.md

Quick start

Requirements: Node.js 18+ and npm.

# 1. Install dependencies
npm install

# 2. Run the dev server
npm run dev

# 3. Open the dashboard
# http://localhost:3000

To build for production:

npm run build
npm run start

If next, framer-motion, or React aren't already present in your package.json, add them before running the dev server:

npm install next react react-dom framer-motion

Note on Tailwind: this project uses Tailwind CSS v4, which is configured natively in CSS (app/globals.css — @import "tailwindcss"; plus an @theme block) rather than a tailwind.config.ts file, and uses the @tailwindcss/postcss PostCSS plugin instead of the old tailwindcss plugin + autoprefixer pairing from v3. If you see a Parsing CSS source code failed / Unknown at rule: @tailwind warning, it means the classic v3-style @tailwind base/components/utilities directives are present somewhere instead of the v4 @import "tailwindcss"; syntax — check app/globals.css matches the version in this repo.

How it works

1. Drop or select a file (TXT, PDF, DOCX, RTF, MD, CSV, LOG, JSON — up to 15MB). The size and format are validated client-side before anything is read.
2. The file's bytes are handed to a Web Worker so the UI thread never blocks, even on a full 15MB scan. The worker retains the buffer in memory afterward so Manual Signature Search can query it later without re-reading the file.
3. The worker runs a sliding-window carving pass across the entire byte range for known file signatures (images, archives, executables, audio/video containers). For each hit:
   • Header validation — where a format allows it, more than a raw byte match is required. JPEG, for example, requires the byte immediately after the SOI marker to be a real marker code (0xC0–0xFE), not an arbitrary byte — this is what rejects coincidental 3-byte matches that turn up inside compressed streams (a real false positive found during testing against a live PDF: FF D8 FF occurring by chance inside Flate-compressed content, with a structurally invalid 4th byte).
   • Footer/end-marker search — bounding the carve with a format-appropriate end marker (PNG's IEND chunk + CRC, PDF's %%EOF, ZIP's End-of-Central-Directory record, GIF's 00 3B trailer with a bare-3B fallback, JPEG's FF D9). Whether this search actually found a real footer — or fell back to the next detected file's start offset — is preserved and shown, not hidden.
   • Interior structure scan (PDF only, currently) — header and footer bytes only prove the boundaries look right; they say nothing about what's between them. For PDF, the carved range is scanned for the ASCII tokens intrinsic to its object model (obj/endobj, stream, xref, trailer, /Root), and a confidence tier is derived from what's actually found — catching the case where a %PDF/%%EOF pair matched coincidentally with no real PDF content between them.
4. Independently, the worker also scans for Base64-encoded content in the document's text, routed by format: plain-text formats (TXT/MD/ CSV/LOG/JSON) are read and scanned directly; PDF is extracted via pdfjs-dist first; DOCX via mammoth. This finds Base64 blocks sitting in a document's visible text — a case raw-buffer carving alone can't cover for PDF/DOCX, since their visible text is stored in structured/font-encoded form in the file, not as plain ASCII bytes. Extraction failure (a malformed PDF, a corrupted DOCX) is reported as its own status and never blocks the raw-buffer carving results, which already ran independently against the same file.
5. Every detected payload appears as a row in the Telemetry Matrix, with its signature, a footer-confidence badge, entropy score, size, and a position indicator — either a byte offset in the raw buffer, or a character offset in the extracted text, depending on which pass found it. These are two different scales and are never plotted against the same ruler. PDF rows carved from the raw buffer also carry a structure-confidence badge from the interior scan above.
6. Selecting a row opens it in the Reconstruction Canvas: the matched header and footer bytes in hex (so a carve can be manually verified, not just trusted), a live preview (image, text, or sandboxed markup) alongside a full hex dump, and a one-click, non-executable download.
7. Manual Signature Search (collapsed by default, below the canvas) lets you query the loaded buffer directly for a hex, ASCII, or decimal byte pattern — independent of the automatic carving pass. Selecting a result jumps to it in the canvas if it falls inside an already-carved file.

Signature confidence

Every carved file is labeled with how confident its boundaries are, shown as a badge next to its signature name in both the Telemetry Matrix and the Reconstruction Canvas:

• footer confirmed (teal) — a real end-of-file marker was located for this format; the offset range is a fact, not an estimate.
• unbounded (red) — this format has a standard footer, but it couldn't be found; the end offset falls back to either the next detected file's start offset or the rest of the buffer if nothing follows, and should be treated as an estimate, not a confirmed boundary. It will never overlap another carved file's range, though — see the fallback note under What it does above.
• no std. footer (neutral) — this format has no standardized trailing marker at all (MP3 stream frames, MP4 atom boxes, WAV's header-declared size, EXE, RAR, 7-Zip, GZIP, TAR, TIFF, WebP) — the size shown is derived from format-specific structure where possible, and a missing footer here is expected, not a red flag.
• weak sig (amber) — the header itself is short or easily-collided (e.g. MZ for PE, BM for BMP); these formats are corroborated with an additional structural check (PE walks its actual e_lfanew → PE\0\0 header pointer; BMP validates its declared file-size field) but are still flagged so a hit isn't presented with unearned confidence.

Content-based corroboration (PDF only, for now). A second, independent badge appears next to PDF hits specifically, reflecting whether real object-model structure was found inside the carved range — not just claimed by its header/footer bytes:

• structure verified (teal) — obj/endobj pairs found, plus at least one of stream/endstream/xref/trailer//Root. Genuine PDF content, high confidence.
• structure partial (amber) — obj/endobj found, but none of the other tokens. Still real structure — plausible for PDFs using compressed cross-reference streams (PDF 1.5+), which don't use the classic xref/trailer keywords at all — just with less corroborating evidence.
• structure not found (red) — no obj/endobj pairing anywhere in the range. Every real PDF with at least one object has this; its total absence is a strong signal that the %PDF/%%EOF match is coincidental rather than genuine PDF content. The Reconstruction Canvas shows the exact markers found (or their absence) and the raw obj/endobj counts for this hit.

Usability notes

• Everything stays local. No network request is made with file content at any point — the worker never has network access, and no server endpoint exists to receive uploaded bytes.
• Header and footer bytes are always shown, not just claimed. The Reconstruction Canvas displays the actual matched header and footer hex for the selected payload, so a carve can be independently verified rather than taken on faith.
• Entropy is contextualized, not just reported. Low-sample readings are marked low-n rather than presented with false precision, and a payload whose entropy doesn't match what's expected for its claimed type is flagged inconsistent — itself a useful forensic signal. Container formats with legitimately huge entropy variance (PDF, TAR) are given wide expected ranges so normal files — like a PDF embedding a compressed image — aren't falsely flagged.
• Executable content is inert by design. PE and ELF payloads are never rendered or given a live preview URL; they can only be hex-inspected or downloaded as a generic .bin file, which won't auto-execute on save.
• Markup is sandboxed. SVG and HTML-shaped payloads render inside an <iframe sandbox=""> with scripting disabled, since these formats can carry executable content of their own.
• Manual search results respect the same offset system as carving — clicking a search hit jumps to its containing carved file in the canvas when one exists, keeping the two inspection modes connected rather than siloed.
• Keyboard accessible throughout. The drop zone, every Telemetry Matrix row, and the search panel are focusable and operable via keyboard (Enter/Space), with a visible focus ring.
• Reduced motion respected. Users with prefers-reduced-motion set get instant state changes instead of the drop-zone and scan animations.

Design system

• Palette: dark graphite base with three semantic accents — amber (active/primary), teal (verified/consistent), red (high-entropy/danger) — used functionally for forensic triage rather than decoratively.
• Type: IBM Plex Sans for UI chrome, IBM Plex Mono for offsets, hex, and all byte-level data — a family designed for technical readouts.
• Signature element: the byte ruler — a tick-marked offset strip shown above the Reconstruction Canvas and as a compact position-bar per Telemetry Matrix row, so the buffer position of every payload is always visible, not buried in a table column. The same motif carries into assets/rawblob-banner.svg, the animated README banner, so the app and its branding read as one product.

Known limitations / not yet wired

• RTF text extraction is not implemented. RTF needs its own control-word parser (it's not XML/zip-based like DOCX or object-based like PDF), which this doesn't have yet — RTF files still get full raw-buffer carving, just no Base64-in-text scanning.
• Bundler resolution of pdfjs-dist's worker script hasn't been confirmed working end-to-end. GlobalWorkerOptions.workerSrc is now explicitly set via new URL('pdfjs-dist/legacy/build/pdf.worker.mjs', import.meta.url) in textExtraction.ts — a real-world regression from an earlier approach (see below) — which is the standard bundler-resolved asset pattern webpack 5+/Turbopack support, but actual resolution through Next.js's Turbopack in this project hasn't been confirmed with a live test yet. If you see a 404 or resolution error for pdf.worker.mjs, that's the next thing to debug — share the exact error and it can be fixed directly.
• PDF/DOCX text extraction's core logic was verified end-to-end against real generated files (a PDF and a DOCX, each containing a genuine embedded Base64 payload) — both libraries correctly extract text and it correctly feeds into the Base64 scanner. One specific earlier claim did NOT hold up under real-world use, worth stating plainly: leaving GlobalWorkerOptions.workerSrc unset was verified by reading pdfjs-dist's source to trigger a safe internal fallback, and that held for a simple test PDF — but real testing against a wider variety of real PDFs surfaced a No "GlobalWorkerOptions.workerSrc" specified failure for some of them, meaning some internal pdfjs code path (likely font-handling or a specific content-stream operation) references it outside the fallback's try/catch. The fix — explicitly setting workerSrc — sidesteps the issue rather than patching each internal call site, but is itself a step less fully proven than the extraction logic it wraps (see the bundler-resolution note above).
• Optional PDF font/CJK assets aren't wired up yet. pdfjs-dist ships standard_fonts and cmaps directories for full-fidelity rendering of PDFs using non-embedded standard fonts or CJK/custom character maps. Text extraction works without them for the common case (embedded fonts, Latin text) — you'll just see a console warning from pdfjs. To silence it and support more PDFs fully, copy node_modules/pdfjs-dist/standard_fonts to public/standard_fonts and node_modules/pdfjs-dist/cmaps to public/pdf-cmaps (paths already configured in textExtraction.ts to expect them there).
• WebP, TIFF, and a few other RIFF/chunked formats are marked hasStandardFooter: false and carve up to the next detected file (or the rest of the buffer if nothing follows) rather than parsing their internal chunk/size fields for a precise bound — functionally correct for classification and now safely non-overlapping with neighboring carves, but their reported byte length should still be read as an upper bound, not exact, until that internal-length parsing is added.
• Very large extracted text blobs are scanned in a single synchronous pass inside the worker; fine at the current 15MB ceiling, but worth chunking with a match cap if that ceiling is ever raised.
• Manual Signature Search runs against whatever buffer was last analyzed; it has no awareness of Base64-decoded sub-buffers once text extraction is wired up, so it will need to gain a "search scope" selector at that point.
• Content-based structural corroboration currently covers PDF only. The same pattern (scan the carved interior for tokens intrinsic to the format, not just its boundary bytes) extends naturally to other container formats — e.g. cross-checking ZIP's central directory entry count against the number of local file headers actually found — but isn't implemented yet. Detecting PDFs with deliberately obfuscated header/footer bytes (no %PDF/%%EOF to anchor on at all) is a related but separate, harder problem: it would mean scanning for trailer/ xref/obj clusters with no matching header nearby, which the current carving pass doesn't attempt since it only starts from a signature match.