pkcs11-fuzzer

Continuous fuzzing for the PKCS#11 stack: OpenSSL + libp11 + SoftHSM2 + OpenSC.

The repo builds its own instrumented target libraries, restores a reusable SoftHSM token snapshot, runs libFuzzer harnesses across several layers of the stack, and includes tooling for coverage, crash triage, patch generation, and cleanup.

Overview

This project fuzzes four distinct surfaces:

Surface	What is exercised
PKCS#11 C API	Direct calls into SoftHSM2 (C_CreateObject, C_FindObjects, C_Sign, C_Decrypt, C_GenerateKeyPair, session/auth APIs, state-machine sequences, concurrency, token lifecycle)
OpenSSL via libp11	EVP and TLS operations routed through the libp11 engine into a PKCS#11 module
OpenSC PKCS#11 module	opensc-pkcs11.so initialization, slot/mechanism enumeration, and attribute-template parsing
Coverage and crash workflow	Corpus management, replay against coverage builds, crash minimization, analysis, and patch generation

When a crash is found, the repo can:

1. capture the crashing input
2. triage and deduplicate it
3. minimize the reproducer
4. generate a maintainer-oriented report and patch template

Repository Layout

pkcs11-fuzzer/
├── build-scripts/   # Build orchestration and token initialization
├── builds/          # Installed trees: libfuzzer, tsan, coverage
├── harnesses/       # libFuzzer targets and shared harness support
├── fuzzing/         # Runtime scripts: launch, coverage, minimize, notify
├── tools/           # Analysis, reporting, cleanup, workflow helpers
├── corpus/          # Seed and evolved corpora per harness
├── crashes/         # Raw crashes, deduplicated inputs, analysis output
├── coverage/        # HTML coverage reports and trending logs
├── token-template/  # Read-only SoftHSM token snapshot
└── src/             # Cloned upstream source trees (generated by builds)

Build Trees

Tree	Purpose	Key flags
builds/libfuzzer	Main fuzzing tree	ASan + UBSan + libFuzzer
builds/tsan	Race detection	ThreadSanitizer
builds/coverage	Source coverage replay	-fprofile-instr-generate -fcoverage-mapping

All three trees are optional in the sense that you can build only what you need, but meaningful library coverage requires the coverage tree.

Prerequisites

Ubuntu packages used by the current scripts:

sudo apt-get install -y \
    git make autoconf automake libtool \
    pkg-config gettext xsltproc \
    build-essential \
    clang lld \
    libsqlite3-dev libpcsclite-dev \
    libboost-filesystem-dev \
    zlib1g-dev

Notes:

1. No system OpenSSL or SoftHSM2 install is required.
2. The build scripts use the system clang in PATH.
3. Disk usage is several GB once all trees and corpora exist.

Quick Start

Build the default fuzzing tree:

git clone <this-repo> pkcs11-fuzzer
cd pkcs11-fuzzer

./build-scripts/build-all.sh
make -C harnesses
make -C harnesses smoke-test

Typical build for current upstream fuzzing work:

./build-scripts/build-all.sh \
    --upstream-softhsm2 \
    --upstream-libp11 \
    --upstream-opensc \
    --coverage-tree

This intentionally keeps OpenSSL on the pinned version. OpenSSL upstream/4.x is currently incompatible with the rest of the stack often enough that it is not the recommended default for fuzzing this project.

Start a fuzzing session:

bash fuzzing/run-libfuzzer.sh --time 3600

Or use the long-running orchestrator:

screen -dmS fuzz bash fuzzing/continuous.sh
screen -r fuzz

fuzzing/continuous.sh now also snapshots crash-time context and runs analysis automatically. For each new artifact in crashes/raw/, it can capture:

1. fuzzing-session.json
2. fuzzing-session.env
3. recent harness and libFuzzer logs
4. matching /tmp/asan.* sidecar reports

Component Roles

Component	Role in this project
OpenSSL	Crypto backend for SoftHSM2 and static link target for the TLS/libp11 harnesses
SoftHSM2	Main PKCS#11 implementation under test
libp11	OpenSSL engine/provider bridge into PKCS#11
OpenSC	Separate PKCS#11 implementation and tooling
p11-kit	Not used; SoftHSM2 is built without it so installs stay self-contained

The harnesses usually load modules directly by path, for example:

harness -> dlopen(builds/libfuzzer/lib/softhsm/libsofthsm2.so)

That is why p11-kit is intentionally excluded.

Harnesses

There are currently 20 harnesses:

Direct PKCS#11 / SoftHSM2 harnesses

Harness	Focus
pkcs11_sign_fuzz	C_SignInit + C_Sign
pkcs11_decrypt_fuzz	C_DecryptInit + C_Decrypt
pkcs11_findobj_fuzz	C_FindObjectsInit + C_FindObjects + attribute retrieval
pkcs11_createobj_fuzz	C_CreateObject, C_GetAttributeValue, C_CopyObject, C_DestroyObject
pkcs11_session_fuzz	C_OpenSession, C_CloseSession, C_Login, C_Logout, operation-state APIs
pkcs11_state_machine_fuzz	Mixed multi-call PKCS#11 bytecode-style sequences
pkcs11_concurrency_fuzz	Concurrent session and crypto activity for race hunting
pkcs11_token_reset_fuzz	Per-iteration isolated C_InitPIN, C_SetPIN, C_InitToken
pkcs11_wrap_fuzz	C_WrapKey, C_UnwrapKey, C_DeriveKey
pkcs11_attrs_fuzz	C_GetAttributeValue, C_SetAttributeValue, C_CopyObject
pkcs11_digest_fuzz	Single- and multi-part digest APIs
pkcs11_multipart_fuzz	multipart sign/verify/encrypt/decrypt APIs
pkcs11_keygen_fuzz	C_GenerateKeyPair and C_GenerateKey
pkcs11_encrypt_fuzz	C_EncryptInit + C_Encrypt
pkcs11_verify_fuzz	C_VerifyInit + C_Verify
pkcs11_random_fuzz	random generation and seeding APIs
pkcs11_hmac_fuzz	HMAC and CMAC paths

Higher-layer harnesses

Harness	Focus
libp11_evp_fuzz	OpenSSL EVP -> libp11 -> PKCS#11
opensc_pkcs11_fuzz	Multi-step OpenSC PKCS#11 command stream: slot, mechanism, session, object, login/logout, reinit
tls_pkcs11_fuzz	Multi-step client/server TLS script using a PKCS#11-backed server key

Common harness model:

byte 0      selector / opcode / mechanism choice
byte 1..N   payload, parameters, or operation stream

That keeps mutation focused on semantically meaningful paths instead of spending most cycles in immediate reject logic.

Token Model

Most PKCS#11 harnesses call pkcs11_init() from harnesses/common.h. That setup:

1. restores token-template/ into a per-process temp directory
2. opens a PKCS#11 session against SoftHSM2
3. logs in with the default fuzz user PIN
4. reuses pre-created keys for speed

Default token contents include:

1. RSA key pair
2. EC P-256 key pair
3. AES key
4. HMAC-capable secret material used by some harnesses

pkcs11_token_reset_fuzz is the exception: it intentionally creates an isolated token copy per iteration so token-management APIs can be exercised without poisoning later iterations.

Running the Fuzzers

Build once:

./build-scripts/build-all.sh
make -C harnesses

Generate or refresh the seed corpus:

make -C harnesses seeds

The seed recipes are written to emit real binary bytes even when make runs under /bin/sh. This matters because many harnesses use selector- and length-driven formats, and textual \xNN seeds degrade mutation quality and coverage growth.

Run all harnesses in parallel:

bash fuzzing/run-libfuzzer.sh

Run for a fixed time:

bash fuzzing/run-libfuzzer.sh --time 3600

Useful environment notes:

1. run-libfuzzer.sh sets ASAN_OPTIONS, UBSAN_OPTIONS, and LSAN_OPTIONS.
2. Crash artifacts are written to crashes/raw/.
3. Per-harness logs are written to coverage/<harness>.log.
4. The active fuzzing runtime environment is also saved to coverage/fuzzing-runtime.env.

Coverage Workflow

Real-time fuzz growth

Watch the libFuzzer counters while fuzzing:

tail -f coverage/pkcs11_state_machine_fuzz.log | grep -E 'NEW|pulse|DONE'

Useful fields:

Field	Meaning
cov	edge/block coverage observed by libFuzzer
ft	feature count, usually more sensitive than cov
corp	corpus size and bytes

Source coverage replay

Build the coverage tree first:

./build-scripts/build-all.sh --coverage-tree

Then replay the corpus through coverage-instrumented binaries:

make -C harnesses coverage
bash tools/show-coverage

Artifacts:

1. coverage/<harness>/index.html for per-harness HTML reports
2. coverage/coverage.log for trend summaries
3. bash tools/show-coverage --totals for per-harness totals
4. bash tools/show-coverage --harness <name> for one harness view

Notes:

1. tools/show-coverage reports the best measured harness per component from coverage/coverage.log.
2. libp11 and opensc coverage only move meaningfully when their dedicated harnesses are fuzzed and replayed into the coverage tree.
3. Binary seed quality matters a lot for these results; malformed textual seeds undercut opcode-driven harnesses badly.

About default.profraw

default.profraw is a raw LLVM coverage profile left behind by a coverage-instrumented run when LLVM_PROFILE_FILE is not set.

It is safe to delete after coverage has been generated.

tools/cleanup.sh --artifacts now removes stray *.profraw and *.profdata files, including a root-level default.profraw.

Crash Workflow

Crash inputs land in:

crashes/raw/

Recommended workflow:

./tools/analyze.sh crashes/raw/<crash>

Analyze everything currently in crashes/raw/:

./tools/analyze.sh --all

The analyzer now tries harder than a single replay:

1. replays each crash multiple times before calling it non-reproducible
2. writes repro_attempts.log
3. emits a non-repro report.md and analysis.json instead of stopping early
4. prefers richer captured asan-report-*.log sidecars when stdout only showed a generic SEGV or DEADLYSIGNAL

Or use the full patch lifecycle tool:

python3 tools/patch_workflow.py crashes/raw/<crash>

That workflow can:

1. verify reproduction
2. minimize the crash input
3. classify likely false positives
4. generate a candidate patch
5. rebuild the affected component
6. re-run the reproducer to verify the fix

Relevant directories:

Path	Meaning
crashes/raw/	raw crash artifacts from libFuzzer
crashes/deduplicated/	one input per unique stack key
crashes/analysis/	reports, reproducers, replay logs, sidecar ASan logs, fuzzing-session snapshots, generated patches

Build Notes

Pinned vs upstream builds

Pinned versions are defined in build-scripts/common.sh.

Use upstream component heads selectively:

./build-scripts/build-all.sh --upstream-softhsm2 --upstream-libp11
./build-scripts/build-all.sh --upstream-opensc --coverage-tree

Recommended full command for day-to-day use:

./build-scripts/build-all.sh --upstream-softhsm2 --upstream-libp11 --upstream-opensc --coverage-tree

Avoid using upstream OpenSSL as the default. OpenSSL 4.0/upstream has known compatibility problems with other components in this stack, so the repo should normally keep OpenSSL pinned while moving SoftHSM2, libp11, and OpenSC forward.

When mixing upstream heads, always run:

make -C harnesses smoke-test

Incremental rebuilds

Rebuild one tree:

rm -rf builds/libfuzzer
./build-scripts/build-all.sh

Rebuild one component in one tree:

rm -rf src/softhsm2/build-libfuzzer
bash build-scripts/03-build-softhsm2.sh libfuzzer

Sanitizer details

This repo intentionally disables some checks that are noisy across dlopen boundaries and C-style PKCS#11 function tables:

1. -fno-sanitize=vptr
2. -fno-sanitize=function
3. -fno-sanitize-recover=undefined

OpenSSL is built static, then linked into instrumented shared libraries so the full stack stays visible to sanitizers and coverage tools.

For tls_pkcs11_fuzz, use ASAN_OPTIONS=...:detect_odr_violation=0 during smoke tests and fuzzing. The harness links static OpenSSL while SoftHSM also loads an instrumented OpenSSL-backed library, and ASan's ODR check is noisy in that setup even when the target logic is fine.

OpenSC and libp11 Coverage Caveat

SoftHSM2 is the strongest-covered component today because most harnesses call it directly. libp11_evp_fuzz, tls_pkcs11_fuzz, and opensc_pkcs11_fuzz are the main paths for growing non-SoftHSM coverage.

Recent harness changes improve that situation:

1. opensc_pkcs11_fuzz is now a command-stream harness instead of one-op-per-input.
2. tls_pkcs11_fuzz now drives a client/server SSL script instead of a single SSL_do_handshake() call.
3. pkcs11_state_machine_fuzz now uses longer multi-session command streams, which indirectly improves OpenSSL reach through richer SoftHSM crypto states.

If OpenSC coverage remains low, that is expected unless you put time into:

1. richer OpenSC seeds
2. more OpenSC-specific harness logic
3. environments that expose deeper smart-card style behavior

Cleanup

Show a summary only:

bash tools/cleanup.sh --summary

Artifacts only:

bash tools/cleanup.sh --soft

Artifacts plus target builds:

bash tools/cleanup.sh --hard

Full teardown:

bash tools/cleanup.sh --all

Dry run:

bash tools/cleanup.sh --dry-run --all

Archive findings before deleting:

bash tools/cleanup.sh --archive=/path/to/handoff --summary

--artifacts removes:

1. crash outputs
2. coverage reports
3. compiled harness binaries
4. token snapshot
5. stray *.profraw and *.profdata files
6. corpus files

Troubleshooting

Build fails trying to write into /usr/share/p11-kit

SoftHSM2 should be built with --without-p11-kit.

SoftHSM2 loses ECC support during configure

Delete the component build dir and rebuild. Autoconf feature probes can be sensitive to sanitizer settings.

rm -rf src/softhsm2/build-libfuzzer
bash build-scripts/03-build-softhsm2.sh libfuzzer

OpenSC configure complains about reader-driver selection

The OpenSC build needs exactly one valid driver configuration. The current build scripts handle that; if you changed them, re-check the OpenSC configure flags.

Coverage looks like 0.00%

Usually one of these is true:

1. builds/coverage/ was not built
2. the harness aborted before useful replay
3. you are looking at harness-only fallback output

Useful Files

Path	Description
build-scripts/build-all.sh	master build entry point
build-scripts/common.sh	pinned versions, flags, clone policy
build-scripts/init-token.sh	token snapshot creation
harnesses/Makefile	build, seeds, smoke-test, and coverage targets
harnesses/common.h	shared SoftHSM session/token bootstrap
fuzzing/run-libfuzzer.sh	launches all harnesses in parallel
fuzzing/gen-coverage.sh	coverage replay and HTML report generation
tools/show-coverage	compact coverage summary
tools/analyze.py	crash analysis and minimization
tools/patch_workflow.py	full analyze -> patch -> rebuild -> verify flow
tools/cleanup.sh	archive and cleanup workflow

Typical Session

./build-scripts/build-all.sh --upstream-softhsm2 --upstream-libp11 --upstream-opensc --coverage-tree
make -C harnesses
make -C harnesses seeds
bash fuzzing/run-libfuzzer.sh --time 3600
make -C harnesses coverage
bash tools/show-coverage

That is the shortest end-to-end path from checkout to fuzzing to coverage.