TRACEMARK · POST-GENERATION LLM FORENSICS
Can plain English carry a cryptographic fingerprint?
TraceMark explores employee-level forensic attribution for LLM-generated text.
A model-agnostic Python gateway that embeds keyed statistical fingerprints into completed LLM responses using meaning-preserving linguistic choices.
The model never needs to know.
Most text-watermarking techniques operate while a model generates tokens. TraceMark instead operates on the completed response. That makes the watermark layer provider-agnostic — OpenAI, Claude and DeepSeek all work unchanged.
Same meaning. Different fingerprint.
Two secret keys make different tiny choices at the same places. The text stays natural either way — but the pattern is unique per employee.
We do not believe the "forecast" is realistic, and we will not accept the plan. The proposal covers revenue, expenses and liabilities — it really is a balanced budget. The manager's review said... "We cannot approve this today."
Interactive visualization. The production implementation uses the Python TraceMark NLP and detection engine.
// THREE MONTHS LATER
The watermarked text gets copied, pasted, re-saved, sent. Metadata dies. The text survives.
"We don't believe the "forecast" is realistic, and we won't accept the plan. The proposal covers revenue, expenses, and liabilities — ..."
The metadata is gone. The linguistic choices remain.
Illustrative detector walkthrough — the real benchmark statistics appear below.
Attribution is a statistics problem.
An unrelated fingerprint should match roughly half of the linguistic choices by chance. The correct fingerprint should match substantially more.
Detection math
- HMAC-SHA256
- HKDF-SHA256
- sentence-local opportunity IDs
- binomial significance testing
- Bonferroni multiple-testing correction
The synthetic benchmark was wrong.
A demo on hand-written sentences is easy. Real text is not. I ran the full pipeline over five hundred thousand real documents.
What real text actually contains
Synthetic demo text was far more "watermarkable" than reality. Safe linguistic choices per 100 words:
Real text was roughly 10× less watermarkable than the initial synthetic benchmark suggested.
V2 broke my original assumption: real text had roughly 10× fewer usable transformations than the synthetic benchmark. The encoding and detection machinery behaved as designed.
415,671 Enron emails later
usable watermark opportunity
in the median Enron email
of Enron emails reached the original
20-opportunity evidence threshold
words most documents need
before enough signal exists
How much text does attribution need?
Drag the two controls — the numbers are the measured attribution accuracy from the V2 benchmark.
Measured on the V2 benchmark.
Null model confirmed
Binomial(0.5) null confirmed across every tested configuration.
Match indicators are effectively uncorrelated.
Across 113 Enron authors, mean unrelated-fingerprint match.
Adjusted-significant rate in the measured false-positive experiment.
Measured false-positive rate over unwatermarked Enron text. Clopper-Pearson upper bound: ~0.30–0.45%.
Where it broke
Real business language contains few safe transformations. The median Enron email had exactly one usable opportunity.
Apostrophes + quotes contributed ~75% of observed capacity — and typography normalization removes them.
Opportunity-ID collision rate. Common phrases create the same linguistic opportunity across many documents.
Case-insensitive canonicalization roughly doubled lowercase-attack survival without hurting clean detection. Documented, not necessarily deployed — the tradeoff is under evaluation.
System architecture
All metrics are benchmark measurements from the V2 research sprint.
Where it actually works
- Cryptographic fingerprint derivation
- Statistical differentiation
- Author-style independence
- Long clean-document attribution
- Fast local processing
- Short-email attribution
- High-density watermarking of real prose
- Robustness to normalization
- Adversarial rewriting
- At 2,000 words: 80% attribution against 10,000 candidate fingerprints
- At 500 words: 7%
- The math holds; the channel is the constraint
The bottleneck is channel capacity, not cryptography.
Next problem: channel capacity
The fingerprinting and detection machinery behaved as expected. The unresolved problem is finding enough safe linguistic transformations in real text.