A linter for AI-generated specs that uses no AI

What it found in a real 60,000-character PRD, and what it falsely flagged in a Kubernetes KEP and in RFC 9110.

Every spec I receive now is partly machine-written. That is fine. What is not fine is that the machine-written parts are typeset exactly like the decided parts, and I can't tell them apart until I'm three days into building.

The problem isn't that AI wrote it

AI-detection tools answer one question: was this written by a model? In real work that question is useless. The answer is always yes, and it should be — nobody hand-writes documents any more, and nobody should have to.

The question I actually need answered is narrower and much more useful:

Which parts of this document are undecided, but typeset as if they were decided?

That's the failure mode. A parameter table gives a precise default on the left and admits in small type on the right that nobody has signed off. A data-source list says TBD on row one, and eleven chapters of formulas downstream compute perfectly anyway. The reader remembers the defaults and the formulas. The chain gets built on an input that does not exist.

Humans rarely produce this. A model produces it constantly, because it generates sections that don't know about each other, and because looking complete is the only thing it is optimising for.

Why it uses no model

The obvious build is "feed the doc to an LLM and ask what's wrong." I didn't, for three reasons:

• Using AI to audit AI inflation produces inflated audits. I wanted findings I could take into a meeting without re-verifying them first.
• The documents are confidential. The only version of this tool that colleagues will actually paste a real spec into is one with no backend and no network calls. That constraint is the product, not a limitation of it.
• Determinism is checkable. Every finding points at a line number. Nothing is generated. It never hallucinates — and it is never clever. That trade is deliberate.

So it's regex. One HTML file, no build step, runs entirely in the browser. You can read the whole rule engine in one sitting.

What it found in a real 60,000-character PRD

An internal requirements document for an industry-analysis product. 2,545 lines, 260 headings, 102 tables. Eleven chapters, every one of them filled in. It looks extremely thorough.

Ten seconds, in the browser:

The four blocking ones, verbatim from the output:

And two definition clashes that I think are the most interesting category, because a human almost never makes them:

• β is defined twice. In the parameter table it's an LDA hyperparameter (0.01). Four chapters later it's a growth-slope threshold (>0.001). Whoever implements chapter five reads chapter one's definition and silently builds the wrong thing. The formula compiles. The output looks plausible.
• The same financing metric is monthly in chapter 2 and quarterly in chapter 5. Same name, same unit, different granularity. Two numbers that look comparable and aren't.

In a sibling document from the same set, a line reads "the five L3 items:" and then lists six. Pure arithmetic. Nobody caught it because the list and the count were generated in separate passes.

The rules that earn their keep need no domain knowledge

The tool cannot tell whether a formula is sound — it never pretends to. But one family of problems is pure arithmetic, and that family turns out to be the valuable one, because there is nothing to argue about:

• Percentages declared as weights that sum to 105%, not 100%
• "Five items:" followed by a list of six
• "100% automated" sitting next to "provided the data source can be connected"
• One symbol defined as two different quantities
• The same metric computed over 6 months in one place and 12 in another

Nobody has to debate whether the document is well written. They only have to say which number is correct. That's also why the output is a list of questions and never a score: a score invites argument ("who says I'm a 60?"), a question doesn't. You either answer it or you can't, and the ones you can't are the document's real progress.

The part I think matters most, and it isn't technical

The tool flags passages that look one-pass generated. It would be easy to make that an accusation. It is deliberately the opposite.

"That part was my AI, just delete it" is a hundred times easier to say than "I got it wrong."

If pointing at a problem means accusing a colleague of incompetence, they can only dig in, and the conversation dies — and then the problem ships anyway. Leaving the blame with the model gives people a way out, and that's when the conversation actually happens.

So the AI-trace tier is counted separately and never added to the questions that must be answered. The phrasing is softer on purpose: "If any of these points were the model's addition rather than your intent, just say so and I'll drop them — no explanation needed."

For the same reason the default mode is self-check, not review. A tool that can only be used to find fault with other people gets resented the first time it's used. You need to be able to say "I run this on everything I send" before anyone else will touch it.

Measuring the false positives, on documents that deserve respect

Synthetic fixtures prove a rule fires. They prove nothing about precision. So I ran it against two public English documents written by people who are very good at writing specifications, where every hit is by definition suspect:

curl -sL https://raw.githubusercontent.com/kubernetes/enhancements/master/keps/NNNN-kep-template/README.md
curl -sL https://www.rfc-editor.org/rfc/rfc9110.txt

The first run was bad, and in an instructive way.

Two real defects came out of this:

RFC 9110 was flagged for having no acceptance criteria. It has conformance requirements — MUST, SHALL, normative references. It defines correctness more rigorously than any PRD I have ever received; it just doesn't use the word "acceptance." The vocabulary now covers RFC 2119 keywords.

"Unverified specifics" was firing on citations. The rule's premise is a model invented these names. In a real engineering document, identifiers are simply normal — and when I looked, the KEP template contains only five backticked identifiers. The other thirty-two hits were documentation links. Three changes fixed it: skip fenced code blocks, skip Markdown link and reference syntax, and treat any identifier appearing three or more times as real, because an invented name is usually mentioned once and a load-bearing one is mentioned repeatedly.

False positives dropped by roughly 70% and the true-positive counts did not move. The baseline numbers and those two curl commands are now in the repo, and any rule change has to re-run them — hits may go down, never up.

One known noise source I did not fix: on RFC 9110, "unsourced numbers" still returns 82, because the English vocabulary includes window, weight and ratio, which are real HTTP terms. It's a clarify-tier finding, not a blocking one, and tightening it costs recall on the documents this is actually for. So it stays, documented.

What it cannot do

Stating this is the whole reason the tool gets to exist. A document that refuses to say what it can't do is exactly what it was built to find.

• It can't tell right from wrong. It doesn't know your business.
• Definition clashes only at the numeric level. Two chapters disagreeing about what "active user" means, in prose, with no numbers attached — invisible to it. I tried; deterministic rules either found nothing or barked at every legitimate difference. Still a gap.
• It can't see "correct but undoable." Headcount, budget, access — not in the document, not visible.
• A hit is not a verdict. Every one means "a human needs to answer this here."

Try it

Paste a spec and see. It runs in your browser; nothing is uploaded, there are no network calls, and it works offline. Don't take that sentence's word for it — the source is one HTML file. Search it for fetch; there isn't one.