Resources · Governance

Living documentation: why generated docs rot and what keeps them true

In short

Documentation generated from code produces volume, not coverage: it records what the code does and misses intent, exceptions and the rules that live in data and in people's heads. Documentation stays true only when each page is tied to the work that changes the system and its health is measured: coverage of important behavior, open gaps and time since an owner last confirmed it.

Abstract illustration: document pages linked to work items

At some point in every modernization conversation, somebody asks for the documentation. What comes back is usually one of two things: a wiki nobody has touched since 2019, or a fresh export generated from the code last week, hundreds of pages long, that your engineers politely ignore.

Both have the same problem. You can't tell, page by page, whether what they say is still true. Documentation you can't trust is worse than none, because it costs time to read and then costs more time to disprove.

Documentation is worth exactly as much as your ability to trust it, and trust comes from three things: coverage of the behavior that matters, a link between each page and the work that changes the system, and a few health measures you check as routinely as uptime. The rest is volume.

Volume is not coverage

When a tool produces documentation "from the code", the output is measured in pages. Every class, endpoint and table gets its paragraph, and the question "is it complete?" seems answered.

It isn't. Completeness against the code is not coverage of what the business depends on. A billing system might have 400 endpoints, of which 30 carry the rules that decide what customers are charged. A generated document gives all 400 equal weight and tells you nothing about which 30 deserve an owner's signature.

Code is a reliable record of what a system does, a poor record of why, and silent about what happens outside it. In our experience, what generated documentation misses falls into five groups.

Cyrille Martraire puts it this way in Living Documentation: most knowledge worth sharing is already somewhere in the system, but not in a convenient form, and part of it is only in people's heads. Generating text from source captures the first part and leaves the second where it was.

Why documentation rots

Each change to the system is a chance for a page to drift, and nobody is paid to notice.

Nygard's observation from 2011 still holds: "Large documents are never kept up to date. Small, modular documents have at least a chance at being updated." A 200-page system manual has one owner and no trigger to review it. A one-page record of one rule can be tied to the one change that affects it.

The cost of the drift shows up as time. In Stack Overflow's 2024 Developer Survey, 30% of professional developers said knowledge silos hurt their productivity ten or more times a week, and fewer than half agreed they could easily find up-to-date information inside their own organization. Those are hours spent re-deriving what the last document failed to keep.

The other side is measured too. DORA's research on documentation quality found that teams with above-average documentation got a far larger lift to organizational performance from every technical practice studied. DORA scores documentation on clarity, findability and reliability, not on page count.

What "living" actually means

Martraire sets four tests for living documentation: it is reliable, it takes low effort to maintain, it is collaborative, and it is insightful. Reliable comes first for a reason. If the reader has to verify a page before using it, the page has failed. Two practices from the last fifteen years make reliability achievable without heroics.

Docs as code

The docs-as-code approach, as the Write the Docs community describes it, keeps documentation in version control, reviews it like code, and lets a team block the merge of a change that doesn't update the pages it touches. The document then shares the system's history and reviewers.

Decision records

An ADR is a one-or-two-page record of one significant decision: the context, the choice, its status and its consequences. When a decision is reversed, the old record stays and is marked "superseded". That gives you something a generated document never has: a trail of what was true, when, and why it stopped being true.

Neither practice is about tooling. Both attach a page to the thing that changes.

Link every page to the work that changes it

This is the mechanism that keeps documentation true, and it is where most documentation efforts quietly stop.

  1. Give each rule a stable identifier. An ID that survives renames and reorganization, not a page title.
  2. Record its source. Which code, which table, which observed behavior or which conversation with an owner supports the claim. A rule with no source is an opinion.
  3. Reference the ID from the work item. When a ticket or pull request changes a rule, it names the rule. The reviewer checks the page as part of the review.
  4. Mark the page when the work completes. The completed change updates the page, or flags it for the owner to confirm. Either way, the page's "last confirmed" date moves.
  5. Keep superseded versions. Yesterday's rule is still evidence. Mark it, don't delete it.

A publisher we worked with had a subscription system where the grace period for lapsed subscribers was documented as 14 days. The code agreed. The live behavior was closer to 21 days, because the job that enforced the cutoff ran weekly, and the support team had built its scripts around the longer figure. The page had no link to the job or to the team's practice, so it stayed at 14 for years. Once the rule had an ID, a source that included the job schedule and a named owner, the next change to that job updated the page the same week.

Three numbers that tell you if documentation is healthy

You don't need a dashboard with twenty metrics. Three, reviewed monthly, are enough.

Coverage of important behavior

Start with the workflows the business can't afford to get wrong: invoicing, entitlements, pricing, regulatory reporting. For each, ask what share of its rules are written down with a source and an owner's confirmation. Report that share, not the page count. "Invoicing: 38 of 44 rules confirmed" is a sentence a CFO can act on.

Open gaps

A gap is behavior you know exists but haven't confirmed: a path never observed in production, a rule an owner couldn't explain, a job nobody has watched run. Count them, name them and give each an owner. A falling gap count is progress. A gap count of zero on a 15-year-old system means nobody is looking.

Age since last confirmation

For every page, when did someone with authority last say "yes, still true"? Not "last edited", which a formatting fix would reset. Set a threshold that fits the rate of change, say 90 days for pricing rules and a year for a stable integration, and list what has passed it. Pages past the threshold are read-only until an owner renews them.

"We don't have time to maintain all that." You don't have to. These measures apply to the important behavior you listed first, which in most legacy systems is a few dozen rules per workflow, not thousands. Everything else can stay generated and labeled as such.

Where Fabrica stands

Fabrica's CLEAR method treats documentation as a by-product of the work rather than a separate deliverable. In Capture, behavior that hasn't yet been observed is recorded as a gap, and in Lock, owners review and approve a versioned System Specification in which each requirement has a stable ID linked back to its evidence and forward to the work item that implements it. When linked work is completed the relevant pages are updated, and later changes go through a new approved version rather than a silent edit. The stages are described on How it works and the specification on Features.

Start with the rules you can't afford to get wrong

If you are deciding what to do about a legacy system, the state of its documentation is part of the decision. Not whether it exists, but whether anyone trusts it, and whether something keeps it true after the next change.

Pick the two or three workflows the business runs on. Write their rules as short, sourced, owned statements with stable IDs, and tie each to the work that changes it. Then watch three numbers. If they move the right way for a quarter, you have living documentation. If they don't, you have learned something important before spending real money.

Questions to ask about any documentation you're offered

Keep reading

Related

Which priority is your system holding up?

Tell us about it on a 30-minute call. We'll suggest a first scope and what a technical review would need to confirm.

Book a 30-minute call