Organizational telemetry

Instrumenting
the Organization

A factory that cannot tell you where the line is stalled is not a factory, it is a hope. Most large organizations run on hope. They are measuring activity, and paying for flow.

Definition. Organizational telemetry is the practice of instrumenting a company the way you would instrument a production line: measuring the movement of work through the system rather than the activity of the people inside it, so that leadership interventions can be designed and verified rather than guessed.

Every organization is already a factory

I have spent most of my career writing software and over the last 8 years running the operations and factories that build physical things. The two aspects are not as different as their vocabularies suggest. A factory takes a design, moves it through a sequence of stations, and produces a unit. A company takes an intention, moves it through a sequence of teams, and produces a change in the product. Both are flow systems. Both have queues, handoffs, rework, and a constraint that sets the pace for everything upstream and downstream of it.

The difference is instrumentation. Walk onto a well-run manufacturing floor and you can find the constraint in about ninety seconds, because the floor is wired to tell you. Cycle time per station. Work in progress between stations. First-pass yield. Scrap and rework. Nobody argues about where the line is stalled, because the line says so.

Now walk into a three-thousand-person software organization and ask the same question: where is work waiting right now, and why? You will get opinions. Good ones, from smart people, and they will not agree. That disagreement is not a people problem. It is a missing instrument.

Activity is not flow

Nearly every productivity dashboard I have inherited measured activity: tickets closed, commits landed, story points burned, meetings held, hours logged. Activity metrics are easy to collect because they are byproducts of tools that were already running. They are also nearly useless for management, for one structural reason: activity is what the organization does, and flow is what the organization is paid for. A team can be at maximum activity and zero flow. That is, precisely, what a stalled line looks like from the inside — everyone busy, nothing arriving.

What most dashboards count What actually predicts outcomes
Commits, PRs, story points per engineer Flow time from decision to production, and the share of it spent waiting
Sprint velocity Queue depth and handoff count between organizational boundaries
Tickets closed Rework rate — the fraction of shipped work that had to be built again
Utilization ("are people busy?") Decision latency — how long a blocking question waits for an answer
Release count Batch size, and the change-failure rate that batch size buys you

The right-hand column has an inconvenient property: none of it is a byproduct of a single tool. Flow time crosses the ticket tracker, the repository, the build system, the release process, and usually a spreadsheet somebody maintains by hand. That is exactly why organizations settle for the left-hand column. It is not that leaders prefer vanity metrics. It is that the honest metrics span systems that were never designed to speak to one another, and someone has to do the unglamorous work of making the seams visible.

What the factory taught me about software

The most useful thing I learned running manufacturing and operations organizations — at Apple, then Ford, now Snowcap — is that the interesting number is almost never the one inside a station. It is the one between stations.

Take the path from a design change to a production line. Engineering makes a change in CAD. That change has to reach tooling, process planning, quality, suppliers, and the line itself. Every one of those groups is competent. Every one of those groups has a system. And in most companies, the change is retyped, re-interpreted, or reconciled by hand at every boundary, because the systems do not share a vocabulary. When you measure the end-to-end path, you find the same shape you find in software: the time spent doing the work is a small fraction of the elapsed time, and the majority is queue and translation.

What made the difference was not a faster tool at any one station. It was making the seam visible — instrumenting the handoff itself, so that "waiting on tooling for eleven days" became a number on a screen instead of a thing everybody vaguely knew. Once the seam is a number, it becomes an engineering problem, and engineering problems get solved. Before it is a number, it is a personality conflict, and those never get solved.

The general lesson. Local optimization at a station is nearly always cheap and nearly always irrelevant. The constraint lives in the handoff. Instrument the handoff first, and you will usually find that the intervention you were about to fund was aimed at the wrong place.

This is the same observation that drove Domain-Adaptive Manufacturing: if the seams between systems are where the time goes, then the architecture should absorb the differences between manufacturers rather than forcing every plant to re-translate the same physics by hand. Standardize the physics, bespoke the experience. Telemetry is how you find out whether you actually did.

Four things worth instrumenting

A useful organizational instrument does not need to be large. It needs to be honest, cross-boundary, and small enough that leadership actually looks at it. Four measures carry most of the weight.

And one measure you must refuse to build: anything that ranks individuals. Not for sentimental reasons. For mechanical ones. The moment a measure is attached to a person's review, the measure becomes a target, the target gets optimized, and you have destroyed the sensor. Deming was right and he was right about factories first: the performance you are looking at is a property of the system, not of the people in it. Telemetry that grades people is gamed within a quarter and lies forever after.

AI changes the instrument, not just the work

Most of the conversation about AI and productivity is about the station: can an assistant help an engineer write code faster. That is a real question with a modest answer. The larger opportunity is that AI changes what an organization is able to measure.

Organizational telemetry historically stopped at whatever a database could count, because everything else — the design review, the ticket thread, the architecture decision, the Slack conversation where the requirement actually changed — was unstructured text. That constraint is gone. A model can read the pull request and the ticket and tell you whether this was new work or rework. It can read the decision record and tell you what the decision was waiting on. It can read across the CAD change and the process plan and flag that they no longer agree. These are semantic measures, and until very recently they were not available at any price.

The reframing I would offer any leadership team: agents are sensors and actuators in the workflow, not chat windows beside it. As a sensor, an agent observes the seam continuously and reports in the vocabulary of flow. As an actuator, it does the small mechanical thing the seam requires — opens the ticket, propagates the change, runs the gate, files the discrepancy — which removes the queue rather than merely reporting it. A chat window makes one person faster. A sensor-and-actuator pair makes the line move.

Two disciplines keep this from becoming a new source of confident noise. The first is the rule I argue for in After Effort: the machine helps most when it arrives after the human effort, not in place of it. An agent that proposes an answer before the organization has framed the question will produce something plausible, and the organization will believe it, and that is worse than having no instrument at all.

The second is that a measurement must not be arguable. In Shippable States Development, gates read counters rather than narratives, for exactly this reason — a gate a report can argue with is not a gate. The same rule applies to an organizational instrument. If a number can be explained away in the meeting where it is presented, it is not telemetry. It is a slide.

Telemetry exists so interventions can be designed

Here is the part that matters to anyone running an organization at scale, and it is not about dashboards at all.

Most executive interventions are guesses wearing the clothes of decisions. A reorg, a tooling mandate, a new process, a headcount reallocation. Each is expensive, each takes two quarters to play out, and in most companies nobody can say afterward whether it worked, because there was no instrument before it and none after. So the organization accumulates interventions the way a codebase accumulates scar tissue: every one of them was a rational response to a real pain, and collectively they are incoherent.

An instrumented organization gets to work differently. You can say: the constraint is the twelve-day wait at the security review; here is the change we are making; here is the number we expect to move; here is when we will read it. That is an experiment. It has a hypothesis, a treatment, and a measurement. It can be wrong, which is the property that makes it valuable — a leadership team that can be measurably wrong will get right faster than one that cannot.

This is also the honest answer to the question of what leadership is for in an organization full of capable people. It is not to supply the effort; there is plenty of effort. It is to see the whole system, find the place where the system is losing, and change that place on purpose. You cannot do that without an instrument. With one, most of the job becomes tractable.

Quotable. "Measure the seams, not the stations. Activity is what an organization does; flow is what it is paid for." — Alex Horovitz, InsanelyGreat

Where to start on Monday

Further reading

Find the constraint

Then change it on purpose.

The same discipline, applied to the code: keep the system in a state you could ship from, and measure whether you actually can.

Read SSD