Number Interpretation

Explain the Number

by michael-denyer · MIT license · ★ 1,003 on GitHub

Apply before you trust, report, or act on a number you measured: a speedup, a regression, a throughput, a latency, or an eval result. Find what limits it, and rule out that it measured something other than the work you think.

---
name: principle-explain-the-number
description: "Apply before you trust, report, or act on a number you measured: a speedup, a regression, a throughput, a latency, or an eval result. Find what limits it, and rule out that it measured something other than the work you think."
user-invocable: false
---

# Explain the Number

A measured number is a claim about a system. Before you trust it, report it, or act on it, find what limits it and rule out that it measured something else.

**Why:** A run that went wrong still prints a plausible number. Requests that failed, a cache that skipped the work, code that never ran, a side left on default settings, and run-to-run noise all produce results that look fine. If you cannot say why the number is not twice as good, you do not know what you measured.

**Pattern:**

- **Ask "why not double?"** Name the resource or code path that bounds the result, such as a core, a lock, the disk, the network, or the load generator itself. Get it from a profile or from system counters taken during a run, then map it to source. A guess from reading the code is not a limiter.
- **List what else the number could be measuring, and rule out each one with evidence.** The usual suspects are errors, skipped or cached work, an untuned side, noise, and a piece too small to matter end to end.
- **Keep the evidence with the number.** Put the run count, the spread, and the limiter in the notes or a linked artifact, so a reader can check the claim.

Overview

A measured number is a claim about a system. Before you trust it, report it, or act on it, find what limits it and rule out that it measured something else.

Manual install (npx, ZIP)

Claude Codenpx skills add michael-denyer/pstack-claude --skill principle-explain-the-number -g -a claude-code

Claude Code (degit)

npx degit michael-denyer/pstack-claude/plugins/pstack/skills/principle-explain-the-number#main ~/.claude/skills/principle-explain-the-number

Claude.ai / desktopZip the folder and upload it under Settings → Capabilities → Skills.

Codex, Cursor, other agentsPoint them at SKILL.md. The other files are plain text.

Connects with Agent Alley MCP · agentalley.io/mcp

Files included

SKILL.md2.1 KB

Read SKILL.md here

michael-denyer/main1 file shown

SKILL.md2.1 KB

Browse every file on GitHub↗

SKILL.md

Show the full text24 lines

principle-explain-the-number/SKILL.md24 lines · 2.1 KB

PreviewCodeRawView on GitHub

Explain the Number

A measured number is a claim about a system. Before you trust it, report it, or act on it, find what limits it and rule out that it measured something else.

Why: A run that went wrong still prints a plausible number. Requests that failed, a cache that skipped the work, code that never ran, a side left on default settings, and run-to-run noise all produce results that look fine. If you cannot say why the number is not twice as good, you do not know what you measured.

Pattern:

Ask "why not double?" Name the resource or code path that bounds the result, such as a core, a lock, the disk, the network, or the load generator itself. Get it from a profile or from system counters taken during a run, then map it to source. A guess from reading the code is not a limiter.
List what else the number could be measuring, and rule out each one with evidence. The usual suspects are errors, skipped or cached work, an untuned side, noise, and a piece too small to matter end to end.
Keep the evidence with the number. Put the run count, the spread, and the limiter in the notes or a linked artifact, so a reader can check the claim.

For a performance number, run the full procedure with the benchmark-checklist skill. For an eval result, ask the same of the trials: did every run do the task, does the gap hold across trials and models, and does the scenario matter.

You skipped this when the evidence behind a number has no run count, no spread, or no named limiter, or when the time saved is larger than the time the changed piece took.

Distinct from Prove It Works, which checks that an output is real. This checks that a measured number means what you say it means.

1	---
2	name: principle-explain-the-number
3	description: "Apply before you trust, report, or act on a number you measured: a speedup, a regression, a throughput, a latency, or an eval result. Find what limits it, and rule out that it measured something other than the work you think."
4	user-invocable: false
5	---
6
7	# Explain the Number
8
9	A measured number is a claim about a system. Before you trust it, report it, or act on it, find what limits it and rule out that it measured something else.
10
11	Why: A run that went wrong still prints a plausible number. Requests that failed, a cache that skipped the work, code that never ran, a side left on default settings, and run-to-run noise all produce results that look fine. If you cannot say why the number is not twice as good, you do not know what you measured.
12
13	Pattern:
14
15	- Ask "why not double?" Name the resource or code path that bounds the result, such as a core, a lock, the disk, the network, or the load generator itself. Get it from a profile or from system counters taken during a run, then map it to source. A guess from reading the code is not a limiter.
16	- List what else the number could be measuring, and rule out each one with evidence. The usual suspects are errors, skipped or cached work, an untuned side, noise, and a piece too small to matter end to end.
17	- Keep the evidence with the number. Put the run count, the spread, and the limiter in the notes or a linked artifact, so a reader can check the claim.
18
19	For a performance number, run the full procedure with the [benchmark-checklist](../benchmark-checklist/SKILL.md) skill. For an eval result, ask the same of the trials: did every run do the task, does the gap hold across trials and models, and does the scenario matter.
20
21	You skipped this when the evidence behind a number has no run count, no spread, or no named limiter, or when the time saved is larger than the time the changed piece took.
22
23	Distinct from [Prove It Works](../principle-prove-it-works/SKILL.md), which checks that an output is real. This checks that a measured number means what you say it means.
24