Find out which of your agent rules actually fire.

Sixty seconds

uvx ruleprobe report

That reads ~/.claude/projects/**/*.jsonl (and ~/.codex/sessions/**/*.jsonl and ~/.gemini/tmp/*/chats/, if Codex or Gemini CLI wrote any), runs the detectors over them, and prints:

detector                                 hits  sessions    of    share  note
----------------------------------------------------------------------------
cache-hygiene/compact                       0         0   339       0%  unobserved
cache-hygiene/model-switch                 11        11   339       3%
secrets/secret-in-write                     3         2   339       1%
transcript-hygiene/unfiltered-find          0         0   339       0%  unobserved
transcript-hygiene/whole-file-cat          89        22   339       6%
verification/no-verify                      0         0   339       0%  unobserved

That is a real run, over one week of one person’s transcripts. Two detectors have never fired on this machine, which is itself the finding: unfiltered-find and no-verify are guarding against something that is not happening, and the rules behind them are paying rent in the context window for nothing.

share is the fraction of sessions the detector fired in at least once. unobserved means it has never fired in the window. frequent means it fired in more than 30 percent of sessions; it describes the share and advises nothing. Both notes stay blank until there are twenty measured sessions, because a share over five sessions is noise.

Nothing is sent anywhere, no model is asked anything, report and explain write nothing, label writes only under the directory you name, and the same transcript gives the same answer every time. Python 3.9 or newer, standard library only.

Other groupings, and a window:

uvx ruleprobe report --by repo --since 30      # last 30 days, one line per repository
uvx ruleprobe report --by stance --stance commits=conventional   # grouped by configuration
uvx ruleprobe report --root ./transcripts      # a directory of your own
uvx ruleprobe report --rules ./docs/rules      # bind detectors to rule files, and name the gaps
uvx ruleprobe detectors                        # what would run
uvx ruleprobe corpus                           # how good each detector is, over the labelled corpus
uvx ruleprobe report --json                    # the same numbers as data, rows included

Each row and the --json result carry schema_version, and a row with a version this release does not know is left out of every count and tallied in unknown_schema. The result also carries renamed, the effective fold map of retired detector id to current id that its counts were read through. It is complete, so a saved result folds a retired id by applying it as it stands, never by merging it again with a later release’s shipped map. The result also carries coverage, on every run: measured, dark and unmeasured count rules, not sessions as the top-level measured and unmeasured do, and share is the measured share of them; catalog counts the measured rules the shipped catalog binds. With no rules read, the counts are zero and share is null.

A row names each detector that raised in rules_errors, and that session leaves the detector’s hit denominator only. A row also carries compliance, detector id to {"opportunities", "followed", "undecided"}, for each enabled detector that defines opportunities; an undecided opportunity counts in undecided alone. A point that names a tool_use_id appears at most once per turn, and a repeat is malformed; an id-less point, None or "", counts once per occurrence. When that callable raises, or returns something other than (turn, tool_use_id, followed) triples (recorded as MalformedOpportunities), its rules_errors entry carries "hook": "opportunities": the detector loses its compliance entry, and its hit figures are untouched.

The report sums compliance beside hits per session, never in place of them. A detector that has any adds opportunities, followed, undecided and rate columns to its line, and opportunities, followed, undecided and compliance_rate to its --json entry; one without has none of them. compliance_rate is followed / opportunities, and it stays null, printed -, until the line holds twenty opportunities (--min-opportunities N); undecided points never count toward that. --by repo and --by stance group compliance as they group hits, under each group’s compliance key and its own minimum. Retired ids fold as hits do. A session whose opportunities failed, or whose detector raised, is out of that detector’s compliance figures, as is one whose stored compliance entry cannot be read; opportunity_errors and malformed_compliance count those sessions per detector, and a note names them.

A transcript does not record the configuration it ran under, so --stance dimension=variant is how you say what it was. It is repeatable, it is what --by stance groups on, and it is what a detector’s gate: block reads: a gated detector with no stance passed never fires, and ruleprobe detectors names the stance each one is waiting for.

What it actually covers

Be clear-eyed about the scope, because the name promises more than version 0.1 delivers.

Six detectors ship with the package, and they are the generic ones: reading a whole file into the context window, an unfiltered find, a commit or push that walks past the repository’s hooks, a secret-shaped string written to a file, a context compaction, and a model change mid-session. They are in the package because they mean the same thing in every repository, and none of them needs to know what your rules say.

Your own rules take a detector you write, as data in the declarative format above or, when the shape is past what a matcher can say, as Python:

from ruleprobe import DEFAULT, Detector, iter_sessions, measure, report

def sudo_install(events, ctx):
    return [(p.turn, p.id) for p in ctx.bash
            if any(seg[:2] == ["sudo", "pip"] for pipe in p.pipelines for seg in pipe)]

DEFAULT.add(Detector("house-style/sudo-install", "house-style", "bash", sudo_install))
print(report([measure(s) for s in iter_sessions(since=30)]))

measure, DEFAULT, Detector imported from the root, and Parsed’s .turn, .id and .pipelines are importable but not declared (see the public API). A tool pinning a version has declared alternatives for some: Detector from ruleprobe.registry, hit(p.event) for (p.turn, p.id), and pipelines(p.command) for p.pipelines. No declared function yet produces a row, so measure has none.

The two are the same engine: ruleprobe/detectors/common.yaml is the shipped six written as data, and a test asserts it produces hit-for-hit what the Python in ruleprobe/detectors/common.py produces over the corpus. Python remains the escape hatch, and the seam a third compiler plugs into is still register_compiler and from_spec.

What a matcher cannot say. It reads one event, or one of the three session shapes, and nothing else. There is no arithmetic, no counting (“more than three reads in a turn”), no reading a tool’s result, no comparing one argument with another, and no state carried across turns beyond order, absent and change. The command matcher inherits every known miss of the shell parse in ruleprobe/shell.py - a command inside a substitution is invisible, and so is a variable’s value. A rule whose shape needs any of that is a Python detector, and the report will not pretend otherwise.

Unbound rule files split into one rule per section. A file whose front matter gives detector: or opt_out: a value is one rule, named by its rule: key or its file name; a key with no value binds nothing and is a finding. Any other file is split at its ATX (#) headings, so a CLAUDE.md of twelve sections is twelve entries in the coverage block, each with the id <path>#<heading-slug> under --rules, and a repeated heading takes -2. A section holding only a code block, an HTML comment, a table, a blockquote, an image or a link reference is not a rule. A file with no heading, or with no section that is a rule, stays one rule, named by its rule: key or else its file name, and its whole text - text above the first heading included - binds the catalog as one unit: measured when exactly one entry matches, unmeasured otherwise. Known misses: in a file with a rule section, text above the first heading is no rule, a setext heading (text underlined with === or ---) does not split, and list items are never split.

Detector validity is measured, and the measurement is small. Every detector is scored against a hand-labelled corpus that ships with the package - see How good are the detectors? below - but that corpus is synthetic and it is ten sessions, so it catches a detector that is wrong about a shape it was shown and says nothing about a shape nobody thought of. The detectors deliberately under-count: a missed hit is a quieter report, a false hit is a wrong one.

Gemini CLI records less, so it is measured less. Its sessions are read into the same events, with write_file read as Write, replace as Edit and run_shell_command as Bash, and six gaps under-count on it rather than guess:

• A context compaction is not told apart from truncation, tool-output masking and rollback, which rewrite the history the same way, so cache-hygiene/compact never fires on Gemini.
• The platform is not recorded. A shell call is Bash only when .project_root, in the project directory above chats/, holds a POSIX path. Every other root keeps the native name, so no Bash detector reads it: a drive letter or a UNC path, where the command is PowerShell, and a missing root, where the platform is unknown.
• The model changes by Gemini’s router and quota fallback as well as by choice, and the transcript does not say which, so a Gemini event’s model is left empty and cache-hygiene/model-switch never fires on Gemini.
• invoke_agent keeps its native name, so a detector on Agent does not see Gemini’s subagent calls; each subagent’s own session is read as a session of its own, with the id <parent session id>/<its own id>.
• Turns are not recorded. One starts at each user record that is not only tool responses.
• Edits the user made before accepting a write or an edit are not measured as writes. The call keeps its native name, write_file or replace, and its input drops content, new_string, old_string and ai_proposed_content, which may hold the user’s text rather than the agent’s. Its file_path and modified_by_user stay.

Every other shipped and catalog detector is scored on labelled Gemini sessions under a POSIX root, near-misses included. A session under a Windows root checks that the shell detectors stay silent there; it does not measure them on PowerShell.

Gemini deletes sessions older than 30 days by default, so a longer --since finds fewer.

What a count is not. A detector fires on a shape in a transcript, not on an intention. whole-file-cat firing 89 times above does not prove the agent wasted context; it proves it read 89 files whole, which is a fact worth having and an argument worth starting.

Why did that fire?

A count you are surprised by is one you want to check. ruleprobe explain reruns the detectors over the same transcripts and prints every hit report counts, with the event behind it:

uvx ruleprobe explain                                   # every hit
uvx ruleprobe explain --detector verification/no-verify # one detector's
uvx ruleprobe explain --session claude-code:sess-1 --key 1:toolu_2

session   claude-code:sess-1
key       1:toolu_2
detector  verification/no-verify
turn      1
tool use  toolu_2 (Bash)
command   git commit --no-verify -m 'fix: the thing'

A hit’s address is its session, <runtime>:<session id>, plus its key, <turn>:<tool use id>, or <turn>:- for a hit that names no tool use - a compaction, a model switch - and so has no single event to show, only its turn. A key is unique only within its session, so both are printed. What is shown is the Bash command, or any other tool’s whole input as JSON, because a hit does not record which field of the event matched. --session takes the address or the bare session id; --session, --detector and --key each narrow the output, and one that matches nothing prints nothing and exits 0; --detector takes a renamed id too. It takes report’s --root, --runtime, --since, --stance and --plugins, and --rules, --detectors and --no-config, so a gated or plugin detector report counts is listed as well; what a detector file skipped goes to stderr.

Every line it prints, to either stream, passes through redact first. It redacts AWS access key ids, Bearer tokens, private keys (header to footer), Slack, GitHub, OpenAI and Anthropic style tokens, a URL’s password, and the value after a key name - aws_secret_access_key, client_secret, and password, passwd, pwd, token, api_key, apikey, secret or access_key followed by : or = - through the end of its line, its continuations, and the indented lines below when the value starts there. Any other credential can still print, so read explain’s output before you share it. A control character or a Unicode format character is printed as its \xNN or \uNNNN escape, so a transcript cannot drive your terminal or reorder a line, and one event’s text is cut at 4,000 characters with a note of how many were cut. Like report, it writes nothing and sends nothing, and running it changes no report number. A stored row keeps counts only, not the events behind them, so it cannot be explained after the fact; an undeclared helper in ruleprobe.report says so for a row and names the rerun over its session that would.

Origins and neighbours

The engine was carved out of agent-harness, where it grew as a hook that measured that project’s own always-loaded rules; the detectors that were about agent-harness’s rules stayed there, and the rule-agnostic half is this package. The nearest neighbour is Burnd, which also reads Claude Code transcripts locally, for token spend rather than for rule compliance.

MIT licensed.