You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: .claude-plugin/plugin.json
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
{
2
2
"name": "posthog",
3
3
"description": "Access PostHog analytics, feature flags, experiments, error tracking, and insights directly from your AI coding tool. Optionally capture Claude Code sessions to PostHog LLM Analytics.",
Copy file name to clipboardExpand all lines: skills/authoring-scouts/SKILL.md
+41-11Lines changed: 41 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,12 +6,15 @@ description: >
6
6
customize a canonical scout for their own setup (narrow its scope, retune its
7
7
thresholds, add disqualifiers), tweak a scout's schedule or dry-run posture, or
8
8
write a brand-new scout from scratch for a specific use case (a custom event, a
9
-
product surface no canonical scout covers). Covers the scout SKILL.md anatomy, the
10
-
report contract, the dedupe + scratchpad-memory conventions, the per-team skills-store
9
+
product surface no canonical scout covers), or steer a scout without editing it at all
10
+
by leaving it a note. Covers the scout SKILL.md anatomy, the
11
+
report contract, the dedupe + scratchpad-memory conventions, the scout-notes steering
12
+
channel, the per-team skills-store
11
13
path vs the canonical in-repo path, and the write-and-inspect test loop (with dry-run as an
12
14
optional safety net). Trigger on
13
15
"write/edit/customize a signals scout", "new scout for X", "tune my scout schedule",
14
-
"make a scout that watches <event>".
16
+
"make a scout that watches <event>", "leave a note for / give feedback to a scout",
17
+
"tell the scouts about X".
15
18
metadata:
16
19
owner_team: signals
17
20
---
@@ -57,13 +60,14 @@ There are two independent decisions: **what** you're building, and **where** it
57
60
| A canonical scout is close but too broad / too noisy / missing a disqualifier for this project |**Adapt** it — narrow the scope, add disqualifiers, retune thresholds. |
58
61
| You want a surface no canonical scout covers (a custom event, a product-specific funnel) |**New scout from scratch** — copy the closest canonical scout as scaffolding, replace the domain discriminator + explore patterns. |
59
62
| You only want to change _when_ / _whether_ a scout runs |**No authoring** — just tune the config (see Run posture). |
63
+
| You have one-off feedback, a pointer, or short-lived context for a scout |**No authoring** — leave a note (see Steering with notes). |
|**Per-team** (the common user path) |Create/edit a `signals-scout-*``LLMSkill` row in the project's skills store via `posthog:skill-create` / `-update` / `-file-create`, then register its config immediately via `posthog:scout-config-create`. | Customizing for one project. The harness globs the row in on the next tick; canonical sync leaves your edited ("diverged") row alone. |
66
-
|**Canonical** (PostHog contributors) | Edit disk under `products/signals/skills/signals-scout-*/`, lint/build, open a PR. | Improving a scout for _every_ enrolled project. `lazy_seed` mirrors it onto all enrolled teams on the next tick. |
|**Per-team** (the common user path) |Prepare a new runnable scout via `posthog:scout-create-prepare`, show its confirmation message, wait for the user to type `confirm`, then call `posthog:scout-create-execute`; edit its prompt or files later via `posthog:skill-update` / `-file-create`, and tune its runtime config via `posthog:scout-config-update`. | Customizing for one project. The harness globs the row in on the next tick; canonical sync leaves your edited ("diverged") row alone. |
70
+
|**Canonical** (PostHog contributors) | Edit disk under `products/signals/skills/signals-scout-*/`, lint/build, open a PR. | Improving a scout for _every_ enrolled project. `lazy_seed` mirrors it onto all enrolled teams on the next tick. |
67
71
68
72
**Adapting-in-place tradeoff:** editing a canonical scout's row for your team marks it **diverged** — you stop receiving upstream improvements to that scout.
69
73
If you only need an _additional_ behavior, prefer authoring a **new, differently-named** scout (`signals-scout-<your-scope>`) and leaving the canonical one intact.
@@ -94,8 +98,9 @@ Name it explicitly near the top of the body so every run anchors on it.
94
98
## Run posture (config)
95
99
96
100
A scout's schedule and emit behavior live on its `SignalScoutConfig`, separate from the skill body.
97
-
For a **brand-new scout**, register the config immediately after creating the skill with `posthog:scout-config-create {"skill_name": "signals-scout-<scope>", ...}`, setting any of the fields below in the same call — including creating it disabled or in dry-run **before it ever runs**.
98
-
(It's an upsert: if the coordinator already auto-registered the row, your fields are applied to it.)
101
+
For a **brand-new scout**, pass these settings in the nested `config` object of the `posthog:scout-create-prepare` call, including creating it disabled or in dry-run **before it ever runs**.
102
+
Show the returned confirmation message, wait for the user to type `confirm`, then call `posthog:scout-create-execute` with the returned `confirmation_hash` and that literal confirmation.
103
+
The endpoint creates the skill and config atomically, always opts the scout into the report channel, and safely re-applies config fields when the same definition is retried.
99
104
Otherwise the coordinator auto-registers an enabled config on the default every-24-hours schedule on its next tick (up to ~30 min).
100
105
For an **existing scout**, tune with `posthog:scout-config-update` (find the `id` via `-config-list`):
101
106
@@ -108,6 +113,31 @@ For an **existing scout**, tune with `posthog:scout-config-update` (find the `id
108
113
Set **`emit=false` (dry-run)** only when you want to be extra careful: the scout still runs and logs its reasoning but writes nothing to the inbox.
109
114
Reach for dry-run on a scout you expect to be chatty, expensive, or high-stakes; for most scouts, just writing and watching the inbox is the better loop.
110
115
116
+
## Steering with notes (no authoring needed)
117
+
118
+
Sometimes you don't want to change the scout — you want to _tell it something_.
119
+
That's what **scout notes** are for: short steering messages any team member (or an agent acting for one) leaves for the fleet, which every run picks up as prior context alongside its scratchpad and run history.
120
+
Reach for a note instead of an edit when the steer is feedback, a pointer, or context with a shelf life:
121
+
122
+
- Feedback on output: "the staging traffic spike you keep flagging is known noise, stop reporting it".
123
+
- A pointer: "dig into the EU signup funnel this week — we think something regressed".
124
+
- Context the scout couldn't know: "we shipped a new checkout on Tuesday, treat conversion shifts after that as expected".
125
+
126
+
The tools (reads on the public `signal_scout:read` scope; because scouts read notes verbatim, writing or deleting one requires the same authorization as editing a scout's skill — the `llm_skill:write` scope plus skill editor access):
127
+
128
+
-`posthog:scout-notes-create {"content": "...", "skill_name": "signals-scout-web-analytics"}` — address one scout by its exact skill name (roster via `scout-config-list`; the skill must already exist, so a typo'd target is rejected instead of silently steering no one), or omit `skill_name` for a general note every scout sees.
129
+
Optionally set `expires_at` so a time-boxed note ("watch closely this week") retires itself.
130
+
-`posthog:scout-notes-list` — browse the active notes; pass `skill_name` to see what a given scout will read.
131
+
-`posthog:scout-notes-delete {"id": "..."}` — retire a note that's been acted on or no longer applies.
132
+
133
+
How scouts treat notes: every run reads its notes in step 1 and is told to let a fresh note visibly shape what it investigates — but notes are **advisory**.
134
+
They direct attention; they don't lower the scout's evidence bar or force a report, so a note saying "report X" still gets an honest investigation, not an automatic emit.
135
+
The scout closes the loop in its run summary (which notes it acted on and how) and folds absorbed guidance into its scratchpad.
136
+
137
+
Choosing between a note and an edit: a note is the right tool for _this project, right now_ steering and for trying a nudge before committing to it; a skill edit is the right tool once the steer is permanent policy (a disqualifier, a threshold, a scope change).
138
+
A note that you keep re-leaving is a skill edit waiting to happen — promote it.
139
+
Note lifecycle stays with humans: scouts never delete notes, so retire acted-on notes yourself (or set `expires_at` up front) to keep the channel high-signal.
140
+
111
141
## Test loop
112
142
113
143
**Dogfood the scout yourself before you ever spend a real run.** You — the agent authoring the scout — have the same PostHog MCP tools a scout uses at runtime (`execute-sql`, `read-data-schema`, the per-product list tools, `scout-project-profile-get`).
@@ -134,7 +164,7 @@ The standard loop is **dogfood → run once ready → inspect**:
134
164
135
165
1. Dogfood the discriminator + explore patterns yourself against the live project (above).
136
166
Refine the body until the logic holds on real data — this is the cheap, iterable part.
137
-
2.Author the scout and register its config (`-config-create`, the default `emit=true`), then spend one `-run-now` to watch the whole scout execute end-to-end.
167
+
2.Create the scout and its config together via `posthog:scout-create-prepare` → `-execute` (schedule and the default `emit=true` go in the nested `config`), then spend one `-run-now` to watch the whole scout execute end-to-end.
138
168
Leave `run_interval_minutes` at a sustainable value — you no longer need a short interval to force an early run.
139
169
3. After the run finishes, read what it did: `posthog:inbox-reports-list` (the reports it actually wrote), `posthog:scout-runs-list` (run summaries), `-runs-retrieve` (full reasoning for one run), and `-scratchpad-search` (the durable memory it wrote).
140
170
4. If it needs work, go back to dogfooding the queries by hand for the iteration — only spend another `-run-now` once you've batched a meaningful change worth a fresh end-to-end run.
@@ -147,7 +177,7 @@ Treat suggestions as input, not instructions — the owner decides.
147
177
The scratchpad is writable only from inside a scout run, so you can't clear an entry from here after applying it via `posthog:skill-update` — the scout reconciles on its own: a later run sees the updated skill body, re-checks the suggestion, and forgets or rewrites the entry once it's addressed.
148
178
(Canonical scouts don't write these — their bodies sync from PostHog's fleet, and skill-level fixes to them belong upstream.)
149
179
150
-
**Want to be extra careful?** Set `emit=false` to dry-run first — create the config with `emit=false` via `-config-create`, then trigger it with `-run-now`: it runs and logs what it _would_ have written (visible via `-runs-list` / `-runs-retrieve`) without writing to the inbox.
180
+
**Want to be extra careful?** Set `emit=false` to dry-run first — pass `emit=false` in the nested `config` at `scout-create-prepare` time (or flip it later with `-config-update`), then trigger it with `-run-now`: it runs and logs what it _would_ have written (visible via `-runs-list` / `-runs-retrieve`) without writing to the inbox.
151
181
Inspect, refine, then flip `emit=true` and run it again.
152
182
Worth it for a scout you expect to be chatty, expensive, or high-stakes; otherwise just writing and watching the inbox is the faster path to a calibrated scout.
Copy file name to clipboardExpand all lines: skills/authoring-scouts/references/lifecycle-and-testing.md
+9-8Lines changed: 9 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,9 @@ How scouts get discovered, scheduled, and dispatched; the two distribution paths
9
9
No registration step.
10
10
-**Config.** Each scout has one `SignalScoutConfig` per `(project, skill_name)` carrying `run_interval_minutes` (default 1440), `enabled`, `emit`, and a `last_run_at` stamp.
11
11
A config is **auto-registered** the first time the coordinator sees a `signals-scout-*` skill without one — authoring the skill is enough to get a scout.
12
-
To configure a fresh scout immediately (instead of waiting for the tick), register the config yourself with `posthog:scout-config-create`, setting the schedule / emit posture in the same call; until one of those happens, the scout has no config row and won't show in `-config-list`.
12
+
Prepare a fresh per-team scout and its config together with `posthog:scout-create-prepare`; the nested `config` object sets its schedule, emit posture, and destinations before it can run.
13
+
Show the returned confirmation message, wait for the user to type `confirm`, then call `posthog:scout-create-execute` with the returned `confirmation_hash` and that literal confirmation.
14
+
The lower-level `posthog:scout-config-create` remains available when a skill already exists without a config.
13
15
Config responses also carry the scout's `description`, read live from the skill's frontmatter — not a config field you set.
14
16
-**Coordinator.** A periodic Temporal workflow ticks (~every 30 min).
15
17
Each tick it bounds candidates to projects enrolled via the `signals-scout` feature-flag allowlist, then dispatches every **enabled** scout whose schedule is **due** (`last_run_at is None`, or `now - last_run_at ≥ run_interval_minutes`), most-overdue first, capped per tick.
@@ -21,7 +23,7 @@ How scouts get discovered, scheduled, and dispatched; the two distribution paths
21
23
Pausing a scout = `enabled=false`.
22
24
Slowing it = a larger `run_interval_minutes`.
23
25
Dry-running it = `emit=false`.
24
-
All three via `posthog:scout-config-update` (get the `id` from `-config-list`), or set at creation time via `-config-create`.
26
+
All three via `posthog:scout-config-update` (get the `id` from `-config-list`), or set at creation time in the nested `config` object passed to `posthog:scout-create-prepare`.
# Adapt an existing per-team scout — use the SMALLEST primitive (find/replace, not full-body)
46
47
posthog:skill-get {"skill_name": "signals-scout-<scope>"} # get current version first
@@ -95,7 +96,7 @@ There's no free test run, and it's slow (async, one run per call): firing the sa
95
96
The loop is **dogfood → run once ready → inspect**:
96
97
97
98
1. Dogfood the discriminator + explore patterns yourself against the live project (above), refining the body until the logic holds — the cheap, iterable part.
98
-
2.Author the scout and register its config (`-config-create`, default `emit=true`), leaving `run_interval_minutes` at a sustainable value — no short-interval trick needed.
99
+
2.Create the scout and its config together via `posthog:scout-create-prepare` → `-execute` (the default `emit=true` goes in the nested `config`), leaving `run_interval_minutes` at a sustainable value — no short-interval trick needed.
99
100
Then spend one `-run-now` to watch the whole scout execute end-to-end, and inspect once it finishes:
100
101
-`posthog:inbox-reports-list` — the reports it actually wrote.
0 commit comments