Skip to content

Commit 1096d16

Browse files
committed
feat(usage-collector): add OpenTelemetry metrics/observability layer
Instruments the usage-collector gear with the operational metric inventory from DESIGN §3.11.5, pushed via OTLP through ToolKit's global SdkMeterProvider. Domain - Add the `UsageCollectorMetrics` port (domain/ports/metrics.rs) with a no-op default impl, plus typed, bounded label vocabularies (PdpOp, PluginOp, PdpFailureCause, AuthzDecision, PluginErrorCategory, RequestOutcome, RecordKind, QueryKind, …) so the `uc_*` instruments only ever carry closed-set label values. - Instrument the service across every flow: PDP decisions/failures and readiness, plugin call duration / accept errors and readiness, ingestion (batch size, duration, per-record and per-request outcomes, metadata bytes), query (inflight gauge, result rows, request outcomes), deactivation requests, and usage-type requests. Infra - Implement the port over `opentelemetry::global::meter_with_scope`, constructing the full `uc_*` instrument set at gear bootstrap. uc_usage_types catalog gauge - Refresh the `uc_usage_types` gauge from a 60s serve-lifecycle loop rather than per-mutation, paginating the whole catalog via `CursorV1` and summing item counts across pages. Bounded by the refresh timeout, a page-size limit, and a page-count safety cap; any SPI error, undecodable cursor, or cap breach leaves the gauge at its prior value (best-effort, never a partial-count publish). Supports multi-instance aggregation. Supporting changes - Add gear config for the metrics layer. - Promote tokio-util to a runtime dependency for the serve lifecycle. - Add domain, infra, and module tests (incl. a multi-page list_usage_types queue in HappyPathPlugin) and update DESIGN / feature docs. Signed-off-by: capybutler <capybutler@gmail.com>
1 parent 6ae9504 commit 1096d16

30 files changed

Lines changed: 5777 additions & 579 deletions

Cargo.lock

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

deny.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,8 @@ ignore = [
3939
"RUSTSEC-2023-0071", # Marvin Attack timing sidechannel in rsa 0.9 — dev-only dep of rustls-corecrypto-provider; used solely for RSA-PSS *verification* (public-key op, no private key involved) in rsa_pss_apple_salt_length_matches_rfc8017 test; no safe upgrade available; advisory itself permits "local use on a non-compromised computer"
4040
"RUSTSEC-2026-0173", # proc-macro-error2 unmaintained — direct dep of our proc-macro crate cf-gears-toolkit-db-macros (compile-time only, never in any runtime artifact); no advisory of a vulnerability, only maintenance status. Tracked for migration to std syn::Error -> compile_error! emission.
4141
"RUSTSEC-2026-0187", # lopdf <0.42 unbounded-recursion DoS on crafted PDFs — transitive via kreuzberg→cf-gears-file-parser→cf-gears-example-server (example app only, never parses untrusted PDFs). No usable fix: kreuzberg 4.9.x pins lopdf "^0.41" (<0.42) and the fix landed in 0.42.0; 5.x drops the bundled-pdfium feature we use. Remove once kreuzberg ships a release allowing lopdf >=0.42.
42+
"RUSTSEC-2026-0194", # quick-xml <0.41 quadratic attribute-duplicate check → CPU-exhaustion DoS on crafted XML — transitive via kreuzberg (0.37/0.39/0.40)→cf-gears-file-parser→cf-gears-example-server (example app only) and starship-battery→plist (0.38) in cf-gears-toolkit-node-info (reads local battery/plist data, not untrusted XML). No usable fix: 0.41 is a semver-incompatible bump and biblib/calamine/kreuzberg/plist pin the older 0.3x/0.40 ranges. Remove once those upstreams allow quick-xml >=0.41.
43+
"RUSTSEC-2026-0195", # quick-xml <0.41 unbounded NsReader namespace-declaration allocation → memory-exhaustion DoS — same transitive chains and rationale as RUSTSEC-2026-0194. Remove once upstreams allow quick-xml >=0.41.
4244
]
4345

4446
# Note:

gears/system/usage-collector/docs/ADR/0001-pdp-centric-authorization.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ The Usage Collector exposes ingestion, query, deactivation, and usage-type-lifec
4444

4545
Chosen option: "PDP-centric authorization", because it is the only option that satisfies the fail-closed posture and authorization NFRs without re-introducing a parallel access store inside the metering substrate. Every ingestion, query, deactivation, and UsageType lifecycle operation runs a single PDP check, and the PDP-returned constraints are applied to queries before the active plugin executes them. The collector deliberately keeps no per-tenant or per-UsageType access state and no PDP-decision cache, so policy evolution in the platform takes effect immediately and there is no drift surface to manage.
4646

47-
PDP enforcement is performed **per domain component**, not via a centralized adapter. Each domain component — ingestion-gateway, query-gateway, deactivation-handler, and usage-type-catalog — accepts `ctx: &SecurityContext` as the first parameter on every operation and calls `authz-resolver-sdk`'s `PolicyEnforcer::access_scope_with(ctx, ...)` inline through a thin shared helper. The shared `authz_scope` helper collapses the per-service authorization surface into a single definition site while keeping the call site local to each domain component. The collector defines this helper once and reuses it across its four domain components.
47+
PDP enforcement is performed **per domain component**, not via a centralized adapter. Each domain component — ingestion-gateway, query-gateway, deactivation-handler, and usage-type-catalog — accepts `ctx: &SecurityContext` as the first parameter on every operation and calls `authz-resolver-sdk`'s `PolicyEnforcer::access_scope_with(ctx, ...)` inline through a thin shared helper. The shared `access_scope_with` helper collapses the per-service authorization surface into a single definition site while keeping the call site local to each domain component. The collector defines this helper once and reuses it across its four domain components.
4848

4949
Authentication is owned upstream by the ToolKit gateway (REST surface) via `OperationBuilder::authenticated()` or supplied directly by the in-process caller (SDK trait surface). The collector accepts only a pre-resolved `SecurityContext` and never synthesizes identity, never resolves credentials, and never consumes a platform AuthN contract. Any operation arriving without a resolved `SecurityContext` is rejected at the surface boundary before any domain component runs.
5050

@@ -56,12 +56,12 @@ Authentication is owned upstream by the ToolKit gateway (REST surface) via `Oper
5656
- Query result scoping is uniformly driven by PDP-returned constraint filters; user-supplied filters can only narrow within the authorized scope.
5757
- The collector cannot independently authorize an emission or read; if the PDP is unreachable, the operation fails closed (no shadow allow path, no synthesized identity, no degraded mode).
5858
- Locality: PDP enforcement lives next to the operation it guards inside each domain component. Each component's helper invocation is reviewable in isolation alongside the business logic it gates.
59-
- Helper-consistency obligation: because the call is inlined per component rather than funneled through a single adapter, the shared helper signature (resource type, action verb, owner tenant, optional resource id, `require_constraints(true)`) must be maintained consistently across ingestion-gateway, query-gateway, deactivation-handler, and usage-type-catalog. Drift in the helper surface — or any component bypassing the helper — would weaken the uniformity guarantee that a centralized adapter would have made structural.
59+
- Helper-consistency obligation: because the call is inlined per component rather than funneled through a single adapter, the shared helper signature (resource type, action verb, owner tenant, optional resource id, and a `require_constraints` flag — `false` on the platform-global catalog path, `true` on the tenant-scoped usage-record and list paths, which additionally gate the returned scope against the operation's attribution) must be maintained consistently across ingestion-gateway, query-gateway, deactivation-handler, and usage-type-catalog. Drift in the helper surface — or any component bypassing the helper — would weaken the uniformity guarantee that a centralized adapter would have made structural.
6060
- Authentication boundary is external: because the collector accepts only a pre-resolved `SecurityContext`, its readiness model does not include an AuthN-client readiness fact. Only PDP-client readiness (`authz-resolver`) is probed at startup, simplifying the collector's failure surface but pushing the entire AuthN failure mode to the ToolKit gateway upstream.
6161

6262
### Confirmation
6363

64-
Compliance is confirmed through (a) design review against the §3.2 component model showing every domain component (ingestion-gateway, query-gateway, deactivation-handler, usage-type-catalog) routes through the per-service `authz_scope` helper before plugin dispatch, (b) authorization conformance tests covering permit / deny / constraint-filtered scenarios on every operation, and (c) negative tests confirming PDP unavailability surfaces as deterministic failure rather than fallback admission.
64+
Compliance is confirmed through (a) design review against the §3.2 component model showing every domain component (ingestion-gateway, query-gateway, deactivation-handler, usage-type-catalog) routes through the per-service `access_scope_with` helper before plugin dispatch, (b) authorization conformance tests covering permit / deny / constraint-filtered scenarios on every operation, and (c) negative tests confirming PDP unavailability surfaces as deterministic failure rather than fallback admission.
6565

6666
## Pros and Cons of the Options
6767

0 commit comments

Comments
 (0)