fix(cron): halt Sentry flood when a local LLM provider is offline (#4408)

[oxoxDev]

Summary

• Stop a cron agent_job from flooding Sentry (2802 events / 29 users, TAURI-RUST-12K) when it runs against a local LLM provider (LM Studio on localhost:1234) whose server isn't running.
• Add a 5th retry-halt guard to execute_job_with_retry for a loopback connection-refused — halt on first occurrence and skip the bypassing report, mirroring the existing 401 / 402 / 400 / api-key guards.
• Fix a locale bug in is_loopback_unavailable: it only matched the English "connection refused (os error N)" string, so non-English-locale hosts (this crash: a zh-CN Windows box) misrouted to the broad NetworkUnreachable bucket.

Problem

A cron agent job (e.g. morning_briefing) pinned to a keyless local provider fails at the TCP layer with error sending request for url (http://localhost:1234/v1/chat/completions): … tcp connect error … (os error 10061) — the user's local model server is offline.

The provider/agent emit sites already demote this via the expected_error_kind classifier (LoopbackUnavailable), so it never reaches Sentry there. But execute_job_with_retry burns every retry and then fires a bare report_error(failure=retries_exhausted) that bypasses the classifier demotion — exactly the leak already fixed for billing (#3913) and api-key (#4166) states, just for a different error class. Result: 2802 events / 29 users for a condition the app has no lever over.

Secondary: is_loopback_unavailable required the literal English connection refused (os error N). On a non-English OS the kernel renders WSAECONNREFUSED/ECONNREFUSED translated, so a genuine loopback-refused body leaked into the broader NetworkUnreachable bucket, blurring "local server down" vs "user network problem".

Solution

• observability.rs — add a locale-independent arm to is_loopback_unavailable: loopback host + the transport-stable tcp connect error marker + the connect-refused errno (61/111/10061). The distinct timeout errnos (60/110/10060) are deliberately not matched. Expose is_local_provider_unreachable_message as a single-source wrapper so the cron guard can't drift from the classifier.
• scheduler.rs — add is_local_provider_unreachable_failure (Agent-only, last_agent_error-first) and a local_unreachable flag; halt on the first occurrence and gate out the terminal report. Narrow by design — only loopback connection-refused matches, so a transient remote provider/backend error still retries and still reports. Modeled on the session_expired halt (no UserErrorCenter swap); the failure still surfaces via run history + cron alert.

S3.5 bucket: genuinely-unpreventable — the app has no lever to start a user's local model server, and the classifier already deems this kind expected at every other emit site. This is a consistency fix (the lone cron path that bypassed the existing demotion), not new suppression, and not on a data path.

Submission Checklist

• Tests added or updated (happy path + at least one failure / edge case) per Testing Strategy
• Diff coverage ≥ 80% — local diff-cover (cargo-llvm-cov lcov vs upstream/main) = 95% (observability.rs 100%, scheduler.rs 88%). All new logic is covered incl. the end-to-end integration test that drives execute_job_with_retry to a real loopback halt. The 3 "missing" scheduler.rs lines are pre-existing sibling-guard branches (credits/budget log arg + the non-halt terminal condition) exercised only by the credits/budget/exhaust tests that this focused run's filter excluded — green under the full CI suite.
• Coverage matrix updated — N/A: behaviour-only change (Sentry-noise suppression + classifier fix; no new feature row)
• All affected feature IDs from the matrix are listed in the PR description under ## Related
• No new external network dependencies introduced (test drives a keyless local provider against a dead loopback port — no server/mock needed; connection-refusal is the condition under test)
• Manual smoke checklist updated if this touches release-cut surfaces — N/A: does not change a release-cut surface
• Linked issue closed via Closes #NNN in the ## Related section

Impact

• Desktop (all platforms). Cron agent jobs only. No behaviour change for successful runs or for remote-provider failures.
• Removes a high-volume Sentry flood at source and keeps a real regression visible (the classifier still routes a build-≥-fix-release recurrence back to unresolved).
• No migration, no schema, no new deps. Halting a dead-local-provider job early also saves the wasted backoff retries.

Related

• Closes cron agent jobs flood Sentry when a local LLM provider (loopback) is offline #4408
• Sentry-Issue: TAURI-RUST-12K
• Siblings (same halt-guard pattern): fix(cron): stop cron billing-state Sentry floods — 402 credits + 400 budget (TAURI-RUST-514 / -BMW) #3913 (402/400), fix(cron): halt + stop Sentry flood when an agent job hits "API key not set" (#4164) #4166 (api-key-unset)
• Follow-up PR(s)/TODOs: optional — surface an offline-local-provider state in the UserErrorCenter (reuses feat(user-errors): surface missing-API-key + cron failures in UserErrorCenter (#4165) #4169 machinery); not needed to stop the flood.

---

AI Authored PR Metadata (required for Codex/Linear PRs)

Linear Issue

• Key: N/A (tracked on GitHub as cron agent jobs flood Sentry when a local LLM provider (loopback) is offline #4408)
• URL: cron agent jobs flood Sentry when a local LLM provider (loopback) is offline #4408

Commit & Branch

• Branch: fix/4408-cron-local-provider-unreachable
• Commit SHA: 7c63e2e37 (observability), 1fd148d84 (cron)

Validation Run

• N/A — pnpm --filter openhuman-app format:check: no frontend changes
• N/A — pnpm typecheck: no frontend changes
• Focused tests: cargo test --lib local_provider (14 passed: 5 guard predicate + classifier arms + 1 end-to-end integration)
• Rust fmt/check (if changed): cargo fmt --check clean; cargo check clean; cargo clippy --lib clean (pre-existing warning only)
• N/A — Tauri fmt/check (if changed): no app/src-tauri changes

Validation Blocked

• command: N/A
• error: N/A
• impact: N/A

Behavior Changes

• Intended behavior change: a cron agent job that fails because a local provider is offline halts on the first attempt and no longer emits a failure=retries_exhausted Sentry event.
• User-visible effect: none in-app (failure still shown in run history + cron alert); the observable effect is the Sentry flood stopping.

Parity Contract

• Legacy behavior preserved: remote/transient provider errors still retry across the backoff loop and still report retries_exhausted; the 4 existing halt guards are unchanged.
• Guard/fallback/dispatch parity checks: new guard is JobType::Agent-only and keyed on the shared classifier matcher; negative tests assert remote-host refused + shell jobs do NOT halt.

Duplicate / Superseded PR Handling

• Duplicate PR(s): none (checked open + 45d-merged; fix(provider): demote lm_studio 'No models loaded' 400 in chat #4189 is a sibling for a different error — lm_studio 400 "No models loaded", not connect-refused)
• Canonical PR: this
• Resolution: N/A

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 613f4487-57ce-46e1-87a4-0a3ac495bd94

📥 Commits

Reviewing files that changed from the base of the PR and between f979bfa and 1fd148d.

📒 Files selected for processing (3)

• src/core/observability.rs
• src/openhuman/cron/scheduler.rs
• src/openhuman/cron/scheduler_tests.rs

👮 Files not reviewed due to content moderation or server errors (3)

• src/core/observability.rs
• src/openhuman/cron/scheduler_tests.rs
• src/openhuman/cron/scheduler.rs

---

📝 Walkthrough

[!WARNING]

Walkthrough skipped

File diffs could not be summarized.

---

_{Comment @coderabbitai help to get the list of available commands.}