Skip to content

fix(sancta-claw): vendor agent-browser-cli via importCargoLock (crates.io 403s python-requests UA)#461

Merged
alexandru-savinov merged 1 commit into
mainfrom
fix/crates-io-403-vendor-ua
Jun 3, 2026
Merged

fix(sancta-claw): vendor agent-browser-cli via importCargoLock (crates.io 403s python-requests UA)#461
alexandru-savinov merged 1 commit into
mainfrom
fix/crates-io-403-vendor-ua

Conversation

@alexandru-savinov
Copy link
Copy Markdown
Owner

Problem

The NixOS Configuration Check → Build x86_64 Configs → Build sancta-claw job has been red on main for every push since the agent-browser-cli 0.14.0 vendor FOD fell out of the binary cache:

Exception: Failed to fetch file from https://crates.io/api/v1/crates/memchr/2.7.6/download. Status code: 403
error: 1 dependencies of derivation '...agent-browser-cli-0.14.0-vendor.drv' failed to build

Root cause (verified empirically)

rustPlatform.fetchCargoVendor (triggered by cargoHash) vendors crates with a Python requests-based fetcher (fetch-cargo-vendor-util.py) that sets no User-Agent, so it sends python-requests/*. crates.io's crawler policy now 403-blocks that UA. Confirmed from a clean host:

User-Agent Result
python-requests/2.32.3 403
curl/8.x 403
descriptive / empty UA 302 ✅
Nix's libcurl fetchurl (nix store prefetch-file) ✅ (not blocked)

This is the upstream issue rust-lang/crates.io#13482. The nixpkgs fix is NixOS/nixpkgs#512735 (set a descriptive UA), merged 2026-04-26 and backported to release-25.11 — but this flake's nixpkgs pin is 2026-03-18, before the backport.

Fix

Switch the single agent-browser-cli buildRustPackage call from cargoHash to cargoLock (importCargoLock), which fetches each crate via Nix's libcurl fetchurl — structurally immune to the crates.io UA policy. cli/Cargo.lock (v4) has only crates.io-registry deps and zero git sources, so no cargoLock.outputHashes are needed; crate hashes come from the lockfile's own checksums (one fewer hash to maintain than cargoHash).

  • Blast radius: minimal — 2-line change to one package; no overlay, no nixpkgs bump, no new files. Only sancta-claw is touched.
  • Revert to cargoHash once the nixpkgs pin carries the #512735 backport.

Alternatives considered (via a multi-agent design + judge workflow)

  • Overlay-patching fetchCargoVendor's UA (mirrors #512735) — fixes all Rust builds repo-wide, but rustPlatform.overrideScope rewrites the vendor builder on every host, and hardcodes the UA string crates.io currently accepts. Runner-up.
  • Bumping the nixpkgs input — ~3656 commits / ~2.3 months of churn rebuilding every host to unblock one CI job. Rejected as primary; fine as later routine maintenance.

Verification (local, aarch64-darwin)

  • nixpkgs-fmt --check clean
  • nix eval .#nixosConfigurations.sancta-claw...toplevel.drvPath → valid drv (IFD reading cli/Cargo.lock works)
  • importCargoLock vendor dir builds end-to-end — all 43 crates fetched via libcurl, zero 403 (/nix/store/...-cargo-vendor-dir)
  • rpi5-full still evaluates (unaffected)
  • The x86_64-linux compile/link is CI-only — exercised by the previously-failing Build sancta-claw job.

🤖 Generated with Claude Code

…s.io 403s python-requests UA)

The "Build sancta-claw" CI job has failed on every push since the
agent-browser-cli 0.14.0 vendor FOD fell out of the binary cache:
crates.io returns HTTP 403 to rustPlatform.fetchCargoVendor because its
Python `requests`-based fetcher sends a `python-requests/*` User-Agent,
which crates.io's crawler policy now blocks (rust-lang/crates.io#13482).

Switch the single agent-browser-cli buildRustPackage call from cargoHash
to cargoLock (importCargoLock), which fetches each crate via Nix's
libcurl fetchurl — not blocked by crates.io. The CLI lockfile
(cli/Cargo.lock, v4) has only crates.io-registry deps and zero git
sources, so no cargoLock.outputHashes are needed and there is one fewer
hash to maintain than cargoHash.

Upstream fix is NixOS/nixpkgs#512735 (set a descriptive User-Agent),
backported to release-25.11 on 2026-04-26 but not yet in this flake's
nixpkgs pin (2026-03-18); revert to cargoHash once the pin carries it.

Verified locally on aarch64-darwin: sancta-claw evaluates to a valid
drvPath, and the importCargoLock vendor dir builds end-to-end (all 43
crates fetched via libcurl, zero 403). The x86_64-linux compile/link is
exercised by the previously-failing "Build sancta-claw" CI job.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 3, 2026

Code Review PR 461: Switch Rust CLI to cargoLock.lockFile. The change replaces cargoHash with cargoLock.lockFile to work around crates.io HTTP 403ing the python-requests User-Agent used by fetchCargoVendor. FINDINGS: LOW - verbose comment but well justified. ANALYSIS: (1) cargoLock.lockFile path reference is valid Nix syntax. (2) No outputHashes needed since lockfile has only crates.io-registry deps. (3) version/cargoRoot/buildAndTestSubdir unchanged. (4) Integrity equivalent to cargoHash. (5) Revert condition documented. VERDICT: PASS - no critical or high severity issues.

@alexandru-savinov alexandru-savinov merged commit 0e14aab into main Jun 3, 2026
14 checks passed
@alexandru-savinov alexandru-savinov deleted the fix/crates-io-403-vendor-ua branch June 3, 2026 17:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant