fix(cli): enforce --obey-robots

[lisa0314]

What changed

• make --obey-robots a global option and apply it to fetch and scrape
• fetch and cache /robots.txt per URL origin, including non-default ports
• stop before requesting a target disallowed by the selected robots rules
• forward the setting to scrape workers
• cover blocked, allowed, disabled, and scrape-worker behavior with a local HTTP fixture

Validation

• cargo nextest run --release --features render -p obscura-cli --test obey_robots --no-fail-fast: 4 passed
• cargo nextest run --release --features render -p obscura-net -p obscura-browser -p obscura-cli --no-fail-fast: 212 passed
• cargo build --release -p obscura-cli --bins --no-default-features
• cargo build --release -p obscura-cli --bins --features render
• full release/render nextest: 1408 passed; 2 timing-sensitive tests failed under the full parallel run and passed individually
• git diff --check

Rendering

Not applicable. The change affects CLI option handling and network request policy, not layout, paint, screenshots, or rendering fixtures.

Performance

No standalone CPU or memory benchmark was run. The default disabled path retains its previous behavior. When enabled, robots rules and empty policy results are cached per full URL origin so repeated targets do not refetch the same policy.

Checklist

• The change is focused on enforcing the existing CLI option.
• Focused integration coverage proves blocked targets are not requested.
• Allowed, disabled, non-default-port, cache, and scrape-worker behavior are covered.
• Render and no-render release builds passed.
• Rendering behavior is unaffected.

Fixes #642.