Skip to content

The limit for client-initiated navigations becomes adjustable and is named after what it counts - #666

Open
aech wants to merge 1 commit into
h4ckf0r0day:mainfrom
aech:fix/navigation-chain-limit
Open

The limit for client-initiated navigations becomes adjustable and is named after what it counts#666
aech wants to merge 1 commit into
h4ckf0r0day:mainfrom
aech:fix/navigation-chain-limit

Conversation

@aech

@aech aech commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

What this is about

The loop in navigate_with_wait_post_inner limits how many documents a navigation chain may load. It counts navigations the page itself triggers after each load: assignments to location and form submissions. In the language of CDP these are scriptInitiated and formSubmissionGet/formSubmissionPost from Page.ClientNavigationReason.

HTTP 3xx redirects never reach this loop. They are followed one layer down, in obscura-net, with their own limit and their own error.

Two things were wrong here.

The name pointed at the wrong layer

The message read Too many redirects (limit 10). A chain of client-initiated navigations thus read as a redirect storm.

The name costs time. I investigated an abort in an SAP login chain and measured the redirect chain, because the message spoke of redirects. The cause was one layer up, in a JavaScript loop.

On top of that, the name was assigned twice. ObscuraNetError::TooManyRedirects means real HTTP redirects. After this change, "Too many redirects" means only that across the whole tree.

The limit could not be raised

const REDIRECT_LIMIT: usize = 10 stood inside the function. No CDP parameter, no env, no setter.

The low default is right and stays. It is what stops a page that resets location on every load. What was wrong is only that an endpoint that chains longer for good reasons remained unreachable.

What changes

The limit gets the shape navigation_timeout already has: a per-page field, a setter, a getter, and a fallback to the environment.

page.set_navigation_chain_limit(20)   programmatic, takes precedence
OBSCURA_NAV_CHAIN_LIMIT=20            in operation
neither set                           unchanged default 10

The number means how many documents a chain may load, the first one included. So the default of 10 allows the requested document and nine client-initiated navigations on top. That is exactly the previous behaviour.

The message names this unit explicitly, because the number is now something an operator acts on. If the word "navigations" had only limit 10 next to it, one would read ten navigations and, when eleven are needed, set the limit to eleven, exactly one too low.

Values below 1 are raised to 1 on both paths. A zero would let the loop run without a single iteration, the page would report success and have loaded nothing. Only an unreadable value from the environment falls back to the default. OBSCURA_NAV_CHAIN_LIMIT=0 and set_navigation_chain_limit(0) therefore mean the same: load the first document and do not chain further.

The new switch stands in docs/Environment-variables.md next to its twin OBSCURA_NAV_TIMEOUT_MS.

Deliberately not included: a CDP parameter on Page.navigate and a switch on the CLI. navigation_timeout manages without both, and a non-standard parameter on a CDP command would be the larger intervention.

Tests

Seven new tests in crates/obscura-browser/src/page.rs. Three check the pure function that reads the environment, as the tests for navigation_timeout demonstrate. Four drive a real chain against a local server whose pages set location.href.

The four chain tests always set their limit themselves. If they inherited it, the page would read OBSCURA_NAV_CHAIN_LIMIT from the process environment, and anyone running the suite with exactly the switch this PR introduces would see it fail through no fault of the code. Measured: with OBSCURA_NAV_CHAIN_LIMIT at 20, 3, 0, and an unreadable value, all seven stay green.

Seven mutations, each caught by exactly the test that means it, plus two unmutated runs as a baseline, one of them with the environment variable set:

Mutation caught by
loop uses the constant again raised limit reaches the longer chain
default 10 becomes 11 chain beyond the default reports the error
env path no longer raises 0 to 1 zero is raised to 1
setter no longer raises 0 to 1 limit 0 still loads the first document
old error text restored chain beyond the default reports the error
message hides the unit again chain beyond the default reports the error
fixture inherits the environment again chain beyond the default reports the error

cargo nextest run --features render -p obscura-browser runs green with 70 tests, cargo check --workspace and --features render,stealth without errors.

Note on scope

The rename changes a public variant of PageError and with it the string a CDP client sees. domains/page.rs passes the error through via map_err(|e| e.to_string()). Exactly this string is the subject of the change. No other place in the tree reads the variant or the text, checked across the whole tree.

Part of #664.

The loop in navigate_with_wait_post_inner limits how many documents
a navigation chain may load. It counts navigations the page itself
triggers after each load: assignments to location and form
submissions, which CDP's Page.ClientNavigationReason calls
scriptInitiated and formSubmission*. HTTP 3xx redirects never reach
this loop. They are followed one layer down, in obscura-net, with
their own limit and their own error.

The message still read "Too many redirects (limit 10)", so a chain
of client-initiated navigations looked like a redirect storm. This
name sends an investigation to the wrong layer. It points at the
redirect chain while the cause is a JavaScript loop.

The limit was also a function-local constant with no way to raise
it. The low default is right and stays 10, but an endpoint that
chains longer than nine navigations for good reasons was simply
unreachable. It is now a per-page value with a fallback to the
environment, in the same form navigation_timeout already has:

  page.set_navigation_chain_limit(20)   programmatic, takes precedence
  OBSCURA_NAV_CHAIN_LIMIT=20            in operation
  neither set                           unchanged default 10

The number counts documents, the first navigation included. So 10
allows the requested document and nine client-initiated navigations
on top. The error message names this unit. The number is now
something an operator acts on, and "limit 10" next to the word
"navigations" would read as ten navigations and send them exactly
one too low.

Values below 1 are raised to 1 on both paths. A zero would let the
loop run without a single iteration, the page would report success
and have loaded nothing. Only an unreadable value from the
environment falls back to the default. A 0 therefore means the same
on both paths: load the first document and do not chain further.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant