Fix session persistence, Cloudflare challenge, and empty get-my-articles - #2
Open
emregulerr wants to merge 2 commits into
Open
Fix session persistence, Cloudflare challenge, and empty get-my-articles#2emregulerr wants to merge 2 commits into
emregulerr wants to merge 2 commits into
Conversation
The authenticated tools returned empty results and forced repeated logins. Several overlapping root causes (see jackyckma#1): - Re-inject a full storageState snapshot on every launch so Medium's session cookies (sid/xsrf) survive the headed->headless switch and server restarts. A persistent profile alone drops pure session cookies on reopen. - Use one consistent User-Agent for both the login window and headless runs, and wait out Cloudflare's "Just a moment..." interstitial after navigation. The UA mismatch (Chrome vs HeadlessChrome) was invalidating cf_clearance. - Navigate to the correct published tab (?tab=posts-published) instead of /me/stories/public, which now redirects to Drafts. - Detect login from the actual page (no header Sign-in button) instead of cookie presence, since Medium sets sid/uid for anonymous visitors too. - Run headless in the background by default; the visible window only opens for the one-time login. Auto-login is triggered on demand by the other tools. This also fixes the documented Google-login persistence limitation: a Google login is now captured and reused silently (verified end-to-end). Closes jackyckma#1
Defense-in-depth: explicitly ignore the persistent auth session snapshot and Chrome profile directory (they hold login cookies). They live in the user's home dir today, but this guards against any future path change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1.
The authenticated tools returned empty results and forced repeated logins. While debugging I found several overlapping root causes in the browser layer and fixed them together.
What was wrong
sid/xsrf). A persistent profile keeps cookies with an expiry, but Chromium drops pure session cookies on close/reopen — so the headed→headless switch and server restarts silently logged the user out.cf_clearanceand re-triggered the challenge./me/stories/publicnow redirects to the Drafts tab; the published list is at?tab=posts-published.uid/sidfor anonymous visitors too, so the page rendered logged-out while the code thought it was authenticated.What changed
storageStatesnapshot on every launch (carries session cookies through the headed→headless switch and across restarts).Bonus: fixes the documented Google-login limitation
The README notes Google login sessions couldn't be persisted. With the persistent profile + storageState re-seeding, a Google login is captured and reused silently in the background — verified end-to-end (
get-my-articlesreturns all published stories headless, no window, no re-login).Testing
get-my-articlesreturns the full published list headless with no visible window and no re-login.npm run build/tsc --noEmitpass.Notes