Skip to content

Fix session persistence, Cloudflare challenge, and empty get-my-articles - #2

Open
emregulerr wants to merge 2 commits into
jackyckma:mainfrom
emregulerr:fix/session-persistence-cloudflare
Open

Fix session persistence, Cloudflare challenge, and empty get-my-articles#2
emregulerr wants to merge 2 commits into
jackyckma:mainfrom
emregulerr:fix/session-persistence-cloudflare

Conversation

@emregulerr

Copy link
Copy Markdown

Fixes #1.

The authenticated tools returned empty results and forced repeated logins. While debugging I found several overlapping root causes in the browser layer and fixed them together.

What was wrong

  1. Session cookies lost across context restarts. Medium's auth relies on session cookies (sid/xsrf). A persistent profile keeps cookies with an expiry, but Chromium drops pure session cookies on close/reopen — so the headed→headless switch and server restarts silently logged the user out.
  2. Cloudflare "Just a moment…" challenge was never waited out, so extraction ran against the interstitial. Made worse by a User-Agent mismatch (headed "Chrome" vs headless "HeadlessChrome"), which invalidated cf_clearance and re-triggered the challenge.
  3. Wrong stories URL/me/stories/public now redirects to the Drafts tab; the published list is at ?tab=posts-published.
  4. Login detected from cookie presence only — Medium sets uid/sid for anonymous visitors too, so the page rendered logged-out while the code thought it was authenticated.
  5. Outdated selectors for the stories list.

What changed

  • Re-inject a full storageState snapshot on every launch (carries session cookies through the headed→headless switch and across restarts).
  • One consistent User-Agent for login + headless runs, plus an explicit Cloudflare wait after each navigation.
  • Correct published-tab URL and more robust article extraction.
  • Auth detected from the actual page (absence of the header Sign-in button).
  • Headless background by default; the visible window opens only for the one-time login, and the other tools trigger auto-login on demand.

Bonus: fixes the documented Google-login limitation

The README notes Google login sessions couldn't be persisted. With the persistent profile + storageState re-seeding, a Google login is captured and reused silently in the background — verified end-to-end (get-my-articles returns all published stories headless, no window, no re-login).

Testing

  • Reproduced the session-cookie drop with a minimal persistent-context add/close/reopen test.
  • Verified end-to-end: after a one-time login, get-my-articles returns the full published list headless with no visible window and no re-login.
  • npm run build / tsc --noEmit pass.

Notes

  • The User-Agent pins a Chrome major version to match the bundled Chromium; it may need a bump on future Playwright upgrades.
  • The saved session file contains auth cookies and should be treated as sensitive (kept in the user's home dir, git-ignored).

The authenticated tools returned empty results and forced repeated logins.
Several overlapping root causes (see jackyckma#1):

- Re-inject a full storageState snapshot on every launch so Medium's session
  cookies (sid/xsrf) survive the headed->headless switch and server restarts.
  A persistent profile alone drops pure session cookies on reopen.
- Use one consistent User-Agent for both the login window and headless runs,
  and wait out Cloudflare's "Just a moment..." interstitial after navigation.
  The UA mismatch (Chrome vs HeadlessChrome) was invalidating cf_clearance.
- Navigate to the correct published tab (?tab=posts-published) instead of
  /me/stories/public, which now redirects to Drafts.
- Detect login from the actual page (no header Sign-in button) instead of
  cookie presence, since Medium sets sid/uid for anonymous visitors too.
- Run headless in the background by default; the visible window only opens for
  the one-time login. Auto-login is triggered on demand by the other tools.

This also fixes the documented Google-login persistence limitation: a Google
login is now captured and reused silently (verified end-to-end).

Closes jackyckma#1
Defense-in-depth: explicitly ignore the persistent auth session snapshot and
Chrome profile directory (they hold login cookies). They live in the user's
home dir today, but this guards against any future path change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix: empty get-my-articles + repeated logins (session persistence, Cloudflare, wrong stories tab)

1 participant