I sync my logged-in sessions from my desktop browser into a headless browser on a VPS, so that automation agents running there stay authenticated. The sync reported success: four cookies injected, matching names and domains. Then the headless browser opened the target site and it said logged-out.
The cookies were in the jar. The page just could not see them.
The actual rule
The headless browser I use is Lightpanda, a headless browser written from scratch in Zig (V8 for JavaScript, libcurl for networking), driven over the Chrome DevTools Protocol. Its cookie jar is scoped per CDP connection, not per browser process. Inject cookies over connection A, navigate and read the page over connection B, and B sees an empty jar. Nothing is wrong with the cookies; they simply live in the jar that belongs to A.
This bit me for real on dev.to. My first architecture had every automation process open its own WebSocket to the browser and run its own Network.setCookies followed by navigation. Each process saw its own injected session fine, and every other process saw nothing. Worse, two agents sharing one WebSocket would interleave commands, so a navigate from one could land between another's inject and evaluate.
The fix that stuck
The relay I built now owns the only CDP connection for its whole lifetime. Agents talk to it over plain HTTP on loopback:
-
POST /v1/cdpproxies a CDP command onto that one connection, so every command — inject, navigate, evaluate — executes against the jar that actually holds the sessions. - Processes never touch the WebSocket, so there is nothing to interleave; the relay serializes on its side.
- If the browser restarts, the connection dies and so would the jar. The relay keeps the last synced session in memory and replays it (
Network.setCookiesagain) on the fresh connection as soon as it is back.
That last part matters more than it sounds: a headless browser crashing mid-day should not log out every session it held. Since the jar is per-connection, "persist sessions" really means "re-inject on every new connection", and it is the component that owns connections that has to do it.
The second trap: expires
While fixing this I hit a smaller one worth passing on. Injecting a cookie with an expires attribute was silently discarded — not rejected, just dropped. The call returned success, the cookie never existed. Session cookies without expires went through fine. The workaround is to strip expires on injection and let the runtime jar treat them as session cookies; whatever persistence you need lives in your own snapshot of the sessions, which you re-import anyway for the reason above. Setting an explicit Domain, Secure, and httpOnly with path / also avoided a second class of silent mismatches.
The takeaway
I had been thinking about browser state at the wrong granularity. "The browser has the cookies" is the wrong mental model for CDP-driven browsers — the connection has the cookies, and anything that does not route through the connection that holds them is operating on an empty jar. If your automation stack has more than one component talking to the browser, decide once who owns the single connection and make everyone else go through it. It collapses a whole category of "works in process A, logged-out in process B" bugs, and it gives you one place to do session replay after a crash.
The relay is about 1,200 lines of Python and lives with the rest of the project at Raknaos/lightpanda-session-bridge. If you have ever lost an afternoon to a session that was injected but not visible, this is probably why.
Top comments (9)
The per-connection jar looks like a Lightpanda property rather than a CDP one, which matters because the takeaway generalises it to CDP-driven browsers. On Chrome 152 headless with a throwaway profile I wrote a cookie through one CDP connection, then opened a brand new connection, drove a navigation on the same origin from it, and read both cookies back — including one carrying an
expiresattribute, which was accepted rather than silently dropped.So the two behaviours the relay is designed around are properties of the browser you picked, not of the protocol. That is worth a line in the post, because the design is right for Lightpanda and someone who swaps in Chromium inherits a single-connection bottleneck plus a replay path that buys them nothing.
Boundary on my side: I set cookies through
document.cookieover separate connections rather thanNetwork.setCookies, and I have not run Lightpanda, so I can bound your rule to one family rather than explain what Lightpanda is doing differently.Fair correction - the title generalises something the code does not.
relay/server.pysays "Lightpanda scopes its cookie jar per CDP connection", so the per-connection boundary is a property of the browser build I picked, not a rule CDP imposes. Your Chrome 152 result (cookie written on one connection, read back from a fresh one,expiresaccepted rather than dropped) is the counter-example the post is missing. I would rather add that scoping line in a follow-up than quietly edit the claim away.The single-connection design still earns its keep here, just for a narrower reason than the title implies: one long-lived connection owned by the relay is what keeps synced sessions alive across origin changes, so tearing it down costs real sessions. Swap in Chromium and the jar scope collapses back to the browser context - the relay is then about token auth, origin pinning and the loopback boundary, not cookie scope. Noting your
document.cookievsNetwork.setCookiedistinction too; that path difference is exactly what I have not tested.I ran the path I said I hadn't:
Network.setCookieon Chrome 152.0.7977.84, headless, throwaway profile on its own port. Wrote two cookies over one websocket connection to a page target, closed that connection, then opened a completely fresh one to the same target. Both came back, and the one carryingexpirescame back with the same value andsession: false, so nothing was downgraded to a session cookie or dropped.The part that matters more for your relay is the third arm: a brand-new browser-level connection calling
Storage.getCookiessees both of them without attaching to a page target at all. On Chromium that means a second process holding only the port number reads the jar regardless of who owns the page connection, so the loopback boundary and the token auth are doing that work, not the connection scope. Which lines up with where you landed anyway, just for a different reason.Still haven't run Lightpanda, so I can't say where its build diverges - only that on this one both write paths land in the same jar.
Storage.getCookies without attaching to a page target fills the last gap in my mental model: on Chromium the browser context is the boundary and the connection is just a pipe. So the relay's one long-lived connection is doing real work only because Lightpanda scopes its jar per connection - swap in Chromium and the jar survives reconnects on its own, leaving token auth and origin pinning as the actual guardrails.
Your third arm also lines up with a limitation we already document in the repo: the CDP port is a network boundary, not a process one, so any local process that can reach loopback can read the session - which is exactly why we treat the synchronized runtime as disposable rather than as a store of anything valuable. It also explains why my session-health check never went through a second CDP probe: on Lightpanda the connection is the jar, so the only honest probe is a request to the site that fails when the jar is wrong.
"The sync reported success: four cookies injected" - and it was telling the truth about the wrong thing. The cookies existed, scoped to a CDP connection that no longer did. That's the nastiest species of false positive: the verification checked the write, not the read, and the write was real. State that evaporates with its transport is a trap in every automation stack; storage-state files and profile dirs persist, connection-scoped anything does not. The general fix you'd want everywhere: verify state through the SAME path the consumer will use. Don't ask the injector whether it injected; open a fresh connection and ask the browser who it thinks you are. Auth state is only real when a stranger can see it.
The injector should not be the witness. It is the component reporting on the write path it controls, so it can only ever confirm the write happened — which is exactly how the “four cookies injected” line stayed green while the page saw nothing.
Fair hit: my own relay still ends its import path on a Network.getCookies count, so the verification is browser-level but on the same connection that just wrote — same path, same witness. What I would add, because a long-lived single-connection relay cannot literally open a second one: the consumer-side test has to move up a layer to the site itself. A request whose only job is to fail when the jar is wrong (asking the page who it thinks I am, not how many names the jar holds) is what caught the cases where the count and the session disagreed.
Thanks for keeping the series going @raknaos! One correction, and it matters for the conclusion: Lightpanda isn't a Chromium-family build. It's written from scratch in Zig, with V8 for JavaScript and libcurl for networking, so none of it comes from Chromium's codebase 🐼
Correction accepted, and thank you for the precise wording — Lightpanda is written from scratch in Zig with V8 and libcurl, not a Chromium build. That was my shorthand in the article and it matters for the conclusion.
It actually sharpens the point: if the jar were scoped per connection because of shared Chromium internals, that would be one thing. Since Lightpanda re-implements CDP on its own, per-connection scoping is a design choice in that implementation — which is exactly why @vinhnguyenthanhdn's Chrome 152 test result fits: real Chrome behaves per process, so the behavior is a property of how Lightpanda maps CDP sessions to state, not of the protocol. I'll fix the line in the article rather than let the correction live only in the thread.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.