Skip to content

browser.md

Latest commit

 

History

History
168 lines (124 loc) · 9.72 KB

browser.md

File metadata and controls

168 lines (124 loc) · 9.72 KB
Raw

The live browser pane

Cardloop embeds a real browser inside the cockpit that the agent drives and you watch — live, over a CDP screencast. You can also drive it yourself, alongside the agent (co-control): click, scroll, and type — including logins — on both desktop and mobile. When you ask the agent to "open", "launch", or "use the browser", it drives this pane, not some invisible headless process.

It's an opt-in module (off by default). Turn it on in Settings → Extensions → Browser; a 🌐 Browser tab then appears in each project.

Three backends (all optional, pick per taste)

The browser backend is pluggable — one swap point, three tiers. The built-in default works out of the box; the other two are opt-in and you can switch any time in Extensions → Browser.

Backend What it is Setup Cost
Built-in Chromium (default) Vanilla Chromium via Playwright — no stealth one Playwright install (below) free
CloakBrowser Anti-detect stealth Chromium (beats most bot detection) one-click install in the UI free tier (MIT)
External CDP / Cloak Manager Connect to any CDP browser, or to persistent logged-in profiles bring your own endpoint / Manager your own infra

A. Built-in Chromium — the default

Zero config beyond a one-time Playwright install. Nothing stealthy, but perfect for general browsing and scraping where you don't need to defeat bot detection.

venv/bin/pip install playwright && venv/bin/playwright install chromium

Then enable the Browser module and leave the backend on Built-in Chromium.

B. CloakBrowser — free stealth, one click

cloakbrowser is a public, MIT-licensed package with a free tier: a patched Chromium (downloaded from GitHub Releases, no key, no limit) that defeats most anti-bot fingerprinting (Cloudflare Turnstile, FingerprintJS, reCAPTCHA v3). navigator.webdriver reads false, the UA looks like a real desktop browser, etc.

In Extensions → Browser, pick CloakBrowser (stealth) → click Install CloakBrowser (free). That runs, detached:

venv/bin/pip install cloakbrowser && venv/bin/python -m cloakbrowser install

A Pro tier (latest Chromium, subscription) exists too, but the free tier is all you need.

C. External CDP / Cloak Manager — persistent logged-in profiles

Point the pane at any browser speaking the Chrome DevTools Protocol. Two ways:

  • Static CDP URL — any Chrome started with --remote-debugging-port, a Browserless/Steel endpoint, etc. Just paste the URL.
  • Cloak Manager — a service that manages persistent profiles: each profile is an isolated fingerprint with its own cookies/localStorage that survive restarts. You log into a site once (handling the captcha/2FA yourself in the Manager's built-in noVNC viewer), and from then on the agent reuses that real, authenticated session over CDP. Set the Manager URL + token in Extensions → Browser → Cloak Manager, click Load profiles, and Use the one you want.

Cardloop ships only the client. There is no bundled Manager and no hardcoded URL or token — you run your own Manager (or use any CDP browser) and enter your endpoint in the UI. Your credentials go to the encrypted secret vault, never to modules.json or git.

Profiles are stopped again when idle. Cardloop launches a Manager profile on demand, so it also stops it: once the last project detaches and CLOAK_PROFILE_IDLE_STOP seconds pass (default 900) the profile is shut down. Cookies and logins live in the profile's on-disk user-data-dir, not in the running process, so nothing is lost — the next use starts it again with the session intact. A profile that was already running when Cardloop connected is treated as yours and is never stopped, no matter how long it idles. Set CLOAK_PROFILE_IDLE_STOP=0 to disable this entirely.

This matters more than it sounds on a VM without GPU passthrough: Chrome falls back to swiftshader (software GL on the CPU), so an idle-but-open profile is not free. Two forgotten profiles once held 55 Chrome processes between them and pinned the host at 445% CPU / 70°C for ten hours. GET /api/browser/profile-usage shows what the cockpit currently holds open and which sessions are attached.

REST vs CDP host split. If your Manager's REST API sits behind a CDN/WAF (fine for JSON) but raw CDP websockets need a directly-reachable address, set CLOAK_MANAGER_CDP_BASE in .env to the internal host (e.g. http://10.0.0.5:8080). REST keeps using the Manager URL; CDP uses this base. Unset → both use the Manager URL.

Co-control: you and the agent, together

The pane is not a one-way screencast. The same browser session is driven by both the agent (via its tools) and you (mouse/keyboard in the pane):

  • Desktop — click to focus the pane, then click/scroll/type normally. Editing keys (Backspace, Delete, Tab, Esc, Home/End, arrows) and modifier shortcuts (Ctrl/⌘+A, Shift-select) are forwarded with their real virtual key codes, so the remote page treats them as a physical keyboard would.
  • Mobile — tap = click, swipe = scroll, and tapping a field raises the on-screen keyboard so you can type logins/passwords from your phone. A key row above the frame adds what a soft keyboard lacks: Esc ⇥ ⌫ Del ← → ↑ ↓ ⏎ and a 📋 paste button.
  • Paste — Ctrl/⌘+V (or 📋) pastes your clipboard into the remote page. It has to work this way: the remote Chromium has its own, empty clipboard, so a forwarded Ctrl+V would paste nothing. The text is inserted at the caret via Input.insertText.
  • History — ← → ⟳ next to the URL bar; tabs are on the strip above it.

This is the intended workflow for logged-in profiles: the agent navigates, and you handle the sensitive bits (passwords, captcha, 2FA) right there in the same live session.

What persists. Cookies and localStorage live in the browser profile, not in the pane — so with a Cloak Manager profile (or any external CDP browser with a real user-data dir) a login you do in the pane survives restarts and is reused by every project on that profile. The built-in and CloakBrowser backends launch a fresh, empty browser each time — nothing persists there. Chromium's own password manager ("Save password?") is browser chrome, not page content, so it never appears in the screencast: what carries over is the session, not a saved password. Keep passwords in the vault (secret set) and paste them in.

Agent-action safety gate

Because a logged-in profile means the agent acts as your identity, mutating actions are gated. In Extensions → Browser → Agent actions:

  • Read only (default) — the agent may navigate and read the page (snapshot), nothing more.
  • Full — the agent may also click and type (submit forms, post, etc.).

Read tools are always allowed; click/type are refused with a note until you flip the gate to Full. Keep it on Read only for logged-in profiles unless you explicitly want the agent acting on your behalf.

The agent reaches the pane through MCP tools browser_navigate, browser_snapshot, browser_click, and browser_type — exposed only while the Browser module is enabled.

Captchas (optional)

Set TWOCAPTCHA_API_KEY in .env (or store twocaptcha_api_key in the encrypted safe) and the agent gains browser_solve_captcha. Leave it unset and the tool reports "not configured" — the agent is never even told it exists, so it can't waste a turn on it.

The image grid is never answered. For reCAPTCHA v2/v3, hCaptcha and Turnstile the widget is not touched at all: the captcha is solved externally, and the resulting response token is written into the page's hidden field and handed to the site's callback. "Select all fire hydrants" and the plain "I'm not a robot" checkbox are the same task type — the tiles never have to be clicked. Old-style distorted-text captchas are different: pass image_selector and the answer comes back as text for the agent to type, since there's no token field to inject into.

Gated behind the same Full agent-actions switch as click/type — injecting a token can submit a form as your logged-in identity. Each call costs real money (~$0.001–0.003), so the tool is documented as "confirm a captcha is actually there first".

What it will refuse. A full-page Cloudflare interstitial ("Verify you are human", not a widget in a form) is detected and declined with an explanation instead of being paid for. That flow needs action/cData/chlPageData scraped out of the page's own turnstile.render call, and the token is bound to the solver's IP — a token solved from someone else's address is rejected when replayed from yours. Pass that one by hand in the pane; with a persistent profile the session then sticks.

Note that solving captchas automatically is against the terms of service of many sites, and repeated anti-bot triggers get the whole profile flagged, not just one request. This is a tool for getting your own automation through your own accounts — treat the cost and the ban risk as real.

TL;DR

  • A fresh install gets the built-in Chromium (Playwright) — safe, no stealth, one install command.
  • Want stealth? One click installs the free CloakBrowser tier.
  • Want the agent to work inside your already-logged-in sessions? Run your own Cloak Manager (or any CDP browser) and point the cockpit at it — nothing is shared or hardcoded.
  • The agent is read-only by default; you decide when it may click and type.
  • Captchas: optional, off unless you add a 2captcha key. Widgets yes, full-page Cloudflare no.