Trawley

— MCP / GETTING STARTED

Connect Trawley to your IDE

Trawley exposes a hosted MCP server that lets Claude Code, Cursor, or Codex build and query Trawley scrapers from inside your editor. Generate a key, paste a snippet, run your first tool call. Under five minutes.

— 01 / KEY

Generate an API key

Open the dashboard, pick a team, and create a key. You'll see it once.

— 02 / CONNECT

Paste the config snippet

Drop the JSON or CLI command into Claude Code, Cursor, or Codex.

— 03 / CALL

Run your first tool

Ask the agent to list your scrapers. That's the cheapest smoke test.

— STEP 01

Generate an API key

  1. 1. Open the API keys page in the dashboard. You need to be a team admin.
  2. 2. Click Generate API key, give it a name that identifies the machine (e.g. Claude Code (laptop)), and copy the key when it appears.
  3. 3. The key is shown once. Store it in a password manager, a secret store, or an environment variable. If you lose it, revoke it and generate a new one.

Keys are team-scoped and full-access

A key grants read and write to every scraper in its team. There are no per-scraper or read-only keys in v0.5. Treat them like deploy keys: one per machine, revoke immediately if leaked.

The rest of this guide assumes you've set TRAWLEY_API_KEY in your shell environment.

export TRAWLEY_API_KEY="trw_t_..."

— STEP 02

Paste the config into your IDE

Trawley speaks MCP over Streamable HTTP, the transport every major IDE client supports in 2026. Pick your editor:

The fastest path is the claude mcp add CLI:

claude mcp add --transport http trawley https://api.trawley.ai/api/mcp \
  --header "Authorization: Bearer $TRAWLEY_API_KEY"

Or drop the JSON form directly into ~/.config/claude/mcp_servers.json (path may vary by platform):

{
  "mcpServers": {
    "trawley": {
      "type": "http",
      "url": "https://api.trawley.ai/api/mcp",
      "headers": {
        "Authorization": "Bearer ${TRAWLEY_API_KEY}"
      }
    }
  }
}

Restart Claude Code after editing the file. Run claude mcp list to confirm the server is connected.

Progress events are best-effort

Some IDE versions (notably Claude Code prior to the fix for regression #51713 and Cursor before 1.7.28) don't render MCP progress notifications inline. Trawley emits progress while a build is running, but the final tool reply is the load-bearing surface: scraper id, final message, and any error always arrive in the terminal CallToolResult regardless of progress rendering.

— STEP 03

Run your first tool call

The cheapest smoke test is trawley_list_scrapers: zero arguments, fast, no billing impact. Ask the agent to list your scrapers and confirm you see expected names.

User:    "List the scrapers on my Trawley team."
Agent:    (calls trawley_list_scrapers)
Trawley:  {
  "scrapers": [
    {
      "id": "scr_...",
      "name": "Acme Homes — Bristol 3-beds",
      "url": "https://acmehomes.co.uk/bristol/3-bed",
      "type": "LIST",
      "updated_at": "2026-05-29T10:14:00Z",
      "last_job": { "status": "COMPLETED", "result_count": 47 }
    }
  ]
}

From there:

  • Build a new scraper: "Use Trawley to build a scraper for acmehomes.co.uk that extracts price, bedrooms, and location."
  • Query an existing scraper: "Find me 3-bed houses with a garden under £500k from the Acme Homes scraper."
  • Check job health: "Is the latest scrape job on my Acme Homes scraper healthy?"

Full per-tool input and return shapes live in the tool reference .

Scraper IDs are confidential

Trawley's existing /api/v1/scrapers/[id]/* endpoints currently lack team-isolation enforcement (rate-limited only, no auth check). The MCP layer enforces team isolation correctly, but the underlying v1 surface is still callable by anyone with a scraper id. A follow-up plan is committed to migrate within 30 days of v0.5 ship.

Until that ships, treat scraper ids as confidential:

  • ·Don't paste them into public transcripts, blog posts, or commits.
  • ·Don't log them where third parties (CI, error tracking) might mirror them.
  • ·Rotate scrapers if an id was shared accidentally.

Troubleshooting

The IDE says the server is connected, but no tools show up.

Check that the Authorization header evaluates correctly. A missing or unexpanded $TRAWLEY_API_KEY will let the transport connect (the request reaches the server) but every tools/list call returns 401. Run echo $TRAWLEY_API_KEY in the same shell that launches the IDE.

I get RATE_LIMIT_PER_SECOND immediately.

The transport enforces a per-key cap of 5 requests/sec. Most IDEs fan out tools/list + tools/call in parallel — short bursts are normal. If you're hitting the cap repeatedly, the agent is probably retrying in a tight loop. See the error reference for back-off guidance.

A build takes minutes and my IDE times out the request.

v0.5 runs builds inside a single HTTP request: if your IDE's transport timeout is lower than the build duration, the connection drops and the build is lost mid-flight. Mitigation: ask the agent to break the goal into smaller steps, or rerun and pass the scraperId from the partial commit to continue editing. Stateful, resumable sessions are on the v1 roadmap.

I see SCRAPER_TEAM_MISMATCH when calling tools.

The scraperId you passed belongs to a different team than the API key. Common cause: a personal-team key used against a work-team scraper id, or copy-pasted ids between transcripts. Call trawley_list_scrapers to confirm what the current key can see.

Every code is documented in the error reference with the recommended next action.

Browse the tools

Seven tools cover build, list, inspect, and query. Each one is self-describing — your agent can pick the right tool on the first turn.

Tool reference