Trawley
Getting started

How Trawley works

The handful of ideas behind scrapers, fields, jobs, and results.

Trawley has a small vocabulary. Learn these five words and the rest of the product makes sense.

Scraper

A scraper points at a website and extracts a list of items from it, for example every property on a listings page or every product in a catalogue. A scraper is the thing you build once and reuse.

Every Trawley scraper works on lists. You give it a page that shows many items, and it captures each one as a record.

Field

A field is a single piece of data you want from each item, like price, bedrooms, or location. You choose the fields when you build the scraper, and the wizard helps you map them to the right part of the page.

Job

A job is one run of a scraper. Each time the scraper collects data, whether you run it by hand or on a schedule, that is a job. Jobs move through a simple lifecycle: pending, running, then completed or failed.

Result

A result is one item the scraper captured during a job, with a value for each field. A run over a listings page produces one result per listing.

Team

Your scrapers belong to a team. Teams let you share scrapers and collaborate, and they are where billing and plan limits live. You can belong to more than one team and switch between them.

How they fit together

text
Team
 └── Scraper (points at a site, defines fields)
       └── Job (one run)
             └── Result (one captured item, with field values)

What's next