Crawl API
Getting Started
There are two ways to use Crawl API:
- API-Based collection
- No-Code collection (via Control Panel)
API-Based collection
- Trigger a data collection job via a simple HTTP POST
- Specify the URLs and output format
- Receive a
snapshot_id
to retrieve results later.
Code Example
Query Parameters:
Your dataset ID (e.g., gd_m6gjtfmeh43we6cqc
)
include error logs in results
markdown
, html
, ld_json
, etc. \
Choose the format that best fits your workflow:
Delivery
Deliver results to:
- Webhooks
- External storage (S3, GCS, etc.)
- Direct download via API or Control Panel
No-Code Scraper (Control Panel)
Use our Control Panel to launch crawls without writing a single line of code. Steps:
- Open the Crawl API Control Panel
- Enter the target domain or URLs
- Choose your output format
- Start the crawl
- Download results directly from the dashboard