Apify logo

Crawl Website Content (Apify)

Apify → Crawl Website Content (Apify)

apify-website-content-crawlerenrichmentReturns in seconds

Crawls website pages and extracts structured content including text, metadata, and URLs.

Input

What it needs.

Map each field to a column in your table. Optional fields can stay empty.

FieldType
  • url
    Website URL
    text

When to use this.

The "Crawl Website Content (Apify)" action is designed to efficiently crawl specified website pages and extract structured content, including text, metadata, and URLs. By providing a single input parameter, the website URL, users can initiate the crawling process. The action outputs essential data fields such as page text, title, description, canonical URL, language code, loaded URL, and referrer URL, all formatted as text or URL types. This action is particularly useful for data enrichment, enabling businesses to gather insights from web content for SEO analysis, competitive research, and content aggregation. With its ability to deliver structured data, it supports various applications in digital marketing, content management, and web analytics.

Output

What it returns.

Every field this action writes back to the row. Click a line to expand nested objects.

FieldType
  • page_text
    Page Text
    text
  • page_title
    Page Title
    text
  • page_description
    Page Description
    text
  • page_canonical_url
    Page Canonical Url
    nullabletext
  • page_language_code
    Page Language Code
    nullabletext
  • crawl_loaded_url
    Crawl Loaded Url
    text
  • crawl_referrer_url
    Crawl Referrer Url
    nullabletext

Workflow

A real example.

Trigger → crawl website content (apify) → enrich → push to your CRM. You set it up once; after that it runs without you.

Step 1TriggerNew row, form submit, or schedule.
Step 2Crawl Website Content (Apify)The Apify action runs.
Step 3EnrichWaterfall fills missing fields.
Step 4Push to CRMHubSpot / Salesforce / Pipedrive.

Built for

Who runs this.

GTM EngineerRevOpsAgency

Reliability

Throughput & reliability.

How fast it drains a table, and what happens when the upstream provider has a bad day.

  • Per-minute limit30 / min
  • Per-day limit5,000 / day
  • RetriesAutomatic w/ backoff
  • Failed lookupsZero credits

Pricing

What it costs to run.

Standard tier

Pricing tier on /pricing. Per-action credit cost is private.

FAQ.

  • Is this real-time?

    Mostly. Fast actions fill the cell in a few seconds. Long-running ones run in the background and write back to the row when they finish, so you can close the tab.

  • Do I get charged on failure?

    No. A failed lookup costs zero credits. When a provider misses or times out, the waterfall moves to the next one and you only pay for the match you actually got.

  • Do I have to run it row by row?

    No. Run it on one cell, on a selection, on the whole table, or on a schedule. Same action either way.

  • What CRMs can I push results to?

    HubSpot, Salesforce, Pipedrive, Zoho, and GoHighLevel are bidirectional. Smartlead, Instantly, Lemlist, HeyReach, Apollo Sequences, and Reply.io for outbound.

Run Crawl Website Content (Apify) in 60 seconds.

Connect the account, drop the action on a column, run it on your rows.