Apify Ultimate Scraper

Universal Web Scraper

AI-driven data extraction from 55+ Actors across all major platforms. This skill automatically selects the best Actor for your task.

Prerequisites

(No need to check it upfront)

• .env file with APIFY_TOKEN
• Node.js 20.6+ (for native --env-file support)

Workflow

Copy this checklist and track progress:

Copy code
Task Progress:
- [ ] Step 1: Understand user goal and select Actor
- [ ] Step 2: Fetch Actor schema
- [ ] Step 3: Ask user preferences (format, filename)
- [ ] Step 4: Run the scraper script
- [ ] Step 5: Summarize results and offer follow-ups

Step 1: Understand User Goal and Select Actor

First, understand what the user wants to achieve. Then select the best Actor from the options below.

Instagram Actors (12)

Facebook Actors (14)

TikTok Actors (14)

YouTube Actors (5)

Google Maps Actors (4)

Other Actors (6)

Actor Selection by Use Case

Multi-Actor Workflows

For complex tasks, chain multiple Actors:

Can't Find a Suitable Actor?

If none of the Actors above match the user's request, search the Apify Store directly:

bashCopy code
node ${CLAUDE_PLUGIN_ROOT}/reference/scripts/search_actors.js --query "SEARCH_KEYWORDS"

Replace SEARCH_KEYWORDS with 1-3 simple terms (e.g., "LinkedIn profiles", "Amazon products", "Twitter").

Step 2: Fetch Actor Schema

Fetch the Actor's input schema and details:

bashCopy code
node --env-file=.env ${CLAUDE_PLUGIN_ROOT}/reference/scripts/fetch_actor_details.js --actor "ACTOR_ID"

Replace ACTOR_ID with the selected Actor (e.g., compass/crawler-google-places).

This returns:

• Actor info (title, description, URL, categories, stats, rating)
• README summary
• Input schema (required and optional parameters)

Step 3: Ask User Preferences

Skip this step for simple lookups (e.g., "what's Nike's follower count?", "find me 5 coffee shops in Prague") — just use quick answer mode and move to Step 4.

For larger scraping tasks, ask:

1. Output format:
   • Quick answer - Display top few results in chat (no file saved)
   • CSV - Full export with all fields
   • JSON - Full export in JSON format
2. Number of results: Based on character of use case

Cost safety: Always set a sensible result limit in the Actor input (e.g., maxResults, resultsLimit, maxCrawledPages, or equivalent field from the input schema). Default to 100 results unless the user explicitly asks for more. Warn the user before running large scrapes (1000+ results) as they consume more Apify credits.

Step 4: Run the Script

Quick answer (display in chat, no file):

bashCopy code
node --env-file=.env ${CLAUDE_PLUGIN_ROOT}/reference/scripts/run_actor.js \
  --actor "ACTOR_ID" \
  --input 'JSON_INPUT'

CSV:

bashCopy code
node --env-file=.env ${CLAUDE_PLUGIN_ROOT}/reference/scripts/run_actor.js \
  --actor "ACTOR_ID" \
  --input 'JSON_INPUT' \
  --output YYYY-MM-DD_OUTPUT_FILE.csv \
  --format csv

JSON:

bashCopy code
node --env-file=.env ${CLAUDE_PLUGIN_ROOT}/reference/scripts/run_actor.js \
  --actor "ACTOR_ID" \
  --input 'JSON_INPUT' \
  --output YYYY-MM-DD_OUTPUT_FILE.json \
  --format json

Step 5: Summarize Results and Offer Follow-ups

After completion, report:

• Number of results found
• File location and name
• Key fields available
• Suggested follow-up workflows based on results:

Error Handling

APIFY_TOKEN not found - Ask user to create .env with APIFY_TOKEN=your_token Actor not found - Check Actor ID spelling Run FAILED - Ask user to check Apify console link in error output Timeout - Reduce input size or increase --timeout