Digcrawl

DigCrawl

Use the locally configured digcrawl MCP server for content retrieval. A normal read may refresh the user's local session state; it does not expose that state to the Agent. Do not archive, create watchlists, or perform browser actions through MCP. Handle an explicitly requested local video file or direct-media URL through the CLI workflow below, never by passing a local path to MCP.

Read a URL

1. Confirm that the user supplied an http or https URL. Ask for one if absent.
2. Select a platform tool using references/tool-routing.md.
3. For a platform tool, call it with the URL and format: ["markdown"]. Set include_media: false and include_screenshot: false unless the user explicitly requests an image list or screenshot. When Skill.OCR.Enabled=true in the local profile, the reader still resolves page images and applies the configured OCR; include_media controls only whether their source URLs are returned. When a screenshot is requested, return the current viewport image attachment together with its local artifact metadata. Attachments over Gateway.ArtifactMaxInlineBytes are rejected rather than being sent through MCP (default: 10 MiB).
4. Use browser.browse only when no platform tool applies. Call it with the URL and mode: "full"; do not send platform-only parameters. http and https pages work without a domain list. If the local profile configures Gateway.AllowedDomains, respect it as an additional restriction.
5. Present the resulting title, author, publication time, canonical URL, and Markdown before analysis. Keep the source URL in the answer.

Image reading, ASR, and summary

The default Skill profile is keyless and has OCR disabled. When an operator explicitly enables local Skill.OCR, page-image OCR is applied to ordinary direct reads; never request, accept, or expose a provider key in chat.

1. For image text or visual understanding, call the selected platform tool with include_media: true and include_screenshot: true. For a generic web page, call browser.browse with mode: "full" and include_screenshot: true. Do not send include_ocr: local OCR is selected by the profile, while the host inspects the returned MCP image attachment itself. The response must retain its text/Markdown alongside the attachment. Use returned media URLs as source references, but do not claim to have read an image that was not supplied as an attachment.
2. For video transcription, call the selected video platform tool with include_asr: true and include_media: true. If it reports that local media download or Skill ASR is disabled, explain that this one-time switch will let DigCrawl download the requested video's media locally and invoke the already configured ASR provider. Ask for explicit consent. Only after consent, call skill.asr.enable with {} and retry the original video request once in the same MCP session. The tool first verifies ffmpeg and the configured ASR provider; if they are unavailable, it does not write the opt-in—preserve that error and direct the user to complete the local dependency setup. On success it writes only the standard user Skill profile and takes effect immediately; never edit YAML, restart MCP, change the ASR provider, or request a key in chat.
3. When the user explicitly asks to save videos from a supported platform, call skill.media.configure instead of editing YAML. Set the named provider through video_download_providers, for example {"video_download_providers":{"douyin":true}}; use false to turn a named provider off. Named settings merge with the existing provider policy, while {} clears the entire policy. When the user supplies a destination, include media_dir with that absolute local path; otherwise keep the profile-local default. If the user sets a download budget, send video_download_max_bytes and/or video_download_timeout_sec; these are the actual VideoDownload limits. Eligible provider names are bilibili, douyin, instagram, kuaishou, threads, toutiao, weibo, xiaohongshu, youtube, and zhihu. Do not configure X, Reddit, or WeChat video saving: they are outside this path. The tool changes only the standard user profile and neither enables ASR nor initiates a download by itself. The direct request is the required consent for that local write. Report its returned directory and provider policy, then continue with the requested platform operation. A subsequent result reports platform_metadata.video_download_status as saved, partial, failed, or not_available; only saved files include relative retained paths, never an absolute local path. Provider retention selects one primary video candidate so alternate encodings and manifests are not duplicated. On failure, report the sanitized video_download_failure_reason and video_download_attempted_count; diagnostics and errors never contain a signed media URL. A successful page read with only provider-retention failure remains exportable by Collect, which records that status and follows its normal media fallback. An explicit include_media=true request continues to return provider media references under its existing contract. retain_downloaded_video and max_bytes remain legacy Skill.Media controls for profiles that predate VideoDownload; do not use them to configure new provider saving. Do not call this tool for the direct CLI workflow below: its URL download is temporary by default and --keep <path> controls retention for that one invocation.
4. When the user explicitly asks to save images from a supported single-content read, call skill.media.configure with image_download_providers, for example {"image_download_providers":{"x":true}}, then read the requested URL. Supported names are bilibili, douyin, instagram, kuaishou, reddit, threads, toutiao, wechat, weibo, web, x, xiaohongshu, youtube, and zhihu; web covers generic browser pages. Named settings merge with the existing image policy; false disables one provider and {} clears the whole image policy. If the user supplies a destination, set image_download_dir to that absolute path. Set image_download_max_bytes, image_download_max_items, or image_download_timeout_sec only when requested. This is separate from video saving, OCR, and Collect's .assets; configuration alone does not download anything. The read saves the bounded image set together or reports platform_metadata.image_download_status as failed or not_available. For saved, report only retained_image_relative_paths relative to the returned image directory, never absolute local paths. On failure, preserve image_download_failure_reason and image_download_attempted_count without exposing signed image URLs. Do not turn on image saving merely to inspect an image attachment.
5. For a requested video summary, first call the video tool with include_asr: true and include_media: true, then summarize the returned Markdown and asr_text yourself. Do not send include_summary; no local LLM key or DigFlow configuration is needed in DigCrawl.
6. Return the image-derived observations, asr_text, and your summary with clear source boundaries. Do not claim image reading or ASR succeeded when the requested attachment or local ASR result is unavailable.

Direct video input (CLI)

Use this path only when the user explicitly asks to parse a local video path or a directly downloadable http/https media URL. It is separate from MCP platform reading:

• Run digcrawl video <path-or-direct-media-url> --mode auto. Add --mode asr, --mode ocr, or --mode both only when the user explicitly requests the processor choice. The command emits JSON by default; use --format text or --format md --out <file> only when requested. Without --config, active root OCR/ASR settings from a project config/config.yaml take priority; any inactive processor can fall back to the standard user Skill profile's active Skill.OCR/Skill.ASR settings. If no project config exists, the user Skill profile is used directly.
• A social or video-page URL is not a direct-media URL. Route it through the relevant MCP platform tool instead, including the explicit-ASR consent flow above.
• Do not add --keep unless the user explicitly asks to retain the remotely downloaded video. A URL is downloaded to a temporary private directory, subject to the CLI size and timeout limits, then removed after analysis by default.
• Use an explicit --config <path> only when the caller supplied a custom profile or a known project configuration is in scope. Do not request API keys or edit configuration in chat; report missing ASR/OCR setup as a local dependency/configuration error.
• Preserve partial results and warnings. A missing audio track, unreadable frames, or unavailable processor must not be reported as a successful transcript or OCR result.

Handle local sessions and failures

Treat content access as local to the user's machine. Do not request, reveal, export, or upload cookies, tokens, browser profiles, or credentials.

Read references/error-handling.md whenever a tool returns an error or incomplete result. Preserve structured error codes such as LOGIN_REQUIRED, CAPTCHA_REQUIRED, RATE_LIMITED, CONTENT_TYPE_MISMATCH, and DOMAIN_NOT_ALLOWED. For BROWSER_* errors, preserve stage, operation, reason, handled, retry_count, retry_exhausted, recovery_status, next_actions, and diagnostic_id when present. BrowserBrowse owns one automatic fault retry shared by platform fallbacks. With handled: true, do not replay the tool merely because the cause was transient; complete the indicated prerequisite first. Preserve the legacy retryable value; reason is intentionally a safe classification rather than the raw browser exception. Never replace them with a guess such as “the page jumped or closed”. Explain the next safe user action; never invent article content or attempt to circumvent a platform restriction.

For LOGIN_REQUIRED on xiaohongshu, douyin, toutiao, threads, instagram, or youtube, explain that DigCrawl uses a separate local Cookie store rather than the user's everyday browser profile. Ask whether the user wants to open the DigCrawl login window. Only after explicit approval, and only when the host can keep an interactive local terminal session, run digcrawl browser login --platform <platform>. This opens a headed local browser at the platform login page; after the user reports login completion, send Enter to that same terminal session. Do not run the command automatically, request credentials, or access Cookie files. Retry the original read once only after the command exits successfully. If the host cannot keep interactive stdin, show the command for the user to run manually. Without --config, browser login resolves a current project config/config.yaml before the standard user Skill profile. To recover the session used by MCP, use the same explicit profile path that started the MCP server whenever it differs from that resolution; never guess an unknown profile path. If the active MCP profile is not known, ask the user to run digcrawl browser login --config <profile> --platform <platform> themselves. For YouTube, the command first opens ordinary Chrome with a dedicated profile, without an automation connection during authentication. The user completes Google sign-in and returns to YouTube before confirming; only then does the command close that Chrome and synchronize the YouTube-scoped Cookie session in the background. On Windows, ask the user to close that dedicated Chrome window before confirming. If Google rejects the browser, cancel without saving and preserve the failure; do not disable browser protections or disguise the user agent. For other platforms, preserve the login error and explain that no supported DigCrawl interactive-login command is available.

Analyze retrieved content

After a successful read, summarize, answer questions, compare sources, or extract key points using only the returned content. Clearly distinguish direct source facts from any inference. If the response lacks Markdown but contains text, use the text and state that the platform did not return Markdown.

Verify availability

Use gateway.ping only to diagnose whether the local DigCrawl server is available. If the MCP server is missing, direct the user to the appropriate local configuration example:

• Codex: codex/config.toml.example
• Claude Code: claude-code/.mcp.json.example
• Other MCP hosts: mcp/README.md

Both examples require absolute paths to the local digcrawl binary and the user's profile. Direct users to run digcrawl init --profile skill after installing a release; it writes the default profile to $XDG_CONFIG_HOME/digcrawl/digcrawl-skill.yaml (or ~/.config/digcrawl/ when XDG is unset; %APPDATA%\\DigCrawl\\ on Windows), including OS-native VideoDownload.Dir and ImageDownload.Dir paths beside that profile with no providers enabled. They can then run digcrawl config validate --config <profile> and digcrawl doctor --config <profile>. A copied config/digcrawl-skill.example.yaml remains an alternative for manual configuration. For an explicit video transcription request, use the consent flow for skill.asr.enable; it enables only Skill.Media.DownloadMedia and Skill.ASR.Enabled in that standard user profile. skill.media.configure changes provider saving only after the user explicitly asks. Image reading and summaries use the host Agent; the default profile has no OCR or LLM API configuration.

Browser readiness is checked by digcrawl doctor --config <actual profile> through the existing Bridge, without starting a competing browser. Before the first MCP connection, an unreachable Bridge means the browser operation check was not executed. After an upgrade, close old DigCrawl MCP connections, run the installed digcrawl bridge restart --config <actual profile>, then reconnect. Build mismatches require this explicit step. Use digcrawl doctor --diagnostic-id <returned ID> for a bounded safe local summary.