Subdomain Enumeration Skill
Comprehensive subdomain discovery using certificate transparency logs (crt.sh), DNS brute force, and passive sources. The first step in any recon pipeline — you can't attack what you don't know exists. Subdomain enumeration consistently reveals staging environments, internal admin panels, API gateways, and forgotten WordPress installs that are softer targets than the production site.
When to Use
- Starting recon on any target domain.
- Production site is well-secured — find softer entry points.
- After
skill_view(name='wp-mass-recon')— enumerate subdomains for each WordPress target. - Building a complete asset inventory for a target organization.
Prerequisites
curl,httpx,dig,jq,dnsx, andsubfinder.- A DNS brute-force wordlist, supplied by the operator.
- Approval to disclose the domain to third-party passive sources such as crt.sh, crt.name, and subfinder providers.
How to Run
bashDOMAIN="${1:-example.com}" OUTPUT_ROOT="${OUTPUT_DIR:-./output}" OUTDIR="$OUTPUT_ROOT/subdomains/$DOMAIN" if [[ ! "$DOMAIN" =~ ^[A-Za-z0-9.-]+$ ]]; then echo "Invalid domain: $DOMAIN" >&2 exit 2 fi mkdir -p "$OUTDIR" # Passive: crt.sh (retry: it frequently returns 502 while its database refreshes) for attempt in 1 2 3; do curl -fsS --max-time 30 --connect-timeout 10 \ "https://crt.sh/?q=%25.$DOMAIN&output=json" -o "$OUTDIR/crtsh_raw.json" && break sleep 15 done jq -r '.[].name_value' "$OUTDIR/crtsh_raw.json" 2>/dev/null \ | sed 's/\*\.//g' \ | sort -u > "$OUTDIR/crtsh.txt" # If crt.sh stays down, crtsh.txt ends up empty: keep going with crt.name/subfinder. # crt.name (historical CT data; resolve names before probing) curl -fsS --max-time 30 --connect-timeout 10 "https://crt.name/v1/search?apex=$DOMAIN" \ | sed 's/\r$//' \ | sed 's/^\*\.//' \ | sort -u > "$OUTDIR/crtname.txt" # Passive: subfinder subfinder -d "$DOMAIN" -silent -timeout 30 > "$OUTDIR/subfinder.txt" # Merge and deduplicate cat "$OUTDIR/crtsh.txt" "$OUTDIR/crtname.txt" "$OUTDIR/subfinder.txt" \ | grep -E '^[A-Za-z0-9.-]+\.[A-Za-z]{2,}$' \ | sort -u > "$OUTDIR/all_subs.txt" # Probe live hosts httpx -silent -l "$OUTDIR/all_subs.txt" -threads 50 -rate-limit 2 \ -status-code -tech-detect -title -o "$OUTDIR/alive.txt"
Quick Reference
| Source | Method | Coverage | Speed |
|---|---|---|---|
| crt.sh | Certificate transparency | Excellent (most certs) | Fast (1-5s) |
| subfinder | Passive APIs (VirusTotal, Shodan, DNSdumpster, etc.) | Very good | Fast (30-60s) |
| dnsx | Bulk DNS A/AAAA/CNAME resolution (100x faster than dig) | Good (uncovers non-HTTP) | Fast (10-30s) |
| httpx probe | Live HTTP/HTTPS check | Best for web attack surface | Fast (30-60s) |
| Google dork | site:example.com | Supplemental | Manual |
Procedure
Step 1 — Passive Enumeration (crt.sh + subfinder)
bashDOMAIN="$1" OUTPUT_ROOT="${OUTPUT_DIR:-./output}" OUTDIR="$OUTPUT_ROOT/subdomains/$DOMAIN" if [[ ! "$DOMAIN" =~ ^[A-Za-z0-9.-]+$ ]]; then echo "Invalid domain: $DOMAIN" >&2 exit 2 fi mkdir -p "$OUTDIR" echo "[*] Passive enumeration for $DOMAIN..." # crt.sh — certificate transparency logs (retry: it frequently returns 502 while its database refreshes) echo "[*] crt.sh query..." for attempt in 1 2 3; do curl -fsS --max-time 30 --connect-timeout 10 \ "https://crt.sh/?q=%25.$DOMAIN&output=json" -o "$OUTDIR/crtsh_raw.json" 2>/dev/null && break sleep 15 done jq -r '.[].name_value' "$OUTDIR/crtsh_raw.json" 2>/dev/null | \ sed 's/\*\.//g' | \ sed 's/^www\.//' | \ sort -u > "$OUTDIR/crtsh.txt" crt_count=$(wc -l < "$OUTDIR/crtsh.txt") echo " crt.sh: $crt_count entries" if [[ ! -s "$OUTDIR/crtsh.txt" ]]; then echo " [!] crt.sh unavailable or empty after retries; results are incomplete" >&2 fi # crt.name — historical certificate-transparency data echo "[*] crt.name query..." curl -fsS --max-time 30 --connect-timeout 10 "https://crt.name/v1/search?apex=$DOMAIN" 2>/dev/null | \ sed 's/\r$//' | \ sed 's/^\*\.//' | \ sort -u > "$OUTDIR/crtname.txt" crtname_count=$(wc -l < "$OUTDIR/crtname.txt") echo " crt.name: $crtname_count entries (historical; resolve before probing)" # Also query with %25. (wildcard) for attempt in 1 2 3; do curl -fsS --max-time 30 --connect-timeout 10 \ "https://crt.sh/?q=%25.%25.$DOMAIN&output=json" -o "$OUTDIR/crtsh_wildcard_raw.json" 2>/dev/null && break sleep 15 done jq -r '.[].name_value' "$OUTDIR/crtsh_wildcard_raw.json" 2>/dev/null | \ sed 's/\*\.//g' | \ sort -u > "$OUTDIR/crtsh_wildcard.txt" # subfinder — passive API aggregation echo "[*] subfinder..." subfinder -d "$DOMAIN" -silent -timeout 30 2>/dev/null | sort -u > "$OUTDIR/subfinder.txt" subf_count=$(wc -l < "$OUTDIR/subfinder.txt") echo " subfinder: $subf_count entries" # Merge all passive sources cat "$OUTDIR"/crtsh.txt "$OUTDIR"/crtsh_wildcard.txt "$OUTDIR"/crtname.txt "$OUTDIR"/subfinder.txt 2>/dev/null | \ sed 's/^www\.//' | \ grep -E '^[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$' | \ sort -u > "$OUTDIR/all_passive.txt" total=$(wc -l < "$OUTDIR/all_passive.txt") echo "" echo "[+] Total unique subdomains (passive): $total"
Step 2 — DNS Resolution
bashDOMAIN="$1" OUTPUT_ROOT="${OUTPUT_DIR:-./output}" OUTDIR="$OUTPUT_ROOT/subdomains/$DOMAIN" echo "[*] Resolving subdomains..." # Batch resolve with dnsx (100x faster than dig loop) dnsx -silent -l "$OUTDIR/all_passive.txt" -a -resp-only -o "$OUTDIR/resolved_raw.txt" 2>/dev/null # Format output: subdomain => IP while read -r line; do sub=$(echo "$line" | cut -d' ' -f1) ip=$(echo "$line" | cut -d' ' -f2) echo "$sub => $ip" done < "$OUTDIR/resolved_raw.txt" > "$OUTDIR/resolved.txt" resolved=$(wc -l < "$OUTDIR/resolved.txt") echo "[+] $resolved subdomains resolved to IPs" # Count unique IPs unique_ips=$(awk '{print $3}' "$OUTDIR/resolved.txt" | sort -u | wc -l) echo "[+] $unique_ips unique IPs" # Identify shared hosting (many subdomains → same IP) echo "" echo "[*] Shared hosting clusters:" awk '{print $3}' "$OUTDIR/resolved.txt" | sort | uniq -c | sort -rn | head -10 | while read -r count ip; do [[ "$count" -gt 1 ]] && echo " $ip: $count subdomains" done
Step 3 — Live Host Discovery
bashDOMAIN="$1" OUTPUT_ROOT="${OUTPUT_DIR:-./output}" OUTDIR="$OUTPUT_ROOT/subdomains/$DOMAIN" echo "[*] Probing live hosts..." # httpx with tech detection httpx -silent -l "$OUTDIR/all_passive.txt" -threads 50 \ -status-code -tech-detect -title -location \ -o "$OUTDIR/alive.txt" 2>/dev/null alive=$(wc -l < "$OUTDIR/alive.txt") echo "[+] $alive live hosts" # Categorize by status code echo "" echo "[*] By HTTP status:" echo " 200: $(grep -c '\[200\]' "$OUTDIR/alive.txt")" echo " 301/302: $(grep -cE '\[301\]|\[302\]' "$OUTDIR/alive.txt")" echo " 403: $(grep -c '\[403\]' "$OUTDIR/alive.txt")" echo " 404: $(grep -c '\[404\]' "$OUTDIR/alive.txt")" # Categorize by technology echo "" echo "[*] By technology:" grep -Eo '\[[a-z-]+\]' "$OUTDIR/alive.txt" | tr -d '[]' | sort | uniq -c | sort -rn | head -15 # WordPress subdomains echo "" echo "[*] WordPress subdomains:" grep -i 'wordpress' "$OUTDIR/alive.txt" | awk '{print $1}' | head -10 # Non-HTTP services (from DNS resolution) echo "" echo "[*] Interesting non-standard ports from DNS (MX, NS, etc.):" dig +short "$DOMAIN" MX 2>/dev/null | head -5 dig +short "$DOMAIN" NS 2>/dev/null | head -5 dig +short "$DOMAIN" TXT 2>/dev/null | grep -i 'spf\|v=spf' | head -3
Step 4 — Subdomain Categorization
bashDOMAIN="$1" OUTPUT_ROOT="${OUTPUT_DIR:-./output}" OUTDIR="$OUTPUT_ROOT/subdomains/$DOMAIN" echo "[*] Categorizing subdomains..." # Staging/Dev echo "" echo "=== STAGING / DEV ===" grep -iE 'staging|stage|dev\.|development|test|uat|beta|sandbox|preview|qa' "$OUTDIR/all_passive.txt" # Admin/Internal echo "" echo "=== ADMIN / INTERNAL ===" grep -iE 'admin|portal|internal|dashboard|manage|cp\.|control|panel|cpanel|webmail|mail\.' "$OUTDIR/all_passive.txt" # API echo "" echo "=== API ===" grep -iE 'api|rest|graphql|ws\.|websocket' "$OUTDIR/all_passive.txt" # Infrastructure echo "" echo "=== CDN / STATIC ===" grep -iE 'cdn|static|assets|media|img|images|files|download|origin|proxy' "$OUTDIR/all_passive.txt" # Email echo "" echo "=== EMAIL ===" grep -iE 'mail\.|smtp|imap|pop|email|webmail|autodiscover' "$OUTDIR/all_passive.txt" # Cloud echo "" echo "=== CLOUD ===" grep -iE 'aws|azure|gcp|cloud|s3|bucket|firebase' "$OUTDIR/all_passive.txt" # Legacy echo "" echo "=== LEGACY / OLD ===" grep -iE 'old|old\.|v1|v2|legacy|archive|backup|bak' "$OUTDIR/all_passive.txt"
Step 5 — Subdomain Takeover Check
bashDOMAIN="$1" OUTPUT_ROOT="${OUTPUT_DIR:-./output}" OUTDIR="$OUTPUT_ROOT/subdomains/$DOMAIN" echo "[*] Checking for subdomain takeover opportunities..." # Check for dangling CNAMEs (subdomains pointing to non-existent services) # For bulk CNAME check: dnsx -silent -l all_passive.txt -cname -resp-only while read -r sub; do cname=$(dig +short "$sub" CNAME 2>/dev/null) if [[ -n "$cname" ]]; then # Check if the CNAME target resolves cname_ip=$(dig +short "$cname" A 2>/dev/null) if [[ -z "$cname_ip" ]]; then echo "[TAKEOVER?] $sub => $cname (NOT RESOLVING)" # Identify provider from CNAME if echo "$cname" | grep -qi 'amazonaws.com'; then echo " Provider: AWS (S3/CloudFront) — check if bucket/domain is claimable" elif echo "$cname" | grep -qi 'azure'; then echo " Provider: Azure — check if resource is claimable" elif echo "$cname" | grep -qi 'github.io'; then echo " Provider: GitHub Pages — check if repo name is available" elif echo "$cname" | grep -qi 'herokuapp.com'; then echo " Provider: Heroku — check if app name is available" elif echo "$cname" | grep -qi 'vercel-dns.com'; then echo " Provider: Vercel — check if project is claimable" elif echo "$cname" | grep -qi 'zendesk.com'; then echo " Provider: Zendesk — check if help desk is claimable" fi fi fi sleep 0.5 done < "$OUTDIR/all_passive.txt"
Pitfalls
- crt.sh rate limiting. crt.sh may return empty JSON if rate-limited. Use delays or query the PostgreSQL dump directly.
- subfinder requires API keys. Some sources (VirusTotal, Shodan) require API keys in
~/.config/subfinder/provider-config.yaml. Without them, results are limited. - Wildcard DNS. If
*.example.comresolves to the same IP, all subdomains will appear "live" in httpx. Check for wildcard by resolving a random string:dig RANDOMSTRING.example.com. - Cloudflare proxying. Subdomains behind Cloudflare will show Cloudflare IPs, not origin IPs. Use SecurityTrails or DNSDumpster for historical DNS records.
Verification
- Every subdomain MUST be probed with httpx to confirm it serves HTTP/HTTPS.
- Resolved IPs MUST be cross-referenced with known CDN IPs (Cloudflare, CloudFront, Fastly) to avoid mistaking CDN IPs for origin.
- Subdomain takeover candidates MUST have their CNAME target manually verified as unclaimed.
- All live subdomains should be documented with: URL, HTTP status, technology stack, and page title.
Phase 7 — Permutation & Prediction
Generate smart mutations from already-discovered subdomains to find hidden services:
bash# gotator — generates permutations gotator -sub all_subs.txt -perm permutations.txt -depth 1 -numbers 3 -md | sort -u > subs_permuted.txt # Resolve permutations puredns resolve subs_permuted.txt -r resolvers.txt -o subs_permuted_alive.txt # Common permutation patterns for the wordlist # %s-dev, dev-%s, %s-staging, staging-%s, %s-prod, %s-internal # %s-admin, admin-%s, %s-api, api-%s, %s-test, test-%s # %s-stg, stg-%s, %s-uat, uat-%s, %s-www, www-%s
Phase 8 — TLD Expansion
A company that owns target.com often neglects target.io, target.net, target.xyz:
bash# tldbrute — discovers registered TLD variants tldbrute -d target.com # Manual IANA TLD list approach wget -q https://data.iana.org/TLD/tlds-alpha-by-domain.txt ROOT=$(echo "target.com" | cut -d. -f1) cat tlds-alpha-by-domain.txt | tr '[:upper:]' '[:lower:]' \ | while read tld; do echo "$ROOT.$tld"; sleep 0.2; done \ | httpx -silent -mc 200 > tlds_alive.txt # Expand existing subdomains across TLDs cat all_subs.txt | while read sub; do cat tlds-alpha-by-domain.txt | tr '[:upper:]' '[:lower:]' \ | sed "s/^/$sub./" done | dnsx -silent > subs_tld_expanded.txt
Phase 9 — Live Certificate Monitoring
Catch new subdomains the moment they're issued:
bash# gungnir — real-time certificate transparency monitoring gungnir -d target.com # certwatcher — alternative CT log monitor certwatcher -d target.com --webhook https://hooks.slack.com/xxx
