RenderDoc GPU Debugging Skill
Overview
This skill enables GPU frame capture, inspection, and debugging using rdc-cli, a 66-command CLI wrapping RenderDoc's Python API. It works with Vulkan, D3D11, D3D12, and OpenGL applications.
Prerequisites
Before any GPU debugging, verify the environment:
bashrdc doctor
All checks should pass. If rdc doctor fails, check:
RENDERDOC_PYTHON_PATHis set to the directory containingrenderdoc.pydrenderdoc.dllis in the same directory asrenderdoc.pydrenderdoccmd.exeis on PATH or in the module directory
1. Session Lifecycle
Every inspection session follows open-work-close:
bashrdc open path/to/capture.rdc # Start daemon, load capture # ... inspection commands ... rdc close # Release resources, stop daemon
Check session state with rdc status. Only one capture can be open per session (use --session name for parallel sessions).
IMPORTANT: Always close sessions when done. Leaked daemon processes consume GPU memory.
2. Capture Workflow
Capture from an executable
bashrdc capture -o output.rdc -- /path/to/app [app-args...]
Key options:
-o, --output: Output .rdc file path--frame N: Capture frame N (default: first presented frame)--timeout S: Capture timeout in seconds (default: 60)--api-validation: Enable API validation layer--ref-all-resources: Reference all resources (larger capture, more complete)--wait-for-exit: Wait for the app to exit after capture
CWD matters
Many applications resolve assets relative to CWD. Always set CWD appropriately:
bashcd /path/to/app/root && rdc capture -o /output/path/capture.rdc -- ./app
For lux-playground specifically:
bashcd D:/shaderlang && python D:/renderdoc/capture_frame.py
Vulkan layer requirement
For Vulkan apps, the RenderDoc Vulkan layer must be registered:
- Registry:
HKCU\SOFTWARE\Khronos\Vulkan\ImplicitLayersmust containD:\renderdoc\module\renderdoc.json(DWORD 0) - Environment:
ENABLE_VULKAN_RENDERDOC_CAPTURE=1must be set
If headless capture fails
Headless Vulkan apps (no swapchain / no vkQueuePresentKHR) cannot be captured by RenderDoc since frame boundaries are defined by present calls. Options:
- Use
--interactivemode if the app supports it (opens a window with a swapchain) - Use the Python API via
capture_frame.pyfor reliable programmatic capture - Use
--triggermode (inject without auto-capture, thenrdc capture-trigger) - Fall back to
renderdoccmd capturedirectly
3. Frame Exploration
After opening a capture, start with high-level overview:
bashrdc info --json # API, GPU, driver, resolution, frame number rdc stats --json # Per-pass breakdown, top draws, largest resources rdc passes # List render passes (debug markers / implicit passes) rdc draws --limit 20 # First 20 draw calls rdc count draws # Total draw call count
Navigate by pass
bashrdc draws --pass "Shadow Pass" --json # Draws in a specific pass rdc draws --pass "GBuffer" --limit 10 # First 10 GBuffer draws
Navigate by event
bashrdc events --limit 50 # All API events (not just draws) rdc events --type DrawIndexed --json # Filter by event type rdc event 42 --json # Detail for one event
VFS browsing
The virtual filesystem provides a structured view of the capture:
bashrdc ls / # Root directories rdc ls /textures -l # List textures with metadata rdc tree /pipelines --depth 2 # Pipeline tree rdc cat /events/42 # Event detail
4. Pipeline State Inspection
Get full pipeline state at any draw call:
bashrdc pipeline EID --json # Full pipeline state rdc pipeline EID vs --json # Vertex shader section only rdc pipeline EID ps --json # Pixel/fragment shader section only rdc pipeline EID om --json # Output merger (blend, depth, stencil) rdc pipeline EID rs --json # Rasterizer state (culling, viewport) rdc pipeline EID ia --json # Input assembler (vertex buffers, topology) rdc pipeline EID ds --json # Depth-stencil state
Bound resources
bashrdc bindings EID --json # All bindings for all stages rdc bindings EID --set 0 --json # Descriptor set 0 only
5. Shader Inspection
View shader metadata and source
bashrdc shader EID vs --json # Vertex shader metadata rdc shader EID ps --json # Fragment/pixel shader metadata rdc shader EID ps --source # Debug source code (if available) rdc shader EID ps --reflect --json # Reflection: inputs, outputs, cbuffers rdc shader EID ps --constants --json # Constant buffer values at this draw
Disassembly targets
bashrdc shader EID ps --targets # List available disassembly formats rdc shader EID ps --target spirv # SPIR-V disassembly rdc shader EID ps --target glsl # GLSL (cross-compiled, if available)
Search shaders
bashrdc search "shadow" --json # Search all shader disassembly for "shadow" rdc search "main" --stage ps # Search only pixel shaders rdc shaders --stage ps --json # List all unique pixel shaders rdc shader-map --json # EID-to-shader mapping
6. Visual Inspection: Export-View-Analyze
This is the core pattern for visual debugging. Always export to PNG, then use Claude Code's Read tool to view the image.
Export render targets
bashrdc rt EID -o D:/renderdoc/captures/analysis/render_target.png rdc rt EID --target 1 -o D:/renderdoc/captures/analysis/rt_target1.png # MRT target 1
Export textures
bashrdc texture RESID -o D:/renderdoc/captures/analysis/texture.png rdc texture RESID --mip 2 -o D:/renderdoc/captures/analysis/mip2.png
Export thumbnail
bashrdc thumbnail -o D:/renderdoc/captures/analysis/thumb.png
View exported images
After exporting a PNG, use the Read tool to view it (Claude Code is multimodal):
Read tool: D:/renderdoc/captures/analysis/render_target.png
Do NOT use cat or bash to view images. The Read tool renders them visually.
Analyze
Correlate what you see in the image with pipeline state data:
- Export the render target at the suspicious draw
- View it with the Read tool
- Check pipeline state (
rdc pipeline EID --json) - Check shader constants (
rdc shader EID ps --constants --json) - Form hypothesis and verify
7. Pixel Debugging
Pixel history
Find all draws that wrote to a pixel:
bashrdc pixel X Y --json # Full pixel history at current event rdc pixel X Y EID --json # Pixel history up to EID
Pick pixel color
Read the current color at a pixel:
bashrdc pick-pixel X Y --json # Color at (X,Y) from current render target rdc pick-pixel X Y EID --json # Color at specific event
Debug pixel shader
Step through shader execution for a specific pixel:
bashrdc debug pixel EID X Y --json # Shader inputs/outputs summary rdc debug pixel EID X Y --trace # Full execution trace (every line) rdc debug pixel EID X Y --dump-at 42 # Variable snapshot at line 42
Debug vertex shader
bashrdc debug vertex EID VTXID --json # Vertex shader debug for vertex ID rdc debug vertex EID VTXID --trace # Full trace
Debug compute shader
bashrdc debug thread EID GX GY GZ TX TY TZ --json # Compute thread debug
8. Shader Edit-Replay
Modify shaders without recompiling the application:
bash# 1. Check available encodings rdc shader-encodings --json # 2. Extract shader source rdc shader EID ps --source -o D:/renderdoc/captures/analysis/shader.frag # 3. Edit the shader (use Edit tool) # 4. Build the modified shader rdc shader-build D:/renderdoc/captures/analysis/shader.frag --encoding GLSL --stage ps --json # 5. Replace (use the shader_id from build output) rdc shader-replace EID ps --with SHADER_ID --json # 6. Export to verify the change rdc rt EID -o D:/renderdoc/captures/analysis/after_edit.png # 7. Restore when done rdc shader-restore EID ps # or: rdc shader-restore-all
9. Frame Comparison
Compare two captures:
bash# Quick summary of differences rdc diff capture_a.rdc capture_b.rdc --shortstat # Detailed comparisons rdc diff capture_a.rdc capture_b.rdc --draws --json # Draw call differences rdc diff capture_a.rdc capture_b.rdc --resources --json # Resource differences rdc diff capture_a.rdc capture_b.rdc --passes --json # Pass structure differences rdc diff capture_a.rdc capture_b.rdc --framebuffer --json # Final framebuffer diff # Visual diff rdc diff capture_a.rdc capture_b.rdc --framebuffer --diff-output D:/renderdoc/captures/analysis/diff.png
10. Debugging Recipes
Recipe: Object is invisible
bash# 1. Find the draw call that should render the object rdc draws --json | jq '.[] | select(.name | contains("ObjectName"))' # or search by pass: rdc draws --pass "Main Pass" --json # 2. Check if it's being culled rdc pipeline EID rs --json # Look at CullMode, FrontFace # 3. Check depth state rdc pipeline EID ds --json # DepthEnable, DepthFunc, DepthWriteMask # 4. Check blend state (maybe alpha is 0) rdc pipeline EID om --json # BlendEnable, SrcBlend, DestBlend # 5. Check vertex transform rdc debug vertex EID 0 --json # Is the position off-screen or behind camera? # 6. Check if the draw is even issuing primitives rdc draw EID --json # VertexCount, InstanceCount, IndexCount
Recipe: Colors are wrong
bash# 1. Export the render target to see what's there rdc rt EID -o D:/renderdoc/captures/analysis/wrong_color.png # 2. Pick the problematic pixel rdc pick-pixel X Y EID --json # 3. Check texture bindings — is the right texture bound? rdc bindings EID --json # 4. Export the bound texture rdc texture RESID -o D:/renderdoc/captures/analysis/bound_texture.png # 5. Check shader constants — wrong material colors? rdc shader EID ps --constants --json # 6. Check blend state — additive when it should be alpha? rdc pipeline EID om --json # 7. Debug the pixel shader to trace the calculation rdc debug pixel EID X Y --trace
Recipe: Shadows are broken
bash# 1. Find the shadow pass rdc passes --json rdc draws --pass "Shadow*" --json # 2. Export the shadow map SHADOW_EID=$(rdc draws --pass "Shadow*" -q | tail -1) rdc rt $SHADOW_EID -o D:/renderdoc/captures/analysis/shadow_map.png # 3. Check shadow map resolution (is it too small?) rdc bindings $SHADOW_EID --json # Look at render target dimensions # 4. Check depth bias rdc pipeline $SHADOW_EID rs --json # DepthBias, SlopeScaledDepthBias # 5. Find the lighting pass that reads the shadow map rdc draws --pass "Raster*" --json LIGHT_EID=$(rdc draws --pass "Raster*" -q | head -1) # 6. Check how the shadow map is sampled rdc shader $LIGHT_EID ps --source # Look for shadow sampling code rdc shader $LIGHT_EID ps --constants --json # Light matrices, bias values # 7. Debug a shadowed pixel rdc debug pixel $LIGHT_EID X Y --trace
Recipe: Performance is bad
bash# 1. Get frame overview rdc stats --json # 2. Count draws per pass rdc passes --json # Look for passes with excessive draw counts # 3. Look for redundant state changes rdc events --limit 500 --json | jq 'group_by(.type) | map({type: .[0].type, count: length}) | sort_by(-.count)' # 4. Check for large resources rdc resources --sort size --json | jq '.[-10:]' # Top 10 largest resources # 5. Look for overdraw (if GPU counters available) rdc counters --list rdc counters --name "overdraw" --json # 6. Export with wireframe overlay to visualize overdraw rdc rt EID --overlay wireframe -o D:/renderdoc/captures/analysis/wireframe.png
Recipe: What changed between two frames
bash# Quick diff rdc diff before.rdc after.rdc --shortstat # Detailed diffs rdc diff before.rdc after.rdc --draws --json rdc diff before.rdc after.rdc --framebuffer --diff-output D:/renderdoc/captures/analysis/frame_diff.png # Compare pipeline state at specific draw rdc diff before.rdc after.rdc --pipeline EID --json
Recipe: Debug this pixel
bash# 1. Get pixel history — which draws touched this pixel? rdc pixel X Y --json # 2. Pick the draw that produced the final color # (usually the last non-failing entry in pixel history) # 3. Get the pixel's current value rdc pick-pixel X Y EID --json # 4. Debug the shader at that pixel rdc debug pixel EID X Y --json # 5. Get full execution trace if needed rdc debug pixel EID X Y --trace # 6. Check variable values at a specific shader line rdc debug pixel EID X Y --dump-at LINE_NUMBER
11. Output Size Management
GPU captures can produce enormous output. Follow these rules:
- Always use
--limitfor exploration:rdc draws --limit 20,rdc events --limit 50 - Filter by pass:
rdc draws --pass "Shadow Pass"instead of all draws - Use
-qfor ID lists:rdc draws -qreturns only EIDs, not full details - Use
--jsonselectively: JSON is verbose; use TSV default for scanning - Pipeline sections:
rdc pipeline EID vsinstead of fullrdc pipeline EID - Limit search results:
rdc search "pattern" --limit 10
Size estimates
rdc info: ~20 linesrdc draws --limit 20: ~25 linesrdc pipeline EID --json: ~200-500 lines (full), ~50 lines (per section)rdc shader EID ps --source: ~50-500 lines depending on shaderrdc debug pixel EID X Y --trace: ~100-1000 lines depending on shader complexity
12. Error Handling
rdc not found
bashpip install rdc-cli
rdc doctor fails
Check RENDERDOC_PYTHON_PATH points to directory containing renderdoc.pyd and renderdoc.dll.
Capture fails with "no swapchain"
The app may not present frames in the expected way. Try:
bashrdc capture --trigger -- /path/to/app # Inject-only mode # Then manually trigger: rdc capture-trigger
Daemon not responding
bashrdc status # Check if daemon is alive rdc close # Force close rdc open capture.rdc # Re-open
GPU feature not supported
Some features (GPU counters, pixel history) depend on the GPU and driver. Check:
bashrdc gpus --json # GPU capabilities rdc counters --list # Available counters (empty if not supported)
Command Reference
For the complete list of all 66 commands with arguments, options, types, and defaults, see references/commands-quick-ref.md.
For extended debugging recipes with expected output shapes, see references/debugging-recipes.md.
