Cloud Monitoring Promql Query

Cloud Monitoring PromQL Generator

Use this skill to generate a valid PromQL query from any Cloud Monitoring metric type. This guide applies to all Cloud Monitoring metric types by mapping Cloud Monitoring metric and resource descriptors to PromQL structures.

Workflow

Resolve Project ID (CRITICAL & BLOCKING)

Before performing any other actions (such as searching code, reading references, or running validation), you MUST verify whether the Google Cloud Project ID is available:

1. Check Prompt/Payload: Look for the Project ID in the user's prompt or input.
2. Check Environment: If the Project ID is not present in the prompt, you MUST run gcloud config get-value project to attempt to resolve it from the environment.
3. Ask for Clarification (BLOCKING): If the Project ID is not in the prompt AND the gcloud command fails, returns an empty string, or is unavailable, you MUST immediately stop. Do NOT generate a PromQL query, do not run the validation script, and do not use placeholders (like YOUR_PROJECT_ID). You must refuse to proceed and ask the user to provide the Project ID.

Inspect Metric and Resource Descriptors

1. Use Provided Descriptors First: If the user's prompt already includes metric descriptor details (such as metric.type, metricKind, valueType, or monitoredResourceTypes) or specific resource filter values, use those values directly instead of calling the Cloud Monitoring API.
2. Discover Missing Descriptors: If exact metric descriptors (metric.type, metricKind, valueType) are missing or underspecified, resolve the target metric type's descriptor using one of these paths:
   • Vague Query: If the prompt is vague (for example, "VM CPU usage"), use the cloud-monitoring-metric-selection skill first to identify the specific metric type.
   • Known Metric Type: If you already have the specific metric type name (for example, compute.googleapis.com/instance/cpu/utilization) but need its descriptor, call the google-cloud-monitoring:list_metric_descriptors MCP tool. If the tool is missing, refer to the cloud-monitoring-metric-selection skill to configure the Cloud Monitoring MCP server.
   • Fallback: If the MCP tool cannot be configured, fall back to making a direct Cloud Monitoring API call.
3. Identify Key Fields: From the retrieved descriptor, identify four key schema attributes:
   • type: The Cloud Monitoring metric type string.
   • metricKind: GAUGE, DELTA, or CUMULATIVE.
   • valueType: INT64, DOUBLE, DISTRIBUTION, or BOOL.
   • monitoredResourceTypes: Compatible resource.type strings required for resource scoping and grouping.

Resolve Resource Filters & Discovery Protocol

To filter data by a specific resource instance, apply these resource rules and discovery protocols:

1. Monitored Resource Filter: Always include the monitored_resource="<type>" filter in your query to prevent collisions across services that share metric names.

   • Example: monitored_resource="gae_app"

2. Preserve User Literals (CRITICAL): ALWAYS use the literal resource names, namespaces, and IDs provided in the user's prompt. Do NOT override or replace these values with active resource names found during Cloud Monitoring discovery unless the user explicitly asked you to find active resources. Telemetry discovery must only be used to identify metric type names and label keys, not to override user input.

3. Resource Identifier Mapping:

   • Direct & Specific Keys: Use the most specific resource identifier available. Example: version_id, cluster_name.
   • Name-to-ID Resolution: If the user filters by a resource name (such as "instance-1"), but the resource schema uses numeric IDs (like instance_id), use PromQL string name labels instead of numeric ID labels. Example: instance_name, metadata_system_name.
   • Composite Identifiers: For resources with hierarchical identifiers (such as Cloud SQL databases), format the filter as a single composite key. Do NOT split them into separate project_id and sub-resource labels. Example: database_id="{project_id}:{instance_name}".

4. Resource Label Discovery: The google-cloud-monitoring:list_metric_descriptors tool only returns metric-specific labels. If the label schema for a monitored resource is unknown, fetch the resource descriptor directly from the Cloud Monitoring v3 REST API (projects.monitoredResourceDescriptors.get):

   bashCopy code
   TOKEN=$(gcloud auth application-default print-access-token 2>/dev/null || gcloud auth print-access-token)
   curl -s -H "Authorization: Bearer ${TOKEN}" \
   "https://monitoring.googleapis.com/v3/projects/{project_id}/monitoredResourceDescriptors/{monitored_resource_type}"
   

   An HTTP 200 OK response returns the MonitoredResourceDescriptor object containing the labels array with the exact resource label keys for that resource.

Choose Aggregation Structure & Defaults

The query structure and aggregation functions (such as rate, histogram_quantile, sum, or avg) depend on the metric type and how it is visualized.

1. Consult the Reference: Consult the Cloud Monitoring to PromQL Basic Aggregations Reference as the single source of truth to map Cloud Monitoring properties (Metric Kind, Value Type, Aligner, Reducer) to their PromQL structures.
2. SRE Aggregation & Visualization Rules:
   • Do NOT sum or average ratio/percentage utilization metrics (like CPU % or Memory limit utilization) across resource instances. Instead, keep them unaggregated (raw metric), group by instance, or wrap in topk(30, avg_over_time(...)).
   • State Label Filtering (CRITICAL): Only the metrics agent.googleapis.com/memory/percent_used and agent.googleapis.com/disk/percent_used require {state!="free"}. Do NOT filter by {state="used"}.

Format & Validate Query

Before presenting any PromQL queries, validate them using the linter:

Python Dependencies

Before executing the validation script (scripts/validate_promql.py), install the required Python dependencies:

bashCopy code
python3 -c "import promql_parser" || pip install promql-parser

Validation Procedure

1. Format Constraints:
   • Metric Name Normalization: Convert Cloud Monitoring metric types to PromQL metric names using this recipe:
     1. Split Domain and Path: Split the Cloud Monitoring metric type by the first slash (/) to separate the domain from the path.
        • Example: storage.googleapis.com/network/received_bytes_count -> domain storage.googleapis.com, path network/received_bytes_count
     2. Normalize Domain: Replace all periods (.) in the domain with underscores (_).
        • Example: storage.googleapis.com -> storage_googleapis_com
     3. Normalize Path: Replace all periods (.) and slashes (/) in the path with underscores (_).
        • Example: network/received_bytes_count -> network_received_bytes_count
     4. Join with Colon: Join the normalized domain and normalized path with a colon (:).
        • Example: storage_googleapis_com:network_received_bytes_count
     5. Native Prometheus Metrics: If the metric type has no slash, keep it as-is.
        • Example: up -> up, http_requests_total -> http_requests_total
     6. Distribution Suffix: If the metric's valueType is DISTRIBUTION, append _bucket to the end of the normalized name.
        • Example: cloudfunctions.googleapis.com/function/execution_times -> cloudfunctions_googleapis_com:function_execution_times_bucket
   • Ensure the final query is a single line with no comments (no # or //). Cloud Monitoring query translation collapses whitespace and can cause code trailing a comment to be ignored or throw parsing errors.
   • Grouping Clause Syntax: Ensure grouping clauses (such as by (label)) only follow aggregation operators (such as sum, avg, min, max, or count). Never place a grouping clause directly after a metric selector.
     • Incorrect: metric{...} by (label)
     • Correct: sum(rate(metric{...}[5m])) by (label)
   • Fenced Output Code Block: ALWAYS wrap the final verified PromQL query in a fenced promql code block in your final response.
2. Linter Verification:
   • Validate all generated queries in a single batch: python3 <path_to_skill>/scripts/validate_promql.py --query '<q1>' '<q2>'
   • If validation fails, read PromQL Error Recovery Guide to diagnose and fix common type mismatches and syntax errors before repeating the loop.

References

• Cloud Monitoring PromQL Basic Aggregations Reference
• Cloud Monitoring PromQL Error Recovery Guide
• Cloud Monitoring PromQL Documentation
• Cloud Monitoring Monitored Resource Types Reference