Whizard Logging

WizTelemetry Logging

Overview

WizTelemetry Logging is an extension component in the KubeSphere Observability Platform for log collection, processing, and storage.

When to Use

• Installing or configuring the WizTelemetry Logging extension
• Understanding log collection architecture (container logs + disk log collection)
• Using the log query API to query logs

Components

Dependencies

• WizTelemetry Platform Service (whizard-telemetry): Required
• WizTelemetry Data Pipeline (vector): Required
• OpenSearch (opensearch): Required

Installation

Prerequisites

REQUIRED: Complete all steps in order before generating InstallPlan.

Step 1: Get Available Clusters and Confirm Target

⚠️ CRITICAL: DO NOT proceed until target clusters are determined.

Step 1.1: Get available clusters

bashCopy code
kubectl get clusters -o jsonpath='{.items[*].metadata.name}'

Step 1.2: Determine target clusters

• If user explicitly specified target clusters in the request → Use those clusters directly, proceed to Step 2
• If user did NOT specify target clusters → Ask user to confirm which clusters to deploy to, then proceed to Step 2

Ask user (if not specified):

Copy code
Available clusters: host, dev
Which clusters do you want to deploy WizTelemetry Logging to?

Step 2: Get Latest Version (if not provided by user)

MUST do this to get the latest version:

bashCopy code
kubectl get extensionversions -l kubesphere.io/extension-ref=whizard-logging -o jsonpath='{range .items[*]}{.spec.version}{"\n"}{end}' | sort -V | tail -1

This outputs the latest version (e.g., 1.4.0). Note this down - you'll use it in the InstallPlan.

Install WizTelemetry Logging

⚠️ IMPORTANT: Complete prerequisite steps BEFORE this step.

Based on your selections:

• Target clusters: User-confirmed cluster names

⚠️ CRITICAL: InstallPlan metadata.name MUST be whizard-logging. DO NOT use any other name.

⚠️ CRITICAL: config field is YAML format. You MUST:

• Use the config structure exactly as shown in the template
• DO NOT add configuration fields that are not shown in the template
• DO NOT modify the structure or hierarchy

⚠️ CRITICAL: All placeholders MUST be replaced with actual values. DO NOT leave them as placeholders.

Template

yamlCopy code
apiVersion: kubesphere.io/v1alpha1
kind: InstallPlan
metadata:
  name: whizard-logging
spec:
  extension:
    name: whizard-logging
    version: <VERSION>  # From Step 2
  enabled: true
  upgradeStrategy: Manual
  clusterScheduling:
    placement:
      clusters:
        - <TARGET_CLUSTERS>

Replace placeholders:

• <VERSION>: From Step 2 (e.g., 1.4.0)
• <TARGET_CLUSTERS>: User-confirmed cluster names

Note: OpenSearch sink configuration (endpoints, auth) is provided by the vector extension. Make sure vector is installed and configured with OpenSearch before installing logging.

Enable Disk Log Collection

To enable disk log collection, add logsidecar-injector to the config:

yamlCopy code
apiVersion: kubesphere.io/v1alpha1
kind: InstallPlan
metadata:
  name: whizard-logging
spec:
  extension:
    name: whizard-logging
    version: <VERSION>  # From Step 2
  enabled: true
  upgradeStrategy: Manual
  config: |
    logsidecar-injector:
      enabled: true
    vector-logging:
      filter:
        extraLabelSelector: "app.kubernetes.io/name!=kube-events-exporter"
      calico:
        enabled: true
      systemd:
        docker:
          enabled: true
        kubelet:
          enabled: true
      sinks:
        opensearch:
          enabled: true
          index:
            prefix: "{{ .cluster }}-logs"
            timestring: "%Y.%m.%d"
  clusterScheduling:
    placement:
      clusters:
        - <TARGET_CLUSTERS>

Configuration Parameters

Logsidecar Injector Parameters

Vector Logging Parameters

Filter Parameters

Calico Log Parameters

Systemd Log Parameters

OpenSearch Sink Parameters

ISM Policy Parameters

Log Query API

Query Logs

bashCopy code
curl -X GET "http://whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/logging.kubesphere.io/v1alpha2/logs?operation=query&log_query=error&size=10&cluster=host&sort=desc" \
  -H "X-Remote-User: admin"

Query Parameters:

Extension Operations

Check Extension Status

bashCopy code
kubectl get installplan whizard-logging
kubectl get extensionversions -l kubesphere.io/extension-ref=whizard-logging

Uninstall Extension

Uninstall from all clusters:

bashCopy code
kubectl delete installplan whizard-logging

Uninstall from specific cluster:

To remove WizTelemetry Logging from a specific cluster, update the InstallPlan by removing that cluster from clusterScheduling.placement.clusters:

yamlCopy code
apiVersion: kubesphere.io/v1alpha1
kind: InstallPlan
metadata:
  name: whizard-logging
spec:
  extension:
    name: whizard-logging
    version: <VERSION>
  enabled: true
  upgradeStrategy: Manual
  clusterScheduling:
    placement:
      clusters:
        - <REMAINING_CLUSTERS>  # Remove the cluster you want to uninstall from