Kubernetes

On Kubernetes, the shortest route into Fixter is the Fixter collector: a standard OpenTelemetry Collector distribution, packaged as a Helm chart. Installing it gets your pod logs, Kubernetes events, and cluster and host metrics into Fixter with no application changes at all. Your own services’ traces and logs are added on top, by pointing them at the collector instead of at Fixter directly.

This page covers the case where you want Fixter to run a collector for you. If you already run an OpenTelemetry Collector, keep it and add a Fixter exporter instead: see Send data with OpenTelemetry.

Before you start

You need an API key. Get one from Settings > API keys in the app, or ask an agent connected over MCP for get_ingestion_credentials, which returns a one-time link you open in a browser to reveal the key.

Kubernetes 1.23 or later, and kubectl and helm pointed at the cluster you want to monitor.

Install into a namespace that is not under a restricted Pod Security Admission policy. The log-collecting agent mounts host paths and runs as root, both of which a restricted namespace rejects. A plain namespace created as below is fine.

Install

Create the namespace and put the key in a Secret first, so it never reaches your shell history or the pod’s process list:

kubectl create namespace fixter
kubectl create secret generic fixter-credentials \
  --namespace fixter \
  --from-literal=api-key=<your-key>

Then install the chart pointing at that Secret:

helm install fixter-collector \
  oci://ghcr.io/fixter-dev/charts/fixter-collector \
  --namespace fixter \
  --set fixter.existingSecret=fixter-credentials \
  --set fixter.existingSecretKey=api-key \
  --set fixter.clusterName=<your-cluster>

The endpoint already defaults to https://ingest.fixter.dev, so there is nothing else you have to set. Pin a version with --version <x.y.z> for reproducible deploys; without it you get the latest published chart.

The chart also accepts --set fixter.apiKey=<your-key> directly, which is mutually exclusive with existingSecret. It is the faster way to try this on a scratch cluster and the wrong way to run it anywhere real.

If a GitOps or Terraform system already manages your Helm releases, the Claude Code plugin handles that case rather than you running the command by hand.

Check the key immediately

A wrong API key fails silently. The collector starts, reports healthy, and ships nothing. Run the bundled test straight after installing:

helm test fixter-collector -n fixter

It posts an empty metrics payload and checks the response, which proves the key is valid and Fixter is reachable. It does not prove every export succeeds, so also confirm data is arriving on the Logs page within a few minutes.

What gets installed

Two workloads:

Component	Kind	Collects
agent	DaemonSet, one per node	kubelet and host metrics, pod logs, and an OTLP relay on ports 4317 and 4318
cluster	Deployment, one replica	cluster-state metrics, Kubernetes events, Prometheus targets

The cluster deployment runs exactly one replica and that is not configurable. Its receivers each produce a full copy of their output, so a second replica would double count everything.

Kubernetes events arrive as logs, one per event as it happens. They carry the reason a pod changed state, OOMKilled, CrashLoopBackOff, FailedScheduling, Unhealthy, which cluster metrics cannot express on their own. Volume is low on a healthy cluster and rises during an incident, which is when you want them. Turn them off with --set cluster.events=false.

Name every cluster. fixter.clusterName is not detected automatically, and an unset name means the cluster is simply not identified in your telemetry. That is survivable with one cluster and ambiguous the moment you have two, which is why it is in the install command above.

Send your applications’ telemetry through it

The agent exposes an OTLP endpoint on every node. Point your workloads at its Service, named <release>-fixter-collector-agent, which for the install above is fixter-collector-agent. Set these on your application deployments:

OTEL_EXPORTER_OTLP_ENDPOINT=http://fixter-collector-agent.fixter.svc.cluster.local:4318
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf

Traffic stays on the node it started from. Your applications still need OpenTelemetry instrumentation to produce traces and logs in the first place: see Send data with OpenTelemetry for that half.

Service names are filled in, never overwritten. Pod logs and kubelet metrics carry no service identity of their own, so the agent supplies one from Kubernetes metadata: the deployment name (falling back to the container name), the namespace, and the pod name. Telemetry your applications send keeps whatever service.name they set. Node and host metrics stay serviceless, which is correct for infrastructure.

Log volume

Pod logs are collected by default. kube-system and the collector’s own pods are already excluded. Before you leave this running, do the arithmetic: roughly 216-byte JSON lines at 10,000 logs per second is about 2.2 MB/s, or 186 GB a day.

Trim by namespace or pod. Put this in a values.yaml next to wherever you keep your Helm configuration:

# values.yaml
agent:
  logs:
    excludeNamespaces: [kube-system, istio-system]
    excludePods: [chatty-app]

Apply it, keeping the settings from your original install:

helm upgrade fixter-collector \
  oci://ghcr.io/fixter-dev/charts/fixter-collector \
  --namespace fixter --reuse-values -f values.yaml

Or turn pod logs off entirely and rely on your applications exporting their own:

helm upgrade fixter-collector \
  oci://ghcr.io/fixter-dev/charts/fixter-collector \
  --namespace fixter --reuse-values \
  --set agent.logs.enabled=false

How log severity is worked out

A pod log line carries no severity of its own, and a stack trace arrives as one line per frame. What the collector can do about that depends on whether it recognises the format.

Out of the box it reads only what a line’s own shape reveals, and does not guess:

Format	Severity	Record boundaries
JSON with a string level (zap, winston, logback)	yes, from level	one per line
JSON with a numeric level (pino, bunyan)	yes, 10 to 60 mapped	one per line
glog and klog, used by Kubernetes components	yes, from the I/W/E/F prefix	one per line
any other text format	none	one per line

That last row is deliberate. Guessing a level from an unknown format goes wrong in ways that are worse than having no level: a service named trace-service read as TRACE, a successful nginx line containing /error/ read as ERROR. Joining continuation lines is riskier still, because a wrong join merges unrelated events into one record and buries an ERROR inside an INFO. Nothing is ever dropped for being unparseable; an unrecognised line still reaches Fixter, just without a severity.

This is the practical argument for logging JSON from your own services. Do that and severity, stack traces, and trace correlation all work with no configuration.

Common datastores are handled automatically, because they log under predictable container and pod names: ClickHouse, Doris, PostgreSQL, MySQL, Kafka, and the AWS VPC CNI. Turn one off with agent.logs.builtinFormats.mysql.enabled: false in your values file.

For a text format the collector does not know, point a preset at the pods that emit it, in the same values.yaml and applied the same way:

# values.yaml
agent:
  logs:
    formats:
      - preset: spring
        include: ["/var/log/pods/prod_*/*/*.log"]
      - preset: python
        include: ["/var/log/pods/*_worker-*/*/*.log"]

Presets available: spring, clickhouse, postgres, mysql, kafka, doris-fe, logfmt, python, dotnet, go-stdlib, and zap-console.

Two things to know before writing one:

• Formats are selected by log file path, in the shape /var/log/pods/<namespace>_<pod>_<uid>/<container>/0.log, so you target a namespace, a pod name prefix, or a container. Pod labels and annotations are not available at the point the line is parsed.
• First match wins, and each format automatically excludes the paths of every format above it. Put narrow patterns before broad ones, and do not add those exclusions by hand: two receivers reading one file duplicates every record.

Scraping Prometheus endpoints

If services already expose Prometheus metrics, the cluster collector can scrape them. There are two ways to say so in your values file, and you can use both at once.

A list of targets, when you know the addresses:

# values.yaml
integrations:
  prometheus:
    targets:
      - job: my-service
        endpoints: ["my-service.default.svc.cluster.local:9090"]
        interval: 30s

A raw scrape_configs block, when you need service discovery or relabeling. The collector ships the full upstream Prometheus receiver, and what you put here is passed to it verbatim, so any config from your existing Prometheus setup works unchanged:

# values.yaml
integrations:
  prometheus:
    scrapeConfigs:
      - job_name: kubernetes-pods
        kubernetes_sd_configs:
          - role: pod
        relabel_configs:
          - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
            action: keep
            regex: "true"

Either key enables the receiver. Apply it the same way as any other values change:

helm upgrade fixter-collector \
  oci://ghcr.io/fixter-dev/charts/fixter-collector \
  --namespace fixter --reuse-values -f values.yaml

The cluster collector restarts with the new scrape config, and the scraped series show up on the Metrics page under the service the targets belong to.

What this does not collect

Worth knowing before you conclude something is broken:

• Control plane logs on managed Kubernetes. On EKS, apiserver, etcd, scheduler and controller logs go to CloudWatch rather than to pods, and kubelet and containerd log to journald on the node. The collector reads neither.
• kube-system pods are excluded by default for volume. CoreDNS, Karpenter, external-dns and the CSI and load balancer controllers do write pod logs, so if you want those, set agent.logs.excludeNamespaces: [] and accept the extra volume.
• Datastore metrics. Datastore logs are parsed, as above, but there are no metrics receivers for ClickHouse, Doris, PostgreSQL, MySQL or Redis yet.
• Managed cloud services whose telemetry lives in CloudWatch rather than in your cluster, such as RDS. For AWS logs and metrics that reach a Firehose stream, see AWS Firehose.

If nothing shows up

• The pods are Ready but no data arrives. Almost always the API key. Run helm test fixter-collector -n fixter; a failure there is a bad or wrong key.
• The agent pod will not schedule. The namespace is likely under a restricted Pod Security Admission policy. The agent needs read-only host path mounts for /var/log/pods and host metrics, so install it into a namespace that permits them.
• Metrics arrive but no pod logs. Check agent.logs.enabled and your namespace and pod exclusions. Note that kube-system is excluded by default.
• Logs arrive with no severity, or stack traces split across records. The format is not one the collector recognises. Log JSON, or point a preset at those pods, as above.
• You cannot tell two clusters apart. fixter.clusterName is not detected automatically; set it on each install.

Still stuck? Email info@fixter.dev and one of our founders will help you fix your problem asap.

Related

• Send data with OpenTelemetry: instrumenting your applications, and the exporter settings if you already run your own collector
• Telemetry: what each signal answers and why you want all three
• Data model: the shape of what arrives