NewAgent Observability — find failures before customers do.
Back to directory
NVIDIA DCGM logo

NVIDIA DCGM MCP server: GPU metrics via community servers

DCGM telemetry usually lands in Prometheus through dcgm-exporter. An agent can query it there, or use a community server that inspects GPUs on the node itself.

Connector
Community only
Coverage
GPU health, utilization, XID errors
Tools
9 in k8s-gpu-mcp-server
Working route
dcgm-exporter, then Prometheus MCP

Checked 3 October 2026 · from NVIDIA's DCGM and dcgm-exporter repositories and docs, the community server repositories, npm and the GitHub container registry

One of 91 guides to monitoring and developer tools. Each covers what an AI coding agent can get out of the tool through MCP, the protocol that lets the agent query the tool directly.

Summary

DCGM is NVIDIA's Data Center GPU Manager, the daemon that collects health and utilization data from data-center GPUs. As of 3 October 2026, NVIDIA has not published an MCP server for it.

Three community projects reach DCGM data over MCP, the Model Context Protocol agents use to call tools. The fullest is ArangoGutierrez/k8s-gpu-mcp-server, a personal project by an NVIDIA engineer.

Most clusters already run dcgm-exporter, which turns DCGM fields into Prometheus metrics. Connecting Prometheus's first-party MCP server gives an agent those series with no GPU-specific install.

How NVIDIA DCGM works with MCP

DCGM runs on each GPU node. It reads temperatures, clocks, power, memory use, NVLink traffic and XID errors. XID errors are the codes the NVIDIA driver logs when a GPU faults.

dcgm-exporter is NVIDIA's companion that publishes those readings as Prometheus metrics, such as DCGM_FI_DEV_GPU_UTIL for utilization. It has 1,890 GitHub stars and was last pushed on 18 September 2026.

DCGM also runs active diagnostics, which stress a GPU to confirm it is healthy. Google Cloud's Cluster Director MCP server exposes those diagnostics as a single tool.

Every vendor here gets the same four checks: official, maintained, distributed and usable. NVIDIA DCGM passes 1 of them.

CheckResultEvidence
OfficialFailAs of 3 October 2026, NVIDIA has not published a DCGM MCP server. The two MCP-named repositories in the NVIDIA GitHub organization, SkillSpector and elements, do not read DCGM.
MaintainedUnknownThe most complete community server, ArangoGutierrez/k8s-gpu-mcp-server, was last pushed on 8 June 2026 and has 4 stars.
DistributedPassk8s-gpu-mcp-server publishes container images at ghcr.io/arangogutierrez/k8s-gpu-mcp-server, tagged 0.1.0 and 0.2.0 among others. Its README also gives an npx command, but npm returned "Not found" for that package on 3 October 2026.
UsableUnknownThe community servers need real NVIDIA GPUs, and k8s-gpu-mcp-server also needs a Kubernetes cluster, so their tool calls could not be exercised from outside.
Part of the workWhat NVIDIA DCGM has hereCan an agent reach it
Production observabilityGPU utilization, temperature, memory, power, NVLink and XID errorsPartial. Prometheus's server reads the exported series; community servers read the node directly
Agent observability & evalsGPU load behind model training and inferencePartial. Utilization per GPU, but nothing about the model requests running on it
AI cost managementUtilization and energy use as a proxy for GPU spendPartial. DCGM_FI_DEV_TOTAL_ENERGY_CONSUMPTION and utilization, with no prices attached

DCGM answers whether a GPU is busy, hot or failing. For which job caused the load, pair it with a Kubernetes or scheduler view. For what the GPU hours cost, use a cost tool.

What can an AI agent do with NVIDIA DCGM?

A DCGM metric the agent asks for returns no series

dcgm-exporter publishes only the fields listed in its counters file. The default list is etc/default-counters.csv, and any field missing from it never reaches Prometheus.

Profiling fields such as DCGM_FI_PROF_PIPE_TENSOR_ACTIVE also depend on the GPU model supporting them.

Guard: List the DCGM_ metric names Prometheus holds before the agent writes a query, and add missing fields to the exporter's counters file.

The agent reports an XID error count that is really an error code

DCGM_FI_DEV_XID_ERRORS is a gauge holding the value of the last XID error, not a running total. A reading of 79 means XID 79, which NVIDIA documents as a GPU that has fallen off the bus.

Guard: Tell the agent to look up the XID number in NVIDIA's XID catalogue rather than summing or averaging the metric.

The k8s-gpu-mcp-server npx command fails to install

The README's quick start runs npx k8s-gpu-mcp-server@latest. On 3 October 2026 the npm registry returned "Not found" for that package name.

Guard: Use the container image from ghcr.io/arangogutierrez/k8s-gpu-mcp-server or build from source with make agent, as the README also documents.

How to configure NVIDIA DCGM MCP for agents

  1. 1

    Read-only first

    Prometheus's core tool profile leaves out its administrative tools, which can delete series or shut the server down.

  2. 2

    Node access is a wider grant

    k8s-gpu-mcp-server runs on GPU nodes and gpu-watch uses SSH, so both see more than a Prometheus query does.

  3. 3

    Keep the operator tools off

    k8s-gpu-mcp-server plans kill_gpu_process and reset_gpu behind an explicit operator mode; its default is --mode=read-only.

Paste this into your agent’s instructions

Terminal
When you answer from DCGM data, state:
- The metric names, the GPUs or nodes, and the time range.
- Whether the data came from Prometheus or from a node-level tool.
Report an XID value as an error code, not a count.
If no DCGM series exist, say the exporter may not be scraped.

Do you need the MCP server at all?

Prometheus. The prometheus/prometheus-mcp server is first-party, with 115 stars and a push on 2 October 2026. The agent writes PromQL, the Prometheus query language, against the DCGM_ series.

Grafana. If dashboards and alerts already sit on that Prometheus, Grafana's server adds alert rules and panels over the same GPU data.

`ArangoGutierrez/k8s-gpu-mcp-server`. It inspects GPUs on each Kubernetes node through NVML, the NVIDIA Management Library, and uses DCGM when available. Tools include get_gpu_health, analyze_xid_errors and get_pod_gpu_allocation.

`GoogleCloudPlatform/cluster-director-mcp`. For clusters built with Google Cloud Cluster Director, its run_dcgm_test tool runs DCGM diagnostics on the GPU nodes. It is a Gemini CLI extension meant for Cloud Shell.

`01p5/gpu-watch`. A read-only Slurm-oriented monitor that reaches nodes over SSH and parses dcgm-exporter's /metrics endpoint directly. It has 0 stars and was last pushed on 1 June 2026.

Query dcgm-exporter through Prometheus for trends and alerts. Add a node-level community server only when the agent must diagnose a specific failing GPU.

NVIDIA DCGM integration setup

The working route needs no DCGM-specific server. Point Prometheus's MCP server at the Prometheus that scrapes dcgm-exporter.

Connect Prometheus in Claude Code

Use the URL of the Prometheus instance that scrapes dcgm-exporter. The core profile keeps the tool list small and read-only.

Terminal
claude mcp add-json "prometheus" '{"command":
  "prometheus-mcp-server",
  "args":["--mcp.tools=core",
          "--prometheus.url=http://prometheus:9090"]}'

Check the DCGM series exist

Run this before connecting anything. An empty result means Prometheus is not scraping dcgm-exporter, and connecting a server will not change that.

Terminal
curl -s "http://prometheus:9090/api/v1/label/__name__/values" \
  | grep -o '"DCGM_[^"]*"' | head

A first query for the agent

Average utilization per GPU over the last hour is a useful smoke test once the series are present.

Terminal
avg_over_time(DCGM_FI_DEV_GPU_UTIL[1h])

Every other client

Each block below is the configuration for one client, with the file path and the key that client expects.

Node-level diagnosis with k8s-gpu-mcp-server

The README documents a Helm install and a source build with make agent. A mock NVML mode lets you inspect the tool output without GPUs.

Prebuilt images are published at ghcr.io/arangogutierrez/k8s-gpu-mcp-server. The author's GitHub profile lists NVIDIA as employer, but the repository is personal, not an NVIDIA product.

What engineers report

Adoption of DCGM MCP servers is small. k8s-gpu-mcp-server has 4 stars, Cluster Director's server has 9, and gpu-watch has none. dcgm-exporter itself has 1,890 stars, which reflects how most teams already consume DCGM data.

Should you connect an agent to NVIDIA DCGM?

Yes, through Prometheus, if your cluster already scrapes dcgm-exporter. Best for platform teams checking GPU utilization, temperature and XID errors across a cluster.

Strongest fit
Yes, through Prometheus, if your cluster already scrapes dcgm-exporter.
Main advantage
GPU telemetry is usually already in Prometheus, and Prometheus has a maintained first-party server.
Main weakness
NVIDIA ships no GPU-aware tools, so the agent works from raw metric series.
Operational risk
Community servers that run on GPU nodes or over SSH hold far wider access than a read-only Prometheus query.

From Oodle

One platform for agent traces and infrastructure

Agent traces usually sit in a different product from the rest of your telemetry, so when a slow database makes an agent slow the symptom is in one tool and the cause is in another. Oodle keeps both in one query surface.

See agent observability

Frequently asked questions

Does NVIDIA publish a DCGM MCP server?

Not as of 3 October 2026. Community servers exist, including ArangoGutierrez/k8s-gpu-mcp-server, a personal project by an NVIDIA engineer, and Google Cloud's cluster-director-mcp.

How can an agent read DCGM metrics today?

Through Prometheus. dcgm-exporter publishes DCGM fields as Prometheus metrics, and the prometheus/prometheus-mcp server lets the agent query them with PromQL.

Can an agent run DCGM diagnostics?

Yes, on Google Cloud Cluster Director clusters. The run_dcgm_test tool in cluster-director-mcp runs DCGM tests on the cluster's GPU nodes.

Does DCGM data show what a GPU costs?

No. It reports utilization and energy use, which help explain spend. Prices and billing come from a cloud or Kubernetes cost tool.

Changelog

  • 14 November 2025. Google Cloud created cluster-director-mcp, which includes the run_dcgm_test tool.
  • 3 January 2026. ArangoGutierrez/k8s-gpu-mcp-server was created.

Sources

Every number on the page traces to one of these

  1. NVIDIA DCGM documentation, read 3 October 2026
  2. dcgm-exporter documentation, read 3 October 2026
  3. NVIDIA Xid catalog, read 3 October 2026
  4. NVIDIA/dcgm-exporter default counters, read 3 October 2026
  5. ArangoGutierrez/k8s-gpu-mcp-server, created 3 January 2026, read 3 October 2026
  6. GoogleCloudPlatform/cluster-director-mcp, created 14 November 2025, read 3 October 2026
  7. 01p5/gpu-watch, read 3 October 2026
  8. prometheus/prometheus-mcp, read 3 October 2026
  9. npm registry lookup for k8s-gpu-mcp-server, 3 October 2026: "Not found"

For the metrics dcgm-exporter publishes:

For what the GPUs cost:

For what the GPUs are serving: