GPU usage metrics in Kubernetes fall short for shared workloads

If you run GPUs in Kubernetes, you'll find that tracking who uses what is a headache. Unlike CPU and memory, where usage is reported per container, GPU metrics on Kubernetes are crude—especially when several workloads share one GPU. The default NVIDIA tool only shows totals at the device level, overstating overall usage and making cost tracking unreliable. You'll need custom metrics just to know how resources are actually being used.

• Kubernetes lacks good per-container GPU usage metrics
• NVIDIA's tool only reports device-level totals
• Shared GPUs often appear over-allocated in dashboards
• Standard CPU and memory tracking doesn’t work for GPUs