Advertise Extended Node Resources for a Kubernetes Node

Cluster administrators often need to advertise custom hardware or software resources to Kubernetes. Extended resources let you declare these node-level assets so the scheduler can allocate them. This article shows static and dynamic methods to configure and use extended node resources.

TL;DR

• Extended resources let you add custom node capacity and allocatable values.
• Enable ExtendedResourcePodScheduling feature gate in Kubelet.
• Use KubeletConfiguration or kubectl patch for static resources.
• Device plugins offer dynamic extended resources (GPUs, NICs).
• Scheduler respects resource namespaced as “domain/name”.
• Pods request extended resources via spec.resources.requests.

Extended Node Resources Overview

Extended resources appear under Node.status.capacity and Node.status.allocatable. Kubernetes treats them like CPU or memory but with custom names. You declare resourceName in the form domain/name. The scheduler matches Pod.spec.resources.requests.name to Node.allocatable.name.

Configuring Extended Node Resources

Two main methods exist: static via KubeletConfiguration file and dynamic via device plugins.

Static Method: KubeletConfiguration

Edit your Kubelet config file (e.g., /var/lib/kubelet/config.yaml) to include:

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
featureGates:
  ExtendedResourcePodScheduling: true
nodeStatusUpdateFrequency: "10s"
extendedResourceAllocation:
  resourceList:
    example.com/foo: "4"

Then restart Kubelet. It will update Node.status.capacity and allocatable with 4 units of example.com/foo.

Static Method: kubectl patch

For quick tests, use kubectl patch:

kubectl patch node worker1 \
  -p '{"status":{"capacity":{"example.com/foo":"2"},"allocatable":{"example.com/foo":"2"}}}' \
  --type=merge

This command sets both capacity and allocatable of example.com/foo to 2 on node worker1.

Extended Node Resources Scheduling

Pods request extended resources under resources.requests:

apiVersion: v1
kind: Pod
metadata:
  name: foo-user
spec:
  containers:
  - name: app
    image: busybox
    resources:
      requests:
        example.com/foo: "1"
    command: ["sleep","3600"]

The scheduler picks a node with ≥1 example.com/foo allocatable.

Dynamic Extended Node Resources

Use device plugins for hardware resources like GPUs or NICs. The plugin advertises resources through the Kubelet Device Plugin API. The Kubelet then populates Node.status with resource name and count.

Security and Access Controls

Extended resources follow the same RBAC and PodSecurityPolicies as standard resources. You can restrict which namespaces or service accounts may request custom resources.

References

• Advertise Extended Resources for a Node | Kubernetes
• KubeletConfiguration API reference
• Device Plugins

Suggested Reading

• How to Drain a Node in Kubernetes
• Understanding udev: Command Usage and Examples
• Kubernetes API for Cluster Administration
• The difference between kubectl apply and kubectl create
• Kubernetes Pod Evicted: Causes and Resolutions

• Which Kubernetes apiVersion Should You Use?

PostHashID: 87f1f295ea728eb811f33e7c528d14bfdf6d8078af9afd497dcc6f0d4f822972