Kubernetes clusters require TLS certificates to authenticate API server, kubelets, and control plane components. Weak or expired certificates expose cluster to man-in-the-middle attacks and unauthorized access. This article explains certificate generation, signing, rotation, and revocation. It guides you through CA setup, server and client certificates, Certificate Signing Requests (CSR), rotation strategies, CSI integration, access controls, and security contexts.
TL;DR
- Set up a dedicated root CA and intermediate CA for cluster certificates.
- Issue server certificates for API server, kubelets, etcd, and controller-manager.
- Use CSR API and kubectl to request and approve certs dynamically.
- Automate certificate rotation via kube-controller-manager and external tools.
- Apply
securityContext, quotas, and RBAC to limit certificate usage. - Integrate certificates with CSI drivers for encrypted volumes and event flows.
Kubernetes Certificates Overview
Kubernetes Certificates secure every communication channel. Certificates rely on Public Key Infrastructure (PKI). You need a root Certificate Authority (CA) to issue leaf certificates. Kubernetes ships with a default CA in kube-controller-manager, but production clusters benefit from custom CA for compliance and rotation control.
Key certificate types:
- Root CA: signs intermediate CAs and leaf certs.
- Server certificates: API server, etcd server, admission controllers.
- Client certificates: kubelet clients, controllers, scheduler.
- CSR objects: dynamic cert requests via Kubernetes API.
Generating Kubernetes Certificates
Follow these steps to generate certificates using cfssl. You can substitute openssl.
Note: This example uses Cloudflare’s cfssl tool. Here’s a nice article on how to use the tool: How to use cfssl to create self signed certificates. You may alternatively use openssl if you are comfortable with the same.
# Create CA config
cat > ca-config.json <<EOF
{
"signing": {
"default": {"expiry": "8760h"},
"profiles": {"kubernetes": {"usages": ["signing","key encipherment","server auth","client auth"],"expiry": "8760h"}}
}
}
EOF
# Generate CA
cfssl gencert -initca ca-csr.json | cfssljson -bare ca
Next, generate server certificate for API server:
# apiserver-csr.json
{
"CN": "kube-apiserver",
"hosts": ["10.0.0.1","127.0.0.1","kubernetes.default"],
"key": {"algo": "rsa","size": 2048},
"names": [{"O": "kubernetes"}]
}
# Sign cert
cfssl gencert \
-ca=ca.pem -ca-key=ca-key.pem \
-config=ca-config.json \
-profile=kubernetes apiserver-csr.json | cfssljson -bare apiserver
Repeat for etcd, controller-manager, scheduler, and kubelet certificates. Store files under /etc/kubernetes/pki.
Certificate Signing Requests in Kubernetes
Kubernetes CSR API enables dynamic requests. Client components generate a private key and CSR. They post a CertificateSigningRequest resource. Cluster admin approves via kubectl.
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
name: kube-node-csr
spec:
request: $BASE64_CSR
signerName: kubernetes.io/kubelet-serving
usages:
- digital signature
- key encipherment
- server auth
Approve CSR:
kubectl certificate approve kube-node-csrFetch the issued certificate:
kubectl get csr kube-node-csr -o jsonpath='{.status.certificate}' | base64 -d > kubelet.crtCertificate Rotation in Kubernetes
Kubernetes supports automatic rotation of kubelet client certificates. kube-controller-manager issues new certs before expiry. You control rotation via --rotate-certificates flag in kubelet.
Configure kubelet:
# /var/lib/kubelet/config.yaml
clientCAFile: "/etc/kubernetes/pki/ca.crt"
serverTLSBootstrap: true
rotateCertificates: true
To rotate control plane certs, replace CA and leaf certificates periodically. Use scripts or cert-manager for automation. Ensure apiserver and etcd receive updated certificates simultaneously to avoid downtime.
Integrating Certificates with CSI Drivers
Container Storage Interface (CSI) drivers often require TLS for controller-endpoint communication. Use Kubernetes Certificates to secure CSIDriver endpoints. Store secrets in Secret resources with Data keys tls.crt and tls.key. Mount secrets into CSI driver pods.
apiVersion: v1
kind: Secret
metadata:
name: csi-tls-secret
type: kubernetes.io/tls
data:
tls.crt:
tls.key:
Volume mount in CSI driver Deployment:
volumes:
- name: tls-creds
secret:
secretName: csi-tls-secret
containers:
- name: csi-controller
volumeMounts:
- mountPath: "/etc/tls"
name: tls-creds
Certificate Lifecycle
Security Contexts, Quotas, and Access Controls
Limit certificate access via Pod securityContext. Restrict file permissions on /etc/kubernetes/pki to root:root and chmod 600. Apply ResourceQuota on secrets to limit total TLS secrets.[3]
apiVersion: v1
kind: ResourceQuota
metadata:
name: tls-secret-quota
spec:
hard:
secrets: "10"
requests.cpu: "2"
requests.memory: "4Gi"
Enforce RBAC on certificate operations. Grant only kube-controller-manager and cert-manager ServiceAccounts permission to approve CSRs. Sample ClusterRoleBinding:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: csr-approver-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:certificates.k8s.io:certificatesigningrequests:nodeclient
subjects:
- kind: ServiceAccount
name: cert-manager
namespace: cert-manager
Use-Cases for Kubernetes Certificates
Event-driven flows benefit from certificates. For example, KEDA triggers scale events based on MQTT messages secured via TLS. Each scale target holds client certificates issued by cluster CA. Use CSI volumes encrypted with mTLS between nodes and brokers. This ensures end-to-end security in event pipelines.
Best Practices for Kubernetes Certificates
- Use short-lived certificates with automated rotation.
- Isolate CA hierarchy: root CA offline, intermediate CA online.
- Audit CSR approvals in logs for compliance.
- Encrypt certificate private keys at rest using Hardware Security Modules.
- Monitor certificate expiry and get alerts via Prometheus rules.
References
Suggested Reading
PostHashID: 8e71fef1364aded1ea9fc07a3ba1adc8504c7419ba7cb0fec9573332b8f1fdf0
