Kubelet 开启 PodPidsLimit

背景

避免某应用泄露问题导致整个集群不可用，将针对集群中的Pod最大只能运行4096个进程。

实现

kubelet 中追加以下参数

# cat /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS="--pod-max-pids=4096"

重启生效

# systemctl restart kubelet

验证

# kubectl get --raw /api/v1/nodes/主机名/proxy/configz | jq .kubeletconfig.podPidsLimit

验证实例

在集群中创建以下Pod应用

apiVersion: v1
kind: Pod
metadata:
  name: pids-stress-test
spec:
  restartPolicy: Never
  containers:
  - name: pids-stress
    image: registry.cn-shenzhen.aliyuncs.com/shuhui/busybox:1.37.0
    command:
    - /bin/sh
    - -c
    - |
      echo "=== PID上限测试 ==="
      echo "当前 pids.max: $(cat /sys/fs/cgroup/pids/pids.max 2>/dev/null)"
      echo ""
      echo "开始消耗进程数..."
      i=0
      while true; do
        i=$((i+1))
        (
          # 模拟常驻进程
          sleep infinity
        ) &
        echo "已创建进程数: $i"
        sleep 0.001
      done

查看日志

# k logs -f pids-stress-test
...
已创建进程数: 4090
已创建进程数: 4091
已创建进程数: 4092
已创建进程数: 4093
已创建进程数: 4094
/bin/sh: can't fork: Resource temporarily unavailable

可以看到，触发阈值后Pod立马被中断

另一种验证

 # kubectl get pods pids-stress-test -o yaml | grep -E 'uid|qos'
  uid: a65a278d-6960-4b78-99c3-bd1e6a460541
  qosClass: BestEffort
  

# systemctl |grep a65a278d |grep -i BestEffort
  kubepods-besteffort-poda65a278d_6960_4b78_99c3_bd1e6a460541.slice                                                                                                   loaded active active    libcontainer container kubepods-besteffort-poda65a278d_6960_4b78_99c3_bd1e6a460541.slice

找到对应的systemd控制名称后，使用以下来验证

# systemctl cat kubepods-besteffort-poda65a278d_6960_4b78_99c3_bd1e6a460541.slice | grep -i tasksmax
# /run/systemd/transient/kubepods-besteffort-poda65a278d_6960_4b78_99c3_bd1e6a460541.slice.d/50-TasksMax.conf
TasksMax=4096

提示：发现已经追加了在kubelet中声明的4096！

通过对应cgroup来验证

# find  /sys/fs/cgroup/ -name "*a65a278d*"
/sys/fs/cgroup/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda65a278d_6960_4b78_99c3_bd1e6a460541.slice
[root@kube-m-01 stress]# cat /sys/fs/cgroup/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda65a278d_6960_4b78_99c3_bd1e6a460541.slice/pids.max
4096