当用户重启kubelet后,发现NPU Exporter没有上报容器的相关信息。
kubelet重启后会重新创建新dockershim.sock文件,但是NPU Exporter获取的是旧dockershim.sock文件,导致无法获取当前容器的数据信息。
选择以下方式之一进行处理。
kubelet重启后,容器会自动退出,由K8s重新拉起容器后,NPU Exporter即可正常获取容器信息。容器拉起期间(大约10s),NPU Exporter会丢失所有数据信息。
kubectl delete pod -n npu-exporter <npu-exporter-podname>
删除容器后,容器会被重新拉起(大约10s),NPU Exporter会丢失期间所有数据信息。
vi npu-exporter-v{version}.yaml
...
volumeMounts:
- name: log-npu-exporter
...
- name: sys
mountPath: /sys
readOnly: true
- name: docker-shim # 删除以下加粗字段
mountPath: /var/run/dockershim.sock
readOnly: true
- name: docker
mountPath: /var/run/docker
readOnly: true
- name: cri-dockerd
mountPath: /var/run/cri-dockerd.sock
readOnly: true
- name: containerd # delete when only use isula
mountPath: /run/containerd
...
volumes:
- name: log-npu-exporter
...
- name: sys
hostPath:
path: /sys
- name: docker-shim # 删除以下加粗字段
hostPath:
path: /var/run/dockershim.sock
- name: docker
hostPath:
path: /var/run/docker
- name: cri-dockerd
hostPath:
path: /var/run/cri-dockerd.sock
- name: containerd
hostPath:
path: /run/containerd
...
... volumeMounts: - name: log-npu-exporter ... - name: sys mountPath: /sys readOnly: true - name: sock mountPath: /var/run # 以实际的dockershim.sock文件目录为准 ... volumes: - name: log-npu-exporter ... - name: sys hostPath: path: /sys - name: sock hostPath: path: /var/run # 以实际的dockershim.sock文件目录为准 - name: containerd hostPath: path: /run/containerd ...