0171.K pod日志提示open prometheus queries.active permission denied解决

kubernetes V1.23.5;

[root@rh-master01 ~]# kubectl get pod -n kuboard
NAME                                     READY   STATUS             RESTARTS         AGE
...
prometheus-k8s-0                         1/2     CrashLoopBackOff   34 (3m33s ago)   153m
prometheus-k8s-1                         1/2     CrashLoopBackOff   34 (3m41s ago)   153m
prometheus-operator-5d7cc5dc4d-wl97p     2/2     Running            0                156m
system-monitor-config-864f784987-ngqk2   1/1     Running            0                153m

[root@rh-master01 ~]# kubectl logs prometheus-k8s-0 -n kuboard
...
level=info ts=2022-05-26T03:07:52.844Z caller=main.go:428 msg="Starting Prometheus" version="(version=2.29.1, branch=HEAD, revision=dcb07e8eac34b5ea37cd229545000b857f1c1637)"
level=info ts=2022-05-26T03:07:52.844Z caller=main.go:433 build_context="(go=go1.16.7, user=root@364730518a4e, date=20210811-14:48:27)"
level=info ts=2022-05-26T03:07:52.844Z caller=main.go:434 host_details="(Linux 5.17.8-1.el7.elrepo.x86_64 #1 SMP PREEMPT Sat May 14 07:21:57 EDT 2022 x86_64 prometheus-k8s-0 (none))"
level=info ts=2022-05-26T03:07:52.844Z caller=main.go:435 fd_limits="(soft=1048576, hard=1048576)"
level=info ts=2022-05-26T03:07:52.844Z caller=main.go:436 vm_limits="(soft=unlimited, hard=unlimited)"
level=error ts=2022-05-26T03:07:52.845Z caller=query_logger.go:87 component=activeQueryTracker msg="Error opening query log file" file=/prometheus/queries.active err="open prometheus/queries.active: permission denied"
panic: Unable to create mmap-ed active query log
...

msg="Error opening query log file" file=/prometheus/queries.active err="open prometheus/queries.active: permission denied" panic: Unable to create mmap-ed active query log

权限问题，Kuboard 监控套件基于kube-prometheus构建,prometheus的镜像中文件/prometheus/queries.active属主为1000这个用户，当前nfs路径prometheus-k8s-db-prometheus-k8s-0属主是root用户(有权限风险),从而导致写入失败。

[root@rh-master01 ~]# chmod -R 777 vm/dev-nfs/kuboard_pv/prometheus-k8s-db-prometheus-k8s-0
[root@rh-master01 ~]# chmod -R 777 vm/dev-nfs/kuboard_pv/prometheus-k8s-db-prometheus-k8s-1


等待数分钟之后，prometheus-k8s-0和prometheus-k8s-0状态便为Running,问题解决
[root@rh-master01 ~]# kubectl get po -n kuboard
NAME                                     READY   STATUS    RESTARTS       AGE
...
prometheus-k8s-0                         2/2     Running   80 (16m ago)   6h35m
prometheus-k8s-1                         2/2     Running   81 (11m ago)   6h35m
...

[root@rh-master01 ~]# kubectl exec -it prometheus-k8s-0 sh -n kuboard
/prometheus $ ls -l prometheus/queries.active
-rw-r--r--    1 1000     root         20001 May 27 00:45 prometheus/queries.active

cat etc/exports
/vm/dev-nfs    192.168.80.0/24(rw,sync,no_root_squash)

http://events.jianshu.io/p/8b9fd5493682
https://github.com/prometheus/prometheus/issues/9704