k8s–基础–26.6–监控告警系统–kube-state-metrics

1、kube-state-metrics是什么？

1. kube-state-metrics 会监听API Server生成有关资源对象的状态指标，比如Deployment、Node、Pod。
2. kube-state-metrics只是简单的提供一个metrics数据，并不会存储这些指标数据，我们可以使用Prometheus来抓取这些数据然后存储。
3. 指标数据
   1. Deployment、Pod、副本状态等
   2. 调度了多少个replicas
   3. 现在可用的pod有几个
   4. 多少个Pod是running/stopped/terminated状态
   5. Pod重启了多少次
   6. 我有多少job在运行中。

2、安装kube-state-metrics前

各个节点都要操作

2.1、上传镜像

2.2、解压

# 解压缩
docker load -i /root/k8s/monitor/kube-state-metrics_1_9_0.tar.gz
 
# 删除
rm -rf /root/k8s/monitor/kube-state-metrics_1_9_0.tar.gz

3、安装kube-state-metrics

1. 以下操作都在master1节点上

3.1、创建sa，并对sa授权

3.1.1、脚本

vi /root/k8s/monitor/kube-state-metrics-rbac.yaml

内容

# 创建sa
---
apiVersion: v1
kind: ServiceAccount
metadata:
  # sa 账号名称
  name: kube-state-metrics
  # sa 账号名称空间
  namespace: kube-system
---
# 对sa账号进行rbac授权
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kube-state-metrics
rules:
- apiGroups: [""]
  resources: ["nodes", "pods", "services", "resourcequotas", "replicationcontrollers", "limitranges", "persistentvolumeclaims", "persistentvolumes", "namespaces", "endpoints"]
  verbs: ["list", "watch"]
- apiGroups: ["extensions"]
  resources: ["daemonsets", "deployments", "replicasets"]
  verbs: ["list", "watch"]
- apiGroups: ["apps"]
  resources: ["statefulsets"]
  verbs: ["list", "watch"]
- apiGroups: ["batch"]
  resources: ["cronjobs", "jobs"]
  verbs: ["list", "watch"]
- apiGroups: ["autoscaling"]
  resources: ["horizontalpodautoscalers"]
  verbs: ["list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kube-state-metrics
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kube-state-metrics
subjects:
- kind: ServiceAccount
  name: kube-state-metrics
  namespace: kube-system

3.1.2、执行

kubectl apply  -f  /root/k8s/monitor/kube-state-metrics-rbac.yaml

# 查看
kubectl get sa -n kube-system | grep metrics

3.2、安装kube-state-metrics组件

3.2.1、脚本

vi /root/k8s/monitor/kube-state-metrics-deploy.yaml

内容

apiVersion: apps/v1
kind: Deployment
metadata:
  # Deployment的名称和 使用的名称空间
  name: kube-state-metrics
  namespace: kube-system
spec:
  # 副本实例
  replicas: 1
  # 标签选择器，选择哪个template
  selector:
    matchLabels:
      app: kube-state-metrics
  # 定义template
  template:
    metadata:
      labels:
        app: kube-state-metrics
    # 定义容器
    spec:
      # sa账号
      serviceAccountName: kube-state-metrics 
      containers:
      - name: kube-state-metrics
#        image: gcr.io/google_containers/kube-state-metrics-amd64:v1.3.1
        image: quay.io/coreos/kube-state-metrics:v1.9.0
        ports:
        - containerPort: 8080

3.2.2、执行

kubectl apply  -f /root/k8s/monitor/kube-state-metrics-deploy.yaml

# 查看
kubectl get ConfigMap -n monitor-sa

4、在grafana中导入kube-state-metrics的配置

4.1、配置文件

4.2、导入Kubernetes Cluster(Prometheus)-1577674936972.json后，效果图

4.3、导入Kubernetes cluster monitoring(via Prometheus)(k8s 1.16)-1577691996738.json后，效果图