36.自定义指标HPA

自定义指标HPA

Kubernetes中不仅支持CPU、内存为指标的HPA,还支持自定义指标的HPA,例如QPS

设置自定义指标

Kubernetes 1.6

Kubernetes1.6集群中配置自定义指标的HPA的说明已废弃。

在设置定义指标HPA之前需要先进行如下配置:

  • heapster的启动参数 --api-server 设置为true

  • 启用custom metric API

  • kube-controller-manager的启动参数中 --horizontal-pod-autoscaler-use-rest-clients 设置为true,并指定 --masterAPI server地址,如 --master=http://172.20.0.113:8080

Kubernetes1.5以前很容易设置,参考 1.6以前版本的kubernetes中开启自定义HPA,而在1.6中因为取消了原来的annotation方式设置custom metric,只能通过API serverkube-aggregator来获取custom metric,因为只有两种方式来设置了,一是直接通过API server获取heapstermetrics,二是部署 kube-aggragator 来实现。

我们将在Kubernetes1.8版本的Kubernetes中,使用聚合的API server来实现自定义指标的HPA

Kuberentes1.7+

确认您的Kubernetes版本在1.7或以上,修改以下配置:

  • kube-controller-manager的启动参数中 --horizontal-pod-autoscaler-use-rest-clients 设置为true,并指定 --masterAPI server地址,如 --master=http://172.20.0.113:8080
  • 修改kube-apiserver的配置文件apiserver,增加一条配置 --requestheader-client-ca-file=/etc/kubernetes/ssl/ca.pem --requestheader-allowed-names=aggregator --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --proxy-client-cert-file=/etc/kubernetes/ssl/kubernetes.pem --proxy-client-key-file=/etc/kubernetes/ssl/kubernetes-key.pem,用来配置aggregatorCA证书。

已经内置了apiregistration.k8s.io/v1beta1API,可以直接定义APIService,如:

apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  name: v1.custom-metrics.metrics.k8s.io
spec:
  insecureSkipTLSVerify: true
  group: custom-metrics.metrics.k8s.io
  groupPriorityMinimum: 1000
  versionPriority: 5
  service:
    name: api
    namespace: custom-metrics
  version: v1alpha1

部署Prometheus

使用 prometheus-operator.yaml 文件部署Prometheus operator

注意: 将镜像修改为你自己的镜像仓库地址。

这产生一个自定义的API,可以通过浏览器访问,还可以使用下面的命令可以检查该API

$ kubectl get --raw=apis/custom-metrics.metrics.k8s.io/v1alpha1
{"kind":"APIResourceList","apiVersion":"v1","groupVersion":"custom-metrics.metrics.k8s.io/v1alpha1","resources":[{"name":"jobs.batch/http_requests","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/http_requests","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"jobs.batch/up","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/up","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"services/scrape_samples_scraped","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/scrape_samples_scraped","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/scrape_duration_seconds","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"services/scrape_duration_seconds","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/http_requests","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/scrape_samples_post_metric_relabeling","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"jobs.batch/scrape_samples_scraped","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"jobs.batch/scrape_duration_seconds","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/scrape_duration_seconds","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/scrape_samples_post_metric_relabeling","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]},{"name":"services/scrape_samples_post_metric_relabeling","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"services/up","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"pods/scrape_samples_scraped","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"services/http_requests","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"jobs.batch/scrape_samples_post_metric_relabeling","singularName":"","namespaced":true,"kind":"MetricValueList","verbs":["get"]},{"name":"namespaces/up","singularName":"","namespaced":false,"kind":"MetricValueList","verbs":["get"]}]}

参考