一、背景
Ingress是k8s中实现7层负载的实现方式，是公开集群外部流量到集群内服务的HTTP和HTTPS路由
二、Ingress基础
通常Ingress实现由Ingress 控制器和Ingress组成，Ingress控制器负责具体实现反向代理及负载均衡，Ingress负责定义匹配规则和路由
Ingress-nginx控制器部署参见：
Installation Guide – Ingress-Nginx Controller
Ingress流程示意图：
三、示例
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: minimal-ingress annotations: nginx.ingress.kubernetes.io/rewrite-target: / spec: ingressClassName: nginx-example rules: - http: paths: - path: /testpath pathType: Prefix backend: service: name: test port: number: 80
参数解释说明：
1. ingressClassName： 指定Ingressclass名称，集群可以有多个class
2. rules：规则具体定义
3. paths：指定匹配URI
4. backend: 指定匹配到URI后转发到后端的service
5. annotations： 配合Ingress控制器来配置；不同的控制器，这里注解不一样
其他参数：
host：未指定host，则规则适用于通过指定IP地址的所有入站流量；反之，则只是适用于特定的host
backend: service加端口的组合；
defaultBackend：默认后端，通常在Ingress控制器配置，实现无法和rules匹配的其他路由处理
四、路径类型
pathType有如下选项：
1. Exact: 精确匹配URL路径，且区分大小写
2. Prefix：基于/分隔的URL路径前缀匹配，区分大小写
3.
ImplementationSpecific
：对于这种路径类型，匹配方法取决于 IngressClass
五、Ingress类
Ingress可以有不同的控制器，通常也使用不同的配置；每个Ingress应当改指定一个类，也就是对ingressClass资源的引用
apiVersion: networking.k8s.io/v1 kind: IngressClass metadata: name: external-lb spec: controller: example.com/ingress-controller parameters: apiGroup: k8s.example.com kind: IngressParameters name: external-lb
六、Ingress部署
注意：ingress-controller和kubernetes版本要保持匹配，我这里kubernetes是1.20，使用的ingress-controller版本是v1.3.0
6.1 部署ingress-controller：ingress-controller.yaml
apiVersion: v1
kind: Namespace
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
name: ingress-nginx
—
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
namespace: ingress-nginx
—
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
namespace: ingress-nginx
—
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
namespace: ingress-nginx
rules:
– apiGroups:
– “”
resources:
– namespaces
verbs:
– get
– apiGroups:
– “”
resources:
– configmaps
– pods
– secrets
– endpoints
verbs:
– get
– list
– watch
– apiGroups:
– “”
resources:
– services
verbs:
– get
– list
– watch
– apiGroups:
– networking.k8s.io
resources:
– ingresses
verbs:
– get
– list
– watch
– apiGroups:
– networking.k8s.io
resources:
– ingresses/status
verbs:
– update
– apiGroups:
– networking.k8s.io
resources:
– ingressclasses
verbs:
– get
– list
– watch
– apiGroups:
– “”
resourceNames:
– ingress-controller-leader
resources:
– configmaps
verbs:
– get
– update
– apiGroups:
– “”
resources:
– configmaps
verbs:
– create
– apiGroups:
– coordination.k8s.io
resourceNames:
– ingress-controller-leader
resources:
– leases
verbs:
– get
– update
– apiGroups:
– coordination.k8s.io
resources:
– leases
verbs:
– create
– apiGroups:
– “”
resources:
– events
verbs:
– create
– patch
—
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
namespace: ingress-nginx
rules:
– apiGroups:
– “”
resources:
– secrets
verbs:
– get
– create
—
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
rules:
– apiGroups:
– “”
resources:
– configmaps
– endpoints
– nodes
– pods
– secrets
– namespaces
verbs:
– list
– watch
– apiGroups:
– coordination.k8s.io
resources:
– leases
verbs:
– list
– watch
– apiGroups:
– “”
resources:
– nodes
verbs:
– get
– apiGroups:
– “”
resources:
– services
verbs:
– get
– list
– watch
– apiGroups:
– networking.k8s.io
resources:
– ingresses
verbs:
– get
– list
– watch
– apiGroups:
– “”
resources:
– events
verbs:
– create
– patch
– apiGroups:
– networking.k8s.io
resources:
– ingresses/status
verbs:
– update
– apiGroups:
– networking.k8s.io
resources:
– ingressclasses
verbs:
– get
– list
– watch
—
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
rules:
– apiGroups:
– admissionregistration.k8s.io
resources:
– validatingwebhookconfigurations
verbs:
– get
– update
—
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx
subjects:
– kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
—
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
namespace: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: ingress-nginx-admission
subjects:
– kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
—
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx
subjects:
– kind: ServiceAccount
name: ingress-nginx
namespace: ingress-nginx
—
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: ingress-nginx-admission
subjects:
– kind: ServiceAccount
name: ingress-nginx-admission
namespace: ingress-nginx
—
apiVersion: v1
data:
allow-snippet-annotations: “true”
kind: ConfigMap
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-controller
namespace: ingress-nginx
—
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
externalTrafficPolicy: Local
ipFamilies:
– IPv4
ipFamilyPolicy: SingleStack
ports:
– appProtocol: http
name: http
port: 80
protocol: TCP
targetPort: http
– appProtocol: https
name: https
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: NodePort
—
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-controller-admission
namespace: ingress-nginx
spec:
ports:
– appProtocol: https
name: https-webhook
port: 443
targetPort: webhook
selector:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
type: NodePort
—
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
minReadySeconds: 0
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
spec:
containers:
– args:
– /nginx-ingress-controller
– –publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
– –election-id=ingress-controller-leader
– –controller-class=k8s.io/ingress-nginx
– –ingress-class=nginx
– –configmap=$(POD_NAMESPACE)/ingress-nginx-controller
– –validating-webhook=:8443
– –validating-webhook-certificate=/usr/local/certificates/cert
– –validating-webhook-key=/usr/local/certificates/key
env:
– name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
– name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
– name: LD_PRELOAD
value: /usr/local/lib/libmimalloc.so
image: ingress-nginx-controller:v1.3.0
imagePullPolicy: IfNotPresent
lifecycle:
preStop:
exec:
command:
– /wait-shutdown
livenessProbe:
failureThreshold: 5
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: controller
ports:
– containerPort: 80
name: http
protocol: TCP
– containerPort: 443
name: https
protocol: TCP
– containerPort: 8443
name: webhook
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources:
requests:
cpu: 100m
memory: 90Mi
securityContext:
allowPrivilegeEscalation: true
capabilities:
add:
– NET_BIND_SERVICE
drop:
– ALL
runAsUser: 101
volumeMounts:
– mountPath: /usr/local/certificates/
name: webhook-cert
readOnly: true
dnsPolicy: ClusterFirst
nodeSelector:
kubernetes.io/os: linux
serviceAccountName: ingress-nginx
terminationGracePeriodSeconds: 300
volumes:
– name: webhook-cert
secret:
secretName: ingress-nginx-admission
—
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission-create
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission-create
spec:
containers:
– args:
– create
– –host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
– –namespace=$(POD_NAMESPACE)
– –secret-name=ingress-nginx-admission
env:
– name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: kube-webhook-certgen:v1.1.1
imagePullPolicy: IfNotPresent
name: create
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
—
apiVersion: batch/v1
kind: Job
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission-patch
namespace: ingress-nginx
spec:
template:
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission-patch
spec:
containers:
– args:
– patch
– –webhook-name=ingress-nginx-admission
– –namespace=$(POD_NAMESPACE)
– –patch-mutating=false
– –secret-name=ingress-nginx-admission
– –patch-failure-policy=Fail
env:
– name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
image: kube-webhook-certgen:v1.1.1
imagePullPolicy: IfNotPresent
name: patch
securityContext:
allowPrivilegeEscalation: false
nodeSelector:
kubernetes.io/os: linux
restartPolicy: OnFailure
securityContext:
fsGroup: 2000
runAsNonRoot: true
runAsUser: 2000
serviceAccountName: ingress-nginx-admission
—
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: nginx
spec:
controller: k8s.io/ingress-nginx
—
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
labels:
app.kubernetes.io/component: admission-webhook
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-admission
webhooks:
– admissionReviewVersions:
– v1
clientConfig:
service:
name: ingress-nginx-controller-admission
namespace: ingress-nginx
path: /networking/v1/ingresses
failurePolicy: Fail
matchPolicy: Equivalent
name: validate.nginx.ingress.kubernetes.io
rules:
– apiGroups:
– networking.k8s.io
apiVersions:
– v1
operations:
– CREATE
– UPDATE
resources:
– ingresses
sideEffects: None—
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-mytest
namespace: default #必须与service的名称空间一致
annotations:
kubernetes.io/ingress.class: “nginx”
spec:
rules:
– host: “mytest.k8s.com”
http:
paths:
– path: “/”
pathType: Exact
backend:
service:
name: mytest #service名称
port:
number: 80 #service端口
apiVersion: apps/v1
kind: Deployment
metadata:
name: mytest
namespace: default
spec:
replicas: 1
selector:
matchLabels:
app: mytest
release: v1
env: test
template:
metadata:
labels:
app: mytest
release: v1
env: test
spec:
containers:
– name: mytest
image: nginx
imagePullPolicy: IfNotPresent
ports:
– name: http
containerPort: 80
—
apiVersion: v1
kind: Service
metadata:
name: mytest
namespace: default
spec:
type: ClusterIP #默认类型
selector:
app: mytest
release: v1
env: test
ports:
– name: http
port: 80
targetPort: 80
6.2 ingressg规则：ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-app
namespace: default #必须与service的名称空间一致
annotations:
kubernetes.io/ingress.class: “nginx”
spec:
rules:
– host: “app.k8s.com”
http:
paths:
– path: “/”
pathType: Exact
backend:
service:
name: ng-app #service名称
port:
number: 80 #service端口
6.3 部署应用
apiVersion: apps/v1
kind: Deployment
metadata:
name: ng-test
namespace: default
spec:
replicas: 1
selector:
matchLabels:
env: test
template:
metadata:
labels:
env: test
spec:
containers:
– name: ng-test
image: nginx
imagePullPolicy: IfNotPresent
ports:
– name: http
containerPort: 80
—
apiVersion: v1
kind: Service
metadata:
name: ng-app
namespace: default
spec:
type: ClusterIP #默认类型
selector:
env: test
ports:
– name: http
port: 80
targetPort: 80
七、测试
注意事项：
1. ingress中定义的host，需要编辑/etc/hosts进行解析，端口为ingress-controller的service对应的80的NodePort端口
[root@zhoupei-v6 ingress]# curl -L http://app.k8s.com:25767
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p><p>For online documentation and support please refer to
<a href=”http://nginx.org/”>nginx.org</a>.<br/>
Commercial support is available at
<a href=”http://nginx.com/”>nginx.com</a>.</p><p><em>Thank you for using nginx.</em></p>
</body>
</html>