1什么是network filter
network filter是envoy的网络过滤器,作用在L3/L4。其中最重要的算是tcp_proxy和http_connection_manager这两个,分别表示tcp连接和http连接。本小节介绍的网络过滤器配置都比较简单,所以放在一章来介绍。
2本节介绍的filter列表
client_ssl_auth
connection_limit
envoy_mobile_http_connection_manager
direct_response
echo
sni_cluster
sni_dynamic_forward_proxy
postgres_proxy
mysql_proxy
mongo_proxy
zookeeper_proxy
3配置
3.1client_ssl_auth
{"auth_api_cluster": "...",认证服务cluster"stat_prefix": "...",stat前缀"refresh_delay": "{...}",刷新抖动"ip_white_list": []ip白名单}
3.2connection_limit
{"stat_prefix": "...",stat前缀"max_connections": "{...}",最大并发连接数"delay": "{...}",拒绝连接延迟"runtime_enabled": "{...}"是否启用,默认启用}
3.3envoy_mobile_http_connection_manager
{"config": "{...}"http_connection_manager 配置}
3.4direct_response
{"response": "{...}"响应内容}
response:
{"filename": "...",来自文件内容"inline_bytes": "...",bytes内容"inline_string": "...",string内容"environment_variable": "..."环境变量内容}
3.5echo
无
3.6sni_cluster
无
3.7sni_dynamic_forward_proxy
{"dns_cache_config": "{...}",dns缓存配置"port_value": "..."连接upstream的端口}
dns_cache_config:
{"name": "...",缓存的名字"dns_lookup_family": "...",ip类型"dns_refresh_rate": "{...}",dns刷新频率,默认60s"host_ttl": "{...}",host没有使用的生存时间,默认5m"max_hosts": "{...}",缓存存放的最大host数量,默认1024"dns_failure_refresh_rate": "{...}",dsn请求失败的刷新频率,默认dns_refresh_rate一样"dns_cache_circuit_breaker": "{...}",缓存断路器"use_tcp_for_dns_lookups": "...",使用tcp查询"dns_resolution_config": "{...}",dns查询服务器配置,废弃"typed_dns_resolver_config": "{...}",dns查询服务器配置"preresolve_hostnames": [],预先定义的host"dns_query_timeout": "{...}"查询超时时间}
dns_lookup_family:
•AUTO(DEFAULT) •V4_ONLY•V6_ONLY•V4_PREFERRED
dns_cache_circuit_breaker:
{"max_pending_requests": "{...}"最大等待请求数量}
typed_dns_resolver_config :
envoy.network.dns_resolver.appleenvoy.network.dns_resolver.cares
envoy.network.dns_resolver.cares:
{"resolvers": [],服务器地址"dns_resolver_options": "{...}"选项}
dns_resolver_options:
{"use_tcp_for_dns_lookups": "...",使用tcp查询"no_default_search_domain": "..."是否有默认查询domain}
preresolve_hostnames:
{"protocol": "...",协议"address": "...",地址"port_value": "...",端口"named_port": "...",命名端口"resolver_name": "...",解析器名字"ipv4_compat": "..."兼容ipv4}
3.8postgres_proxy
{"stat_prefix": "...",stat前缀"enable_sql_parsing": "{...}",是否启用sql解析,默认true"terminate_ssl": "..."是否终止ssl}
3.9mysql_proxy
{"stat_prefix": "..."stat前缀}
3.10mongo_proxy
{"stat_prefix": "...",stat前缀"access_log": "...",访问日志路径"delay": "{...}",注入延迟"emit_dynamic_metadata": "...",是否产生动态元数据,默认false"commands": []产生metrics的命令}
delay:
{"fixed_delay": "{...}",固定延迟"header_delay": "{...}",由头控制的延迟"percentage": "{...}"延迟比列}
3.11zookeeper_proxy
{"stat_prefix": "...",stat前缀"max_packet_bytes": "{...}"消息最大大小,默认1Mb,istio未实现}
4实战
4.1client_ssl_auth
先部署tcp-echo
simple/tcp-echo-services.yaml
kubectl apply -f tcp-echo-services.yaml -n istio
apiVersion: v1kind: Servicemetadata:name: tcp-echolabels:app: tcp-echoservice: tcp-echospec:ports:- name: tcpport: 9000- name: tcp-otherport: 9001# Port 9002 is omitted intentionally for testing the pass through filter chain.selector:app: tcp-echo---apiVersion: apps/v1kind: Deploymentmetadata:name: tcp-echo-v1labels:app: tcp-echoversion: v1spec:replicas: 1selector:matchLabels:app: tcp-echoversion: v1template:metadata:labels:app: tcp-echoversion: v1spec:containers:- name: tcp-echoimage: docker.io/istio/tcp-echo-server:1.2imagePullPolicy: IfNotPresentargs: [ "9000,9001,9002", "one" ]ports:- containerPort: 9000- containerPort: 9001---apiVersion: apps/v1kind: Deploymentmetadata:name: tcp-echo-v2labels:app: tcp-echoversion: v2spec:replicas: 1selector:matchLabels:app: tcp-echoversion: v2template:metadata:labels:app: tcp-echoversion: v2spec:containers:- name: tcp-echoimage: docker.io/istio/tcp-echo-server:1.2imagePullPolicy: IfNotPresentargs: [ "9000,9001,9002", "two" ]ports:- containerPort: 9000- containerPort: 9001
tcp-echo-all-v1.yaml
kubectl apply -f tcp-echo-all-v1.yaml -n istio
apiVersion: networking.istio.io/v1alpha3kind: Gatewaymetadata:name: tcp-echo-gatewayspec:selector:istio: ingressgatewayservers:- port:number: 31400name: tcpprotocol: TCPhosts:- "*"---apiVersion: networking.istio.io/v1alpha3kind: DestinationRulemetadata:name: tcp-echo-destinationspec:host: tcp-echosubsets:- name: v1labels:version: v1- name: v2labels:version: v2---apiVersion: networking.istio.io/v1alpha3kind: VirtualServicemetadata:name: tcp-echospec:hosts:- "*"gateways:- tcp-echo-gatewaytcp:- match:- port: 31400route:- destination:host: tcp-echoport:number: 9000subset: v1
This functionality can be used to build edge proxy VPN support for web infrastructure.
ef-client_ssl_auth.yaml
kubectl apply -f ef-client_ssl_auth.yaml -n istio-system
apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:name: simplenamespace: istio-systemspec:workloadSelector:labels:istio: ingressgatewayconfigPatches:- applyTo: NETWORK_FILTERmatch:context: GATEWAYlistener:portNumber: 31400filterChain:filter:name: "envoy.filters.network.tcp_proxy"patch:operation: INSERT_BEFOREvalue:name: envoy.filters.network.client_ssl_authtyped_config:'@type': type.googleapis.com/envoy.extensions.filters.network.client_ssl_auth.v3.ClientSSLAuthauth_api_cluster: auth_api_clusterstat_prefix: testrefresh_delay: 60000msip_white_list:- address_prefix: 0.0.0.0prefix_len: 0- applyTo: CLUSTERpatch:operation: ADDvalue:name: auth_api_clusterconnect_timeout: 0.25stype: STRICT_DNSlb_policy: ROUND_ROBINload_assignment:cluster_name: auth_api_clusterendpoints:- lb_endpoints:- endpoint:address:socket_address:address: authport_value: 80
由于我们没有auth_api_cluster实现,所以白名单以外的情况暂时不能验证。
4.2connection_limit
ef-connection_limit.yaml
kubectl apply -f ef-connection_limit.yaml -n istio-system
apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:name: simplenamespace: istio-systemspec:workloadSelector:labels:istio: ingressgatewayconfigPatches:- applyTo: NETWORK_FILTERmatch:context: GATEWAYlistener:portNumber: 31400filterChain:filter:name: "envoy.filters.network.tcp_proxy"patch:operation: INSERT_BEFOREvalue:name: envoy.filters.network.connection_limittyped_config:'@type': type.googleapis.com/envoy.extensions.filters.network.connection_limit.v3.ConnectionLimitstat_prefix: connectionLimitmax_connections: 1delay: 1sruntime_enabled:default_value: trueruntime_key: connectionlimit.enabled
开启两个telnet,第二个1秒后就关闭了
4.3envoy_mobile_http_connection_manager
配置方式和http_connection_manager 差不多,请参考http_connection_manager
4.4direct_response
ef-direct_response.yaml
kubectl apply -f ef-direct_response.yaml -n istio-system
apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:name: simplenamespace: istio-systemspec:workloadSelector:labels:istio: ingressgatewayconfigPatches:- applyTo: NETWORK_FILTERmatch:context: GATEWAYlistener:portNumber: 31400filterChain:filter:name: "envoy.filters.network.tcp_proxy"patch:operation: REPLACEvalue:name: envoy.filters.network.direct_responsetyped_config:'@type': type.googleapis.com/envoy.extensions.filters.network.direct_response.v3.Configresponse:inline_string: "this is a test"
telnet 192.168.229.134 32691
4.5echo
ef-echo.yaml
kubectl apply -f ef-echo.yaml -n istio-system
apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:name: simplenamespace: istio-systemspec:workloadSelector:labels:istio: ingressgatewayconfigPatches:- applyTo: NETWORK_FILTERmatch:context: GATEWAYlistener:portNumber: 31400filterChain:filter:name: "envoy.filters.network.tcp_proxy"patch:operation: REPLACEvalue:name: envoy.filters.network.echotyped_config:'@type': type.googleapis.com/envoy.extensions.filters.network.echo.v3.Echo
4.6sni_cluster
Set the upstream cluster name from the SNI field in the TLS connection
ef-sni_cluster.yaml
kubectl apply -f ef-sni_cluster.yaml -n istio-system
apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:name: simplenamespace: istio-systemspec:workloadSelector:labels:istio: ingressgatewayconfigPatches:- applyTo: NETWORK_FILTERmatch:context: GATEWAYlistener:portNumber: 8443filterChain:filter:name: "envoy.filters.network.http_connection_manager"patch:operation: INSERT_BEFOREvalue:name: envoy.filters.network.sni_clustertyped_config:'@type': type.googleapis.com/envoy.extensions.filters.network.sni_cluster.v3.SniCluster
"sni": "outbound_.443_._.istio-ingressgateway.istio-system.svc.cluster.local"
上游cluster name设置成这个
4.7sni_dynamic_forward_proxy
ef-sni_dynamic_forward_proxy.yaml
kubectl apply -f ef-sni_dynamic_forward_proxy.yaml -n istio-system
apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:name: simplenamespace: istio-systemspec:workloadSelector:labels:istio: ingressgatewayconfigPatches:- applyTo: NETWORK_FILTERmatch:context: GATEWAYlistener:portNumber: 8443filterChain:filter:name: "envoy.filters.network.http_connection_manager"patch:operation: INSERT_BEFOREvalue:name: envoy.filters.network.sni_dynamic_forward_proxytyped_config:"@type": type.googleapis.com/envoy.extensions.filters.network.sni_dynamic_forward_proxy.v3alpha.FilterConfigdns_cache_config:name: dynamic_forward_proxy_cache_configdns_lookup_family: V4_ONLYdns_resolution_config:resolvers:- socket_address:address: "8.8.8.8"port_value: 53dns_resolver_options:use_tcp_for_dns_lookups: trueno_default_search_domain: true- applyTo: CLUSTERpatch:operation: ADDvalue:name: dynamic_forward_proxy_clusterlb_policy: CLUSTER_PROVIDEDcluster_type:name: envoy.clusters.dynamic_forward_proxytyped_config:"@type": type.googleapis.com/envoy.extensions.clusters.dynamic_forward_proxy.v3.ClusterConfigdns_cache_config:name: dynamic_forward_proxy_cache_configdns_lookup_family: V4_ONLYdns_resolution_config:resolvers:- socket_address:address: "8.8.8.8"port_value: 53dns_resolver_options:use_tcp_for_dns_lookups: trueno_default_search_domain: true
必须有ype.googleapis.com/envoy.extensions.filters.listener.tls_inspector.v3.TlsInspector这个listenerfilter
sni_dynamic_forward_proxy 还在研究,不知是怎么配置目的domain,没发现配置点
4.8postgres_proxy
部署postgres
postgres-deploy.yaml
kubectl apply -f postgres-deploy.yaml -n istio
apiVersion: v1kind: ConfigMapmetadata:name: postgres-configlabels:app: postgresdata:POSTGRES_DB: masterPOSTGRES_USER: postgresPOSTGRES_PASSWORD: postgres---apiVersion: apps/v1kind: StatefulSetmetadata:name: postgresspec:serviceName: "postgres"replicas: 1selector:matchLabels:app: postgrestemplate:metadata:labels:app: postgresspec:containers:- name: postgresimage: postgres:9.5envFrom:- configMapRef:name: postgres-configports:- containerPort: 5432name: postgredbvolumeMounts:# - name: postgres-data# mountPath: var/lib/postgresql/data# subPath: postgres# volumeClaimTemplates:# - metadata:# name: postgres-data# spec:# accessModes: ["ReadWriteOnce"]# resources:# requests:# storage: 2Gi---apiVersion: v1kind: Servicemetadata:name: postgreslabels:app: postgresspec:ports:- port: 5432protocol: TCPtargetPort: 5432name: postgresselector:app: postgres
ef-postgres_proxy.yaml
kubectl apply -f ef-postgres_proxy.yaml -n istio
apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:name: simplenamespace: istio-systemspec:workloadSelector:labels:istio: ingressgatewayconfigPatches:- applyTo: NETWORK_FILTERmatch:context: GATEWAYlistener:portNumber: 31400filterChain:filter:name: "envoy.filters.network.tcp_proxy"patch:operation: INSERT_BEFOREvalue:name: envoy.filters.network.postgres_proxytyped_config:'@type': type.googleapis.com/envoy.extensions.filters.network.postgres_proxy.v3alpha.PostgresProxystat_prefix: postgreenable_sql_parsing: trueterminate_ssl: false
vs-gw-postgre.yaml
kubectl apply -f vs-gw-postgre.yaml -n istio
apiVersion: networking.istio.io/v1alpha3kind: Gatewaymetadata:name: tcp-gatewayspec:selector:istio: ingressgatewayservers:- port:number: 31400name: tcpprotocol: TCPhosts:- "*"---apiVersion: networking.istio.io/v1alpha3kind: VirtualServicemetadata:name: postgrespec:hosts:- "*"gateways:- tcp-gatewaytcp:- match:- port: 31400route:- destination:host: postgresport:number: 5432
4.9mysql_proxy
部署mysql
mysql-deploy.yaml
kubectl apply -f mysql-deploy.yaml -n istio
apiVersion: v1kind: Secretmetadata:name: mysql-credentialstype: Opaquedata:rootpasswd: cGFzc3dvcmQ=---apiVersion: v1kind: Servicemetadata:name: mysqldblabels:app: mysqldbservice: mysqldbspec:ports:- port: 3306name: tcpselector:app: mysqldb---apiVersion: apps/v1kind: Deploymentmetadata:name: mysqldb-v1labels:app: mysqldbversion: v1spec:replicas: 1selector:matchLabels:app: mysqldbversion: v1template:metadata:labels:app: mysqldbversion: v1spec:containers:- name: mysqldbimage: docker.io/istio/examples-bookinfo-mysqldb:1.16.2imagePullPolicy: IfNotPresentports:- containerPort: 3306env:- name: MYSQL_ROOT_PASSWORDvalueFrom:secretKeyRef:name: mysql-credentialskey: rootpasswdargs: ["--default-authentication-plugin","mysql_native_password"]volumeMounts:- name: var-lib-mysqlmountPath: /var/lib/mysqlvolumes:- name: var-lib-mysqlemptyDir: {}
vs-gw-mysql.yaml
kubectl apply -f vs-gw-mysql.yaml -n istio
apiVersion: networking.istio.io/v1alpha3kind: Gatewaymetadata:name: tcp-gatewayspec:selector:istio: ingressgatewayservers:- port:number: 31400name: tcpprotocol: TCPhosts:- "*"---apiVersion: networking.istio.io/v1alpha3kind: VirtualServicemetadata:name: mysqldbspec:hosts:- "*"gateways:- tcp-gatewaytcp:- match:- port: 31400route:- destination:host: mysqldbport:number: 3306
ef-mysql_proxy.yaml
kubectl apply -f ef-mysql_proxy.yaml -n istio
apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:name: simplenamespace: istio-systemspec:workloadSelector:labels:istio: ingressgatewayconfigPatches:- applyTo: NETWORK_FILTERmatch:context: GATEWAYlistener:portNumber: 31400filterChain:filter:name: "envoy.filters.network.tcp_proxy"patch:operation: INSERT_BEFOREvalue:name: envoy.filters.network.mysql_proxytyped_config:'@type': type.googleapis.com/envoy.extensions.filters.network.mysql_proxy.v3.MySQLProxystat_prefix: mysql
4.10mongo_proxy
部署mongodb
mongodb-deploy.yaml
kubectl apply -f mongodb-deploy.yaml -n istio
apiVersion: v1kind: Servicemetadata:name: mongodblabels:app: mongodbservice: mongodbspec:ports:- port: 27017name: mongoselector:app: mongodb---apiVersion: apps/v1kind: Deploymentmetadata:name: mongodb-v1labels:app: mongodbversion: v1spec:replicas: 1selector:matchLabels:app: mongodbversion: v1template:metadata:labels:app: mongodbversion: v1spec:containers:- name: mongodbimage: docker.io/istio/examples-bookinfo-mongodb:1.16.2imagePullPolicy: IfNotPresentports:- containerPort: 27017volumeMounts:- name: data-dbmountPath: /data/dbvolumes:- name: data-dbemptyDir: {}
vs-gw-mongodb.yaml
kubectl apply -f vs-gw-mongodb.yaml -n istio
apiVersion: networking.istio.io/v1alpha3kind: Gatewaymetadata:name: tcp-gatewayspec:selector:istio: ingressgatewayservers:- port:number: 31400name: tcpprotocol: TCPhosts:- "*"---apiVersion: networking.istio.io/v1alpha3kind: VirtualServicemetadata:name: mongodbspec:hosts:- "*"gateways:- tcp-gatewaytcp:- match:- port: 31400route:- destination:host: mongodbport:number: 27017
ef-mongo_proxy.yaml
kubectl apply -f ef-mongo_proxy.yaml -n istio
apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:name: simplenamespace: istio-systemspec:workloadSelector:labels:istio: ingressgatewayconfigPatches:- applyTo: NETWORK_FILTERmatch:context: GATEWAYlistener:portNumber: 31400filterChain:filter:name: "envoy.filters.network.tcp_proxy"patch:operation: INSERT_BEFOREvalue:name: envoy.filters.network.mongo_proxytyped_config:'@type': type.googleapis.com/envoy.extensions.filters.network.mongo_proxy.v3.MongoProxystat_prefix: mongodbemit_dynamic_metadata: truedelay:fixed_delay: 1spercentage:numerator: 20denominator: HUNDREDcommands:- delete- insert- updateaccess_log: /dev/stdout
4.11zookeeper_proxy
部署zookeeper
zookeeper-deploy.yaml
kubectl apply -f zookeeper-deploy.yaml -n istio
apiVersion: v1kind: Servicemetadata:name: zookeeperlabels:app: zookeeperservice: zookeeperspec:ports:- port: 2181name: zookeeperselector:app: zookeeper---apiVersion: apps/v1kind: Deploymentmetadata:name: zookeeperlabels:app: zookeeperversion: v1spec:replicas: 1selector:matchLabels:app: zookeeperversion: v1template:metadata:labels:app: zookeeperversion: v1spec:containers:- name: zookeeperimage: docker.io/zookeeper:3.7.0imagePullPolicy: IfNotPresentports:- containerPort: 2181
vs-gw-zookeeper.yaml
kubectl apply -f vs-gw-zookeeper.yaml -n istio
apiVersion: networking.istio.io/v1alpha3kind: Gatewaymetadata:name: tcp-gatewayspec:selector:istio: ingressgatewayservers:- port:number: 31400name: tcpprotocol: TCPhosts:- "*"---apiVersion: networking.istio.io/v1alpha3kind: VirtualServicemetadata:name: zookeeperspec:hosts:- "*"gateways:- tcp-gatewaytcp:- match:- port: 31400route:- destination:host: zookeeperport:number: 2181
ef-zookeeper.yaml
kubectl apply -f ef-zookeeper.yaml -n istio-system
apiVersion: networking.istio.io/v1alpha3kind: EnvoyFiltermetadata:name: simplenamespace: istio-systemspec:workloadSelector:labels:istio: ingressgatewayconfigPatches:- applyTo: NETWORK_FILTERmatch:context: GATEWAYlistener:portNumber: 31400filterChain:filter:name: "envoy.filters.network.tcp_proxy"patch:operation: INSERT_BEFOREvalue:name: envoy.filters.network.mongo_proxytyped_config:'@type': type.googleapis.com/envoy.extensions.filters.network.mongo_proxy.v3.MongoProxystat_prefix: zookeeper#max_packet_bytes: 10241024




