一、Calico简介
1.1、calico特性
Calico是另一款主流的开源虚拟化网络方案， 用于为云原生应用实现互联与策略控制， 可以整合进大多数主流的编排系统， 例如Kubernetes、 Apache Mesos、 Docker和OpenStack等。 与Flannel相比，Calico的一个显著优势是对网络策略的支持， 它允许用户动态定义访问控制规则以管控进出容器的数据报文， 从而为Pod间通信按需施加安全策略。
Calico是一个三层的虚拟网络解决方案， 它把每个节点都当作虚拟路由器（vRouter） ， 并把每个节点上的Pod都当作是“节点路由器”后的一个终端设备并为其分配一个IP地址。 各节点路由器通过BGP协议学习生成路由规则， 从而实现不同节点上Pod间的互联互通。
1.2、calico通信机制
BGP是互联网上一个核心的去中心化自治路由协议， 它通过维护IP路由表或“前缀”表来实现自治系统（AS） 之间的可达性， 通常作为大规模数据中心维护不同的自治系统之间路由信息的矢量路由协议。 Linux内核原生支持BGP， 因而我们可轻易把一台Linux主机配置成为边界网关。
Calico把Kubernetes集群环境中的每个节点上的Pod所组成的网络视为一个自治系统， 而每个节点也就自然由各自的Pod对象组成虚拟网络， 进而形成自治系统的边界网关。 各节点间通过BGP协议交换路由信息并生成路由规则。 但考虑到并非所有网络都能支持BGP， 而且BGP路由模型要求所有节点必须要位于同一个二层网络， 所以Calico还支持基于IPIP和VXLAN的Overlay网络模型， 它们的工作模式与Flannel的VXLAN和IPIP模型并无显著不同。
类似Flannel在VXLAN后端启用DirectRouting时的网络模型， Calico也支持混合使用路由和Overlay网络模型， BGP路由模型用于二层网络的高性能通信， IP-IP或VXLAN用于跨子网的节点间报文转发。 IP-IP协议包头非常小， 理论上它的速度要比VXLAN稍快一点，但安全性更差。需要注意的是， Calico网络提供的在线部署清单中默认使用的是IPIP隧道网络， 而非BGP或者混合模型， 因为它假设节点的底层网络不支持BGP协议。
1.3、calico架构
Calico的系统组件主要有Felix、 BGP路由反射器、 编排系统插件、BIRD和etcd存储系统等。
BGP模式下的Calico所承载的各Pod资源直接基于vRouter经由基础网络进行互联， 它非叠加、 无隧道、 不使用VRF表， 也不依赖于NAT， 因此每个工作负载都可以直接配置使用公网IP接入互联网， 当然， 也可以按需使用网络策略控制它的网络连通性。
（1） Felix
Felix是运行于各节点上守护进程， 它主要负责完成接口管理、 路由规划、 ACL规划和状态报告几个核心任务， 从而为各端点（VM或Container） 生成连接机制。
- 接口管理， 负责创建网络接口、 生成必要信息并送往内核， 以确保内核能正确处理各端点的流量， 尤其是要确保目标节点MAC能响应当前节点上各工作负载的MAC地址的ARP请求， 以及为Felix管理的接口打开转发功能。 另外， 接口管理还要监控各接口的变动以确保规则能得到正确应用。
- 路由规划， 负责为当前节点上运行的各端点在内核FIB（ Forwarding Information Base） 中生成路由信息， 以保证到达当前节点的报文可正确转发给端点。
- ACL规划， 负责在Linux内核中生成ACL， 实现仅放行端点间的合规流量， 并确保流量不能绕过Calico等安全措施。
- 状态报告， 负责提供网络健康状态的相关数据， 尤其是报告由Felix管理的节点上的错误和问题。 这些报告数据会存储在etcd， 供其他组件或网络管理员使用。
（ 2） 编排系统插件
编排系统插件的主要功能是将Calico整合进所在的编排系统中， 例如Kubernetes或OpenStack等。 它主要负责完成API转换， 从而让管理员和用户能够无差别地使用Calico的网络功能。 换句话说， 编排系统通常有自己的网络管理API， 相应的插件要负责将对这些API的调用转换为Calico的数据模型， 并存储到Calico的存储系统中。 因而， 编排插件的具体实现依赖于底层编排系统， 不同的编排系统有各自专用的插件。
（3）etcd存储系统利用etcd， Calico网络可实现为有明确状态（ 正常或故障） 的系统， 且易于通过扩展应对访问压力的提升， 避免自身成为系统瓶颈。 另外， etcd也是Calico各组件的通信总线。
（4）BGP客户端Calico要求在每个运行着Felix的节点上同时运行一个称为BIRD的守护进程， 它是BGP协议的客户端， 负责将Felix生成的路由信息载入内核并通告给整个网络中。
（5）BGP路由反射器
Calico在每一个计算节点利用Linux内核实现了一个高效的vRouter（虚拟路由器） 进行报文转发。 每个vRouter通过BGP协议将自身所属节点运行的Pod资源的IP地址信息， 基于节点上的专用代理程序（Felix） 生成路由规则向整个Calico网络内传播。 尽管小规模部署能够直接使用BGP网格模型， 但随着节点数量（假设为N） 的增加， 这些连接的数量就会以N2的规模快速增长， 从而给集群网络带来巨大的压力。因此， 一般建议大规模的节点网络使用BGP路由反射器进行路由学习，BGP的点到点通信也就转为与中心点的单路通信模型。 另外， 出于冗余考虑， 生产实践中应该部署多个BGP路由反射器。 而对于Calico来说，BGP客户端程序除了作为客户端使用外， 也可以配置为路由反射器。
另外， Calico可将关键配置抽象成资源类型， 并允许用户按需定义资源对象以完成系统配置， 这些资源对象保存在Datastore中， Datastore可以是独立管理的etcd存储系统， 也可以是Kubernetes API封装的集群状态存储系统（即Kubernetes使用的etcd存储系统） 。 Calico专有的资源类型有十几种， 包括IPPool（IP地址池） 、 NetworkPolicy（网络策略） 、BGPConfiguration（BGP配置参数） 和FelixConfiguration（Felix配置参数） 等。 类似于Kubernetes API资源的定义， 这些资源的配置格式同样以JSON使用apiVersion、 kind、 metadata和spec等一级字段进行定义， 并能够使用calicoctl客户端工具进行管理， 也支持由kublet借助CRD进行这类资源的管理。
二、k8s环境部署calico组件
#环境准备 192.168.247.91 k8s-calico-master-01 kubeapi.calico.com 2vcpu 4G 50G ubuntu20.04 192.168.247.94 k8s-calico-node-01 2vcpu 4G 50G ubuntu20.04 192.168.247.95 k8s-calico-node-02 2vcpu 4G 50G ubuntu20.04 #关闭swap swapoff -a && sed -i 's@/swap.img@#/swap.img@g' /etc/fstab #关闭防火墙 ufw disable && ufw status #配置时间同步 apt-get update && apt install chrony -y cat > /etc/chrony/chrony.conf <<EOF server ntp.aliyun.com iburst stratumweight 0 driftfile /var/lib/chrony/drift rtcsync makestep 10 3 bindcmdaddress 127.0.0.1 bindcmdaddress ::1 keyfile /etc/chrony.keys commandkey 1 generatecommandkey logchange 0.5 logdir /var/log/chrony EOF systemctl restart chrony #安装并启用docker apt -y install apt-transport-https ca-certificates curl software-properties-common curl -fsSL http://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | apt-key add - add-apt-repository "deb [arch=amd64] http://mirrors.aliyun.com/docker-ce/linux/ubuntu $(lsb_release -cs) stable" apt update && apt install docker-ce -y #配置docker加速 cat > /etc/docker/daemon.json <<EOF { "registry-mirrors": [ "https://docker.mirrors.ustc.edu.cn", "https://hub-mirror.c.163.com", "https://reg-mirror.qiniu.com", "https://registry.docker-cn.com", "https://a7h8080e.mirror.aliyuncs.com" ], "exec-opts": ["native.cgroupdriver=systemd"], "log-driver": "json-file", "log-opts": { "max-size": "200m" }, "storage-driver": "overlay2" } EOF systemctl restart docker docker info #查看docker版本及加速是否生效 #安装cri-dockerd curl -LO https://github.com/Mirantis/cri-dockerd/releases/download/v0.2.6/cri-dockerd_0.2.6.3-0.ubuntu-focal_amd64.deb apt install ./cri-dockerd_0.2.6.3-0.ubuntu-focal_amd64.deb -y #安装kubelet、kubeadm、kubectl apt update && apt install -y apt-transport-https curl curl -fsSL https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -cat <<EOF >/etc/apt/sources.list.d/kubernetes.list deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main EOF apt-get update && apt install -y kubelet kubeadm kubectl systemctl enable kubelet #整合kubelet和cri-dockerd # vim /usr/lib/systemd/system/cri-docker.service # cat /usr/lib/systemd/system/cri-docker.service [Unit] Description=CRI Interface for Docker Application Container Engine Documentation=https://docs.mirantis.com After=network-online.target firewalld.service docker.service Wants=network-online.target Requires=cri-docker.socket [Service] Type=notify #ExecStart=/usr/bin/cri-dockerd --container-runtime-endpoint fd:// ExecStart=/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --network-plugin=cni --cni-bin-dir=/opt/cni/bin --cni-cache-dir=/var/lib/cni/cache --cni-conf-dir=/etc/cni/net.d ExecReload=/bin/kill -s HUP $MAINPID TimeoutSec=0 RestartSec=2 Restart=always StartLimitBurst=3 StartLimitInterval=60s LimitNOFILE=infinity LimitNPROC=infinity LimitCORE=infinity TasksMax=infinity Delegate=yes KillMode=process [Install] WantedBy=multi-user.target systemctl daemon-reload && systemctl restart cri-docker #配置kubelet mkdir /etc/sysconfig/ cat > /etc/sysconfig/kubelet <<EOF KUBELET_KUBEADM_ARGS="--container-runtime=remote --container-runtime-endpoint=/run/cri-dockerd.sock" EOF
#初始化master # kubeadm config images list #查看所需的容器镜像 registry.k8s.io/kube-apiserver:v1.25.3 registry.k8s.io/kube-controller-manager:v1.25.3 registry.k8s.io/kube-scheduler:v1.25.3 registry.k8s.io/kube-proxy:v1.25.3 registry.k8s.io/pause:3.8 registry.k8s.io/etcd:3.5.4-0 registry.k8s.io/coredns/coredns:v1.9.3#拉取镜像 # kubeadm config images pull --cri-socket unix:///run/cri-dockerd.sock #拉取镜像 #初始化集群 kubeadm init --control-plane-endpoint="kubeapi.calico.com" \ --kubernetes-version=v1.25.3 \ --pod-network-cidr=10.200.0.0/16 \ --service-cidr=10.100.0.0/12 \ --token-ttl=0 \ --cri-socket unix:///run/cri-dockerd.sock \ --upload-certs #master添加 kubeadm join kubeapi.calico.com:6443 --token z4mff1.o81fx972z3itwsja \ --discovery-token-ca-cert-hash sha256:75bf3bbd22ceee393014359131b7238074671d1d259704610c7691cf7edf96b3 \ --control-plane --certificate-key ca96cb8c0a0d4e33f8561155419dec8c42d7c83c361b3f6dbc0848c2de203d78 \ --cri-socket unix:///run/cri-dockerd.sock #node添加 kubeadm join kubeapi.calico.com:6443 --token z4mff1.o81fx972z3itwsja \ --discovery-token-ca-cert-hash sha256:75bf3bbd22ceee393014359131b7238074671d1d259704610c7691cf7edf96b3 \ --cri-socket unix:///run/cri-dockerd.sock [root@k8s-calico-master-01 ~]# mkdir -p $HOME/.kube [root@k8s-calico-master-01 ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config [root@k8s-calico-master-01 ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config [root@k8s-calico-master-01 ~]# kubectl get node -owide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME k8s-calico-master-01 NotReady control-plane 2m41s v1.25.3 192.168.247.91 <none> Ubuntu 20.04.3 LTS 5.4.0-122-generic docker://20.10.21 k8s-calico-node-01 NotReady <none> 50s v1.25.3 192.168.247.94 <none> Ubuntu 20.04.3 LTS 5.4.0-122-generic docker://20.10.21 k8s-calico-node-02 NotReady <none> 46s v1.25.3 192.168.247.95 <none> Ubuntu 20.04.3 LTS 5.4.0-122-generic docker://20.10.21 [root@k8s-calico-master-01 ~]# kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-565d847f94-nd5fm 0/1 Pending 0 2m29s kube-system coredns-565d847f94-tnqxt 0/1 Pending 0 2m29s kube-system etcd-k8s-calico-master-01 1/1 Running 0 2m41s kube-system kube-apiserver-k8s-calico-master-01 1/1 Running 0 2m41s kube-system kube-controller-manager-k8s-calico-master-01 1/1 Running 0 2m42s kube-system kube-proxy-p65xq 1/1 Running 0 55s kube-system kube-proxy-rjkz7 1/1 Running 0 51s kube-system kube-proxy-zlwlt 1/1 Running 0 2m29s kube-system kube-scheduler-k8s-calico-master-01 1/1 Running 0 2m41s [root@k8s-calico-master-01 ~]# #下载calico网络组件安装包 wget https://github.com/projectcalico/calico/archive/refs/tags/v3.24.4.zip
unzip v3.24.4.zip cd calico-3.24.4/manifests [root@k8s-calico-master-01 manifests]# kubectl apply -f calico.yaml poddisruptionbudget.policy/calico-kube-controllers created serviceaccount/calico-kube-controllers created serviceaccount/calico-node created configmap/calico-config created customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created clusterrole.rbac.authorization.k8s.io/calico-kube-controllers created clusterrole.rbac.authorization.k8s.io/calico-node created clusterrolebinding.rbac.authorization.k8s.io/calico-kube-controllers created clusterrolebinding.rbac.authorization.k8s.io/calico-node created daemonset.apps/calico-node created deployment.apps/calico-kube-controllers created [root@k8s-calico-master-01 manifests]# [root@k8s-calico-master-01 manifests]# kubectl get node -owide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME k8s-calico-master-01 Ready control-plane 23m v1.25.3 192.168.247.91 <none> Ubuntu 20.04.3 LTS 5.4.0-122-generic docker://20.10.21 k8s-calico-node-01 Ready <none> 22m v1.25.3 192.168.247.94 <none> Ubuntu 20.04.3 LTS 5.4.0-122-generic docker://20.10.21 k8s-calico-node-02 Ready <none> 22m v1.25.3 192.168.247.95 <none> Ubuntu 20.04.3 LTS 5.4.0-122-generic docker://20.10.21 [root@k8s-calico-master-01 manifests]# [root@k8s-calico-master-01 manifests]# kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-f79f7749d-lkr96 1/1 Running 0 6m51s kube-system calico-node-5kl79 1/1 Running 0 6m51s kube-system calico-node-6fjqj 1/1 Running 0 6m51s kube-system calico-node-7bz4x 1/1 Running 0 6m51s kube-system coredns-565d847f94-m54rw 1/1 Running 0 17s kube-system coredns-565d847f94-mqlk9 1/1 Running 0 21m kube-system etcd-k8s-calico-master-01 1/1 Running 0 22m kube-system kube-apiserver-k8s-calico-master-01 1/1 Running 0 21m kube-system kube-controller-manager-k8s-calico-master-01 1/1 Running 0 21m kube-system kube-proxy-76zhv 1/1 Running 0 20m kube-system kube-proxy-8kl86 1/1 Running 0 20m kube-system kube-proxy-wl9cl 1/1 Running 0 21m kube-system kube-scheduler-k8s-calico-master-01 1/1 Running 0 22m [root@k8s-calico-master-01 manifests]# [root@k8s-calico-master-01 manifests]# kubectl create deployment nginx --image=nginx:1.23.2-alpine --replicas=3 deployment.apps/nginx created [root@k8s-calico-master-01 manifests]# [root@k8s-calico-master-01 manifests]# kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE default nginx-5cd78c5d7-4nl9q 1/1 Running 0 2m23s default nginx-5cd78c5d7-57pmr 1/1 Running 0 2m23s default nginx-5cd78c5d7-96qf2 1/1 Running 0 2m23s kube-system calico-kube-controllers-f79f7749d-lkr96 1/1 Running 0 9m52s kube-system calico-node-5kl79 1/1 Running 0 9m52s kube-system calico-node-6fjqj 1/1 Running 0 9m52s kube-system calico-node-7bz4x 1/1 Running 0 9m52s kube-system coredns-565d847f94-m54rw 1/1 Running 0 3m18s kube-system coredns-565d847f94-mqlk9 1/1 Running 0 24m kube-system etcd-k8s-calico-master-01 1/1 Running 0 25m kube-system kube-apiserver-k8s-calico-master-01 1/1 Running 0 24m kube-system kube-controller-manager-k8s-calico-master-01 1/1 Running 0 24m kube-system kube-proxy-76zhv 1/1 Running 0 23m kube-system kube-proxy-8kl86 1/1 Running 0 23m kube-system kube-proxy-wl9cl 1/1 Running 0 24m kube-system kube-scheduler-k8s-calico-master-01 1/1 Running 0 25m [root@k8s-calico-master-01 manifests]#
[root@k8s-calico-master-01 manifests]# kubectl create service nodeport nginx --tcp=80:80 service/nginx created [root@k8s-calico-master-01 manifests]#
[root@k8s-calico-master-01 manifests]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 25m nginx NodePort 10.105.22.18 <none> 80:30616/TCP 15s [root@k8s-calico-master-01 manifests]#
三、calico通信流程
calico网络示意图
通信流程
#准备测试pod [root@easzlab-deploy ~]# docker pull centos:centos7.9.2009 #提前下载镜像 centos7.9.2009: Pulling from library/centos 2d473b07cdd5: Pull complete Digest: sha256:c73f515d06b0fa07bb18d8202035e739a494ce760aa73129f60f4bf2bd22b407 Status: Downloaded newer image for centos:centos7.9.2009 docker.io/library/centos:centos7.9.2009 [root@easzlab-deploy ~]# [root@easzlab-deploy ~]# docker tag centos:centos7.9.2009 harbor.magedu.net/baseimages/centos7.9.2009 [root@easzlab-deploy ~]# docker push harbor.magedu.net/baseimages/centos7.9.2009 #为了部署方便，减小下载镜像时间，提前上传harbor私有仓库 Using default tag: latest The push refers to repository [harbor.magedu.net/baseimages/centos7.9.2009] 174f56854903: Pushed latest: digest: sha256:dead07b4d8ed7e29e98de0f4504d87e8880d4347859d839686a31da35a3b532f size: 529 [root@easzlab-deploy ~]# [root@easzlab-deploy ~]# kubectl run net-test1 --image=harbor.magedu.net/baseimages/centos7.9.2009 sleep 360000 pod/net-test1 created [root@easzlab-deploy ~]# [root@easzlab-deploy ~]# kubectl run net-test2 --image=harbor.magedu.net/baseimages/centos7.9.2009 sleep 360000 pod/net-test2 created [root@easzlab-deploy ~]# [root@easzlab-deploy ~]# kubectl get pod -owide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES net-test1 1/1 Running 0 17m 10.200.40.224 172.16.88.160 <none> <none> net-test2 1/1 Running 0 14s 10.200.40.225 172.16.88.160 <none> <none> [root@easzlab-deploy ~]# [root@easzlab-deploy ~]# kubectl exec -it net-test1 bash kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead. [root@net-test1 /]# [root@net-test1 /]# yum install -y vim wget tree lrzsz iproute net-tools iotop tcpdump ethtool [root@net-test1 /]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000 link/ipip 0.0.0.0 brd 0.0.0.0 4: eth0@if35: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether 92:b0:25:85:c9:44 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 10.200.40.224/32 brd 10.200.40.224 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::90b0:25ff:fe85:c944/64 scope link valid_lft forever preferred_lft forever [root@net-test1 /]# [root@net-test2 /]# yum install -y vim wget tree lrzsz iproute net-tools iotop tcpdump ethtool [root@net-test2 /]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000 link/ipip 0.0.0.0 brd 0.0.0.0 4: eth0@if36: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether b2:76:7a:81:77:34 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 10.200.40.225/32 brd 10.200.40.225 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::b076:7aff:fe81:7734/64 scope link valid_lft forever preferred_lft forever [root@net-test2 /]# [root@easzlab-k8s-node-01 ~]# ip a |grep ^36 36: cali2b2e7c9e43e@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default [root@easzlab-k8s-node-01 ~]# [root@easzlab-k8s-node-01 ~]# ip a |grep ^35 35: cali073b411309a@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default [root@easzlab-k8s-node-01 ~]#
抓包测试
pod1–ping—>pod2
[root@net-test2 /]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1000 link/ipip 0.0.0.0 brd 0.0.0.0 4: eth0@if36: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default link/ether b2:76:7a:81:77:34 brd ff:ff:ff:ff:ff:ff link-netnsid 0 inet 10.200.40.225/32 brd 10.200.40.225 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::b076:7aff:fe81:7734/64 scope link valid_lft forever preferred_lft forever [root@net-test2 /]# [root@net-test1 /]# ping 10.200.40.225 PING 10.200.40.225 (10.200.40.225) 56(84) bytes of data. 64 bytes from 10.200.40.225: icmp_seq=1 ttl=63 time=0.331 ms 64 bytes from 10.200.40.225: icmp_seq=2 ttl=63 time=0.196 ms 64 bytes from 10.200.40.225: icmp_seq=3 ttl=63 time=0.166 ms 64 bytes from 10.200.40.225: icmp_seq=4 ttl=63 time=0.158 ms 64 bytes from 10.200.40.225: icmp_seq=5 ttl=63 time=0.263 ms 64 bytes from 10.200.40.225: icmp_seq=6 ttl=63 time=0.303 ms 64 bytes from 10.200.40.225: icmp_seq=7 ttl=63 time=0.350 ms 64 bytes from 10.200.40.225: icmp_seq=8 ttl=63 time=0.290 ms 64 bytes from 10.200.40.225: icmp_seq=9 ttl=63 time=0.262 ms 64 bytes from 10.200.40.225: icmp_seq=10 ttl=63 time=0.331 ms 64 bytes from 10.200.40.225: icmp_seq=11 ttl=63 time=0.255 ms 64 bytes from 10.200.40.225: icmp_seq=12 ttl=63 time=0.302 ms 64 bytes from 10.200.40.225: icmp_seq=13 ttl=63 time=0.299 ms 64 bytes from 10.200.40.225: icmp_seq=14 ttl=63 time=0.332 ms 64 bytes from 10.200.40.225: icmp_seq=15 ttl=63 time=0.351 ms 64 bytes from 10.200.40.225: icmp_seq=16 ttl=63 time=0.359 ms ^C --- 10.200.40.225 ping statistics --- 16 packets transmitted, 16 received, 0% packet loss, time 15334ms rtt min/avg/max/mdev = 0.158/0.284/0.359/0.063 ms [root@net-test1 /]# [root@easzlab-k8s-node-01 ~]# tcpdump -nn -vvv -i cali2b2e7c9e43e -vvv -nn ! port 22 and ! port 2379 and ! port 6443 and ! port 10250 and ! arp and ! port 53 tcpdump: listening on cali2b2e7c9e43e, link-type EN10MB (Ethernet), capture size 262144 bytes 10:15:05.673636 IP (tos 0x0, ttl 63, id 37108, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 5, length 64 10:15:05.673725 IP (tos 0x0, ttl 64, id 13898, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 5, length 64 10:15:06.697644 IP (tos 0x0, ttl 63, id 37296, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 6, length 64 10:15:06.697718 IP (tos 0x0, ttl 64, id 13994, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 6, length 64 10:15:07.721723 IP (tos 0x0, ttl 63, id 37404, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 7, length 64 10:15:07.721792 IP (tos 0x0, ttl 64, id 14025, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 7, length 64 10:15:08.745614 IP (tos 0x0, ttl 63, id 37478, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 8, length 64 10:15:08.745677 IP (tos 0x0, ttl 64, id 14197, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 8, length 64 10:15:09.769666 IP (tos 0x0, ttl 63, id 37689, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 9, length 64 10:15:09.769717 IP (tos 0x0, ttl 64, id 14424, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 9, length 64 10:15:10.793676 IP (tos 0x0, ttl 63, id 37914, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 10, length 64 10:15:10.793747 IP (tos 0x0, ttl 64, id 14635, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 10, length 64 10:15:11.817754 IP (tos 0x0, ttl 63, id 38065, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 11, length 64 10:15:11.817813 IP (tos 0x0, ttl 64, id 14675, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 11, length 64 10:15:12.841658 IP (tos 0x0, ttl 63, id 38193, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 12, length 64 10:15:12.841719 IP (tos 0x0, ttl 64, id 14868, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 12, length 64 10:15:13.865719 IP (tos 0x0, ttl 63, id 38305, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 13, length 64 10:15:13.865779 IP (tos 0x0, ttl 64, id 14943, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 13, length 64 10:15:14.889792 IP (tos 0x0, ttl 63, id 38327, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 14, length 64 10:15:14.889859 IP (tos 0x0, ttl 64, id 15081, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 14, length 64 10:15:15.914342 IP (tos 0x0, ttl 63, id 38546, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 15, length 64 10:15:15.914410 IP (tos 0x0, ttl 64, id 15132, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 15, length 64 10:15:16.915587 IP (tos 0x0, ttl 63, id 38793, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 16, length 64 10:15:16.915683 IP (tos 0x0, ttl 64, id 15196, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 16, length 64 [root@easzlab-k8s-node-01 ~]# tcpdump -nn -vvv -i cali073b411309a -vvv -nn ! port 22 and ! port 2379 and ! port 6443 and ! port 10250 and ! arp and ! port 53 tcpdump: listening on cali073b411309a, link-type EN10MB (Ethernet), capture size 262144 bytes 10:15:07.721570 IP (tos 0x0, ttl 64, id 37404, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 7, length 64 10:15:07.721813 IP (tos 0x0, ttl 63, id 14025, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 7, length 64 10:15:08.745504 IP (tos 0x0, ttl 64, id 37478, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 8, length 64 10:15:08.745700 IP (tos 0x0, ttl 63, id 14197, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 8, length 64 10:15:09.769557 IP (tos 0x0, ttl 64, id 37689, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 9, length 64 10:15:09.769736 IP (tos 0x0, ttl 63, id 14424, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 9, length 64 10:15:10.793539 IP (tos 0x0, ttl 64, id 37914, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 10, length 64 10:15:10.793770 IP (tos 0x0, ttl 63, id 14635, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 10, length 64 10:15:11.817656 IP (tos 0x0, ttl 64, id 38065, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 11, length 64 10:15:11.817832 IP (tos 0x0, ttl 63, id 14675, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 11, length 64 10:15:12.841527 IP (tos 0x0, ttl 64, id 38193, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 12, length 64 10:15:12.841739 IP (tos 0x0, ttl 63, id 14868, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 12, length 64 10:15:13.865586 IP (tos 0x0, ttl 64, id 38305, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 13, length 64 10:15:13.865798 IP (tos 0x0, ttl 63, id 14943, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 13, length 64 10:15:14.889651 IP (tos 0x0, ttl 64, id 38327, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 14, length 64 10:15:14.889879 IP (tos 0x0, ttl 63, id 15081, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 14, length 64 10:15:15.914216 IP (tos 0x0, ttl 64, id 38546, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 15, length 64 10:15:15.914448 IP (tos 0x0, ttl 63, id 15132, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 15, length 64 10:15:16.915467 IP (tos 0x0, ttl 64, id 38793, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo request, id 126, seq 16, length 64 10:15:16.915714 IP (tos 0x0, ttl 63, id 15196, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo reply, id 126, seq 16, length 64
pod2–ping–>pod1
[root@net-test1 /]# ifconfig eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 10.200.40.224 netmask 255.255.255.255 broadcast 10.200.40.224 inet6 fe80::90b0:25ff:fe85:c944 prefixlen 64 scopeid 0x20<link> ether 92:b0:25:85:c9:44 txqueuelen 0 (Ethernet) RX packets 33115 bytes 50425930 (48.0 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 12864 bytes 851045 (831.0 KiB) TX errors 0 dropped 1 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1000 (Local Loopback) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 [root@net-test1 /]# [root@net-test2 /]# ping 10.200.40.224 PING 10.200.40.224 (10.200.40.224) 56(84) bytes of data. 64 bytes from 10.200.40.224: icmp_seq=1 ttl=63 time=0.280 ms 64 bytes from 10.200.40.224: icmp_seq=2 ttl=63 time=0.218 ms 64 bytes from 10.200.40.224: icmp_seq=3 ttl=63 time=0.233 ms 64 bytes from 10.200.40.224: icmp_seq=4 ttl=63 time=0.212 ms 64 bytes from 10.200.40.224: icmp_seq=5 ttl=63 time=0.250 ms 64 bytes from 10.200.40.224: icmp_seq=6 ttl=63 time=0.147 ms 64 bytes from 10.200.40.224: icmp_seq=7 ttl=63 time=0.275 ms 64 bytes from 10.200.40.224: icmp_seq=8 ttl=63 time=0.311 ms 64 bytes from 10.200.40.224: icmp_seq=9 ttl=63 time=0.310 ms 64 bytes from 10.200.40.224: icmp_seq=10 ttl=63 time=0.250 ms 64 bytes from 10.200.40.224: icmp_seq=11 ttl=63 time=0.414 ms 64 bytes from 10.200.40.224: icmp_seq=12 ttl=63 time=0.251 ms 64 bytes from 10.200.40.224: icmp_seq=13 ttl=63 time=0.269 ms 64 bytes from 10.200.40.224: icmp_seq=14 ttl=63 time=0.270 ms 64 bytes from 10.200.40.224: icmp_seq=15 ttl=63 time=0.264 ms ^C --- 10.200.40.224 ping statistics --- 15 packets transmitted, 15 received, 0% packet loss, time 14335ms rtt min/avg/max/mdev = 0.147/0.263/0.414/0.058 ms [root@net-test2 /]# [root@easzlab-k8s-node-01 ~]# tcpdump -nn -vvv -i cali2b2e7c9e43e -vvv -nn ! port 22 and ! port 2379 and ! port 6443 and ! port 10250 and ! arp and ! port 53 tcpdump: listening on cali2b2e7c9e43e, link-type EN10MB (Ethernet), capture size 262144 bytes 10:12:09.545509 IP (tos 0x0, ttl 64, id 54755, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 5, length 64 10:12:09.545679 IP (tos 0x0, ttl 63, id 8738, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 5, length 64 10:12:10.569773 IP (tos 0x0, ttl 64, id 54882, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 6, length 64 10:12:10.569868 IP (tos 0x0, ttl 63, id 8857, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 6, length 64 10:12:11.593514 IP (tos 0x0, ttl 64, id 54953, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 7, length 64 10:12:11.593716 IP (tos 0x0, ttl 63, id 8885, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 7, length 64 10:12:12.617550 IP (tos 0x0, ttl 64, id 55118, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 8, length 64 10:12:12.617765 IP (tos 0x0, ttl 63, id 9060, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 8, length 64 10:12:13.641520 IP (tos 0x0, ttl 64, id 55343, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 9, length 64 10:12:13.641737 IP (tos 0x0, ttl 63, id 9090, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 9, length 64 10:12:14.665526 IP (tos 0x0, ttl 64, id 55536, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 10, length 64 10:12:14.665702 IP (tos 0x0, ttl 63, id 9245, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 10, length 64 10:12:15.689602 IP (tos 0x0, ttl 64, id 55612, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 11, length 64 10:12:15.689841 IP (tos 0x0, ttl 63, id 9474, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 11, length 64 10:12:16.713636 IP (tos 0x0, ttl 64, id 55799, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 12, length 64 10:12:16.713812 IP (tos 0x0, ttl 63, id 9687, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 12, length 64 10:12:17.737540 IP (tos 0x0, ttl 64, id 55970, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 13, length 64 10:12:17.737729 IP (tos 0x0, ttl 63, id 9931, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 13, length 64 10:12:18.761512 IP (tos 0x0, ttl 64, id 56176, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 14, length 64 10:12:18.761706 IP (tos 0x0, ttl 63, id 10010, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 14, length 64 10:12:19.785658 IP (tos 0x0, ttl 64, id 56210, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 15, length 64 10:12:19.785844 IP (tos 0x0, ttl 63, id 10011, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 15, length 64 [root@easzlab-k8s-node-01 ~]# tcpdump -nn -vvv -i cali073b411309a -vvv -nn ! port 22 and ! port 2379 and ! port 6443 and ! port 10250 and ! arp and ! port 53 tcpdump: listening on cali073b411309a, link-type EN10MB (Ethernet), capture size 262144 bytes 10:12:11.593590 IP (tos 0x0, ttl 63, id 54953, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 7, length 64 10:12:11.593676 IP (tos 0x0, ttl 64, id 8885, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 7, length 64 10:12:12.617673 IP (tos 0x0, ttl 63, id 55118, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 8, length 64 10:12:12.617744 IP (tos 0x0, ttl 64, id 9060, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 8, length 64 10:12:13.641642 IP (tos 0x0, ttl 63, id 55343, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 9, length 64 10:12:13.641716 IP (tos 0x0, ttl 64, id 9090, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 9, length 64 10:12:14.665616 IP (tos 0x0, ttl 63, id 55536, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 10, length 64 10:12:14.665681 IP (tos 0x0, ttl 64, id 9245, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 10, length 64 10:12:15.689732 IP (tos 0x0, ttl 63, id 55612, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 11, length 64 10:12:15.689818 IP (tos 0x0, ttl 64, id 9474, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 11, length 64 10:12:16.713719 IP (tos 0x0, ttl 63, id 55799, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 12, length 64 10:12:16.713791 IP (tos 0x0, ttl 64, id 9687, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 12, length 64 10:12:17.737653 IP (tos 0x0, ttl 63, id 55970, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 13, length 64 10:12:17.737710 IP (tos 0x0, ttl 64, id 9931, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 13, length 64 10:12:18.761615 IP (tos 0x0, ttl 63, id 56176, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 14, length 64 10:12:18.761685 IP (tos 0x0, ttl 64, id 10010, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 14, length 64 10:12:19.785767 IP (tos 0x0, ttl 63, id 56210, offset 0, flags [DF], proto ICMP (1), length 84) 10.200.40.225 > 10.200.40.224: ICMP echo request, id 108, seq 15, length 64 10:12:19.785825 IP (tos 0x0, ttl 64, id 10011, offset 0, flags [none], proto ICMP (1), length 84) 10.200.40.224 > 10.200.40.225: ICMP echo reply, id 108, seq 15, length 64
保存tcpdump抓包过程，可以通过Wireshark进行分析
[root@easzlab-k8s-node-01 ~]# tcpdump -nn -vvv -i cali2b2e7c9e43e -vvv -nn ! port 22 and ! port 2379 and ! port 6443 and ! port 10250 and ! arp and ! port 53 -w cali2b2e7c9e43e.cap
[root@easzlab-k8s-node-01 ~]# tcpdump -nn -vvv -i cali073b411309a -vvv -nn ! port 22 and ! port 2379 and ! port 6443 and ! port 10250 and ! arp and ! port 53 -w cali073b411309a.cap
当源pod发起请求：[root@net-test2 /]# ping 10.200.40.224, 报文先到达宿主机与pod对应的网卡：cali2b2e7c9e43e，在到达在宿主机与pod对应的网卡：cali073b411309a
此时报文中源IP为pod的eth0的ip,源mac 为pod的eth0的mac，目的IP为10.200.40.225，下一跳为网关地址169.254.1.1 ,目的mac为ee:ee:ee:ee:ee:ee(默认网关169.254.1.1的MAC地址)。
[root@k8s-calico-master-01 manifests]# kubectl get node -owideNAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIMEk8s-calico-master-01 Ready control-plane 23m v1.25.3 192.168.247.91 <none> Ubuntu 20.04.3 LTS 5.4.0-122-generic docker://20.10.21k8s-calico-node-01 Ready <none> 22m v1.25.3 192.168.247.94 <none> Ubuntu 20.04.3 LTS 5.4.0-122-generic docker://20.10.21k8s-calico-node-02 Ready <none> 22m v1.25.3 192.168.247.95 <none> Ubuntu 20.04.3 LTS 5.4.0-122-generic docker://20.10.21[root@k8s-calico-master-01 manifests]#
原文地址：http://www.cnblogs.com/cyh00001/p/16833584.html
1. 本站所有资源来源于用户上传和网络，如有侵权请邮件联系站长！
2. 分享目的仅供大家学习和交流，请务用于商业用途!
3. 如果你也有好源码或者教程，可以到用户中心发布，分享有积分奖励和额外收入！
4. 本站提供的源码、模板、插件等等其他资源，都不包含技术服务请大家谅解！
5. 如有链接无法下载、失效或广告，请联系管理员处理！
6. 本站资源售价只是赞助，收取费用仅维持本站的日常运营所需！
7. 如遇到加密压缩包，默认解压密码为"gltf",如遇到无法解压的请联系管理员！
8. 因为资源和程序源码均为可复制品，所以不支持任何理由的退款兑现，请斟酌后支付下载
声明：如果标题没有注明"已测试"或者"测试可用"等字样的资源源码均未经过站长测试.特别注意没有标注的源码不保证任何可用性
