AWS GWLB对访问NLB流量做安全检测
B站视频：https://www.bilibili.com/video/BV1wt4y1K7kz/?spm_id_from=333.999.0.0
一、架构图
这个设计的核心在于路由表的设计，可以根据下面的架构图示，理解路由流量走向。（我发现公众号对图片压缩特别厉害，暂时没有找到上传原图的办法。如果你和我一样有强迫症，可以找我要高清图片）
实验环境一共有两个VPC，左边的是业务VPC，里面有两个APP模拟HTTP的业务，EC2放在私有子网里面，在不同的AZ。
有一个面向互联网的Network Load Balancer，向公网发布了这个HTTP的服务，这样互联网上的用户，就可以通过NLB访问后面的业务。我们要做的是把这个流量，引导到防火墙上去，做安全检测。这里使用Linux的
iptables
来模拟防火墙。
另外，APP可以通过NAT GW上网，这个APP主动访问互联网的流量，也需要送到防火墙上去，做安全检测。所以，APP有两种流量都需要送到防火墙上去做安全检测，下面看一下APP的两种流量路径。
一、来自于互联网对NLB访问的流量路径
。
-
首先，互联网上的用户，对NLB的公有DNS发起请求，这个DNS请求，会解析到NLB的两个公网IP地址，然后流量通过ISP路由，到达AWS的IGW上。
-
因为IGW关联了一个Ingress Route Table，发现去往NLB所在子网的路由指向GWLB endpoint1，和GWLB endpoint2，这里假设现在解析的主IP是Endpoint2所在的网段，这样路由会送到GWLB Endpoint2。
-
Endpoint2收到流量之后，会通过Private Link，把流量发送到GWLB上。
-
GWLB会通过GENEVE封装报文，把流量发送到防火墙。
-
防火墙做完安全检测之后，又会把流量送回给GWLB，然后通过Private Link，送到GWLB Endpoint2。
-
注意，到达GWLB Endpoint2之后，查询的是GWLB Endpoint2关联的Public Route Table，因为目的地址是NLB subnet2网段，所以匹配到local路由，将流量送给NLB。
-
流量到NLB之后，因为侦听组关联了APP实例，所以NLB将流量发送到实例的主接口，这里查询的是NLB所在子网的local路由。
-
最后，流量抵达了APP。以上就是来自互联网，对NLB发起的请求，流量经过防火墙之后，再抵达APP的过程。
继续看一下回包流程。
-
注意，对于来自NLB的流量，EC2并不查询所在子网的路由表，而是直接返回给NLB（这个点很有意思，有机会单独再聊NLB的行为）。
-
NLB收到报文之后，查询所在子网路由表，因为回包是互联网上的地址，所以会匹配到默认路由，将报文发送到GWLB Endpoint2。
-
接下来又是一样的流程，流量会经过Private Link到防火墙绕一圈再回来。
-
流量回到GWLB Endpoint2上之后，会匹配到默认路由，将流量通过IGW发送到互联网上。
以上就是完整的从互联网对NLB发起请求和回包的流程。
二、APP1主动访问互联网的流量路径
-
APP1对互联网地址发起请求，APP1查询子网关联的路由表，匹配到默认路由，流量送到NAT GW1。
-
NAT GW1收到流量后，查询路由表，默认路由把流量送给GWLB Endpoint1，发送时会对源IP地址做NAT转换。
-
流量到GWLB Endpoint1之后，通过Private Link把流量送到防火墙检测，然后再发回来。
-
GWLB Endpoint1收到流量之后，默认路由将流量送往IGW到达互联网，源地址是NAT GW1的公网IP地址。
继续看回包流程。
-
互联网上的主机收到报文以后，源地址是NAT GW1的公网IP地址，报文通过ISP路由，流量到达AWS的IGW。
-
到达IGW之后，会查询Ingress Route Table，去往NAT GW1网段的路由送到GWLB Endpoint1。
-
流量到GWLB Endpoint1之后，通过Private Link把流量送到防火墙检测，然后再发回来。
-
GWLB Endpoint1收到流量之后，匹配到local路由，将流量发送到NAT GW1。
-
NAT GW1收到报文之后，查询NAT转换表项，目的地址匹配到local路由表，将流量发送到APP1。
二、创建实验环境
通过CloudFormation创建实验环境，堆栈需要7分钟左右的时间创建完成。
上传堆栈文件。
编辑堆栈名称，修改实例密钥。
允许创建IAM资源。
Parameters:
EC2InstanceAmiId:
Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
Default: '/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2'
Environment:
Type: String
AllowedValues:
- dev
- prod
Default: dev
MyKeyPair:
Description: Amazon EC2 Key Pair
Type: AWS::EC2::KeyPair::KeyName
Default: Global_Tokyo_KeyPair
WebServerPort:
Description: Apache Http Server Port
Type: String
Default: 8443
AllowedValues:
- 8443
- 8888
- 8088
Resources:
BastionSsmRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Action:
- 'sts:AssumeRole'
Path: /
BastionSsmPolicy:
Type: AWS::IAM::Policy
Properties:
PolicyName: PrivatelianceInstanceAccess
PolicyDocument:
Statement:
- Effect: Allow
Action:
- ssm:DescribeAssociation
- ssm:GetDeployablePatchSnapshotForInstance
- ssm:GetDocument
- ssm:DescribeDocument
- ssm:GetManifest
- ssm:GetParameter
- ssm:GetParameters
- ssm:ListAssociations
- ssm:ListInstanceAssociations
- ssm:PutInventory
- ssm:PutComplianceItems
- ssm:PutConfigurePackageResult
- ssm:UpdateAssociationStatus
- ssm:UpdateInstanceAssociationStatus
- ssm:UpdateInstanceInformation
Resource: "*"
- Effect: Allow
Action:
- ssmmessages:CreateControlChannel
- ssmmessages:CreateDataChannel
- ssmmessages:OpenControlChannel
- ssmmessages:OpenDataChannel
Resource: "*"
- Effect: Allow
Action:
- ec2messages:AcknowledgeMessage
- ec2messages:DeleteMessage
- ec2messages:FailMessage
- ec2messages:GetEndpoint
- ec2messages:GetMessages
- ec2messages:SendReply
Resource: "*"
Roles:
- !Ref BastionSsmRole
BastionSsmProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles:
- !Ref BastionSsmRole
#=========================================SecVpc========================================#
# 创建SecVpc
SecVpc:
Type: AWS::EC2::VPC
Properties:
CidrBlock: 10.100.10.0/16
EnableDnsSupport: 'true'
EnableDnsHostnames: 'true'
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc
# 创建IGW并且关联到VPC
SecVpcIGW:
Type: "AWS::EC2::InternetGateway"
Properties:
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpcIGW
SecVpcAttachIgw:
Type: "AWS::EC2::VPCGatewayAttachment"
Properties:
VpcId: !Ref SecVpc
InternetGatewayId: !Ref SecVpcIGW
#---------------------------SecVpc创建6个子网-------------------------------------#
# SecVpc AZ1内创建公有子网
SecVpcAz1PublicSubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref SecVpc
CidrBlock: 10.100.10.0/24
AvailabilityZone:
Fn::Select:
- 0
- Fn::GetAZs: ""
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc-AZ1-Public-Subnet
# SecVpc AZ2内创建公有子网
SecVpcAz2PublicSubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref SecVpc
CidrBlock: 10.100.20.0/24
AvailabilityZone:
Fn::Select:
- 1
- Fn::GetAZs: ""
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc-AZ2-Public-Subnet
# SecVpc AZ1内创建私有子网
SecVpcAz1PrivateSubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref SecVpc
CidrBlock: 10.100.30.0/24
AvailabilityZone:
Fn::Select:
- 0
- Fn::GetAZs: ""
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc-AZ1-Private-Subnet
# SecVpc AZ2内创建私有子网
SecVpcAz2PrivateSubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref SecVpc
CidrBlock: 10.100.40.0/24
AvailabilityZone:
Fn::Select:
- 1
- Fn::GetAZs: ""
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc-AZ2-Private-Subnet
# SecVpc AZ1内创建TGW子网
SecVpcAz1TgwSubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref SecVpc
CidrBlock: 10.100.50.0/24
AvailabilityZone:
Fn::Select:
- 0
- Fn::GetAZs: ""
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc-AZ1-TGW-Subnet
# SecVpc AZ2内创建TGW子网
SecVpcAz2TgwSubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref SecVpc
CidrBlock: 10.100.60.0/24
AvailabilityZone:
Fn::Select:
- 1
- Fn::GetAZs: ""
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc-AZ2-TGW-Subnet
#---------------------------SecVpc创建路由表-------------------------------------#
# 公有子网路由表及关联
SecVpcAz1PublicRouteTable:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref SecVpc
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc-AZ1-Public-RouteTable
SecVpcAz1PublicRouteTableAssociation:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
RouteTableId: !Ref SecVpcAz1PublicRouteTable
SubnetId: !Ref SecVpcAz1PublicSubnet
SecVpcAz2PublicRouteTable:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref SecVpc
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc-AZ2-Public-RouteTable
SecVpcAz2PublicRouteTableAssociation:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
RouteTableId: !Ref SecVpcAz2PublicRouteTable
SubnetId: !Ref SecVpcAz2PublicSubnet
# Private子网路由表及关联
SecVpcAz1PrivateRouteTable:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref SecVpc
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc-AZ1-Private-RouteTable
SecVpcAz1PrivateRouteTableAssociation:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
RouteTableId: !Ref SecVpcAz1PrivateRouteTable
SubnetId: !Ref SecVpcAz1PrivateSubnet
SecVpcAz2PrivateRouteTable:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref SecVpc
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc-AZ2-Private-RouteTable
SecVpcAz2PrivateRouteTableAssociation:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
RouteTableId: !Ref SecVpcAz2PrivateRouteTable
SubnetId: !Ref SecVpcAz2PrivateSubnet
# Tgw路由表及关联
SecVpcAz1TgwRouteTable:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref SecVpc
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc-AZ1-Tgw-RouteTable
SecVpcAz1TgwRouteTableAssociation:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
RouteTableId: !Ref SecVpcAz1TgwRouteTable
SubnetId: !Ref SecVpcAz1TgwSubnet
SecVpcAz2TgwRouteTable:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref SecVpc
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc-AZ2-Tgw-RouteTable
SecVpcAz2TgwRouteTableAssociation:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
RouteTableId: !Ref SecVpcAz2TgwRouteTable
SubnetId: !Ref SecVpcAz2TgwSubnet
#---------------------------NAT Gateway------------------------------------#
# AZ1 NAT GW
SecVpcAz1NatGatewayEIP:
Type: AWS::EC2::EIP
Properties:
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc-AZ1-NatGateway-EIP
SecVpcAz1NatGateway:
Type: AWS::EC2::NatGateway
Properties:
AllocationId: !GetAtt SecVpcAz1NatGatewayEIP.AllocationId
SubnetId: !Ref SecVpcAz1PublicSubnet
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc-AZ1-NatGateway
# AZ2 NAT GW
SecVpcAz2NatGatewayEIP:
Type: AWS::EC2::EIP
Properties:
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc-AZ2-NatGateway-EIP
SecVpcAz2NatGateway:
Type: AWS::EC2::NatGateway
Properties:
AllocationId: !GetAtt SecVpcAz2NatGatewayEIP.AllocationId
SubnetId: !Ref SecVpcAz2PublicSubnet
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc-AZ2-NatGateway
SecVpcAz1PrivateSubnetToInternetRoute:
DependsOn: SecVpcAz1NatGateway
Type: AWS::EC2::Route
Properties:
RouteTableId: !Ref SecVpcAz1PrivateRouteTable
DestinationCidrBlock: '0.0.0.0/0'
NatGatewayId: !Ref SecVpcAz1NatGateway
SecVpcAz2PrivateSubnetToInternetRoute:
DependsOn: SecVpcAz2NatGateway
Type: AWS::EC2::Route
Properties:
RouteTableId: !Ref SecVpcAz2PrivateRouteTable
DestinationCidrBlock: '0.0.0.0/0'
NatGatewayId: !Ref SecVpcAz2NatGateway
#---------------------------添加路由------------------------------------#
# 公有子网添加默认路由去往IGW
SecVpcAz1PublicSubnetToInternetRoute:
Type: "AWS::EC2::Route"
DependsOn: SecVpcIGW
Properties:
RouteTableId: !Ref SecVpcAz1PublicRouteTable
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref SecVpcIGW
SecVpcAz2PublicSubnetToInternetRoute:
Type: "AWS::EC2::Route"
DependsOn: SecVpcIGW
Properties:
RouteTableId: !Ref SecVpcAz2PublicRouteTable
DestinationCidrBlock: 0.0.0.0/0
GatewayId: !Ref SecVpcIGW
#---------------------------SecVpc创建安全组------------------------------------#
SecVpcSg:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: SG to test ping
VpcId: !Ref SecVpc
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 22
ToPort: 22
CidrIp: 0.0.0.0/0
- IpProtocol: icmp
FromPort: -1
ToPort: -1
CidrIp: 0.0.0.0/0
- IpProtocol: -1
FromPort: -1
ToPort: -1
CidrIp: 10.100.0.0/16
- IpProtocol: tcp
FromPort: 3389
ToPort: 3389
CidrIp: 0.0.0.0/0
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpcSg
#---------------------------SecVpc创建EC2实例------------------------------------#
#--------------------------IAM Instance Role and Profile------------------------------------#
ApplianceRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub "${AWS::StackName}-appliance-role"
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore"
AssumeRolePolicyDocument:
Statement:
- Effect: Allow
Principal:
Service:
- ec2.amazonaws.com
Action:
- 'sts:AssumeRole'
Path: /
AppliancePolicy:
Type: AWS::IAM::Policy
Properties:
PolicyName: AppServer
PolicyDocument:
Statement:
- Effect: Allow
Action:
- ec2:DescribeNetworkInterfaces
Resource: '*'
Roles:
- !Ref ApplianceRole
ApplianceProfile:
Type: AWS::IAM::InstanceProfile
Properties:
Path: /
Roles:
- !Ref ApplianceRole
#---------------------------SecVpc创建GWLB------------------------------------#
# Gateway Load Balancer (GWLB), Target Group, Listener
Gwlb:
Type: AWS::ElasticLoadBalancingV2::LoadBalancer
Properties:
LoadBalancerAttributes:
- Key: load_balancing.cross_zone.enabled
Value: true
Name: gwlb1
Type: gateway
Subnets:
- !Ref SecVpcAz1PrivateSubnet
- !Ref SecVpcAz2PrivateSubnet
Tags:
- Key: Name
Value: !Sub "${AWS::StackName}-gwlb-1"
# Target Group:
TargetGroup:
Type: AWS::ElasticLoadBalancingV2::TargetGroup
Properties:
Name: tg1
Port: 6081
Protocol: GENEVE
TargetGroupAttributes:
- Key: deregistration_delay.timeout_seconds
Value: "20"
VpcId: !Ref SecVpc
HealthCheckPort: 80
HealthCheckProtocol: HTTP
TargetType: instance
Targets:
- Id: !Ref Appliance1
- Id: !Ref Appliance2
Tags:
- Key: Name
Value: !Sub "${AWS::StackName}-tg-1"
# Listener:
Listener:
Type: AWS::ElasticLoadBalancingV2::Listener
Properties:
DefaultActions:
- Type: forward
TargetGroupArn: !Ref TargetGroup
LoadBalancerArn: !Ref Gwlb
#---------------------------SecVpc创建EC2实例------------------------------------#
# EC2 Instances (Appliances acting as target for GWLB):
Appliance1:
DependsOn: [Gwlb, SecVpcAz1PrivateSubnetToInternetRoute]
Type: AWS::EC2::Instance
Properties:
ImageId: !Ref EC2InstanceAmiId
KeyName: !Ref MyKeyPair
InstanceType: t2.micro
IamInstanceProfile: !Ref ApplianceProfile
SecurityGroupIds:
- !Ref SecVpcSg
SubnetId: !Ref SecVpcAz1PrivateSubnet
Tags:
- Key: Name
Value: !Sub "${AWS::StackName}-appliance-1"
UserData:
Fn::Base64: |
#!/bin/bash -ex
# Install packages:
yum update -y;
yum install jq -y;
yum install httpd -y;
yum install htop -y;
sudo yum install iptables-services -y;
# Enable IP Forwarding:
echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.d/00-defaults.conf;
sysctl -p /etc/sysctl.d/00-defaults.conf;
# Configure hostname:
hostnamectl set-hostname gwlb-target-1;
# Configure SSH client alive interval for ssh session timeout:
echo 'ClientAliveInterval 60' | sudo tee --append /etc/ssh/sshd_config;
service sshd restart;
# Set dark background for vim:
touch /home/ec2-user/.vimrc;
echo "set background=dark" >> /home/ec2-user/.vimrc;
# Define variables:
curl --silent http://169.254.169.254/latest/dynamic/instance-identity/document > /home/ec2-user/iid;
export instance_interface=$(curl --silent http://169.254.169.254/latest/meta-data/network/interfaces/macs/);
export instance_vpcid=$(curl --silent http://169.254.169.254/latest/meta-data/network/interfaces/macs/$instance_interface/vpc-id);
export instance_az=$(cat /home/ec2-user/iid |grep 'availability' | awk -F': ' '{print $2}' | awk -F',' '{print $1}');
export instance_ip=$(cat /home/ec2-user/iid |grep 'privateIp' | awk -F': ' '{print $2}' | awk -F',' '{print $1}' | awk -F'"' '{print$2}');
export instance_region=$(cat /home/ec2-user/iid |grep 'region' | awk -F': ' '{print $2}' | awk -F',' '{print $1}' | awk -F'"' '{print$2}');
export local_az_gwlb_ip=$(aws --region $instance_region ec2 describe-network-interfaces --filters Name=vpc-id,Values=$instance_vpcid | jq ' .NetworkInterfaces[] | select(.AvailabilityZone=='$instance_az') | select(.InterfaceType=="gateway_load_balancer") |.PrivateIpAddress' -r);
export remote_az_gwlb_ip=$(aws --region $instance_region ec2 describe-network-interfaces --filters Name=vpc-id,Values=$instance_vpcid | jq ' .NetworkInterfaces[] | select(.AvailabilityZone!='$instance_az') | select(.InterfaceType=="gateway_load_balancer") |.PrivateIpAddress' -r);
# Start http and configure index.html:
systemctl enable httpd;
systemctl start httpd;
touch /var/www/html/index.html;
echo "<html>" >> /var/www/html/index.html
echo " <head>" >> /var/www/html/index.html
echo " <title>Gateway Load Balancer POC</title>" >> /var/www/html/index.html
echo " <meta http-equiv='Content-Type' content='text/html; charset=ISO-8859-1'>" >> /var/www/html/index.html
echo " </head>" >> /var/www/html/index.html
echo " <body>" >> /var/www/html/index.html
echo " <h1>Welcome to Gateway Load Balancer POC:</h1>" >> /var/www/html/index.html
echo " <h2>This is appliance running in $instance_az. Happy testing!</h2>" >> /var/www/html/index.html
echo " </body>" >> /var/www/html/index.html
echo "</html>" >> /var/www/html/index.html
# Start and configure iptables:
systemctl enable iptables;
systemctl start iptables;
# Configuration below allows allows all traffic:
# Set the default policies for each of the built-in chains to ACCEPT:
iptables -P INPUT ACCEPT;
iptables -P FORWARD ACCEPT;
iptables -P OUTPUT ACCEPT;
# Flush the nat and mangle tables, flush all chains (-F), and delete all non-default chains (-X):
iptables -t nat -F;
iptables -t mangle -F;
iptables -F;
iptables -X;
# Configure nat table to hairpin traffic back to GWLB:
iptables -t nat -A PREROUTING -p udp -s $local_az_gwlb_ip -d $instance_ip -i eth0 -j DNAT --to-destination $local_az_gwlb_ip:6081;
iptables -t nat -A POSTROUTING -p udp --dport 6081 -s $local_az_gwlb_ip -d $local_az_gwlb_ip -o eth0 -j MASQUERADE;
iptables -t nat -A PREROUTING -p udp -s $remote_az_gwlb_ip -d $instance_ip -i eth0 -j DNAT --to-destination $remote_az_gwlb_ip:6081;
iptables -t nat -A POSTROUTING -p udp --dport 6081 -s $remote_az_gwlb_ip -d $remote_az_gwlb_ip -o eth0 -j MASQUERADE;
# Save iptables:
service iptables save;
Appliance2:
DependsOn: [ Gwlb, SecVpcAz2PrivateSubnetToInternetRoute ]
Type: AWS::EC2::Instance
Properties:
ImageId: !Ref EC2InstanceAmiId
KeyName: !Ref MyKeyPair
InstanceType: t2.micro
IamInstanceProfile: !Ref ApplianceProfile
SecurityGroupIds:
- !Ref SecVpcSg
SubnetId: !Ref SecVpcAz2PrivateSubnet
Tags:
- Key: Name
Value: !Sub "${AWS::StackName}-appliance-2"
UserData:
Fn::Base64: |
#!/bin/bash -ex
# Install packages:
yum update -y;
yum install jq -y;
yum install httpd -y;
yum install htop -y;
sudo yum install iptables-services -y;
# Enable IP Forwarding:
echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.d/00-defaults.conf;
sysctl -p /etc/sysctl.d/00-defaults.conf;
# Configure hostname:
hostnamectl set-hostname gwlb-target-2;
# Configure SSH client alive interval for ssh session timeout:
echo 'ClientAliveInterval 60' | sudo tee --append /etc/ssh/sshd_config;
service sshd restart;
# Set dark background for vim:
touch /home/ec2-user/.vimrc;
echo "set background=dark" >> /home/ec2-user/.vimrc;
# Define variables:
curl --silent http://169.254.169.254/latest/dynamic/instance-identity/document > /home/ec2-user/iid;
export instance_interface=$(curl --silent http://169.254.169.254/latest/meta-data/network/interfaces/macs/);
export instance_vpcid=$(curl --silent http://169.254.169.254/latest/meta-data/network/interfaces/macs/$instance_interface/vpc-id);
export instance_az=$(cat /home/ec2-user/iid |grep 'availability' | awk -F': ' '{print $2}' | awk -F',' '{print $1}');
export instance_ip=$(cat /home/ec2-user/iid |grep 'privateIp' | awk -F': ' '{print $2}' | awk -F',' '{print $1}' | awk -F'"' '{print$2}');
export instance_region=$(cat /home/ec2-user/iid |grep 'region' | awk -F': ' '{print $2}' | awk -F',' '{print $1}' | awk -F'"' '{print$2}');
export local_az_gwlb_ip=$(aws --region $instance_region ec2 describe-network-interfaces --filters Name=vpc-id,Values=$instance_vpcid | jq ' .NetworkInterfaces[] | select(.AvailabilityZone=='$instance_az') | select(.InterfaceType=="gateway_load_balancer") |.PrivateIpAddress' -r);
export remote_az_gwlb_ip=$(aws --region $instance_region ec2 describe-network-interfaces --filters Name=vpc-id,Values=$instance_vpcid | jq ' .NetworkInterfaces[] | select(.AvailabilityZone!='$instance_az') | select(.InterfaceType=="gateway_load_balancer") |.PrivateIpAddress' -r);
# Start http and configure index.html:
systemctl enable httpd;
systemctl start httpd;
touch /var/www/html/index.html;
echo "<html>" >> /var/www/html/index.html
echo " <head>" >> /var/www/html/index.html
echo " <title>Gateway Load Balancer POC</title>" >> /var/www/html/index.html
echo " <meta http-equiv='Content-Type' content='text/html; charset=ISO-8859-1'>" >> /var/www/html/index.html
echo " </head>" >> /var/www/html/index.html
echo " <body>" >> /var/www/html/index.html
echo " <h1>Welcome to Gateway Load Balancer POC:</h1>" >> /var/www/html/index.html
echo " <h2>This is appliance running in $instance_az. Happy testing!</h2>" >> /var/www/html/index.html
echo " </body>" >> /var/www/html/index.html
echo "</html>" >> /var/www/html/index.html
# Start and configure iptables:
systemctl enable iptables;
systemctl start iptables;
# Configuration below allows allows all traffic:
# Set the default policies for each of the built-in chains to ACCEPT:
iptables -P INPUT ACCEPT;
iptables -P FORWARD ACCEPT;
iptables -P OUTPUT ACCEPT;
# Flush the nat and mangle tables, flush all chains (-F), and delete all non-default chains (-X):
iptables -t nat -F;
iptables -t mangle -F;
iptables -F;
iptables -X;
# Configure nat table to hairpin traffic back to GWLB:
iptables -t nat -A PREROUTING -p udp -s $local_az_gwlb_ip -d $instance_ip -i eth0 -j DNAT --to-destination $local_az_gwlb_ip:6081;
iptables -t nat -A POSTROUTING -p udp --dport 6081 -s $local_az_gwlb_ip -d $local_az_gwlb_ip -o eth0 -j MASQUERADE;
iptables -t nat -A PREROUTING -p udp -s $remote_az_gwlb_ip -d $instance_ip -i eth0 -j DNAT --to-destination $remote_az_gwlb_ip:6081;
iptables -t nat -A POSTROUTING -p udp --dport 6081 -s $remote_az_gwlb_ip -d $remote_az_gwlb_ip -o eth0 -j MASQUERADE;
# Save iptables:
service iptables save;
SecVpcBastionLinux:
Type: AWS::EC2::Instance
Properties:
IamInstanceProfile: !Ref BastionSsmProfile
ImageId: !Ref EC2InstanceAmiId
KeyName: !Ref MyKeyPair
InstanceType: t3.nano
NetworkInterfaces:
- AssociatePublicIpAddress: true
DeviceIndex: 0
GroupSet:
- Ref: SecVpcSg
SubnetId: !Ref SecVpcAz1PublicSubnet
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-SecVpc-Bastion-Linux
# Create VPC Endpoint Service:
VpcEndpointService:
Type: AWS::EC2::VPCEndpointService
Properties:
GatewayLoadBalancerArns:
- !Ref Gwlb
AcceptanceRequired: false
# Create Lambda Custom Resource to retrieve VPC Endpoint Service Name:
VpceServiceLambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Path: /
Policies:
- PolicyName: root
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- logs:CreateLogStream
- logs:PutLogEvents
Resource: arn:aws:logs:*:*:*
- Effect: Allow
Action:
- ec2:DescribeVpcEndpointServiceConfigurations
- ec2:DescribeVpcEndpointServicePermissions
- ec2:DescribeVpcEndpointServices
Resource: "*"
VpceServiceLogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Sub /aws/lambda/${AWS::StackName}-service
RetentionInDays: 1
VpceServiceName:
Type: AWS::Lambda::Function
DependsOn: VpceServiceLogGroup
Properties:
FunctionName: !Sub ${AWS::StackName}-service
Handler: "index.handler"
Role: !GetAtt VpceServiceLambdaExecutionRole.Arn
Code:
ZipFile: |
import json
import logging
import time
import boto3
import cfnresponse
from botocore.exceptions import ClientError
try:
ec2 = boto3.client('ec2')
except ClientError as e:
logger.error(f"ERROR: failed to connect to EC2 client: {e}")
sys.exit(1)
def handler(event, context):
logger = logging.getLogger()
logger.setLevel(logging.INFO)
logger.info('Received event: {}'.format(json.dumps(event)))
responseData = {}
responseStatus = cfnresponse.FAILED
try:
serviceid = event["ResourceProperties"]["VpceServiceId"]
except Exception as e:
logger.info('Attribute retrival failure: {}'.format(e))
try:
if event["RequestType"] == "Delete":
responseStatus = cfnresponse.SUCCESS
cfnresponse.send(event, context, responseStatus, responseData)
except Exception:
logger.exception("Signaling failure to CloudFormation.")
cfnresponse.send(event, context, cfnresponse.FAILED, {})
if event["RequestType"] == "Create":
logger.info("Retrieving VPC Endpoint Service Name:")
try:
response = ec2.describe_vpc_endpoint_service_configurations(
Filters=[
{
'Name': 'service-id',
'Values': [serviceid]
}
]
)
except Exception as e:
logger.info('ec2.describe_vpc_endpoint_service_configurations failure: {}'.format(e))
service_name = response['ServiceConfigurations'][0]['ServiceName']
time.sleep(120)
responseData['ServiceName'] = service_name
responseStatus = cfnresponse.SUCCESS
cfnresponse.send(event, context, responseStatus, responseData)
Runtime: python3.7
Timeout: 150
RetrieveVpceServiceName:
Type: Custom::RetrieveAttributes
Properties:
ServiceToken: !GetAtt VpceServiceName.Arn
VpceServiceId: !Ref VpcEndpointService
# Create Gateway Load Balancer Endpoint:
GwlbEndpoint1:
Type: AWS::EC2::VPCEndpoint
Properties:
VpcId: !Ref SecVpc
ServiceName: !GetAtt RetrieveVpceServiceName.ServiceName
VpcEndpointType: GatewayLoadBalancer
SubnetIds:
- !Ref SecVpcAz1PrivateSubnet
GwlbEndpoint2:
Type: AWS::EC2::VPCEndpoint
Properties:
VpcId: !Ref SecVpc
ServiceName: !GetAtt RetrieveVpceServiceName.ServiceName
VpcEndpointType: GatewayLoadBalancer
SubnetIds:
- !Ref SecVpcAz2PrivateSubnet
#=========================================VpcA========================================#
VpcA:
Type: AWS::EC2::VPC
Properties:
CidrBlock: 10.110.10.0/16
EnableDnsSupport: 'true'
EnableDnsHostnames: 'true'
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VpcA
# 创建IGW并且关联到VPC
VpcAIGW:
Type: "AWS::EC2::InternetGateway"
Properties:
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VpcAIGW
VpcAAttachIgw:
Type: "AWS::EC2::VPCGatewayAttachment"
Properties:
VpcId: !Ref VpcA
InternetGatewayId: !Ref VpcAIGW
#---------------------------VpcA创建8个子网-------------------------------------#
# VpcA AZ1内创建公有子网
VpcAAz1PublicSubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VpcA
CidrBlock: 10.110.10.0/24
AvailabilityZone:
Fn::Select:
- 0
- Fn::GetAZs: ""
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VpcA-AZ1-Public-Subnet
# VpcA AZ2内创建公有子网
VpcAAz2PublicSubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VpcA
CidrBlock: 10.110.20.0/24
AvailabilityZone:
Fn::Select:
- 1
- Fn::GetAZs: ""
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VpcA-AZ2-Public-Subnet
# VpcA AZ1内创建APP子网
VpcAAz1NlbSubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VpcA
CidrBlock: 10.110.30.0/24
AvailabilityZone:
Fn::Select:
- 0
- Fn::GetAZs: ""
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VpcA-AZ1-Nlb-Subnet
# VpcA AZ2内创建APP子网
VpcAAz2NlbSubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VpcA
CidrBlock: 10.110.40.0/24
AvailabilityZone:
Fn::Select:
- 1
- Fn::GetAZs: ""
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VpcA-AZ2-Nlb-Subnet
# VpcA AZ1内创建GWLB子网
VpcAAz1AppSubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VpcA
CidrBlock: 10.110.50.0/24
AvailabilityZone:
Fn::Select:
- 0
- Fn::GetAZs: ""
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VpcA-AZ1-App-Subnet
# VpcA AZ2内创建GWLB子网
VpcAAz2AppSubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VpcA
CidrBlock: 10.110.60.0/24
AvailabilityZone:
Fn::Select:
- 1
- Fn::GetAZs: ""
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VpcA-AZ2-App-Subnet
# VpcA AZ1内创建TGW子网
VpcAAz1TgwSubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VpcA
CidrBlock: 10.110.70.0/24
AvailabilityZone:
Fn::Select:
- 0
- Fn::GetAZs: ""
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VpcA-AZ1-TGW-Subnet
# VpcA AZ2内创建TGW子网
VpcAAz2TgwSubnet:
Type: AWS::EC2::Subnet
Properties:
VpcId: !Ref VpcA
CidrBlock: 10.110.80.0/24
AvailabilityZone:
Fn::Select:
- 1
- Fn::GetAZs: ""
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VpcA-AZ2-TGW-Subnet
#---------------------------VpcA创建路由表-------------------------------------#
# 公有子网路由表及关联
VpcAAz1PublicRouteTable:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref VpcA
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VpcA-AZ1-Public-RouteTable
VpcAAz1PublicRouteTableAssociation:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
RouteTableId: !Ref VpcAAz1PublicRouteTable
SubnetId: !Ref VpcAAz1PublicSubnet
# IGW路由表及关联
VpcAIgwRouteTable:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref VpcA
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VpcA-Ingress-RouteTable
VpcAIgwRouteTableAssociation:
Type: "AWS::EC2::GatewayRouteTableAssociation"
Properties:
GatewayId: !Ref VpcAIGW
RouteTableId: !Ref VpcAIgwRouteTable
VpcAAz2PublicRouteTableAssociation:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
RouteTableId: !Ref VpcAAz1PublicRouteTable
SubnetId: !Ref VpcAAz2PublicSubnet
# Nlb子网路由表及关联
VpcAAz1NlbRouteTable:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref VpcA
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VpcA-AZ1-Nlb-RouteTable
VpcAAz1NlbRouteTableAssociation:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
RouteTableId: !Ref VpcAAz1NlbRouteTable
SubnetId: !Ref VpcAAz1NlbSubnet
VpcAAz2NlbRouteTable:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref VpcA
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VpcA-AZ2-Nlb-RouteTable
VpcAAz2NlbRouteTableAssociation:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
RouteTableId: !Ref VpcAAz2NlbRouteTable
SubnetId: !Ref VpcAAz2NlbSubnet
# App子网路由表及关联
VpcAAz1AppRouteTable:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref VpcA
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VpcA-AZ1-App-RouteTable
VpcAAz1AppRouteTableAssociation:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
RouteTableId: !Ref VpcAAz1AppRouteTable
SubnetId: !Ref VpcAAz1AppSubnet
VpcAAz2AppRouteTable:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref VpcA
Tags:
- Key: Name
Value: !Sub ${AWS::StackName}-VpcA-AZ2-App-RouteTable
VpcAAz2AppRouteTableAssociation:
Type: "AWS::EC2::SubnetRouteTableAssociation"
Properties:
RouteTableId: !Ref VpcAAz2AppRouteTable
SubnetId: !Ref VpcAAz2AppSubnet
# Tgw路由表及关联
VpcAAz1TgwRouteTable:
Type: "AWS::EC2::RouteTable"
Properties:
VpcId: !Ref VpcA
版权声明：本文为liuqianglong_liu原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。