HBN 服务配置
要开始配置 HBN,请登录到 HBN 容器
sudo crictl exec -it $(crictl ps | grep hbn | awk '{print $1;}'
) bash
扁平文件配置
将网络接口和 FRR 配置文件添加到 HBN 以实现所需的配置
/etc/network/interfaces
注意有关更多信息,请参阅 NVIDIA® Cumulus® Linux 文档。
/etc/frr/frr.conf
;/etc/frr/daemons
注意有关更多信息,请参阅 NVIDIA® Cumulus® Linux 文档。
本节假定您熟悉 NVIDIA 用户体验 (NVUE) Cumulus Linux 文档。以下小节仅扩展了 NVUE 的 HBN 特定方面。
NVUE 服务
HBN 默认安装 NVUE 并在启动时启用 NVUE 服务。
NVUE REST API
HBN 默认启用 REST API,但仅限本地主机访问。默认情况下,用户无法从外部访问 REST API。
要启用 REST API 访问,请参阅“启用 REST API 访问”部分。
用户可以从命令行运行 cURL 命令。使用默认的 HBN 用户名 nvidia
和密码,在通过 HBN 准备脚本启用 REST API 时必须更新密码。
要更改 nvidia
用户的默认密码或为 NVUE 访问添加其他用户,请参阅“HBN NVUE 用户凭据”部分。
REST API 示例
curl -u 'nvidia:nvidia' --insecure https://<mgmt_ip>:8765/nvue_v1/vrf/default/router/bgp
{
"configured-neighbors": 2,
"established-neighbors": 2,
"router-id": "10.10.10.201"
}
通过 CLI 管理 NVUE REST API
要启用 REST API 服务,请运行
nv set system api state enabled
要禁用 REST API 服务
nv set system api state disabled
要将 REST API 服务绑定到特定地址
nv set system api listening-address <localhost|ipv4|ipv6|0.0.0.0>
有关使用 NVUE REST API 的信息,请参阅 NVUE API 文档。
NVUE CLI
有关使用 NVUE CLI 的信息,请参阅 NVUE CLI 文档
NVUE 启动配置文件
当使用 NVUE 保存网络配置时,HBN 将配置写入到 /etc/nvue.d/startup.yaml
文件。
启动配置在启动时由 supervisor 守护程序应用。nvued-startup
在应用启动配置后将显示在 EXITED
状态中。
# supervisorctl status nvued-startup
nvued-startup EXITED Apr 17 10:04 AM
nv config apply startup
应用保存在 /etc/nvue.d/
的 yaml 配置。
nv config save
将运行配置保存到 /etc/nvue.d/startup.yaml
。
HBN 默认配置
在全新安装 HBN 后,默认的 /etc/network/interfaces
文件应仅包含环回接口的声明
source /etc/network/interfaces.d/*.intf
auto lo
iface lo inet loopback
FRR 配置文件也应存在于 /etc/frr/
下,但不会启用任何配置。
三层路由
下一跳 ID 和下一跳组
在 HBN 中,路由编程使用下一跳 ID 来指定路由的下一跳。对于具有多条路径的路由,使用下一跳组来标识等价多路径 (ECMP) 下一跳集。此分组使系统能够高效地处理和修改路由和下一跳对象,从而提高可扩展性、网络收敛性和性能。用户可能会注意到内核操作命令输出中路由信息显示的变化。
使用 BGP 和 ECMP 的原生路由
HBN 支持使用 BGP 和 ECMP 的 IPv4 和 IPv6 流量单播路由。ECMP 通过使用基于源 IP、目标 IP 和 IP 标头协议类型的哈希计算来分配流量来实现。
对于 TCP 和 UDP 数据包,它还包括源端口和目标端口。
ECMP 示例
只要路由在 Uplink 或主机端口上有多条路径,就会实施 ECMP。例如,20.20.20.0/24 有 2 条使用两个 Uplink 的路径,因此根据 IP 标头的哈希值选择路径。
# ip route show 20.20.20.0/24
20.20.20.0/24 nhid 106 proto bgp metric 20
# ip nexthop show group id 106
id
106 group 105/107 proto zebra
# ip nexthop show id 105
id
105 via fe80::202:ff:fe00:7 dev p0_if scope link proto zebra
# ip nexthop show id 107
id
107 via fe80::202:ff:fe00:c dev p1_if scope link proto zebra
HBN 最多支持 16 条 ECMP 路径。
使用面向主机的端口作为接入端口的原生路由的 NVUE 配置示例
nv set interface lo ip address 10.10.10.1/32
nv set interface lo ip address 2010:10:10::1/128
nv set interface vlan100 type svi
nv set interface vlan100 vlan 100
nv set interface vlan100 base-interface br_default
nv set interface vlan100 ip address 2030:30:30::1/64
nv set interface vlan100 ip address 30.30.30.1/24
nv set bridge domain br_default vlan 100
nv set interface pf0hpf_if,pf1hpf_if bridge domain br_default access 100
nv set vrf default router bgp router-id 10.10.10.1
nv set vrf default router bgp autonomous-system 65501
nv set vrf default router bgp path-selection multipath aspath-ignore on
nv set vrf default router bgp address-family ipv4-unicast enable on
nv set vrf default router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf default router bgp address-family ipv6-unicast enable on
nv set vrf default router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf default router bgp neighbor p0_if remote-as external
nv set vrf default router bgp neighbor p0_if type unnumbered
nv set vrf default router bgp neighbor p0_if address-family ipv4-unicast enable on
nv set vrf default router bgp neighbor p0_if address-family ipv6-unicast enable on
nv set vrf default router bgp neighbor p1_if remote-as external
nv set vrf default router bgp neighbor p1_if type unnumbered
nv set vrf default router bgp neighbor p1_if address-family ipv4-unicast enable on
nv set vrf default router bgp neighbor p1_if address-family ipv6-unicast enable on
使用面向主机的端口作为 Trunk 端口的原生路由的 NVUE 配置示例
nv set interface lo ip address 10.10.10.1/32
nv set interface lo ip address 2010:10:10::1/128
nv set interface vlan100 type svi
nv set interface vlan100 vlan 100
nv set interface vlan100 base-interface br_default
nv set interface vlan100 ip address 2030:30:30::1/64
nv set interface vlan100 ip address 30.30.30.1/24
nv set bridge domain br_default vlan 100
nv set interface pf0hpf_if,pf1hpf_if bridge domain br_default
nv set vrf default router bgp router-id 10.10.10.1
nv set vrf default router bgp autonomous-system 65501
nv set vrf default router bgp path-selection multipath aspath-ignore on
nv set vrf default router bgp address-family ipv4-unicast enable on
nv set vrf default router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf default router bgp address-family ipv6-unicast enable on
nv set vrf default router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf default router bgp neighbor p0_if remote-as external
nv set vrf default router bgp neighbor p0_if type unnumbered
nv set vrf default router bgp neighbor p0_if address-family ipv4-unicast enable on
nv set vrf default router bgp neighbor p0_if address-family ipv6-unicast enable on
nv set vrf default router bgp neighbor p1_if remote-as external
nv set vrf default router bgp neighbor p1_if type unnumbered
nv set vrf default router bgp neighbor p1_if address-family ipv4-unicast enable on
nv set vrf default router bgp neighbor p1_if address-family ipv6-unicast enable on
使用面向主机的端口作为接入端口的原生路由的扁平文件配置示例
/etc/network/interfaces 配置示例
auto lo
iface lo inet loopback
address 10.10.10.1/32
address 2010:10:10::1/128
auto p0_if
iface p0_if
auto p1_if
iface p1_if
auto pf0hpf_if
iface pf0hpf_if
bridge-access 100
auto pf1hpf_if
iface pf1hpf_if
bridge-access 100
auto vlan100
iface vlan100
address 2030:30:30::1/64
address 30.30.30.1/24
vlan-raw-device br_default
vlan-id 100
auto br_default
iface br_default
bridge-ports pf0hpf_if pf1hpf_if
bridge-vlan-aware yes
bridge-vids 100
bridge-pvid 1
使用面向主机的端口作为 Trunk 端口的原生路由的扁平文件配置示例
/etc/network/interfaces 配置示例
auto lo
iface lo inet loopback
address 10.10.10.1/32
address 2010:10:10::1/128
auto p0_if
iface p0_if
auto p1_if
iface p1_if
auto pf0hpf_if
iface pf0hpf_if
auto pf1hpf_if
iface pf1hpf_if
auto vlan100
iface vlan100
address 2030:30:30::1/64
address 30.30.30.1/24
vlan-raw-device br_default
vlan-id 100
auto br_default
iface br_default
bridge-ports pf0hpf_if pf1hpf_if
bridge-vlan-aware yes
bridge-vids 100
bridge-pvid 1
/etc/frr/daemons 配置示例
bgpd=yes
vtysh_enable=yes
FRR Config file @ /etc/frr/frr.conf -
!
frr version 7.5+cl5.3.0u0
frr defaults datacenter
hostname BLUEFIELD2
log syslog informational
no zebra nexthop kernel enable
!
router bgp 65501
bgp router-id 10.10.10.1
bgp bestpath as-path multipath-relax
neighbor p0_if interface remote-as external
neighbor p0_if advertisement-interval 0
neighbor p0_if timers 3 9
neighbor p0_if timers connect 10
neighbor p1_if interface remote-as external
neighbor p1_if advertisement-interval 0
neighbor p1_if timers 3 9
neighbor p1_if timers connect 10
!
address-family ipv4 unicast
redistribute connected
maximum-paths 64
maximum-paths ibgp 64
exit-address-family
!
address-family ipv6 unicast
redistribute connected
neighbor p0_if activate
neighbor p1_if activate
maximum-paths 64
maximum-paths ibgp 64
exit-address-family
!
line vty
!
end
面向主机的接口上的直接路由
面向主机的接口(PF 和 VF)不限于成为路由桥的一部分。HBN 支持仅 L3 配置,并在面向主机的 PF 和 VF 上进行直接路由。
NVUE 配置示例
nv set interface pf0hpf_if ip address 30.30.11.1/24
nv set interface pf0hpf_if ip address 2030:30:11::1/64
nv set interface pf0vf0_if ip address 30.30.13.1/24
nv set interface pf0vf0_if ip address 2030:30:13::1/64
扁平文件配置示例
auto pf0hpf_if
iface pf0hpf_if
address 2030:30:11::1/64
address 30.30.11.1/24
auto pf0vf0_if
iface pf0vf0_if
address 2030:30:13::1/64
address 30.30.13.1/24
与主机建立 BGP 对等连接
HBN 支持在主机和 BlueField Arm 上运行的 HBN 服务之间建立 BGP 会话,并允许主机通过 BlueField 将任意路由前缀通告到下层结构。主机可以使用任何标准 BGP 协议栈实现来与 HBN 建立 BGP 对等互连。
主机上端点之间的流量将被卸载。
支持 IPv4 和 IPv6 单播 AFI/SAFI。
可以为这些前缀应用路由过滤,以限制此配置中潜在的安全影响。
主机 BGP 对等互连的 NVUE 配置示例
以下代码块显示了与地址为 45.3.0.4
和 2001:cafe:1ead::4
的主机建立对等连接的配置。BGP 会话可以使用 IPv4 或 IPv6 地址建立。
这些会话中的任何一个都可以支持 IPv4 单播和 IPv6 单播 AFI/SAFI。
与主机建立对等互连的 NVUE 配置
nv set vrf default router bgp autonomous-system 63642
nv set vrf default router bgp enable on
nv set vrf default router bgp neighbor 45.3.0.4 nexthop-connected-check off
nv set vrf default router bgp neighbor 45.3.0.4 peer-group dpu_host
nv set vrf default router bgp neighbor 45.3.0.4 type numbered
nv set vrf default router bgp neighbor 2001:cafe:1ead::4 nexthop-connected-check off
nv set vrf default router bgp neighbor 2001:cafe:1ead::4 peer-group dpu_host
nv set vrf default router bgp neighbor 2001:cafe:1ead::4 type numbered
nv set vrf default router bgp peer-group dpu_host address-family ipv4-unicast enable on
nv set vrf default router bgp peer-group dpu_host address-family ipv6-unicast enable on
nv set vrf default router bgp peer-group dpu_host remote-as external
主机 BGP 对等互连的扁平文件配置示例
以下代码块显示了与地址为 45.3.0.4
和 2001:cafe:1ead::4
的主机建立对等连接的配置。BGP 会话可以使用 IPv4 或 IPv6 地址建立。
frr.conf
文件
router bgp 63642
bgp router-id 27.0.0.4
bgp bestpath as-path multipath-relax
neighbor dpu_host peer-group
neighbor dpu_host remote-as external
neighbor dpu_host advertisement-interval 0
neighbor dpu_host timers 3 9
neighbor dpu_host timers connect 10
neighbor dpu_host disable-connected-check
neighbor fabric peer-group
neighbor fabric remote-as external
neighbor fabric advertisement-interval 0
neighbor fabric timers 3 9
neighbor fabric timers connect 10
neighbor 45.3.0.4 peer-group dpu_host
neighbor 2001:cafe:1ead::4 peer-group dpu_host
neighbor p0_if interface peer-group fabric
neighbor p1_if interface peer-group fabric
!
address-family ipv4 unicast
neighbor dpu_host activate
!
address-family ipv6 unicast
neighbor dpu_host activate
主机上的 FRR 配置示例
任何 BGP 实现都可以在主机上使用,以与 HBN 建立对等连接并通告端点。以下是使用 FRR BGP 的示例
主机上的 FRR 配置示例
bf2-s12# sh run
Building configuration...
Current configuration:
!
frr version 7.2.1
frr defaults traditional
hostname bf2-s12
no ip forwarding
no ipv6 forwarding
!
router bgp 1000008
!
router bgp 1000008 vrf v_200_2000
neighbor 45.3.0.2 remote-as external
neighbor 2001:cafe:1ead::2 remote-as external
!
address-family ipv4 unicast
redistribute connected
exit-address-family
!
address-family ipv6 unicast
redistribute connected
neighbor 45.3.0.2 activate
neighbor 2001:cafe:1ead::2 activate
exit-address-family
!
line vty
!
end
主机上的接口配置示例
root@bf2-s12:/home/cumulus# ifquery -a
auto lo
iface lo inet loopback
address 27.0.0.7/32
address 2001:c000:10ff:f00d::7/128
auto v_200_2000
iface v_200_2000
address 60.1.0.1
address 60.1.0.2
address 60.1.0.3
address 2001:60:1::1
address 2001:60:1::2
address 2001:60:1::3
vrf-table auto
auto ens1f0np0
iface ens1f0np0
address 45.3.0.4/24
address 2001:cafe:1ead::4/64
gateway 45.3.0.1
gateway 2001:cafe:1ead::1
vrf v_200_2000
hwaddress 00:03:00:08:00:12
mtu 9162
VRF 路由泄漏
当需要多个独立的路由和转发表时,通常使用 VRF。但是,用户可能希望从一个 VRF 到达另一个 VRF 中的目标,如下列情况所示
使服务(例如防火墙)可用于多个 VRF
为多个 VRF 启用到外部网络或 Internet 的路由,其中外部网络本身可以通过特定的 VRF 到达
路由泄漏可用于到达远程目标以及另一个 VRF 中直接连接的目标。多个 VRF 可以从单个源 VRF 导入路由,并且一个 VRF 可以从多个源 VRF 导入路由。当单个 VRF 为其他 VRF 提供到外部网络或共享服务的连接时,可以使用此功能。可以使用路由图控制跨 VRF 动态泄漏的路由。
当使用路由泄漏时
redistribute 命令(而不是 network 命令)必须在 BGP 中使用,以泄漏非 BGP 路由(连接路由或静态路由)
无法在默认 VRF 和非默认 VRF 之间泄漏路由
3769309 – 即使在这两个 VRF 之间启用了 VRF 路由泄漏,从 vrfX 中本地连接的主机到 vrfY 中 BlueField/HBN 上的本地接口 IP 地址的 Ping 或其他 IP 流量也无法工作。
在以下示例命令中,VRF BLUE 的 BGP 路由表中的路由动态泄漏到 VRF RED 中
nv set vrf RED router bgp address-family ipv4-unicast route-import
from-vrf list BLUE
nv config apply
以下示例命令从 VRF BLUE 中删除泄漏到 VRF RED 的路由
nv unset vrf RED router bgp address-family ipv4-unicast route-import
from-vrf list BLUE
nv config apply
要从导入过程中排除某些前缀,请在路由图中配置这些前缀。
以下示例配置路由图以匹配源协议 BGP,并将路由从 VRF BLUE 导入到 VRF RED。对于导入的路由,VRF RED 中的 community 为 11:11。
nv set vrf RED router bgp address-family ipv4-unicast route-import
from-vrf list BLUE
nv set router policy route-map BLUEtoRED rule 10
match type ipv4
nv set router policy route-map BLUEtoRED rule 10
match source-protocol bgp
nv set router policy route-map BLUEtoRED rule 10
action permit
nv set router policy route-map BLUEtoRED rule 10
set community 11
:11
nv set vrf RED router bgp address-family ipv4-unicast route-import
from-vrf route-map BLUEtoRED
nv config
要检查 VRF 路由泄漏的状态,请运行
NVUE 命令
nv show vrf <vrf-name> router bgp address-family ipv4-unicast route-
import
Vtysh 命令
show ip bgp vrf <vrf-name> ipv4|ipv6 unicast route-leak command.
例如
nv show vrf RED router bgp address-family ipv4-unicast route-
import
operational applied -------------- ------------ --------- from-vrf enable on route-map BLUEtoRED [list] BLUE BLUE [route-target]10.10
.10.1
:3
要显示更详细的状态信息,可以使用以下 NVUE 命令
nv show vrf <vrf-name> router bgp address-family ipv4-unicast route-import from-vrf
nv show vrf <vrf-name> router bgp address-family ipv4-unicast route-import from-vrf list
nv show vrf <vrf-name> router bgp address-family ipv4-unicast route-import from-vrf list <leak-vrf-id>
要查看 BGP 路由表,请运行
NVUE 命令
nv show vrf <vrf-name> router bgp address-family ipv4-unicast
Vtysh 命令
show ip bgp vrf <vrf-name> ipv4|ipv6 unicast
要查看 FRR IP 路由表,请运行
Vtysh 命令
show ip route vrf <vrf-name>
或
net show route vrf <vrf-name>
信息这些命令显示所有路由,包括从其他 VRF 泄漏的路由。
VLAN 子接口
VLAN 子接口是接口上的 VLAN 设备。VLAN ID 使用点 (.
) VLAN 表示法附加到父接口,这是在 Linux 中指定 VLAN 设备的标准方法。
例如
ID 为 100 的 VLAN(它是
p0_if
的子接口)被注释为p0_if.100
子接口
p0_if.100
仅接收在端口p0_if
上具有 VLAN 100 标记的数据包从
p0_if.100
传输的任何数据包都将具有 VLAN 标记 100
在 HBN 中,VLAN 子接口可以在 Uplink 端口以及面向主机的 PF 和 VF 端口上创建。VLAN 子接口仅接收标记为该 VLAN 的流量。
VLAN 子接口是 L3 接口,不应添加到网桥。
在以下示例中,创建了 VLAN ID 为 10 的 p0_if
上的 Uplink 子接口和 VLAN ID 为 999 的 VF 端口 pf1vf0_if
上的面向主机的子接口。面向主机的子接口也分配了 IPv4 和 IPv6 地址。
使用 NVUE 命令进行子接口配置
nv set interface p0_if.10 base-interface p0_if
nv set interface p0_if.10 type sub
nv set interface p0_if.10 vlan 10
nv set interface pf1vf0_if type swp
nv set interface pf1vf0_if.999 base-interface pf1vf0_if
nv set interface pf1vf0_if.999 type sub
nv set interface pf1vf0_if.999 vlan 999
nv set interface pf1vf0_if ip address 30.30.14.1/24
nv set interface pf1vf0_if ip address 2030:30:14::1/64
使用 /etc/network/interfaces
中的示例扁平文件进行相同配置
子接口配置 e/n/i 文件
auto p0_if.10
iface p0_if.10
auto pf1vf0_if.999
iface pf1vf0_if.999
address 2030:30:40::1/64
address 30.30.40.1/24
以太网虚拟专用网络 – EVPN
HBN 支持具有 EVPN 控制平面的 VXLAN,用于覆盖网络中 IPv4 和 IPv6 流量的子网内桥接 (L2) 服务。
对于底层网络,仅支持 IPv4 或 BGP 无编号配置。
HBN 仅支持通过 Uplink 父接口进行 VXLAN 封装。
单个 VXLAN 设备
使用单个 VXLAN 设备,一组 VXLAN 网络标识符 (VNI) 代表单个设备模型。单个 VXLAN 设备具有一组属于 VXLAN 构造的属性。单个 VNI 包括 VLAN 到 VNI 的映射,这允许用户指定哪些 VLAN 与哪些 VNI 关联。单个 VXLAN 设备通过用单个 VXLAN 设备替换多个传统 VXLAN 设备来简化配置并减少开销。
用户可以使用 NVUE 自动配置单个 VXLAN 设备,或通过手动编辑 /etc/network/interfaces
文件进行配置。当用户使用 NVUE 配置单个 VXLAN 设备时,NVUE 使用网桥名称作为哈希键,以下列格式为设备创建唯一名称:vxlan<id>
。
此示例配置执行以下步骤
创建单个 VXLAN 设备 (vxlan21)。
将 VLAN 10 映射到 VNI 10,并将 VLAN 20 映射到 VNI 20。
将 VXLAN 设备添加到默认网桥。
cumulus@leaf01:~$ nv set bridge domain bridge vlan 10 vni 10
cumulus@leaf01:~$ nv set bridge domain bridge vlan 20 vni 20
cumulus@leaf01:~$ nv set nve vxlan source address 10.10.10.1
cumulus@leaf01:~$ nv config apply
或者,用户可以按如下方式编辑文件 /etc/network/interfaces
,然后运行 ifreload -a
命令以应用 SVD 配置。
auto lo
iface lo inet loopback
vxlan-local-tunnelip 10.10.10.1
auto vxlan21
iface vxlan21
bridge-vlan-vni-map 10=10 20=20
bridge-learning off
auto bridge
iface bridge
bridge-vlan-aware yes
bridge-ports vxlan21 pf0hpf_if pf1hpf_if
bridge-vids 10 20
bridge-pvid 1
用户不得同时使用单个和传统 VXLAN 设备。
EVPN 的交换机配置示例
以下是底层交换机(NVIDIA® Spectrum® 与 Cumulus Linux)的 NVUE 配置示例,用于启用使用 HBN 的 EVPN 部署。
它假定所有 BlueField 设备上的 Uplink 都连接到交换机上的端口 swp1-4
。
nv set evpn enable on
nv set router bgp enable on
nv set vrf default router bgp address-family ipv4-unicast enable on
nv set vrf default router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf default router bgp address-family l2vpn-evpn enable on
nv set vrf default router bgp autonomous-system 63640
nv set vrf default router bgp enable on
nv set vrf default router bgp neighbor swp1 peer-group fabric
nv set vrf default router bgp neighbor swp1 type unnumbered
nv set vrf default router bgp neighbor swp2 peer-group fabric
nv set vrf default router bgp neighbor swp2 type unnumbered
nv set vrf default router bgp neighbor swp3 peer-group fabric
nv set vrf default router bgp neighbor swp3 type unnumbered
nv set vrf default router bgp neighbor swp4 peer-group fabric
nv set vrf default router bgp neighbor swp4 type unnumbered
nv set vrf default router bgp path-selection multipath aspath-ignore on
nv set vrf default router bgp peer-group fabric address-family ipv4-unicast enable on
nv set vrf default router bgp peer-group fabric address-family ipv6-unicast enable on
nv set vrf default router bgp peer-group fabric address-family l2vpn-evpn add-path-tx off
nv set vrf default router bgp peer-group fabric address-family l2vpn-evpn enable on
nv set vrf default router bgp peer-group fabric remote-as external
nv set vrf default router bgp router-id 27.0.0.10
nv set interface lo ip address 2001:c000:10ff:f00d::10/128
nv set interface lo ip address 27.0.0.10/32
nv set interface lo type loopback
nv set interface swp1,swp2,swp3,swp4 type swp
二层 EVPN
使用面向主机的端口作为接入端口的 L2 EVPN 的 NVUE 配置示例
以下是 NVUE 配置示例,其中包含用于 BlueField 上 EVPN 桥接的 L2-VNI (2000
, 2001
)。
nv set bridge domain br_default encap 802.1Q
nv set bridge domain br_default type vlan-aware
nv set bridge domain br_default vlan 200 vni 2000 flooding enable auto
nv set bridge domain br_default vlan 200 vni 2000 mac-learning off
nv set bridge domain br_default vlan 201 vni 2001 flooding enable auto
nv set bridge domain br_default vlan 201 vni 2001 mac-learning off
nv set evpn enable on
nv set nve vxlan arp-nd-suppress on
nv set nve vxlan enable on
nv set nve vxlan mac-learning off
nv set nve vxlan source address 27.0.0.4
nv set router bgp enable on
nv set system global anycast-mac 44:38:39:42:42:07
nv set vrf default router bgp address-family ipv4-unicast enable on
nv set vrf default router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf default router bgp address-family l2vpn-evpn enable on
nv set vrf default router bgp autonomous-system 63642
nv set vrf default router bgp enable on
nv set vrf default router bgp neighbor p0_if peer-group fabric
nv set vrf default router bgp neighbor p0_if type unnumbered
nv set vrf default router bgp neighbor p1_if peer-group fabric
nv set vrf default router bgp neighbor p1_if type unnumbered
nv set vrf default router bgp path-selection multipath aspath-ignore on
nv set vrf default router bgp peer-group fabric address-family ipv4-unicast enable on
nv set vrf default router bgp peer-group fabric address-family ipv4-unicast policy outbound route-map MY_ORIGIN_ASPATH_ONLY
nv set vrf default router bgp peer-group fabric address-family ipv6-unicast enable on
nv set vrf default router bgp peer-group fabric address-family ipv6-unicast policy outbound route-map MY_ORIGIN_ASPATH_ONLY
nv set vrf default router bgp peer-group fabric address-family l2vpn-evpn add-path-tx off
nv set vrf default router bgp peer-group fabric address-family l2vpn-evpn enable on
nv set vrf default router bgp peer-group fabric remote-as external
nv set vrf default router bgp router-id 27.0.0.4
nv set interface lo ip address 2001:c000:10ff:f00d::4/128
nv set interface lo ip address 27.0.0.4/32
nv set interface lo type loopback
nv set interface p0_if,p1_if,pf0hpf_if,pf1hpf_if type swp
nv set interface pf0hpf_if bridge domain br_default access 200
nv set interface pf1hpf_if bridge domain br_default access 201
nv set interface vlan200-201 base-interface br_default
nv set interface vlan200-201 ip ipv4 forward on
nv set interface vlan200-201 ip ipv6 forward on
nv set interface vlan200-201 ip vrr enable on
nv set interface vlan200-201 ip vrr state up
nv set interface vlan200-201 link mtu 9050
nv set interface vlan200-201 type svi
nv set interface vlan200 ip address 2001:cafe:1ead::3/64
nv set interface vlan200 ip address 45.3.0.2/24
nv set interface vlan200 ip vrr address 2001:cafe:1ead::1/64
nv set interface vlan200 ip vrr address 45.3.0.1/24
nv set interface vlan200 vlan 200
nv set interface vlan201 ip address 2001:cafe:1ead:1::3/64
nv set interface vlan201 ip address 45.3.1.2/24
nv set interface vlan201 ip vrr address 2001:cafe:1ead:1::1/64
nv set interface vlan201 ip vrr address 45.3.1.1/24
nv set interface vlan201 vlan 201
使用面向主机的端口作为 Trunk 端口的 L2 EVPN 的 NVUE 配置示例
以下是 NVUE 配置示例,其中包含用于 BlueField 上 EVPN 桥接的 L2-VNI (2000
, 2001
)。
nv set bridge domain br_default encap 802.1Q
nv set bridge domain br_default type vlan-aware
nv set bridge domain br_default vlan 200 vni 2000 flooding enable auto
nv set bridge domain br_default vlan 200 vni 2000 mac-learning off
nv set bridge domain br_default vlan 201 vni 2001 flooding enable auto
nv set bridge domain br_default vlan 201 vni 2001 mac-learning off
nv set evpn enable on
nv set nve vxlan arp-nd-suppress on
nv set nve vxlan enable on
nv set nve vxlan mac-learning off
nv set nve vxlan source address 27.0.0.4
nv set router bgp enable on
nv set system global anycast-mac 44:38:39:42:42:07
nv set vrf default router bgp address-family ipv4-unicast enable on
nv set vrf default router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf default router bgp address-family l2vpn-evpn enable on
nv set vrf default router bgp autonomous-system 63642
nv set vrf default router bgp enable on
nv set vrf default router bgp neighbor p0_if peer-group fabric
nv set vrf default router bgp neighbor p0_if type unnumbered
nv set vrf default router bgp neighbor p1_if peer-group fabric
nv set vrf default router bgp neighbor p1_if type unnumbered
nv set vrf default router bgp path-selection multipath aspath-ignore on
nv set vrf default router bgp peer-group fabric address-family ipv4-unicast enable on
nv set vrf default router bgp peer-group fabric address-family ipv4-unicast policy outbound route-map MY_ORIGIN_ASPATH_ONLY
nv set vrf default router bgp peer-group fabric address-family ipv6-unicast enable on
nv set vrf default router bgp peer-group fabric address-family ipv6-unicast policy outbound route-map MY_ORIGIN_ASPATH_ONLY
nv set vrf default router bgp peer-group fabric address-family l2vpn-evpn add-path-tx off
nv set vrf default router bgp peer-group fabric address-family l2vpn-evpn enable on
nv set vrf default router bgp peer-group fabric remote-as external
nv set vrf default router bgp router-id 27.0.0.4
nv set interface lo ip address 2001:c000:10ff:f00d::4/128
nv set interface lo ip address 27.0.0.4/32
nv set interface lo type loopback
nv set interface p0_if,p1_if,pf0hpf_if,pf1hpf_if type swp
nv set interface pf0hpf_if bridge domain br_default
nv set interface pf1hpf_if bridge domain br_default
nv set interface vlan200-201 base-interface br_default
nv set interface vlan200-201 ip ipv4 forward on
nv set interface vlan200-201 ip ipv6 forward on
nv set interface vlan200-201 ip vrr enable on
nv set interface vlan200-201 ip vrr state up
nv set interface vlan200-201 link mtu 9050
nv set interface vlan200-201 type svi
nv set interface vlan200 ip address 2001:cafe:1ead::3/64
nv set interface vlan200 ip address 45.3.0.2/24
nv set interface vlan200 ip vrr address 2001:cafe:1ead::1/64
nv set interface vlan200 ip vrr address 45.3.0.1/24
nv set interface vlan200 vlan 200
nv set interface vlan201 ip address 2001:cafe:1ead:1::3/64
nv set interface vlan201 ip address 45.3.1.2/24
nv set interface vlan201 ip vrr address 2001:cafe:1ead:1::1/64
nv set interface vlan201 ip vrr address 45.3.1.1/24
nv set interface vlan201 vlan 201
使用面向主机的端口作为接入端口的 L2 EVPN 的扁平文件配置示例
以下是扁平文件配置示例,其中包含用于 BlueField 上 EVPN 桥接的 L2-VNI (vx-2000
, vx-2001
)。
此文件位于 /etc/network/interfaces
auto lo
iface lo inet loopback
address 2001:c000:10ff:f00d::4/128
address 27.0.0.4/32
vxlan-local-tunnelip 27.0.0.4
auto p0_if
iface p0_if
auto p1_if
iface p1_if
auto pf0hpf_if
iface pf0hpf_if
bridge-access 200
auto pf1hpf_if
iface pf1hpf_if
bridge-access 201
auto vlan200
iface vlan200
address 2001:cafe:1ead::3/64
address 45.3.0.2/24
mtu 9050
address-virtual 00:00:5e:00:01:01 2001:cafe:1ead::1/64 45.3.0.1/24
vlan-raw-device br_default
vlan-id 200
auto vlan201
iface vlan201
address 2001:cafe:1ead:1::3/64
address 45.3.1.2/24
mtu 9050
address-virtual 00:00:5e:00:01:01 2001:cafe:1ead:1::1/64 45.3.1.1/24
vlan-raw-device br_default
vlan-id 201
auto vxlan48
iface vxlan48
bridge-vlan-vni-map 200=2000 201=2001
217=2017
bridge-learning off
auto br_default
iface br_default
bridge-ports pf0hpf_if pf1hpf_if vxlan48
bridge-vlan-aware yes
bridge-vids 200 201
bridge-pvid 1
使用面向主机的端口作为 Trunk 端口的 L2 EVPN 的扁平文件配置示例
以下是扁平文件配置示例,其中包含用于 BlueField 上 EVPN 桥接的 L2-VNI (vx-2000
, vx-2001
)。
此文件位于 /etc/network/interfaces
auto lo
iface lo inet loopback
address 2001:c000:10ff:f00d::4/128
address 27.0.0.4/32
vxlan-local-tunnelip 27.0.0.4
auto p0_if
iface p0_if
auto p1_if
iface p1_if
auto pf0hpf_if
iface pf0hpf_if
auto pf1hpf_if
iface pf1hpf_if
auto vlan200
iface vlan200
address 2001:cafe:1ead::3/64
address 45.3.0.2/24
mtu 9050
address-virtual 00:00:5e:00:01:01 2001:cafe:1ead::1/64 45.3.0.1/24
vlan-raw-device br_default
vlan-id 200
auto vlan201
iface vlan201
address 2001:cafe:1ead:1::3/64
address 45.3.1.2/24
mtu 9050
address-virtual 00:00:5e:00:01:01 2001:cafe:1ead:1::1/64 45.3.1.1/24
vlan-raw-device br_default
vlan-id 201
auto vxlan48
iface vxlan48
bridge-vlan-vni-map 200=2000 201=2001
217=2017
bridge-learning off
auto br_default
iface br_default
bridge-ports pf0hpf_if pf1hpf_if vxlan48
bridge-vlan-aware yes
bridge-vids 200 201
bridge-pvid 1
此文件告知 frr
软件包要启动哪个守护程序,并且位于 /etc/frr/daemons
bgpd=yes
ospfd=no
ospf6d=no
isisd=no
pimd=no
ldpd=no
pbrd=no
vrrpd=no
fabricd=no
nhrpd=no
eigrpd=no
babeld=no
sharpd=no
fabricd=no
ripngd=no
ripd=no
vtysh_enable=yes
zebra_options=" -M cumulus_mlag -M snmp -A 127.0.0.1 -s 90000000"
bgpd_options=" -M snmp -A 127.0.0.1"
ospfd_options=" -M snmp -A 127.0.0.1"
ospf6d_options=" -M snmp -A ::1"
ripd_options=" -A 127.0.0.1"
ripngd_options=" -A ::1"
isisd_options=" -A 127.0.0.1"
pimd_options=" -A 127.0.0.1"
ldpd_options=" -A 127.0.0.1"
nhrpd_options=" -A 127.0.0.1"
eigrpd_options=" -A 127.0.0.1"
babeld_options=" -A 127.0.0.1"
sharpd_options=" -A 127.0.0.1"
pbrd_options=" -A 127.0.0.1"
staticd_options="-A 127.0.0.1"
fabricd_options="-A 127.0.0.1"
vrrpd_options=" -A 127.0.0.1"
frr_profile="datacenter"
FRR 配置文件位于 /etc/frr/frr.conf
!---- Cumulus Defaults ----
frr defaults datacenter
log syslog informational
no zebra nexthop kernel enable
vrf default
outer bgp 63642 vrf default
bgp router-id 27.0.0.4
bgp bestpath as-path multipath-relax
timers bgp 3 9
bgp deterministic-med
! Neighbors
neighbor fabric peer-group
neighbor fabric remote-as external
neighbor fabric timers 3 9
neighbor fabric timers connect 10
neighbor fabric advertisement-interval 0
neighbor p0_if interface peer-group fabric
neighbor p1_if interface peer-group fabric
address-family ipv4 unicast
maximum-paths ibgp 64
maximum-paths 64
distance bgp 20 200 200
neighbor fabric activate
exit-address-family
address-family ipv6 unicast
maximum-paths ibgp 64
maximum-paths 64
distance bgp 20 200 200
neighbor fabric activate
exit-address-family
address-family l2vpn evpn
advertise-all-vni
neighbor fabric activate
exit-address-family
具有对称路由的三层 EVPN
在分布式对称路由中,每个 VXLAN 端点 (VTEP) 都充当三层网关,为其连接的主机执行路由。但是,入口 VTEP 和出口 VTEP 都会路由数据包(类似于路由到下一跳路由器的传统路由行为)。在 VXLAN 封装的数据包中,内部目标 MAC 地址是出口 VTEP 的路由器 MAC 地址,以指示出口 VTEP 是下一跳,并且它也必须执行路由。
所有路由都在租户 (VRF) 的上下文中发生。对于入口 VTEP 从本地连接的主机接收的数据包,与 VLAN 对应的 SVI 接口确定 VRF。对于出口 VTEP 通过 VXLAN 隧道接收的数据包,数据包中的 VNI 必须指定 VRF。对于对称路由,这是一个与租户对应的 VNI,并且与源 VNI 或目标 VNI 都不同。此 VNI 是三层 VNI 或互连 VNI。映射 VLAN 的常规 VNI 是二层 VNI。
有关此的更多详细信息,请参阅 Cumulus Linux 用户手册。
HBN 在 L3 VNI 和租户 (VRF) 之间使用一对一映射。
VRF 到 L3 VNI 的映射在所有 VTEP 中必须保持一致。
L3 VNI 和 L2 VNI 不能具有相同的 ID。
在 EVPN 对称路由配置中,当交换机通告类型 2 (MAC/IP) 路由时,除了包含两个 VNI(L2 和 L3 VNI)外,该路由还包含 L2 和 L3 的单独路由目标 (RT)。L3 RT 将路由与租户 VRF 关联。默认情况下,这是使用 L3 VNI 而不是 L2 VNI 自动派生的。但是,这是可配置的。
对于 EVPN 对称路由,用户必须执行以下小节中列出的配置。可选配置包括为租户 VRF 配置路由区分符 (RD) 和 RT,以及通告本地连接的子网。
使用面向主机的端口作为接入端口的 L3 EVPN 的 NVUE 配置示例
如果使用 NVUE 配置 EVPN 对称路由,则以下是使用 NVUE 命令的配置示例
nv set bridge domain br_default vlan 111
vni 1000111
nv set bridge domain br_default vlan 112
vni 1000112
nv set bridge domain br_default vlan 213
vni 1000213
nv set bridge domain br_default vlan 214
vni 1000214
nv set evpn enable on
nv set interface
lo ip address 6.0
.0.19
/32
nv set interface
lo type loopback
nv set interface
p0_if description 'alias p0_if to leaf-21 swp3'
nv set interface
p0_if,p1_if,pf0hpf_if,pf0vf0_if,pf1hpf_if,pf1vf0_if type swp
nv set interface
p1_if description 'alias p1_if to leaf-22 swp3'
nv set interface
pf0hpf_if bridge domain br_default access 111
nv set interface
pf0hpf_if description 'alias pf0hpf_if to host-211 ens2f0np0'
nv set interface
pf0vf0_if bridge domain br_default access 112
nv set interface
pf0vf0_if description 'alias pf0vf0_if to host-211 ens2f0np0v0'
nv set interface
pf1hpf_if bridge domain br_default access 213
nv set interface
pf1hpf_if description 'alias pf1hpf_if to host-211 ens2f1np1'
nv set interface
pf1vf0_if bridge domain br_default access 214
nv set interface
pf1vf0_if description 'alias pf1vf0_if to host-211 ens2f1np0v0'
nv set interface
vlan111 ip address 60.1
.1.21
/24
nv set interface
vlan111 ip address 2060
:1
:1
:1
::21
/64
nv set interface
vlan111 ip vrr address 60.1
.1.250
/24
nv set interface
vlan111 ip vrr address 2060
:1
:1
:1
::250
/64
nv set interface
vlan111 vlan 111
nv set interface
vlan111,213
ip vrf vrf2
nv set interface
vlan111-112
,213
-214
ip vrr enable on
nv set interface
vlan111-112
,213
-214
ip vrr mac-address 00
:00
:5e:00
:01
:01
nv set interface
vlan111-112
,213
-214
ip ipv4 forward on
nv set interface
vlan111-112
,213
-214
ip ipv6 forward on
nv set interface
vlan111-112
,213
-214
type svi
nv set interface
vlan112 ip address 50.1
.1.21
/24
nv set interface
vlan112 ip address 2050
:1
:1
:1
::21
/64
nv set interface
vlan112 ip vrr address 50.1
.1.250
/24
nv set interface
vlan112 ip vrr address 2050
:1
:1
:1
::250
/64
nv set interface
vlan112 vlan 112
nv set interface
vlan112,214
ip vrf vrf1
nv set interface
vlan213 ip address 60.1
.210.21
/24
nv set interface
vlan213 ip address 2060
:1
:1
:210
::21
/64
nv set interface
vlan213 ip vrr address 60.1
.210.250
/24
nv set interface
vlan213 ip vrr address 2060
:1
:1
:210
::250
/64
nv set interface
vlan213 vlan 213
nv set interface
vlan214 ip address 50.1
.210.21
/24
nv set interface
vlan214 ip address 2050
:1
:1
:210
::21
/64
nv set interface
vlan214 ip vrr address 50.1
.210.250
/24
nv set interface
vlan214 ip vrr address 2050
:1
:1
:210
::250
/64
nv set interface
vlan214 vlan 214
nv set nve vxlan arp-nd-suppress on
nv set nve vxlan enable on
nv set nve vxlan source address 6.0
.0.19
nv set platform
nv set router bgp enable on
nv set router policy route-map ALLOW_LOBR rule 10
action permit
nv set router policy route-map ALLOW_LOBR rule 10
match interface
lo
nv set router policy route-map ALLOW_LOBR rule 20
action permit
nv set router policy route-map ALLOW_LOBR rule 20
match interface
br_default
nv set router policy route-map ALLOW_VRF1 rule 10
action permit
nv set router policy route-map ALLOW_VRF1 rule 10
match interface
vrf1
nv set router policy route-map ALLOW_VRF2 rule 10
action permit
nv set router policy route-map ALLOW_VRF2 rule 10
match interface
vrf2
nv set router vrr enable on
nv set system global system-mac 00
:01
:00
:00
:1e:03
nv set vrf default
router bgp address-family ipv4-unicast enable on
nv set vrf default
router bgp address-family ipv4-unicast multipaths ebgp 16
nv set vrf default
router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf default
router bgp address-family ipv4-unicast redistribute connected route-map ALLOW_LOBR
nv set vrf default
router bgp address-family l2vpn-evpn enable on
nv set vrf default
router bgp autonomous-system 650019
nv set vrf default
router bgp enable on
nv set vrf default
router bgp neighbor p0_if address-family l2vpn-evpn add-path-tx off
nv set vrf default
router bgp neighbor p0_if address-family l2vpn-evpn enable on
nv set vrf default
router bgp neighbor p0_if peer-group TOR_LEAF_SPINE
nv set vrf default
router bgp neighbor p0_if remote-as external
nv set vrf default
router bgp neighbor p0_if type unnumbered
nv set vrf default
router bgp neighbor p1_if address-family l2vpn-evpn add-path-tx off
nv set vrf default
router bgp neighbor p1_if address-family l2vpn-evpn enable on
nv set vrf default
router bgp neighbor p1_if peer-group TOR_LEAF_SPINE
nv set vrf default
router bgp neighbor p1_if remote-as external
nv set vrf default
router bgp neighbor p1_if type unnumbered
nv set vrf default
router bgp path-selection multipath aspath-ignore on
nv set vrf default
router bgp path-selection routerid-compare on
nv set vrf default
router bgp peer-group TOR_LEAF_SPINE address-family ipv4-unicast enable on
nv set vrf default
router bgp router-id 6.0
.0.19
nv set vrf vrf1 evpn enable on
nv set vrf vrf1 evpn vni 104001
nv set vrf vrf1 loopback ip address 50.1
.21.21
/32
nv set vrf vrf1 loopback ip address 2050
:50
:50
:21
::21
/128
nv set vrf vrf1 router bgp address-family ipv4-unicast enable on
nv set vrf vrf1 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf vrf1 router bgp address-family ipv4-unicast redistribute connected route-map ALLOW_VRF1
nv set vrf vrf1 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf vrf1 router bgp address-family ipv6-unicast enable on
nv set vrf vrf1 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf vrf1 router bgp address-family ipv6-unicast redistribute connected route-map ALLOW_VRF1
nv set vrf vrf1 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf vrf1 router bgp autonomous-system 650019
nv set vrf vrf1 router bgp enable on
nv set vrf vrf1 router bgp router-id 50.1
.21.21
nv set vrf vrf2 evpn enable on
nv set vrf vrf2 evpn vni 104002
nv set vrf vrf2 loopback ip address 60.1
.21.21
/32
nv set vrf vrf2 loopback ip address 2060
:60
:60
:21
::21
/128
nv set vrf vrf2 router bgp address-family ipv4-unicast enable on
nv set vrf vrf2 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf vrf2 router bgp address-family ipv4-unicast redistribute connected route-map ALLOW_VRF2
nv set vrf vrf2 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf vrf2 router bgp address-family ipv6-unicast enable on
nv set vrf vrf2 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf vrf2 router bgp address-family ipv6-unicast redistribute connected route-map ALLOW_VRF2
nv set vrf vrf2 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf vrf2 router bgp autonomous-system 650019
nv set vrf vrf2 router bgp enable on
nv set vrf vrf2 router bgp router-id 60.1
.21.21
使用面向主机的端口作为 Trunk 端口的 L3 EVPN 的 NVUE 配置示例
如果使用 NVUE 配置 EVPN 对称路由,则以下是使用 NVUE 命令的配置示例
nv set bridge domain br_default vlan 111
vni 1000111
nv set bridge domain br_default vlan 112
vni 1000112
nv set bridge domain br_default vlan 213
vni 1000213
nv set bridge domain br_default vlan 214
vni 1000214
nv set evpn enable on
nv set interface
lo ip address 6.0
.0.19
/32
nv set interface
lo type loopback
nv set interface
p0_if description 'alias p0_if to leaf-21 swp3'
nv set interface
p0_if,p1_if,pf0hpf_if,pf0vf0_if,pf1hpf_if,pf1vf0_if type swp
nv set interface
p1_if description 'alias p1_if to leaf-22 swp3'
nv set interface
pf0hpf_if bridge domain br_default
nv set interface
pf0hpf_if description 'alias pf0hpf_if to host-211 ens2f0np0'
nv set interface
pf0vf0_if bridge domain br_default
nv set interface
pf0vf0_if description 'alias pf0vf0_if to host-211 ens2f0np0v0'
nv set interface
pf1hpf_if bridge domain br_default
nv set interface
pf1hpf_if description 'alias pf1hpf_if to host-211 ens2f1np1'
nv set interface
pf1vf0_if bridge domain br_default
nv set interface
pf1vf0_if description 'alias pf1vf0_if to host-211 ens2f1np0v0'
nv set interface
vlan111 ip address 60.1
.1.21
/24
nv set interface
vlan111 ip address 2060
:1
:1
:1
::21
/64
nv set interface
vlan111 ip vrr address 60.1
.1.250
/24
nv set interface
vlan111 ip vrr address 2060
:1
:1
:1
::250
/64
nv set interface
vlan111 vlan 111
nv set interface
vlan111,213
ip vrf vrf2
nv set interface
vlan111-112
,213
-214
ip vrr enable on
nv set interface
vlan111-112
,213
-214
ip vrr mac-address 00
:00
:5e:00
:01
:01
nv set interface
vlan111-112
,213
-214
ip ipv4 forward on
nv set interface
vlan111-112
,213
-214
ip ipv6 forward on
nv set interface
vlan111-112
,213
-214
type svi
nv set interface
vlan112 ip address 50.1
.1.21
/24
nv set interface
vlan112 ip address 2050
:1
:1
:1
::21
/64
nv set interface
vlan112 ip vrr address 50.1
.1.250
/24
nv set interface
vlan112 ip vrr address 2050
:1
:1
:1
::250
/64
nv set interface
vlan112 vlan 112
nv set interface
vlan112,214
ip vrf vrf1
nv set interface
vlan213 ip address 60.1
.210.21
/24
nv set interface
vlan213 ip address 2060
:1
:1
:210
::21
/64
nv set interface
vlan213 ip vrr address 60.1
.210.250
/24
nv set interface
vlan213 ip vrr address 2060
:1
:1
:210
::250
/64
nv set interface
vlan213 vlan 213
nv set interface
vlan214 ip address 50.1
.210.21
/24
nv set interface
vlan214 ip address 2050
:1
:1
:210
::21
/64
nv set interface
vlan214 ip vrr address 50.1
.210.250
/24
nv set interface
vlan214 ip vrr address 2050
:1
:1
:210
::250
/64
nv set interface
vlan214 vlan 214
nv set nve vxlan arp-nd-suppress on
nv set nve vxlan enable on
nv set nve vxlan source address 6.0
.0.19
nv set platform
nv set router bgp enable on
nv set router policy route-map ALLOW_LOBR rule 10
action permit
nv set router policy route-map ALLOW_LOBR rule 10
match interface
lo
nv set router policy route-map ALLOW_LOBR rule 20
action permit
nv set router policy route-map ALLOW_LOBR rule 20
match interface
br_default
nv set router policy route-map ALLOW_VRF1 rule 10
action permit
nv set router policy route-map ALLOW_VRF1 rule 10
match interface
vrf1
nv set router policy route-map ALLOW_VRF2 rule 10
action permit
nv set router policy route-map ALLOW_VRF2 rule 10
match interface
vrf2
nv set router vrr enable on
nv set system global system-mac 00
:01
:00
:00
:1e:03
nv set vrf default
router bgp address-family ipv4-unicast enable on
nv set vrf default
router bgp address-family ipv4-unicast multipaths ebgp 16
nv set vrf default
router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf default
router bgp address-family ipv4-unicast redistribute connected route-map ALLOW_LOBR
nv set vrf default
router bgp address-family l2vpn-evpn enable on
nv set vrf default
router bgp autonomous-system 650019
nv set vrf default
router bgp enable on
nv set vrf default
router bgp neighbor p0_if address-family l2vpn-evpn add-path-tx off
nv set vrf default
router bgp neighbor p0_if address-family l2vpn-evpn enable on
nv set vrf default
router bgp neighbor p0_if peer-group TOR_LEAF_SPINE
nv set vrf default
router bgp neighbor p0_if remote-as external
nv set vrf default
router bgp neighbor p0_if type unnumbered
nv set vrf default
router bgp neighbor p1_if address-family l2vpn-evpn add-path-tx off
nv set vrf default
router bgp neighbor p1_if address-family l2vpn-evpn enable on
nv set vrf default
router bgp neighbor p1_if peer-group TOR_LEAF_SPINE
nv set vrf default
router bgp neighbor p1_if remote-as external
nv set vrf default
router bgp neighbor p1_if type unnumbered
nv set vrf default
router bgp path-selection multipath aspath-ignore on
nv set vrf default
router bgp path-selection routerid-compare on
nv set vrf default
router bgp peer-group TOR_LEAF_SPINE address-family ipv4-unicast enable on
nv set vrf default
router bgp router-id 6.0
.0.19
nv set vrf vrf1 evpn enable on
nv set vrf vrf1 evpn vni 104001
nv set vrf vrf1 loopback ip address 50.1
.21.21
/32
nv set vrf vrf1 loopback ip address 2050
:50
:50
:21
::21
/128
nv set vrf vrf1 router bgp address-family ipv4-unicast enable on
nv set vrf vrf1 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf vrf1 router bgp address-family ipv4-unicast redistribute connected route-map ALLOW_VRF1
nv set vrf vrf1 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf vrf1 router bgp address-family ipv6-unicast enable on
nv set vrf vrf1 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf vrf1 router bgp address-family ipv6-unicast redistribute connected route-map ALLOW_VRF1
nv set vrf vrf1 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf vrf1 router bgp autonomous-system 650019
nv set vrf vrf1 router bgp enable on
nv set vrf vrf1 router bgp router-id 50.1
.21.21
nv set vrf vrf2 evpn enable on
nv set vrf vrf2 evpn vni 104002
nv set vrf vrf2 loopback ip address 60.1
.21.21
/32
nv set vrf vrf2 loopback ip address 2060
:60
:60
:21
::21
/128
nv set vrf vrf2 router bgp address-family ipv4-unicast enable on
nv set vrf vrf2 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf vrf2 router bgp address-family ipv4-unicast redistribute connected route-map ALLOW_VRF2
nv set vrf vrf2 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf vrf2 router bgp address-family ipv6-unicast enable on
nv set vrf vrf2 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf vrf2 router bgp address-family ipv6-unicast redistribute connected route-map ALLOW_VRF2
nv set vrf vrf2 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf vrf2 router bgp autonomous-system 650019
nv set vrf vrf2 router bgp enable on
nv set vrf vrf2 router bgp router-id 60.1
.21.21
将面向主机的接口作为接入端口的 L3 EVPN 的扁平文件配置示例
以下是扁平文件配置示例,其中包含用于 BlueField 上 EVPN 桥接和对称路由的 L2 VNI 和 L3 VNI。
此文件位于 /etc/network/interfaces
auto lo
iface lo inet loopback
address 6.0.0.19/32
vxlan-local-tunnelip 6.0.0.19
auto vrf1
iface vrf1
address 2050:50:50:21::21/128
address 50.1.21.21/32
vrf-table auto
auto vrf2
iface vrf2
address 2060:60:60:21::21/128
address 60.1.21.21/32
vrf-table auto
auto p0_if
iface p0_if
alias alias p0_if to leaf-21 swp3
auto p1_if
iface p1_if
alias alias p1_if to leaf-22 swp3
auto pf0hpf_if
iface pf0hpf_if
alias alias pf0hpf_if to host-211 ens2f0np0
bridge-access 111
auto pf0vf0_if
iface pf0vf0_if
alias alias pf0vf0_if to host-211 ens2f0np0v0
bridge-access 112
auto pf1hpf_if
iface pf1hpf_if
alias alias pf1hpf_if to host-211 ens2f1np1
bridge-access 213
auto pf1vf0_if
iface pf1vf0_if
alias alias pf1vf0_if to host-211 ens2f1np0v0
bridge-access 214
auto vlan111
iface vlan111
address 2060:1:1:1::21/64
address 60.1.1.21/24
address-virtual 00:00:5e:00:01:01 2060:1:1:1::250/64 60.1.1.250/24
hwaddress 00:01:00:00:1e:03
vrf vrf2
vlan-raw-device br_default
vlan-id 111
auto vlan112
iface vlan112
address 2050:1:1:1::21/64
address 50.1.1.21/24
address-virtual 00:00:5e:00:01:01 2050:1:1:1::250/64 50.1.1.250/24
hwaddress 00:01:00:00:1e:03
vrf vrf1
vlan-raw-device br_default
vlan-id 112
auto vlan213
iface vlan213
address 2060:1:1:210::21/64
address 60.1.210.21/24
address-virtual 00:00:5e:00:01:01 2060:1:1:210::250/64 60.1.210.250/24
hwaddress 00:01:00:00:1e:03
vrf vrf2
vlan-raw-device br_default
vlan-id 213
auto vlan214
iface vlan214
address 2050:1:1:210::21/64
address 50.1.210.21/24
address-virtual 00:00:5e:00:01:01 2050:1:1:210::250/64 50.1.210.250/24
hwaddress 00:01:00:00:1e:03
vrf vrf1
vlan-raw-device br_default
vlan-id 214
auto vlan4058_l3
iface vlan4058_l3
vrf vrf1
vlan-raw-device br_default
address-virtual none
vlan-id 4058
auto vlan4059_l3
iface vlan4059_l3
vrf vrf2
vlan-raw-device br_default
address-virtual none
vlan-id 4059
auto vxlan48
iface vxlan48
bridge-vlan-vni-map 111=1000111 112=1000112 213=1000213 214=1000214 4058=104001 4059=104002
bridge-learning off
auto br_default
iface br_default
bridge-ports pf0hpf_if pf0vf0_if pf1hpf_if pf1vf0_if vxlan48
hwaddress 00:01:00:00:1e:03
bridge-vlan-aware yes
bridge-vids 111 112 213 214
bridge-pvid 1
将面向主机的接口作为 Trunk 端口的 L3 EVPN 的扁平文件配置示例
以下是扁平文件配置示例,其中包含用于 BlueField 上 EVPN 桥接和对称路由的 L2 VNI 和 L3 VNI。
此文件位于 /etc/network/interfaces
auto lo
iface lo inet loopback
address 6.0.0.19/32
vxlan-local-tunnelip 6.0.0.19
auto vrf1
iface vrf1
address 2050:50:50:21::21/128
address 50.1.21.21/32
vrf-table auto
auto vrf2
iface vrf2
address 2060:60:60:21::21/128
address 60.1.21.21/32
vrf-table auto
auto p0_if
iface p0_if
alias alias p0_if to leaf-21 swp3
auto p1_if
iface p1_if
alias alias p1_if to leaf-22 swp3
auto pf0hpf_if
iface pf0hpf_if
alias alias pf0hpf_if to host-211 ens2f0np0
auto pf0vf0_if
iface pf0vf0_if
alias alias pf0vf0_if to host-211 ens2f0np0v0
auto pf1hpf_if
iface pf1hpf_if
alias alias pf1hpf_if to host-211 ens2f1np1
auto pf1vf0_if
iface pf1vf0_if
alias alias pf1vf0_if to host-211 ens2f1np0v0
auto vlan111
iface vlan111
address 2060:1:1:1::21/64
address 60.1.1.21/24
address-virtual 00:00:5e:00:01:01 2060:1:1:1::250/64 60.1.1.250/24
hwaddress 00:01:00:00:1e:03
vrf vrf2
vlan-raw-device br_default
vlan-id 111
auto vlan112
iface vlan112
address 2050:1:1:1::21/64
address 50.1.1.21/24
address-virtual 00:00:5e:00:01:01 2050:1:1:1::250/64 50.1.1.250/24
hwaddress 00:01:00:00:1e:03
vrf vrf1
vlan-raw-device br_default
vlan-id 112
auto vlan213
iface vlan213
address 2060:1:1:210::21/64
address 60.1.210.21/24
address-virtual 00:00:5e:00:01:01 2060:1:1:210::250/64 60.1.210.250/24
hwaddress 00:01:00:00:1e:03
vrf vrf2
vlan-raw-device br_default
vlan-id 213
auto vlan214
iface vlan214
address 2050:1:1:210::21/64
address 50.1.210.21/24
address-virtual 00:00:5e:00:01:01 2050:1:1:210::250/64 50.1.210.250/24
hwaddress 00:01:00:00:1e:03
vrf vrf1
vlan-raw-device br_default
vlan-id 214
auto vlan4058_l3
iface vlan4058_l3
vrf vrf1
vlan-raw-device br_default
address-virtual none
vlan-id 4058
auto vlan4059_l3
iface vlan4059_l3
vrf vrf2
vlan-raw-device br_default
address-virtual none
vlan-id 4059
auto vxlan48
iface vxlan48
bridge-vlan-vni-map 111=1000111 112=1000112 213=1000213 214=1000214 4058=104001 4059=104002
bridge-learning off
auto br_default
iface br_default
bridge-ports pf0hpf_if pf0vf0_if pf1hpf_if pf1vf0_if vxlan48
hwaddress 00:01:00:00:1e:03
bridge-vlan-aware yes
bridge-vids 111 112 213 214
bridge-pvid 1
FRR 配置位于 /etc/frr/frr.conf
frr version 8.4.3
frr defaults datacenter
hostname doca-hbn-service-bf3-s05-1-ipmi
log syslog informational
no zebra nexthop kernel enable
service integrated-vtysh-config
!
vrf vrf1
vni 104001
exit-vrf
!
vrf vrf2
vni 104002
exit-vrf
!
router bgp 650019
bgp router-id 6.0.0.19
bgp bestpath as-path multipath-relax
bgp bestpath compare-routerid
neighbor TOR_LEAF_SPINE peer-group
neighbor TOR_LEAF_SPINE advertisement-interval 0
neighbor TOR_LEAF_SPINE timers 3 9
neighbor TOR_LEAF_SPINE timers connect 10
neighbor p0_if interface peer-group TOR_LEAF_SPINE
neighbor p0_if remote-as external
neighbor p0_if advertisement-interval 0
neighbor p0_if timers 3 9
neighbor p0_if timers connect 10
neighbor p1_if interface peer-group TOR_LEAF_SPINE
neighbor p1_if remote-as external
neighbor p1_if advertisement-interval 0
neighbor p1_if timers 3 9
neighbor p1_if timers connect 10
!
address-family ipv4 unicast
redistribute connected route-map ALLOW_LOBR
maximum-paths 16
maximum-paths ibgp 64
exit-address-family
!
address-family l2vpn evpn
neighbor p0_if activate
neighbor p1_if activate
advertise-all-vni
exit-address-family
exit
!
router bgp 650019 vrf vrf1
bgp router-id 50.1.21.21
!
address-family ipv4 unicast
redistribute connected route-map ALLOW_VRF1
maximum-paths 64
maximum-paths ibgp 64
exit-address-family
!
address-family ipv6 unicast
redistribute connected route-map ALLOW_VRF1
maximum-paths 64
maximum-paths ibgp 64
exit-address-family
!
address-family l2vpn evpn
advertise ipv4 unicast
advertise ipv6 unicast
exit-address-family
exit
!
router bgp 650019 vrf vrf2
bgp router-id 60.1.21.21
!
address-family ipv4 unicast
redistribute connected route-map ALLOW_VRF2
maximum-paths 64
maximum-paths ibgp 64
exit-address-family
!
address-family ipv6 unicast
redistribute connected route-map ALLOW_VRF2
maximum-paths 64
maximum-paths ibgp 64
exit-address-family
!
address-family l2vpn evpn
advertise ipv4 unicast
advertise ipv6 unicast
exit-address-family
exit
!
route-map ALLOW_LOBR permit 10
match interface lo
exit
!
route-map ALLOW_LOBR permit 20
match interface br_default
exit
!
route-map ALLOW_VRF1 permit 10
match interface vrf1
exit
!
route-map ALLOW_VRF2 permit 10
match interface vrf2
exit
EVPN 的多跳 eBGP 对等连接(对称 EVPN 路由中的路由服务器)
在 EVPN 拓扑中,用于 EVPN 支持的 eBGP 多跳对等互连在类似路由服务器的角色中,允许在任何支持 IP 传输的云上部署 EVPN。
路由服务器和 BF/HBN VTEP 通过 IP 云连接。
即
云提供商中的交换机无需了解 EVPN
提供商网络结构中的交换机提供 IPv4 和 IPv6 传输,并且不必支持 EVPN
EVPN 的路由服务器配置示例
root@sn1
:/home/cumulus# uname -a
Linux sn1 5.15
.0
-88
-generic #98
-Ubuntu SMP Mon Oct 2
15
:18
:56
UTC 2023
x86_64 x86_64 x86_64 GNU/Linux
root@sn1
:/home/cumulus# dpkg -l frr
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-==============-=====================-============-=============================================================
ii frr 9.0
.1
-0
~ubuntu22.04.1
amd64 FRRouting suite of internet protocols (BGP, OSPF, IS-IS, ...)
root@sn1
:/home/cumulus#
以下是以 Ubuntu 服务器(运行 FRR 9.0 稳定版)的配置示例,该服务器配置为 EVPN 路由服务器和一个 HBN VTEP,该 HBN VTEP 与两个 spine 交换机建立 IP 连接对等互连,并与 3 个路由服务器建立 EVPN 覆盖控制对等互连。
sn1# sh run
Building configuration...
Current configuration:
!
frr version 9.0
.1
frr defaults datacenter
hostname sn1
no ip forwarding
no ipv6 forwarding
service integrated-vtysh-config
!
router bgp 4200065507
bgp router-id 6.0
.0.7
timers bgp 60
180
neighbor rclients peer-group
neighbor rclients remote-as external
neighbor rclients ebgp-multihop 10
neighbor rclients update-source lo
neighbor rclients advertisement-interval 0
neighbor rclients timers 3
9
neighbor rclients timers connect 10
neighbor rcsuper peer-group
neighbor rcsuper remote-as external
neighbor rcsuper advertisement-interval 0
neighbor rcsuper timers 3
9
neighbor rcsuper timers connect 10
neighbor swp1 interface
peer-group rcsuper
bgp listen range 6.0
.0.0
/24
peer-group rclients
!
address-family ipv4 unicast
redistribute connected
neighbor fabric route-map pass in
neighbor fabric route-map pass out
no neighbor rclients activate
maximum-paths 64
maximum-paths ibgp 64
exit-address-family
!
address-family l2vpn evpn
neighbor rclients activate
neighbor rcsuper activate
exit-address-family
exit
!
route-map pass permit 10
set community 11
:11
additive
exit
!
end
sn1#
FRR 配置 (frr.conf
)
root@sn1
:/home/cumulus# ifquery -a
auto lo
iface lo inet loopback
address 6.0
.0.7
/32
auto lo
iface lo inet loopback
auto swp1
iface swp1
auto eth0
iface eth0
address 192.168
.0.15
/24
gateway 192.168
.0.2
root@sn1
:/home/cumulus#
接口配置 (/etc/network/interfaces
)
root@doca
-hbn-service-bf2-s12-1
-ipmi:/tmp# nv config show -o commands
nv set bridge domain br_default vlan 101
vni 10101
nv set bridge domain br_default vlan 102
vni 10102
nv set bridge domain br_default vlan 201
vni 10201
nv set bridge domain br_default vlan 202
vni 10202
nv set evpn enable on
nv set evpn route-advertise svi-ip off
nv set interface
ilan3200 ip vrf internet1
nv set interface
ilan3200 vlan 3200
nv set interface
ilan3200,slan3201,vlan101-102
,201
-202
,3001
-3002
base-interface
br_default
nv set interface
ilan3200,slan3201,vlan101-102
,201
-202
,3001
-3002
type svi
nv set interface
lo ip address 6.0
.0.13
/32
nv set interface
lo ip address 2001
::13
/128
nv set interface
lo type loopback
nv set interface
p0_if,p1_if,pf0hpf_if,pf0vf0_if,pf0vf1_if,pf0vf2_if,pf0vf3_if,pf1hpf_if type swp
nv set interface
pf0vf0_if bridge domain br_default access 101
nv set interface
pf0vf1_if bridge domain br_default access 102
nv set interface
pf0vf2_if bridge domain br_default access 201
nv set interface
pf0vf3_if bridge domain br_default access 202
nv set interface
slan3201 ip vrf special1
nv set interface
slan3201 vlan 3201
nv set interface
vlan101 ip address 21.1
.0.13
/16
nv set interface
vlan101 ip address 2020
:0
:1
:1
::13
/64
nv set interface
vlan101 ip vrr address 21.1
.0.250
/16
nv set interface
vlan101 ip vrr address 2020
:0
:1
:1
::250
/64
nv set interface
vlan101 ip vrr mac-address 00
:00
:01
:00
:00
:65
nv set interface
vlan101 vlan 101
nv set interface
vlan101-102
,201
-202
ip vrr enable on
nv set interface
vlan101-102
,3001
ip vrf tenant1
nv set interface
vlan102 ip address 21.2
.0.13
/16
nv set interface
vlan102 ip address 2020
:0
:1
:2
::13
/64
nv set interface
vlan102 ip vrr address 21.2
.0.250
/16
nv set interface
vlan102 ip vrr address 2020
:0
:1
:2
::250
/64
nv set interface
vlan102 ip vrr mac-address 00
:00
:01
:00
:00
:66
nv set interface
vlan102 vlan 102
nv set interface
vlan201 ip address 22.1
.0.13
/16
nv set interface
vlan201 ip address 2020
:0
:2
:1
::13
/64
nv set interface
vlan201 ip vrr address 22.1
.0.250
/16
nv set interface
vlan201 ip vrr address 2020
:0
:2
:1
::250
/64
nv set interface
vlan201 ip vrr mac-address 00
:00
:02
:00
:00
:c9
nv set interface
vlan201 vlan 201
nv set interface
vlan201-202
,3002
ip vrf tenant2
nv set interface
vlan202 ip address 22.2
.0.13
/16
nv set interface
vlan202 ip address 2020
:0
:2
:2
::13
/64
nv set interface
vlan202 ip vrr address 22.2
.0.250
/16
nv set interface
vlan202 ip vrr address 2020
:0
:2
:2
::250
/64
nv set interface
vlan202 ip vrr mac-address 00
:00
:02
:00
:00
:ca
nv set interface
vlan202 vlan 202
nv set interface
vlan3001 vlan 3001
nv set interface
vlan3002 vlan 3002
nv set nve vxlan arp-nd-suppress on
nv set nve vxlan enable on
nv set nve vxlan source address 6.0
.0.13
nv set platform
nv set router bgp autonomous-system 4200065011
nv set router bgp enable on
nv set router bgp router-id 6.0
.0.13
nv set router vrr enable on
nv set system config snippet
nv set system global
nv set vrf default
router bgp address-family ipv4-unicast enable on
nv set vrf default
router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf default
router bgp address-family ipv6-unicast enable on
nv set vrf default
router bgp address-family l2vpn-evpn enable on
nv set vrf default
router bgp enable on
nv set vrf default
router bgp neighbor 6.0
.0.7
peer-group rservers
nv set vrf default
router bgp neighbor 6.0
.0.7
type numbered
nv set vrf default
router bgp neighbor 6.0
.0.8
peer-group rservers
nv set vrf default
router bgp neighbor 6.0
.0.8
type numbered
nv set vrf default
router bgp neighbor 6.0
.0.9
peer-group rservers
nv set vrf default
router bgp neighbor 6.0
.0.9
type numbered
nv set vrf default
router bgp neighbor p0_if peer-group fabric
nv set vrf default
router bgp neighbor p0_if type unnumbered
nv set vrf default
router bgp neighbor p1_if peer-group fabric
nv set vrf default
router bgp neighbor p1_if type unnumbered
nv set vrf default
router bgp peer-group fabric address-family ipv4-unicast enable on
nv set vrf default
router bgp peer-group fabric address-family ipv6-unicast enable on
nv set vrf default
router bgp peer-group fabric remote-as external
nv set vrf default
router bgp peer-group rservers address-family ipv4-unicast enable off
nv set vrf default
router bgp peer-group rservers address-family l2vpn-evpn add-path-tx off
nv set vrf default
router bgp peer-group rservers address-family l2vpn-evpn enable on
nv set vrf default
router bgp peer-group rservers multihop-ttl 3
nv set vrf default
router bgp peer-group rservers remote-as external
nv set vrf default
router bgp peer-group rservers update-source lo
nv set vrf internet1 evpn enable on
nv set vrf internet1 evpn vni 42000
nv set vrf internet1 loopback ip address 8.1
.0.13
/32
nv set vrf internet1 loopback ip address 2008
:0
:1
::13
/64
nv set vrf internet1 router bgp address-family ipv4-unicast enable on
nv set vrf internet1 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf internet1 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf internet1 router bgp enable on
nv set vrf special1 evpn enable on
nv set vrf special1 evpn vni 42001
nv set vrf special1 loopback ip address 9.1
.0.13
/32
nv set vrf special1 loopback ip address 2009
:0
:1
::13
/64
nv set vrf special1 router bgp address-family ipv4-unicast enable on
nv set vrf special1 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf special1 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf special1 router bgp enable on
nv set vrf tenant1 evpn enable on
nv set vrf tenant1 evpn vni 30001
nv set vrf tenant1 router bgp address-family ipv4-unicast enable on
nv set vrf tenant1 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf tenant1 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf tenant1 router bgp enable on
nv set vrf tenant1 router bgp router-id 6.0
.0.13
nv set vrf tenant2 evpn enable on
nv set vrf tenant2 evpn vni 30002
nv set vrf tenant2 router bgp address-family ipv4-unicast enable on
nv set vrf tenant2 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf tenant2 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf tenant2 router bgp enable on
nv set vrf tenant2 router bgp router-id 6.0
.0.13
root@doca
-hbn-service-bf2-s12-1
-ipmi:/tmp#
使用 EVPN 路由服务器进行部署的 HBN 配置示例
doca-hbn-service-bf2-s12-1
-ipmi# sh bgp sum
IPv4 Unicast Summary (VRF default
):
BGP router identifier 6.0
.0.13
, local AS number 4200065011
vrf-id 0
BGP table version 20
RIB entries 21
, using 4032
bytes of memory
Peers 2
, using 40
KiB of memory
Peer groups 2
, using 128
bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
spine11(p0_if) 4
65201
30617
30620
0
0
0
1d01h30m 9
11
N/A
spine12(p1_if) 4
65201
30620
30623
0
0
0
1d01h30m 9
11
N/A
Total number of neighbors 2
IPv6 Unicast Summary (VRF default
):
BGP router identifier 6.0
.0.13
, local AS number 4200065011
vrf-id 0
BGP table version 0
RIB entries 0
, using 0
bytes of memory
Peers 2
, using 40
KiB of memory
Peer groups 2
, using 128
bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
spine11(p0_if) 4
65201
30617
30620
0
0
0
1d01h30m 0
0
N/A
spine12(p1_if) 4
65201
30620
30623
0
0
0
1d01h30m 0
0
N/A
Total number of neighbors 2
L2VPN EVPN Summary (VRF default
):
BGP router identifier 6.0
.0.13
, local AS number 4200065011
vrf-id 0
BGP table version 0
RIB entries 79
, using 15
KiB of memory
Peers 3
, using 60
KiB of memory
Peer groups 2
, using 128
bytes of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
sn1(6.0
.0.7
) 4
4200065507
31410
31231
0
0
0
00
:27
:51
69
95
N/A
sn2(6.0
.0.8
) 4
4200065508
31169
31062
0
0
0
02
:34
:47
69
95
N/A
sn3(6.0
.0.9
) 4
4200065509
31285
31059
0
0
0
02
:34
:47
69
95
N/A
Total number of neighbors 3
doca-hbn-service-bf2-s12-1
-ipmi#
验证 HBN 中的 BGP 会话
命令输出显示 HBN 与 spine 交换机之间存在 BGP 会话,用于交换 IPv4/IPv6 单播。与路由服务器 sn1
、sn2
和 sn3
的 BGP 会话仅交换 L2VPN EVPN AFI/SAFI。
下游 VNI (DVNI)
下游 VNI(对称 EVPN 路由泄漏)允许用户泄漏远程 EVPN 路由,而无需本地配置源租户 VRF。常见的用例是上游交换机从下游叶子交换机学习 L3VNI,并将学习到的 L3VNI 强加于路由到关联 VRF 的流量 VXLAN。这消除了在所有叶子交换机上配置 L3VNI-SVI 接口的需求,并启用了共享服务和中心辐射型场景。
要配置对特定 VRF 中共享服务的访问,用户必须
配置路由目标导入语句,有效地将路由从远程租户泄漏到共享 VRF。
在远程节点导入共享 VRF 的路由目标。
route-target import|export <asn>:<vni>
例如
route-target import 65101:6000
路由目标导入或导出语句采用以下格式
对于路由目标导入语句,用户可以使用 route-target import ANY:<vni>
用于 NVUE 命令,或在 /etc/frr/frr.conf
文件中使用 route-target import *:<vni>
。ANY
在 NVUE 命令中或 /etc/frr/frr.conf
文件中的星号 (*
) 使用任何 ASN(自治系统号)作为通配符。
NVUE 命令如下
nv set vrf <vrf> router bgp route-import from-evpn route-target <asn>:<vni>
要配置路由导入语句
nv set vrf <vrf> router bgp route-export from-evpn route-target <asn>:<vni>
要配置路由导出语句
实施 DVNI 配置时的重要注意事项
EVPN 对称模式仅支持具有 L3 VNI 和单个 VXLAN 设备的下游 VNI
您可以在 VRF 中配置多个导入和导出路由目标
您不能将重叠的租户前缀泄漏(导入)到同一目标 VRF 中
如果对称 EVPN 配置正在使用自动导入/导出(通常是这种情况),则当配置 DVNI 时,将禁用租户 VNI 的自动导入,这将使 VRF 与租户隔离。为避免此问题,请将 route-import from-evpn route-target auto
添加到命令行。
共享 Internet 服务的 DVNI 配置
此处的配置示例考虑了一种场景,其中外部/Internet 连接通过防火墙 (FW) 提供,该防火墙连接到共享 VRF(本示例中为 vrf external
)。
超级 spine 交换机上的路由配置了 external VRF,其中导入了来自远程租户的路由目标。
在使用 HBN 的 BlueField 设备上,本地租户 VRF 导入与共享 external VRF 对应的路由目标。
L3VNI | 在使用 HBN 的 BlueField 设备上,本地租户 VRF 导入与共享 external VRF 对应的路由目标。 | |
租户 | 30001 | 租户 1 |
在 HBN VTEP 上 | 30002 | 租户 1 |
租户 2 | 30003 | 租户 1 |
租户 3 | 30004 | 租户 1 |
租户 4 | 30005 | 租户 1 |
租户 5 | 30006 | 租户 1 |
租户 6 | 60000 | 外部 |
在超级 spine 上配置,并连接到外部世界
nv set vrf tenant1 router bgp route-import
from-evpn route-target ANY:60000
nv set vrf tenant1 router bgp route-import
from-evpn route-target auto
nv set vrf tenant2 router bgp route-import
from-evpn route-target ANY:60000
nv set vrf tenant2 router bgp route-import
from-evpn route-target auto
nv set vrf tenant3 router bgp route-import
from-evpn route-target ANY:60000
nv set vrf tenant3 router bgp route-import
from-evpn route-target auto
nv set vrf tenant4 router bgp route-import
from-evpn route-target ANY:60000
nv set vrf tenant4 router bgp route-import
from-evpn route-target auto
nv set vrf tenant5 router bgp route-import
from-evpn route-target ANY:60000
nv set vrf tenant5 router bgp route-import
from-evpn route-target auto
nv set vrf tenant6 router bgp route-import
from-evpn route-target ANY:60000
nv set vrf tenant6 router bgp route-import
from-evpn route-target auto
root@doca
-hbn-service-bf3-s06-1
-ipmi:/tmp#
在使用 HBN 的 BlueField 设备上,HBN 上的每个租户 VRF 都必须导入共享 external VRF 的 VNI
nv set vrf external router bgp route-import
from-evpn route-target ANY:30001
nv set vrf external router bgp route-import
from-evpn route-target ANY:30002
nv set vrf external router bgp route-import
from-evpn route-target ANY:30003
nv set vrf external router bgp route-import
from-evpn route-target ANY:30004
nv set vrf external router bgp route-import
from-evpn route-target ANY:30005
nv set vrf external router bgp route-import
from-evpn route-target ANY:30006
nv set vrf external router bgp route-import
from-evpn route-target auto
root@superspine1
:mgmt:/home/cumulus#
在超级 spine 交换机(本示例中的 SS1)上,每个需要访问共享服务的远程租户 VRF 都必须泄漏到共享 external VRF。
在这种情况下,所有超级 spine 都需要此配置。
HBN 的 VRF 表中的 DVNI 泄漏路由
此处的每个超级 spine 都在通告可达性,从而提供 4 路覆盖 ECMP。
root@doca
-hbn-service-bf3-s06-1
-ipmi:/tmp# ip -4
route show table all 6.0
.0.4
/32
6.0
.0.4
table tenant1 proto bgp metric 20
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.12
ttl 0
tos 0
via 6.0
.0.12
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.13
ttl 0
tos 0
via 6.0
.0.13
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.14
ttl 0
tos 0
via 6.0
.0.14
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.15
ttl 0
tos 0
via 6.0
.0.15
dev vxlan48 weight 1
onlink
6.0
.0.4
table tenant2 proto bgp metric 20
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.12
ttl 0
tos 0
via 6.0
.0.12
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.13
ttl 0
tos 0
via 6.0
.0.13
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.14
ttl 0
tos 0
via 6.0
.0.14
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.15
ttl 0
tos 0
via 6.0
.0.15
dev vxlan48 weight 1
onlink
6.0
.0.4
table tenant3 proto bgp metric 20
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.12
ttl 0
tos 0
via 6.0
.0.12
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.13
ttl 0
tos 0
via 6.0
.0.13
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.14
ttl 0
tos 0
via 6.0
.0.14
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.15
ttl 0
tos 0
via 6.0
.0.15
dev vxlan48 weight 1
onlink
6.0
.0.4
table tenant4 proto bgp metric 20
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.12
ttl 0
tos 0
via 6.0
.0.12
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.13
ttl 0
tos 0
via 6.0
.0.13
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.14
ttl 0
tos 0
via 6.0
.0.14
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.15
ttl 0
tos 0
via 6.0
.0.15
dev vxlan48 weight 1
onlink
6.0
.0.4
table tenant5 proto bgp metric 20
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.12
ttl 0
tos 0
via 6.0
.0.12
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.13
ttl 0
tos 0
via 6.0
.0.13
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.14
ttl 0
tos 0
via 6.0
.0.14
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.15
ttl 0
tos 0
via 6.0
.0.15
dev vxlan48 weight 1
onlink
6.0
.0.4
table tenant6 proto bgp metric 20
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.12
ttl 0
tos 0
via 6.0
.0.12
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.13
ttl 0
tos 0
via 6.0
.0.13
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.14
ttl 0
tos 0
via 6.0
.0.14
dev vxlan48 weight 1
onlink
nexthop encap ip id 60000
src 0.0
.0.0
dst 6.0
.0.15
ttl 0
tos 0
via 6.0
.0.15
dev vxlan48 weight 1
onlink
root@doca
-hbn-service-bf3-s06-1
-ipmi:/tmp#
所有租户 VRF 的内核表,显示导入的共享服务
root@doca
-hbn-service-bf3-s06-1
-ipmi:/tmp# vtysh
Hello, this
is FRRouting (version 8.4
.3
).
Copyright 1996
-2005
Kunihiro Ishiguro, et al.
doca-hbn-service-bf3-s06-1
-ipmi# sh ip route vrf tenant1
Codes: K - kernel route, C - connected, S - static
, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
Z - FRR,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
VRF tenant1:
K>* 0.0
.0.0
/0
[255
/8192
] unreachable (ICMP unreachable), 00
:10
:36
B>* 6.0
.0.4
/32
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:38
* via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:38
* via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:38
* via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:38
B>* 6.6
.0.12
/32
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:38
B>* 6.6
.0.13
/32
[20
/0
] via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:38
B>* 6.6
.0.14
/32
[20
/0
] via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:38
B>* 6.6
.0.15
/32
[20
/0
] via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:38
B>* 7.1
.0.6
/32
[20
/0
] via 6.0
.0.6
, vlan4052_l3 onlink, weight 1
, 00
:05
:37
C>* 7.1
.0.16
/32
is directly connected, tenant1, 00
:10
:36
B>* 7.1
.0.18
/32
[20
/0
] via 6.0
.0.18
, vlan4052_l3 onlink, weight 1
, 00
:05
:37
B>* 7.1
.0.20
/32
[20
/0
] via 6.0
.0.20
, vlan4052_l3 onlink, weight 1
, 00
:05
:37
C>* 21.1
.0.0
/16
is directly connected, vlan101, 00
:10
:36
C * 21.1
.0.0
/16
[0
/1024
] is directly connected, vlan101-v0, 00
:10
:36
C * 21.2
.0.0
/16
[0
/1024
] is directly connected, vlan102-v0, 00
:10
:36
C>* 21.2
.0.0
/16
is directly connected, vlan102, 00
:10
:36
B>* 101.12
.4.0
/24
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:38
B>* 101.13
.4.0
/24
[20
/0
] via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:38
B>* 101.14
.4.0
/24
[20
/0
] via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:38
B>* 101.15
.4.0
/24
[20
/0
] via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:38
doca-hbn-service-bf3-s06-1
-ipmi# sh ip route vrf all
Codes: K - kernel route, C - connected, S - static
, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, A - Babel, D - SHARP, F - PBR, f - OpenFabric,
Z - FRR,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
VRF default
:
B>* 6.0
.0.6
/32
[20
/0
] via fe80::202
:ff:fe00:1f, p0_if, weight 1
, 00
:06
:47
* via fe80::202
:ff:fe00:27
, p1_if, weight 1
, 00
:06
:47
B>* 6.0
.0.7
/32
[20
/0
] via fe80::202
:ff:fe00:1f, p0_if, weight 1
, 00
:05
:48
* via fe80::202
:ff:fe00:27
, p1_if, weight 1
, 00
:05
:48
B>* 6.0
.0.8
/32
[20
/0
] via fe80::202
:ff:fe00:1f, p0_if, weight 1
, 00
:05
:38
* via fe80::202
:ff:fe00:27
, p1_if, weight 1
, 00
:05
:38
B>* 6.0
.0.9
/32
[20
/0
] via fe80::202
:ff:fe00:1f, p0_if, weight 1
, 00
:05
:28
* via fe80::202
:ff:fe00:27
, p1_if, weight 1
, 00
:05
:28
B>* 6.0
.0.10
/32
[20
/0
] via fe80::202
:ff:fe00:1f, p0_if, weight 1
, 00
:06
:49
B>* 6.0
.0.11
/32
[20
/0
] via fe80::202
:ff:fe00:27
, p1_if, weight 1
, 00
:06
:47
B>* 6.0
.0.12
/32
[20
/0
] via fe80::202
:ff:fe00:1f, p0_if, weight 1
, 00
:06
:47
* via fe80::202
:ff:fe00:27
, p1_if, weight 1
, 00
:06
:47
B>* 6.0
.0.13
/32
[20
/0
] via fe80::202
:ff:fe00:1f, p0_if, weight 1
, 00
:06
:47
* via fe80::202
:ff:fe00:27
, p1_if, weight 1
, 00
:06
:47
B>* 6.0
.0.14
/32
[20
/0
] via fe80::202
:ff:fe00:1f, p0_if, weight 1
, 00
:06
:47
* via fe80::202
:ff:fe00:27
, p1_if, weight 1
, 00
:06
:47
B>* 6.0
.0.15
/32
[20
/0
] via fe80::202
:ff:fe00:1f, p0_if, weight 1
, 00
:06
:47
* via fe80::202
:ff:fe00:27
, p1_if, weight 1
, 00
:06
:47
C>* 6.0
.0.16
/32
is directly connected, lo, 00
:10
:42
B>* 6.0
.0.18
/32
[20
/0
] via fe80::202
:ff:fe00:1f, p0_if, weight 1
, 00
:06
:47
* via fe80::202
:ff:fe00:27
, p1_if, weight 1
, 00
:06
:47
B>* 6.0
.0.20
/32
[20
/0
] via fe80::202
:ff:fe00:1f, p0_if, weight 1
, 00
:06
:47
* via fe80::202
:ff:fe00:27
, p1_if, weight 1
, 00
:06
:47
B>* 192.168
.0.0
/24
[20
/0
] via fe80::202
:ff:fe00:1f, p0_if, weight 1
, 00
:05
:48
* via fe80::202
:ff:fe00:27
, p1_if, weight 1
, 00
:05
:48
VRF internet1:
K>* 0.0
.0.0
/0
[255
/8192
] unreachable (ICMP unreachable), 00
:10
:42
B>* 8.1
.0.6
/32
[20
/0
] via 6.0
.0.6
, vlan4004_l3 onlink, weight 1
, 00
:05
:43
C>* 8.1
.0.16
/32
is directly connected, internet1, 00
:10
:42
B>* 8.1
.0.18
/32
[20
/0
] via 6.0
.0.18
, vlan4004_l3 onlink, weight 1
, 00
:05
:43
B>* 8.1
.0.20
/32
[20
/0
] via 6.0
.0.20
, vlan4004_l3 onlink, weight 1
, 00
:05
:43
VRF mgmt:
K>* 0.0
.0.0
/0
[255
/8192
] unreachable (ICMP unreachable), 00
:10
:42
C>* 10.88
.0.0
/16
is directly connected, eth0, 00
:10
:42
VRF special1:
K>* 0.0
.0.0
/0
[255
/8192
] unreachable (ICMP unreachable), 00
:10
:42
B>* 9.1
.0.6
/32
[20
/0
] via 6.0
.0.6
, vlan4033_l3 onlink, weight 1
, 00
:05
:43
C>* 9.1
.0.16
/32
is directly connected, special1, 00
:10
:42
B>* 9.1
.0.18
/32
[20
/0
] via 6.0
.0.18
, vlan4033_l3 onlink, weight 1
, 00
:05
:43
B>* 9.1
.0.20
/32
[20
/0
] via 6.0
.0.20
, vlan4033_l3 onlink, weight 1
, 00
:05
:43
VRF tenant1:
K>* 0.0
.0.0
/0
[255
/8192
] unreachable (ICMP unreachable), 00
:10
:42
B>* 6.0
.0.4
/32
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.12
/32
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.13
/32
[20
/0
] via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.14
/32
[20
/0
] via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.15
/32
[20
/0
] via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 7.1
.0.6
/32
[20
/0
] via 6.0
.0.6
, vlan4052_l3 onlink, weight 1
, 00
:05
:43
C>* 7.1
.0.16
/32
is directly connected, tenant1, 00
:10
:42
B>* 7.1
.0.18
/32
[20
/0
] via 6.0
.0.18
, vlan4052_l3 onlink, weight 1
, 00
:05
:43
B>* 7.1
.0.20
/32
[20
/0
] via 6.0
.0.20
, vlan4052_l3 onlink, weight 1
, 00
:05
:43
C>* 21.1
.0.0
/16
is directly connected, vlan101, 00
:10
:42
C * 21.1
.0.0
/16
[0
/1024
] is directly connected, vlan101-v0, 00
:10
:42
C * 21.2
.0.0
/16
[0
/1024
] is directly connected, vlan102-v0, 00
:10
:42
C>* 21.2
.0.0
/16
is directly connected, vlan102, 00
:10
:42
B>* 101.12
.4.0
/24
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.13
.4.0
/24
[20
/0
] via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.14
.4.0
/24
[20
/0
] via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.15
.4.0
/24
[20
/0
] via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
VRF tenant2:
K>* 0.0
.0.0
/0
[255
/8192
] unreachable (ICMP unreachable), 00
:10
:42
B>* 6.0
.0.4
/32
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.12
/32
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.13
/32
[20
/0
] via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.14
/32
[20
/0
] via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.15
/32
[20
/0
] via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 7.2
.0.6
/32
[20
/0
] via 6.0
.0.6
, vlan4037_l3 onlink, weight 1
, 00
:05
:43
C>* 7.2
.0.16
/32
is directly connected, tenant2, 00
:10
:42
B>* 7.2
.0.18
/32
[20
/0
] via 6.0
.0.18
, vlan4037_l3 onlink, weight 1
, 00
:05
:43
B>* 7.2
.0.20
/32
[20
/0
] via 6.0
.0.20
, vlan4037_l3 onlink, weight 1
, 00
:05
:43
C * 22.1
.0.0
/16
[0
/1024
] is directly connected, vlan201-v0, 00
:10
:42
C>* 22.1
.0.0
/16
is directly connected, vlan201, 00
:10
:42
C * 22.2
.0.0
/16
[0
/1024
] is directly connected, vlan202-v0, 00
:10
:42
C>* 22.2
.0.0
/16
is directly connected, vlan202, 00
:10
:42
B>* 101.12
.4.0
/24
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.13
.4.0
/24
[20
/0
] via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.14
.4.0
/24
[20
/0
] via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.15
.4.0
/24
[20
/0
] via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
VRF tenant3:
K>* 0.0
.0.0
/0
[255
/8192
] unreachable (ICMP unreachable), 00
:10
:42
B>* 6.0
.0.4
/32
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.12
/32
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.13
/32
[20
/0
] via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.14
/32
[20
/0
] via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.15
/32
[20
/0
] via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 7.3
.0.6
/32
[20
/0
] via 6.0
.0.6
, vlan4022_l3 onlink, weight 1
, 00
:05
:43
C>* 7.3
.0.16
/32
is directly connected, tenant3, 00
:10
:42
B>* 7.3
.0.18
/32
[20
/0
] via 6.0
.0.18
, vlan4022_l3 onlink, weight 1
, 00
:05
:43
B>* 7.3
.0.20
/32
[20
/0
] via 6.0
.0.20
, vlan4022_l3 onlink, weight 1
, 00
:05
:43
C>* 23.17
.0.0
/16
is directly connected, pf0vf4_if.3
, 00
:10
:42
B>* 23.19
.0.0
/16
[20
/0
] via 6.0
.0.18
, vlan4022_l3 onlink, weight 1
, 00
:05
:43
B>* 23.21
.0.0
/16
[20
/0
] via 6.0
.0.20
, vlan4022_l3 onlink, weight 1
, 00
:05
:43
B>* 101.12
.4.0
/24
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.13
.4.0
/24
[20
/0
] via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.14
.4.0
/24
[20
/0
] via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.15
.4.0
/24
[20
/0
] via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
VRF tenant4:
K>* 0.0
.0.0
/0
[255
/8192
] unreachable (ICMP unreachable), 00
:10
:42
B>* 6.0
.0.4
/32
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.12
/32
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.13
/32
[20
/0
] via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.14
/32
[20
/0
] via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.15
/32
[20
/0
] via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 7.4
.0.6
/32
[20
/0
] via 6.0
.0.6
, vlan4017_l3 onlink, weight 1
, 00
:05
:43
C>* 7.4
.0.16
/32
is directly connected, tenant4, 00
:10
:42
B>* 7.4
.0.18
/32
[20
/0
] via 6.0
.0.18
, vlan4017_l3 onlink, weight 1
, 00
:05
:43
B>* 7.4
.0.20
/32
[20
/0
] via 6.0
.0.20
, vlan4017_l3 onlink, weight 1
, 00
:05
:43
C>* 24.17
.0.0
/16
is directly connected, pf0vf4_if.4
, 00
:10
:42
B>* 24.19
.0.0
/16
[20
/0
] via 6.0
.0.18
, vlan4017_l3 onlink, weight 1
, 00
:05
:43
B>* 24.21
.0.0
/16
[20
/0
] via 6.0
.0.20
, vlan4017_l3 onlink, weight 1
, 00
:05
:43
B>* 101.12
.4.0
/24
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.13
.4.0
/24
[20
/0
] via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.14
.4.0
/24
[20
/0
] via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.15
.4.0
/24
[20
/0
] via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
VRF tenant5:
K>* 0.0
.0.0
/0
[255
/8192
] unreachable (ICMP unreachable), 00
:10
:42
B>* 6.0
.0.4
/32
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.12
/32
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.13
/32
[20
/0
] via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.14
/32
[20
/0
] via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.15
/32
[20
/0
] via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 7.5
.0.6
/32
[20
/0
] via 6.0
.0.6
, vlan4046_l3 onlink, weight 1
, 00
:05
:43
C>* 7.5
.0.16
/32
is directly connected, tenant5, 00
:10
:42
B>* 7.5
.0.18
/32
[20
/0
] via 6.0
.0.18
, vlan4046_l3 onlink, weight 1
, 00
:05
:43
B>* 7.5
.0.20
/32
[20
/0
] via 6.0
.0.20
, vlan4046_l3 onlink, weight 1
, 00
:05
:43
C>* 25.17
.0.0
/16
is directly connected, pf0vf4_if.5
, 00
:10
:42
B>* 25.19
.0.0
/16
[20
/0
] via 6.0
.0.18
, vlan4046_l3 onlink, weight 1
, 00
:05
:43
B>* 25.21
.0.0
/16
[20
/0
] via 6.0
.0.20
, vlan4046_l3 onlink, weight 1
, 00
:05
:43
B>* 101.12
.4.0
/24
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.13
.4.0
/24
[20
/0
] via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.14
.4.0
/24
[20
/0
] via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.15
.4.0
/24
[20
/0
] via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
VRF tenant6:
K>* 0.0
.0.0
/0
[255
/8192
] unreachable (ICMP unreachable), 00
:10
:42
B>* 6.0
.0.4
/32
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
* via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.12
/32
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.13
/32
[20
/0
] via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.14
/32
[20
/0
] via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 6.6
.0.15
/32
[20
/0
] via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 7.6
.0.6
/32
[20
/0
] via 6.0
.0.6
, vlan4041_l3 onlink, weight 1
, 00
:05
:43
C>* 7.6
.0.16
/32
is directly connected, tenant6, 00
:10
:42
B>* 7.6
.0.18
/32
[20
/0
] via 6.0
.0.18
, vlan4041_l3 onlink, weight 1
, 00
:05
:43
B>* 7.6
.0.20
/32
[20
/0
] via 6.0
.0.20
, vlan4041_l3 onlink, weight 1
, 00
:05
:43
C>* 26.17
.0.0
/16
is directly connected, pf0vf4_if.6
, 00
:10
:42
B>* 26.19
.0.0
/16
[20
/0
] via 6.0
.0.18
, vlan4041_l3 onlink, weight 1
, 00
:05
:43
B>* 26.21
.0.0
/16
[20
/0
] via 6.0
.0.20
, vlan4041_l3 onlink, weight 1
, 00
:05
:43
B>* 101.12
.4.0
/24
[20
/0
] via 6.0
.0.12
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.13
.4.0
/24
[20
/0
] via 6.0
.0.13
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.14
.4.0
/24
[20
/0
] via 6.0
.0.14
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
B>* 101.15
.4.0
/24
[20
/0
] via 6.0
.0.15
, vxlan48 (vrf default
) onlink, label 60000
, weight 1
, 00
:05
:44
doca-hbn-service-bf3-s06-1
-ipmi#
FRR RIB 表
DVNI 调试
May 7
20
:59
:49
doca-hbn-service-bf3-s06-1
-ipmi bgpd[1775018
]: [GKC5Y-XBAX9] vrf tenant1: import
evpn prefix [5
]:[0
]:[32
]:[6.0
.0.4
] parent 0xaaaafda63a90
flags 0x410
May 7
20
:59
:49
doca-hbn-service-bf3-s06-1
-ipmi bgpd[1775018
]: [KZNVF-SX7KT] ... new
pi dest 0xaaaafe524650
(l 2
) pi 0xaaaafe5ae400
(l 1
, f 0x4010
)
May 7
20
:59
:49
doca-hbn-service-bf3-s06-1
-ipmi bgpd[1775018
]: [GKC5Y-XBAX9] vrf tenant2: import
evpn prefix [5
]:[0
]:[32
]:[6.0
.0.4
] parent 0xaaaafda63a90
flags 0x410
May 7
20
:59
:49
doca-hbn-service-bf3-s06-1
-ipmi bgpd[1775018
]: [KZNVF-SX7KT] ... new
pi dest 0xaaaafe51c420
(l 2
) pi 0xaaaafe55d230
(l 1
, f 0x4010
)
May 7
20
:59
:49
doca-hbn-service-bf3-s06-1
-ipmi bgpd[1775018
]: [GKC5Y-XBAX9] vrf tenant3: import
evpn prefix [5
]:[0
]:[32
]:[6.0
.0.4
] parent 0xaaaafda63a90
flags 0x410
May 7
20
:59
:49
doca-hbn-service-bf3-s06-1
-ipmi bgpd[1775018
]: [KZNVF-SX7KT] ... new
pi dest 0xaaaafe51a670
(l 2
) pi 0xaaaafe674820
(l 1
, f 0x4010
)
May 7
20
:59
:49
doca-hbn-service-bf3-s06-1
-ipmi bgpd[1775018
]: [GKC5Y-XBAX9] vrf tenant4: import
evpn prefix [5
]:[0
]:[32
]:[6.0
.0.4
] parent 0xaaaafda63a90
flags 0x410
May 7
20
:59
:49
doca-hbn-service-bf3-s06-1
-ipmi bgpd[1775018
]: [KZNVF-SX7KT] ... new
pi dest 0xaaaafe519fb0
(l 2
) pi 0xaaaafe675e40
(l 1
, f 0x4010
)
May 7
20
:59
:49
doca-hbn-service-bf3-s06-1
-ipmi bgpd[1775018
]: [GKC5Y-XBAX9] vrf tenant5: import
evpn prefix [5
]:[0
]:[32
]:[6.0
.0.4
] parent 0xaaaafda63a90
flags 0x410
May 7
20
:59
:49
doca-hbn-service-bf3-s06-1
-ipmi bgpd[1775018
]: [KZNVF-SX7KT] ... new
pi dest 0xaaaafe55ae50
(l 2
) pi 0xaaaafe5482f0
(l 1
, f 0x4010
)
May 7
20
:59
:49
doca-hbn-service-bf3-s06-1
-ipmi bgpd[1775018
]: [GKC5Y-XBAX9] vrf tenant6: import
evpn prefix [5
]:[0
]:[32
]:[6.0
.0.4
] parent 0xaaaafda63a90
flags 0x410
May 7
20
:59
:49
doca-hbn-service-bf3-s06-1
-ipmi bgpd[1775018
]: [KZNVF-SX7KT] ... new
pi dest 0xaaaafdaf3590
(l 2
) pi 0xaaaafe48fbf0
(l 1
, f 0x4010
)
BGP/Zebra 调试
root@doca
-hbn-service-bf3-s06-1
-ipmi:/tmp# cat /cumulus/nl2docad/run/software-tables/15
{
"table"
: {
"id"
: 15
,
"name"
: "HAL Downstream-VNI Table "
,
"count"
: 1
,
"records"
: [
{
"vni"
: 60000
,
"fid"
: 4098
,
"mark-for-del"
: 0
,
"vtep-users"
:
{
"count"
: 4
,
"vtep-user-list"
: [
{
"dest-vtep"
: "6.0.0.12"
,
"dest-mac"
: "44:38:39:f0:00:12"
,
"is-dmac-null"
: 0
,
"ref-cnt"
: 36
},
{
"dest-vtep"
: "6.0.0.14"
,
"dest-mac"
: "44:38:39:f0:00:14"
,
"is-dmac-null"
: 0
,
"ref-cnt"
: 36
},
{
"dest-vtep"
: "6.0.0.13"
,
"dest-mac"
: "44:38:39:f0:00:13"
,
"is-dmac-null"
: 0
,
"ref-cnt"
: 36
},
{
"dest-vtep"
: "6.0.0.15"
,
"dest-mac"
: "44:38:39:f0:00:15"
,
"is-dmac-null"
: 0
,
"ref-cnt"
: 36
}
]
}
}
]
}
}root@doca
-hbn-service-bf3-s06-1
-ipmi:/tmp#
DVNI 表
DVNI 配置示例
root@doca
-hbn-service-bf3-s06-1
-ipmi:/tmp# nv config show -o commands
nv set bridge domain br_default vlan 101
vni 10101
nv set bridge domain br_default vlan 102
vni 10102
nv set bridge domain br_default vlan 201
vni 10201
nv set bridge domain br_default vlan 202
vni 10202
nv set evpn enable on
nv set evpn route-advertise svi-ip off
nv set interface
ilan3200 ip vrf internet1
nv set interface
ilan3200 vlan 3200
nv set interface
ilan3200,slan3201,vlan101-102
,201
-202
,3001
-3006
base-interface
br_default
nv set interface
ilan3200,slan3201,vlan101-102
,201
-202
,3001
-3006
type svi
nv set interface
lo ip address 6.0
.0.16
/32
nv set interface
lo ip address 2001
::16
/128
nv set interface
lo type loopback
nv set interface
p0_if,p1_if,pf0hpf_if,pf0vf0_if,pf0vf1_if,pf0vf2_if,pf0vf3_if,pf0vf4_if,pf1hpf_if type swp
nv set interface
pf0vf0_if bridge domain br_default access 101
nv set interface
pf0vf1_if bridge domain br_default access 102
nv set interface
pf0vf2_if bridge domain br_default access 201
nv set interface
pf0vf3_if bridge domain br_default access 202
nv set interface
pf0vf4_if.3
ip address 23.17
.0.16
/16
nv set interface
pf0vf4_if.3
ip address 2020
:0
:3
:17
::16
/64
nv set interface
pf0vf4_if.3
vlan 3
nv set interface
pf0vf4_if.3
,vlan3003 ip vrf tenant3
nv set interface
pf0vf4_if.3
-6
base-interface
pf0vf4_if
nv set interface
pf0vf4_if.3
-6
type sub
nv set interface
pf0vf4_if.4
ip address 24.17
.0.16
/16
nv set interface
pf0vf4_if.4
ip address 2020
:0
:4
:17
::16
/64
nv set interface
pf0vf4_if.4
vlan 4
nv set interface
pf0vf4_if.4
,vlan3004 ip vrf tenant4
nv set interface
pf0vf4_if.5
ip address 25.17
.0.16
/16
nv set interface
pf0vf4_if.5
ip address 2020
:0
:5
:17
::16
/64
nv set interface
pf0vf4_if.5
vlan 5
nv set interface
pf0vf4_if.5
,vlan3005 ip vrf tenant5
nv set interface
pf0vf4_if.6
ip address 26.17
.0.16
/16
nv set interface
pf0vf4_if.6
ip address 2020
:0
:6
:17
::16
/64
nv set interface
pf0vf4_if.6
vlan 6
nv set interface
pf0vf4_if.6
,vlan3006 ip vrf tenant6
nv set interface
slan3201 ip vrf special1
nv set interface
slan3201 vlan 3201
nv set interface
vlan101 ip address 21.1
.0.16
/16
nv set interface
vlan101 ip address 2020
:0
:1
:1
::16
/64
nv set interface
vlan101 ip vrr address 21.1
.0.250
/16
nv set interface
vlan101 ip vrr address 2020
:0
:1
:1
::250
/64
nv set interface
vlan101 ip vrr mac-address 00
:00
:01
:00
:00
:65
nv set interface
vlan101 vlan 101
nv set interface
vlan101-102
,201
-202
ip vrr enable on
nv set interface
vlan101-102
,3001
ip vrf tenant1
nv set interface
vlan102 ip address 21.2
.0.16
/16
nv set interface
vlan102 ip address 2020
:0
:1
:2
::16
/64
nv set interface
vlan102 ip vrr address 21.2
.0.250
/16
nv set interface
vlan102 ip vrr address 2020
:0
:1
:2
::250
/64
nv set interface
vlan102 ip vrr mac-address 00
:00
:01
:00
:00
:66
nv set interface
vlan102 vlan 102
nv set interface
vlan201 ip address 22.1
.0.16
/16
nv set interface
vlan201 ip address 2020
:0
:2
:1
::16
/64
nv set interface
vlan201 ip vrr address 22.1
.0.250
/16
nv set interface
vlan201 ip vrr address 2020
:0
:2
:1
::250
/64
nv set interface
vlan201 ip vrr mac-address 00
:00
:02
:00
:00
:c9
nv set interface
vlan201 vlan 201
nv set interface
vlan201-202
,3002
ip vrf tenant2
nv set interface
vlan202 ip address 22.2
.0.16
/16
nv set interface
vlan202 ip address 2020
:0
:2
:2
::16
/64
nv set interface
vlan202 ip vrr address 22.2
.0.250
/16
nv set interface
vlan202 ip vrr address 2020
:0
:2
:2
::250
/64
nv set interface
vlan202 ip vrr mac-address 00
:00
:02
:00
:00
:ca
nv set interface
vlan202 vlan 202
nv set interface
vlan3001 vlan 3001
nv set interface
vlan3002 vlan 3002
nv set interface
vlan3003 vlan 3003
nv set interface
vlan3004 vlan 3004
nv set interface
vlan3005 vlan 3005
nv set interface
vlan3006 vlan 3006
nv set nve vxlan arp-nd-suppress on
nv set nve vxlan enable on
nv set nve vxlan source address 6.0
.0.16
nv set platform
nv set router bgp autonomous-system 65011
nv set router bgp enable on
nv set router bgp router-id 6.0
.0.16
nv set router vrr enable on
nv set system config snippet
nv set system global
nv set vrf default
router bgp address-family ipv4-unicast enable on
nv set vrf default
router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf default
router bgp address-family ipv6-unicast enable on
nv set vrf default
router bgp address-family l2vpn-evpn enable on
nv set vrf default
router bgp enable on
nv set vrf default
router bgp neighbor 6.0
.0.7
peer-group rservers
nv set vrf default
router bgp neighbor 6.0
.0.7
type numbered
nv set vrf default
router bgp neighbor 6.0
.0.8
peer-group rservers
nv set vrf default
router bgp neighbor 6.0
.0.8
type numbered
nv set vrf default
router bgp neighbor 6.0
.0.9
peer-group rservers
nv set vrf default
router bgp neighbor 6.0
.0.9
type numbered
nv set vrf default
router bgp neighbor p0_if peer-group fabric
nv set vrf default
router bgp neighbor p0_if type unnumbered
nv set vrf default
router bgp neighbor p1_if peer-group fabric
nv set vrf default
router bgp neighbor p1_if type unnumbered
nv set vrf default
router bgp peer-group fabric address-family ipv4-unicast enable on
nv set vrf default
router bgp peer-group fabric address-family ipv6-unicast enable on
nv set vrf default
router bgp peer-group fabric bfd detect-multiplier 3
nv set vrf default
router bgp peer-group fabric bfd enable on
nv set vrf default
router bgp peer-group fabric bfd min-rx-interval 1000
nv set vrf default
router bgp peer-group fabric bfd min-tx-interval 1000
nv set vrf default
router bgp peer-group fabric remote-as external
nv set vrf default
router bgp peer-group rservers address-family ipv4-unicast enable off
nv set vrf default
router bgp peer-group rservers address-family l2vpn-evpn add-path-tx off
nv set vrf default
router bgp peer-group rservers address-family l2vpn-evpn enable on
nv set vrf default
router bgp peer-group rservers multihop-ttl 10
nv set vrf default
router bgp peer-group rservers remote-as external
nv set vrf default
router bgp peer-group rservers update-source lo
nv set vrf internet1 evpn enable on
nv set vrf internet1 evpn vni 42000
nv set vrf internet1 loopback ip address 8.1
.0.16
/32
nv set vrf internet1 loopback ip address 2008
:0
:1
::16
/64
nv set vrf internet1 router bgp address-family ipv4-unicast enable on
nv set vrf internet1 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf internet1 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf internet1 router bgp address-family ipv6-unicast enable on
nv set vrf internet1 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf internet1 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf internet1 router bgp enable on
nv set vrf special1 evpn enable on
nv set vrf special1 evpn vni 42001
nv set vrf special1 loopback ip address 9.1
.0.16
/32
nv set vrf special1 loopback ip address 2009
:0
:1
::16
/64
nv set vrf special1 router bgp address-family ipv4-unicast enable on
nv set vrf special1 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf special1 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf special1 router bgp address-family ipv6-unicast enable on
nv set vrf special1 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf special1 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf special1 router bgp enable on
nv set vrf tenant1 evpn enable on
nv set vrf tenant1 evpn vni 30001
nv set vrf tenant1 loopback ip address 7.1
.0.16
/32
nv set vrf tenant1 loopback ip address 2007
:0
:1
::16
/64
nv set vrf tenant1 router bgp address-family ipv4-unicast enable on
nv set vrf tenant1 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf tenant1 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf tenant1 router bgp address-family ipv6-unicast enable on
nv set vrf tenant1 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf tenant1 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf tenant1 router bgp enable on
nv set vrf tenant1 router bgp neighbor 21.1
.0.17
peer-group hostgroup
nv set vrf tenant1 router bgp neighbor 21.1
.0.17
type numbered
nv set vrf tenant1 router bgp peer-group hostgroup address-family ipv4-unicast enable on
nv set vrf tenant1 router bgp peer-group hostgroup address-family ipv6-unicast enable on
nv set vrf tenant1 router bgp peer-group hostgroup remote-as external
nv set vrf tenant1 router bgp route-import
from-evpn route-target ANY:60000
nv set vrf tenant1 router bgp route-import
from-evpn route-target auto
nv set vrf tenant1 router bgp router-id 6.0
.0.16
nv set vrf tenant2 evpn enable on
nv set vrf tenant2 evpn vni 30002
nv set vrf tenant2 loopback ip address 7.2
.0.16
/32
nv set vrf tenant2 loopback ip address 2007
:0
:2
::16
/64
nv set vrf tenant2 router bgp address-family ipv4-unicast enable on
nv set vrf tenant2 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf tenant2 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf tenant2 router bgp address-family ipv6-unicast enable on
nv set vrf tenant2 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf tenant2 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf tenant2 router bgp enable on
nv set vrf tenant2 router bgp neighbor 22.1
.0.17
peer-group hostgroup
nv set vrf tenant2 router bgp neighbor 22.1
.0.17
type numbered
nv set vrf tenant2 router bgp peer-group hostgroup address-family ipv4-unicast enable on
nv set vrf tenant2 router bgp peer-group hostgroup address-family ipv6-unicast enable on
nv set vrf tenant2 router bgp peer-group hostgroup remote-as external
nv set vrf tenant2 router bgp route-import
from-evpn route-target ANY:60000
nv set vrf tenant2 router bgp route-import
from-evpn route-target auto
nv set vrf tenant2 router bgp router-id 6.0
.0.16
nv set vrf tenant3 evpn enable on
nv set vrf tenant3 evpn vni 30003
nv set vrf tenant3 loopback ip address 7.3
.0.16
/32
nv set vrf tenant3 loopback ip address 2007
:0
:3
::16
/64
nv set vrf tenant3 router bgp address-family ipv4-unicast enable on
nv set vrf tenant3 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf tenant3 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf tenant3 router bgp address-family ipv6-unicast enable on
nv set vrf tenant3 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf tenant3 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf tenant3 router bgp enable on
nv set vrf tenant3 router bgp neighbor 23.17
.0.17
peer-group hostgroup
nv set vrf tenant3 router bgp neighbor 23.17
.0.17
type numbered
nv set vrf tenant3 router bgp peer-group hostgroup address-family ipv4-unicast enable on
nv set vrf tenant3 router bgp peer-group hostgroup address-family ipv6-unicast enable on
nv set vrf tenant3 router bgp peer-group hostgroup remote-as external
nv set vrf tenant3 router bgp route-import
from-evpn route-target ANY:60000
nv set vrf tenant3 router bgp route-import
from-evpn route-target auto
nv set vrf tenant3 router bgp router-id 6.0
.0.16
nv set vrf tenant3 table auto
nv set vrf tenant4 evpn enable on
nv set vrf tenant4 evpn vni 30004
nv set vrf tenant4 loopback ip address 7.4
.0.16
/32
nv set vrf tenant4 loopback ip address 2007
:0
:4
::16
/64
nv set vrf tenant4 router bgp address-family ipv4-unicast enable on
nv set vrf tenant4 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf tenant4 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf tenant4 router bgp address-family ipv6-unicast enable on
nv set vrf tenant4 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf tenant4 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf tenant4 router bgp enable on
nv set vrf tenant4 router bgp neighbor 24.17
.0.17
peer-group hostgroup
nv set vrf tenant4 router bgp neighbor 24.17
.0.17
type numbered
nv set vrf tenant4 router bgp peer-group hostgroup address-family ipv4-unicast enable on
nv set vrf tenant4 router bgp peer-group hostgroup address-family ipv6-unicast enable on
nv set vrf tenant4 router bgp peer-group hostgroup remote-as external
nv set vrf tenant4 router bgp route-import
from-evpn route-target ANY:60000
nv set vrf tenant4 router bgp route-import
from-evpn route-target auto
nv set vrf tenant4 router bgp router-id 6.0
.0.16
nv set vrf tenant4 table auto
nv set vrf tenant5 evpn enable on
nv set vrf tenant5 evpn vni 30005
nv set vrf tenant5 loopback ip address 7.5
.0.16
/32
nv set vrf tenant5 loopback ip address 2007
:0
:5
::16
/64
nv set vrf tenant5 router bgp address-family ipv4-unicast enable on
nv set vrf tenant5 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf tenant5 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf tenant5 router bgp address-family ipv6-unicast enable on
nv set vrf tenant5 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf tenant5 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf tenant5 router bgp enable on
nv set vrf tenant5 router bgp neighbor 25.17
.0.17
peer-group hostgroup
nv set vrf tenant5 router bgp neighbor 25.17
.0.17
type numbered
nv set vrf tenant5 router bgp peer-group hostgroup address-family ipv4-unicast enable on
nv set vrf tenant5 router bgp peer-group hostgroup address-family ipv6-unicast enable on
nv set vrf tenant5 router bgp peer-group hostgroup remote-as external
nv set vrf tenant5 router bgp route-import
from-evpn route-target ANY:60000
nv set vrf tenant5 router bgp route-import
from-evpn route-target auto
nv set vrf tenant5 router bgp router-id 6.0
.0.16
nv set vrf tenant5 table auto
nv set vrf tenant6 evpn enable on
nv set vrf tenant6 evpn vni 30006
nv set vrf tenant6 loopback ip address 7.6
.0.16
/32
nv set vrf tenant6 loopback ip address 2007
:0
:6
::16
/64
nv set vrf tenant6 router bgp address-family ipv4-unicast enable on
nv set vrf tenant6 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf tenant6 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf tenant6 router bgp address-family ipv6-unicast enable on
nv set vrf tenant6 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf tenant6 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf tenant6 router bgp enable on
nv set vrf tenant6 router bgp neighbor 26.17
.0.17
peer-group hostgroup
nv set vrf tenant6 router bgp neighbor 26.17
.0.17
type numbered
nv set vrf tenant6 router bgp peer-group hostgroup address-family ipv4-unicast enable on
nv set vrf tenant6 router bgp peer-group hostgroup address-family ipv6-unicast enable on
nv set vrf tenant6 router bgp peer-group hostgroup remote-as external
nv set vrf tenant6 router bgp route-import
from-evpn route-target ANY:60000
nv set vrf tenant6 router bgp route-import
from-evpn route-target auto
nv set vrf tenant6 router bgp router-id 6.0
.0.16
nv set vrf tenant6 table auto
root@doca
-hbn-service-bf3-s06-1
-ipmi:/tmp#
BlueField 设备的 HBN 配置示例
root@superspine1
:mgmt:/home/cumulus# nv config show -o commands
nv set bridge domain br_default vlan 101
vni 10101
nv set bridge domain br_default vlan 102
vni 10102
nv set bridge domain br_default vlan 201
vni 10201
nv set bridge domain br_default vlan 202
vni 10202
nv set evpn enable on
nv set interface
eth0 ip address 192.168
.0.15
/24
nv set interface
eth0 ip gateway 192.168
.0.2
nv set interface
eth0 type eth
nv set interface
lo ip address 6.0
.0.12
/32
nv set interface
lo ip address 2001
::12
/128
nv set interface
lo type loopback
nv set interface
swp1-6
type swp
nv set interface
swp6 ip address 101.12
.4.12
/24
nv set interface
swp6 ip address 2101
:12
::4
:12
/112
nv set interface
swp6 ip vrf external
nv set nve vxlan arp-nd-suppress on
nv set nve vxlan enable on
nv set nve vxlan source address 6.0
.0.12
nv set platform
nv set router bgp autonomous-system 65300
nv set router bgp enable on
nv set router bgp router-id 6.0
.0.12
nv set system config snippet
nv set system global system-mac 44
:38
:39
:f0:00
:12
nv set system hostname superspine1
nv set system ssh-server permit-root-login enabled
nv set vrf default
router bgp address-family ipv4-unicast enable on
nv set vrf default
router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf default
router bgp address-family ipv6-unicast enable on
nv set vrf default
router bgp address-family l2vpn-evpn enable on
nv set vrf default
router bgp enable on
nv set vrf default
router bgp neighbor swp1 peer-group fabric
nv set vrf default
router bgp neighbor swp1 type unnumbered
nv set vrf default
router bgp neighbor swp2 peer-group fabric
nv set vrf default
router bgp neighbor swp2 type unnumbered
nv set vrf default
router bgp neighbor swp3 peer-group rservers
nv set vrf default
router bgp neighbor swp3 type unnumbered
nv set vrf default
router bgp neighbor swp4 peer-group rservers
nv set vrf default
router bgp neighbor swp4 type unnumbered
nv set vrf default
router bgp neighbor swp5 peer-group rservers
nv set vrf default
router bgp neighbor swp5 type unnumbered
nv set vrf default
router bgp peer-group fabric address-family ipv4-unicast enable on
nv set vrf default
router bgp peer-group fabric address-family ipv6-unicast enable on
nv set vrf default
router bgp peer-group fabric bfd detect-multiplier 3
nv set vrf default
router bgp peer-group fabric bfd enable on
nv set vrf default
router bgp peer-group fabric bfd min-rx-interval 1000
nv set vrf default
router bgp peer-group fabric bfd min-tx-interval 1000
nv set vrf default
router bgp peer-group fabric remote-as external
nv set vrf default
router bgp peer-group rservers address-family ipv4-unicast enable on
nv set vrf default
router bgp peer-group rservers address-family l2vpn-evpn add-path-tx off
nv set vrf default
router bgp peer-group rservers address-family l2vpn-evpn enable on
nv set vrf default
router bgp peer-group rservers remote-as external
nv set vrf external evpn enable on
nv set vrf external evpn vni 60000
nv set vrf external loopback ip address 6.6
.0.12
/32
nv set vrf external loopback ip address 2006
:0
:6
::12
/64
nv set vrf external router bgp address-family ipv4-unicast enable on
nv set vrf external router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf external router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf external router bgp address-family ipv6-unicast enable on
nv set vrf external router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf external router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf external router bgp address-family l2vpn-evpn enable on
nv set vrf external router bgp enable on
nv set vrf external router bgp neighbor swp6 peer-group peer-group-fw
nv set vrf external router bgp neighbor swp6 type unnumbered
nv set vrf external router bgp peer-group peer-group-fw address-family ipv4-unicast enable on
nv set vrf external router bgp peer-group peer-group-fw address-family ipv6-unicast enable on
nv set vrf external router bgp peer-group peer-group-fw remote-as external
nv set vrf external router bgp route-import
from-evpn route-target ANY:30001
nv set vrf external router bgp route-import
from-evpn route-target ANY:30002
nv set vrf external router bgp route-import
from-evpn route-target ANY:30003
nv set vrf external router bgp route-import
from-evpn route-target ANY:30004
nv set vrf external router bgp route-import
from-evpn route-target ANY:30005
nv set vrf external router bgp route-import
from-evpn route-target ANY:30006
nv set vrf external router bgp route-import
from-evpn route-target auto
root@superspine1
:mgmt:/home/cumulus#
使用下游 VNI 和子接口的网关应用程序
SS1 交换机配置示例
运行 HBN 服务的 DPU 可以使用 HBN 功能的组合部署为边界网关,具体来说,包括 EVPN 对称路由、下游 VNI、VRF 路由泄漏和 VLAN 子接口。这样的边界网关可以为一个或多个租户执行北向流量切换(到外部网络或 Internet)。在此网关配置中,BlueField 的 Uplink 必须同时承载租户流量(将在“覆盖网络”中并进行 VXLAN 封装)以及与外部网络或 Internet 之间的流量(将在“底层网络”中进行直接路由)。这是通过在 Uplink 接口上配置和运行 VXLAN-EVPN,同时在这些相同的 Uplink 上配置和使用额外的 VLAN 子接口来完成的,用于与外部网络之间的流量。这些 VLAN 子接口将配置到 Internet 或外部 VRF 中,以便与在默认 VRF 上承载的 VXLAN 封装流量分离。
借助运行 HBN 且能够充当边界网关的 BlueField,不再依赖物理交换机和路由器来终止 VXLAN 流量并执行此角色,因此对底层网络的要求仅是提供端到端 IP/UDP 连接并促进在顶部设置覆盖网络。此外,可以在网络中轻松部署多个边界网关,包括每个租户的专用网关或租户组的共享网关。
由于 HBN 目前不支持网络地址转换 (NAT),因此对于具有重叠 IP 地址的租户,必须为每个租户部署一个专用边界网关。
有关更多详细信息以及共同启用边界网关功能的某些关键功能的配置,请参阅关于 下游 VNI 和 VLAN 子接口 的部分。
网关应用程序示例
以下拓扑图和相关配置代码段显示了边界网关部署的两个不同用例
tenant1
是托管在具有非网关 BlueField 的服务器上的租户示例,使用 BlueField Gw-HBN1 上的专用边界网关进行 Internet 连接。此租户往返 Internet 的流量以粉红色标记。

gw_tenant1
是托管在具有网关 BlueField 的服务器上的租户示例。在这种情况下,此租户的边界网关由 BlueField Gw-HBN2 提供。此租户往返 Internet 的流量以蓝色描绘。
L3 VNI 源映射 | HBN | VRF |
L3 VNI |
|
|
L3 VNI |
|
|
|
|
|
tenant-hbn3
和 tenant-hbn4
Internet VRF 的配置代码段
Internet VRF 在 BGP 会话中使用子接口功能与底层交换机建立(即,
p0_if.60
和p1_if.60
)
nv set interface
p0_if.60
,p1_if.60
,vlan10 ip vrf internet1
nv set vrf internet1 evpn enable on
nv set vrf internet1 evpn vni 10000
nv set vrf internet1 loopback ip address 6.2
.0.1
/32
nv set vrf internet1 loopback ip address 2001
:cafe:feed::1
/128
nv set vrf internet1 router bgp address-family ipv4-unicast enable on
nv set vrf internet1 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf internet1 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf internet1 router bgp address-family ipv6-unicast enable on
nv set vrf internet1 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf internet1 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf internet1 router bgp address-family l2vpn-evpn enable on
nv set vrf internet1 router bgp autonomous-system 65552
nv set vrf internet1 router bgp enable on
nv set vrf internet1 router bgp neighbor p0_if.60
capabilities source-address internet1
nv set vrf internet1 router bgp neighbor p0_if.60
peer-group l3_pg1
nv set vrf internet1 router bgp neighbor p0_if.60
type unnumbered
nv set vrf internet1 router bgp neighbor p1_if.60
capabilities source-address internet1
nv set vrf internet1 router bgp neighbor p1_if.60
peer-group l3_pg1
nv set vrf internet1 router bgp neighbor p1_if.60
type unnumbered
nv set vrf internet1 router bgp peer-group l3_pg1 address-family ipv4-unicast enable on
nv set vrf internet1 router bgp peer-group l3_pg1 address-family ipv6-unicast enable on
nv set vrf internet1 router bgp peer-group l3_pg1 remote-as external
nv set vrf internet1 router bgp route-export to-evpn route-target 65552
:10000
nv set vrf internet1 router bgp route-import
from-evpn route-target ANY:20000
nv set vrf internet1 router bgp route-import
from-evpn route-target ANY:30000
nv set vrf internet1 router bgp route-import
from-evpn route-target auto
nv set vrf internet1 router bgp router-id 27.0
.0.5
Internet VRF 还使用下游 VNI 功能和 from-EVPN 语法导入所有租户 VRF(本地和远程)。
网关本地租户的配置代码段
gw_tenant
跨越 2 个网关并使用 L3 VNI 连接gw_tenant
具有多个 SVI,它们表示为vlan30
和vlan31
SVI
Internet L3 VNI 使用 DVNI 导入。该示例还显式添加了使用 auto 的路由目标。
nv set interface
vlan30-31
ip vrf gw_tenant1
nv set vrf gw_tenant1 evpn enable on
nv set vrf gw_tenant1 evpn vni 30000
nv set vrf gw_tenant1 loopback ip address 15.3
.0.1
/32
nv set vrf gw_tenant1 loopback ip address 2001
:bad:c0de::1
/128
nv set vrf gw_tenant1 router bgp address-family ipv4-unicast enable on
nv set vrf gw_tenant1 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf gw_tenant1 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf gw_tenant1 router bgp address-family ipv6-unicast enable on
nv set vrf gw_tenant1 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf gw_tenant1 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf gw_tenant1 router bgp address-family l2vpn-evpn enable on
nv set vrf gw_tenant1 router bgp autonomous-system 65552
nv set vrf gw_tenant1 router bgp enable on
nv set vrf gw_tenant1 router bgp route-export to-evpn route-target 65552
:30000
nv set vrf gw_tenant1 router bgp route-import
from-evpn route-target ANY:10000
nv set vrf gw_tenant1 router bgp route-import
from-evpn route-target auto
nv set vrf gw_tenant1 router bgp router-id 27.0
.0.5
gw_tenant
VRF
远程租户的配置代码段
tenant1
跨越 2 个远程 HBN VTEP 并使用 L3 VNI 连接
tenant1
在 tenant1 中导入 Internet L3 VNI 路由,并使用 route-target auto 添加自己的路由。
nv set interface
vlan20-21
ip vrf tenant1
nv set vrf tenant1 evpn enable on
nv set vrf tenant1 evpn vni 20000
nv set vrf tenant1 loopback ip address 15.1
.0.1
/32
nv set vrf tenant1 loopback ip address 2001
:c001:c0de::1
/128
nv set vrf tenant1 router bgp address-family ipv4-unicast enable on
nv set vrf tenant1 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf tenant1 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf tenant1 router bgp address-family ipv6-unicast enable on
nv set vrf tenant1 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf tenant1 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf tenant1 router bgp address-family l2vpn-evpn enable on
nv set vrf tenant1 router bgp autonomous-system 6300656
nv set vrf tenant1 router bgp enable on
nv set vrf tenant1 router bgp route-export to-evpn route-target 6300656
:20000
nv set vrf tenant1 router bgp route-import
from-evpn route-target ANY:10000
nv set vrf tenant1 router bgp route-import
from-evpn route-target auto
nv set vrf tenant1 router bgp router-id 27.0
.0.17
租户 VRF
HBN 加速路由计划
以下小节从上面的代码段中选择一些 IP 端点,并检查它们的路由分发。
网关设备具有远程租户
Internet 路由使用来自出口节点的默认发起者注入。
网关-1 路由信息
BGP 通过 Internet VRF 中的子接口功能共享 Uplink。
root
@doca
-hbn-service # ip -4
route showdefault
vrf internet1default
nhid248
proto bgp metric20
root@doca
-hbn-service-bf2-s12-1
-ipmi:~# ip -6
route showdefault
vrf internet1default
nhid248
proto bgp metric20
pref medium root@doca
-hbn-service # ip nexthop get id248
; ip nexthop get id249
; ip nexthop get id250
id248
group249
/250
proto zebra id249
via fe80::202
:ff:fe00:1d dev p0_if.60
scope link proto zebra id250
via fe80::202
:ff:fe00:26
dev p1_if.60
scope link proto zebragateway1 - 外部路由 Internet VRF
本地租户路由信息:Internet 通过对等网关使用 L3 VNI 到达。
root
@hbn
:/# ip -4
route show vrf gw_tenant1default
default
encap ip id10000
src0.0
.0.0
dst27.0
.0.7
ttl0
tos0
via27.0
.0.7
dev vxlan48 proto bgp metric20
onlink root@hbn
:/# ip -6
route show vrf gw_tenant1default
default
encap ip id10000
src0.0
.0.0
dst27.0
.0.7
ttl0
tos0
via ::ffff:27.0
.0.7
dev vxlan48 proto bgp metric20
onlink pref mediumgateway1 - 外部路由 gw_tenant VRF
通过使用 DVNI CFG 的 gateway1 实现远程租户路由可达性。
考虑来自 Tenant-HBN3 上远程
tenant1
VRF 的 IP 端点。root
@hbn
:/# ip -4
route show vrf internet115.1
.0.1
/32
15.1
.0.1
encap ip id20000
src0.0
.0.0
dst27.0
.0.17
ttl0
tos0
via27.0
.0.17
dev vxlan48 proto bgp metric20
onlink root@hbn
:/# ip -6
route show vrf internet12001
:c001:c0de::1
/128
2001
:c001:c0de::1
encap ip id20000
src0.0
.0.0
dst27.0
.0.17
ttl0
tos0
via ::ffff:27.0
.0.17
dev vxlan48 proto bgp metric20
onlink pref medium
gateway1 - 路由 Internet VRF
Tenant-HBN3 路由信息
IP 端点作为 gateway1 VRF 环回,并且 VNI 的 DVNI 切换正在到达
gateway1
节点。root
@hbn
:/# ip -4
route show vrf tenant16.2
.0.1
/32
6.2
.0.1
encap ip id10000
src0.0
.0.0
dst27.0
.0.5
ttl0
tos0
via27.0
.0.5
dev vxlan48 proto bgp metric20
onlink root@hbn
:/# ip -6
route show vrf tenant12001
:cafe:feed::1
/128
2001
:cafe:feed::1
encap ip id10000
src0.0
.0.0
dst27.0
.0.5
ttl0
tos0
via ::ffff:27.0
.0.5
dev vxlan48 proto bgp metric20
onlink pref mediumtenant-hbn3 - 路由 租户 VRF
Internet VRF 默认路由正在到达远程租户 VRF。
root
@hbn
:/# ip -4
route show vrf tenant1default
default
proto bgp metric20
nexthop encap ip id10000
src0.0
.0.0
dst27.0
.0.5
ttl0
tos0
via27.0
.0.5
dev vxlan48 weight1
onlink nexthop encap ip id10000
src0.0
.0.0
dst27.0
.0.7
ttl0
tos0
via27.0
.0.7
dev vxlan48 weight1
onlink root@hbn
:/# ip -6
route show vrf tenant1default
default
proto bgp metric20
pref medium nexthop encap ip id10000
src0.0
.0.0
dst27.0
.0.5
ttl0
tos0
via ::ffff:27.0
.0.5
dev vxlan48 weight1
onlink nexthop encap ip id10000
src0.0
.0.0
dst27.0
.0.7
ttl0
tos0
via ::ffff:27.0
.0.7
dev vxlan48 weight1
onlink
tenant-hbn3 外部 - 路由 租户 VRF
网关和租户完整配置示例
网关-1 完整配置
nv set bridge domain br_default encap 802
.1Q
nv set bridge domain br_default type vlan-aware
nv set bridge domain br_default untagged 1
nv set bridge domain br_default vlan 10
,30
-31
nv set evpn enable on
nv set interface
lo ip address 27.0
.0.5
/32
nv set interface
lo ip address 2001
:c001:ff:f00d::5
/128
nv set interface
lo type loopback
nv set interface
p0_if,p1_if,pf0hpf_if,pf0vf0_if,pf0vf1_if,pf0vf2_if,pf0vf3_if,pf0vf4_if,pf1hpf_if type swp
nv set interface
p0_if.60
base-interface
p0_if
nv set interface
p0_if.60
,p1_if.60
type sub
nv set interface
p0_if.60
,p1_if.60
vlan 60
nv set interface
p0_if.60
,p1_if.60
,vlan10 ip vrf internet1
nv set interface
p1_if.60
base-interface
p1_if
nv set interface
pf0hpf_if bridge domain br_default access 30
nv set interface
pf0vf0_if bridge domain br_default access 31
nv set interface
vlan10 ip address 12.2
.0.1
/24
nv set interface
vlan10 ip address 2001
:c001:d00d::1
/96
nv set interface
vlan10 vlan 10
nv set interface
vlan10,30
-31
ip ipv4 forward on
nv set interface
vlan10,30
-31
ip ipv6 forward on
nv set interface
vlan10,30
-31
type svi
nv set interface
vlan30 ip address 45.3
.0.1
/24
nv set interface
vlan30 ip address 2001
:b055:b00c::1
/96
nv set interface
vlan30 vlan 30
nv set interface
vlan30-31
ip vrf gw_tenant1
nv set interface
vlan31 ip address 45.3
.1.1
/24
nv set interface
vlan31 ip address 2001
:b055:b00c::1
:0
:1
/96
nv set interface
vlan31 vlan 31
nv set nve vxlan arp-nd-suppress on
nv set nve vxlan enable on
nv set nve vxlan mac-learning off
nv set nve vxlan source address 27.0
.0.5
nv set platform
nv set router bgp enable on
nv set system config snippet
nv set system global anycast-mac 44
:38
:39
:42
:42
:17
nv set vrf default
router bgp address-family ipv4-unicast enable on
nv set vrf default
router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf default
router bgp address-family ipv6-unicast enable on
nv set vrf default
router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf default
router bgp address-family l2vpn-evpn enable on
nv set vrf default
router bgp autonomous-system 65552
nv set vrf default
router bgp enable on
nv set vrf default
router bgp neighbor 27.0
.0.11
peer-group rs_client
nv set vrf default
router bgp neighbor 27.0
.0.11
type numbered
nv set vrf default
router bgp neighbor 27.0
.0.12
peer-group rs_client
nv set vrf default
router bgp neighbor 27.0
.0.12
type numbered
nv set vrf default
router bgp neighbor p0_if capabilities source-address lo
nv set vrf default
router bgp neighbor p0_if peer-group fabric
nv set vrf default
router bgp neighbor p0_if type unnumbered
nv set vrf default
router bgp neighbor p1_if capabilities source-address lo
nv set vrf default
router bgp neighbor p1_if peer-group fabric
nv set vrf default
router bgp neighbor p1_if type unnumbered
nv set vrf default
router bgp path-selection multipath aspath-ignore on
nv set vrf default
router bgp peer-group fabric address-family ipv4-unicast enable on
nv set vrf default
router bgp peer-group fabric address-family ipv6-unicast enable on
nv set vrf default
router bgp peer-group fabric address-family l2vpn-evpn add-path-tx off
nv set vrf default
router bgp peer-group fabric address-family l2vpn-evpn enable off
nv set vrf default
router bgp peer-group fabric remote-as external
nv set vrf default
router bgp peer-group fabric timers connection-retry 5
nv set vrf default
router bgp peer-group fabric timers hold 30
nv set vrf default
router bgp peer-group fabric timers keepalive 10
nv set vrf default
router bgp peer-group rs_client address-family ipv4-unicast enable off
nv set vrf default
router bgp peer-group rs_client address-family ipv6-unicast enable off
nv set vrf default
router bgp peer-group rs_client address-family l2vpn-evpn add-path-tx off
nv set vrf default
router bgp peer-group rs_client address-family l2vpn-evpn enable on
nv set vrf default
router bgp peer-group rs_client multihop-ttl 5
nv set vrf default
router bgp peer-group rs_client remote-as external
nv set vrf default
router bgp peer-group rs_client timers connection-retry 5
nv set vrf default
router bgp peer-group rs_client timers hold 30
nv set vrf default
router bgp peer-group rs_client timers keepalive 10
nv set vrf default
router bgp router-id 27.0
.0.5
nv set vrf gw_tenant1 evpn enable on
nv set vrf gw_tenant1 evpn vni 30000
nv set vrf gw_tenant1 loopback ip address 15.3
.0.1
/32
nv set vrf gw_tenant1 loopback ip address 2001
:bad:c0de::1
/128
nv set vrf gw_tenant1 router bgp address-family ipv4-unicast enable on
nv set vrf gw_tenant1 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf gw_tenant1 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf gw_tenant1 router bgp address-family ipv6-unicast enable on
nv set vrf gw_tenant1 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf gw_tenant1 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf gw_tenant1 router bgp address-family l2vpn-evpn enable on
nv set vrf gw_tenant1 router bgp autonomous-system 65552
nv set vrf gw_tenant1 router bgp enable on
nv set vrf gw_tenant1 router bgp route-export to-evpn route-target 65552
:30000
nv set vrf gw_tenant1 router bgp route-import
from-evpn route-target ANY:10000
nv set vrf gw_tenant1 router bgp route-import
from-evpn route-target auto
nv set vrf gw_tenant1 router bgp router-id 27.0
.0.5
nv set vrf internet1 evpn enable on
nv set vrf internet1 evpn vni 10000
nv set vrf internet1 loopback ip address 6.2
.0.1
/32
nv set vrf internet1 loopback ip address 2001
:cafe:feed::1
/128
nv set vrf internet1 router bgp address-family ipv4-unicast enable on
nv set vrf internet1 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf internet1 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf internet1 router bgp address-family ipv6-unicast enable on
nv set vrf internet1 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf internet1 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf internet1 router bgp address-family l2vpn-evpn enable on
nv set vrf internet1 router bgp autonomous-system 65552
nv set vrf internet1 router bgp enable on
nv set vrf internet1 router bgp neighbor p0_if.60
capabilities source-address internet1
nv set vrf internet1 router bgp neighbor p0_if.60
peer-group l3_pg1
nv set vrf internet1 router bgp neighbor p0_if.60
type unnumbered
nv set vrf internet1 router bgp neighbor p1_if.60
capabilities source-address internet1
nv set vrf internet1 router bgp neighbor p1_if.60
peer-group l3_pg1
nv set vrf internet1 router bgp neighbor p1_if.60
type unnumbered
nv set vrf internet1 router bgp peer-group l3_pg1 address-family ipv4-unicast enable on
nv set vrf internet1 router bgp peer-group l3_pg1 address-family ipv6-unicast enable on
nv set vrf internet1 router bgp peer-group l3_pg1 remote-as external
nv set vrf internet1 router bgp route-export to-evpn route-target 65552
:10000
nv set vrf internet1 router bgp route-import
from-evpn route-target ANY:20000
nv set vrf internet1 router bgp route-import
from-evpn route-target ANY:30000
nv set vrf internet1 router bgp route-import
from-evpn route-target auto
nv set vrf internet1 router bgp router-id 27.0
.0.5
网关-HBN-1
网关-2 完整配置
nv set bridge domain br_default encap 802
.1Q
nv set bridge domain br_default type vlan-aware
nv set bridge domain br_default untagged 1
nv set bridge domain br_default vlan 10
,30
-31
nv set evpn enable on
nv set interface
lo ip address 27.0
.0.7
/32
nv set interface
lo ip address 2001
:c001:ff:f00d::7
/128
nv set interface
lo type loopback
nv set interface
p0_if,p1_if,pf0hpf_if,pf0vf0_if,pf0vf1_if,pf0vf2_if,pf0vf3_if,pf0vf4_if,pf1hpf_if type swp
nv set interface
p0_if.60
base-interface
p0_if
nv set interface
p0_if.60
,p1_if.60
type sub
nv set interface
p0_if.60
,p1_if.60
vlan 60
nv set interface
p0_if.60
,p1_if.60
,vlan10 ip vrf internet1
nv set interface
p1_if.60
base-interface
p1_if
nv set interface
pf0hpf_if bridge domain br_default access 30
nv set interface
pf0vf0_if bridge domain br_default access 31
nv set interface
vlan10 ip address 12.2
.1.1
/24
nv set interface
vlan10 ip address 2001
:c001:d00d::1
:0
:1
/96
nv set interface
vlan10 vlan 10
nv set interface
vlan10,30
-31
ip ipv4 forward on
nv set interface
vlan10,30
-31
ip ipv6 forward on
nv set interface
vlan10,30
-31
type svi
nv set interface
vlan30 ip address 45.3
.2.1
/24
nv set interface
vlan30 ip address 2001
:b055:b00c::2
:0
:1
/96
nv set interface
vlan30 vlan 30
nv set interface
vlan30-31
ip vrf gw_tenant1
nv set interface
vlan31 ip address 45.3
.3.1
/24
nv set interface
vlan31 ip address 2001
:b055:b00c::3
:0
:1
/96
nv set interface
vlan31 vlan 31
nv set nve vxlan arp-nd-suppress on
nv set nve vxlan enable on
nv set nve vxlan mac-learning off
nv set nve vxlan source address 27.0
.0.7
nv set platform
nv set router bgp enable on
nv set system config snippet
nv set system global anycast-mac 44
:38
:39
:42
:42
:19
nv set vrf default
router bgp address-family ipv4-unicast enable on
nv set vrf default
router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf default
router bgp address-family ipv6-unicast enable on
nv set vrf default
router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf default
router bgp address-family l2vpn-evpn enable on
nv set vrf default
router bgp autonomous-system 65554
nv set vrf default
router bgp enable on
nv set vrf default
router bgp neighbor 27.0
.0.11
peer-group rs_client
nv set vrf default
router bgp neighbor 27.0
.0.11
type numbered
nv set vrf default
router bgp neighbor 27.0
.0.12
peer-group rs_client
nv set vrf default
router bgp neighbor 27.0
.0.12
type numbered
nv set vrf default
router bgp neighbor p0_if capabilities source-address lo
nv set vrf default
router bgp neighbor p0_if peer-group fabric
nv set vrf default
router bgp neighbor p0_if type unnumbered
nv set vrf default
router bgp neighbor p1_if capabilities source-address lo
nv set vrf default
router bgp neighbor p1_if peer-group fabric
nv set vrf default
router bgp neighbor p1_if type unnumbered
nv set vrf default
router bgp path-selection multipath aspath-ignore on
nv set vrf default
router bgp peer-group fabric address-family ipv4-unicast enable on
nv set vrf default
router bgp peer-group fabric address-family ipv6-unicast enable on
nv set vrf default
router bgp peer-group fabric address-family l2vpn-evpn add-path-tx off
nv set vrf default
router bgp peer-group fabric address-family l2vpn-evpn enable off
nv set vrf default
router bgp peer-group fabric remote-as external
nv set vrf default
router bgp peer-group fabric timers connection-retry 5
nv set vrf default
router bgp peer-group fabric timers hold 30
nv set vrf default
router bgp peer-group fabric timers keepalive 10
nv set vrf default
router bgp peer-group rs_client address-family ipv4-unicast enable off
nv set vrf default
router bgp peer-group rs_client address-family ipv6-unicast enable off
nv set vrf default
router bgp peer-group rs_client address-family l2vpn-evpn add-path-tx off
nv set vrf default
router bgp peer-group rs_client address-family l2vpn-evpn enable on
nv set vrf default
router bgp peer-group rs_client multihop-ttl 5
nv set vrf default
router bgp peer-group rs_client remote-as external
nv set vrf default
router bgp peer-group rs_client timers connection-retry 5
nv set vrf default
router bgp peer-group rs_client timers hold 30
nv set vrf default
router bgp peer-group rs_client timers keepalive 10
nv set vrf default
router bgp router-id 27.0
.0.7
nv set vrf gw_tenant1 evpn enable on
nv set vrf gw_tenant1 evpn vni 30000
nv set vrf gw_tenant1 loopback ip address 15.3
.0.2
/32
nv set vrf gw_tenant1 loopback ip address 2001
:bad:c0de::2
/128
nv set vrf gw_tenant1 router bgp address-family ipv4-unicast enable on
nv set vrf gw_tenant1 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf gw_tenant1 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf gw_tenant1 router bgp address-family ipv6-unicast enable on
nv set vrf gw_tenant1 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf gw_tenant1 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf gw_tenant1 router bgp address-family l2vpn-evpn enable on
nv set vrf gw_tenant1 router bgp autonomous-system 65554
nv set vrf gw_tenant1 router bgp enable on
nv set vrf gw_tenant1 router bgp route-export to-evpn route-target 65554
:30000
nv set vrf gw_tenant1 router bgp route-import
from-evpn route-target ANY:10000
nv set vrf gw_tenant1 router bgp route-import
from-evpn route-target auto
nv set vrf gw_tenant1 router bgp router-id 27.0
.0.7
nv set vrf internet1 evpn enable on
nv set vrf internet1 evpn vni 10000
nv set vrf internet1 loopback ip address 6.2
.0.2
/32
nv set vrf internet1 loopback ip address 2001
:cafe:feed::2
/128
nv set vrf internet1 router bgp address-family ipv4-unicast enable on
nv set vrf internet1 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf internet1 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf internet1 router bgp address-family ipv6-unicast enable on
nv set vrf internet1 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf internet1 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf internet1 router bgp address-family l2vpn-evpn enable on
nv set vrf internet1 router bgp autonomous-system 65554
nv set vrf internet1 router bgp enable on
nv set vrf internet1 router bgp neighbor p0_if.60
capabilities source-address internet1
nv set vrf internet1 router bgp neighbor p0_if.60
peer-group l3_pg1
nv set vrf internet1 router bgp neighbor p0_if.60
type unnumbered
nv set vrf internet1 router bgp neighbor p1_if.60
capabilities source-address internet1
nv set vrf internet1 router bgp neighbor p1_if.60
peer-group l3_pg1
nv set vrf internet1 router bgp neighbor p1_if.60
type unnumbered
nv set vrf internet1 router bgp peer-group l3_pg1 address-family ipv4-unicast enable on
nv set vrf internet1 router bgp peer-group l3_pg1 address-family ipv6-unicast enable on
nv set vrf internet1 router bgp peer-group l3_pg1 remote-as external
nv set vrf internet1 router bgp route-export to-evpn route-target 65554
:10000
nv set vrf internet1 router bgp route-import
from-evpn route-target ANY:20000
nv set vrf internet1 router bgp route-import
from-evpn route-target ANY:30000
nv set vrf internet1 router bgp route-import
from-evpn route-target auto
nv set vrf internet1 router bgp router-id 27.0
.0.7
网关-HBN-2
租户-HBN-3 完整配置
nv set bridge domain br_default encap 802
.1Q
nv set bridge domain br_default type vlan-aware
nv set bridge domain br_default untagged 1
nv set bridge domain br_default vlan 20
-21
nv set evpn enable on
nv set interface
lo ip address 27.0
.0.17
/32
nv set interface
lo ip address 2001
:c001:ff:f00d::11
/128
nv set interface
lo type loopback
nv set interface
p0-1
,pf0hpf,pf0vf0-12
,pf1hpf,pf1vf0-4
type swp
nv set interface
pf0hpf bridge domain br_default access 20
nv set interface
pf0vf0 bridge domain br_default access 21
nv set interface
vlan20 ip address 45.1
.0.1
/24
nv set interface
vlan20 ip address 2001
:c001:b00c::1
/96
nv set interface
vlan20 vlan 20
nv set interface
vlan20-21
ip ipv4 forward on
nv set interface
vlan20-21
ip ipv6 forward on
nv set interface
vlan20-21
ip vrf tenant1
nv set interface
vlan20-21
type svi
nv set interface
vlan21 ip address 45.1
.1.1
/24
nv set interface
vlan21 ip address 2001
:c001:b00c::1
:0
:1
/96
nv set interface
vlan21 vlan 21
nv set nve vxlan arp-nd-suppress on
nv set nve vxlan enable on
nv set nve vxlan mac-learning off
nv set nve vxlan source address 27.0
.0.17
nv set platform
nv set router bgp enable on
nv set system global anycast-mac 44
:38
:39
:42
:42
:21
nv set vrf default
router bgp address-family ipv4-unicast enable on
nv set vrf default
router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf default
router bgp address-family ipv6-unicast enable on
nv set vrf default
router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf default
router bgp address-family l2vpn-evpn enable on
nv set vrf default
router bgp autonomous-system 6300656
nv set vrf default
router bgp enable on
nv set vrf default
router bgp neighbor 27.0
.0.11
peer-group rs_client
nv set vrf default
router bgp neighbor 27.0
.0.11
type numbered
nv set vrf default
router bgp neighbor 27.0
.0.12
peer-group rs_client
nv set vrf default
router bgp neighbor 27.0
.0.12
type numbered
nv set vrf default
router bgp neighbor p0 capabilities source-address lo
nv set vrf default
router bgp neighbor p0 peer-group fabric
nv set vrf default
router bgp neighbor p0 type unnumbered
nv set vrf default
router bgp neighbor p1 capabilities source-address lo
nv set vrf default
router bgp neighbor p1 peer-group fabric
nv set vrf default
router bgp neighbor p1 type unnumbered
nv set vrf default
router bgp path-selection multipath aspath-ignore on
nv set vrf default
router bgp peer-group fabric address-family ipv4-unicast enable on
nv set vrf default
router bgp peer-group fabric address-family ipv6-unicast enable on
nv set vrf default
router bgp peer-group fabric address-family l2vpn-evpn add-path-tx off
nv set vrf default
router bgp peer-group fabric address-family l2vpn-evpn enable off
nv set vrf default
router bgp peer-group fabric remote-as external
nv set vrf default
router bgp peer-group fabric timers connection-retry 5
nv set vrf default
router bgp peer-group fabric timers hold 30
nv set vrf default
router bgp peer-group fabric timers keepalive 10
nv set vrf default
router bgp peer-group rs_client address-family ipv4-unicast enable off
nv set vrf default
router bgp peer-group rs_client address-family ipv6-unicast enable off
nv set vrf default
router bgp peer-group rs_client address-family l2vpn-evpn add-path-tx off
nv set vrf default
router bgp peer-group rs_client address-family l2vpn-evpn enable on
nv set vrf default
router bgp peer-group rs_client multihop-ttl 5
nv set vrf default
router bgp peer-group rs_client remote-as external
nv set vrf default
router bgp peer-group rs_client timers connection-retry 5
nv set vrf default
router bgp peer-group rs_client timers hold 30
nv set vrf default
router bgp peer-group rs_client timers keepalive 10
nv set vrf default
router bgp router-id 27.0
.0.17
nv set vrf tenant1 evpn enable on
nv set vrf tenant1 evpn vni 20000
nv set vrf tenant1 loopback ip address 15.1
.0.1
/32
nv set vrf tenant1 loopback ip address 2001
:c001:c0de::1
/128
nv set vrf tenant1 router bgp address-family ipv4-unicast enable on
nv set vrf tenant1 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf tenant1 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf tenant1 router bgp address-family ipv6-unicast enable on
nv set vrf tenant1 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf tenant1 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf tenant1 router bgp address-family l2vpn-evpn enable on
nv set vrf tenant1 router bgp autonomous-system 6300656
nv set vrf tenant1 router bgp enable on
nv set vrf tenant1 router bgp route-export to-evpn route-target 6300656
:20000
nv set vrf tenant1 router bgp route-import
from-evpn route-target ANY:10000
nv set vrf tenant1 router bgp route-import
from-evpn route-target auto
nv set vrf tenant1 router bgp router-id 27.0
.0.17
租户-HBN-3
租户-HBN-4 完整配置
nv set bridge domain br_default encap 802
.1Q
nv set bridge domain br_default type vlan-aware
nv set bridge domain br_default untagged 1
nv set bridge domain br_default vlan 20
-21
nv set evpn enable on
nv set interface
lo ip address 27.0
.0.19
/32
nv set interface
lo ip address 2001
:c001:ff:f00d::13
/128
nv set interface
lo type loopback
nv set interface
p0-1
,pf0hpf,pf0vf0-12
,pf1hpf,pf1vf0-4
type swp
nv set interface
pf0hpf bridge domain br_default access 20
nv set interface
pf0vf0 bridge domain br_default access 21
nv set interface
vlan20 ip address 45.1
.2.1
/24
nv set interface
vlan20 ip address 2001
:c001:b00c::2
:0
:1
/96
nv set interface
vlan20 vlan 20
nv set interface
vlan20-21
ip ipv4 forward on
nv set interface
vlan20-21
ip ipv6 forward on
nv set interface
vlan20-21
ip vrf tenant1
nv set interface
vlan20-21
type svi
nv set interface
vlan21 ip address 45.1
.3.1
/24
nv set interface
vlan21 ip address 2001
:c001:b00c::3
:0
:1
/96
nv set interface
vlan21 vlan 21
nv set nve vxlan arp-nd-suppress on
nv set nve vxlan enable on
nv set nve vxlan mac-learning off
nv set nve vxlan source address 27.0
.0.19
nv set platform
nv set router bgp enable on
nv set system global anycast-mac 44
:38
:39
:42
:42
:23
nv set vrf default
router bgp address-family ipv4-unicast enable on
nv set vrf default
router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf default
router bgp address-family ipv6-unicast enable on
nv set vrf default
router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf default
router bgp address-family l2vpn-evpn enable on
nv set vrf default
router bgp autonomous-system 6300658
nv set vrf default
router bgp enable on
nv set vrf default
router bgp neighbor 27.0
.0.11
peer-group rs_client
nv set vrf default
router bgp neighbor 27.0
.0.11
type numbered
nv set vrf default
router bgp neighbor 27.0
.0.12
peer-group rs_client
nv set vrf default
router bgp neighbor 27.0
.0.12
type numbered
nv set vrf default
router bgp neighbor p0 capabilities source-address lo
nv set vrf default
router bgp neighbor p0 peer-group fabric
nv set vrf default
router bgp neighbor p0 type unnumbered
nv set vrf default
router bgp neighbor p1 capabilities source-address lo
nv set vrf default
router bgp neighbor p1 peer-group fabric
nv set vrf default
router bgp neighbor p1 type unnumbered
nv set vrf default
router bgp path-selection multipath aspath-ignore on
nv set vrf default
router bgp peer-group fabric address-family ipv4-unicast enable on
nv set vrf default
router bgp peer-group fabric address-family ipv6-unicast enable on
nv set vrf default
router bgp peer-group fabric address-family l2vpn-evpn add-path-tx off
nv set vrf default
router bgp peer-group fabric address-family l2vpn-evpn enable off
nv set vrf default
router bgp peer-group fabric remote-as external
nv set vrf default
router bgp peer-group fabric timers connection-retry 5
nv set vrf default
router bgp peer-group fabric timers hold 30
nv set vrf default
router bgp peer-group fabric timers keepalive 10
nv set vrf default
router bgp peer-group rs_client address-family ipv4-unicast enable off
nv set vrf default
router bgp peer-group rs_client address-family ipv6-unicast enable off
nv set vrf default
router bgp peer-group rs_client address-family l2vpn-evpn add-path-tx off
nv set vrf default
router bgp peer-group rs_client address-family l2vpn-evpn enable on
nv set vrf default
router bgp peer-group rs_client multihop-ttl 5
nv set vrf default
router bgp peer-group rs_client remote-as external
nv set vrf default
router bgp peer-group rs_client timers connection-retry 5
nv set vrf default
router bgp peer-group rs_client timers hold 30
nv set vrf default
router bgp peer-group rs_client timers keepalive 10
nv set vrf default
router bgp router-id 27.0
.0.19
nv set vrf tenant1 evpn enable on
nv set vrf tenant1 evpn vni 20000
nv set vrf tenant1 loopback ip address 15.1
.0.2
/32
nv set vrf tenant1 loopback ip address 2001
:c001:c0de::2
/128
nv set vrf tenant1 router bgp address-family ipv4-unicast enable on
nv set vrf tenant1 router bgp address-family ipv4-unicast redistribute connected enable on
nv set vrf tenant1 router bgp address-family ipv4-unicast route-export to-evpn enable on
nv set vrf tenant1 router bgp address-family ipv6-unicast enable on
nv set vrf tenant1 router bgp address-family ipv6-unicast redistribute connected enable on
nv set vrf tenant1 router bgp address-family ipv6-unicast route-export to-evpn enable on
nv set vrf tenant1 router bgp address-family l2vpn-evpn enable on
nv set vrf tenant1 router bgp autonomous-system 6300658
nv set vrf tenant1 router bgp enable on
nv set vrf tenant1 router bgp route-export to-evpn route-target 6300658
:20000
nv set vrf tenant1 router bgp route-import
from-evpn route-target ANY:10000
nv set vrf tenant1 router bgp route-import
from-evpn route-target auto
nv set vrf tenant1 router bgp router-id 27.0
.0.19
访问控制列表
租户-HBN4
访问控制列表 (ACL) 是一组用于过滤网络流量的规则。这些规则用于指定必须在网络设备接口处允许或阻止的流量流。ACL 有两种类型
无状态 ACL – 应用于单个数据包的规则。它们单独检查每个数据包,并根据数据包标头信息和规则指定的匹配条件来允许/阻止数据包。
无状态 ACL
有状态 ACL – 应用于流量会话/连接的规则。它们根据数据包所属的会话/连接的状态检查每个数据包,以确定是否允许/阻止数据包。
HBN 支持为 IPv4 数据包、IPv6 数据包和以太网 (MAC) 帧配置无状态 ACL。以下示例描述了如何使用 NVUE 和扁平文件 (cl-acltool
) 为每种情况配置无状态 ACL。
无状态 ACL 可以绑定到主机表示端口(例如,pf0hpf_if、pf0vf0_if 等)和主机表示端口的子接口(pf0hpf_if.500、pf0vf0_if.999 等)。
无状态 ACL 的 NVUE 示例
NVUE IPv4 ACL 示例
root@hbn01-host01:~# nv set acl acl1_ingress type ipv4
root@hbn01-host01:~# nv set acl acl1_ingress rule 100 match ip protocol udp
root@hbn01-host01:~# nv set acl acl1_ingress rule 100 match ip dest-port 67
root@hbn01-host01:~# nv set acl acl1_ingress rule 100 match ip source-port 68
root@hbn01-host01:~# nv set acl acl1_ingress rule 100 action permit
以下是一个入口 IPv4 ACL 的示例,该 ACL 允许 DHCP 请求数据包在 pf0vf1_if.999
接口上向 DHCP 服务器入口。
root@hbn01-host01:~# nv set interface pf0vf1_if.999 acl acl1_ingress inbound
root@hbn01-host01:~# nv config apply
将入口 IPv4 ACL 绑定到 BlueField 的主机表示端口 pf0vf1_if.999
的入站方向
root@hbn01-host01:~# nv set acl acl2_egress type ipv4
root@hbn01-host01:~# nv set acl acl2_egress rule 200 match ip protocol udp
root@hbn01-host01:~# nv set acl acl2_egress rule 200 match ip dest-port 68
root@hbn01-host01:~# nv set acl acl2_egress rule 200 match ip source-port 67
root@hbn01-host01:~# nv set acl acl2_egress rule 200 action permit
以下是一个出口 IPv4 ACL 的示例,该 ACL 允许 DHCP 响应数据包从 pf0vf1_if.999
端口向 DHCP 客户端出口。
root@hbn01-host01:~# nv set interface pf0vf1_if.999 acl acl2_egress outbound
root@hbn01-host01:~# nv config apply
将出口 IPv4 ACL 绑定到 BlueField 的主机表示端口 pf0vf1_if.999
的出站方向
NVUE IPv6 ACL 示例
root@hbn01-host01:~# nv set acl acl5_ingress type ipv6
root@hbn01-host01:~# nv set acl acl5_ingress rule 100 match ip protocol tcp
root@hbn01-host01:~# nv set acl acl5_ingress rule 100 match ip dest-ip 48:2034::80:9
root@hbn01-host01:~# nv set acl acl5_ingress rule 100 action permit
以下是一个入口 IPv6 ACL 的示例,该 ACL 允许在端口 pf0hpf_if
上入口的具有匹配 dest-ip
和协议 tcp 的流量。
root@hbn01-host01:~# nv set interface pf0hpf_if acl acl5_ingress inbound
root@hbn01-host01:~# nv config apply
将入口 IPv6 ACL 绑定到 BlueField 的主机表示端口 pf0hpf_if
的入站方向
root@hbn01-host01:~# nv set acl acl6_egress type ipv6
root@hbn01-host01:~# nv set acl acl6_egress rule 101 match ip protocol tcp
root@hbn01-host01:~# nv set acl acl6_egress rule 101 match ip source-ip 48:2034::80:9
root@hbn01-host01:~# nv set acl acl6_egress rule 101 action permit
以下是一个出口 IPv6 ACL 的示例,该 ACL 允许从端口 pf0hpf_if
出口的具有匹配 source-ip
和协议 tcp 的流量。
root@hbn01-host01:~# nv set interface pf0hpf_if acl acl6_egress outbound
root@hbn01-host01:~# nv config apply
将出口 IPv6 ACL 绑定到 BlueField 的主机表示端口 pf0hpf_if
的出站方向
NVUE MAC ACL 示例
root@hbn01-host01:~# nv set acl acl3_ingress type mac
root@hbn01-host01:~# nv set acl acl3_ingress rule 1 match mac source-mac 00:00:00:00:00:0a
root@hbn01-host01:~# nv set acl acl3_ingress rule 1 match mac dest-mac 00:00:00:00:00:0b
root@hbn01-host01:~# nv set interface pf0hpf_if acl acl3_ingress inbound
以下是一个入口 MAC ACL 的示例,该 ACL 允许入口到端口 pf0hpf_if
的具有匹配 source-mac
和 dest-mac
的流量。
root@hbn01-host01:~# nv set interface pf0hpf_if acl acl3_ingress inbound
root@hbn01-host01:~# nv config apply
将入口 MAC ACL 绑定到 BlueField 的主机表示端口 pf0hpf_if
的入站方向
root@hbn01-host01:~# nv set acl acl4_egress type mac
root@hbn01-host01:~# nv set acl acl4_egress rule 2 match mac source-mac 00:00:00:00:00:0b
root@hbn01-host01:~# nv set acl acl4_egress rule 2 match mac dest-mac 00:00:00:00:00:0a
root@hbn01-host01:~# nv set acl acl4_egress rule 2 action permit
以下是一个出口 MAC ACL 的示例,该 ACL 允许从端口 pf0hpf_if
出口的具有匹配 source-mac
和 dest-mac
的流量。
root@hbn01-host01:~# nv set interface pf0hpf_if acl acl4_egress outbound
root@hbn01-host01:~# nv config apply
将出口 MAC ACL 绑定到 BlueField 的主机表示端口 pf0hpf_if
的出站方向
无状态 ACL 的扁平文件 (cl-acltool) 示例
对于上面引用的相同示例,以下是相应的 ACL 规则,必须在 /etc/cumulus/acl/policy.d/<rule_name.rules>
下配置,然后调用 cl-acltool -i
。 /etc/cumulus/acl/policy.d/<rule_name.rules>
中的规则是使用 Linux iptables/ip6tables/ebtables 配置的。
扁平文件 IPv4 ACL 示例
[iptables]
## ACL acl1_ingress in dir inbound on interface pf1vf1_if.999 ##
-t filter -A FORWARD -m physdev --physdev-in pf1vf1_if.999 -p udp --sport 68 --dport 67 -j ACCEPT
以下示例配置一个入口 IPv4 ACL 规则,该规则在 /etc/cumulus/acl/policy.d/<rule_name.rules>
下匹配 DHCP 请求,入口接口作为 BlueField 的主机表示器,然后调用 cl-acltool -i
[iptables]
## ACL acl2_egress in dir outbound on interface pf1vf1_if.999 ##
-t filter -A FORWARD -m physdev --physdev-out pf1vf1_if.999 -p udp --sport 67 --dport 68 -j ACCEPT
以下示例配置一个出口 IPv4 ACL 规则,该规则在 /etc/cumulus/acl/policy.d/<rule_name.rules>
下匹配 DHCP 响应,出口接口作为 BlueField 的主机表示器,然后调用 cl-acltool -i
扁平文件 IPv6 ACL 示例
[ip6tables]
## ACL acl5_ingress in dir inbound on interface pf0hpf_if ##
-t filter -A FORWARD -m physdev --physdev-in pf0hpf_if -d 48:2034::80:9 -p tcp -j ACCEPT
以下示例配置了一个出口 IPv6 ACL 规则,该规则在 /etc/cumulus/acl/policy.d/<rule_name.rules>
下使用 source-ip
和 tcp
协议进行匹配,并将出口接口作为 BlueField 的主机表示,然后调用 cl-acltool -i
[ip6tables]
## ACL acl6_egress in dir outbound on interface pf0hpf_if ##
-t filter -A FORWARD -m physdev --physdev-out pf0hpf_if -s 48:2034::80:9 -p tcp -j ACCEPT
扁平文件 MAC ACL 示例
以下示例配置了一个入口 MAC ACL 规则,该规则在 /etc/cumulus/acl/policy.d/<rule_name.rules>
下使用 source-mac
和 dest-mac
进行匹配,并将入口接口作为 BlueField 的主机表示,然后调用 cl-acltool -i
[ebtables]
## ACL acl3_ingress in dir inbound on interface pf0hpf_if ##
-t filter -A FORWARD -m physdev --physdev-in pf0hpf_if -s 00:00:00:00:00:0a/ff:ff:ff:ff:ff:ff -d 00:00:00:00:00:0b/ff:ff:ff:ff:ff:ff -j ACCEPT
以下示例配置了一个出口 MAC ACL 规则,该规则在 /etc/cumulus/acl/policy.d/<rule_name.rules>
下使用 source-mac
和 dest-mac
进行匹配,并将出口接口作为 BlueField 的主机表示,然后调用 cl-acltool -i
[ebtables]
## ACL acl4_egress in dir outbound on interface pf0hpf_if ##
-t filter -A FORWARD -m physdev --physdev-out pf0hpf_if -s 00:00:00:00:00:0b/ff:ff:ff:ff:ff:ff -d 00:00:00:00:00:0a/ff:ff:ff:ff:ff:ff -j ACCEPT
有状态 ACL
状态 ACL 有助于监控和跟踪流量流,以实施基于流的流量过滤(与基于数据包的无状态 ACL 不同)。HBN 使用反射 ACL 机制支持状态 ACL。反射 ACL 机制用于允许从“网络内部”到“网络外部”发起连接,并仅允许从“网络外部”到“网络内部”对已发起连接的回复(或反之亦然)。
HBN 支持 IPv4 流量的状态 ACL 配置。TCP、UDP 和 ICMP 协议支持状态 ACL 配置。
状态 ACL 可以绑定到主机表示端口(例如,pf0hpf_if、pf0vf0_if 等)和主机表示端口的子接口(pf0hpf_if.500、pf0vf0_if.999 等)。
状态 ACL 可以应用于原生路由流量(EVPN 部署中的南北向底层路由流量)、EVPN 桥接流量(EVPN 部署中的东西向叠加桥接/L2 流量)和 EVPN 路由流量(EVPN 部署中的东西向叠加路由流量)。应用于原生路由流量的状态 ACL 称为“原生 L3 状态 ACL”。应用于 EVPN 桥接流量和 EVPN 路由流量的状态 ACL 分别称为“EVPN-L2 状态 ACL”和“EVPN-L3 状态 ACL”。
HBN 中的状态 ACL 默认启用。要启用状态 ACL 功能,请使用以下 NVUE 命令
root@hbn03-host00:~# nv set system reflexive-acl enable
root@hbn03-host00:~# nv config apply
如果使用扁平文件配置(而不是 NVUE),请编辑文件 /etc/cumulus/nl2docad.d/acl.conf
并将 knob rflx.reflexive_acl_enable
设置为 TRUE
。要应用此更改,请执行
root@hbn03-host00:~# supervisorctl start nl2doca-reload
状态 ACL 的 NVUE 示例
以下示例说明如何允许主机(BlueField 所在的主机)发起的 HTTP (TCP) 连接连接到外部网络上的 HTTP 服务器(IP 地址为 11.11.11.11)。必须为 CONNTRACK 条目配置两组与 CONNTRACK 状态匹配的 ACL,才能在内核中建立 CONNTRACK 条目,该条目将被卸载到硬件
配置一个 ACL 规则,该规则匹配 TCP/HTTP 连接/流详细信息以及 NEW、ESTABLISHED 的 CONNTRACK 状态,并将其绑定到入站方向的 SVI。
配置一个 ACL 规则,该规则匹配 TCP/HTTP 连接/流详细信息以及 ESTABLISHED 的 CONNTRACK 状态,并将其绑定到出站方向的 SVI。
状态 ACL 应绑定到物理接口。在此示例中,物理接口是 pf1vf7_if
。
配置入口 ACL 规则
root@hbn03-host00:~# nv set acl allow_tcp_conn_from_host rule 11 action permit root@hbn03-host00:~# nv set acl allow_tcp_conn_from_host rule 11 match conntrack new root@hbn03-host00:~# nv set acl allow_tcp_conn_from_host rule 11 match conntrack established root@hbn03-host00:~# nv set acl allow_tcp_conn_from_host rule 11 match ip dest-ip 11.11.11.11/32 root@hbn03-host00:~# nv set acl allow_tcp_conn_from_host rule 11 match ip dest-port 80 root@hbn03-host00:~# nv set acl allow_tcp_conn_from_host rule 11 match ip protocol tcp root@hbn03-host00:~# nv set acl allow_tcp_conn_from_host type ipv4
将此 ACL 绑定到入站方向的物理接口
root@hbn03-host00:~# nv set interface pf1vf7_if acl allow_tcp_conn_from_host inbound root@hbn03-host00:~# nv config apply
配置出口 ACL 规则
root@hbn03-host00:~# nv set acl allow_tcp_resp_from_server rule 21 action permit root@hbn03-host00:~# nv set acl allow_tcp_resp_from_server rule 21 match conntrack established root@hbn03-host00:~# nv set acl allow_tcp_resp_from_server rule 21 match ip protocol tcp root@hbn03-host00:~# nv set acl allow_tcp_resp_from_server type ipv4 root@hbn03-host00:~# nv config apply
将此 ACL 绑定到出站方向的物理接口
root@hbn03-host00:~# nv set interface pf1vf7_if acl allow_tcp_resp_from_server outbound root@hbn03-host00:~# nv config apply
状态 ACL 的扁平文件 (cl-acltool) 示例
对于上面引用的状态 ACL 的相同 NVUE 示例(IP 地址为 11.11.11.11 的外部网络上的 HTTP 服务器),以下是在 /etc/cumulus/acl/policy.d/<rule_name.rules>
下必须配置的相应 ACL 规则,然后调用 cl-acltool -i
以在 BlueField 硬件中安装规则。
配置一个入口 ACL 规则,该规则匹配 TCP 流详细信息和 NEW、ESTABLISHED 的 CONNTRACK 状态,位于
/etc/cumulus/acl/policy.d/stateful_acl.rules
下,并将入口接口作为 SVI,然后调用cl-acltool -i
[iptables] ## ACL allow_tcp_conn_from_host in dir inbound on interface pf1vf7_if ## -t filter -A FORWARD -m physdev --physdev-in pf1vf7_if -p tcp –d 11.11.11.11/32 --dport 80 -m conntrack --ctstate EST,NEW -j ACCEPT -m mark --mark 0xdead
配置一个出口 ACL 规则,该规则匹配 TCP 流和 ESTABLISHED、RELATED 的 CONNTRACK 状态,位于
/etc/cumulus/acl/policy.d/stateful_acl.rules
文件下,并将出口接口作为 SVI,然后调用cl-acltool -i
[iptables] ## ACL allow_tcp_resp_from_server in dir outbound on interface pf1vf7_if ## -t filter -A FORWARD -m physdev --physdev-out pf1vf7_if -p tcp -s 11.11.11.11/32 --sport 80 -m conntrack --ctstate EST -j ACCEPT -m mark --mark 0xdead
网络地址转换
网络地址转换 (NAT) 允许网络对内部流量使用一组 IP 地址,而对外部流量使用另一组 IP 地址。除了防止 IPv4 地址耗尽外,NAT 还允许在内部使用私有地址空间,同时仍然提供对 Internet 的访问。NAT 本质上由一组用于在公共和私有 IP 地址和端口之间进行转换的规则组成。静态和动态 NAT 规则都在出站或入站上行链路接口或上行链路端口的子接口上配置。这些 NAT 规则应应用于所有面向公共出口和入口的上行链路接口及其子接口。
HBN 支持源 NAT (SNAT),它允许来自私有网络的流量通过更改数据包的源地址到达公共网络。HBN 还支持目标 NAT (DNAT),它允许来自公共网络的入站流量通过更改数据包的目标地址到达私有网络。HBN SNAT 和 DNAT 支持 IPv4 和三种协议:ICMP、TCP 和 UDP。
SNAT 和 DNAT 可以配置为两种类型
静态 NAT – 提供一个私有 IP 地址与单个公共地址之间的永久映射。静态 NAT 是无状态的,不使用连接跟踪。静态 NAT 默认启用,并支持端口地址转换 (PAT),私有端口和公共端口之间存在一对一映射。静态 NAT 映射规则不会从规则表中刷新或超时。
动态 NAT – 这是一种有状态 NAT,它使用 OVS 连接跟踪基础设施来监控 NAT 连接。动态 NAT 默认禁用,用户必须先启用动态模式,然后才能设置动态 NAT 规则(请参阅“动态 NAT 配置”部分)。动态 NAT 将私有 IP 地址映射到一系列公共地址,并支持 PAT 的端口范围。用户负责管理动态 NAT 和 PAT 配置中使用的 IP 地址和端口池。建议使用 2 的幂的 NAT IP 地址范围。
源 NAT (SNAT)
静态 SNAT 配置
静态 NAT 默认启用,并且是无状态的。
静态 SNAT 的 NVUE 示例
NVUE ICMP 示例
以下示例说明了一个 SNAT 规则,该规则将匹配源 IP 30.30.30.2 的 ICMP 数据包转换为源 IP 100.100.100.2
nv set acl acl_1 rule 1
action source-nat translate-ip 100.100
.100.1
nv set acl acl_1 rule 1
match ip protocol icmp
nv set acl acl_1 rule 1
match ip source-ip 30.30
.30.2
nv set acl acl_1 type ipv4
将出口 ICMP NAT ACL 绑定到 BlueField 的上行链路端口 p0_if
和 p1_if
的出站方向
nv set interface
p0_if acl acl_1 outbound
nv set interface
p1_if acl acl_1 outbound
带 PAT 的 NVUE UDP 示例
以下示例说明了一个 SNAT 规则,该规则将匹配源 IP 21.1.0.18 和源端口 10011 的 UDP 数据包转换为源 IP 199.5.1.1 和源端口 11011
nv set acl stnatpat rule 1
action source-nat translate-ip 199.5
.1.1
nv set acl stnatpat rule 1
action source-nat translate-port 11011
nv set acl stnatpat rule 1
match ip protocol udp
nv set acl stnatpat rule 1
match ip source-ip 21.1
.0.18
nv set acl stnatpat rule 1
match ip source-port 10011
nv set acl stnatpat type ipv4
将出口 UDP NAT ACL 绑定到 BlueField 的上行链路端口 p0_if.10
和 p1_if.10
的出站方向
nv set interface
p0_if.10
,p1_if.10
acl stnatpat outbound
静态 SNAT 的扁平文件 (cl-acltool) 示例
[iptables]
## ACL acl_1 in dir outbound on interface
p0_if ##
-t nat -A POSTROUTING -m physdev --physdev-out p0_if -s 30.30
.30.2
-p icmp -j SNAT --to-source 100.100
.100.1
-m mark --mark 0xdead
## ACL acl_1 in dir outbound on interface
p1_if ##
-t nat -A POSTROUTING -m physdev --physdev-out p1_if -s 30.30
.30.2
-p icmp -j SNAT --to-source 100.100
.100.1
-m mark --mark 0xdead
## ACL stnatpat in dir outbound on interface
p0_if.10
##
-t nat -A POSTROUTING -o p0_if.10
-s 21.1
.0.18
-p udp --sport 10011
-j SNAT --to-source 199.5
.1.1
:11011
-m mark --mark 0xdead
## ACL stnatpat in dir outbound on interface
p1_if.10
##
-t nat -A POSTROUTING -o p1_if.10
-s 21.1
.0.18
-p udp --sport 10011
-j SNAT --to-source 199.5
.1.1
:11011
-m mark --mark 0xdead
动态 NAT 配置
动态 NAT 是有状态的,并且依赖于底层连接跟踪。
启用 NVUE 动态 NAT
nv set system nat mode dynamic
动态 SNAT 的 NVUE 示例
带 TCP 的 NVUE 动态 SNAT
以下示例说明了一个 SNAT 规则,该规则将匹配源 IP 21.1.0.0/16 的 TCP 数据包转换为 199.10.0.1 到 199.10.255.254 的 IP 范围
nv set acl dnat1 rule 1
action source-nat translate-ip 199.10
.0.1
to 199.10
.255.254
nv set acl dnat1 rule 1
match ip protocol tcp
nv set acl dnat1 rule 1
match ip source-ip 21.1
.0.0
/16
nv set acl dnat1 type ipv4
将出口 TCP NAT 绑定到上行链路子接口 p0_if.10
和 p1_if.10
nv set interface
p0_if.10
,p1_if.10
acl dnat1 outbound
带 TCP 和 PAT 的 NVUE 动态 SNAT
以下示例说明了一个 SNAT 规则,该规则将匹配源 IP 30.30.30.0/29 和源端口 5000 的 TCP 数据包转换为 100.100.100.8 到 100.100.100.15 的 IP 范围和源端口 10000
nv set acl acl_2 rule 1
action source-nat translate-ip 100.100
.100.8
to 100.100
.100.15
nv set acl acl_2 rule 1
action source-nat translate-port 10000
nv set acl acl_2 rule 1
match ip protocol tcp
nv set acl acl_2 rule 1
match ip source-ip 30.30
.30.0
/29
nv set acl acl_2 rule 1
match ip source-port 5000
nv set acl acl_2 type ipv4
将出口 TCP NAT 绑定到上行链路子接口 p0_if
和 p1_if
nv set interface
p0_if,p1_if acl acl_2 outbound
动态 SNAT 的扁平文件 (cl-acltool) 示例
## ACL dnat1 in dir outbound on interface
p0_if.10
##
-t nat -A POSTROUTING -o p0_if.10
-s 21.1
.0.0
/16
-p tcp -j SNAT --to-source 199.10
.0.1
-199.10
.255.254
-m mark --mark 0xdead
## ACL dnat1 in dir outbound on interface
p1_if.10
##
-t nat -A POSTROUTING -o p1_if.10
-s 21.1
.0.0
/16
-p tcp -j SNAT --to-source 199.10
.0.1
-199.10
.255.254
-m mark --mark 0xdead
## ACL acl_2 in dir outbound on interface
p0_if ##
-t nat -A POSTROUTING -m physdev --physdev-out p0_if -s 30.30
.30.0
/29
-p tcp --sport 5000
-j SNAT --to-source 100.100
.100.8
-100.100
.100.15
:10000
-m mark --mark 0xdead
## ACL acl_2 in dir outbound on interface
p1_if ##
-t nat -A POSTROUTING -m physdev --physdev-out p1_if -s 30.30
.30.0
/29
-p tcp --sport 5000
-j SNAT --to-source 100.100
.100.8
-100.100
.100.15
:10000
-m mark --mark 0xdead
以下示例说明了如何使用可用范围内的唯一端口号将私有 IP 地址池转换为公共地址
nv set acl dnatpat1 rule 1
action source-nat translate-ip 199.9
.1.1
nv set acl dnatpat1 rule 1
match ip protocol tcp
nv set acl dnatpat1 rule 1
match ip source-ip 21.1
.0.0
/16
nv set acl dnatpat1 rule 1
action source-nat translate-port 10001
-24000
nv set acl dnatpat1 type ipv4
nv set interface
p0_if.10
,p1_if.10
acl dnatpat1 outbound
目标 NAT (DNAT)
静态 DNAT 配置
静态 NAT 默认启用,并且是无状态的。
静态 DNAT 的 NVUE 示例
NVUE ICMP 示例
以下示例说明了一个 SNAT 规则,该规则将匹配入站目标 IP 100.100.100.1 的 ICMP 数据包转换为目标 IP 30.30.30.2
nv set acl acl_1 rule 1
action dest-nat translate-ip 30.30
.30.2
nv set acl acl_1 rule 1
match ip protocol icmp
nv set acl acl_1 rule 1
match ip dest-ip 100.100
.100.1
nv set acl acl_1 type ipv4
将入口 ICMP NAT ACL 绑定到 BlueField 的上行链路端口 p0_if 和 p1_if 的入站方向
nv set interface
p0_if acl acl_1 inbound
nv set interface
p1_if acl acl_1 inbound
带 PAT 的 NVUE UDP 示例
静态 DNAT 的扁平文件 (cl-acltool) 示例
## ACL acl_1 in dir inbound on interface
p0_if ##
-t nat -A PREROUTING -m physdev --physdev-in p0_if -d 100.100
.100.1
-p icmp -j DNAT --to-destination 30.30
.30.2
-m mark --mark 0xdead
## ACL acl_1 in dir inbound on interface
p1_if ##
-t nat -A PREROUTING -m physdev --physdev-in p1_if -d 100.100
.100.1
-p icmp -j DNAT --to-destination 30.30
.30.2
-m mark --mark 0xdead
动态 DNAT 配置
这是 DNAT 和 SNAT 的通用配置,因此请参阅“动态 NAT 配置”部分。
启用 NVUE 动态 DNAT
动态 DNAT 的 NVUE 示例
带 TCP 和 PAT 的 NVUE 动态 DNAT
以下示例说明了一个 D NAT 规则,该规则将匹配目标 IP 100.100.100.1/29 和目标端口 5000 的 TCP 数据包转换为 30.30.30.1 到 30.30.30.6 的 IP 范围和转换后的目标端口 1234
nv set acl acl_3 rule 1
action dest-nat translate-ip 30.30
.30.1
to 30.30
.30.6
nv set acl acl_3 rule 1
action dest-nat translate-port 1234
nv set acl acl_3 rule 1
match ip protocol tcp
nv set acl acl_3 rule 1
match ip dest-ip 100.100
.100.1
/29
nv set acl acl_3 rule 1
match ip dest-port 5000
nv set acl acl_3 type ipv4
将入口 TCP NAT ACL 绑定到上行链路子接口 p0_if
和 p1_if
nv set interface
p0_if acl acl_6 inbound
nv set interface
p1_if acl acl_6 inbound
动态 DNAT 的扁平文件 (cl-acltool) 示例
## ACL acl_6 in dir inbound on interface
p0_if ##
-t nat -A PREROUTING -m physdev --physdev-in p0_if -d 100.100
.100.1
/29
-p udp -j DNAT --to-destination 30.30
.30.1
-30.30
.30.6
-m mark --mark 0xdead
## ACL acl_6 in dir inbound on interface
p1_if ##
-t nat -A PREROUTING -m physdev --physdev-in p1_if -d 100.100
.100.1
/29
-p udp -j DNAT --to-destination 30.30
.30.1
-30.30
.30.6
-m mark --mark 0xdead
带有 NVUE 的动态 DNAT 的更多示例
实际上,可以组合多个规则,并且可以将多个 dnat acl 应用于相同的上行链路端口。
ICMP
nv set acl dyndnaticmp rule 1
action dest-nat translate-ip 21.1
.0.19
to 21.1
.0.20
nv set acl dyndnaticmp rule 1
match ip dest-ip 199.101
.6.0
/30
nv set acl dyndnaticmp rule 1
match ip protocol icmp
nv set acl dyndnaticmp rule 2
action dest-nat translate-ip 120.19
.0.19
to 120.19
.0.20
nv set acl dyndnaticmp rule 2
match ip dest-ip 199.101
.6.4
/30
nv set acl dyndnaticmp rule 2
match ip protocol icmp
nv set acl dyndnaticmp rule 3
action dest-nat translate-ip 21.1
.0.15
to 21.1
.0.16
nv set acl dyndnaticmp rule 3
match ip dest-ip 199.101
.6.8
/30
nv set acl dyndnaticmp rule 3
match ip protocol icmp
nv set acl dyndnaticmp type ipv4
TCP
nv set acl dyndnattcp rule 1
action dest-nat translate-ip 21.1
.0.19
to 21.1
.0.20
nv set acl dyndnattcp rule 1
match ip dest-ip 199.101
.7.0
/30
nv set acl dyndnattcp rule 1
match ip protocol tcp
nv set acl dyndnattcp rule 2
action dest-nat translate-ip 120.19
.0.19
to 120.19
.0.20
nv set acl dyndnattcp rule 2
match ip dest-ip 199.101
.7.4
/30
nv set acl dyndnattcp rule 2
match ip protocol tcp
nv set acl dyndnattcp rule 3
action dest-nat translate-ip 21.1
.0.15
to 21.1
.0.16
nv set acl dyndnattcp rule 3
match ip dest-ip 199.101
.7.8
/30
nv set acl dyndnattcp rule 3
match ip protocol tcp
nv set acl dyndnattcp type ipv4
UDP
nv set acl dyndnatudp rule 1
action dest-nat translate-ip 21.1
.0.19
to 21.1
.0.20
nv set acl dyndnatudp rule 1
match ip dest-ip 199.101
.8.0
/30
nv set acl dyndnatudp rule 1
match ip protocol udp
nv set acl dyndnatudp rule 2
action dest-nat translate-ip 120.19
.0.19
to 120.19
.0.20
nv set acl dyndnatudp rule 2
match ip dest-ip 199.101
.8.4
/30
nv set acl dyndnatudp rule 2
match ip protocol udp
nv set acl dyndnatudp rule 3
action dest-nat translate-ip 21.1
.0.15
to 21.1
.0.16
nv set acl dyndnatudp rule 3
match ip dest-ip 199.101
.8.8
/30
nv set acl dyndnatudp rule 3
match ip protocol udp
nv set acl dyndnatudp type ipv4
将这些应用于上行链路端口
nv set interface
p0_if.10
,p1_if.10
acl dyndnaticmp inbound
nv set interface
p0_if.10
,p1_if.10
acl dyndnattcp inbound
nv set interface
p0_if.10
,p1_if.10
acl dyndnatudp inbound
控制平面策略 (CoPP)
控制平面策略是一种保护控制平面的机制。这是通过策略控制流向控制平面的流量来实现的。控制平面由路由器、管理协议处理程序和系统服务组成。CoPP 确保只有必要的流量到达控制平面,而不会超过预定义的包速率。CoPP 使用过滤和速率限制流向控制平面的流量。这可以防止 CPU 和其他系统资源因过多的数据包(即拒绝服务 (DoS) 攻击)而过载。根据预定义或用户定义的配置,对超出流量进行速率限制并丢弃。CoPP 配置全局应用于控制平面。
我们为每种协议或数据包组分别设置了以下策略器。每个值的单位是每秒数据包数。
序号 | 策略器名称 | 协议 | 默认状态 | 默认速率 | 默认突发 |
1. | arp | ARP、RARP | 已启用 | 800 | 800 |
2. | bgp | BGP | 已启用 | 2000 | 2000 |
3. | bfd | BFD | 已启用 | 2000 | 2000 |
4. | icmp | ICMP、ICMPv6 | 已启用 | 1000 | 1000 |
5. | icmp6-neigh | IPv6 NDP | 已启用 | 1000 | 1000 |
6. | dhcp | DHCPv4、DHCPv6 | 已启用 | 2000 | 2000 |
7. | ip2me | 所有其他发往 HBN 的 IPv4 或 IPv6 流量 | 已启用 | 1000 | 1000 |
8. | catch-all | 上述规则未匹配的所有其他流量 | 已启用 | 100 | 100 |
我们默认在 HBN 上启用 CoPP 配置,并使用预定义的值。
CoPP 配置
启用策略器
在此示例中,我们启用了 bgp 策略器。
nv set system control-plane policer bgp enable on
更新策略器
在此示例中,我们更改了 bgp 策略器的速率和突发。
nv set system control-plane policer bgp rate 3000
nv set system control-plane policer bgp burst 3000
禁用策略器
nv set system control-plane policer bgp enable off
CoPP 显示命令
CoPP 摘要
以下 CLI 显示了所有策略器的摘要。这包括配置和各自的统计信息
nv show system control-plane policer
相应的示例输出
Policer State Policer Rate Policer Burst To CPU Pkts To CPU Bytes Violated Packets Violated Bytes
----------- ----- ------------ ------------- ----------- ------------ ---------------- --------------
arp on 800
800
1327
133663
0
0
bfd on 2000
2000
0
0
0
0
bgp on 2000
2000
167470
16689652
0
0
catch
-all on 100
100
0
0
0
0
dhcp on 2000
2000
0
0
0
0
icmp on 1000
1000
0
0
0
0
icmp6-neigh on 1000
1000
426605
36641706
0
0
ip2me on 1000
1000
0
0
0
0
特定策略器的 Show CLI
以下 CLI 显示了各个策略器的详细信息。在此特定示例中,将显示 bgp 策略器详细信息
nv show system control-plane policer bgp
相应的示例输出
operational applied
---------------- ----------- -------
state on on
burst 3000
3000
rate 3000
3000
statistics
to-cpu-pkts 167686
to-cpu-bytes 16710895
violated-pkts 0
violated-bytes 0
HBN 上的 DHCP 中继
DHCP 是一种客户端服务器协议,可自动为 IP 主机提供 IP 地址和其他相关的配置信息。DHCP 中继(代理)是转发客户端和服务器之间 DHCP 数据包的主机。DHCP 中继在不在同一物理子网上的客户端和服务器之间转发请求和回复。
可以使用扁平文件(supervisord 配置)或通过 NVUE 配置 DHCP 中继。
配置
HBN 是一个非 systemd 的容器。因此,必须按照以下小节中的说明配置 DHCP 中继。
扁平文件配置 (Supervisord)
HBN 初始化脚本在 BlueField 上的 /var/lib/hbn/etc/supervisor/conf.d/
中安装默认配置文件。BlueField 目录已挂载到 /etc/supervisor/conf.d
,从而实现配置持久性。
默认情况下,DHCP 中继已禁用。默认配置适用于默认 VRF 中的 DHCPv4 中继和 DHCPv6 中继的一个实例。
NVUE 配置
用户可以使用 NVUE 通过 CLI 和 REST API 配置和维护 DHCPv4 和 DHCPv6 中继。NVUE 生成所有必需的配置并维护中继服务。
DHCPv4 中继配置
NVUE 示例
以下配置启动一个中继服务,该服务侦听 p0_if
、p1_if
和 vlan482
上的 DHCP 消息,并将请求中继到 DHCP 服务器 10.89.0.1,其中 gateway-interface
为 lo
。
nv set service dhcp-relay default gateway-interface lo
nv set service dhcp-relay default interface p0_if
nv set service dhcp-relay default interface p1_if
nv set service dhcp-relay default interface vlan482 downstream
nv set service dhcp-relay default server 10.89.0.1
扁平文件示例
[program: isc-dhcp-relay-default]
command = /usr/sbin/dhcrelay --nl -d -i p0_if -i p1_if -id vlan482 -U lo 10.89.0.1
autostart = true
autorestart = unexpected
startsecs = 3
startretries = 3
exitcodes = 0
stopsignal = TERM
stopwaitsecs = 3
其中
选项 | 描述 |
| 要侦听请求和回复的网络接口 |
| 上游网络接口 |
| 下游网络接口 |
| 网关 IP 地址接口。对 |
| 调试日志记录。位置: |
| 在将代理选项字段附加到每个请求之前,先使用 |
| 设置自定义远程 ID 字符串(最多 255 个字符)。要使用此选项,您还必须启用 |
| 将接收数据包的底层物理接口设置为 |
DHCPv4 中继选项 82
NVUE 示例
以下 NVUE 命令用于在 DHCP 数据包中启用选项 82 插入,并使用默认值
nv set service dhcp-relay default agent enable on
要使用 NVUE 提供自定义 remote-id
(例如,host10)
nv set service dhcp-relay default agent remote-id host10
要使用 NVUE 将接收请求的底层物理接口用作 circuit-id
nv set service dhcp-relay default agent use-pif-circuit-id enable on
扁平文件示例
[program: isc-dhcp-relay-default]
command = /usr/sbin/dhcrelay --nl -d -i p0_if -i p1_if -id vlan482 -U lo -a --use-pif-circuit-id -r host10 10.89.0.1
autostart = true
autorestart = unexpected
startsecs = 3
startretries = 3
exitcodes = 0
stopsignal = TERM
stopwaitsecs = 3
DHCPv6 中继配置
NVUE 示例
以下 NVUE 命令启动 DHCPv6 中继服务,该服务侦听 vlan482
上的 DHCPv6 请求,并将中继的 DHCPv6 请求发送到 p0_if
和 p1_if
。
nv set service dhcp-relay6 default interface downstream vlan482
nv set service dhcp-relay6 default interface upstream p0_if
nv set service dhcp-relay6 default interface upstream p1_if
扁平文件示例
[program: isc-dhcp-relay6-default]
command = /usr/sbin/dhcrelay --nl -6 -d -l vlan482 -u p0_if -u p1_if
autostart = true
autorestart = unexpected
startsecs = 3
startretries = 3
exitcodes = 0
stopsignal = TERM
stopwaitsecs = 3
其中
选项 | 描述 |
| 下游接口。对 |
| 上游接口。对 |
| IPv6 |
| 调试日志记录位于 |
DHCP 中继和 VRF 注意事项
可以在 VRF 上下文中生成 DHCP 中继,以处理该 VRF 中的 DHCP 请求。每个 VRF 只能有 1 个 DHCPv4 中继和 1 个 DHCPv6 中继实例。要实现这一点,用户可以遵循以下准则
默认 VRF 上的 DHCPv4
/usr/sbin/dhcrelay --nl -i <interface> -U [address]%%<interface> <server_ip>
VRF 上的 DHCPv4
/usr/sbin/ip vrf exec <vrf> /usr/sbin/dhcrelay –-nl -i <interface> -U [address]%%<interface> <server_ip>
默认 VRF 上的 DHCPv6
/usr/sbin/dhcrelay --nl -6 -l <interface> -u <interface>
VRF 上的 DHCPv6
/usr/sbin/ip vrf exec <vrf> /usr/sbin/dhcrelay --nl -6 -l <interface> -u <interface>