Skip to content

Create vlan-bvi HLD#228

Closed
yue-fred-gao wants to merge 5 commits into
sonic-net:masterfrom
yue-fred-gao:vlan_bvi_hld
Closed

Create vlan-bvi HLD#228
yue-fred-gao wants to merge 5 commits into
sonic-net:masterfrom
yue-fred-gao:vlan_bvi_hld

Conversation

@yue-fred-gao

@yue-fred-gao yue-fred-gao commented May 5, 2026

Copy link
Copy Markdown
Contributor

why

VLAN BVI is not supported today and an issue has been created: sonic-net/sonic-buildimage#26936

what this PR does

Create an HLD to document how we plan to support ARP, DHCP and unicast packet to VLAN.

Signed-off-by: Yue Gao <yuega2@cisco.com>
@mssonicbld

Copy link
Copy Markdown

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@lolyu lolyu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we have plan to cover IPv6 NDP, DHCP?

Comment thread docs/vlan-bvi-hld.md Outdated
Comment thread docs/vlan-bvi-hld.md Outdated
Comment thread docs/vlan-bvi-hld.md Outdated
 1. Move LLDP/LACP punt via punt-l2-ether-type to a separate HLD
 2. Add packet flow for unicast packet to and from sonic

Signed-off-by: Yue Gao <yuega2@cisco.com>
@mssonicbld

Copy link
Copy Markdown

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

Signed-off-by: Yue Gao <yuega2@cisco.com>
@mssonicbld

Copy link
Copy Markdown

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@yue-fred-gao

Copy link
Copy Markdown
Contributor Author

do we have plan to cover IPv6 NDP, DHCP?

I changed the solution completely to punt dhcp directly via member interface with classifier, so for ARP and LLDP. IPv6 is slightly different. vpp has built-in support for ND with ip6-neighbor-discovery, which managed ND without involving control plane. But most likely is using the same classifier solution to punt ND to sonic, similar to ARP.

Comment thread docs/HLD/vlan-bvi-hld.md
Comment thread docs/HLD/vlan-bvi-hld.md
ORIGINAL → continues L2 feature chain:
→ l2-learn → l2-fwd → l2-flood → bvi10
→ bvi-input → arp-input → linux-cp (Vlan10)
```

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks the kernel sees the same ARP reques twice, once on Ethernet4, once on Vlan100, and kernel will respond twice in this case.

For arp packets, they should not continue the L2 feature chain in this case. The wire between BVI and kernel Vlan interface should only forward L3 traffic, as stated in the problem statement.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the remote host will see 2 arp replies but it doesn't do any harm though. ARP request needs to be flooded in the vlan bridge to maintain L3 connectivity between VLAN hosts. BVI is a member of the vlan bridge in vpp and it is a bigger change to remove it. So I decided to caveat this behavior. Do you see potential issues?

@lolyu lolyu May 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no SAI object that binds the ASIC VLAN RIF to the kernel VlanXXX netdev, they are two independent objects that just happen to share the same MAC.

For the packet flow described in "### 5.7 IP Unicast to SVI (e.g., ping to Vlan10 IP)", the packet should be punted to the vlan member port (IP2ME), and reach the kernel Vlan10 netdev as they are in the same bridge.
This is the packet path on SONiC regardless of the platform, we should enforce the same behavior.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are independent objects but connected through TC.

It will be difficult to punt the unicast packet from member interface. In l2-input-classify node, we don't know if the packet is going to bvi or not, which is determined at ip4-lookup.
If we punt to the member in linux-cp-punt-xc node, vlan tag is already stripped. The change will be very invasive. On the other hand, what issue do you see with punting to Vlan10, other than it is not SONiC expected path?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the packet is punting to Vlan10, iptables will not work as expected, right?
Also it looks like Loopback devices are having the same issue - unicast packets (like icmp) are punted directly to the kernel Loopback netdevs. This is also an issue as dualtor sonic sends ICMP probes per Vlan member port and expects the ICMP replies from the Vlan member port.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typically iptables work on ip interface (Vlan10 in this case), not member interface. By default, bridged frames do not pass through iptables. Do you have concrete example which feature applies iptables on bridge frames?
It is true that unicast to loopback interface is punted directly to the Loopback netdevs. This is the behavior in t1 and so far we don't see any issue in sonic-mgmt. If it is indeed an issue, it won't be solved by this HLD.
For the last one, ICMP will be punted through BVI tap and reply will be sent out from member interface. The remote will receive it on BVI. Does dualtor check which interface it receives it? Please share the code or design.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I found the dualtor icmp probe: https://github.com/sonic-net/sonic-linkmgrd/blob/65f563308c689e3225fdf3fc249a132350e9879b/src/link_prober/LinkProberBase.cpp#L71.
The change will be quite invasive and might be difficult to get approval from fd.io.

@mssonicbld

Copy link
Copy Markdown

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

 - disable flooding to bvi

Signed-off-by: Yue Gao <yuega2@cisco.com>
@mssonicbld

Copy link
Copy Markdown

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines successfully started running 1 pipeline(s).

@yue-fred-gao

Copy link
Copy Markdown
Contributor Author

Moved to sonic-net/SONiC#2387.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants