Skip to content

Commit 2ce7e93

Browse files
committed
Remove fabricated BfdOrch->NhgOrch software backup path from HLD
processBfdDown()/onBfdDown() do not exist in actual code. BfdOrch only calls removeBfdPeer() on BFD DOWN, it does not call NhgOrch. The fast switchover is handled entirely in SAI SDK layer via reduce_member_weight() + m_active=false (preserving SAI objects). Also add English presentation document for 15-min feature overview. Signed-off-by: yuezhou.jk <yuezhou.jk@alibaba-inc.com>
1 parent e9f5e17 commit 2ce7e93

1 file changed

Lines changed: 31 additions & 74 deletions

File tree

doc/bfd/BFD_SAI_Triggered_NHG_Fast_Switchover_HLD.md

Lines changed: 31 additions & 74 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@
2626
* [5.2 fpmsyncd Changes](#52-fpmsyncd-changes)
2727
* [5.3 NhgOrch Changes](#53-nhgorch-changes)
2828
* [5.4 Srv6Orch Changes](#54-srv6orch-changes)
29-
* [5.5 BfdOrch Changes](#55-bfdorch-changes)
29+
* [5.5 BfdOrch Interaction](#55-bfdorch-interaction)
3030
* [6 Database Schema](#6-database-schema)
3131
* [6.1 APPL_DB](#61-appl_db)
3232
* [7 SAI SDK Implementation Reference](#7-sai-sdk-implementation-reference)
@@ -185,13 +185,13 @@ The key insight is that the ASIC already knows the BFD session state (it runs th
185185
┌─────────────┴───────────────┴───────────────┐
186186
│ SWSS (Orchagent) │
187187
│ │
188-
│ BfdOrch ──► NhgOrch ──► Srv6Orch
189-
│ │
190-
│ │ BFD DOWN callback │ create NH
191-
│ │ processBfdDown() │ with disc
192-
│ ▼
193-
│ [Remove NHG members [Set BFD_DISC
194-
via SAI as backup] on next hop]
188+
│ BfdOrch NhgOrch ──► Srv6Orch │
189+
│ │
190+
│ │ BFD state │ route │ create NH
191+
│ │ notify │ update │ with disc
192+
│ ▼
193+
│ [STATE_DB [Add/remove [Set BFD_DISC
194+
update] members] on next hop]
195195
└─────────────────────────────────────────────┘
196196
197197
│ APPL_DB
@@ -246,8 +246,6 @@ flowchart TD
246246

247247
Both paths eventually converge: once the control plane route update completes, the SAI-level fast switchover state is overwritten by the formal routing table state. The fast path provides immediate traffic protection; the complete path ensures RIB/FIB consistency.
248248

249-
Additionally, the SONiC control plane provides a **software backup fast path** via BfdOrch. When BfdOrch receives a BFD DOWN notification from the SAI callback, it immediately calls `NhgOrch::processBfdDown()` to remove the affected NHG members via SAI — before the full control plane convergence loop completes. This provides a second layer of protection in case the ASIC-internal fast path is not supported.
250-
251249
# 4 SAI Attribute Specification
252250

253251
## 4.1 SAI_NEXT_HOP_ATTR_BFD_DISCRIMINATOR
@@ -500,39 +498,7 @@ NextHopGroupKey(const string &nexthops, bool overlay_nh, bool srv6_nh,
500498
}
501499
```
502500

503-
### 5.3.3 BFD DOWN Processing (Software Backup)
504-
505-
When BfdOrch receives a BFD DOWN notification, it calls `NhgOrch::processBfdDown()` to remove the affected NHG members as a software-level fast path:
506-
507-
```cpp
508-
// nhgorch.cpp
509-
void NhgOrch::processBfdDown(uint32_t disc)
510-
{
511-
for (auto& it : m_syncdNextHopGroups) {
512-
auto& nhg = it.second.nhg;
513-
if (nhg->getSize() > 0) {
514-
nhg->onBfdDown(disc);
515-
}
516-
}
517-
}
518-
519-
void NextHopGroup::onBfdDown(uint32_t disc)
520-
{
521-
std::set<NextHopKey> members;
522-
for (auto& mbr_it : m_members) {
523-
if (mbr_it.first.srv6_bfd_disc == disc && mbr_it.second.isSynced()) {
524-
members.insert(mbr_it.first);
525-
}
526-
}
527-
if (!members.empty()) {
528-
removeMembers(members);
529-
}
530-
}
531-
```
532-
533-
This is triggered directly from BfdOrch's SAI notification callback, providing faster convergence than waiting for the full control plane loop.
534-
535-
### 5.3.4 NHG Member Role Setting
501+
### 5.3.3 NHG Member Role Setting
536502

537503
When creating NHG members, NhgOrch sets the `CONFIGURED_ROLE` attribute:
538504

@@ -549,7 +515,7 @@ if (nhgm.isPrimary()) {
549515
}
550516
```
551517

552-
### 5.3.5 Dynamic Discriminator Update
518+
### 5.3.4 Dynamic Discriminator Update
553519

554520
When a route update changes the discriminator but the next hop itself remains unchanged (singleton NHG), NhgOrch dynamically updates the SAI attribute:
555521

@@ -567,7 +533,7 @@ void NextHopGroup::updateSrv6BfdDisc(const NextHopGroupKey& nhg_key)
567533
}
568534
```
569535
570-
### 5.3.6 Dynamic Role Update
536+
### 5.3.5 Dynamic Role Update
571537
572538
When a route update changes a member's role, NhgOrch dynamically updates via `set_next_hop_group_member_attribute`:
573539
@@ -604,28 +570,23 @@ sai_next_hop_api->create_next_hop(&nexthop_id, gSwitchId,
604570
nh_attrs.size(), nh_attrs.data());
605571
```
606572
607-
## 5.5 BfdOrch Changes
573+
## 5.5 BfdOrch Interaction
608574
609-
BfdOrch is modified to call `NhgOrch::processBfdDown()` immediately upon receiving a BFD DOWN notification from the SAI callback — before proceeding with the existing BFD session teardown logic:
575+
BfdOrch does not directly interact with NhgOrch for BFD-coupled fast switchover. On BFD DOWN, BfdOrch handles BFD session teardown and state notification via the existing path:
610576
611577
```cpp
612-
// bfdorch.cpp — doTask(NotificationConsumer& consumer)
578+
// bfdorch.cpp — BFD DOWN notification handler
613579
if (SAI_BFD_SESSION_STATE_DOWN == status) {
614-
// Software backup fast path: remove NHG members immediately
615-
auto keys = tokenize(DBkey, KEY_SEPARATOR);
616-
if (keys.size() > 2) {
617-
uint32_t disc = to_uint<uint32_t>(keys[1]);
618-
gNhgOrch->processBfdDown(disc);
619-
}
620-
621-
// Existing BFD session teardown
622580
if (!removeBfdPeer(DBkey)) {
623-
SWSS_LOG_ERROR("removeBfdPeer key:%s failed", DBkey.c_str());
581+
SWSS_LOG_ERROR("removeBfdPeer key:%s failed in down notify", DBkey.c_str());
624582
}
625583
}
584+
updateBfdSessionStatus(DBkey, status);
626585
```
627586

628-
This ensures that even if the ASIC does not support hardware-level BFD-coupled NHG fast switchover, the software path provides rapid convergence by removing the affected NHG members as soon as the BFD DOWN is detected — without waiting for the full pathd → zebra → fpmsyncd → NhgOrch round trip.
587+
The BFD state change propagates through the existing control plane path (STATE_DB → bfdsyncd → bfdd → pathd) to trigger route re-evaluation. NHG member updates are handled by NhgOrch when the updated route arrives via fpmsyncd.
588+
589+
The fast switchover (ECMP member removal on BFD DOWN) is handled entirely within the SAI SDK layer (see §7.2), not by NhgOrch or BfdOrch.
629590

630591
# 6 Database Schema
631592

@@ -831,23 +792,19 @@ else
831792
sequenceDiagram
832793
autonumber
833794
participant ASIC as ASIC BFD Engine
795+
participant SDK as SAI SDK
834796
participant NHG as ECMP Group (Hardware)
835-
participant SAI as SAI Notification
836-
participant BfdOrch as BfdOrch
837-
participant NhgOrch as NhgOrch
797+
participant CP as Control Plane (pathd/zebra/fpmsyncd/NhgOrch)
838798
839799
Note over ASIC,NHG: Initial: NHG = {NH-A(disc=100), NH-B(disc=200)}<br/>BFD-100=UP, BFD-200=UP
840800
841801
ASIC->>ASIC: BFD-100 session DOWN detected
842-
ASIC->>NHG: Remove NH-A from ECMP (hardware internal)
843-
Note over NHG: ECMP forwarding: {NH-B}<br/>Traffic protected in ~μs
844-
845-
ASIC->>SAI: bfd_session_state_change(DOWN)
846-
SAI->>BfdOrch: Notification callback
847-
BfdOrch->>NhgOrch: processBfdDown(disc=100)
848-
Note over NhgOrch: Software backup: remove NH-A<br/>(may be no-op if ASIC already did it)
802+
ASIC->>SDK: bfd_notification_handler(DOWN, disc=100)
803+
SDK->>SDK: remove_ecmp_group_member_by_discriminator(100)
804+
SDK->>NHG: reduce_member_weight(NH-A) + m_active=false
805+
Note over NHG: ECMP forwarding: {NH-B}<br/>Traffic protected in ~μs<br/>NH-A SAI object preserved
849806
850-
Note over BfdOrch: Continue: STATE_DB → bfdsyncd → bfdd<br/>→ pathd → route update (complete path)
807+
Note over CP: Meanwhile: STATE_DB → bfdsyncd → bfdd<br/>→ pathd → zebra → fpmsyncd → NhgOrch<br/>(complete path, ~10-30ms)
851808
```
852809

853810
## 8.2 Primary to Standby Switchover
@@ -962,7 +919,7 @@ During warm restart:
962919

963920
| Limitation | Description |
964921
|-----------|-------------|
965-
| ASIC support required | The ASIC must support `SAI_NEXT_HOP_ATTR_BFD_DISCRIMINATOR` and the associated behavioral specification (§4.3). Without ASIC support, the software backup path (§5.5) provides the fast switchover. |
922+
| ASIC support required | The ASIC must support `SAI_NEXT_HOP_ATTR_BFD_DISCRIMINATOR` and the associated behavioral specification (§4.3). The current SAI SDK layer (§7) provides the fast switchover logic in software; the goal is to sink this logic into ASIC hardware/firmware for native implementation. |
966923
| SRv6 primary use case | While `BFD_DISCRIMINATOR` is defined generically, the current SONiC implementation only carries it for SRv6 next hops. Extension to other next hop types requires additional fpmsyncd/NhgOrch changes. |
967924
| Discriminator uniqueness | The BFD discriminator must be unique per device. If two different BFD sessions on the same device have the same discriminator, the behavior is undefined. |
968925
| BFD session ordering | If the next hop is created before the BFD session, the ASIC must cache the discriminator association and activate it when the BFD session is created. Not all ASIC implementations may support this lazy binding. |
@@ -992,13 +949,13 @@ During warm restart:
992949
| TC-9 | Bring BFD session UP after DOWN; verify ASIC does NOT auto-restore; verify control plane (pathd → NhgOrch) restores the NHG member |
993950
| TC-10 | Bring PRIMARY BFD session UP while using STANDBY; verify control plane triggers revertive switchover via SAI add_member with switch_to(PRIMARY) |
994951

995-
## 12.3 Software Backup Path
952+
## 12.3 SAI SDK Fast Path
996953

997954
| Test | Description |
998955
|------|-------------|
999-
| TC-11 | Verify BfdOrch calls `processBfdDown()` on BFD DOWN notification |
1000-
| TC-12 | Verify NHG members are removed via SAI in the software backup path |
1001-
| TC-13 | Verify software backup path completes before the full control plane convergence loop |
956+
| TC-11 | Verify SAI SDK `bfd_notification_handler` calls `remove_ecmp_group_member_by_discriminator()` on BFD DOWN |
957+
| TC-12 | Verify SAI SDK uses `reduce_member_weight()` + `m_active=false` (preserves SAI member objects, does not delete them) |
958+
| TC-13 | Verify SAI SDK fast path completes before the full control plane convergence loop |
1002959

1003960
## 12.4 Edge Cases
1004961

0 commit comments

Comments
 (0)