Bidirectional Forwarding Detection (BFD) on Link Aggregation Group (LAG) Interfaces

The Bidirectional Forwarding Detection (BFD) protocol provides a mechanism to detect faults in the bidirectional path between two forwarding engines, including interfaces, data link(s), and to the extent possible the forwarding engines themselves, with potentially very low latency. BFD protocol provides as well a fast mechanism for detecting communication failures on any data links and the protocol can run over any media and at any protocol layer. Link aggregation (LAG) as defined in provides mechanisms to combine multiple physical links to a single logical link. The goal is to provide higher bandwidth with the aggregated logical link and better resiliency, since if one of the physical member links fails, the aggregate logical link can continue to forward traffic over the remaining operational physical member links. Currently Link aggregation control protocol (LACP) is used to detect failures on a per physical member link. However, the use of BFD for failure detection would (1) provide a faster detection (2) provide detection in the absence of LACP (3) and would be able to verify L3 connectivity per member link. Running a single BFD session over the LAG virtual port, and without internal knowledge of the LAG virtual port physical members, will make it impossible for BFD to guarantee a detection of the physical member link failures. The goal is to verify every member link connectivity. The approach is to run BFD async session over each member link and make BFD control whether the member link should be part of the L2 Loadbalance table of the LAG virtual port in the presence and in the absence of LACP. This document describes how to establish a BFD async session per physical member link of the Link Aggregation (LAG) virtual port. While there are native Ethernet mechanisms to detect failures (802.1ax, .3ah) that could be used for LAG, the solution proposed in this document enables operators who have already deployed BFD over different technologies (e.g. IP, MPLS) to use a common failure detection mechanism.

The mechanism proposed for a fast detection of LAG member link failure is running BFD async sessions on every LAG member link. We call these per LAG member link BFD sessions as micro BFD sessions in the rest of this document.

Only one address family MUST be used for all micro BFD async sessions running on all LAG member links. i.e. all member link async BFD sessions MUST either use IPv4 or IPv6.

A single micro BFD session runs on each member link of the LAG. The micro BFD async sessions negotiation MUST follow the same procedures defined in and . Only asynchronous mode is considered in this document. The echo function is outside the document's scope. At least one system MUST take the Active role (possibly both). The micro BFD sessions on the member links are independent BFD sessions. They use their own unique, local discriminator values, maintain their own set of state variables and have their own independent state machine. Timer values MAY be different, even among the micro BFD sessions belonging to the same LAG virtual port, although it is expected that micro BFD sessions belonging to the same LAG virtual port use the same timer values. The demultiplexing of a received packet is solely based on the Your Discriminator field, if this field is nonzero. For the initial Down BFD packets of a BFD session this value may be zero. In this case demultiplexing MUST be based on some combination of other fields which MUST include the interface information of the member link. When receiving a BFD packet for a micro session with a valid, non-zero Your Discriminator, a check MUST be done if the packet was received on the correct member link interface. If the check fails then the packet MUST be discarded. This needs to be done before state variables for the BFD sessions are updated by the received packet. The BFD Control packets for each micro BFD session are IP/UDP encapsulated as defined in , but with a new UDP destination port "BfdBndlPort" (to be assigned by IANA). Control packets use a destination IP address that is the peer's remote IP address. The details of how this destination IP address is learnt is outside the scope of the document . On Ethernet-based LAG member links the destination MAC is a dedicated MAC address (to be assigned by IANA according to ) to be the immediate next hop. This will be used for the initial BFD down packets of a session, while sending BFD Init and Up packets with the MAC address learned from the received BFD packets. On Ethernet-based LAG member links the source MAC is the embedded MAC address of the port transmitting the packet. This mechanism helps to reduce the use of additional MAC addresses, which reduces the requirements for the Ethernet hardware on the receiving port.

The LAG Management Module (LMM) could be envisaged as a client of BFD, i.e. the LMM requests a micro BFD session per member link. LMM then uses the micro BFD session state, in addition to LACP state, to monitor the health of the individual members links of the LAG. The micro BFD session for a particular port MUST be requested when the port is attached to an aggregator. The session MUST be deleted when the port is detached from the aggregator.

Once LACP has determined that a link is suitable for aggregation within its selected LAG, and has completed negotiations with the partner device so as to bring that link to Distributing state, at that point the micro BFD session can be started on the link. BFD, as a layer 3 protocol, is viewed as running across the LAG, with load balancing constraints ensuring particular micro BFD sessions are effectively bound to particular member links. Although the link is in LACP Distributing state it should not be used for carrying traffic other than the micro BFD session. BFD is used to verify that a link is ready to be an active member of its LAG for the purpose of carrying LAG level data traffic. Only when the micro BFD session is up should the link become active for forwarding general traffic over the bundle. In all cases where a micro BFD session goes down the corresponding link is removed from the active set of members of the bundle. LACP remains in Distributing state as BFD runs above the LACP layer but the link is inactive until such time as the micro BFD session is restored. This applies also to the case where a BFD session is started on an existing active link and the BFD session never comes up.

Use of LACP for link aggregation, is strictly optional. It is equally possible to use no aggregation control protocol and to step directly from the layer 1 or layer 2 OAM state becoming operational to starting the micro BFD session.

A possible exception to this would apply if the BFD over LAG feature were provisioned after the link was already active within the LAG. In such cases it may be desirable to retain the link in its active status to avoid disruption to traffic forwarding while the micro BFD session is brought up. Another exception is when the BFD over LAG feature is un-provisioned after links were already BFD validated and active within the LAG. In such case, it may be desirable to retain those links in their active status to avoid disruption to traffic forwarding since expectation is that the peer device will soon un-provision the BFD over LAG feature. Note that if one device is not operating a micro BFD session on a link, while the other device is and perceives the session to be down, this will result in the two devices having a different view of the status of the link. This would likely lead to traffic loss across the LAG. The use of another protocol to bootstrap BFD can detect such mismatched config, since the side that's not configured can send a rejection error.

Layer 3 protocols like e.g. OSPF may use a micro BFD session to detect failures on the LAG virtual port, or may establish a new BFD session over the logical LAG virtual port.

When a micro BFD session goes down then this member link MUST be taken out of the LAG L2 load balance table.

This document does not introduce any additional security issues and the security mechanisms defined in apply in this document.

The IANA is requested to assign a well-known port number for the UDP encapsulated micro BFD sessions. IANA is also requested to assign a dedicated MAC address according to RFC 5342 .

We would like to thank Dave Katz, Alexander Vainshtein, Greg Mirsky and Jeff Tantsura for their comments. The initial event to start the current discussion was the distribution of draft-chen-bfd-interface-00.

Paul Hitchen BT Email: paul.hitchen@bt.com George Swallow Cisco Systems Email: swallow@cisco.com Wim Henderickx Alcatel-Lucent Email: wim.henderickx@alcatel-lucent.com Nobo Akiya Cisco Systems Email: nobo@cisco.com Neil Ketley Cisco Systems Email: nketley@cisco.com Carlos Pignataro Cisco Systems Email: cpignata@cisco.com Nitin Bahadur Juniper Networks Email: nitinb@juniper.net Zuliang Wang Huawei Technologies Email: liang_tsing@huawei.com Liang Guo China Telecom Email: guoliang@gsta.com Jeff Tantsura Ericsson Email: jeff.tantsura@ericsson.com