kernel: bridge: Fix a deadlock when enabling multicast snooping
[ Upstream commit 851d0a73c90e6c8c63fef106c6c1e73df7e05d9d ]
From: Joseph Huang <Joseph.Huang@garmin.com>
When enabling multicast snooping, bridge module deadlocks on multicast_lock
if 1) IPv6 is enabled, and 2) there is an existing querier on the same L2
network.
The deadlock was caused by the following sequence: While holding the lock,
br_multicast_open calls br_multicast_join_snoopers, which eventually causes
IP stack to (attempt to) send out a Listener Report (in igmp6_join_group).
Since the destination Ethernet address is a multicast address, br_dev_xmit
feeds the packet back to the bridge via br_multicast_rcv, which in turn
calls br_multicast_add_group, which then deadlocks on multicast_lock.
The fix is to move the call br_multicast_join_snoopers outside of the
critical section. This works since br_multicast_join_snoopers only deals
with IP and does not modify any multicast data structures of the bridge,
so there's no need to hold the lock.
Steps to reproduce:
1. sysctl net.ipv6.conf.all.force_mld_version=1
2. have another querier
3. ip link set dev bridge type bridge mcast_snooping 0 && \
ip link set dev bridge type bridge mcast_snooping 1 < deadlock >
A typical call trace looks like the following:
[ 936.251495] _raw_spin_lock+0x5c/0x68
[ 936.255221] br_multicast_add_group+0x40/0x170 [bridge]
[ 936.260491] br_multicast_rcv+0x7ac/0xe30 [bridge]
[ 936.265322] br_dev_xmit+0x140/0x368 [bridge]
[ 936.269689] dev_hard_start_xmit+0x94/0x158
[ 936.273876] __dev_queue_xmit+0x5ac/0x7f8
[ 936.277890] dev_queue_xmit+0x10/0x18
[ 936.281563] neigh_resolve_output+0xec/0x198
[ 936.285845] ip6_finish_output2+0x240/0x710
[ 936.290039] __ip6_finish_output+0x130/0x170
[ 936.294318] ip6_output+0x6c/0x1c8
[ 936.297731] NF_HOOK.constprop.0+0xd8/0xe8
[ 936.301834] igmp6_send+0x358/0x558
[ 936.305326] igmp6_join_group.part.0+0x30/0xf0
[ 936.309774] igmp6_group_added+0xfc/0x110
[ 936.313787] __ipv6_dev_mc_inc+0x1a4/0x290
[ 936.317885] ipv6_dev_mc_inc+0x10/0x18
[ 936.321677] br_multicast_open+0xbc/0x110 [bridge]
[ 936.326506] br_multicast_toggle+0xec/0x140 [bridge]
Fixes: 4effd28c1245 ("bridge: join all-snoopers multicast address")
Signed-off-by: Joseph Huang <Joseph.Huang@garmin.com>
Acked-by: Nikolay Aleksandrov <nikolay@nvidia.com>
Link: https://lore.kernel.org/r/20201204235628.50653-1-Joseph.Huang@garmin.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
[linus.luessing@c0d3.blue: backported to 4.4]
This commit is contained in:
Linus Lüssing
parent
9df297bdde
commit
13cb7504f4
@ -0,0 +1,275 @@
|
|||||||
|
From: Linus Lüssing <linus.luessing@c0d3.blue>
|
||||||
|
Date: Mon, 25 Jan 2021 01:38:19 +0100
|
||||||
|
Subject: kernel: bridge: Fix a deadlock when enabling multicast snooping
|
||||||
|
|
||||||
|
[ Upstream commit 851d0a73c90e6c8c63fef106c6c1e73df7e05d9d ]
|
||||||
|
|
||||||
|
From: Joseph Huang <Joseph.Huang@garmin.com>
|
||||||
|
|
||||||
|
When enabling multicast snooping, bridge module deadlocks on multicast_lock
|
||||||
|
if 1) IPv6 is enabled, and 2) there is an existing querier on the same L2
|
||||||
|
network.
|
||||||
|
|
||||||
|
The deadlock was caused by the following sequence: While holding the lock,
|
||||||
|
br_multicast_open calls br_multicast_join_snoopers, which eventually causes
|
||||||
|
IP stack to (attempt to) send out a Listener Report (in igmp6_join_group).
|
||||||
|
Since the destination Ethernet address is a multicast address, br_dev_xmit
|
||||||
|
feeds the packet back to the bridge via br_multicast_rcv, which in turn
|
||||||
|
calls br_multicast_add_group, which then deadlocks on multicast_lock.
|
||||||
|
|
||||||
|
The fix is to move the call br_multicast_join_snoopers outside of the
|
||||||
|
critical section. This works since br_multicast_join_snoopers only deals
|
||||||
|
with IP and does not modify any multicast data structures of the bridge,
|
||||||
|
so there's no need to hold the lock.
|
||||||
|
|
||||||
|
Steps to reproduce:
|
||||||
|
1. sysctl net.ipv6.conf.all.force_mld_version=1
|
||||||
|
2. have another querier
|
||||||
|
3. ip link set dev bridge type bridge mcast_snooping 0 && \
|
||||||
|
ip link set dev bridge type bridge mcast_snooping 1 < deadlock >
|
||||||
|
|
||||||
|
A typical call trace looks like the following:
|
||||||
|
|
||||||
|
[ 936.251495] _raw_spin_lock+0x5c/0x68
|
||||||
|
[ 936.255221] br_multicast_add_group+0x40/0x170 [bridge]
|
||||||
|
[ 936.260491] br_multicast_rcv+0x7ac/0xe30 [bridge]
|
||||||
|
[ 936.265322] br_dev_xmit+0x140/0x368 [bridge]
|
||||||
|
[ 936.269689] dev_hard_start_xmit+0x94/0x158
|
||||||
|
[ 936.273876] __dev_queue_xmit+0x5ac/0x7f8
|
||||||
|
[ 936.277890] dev_queue_xmit+0x10/0x18
|
||||||
|
[ 936.281563] neigh_resolve_output+0xec/0x198
|
||||||
|
[ 936.285845] ip6_finish_output2+0x240/0x710
|
||||||
|
[ 936.290039] __ip6_finish_output+0x130/0x170
|
||||||
|
[ 936.294318] ip6_output+0x6c/0x1c8
|
||||||
|
[ 936.297731] NF_HOOK.constprop.0+0xd8/0xe8
|
||||||
|
[ 936.301834] igmp6_send+0x358/0x558
|
||||||
|
[ 936.305326] igmp6_join_group.part.0+0x30/0xf0
|
||||||
|
[ 936.309774] igmp6_group_added+0xfc/0x110
|
||||||
|
[ 936.313787] __ipv6_dev_mc_inc+0x1a4/0x290
|
||||||
|
[ 936.317885] ipv6_dev_mc_inc+0x10/0x18
|
||||||
|
[ 936.321677] br_multicast_open+0xbc/0x110 [bridge]
|
||||||
|
[ 936.326506] br_multicast_toggle+0xec/0x140 [bridge]
|
||||||
|
|
||||||
|
Fixes: 4effd28c1245 ("bridge: join all-snoopers multicast address")
|
||||||
|
Signed-off-by: Joseph Huang <Joseph.Huang@garmin.com>
|
||||||
|
Acked-by: Nikolay Aleksandrov <nikolay@nvidia.com>
|
||||||
|
Link: https://lore.kernel.org/r/20201204235628.50653-1-Joseph.Huang@garmin.com
|
||||||
|
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
||||||
|
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
|
||||||
|
[linus.luessing@c0d3.blue: backported to 4.4]
|
||||||
|
|
||||||
|
diff --git a/target/linux/generic/backport-4.14/120-bridge-Fix-a-deadlock-when-enabling-multicast-snoopi.patch b/target/linux/generic/backport-4.14/120-bridge-Fix-a-deadlock-when-enabling-multicast-snoopi.patch
|
||||||
|
new file mode 100644
|
||||||
|
index 0000000000000000000000000000000000000000..e89d9e3eb2063b1f478c0217aaa3c89d14880043
|
||||||
|
--- /dev/null
|
||||||
|
+++ b/target/linux/generic/backport-4.14/120-bridge-Fix-a-deadlock-when-enabling-multicast-snoopi.patch
|
||||||
|
@@ -0,0 +1,209 @@
|
||||||
|
+From 0e125bd346d3b9d6997e525b263834f6b650cf60 Mon Sep 17 00:00:00 2001
|
||||||
|
+From: Joseph Huang <Joseph.Huang@garmin.com>
|
||||||
|
+Date: Fri, 4 Dec 2020 18:56:28 -0500
|
||||||
|
+Subject: [PATCH] bridge: Fix a deadlock when enabling multicast snooping
|
||||||
|
+
|
||||||
|
+[ Upstream commit 851d0a73c90e6c8c63fef106c6c1e73df7e05d9d ]
|
||||||
|
+
|
||||||
|
+When enabling multicast snooping, bridge module deadlocks on multicast_lock
|
||||||
|
+if 1) IPv6 is enabled, and 2) there is an existing querier on the same L2
|
||||||
|
+network.
|
||||||
|
+
|
||||||
|
+The deadlock was caused by the following sequence: While holding the lock,
|
||||||
|
+br_multicast_open calls br_multicast_join_snoopers, which eventually causes
|
||||||
|
+IP stack to (attempt to) send out a Listener Report (in igmp6_join_group).
|
||||||
|
+Since the destination Ethernet address is a multicast address, br_dev_xmit
|
||||||
|
+feeds the packet back to the bridge via br_multicast_rcv, which in turn
|
||||||
|
+calls br_multicast_add_group, which then deadlocks on multicast_lock.
|
||||||
|
+
|
||||||
|
+The fix is to move the call br_multicast_join_snoopers outside of the
|
||||||
|
+critical section. This works since br_multicast_join_snoopers only deals
|
||||||
|
+with IP and does not modify any multicast data structures of the bridge,
|
||||||
|
+so there's no need to hold the lock.
|
||||||
|
+
|
||||||
|
+Steps to reproduce:
|
||||||
|
+1. sysctl net.ipv6.conf.all.force_mld_version=1
|
||||||
|
+2. have another querier
|
||||||
|
+3. ip link set dev bridge type bridge mcast_snooping 0 && \
|
||||||
|
+ ip link set dev bridge type bridge mcast_snooping 1 < deadlock >
|
||||||
|
+
|
||||||
|
+A typical call trace looks like the following:
|
||||||
|
+
|
||||||
|
+[ 936.251495] _raw_spin_lock+0x5c/0x68
|
||||||
|
+[ 936.255221] br_multicast_add_group+0x40/0x170 [bridge]
|
||||||
|
+[ 936.260491] br_multicast_rcv+0x7ac/0xe30 [bridge]
|
||||||
|
+[ 936.265322] br_dev_xmit+0x140/0x368 [bridge]
|
||||||
|
+[ 936.269689] dev_hard_start_xmit+0x94/0x158
|
||||||
|
+[ 936.273876] __dev_queue_xmit+0x5ac/0x7f8
|
||||||
|
+[ 936.277890] dev_queue_xmit+0x10/0x18
|
||||||
|
+[ 936.281563] neigh_resolve_output+0xec/0x198
|
||||||
|
+[ 936.285845] ip6_finish_output2+0x240/0x710
|
||||||
|
+[ 936.290039] __ip6_finish_output+0x130/0x170
|
||||||
|
+[ 936.294318] ip6_output+0x6c/0x1c8
|
||||||
|
+[ 936.297731] NF_HOOK.constprop.0+0xd8/0xe8
|
||||||
|
+[ 936.301834] igmp6_send+0x358/0x558
|
||||||
|
+[ 936.305326] igmp6_join_group.part.0+0x30/0xf0
|
||||||
|
+[ 936.309774] igmp6_group_added+0xfc/0x110
|
||||||
|
+[ 936.313787] __ipv6_dev_mc_inc+0x1a4/0x290
|
||||||
|
+[ 936.317885] ipv6_dev_mc_inc+0x10/0x18
|
||||||
|
+[ 936.321677] br_multicast_open+0xbc/0x110 [bridge]
|
||||||
|
+[ 936.326506] br_multicast_toggle+0xec/0x140 [bridge]
|
||||||
|
+
|
||||||
|
+Fixes: 4effd28c1245 ("bridge: join all-snoopers multicast address")
|
||||||
|
+Signed-off-by: Joseph Huang <Joseph.Huang@garmin.com>
|
||||||
|
+Acked-by: Nikolay Aleksandrov <nikolay@nvidia.com>
|
||||||
|
+Link: https://lore.kernel.org/r/20201204235628.50653-1-Joseph.Huang@garmin.com
|
||||||
|
+Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
||||||
|
+Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
|
||||||
|
+[linus.luessing@c0d3.blue: backported to 4.4]
|
||||||
|
+---
|
||||||
|
+ net/bridge/br_device.c | 6 ++++++
|
||||||
|
+ net/bridge/br_multicast.c | 34 +++++++++++++++++++++++++---------
|
||||||
|
+ net/bridge/br_private.h | 10 ++++++++++
|
||||||
|
+ 3 files changed, 41 insertions(+), 9 deletions(-)
|
||||||
|
+
|
||||||
|
+diff --git a/net/bridge/br_device.c b/net/bridge/br_device.c
|
||||||
|
+index b7cc322acdc8..4bd8a7ba002a 100644
|
||||||
|
+--- a/net/bridge/br_device.c
|
||||||
|
++++ b/net/bridge/br_device.c
|
||||||
|
+@@ -140,6 +140,9 @@ static int br_dev_open(struct net_device *dev)
|
||||||
|
+ br_stp_enable_bridge(br);
|
||||||
|
+ br_multicast_open(br);
|
||||||
|
+
|
||||||
|
++ if (!br->multicast_disabled)
|
||||||
|
++ br_multicast_join_snoopers(br);
|
||||||
|
++
|
||||||
|
+ return 0;
|
||||||
|
+ }
|
||||||
|
+
|
||||||
|
+@@ -160,6 +163,9 @@ static int br_dev_stop(struct net_device *dev)
|
||||||
|
+ br_stp_disable_bridge(br);
|
||||||
|
+ br_multicast_stop(br);
|
||||||
|
+
|
||||||
|
++ if (!br->multicast_disabled)
|
||||||
|
++ br_multicast_leave_snoopers(br);
|
||||||
|
++
|
||||||
|
+ netif_stop_queue(dev);
|
||||||
|
+
|
||||||
|
+ return 0;
|
||||||
|
+diff --git a/net/bridge/br_multicast.c b/net/bridge/br_multicast.c
|
||||||
|
+index 6d8d1cf2ddc6..1b1d06549583 100644
|
||||||
|
+--- a/net/bridge/br_multicast.c
|
||||||
|
++++ b/net/bridge/br_multicast.c
|
||||||
|
+@@ -2314,7 +2314,7 @@ static inline void br_ip6_multicast_join_snoopers(struct net_bridge *br)
|
||||||
|
+ }
|
||||||
|
+ #endif
|
||||||
|
+
|
||||||
|
+-static void br_multicast_join_snoopers(struct net_bridge *br)
|
||||||
|
++void br_multicast_join_snoopers(struct net_bridge *br)
|
||||||
|
+ {
|
||||||
|
+ br_ip4_multicast_join_snoopers(br);
|
||||||
|
+ br_ip6_multicast_join_snoopers(br);
|
||||||
|
+@@ -2345,7 +2345,7 @@ static inline void br_ip6_multicast_leave_snoopers(struct net_bridge *br)
|
||||||
|
+ }
|
||||||
|
+ #endif
|
||||||
|
+
|
||||||
|
+-static void br_multicast_leave_snoopers(struct net_bridge *br)
|
||||||
|
++void br_multicast_leave_snoopers(struct net_bridge *br)
|
||||||
|
+ {
|
||||||
|
+ br_ip4_multicast_leave_snoopers(br);
|
||||||
|
+ br_ip6_multicast_leave_snoopers(br);
|
||||||
|
+@@ -2364,9 +2364,6 @@ static void __br_multicast_open(struct net_bridge *br,
|
||||||
|
+
|
||||||
|
+ void br_multicast_open(struct net_bridge *br)
|
||||||
|
+ {
|
||||||
|
+- if (!br->multicast_disabled)
|
||||||
|
+- br_multicast_join_snoopers(br);
|
||||||
|
+-
|
||||||
|
+ __br_multicast_open(br, &br->ip4_own_query);
|
||||||
|
+ #if IS_ENABLED(CONFIG_IPV6)
|
||||||
|
+ __br_multicast_open(br, &br->ip6_own_query);
|
||||||
|
+@@ -2382,9 +2379,6 @@ void br_multicast_stop(struct net_bridge *br)
|
||||||
|
+ del_timer_sync(&br->ip6_other_query.timer);
|
||||||
|
+ del_timer_sync(&br->ip6_own_query.timer);
|
||||||
|
+ #endif
|
||||||
|
+-
|
||||||
|
+- if (!br->multicast_disabled)
|
||||||
|
+- br_multicast_leave_snoopers(br);
|
||||||
|
+ }
|
||||||
|
+
|
||||||
|
+ void br_multicast_dev_del(struct net_bridge *br)
|
||||||
|
+@@ -2540,6 +2534,7 @@ int br_multicast_toggle(struct net_bridge *br, unsigned long val)
|
||||||
|
+ {
|
||||||
|
+ struct net_bridge_mdb_htable *mdb;
|
||||||
|
+ struct net_bridge_port *port;
|
||||||
|
++ bool change_snoopers = false;
|
||||||
|
+ int err = 0;
|
||||||
|
+
|
||||||
|
+ spin_lock_bh(&br->multicast_lock);
|
||||||
|
+@@ -2549,7 +2544,7 @@ int br_multicast_toggle(struct net_bridge *br, unsigned long val)
|
||||||
|
+ br_mc_disabled_update(br->dev, !val);
|
||||||
|
+ br->multicast_disabled = !val;
|
||||||
|
+ if (br->multicast_disabled) {
|
||||||
|
+- br_multicast_leave_snoopers(br);
|
||||||
|
++ change_snoopers = true;
|
||||||
|
+ goto unlock;
|
||||||
|
+ }
|
||||||
|
+
|
||||||
|
+@@ -2575,9 +2570,30 @@ int br_multicast_toggle(struct net_bridge *br, unsigned long val)
|
||||||
|
+ list_for_each_entry(port, &br->port_list, list)
|
||||||
|
+ __br_multicast_enable_port(port);
|
||||||
|
+
|
||||||
|
++ change_snoopers = true;
|
||||||
|
++
|
||||||
|
+ unlock:
|
||||||
|
+ spin_unlock_bh(&br->multicast_lock);
|
||||||
|
+
|
||||||
|
++ /* br_multicast_join_snoopers has the potential to cause
|
||||||
|
++ * an MLD Report/Leave to be delivered to br_multicast_rcv,
|
||||||
|
++ * which would in turn call br_multicast_add_group, which would
|
||||||
|
++ * attempt to acquire multicast_lock. This function should be
|
||||||
|
++ * called after the lock has been released to avoid deadlocks on
|
||||||
|
++ * multicast_lock.
|
||||||
|
++ *
|
||||||
|
++ * br_multicast_leave_snoopers does not have the problem since
|
||||||
|
++ * br_multicast_rcv first checks BROPT_MULTICAST_ENABLED, and
|
||||||
|
++ * returns without calling br_multicast_ipv4/6_rcv if it's not
|
||||||
|
++ * enabled. Moved both functions out just for symmetry.
|
||||||
|
++ */
|
||||||
|
++ if (change_snoopers) {
|
||||||
|
++ if (!br->multicast_disabled)
|
||||||
|
++ br_multicast_join_snoopers(br);
|
||||||
|
++ else
|
||||||
|
++ br_multicast_leave_snoopers(br);
|
||||||
|
++ }
|
||||||
|
++
|
||||||
|
+ return err;
|
||||||
|
+ }
|
||||||
|
+
|
||||||
|
+diff --git a/net/bridge/br_private.h b/net/bridge/br_private.h
|
||||||
|
+index bbf7513c1d16..cb701c31199d 100644
|
||||||
|
+--- a/net/bridge/br_private.h
|
||||||
|
++++ b/net/bridge/br_private.h
|
||||||
|
+@@ -611,6 +611,8 @@ void br_multicast_del_port(struct net_bridge_port *port);
|
||||||
|
+ void br_multicast_enable_port(struct net_bridge_port *port);
|
||||||
|
+ void br_multicast_disable_port(struct net_bridge_port *port);
|
||||||
|
+ void br_multicast_init(struct net_bridge *br);
|
||||||
|
++void br_multicast_join_snoopers(struct net_bridge *br);
|
||||||
|
++void br_multicast_leave_snoopers(struct net_bridge *br);
|
||||||
|
+ void br_multicast_open(struct net_bridge *br);
|
||||||
|
+ void br_multicast_stop(struct net_bridge *br);
|
||||||
|
+ void br_multicast_dev_del(struct net_bridge *br);
|
||||||
|
+@@ -736,6 +738,14 @@ static inline void br_multicast_init(struct net_bridge *br)
|
||||||
|
+ {
|
||||||
|
+ }
|
||||||
|
+
|
||||||
|
++static inline void br_multicast_join_snoopers(struct net_bridge *br)
|
||||||
|
++{
|
||||||
|
++}
|
||||||
|
++
|
||||||
|
++static inline void br_multicast_leave_snoopers(struct net_bridge *br)
|
||||||
|
++{
|
||||||
|
++}
|
||||||
|
++
|
||||||
|
+ static inline void br_multicast_open(struct net_bridge *br)
|
||||||
|
+ {
|
||||||
|
+ }
|
||||||
|
+--
|
||||||
|
+2.29.2
|
||||||
|
+
|
||||||
Loading…
Reference in New Issue
Block a user