Message ID | 20220808175845.484968-1-ivecera@redhat.com (mailing list archive) |
---|---|
State | Awaiting Upstream |
Delegated to: | Netdev Maintainers |
Headers | show |
Series | [net] iavf: Fix deadlock in initialization | expand |
> -----Original Message----- > From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of > Ivan Vecera > Sent: Monday, August 8, 2022 7:59 PM > To: netdev@vger.kernel.org > Cc: moderated list:INTEL ETHERNET DRIVERS <intel-wired- > lan@lists.osuosl.org>; Brandeburg, Jesse <jesse.brandeburg@intel.com>; > open list <linux-kernel@vger.kernel.org>; Stefan Assmann > <sassmann@kpanic.de>; Eric Dumazet <edumazet@google.com>; Jakub > Kicinski <kuba@kernel.org>; Paolo Abeni <pabeni@redhat.com>; David S. > Miller <davem@davemloft.net> > Subject: [Intel-wired-lan] [PATCH net] iavf: Fix deadlock in initialization > > Fix deadlock that occurs when iavf interface is a part of failover configuration. > > 1. Mutex crit_lock is taken at the beginning of iavf_watchdog_task() 2. > Function iavf_init_config_adapter() is called when adapter > state is __IAVF_INIT_CONFIG_ADAPTER > 3. iavf_init_config_adapter() calls register_netdevice() that emits > NETDEV_REGISTER event > 4. Notifier function failover_event() then calls > net_failover_slave_register() that calls dev_open() 5. dev_open() calls > iavf_open() that tries to take crit_lock in > end-less loop > > Stack trace: > ... > [ 790.251876] usleep_range_state+0x5b/0x80 [ 790.252547] > iavf_open+0x37/0x1d0 [iavf] [ 790.253139] __dev_open+0xcd/0x160 [ > 790.253699] dev_open+0x47/0x90 [ 790.254323] > net_failover_slave_register+0x122/0x220 [net_failover] [ 790.255213] > failover_slave_register.part.7+0xd2/0x180 [failover] [ 790.256050] > failover_event+0x122/0x1ab [failover] [ 790.256821] > notifier_call_chain+0x47/0x70 [ 790.257510] > register_netdevice+0x20f/0x550 [ 790.258263] > iavf_watchdog_task+0x7c8/0xea0 [iavf] [ 790.259009] > process_one_work+0x1a7/0x360 [ 790.259705] worker_thread+0x30/0x390 > > To fix the situation we should check the current adapter state after first > unsuccessful mutex_trylock() and return with -EBUSY if it is > __IAVF_INIT_CONFIG_ADAPTER. > > Fixes: 226d528512cf ("iavf: fix locking of critical sections") > Signed-off-by: Ivan Vecera <ivecera@redhat.com> > --- > drivers/net/ethernet/intel/iavf/iavf_main.c | 11 ++++++++++- > 1 file changed, 10 insertions(+), 1 deletion(-) > > diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c > b/drivers/net/ethernet/intel/iavf/iavf_main.c > index 45d097a164ad..f9dcaadc7ea0 100644 > --- a/drivers/net/ethernet/intel/iavf/iavf_main.c > +++ b/drivers/net/ethernet/intel/iavf/iavf_main.c > @@ -4085,8 +4085,17 @@ static int iavf_open(struct net_device *netdev) Tested-by: Konrad Jankowski <konrad0.jankowski@intel.com>
diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c index 45d097a164ad..f9dcaadc7ea0 100644 --- a/drivers/net/ethernet/intel/iavf/iavf_main.c +++ b/drivers/net/ethernet/intel/iavf/iavf_main.c @@ -4085,8 +4085,17 @@ static int iavf_open(struct net_device *netdev) return -EIO; } - while (!mutex_trylock(&adapter->crit_lock)) + while (!mutex_trylock(&adapter->crit_lock)) { + /* If we are in __IAVF_INIT_CONFIG_ADAPTER state the crit_lock + * is already taken and iavf_open is called from an upper + * device's notifier reacting on NETDEV_REGISTER event. + * We have to leave here to avoid dead lock. + */ + if (adapter->state == __IAVF_INIT_CONFIG_ADAPTER) + return -EBUSY; + usleep_range(500, 1000); + } if (adapter->state != __IAVF_DOWN) { err = -EBUSY;
Fix deadlock that occurs when iavf interface is a part of failover configuration. 1. Mutex crit_lock is taken at the beginning of iavf_watchdog_task() 2. Function iavf_init_config_adapter() is called when adapter state is __IAVF_INIT_CONFIG_ADAPTER 3. iavf_init_config_adapter() calls register_netdevice() that emits NETDEV_REGISTER event 4. Notifier function failover_event() then calls net_failover_slave_register() that calls dev_open() 5. dev_open() calls iavf_open() that tries to take crit_lock in end-less loop Stack trace: ... [ 790.251876] usleep_range_state+0x5b/0x80 [ 790.252547] iavf_open+0x37/0x1d0 [iavf] [ 790.253139] __dev_open+0xcd/0x160 [ 790.253699] dev_open+0x47/0x90 [ 790.254323] net_failover_slave_register+0x122/0x220 [net_failover] [ 790.255213] failover_slave_register.part.7+0xd2/0x180 [failover] [ 790.256050] failover_event+0x122/0x1ab [failover] [ 790.256821] notifier_call_chain+0x47/0x70 [ 790.257510] register_netdevice+0x20f/0x550 [ 790.258263] iavf_watchdog_task+0x7c8/0xea0 [iavf] [ 790.259009] process_one_work+0x1a7/0x360 [ 790.259705] worker_thread+0x30/0x390 To fix the situation we should check the current adapter state after first unsuccessful mutex_trylock() and return with -EBUSY if it is __IAVF_INIT_CONFIG_ADAPTER. Fixes: 226d528512cf ("iavf: fix locking of critical sections") Signed-off-by: Ivan Vecera <ivecera@redhat.com> --- drivers/net/ethernet/intel/iavf/iavf_main.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)