From patchwork Mon Feb 24 19:06:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Nguyen X-Patchwork-Id: 13988822 X-Patchwork-Delegate: kuba@kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5C4521EA7FA for ; Mon, 24 Feb 2025 19:06:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740424018; cv=none; b=O2NeSH0ZXy12jQfub9hiEKLc15Oq1i3ZEd5TvJR774as+IGRG5GztUwT2AXJoDAiFYz4QvEMUIvW1nmdP82bEqi6aswHts06AV027NSV04jaWEmxv9s6bfRAkIytO1rLQnrCVUCTlsd69LWOK3f1Y2DYshzGe7wgu8p7hgOu7rA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740424018; c=relaxed/simple; bh=5oi5eWMS/VxfQDViWUUaWJURI+rH8oMwrGvNj8xckZk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JkkxGOnIaWUh5BBeMFlmfEE9Xf6Gh6yHBYbyNBh+XvcihVYg/QREpJFhuB1CxuSDYZJBJYnHs9ZXCMDKQms6GINNOrCXBkvR1ihhfMi1O0ZAWAOmAuwIElT+0IvpqF4juLeJxBSfBUYq0nOK7BEztFQAkfCoCkJvUMxqzfqIBOU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=W6WiYiWy; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="W6WiYiWy" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1740424016; x=1771960016; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5oi5eWMS/VxfQDViWUUaWJURI+rH8oMwrGvNj8xckZk=; b=W6WiYiWyCB2sqziafNP5EjHCD0i/CD7d2VBwjjmxtAgd8JQOY+q2zsvK D0NpHI5EdcfoZFzy4Uhx0efCpqPL4svaJRpEJsyTjsCVscGIMQ7ZpwqCc IbSuGvs9Jw3VYa4d1AXzWPekPCu+o6xq0ynSsaX/3xwBzdodZfDufSUhJ eEJdOZDX6R+dv77HP9SLsdi7W/5xDC2K/G7SlNpPUCtL8RP2Wvb7lLykD Df6feoEm1LEY+rSUWelyyglwG9SvbgKabAl/FXMGPdApEPhT4W0aaXLDF ktWmUo25WbYJDG6lmCOESCMvoD8C4sXFy2L+sK0nL+OopMSbYaBxADKs1 w==; X-CSE-ConnectionGUID: xfKUVpY9QE2nwdb0+7wVkA== X-CSE-MsgGUID: eEceMJIYTJiPrJ15O3WNHQ== X-IronPort-AV: E=McAfee;i="6700,10204,11355"; a="58614189" X-IronPort-AV: E=Sophos;i="6.13,312,1732608000"; d="scan'208";a="58614189" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Feb 2025 11:06:54 -0800 X-CSE-ConnectionGUID: LiCucU4jR7Kv99uOIfABAw== X-CSE-MsgGUID: m1xA0oDwTlCY/ScD7uPdJQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="153358456" Received: from anguy11-upstream.jf.intel.com ([10.166.9.133]) by orviesa001.jf.intel.com with ESMTP; 24 Feb 2025 11:06:53 -0800 From: Tony Nguyen To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, andrew+netdev@lunn.ch, netdev@vger.kernel.org Cc: Marcin Szycik , anthony.l.nguyen@intel.com, michal.swiatkowski@linux.intel.com, Sujai Buvaneswaran , Martyna Szapar-Mudlaw , Simon Horman Subject: [PATCH net 1/5] ice: Fix deinitializing VF in error path Date: Mon, 24 Feb 2025 11:06:41 -0800 Message-ID: <20250224190647.3601930-2-anthony.l.nguyen@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250224190647.3601930-1-anthony.l.nguyen@intel.com> References: <20250224190647.3601930-1-anthony.l.nguyen@intel.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Marcin Szycik If ice_ena_vfs() fails after calling ice_create_vf_entries(), it frees all VFs without removing them from snapshot PF-VF mailbox list, leading to list corruption. Reproducer: devlink dev eswitch set $PF1_PCI mode switchdev ip l s $PF1 up ip l s $PF1 promisc on sleep 1 echo 1 > /sys/class/net/$PF1/device/sriov_numvfs sleep 1 echo 1 > /sys/class/net/$PF1/device/sriov_numvfs Trace (minimized): list_add corruption. next->prev should be prev (ffff8882e241c6f0), but was 0000000000000000. (next=ffff888455da1330). kernel BUG at lib/list_debug.c:29! RIP: 0010:__list_add_valid_or_report+0xa6/0x100 ice_mbx_init_vf_info+0xa7/0x180 [ice] ice_initialize_vf_entry+0x1fa/0x250 [ice] ice_sriov_configure+0x8d7/0x1520 [ice] ? __percpu_ref_switch_mode+0x1b1/0x5d0 ? __pfx_ice_sriov_configure+0x10/0x10 [ice] Sometimes a KASAN report can be seen instead with a similar stack trace: BUG: KASAN: use-after-free in __list_add_valid_or_report+0xf1/0x100 VFs are added to this list in ice_mbx_init_vf_info(), but only removed in ice_free_vfs(). Move the removing to ice_free_vf_entries(), which is also being called in other places where VFs are being removed (including ice_free_vfs() itself). Fixes: 8cd8a6b17d27 ("ice: move VF overflow message count into struct ice_mbx_vf_info") Reported-by: Sujai Buvaneswaran Closes: https://lore.kernel.org/intel-wired-lan/PH0PR11MB50138B635F2E5CEB7075325D961F2@PH0PR11MB5013.namprd11.prod.outlook.com Reviewed-by: Martyna Szapar-Mudlaw Signed-off-by: Marcin Szycik Reviewed-by: Simon Horman Tested-by: Sujai Buvaneswaran Signed-off-by: Tony Nguyen --- drivers/net/ethernet/intel/ice/ice_sriov.c | 5 +---- drivers/net/ethernet/intel/ice/ice_vf_lib.c | 8 ++++++++ drivers/net/ethernet/intel/ice/ice_vf_lib_private.h | 1 + 3 files changed, 10 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_sriov.c b/drivers/net/ethernet/intel/ice/ice_sriov.c index b83f99c01d91..8aabf7749aa5 100644 --- a/drivers/net/ethernet/intel/ice/ice_sriov.c +++ b/drivers/net/ethernet/intel/ice/ice_sriov.c @@ -36,6 +36,7 @@ static void ice_free_vf_entries(struct ice_pf *pf) hash_for_each_safe(vfs->table, bkt, tmp, vf, entry) { hash_del_rcu(&vf->entry); + ice_deinitialize_vf_entry(vf); ice_put_vf(vf); } } @@ -193,10 +194,6 @@ void ice_free_vfs(struct ice_pf *pf) wr32(hw, GLGEN_VFLRSTAT(reg_idx), BIT(bit_idx)); } - /* clear malicious info since the VF is getting released */ - if (!ice_is_feature_supported(pf, ICE_F_MBX_LIMIT)) - list_del(&vf->mbx_info.list_entry); - mutex_unlock(&vf->cfg_lock); } diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib.c b/drivers/net/ethernet/intel/ice/ice_vf_lib.c index c7c0c2f50c26..815ad0bfe832 100644 --- a/drivers/net/ethernet/intel/ice/ice_vf_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_vf_lib.c @@ -1036,6 +1036,14 @@ void ice_initialize_vf_entry(struct ice_vf *vf) mutex_init(&vf->cfg_lock); } +void ice_deinitialize_vf_entry(struct ice_vf *vf) +{ + struct ice_pf *pf = vf->pf; + + if (!ice_is_feature_supported(pf, ICE_F_MBX_LIMIT)) + list_del(&vf->mbx_info.list_entry); +} + /** * ice_dis_vf_qs - Disable the VF queues * @vf: pointer to the VF structure diff --git a/drivers/net/ethernet/intel/ice/ice_vf_lib_private.h b/drivers/net/ethernet/intel/ice/ice_vf_lib_private.h index 0c7e77c0a09f..5392b0404986 100644 --- a/drivers/net/ethernet/intel/ice/ice_vf_lib_private.h +++ b/drivers/net/ethernet/intel/ice/ice_vf_lib_private.h @@ -24,6 +24,7 @@ #endif void ice_initialize_vf_entry(struct ice_vf *vf); +void ice_deinitialize_vf_entry(struct ice_vf *vf); void ice_dis_vf_qs(struct ice_vf *vf); int ice_check_vf_init(struct ice_vf *vf); enum virtchnl_status_code ice_err_to_virt_err(int err); From patchwork Mon Feb 24 19:06:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Nguyen X-Patchwork-Id: 13988826 X-Patchwork-Delegate: kuba@kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B29E31EDA26 for ; Mon, 24 Feb 2025 19:06:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740424021; cv=none; b=WAPEAxUMkjFBJfv5A23205YA6mDXAgB6BmEhgdY5WmQKwvS+1hKpucDADP9pDU0H3ohfzfLUs8hvKhrpw50emc3f5ZzOkieu7aQd0rYzJeQ9xTHIcSLfNI6p4UVGTL+/eYUiQst0dv8FC3ABpfW32JB3kajUSJsBHGkgh5y7VVw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740424021; c=relaxed/simple; bh=zXIV4Vs4DTZa9MGglwuulnjPA627lXa9IWLZ1k8YS88=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qoKygHFU9L9UMT1Xvk5dxLs2WdThwdUgQ1BT1WGzg+0aDKT+y8qGX939IcgY3SDbIYIXhsLaBNuufOsQCUBd85HzH0QjxGSNEpx92aLvDM73BlMqWyK1BdJVHhLozkezrJMlgK4XqswXcbJcTxs0g0jfFVbIqfvssRBscb6gfoU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=VJN0uPeY; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="VJN0uPeY" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1740424017; x=1771960017; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zXIV4Vs4DTZa9MGglwuulnjPA627lXa9IWLZ1k8YS88=; b=VJN0uPeYFqzpHkBiZ4pk4JMoJO2Ffm1ROcn+Vhls4wXxHswy+bIW5c8e z1Bx9o5p57WK5DHNihUVtjMawimCdkNwsn5F5S0u4PGyEYxD0sjjJY259 HKMkT89YrD/JhODJGAbOGSoGOgg0ea6snf//XD5hp57IKk2xNz9ZFJu9b R+9YhbjGJfERgRmLh0zSu1DQMCAHweQG8QzKcaDNry1ih58OkCBLHwwMj 8PIqMBxmx7aj2NQWXNM2gkK+ISqmxNbpr98Dkc4Kpaj/geowf81KZyDbP cWsZ4W8azLx4ChR6FeSoWhIvBPgCToT3rxwho+tq/WNiUhCewqjgKFUcI g==; X-CSE-ConnectionGUID: w2qgzB+WT3KsZqUGLQ5bCw== X-CSE-MsgGUID: VVvWsJcgQtCJM1tA0lYbNQ== X-IronPort-AV: E=McAfee;i="6700,10204,11355"; a="58614195" X-IronPort-AV: E=Sophos;i="6.13,312,1732608000"; d="scan'208";a="58614195" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Feb 2025 11:06:55 -0800 X-CSE-ConnectionGUID: G/ozPYuZTLqSemwXHL7tzw== X-CSE-MsgGUID: pO5Huj65RRWoe2RUGpndWw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="153358459" Received: from anguy11-upstream.jf.intel.com ([10.166.9.133]) by orviesa001.jf.intel.com with ESMTP; 24 Feb 2025 11:06:54 -0800 From: Tony Nguyen To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, andrew+netdev@lunn.ch, netdev@vger.kernel.org Cc: Marcin Szycik , anthony.l.nguyen@intel.com, michal.swiatkowski@linux.intel.com, Sujai Buvaneswaran , Martyna Szapar-Mudlaw , Simon Horman Subject: [PATCH net 2/5] ice: Avoid setting default Rx VSI twice in switchdev setup Date: Mon, 24 Feb 2025 11:06:42 -0800 Message-ID: <20250224190647.3601930-3-anthony.l.nguyen@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250224190647.3601930-1-anthony.l.nguyen@intel.com> References: <20250224190647.3601930-1-anthony.l.nguyen@intel.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Marcin Szycik As part of switchdev environment setup, uplink VSI is configured as default for both Tx and Rx. Default Rx VSI is also used by promiscuous mode. If promisc mode is enabled and an attempt to enter switchdev mode is made, the setup will fail because Rx VSI is already configured as default (rule exists). Reproducer: devlink dev eswitch set $PF1_PCI mode switchdev ip l s $PF1 up ip l s $PF1 promisc on echo 1 > /sys/class/net/$PF1/device/sriov_numvfs In switchdev setup, use ice_set_dflt_vsi() instead of plain ice_cfg_dflt_vsi(), which avoids repeating setting default VSI for Rx if it's already configured. Fixes: 50d62022f455 ("ice: default Tx rule instead of to queue") Reported-by: Sujai Buvaneswaran Closes: https://lore.kernel.org/intel-wired-lan/PH0PR11MB50138B635F2E5CEB7075325D961F2@PH0PR11MB5013.namprd11.prod.outlook.com Reviewed-by: Martyna Szapar-Mudlaw Signed-off-by: Marcin Szycik Reviewed-by: Simon Horman Tested-by: Sujai Buvaneswaran Signed-off-by: Tony Nguyen --- drivers/net/ethernet/intel/ice/ice_eswitch.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/drivers/net/ethernet/intel/ice/ice_eswitch.c b/drivers/net/ethernet/intel/ice/ice_eswitch.c index fb527434b58b..d649c197cf67 100644 --- a/drivers/net/ethernet/intel/ice/ice_eswitch.c +++ b/drivers/net/ethernet/intel/ice/ice_eswitch.c @@ -38,8 +38,7 @@ static int ice_eswitch_setup_env(struct ice_pf *pf) if (ice_vsi_add_vlan_zero(uplink_vsi)) goto err_vlan_zero; - if (ice_cfg_dflt_vsi(uplink_vsi->port_info, uplink_vsi->idx, true, - ICE_FLTR_RX)) + if (ice_set_dflt_vsi(uplink_vsi)) goto err_def_rx; if (ice_cfg_dflt_vsi(uplink_vsi->port_info, uplink_vsi->idx, true, From patchwork Mon Feb 24 19:06:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Nguyen X-Patchwork-Id: 13988823 X-Patchwork-Delegate: kuba@kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 24EBF1FC0F1 for ; Mon, 24 Feb 2025 19:06:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740424019; cv=none; b=TBwKgN0A4VbXoRcy6ttQ7qFRnYdQz2kyKjKjv6GkF41mpu4t/wogrt6Bu+ZIuzoBmyF7nkxuNg441iBHcofjbOjUnj1bbuvanvcj3DYJ6Y/V4Rrc5d4Pe1RG0hNdrnpeKTtVaSyKq/RJJDSVl/zb3MexghMT7AMNMPPth67ML6g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740424019; c=relaxed/simple; bh=zCdwYvNSOb0TUw/feMD/Q//2+RlqrrePrcFwiU55D/o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=B9Jvt0psJv+6nkghhiYNfq7Jj6EyhEI7rUmx1P0TXWvFFqxh1MNqOYHb38xhjQRIZ7d1k7voGO0WENp6I+2CurGsCbgK0b/ep6TVB3B/4ZR6YZ34CqShK0kphue860elMcOkipwCFponANZO8zpo4Vw40MvC338X73WYZurMX0k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=SzUMdczp; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="SzUMdczp" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1740424018; x=1771960018; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=zCdwYvNSOb0TUw/feMD/Q//2+RlqrrePrcFwiU55D/o=; b=SzUMdczpvNElGYpfPoH7cGvG4Xf9CcH8pzKfUO+u7jzGcqnHTsWPG6r0 WxP1xpPvnmk6cgcwdHL9WXPEQW0+QmmgH9xESR6ro5B1GiM86J9Dm4Eyc eTEcUKTShdIBmPjmS+7LSOJQu/fXbwcbFhEm/8xoxDR5jbBURQ3mTPGNd wvTb01kyv5Z8NcyXyZ61mBFKH/53kj0upkBCY3Ch26kv8+jsQnHAkiiNx +vQDsL2ayuknRqz70oHjaby9XInX7halbT3vGilqDvV1xFMswDuDEoQdl 98Ym5lw0xSWxEo1m0o6GwaZ5RnTR+njuxlptZkULusGEPOEs7RyzjIeAO Q==; X-CSE-ConnectionGUID: HZ1DmZn9TZyx072+XpajDg== X-CSE-MsgGUID: DMdnvOAmTCaFopX5fFOBbQ== X-IronPort-AV: E=McAfee;i="6700,10204,11355"; a="58614201" X-IronPort-AV: E=Sophos;i="6.13,312,1732608000"; d="scan'208";a="58614201" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Feb 2025 11:06:55 -0800 X-CSE-ConnectionGUID: WzCjF33eRnG1mN+ONGXb8A== X-CSE-MsgGUID: JvpgXMikTLqkv49NgfE0tg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="153358462" Received: from anguy11-upstream.jf.intel.com ([10.166.9.133]) by orviesa001.jf.intel.com with ESMTP; 24 Feb 2025 11:06:54 -0800 From: Tony Nguyen To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, andrew+netdev@lunn.ch, netdev@vger.kernel.org Cc: Ahmed Zaki , anthony.l.nguyen@intel.com, willemb@google.com, Madhu Chittim , Simon Horman , Samuel Salin Subject: [PATCH net 3/5] idpf: synchronize pending IRQs after disable Date: Mon, 24 Feb 2025 11:06:43 -0800 Message-ID: <20250224190647.3601930-4-anthony.l.nguyen@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250224190647.3601930-1-anthony.l.nguyen@intel.com> References: <20250224190647.3601930-1-anthony.l.nguyen@intel.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Ahmed Zaki Wait for pending IRQ handler after it is disabled. This will ensure the IRQ is cleanly freed afterwards. Fixes: d4d558718266 ("idpf: initialize interrupts and enable vport") Reviewed-by: Madhu Chittim Suggested-by: Sudheer Mogilappagari Signed-off-by: Ahmed Zaki Reviewed-by: Simon Horman Tested-by: Samuel Salin Signed-off-by: Tony Nguyen --- drivers/net/ethernet/intel/idpf/idpf_txrx.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/intel/idpf/idpf_txrx.c b/drivers/net/ethernet/intel/idpf/idpf_txrx.c index 9be6a6b59c4e..8006bd9a95f6 100644 --- a/drivers/net/ethernet/intel/idpf/idpf_txrx.c +++ b/drivers/net/ethernet/intel/idpf/idpf_txrx.c @@ -3592,10 +3592,15 @@ static void idpf_vport_intr_rel_irq(struct idpf_vport *vport) static void idpf_vport_intr_dis_irq_all(struct idpf_vport *vport) { struct idpf_q_vector *q_vector = vport->q_vectors; - int q_idx; + int q_idx, vidx, irq_num; + + for (q_idx = 0; q_idx < vport->num_q_vectors; q_idx++) { + vidx = vport->q_vector_idxs[q_idx]; + irq_num = vport->adapter->msix_entries[vidx].vector; - for (q_idx = 0; q_idx < vport->num_q_vectors; q_idx++) writel(0, q_vector[q_idx].intr_reg.dyn_ctl); + synchronize_irq(irq_num); + } } /** From patchwork Mon Feb 24 19:06:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Nguyen X-Patchwork-Id: 13988824 X-Patchwork-Delegate: kuba@kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9CD841FC117 for ; Mon, 24 Feb 2025 19:06:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740424020; cv=none; b=AAcUX1iqsstHUGgKF51RAmYuwsFZuSxBVQPYk4Of22wboviTg/Bg+DR/L3aCETHFbfE0SA/ijMNg6F0acxpd0yadJQrszNUfzS97VeHtBMj4ykBQlSWLtS9Dz5QV51DbHqIMC58/4sgdOZPhhNw++WeHJMt9ZJS+uesARVHf7pE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740424020; c=relaxed/simple; bh=k1PwlPCKImjHJOAH/Zyy1ccbheJTwqF9GXZwpF0lzVE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=p4qZq+3edRaP72xixMnQ60O34hoY/9ALyYtsL/2gkUEr+2RnoFRDELENntNmsV3LcL8N9s9A8ozON6vaFLAhnQqkR9kkMA0CTkabYxd7GocZYA9+0d527tahEORxbp1NzPtwLAVFy4ZdRQWifzf8esp1MTPgsMVx6g1vHTxKgtI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=brMS6lyF; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="brMS6lyF" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1740424019; x=1771960019; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=k1PwlPCKImjHJOAH/Zyy1ccbheJTwqF9GXZwpF0lzVE=; b=brMS6lyF/meTnAn4rEixuEU+qL+EeDK7dQLz/tgEf5KaSiTiS3xw6Ly2 KtK1nybyAMIDEP8BZkzbKoyu6zRfUHgMMd1RpHo+/uNeO9L+2Grrngvap ILodACzPrq8zMOb+Qn8Ifzlj3sXhBajssyuUVvrmSx4Xkfl6ca8+w+OVh qLzxijSI3xqpvIV0/7Naqt9P8cPWVL+zxdUbqnIdRErbM3h0sFKXjzfZQ n4mbYMzDAIJolHCp0PSEZteQNSZlsrsh8QJS567Q7TsSODYrFKffGFZ8V 6CmGCPgPF6Fx8hep8ky5ZA11VHPmTNKmr5RRKaWIq7/AhyHZktCrmNofR Q==; X-CSE-ConnectionGUID: 5Ts/BP8BQBGXA6A88s0vpA== X-CSE-MsgGUID: nhWJIdqjR4mkXxu1os2mog== X-IronPort-AV: E=McAfee;i="6700,10204,11355"; a="58614207" X-IronPort-AV: E=Sophos;i="6.13,312,1732608000"; d="scan'208";a="58614207" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Feb 2025 11:06:55 -0800 X-CSE-ConnectionGUID: H0pONhDPT7KeqPKrPh8dmA== X-CSE-MsgGUID: vyux8bQuQPSlgr6n0QkdGQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="153358465" Received: from anguy11-upstream.jf.intel.com ([10.166.9.133]) by orviesa001.jf.intel.com with ESMTP; 24 Feb 2025 11:06:54 -0800 From: Tony Nguyen To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, andrew+netdev@lunn.ch, netdev@vger.kernel.org Cc: Jacob Keller , anthony.l.nguyen@intel.com, przemyslaw.kitszel@intel.com, Rafal Romanowski Subject: [PATCH net 4/5] iavf: fix circular lock dependency with netdev_lock Date: Mon, 24 Feb 2025 11:06:44 -0800 Message-ID: <20250224190647.3601930-5-anthony.l.nguyen@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250224190647.3601930-1-anthony.l.nguyen@intel.com> References: <20250224190647.3601930-1-anthony.l.nguyen@intel.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Jacob Keller We have recently seen reports of lockdep circular lock dependency warnings when loading the iAVF driver: [ 1504.790308] ====================================================== [ 1504.790309] WARNING: possible circular locking dependency detected [ 1504.790310] 6.13.0 #net_next_rt.c2933b2befe2.el9 Not tainted [ 1504.790311] ------------------------------------------------------ [ 1504.790312] kworker/u128:0/13566 is trying to acquire lock: [ 1504.790313] ffff97d0e4738f18 (&dev->lock){+.+.}-{4:4}, at: register_netdevice+0x52c/0x710 [ 1504.790320] [ 1504.790320] but task is already holding lock: [ 1504.790321] ffff97d0e47392e8 (&adapter->crit_lock){+.+.}-{4:4}, at: iavf_finish_config+0x37/0x240 [iavf] [ 1504.790330] [ 1504.790330] which lock already depends on the new lock. [ 1504.790330] [ 1504.790330] [ 1504.790330] the existing dependency chain (in reverse order) is: [ 1504.790331] [ 1504.790331] -> #1 (&adapter->crit_lock){+.+.}-{4:4}: [ 1504.790333] __lock_acquire+0x52d/0xbb0 [ 1504.790337] lock_acquire+0xd9/0x330 [ 1504.790338] mutex_lock_nested+0x4b/0xb0 [ 1504.790341] iavf_finish_config+0x37/0x240 [iavf] [ 1504.790347] process_one_work+0x248/0x6d0 [ 1504.790350] worker_thread+0x18d/0x330 [ 1504.790352] kthread+0x10e/0x250 [ 1504.790354] ret_from_fork+0x30/0x50 [ 1504.790357] ret_from_fork_asm+0x1a/0x30 [ 1504.790361] [ 1504.790361] -> #0 (&dev->lock){+.+.}-{4:4}: [ 1504.790364] check_prev_add+0xf1/0xce0 [ 1504.790366] validate_chain+0x46a/0x570 [ 1504.790368] __lock_acquire+0x52d/0xbb0 [ 1504.790370] lock_acquire+0xd9/0x330 [ 1504.790371] mutex_lock_nested+0x4b/0xb0 [ 1504.790372] register_netdevice+0x52c/0x710 [ 1504.790374] iavf_finish_config+0xfa/0x240 [iavf] [ 1504.790379] process_one_work+0x248/0x6d0 [ 1504.790381] worker_thread+0x18d/0x330 [ 1504.790383] kthread+0x10e/0x250 [ 1504.790385] ret_from_fork+0x30/0x50 [ 1504.790387] ret_from_fork_asm+0x1a/0x30 [ 1504.790389] [ 1504.790389] other info that might help us debug this: [ 1504.790389] [ 1504.790389] Possible unsafe locking scenario: [ 1504.790389] [ 1504.790390] CPU0 CPU1 [ 1504.790391] ---- ---- [ 1504.790391] lock(&adapter->crit_lock); [ 1504.790393] lock(&dev->lock); [ 1504.790394] lock(&adapter->crit_lock); [ 1504.790395] lock(&dev->lock); [ 1504.790397] [ 1504.790397] *** DEADLOCK *** This appears to be caused by the change in commit 5fda3f35349b ("net: make netdev_lock() protect netdev->reg_state"), which added a netdev_lock() in register_netdevice. The iAVF driver calls register_netdevice() from iavf_finish_config(), as a final stage of its state machine post-probe. It currently takes the RTNL lock, then the netdev lock, and then the device critical lock. This pattern is used throughout the driver. Thus there is a strong dependency that the crit_lock should not be acquired before the net device lock. The change to register_netdevice creates an ABBA lock order violation because the iAVF driver is holding the crit_lock while calling register_netdevice, which then takes the netdev_lock. It seems likely that future refactors could result in netdev APIs which hold the netdev_lock while calling into the driver. This means that we should not re-order the locks so that netdev_lock is acquired after the device private crit_lock. Instead, notice that we already release the netdev_lock prior to calling the register_netdevice. This flow only happens during the early driver initialization as we transition through the __IAVF_STARTUP, __IAVF_INIT_VERSION_CHECK, __IAVF_INIT_GET_RESOURCES, etc. Analyzing the places where we take crit_lock in the driver there are two sources: a) several of the work queue tasks including adminq_task, watchdog_task, reset_task, and the finish_config task. b) various callbacks which ultimately stem back to .ndo operations or ethtool operations. The latter cannot be triggered until after the netdevice registration is completed successfully. The iAVF driver uses alloc_ordered_workqueue, which is an unbound workqueue that has a max limit of 1, and thus guarantees that only a single work item on the queue is executing at any given time, so none of the other work threads could be executing due to the ordered workqueue guarantees. The iavf_finish_config() function also does not do anything else after register_netdevice, unless it fails. It seems unlikely that the driver private crit_lock is protecting anything that register_netdevice() itself touches. Thus, to fix this ABBA lock violation, lets simply release the adapter->crit_lock as well as netdev_lock prior to calling register_netdevice(). We do still keep holding the RTNL lock as required by the function. If we do fail to register the netdevice, then we re-acquire the adapter critical lock to finish the transition back to __IAVF_INIT_CONFIG_ADAPTER. This ensures every call where both netdev_lock and the adapter->crit_lock are acquired under the same ordering. Fixes: afc664987ab3 ("eth: iavf: extend the netdev_lock usage") Signed-off-by: Jacob Keller Tested-by: Przemek Kitszel Reviewed-by: Przemek Kitszel Reviewed-by: Jakub Kicinski Tested-by: Rafal Romanowski Signed-off-by: Tony Nguyen --- drivers/net/ethernet/intel/iavf/iavf_main.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/intel/iavf/iavf_main.c b/drivers/net/ethernet/intel/iavf/iavf_main.c index 852e5b62f0a5..6faa62bced3a 100644 --- a/drivers/net/ethernet/intel/iavf/iavf_main.c +++ b/drivers/net/ethernet/intel/iavf/iavf_main.c @@ -1983,7 +1983,7 @@ static int iavf_reinit_interrupt_scheme(struct iavf_adapter *adapter, bool runni static void iavf_finish_config(struct work_struct *work) { struct iavf_adapter *adapter; - bool netdev_released = false; + bool locks_released = false; int pairs, err; adapter = container_of(work, struct iavf_adapter, finish_config); @@ -2012,19 +2012,22 @@ static void iavf_finish_config(struct work_struct *work) netif_set_real_num_tx_queues(adapter->netdev, pairs); if (adapter->netdev->reg_state != NETREG_REGISTERED) { + mutex_unlock(&adapter->crit_lock); netdev_unlock(adapter->netdev); - netdev_released = true; + locks_released = true; err = register_netdevice(adapter->netdev); if (err) { dev_err(&adapter->pdev->dev, "Unable to register netdev (%d)\n", err); /* go back and try again.*/ + mutex_lock(&adapter->crit_lock); iavf_free_rss(adapter); iavf_free_misc_irq(adapter); iavf_reset_interrupt_capability(adapter); iavf_change_state(adapter, __IAVF_INIT_CONFIG_ADAPTER); + mutex_unlock(&adapter->crit_lock); goto out; } } @@ -2040,9 +2043,10 @@ static void iavf_finish_config(struct work_struct *work) } out: - mutex_unlock(&adapter->crit_lock); - if (!netdev_released) + if (!locks_released) { + mutex_unlock(&adapter->crit_lock); netdev_unlock(adapter->netdev); + } rtnl_unlock(); } From patchwork Mon Feb 24 19:06:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Nguyen X-Patchwork-Id: 13988825 X-Patchwork-Delegate: kuba@kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4E1E1FCFD9 for ; Mon, 24 Feb 2025 19:06:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740424021; cv=none; b=k9BKnWMtBem6Zk9OoUfnMLJZ8zsJ4/SztfS2N29YHdPDdLNKx5UWE9wrw7qNFsbQWP+c/MQkfyX3J323WxoMV9r6HmXqnzVSGDcLxRhnYJjWDVaGaHu63HZaza1SotEgusc675oq/IbZ0HSu1Qkc5bJDdOeYSsjyJWiA5b/W3jo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1740424021; c=relaxed/simple; bh=6KJ3WfDuHrHmdx+KLYqYOepx6d79l0cHoc86UKibcA0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MKscv8rG5myaNQ6idx5Jx7hDs2URmAvoCan7moq0YjB0YQPwsomOMDk/88WJxqnzYI2Byd/2dMLFG86/5ZWspQqbYj8vIV6XcC1MrtYpCZCSLNBX7vuXNEAc/JVGEOxjHrPd7d/viLsEaNfnq7L8R2UckRY7Jutq5AOLqckysfo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=SleyUcsG; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="SleyUcsG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1740424020; x=1771960020; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6KJ3WfDuHrHmdx+KLYqYOepx6d79l0cHoc86UKibcA0=; b=SleyUcsGKHORXDFPzYYim3eSxEASHvMLMjkQKmddo0DS+hp2Xu9puLKx q0zXWBPKnwN3Qv28RgM32zLVFYpZapLKigbeOrkvW9GXlhc2eiR2kM8Ty WMVQU2aNs0JalbwdvpKtUOJduUM2PMEphGy4vmObKinSwH4R54gMCowbi Ec5m72VhyLePPTZ6sTOXO6CLiZZsiaO+5vfqe7hgL9JheXluqeK4qORQQ cyjy2BZxH8hNQprN/Sni/dQVigOPV6CyMYrNJh31PeXsoyLwFAbEP8n+N +x4l5vMA+l/fAAIrjtLrdP5QGX+J00rvX2DyPziZAwOtsNSekaV2BR9zI w==; X-CSE-ConnectionGUID: I6zNdJa4Q3uRfz5fzLnrZQ== X-CSE-MsgGUID: NrWNjn7BQf6K5U3q2MOHtw== X-IronPort-AV: E=McAfee;i="6700,10204,11355"; a="58614213" X-IronPort-AV: E=Sophos;i="6.13,312,1732608000"; d="scan'208";a="58614213" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Feb 2025 11:06:55 -0800 X-CSE-ConnectionGUID: Q3bMzdu5Tr2lZszrf5DbPQ== X-CSE-MsgGUID: teVHScsXRjmKJvOo+E71vA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="153358468" Received: from anguy11-upstream.jf.intel.com ([10.166.9.133]) by orviesa001.jf.intel.com with ESMTP; 24 Feb 2025 11:06:54 -0800 From: Tony Nguyen To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, andrew+netdev@lunn.ch, netdev@vger.kernel.org Cc: Piotr Kwapulinski , anthony.l.nguyen@intel.com, Dan Carpenter , Michal Swiatkowski , Przemek Kitszel , Simon Horman , Bharath R Subject: [PATCH net 5/5] ixgbe: fix media cage present detection for E610 device Date: Mon, 24 Feb 2025 11:06:45 -0800 Message-ID: <20250224190647.3601930-6-anthony.l.nguyen@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250224190647.3601930-1-anthony.l.nguyen@intel.com> References: <20250224190647.3601930-1-anthony.l.nguyen@intel.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Piotr Kwapulinski The commit 23c0e5a16bcc ("ixgbe: Add link management support for E610 device") introduced incorrect checking of media cage presence for E610 device. Fix it. Fixes: 23c0e5a16bcc ("ixgbe: Add link management support for E610 device") Reported-by: Dan Carpenter Closes: https://lore.kernel.org/all/e7d73b32-f12a-49d1-8b60-1ef83359ec13@stanley.mountain/ Reviewed-by: Michal Swiatkowski Reviewed-by: Przemek Kitszel Signed-off-by: Piotr Kwapulinski Reviewed-by: Simon Horman Tested-by: Bharath R Signed-off-by: Tony Nguyen --- drivers/net/ethernet/intel/ixgbe/ixgbe_e610.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_e610.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_e610.c index 683c668672d6..cb07ecd8937d 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_e610.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_e610.c @@ -1122,7 +1122,7 @@ static bool ixgbe_is_media_cage_present(struct ixgbe_hw *hw) * returns error (ENOENT), then no cage present. If no cage present then * connection type is backplane or BASE-T. */ - return ixgbe_aci_get_netlist_node(hw, cmd, NULL, NULL); + return !ixgbe_aci_get_netlist_node(hw, cmd, NULL, NULL); } /**