From patchwork Tue Oct 15 15:23:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yonatan Maman X-Patchwork-Id: 13836623 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2077.outbound.protection.outlook.com [40.107.237.77]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F06831DD0D9; Tue, 15 Oct 2024 15:24:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.77 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729005882; cv=fail; b=DlIAcFCkF7nraenPr8b2FNbVu11WigAYCeGXXOvzJnK9gJdyAjUQwMmQ8dybSSR0Kk4eSCnuTY93PyCfsP3J8M8CklM8BAnzlJKVHjGm30QwK1swd5OZsIyFdENQnk5tovWNBZu9sEHpAKfHZEwwVbyFOAzHY7WxRKODHXqPmH0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729005882; c=relaxed/simple; bh=YMDogKdyPe/Sk54OSkn0IWnlmUZ5jRadf6NKfJQtHv0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=MWxtiL1agFlG/7RxZCAnBzFDClTCNfoDBD0tMqr56cXtHTYh7a+vIVw7IerrtRGXqZC6L8h++EYNaryDdHSMBwDRj6WsmnhBH+98OHd9G/vXqHMaAiiRJ/HN8BhmATMLlMNDCRov99ok4OOB3lGyeaDPYYXIpvijpALSPvfsBfk= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Lh6jSU5c; arc=fail smtp.client-ip=40.107.237.77 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Lh6jSU5c" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=N9YJsP9XsJbz7uAP2K0dp6xNI/hDiO6qGTeK5Uw1Q0rOgi+uNQvSh3vIIFfk0a6cmqCMirB3A+OmDiUorbUGTZXxQBkkQR9Jy/U+JA/ESqvu4DTUk0f6oDqldadlTT2pKUB6v9gpH6W/94pp4PNOrPZ/FtuRxIPNyt3TriwomJcO9vAIKI60ojWMCF072FwSdkZVW2hG09NslJSr4AOOEBKnKaOCExtv+MfCPYYGmemjO+8vz6fj2Xs0mYkV8he/3rtxuZ474TzLi9c//rpZAN+waW8Jxdw3oHEWC2vT4R8TvO3HmBGg6H9a+Xhlff9Ntmp3PODNl9OpF0WQYUn84Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=E/lrVl1VFYPdVp8Dnfq0tyenbmZBRBm95p50EAMlqC8=; b=H1+AM++YtgfGPwve3GXUHlwYEq71HTAIbZRXhwuoSmmhO01BhmDNcFzXcFg2jzi1Bq5AJlY8iVVatCDZ3e/cbZtXMWBv7ByijLKuJaJ1aI18cqz31C0staQxjr9wbScJ3YckPF14WWNqBfK1KnejStmq7VR2j33enI0ZUhNMxafVdIUshIAhM89E7BwZN2i/gwttJh1+iqTVzSnfcytCb+Yo6zkj8fFZWI4ldaZkxoY7R6okgNrDQF8hvBoFFHyS2f+5BoFm6oNMnrXa7q044t7JeF5d/Gijk3/2oIpqOKSRn/CP5p+XBexANTq9+Dc+JQEoRtve11HScZ2Ya/+s5g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=E/lrVl1VFYPdVp8Dnfq0tyenbmZBRBm95p50EAMlqC8=; b=Lh6jSU5cq+AJm3eSwpR5vM3WKFGMiiI0ArrgHRcGy+G26xrIxj5Qm2jXmHwzrHFsLyUg+qfbp4/Ft9Nvp+xWOochIVPJW9EmpinCVjW+oW+d/uCloP9K0zYyFVl9TedC2EFBHJqwztMPmzpbWuWzjYnvtnXcGgbXvhDI0EwWH1Xro72YaVvhY0xuHNiiXilJxBfNnuncUzQ7swa0a4WgtwC2FRTRsmx69EdaBOwYejBhvTiPHKMbsR9hpWp0BQ7stxBAjRH4SlxmIycccI2fCxN1s959plO7279ZwxlXd2xlRVYmCnqrpkemIC+TgRjbuhoHB+kdXF1zXBk/LVmMGg== Received: from BN9PR03CA0263.namprd03.prod.outlook.com (2603:10b6:408:ff::28) by IA0PR12MB8864.namprd12.prod.outlook.com (2603:10b6:208:485::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8048.18; Tue, 15 Oct 2024 15:24:35 +0000 Received: from BN3PEPF0000B36D.namprd21.prod.outlook.com (2603:10b6:408:ff:cafe::1c) by BN9PR03CA0263.outlook.office365.com (2603:10b6:408:ff::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8069.18 via Frontend Transport; Tue, 15 Oct 2024 15:24:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by BN3PEPF0000B36D.mail.protection.outlook.com (10.167.243.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.1 via Frontend Transport; Tue, 15 Oct 2024 15:24:34 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 15 Oct 2024 08:24:21 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Tue, 15 Oct 2024 08:24:20 -0700 Received: from vdi.nvidia.com (10.127.8.9) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Tue, 15 Oct 2024 08:24:17 -0700 From: Yonatan Maman To: , , , , , , , , , , , , , , , CC: Yonatan Maman , Gal Shalom Subject: [PATCH v1 4/4] RDMA/mlx5: Enabling ATS for ODP memory Date: Tue, 15 Oct 2024 18:23:48 +0300 Message-ID: <20241015152348.3055360-5-ymaman@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241015152348.3055360-1-ymaman@nvidia.com> References: <20241015152348.3055360-1-ymaman@nvidia.com> Precedence: bulk X-Mailing-List: linux-rdma@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN3PEPF0000B36D:EE_|IA0PR12MB8864:EE_ X-MS-Office365-Filtering-Correlation-Id: 831e4da0-c848-4740-6d30-08dced2d799a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|7416014|36860700013|82310400026|921020; X-Microsoft-Antispam-Message-Info: 4V6EdxkGyMmUbDNSLHuCOf4a7m//JN836m3OuI9qYscOVWt+U1KV+VogYCWs5hF6D44MvDWdOQ0C7eYTaZCOy8NbHzvaIrv0AssPtniQBp/a2EmvngZRJVVkErCMv42+1DZfO9m9GbLVNurLr7XL3vNp7FbzDuegoZZA0dtZGMnLCIIlc8QHFjvjcAjObQgwmGfD+kqpUlkAMXuM+Qbt0KUJQ2QDp6s4UUfQdRPb5YbtEOYb+BTjBzL7mHrPnmLxclt2sAkn5CSW8HwV/VzOOcf7GQ77hqWBrFuGvBS1NCFQFtL4Ld9+Wtv9ieBydDY59W9biykBJABEWo2iz7D0Ot2Wc0NtE2BJpOYxkZf9SVjH7KviVTnncewtNe0zatpdI9zTj5yq9Cs5QjymV7l/ev4AzLWjtsH6Uj3hNap8Fp7X0bPHseR+oqsS7b98JvRHZqaFk8zaSr4nqkGB12v5YzITQR1P8LhMQpax5+PIp3gKBbK5mTlbbL3JWl1S8rRaVHAgD4vAfO0fzEJS7EKZe5ZeGJpng8GOh6E2gS0Dl31nsQb5bVWOEFG7mJCUtlw0IOqbpffRECcanPk1zfVQ3x2C4s0ORa6CMT4mQ1DODPzLUPa8tqiUCiqVAIigjj0e8mSuP9JOWA1ym02pJkelDoCavGeYOtc3oOhjl/Z8fam6tAG+chcIxdRlfYSfsU4jTUax8anRN6QLr9qkB5J8hVHki1SUVwhigcQylrGIGC60E3OAPGC3sdyKYrU2/iugyiy1rpbuhzd8vMCvJoBh6x9Lzz0zxvxPLzBPFVhkujhPVTr/nfYRulw2jSchtWciOsDEVNwlsH+jLSWWnFELrn3gapfNepkjcn2y3mimX5UK/HfKn+AT1V5MXx3BeYq5hrrbMSbdFKmSDGNcgrT0HobxwO/avBZyed6hbPPin+LvVlaJK7rwSzk6lKkI2ON0jaEwbByphS/wZzQjz7eEZne7KCjv+FEU71nxJpSPJ+mdHYfoG4xJDnEEOy57HC9xYGblU8qKM1XVrDTMYdZL+HkZKuzkdKCVNz4whOhf8GaeH+RAeI6eEIYZm2Z/SiBAcn3BBxpfqslJLQsPTMLbZ1ZBTwliDBtBI0TA91CJ17TUCPx3yWecidpw3bFKgrobR8qNDK7Laz0uYv91rZENBXNikOv9/Cx8quSo1cyuGDlfisgXhs7X8Hoa6nkiPAQaHEc72YjH2X4ZM50ywA0t/NFBgke8porAC9bTCRY8YCZIRbs4JPttKHLk+RQnyQiUHlaxsCfK+bW2pJza7xAPOzv7Yg24cDFfjYUethjgIEMPi+uMxm3O30NKAcshzQAkL1bQY3TGAFmoRXC4Ndgs8Fi4oxmb3g0QVvz1FuHnOHHrD2G0M4UXMmNS255FALDz X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(1800799024)(7416014)(36860700013)(82310400026)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Oct 2024 15:24:34.7211 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 831e4da0-c848-4740-6d30-08dced2d799a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN3PEPF0000B36D.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8864 From: Yonatan Maman ATS (Address Translation Services) mainly utilized to optimize PCI Peer-to-Peer transfers and prevent bus failures. This change employed ATS usage for ODP memory, to optimize DMA P2P for ODP memory. (e.g DMA P2P for private device pages - ODP memory). Signed-off-by: Yonatan Maman Reviewed-by: Gal Shalom --- drivers/infiniband/hw/mlx5/mlx5_ib.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h index 23fd72f7f63d..55ccbc7d9aa3 100644 --- a/drivers/infiniband/hw/mlx5/mlx5_ib.h +++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h @@ -1701,9 +1701,9 @@ static inline bool rt_supported(int ts_cap) static inline bool mlx5_umem_needs_ats(struct mlx5_ib_dev *dev, struct ib_umem *umem, int access_flags) { - if (!MLX5_CAP_GEN(dev->mdev, ats) || !umem->is_dmabuf) - return false; - return access_flags & IB_ACCESS_RELAXED_ORDERING; + if (MLX5_CAP_GEN(dev->mdev, ats) && (umem->is_dmabuf || umem->is_odp)) + return access_flags & IB_ACCESS_RELAXED_ORDERING; + return false; } int set_roce_addr(struct mlx5_ib_dev *dev, u32 port_num,