From patchwork Mon Jan 9 19:04:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 13094119 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 886BDC5479D for ; Mon, 9 Jan 2023 19:04:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234827AbjAITEX (ORCPT ); Mon, 9 Jan 2023 14:04:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45760 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237535AbjAITEK (ORCPT ); Mon, 9 Jan 2023 14:04:10 -0500 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2118.outbound.protection.outlook.com [40.107.102.118]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A5CC138AF6 for ; Mon, 9 Jan 2023 11:04:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=X2zxdYe6BkU4hdzjdXhqCKxF3dUvf8rPisjKHjb3cp1MOfbRel+i0vDi0rWVBnMQUBhtXOCNdHVg2hcKvrjrHQqbZGdMvd0x/9+kFYSxeb2gEHZdDYnyYVeLUzGdTGu0jMyuWVjSGj1S48TybCXnlLTnauAlI2FgFkd7oswhfuj3w++UJWGnvyId0f9hIMElxBGGRqiQu+uSsrHO+vT2RcmQa2Cq3GnSADu1IN1DfifjndlyDvLjNs9lemPe4uAyAd24txfSHBZ62l8Ku5OTbWSp7akX3qCyMZxbvq8igx8axuSAICeVDXFw+YeAPw/Gj+aD2c2FGiVrmn8B6xipwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=z2KykQ1QJG58t54DnoXH0Dsz3JHCsKv82SzoXNRmbXY=; b=JjxuCFSEpqe6RaeeNVVCR0OKg01zgOR8wTu10WHV5BYdV6ojxqcurFyg7s38VaTGTtgSkVkHOHiQbqOvp7x+5vX765Gta+p49L4NqrZD0etaBpKb0O++UIS65qljpt/8VUIWE2ugQvjGEDwwqdW9xZYWWh0Aq+slaup3UBDnIUo1Q5vKLIL3aVhRwUtN+ddAP8SDZFcGlFsEjhNM3Hdh3fhEAubRmK7kWc3+kHxe0bcj1LWJGSLTyTTS/SOcp3gv30vVCUlcx1+9wZathZVKA/FbOxYrZPS1tktGFnBx0W8hmj+g4S8+wcXYt16VwbnA5Dv+4b9035lMFbUwnEkJxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=nvidia.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=z2KykQ1QJG58t54DnoXH0Dsz3JHCsKv82SzoXNRmbXY=; b=BR85GSryw3o1zHSM8IKy4kEfbmoVoFObXIlExFGInfkFm0NILEQYn6oW9qVp9EttxYv0p26FFE8uABzeYHmnJZtdI4b1+sYrY6MRQE/ahCecSYHLdTpcZ96qtSj00jQXtIvvOeF6X1mKPpad9s/Ljqz6FPtZYmzaQcwpSWfvReVEp3qawGdKnpJDTt/vR+qTF031Tef0XnRjsaMkbmlInYjM5Fqex6EID1Iv367L6wqqhZlv1pxPvRf6hK/umGM6mITZHVJ+9+nIQel1J513KkORwqizuTcepE/1sLoR0gSyqkhULyxVTgvUCLAtT6WFhhAMxiWj61ghpa4Rf3QA1w== Received: from BN9P222CA0012.NAMP222.PROD.OUTLOOK.COM (2603:10b6:408:10c::17) by DM5PR0102MB3384.prod.exchangelabs.com (2603:10b6:4:a4::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18; Mon, 9 Jan 2023 19:04:04 +0000 Received: from BN8NAM11FT077.eop-nam11.prod.protection.outlook.com (2603:10b6:408:10c:cafe::fb) by BN9P222CA0012.outlook.office365.com (2603:10b6:408:10c::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18 via Frontend Transport; Mon, 9 Jan 2023 19:04:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; dkim=none (message not signed) header.d=none;dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-02.cornelisnetworks.com; pr=C Received: from awfm-02.cornelisnetworks.com (208.255.156.42) by BN8NAM11FT077.mail.protection.outlook.com (10.13.177.232) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5944.17 via Frontend Transport; Mon, 9 Jan 2023 19:04:04 +0000 Received: from awfm-02.cornelisnetworks.com (localhost [127.0.0.1]) by awfm-02.cornelisnetworks.com (8.16.1/8.16.1) with ESMTP id 309J43bn1477904; Mon, 9 Jan 2023 14:04:03 -0500 Subject: [PATCH for-next 1/7] IB/hfi1: Remove redundant pageidx variable From: Dennis Dalessandro To: jgg@nvidia.com, leonro@nvidia.com Cc: Dean Luick , linux-rdma@vger.kernel.org Date: Mon, 09 Jan 2023 14:04:03 -0500 Message-ID: <167329104365.1472990.14264918308557487946.stgit@awfm-02.cornelisnetworks.com> In-Reply-To: <167328561962.1472990.9463955313515395755.stgit@awfm-02.cornelisnetworks.com> References: <167328561962.1472990.9463955313515395755.stgit@awfm-02.cornelisnetworks.com> User-Agent: StGit/1.5.dev2+g9ce680a5 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT077:EE_|DM5PR0102MB3384:EE_ X-MS-Office365-Filtering-Correlation-Id: 1ea9f496-fd98-4fa9-ff23-08daf27446c3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: BSeT0byl2xgEC2o5ErcOR8Rn2v9iSk830JjJ449iaG8huEhG7/Ms8Pm/6wi6CUuGK9uGb8mY6QA/8gy0rb1E55NtA2yrcun1L5G6oaH4vPe4WgvsTcLy6i4zpkHjFXGGxFAE0CMBPb7d55OBo3hal2KVO4Pj3nbW7S7v3zkGNTFMoIacduc5DWvOc5wLet/2hXt/JMYtmkH7F0d9p8sJ3/GEBRFjOf1FXt2oa2tiTwpseN0oRd3AFzOUjnX8fkSbl4wJ21GouVXP7iWCZH5ClvIfvAgVY1yC0F6wugd+gC06o+UWp5vTGgvt4J7FhioS6r1LyuAXJfC41U7UBDMGvcceV9ewdI2+nzjwKKMkIs4rdAfSCdVtKfE74DZodW3VBHbSGgukLhJCEYh7gqFRlhPvNg3iF1vUrihPQe+4LAfhyIG/Ord2FB9XhD1s2n/KHPTM/uebBPTjgQilSgbKaMclHRBynEiRX7Rskh6jdaVpe0/jwlUpbwV+h4kCqX5TaCyWSXydKdNhtNx5qxlhKwfm05xyzOV1KFUPwmL36lyEpeI0MQqKdndJ1P07QcJNa+mn9htTFEmbdVSd+qlxmwj9mCCQ7IIZN5UsF3md5W97oEB4MvRFDNMon7/MEKh6S3xCqflAb8lGq5ysoCwYuoIMuVvEcsPA8bneUKFWUwNudc8FkZGJ2HR8vBusoTZvpVe5duZAh84cS8WRu47FIA== X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-02.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(136003)(376002)(39840400004)(396003)(346002)(451199015)(36840700001)(46966006)(103116003)(316002)(5660300002)(44832011)(40480700001)(26005)(7696005)(186003)(478600001)(7126003)(426003)(47076005)(41300700001)(336012)(4326008)(70206006)(70586007)(8676002)(82310400005)(8936002)(83380400001)(86362001)(55016003)(36860700001)(356005)(2906002)(81166007)(36900700001);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jan 2023 19:04:04.3412 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1ea9f496-fd98-4fa9-ff23-08daf27446c3 X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-02.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT077.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR0102MB3384 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Dean Luick In hfi1_user_exp_rcv_setup(), variable pageidx mirrors variable tididx. Remove pageidx and its use as an argument to program_rcvarray(). Signed-off-by: Dean Luick Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/user_exp_rcv.c | 22 +++++++++------------- 1 file changed, 9 insertions(+), 13 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/user_exp_rcv.c b/drivers/infiniband/hw/hfi1/user_exp_rcv.c index b02f2f0809c8..c2099510817c 100644 --- a/drivers/infiniband/hw/hfi1/user_exp_rcv.c +++ b/drivers/infiniband/hw/hfi1/user_exp_rcv.c @@ -27,8 +27,7 @@ static bool tid_cover_invalidate(struct mmu_interval_notifier *mni, const struct mmu_notifier_range *range, unsigned long cur_seq); static int program_rcvarray(struct hfi1_filedata *fd, struct tid_user_buf *, - struct tid_group *grp, - unsigned int start, u16 count, + struct tid_group *grp, u16 count, u32 *tidlist, unsigned int *tididx, unsigned int *pmapped); static int unprogram_rcvarray(struct hfi1_filedata *fd, u32 tidinfo); @@ -256,7 +255,7 @@ int hfi1_user_exp_rcv_setup(struct hfi1_filedata *fd, int ret = 0, need_group = 0, pinned; struct hfi1_ctxtdata *uctxt = fd->uctxt; struct hfi1_devdata *dd = uctxt->dd; - unsigned int ngroups, pageidx = 0, pageset_count, + unsigned int ngroups, pageset_count, tididx = 0, mapped, mapped_pages = 0; u32 *tidlist = NULL; struct tid_user_buf *tidbuf; @@ -337,7 +336,7 @@ int hfi1_user_exp_rcv_setup(struct hfi1_filedata *fd, tid_group_pop(&uctxt->tid_group_list); ret = program_rcvarray(fd, tidbuf, grp, - pageidx, dd->rcv_entries.group_size, + dd->rcv_entries.group_size, tidlist, &tididx, &mapped); /* * If there was a failure to program the RcvArray @@ -353,11 +352,10 @@ int hfi1_user_exp_rcv_setup(struct hfi1_filedata *fd, tid_group_add_tail(grp, &uctxt->tid_full_list); ngroups--; - pageidx += ret; mapped_pages += mapped; } - while (pageidx < pageset_count) { + while (tididx < pageset_count) { struct tid_group *grp, *ptr; /* * If we don't have any partially used tid groups, check @@ -379,11 +377,11 @@ int hfi1_user_exp_rcv_setup(struct hfi1_filedata *fd, */ list_for_each_entry_safe(grp, ptr, &uctxt->tid_used_list.list, list) { - unsigned use = min_t(unsigned, pageset_count - pageidx, + unsigned use = min_t(unsigned, pageset_count - tididx, grp->size - grp->used); ret = program_rcvarray(fd, tidbuf, grp, - pageidx, use, tidlist, + use, tidlist, &tididx, &mapped); if (ret < 0) { hfi1_cdbg(TID, @@ -395,11 +393,10 @@ int hfi1_user_exp_rcv_setup(struct hfi1_filedata *fd, tid_group_move(grp, &uctxt->tid_used_list, &uctxt->tid_full_list); - pageidx += ret; mapped_pages += mapped; need_group = 0; /* Check if we are done so we break out early */ - if (pageidx >= pageset_count) + if (tididx >= pageset_count) break; } else if (WARN_ON(ret == 0)) { /* @@ -643,7 +640,6 @@ static u32 find_phys_blocks(struct tid_user_buf *tidbuf, unsigned int npages) * struct tid_pageset holding information on physically contiguous * chunks from the user buffer), and other fields. * @grp: RcvArray group - * @start: starting index into sets array * @count: number of struct tid_pageset's to program * @tidlist: the array of u32 elements when the information about the * programmed RcvArray entries is to be encoded. @@ -663,14 +659,14 @@ static u32 find_phys_blocks(struct tid_user_buf *tidbuf, unsigned int npages) * number of RcvArray entries programmed. */ static int program_rcvarray(struct hfi1_filedata *fd, struct tid_user_buf *tbuf, - struct tid_group *grp, - unsigned int start, u16 count, + struct tid_group *grp, u16 count, u32 *tidlist, unsigned int *tididx, unsigned int *pmapped) { struct hfi1_ctxtdata *uctxt = fd->uctxt; struct hfi1_devdata *dd = uctxt->dd; u16 idx; + unsigned int start = *tididx; u32 tidinfo = 0, rcventry, useidx = 0; int mapped = 0; From patchwork Mon Jan 9 19:04:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 13094120 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9298BC54EBD for ; Mon, 9 Jan 2023 19:04:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234778AbjAITEY (ORCPT ); Mon, 9 Jan 2023 14:04:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45792 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237497AbjAITEN (ORCPT ); Mon, 9 Jan 2023 14:04:13 -0500 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2120.outbound.protection.outlook.com [40.107.243.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 130D138AFB for ; Mon, 9 Jan 2023 11:04:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=btWlHcyU/N4Y4aDRUmbqQIRVMm4LqzlvqIpLVJjHNpJg6OsU6rVopINN+a1B9Kxn65JKddpQr0js3I3kh7OxdhA5arR0V48iGvd3vlT3HP2F5H8HzPDV3qzThHSQRe5NY2ED6+l1oEi6AX2eqfcQONxN9o+g2Ae+IZnCnnuCAFJ1srzpNCqgwi9gDuQQZCdMG7+FirnmgLQQqPEOsOgUfckjfbmhEojqswJ3USGV746Kr+zVzu0Ejh+EysBm4qKNVffmmHCwfPvUmkK6i0wB8f0n4/AesXIjXWNqb3TxKkcjibG3EI96ujwlyEjbo/czJO2T1hb1tBJ9dRlH5srvqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7iKaT6tFy8MyfXTqFBq7FdLI/3o4UJeqe6+x02Qjsi4=; b=Ad34XPHGXUWdnlXvjPMhy425/nSaDHkZVJpS0kSRHMTp9LJxSv6q/XyqxRM2S8K62UZ2vq6Uifvx7T9aDvC3QUw+Rk/5dIeY+hk704piAHgvzAo1tLO3u+hgd2+Nifvc2637WtaNQCn9Xf/6krXMrkdMA//YLYTfh4sRupquEMzBmlE4mgArmAcnNZ5xsgFnrIvASbLIdnplrNAjCbieQApP8MDIO34UZmmedBOyDRKJgGginmmN4UO9xmCkIWnsIG4Npo4Cq31r1KP2+ovvwuocVkW1xWjTlW34IXth5wYrOSPcHZZ+APrd6VNYFzas4ttpHJ3zMFEn4Xob7jHgIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=nvidia.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7iKaT6tFy8MyfXTqFBq7FdLI/3o4UJeqe6+x02Qjsi4=; b=c2ASNwt9x2BxHkdoI4O0jmNvkQ+LXHK7xgIJxgc/L/Ev9oKc1zdAwTBr94HmauY3mI4vl0Fn612L7PfdAE9WL8nttrhSjlUNNy5b0AnLTw1lzxpgMLhtNuPTRmq5qAzkxHZUyJnnFTQWkgxibqtTIY2cEb/uABh3vzMW9r/kvIJ25+gJ/h0sxIrFQEC6qxJNRXSPnyFHSv0DD3b0urnybyLVbXf1yIQ4lf86EA6sQ9BhYiC9gN9yOGd0NmRcQlv+ZihFvl7kWxq6bu9UkYBp9TfaNfGXwEg2OIj5VdSz2BzCPsF3Y5BopVuWQGSUf8mxlhQdi4jRve72b+19PdL6eA== Received: from BN0PR07CA0030.namprd07.prod.outlook.com (2603:10b6:408:141::13) by SN6PR01MB4973.prod.exchangelabs.com (2603:10b6:805:c4::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18; Mon, 9 Jan 2023 19:04:09 +0000 Received: from BN8NAM11FT089.eop-nam11.prod.protection.outlook.com (2603:10b6:408:141:cafe::1c) by BN0PR07CA0030.outlook.office365.com (2603:10b6:408:141::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18 via Frontend Transport; Mon, 9 Jan 2023 19:04:09 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; dkim=none (message not signed) header.d=none;dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-02.cornelisnetworks.com; pr=C Received: from awfm-02.cornelisnetworks.com (208.255.156.42) by BN8NAM11FT089.mail.protection.outlook.com (10.13.176.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18 via Frontend Transport; Mon, 9 Jan 2023 19:04:09 +0000 Received: from awfm-02.cornelisnetworks.com (localhost [127.0.0.1]) by awfm-02.cornelisnetworks.com (8.16.1/8.16.1) with ESMTP id 309J48dL1477912; Mon, 9 Jan 2023 14:04:08 -0500 Subject: [PATCH for-next 2/7] IB/hfi1: Assign npages earlier From: Dennis Dalessandro To: jgg@nvidia.com, leonro@nvidia.com Cc: Dean Luick , linux-rdma@vger.kernel.org Date: Mon, 09 Jan 2023 14:04:08 -0500 Message-ID: <167329104884.1472990.4639750192433251493.stgit@awfm-02.cornelisnetworks.com> In-Reply-To: <167328561962.1472990.9463955313515395755.stgit@awfm-02.cornelisnetworks.com> References: <167328561962.1472990.9463955313515395755.stgit@awfm-02.cornelisnetworks.com> User-Agent: StGit/1.5.dev2+g9ce680a5 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT089:EE_|SN6PR01MB4973:EE_ X-MS-Office365-Filtering-Correlation-Id: d7609788-666d-4017-852d-08daf27449e1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wOs14EnKBlfJTyeA2mDTBInJ8pDKsCf39T18FZ4BS0taspRjPTWvfz9duOdLdDd1dRbMw4gSnzt5e1cv4EJ0WIYAZRxm/eNASST2YGgVISj+ZEICRAMEXBHMcANAW0j3FBAglZbWDzaEAHybPi7x0XthfgfgOXRIA52lGJYwI97fHNCb3pE3F3D0BsHyv/Nil19QJrDGRwZ1mYYfCpyApAgQWG+DcPRnIbjW9W+9kRYgPAchX0R6HCVf4REzrV40mZCmKhoNpCgpuj7kMmLrWGi6CAmlFi5U1WXCagfaCTcqltovi0GF2Z9zI6T2AZMUkyrdr7YGO1c8y3o33F6iBJhIDs51j/4/xetdYQUfl2xZNZtB44VY+9en9O12X0P5F1RaFOpiwpsHfC3jDlt5TVxxaoNMTFcAsdaPJjyLbmvUJy3FVDoEw7prvVar/21ZKRQ48g9IMuL2stxIEPiIlqSD/Oyg/rFAciPb93I/JoRWOqwg4rbfkzHthrycadmL41krJHrTY/Qjg8KRzq9+DcFQNTKxQZqn2R6B+tMG6TBGxw55CbuWXxs9RVbW0RrOrpEZBA4KmzezvSu99Z51uPaXpqjOUSJAnGeLeEJbdKDbnzQj3x3BohEjpuw5FUNV/YfoQ2BzEQlwLe1hie2ZssF+cG2Dg/cegUaGmhxS/Nb8lzdsobmkNQtvnQiiyUJBnhIG+uMrwZ8uCk1Y5o3rPA== X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-02.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(346002)(376002)(396003)(136003)(39840400004)(451199015)(36840700001)(46966006)(83380400001)(2906002)(44832011)(82310400005)(47076005)(336012)(36860700001)(426003)(81166007)(7126003)(5660300002)(55016003)(7696005)(40480700001)(26005)(186003)(8936002)(478600001)(70586007)(70206006)(8676002)(103116003)(41300700001)(86362001)(356005)(4326008)(316002)(36900700001);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jan 2023 19:04:09.5522 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d7609788-666d-4017-852d-08daf27449e1 X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-02.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT089.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR01MB4973 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Dean Luick Improve code clarity and enable earlier use of tidbuf->npages by moving its assignment to structure creation time. Signed-off-by: Dean Luick Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/user_exp_rcv.c | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/user_exp_rcv.c b/drivers/infiniband/hw/hfi1/user_exp_rcv.c index c2099510817c..0e51c95d3f1d 100644 --- a/drivers/infiniband/hw/hfi1/user_exp_rcv.c +++ b/drivers/infiniband/hw/hfi1/user_exp_rcv.c @@ -159,16 +159,11 @@ static void unpin_rcv_pages(struct hfi1_filedata *fd, static int pin_rcv_pages(struct hfi1_filedata *fd, struct tid_user_buf *tidbuf) { int pinned; - unsigned int npages; + unsigned int npages = tidbuf->npages; unsigned long vaddr = tidbuf->vaddr; struct page **pages = NULL; struct hfi1_devdata *dd = fd->uctxt->dd; - /* Get the number of pages the user buffer spans */ - npages = num_user_pages(vaddr, tidbuf->length); - if (!npages) - return -EINVAL; - if (npages > fd->uctxt->expected_count) { dd_dev_err(dd, "Expected buffer too big\n"); return -EINVAL; @@ -195,7 +190,6 @@ static int pin_rcv_pages(struct hfi1_filedata *fd, struct tid_user_buf *tidbuf) return pinned; } tidbuf->pages = pages; - tidbuf->npages = npages; fd->tid_n_pinned += pinned; return pinned; } @@ -273,6 +267,7 @@ int hfi1_user_exp_rcv_setup(struct hfi1_filedata *fd, mutex_init(&tidbuf->cover_mutex); tidbuf->vaddr = tinfo->vaddr; tidbuf->length = tinfo->length; + tidbuf->npages = num_user_pages(tidbuf->vaddr, tidbuf->length); tidbuf->psets = kcalloc(uctxt->expected_count, sizeof(*tidbuf->psets), GFP_KERNEL); if (!tidbuf->psets) { From patchwork Mon Jan 9 19:04:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 13094121 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BA11C61DB3 for ; Mon, 9 Jan 2023 19:04:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237128AbjAITEY (ORCPT ); Mon, 9 Jan 2023 14:04:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45832 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237271AbjAITET (ORCPT ); Mon, 9 Jan 2023 14:04:19 -0500 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2099.outbound.protection.outlook.com [40.107.220.99]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B06C03750A for ; Mon, 9 Jan 2023 11:04:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VbcqOOmLLl5XnooB2gOGtFOk5UyZZOK8brN4KWR6fqKHHiI1e0kBBkM3hbBjLTtlIzMsM/WkkoRx7+/xE2sqvK5uYEM4ljQg4ZYg0nRSoHvV8gsHVamL328WA04HTJTHXTjkYmMuhrbBoSxPr36MT/tq1YaQRc3yCYWWEjvE+HyQCDVLAAY488rlYTIGy9GvV6kpKwvZAmXPRofJN4pZlOepbri0cwpQAO2H55fS8pvdJzjaHogJQozL3eZMZYORMDIAjMx9pdlBmj2dcQXepSgyBZQe4hIDhR4rkFfY1ThIu9wl2dEizwr5MU6l5L90yuK+zkDeK3QZRis/RLJUlA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5rq8hUMv66znCK9moyrQos/tUEw1InqtcthJW316MBY=; b=k02rF2dYrUopfadHpmX7NboY2UEo9917zFNsalVRIs1opa4mS8y5FbhDJgc4QHiPUhOMt0Qbr5T33KQoLmfItyvPOod17tIMs4NXS+P2SJS+gDEV1kxQlzusozWXE9EVWlNh8S1m76rwI3ND+WmeB/ldDi5MYutPrDepnjXmfAbqSuy02dWJ0Apf2XPutVj8JRyBtcdXe+JB+f8l/LN27DfmUamYfsNnudROC6ike8HlL548WoKFdfntElVuY2Vjppce1F/rUn1HEEuDKi50aGtnczVL9fqYKb8Yomhflo3KrtXX0HUJ6zzflDXpJKpfhrZaW9pHxqGkVj4D6+ZKIg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=nvidia.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5rq8hUMv66znCK9moyrQos/tUEw1InqtcthJW316MBY=; b=G8nIr2HmNvhwGHBAve2x9yr3bwiihIvU0/Q3srfpqomh1qD3FTjfFklKZD+Al9gBAtod5p1vOmrVAf0KFa/1oTIFj4b8B3DDlgMFrMTvIk1U+VUol9veIJPHtsN3IGsSYsCX4pgNwTD3iUScsc+qG/kdkAYTwzzMuC7bpeYgGz7Pt2dtqSmWwGP48D/CbvbjxKzRstB6DsdGtSpGtVaaRHfnpIMNGoDnf6cDkGxhK4sWXhf08Bc81H4z0vEXE0P4nkXrXqlkHAKyA37xryHmZ3JpD7DTOd8xmCiwIqH3LHXwlLe4h/trn8oRg93a2nF5nuA+YSxE7zGanZ0IJiM1ag== Received: from BN9PR03CA0949.namprd03.prod.outlook.com (2603:10b6:408:108::24) by DM6PR01MB4953.prod.exchangelabs.com (2603:10b6:5:8::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18; Mon, 9 Jan 2023 19:04:15 +0000 Received: from BN8NAM11FT024.eop-nam11.prod.protection.outlook.com (2603:10b6:408:108:cafe::b) by BN9PR03CA0949.outlook.office365.com (2603:10b6:408:108::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18 via Frontend Transport; Mon, 9 Jan 2023 19:04:15 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; dkim=none (message not signed) header.d=none;dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-02.cornelisnetworks.com; pr=C Received: from awfm-02.cornelisnetworks.com (208.255.156.42) by BN8NAM11FT024.mail.protection.outlook.com (10.13.177.38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18 via Frontend Transport; Mon, 9 Jan 2023 19:04:14 +0000 Received: from awfm-02.cornelisnetworks.com (localhost [127.0.0.1]) by awfm-02.cornelisnetworks.com (8.16.1/8.16.1) with ESMTP id 309J4ER71477920; Mon, 9 Jan 2023 14:04:14 -0500 Subject: [PATCH for-next 3/7] IB/hfi1: Consolidate the creation of user TIDs From: Dennis Dalessandro To: jgg@nvidia.com, leonro@nvidia.com Cc: Dean Luick , linux-rdma@vger.kernel.org Date: Mon, 09 Jan 2023 14:04:14 -0500 Message-ID: <167329105402.1472990.9685946655723333660.stgit@awfm-02.cornelisnetworks.com> In-Reply-To: <167328561962.1472990.9463955313515395755.stgit@awfm-02.cornelisnetworks.com> References: <167328561962.1472990.9463955313515395755.stgit@awfm-02.cornelisnetworks.com> User-Agent: StGit/1.5.dev2+g9ce680a5 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT024:EE_|DM6PR01MB4953:EE_ X-MS-Office365-Filtering-Correlation-Id: 2642d7d5-18ac-4cd2-6184-08daf2744cfe X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: aWacF+dLxxrhJBWAh+OoOrDt0P0F6RjYDY5Ggcu6z4R6G3W8n6aZgqjj7oLzE5CISqHcvZyQxFfg0+4povPg1FPyQQjIvMEe7YKC4MXUr1xVRrj0UgdvczLS94EzpY8ecY/ZZfkj4ZAEVma7pI5DCNmbYFuvsQ5cR00aGUPo5qL6Z/buBjdFHLDpBBQy/fNe5ldgAKp8SxVHAynIrPGRFloXTGn9u8sS76xp97rjehMS9HcmwSFp6jFDepQMimiKjdbRvPGzxK3X2hSejm8ZI9MSWNnV3bHk/SdMh30Ib6TBoJpc3XdEGJVoM4mvzr5uBU1megIMk6UimQxT/5G1RNhzU/Em/hiUpUDoLDCqXvSrBwPDXLTIHUJMmlmU50xjYUPedSaBg+euJny/JXPYeUrQwYfR0SbvSEpYLJqPYVOv8qiiQF1E5Cb/Ccs+TfdqfKT73IbOwgC6xol2mdQiBQJh42yRENrArqkpriPR9JrPvUO+dd+tRC2i6nI+y1mv/7s4fo8i9uweiOqU6tVeIR/0AfTh/Fn973S7su04zEA4ANnSVv24dEVtYFK1OWux7JfoQqE9Qsyl1sYu+GbziOALTV/9crIf+AJcIjEd173yX6mubHz1F9T093XenrNfMV/Ag42PhJ1YzYIqGPM38JeVGo3RGtMf1MPMy4jhrdqsB/PXsTvl/IebsMyd62d2KDG4eR/081pO7JEhOf8QSg== X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-02.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(136003)(376002)(346002)(396003)(39840400004)(451199015)(46966006)(36840700001)(82310400005)(8936002)(2906002)(5660300002)(47076005)(426003)(41300700001)(81166007)(7696005)(70206006)(4326008)(70586007)(8676002)(316002)(356005)(44832011)(186003)(26005)(40480700001)(7126003)(336012)(55016003)(86362001)(83380400001)(36860700001)(103116003)(478600001)(36900700001);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jan 2023 19:04:14.7901 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2642d7d5-18ac-4cd2-6184-08daf2744cfe X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-02.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT024.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR01MB4953 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Dean Luick The function rcventry2tidinfo() only creates part of a TID and all calls to it are only used to make a user TID. Consolidate all usage into a single routine. Signed-off-by: Dean Luick Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/exp_rcv.h | 5 +++-- drivers/infiniband/hw/hfi1/user_exp_rcv.c | 8 +++----- 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/exp_rcv.h b/drivers/infiniband/hw/hfi1/exp_rcv.h index c6291bbf723c..41f7fe5d1839 100644 --- a/drivers/infiniband/hw/hfi1/exp_rcv.h +++ b/drivers/infiniband/hw/hfi1/exp_rcv.h @@ -133,12 +133,13 @@ static inline struct tid_group *tid_group_pop(struct exp_tid_set *set) return grp; } -static inline u32 rcventry2tidinfo(u32 rcventry) +static inline u32 create_tid(u32 rcventry, u32 npages) { u32 pair = rcventry & ~0x1; return EXP_TID_SET(IDX, pair >> 1) | - EXP_TID_SET(CTRL, 1 << (rcventry - pair)); + EXP_TID_SET(CTRL, 1 << (rcventry - pair)) | + EXP_TID_SET(LEN, npages); } /** diff --git a/drivers/infiniband/hw/hfi1/user_exp_rcv.c b/drivers/infiniband/hw/hfi1/user_exp_rcv.c index 0e51c95d3f1d..0229cf210431 100644 --- a/drivers/infiniband/hw/hfi1/user_exp_rcv.c +++ b/drivers/infiniband/hw/hfi1/user_exp_rcv.c @@ -706,8 +706,7 @@ static int program_rcvarray(struct hfi1_filedata *fd, struct tid_user_buf *tbuf, return ret; mapped += npages; - tidinfo = rcventry2tidinfo(rcventry - uctxt->expected_base) | - EXP_TID_SET(LEN, npages); + tidinfo = create_tid(rcventry - uctxt->expected_base, npages); tidlist[(*tididx)++] = tidinfo; grp->used++; grp->map |= 1 << useidx++; @@ -916,9 +915,8 @@ static bool tid_rb_invalidate(struct mmu_interval_notifier *mni, spin_lock(&fdata->invalid_lock); if (fdata->invalid_tid_idx < uctxt->expected_count) { fdata->invalid_tids[fdata->invalid_tid_idx] = - rcventry2tidinfo(node->rcventry - uctxt->expected_base); - fdata->invalid_tids[fdata->invalid_tid_idx] |= - EXP_TID_SET(LEN, node->npages); + create_tid(node->rcventry - uctxt->expected_base, + node->npages); if (!fdata->invalid_tid_idx) { unsigned long *ev; From patchwork Mon Jan 9 19:04:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 13094122 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99E05C54EBD for ; Mon, 9 Jan 2023 19:04:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234322AbjAITE2 (ORCPT ); Mon, 9 Jan 2023 14:04:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45864 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237323AbjAITE0 (ORCPT ); Mon, 9 Jan 2023 14:04:26 -0500 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2099.outbound.protection.outlook.com [40.107.220.99]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0A72D3750A for ; Mon, 9 Jan 2023 11:04:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WJ2SpCvf1/TEMDsfqio9yWB2kICh69UWDoa9menCFW75ckUShNuhF0AcD+Ff8GEHHVtRTsHQo/6j2aEA/jrQ1YAyBDPhkndqeeMKIRG6mlx1IQuwvc7lZm8jrcutzBqO164jfLryuhL4gJYGpXJr8P/yCBGu7J0zC82bKjdPfXbBAeBkp/M+qsaUleO+AGyDzkDaEuDBLl0UAUof/qn4YA/T4i1N2HVAZhit0I81O7871mfqYTCXq14AJ0FuOajOLdt3Syg+1BWBjZufDP5DAxnJ6yWDJ/xYbfAok9gNyCf+tjTXgeYFPH7FpPBaUgGfTGU9j+dtvEsE3TbyFf2EbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1OMom0q0yxg4zGECCkDunAWRLuE1eQ5avHJxkd+qaH4=; b=FQ2SP7d7E4o2bgGrE/KC2JS2QK/GjSgfrFwKiOg4sIhTkJ7FFO1ElrF6wk4QydzZjAlXAlzLKnz345W7Xs+wX4gqjui5ijJVnGYfi9/s+A+VL15k31t17t8OE2it1F6T6nVsp+5T8I1DUQO1j8R130OAe2Zf3zoMxxX00HmYo0FrKbldTWsE2WmSBtubX5s+O6Tg6oGXQ7LucbPzeDpV4Lec/CyOXbsdXa5jYfEm4ZRmU019+bEsg3bf+jGHq9SqmNBE72eAoM3CW6MQmtC3AnHY+EgUEzcuhqeSiKlaf+rn+530HzCtJIGiespdA/XYNCn2BV8bc1toxfPhsoGv5g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=nvidia.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1OMom0q0yxg4zGECCkDunAWRLuE1eQ5avHJxkd+qaH4=; b=nEolYV6pUhocfEUSWm6BSPEOFTzjJefLl0B0mRqmxZx3yuO0bFhXhBhy0KlfeluajoDJxLbc+ict1axMRdEX+gW0j/WdcFhnheLbXgoqpSkCIdP2gXOk02dngFO6eeN6ALJV31J0xvnWB0+ZjsCUkSBHBQeCTz4MbcdYDUC2VLwhi1NJ3UsAceSExIUJ/Ckc8g+btuQ2AsoIzGkqMoDxv/T03bYs+cZrmUUA0iFvm59ffd31wxGtCBZum2wuBwmRlF4P2YKlqn/7i+JFJyB7FzObnz9tCKV9NCrMHcKGxONGbAcBo6Y6uiZDt1EJZGL82mvftkpNbMRv2w/RoHmnMQ== Received: from BN9PR03CA0749.namprd03.prod.outlook.com (2603:10b6:408:110::34) by BN6PR0101MB2980.prod.exchangelabs.com (2603:10b6:405:31::36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18; Mon, 9 Jan 2023 19:04:20 +0000 Received: from BN8NAM11FT097.eop-nam11.prod.protection.outlook.com (2603:10b6:408:110:cafe::a1) by BN9PR03CA0749.outlook.office365.com (2603:10b6:408:110::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18 via Frontend Transport; Mon, 9 Jan 2023 19:04:20 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; dkim=none (message not signed) header.d=none;dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-02.cornelisnetworks.com; pr=C Received: from awfm-02.cornelisnetworks.com (208.255.156.42) by BN8NAM11FT097.mail.protection.outlook.com (10.13.176.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18 via Frontend Transport; Mon, 9 Jan 2023 19:04:19 +0000 Received: from awfm-02.cornelisnetworks.com (localhost [127.0.0.1]) by awfm-02.cornelisnetworks.com (8.16.1/8.16.1) with ESMTP id 309J4JYP1477928; Mon, 9 Jan 2023 14:04:19 -0500 Subject: [PATCH for-next 4/7] IB/hfi1: Improve TID validity checking From: Dennis Dalessandro To: jgg@nvidia.com, leonro@nvidia.com Cc: Dean Luick , linux-rdma@vger.kernel.org Date: Mon, 09 Jan 2023 14:04:19 -0500 Message-ID: <167329105916.1472990.9915542468337924727.stgit@awfm-02.cornelisnetworks.com> In-Reply-To: <167328561962.1472990.9463955313515395755.stgit@awfm-02.cornelisnetworks.com> References: <167328561962.1472990.9463955313515395755.stgit@awfm-02.cornelisnetworks.com> User-Agent: StGit/1.5.dev2+g9ce680a5 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT097:EE_|BN6PR0101MB2980:EE_ X-MS-Office365-Filtering-Correlation-Id: 2748ef14-c371-43fb-15a9-08daf2745005 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0VTydzsTMskiQbGn/Moj5bCv6mshYlsHUB38vWbC85/LvmGT7xU85lqpiKIaRDJXSOSAMoca4zlIRkAAOr+wnvgQU8L/A538EfPcQ7GFaCcFuf/g4/LGwZGMMj+VAzjYKc0ErDdo2MyvX6KVaNLuCIHgoXAmusW93NPgTr3AcWC2ePMn9FQHaoN5dbuIc2Nh3b+wmln991+c5s//VuKW4V/20b8zXpJKvsvjJkZSAQNmn0mkU/bDq3tdtO6avlGm1P6/rLdaQrub/i6/Ns9GtXSVEh///kzhcYDFow28J+zLWcoP0N5HBn3TjFSsxYXtOgBJI/Iw/4A39x4VODpB+vHEtbG+q+efiS8+M9MVB9bHKKV7mSGfZ8oubWRgzCzwJ3LMC5bVkdob2xigAKkYSMuHQfZTk4udKS0MezTdcM+W1ONBlZD0vmDrw7NNLGmBIXDKChYBbuv/oTPHF33Wa4Ze78sm97dcEJurJTpHo0ASmXmIht83yNkmMZU/JblHgaXrZy8WJR46OoZZMhweoJ93vFB3DX/knJetYSXAB2or0Y//2ps5tLb8BQtd1nEztBd6JX/4xOr806/wvKzdy3LioOxu5UywkI++gtlDzgbfkv39HppsVzf809lYT2qMRLIRNBIQpmtmZ1D7OSZ0NhLmDUVCNHVhdEuyTh6ASe9MjrN5x6CxuvDE1M4yskaaKD0MaQsCvDhaI2DmnPWddw== X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-02.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(346002)(39840400004)(376002)(136003)(396003)(451199015)(46966006)(36840700001)(41300700001)(26005)(103116003)(44832011)(8936002)(5660300002)(36860700001)(186003)(478600001)(7126003)(336012)(83380400001)(47076005)(426003)(82310400005)(316002)(8676002)(70206006)(4326008)(86362001)(356005)(40480700001)(55016003)(70586007)(7696005)(81166007)(2906002)(36900700001);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jan 2023 19:04:19.8516 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2748ef14-c371-43fb-15a9-08daf2745005 X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-02.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT097.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR0101MB2980 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Dean Luick Correct and improve validity checking of user supplied TIDs. A tidctrl value of 0 is invalid. Verify that the final index is in range, not an intermediate value. Signed-off-by: Dean Luick Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/user_exp_rcv.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/user_exp_rcv.c b/drivers/infiniband/hw/hfi1/user_exp_rcv.c index 0229cf210431..96058baf36ed 100644 --- a/drivers/infiniband/hw/hfi1/user_exp_rcv.c +++ b/drivers/infiniband/hw/hfi1/user_exp_rcv.c @@ -790,20 +790,20 @@ static int unprogram_rcvarray(struct hfi1_filedata *fd, u32 tidinfo) struct hfi1_ctxtdata *uctxt = fd->uctxt; struct hfi1_devdata *dd = uctxt->dd; struct tid_rb_node *node; - u8 tidctrl = EXP_TID_GET(tidinfo, CTRL); + u32 tidctrl = EXP_TID_GET(tidinfo, CTRL); u32 tididx = EXP_TID_GET(tidinfo, IDX) << 1, rcventry; - if (tididx >= uctxt->expected_count) { - dd_dev_err(dd, "Invalid RcvArray entry (%u) index for ctxt %u\n", - tididx, uctxt->ctxt); - return -EINVAL; - } - - if (tidctrl == 0x3) + if (tidctrl == 0x3 || tidctrl == 0x0) return -EINVAL; rcventry = tididx + (tidctrl - 1); + if (rcventry >= uctxt->expected_count) { + dd_dev_err(dd, "Invalid RcvArray entry (%u) index for ctxt %u\n", + rcventry, uctxt->ctxt); + return -EINVAL; + } + node = fd->entry_to_rb[rcventry]; if (!node || node->rcventry != (uctxt->expected_base + rcventry)) return -EBADF; From patchwork Mon Jan 9 19:04:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 13094123 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DCB3C5479D for ; Mon, 9 Jan 2023 19:04:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237305AbjAITEa (ORCPT ); Mon, 9 Jan 2023 14:04:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45888 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237323AbjAITE3 (ORCPT ); Mon, 9 Jan 2023 14:04:29 -0500 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2104.outbound.protection.outlook.com [40.107.102.104]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 20E85392CA for ; Mon, 9 Jan 2023 11:04:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KAiGRw/ssHty/kmpcs1gQjgCUwbYermkyELMdaYv2X6WgKOUkqom4bK+VL3sRYm32l4GNHFBdZ48gu17NvnYuxstE60PrTKKs1z4pKs0vkbjlvPZbTp6yqdYZyTmfu++nqwJ0kE2flBQNJ+KEjpWcdCjjQbn2Y1WQqJ9njZhecJTMBObGiHx3eWteSMhaaBTmnQqPzJunhmVQZ5NbMMIIzJhgAfqSIjYfgbT4K7HeUPeZMKkNqyNePP9VS0vx0W1iPINWiXxrXx7M0QYY+R3xJXdatCNTbUMWHCmYk8iE0iz6gVNMU2y2oRfuzUelbuMKCbRnUN34e3H+sNXZH5vCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9LJwKoJLP0snXc8RW9WMN8sZHPQML36iDXo9+QCdilU=; b=QHO/e4qdkNyANPUvA03GJLxiMZXmwVn5FJ1Gj5YaJbjED/Aw2EokAq0xSjG0OGcMqHZFDIh/HWMppNf0hFgnXTix5qXYveSZcEUL9xQcqrDfx6wm1NgmoyB2mPPt6uLq58N0FQTTq6ji9iJZYGu4EQW8fgzAiZKFDXClSJgt6LtavMV/dPs+w8SD4MZgZISrDwSi28ZeSPkG+xTpqhKG6rRTeLN7C8bXE+HsWTyEfiBdv9W54f9pQuKiDJQgP9Tar4j4TkpufuUbocdfJreRZB37jpkrXs/IxwNhlop/agvcKClF9TMw7A2zLda5hqCKbC2gDkyKwq1/9jxjfVmTZA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=nvidia.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9LJwKoJLP0snXc8RW9WMN8sZHPQML36iDXo9+QCdilU=; b=ZgmCOUtG5nimvZncEnWL8HZ2GGnBBQiOZxGxTbjcutXDO3Iq+/PkL016soDkoUQC3vEXcuAYonGYpOyOrWPOvyvpmo5oQtEoEfGGv7ra3QBpfWngsk8Jmb5w7bTsMIir/i59bn2Ru6NipViYZpa14GSpjoJRhOeOjikZNX4Q1+LWxntAuq6AAd++BIBLnu2hyiiZblDaaqzYm/VLFUYsIkoeyGWFV4OeuNaXzfVcAi7LmY+0vZpNG4uUGdsFtEbaH1QoA4InCVxkAYAlfL7xXWXXGKOW2WVEIyFBd8ts0Byges2jdRTOavEmrlPOja9y7+4Pnks3xzFlMlth4AEnGg== Received: from MW4PR04CA0073.namprd04.prod.outlook.com (2603:10b6:303:6b::18) by PH0PR01MB7256.prod.exchangelabs.com (2603:10b6:510:105::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18; Mon, 9 Jan 2023 19:04:25 +0000 Received: from CO1NAM11FT102.eop-nam11.prod.protection.outlook.com (2603:10b6:303:6b:cafe::c0) by MW4PR04CA0073.outlook.office365.com (2603:10b6:303:6b::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18 via Frontend Transport; Mon, 9 Jan 2023 19:04:25 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; dkim=none (message not signed) header.d=none;dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-02.cornelisnetworks.com; pr=C Received: from awfm-02.cornelisnetworks.com (208.255.156.42) by CO1NAM11FT102.mail.protection.outlook.com (10.13.175.87) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18 via Frontend Transport; Mon, 9 Jan 2023 19:04:25 +0000 Received: from awfm-02.cornelisnetworks.com (localhost [127.0.0.1]) by awfm-02.cornelisnetworks.com (8.16.1/8.16.1) with ESMTP id 309J4OUL1477937; Mon, 9 Jan 2023 14:04:24 -0500 Subject: [PATCH for-next 5/7] IB/hfi1: Split IB counter allocation From: Dennis Dalessandro To: jgg@nvidia.com, leonro@nvidia.com Cc: Dean Luick , linux-rdma@vger.kernel.org Date: Mon, 09 Jan 2023 14:04:24 -0500 Message-ID: <167329106431.1472990.12587703493884915680.stgit@awfm-02.cornelisnetworks.com> In-Reply-To: <167328561962.1472990.9463955313515395755.stgit@awfm-02.cornelisnetworks.com> References: <167328561962.1472990.9463955313515395755.stgit@awfm-02.cornelisnetworks.com> User-Agent: StGit/1.5.dev2+g9ce680a5 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT102:EE_|PH0PR01MB7256:EE_ X-MS-Office365-Filtering-Correlation-Id: a9e5182b-33d3-4dd6-91e5-08daf2745370 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YJ0qKQdte9qebdAIVr/lphp9ZpxZoWTKDu1pkFFJJFOEtfWRNfYPO19EZRjE9od0yCFwlK2kJFEg4bBzNnQWeFi52fmICnaoExc1ZryZLbzTCt6AMa0Hf9BdN+lvLaclphEASg99JgUETYOrDYwEpdwT+dly3T29HpZ0lAX8ASvUjbk94IQPEnJqLgtZxsLU3Fk8lHCmQIdgry/jyCqoFLS/2wXixJ5AbWjscXT4NeITebazAVSmkJu6LaOmkFz1Xm4TDIPKR/gmDRbm6GnTB8qBROGFK5Du0QeLzrFwWcTzrCg4Auppukka0OF5e7bUwDqxWq/rDunebyo91EQivd7JAJEViVAeIsFm56GEg36VmpvTF9NfVwQfy1LKw4mExsSoc7nmr9+eEDVl3D2mwKWtsUVBMixtqOsALPCoyQWa/Zv1RumALm0hRwO9+ZGn1TG2NanFvQYJhFX0fy3QVsCtzbQpDbBiQsfTkUxd7lIFJg4iDD+p21EFbKJ1RZ/h2nTmLVTfbZ3sfalZuqhCDbBODE+cPgeD/t27WpaqTXv2YEU93XXvejdyOzeshwxJaz3Cj+ZPofeA0daHlRWh/wn2lSypuJ9AXsdldvMX2tb/DfuQeHbnnatVaOB2X5oxm4cKtmMsCGXGnzAqfyKFo0B+9qIKWrTY1oBAcFDRjRSWcuwKTA6DMA4nln2NBIHWonpdAY2cZMO4lJ0kNU/APg== X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-02.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(136003)(376002)(346002)(396003)(39840400004)(451199015)(46966006)(36840700001)(82310400005)(8936002)(2906002)(5660300002)(47076005)(426003)(41300700001)(81166007)(7696005)(70206006)(4326008)(70586007)(8676002)(316002)(356005)(44832011)(186003)(26005)(40480700001)(7126003)(336012)(55016003)(86362001)(83380400001)(36860700001)(103116003)(478600001)(36900700001);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jan 2023 19:04:25.4798 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a9e5182b-33d3-4dd6-91e5-08daf2745370 X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-02.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT102.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR01MB7256 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Dean Luick Split the IB device and port counter allocation. Remove the need for a lock. Clean up pointer usage. Signed-off-by: Dean Luick Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/verbs.c | 81 ++++++++++++++---------------------- 1 file changed, 32 insertions(+), 49 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/verbs.c b/drivers/infiniband/hw/hfi1/verbs.c index e6e17984553c..7f6d7fc7951d 100644 --- a/drivers/infiniband/hw/hfi1/verbs.c +++ b/drivers/infiniband/hw/hfi1/verbs.c @@ -1598,13 +1598,11 @@ static const char * const driver_cntr_names[] = { "DRIVER_EgrHdrFull" }; -static DEFINE_MUTEX(cntr_names_lock); /* protects the *_cntr_names bufers */ static struct rdma_stat_desc *dev_cntr_descs; static struct rdma_stat_desc *port_cntr_descs; int num_driver_cntrs = ARRAY_SIZE(driver_cntr_names); static int num_dev_cntrs; static int num_port_cntrs; -static int cntr_names_initialized; /* * Convert a list of names separated by '\n' into an array of NULL terminated @@ -1615,8 +1613,8 @@ static int init_cntr_names(const char *names_in, const size_t names_len, int num_extra_names, int *num_cntrs, struct rdma_stat_desc **cntr_descs) { - struct rdma_stat_desc *q; - char *names_out, *p; + struct rdma_stat_desc *names_out; + char *p; int i, n; n = 0; @@ -1624,65 +1622,45 @@ static int init_cntr_names(const char *names_in, const size_t names_len, if (names_in[i] == '\n') n++; - names_out = - kzalloc((n + num_extra_names) * sizeof(*q) + names_len, - GFP_KERNEL); + names_out = kzalloc((n + num_extra_names) * sizeof(*names_out) + + names_len, + GFP_KERNEL); if (!names_out) { *num_cntrs = 0; *cntr_descs = NULL; return -ENOMEM; } - p = names_out + (n + num_extra_names) * sizeof(*q); + p = (char *)&names_out[n + num_extra_names]; memcpy(p, names_in, names_len); - q = (struct rdma_stat_desc *)names_out; for (i = 0; i < n; i++) { - q[i].name = p; + names_out[i].name = p; p = strchr(p, '\n'); *p++ = '\0'; } *num_cntrs = n; - *cntr_descs = (struct rdma_stat_desc *)names_out; + *cntr_descs = names_out; return 0; } -static int init_counters(struct ib_device *ibdev) -{ - struct hfi1_devdata *dd = dd_from_ibdev(ibdev); - int i, err = 0; - - mutex_lock(&cntr_names_lock); - if (cntr_names_initialized) - goto out_unlock; - - err = init_cntr_names(dd->cntrnames, dd->cntrnameslen, num_driver_cntrs, - &num_dev_cntrs, &dev_cntr_descs); - if (err) - goto out_unlock; - - for (i = 0; i < num_driver_cntrs; i++) - dev_cntr_descs[num_dev_cntrs + i].name = driver_cntr_names[i]; - - err = init_cntr_names(dd->portcntrnames, dd->portcntrnameslen, 0, - &num_port_cntrs, &port_cntr_descs); - if (err) { - kfree(dev_cntr_descs); - dev_cntr_descs = NULL; - goto out_unlock; - } - cntr_names_initialized = 1; - -out_unlock: - mutex_unlock(&cntr_names_lock); - return err; -} - static struct rdma_hw_stats *hfi1_alloc_hw_device_stats(struct ib_device *ibdev) { - if (init_counters(ibdev)) - return NULL; + if (!dev_cntr_descs) { + struct hfi1_devdata *dd = dd_from_ibdev(ibdev); + int i, err; + + err = init_cntr_names(dd->cntrnames, dd->cntrnameslen, + num_driver_cntrs, + &num_dev_cntrs, &dev_cntr_descs); + if (err) + return NULL; + + for (i = 0; i < num_driver_cntrs; i++) + dev_cntr_descs[num_dev_cntrs + i].name = + driver_cntr_names[i]; + } return rdma_alloc_hw_stats_struct(dev_cntr_descs, num_dev_cntrs + num_driver_cntrs, RDMA_HW_STATS_DEFAULT_LIFESPAN); @@ -1691,8 +1669,16 @@ static struct rdma_hw_stats *hfi1_alloc_hw_device_stats(struct ib_device *ibdev) static struct rdma_hw_stats *hfi_alloc_hw_port_stats(struct ib_device *ibdev, u32 port_num) { - if (init_counters(ibdev)) - return NULL; + if (!port_cntr_descs) { + struct hfi1_devdata *dd = dd_from_ibdev(ibdev); + int err; + + err = init_cntr_names(dd->portcntrnames, dd->portcntrnameslen, + 0, + &num_port_cntrs, &port_cntr_descs); + if (err) + return NULL; + } return rdma_alloc_hw_stats_struct(port_cntr_descs, num_port_cntrs, RDMA_HW_STATS_DEFAULT_LIFESPAN); } @@ -1917,13 +1903,10 @@ void hfi1_unregister_ib_device(struct hfi1_devdata *dd) del_timer_sync(&dev->mem_timer); verbs_txreq_exit(dev); - mutex_lock(&cntr_names_lock); kfree(dev_cntr_descs); kfree(port_cntr_descs); dev_cntr_descs = NULL; port_cntr_descs = NULL; - cntr_names_initialized = 0; - mutex_unlock(&cntr_names_lock); } void hfi1_cnp_rcv(struct hfi1_packet *packet) From patchwork Mon Jan 9 19:04:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 13094124 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 112B6C54EBD for ; Mon, 9 Jan 2023 19:04:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237323AbjAITEf (ORCPT ); Mon, 9 Jan 2023 14:04:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45936 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237225AbjAITEe (ORCPT ); Mon, 9 Jan 2023 14:04:34 -0500 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2106.outbound.protection.outlook.com [40.107.96.106]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5D6EC32EA2 for ; Mon, 9 Jan 2023 11:04:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hU9HastQRX0khIGXpM+9kXKqt+6d7CJtp19uAJ1z1Ce7TcHeBPlavJSMHZouPT4wvRgqHhjjr+mkOqP1gwz1jEFxDdLCbH5ehBwt5BK2sq7vIZI1deO9RQnf64kUDatRrLCNh0CCA3ZVE1CDbkabBKZdT6uQMSB+SpPNvs3/p8U4Kt1j1CTOc6036k/UGjlAQHzw/u3sY8dwPfguDujvRPn5lhXIbPnY9bqyq1L5zomKNw2rMbjP1mA5APt+Q0PAxdkpcmU7WWyHVpfASif2hJMw/6FzgiLhnzg7BFh/spX2yzkNllBCeWM4yg1+fAWuvA/Yoxdg18/Z1Z7yYe8Ujw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=qy28ws4X2FxSiPV+NgaKBr6iU8xsmGUdCP6dSpvcwaE=; b=ZqjFQqdZlSqB4yqlQAvFip9tW7kGjYXHKcbC8CpoqicV1HmjYxLQ9i3fb5UDf5doeKIfFh+ZZ9nsxy9tdgBeSfCa86k02au1nd5EbxOgRdjzc7AhmIioUBCUjhXJbEeG9/eVOBdgMJibsFN6PrPnDhl7UFKS09OGp8/Ttqv8yzyIFvOfJYr/UqQURiXRlah75jLeFaWby9blvSs7oY1TKsIyRuHsmJBVOoOj9V+PzSTebhJ9TmCo7DyZjuCbxA93xqX0L/Elq1ffC52yLbXaAbI1LaStAZhGY1qPoC/FAYcIdfh1tcuocKbSXrvVbPGfHCCkuwXr5jw2QrsagLp5KQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=nvidia.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=qy28ws4X2FxSiPV+NgaKBr6iU8xsmGUdCP6dSpvcwaE=; b=J/Tko96TLCHRUfeCrNGMbX21ciwdfmbtF0vkBN84w4zxnOOfDzXM5d4yJul5mjpJGNeI1iuy/gnsYALoixtujYKHJH5trMM3w5lWc358ZPxJfbZ7wV4G5Y78tsTe8chGW4+0fkjMC0lx5bMvDbHcoDxSJ8Nxswt5jGX2k2nNlU+uxy6/eEzx0yXuwXnOn9GNia48/UuW9qC83nNUj3jRERySipAEdI7rFOHfnVNuqjhpJ3+tMwdTzTbdjoYRMiuPxLfMFM+iZm0h4aedY/6mEAldHRdodTc54eiQy/FIwHDQw5SeUWPNaspZfjT9sJWWsyVJyk/sbHzfA5JMLMQglw== Received: from MW4PR04CA0357.namprd04.prod.outlook.com (2603:10b6:303:8a::32) by MN2PR01MB5472.prod.exchangelabs.com (2603:10b6:208:113::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18; Mon, 9 Jan 2023 19:04:31 +0000 Received: from CO1NAM11FT047.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8a:cafe::df) by MW4PR04CA0357.outlook.office365.com (2603:10b6:303:8a::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18 via Frontend Transport; Mon, 9 Jan 2023 19:04:30 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; dkim=none (message not signed) header.d=none;dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-02.cornelisnetworks.com; pr=C Received: from awfm-02.cornelisnetworks.com (208.255.156.42) by CO1NAM11FT047.mail.protection.outlook.com (10.13.174.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5944.6 via Frontend Transport; Mon, 9 Jan 2023 19:04:30 +0000 Received: from awfm-02.cornelisnetworks.com (localhost [127.0.0.1]) by awfm-02.cornelisnetworks.com (8.16.1/8.16.1) with ESMTP id 309J4TDO1477945; Mon, 9 Jan 2023 14:04:29 -0500 Subject: [PATCH for-next 6/7] IBh/hfi1: Update RMT size calculation From: Dennis Dalessandro To: jgg@nvidia.com, leonro@nvidia.com Cc: Dean Luick , linux-rdma@vger.kernel.org Date: Mon, 09 Jan 2023 14:04:29 -0500 Message-ID: <167329106946.1472990.18385495251650939054.stgit@awfm-02.cornelisnetworks.com> In-Reply-To: <167328561962.1472990.9463955313515395755.stgit@awfm-02.cornelisnetworks.com> References: <167328561962.1472990.9463955313515395755.stgit@awfm-02.cornelisnetworks.com> User-Agent: StGit/1.5.dev2+g9ce680a5 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT047:EE_|MN2PR01MB5472:EE_ X-MS-Office365-Filtering-Correlation-Id: ff91af65-5048-4965-973b-08daf274567e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oZmfo7ElmClcl2gQpQ8aM4UCLFPnC4IGAP/In02Ri3D7QC14rItXL/5FjY5xw8cmddJrUsyzIBuZLo6BcKERERp1WLTtvZPCXADs7im4/4t2ZL9aRJD98MyEcMLiryJWY5SI6WF56iIV0MjsQbdj9F47TXnSnFYfQPuPggxm1TabKcfEy4+044W+i1GtbPWLM58trZHA4G4FmcJDr2Z7ZFHXPH0wMiL4ePnkgPLsZZdGSAJPD2/ge+qcqsW4mal8xsshNbFfD3ZxWVhaR6W3y0WedeeB60nKajzHXHBEgHTqdN7IPd4vccnq38rdIm3VtcGLu//oMBPbNkUo+G3j9/Ww7WetyeAIzcM2RzS/guvIKU6mECCYogmEJO9/VBZAoNI3hSNBuvdoBRnGzeHeImnZa9RAbYFPEfHIJMzQqc5vKUsBgzN0hfxofk3bJZNOFSdU7yEBMzz17nEYmqPoTOZpAlR2fPXsknI4oSCwOJ9/F6+wPPxIKVORCvvLuBaJX+66l2NfI47xoyP0jly6gnnnqrP2mzZ4lUjEeE0GRS83w0DLKaoloYkbLEw52kDNYwf3Yej7yyaLo/fgnes5phCPHKDb1VKLa614lMfwnzWgVgRBu0anx5cVSYHyrk09PqXZ58QsKLtDqs4krhBTEW7+lesFtRQkyIGwl/0HZTC/VY51VH076M1MVtL3ZTrC7ag/7Hz6TB5XCc+kroGVJw== X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-02.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(396003)(136003)(39840400004)(376002)(346002)(451199015)(36840700001)(46966006)(82310400005)(2906002)(15650500001)(8936002)(44832011)(5660300002)(41300700001)(70586007)(8676002)(70206006)(7696005)(316002)(4326008)(103116003)(478600001)(26005)(40480700001)(186003)(336012)(55016003)(7126003)(426003)(86362001)(47076005)(36860700001)(83380400001)(356005)(81166007)(36900700001);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jan 2023 19:04:30.6049 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ff91af65-5048-4965-973b-08daf274567e X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-02.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT047.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR01MB5472 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Dean Luick Fix possible RMT overflow: Use the correct netdev size. Don't allow adjusted user contexts to go negative. Fix QOS calculation: Send kernel context count as an argument since dd->n_krcv_queues is not yet set up in earliest call. Do not include the control context in the QOS calculation. Use the same sized variable to find the max of krcvq[] entries. Update the RMT count explanation to make more sense. Signed-off-by: Dean Luick Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/chip.c | 59 ++++++++++++++++++++----------------- 1 file changed, 32 insertions(+), 27 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/chip.c b/drivers/infiniband/hw/hfi1/chip.c index ebe970f76232..90b672feed83 100644 --- a/drivers/infiniband/hw/hfi1/chip.c +++ b/drivers/infiniband/hw/hfi1/chip.c @@ -1056,7 +1056,7 @@ static void read_link_down_reason(struct hfi1_devdata *dd, u8 *ldr); static void handle_temp_err(struct hfi1_devdata *dd); static void dc_shutdown(struct hfi1_devdata *dd); static void dc_start(struct hfi1_devdata *dd); -static int qos_rmt_entries(struct hfi1_devdata *dd, unsigned int *mp, +static int qos_rmt_entries(unsigned int n_krcv_queues, unsigned int *mp, unsigned int *np); static void clear_full_mgmt_pkey(struct hfi1_pportdata *ppd); static int wait_link_transfer_active(struct hfi1_devdata *dd, int wait_ms); @@ -13362,7 +13362,6 @@ static int set_up_context_variables(struct hfi1_devdata *dd) int ret; unsigned ngroups; int rmt_count; - int user_rmt_reduced; u32 n_usr_ctxts; u32 send_contexts = chip_send_contexts(dd); u32 rcv_contexts = chip_rcv_contexts(dd); @@ -13421,28 +13420,34 @@ static int set_up_context_variables(struct hfi1_devdata *dd) (num_kernel_contexts + n_usr_ctxts), &node_affinity.real_cpu_mask); /* - * The RMT entries are currently allocated as shown below: - * 1. QOS (0 to 128 entries); - * 2. FECN (num_kernel_context - 1 + num_user_contexts + - * num_netdev_contexts); - * 3. netdev (num_netdev_contexts). - * It should be noted that FECN oversubscribe num_netdev_contexts - * entries of RMT because both netdev and PSM could allocate any receive - * context between dd->first_dyn_alloc_text and dd->num_rcv_contexts, - * and PSM FECN must reserve an RMT entry for each possible PSM receive - * context. + * RMT entries are allocated as follows: + * 1. QOS (0 to 128 entries) + * 2. FECN (num_kernel_context - 1 [a] + num_user_contexts + + * num_netdev_contexts [b]) + * 3. netdev (NUM_NETDEV_MAP_ENTRIES) + * + * Notes: + * [a] Kernel contexts (except control) are included in FECN if kernel + * TID_RDMA is active. + * [b] Netdev and user contexts are randomly allocated from the same + * context pool, so FECN must cover all contexts in the pool. */ - rmt_count = qos_rmt_entries(dd, NULL, NULL) + (num_netdev_contexts * 2); - if (HFI1_CAP_IS_KSET(TID_RDMA)) - rmt_count += num_kernel_contexts - 1; - if (rmt_count + n_usr_ctxts > NUM_MAP_ENTRIES) { - user_rmt_reduced = NUM_MAP_ENTRIES - rmt_count; - dd_dev_err(dd, - "RMT size is reducing the number of user receive contexts from %u to %d\n", - n_usr_ctxts, - user_rmt_reduced); - /* recalculate */ - n_usr_ctxts = user_rmt_reduced; + rmt_count = qos_rmt_entries(num_kernel_contexts - 1, NULL, NULL) + + (HFI1_CAP_IS_KSET(TID_RDMA) ? num_kernel_contexts - 1 + : 0) + + n_usr_ctxts + + num_netdev_contexts + + NUM_NETDEV_MAP_ENTRIES; + if (rmt_count > NUM_MAP_ENTRIES) { + int over = rmt_count - NUM_MAP_ENTRIES; + /* try to squish user contexts, minimum of 1 */ + if (over >= n_usr_ctxts) { + dd_dev_err(dd, "RMT overflow: reduce the requested number of contexts\n"); + return -EINVAL; + } + dd_dev_err(dd, "RMT overflow: reducing # user contexts from %u to %u\n", + n_usr_ctxts, n_usr_ctxts - over); + n_usr_ctxts -= over; } /* the first N are kernel contexts, the rest are user/netdev contexts */ @@ -14299,15 +14304,15 @@ static void clear_rsm_rule(struct hfi1_devdata *dd, u8 rule_index) } /* return the number of RSM map table entries that will be used for QOS */ -static int qos_rmt_entries(struct hfi1_devdata *dd, unsigned int *mp, +static int qos_rmt_entries(unsigned int n_krcv_queues, unsigned int *mp, unsigned int *np) { int i; unsigned int m, n; - u8 max_by_vl = 0; + uint max_by_vl = 0; /* is QOS active at all? */ - if (dd->n_krcv_queues <= MIN_KERNEL_KCTXTS || + if (n_krcv_queues < MIN_KERNEL_KCTXTS || num_vls == 1 || krcvqsset <= 1) goto no_qos; @@ -14365,7 +14370,7 @@ static void init_qos(struct hfi1_devdata *dd, struct rsm_map_table *rmt) if (!rmt) goto bail; - rmt_entries = qos_rmt_entries(dd, &m, &n); + rmt_entries = qos_rmt_entries(dd->n_krcv_queues - 1, &m, &n); if (rmt_entries == 0) goto bail; qpns_per_vl = 1 << m; From patchwork Mon Jan 9 19:04:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 13094125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3665EC5479D for ; Mon, 9 Jan 2023 19:04:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237225AbjAITEl (ORCPT ); Mon, 9 Jan 2023 14:04:41 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45964 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236183AbjAITEk (ORCPT ); Mon, 9 Jan 2023 14:04:40 -0500 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2115.outbound.protection.outlook.com [40.107.102.115]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 848593750A for ; Mon, 9 Jan 2023 11:04:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Vj+C8WJn8lWEZ8vbzmyDhsVTf7R9Cg5+cWUiR8K8z/oSNKHdXPtrEGj7OOe4sp4eimJvGY7AbHbKI3TiILG09rIs1Z7Bfg9z4u1HhqLM3Vcg2SoWgML+sxDg1U01U/MnPJr9OxT8hk4cb0NXRgnp0jeZGCSGxgeWd6JEsPelGStYUm775cL1Nky3/fjEcojy2O7IZbOBGAkDcWoRSPluFVGYREUMbRAFNWNeMzgHNHnlpWvzvc9lz7Ey7wvHEjO0578JwDw4GMCRS2dRZWmf84c3ti8lkKJK8CCJAT70P9YkoWExSB9CNmFrLKv5mFiTBrpNenoJRHA1Mp8Z9aBlkQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=x9Gr1MsbFyYrCMFVhwONQDiOyHg5fvC7SXU/PPekGLc=; b=Z48nZiEpK+zHkqzYfxV+58UVCKwF5BiTz9xy9hxpSqtPnvOP36yWCQ/jgLrd46fIhfRdJ74ehseu1Guoa/lu5rJwSTAUxuSix3okUojv3A1SveBSMPMS+2Mm3z2VBUHVkb3xEIWc6isgF88JOCAb9UoTSKoUwLgB3TK1Xz7Ear/kxRn+E+FjxUVilAGclJ/QQPSAet0/AWus7H1LSHqqp6OTDtC/vE80wAz0fCsVaPKbdxn+w0D/d9OvpnK4wkEOzBwNyNQ11tWURBAGEgvQ78DEYluiHxyVnv0jXDyq9TB8t8adLO+4ZRKrIrJiqKRcd6jALvA4xbRMYyTsrbT2Xg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=nvidia.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=x9Gr1MsbFyYrCMFVhwONQDiOyHg5fvC7SXU/PPekGLc=; b=FDBBYVztGhl6Tg4BmpH8cLvFPH4B1nhpA84ZZZF42hkfS6CVRFJHWySKv6wepL8hAZiIzMTr0fvurAWftLUqmPaOWPuUVWL3Iwlr55sRdXuc4PYeemWRASYZiDH27MuFFKsH2L/s9KmF8vPzA+e8o7BQ3DPGBQ8tg/JhBaah3iG5Z+K+8RaI1KB0OXouFm2TsowzCP4X530M1Vst4PSMdoLvkkBVSiwY+aRsSHAJUHWVvySyjBuy/Y2+VBTpyzN2H2z90G/UqhBlQ4+HMMajHUFel7KDmHnEBTLguPaYOQXwXS9xGm8H1pJY/ywXUeP0jGAzgA0pqYdTHCyah9yjhw== Received: from MW4P220CA0012.NAMP220.PROD.OUTLOOK.COM (2603:10b6:303:115::17) by SN6PR0102MB3391.prod.exchangelabs.com (2603:10b6:805:2::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18; Mon, 9 Jan 2023 19:04:36 +0000 Received: from CO1NAM11FT091.eop-nam11.prod.protection.outlook.com (2603:10b6:303:115:cafe::cb) by MW4P220CA0012.outlook.office365.com (2603:10b6:303:115::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18 via Frontend Transport; Mon, 9 Jan 2023 19:04:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; dkim=none (message not signed) header.d=none;dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-02.cornelisnetworks.com; pr=C Received: from awfm-02.cornelisnetworks.com (208.255.156.42) by CO1NAM11FT091.mail.protection.outlook.com (10.13.175.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5986.18 via Frontend Transport; Mon, 9 Jan 2023 19:04:35 +0000 Received: from awfm-02.cornelisnetworks.com (localhost [127.0.0.1]) by awfm-02.cornelisnetworks.com (8.16.1/8.16.1) with ESMTP id 309J4YGA1477954; Mon, 9 Jan 2023 14:04:34 -0500 Subject: [PATCH for-next 7/7] IB/hfi1: Use dma_mmap_coherent for matching buffers From: Dennis Dalessandro To: jgg@nvidia.com, leonro@nvidia.com Cc: Dean Luick , linux-rdma@vger.kernel.org Date: Mon, 09 Jan 2023 14:04:34 -0500 Message-ID: <167329107460.1472990.9090255834533222032.stgit@awfm-02.cornelisnetworks.com> In-Reply-To: <167328561962.1472990.9463955313515395755.stgit@awfm-02.cornelisnetworks.com> References: <167328561962.1472990.9463955313515395755.stgit@awfm-02.cornelisnetworks.com> User-Agent: StGit/1.5.dev2+g9ce680a5 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT091:EE_|SN6PR0102MB3391:EE_ X-MS-Office365-Filtering-Correlation-Id: e01cdcea-252c-4670-509c-08daf274598a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wJj5UrSFA8SXMKQDfM/9oSOHShq9f2BMesnilK0VM8n/hWwX6EZDjgDyzxJ6Oc3lgyh9T9yQyoHy3M+Z0HRuSuDJ3QEL1ITGddCwa8MPFK+bCIpQ5/VlZtM+UyrA5bJfQDM3kyrhbzuCRrHBn6QfpnR8wd3W5yOWuShCOM+Ym0kJ2Zab1PhA94GtDcqNpETws+GLUwIM2LbJWxk2kQ5WiqKXxr+ZGB171vi2uh/tYhxcdIWXuUizOygictnlL9tTly5ngTgnNR3slo9vpwx1TZM2vOkqAuMoRz7ptv2fciRQkJw3y+x07fn+CUmF4gUNrF+rw7Rc1BbsG1JV8YM5m+ESJcH+C0913tPE0Fax6ft4T4hzqm22TSiyueH6RWIcY1svsW97LlSZ//3azQqmjRPLmyWGU6c388TDRdaopmc5uwFkJQ6WYdlzqEE/FLSAnp94SM2e9I5GXvgMt7o/Cz4ffcZBAnMw1S0pxWPEUuNOVF9Itu3ZKuM2Uw+V7xoWmpd8TwQxKwIrRlhIK4V38hgFVG6y+SAdl16uhbgpAJmtIWmJwItwTYNndengwHBqWtY3ZPg9jP9P/0t1eAUuhzApWSPhk7zNlsXBdcDoyjZN98Dre0YzGbcDhPEq3pj/Ko/p5p9ZH2R9JGSEjRykcEWiLQMI5wr3NVSzH+NFZZ6qm2bIXSbx/ZmsvYtXVOEu6yQaIHVCojzQ1TBRxWk8pOJQxufoirtjS9LypaMLvKo= X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-02.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230022)(4636009)(346002)(376002)(396003)(136003)(39840400004)(451199015)(36840700001)(46966006)(83380400001)(2906002)(44832011)(82310400005)(47076005)(336012)(36860700001)(426003)(81166007)(7126003)(5660300002)(55016003)(7696005)(40480700001)(26005)(186003)(8936002)(478600001)(70586007)(70206006)(8676002)(103116003)(41300700001)(86362001)(356005)(4326008)(316002)(36900700001)(309714004);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jan 2023 19:04:35.6992 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e01cdcea-252c-4670-509c-08daf274598a X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-02.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT091.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR0102MB3391 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Dean Luick For memory allocated with dma_alloc_coherent(), use dma_mmap_coherent() to mmap it into user space. Signed-off-by: Dean Luick Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/file_ops.c | 81 +++++++++++++++++++++++---------- 1 file changed, 57 insertions(+), 24 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/file_ops.c b/drivers/infiniband/hw/hfi1/file_ops.c index c9fc913db00c..9703e863ef06 100644 --- a/drivers/infiniband/hw/hfi1/file_ops.c +++ b/drivers/infiniband/hw/hfi1/file_ops.c @@ -306,6 +306,17 @@ static ssize_t hfi1_write_iter(struct kiocb *kiocb, struct iov_iter *from) return reqs; } +static inline void mmap_cdbg(u16 ctxt, u8 subctxt, u8 type, u8 mapio, u8 vmf, + u64 memaddr, void *memvirt, dma_addr_t memdma, + ssize_t memlen, struct vm_area_struct *vma) +{ + hfi1_cdbg(PROC, + "%u:%u type:%u io/vf/dma:%d/%d/%d, addr:0x%llx, len:%lu(%lu), flags:0x%lx", + ctxt, subctxt, type, mapio, vmf, !!memdma, + memaddr ?: (u64)memvirt, memlen, + vma->vm_end - vma->vm_start, vma->vm_flags); +} + static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma) { struct hfi1_filedata *fd = fp->private_data; @@ -315,6 +326,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma) u64 token = vma->vm_pgoff << PAGE_SHIFT, memaddr = 0; void *memvirt = NULL; + dma_addr_t memdma = 0; u8 subctxt, mapio = 0, vmf = 0, type; ssize_t memlen = 0; int ret = 0; @@ -334,6 +346,11 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma) goto done; } + /* + * vm_pgoff is used as a buffer selector cookie. Always mmap from + * the beginning. + */ + vma->vm_pgoff = 0; flags = vma->vm_flags; switch (type) { @@ -355,7 +372,8 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma) vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot); mapio = 1; break; - case PIO_CRED: + case PIO_CRED: { + u64 cr_page_offset; if (flags & VM_WRITE) { ret = -EPERM; goto done; @@ -365,10 +383,11 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma) * second or third page allocated for credit returns (if number * of enabled contexts > 64 and 128 respectively). */ - memvirt = dd->cr_base[uctxt->numa_id].va; - memaddr = virt_to_phys(memvirt) + - (((u64)uctxt->sc->hw_free - - (u64)dd->cr_base[uctxt->numa_id].va) & PAGE_MASK); + cr_page_offset = ((u64)uctxt->sc->hw_free - + (u64)dd->cr_base[uctxt->numa_id].va) & + PAGE_MASK; + memvirt = dd->cr_base[uctxt->numa_id].va + cr_page_offset; + memdma = dd->cr_base[uctxt->numa_id].dma + cr_page_offset; memlen = PAGE_SIZE; flags &= ~VM_MAYWRITE; flags |= VM_DONTCOPY | VM_DONTEXPAND; @@ -378,14 +397,16 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma) * memory been flagged as non-cached? */ /* vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); */ - mapio = 1; break; + } case RCV_HDRQ: memlen = rcvhdrq_size(uctxt); memvirt = uctxt->rcvhdrq; + memdma = uctxt->rcvhdrq_dma; break; case RCV_EGRBUF: { - unsigned long addr; + unsigned long vm_start_save; + unsigned long vm_end_save; int i; /* * The RcvEgr buffer need to be handled differently @@ -404,24 +425,34 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma) goto done; } vma->vm_flags &= ~VM_MAYWRITE; - addr = vma->vm_start; + /* + * Mmap multiple separate allocations into a single vma. From + * here, dma_mmap_coherent() calls dma_direct_mmap(), which + * requires the mmap to exactly fill the vma starting at + * vma_start. Adjust the vma start and end for each eager + * buffer segment mapped. Restore the originals when done. + */ + vm_start_save = vma->vm_start; + vm_end_save = vma->vm_end; + vma->vm_end = vma->vm_start; for (i = 0 ; i < uctxt->egrbufs.numbufs; i++) { memlen = uctxt->egrbufs.buffers[i].len; memvirt = uctxt->egrbufs.buffers[i].addr; - ret = remap_pfn_range( - vma, addr, - /* - * virt_to_pfn() does the same, but - * it's not available on x86_64 - * when CONFIG_MMU is enabled. - */ - PFN_DOWN(__pa(memvirt)), - memlen, - vma->vm_page_prot); - if (ret < 0) + memdma = uctxt->egrbufs.buffers[i].dma; + vma->vm_end += memlen; + mmap_cdbg(ctxt, subctxt, type, mapio, vmf, memaddr, + memvirt, memdma, memlen, vma); + ret = dma_mmap_coherent(&dd->pcidev->dev, vma, + memvirt, memdma, memlen); + if (ret < 0) { + vma->vm_start = vm_start_save; + vma->vm_end = vm_end_save; goto done; - addr += memlen; + } + vma->vm_start += memlen; } + vma->vm_start = vm_start_save; + vma->vm_end = vm_end_save; ret = 0; goto done; } @@ -481,6 +512,7 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma) } memlen = PAGE_SIZE; memvirt = (void *)hfi1_rcvhdrtail_kvaddr(uctxt); + memdma = uctxt->rcvhdrqtailaddr_dma; flags &= ~VM_MAYWRITE; break; case SUBCTXT_UREGS: @@ -529,14 +561,15 @@ static int hfi1_file_mmap(struct file *fp, struct vm_area_struct *vma) } vma->vm_flags = flags; - hfi1_cdbg(PROC, - "%u:%u type:%u io/vf:%d/%d, addr:0x%llx, len:%lu(%lu), flags:0x%lx\n", - ctxt, subctxt, type, mapio, vmf, memaddr, memlen, - vma->vm_end - vma->vm_start, vma->vm_flags); + mmap_cdbg(ctxt, subctxt, type, mapio, vmf, memaddr, memvirt, memdma, + memlen, vma); if (vmf) { vma->vm_pgoff = PFN_DOWN(memaddr); vma->vm_ops = &vm_ops; ret = 0; + } else if (memdma) { + ret = dma_mmap_coherent(&dd->pcidev->dev, vma, + memvirt, memdma, memlen); } else if (mapio) { ret = io_remap_pfn_range(vma, vma->vm_start, PFN_DOWN(memaddr),