From patchwork Fri Apr 7 16:52:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 13205059 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9177BC6FD1D for ; Fri, 7 Apr 2023 16:52:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229575AbjDGQwi (ORCPT ); Fri, 7 Apr 2023 12:52:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229570AbjDGQwh (ORCPT ); Fri, 7 Apr 2023 12:52:37 -0400 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2122.outbound.protection.outlook.com [40.107.92.122]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D46EA55AE for ; Fri, 7 Apr 2023 09:52:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RdYYKHhMaBbie40XVPKAQpk2uj9RSzwL2heVQJ+W32dMzGO2HOCjHP28rinHcsodXTqTGaZi1ycHnKuDOPGHK+4UgPN+od2nPC9Udb6Dlk+ezbMg2I52iJxE+TSgVcI9XfIJNMRJxUWR8k7jcXwtDEed9Zz+W9Uyq9UwhO2nG47g7nWYZaHwucdPp+E7q29cGRLdpAXaQKZn+Nx4pNxDzm3XnDeArKyMrPeL//UPgUT27Lx1IPiVZkxWvXFHR4bPgSnSlnZJGJ6dsnV7RiHiTKCArB6ZoUwmy8J10aIWsJHtjqY7OxjpYbz1tK950acF55Jx6PV//oFqP+htkd+w+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CsBsdWTFxMefyUEAQ/z1I0EvRZ9vHD7gfa015W9deSM=; b=hnbgSPmtHMd1P8W9kVX2j+nZ6BauiaWeAadOayXBDJ7zEl3gFs3N53ggJYGD8sYD6tY90uoUKJqb9scwt5GwTgA2jcC2hJ42TsIhaZP8lCpROGJhj3EQ4GhFxxqk7Kpj06siwYlmcmuQncNFoowMKJnkqgccIM0fs+uMydaMERepzyYCbqQEq18n5zfIMdu75pnSYEa07mtvON4iLb1qmHf55WmAC16HiSJVE3abSS8lPBJRiRaEBULHO7TXMAIRAlyJKqFG2q6H1pGcFTvocbowCaqGTkWtbtp6oyjeR3+bB20ivXeooEBX9+cgryAaYCfNIiXPNZ5j+UIeoCBQpQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=cornelisnetworks.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CsBsdWTFxMefyUEAQ/z1I0EvRZ9vHD7gfa015W9deSM=; b=e0nEjd1gS5s5exu4rgyr1AjgcBmP+/JC+V/Hne7mv4lGGWdWBw82sZ7O/hW1t+TqG7+Sz+T25Xo7FZBpyBHGqWIfPqR/FaNlAfKL1UgmkN227HA6/S7Q6svUkiHnhMWKHppnh731qEltxML0Y8+eqQiLfQW0Ng3rU1YXqatm2deD9m/A0kZjq9l5qfHGt/iiUU0CZWieLZaWiGO4JSq7nmXhG3n00NOhfsDHPY0v/wY9E4MxV2NaZZOSSzf570/bqDpoT7mFdW6VV9ln+KANq/e+f6d6S2iS6kZPNyJM3q/3zIrKEgbbsZVlub0+UK6wzHvHzHVBa5cfQ9q4fKF1IA== Received: from DM6PR07CA0060.namprd07.prod.outlook.com (2603:10b6:5:74::37) by BN6PR0101MB2915.prod.exchangelabs.com (2603:10b6:405:2e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.26; Fri, 7 Apr 2023 16:52:30 +0000 Received: from DM6NAM11FT016.eop-nam11.prod.protection.outlook.com (2603:10b6:5:74:cafe::d4) by DM6PR07CA0060.outlook.office365.com (2603:10b6:5:74::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.31 via Frontend Transport; Fri, 7 Apr 2023 16:52:30 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; dkim=none (message not signed) header.d=none;dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-02.cornelisnetworks.com; pr=C Received: from awfm-02.cornelisnetworks.com (208.255.156.42) by DM6NAM11FT016.mail.protection.outlook.com (10.13.173.139) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.31 via Frontend Transport; Fri, 7 Apr 2023 16:52:29 +0000 Received: from 252.162.96.66.static.eigbox.net (localhost [127.0.0.1]) by awfm-02.cornelisnetworks.com (8.16.1/8.16.1) with ESMTP id 337GqTkT3027374; Fri, 7 Apr 2023 12:52:29 -0400 Subject: [PATCH for-next 1/5] IB/hfi1: Remove trace newlines From: Dennis Dalessandro To: jgg@nvidia.com, leonro@nvidia.com Cc: Dean Luick , linux-rdma@vger.kernel.org Date: Fri, 07 Apr 2023 12:52:29 -0400 Message-ID: <168088634897.3027109.10401662436950683555.stgit@252.162.96.66.static.eigbox.net> In-Reply-To: <168088607365.3027109.2194306496858796762.stgit@252.162.96.66.static.eigbox.net> References: <168088607365.3027109.2194306496858796762.stgit@252.162.96.66.static.eigbox.net> User-Agent: StGit/1.5.dev2+g9ce680a5 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT016:EE_|BN6PR0101MB2915:EE_ X-MS-Office365-Filtering-Correlation-Id: b6d9ab24-6b93-4437-be8c-08db378879b1 X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: s9RmYMz8pETCNWmi5NN4rNRGV37swefUmIPsOqcYptoGr5BXvqhpV2qGmjI3wO17bBS2DLyDpXBZ18QupR5HJfXB2wmVYlstZoskT+kE+bwPo/1JWjZGDGtetrKcqdOs+eyFD/EWasahBCXrNz2YjLkJaCztA1sUjuroDoGqDbLQPbBTN1/gHKA/gmNCB/VGBh8kBwwLSpRVWvgSKMbWilfdeW/+qBu6JkLp8qzs8FCpQ2vVlHN4IoJ3YtdXqh0S/zWSnar8wzLoFcudaiDQu4kgkA1qvAQf1jybabNBovtTT7TRteo/qy5ZOdTDnW1qLNUEXGU57NxLEHths1Mwc/ZJ+cfKOwxej3uHUhPPIptKrA/eugcZgL07PVksNPk3pWZJg3knUk41/sQ/n0pYnzMxlYmq7r3BE20RrwccxAsrhXYRaGB/5ofVptxl/cPoLi+I1DNaIjFfdq1X0eN1NpNuopAelvUOyhgpr14Er68+g3LPQPqcoyz01pXIaQTGupUvZg9xplW0KuhjDBG6a1rpEEKGGuiVApXlPCc/VeykfESaC0FxuWzHV31KWJzgMR9QTYYYLglMXiY+wgcsk/FX9fYKGnuv2hxSGpJVttAe60rqlZh+zrqVnM+VPpR3YeHD2RfH/XTHM9QIHWEHR1uCO2MXTlaClRLzbZZNdjDd+ydcwtdqL+tJxppPT/ZCPVD3wDsLs/6GywYO19lc6kKGX6vkTbp9P6SFivxSFsFkXpRRAfBdhzCxwyaszPXrntmITfheZX6gydFcHGO8eA== X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-02.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(136003)(396003)(346002)(376002)(39840400004)(451199021)(36840700001)(46966006)(7126003)(47076005)(83380400001)(7696005)(426003)(26005)(9686003)(8936002)(186003)(70586007)(336012)(70206006)(44832011)(8676002)(36860700001)(4326008)(55016003)(5660300002)(478600001)(316002)(2906002)(41300700001)(103116003)(40480700001)(356005)(86362001)(82310400005)(81166007)(9916002)(24686002)(36900700001)(102196002);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Apr 2023 16:52:29.8735 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b6d9ab24-6b93-4437-be8c-08db378879b1 X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-02.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT016.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR0101MB2915 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Dean Luick The hfi1_cdbg trace mechanism appends a newline. Remove trailing newlines from all format strings. Signed-off-by: Dean Luick Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/chip.c | 18 +++++++++--------- drivers/infiniband/hw/hfi1/driver.c | 2 +- drivers/infiniband/hw/hfi1/file_ops.c | 2 +- drivers/infiniband/hw/hfi1/init.c | 12 ++++++------ drivers/infiniband/hw/hfi1/pio.c | 2 +- 5 files changed, 18 insertions(+), 18 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/chip.c b/drivers/infiniband/hw/hfi1/chip.c index 90b672feed83..9dbb89e9f4af 100644 --- a/drivers/infiniband/hw/hfi1/chip.c +++ b/drivers/infiniband/hw/hfi1/chip.c @@ -12135,7 +12135,7 @@ void hfi1_rcvctrl(struct hfi1_devdata *dd, unsigned int op, set_intr_bits(dd, IS_RCVURGENT_START + rcd->ctxt, IS_RCVURGENT_START + rcd->ctxt, false); - hfi1_cdbg(RCVCTRL, "ctxt %d rcvctrl 0x%llx\n", ctxt, rcvctrl); + hfi1_cdbg(RCVCTRL, "ctxt %d rcvctrl 0x%llx", ctxt, rcvctrl); write_kctxt_csr(dd, ctxt, RCV_CTXT_CTRL, rcvctrl); /* work around sticky RcvCtxtStatus.BlockedRHQFull */ @@ -12205,10 +12205,10 @@ u32 hfi1_read_cntrs(struct hfi1_devdata *dd, char **namep, u64 **cntrp) hfi1_cdbg(CNTR, "reading %s", entry->name); if (entry->flags & CNTR_DISABLED) { /* Nothing */ - hfi1_cdbg(CNTR, "\tDisabled\n"); + hfi1_cdbg(CNTR, "\tDisabled"); } else { if (entry->flags & CNTR_VL) { - hfi1_cdbg(CNTR, "\tPer VL\n"); + hfi1_cdbg(CNTR, "\tPer VL"); for (j = 0; j < C_VL_COUNT; j++) { val = entry->rw_cntr(entry, dd, j, @@ -12216,21 +12216,21 @@ u32 hfi1_read_cntrs(struct hfi1_devdata *dd, char **namep, u64 **cntrp) 0); hfi1_cdbg( CNTR, - "\t\tRead 0x%llx for %d\n", + "\t\tRead 0x%llx for %d", val, j); dd->cntrs[entry->offset + j] = val; } } else if (entry->flags & CNTR_SDMA) { hfi1_cdbg(CNTR, - "\t Per SDMA Engine\n"); + "\t Per SDMA Engine"); for (j = 0; j < chip_sdma_engines(dd); j++) { val = entry->rw_cntr(entry, dd, j, CNTR_MODE_R, 0); hfi1_cdbg(CNTR, - "\t\tRead 0x%llx for %d\n", + "\t\tRead 0x%llx for %d", val, j); dd->cntrs[entry->offset + j] = val; @@ -12271,7 +12271,7 @@ u32 hfi1_read_portcntrs(struct hfi1_pportdata *ppd, char **namep, u64 **cntrp) hfi1_cdbg(CNTR, "reading %s", entry->name); if (entry->flags & CNTR_DISABLED) { /* Nothing */ - hfi1_cdbg(CNTR, "\tDisabled\n"); + hfi1_cdbg(CNTR, "\tDisabled"); continue; } @@ -12513,7 +12513,7 @@ static void do_update_synth_timer(struct work_struct *work) hfi1_cdbg( CNTR, - "[%d] curr tx=0x%llx rx=0x%llx :: last tx=0x%llx rx=0x%llx\n", + "[%d] curr tx=0x%llx rx=0x%llx :: last tx=0x%llx rx=0x%llx", dd->unit, cur_tx, cur_rx, dd->last_tx, dd->last_rx); if ((cur_tx < dd->last_tx) || (cur_rx < dd->last_rx)) { @@ -12527,7 +12527,7 @@ static void do_update_synth_timer(struct work_struct *work) } else { total_flits = (cur_tx - dd->last_tx) + (cur_rx - dd->last_rx); hfi1_cdbg(CNTR, - "[%d] total flits 0x%llx limit 0x%llx\n", dd->unit, + "[%d] total flits 0x%llx limit 0x%llx", dd->unit, total_flits, (u64)CNTR_32BIT_MAX); if (total_flits >= CNTR_32BIT_MAX) { hfi1_cdbg(CNTR, "[%d] 32bit limit hit, updating", diff --git a/drivers/infiniband/hw/hfi1/driver.c b/drivers/infiniband/hw/hfi1/driver.c index bcc6bc0540f0..f4492fa407e0 100644 --- a/drivers/infiniband/hw/hfi1/driver.c +++ b/drivers/infiniband/hw/hfi1/driver.c @@ -1597,7 +1597,7 @@ static int hfi1_setup_bypass_packet(struct hfi1_packet *packet) return 0; drop: - hfi1_cdbg(PKT, "%s: packet dropped\n", __func__); + hfi1_cdbg(PKT, "%s: packet dropped", __func__); ibp->rvp.n_pkt_drops++; return -EINVAL; } diff --git a/drivers/infiniband/hw/hfi1/file_ops.c b/drivers/infiniband/hw/hfi1/file_ops.c index b1d6ca7e9708..acbc52bdf841 100644 --- a/drivers/infiniband/hw/hfi1/file_ops.c +++ b/drivers/infiniband/hw/hfi1/file_ops.c @@ -977,7 +977,7 @@ static int allocate_ctxt(struct hfi1_filedata *fd, struct hfi1_devdata *dd, ret = -ENOMEM; goto ctxdata_free; } - hfi1_cdbg(PROC, "allocated send context %u(%u)\n", uctxt->sc->sw_index, + hfi1_cdbg(PROC, "allocated send context %u(%u)", uctxt->sc->sw_index, uctxt->sc->hw_context); ret = sc_enable(uctxt->sc); if (ret) diff --git a/drivers/infiniband/hw/hfi1/init.c b/drivers/infiniband/hw/hfi1/init.c index 62b6c5020039..6de37c5d7d27 100644 --- a/drivers/infiniband/hw/hfi1/init.c +++ b/drivers/infiniband/hw/hfi1/init.c @@ -342,7 +342,7 @@ int hfi1_create_ctxtdata(struct hfi1_pportdata *ppd, int numa, INIT_LIST_HEAD(&rcd->flow_queue.queue_head); INIT_LIST_HEAD(&rcd->rarr_queue.queue_head); - hfi1_cdbg(PROC, "setting up context %u\n", rcd->ctxt); + hfi1_cdbg(PROC, "setting up context %u", rcd->ctxt); /* * Calculate the context's RcvArray entry starting point. @@ -400,7 +400,7 @@ int hfi1_create_ctxtdata(struct hfi1_pportdata *ppd, int numa, rcd->egrbufs.count = MAX_EAGER_ENTRIES; } hfi1_cdbg(PROC, - "ctxt%u: max Eager buffer RcvArray entries: %u\n", + "ctxt%u: max Eager buffer RcvArray entries: %u", rcd->ctxt, rcd->egrbufs.count); /* @@ -432,7 +432,7 @@ int hfi1_create_ctxtdata(struct hfi1_pportdata *ppd, int numa, if (rcd->egrbufs.size < hfi1_max_mtu) { rcd->egrbufs.size = __roundup_pow_of_two(hfi1_max_mtu); hfi1_cdbg(PROC, - "ctxt%u: eager bufs size too small. Adjusting to %u\n", + "ctxt%u: eager bufs size too small. Adjusting to %u", rcd->ctxt, rcd->egrbufs.size); } rcd->egrbufs.rcvtid_size = HFI1_MAX_EAGER_BUFFER_SIZE; @@ -1920,7 +1920,7 @@ int hfi1_setup_eagerbufs(struct hfi1_ctxtdata *rcd) rcd->egrbufs.size = alloced_bytes; hfi1_cdbg(PROC, - "ctxt%u: Alloced %u rcv tid entries @ %uKB, total %uKB\n", + "ctxt%u: Alloced %u rcv tid entries @ %uKB, total %uKB", rcd->ctxt, rcd->egrbufs.alloced, rcd->egrbufs.rcvtid_size / 1024, rcd->egrbufs.size / 1024); @@ -1943,13 +1943,13 @@ int hfi1_setup_eagerbufs(struct hfi1_ctxtdata *rcd) rcd->expected_count = MAX_TID_PAIR_ENTRIES * 2; rcd->expected_base = rcd->eager_base + egrtop; - hfi1_cdbg(PROC, "ctxt%u: eager:%u, exp:%u, egrbase:%u, expbase:%u\n", + hfi1_cdbg(PROC, "ctxt%u: eager:%u, exp:%u, egrbase:%u, expbase:%u", rcd->ctxt, rcd->egrbufs.alloced, rcd->expected_count, rcd->eager_base, rcd->expected_base); if (!hfi1_rcvbuf_validate(rcd->egrbufs.rcvtid_size, PT_EAGER, &order)) { hfi1_cdbg(PROC, - "ctxt%u: current Eager buffer size is invalid %u\n", + "ctxt%u: current Eager buffer size is invalid %u", rcd->ctxt, rcd->egrbufs.rcvtid_size); ret = -EINVAL; goto bail_rcvegrbuf_phys; diff --git a/drivers/infiniband/hw/hfi1/pio.c b/drivers/infiniband/hw/hfi1/pio.c index 51ae58c02b15..62e7dc9bea7b 100644 --- a/drivers/infiniband/hw/hfi1/pio.c +++ b/drivers/infiniband/hw/hfi1/pio.c @@ -820,7 +820,7 @@ struct send_context *sc_alloc(struct hfi1_devdata *dd, int type, } hfi1_cdbg(PIO, - "Send context %u(%u) %s group %u credits %u credit_ctrl 0x%llx threshold %u\n", + "Send context %u(%u) %s group %u credits %u credit_ctrl 0x%llx threshold %u", sw_index, hw_context, sc_type_name(type), From patchwork Fri Apr 7 16:52:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 13205061 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3CE3C6FD1D for ; Fri, 7 Apr 2023 16:52:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229570AbjDGQwp (ORCPT ); Fri, 7 Apr 2023 12:52:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33038 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229887AbjDGQwo (ORCPT ); Fri, 7 Apr 2023 12:52:44 -0400 Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2099.outbound.protection.outlook.com [40.107.100.99]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B857555AE for ; Fri, 7 Apr 2023 09:52:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lfuFdZobBQ0LB9b4Qboop7Wr9IjoaCheQO+caYceSe2HU7rkdVrL62vlZZlEi7b1mOPv3Dwb3Uy97GtyZjf8RVJeTaIpKYawdSSt2cxx5dnRoUVZZmI3hDjv9Y9IGeecHPCRjNzGu0rpIOb8/Rc3Sm4fvugQ+LRKNR6tycBaSZbcKbb2HLMpK84D3Me4vhzwC1i0rbzd1LX71IHKa3z+TjVnuzA9wcYlwxkFQEy3zo6AMx8AVqb9LTsBvFhmrSFlyWAJkj4VDS31v8CV1daockbdgMeKoSS4FiN/SZHkAdtc6FmrlK6Hg3w44tuahnSUa7ZHzPEgBvxJ1B7oPkONgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZF4ZhfCr12Z/bU8C2WGNuQrPMXUj6+/zwBjrkUskj1k=; b=evtcjWRw6X9BX/zxu466bQgwBc+bc5yAJ/pZf+GAqEGCEcodtscEQ4jn6ceL/TmYS3Pjw+IP7lQ2AqvcTpDoZSrMwelhbqb1bGvY7VVCBgZT5dWyY0ZlUAn09AZqVD0KL9REuX6zbE8m1JVchC7CyKTvzUm6hQp2Coh2CASESxSw968z7XfeZbDe8AYyrs0HOuY7hxQtf7IqjUjVu/8yECAnIMvVJNV0izvrQHVQnZ7pdtifdf2imCOOYBqba0L6bK3h7uFBXwmZ7zfhN1bii5NQ3kOSZDr72V3ewKsIFieg/Y0xXFExbv0KT6zKnUF4LPlnJeUwqghHpJX8/h2yaQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=nvidia.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZF4ZhfCr12Z/bU8C2WGNuQrPMXUj6+/zwBjrkUskj1k=; b=aV4wVbFDvDqA+oRN5BV9/EbuuAjnhO3mAVqbI+2s6CcI9bcdglfqpAhLCzLannG5eKr3yrLtw2dAqoSACrEAY4ZIMSQKVpT1+Kq7qR/k+2lYYhDwgZda5DghfUkXE56KK+2VTwpcEyyaLBaH1JOFkRAKoW8r458RU5x+LS/TOvSFfSqD7VSJHOaqcU14Gp083pPcMJqk1e6zVyJURweiDCzNgapDF5oR8onCdbTBhqKHztxrtb+swgCg6UtXk5yB8YF7wNyoSv38eWdmcIBccl38h545HHCQoPmfUnm2cb9uuFcRf/n7iS+bgA67hF7yUv7lu9ObFp6owE26PlQc0Q== Received: from DM6PR13CA0005.namprd13.prod.outlook.com (2603:10b6:5:bc::18) by BL0PR0102MB3492.prod.exchangelabs.com (2603:10b6:207:1e::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6298.18; Fri, 7 Apr 2023 16:52:35 +0000 Received: from DM6NAM11FT009.eop-nam11.prod.protection.outlook.com (2603:10b6:5:bc:cafe::b) by DM6PR13CA0005.outlook.office365.com (2603:10b6:5:bc::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6298.20 via Frontend Transport; Fri, 7 Apr 2023 16:52:35 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; dkim=none (message not signed) header.d=none;dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-02.cornelisnetworks.com; pr=C Received: from awfm-02.cornelisnetworks.com (208.255.156.42) by DM6NAM11FT009.mail.protection.outlook.com (10.13.173.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.31 via Frontend Transport; Fri, 7 Apr 2023 16:52:35 +0000 Received: from 252.162.96.66.static.eigbox.net (localhost [127.0.0.1]) by awfm-02.cornelisnetworks.com (8.16.1/8.16.1) with ESMTP id 337GqYBa3027384; Fri, 7 Apr 2023 12:52:34 -0400 Subject: [PATCH for-next 2/5] IB/hfi1: Suppress useless compiler warnings From: Dennis Dalessandro To: jgg@nvidia.com, leonro@nvidia.com Cc: Ehab Ababneh , linux-rdma@vger.kernel.org Date: Fri, 07 Apr 2023 12:52:34 -0400 Message-ID: <168088635415.3027109.5711716700328939402.stgit@252.162.96.66.static.eigbox.net> In-Reply-To: <168088607365.3027109.2194306496858796762.stgit@252.162.96.66.static.eigbox.net> References: <168088607365.3027109.2194306496858796762.stgit@252.162.96.66.static.eigbox.net> User-Agent: StGit/1.5.dev2+g9ce680a5 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT009:EE_|BL0PR0102MB3492:EE_ X-MS-Office365-Filtering-Correlation-Id: 1068add4-1f08-4191-6093-08db37887cc2 X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9S9KGaB0FI0XeG97TeoIwJfDWqUM0EKPfz/7Mtie/2DVyePgwoUrO7s1aaUMM7YUUq7IGrflnvp4ASCaz5dH30ZkJutjsjCzhDy/lsgUVQ9D4OHM8Oy6Z7nkKGePumivwrl0q7U8U98qXJFvglz4DCmdQRcDus7urc905WmpCY6icKnTjWEloQvVNzz9MP8/g0q7cf5Ca7S1YmQ+enzy3/YHV8eS4Iwxz6+/CjjPtsWwjZHte5dZv6rD9WSUIIUTyHZes/ERNE/jPlO0+61gWkX/DCSCh0eWCTMe0wWvH/5hr3yil/0LGJaDyruSxullWHr35tuG0fCs9NJ0f1GgWx1To/wDrAK97jC7J5cmSp2/+wE8qEYb7G1upt5FRieJI/WFUsozU2hBfPGsovDY6nuqdHtZiO/eGw61db4vrjItNUJvQv8QTardLP0rRgBEHEUPxPSs+w9nP6QZw9l3OxuUQPHvpATijmmo4OS5tKIIj+PivDVqMry14LP6iePaw89CaEaiYBJ0fMliPBrqNTjuv0SvVnjLedi8rD25K3rsj55Ar36BC2O8KF0Oq9kHOj1vuZ0tpKyrQVsblPZNPuyNdnzh44TcfaGSoygP3siWksgOIUvRTWUeYtrg4AzJN+CPJHLYiDoiG4Vi/DiElUeRMHPzSK/Ne4GIF1G4tLUKPLKypatSmW5ERWyq7uz3QuRl5zdR3vDF9sjO3tuJy2vHmZviOxvBx4Q0cqet42cLJ+1AGXpxbsPNi/QvLt9bFwQue9sLbYXDsSfXd0+W9Q== X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-02.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(376002)(136003)(396003)(39840400004)(451199021)(46966006)(36840700001)(44832011)(2906002)(103116003)(41300700001)(316002)(40480700001)(55016003)(4326008)(5660300002)(8676002)(8936002)(83380400001)(47076005)(70586007)(86362001)(70206006)(7696005)(36860700001)(9686003)(26005)(186003)(7126003)(426003)(336012)(478600001)(82310400005)(356005)(81166007)(9916002)(24686002)(36900700001)(102196002);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Apr 2023 16:52:35.0156 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1068add4-1f08-4191-6093-08db37887cc2 X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-02.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT009.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR0102MB3492 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Ehab Ababneh These warnings can cause build failure: In file included from ./include/trace/define_trace.h:102, from drivers/infiniband/hw/hfi1/trace_dbg.h:111, from drivers/infiniband/hw/hfi1/trace.h:15, from drivers/infiniband/hw/hfi1/trace.c:6: drivers/infiniband/hw/hfi1/./trace_dbg.h: In function ‘trace_event_get_offsets_hfi1_trace_template’: ./include/trace/trace_events.h:261:9: warning: function ‘trace_event_get_offsets_hfi1_trace_template’ might be a candidate for ‘gnu_printf’ format attribute [-Wsuggest-attribute=format] struct trace_event_raw_##call __maybe_unused *entry; \ ^~~~~~~~~~~~~~~~ drivers/infiniband/hw/hfi1/./trace_dbg.h:25:1: note: in expansion of macro ‘DECLARE_EVENT_CLASS’ DECLARE_EVENT_CLASS(hfi1_trace_template, ^~~~~~~~~~~~~~~~~~~ In file included from ./include/trace/define_trace.h:102, from drivers/infiniband/hw/hfi1/trace_dbg.h:111, from drivers/infiniband/hw/hfi1/trace.h:15, from drivers/infiniband/hw/hfi1/trace.c:6: drivers/infiniband/hw/hfi1/./trace_dbg.h: In function ‘trace_event_raw_event_hfi1_trace_template’: ./include/trace/trace_events.h:386:9: warning: function ‘trace_event_raw_event_hfi1_trace_template’ might be a candidate for ‘gnu_printf’ format attribute [-Wsuggest-attribute=format] struct trace_event_raw_##call *entry; \ ^~~~~~~~~~~~~~~~ drivers/infiniband/hw/hfi1/./trace_dbg.h:25:1: note: in expansion of macro ‘DECLARE_EVENT_CLASS’ DECLARE_EVENT_CLASS(hfi1_trace_template, ^~~~~~~~~~~~~~~~~~~ In file included from ./include/trace/define_trace.h:103, from drivers/infiniband/hw/hfi1/trace_dbg.h:111, from drivers/infiniband/hw/hfi1/trace.h:15, from drivers/infiniband/hw/hfi1/trace.c:6: drivers/infiniband/hw/hfi1/./trace_dbg.h: In function ‘perf_trace_hfi1_trace_template’: ./include/trace/perf.h:70:9: warning: function ‘perf_trace_hfi1_trace_template’ might be a candidate for ‘gnu_printf’ format attribute [-Wsuggest-attribute=format] struct hlist_head *head; \ ^~~~~~~~~~ drivers/infiniband/hw/hfi1/./trace_dbg.h:25:1: note: in expansion of macro ‘DECLARE_EVENT_CLASS’ DECLARE_EVENT_CLASS(hfi1_trace_template, ^~~~~~~~~~~~~~~~~~~ Solution adapted here is similar to the one in fbbc95a49d5b0 Signed-off-by: Ehab Ababneh Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/trace_dbg.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/infiniband/hw/hfi1/trace_dbg.h b/drivers/infiniband/hw/hfi1/trace_dbg.h index 582b6f68df3d..489395bfb5b3 100644 --- a/drivers/infiniband/hw/hfi1/trace_dbg.h +++ b/drivers/infiniband/hw/hfi1/trace_dbg.h @@ -22,6 +22,11 @@ #define MAX_MSG_LEN 512 +#pragma GCC diagnostic push +#ifndef __clang__ +#pragma GCC diagnostic ignored "-Wsuggest-attribute=format" +#endif + DECLARE_EVENT_CLASS(hfi1_trace_template, TP_PROTO(const char *function, struct va_format *vaf), TP_ARGS(function, vaf), @@ -36,6 +41,8 @@ DECLARE_EVENT_CLASS(hfi1_trace_template, __get_str(msg)) ); +#pragma GCC diagnostic pop + /* * It may be nice to macroize the __hfi1_trace but the va_* stuff requires an * actual function to work and can not be in a macro. From patchwork Fri Apr 7 16:52:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 13205062 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49EC1C6FD1D for ; Fri, 7 Apr 2023 16:52:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229935AbjDGQwv (ORCPT ); Fri, 7 Apr 2023 12:52:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33134 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229611AbjDGQwv (ORCPT ); Fri, 7 Apr 2023 12:52:51 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2136.outbound.protection.outlook.com [40.107.220.136]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9CF505B92 for ; Fri, 7 Apr 2023 09:52:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JWDTiVKCxC2UXwy/wOqfYAivolwFZqQsvPPB8tQUL/1UnizKLU5H18l5xzwN2/yQPJgO3y2xwUXFwtlwIy6e9n64VzJeQmw/rKjW0AsKMOYFjoJ9OP4RZ1ZTAygJZ88JZXqyB6OQ3LxfBq9FKvnZmZLfoO0EWE+OB9sPhSgy5WC1ibWM5s059vvf7LD8ON1n2mwK3Z4xz/wWmwjAp49knaiMmNEGKk1htSoi36x24cYMvsUX6X34OrQniKuDBwXaQvzjg1XhkahNqsIhd3xt8wBdja0MwgORudEJjT7VGlczcukxboltfjoEse8TmAtsL9flybm1bxHrvgwCP06hRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=C0VuyUTbjuV/ejAK8NY8T4i8+HA6yLNtJR8SbIhpLGo=; b=CPFLari7nD6Ib8PopUKtDaO4syv1l0owF6FVRDvDRuKn46Muo6CGho/B23LDzi9mq/euXmFMOR153evqguZxLz4GPl2MRm8MoeLV7BzaYKAniajRD5tp8pHWHFtYQg99rFhXZo3ZuaesUipyoocVhm/dUbEraXraWlDxF1JiT8PfAU++8QV9Py1BEVF1yd/95v9llpf0Ck1K97nOqyHOmpSV78FCDTt7WT3YOpBKoPPabrPiu5DUpdHskcsKQYsFQW0xhIa+ML+jsRLimnrm21rMhLvtzt6WBBcUpz8J2WT5I6e1/G4tPjtA0MF2C/15jKIT70vdyLILYTDJBI5Qlg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=nvidia.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C0VuyUTbjuV/ejAK8NY8T4i8+HA6yLNtJR8SbIhpLGo=; b=nNtp3r+sfud+42HlL53Pvtx/90I5H4nNzIgnP5ezso2KMKIWf5CwO41Hz7gG+FGU0JExEM6bUttGua8cEEalpmF7XXeGK1QkC8ssUKUrYZCmIyeX2XTXqp3smy7uR1M3N+YovBR1dt7uK/9FPuerDlj2JinJ/gOzjROByAWU+H9Qf5vFioWMWDA3F9K3TlGXj9l+HllM3x9gOz3M0q9ERQmkgUXOYWPSxsRfBG9qW4zP1vISnA/CVZ7wqjHREU4fKtuTxzatLsvZmnEY8irmqX7df+i5BUWrpHlHTE4TzDb/iomxBgI0LmFKiZ6cD650LLoQlCXffbbUNaV2ukVuUg== Received: from MW4P220CA0012.NAMP220.PROD.OUTLOOK.COM (2603:10b6:303:115::17) by CH0PR01MB7153.prod.exchangelabs.com (2603:10b6:610:ea::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6298.18; Fri, 7 Apr 2023 16:52:40 +0000 Received: from CO1NAM11FT041.eop-nam11.prod.protection.outlook.com (2603:10b6:303:115:cafe::3b) by MW4P220CA0012.outlook.office365.com (2603:10b6:303:115::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.34 via Frontend Transport; Fri, 7 Apr 2023 16:52:40 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; dkim=none (message not signed) header.d=none;dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-02.cornelisnetworks.com; pr=C Received: from awfm-02.cornelisnetworks.com (208.255.156.42) by CO1NAM11FT041.mail.protection.outlook.com (10.13.174.217) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6298.21 via Frontend Transport; Fri, 7 Apr 2023 16:52:40 +0000 Received: from 252.162.96.66.static.eigbox.net (localhost [127.0.0.1]) by awfm-02.cornelisnetworks.com (8.16.1/8.16.1) with ESMTP id 337GqdRb3027393; Fri, 7 Apr 2023 12:52:39 -0400 Subject: [PATCH for-next 3/5] IB/hfi1: Fix SDMA mmu_rb_node not being evicted in LRU order From: Dennis Dalessandro To: jgg@nvidia.com, leonro@nvidia.com Cc: Brendan Cunningham , Patrick Kelsey , linux-rdma@vger.kernel.org Date: Fri, 07 Apr 2023 12:52:39 -0400 Message-ID: <168088635931.3027109.10423156330761536044.stgit@252.162.96.66.static.eigbox.net> In-Reply-To: <168088607365.3027109.2194306496858796762.stgit@252.162.96.66.static.eigbox.net> References: <168088607365.3027109.2194306496858796762.stgit@252.162.96.66.static.eigbox.net> User-Agent: StGit/1.5.dev2+g9ce680a5 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT041:EE_|CH0PR01MB7153:EE_ X-MS-Office365-Filtering-Correlation-Id: 75d739ff-628a-443c-829a-08db3788800e X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CMJRM4GM5072B31ycPeF2SwwgAQQ8CDvc5d9LMf2kmlZQgM3nmEHDvGg9FU8AnSg3JIBRu4j6XfSSY5nTxef8XEjHIhGbxBLEPPOBB9zC3XGQj3OGrsDcG6WZ3wXbA8ZCUVik/yOx8gxpKC6zx2cBow/PT7Nw6rfnrmbUJZVtV3JgUzJh5oMlfI8V0V8LQHF3ACNE1tNBT7gQ0KBvNY7eWbJpw+TXIGQ9MEMQGLvr++nNk52dzQ7H2wa5abiBbNsmVNrwZpzQj2xgrn/DTFKUhbPk6gucFWbV+KC9eI/kqjTAy78I5bg81ENYowVCNDQuLHfPR3j3hBt0px/jPeOLK23gQqmvVCW/nSEW8Y/c8/hLIoy1T6YnC94R2xTu89SJFJUCPe+EliQnvJmKD/i7H6tap05Y3k0fmTBoKVOOb3OLW51Dzoc/ggIHLxxBctXqyTb+zFAZJCwKtgCI2ZrmpHn03mWNtxNXvjCvwEpE3+HPQ3d7na9YlnJR30aVSqquiVvy5nDi0DOWwMiLbBjBe6IBHwFHQgeyJRSrPjFH3eve61JY8RZw1oakiPhC9jE5xtkk5un3qk5jYd3/rFoosEJbpucdsyJU6xqmmkq3i4ijiWoZQok7g/kt9T4jzWIPaS/UD4e3hwUl+PJXaZubghKD3Qqyk/vLpfTQSKvfNpezcbW0mIW9c/aTlIJg9i58JSA9B7/3MyrYd9l840lsAEUf+l6hmpbDS4tHzhvzYmQCW42v1BOOtUGjeQ8VigX5LcsR2NdLAVHWnlla0T6vg== X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-02.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(376002)(396003)(39840400004)(136003)(451199021)(36840700001)(46966006)(40480700001)(36860700001)(336012)(83380400001)(426003)(47076005)(7126003)(54906003)(478600001)(7696005)(186003)(26005)(9686003)(103116003)(2906002)(44832011)(5660300002)(81166007)(316002)(4326008)(356005)(41300700001)(70586007)(86362001)(82310400005)(8676002)(8936002)(70206006)(55016003)(9916002)(24686002)(102196002)(36900700001);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Apr 2023 16:52:40.5009 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 75d739ff-628a-443c-829a-08db3788800e X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-02.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT041.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR01MB7153 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Patrick Kelsey hfi1_mmu_rb_remove_unless_exact() did not move mmu_rb_node objects in mmu_rb_handler->lru_list after getting a cache hit on an mmu_rb_node. As a result, hfi1_mmu_rb_evict() was not guaranteed to evict truly least-recently used nodes. This could be a performance issue for an application when that application: - Uses some long-lived buffers frequently. - Uses a large number of buffers once. - Hits the mmu_rb_handler cache size or pinned-page limits, forcing mmu_rb_handler cache entries to be evicted. In this case, the one-time use buffers cause the long-lived buffer entries to eventually filter to the end of the LRU list where hfi1_mmu_rb_evict() will consider evicting a frequently-used long-lived entry instead of evicting one of the one-time use entries. Fix this by inserting new mmu_rb_node at the tail of mmu_rb_handler->lru_list and move mmu_rb_ndoe to the tail of mmu_rb_handler->lru_list when the mmu_rb_node is a hit in hfi1_mmu_rb_remove_unless_exact(). Change hfi1_mmu_rb_evict() to evict from the head of mmu_rb_handler->lru_list instead of the tail. Fixes: 0636e9ab8355 ("IB/hfi1: Add cache evict LRU list") Signed-off-by: Brendan Cunningham Signed-off-by: Patrick Kelsey Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/mmu_rb.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/mmu_rb.c b/drivers/infiniband/hw/hfi1/mmu_rb.c index 7333646021bb..af46ff203342 100644 --- a/drivers/infiniband/hw/hfi1/mmu_rb.c +++ b/drivers/infiniband/hw/hfi1/mmu_rb.c @@ -130,7 +130,7 @@ int hfi1_mmu_rb_insert(struct mmu_rb_handler *handler, goto unlock; } __mmu_int_rb_insert(mnode, &handler->root); - list_add(&mnode->list, &handler->lru_list); + list_add_tail(&mnode->list, &handler->lru_list); ret = handler->ops->insert(handler->ops_arg, mnode); if (ret) { @@ -181,8 +181,10 @@ bool hfi1_mmu_rb_remove_unless_exact(struct mmu_rb_handler *handler, spin_lock_irqsave(&handler->lock, flags); node = __mmu_rb_search(handler, addr, len); if (node) { - if (node->addr == addr && node->len == len) + if (node->addr == addr && node->len == len) { + list_move_tail(&node->list, &handler->lru_list); goto unlock; + } __mmu_int_rb_remove(node, &handler->root); list_del(&node->list); /* remove from LRU list */ ret = true; @@ -206,8 +208,7 @@ void hfi1_mmu_rb_evict(struct mmu_rb_handler *handler, void *evict_arg) INIT_LIST_HEAD(&del_list); spin_lock_irqsave(&handler->lock, flags); - list_for_each_entry_safe_reverse(rbnode, ptr, &handler->lru_list, - list) { + list_for_each_entry_safe(rbnode, ptr, &handler->lru_list, list) { if (handler->ops->evict(handler->ops_arg, rbnode, evict_arg, &stop)) { __mmu_int_rb_remove(rbnode, &handler->root); @@ -219,9 +220,7 @@ void hfi1_mmu_rb_evict(struct mmu_rb_handler *handler, void *evict_arg) } spin_unlock_irqrestore(&handler->lock, flags); - while (!list_empty(&del_list)) { - rbnode = list_first_entry(&del_list, struct mmu_rb_node, list); - list_del(&rbnode->list); + list_for_each_entry_safe(rbnode, ptr, &del_list, list) { handler->ops->remove(handler->ops_arg, rbnode); } } From patchwork Fri Apr 7 16:52:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 13205064 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 711EFC77B61 for ; Fri, 7 Apr 2023 16:53:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229598AbjDGQxB (ORCPT ); Fri, 7 Apr 2023 12:53:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33428 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229887AbjDGQw6 (ORCPT ); Fri, 7 Apr 2023 12:52:58 -0400 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2131.outbound.protection.outlook.com [40.107.244.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 21BC86A47 for ; Fri, 7 Apr 2023 09:52:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WiP7RFMqINfdxfxosD71O7tlsrFSasORDwV+l2sD+viBN5QS6xA0dgDX4+BxBhAZ2OhrysAKczZJ/aytCgCXfn6zrP4IlbKQvaX7C6PgtQE2xYemSXcBbKN9I5QMeA5cIMFdwv222dqoH4Wf/vHlGXsxUxd3X2Te/pvVxHoKWTSC87MkoJQWknHvdLU+lk+WQorNP82CmoJvYaqdO7R0RHwXcQSeOGXWaWp7hZ5z84Rm6T98Q92TYU5KIAcF4JU666QBCd1GmU+pcv1ouCyTYoYT/JZqYNJB3hTuh3q58bj0GYaqCumY/1MkxUphAKW4iNoWe5yrSkIEB7E6kEbQZg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XTMhFpOmAxlxh5lSOHCkOdq7X/L+adOgRkAAZGOt/nk=; b=W3s4+ViDKhxbnHtfWaXvaxAgNzdEJev+nVzjxMSw3A3oi3Pdu4QNwDXRZDZDUYhWExo0+eFncKZWfwNCa5bLhKcf1wQC2WYR9VSNiJUuQLflwITzKdizRSqV1xD+bR5Cqx/2lLKvtRTn21ri71EviUgXwWER2PFXGiAFm5reXap3kRlDcPfMWNwi9C7ks1/RFS9DjJzhKIyvwIaoJfp75AsCluzNTE1y8RuopeL/8axe2RVD3poS17Ht9A2izVwMJ1YpHqH6PRs3ficLm5nL8bWF4Ip/wH7tPP3xWOtvvOSR0CwMb1Osg9kQW7E+V1ikpTI6WzFosHsg3UDXbe/jlQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=nvidia.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XTMhFpOmAxlxh5lSOHCkOdq7X/L+adOgRkAAZGOt/nk=; b=OkXpkN8BlshN4rWZAarU6rfIOJEYdZ5EAKQDRx0kfedothgwCFcDO7EVrMKygurnY8B7xVQt3o+XfEOyoKZAa28Ig2085wkF/YEUIJejcvqGetavW3Btzc8Ra3zKwBsa8T3wz80GjAR6Ni7/wGI+jK9EdejSyn90hMBJudSOmekyTj1vPm5uhgV+DnjwxP0Pg18s/hcCu+oRqoQEycvvT2WJ0nkIHSU+oyG67BBx64l/hrPFo1AcrhDdk06d9SzZk6r/qMvwyC2C1Ujejm8nNyonDBntUYe0x181hZiW2LcBQKOxt8UZCQ1qtDZsOLHN2QKt+Dx+L3r4yh34HCpc1A== Received: from MW2PR2101CA0030.namprd21.prod.outlook.com (2603:10b6:302:1::43) by DM5PR0101MB3132.prod.exchangelabs.com (2603:10b6:4:2c::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.16; Fri, 7 Apr 2023 16:52:46 +0000 Received: from CO1NAM11FT097.eop-nam11.prod.protection.outlook.com (2603:10b6:302:1:cafe::29) by MW2PR2101CA0030.outlook.office365.com (2603:10b6:302:1::43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6298.16 via Frontend Transport; Fri, 7 Apr 2023 16:52:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; dkim=none (message not signed) header.d=none;dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-02.cornelisnetworks.com; pr=C Received: from awfm-02.cornelisnetworks.com (208.255.156.42) by CO1NAM11FT097.mail.protection.outlook.com (10.13.175.185) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6298.20 via Frontend Transport; Fri, 7 Apr 2023 16:52:45 +0000 Received: from 252.162.96.66.static.eigbox.net (localhost [127.0.0.1]) by awfm-02.cornelisnetworks.com (8.16.1/8.16.1) with ESMTP id 337Gqic43027403; Fri, 7 Apr 2023 12:52:44 -0400 Subject: [PATCH for-next 4/5] IB/hfi1: Fix bugs with non-PAGE_SIZE-end multi-iovec user SDMA requests From: Dennis Dalessandro To: jgg@nvidia.com, leonro@nvidia.com Cc: Brendan Cunningham , Patrick Kelsey , linux-rdma@vger.kernel.org Date: Fri, 07 Apr 2023 12:52:44 -0400 Message-ID: <168088636445.3027109.10054635277810177889.stgit@252.162.96.66.static.eigbox.net> In-Reply-To: <168088607365.3027109.2194306496858796762.stgit@252.162.96.66.static.eigbox.net> References: <168088607365.3027109.2194306496858796762.stgit@252.162.96.66.static.eigbox.net> User-Agent: StGit/1.5.dev2+g9ce680a5 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT097:EE_|DM5PR0101MB3132:EE_ X-MS-Office365-Filtering-Correlation-Id: 0d0a7836-1bfc-4dc3-cd71-08db37888311 X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: q9HTpdINnHYJ75xyxupcXJYz2k7wtDSQ37SjRfnk8jUuZfdzplYUVjp6rwcNlGGTpm7wDcOSxu7JH27paSz+yrtGfgkBno162wenB+r+vQ/hXxwm88Rwu7g0UgN3Wl8piNxTeumPlD0MAnqbtBcUOW9TEQ8IvLIhkT7sjOaxoC5fVUR3vQAUgGbBAFvbl6gbAdEus0ESSFlrAKe1aS7qsjqYM7Nu2vehRkiMvU2axkyRc1XHslt8kEK1h5OXIJrXkvnhNSZDuoAXg7gcc5AyPBp8wiJGVKEyRI8TwyeLpJa6MbSBrpljmgbTk2bDMPew1wpJNfEd3oQst6BbcmC/TnznskUV8xVuiSkXV1vUpA18SlnWtm1wjE4sdIFMPRO2JvNfrD2nqJUH9CSArKIva/A0IXTev9tezUrGdoE43240/XxQbyq7NjC/zPRd+1irg5Zv7y4GGB1c4WNudgQjpprO3NemnBkIwFH/3Lk+eoQnuv03jbb2olC7xf3PK8Cer69AeB0Y7ZuHPqZizfbPVVrgstvZdZtRxw9Yad2JdemTFJFoj7BF9736/HPDC6JJVKf97QXNjeQQvXHe9oAM+iJaHFl+j8dEN56MZkqT6XMwFO84svCB4GatjOBoBwoFw0ILHAA0VuwYbLmDvFGR3q6wDtKDbUJQ2PPyOv24sbwG0wcpRgcqkNgCBRLB9Mm0BMK+hQ2mxeN56upgvLPJnzxWvDwgBHEIzMlWUv+VV3q92o61Jp6Kd7YPzAS6SGntqlVn0IQo0mARQBo1VE+qPQ== X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-02.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(39840400004)(346002)(376002)(136003)(396003)(451199021)(36840700001)(46966006)(70586007)(9686003)(8676002)(70206006)(86362001)(26005)(4326008)(186003)(54906003)(7696005)(316002)(36860700001)(40480700001)(55016003)(7126003)(103116003)(82310400005)(81166007)(44832011)(8936002)(5660300002)(2906002)(30864003)(426003)(47076005)(478600001)(336012)(356005)(41300700001)(83380400001)(9916002)(24686002)(102196002)(36900700001);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Apr 2023 16:52:45.5400 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0d0a7836-1bfc-4dc3-cd71-08db37888311 X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-02.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT097.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR0101MB3132 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Patrick Kelsey hfi1 user SDMA request processing has two bugs that can cause data corruption for user SDMA requests that have multiple payload iovecs where an iovec other than the tail iovec does not run up to the page boundary for the buffer pointed to by that iovec.a Here are the specific bugs: 1. user_sdma_txadd() does not use struct user_sdma_iovec->iov.iov_len. Rather, user_sdma_txadd() will add up to PAGE_SIZE bytes from iovec to the packet, even if some of those bytes are past iovec->iov.iov_len and are thus not intended to be in the packet. 2. user_sdma_txadd() and user_sdma_send_pkts() fail to advance to the next iovec in user_sdma_request->iovs when the current iovec is not PAGE_SIZE and does not contain enough data to complete the packet. The transmitted packet will contain the wrong data from the iovec pages. This has not been an issue with SDMA packets from hfi1 Verbs or PSM2 because they only produce iovecs that end short of PAGE_SIZE as the tail iovec of an SDMA request. Fixing these bugs exposes other bugs with the SDMA pin cache (struct mmu_rb_handler) that get in way of supporting user SDMA requests with multiple payload iovecs whose buffers do not end at PAGE_SIZE. So this commit fixes those issues as well. Here are the mmu_rb_handler bugs that non-PAGE_SIZE-end multi-iovec payload user SDMA requests can hit: 1. Overlapping memory ranges in mmu_rb_handler will result in duplicate pinnings. 2. When extending an existing mmu_rb_handler entry (struct mmu_rb_node), the mmu_rb code (1) removes the existing entry under a lock, (2) releases that lock, pins the new pages, (3) then reacquires the lock to insert the extended mmu_rb_node. If someone else comes in and inserts an overlapping entry between (2) and (3), insert in (3) will fail. The failure path code in this case unpins _all_ pages in either the original mmu_rb_node or the new mmu_rb_node that was inserted between (2) and (3). 3. In hfi1_mmu_rb_remove_unless_exact(), mmu_rb_node->refcount is incremented outside of mmu_rb_handler->lock. As a result, mmu_rb_node could be evicted by another thread that gets mmu_rb_handler->lock and checks mmu_rb_node->refcount before mmu_rb_node->refcount is incremented. 4. Related to #2 above, SDMA request submission failure path does not check mmu_rb_node->refcount before freeing mmu_rb_node object. If there are other SDMA requests in progress whose iovecs have pointers to the now-freed mmu_rb_node(s), those pointers to the now-freed mmu_rb nodes will be dereferenced when those SDMA requests complete. Fixes: 7be85676f1d1 ("IB/hfi1: Don't remove RB entry when not needed.") Fixes: 7724105686e7 ("IB/hfi1: add driver files") Signed-off-by: Brendan Cunningham Signed-off-by: Patrick Kelsey Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/ipoib_tx.c | 1 drivers/infiniband/hw/hfi1/mmu_rb.c | 66 +-- drivers/infiniband/hw/hfi1/mmu_rb.h | 8 drivers/infiniband/hw/hfi1/sdma.c | 21 - drivers/infiniband/hw/hfi1/sdma.h | 16 + drivers/infiniband/hw/hfi1/sdma_txreq.h | 1 drivers/infiniband/hw/hfi1/trace_mmu.h | 4 drivers/infiniband/hw/hfi1/user_sdma.c | 600 ++++++++++++++++++++----------- drivers/infiniband/hw/hfi1/user_sdma.h | 5 drivers/infiniband/hw/hfi1/verbs.c | 4 drivers/infiniband/hw/hfi1/vnic_sdma.c | 1 11 files changed, 423 insertions(+), 304 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/ipoib_tx.c b/drivers/infiniband/hw/hfi1/ipoib_tx.c index 349eb4139136..8973a081d641 100644 --- a/drivers/infiniband/hw/hfi1/ipoib_tx.c +++ b/drivers/infiniband/hw/hfi1/ipoib_tx.c @@ -215,6 +215,7 @@ static int hfi1_ipoib_build_ulp_payload(struct ipoib_txreq *tx, const skb_frag_t *frag = &skb_shinfo(skb)->frags[i]; ret = sdma_txadd_page(dd, + NULL, txreq, skb_frag_page(frag), frag->bv_offset, diff --git a/drivers/infiniband/hw/hfi1/mmu_rb.c b/drivers/infiniband/hw/hfi1/mmu_rb.c index af46ff203342..71b9ac018887 100644 --- a/drivers/infiniband/hw/hfi1/mmu_rb.c +++ b/drivers/infiniband/hw/hfi1/mmu_rb.c @@ -126,7 +126,7 @@ int hfi1_mmu_rb_insert(struct mmu_rb_handler *handler, spin_lock_irqsave(&handler->lock, flags); node = __mmu_rb_search(handler, mnode->addr, mnode->len); if (node) { - ret = -EINVAL; + ret = -EEXIST; goto unlock; } __mmu_int_rb_insert(mnode, &handler->root); @@ -143,6 +143,19 @@ int hfi1_mmu_rb_insert(struct mmu_rb_handler *handler, return ret; } +/* Caller must hold handler lock */ +struct mmu_rb_node *hfi1_mmu_rb_get_first(struct mmu_rb_handler *handler, + unsigned long addr, unsigned long len) +{ + struct mmu_rb_node *node; + + trace_hfi1_mmu_rb_search(addr, len); + node = __mmu_int_rb_iter_first(&handler->root, addr, (addr + len) - 1); + if (node) + list_move_tail(&node->list, &handler->lru_list); + return node; +} + /* Caller must hold handler lock */ static struct mmu_rb_node *__mmu_rb_search(struct mmu_rb_handler *handler, unsigned long addr, @@ -167,34 +180,6 @@ static struct mmu_rb_node *__mmu_rb_search(struct mmu_rb_handler *handler, return node; } -bool hfi1_mmu_rb_remove_unless_exact(struct mmu_rb_handler *handler, - unsigned long addr, unsigned long len, - struct mmu_rb_node **rb_node) -{ - struct mmu_rb_node *node; - unsigned long flags; - bool ret = false; - - if (current->mm != handler->mn.mm) - return ret; - - spin_lock_irqsave(&handler->lock, flags); - node = __mmu_rb_search(handler, addr, len); - if (node) { - if (node->addr == addr && node->len == len) { - list_move_tail(&node->list, &handler->lru_list); - goto unlock; - } - __mmu_int_rb_remove(node, &handler->root); - list_del(&node->list); /* remove from LRU list */ - ret = true; - } -unlock: - spin_unlock_irqrestore(&handler->lock, flags); - *rb_node = node; - return ret; -} - void hfi1_mmu_rb_evict(struct mmu_rb_handler *handler, void *evict_arg) { struct mmu_rb_node *rbnode, *ptr; @@ -225,29 +210,6 @@ void hfi1_mmu_rb_evict(struct mmu_rb_handler *handler, void *evict_arg) } } -/* - * It is up to the caller to ensure that this function does not race with the - * mmu invalidate notifier which may be calling the users remove callback on - * 'node'. - */ -void hfi1_mmu_rb_remove(struct mmu_rb_handler *handler, - struct mmu_rb_node *node) -{ - unsigned long flags; - - if (current->mm != handler->mn.mm) - return; - - /* Validity of handler and node pointers has been checked by caller. */ - trace_hfi1_mmu_rb_remove(node->addr, node->len); - spin_lock_irqsave(&handler->lock, flags); - __mmu_int_rb_remove(node, &handler->root); - list_del(&node->list); /* remove from LRU list */ - spin_unlock_irqrestore(&handler->lock, flags); - - handler->ops->remove(handler->ops_arg, node); -} - static int mmu_notifier_range_start(struct mmu_notifier *mn, const struct mmu_notifier_range *range) { diff --git a/drivers/infiniband/hw/hfi1/mmu_rb.h b/drivers/infiniband/hw/hfi1/mmu_rb.h index 7417be2b9dc8..ed75acdb7b83 100644 --- a/drivers/infiniband/hw/hfi1/mmu_rb.h +++ b/drivers/infiniband/hw/hfi1/mmu_rb.h @@ -52,10 +52,8 @@ void hfi1_mmu_rb_unregister(struct mmu_rb_handler *handler); int hfi1_mmu_rb_insert(struct mmu_rb_handler *handler, struct mmu_rb_node *mnode); void hfi1_mmu_rb_evict(struct mmu_rb_handler *handler, void *evict_arg); -void hfi1_mmu_rb_remove(struct mmu_rb_handler *handler, - struct mmu_rb_node *mnode); -bool hfi1_mmu_rb_remove_unless_exact(struct mmu_rb_handler *handler, - unsigned long addr, unsigned long len, - struct mmu_rb_node **rb_node); +struct mmu_rb_node *hfi1_mmu_rb_get_first(struct mmu_rb_handler *handler, + unsigned long addr, + unsigned long len); #endif /* _HFI1_MMU_RB_H */ diff --git a/drivers/infiniband/hw/hfi1/sdma.c b/drivers/infiniband/hw/hfi1/sdma.c index 8ed20392e9f0..bb2552dd29c1 100644 --- a/drivers/infiniband/hw/hfi1/sdma.c +++ b/drivers/infiniband/hw/hfi1/sdma.c @@ -1593,22 +1593,7 @@ static inline void sdma_unmap_desc( struct hfi1_devdata *dd, struct sdma_desc *descp) { - switch (sdma_mapping_type(descp)) { - case SDMA_MAP_SINGLE: - dma_unmap_single( - &dd->pcidev->dev, - sdma_mapping_addr(descp), - sdma_mapping_len(descp), - DMA_TO_DEVICE); - break; - case SDMA_MAP_PAGE: - dma_unmap_page( - &dd->pcidev->dev, - sdma_mapping_addr(descp), - sdma_mapping_len(descp), - DMA_TO_DEVICE); - break; - } + system_descriptor_complete(dd, descp); } /* @@ -3128,7 +3113,7 @@ int ext_coal_sdma_tx_descs(struct hfi1_devdata *dd, struct sdma_txreq *tx, /* Add descriptor for coalesce buffer */ tx->desc_limit = MAX_DESC; - return _sdma_txadd_daddr(dd, SDMA_MAP_SINGLE, tx, + return _sdma_txadd_daddr(dd, SDMA_MAP_SINGLE, NULL, tx, addr, tx->tlen); } @@ -3167,10 +3152,12 @@ int _pad_sdma_tx_descs(struct hfi1_devdata *dd, struct sdma_txreq *tx) return rval; } } + /* finish the one just added */ make_tx_sdma_desc( tx, SDMA_MAP_NONE, + NULL, dd->sdma_pad_phys, sizeof(u32) - (tx->packet_len & (sizeof(u32) - 1))); tx->num_desc++; diff --git a/drivers/infiniband/hw/hfi1/sdma.h b/drivers/infiniband/hw/hfi1/sdma.h index b023fc461bd5..95aaec14c6c2 100644 --- a/drivers/infiniband/hw/hfi1/sdma.h +++ b/drivers/infiniband/hw/hfi1/sdma.h @@ -594,6 +594,7 @@ static inline dma_addr_t sdma_mapping_addr(struct sdma_desc *d) static inline void make_tx_sdma_desc( struct sdma_txreq *tx, int type, + void *pinning_ctx, dma_addr_t addr, size_t len) { @@ -612,6 +613,7 @@ static inline void make_tx_sdma_desc( << SDMA_DESC0_PHY_ADDR_SHIFT) | (((u64)len & SDMA_DESC0_BYTE_COUNT_MASK) << SDMA_DESC0_BYTE_COUNT_SHIFT); + desc->pinning_ctx = pinning_ctx; } /* helper to extend txreq */ @@ -643,6 +645,7 @@ static inline void _sdma_close_tx(struct hfi1_devdata *dd, static inline int _sdma_txadd_daddr( struct hfi1_devdata *dd, int type, + void *pinning_ctx, struct sdma_txreq *tx, dma_addr_t addr, u16 len) @@ -652,6 +655,7 @@ static inline int _sdma_txadd_daddr( make_tx_sdma_desc( tx, type, + pinning_ctx, addr, len); WARN_ON(len > tx->tlen); tx->num_desc++; @@ -672,6 +676,7 @@ static inline int _sdma_txadd_daddr( /** * sdma_txadd_page() - add a page to the sdma_txreq * @dd: the device to use for mapping + * @pinning_ctx: context to be released at descriptor retirement * @tx: tx request to which the page is added * @page: page to map * @offset: offset within the page @@ -687,6 +692,7 @@ static inline int _sdma_txadd_daddr( */ static inline int sdma_txadd_page( struct hfi1_devdata *dd, + void *pinning_ctx, struct sdma_txreq *tx, struct page *page, unsigned long offset, @@ -714,8 +720,7 @@ static inline int sdma_txadd_page( return -ENOSPC; } - return _sdma_txadd_daddr( - dd, SDMA_MAP_PAGE, tx, addr, len); + return _sdma_txadd_daddr(dd, SDMA_MAP_PAGE, pinning_ctx, tx, addr, len); } /** @@ -749,7 +754,8 @@ static inline int sdma_txadd_daddr( return rval; } - return _sdma_txadd_daddr(dd, SDMA_MAP_NONE, tx, addr, len); + return _sdma_txadd_daddr(dd, SDMA_MAP_NONE, NULL, tx, + addr, len); } /** @@ -795,8 +801,7 @@ static inline int sdma_txadd_kvaddr( return -ENOSPC; } - return _sdma_txadd_daddr( - dd, SDMA_MAP_SINGLE, tx, addr, len); + return _sdma_txadd_daddr(dd, SDMA_MAP_SINGLE, NULL, tx, addr, len); } struct iowait_work; @@ -1030,4 +1035,5 @@ extern uint mod_num_sdma; void sdma_update_lmc(struct hfi1_devdata *dd, u64 mask, u32 lid); +void system_descriptor_complete(struct hfi1_devdata *dd, struct sdma_desc *descp); #endif diff --git a/drivers/infiniband/hw/hfi1/sdma_txreq.h b/drivers/infiniband/hw/hfi1/sdma_txreq.h index e262fb5c5ec6..fad946cb5e0d 100644 --- a/drivers/infiniband/hw/hfi1/sdma_txreq.h +++ b/drivers/infiniband/hw/hfi1/sdma_txreq.h @@ -19,6 +19,7 @@ struct sdma_desc { /* private: don't use directly */ u64 qw[2]; + void *pinning_ctx; }; /** diff --git a/drivers/infiniband/hw/hfi1/trace_mmu.h b/drivers/infiniband/hw/hfi1/trace_mmu.h index 187e9244fe5e..57900ebb7702 100644 --- a/drivers/infiniband/hw/hfi1/trace_mmu.h +++ b/drivers/infiniband/hw/hfi1/trace_mmu.h @@ -37,10 +37,6 @@ DEFINE_EVENT(hfi1_mmu_rb_template, hfi1_mmu_rb_search, TP_PROTO(unsigned long addr, unsigned long len), TP_ARGS(addr, len)); -DEFINE_EVENT(hfi1_mmu_rb_template, hfi1_mmu_rb_remove, - TP_PROTO(unsigned long addr, unsigned long len), - TP_ARGS(addr, len)); - DEFINE_EVENT(hfi1_mmu_rb_template, hfi1_mmu_mem_invalidate, TP_PROTO(unsigned long addr, unsigned long len), TP_ARGS(addr, len)); diff --git a/drivers/infiniband/hw/hfi1/user_sdma.c b/drivers/infiniband/hw/hfi1/user_sdma.c index a71c5a36ceba..ae58b48afe07 100644 --- a/drivers/infiniband/hw/hfi1/user_sdma.c +++ b/drivers/infiniband/hw/hfi1/user_sdma.c @@ -24,7 +24,6 @@ #include "hfi.h" #include "sdma.h" -#include "mmu_rb.h" #include "user_sdma.h" #include "verbs.h" /* for the headers */ #include "common.h" /* for struct hfi1_tid_info */ @@ -39,11 +38,7 @@ static unsigned initial_pkt_count = 8; static int user_sdma_send_pkts(struct user_sdma_request *req, u16 maxpkts); static void user_sdma_txreq_cb(struct sdma_txreq *txreq, int status); static inline void pq_update(struct hfi1_user_sdma_pkt_q *pq); -static void user_sdma_free_request(struct user_sdma_request *req, bool unpin); -static int pin_vector_pages(struct user_sdma_request *req, - struct user_sdma_iovec *iovec); -static void unpin_vector_pages(struct mm_struct *mm, struct page **pages, - unsigned start, unsigned npages); +static void user_sdma_free_request(struct user_sdma_request *req); static int check_header_template(struct user_sdma_request *req, struct hfi1_pkt_header *hdr, u32 lrhlen, u32 datalen); @@ -81,6 +76,11 @@ static struct mmu_rb_ops sdma_rb_ops = { .invalidate = sdma_rb_invalidate }; +static int add_system_pages_to_sdma_packet(struct user_sdma_request *req, + struct user_sdma_txreq *tx, + struct user_sdma_iovec *iovec, + u32 *pkt_remaining); + static int defer_packet_queue( struct sdma_engine *sde, struct iowait_work *wait, @@ -410,6 +410,7 @@ int hfi1_user_sdma_process_request(struct hfi1_filedata *fd, ret = -EINVAL; goto free_req; } + /* Copy the header from the user buffer */ ret = copy_from_user(&req->hdr, iovec[idx].iov_base + sizeof(info), sizeof(req->hdr)); @@ -484,9 +485,8 @@ int hfi1_user_sdma_process_request(struct hfi1_filedata *fd, memcpy(&req->iovs[i].iov, iovec + idx++, sizeof(req->iovs[i].iov)); - ret = pin_vector_pages(req, &req->iovs[i]); - if (ret) { - req->data_iovs = i; + if (req->iovs[i].iov.iov_len == 0) { + ret = -EINVAL; goto free_req; } req->data_len += req->iovs[i].iov.iov_len; @@ -584,7 +584,7 @@ int hfi1_user_sdma_process_request(struct hfi1_filedata *fd, if (req->seqsubmitted) wait_event(pq->busy.wait_dma, (req->seqcomp == req->seqsubmitted - 1)); - user_sdma_free_request(req, true); + user_sdma_free_request(req); pq_update(pq); set_comp_state(pq, cq, info.comp_idx, ERROR, ret); } @@ -696,48 +696,6 @@ static int user_sdma_txadd_ahg(struct user_sdma_request *req, return ret; } -static int user_sdma_txadd(struct user_sdma_request *req, - struct user_sdma_txreq *tx, - struct user_sdma_iovec *iovec, u32 datalen, - u32 *queued_ptr, u32 *data_sent_ptr, - u64 *iov_offset_ptr) -{ - int ret; - unsigned int pageidx, len; - unsigned long base, offset; - u64 iov_offset = *iov_offset_ptr; - u32 queued = *queued_ptr, data_sent = *data_sent_ptr; - struct hfi1_user_sdma_pkt_q *pq = req->pq; - - base = (unsigned long)iovec->iov.iov_base; - offset = offset_in_page(base + iovec->offset + iov_offset); - pageidx = (((iovec->offset + iov_offset + base) - (base & PAGE_MASK)) >> - PAGE_SHIFT); - len = offset + req->info.fragsize > PAGE_SIZE ? - PAGE_SIZE - offset : req->info.fragsize; - len = min((datalen - queued), len); - ret = sdma_txadd_page(pq->dd, &tx->txreq, iovec->pages[pageidx], - offset, len); - if (ret) { - SDMA_DBG(req, "SDMA txreq add page failed %d\n", ret); - return ret; - } - iov_offset += len; - queued += len; - data_sent += len; - if (unlikely(queued < datalen && pageidx == iovec->npages && - req->iov_idx < req->data_iovs - 1)) { - iovec->offset += iov_offset; - iovec = &req->iovs[++req->iov_idx]; - iov_offset = 0; - } - - *queued_ptr = queued; - *data_sent_ptr = data_sent; - *iov_offset_ptr = iov_offset; - return ret; -} - static int user_sdma_send_pkts(struct user_sdma_request *req, u16 maxpkts) { int ret = 0; @@ -769,8 +727,7 @@ static int user_sdma_send_pkts(struct user_sdma_request *req, u16 maxpkts) maxpkts = req->info.npkts - req->seqnum; while (npkts < maxpkts) { - u32 datalen = 0, queued = 0, data_sent = 0; - u64 iov_offset = 0; + u32 datalen = 0; /* * Check whether any of the completions have come back @@ -863,27 +820,17 @@ static int user_sdma_send_pkts(struct user_sdma_request *req, u16 maxpkts) goto free_txreq; } - /* - * If the request contains any data vectors, add up to - * fragsize bytes to the descriptor. - */ - while (queued < datalen && - (req->sent + data_sent) < req->data_len) { - ret = user_sdma_txadd(req, tx, iovec, datalen, - &queued, &data_sent, &iov_offset); - if (ret) - goto free_txreq; - } - /* - * The txreq was submitted successfully so we can update - * the counters. - */ req->koffset += datalen; if (req_opcode(req->info.ctrl) == EXPECTED) req->tidoffset += datalen; - req->sent += data_sent; - if (req->data_len) - iovec->offset += iov_offset; + req->sent += datalen; + while (datalen) { + ret = add_system_pages_to_sdma_packet(req, tx, iovec, + &datalen); + if (ret) + goto free_txreq; + iovec = &req->iovs[req->iov_idx]; + } list_add_tail(&tx->txreq.list, &req->txps); /* * It is important to increment this here as it is used to @@ -920,133 +867,14 @@ static int user_sdma_send_pkts(struct user_sdma_request *req, u16 maxpkts) static u32 sdma_cache_evict(struct hfi1_user_sdma_pkt_q *pq, u32 npages) { struct evict_data evict_data; + struct mmu_rb_handler *handler = pq->handler; evict_data.cleared = 0; evict_data.target = npages; - hfi1_mmu_rb_evict(pq->handler, &evict_data); + hfi1_mmu_rb_evict(handler, &evict_data); return evict_data.cleared; } -static int pin_sdma_pages(struct user_sdma_request *req, - struct user_sdma_iovec *iovec, - struct sdma_mmu_node *node, - int npages) -{ - int pinned, cleared; - struct page **pages; - struct hfi1_user_sdma_pkt_q *pq = req->pq; - - pages = kcalloc(npages, sizeof(*pages), GFP_KERNEL); - if (!pages) - return -ENOMEM; - memcpy(pages, node->pages, node->npages * sizeof(*pages)); - - npages -= node->npages; -retry: - if (!hfi1_can_pin_pages(pq->dd, current->mm, - atomic_read(&pq->n_locked), npages)) { - cleared = sdma_cache_evict(pq, npages); - if (cleared >= npages) - goto retry; - } - pinned = hfi1_acquire_user_pages(current->mm, - ((unsigned long)iovec->iov.iov_base + - (node->npages * PAGE_SIZE)), npages, 0, - pages + node->npages); - if (pinned < 0) { - kfree(pages); - return pinned; - } - if (pinned != npages) { - unpin_vector_pages(current->mm, pages, node->npages, pinned); - return -EFAULT; - } - kfree(node->pages); - node->rb.len = iovec->iov.iov_len; - node->pages = pages; - atomic_add(pinned, &pq->n_locked); - return pinned; -} - -static void unpin_sdma_pages(struct sdma_mmu_node *node) -{ - if (node->npages) { - unpin_vector_pages(mm_from_sdma_node(node), node->pages, 0, - node->npages); - atomic_sub(node->npages, &node->pq->n_locked); - } -} - -static int pin_vector_pages(struct user_sdma_request *req, - struct user_sdma_iovec *iovec) -{ - int ret = 0, pinned, npages; - struct hfi1_user_sdma_pkt_q *pq = req->pq; - struct sdma_mmu_node *node = NULL; - struct mmu_rb_node *rb_node; - struct iovec *iov; - bool extracted; - - extracted = - hfi1_mmu_rb_remove_unless_exact(pq->handler, - (unsigned long) - iovec->iov.iov_base, - iovec->iov.iov_len, &rb_node); - if (rb_node) { - node = container_of(rb_node, struct sdma_mmu_node, rb); - if (!extracted) { - atomic_inc(&node->refcount); - iovec->pages = node->pages; - iovec->npages = node->npages; - iovec->node = node; - return 0; - } - } - - if (!node) { - node = kzalloc(sizeof(*node), GFP_KERNEL); - if (!node) - return -ENOMEM; - - node->rb.addr = (unsigned long)iovec->iov.iov_base; - node->pq = pq; - atomic_set(&node->refcount, 0); - } - - iov = &iovec->iov; - npages = num_user_pages((unsigned long)iov->iov_base, iov->iov_len); - if (node->npages < npages) { - pinned = pin_sdma_pages(req, iovec, node, npages); - if (pinned < 0) { - ret = pinned; - goto bail; - } - node->npages += pinned; - npages = node->npages; - } - iovec->pages = node->pages; - iovec->npages = npages; - iovec->node = node; - - ret = hfi1_mmu_rb_insert(req->pq->handler, &node->rb); - if (ret) { - iovec->node = NULL; - goto bail; - } - return 0; -bail: - unpin_sdma_pages(node); - kfree(node); - return ret; -} - -static void unpin_vector_pages(struct mm_struct *mm, struct page **pages, - unsigned start, unsigned npages) -{ - hfi1_release_user_pages(mm, pages + start, npages, false); - kfree(pages); -} - static int check_header_template(struct user_sdma_request *req, struct hfi1_pkt_header *hdr, u32 lrhlen, u32 datalen) @@ -1388,7 +1216,7 @@ static void user_sdma_txreq_cb(struct sdma_txreq *txreq, int status) if (req->seqcomp != req->info.npkts - 1) return; - user_sdma_free_request(req, false); + user_sdma_free_request(req); set_comp_state(pq, cq, req->info.comp_idx, state, status); pq_update(pq); } @@ -1399,10 +1227,8 @@ static inline void pq_update(struct hfi1_user_sdma_pkt_q *pq) wake_up(&pq->wait); } -static void user_sdma_free_request(struct user_sdma_request *req, bool unpin) +static void user_sdma_free_request(struct user_sdma_request *req) { - int i; - if (!list_empty(&req->txps)) { struct sdma_txreq *t, *p; @@ -1415,21 +1241,6 @@ static void user_sdma_free_request(struct user_sdma_request *req, bool unpin) } } - for (i = 0; i < req->data_iovs; i++) { - struct sdma_mmu_node *node = req->iovs[i].node; - - if (!node) - continue; - - req->iovs[i].node = NULL; - - if (unpin) - hfi1_mmu_rb_remove(req->pq->handler, - &node->rb); - else - atomic_dec(&node->refcount); - } - kfree(req->tids); clear_bit(req->info.comp_idx, req->pq->req_in_use); } @@ -1447,6 +1258,368 @@ static inline void set_comp_state(struct hfi1_user_sdma_pkt_q *pq, idx, state, ret); } +static void unpin_vector_pages(struct mm_struct *mm, struct page **pages, + unsigned int start, unsigned int npages) +{ + hfi1_release_user_pages(mm, pages + start, npages, false); + kfree(pages); +} + +static void free_system_node(struct sdma_mmu_node *node) +{ + if (node->npages) { + unpin_vector_pages(mm_from_sdma_node(node), node->pages, 0, + node->npages); + atomic_sub(node->npages, &node->pq->n_locked); + } + kfree(node); +} + +static inline void acquire_node(struct sdma_mmu_node *node) +{ + atomic_inc(&node->refcount); + WARN_ON(atomic_read(&node->refcount) < 0); +} + +static inline void release_node(struct mmu_rb_handler *handler, + struct sdma_mmu_node *node) +{ + atomic_dec(&node->refcount); + WARN_ON(atomic_read(&node->refcount) < 0); +} + +static struct sdma_mmu_node *find_system_node(struct mmu_rb_handler *handler, + unsigned long start, + unsigned long end) +{ + struct mmu_rb_node *rb_node; + struct sdma_mmu_node *node; + unsigned long flags; + + spin_lock_irqsave(&handler->lock, flags); + rb_node = hfi1_mmu_rb_get_first(handler, start, (end - start)); + if (!rb_node) { + spin_unlock_irqrestore(&handler->lock, flags); + return NULL; + } + node = container_of(rb_node, struct sdma_mmu_node, rb); + acquire_node(node); + spin_unlock_irqrestore(&handler->lock, flags); + + return node; +} + +static int pin_system_pages(struct user_sdma_request *req, + uintptr_t start_address, size_t length, + struct sdma_mmu_node *node, int npages) +{ + struct hfi1_user_sdma_pkt_q *pq = req->pq; + int pinned, cleared; + struct page **pages; + + pages = kcalloc(npages, sizeof(*pages), GFP_KERNEL); + if (!pages) + return -ENOMEM; + +retry: + if (!hfi1_can_pin_pages(pq->dd, current->mm, atomic_read(&pq->n_locked), + npages)) { + SDMA_DBG(req, "Evicting: nlocked %u npages %u", + atomic_read(&pq->n_locked), npages); + cleared = sdma_cache_evict(pq, npages); + if (cleared >= npages) + goto retry; + } + + SDMA_DBG(req, "Acquire user pages start_address %lx node->npages %u npages %u", + start_address, node->npages, npages); + pinned = hfi1_acquire_user_pages(current->mm, start_address, npages, 0, + pages); + + if (pinned < 0) { + kfree(pages); + SDMA_DBG(req, "pinned %d", pinned); + return pinned; + } + if (pinned != npages) { + unpin_vector_pages(current->mm, pages, node->npages, pinned); + SDMA_DBG(req, "npages %u pinned %d", npages, pinned); + return -EFAULT; + } + node->rb.addr = start_address; + node->rb.len = length; + node->pages = pages; + node->npages = npages; + atomic_add(pinned, &pq->n_locked); + SDMA_DBG(req, "done. pinned %d", pinned); + return 0; +} + +static int add_system_pinning(struct user_sdma_request *req, + struct sdma_mmu_node **node_p, + unsigned long start, unsigned long len) + +{ + struct hfi1_user_sdma_pkt_q *pq = req->pq; + struct sdma_mmu_node *node; + int ret; + + node = kzalloc(sizeof(*node), GFP_KERNEL); + if (!node) + return -ENOMEM; + + node->pq = pq; + ret = pin_system_pages(req, start, len, node, PFN_DOWN(len)); + if (ret == 0) { + ret = hfi1_mmu_rb_insert(pq->handler, &node->rb); + if (ret) + free_system_node(node); + else + *node_p = node; + + return ret; + } + + kfree(node); + return ret; +} + +static int get_system_cache_entry(struct user_sdma_request *req, + struct sdma_mmu_node **node_p, + size_t req_start, size_t req_len) +{ + struct hfi1_user_sdma_pkt_q *pq = req->pq; + u64 start = ALIGN_DOWN(req_start, PAGE_SIZE); + u64 end = PFN_ALIGN(req_start + req_len); + struct mmu_rb_handler *handler = pq->handler; + int ret; + + if ((end - start) == 0) { + SDMA_DBG(req, + "Request for empty cache entry req_start %lx req_len %lx start %llx end %llx", + req_start, req_len, start, end); + return -EINVAL; + } + + SDMA_DBG(req, "req_start %lx req_len %lu", req_start, req_len); + + while (1) { + struct sdma_mmu_node *node = + find_system_node(handler, start, end); + u64 prepend_len = 0; + + SDMA_DBG(req, "node %p start %llx end %llu", node, start, end); + if (!node) { + ret = add_system_pinning(req, node_p, start, + end - start); + if (ret == -EEXIST) { + /* + * Another execution context has inserted a + * conficting entry first. + */ + continue; + } + return ret; + } + + if (node->rb.addr <= start) { + /* + * This entry covers at least part of the region. If it doesn't extend + * to the end, then this will be called again for the next segment. + */ + *node_p = node; + return 0; + } + + SDMA_DBG(req, "prepend: node->rb.addr %lx, node->refcount %d", + node->rb.addr, atomic_read(&node->refcount)); + prepend_len = node->rb.addr - start; + + /* + * This node will not be returned, instead a new node + * will be. So release the reference. + */ + release_node(handler, node); + + /* Prepend a node to cover the beginning of the allocation */ + ret = add_system_pinning(req, node_p, start, prepend_len); + if (ret == -EEXIST) { + /* Another execution context has inserted a conficting entry first. */ + continue; + } + return ret; + } +} + +static int add_mapping_to_sdma_packet(struct user_sdma_request *req, + struct user_sdma_txreq *tx, + struct sdma_mmu_node *cache_entry, + size_t start, + size_t from_this_cache_entry) +{ + struct hfi1_user_sdma_pkt_q *pq = req->pq; + unsigned int page_offset; + unsigned int from_this_page; + size_t page_index; + void *ctx; + int ret; + + /* + * Because the cache may be more fragmented than the memory that is being accessed, + * it's not strictly necessary to have a descriptor per cache entry. + */ + + while (from_this_cache_entry) { + page_index = PFN_DOWN(start - cache_entry->rb.addr); + + if (page_index >= cache_entry->npages) { + SDMA_DBG(req, + "Request for page_index %zu >= cache_entry->npages %u", + page_index, cache_entry->npages); + return -EINVAL; + } + + page_offset = start - ALIGN_DOWN(start, PAGE_SIZE); + from_this_page = PAGE_SIZE - page_offset; + + if (from_this_page < from_this_cache_entry) { + ctx = NULL; + } else { + /* + * In the case they are equal the next line has no practical effect, + * but it's better to do a register to register copy than a conditional + * branch. + */ + from_this_page = from_this_cache_entry; + ctx = cache_entry; + } + + ret = sdma_txadd_page(pq->dd, ctx, &tx->txreq, + cache_entry->pages[page_index], + page_offset, from_this_page); + if (ret) { + /* + * When there's a failure, the entire request is freed by + * user_sdma_send_pkts(). + */ + SDMA_DBG(req, + "sdma_txadd_page failed %d page_index %lu page_offset %u from_this_page %u", + ret, page_index, page_offset, from_this_page); + return ret; + } + start += from_this_page; + from_this_cache_entry -= from_this_page; + } + return 0; +} + +static int add_system_iovec_to_sdma_packet(struct user_sdma_request *req, + struct user_sdma_txreq *tx, + struct user_sdma_iovec *iovec, + size_t from_this_iovec) +{ + struct mmu_rb_handler *handler = req->pq->handler; + + while (from_this_iovec > 0) { + struct sdma_mmu_node *cache_entry; + size_t from_this_cache_entry; + size_t start; + int ret; + + start = (uintptr_t)iovec->iov.iov_base + iovec->offset; + ret = get_system_cache_entry(req, &cache_entry, start, + from_this_iovec); + if (ret) { + SDMA_DBG(req, "pin system segment failed %d", ret); + return ret; + } + + from_this_cache_entry = cache_entry->rb.len - (start - cache_entry->rb.addr); + if (from_this_cache_entry > from_this_iovec) + from_this_cache_entry = from_this_iovec; + + ret = add_mapping_to_sdma_packet(req, tx, cache_entry, start, + from_this_cache_entry); + if (ret) { + /* + * We're guaranteed that there will be no descriptor + * completion callback that releases this node + * because only the last descriptor referencing it + * has a context attached, and a failure means the + * last descriptor was never added. + */ + release_node(handler, cache_entry); + SDMA_DBG(req, "add system segment failed %d", ret); + return ret; + } + + iovec->offset += from_this_cache_entry; + from_this_iovec -= from_this_cache_entry; + } + + return 0; +} + +static int add_system_pages_to_sdma_packet(struct user_sdma_request *req, + struct user_sdma_txreq *tx, + struct user_sdma_iovec *iovec, + u32 *pkt_data_remaining) +{ + size_t remaining_to_add = *pkt_data_remaining; + /* + * Walk through iovec entries, ensure the associated pages + * are pinned and mapped, add data to the packet until no more + * data remains to be added. + */ + while (remaining_to_add > 0) { + struct user_sdma_iovec *cur_iovec; + size_t from_this_iovec; + int ret; + + cur_iovec = iovec; + from_this_iovec = iovec->iov.iov_len - iovec->offset; + + if (from_this_iovec > remaining_to_add) { + from_this_iovec = remaining_to_add; + } else { + /* The current iovec entry will be consumed by this pass. */ + req->iov_idx++; + iovec++; + } + + ret = add_system_iovec_to_sdma_packet(req, tx, cur_iovec, + from_this_iovec); + if (ret) + return ret; + + remaining_to_add -= from_this_iovec; + } + *pkt_data_remaining = remaining_to_add; + + return 0; +} + +void system_descriptor_complete(struct hfi1_devdata *dd, + struct sdma_desc *descp) +{ + switch (sdma_mapping_type(descp)) { + case SDMA_MAP_SINGLE: + dma_unmap_single(&dd->pcidev->dev, sdma_mapping_addr(descp), + sdma_mapping_len(descp), DMA_TO_DEVICE); + break; + case SDMA_MAP_PAGE: + dma_unmap_page(&dd->pcidev->dev, sdma_mapping_addr(descp), + sdma_mapping_len(descp), DMA_TO_DEVICE); + break; + } + + if (descp->pinning_ctx) { + struct sdma_mmu_node *node = descp->pinning_ctx; + + release_node(node->rb.handler, node); + } +} + static bool sdma_rb_filter(struct mmu_rb_node *node, unsigned long addr, unsigned long len) { @@ -1493,8 +1666,7 @@ static void sdma_rb_remove(void *arg, struct mmu_rb_node *mnode) struct sdma_mmu_node *node = container_of(mnode, struct sdma_mmu_node, rb); - unpin_sdma_pages(node); - kfree(node); + free_system_node(node); } static int sdma_rb_invalidate(void *arg, struct mmu_rb_node *mnode) diff --git a/drivers/infiniband/hw/hfi1/user_sdma.h b/drivers/infiniband/hw/hfi1/user_sdma.h index ea56eb57e656..a241836371dc 100644 --- a/drivers/infiniband/hw/hfi1/user_sdma.h +++ b/drivers/infiniband/hw/hfi1/user_sdma.h @@ -112,16 +112,11 @@ struct sdma_mmu_node { struct user_sdma_iovec { struct list_head list; struct iovec iov; - /* number of pages in this vector */ - unsigned int npages; - /* array of pinned pages for this vector */ - struct page **pages; /* * offset into the virtual address space of the vector at * which we last left off. */ u64 offset; - struct sdma_mmu_node *node; }; /* evict operation argument */ diff --git a/drivers/infiniband/hw/hfi1/verbs.c b/drivers/infiniband/hw/hfi1/verbs.c index 7f6d7fc7951d..fbdcfecb1768 100644 --- a/drivers/infiniband/hw/hfi1/verbs.c +++ b/drivers/infiniband/hw/hfi1/verbs.c @@ -778,8 +778,8 @@ static int build_verbs_tx_desc( /* add icrc, lt byte, and padding to flit */ if (extra_bytes) - ret = sdma_txadd_daddr(sde->dd, &tx->txreq, - sde->dd->sdma_pad_phys, extra_bytes); + ret = sdma_txadd_daddr(sde->dd, &tx->txreq, sde->dd->sdma_pad_phys, + extra_bytes); bail_txadd: return ret; diff --git a/drivers/infiniband/hw/hfi1/vnic_sdma.c b/drivers/infiniband/hw/hfi1/vnic_sdma.c index c3f0f8d877c3..727eedfba332 100644 --- a/drivers/infiniband/hw/hfi1/vnic_sdma.c +++ b/drivers/infiniband/hw/hfi1/vnic_sdma.c @@ -64,6 +64,7 @@ static noinline int build_vnic_ulp_payload(struct sdma_engine *sde, /* combine physically continuous fragments later? */ ret = sdma_txadd_page(sde->dd, + NULL, &tx->txreq, skb_frag_page(frag), skb_frag_off(frag), From patchwork Fri Apr 7 16:52:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 13205063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18F01C6FD1D for ; Fri, 7 Apr 2023 16:53:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229832AbjDGQxF (ORCPT ); Fri, 7 Apr 2023 12:53:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33684 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229762AbjDGQxE (ORCPT ); Fri, 7 Apr 2023 12:53:04 -0400 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2116.outbound.protection.outlook.com [40.107.102.116]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F28DD55AE for ; Fri, 7 Apr 2023 09:52:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fvzhFOtbeJSExfwMS2qFjXUSnu9VdDg/o10I8FI57g5+WHFjkjd+yImY/8P+n4ERVCSUhzvOUsFSBDNs/ppQW/125rksEeqALo/EXSdijINmabCLvY8/uQUPq7DpC9bJq49/9FJs6rsFwd6EwjFSRrCiykZBxCcx3ZQMMbRUfbT2m8HpSZUk2RLGZamZmPt3f1fYOI12cGOEutJAvo57ebp8YQYNaeTq6EOljUNP+Zgjl4X08AjG2an+OSl08TQ+9tUR3zyarolUD+ehY+Y45vaGdlwJf2V4bpCJcdExP4xE1x/4X1qyCVpegYX5GXDNqV1v3kIqkt0FJS8RQAgsyA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=PhxdTIX9W6foq67chk/9qyJV8tya5ITAeo13v4GUkms=; b=i8uAVVLta0LrvBhjARm5XwExb+rZBLRKuS/JTIzeiSewXDFsM1puGaDcKZ9n+zXp3D4cV28hq/2XJzRBhfcfNEHLiVwj3PW0yfmrNfHKbbssoClWGpK3usS3gv8YeElbtyvwxcUz/wqSulQqBYFlGthlAbjaWuvemVkxtBwGEErFgzjm+LqPrDeUeHbtgHNUJ6ld3P544CTYk4N0/7fmJBmximSh73dozrJ1IgH7lvarxLnViDi3F+Q/sgnUfJansVw/AGNTF9iHUtkDQ+KEPEp3HeTbBFsTw+jvXyQYPtTiopQnYW4Cbf8X/taLNBEeOLUr+9qFCRS5Gjx6sZC5uA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=nvidia.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=PhxdTIX9W6foq67chk/9qyJV8tya5ITAeo13v4GUkms=; b=mWJwppkZr7d0SD3MLBmv49JsHeeKlxvbp6FSjEZw8bhHaObuZjI506u6eMcwhfYwrJjWlPO4guGCltTu8ccy6SVhstUvl4VkpEUk8TE9y0EXmyrVPI/qrcBI9wUg6ToibRqfn3IE/1HHe1t8NlUukHlQjhpSvI3INI01pwNt7O4s4xnl3Bqfcy7mHRzEFMGyUVcD3swYXA9Rx89BuggLPz0/Uednwi6qHawz4natR9bMQeOSkPMwoMIXrJPH9DojNr/T6tC65KzMLCjsG4zsgpo5P3uezRXrOkWqBkaC6f1Kf97F6crrdyk2Z4fN4g2QDVoa4PfXcePk3w2j7myNPA== Received: from MW3PR06CA0016.namprd06.prod.outlook.com (2603:10b6:303:2a::21) by BN7PR01MB3988.prod.exchangelabs.com (2603:10b6:406:89::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.28; Fri, 7 Apr 2023 16:52:51 +0000 Received: from CO1NAM11FT013.eop-nam11.prod.protection.outlook.com (2603:10b6:303:2a:cafe::ae) by MW3PR06CA0016.outlook.office365.com (2603:10b6:303:2a::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.31 via Frontend Transport; Fri, 7 Apr 2023 16:52:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; dkim=none (message not signed) header.d=none;dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-02.cornelisnetworks.com; pr=C Received: from awfm-02.cornelisnetworks.com (208.255.156.42) by CO1NAM11FT013.mail.protection.outlook.com (10.13.174.227) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6298.20 via Frontend Transport; Fri, 7 Apr 2023 16:52:50 +0000 Received: from 252.162.96.66.static.eigbox.net (localhost [127.0.0.1]) by awfm-02.cornelisnetworks.com (8.16.1/8.16.1) with ESMTP id 337GqnVa3027412; Fri, 7 Apr 2023 12:52:49 -0400 Subject: [PATCH for-next 5/5] IB/hfi1: Place struct mmu_rb_handler on cache line start From: Dennis Dalessandro To: jgg@nvidia.com, leonro@nvidia.com Cc: Brendan Cunningham , Patrick Kelsey , linux-rdma@vger.kernel.org Date: Fri, 07 Apr 2023 12:52:49 -0400 Message-ID: <168088636963.3027109.16959757980497822530.stgit@252.162.96.66.static.eigbox.net> In-Reply-To: <168088607365.3027109.2194306496858796762.stgit@252.162.96.66.static.eigbox.net> References: <168088607365.3027109.2194306496858796762.stgit@252.162.96.66.static.eigbox.net> User-Agent: StGit/1.5.dev2+g9ce680a5 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1NAM11FT013:EE_|BN7PR01MB3988:EE_ X-MS-Office365-Filtering-Correlation-Id: b2aef170-d61a-4cba-43f2-08db37888639 X-MS-Exchange-AtpMessageProperties: SA X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iEUzS/Oh9waU0Y1u8DK4dVhxeZPyd+MJyYLsUKg1tJIgimc3R2rbLKDcShZiWIvXeUZ69yIM7TVvV0Eh9BHVDjVdoZj6uBDp0Cogrtz8AvAIk81ucIUfTYM/3H5h20WonhjqOqYM2/94ccjAKzLBCNTntK3ftAuH9qjV+l5wAas+bStABU4yIcwgXDv+LJnYdNTJSW468uWgBWlLyLJ9vOeWsrjWAfAFS6rnWdl4A88+sJJ83n6kb8e/zFHU9sXKP081itC0Trsh90ZoPY5vRhdMd7ym7EI0hkVunY24V6EIVx5Prjh69MaSIXW6kmMY87zHuQGZtbbWsp0X2O3lhSM5VVr/SjLE2mTYD0yVZ1Yv1NYkeGBNFv0zdyLJAPFgSwquWde1wRCoY1T3QMoy4CpyYuR8+7GrPAwzvaG1B4qK1DaaMD5IArUBCWnwsmW0anUG8VgT09izh9A3JaQUquFnmqxbIdIXzhd/FRhkEsxa4yrle3rCtaLtbbEq6NlOFvOaIxjp4UDN4UEvEjdMt74GWVeCjw9S5xhJFKXEDo8YrMRWgobijUhwtAYhrTi5a0mqp8j+yHHgA2LWvmTyYaaA+ajVlyBZl03NAS4ZU31yOZgYa5pHeVRUXHhHWxEvW0WjhOf+xVNR3wxOHD9Hu2BYBtAmmesHDuKpE72OAoanmMrhVYnWSW6Vk8QjFpcUo3/GsoByF1eoXN7E8h3wXSpytS8bguCnKTd5CpIvDcRrmsdiJ7DgQQLAAKHnLUCRyl7B0+JC3DPi8dm+AXRPhlhiRuhGlIpnArdCntjvgWDH5ZW0StJDVrmxOj16w6yNkTZz1AE+0hTMGn3fdaaimg== X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-02.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(39840400004)(396003)(136003)(376002)(451199021)(46966006)(36840700001)(426003)(81166007)(44832011)(40480700001)(5660300002)(356005)(70586007)(8676002)(41300700001)(103116003)(82310400005)(55016003)(70206006)(4326008)(86362001)(8936002)(186003)(36860700001)(54906003)(47076005)(83380400001)(26005)(478600001)(7126003)(336012)(2906002)(9686003)(7696005)(316002)(9916002)(24686002)(36900700001)(102196002);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Apr 2023 16:52:50.8474 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b2aef170-d61a-4cba-43f2-08db37888639 X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-02.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT013.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR01MB3988 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Patrick Kelsey Place struct mmu_rb_handler on cache line start like so: struct mmu_rb_handler *h; void *free_ptr; int ret; free_ptr = kzalloc(sizeof(*h) + cache_line_size() - 1, GFP_KERNEL); if (!free_ptr) return -ENOMEM; h = PTR_ALIGN(free_ptr, cache_line_size()); Additionally, move struct mmu_rb_handler fields "root" and "ops_args" to start after the next cacheline using the "____cacheline_aligned_in_smp" annotation. Allocating an additional cache_line_size() - 1 bytes to place struct mmu_rb_handler on a cache line start does increase memory consumption. However, few struct mmu_rb_handler are created when hfi1 is in use. As mmu_rb_handler->root and mmu_rb_handler->ops_args are accessed frequently, the advantage of having them both within a cache line is expected to outweigh the disadvantage of the additional memory consumption per struct mmu_rb_handler. Signed-off-by: Brendan Cunningham Signed-off-by: Patrick Kelsey Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/mmu_rb.c | 11 +++++++---- drivers/infiniband/hw/hfi1/mmu_rb.h | 14 ++++++++++++-- 2 files changed, 19 insertions(+), 6 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/mmu_rb.c b/drivers/infiniband/hw/hfi1/mmu_rb.c index 71b9ac018887..1cea8b0c78e0 100644 --- a/drivers/infiniband/hw/hfi1/mmu_rb.c +++ b/drivers/infiniband/hw/hfi1/mmu_rb.c @@ -46,12 +46,14 @@ int hfi1_mmu_rb_register(void *ops_arg, struct mmu_rb_handler **handler) { struct mmu_rb_handler *h; + void *free_ptr; int ret; - h = kzalloc(sizeof(*h), GFP_KERNEL); - if (!h) + free_ptr = kzalloc(sizeof(*h) + cache_line_size() - 1, GFP_KERNEL); + if (!free_ptr) return -ENOMEM; + h = PTR_ALIGN(free_ptr, cache_line_size()); h->root = RB_ROOT_CACHED; h->ops = ops; h->ops_arg = ops_arg; @@ -62,10 +64,11 @@ int hfi1_mmu_rb_register(void *ops_arg, INIT_LIST_HEAD(&h->del_list); INIT_LIST_HEAD(&h->lru_list); h->wq = wq; + h->free_ptr = free_ptr; ret = mmu_notifier_register(&h->mn, current->mm); if (ret) { - kfree(h); + kfree(free_ptr); return ret; } @@ -108,7 +111,7 @@ void hfi1_mmu_rb_unregister(struct mmu_rb_handler *handler) /* Now the mm may be freed. */ mmdrop(handler->mn.mm); - kfree(handler); + kfree(handler->free_ptr); } int hfi1_mmu_rb_insert(struct mmu_rb_handler *handler, diff --git a/drivers/infiniband/hw/hfi1/mmu_rb.h b/drivers/infiniband/hw/hfi1/mmu_rb.h index ed75acdb7b83..c4da064188c9 100644 --- a/drivers/infiniband/hw/hfi1/mmu_rb.h +++ b/drivers/infiniband/hw/hfi1/mmu_rb.h @@ -33,15 +33,25 @@ struct mmu_rb_ops { }; struct mmu_rb_handler { + /* + * struct mmu_notifier is 56 bytes, and spinlock_t is 4 bytes, so + * they fit together in one cache line. mn is relatively rarely + * accessed, so co-locating the spinlock with it achieves much of + * the cacheline contention reduction of giving the spinlock its own + * cacheline without the overhead of doing so. + */ struct mmu_notifier mn; - struct rb_root_cached root; - void *ops_arg; spinlock_t lock; /* protect the RB tree */ + + /* Begin on a new cachline boundary here */ + struct rb_root_cached root ____cacheline_aligned_in_smp; + void *ops_arg; struct mmu_rb_ops *ops; struct list_head lru_list; struct work_struct del_work; struct list_head del_list; struct workqueue_struct *wq; + void *free_ptr; }; int hfi1_mmu_rb_register(void *ops_arg,