From patchwork Tue Jul 6 17:23:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 12361005 X-Patchwork-Delegate: jgg@ziepe.ca Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 733ACC07E96 for ; Tue, 6 Jul 2021 17:23:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4EC9B61C5F for ; Tue, 6 Jul 2021 17:23:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230400AbhGFR03 (ORCPT ); Tue, 6 Jul 2021 13:26:29 -0400 Received: from mail-bn8nam08on2108.outbound.protection.outlook.com ([40.107.100.108]:3296 "EHLO NAM04-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S230408AbhGFR02 (ORCPT ); Tue, 6 Jul 2021 13:26:28 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RrlZcmUoFUys1mGnb1HLGdhtFLtPsBMCWiufwjZ6LhckiinSHA8gd/isig5zDNWJCaZC4u3Eng+e+io3oTbO5xD82aU4AR4BJlJkgNfnNck6AClT1wQqXjP5uJ+O8s4Y3mtQLKfTtfzZ5Q0V+/Ei4nY3clvcnxfKxO8nXP6yHBWXZAzpR66R5+bcD1lojRzLTsad6PYuDSAEHcoDpoCfeXZTrsm1EU3eOCA/adveOJVnffm7z9/bu5jC1kDrvJuBZ/WSZqJ9peAjYLQOyShoZtR024wVDOEaLqH5bHLkH9uI4RE0TMnjKxjJlzB/zHIrasQ3FU7SPnOZC6OTuoAGEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IKfEK7L2nNqi9f2OMG0ucDgYjgL/IB1HuU9jWgK7IRE=; b=X3zG8LH/3jvny9sDigQ8lbjptlssS2zBAv2WH3L13MV0uxkVxXrmw8DvveJmGjQsuzZ8O1B/gk/qWGeKDsBB024OetVuMHOQ6s5nNQ+KZt9zNLybWpzamgPuOj9ow7W11Z8R7M047MIT8hdplwNYpiEk6qxLfljLJG+q8rPtXQC6PhnFAAP9JjUz+dWypc4en8wnw7gAp/l3yh57xZgdcLnp7zYH8QUXwtxs3piBY0d4gR2M1b7+eIHMQk2KiLZlMK/dn9f5yDj7jjOTF/aeenG8qRVjIjzvnNUOlwIH5FqfhHdxNx2w/GYohrYaOIB1x6P1Wh9I63Y9FI0w/JaZPA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=cornelisnetworks.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=IKfEK7L2nNqi9f2OMG0ucDgYjgL/IB1HuU9jWgK7IRE=; b=XSMN0sL2zmCV+Dm+yX7x7+5u0J+TFT8I6vrb0K12c1WE4wxjypgrXBdSibSGeeJNdPEZufybd6rMfLVuoohMiT0cWaViLdzIObwrMWgkSo0wd4Y1GbWVy8t6P161fQ4aiyPWHc+iH5FnDpmRANGWA7bc4oTE0r/GKMe4/Ub7jgQtEtmx6BW8mOfQfP8+TdYD0REg9kaqbiX9t8gurKb/B3w74nE5MAqVlpc91z6RxhduIH9b3B0k1BRxz/5zJWzxjGgTP0F5im40l3LkkztDJgVs5yRNU7oXl6AjOHFpieR9jKAvtE0INFs0/TbvO/V7nrrDG3ZlD8YCL5GC7/2h0g== Received: from DM5PR10CA0006.namprd10.prod.outlook.com (2603:10b6:4:2::16) by MW4PR01MB6275.prod.exchangelabs.com (2603:10b6:303:75::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4242.21; Tue, 6 Jul 2021 17:23:46 +0000 Received: from DM6NAM11FT010.eop-nam11.prod.protection.outlook.com (2603:10b6:4:2:cafe::3) by DM5PR10CA0006.outlook.office365.com (2603:10b6:4:2::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4308.20 via Frontend Transport; Tue, 6 Jul 2021 17:23:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; cornelisnetworks.com; dkim=none (message not signed) header.d=none;cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-01.cornelisnetworks.com; Received: from awfm-01.cornelisnetworks.com (208.255.156.42) by DM6NAM11FT010.mail.protection.outlook.com (10.13.172.222) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22 via Frontend Transport; Tue, 6 Jul 2021 17:23:46 +0000 Received: from awfm-01.cornelisnetworks.com (localhost [127.0.0.1]) by awfm-01.cornelisnetworks.com (8.14.7/8.14.7) with ESMTP id 166HNjQl058971; Tue, 6 Jul 2021 13:23:45 -0400 Subject: [PATCH for-rc or next 1/2] IB/hfi1: Indicate DMA wait when txq is queued for wakeup From: Dennis Dalessandro To: jgg@ziepe.ca, dledford@redhat.com Cc: linux-rdma@vger.kernel.org, Mike Marciniszyn Date: Tue, 06 Jul 2021 13:23:45 -0400 Message-ID: <20210706172345.49902.10221.stgit@awfm-01.cornelisnetworks.com> In-Reply-To: <20210706171942.49902.72880.stgit@awfm-01.cornelisnetworks.com> References: <20210706171942.49902.72880.stgit@awfm-01.cornelisnetworks.com> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: aa50bc8b-8094-499c-c78d-08d940a2cfa9 X-MS-TrafficTypeDiagnostic: MW4PR01MB6275: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3968; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iv1NKRumdoPPZOBQbbGYGbmf/9x6bBqVGmSWEyGSXmH3cDpNeselYExbYgwZyWVDqVlou7OK1XP2PoVfbKgL0bnw4Glh6dS8yrZ5bfXqFLYQb2nJFoQO0EK4m8UHvOFu848p0UtSA0EuTRjullmum//9APtZxYm0DJnZD1VxtFeL5mFBlyY/5x/9y9yHnbEXwVC/fbPhoWkKXbRg25FerHRzbcdG4cr3jybZyAGd5NVvB/k1VQgoJStD7/GaXiuhGjUpBi+I6kHfPfSweNRJbnn0gDioWtdN3PCaEx3ftbBEikfgWqJ2DDoKAzG1PjqwXpW5wYy5nkBbKx3W3V5Ld6oKcZiISQdVsjhy6uQN+ZC+OHW2Ik6tFlOdRwb2iIV9q9aLC4I+2oYc8gv1KliI5b57ptJCApdALqg5zrai2cOwTJltHVAOCDkhEx4wycs/7U2mZWbLdxYKBQFOrusukkfjFfJe3Yf6RaYv4E4ZfXOI5ZfK2c3jg0AMseiOdw3t6CdHegunV8d3wYk4FPYSwnUlFJhlO0+v1x6jzq2CuQ//+smGxFJGJpQ4PqQXhotD5wvRTyEftZCDV9YesrFoidQNw26Sl5Uy2MzIfaRml3V0OA8W5eRN6r1MHtxFP9W/oDSsn+EL/NRl1/JuLfKs+A48xPthX6aJGTMaT51OKck43l4x0ag/eyC9Z+ILQdFX97TPtRbUuUSyFk8VSbSFaQ3OLeNVEY+/6ph9cvQK3qnOk6XQ5kggOUfNHZWiKn9OwHv00RtbDP3TXs71HJwi8g== X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-01.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(396003)(39840400004)(136003)(376002)(346002)(46966006)(36840700001)(7696005)(1076003)(107886003)(70206006)(44832011)(70586007)(8676002)(26005)(103116003)(5660300002)(47076005)(36906005)(55016002)(8936002)(83380400001)(316002)(186003)(426003)(336012)(4326008)(34020700004)(36860700001)(7126003)(2906002)(86362001)(81166007)(82310400003)(478600001)(356005)(36900700001);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jul 2021 17:23:46.1816 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: aa50bc8b-8094-499c-c78d-08d940a2cfa9 X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-01.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT010.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR01MB6275 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mike Marciniszyn There is no counter for dmawait in AIP, which hampers debugging performance issues. Add the counter increment when the txq is queued. Fixes: d99dc602e2a5 ("IB/hfi1: Add functions to transmit datagram ipoib packets") Fixes: c4cf5688ea69 ("IB/hfi1: Indicate DMA wait when txq is queued for wakeup") Signed-off-by: Mike Marciniszyn Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/ipoib_tx.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/infiniband/hw/hfi1/ipoib_tx.c b/drivers/infiniband/hw/hfi1/ipoib_tx.c index 993f983..e74ddbe 100644 --- a/drivers/infiniband/hw/hfi1/ipoib_tx.c +++ b/drivers/infiniband/hw/hfi1/ipoib_tx.c @@ -644,10 +644,13 @@ static int hfi1_ipoib_sdma_sleep(struct sdma_engine *sde, /* came from non-list submit */ list_add_tail(&txreq->list, &txq->tx_list); if (list_empty(&txq->wait.list)) { + struct hfi1_ibport *ibp = &sde->ppd->ibport_data; + if (!atomic_xchg(&txq->no_desc, 1)) { trace_hfi1_txq_queued(txq); hfi1_ipoib_stop_txq(txq); } + ibp->rvp.n_dmawait++; iowait_queue(pkts_sent, wait->iow, &sde->dmawait); } From patchwork Tue Jul 6 17:23:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dennis Dalessandro X-Patchwork-Id: 12361007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FC0BC07E96 for ; Tue, 6 Jul 2021 17:23:56 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1750061C5D for ; Tue, 6 Jul 2021 17:23:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230408AbhGFR0e (ORCPT ); Tue, 6 Jul 2021 13:26:34 -0400 Received: from mail-mw2nam08on2120.outbound.protection.outlook.com ([40.107.101.120]:2783 "EHLO NAM04-MW2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S230348AbhGFR0d (ORCPT ); Tue, 6 Jul 2021 13:26:33 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=c7uyAru/7JTUT03f8Cm7d5B36DpGzh8Nh/LOsMAqa+PoLarzz5IjIsfngjG8eOui8JAOBbsQSZbpJ3J2jHyUPLsXDCJ2/7qjAhXH67YGD0X4SWD1+PuWO7OmOr0BkLaZNyK1YfQhS160lEsznISJOTYWMIvN92aQnBnwlb1tKe4igkYWaZxka/QiUzggurH/vkdNyP9VVIgMJMgXgww1puciJ9qFlYFIk6lVxU/E237ceCOG2IBmsw17B1wUVF27pr4fpQY1xBQbn6/BMSfqp+RkxfiwjRR2VwaCERySvyrtT/67VMz89J3Q134nHxKlf9mh6Wb8E3MPoaaoOpoooA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3tx9KDnNCUZzWOtiC3xOouBbZEBcgEIKnGIIlp6M4HY=; b=YE+r/K7CEwae5rGuw3KkAQQ6Aj9yElnbCxX6seCEpldmrvjI8KTjQIZ+Ww5zvQk907LaMREpP3I6Jm5pBdCxAzX3+T/WmbWctXM60wZeIyZDJCoOfcEfqBZcc4USJIqKi5cmAw4PAATwysVOpW4NR72uvULuKwHxGjwOQ72AHcCmB8X85uS23oDLN6tPiufRJcfAtFCgUFvSO27NvZ8LqBHzgHaY8AJjIDuv7+8rxTr3J8+03nZcMQD6E1IbaDNSVkK+G+oBi48sUwVq7KSeonrV79VIjzz6b/IQoSUpfq0QO4X8iIu3A4Z/5gVBXRnxWuWDxfqO4NfODhwBlz4UOw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 208.255.156.42) smtp.rcpttodomain=cornelisnetworks.com smtp.mailfrom=cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cornelisnetworks.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3tx9KDnNCUZzWOtiC3xOouBbZEBcgEIKnGIIlp6M4HY=; b=kikZ7a2fmOQn2sKYwFebNw/WclkFp5wqaT8y5MJpjVpzKa18/PvEfdQsMIAvP2+BAE1+eFBv6Wvz5pCROipmhESosj8bHLha4FqcGzioknwjGuaQLETmauGK5A75ArLuoNAWn1lCtfxExnUfJ4rLhsYo3iCxobKTIXqqdKsK77xqehM6aTIo01S98DeAzzeX8czxtOiSKqeLZ0rrNt05DjOqp5RrK5dBgJ/V/z8brDf2yPphot6JtP1FzMufWfCpiYg3uz6ITB8+RM8mLPCYngd552qJLFQgC14dVWGaiW4Jrh2kmRVN/hIehtMrVL5c6005gUUqLR+nt3SoqnzFhA== Received: from DM3PR08CA0014.namprd08.prod.outlook.com (2603:10b6:0:52::24) by CY1PR01MB1963.prod.exchangelabs.com (2a01:111:e400:c618::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.23; Tue, 6 Jul 2021 17:23:51 +0000 Received: from DM6NAM11FT019.eop-nam11.prod.protection.outlook.com (2603:10b6:0:52:cafe::a1) by DM3PR08CA0014.outlook.office365.com (2603:10b6:0:52::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.23 via Frontend Transport; Tue, 6 Jul 2021 17:23:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 208.255.156.42) smtp.mailfrom=cornelisnetworks.com; cornelisnetworks.com; dkim=none (message not signed) header.d=none;cornelisnetworks.com; dmarc=bestguesspass action=none header.from=cornelisnetworks.com; Received-SPF: Pass (protection.outlook.com: domain of cornelisnetworks.com designates 208.255.156.42 as permitted sender) receiver=protection.outlook.com; client-ip=208.255.156.42; helo=awfm-01.cornelisnetworks.com; Received: from awfm-01.cornelisnetworks.com (208.255.156.42) by DM6NAM11FT019.mail.protection.outlook.com (10.13.172.172) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4287.22 via Frontend Transport; Tue, 6 Jul 2021 17:23:51 +0000 Received: from awfm-01.cornelisnetworks.com (localhost [127.0.0.1]) by awfm-01.cornelisnetworks.com (8.14.7/8.14.7) with ESMTP id 166HNowK059137; Tue, 6 Jul 2021 13:23:50 -0400 Subject: [PATCH for-rc or next 2/2] IB/hfi1: Adjust pkey entry in index 0 From: Dennis Dalessandro To: jgg@ziepe.ca, dledford@redhat.com Cc: linux-rdma@vger.kernel.org, Mike Marciniszyn Date: Tue, 06 Jul 2021 13:23:50 -0400 Message-ID: <20210706172350.49902.57258.stgit@awfm-01.cornelisnetworks.com> In-Reply-To: <20210706171942.49902.72880.stgit@awfm-01.cornelisnetworks.com> References: <20210706171942.49902.72880.stgit@awfm-01.cornelisnetworks.com> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8440bc57-e7bc-40a0-4609-08d940a2d2c3 X-MS-TrafficTypeDiagnostic: CY1PR01MB1963: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +wKLLpUKOrV/ZSenwTzCRUgxcavEz76SoJDTpG/L8TTbiK3clKHBG1G7arMn9+kz8vPMPTZE8MAAgPpBbI4sjoYDI4rC1Ah5PqI/f7XNaFlBFfcQhElAHrBzM4J9tqEKaRDbQpU/9/uhb1YLzkXPN1XdEHrbcZhm0bLn15SCJi+fK1ayeC8CiNSzx6kRbz6HvoDb2gOOiJ4V7t0ow6QI+NQlzO+d1vaWMy7Q1sT/C+PtB9orD85nXLA2qyXaxKrGESntnG8uH//vCzTVDzxK6SWMjgt/uu+Cxbdp1jvvexVqnWxl9chKSSmTPXl7CvEDX9szPHhhvEJb0B5Y5hkiNkKXVbkaauhQSPD5HO3uk6ZVJ9+sbBp76YWPWUOzMmD8DHDoLb4rMOvujtPdJK/Bm0t3Y1x3WBcTCE04MeR3Cp3oa5/3SJqUsTXIj38FxiAsEOpNh4kM539jB96UBz0M9hENffl2Pzom7L3aSg1240wu/flwmNhwViq5Tz+V0WzQER9NGd4LaH9Od3gU0Y/DwczDxmUnbV9iq1eYScRVC/hc7SmW9rHKlcq7ANR1Te48Ch0Fb3wNKwwhfjVM+l3q6yTkJS+sq+mxTcmFhc/FcJwchhpzOBxAqLTWnSSdUC8p57RH3ssRCawrfkrS/wUCBByJXCofwB2NrbZFM8pIVb+4fWxlwNuqUD5i4Me/EQduN5K24Vn8Vf0oi1MJY5t4ekFe1yjfeNj2bU21nt67D9fOETTUoDU12KvBYQCPKMMeztGYHNPyPrn0HR7A7Uji7g== X-Forefront-Antispam-Report: CIP:208.255.156.42;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:awfm-01.cornelisnetworks.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(4636009)(39840400004)(136003)(396003)(346002)(376002)(46966006)(36840700001)(47076005)(478600001)(1076003)(55016002)(36906005)(107886003)(82310400003)(34020700004)(336012)(426003)(4326008)(316002)(7126003)(36860700001)(8676002)(81166007)(7696005)(186003)(70206006)(8936002)(70586007)(26005)(356005)(5660300002)(2906002)(83380400001)(103116003)(86362001)(44832011)(36900700001);DIR:OUT;SFP:1102; X-OriginatorOrg: cornelisnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jul 2021 17:23:51.3778 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8440bc57-e7bc-40a0-4609-08d940a2d2c3 X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a;Ip=[208.255.156.42];Helo=[awfm-01.cornelisnetworks.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT019.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR01MB1963 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Mike Marciniszyn It is possible for the primary IPoIB network device associated with any RDMA device to fail to join certain multicast groups preventing IPv6 neighbor discovery and possibly other network ULPs from working correctly. The IPv4 broadcast group is not affected as the IPoIB network device handles joining that multicast group directly. This is because the primary IPoIB network device uses the pkey at ndex 0 in the associated RDMA device's pkey table. Anytime the pkey value of index 0 changes, the primary IPoIB network device automatically modifies it's broadcast address (i.e. /sys/class/net/[ib0]/broadcast), since the broadcast address includes the pkey value, and then bounces carrier. This includes initial pkey assignment, such as when the pkey at index 0 transitions from the opa default of invalid (0x0000) to some value such as the OPA default pkey for Virtual Fabric 0: 0x8001 or when the fabric manager is restarted with a configuration change causing the pkey at index 0 to change. Many network ULPs are not sensitive to the carrier bounce and are not expecting the broadcast address to change including the linux IPv6 stack. This problem does not affect IPoIB child network devices as their pkey value is constant for all time. To mitigate this issue, change the default pkey in at index 0 to 0x8001 to cover the predominant case and avoid issues as ipoib comes up and the FM sweeps. At some point, ipoib multicast support should automatically fix non-broadcast addresses as it does with the primary broadcast address. Fixes: 7724105686e7 ("IB/hfi1: add driver files") Suggested-by: Josh Collier Signed-off-by: Mike Marciniszyn Signed-off-by: Dennis Dalessandro --- drivers/infiniband/hw/hfi1/init.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/init.c b/drivers/infiniband/hw/hfi1/init.c index 0986aa0..34106e5 100644 --- a/drivers/infiniband/hw/hfi1/init.c +++ b/drivers/infiniband/hw/hfi1/init.c @@ -650,12 +650,7 @@ void hfi1_init_pportdata(struct pci_dev *pdev, struct hfi1_pportdata *ppd, ppd->pkeys[default_pkey_idx] = DEFAULT_P_KEY; ppd->part_enforce |= HFI1_PART_ENFORCE_IN; - - if (loopback) { - dd_dev_err(dd, "Faking data partition 0x8001 in idx %u\n", - !default_pkey_idx); - ppd->pkeys[!default_pkey_idx] = 0x8001; - } + ppd->pkeys[0] = 0x8001; INIT_WORK(&ppd->link_vc_work, handle_verify_cap); INIT_WORK(&ppd->link_up_work, handle_link_up);