From patchwork Thu Jun 6 07:09:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kelvin Cao X-Patchwork-Id: 10978755 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A7E0913AD for ; Thu, 6 Jun 2019 07:09:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9589328390 for ; Thu, 6 Jun 2019 07:09:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 86DEB2875A; Thu, 6 Jun 2019 07:09:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1DA9B28390 for ; Thu, 6 Jun 2019 07:09:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726752AbfFFHJy (ORCPT ); Thu, 6 Jun 2019 03:09:54 -0400 Received: from esa2.microchip.iphmx.com ([68.232.149.84]:30682 "EHLO esa2.microchip.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725782AbfFFHJx (ORCPT ); Thu, 6 Jun 2019 03:09:53 -0400 Received-SPF: SoftFail (esa2.microchip.iphmx.com: domain of kelvin.cao@microchip.com is inclined to not designate 208.19.100.22 as permitted sender) identity=mailfrom; client-ip=208.19.100.22; receiver=esa2.microchip.iphmx.com; envelope-from="kelvin.cao@microchip.com"; x-sender="kelvin.cao@microchip.com"; x-conformance=spf_only; x-record-type="v=spf1"; x-record-text="v=spf1 mx a:ushub1.microchip.com a:smtpout.microchip.com a:mx1.microchip.iphmx.com a:mx2.microchip.iphmx.com include:servers.mcsv.net include:mktomail.com include:spf.protection.outlook.com ~all" Received-SPF: None (esa2.microchip.iphmx.com: no sender authenticity information available from domain of postmaster@smtp.microsemi.com) identity=helo; client-ip=208.19.100.22; receiver=esa2.microchip.iphmx.com; envelope-from="kelvin.cao@microchip.com"; x-sender="postmaster@smtp.microsemi.com"; x-conformance=spf_only Authentication-Results: esa2.microchip.iphmx.com; dkim=none (message not signed) header.i=none; spf=SoftFail smtp.mailfrom=kelvin.cao@microchip.com; spf=None smtp.helo=postmaster@smtp.microsemi.com; dmarc=fail (p=none dis=none) d=microchip.com X-IronPort-AV: E=Sophos;i="5.63,558,1557212400"; d="scan'208";a="36174407" Received: from unknown (HELO smtp.microsemi.com) ([208.19.100.22]) by esa2.microchip.iphmx.com with ESMTP/TLS/AES256-SHA256; 06 Jun 2019 00:09:52 -0700 Received: from AVMBX3.microsemi.net (10.100.34.33) by AVMBX2.microsemi.net (10.100.34.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 6 Jun 2019 00:09:52 -0700 Received: from AVMBX3.microsemi.net (10.100.34.33) by AVMBX3.microsemi.net (10.100.34.33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 6 Jun 2019 00:09:51 -0700 Received: from NTB-Peer.microsemi.net (10.188.116.183) by avmbx3.microsemi.net (10.100.34.33) with Microsoft SMTP Server id 15.1.1713.5 via Frontend Transport; Thu, 6 Jun 2019 00:09:48 -0700 From: Kelvin Cao To: , , , , , , , CC: , Subject: [PATCH 1/3] ntb_hw_switchtec: Remove redundant steps of switchtec_ntb_reinit_peer() function Date: Thu, 6 Jun 2019 15:09:42 +0800 Message-ID: <1559804984-24698-2-git-send-email-kelvin.cao@microchip.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559804984-24698-1-git-send-email-kelvin.cao@microchip.com> References: <1559804984-24698-1-git-send-email-kelvin.cao@microchip.com> MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Joey Zhang When a re-initialization is caused by a link event, the driver will re-setup the shared memory window. But at that time, the shared memory is still valid, and it's unnecessary to free, reallocate and then initialize it again. We only need to reconfigure the hardware registers. Remove the redundant steps from switchtec_ntb_reinit_peer() function. Signed-off-by: Joey Zhang Signed-off-by: Wesley Sheng Reviewed-by: Logan Gunthorpe --- drivers/ntb/hw/mscc/ntb_hw_switchtec.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/drivers/ntb/hw/mscc/ntb_hw_switchtec.c b/drivers/ntb/hw/mscc/ntb_hw_switchtec.c index d905d36..947ed0b 100644 --- a/drivers/ntb/hw/mscc/ntb_hw_switchtec.c +++ b/drivers/ntb/hw/mscc/ntb_hw_switchtec.c @@ -1457,10 +1457,13 @@ static void switchtec_ntb_deinit_db_msg_irq(struct switchtec_ntb *sndev) static int switchtec_ntb_reinit_peer(struct switchtec_ntb *sndev) { - dev_info(&sndev->stdev->dev, "peer reinitialized\n"); - switchtec_ntb_deinit_shared_mw(sndev); - switchtec_ntb_init_mw(sndev); - return switchtec_ntb_init_shared_mw(sndev); + int rc; + + dev_info(&sndev->stdev->dev, "reinitialize shared memory window\n"); + rc = config_rsvd_lut_win(sndev, sndev->mmio_peer_ctrl, 0, + sndev->self_partition, + sndev->self_shared_dma); + return rc; } static int switchtec_ntb_add(struct device *dev, From patchwork Thu Jun 6 07:09:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kelvin Cao X-Patchwork-Id: 10978757 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6B64713AD for ; Thu, 6 Jun 2019 07:10:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 525CD28390 for ; Thu, 6 Jun 2019 07:10:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 432BF2875A; Thu, 6 Jun 2019 07:10:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E99A928390 for ; Thu, 6 Jun 2019 07:10:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726793AbfFFHJ4 (ORCPT ); Thu, 6 Jun 2019 03:09:56 -0400 Received: from esa6.microchip.iphmx.com ([216.71.154.253]:25148 "EHLO esa6.microchip.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725782AbfFFHJ4 (ORCPT ); Thu, 6 Jun 2019 03:09:56 -0400 Received-SPF: SoftFail (esa6.microchip.iphmx.com: domain of kelvin.cao@microchip.com is inclined to not designate 208.19.100.22 as permitted sender) identity=mailfrom; client-ip=208.19.100.22; receiver=esa6.microchip.iphmx.com; envelope-from="kelvin.cao@microchip.com"; x-sender="kelvin.cao@microchip.com"; x-conformance=spf_only; x-record-type="v=spf1"; x-record-text="v=spf1 mx a:ushub1.microchip.com a:smtpout.microchip.com a:mx1.microchip.iphmx.com a:mx2.microchip.iphmx.com include:servers.mcsv.net include:mktomail.com include:spf.protection.outlook.com ~all" Received-SPF: None (esa6.microchip.iphmx.com: no sender authenticity information available from domain of postmaster@smtp.microsemi.com) identity=helo; client-ip=208.19.100.22; receiver=esa6.microchip.iphmx.com; envelope-from="kelvin.cao@microchip.com"; x-sender="postmaster@smtp.microsemi.com"; x-conformance=spf_only Authentication-Results: esa6.microchip.iphmx.com; dkim=none (message not signed) header.i=none; spf=SoftFail smtp.mailfrom=kelvin.cao@microchip.com; spf=None smtp.helo=postmaster@smtp.microsemi.com; dmarc=fail (p=none dis=none) d=microchip.com X-IronPort-AV: E=Sophos;i="5.63,558,1557212400"; d="scan'208";a="33318222" Received: from unknown (HELO smtp.microsemi.com) ([208.19.100.22]) by esa6.microchip.iphmx.com with ESMTP/TLS/AES256-SHA256; 06 Jun 2019 00:09:56 -0700 Received: from AVMBX3.microsemi.net (10.100.34.33) by AVMBX2.microsemi.net (10.100.34.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 6 Jun 2019 00:09:55 -0700 Received: from AVMBX3.microsemi.net (10.100.34.33) by AVMBX3.microsemi.net (10.100.34.33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 6 Jun 2019 00:09:54 -0700 Received: from NTB-Peer.microsemi.net (10.188.116.183) by avmbx3.microsemi.net (10.100.34.33) with Microsoft SMTP Server id 15.1.1713.5 via Frontend Transport; Thu, 6 Jun 2019 00:09:52 -0700 From: Kelvin Cao To: , , , , , , , CC: , Subject: [PATCH 2/3] ntb_hw_switchtec: Skip unnecessary re-setup of shared memory window for crosslink case Date: Thu, 6 Jun 2019 15:09:43 +0800 Message-ID: <1559804984-24698-3-git-send-email-kelvin.cao@microchip.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559804984-24698-1-git-send-email-kelvin.cao@microchip.com> References: <1559804984-24698-1-git-send-email-kelvin.cao@microchip.com> MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Wesley Sheng In case of NTB crosslink topology, the setting of shared memory window in the virtual partition doesn't reset on peer's reboot. So skip the unnecessary re-setup of shared memory window for that case. Signed-off-by: Wesley Sheng --- drivers/ntb/hw/mscc/ntb_hw_switchtec.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/ntb/hw/mscc/ntb_hw_switchtec.c b/drivers/ntb/hw/mscc/ntb_hw_switchtec.c index 947ed0b..6cf15c18 100644 --- a/drivers/ntb/hw/mscc/ntb_hw_switchtec.c +++ b/drivers/ntb/hw/mscc/ntb_hw_switchtec.c @@ -1459,6 +1459,9 @@ static int switchtec_ntb_reinit_peer(struct switchtec_ntb *sndev) { int rc; + if (crosslink_is_enabled(sndev)) + return 0; + dev_info(&sndev->stdev->dev, "reinitialize shared memory window\n"); rc = config_rsvd_lut_win(sndev, sndev->mmio_peer_ctrl, 0, sndev->self_partition, From patchwork Thu Jun 6 07:09:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kelvin Cao X-Patchwork-Id: 10978759 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 40B6513AD for ; Thu, 6 Jun 2019 07:10:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2E27628390 for ; Thu, 6 Jun 2019 07:10:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1FB562875A; Thu, 6 Jun 2019 07:10:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 96AF528390 for ; Thu, 6 Jun 2019 07:10:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726817AbfFFHKB (ORCPT ); Thu, 6 Jun 2019 03:10:01 -0400 Received: from esa2.microchip.iphmx.com ([68.232.149.84]:30692 "EHLO esa2.microchip.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725782AbfFFHKA (ORCPT ); Thu, 6 Jun 2019 03:10:00 -0400 Received-SPF: SoftFail (esa2.microchip.iphmx.com: domain of kelvin.cao@microchip.com is inclined to not designate 208.19.100.22 as permitted sender) identity=mailfrom; client-ip=208.19.100.22; receiver=esa2.microchip.iphmx.com; envelope-from="kelvin.cao@microchip.com"; x-sender="kelvin.cao@microchip.com"; x-conformance=spf_only; x-record-type="v=spf1"; x-record-text="v=spf1 mx a:ushub1.microchip.com a:smtpout.microchip.com a:mx1.microchip.iphmx.com a:mx2.microchip.iphmx.com include:servers.mcsv.net include:mktomail.com include:spf.protection.outlook.com ~all" Received-SPF: None (esa2.microchip.iphmx.com: no sender authenticity information available from domain of postmaster@smtp.microsemi.com) identity=helo; client-ip=208.19.100.22; receiver=esa2.microchip.iphmx.com; envelope-from="kelvin.cao@microchip.com"; x-sender="postmaster@smtp.microsemi.com"; x-conformance=spf_only Authentication-Results: esa2.microchip.iphmx.com; dkim=none (message not signed) header.i=none; spf=SoftFail smtp.mailfrom=kelvin.cao@microchip.com; spf=None smtp.helo=postmaster@smtp.microsemi.com; dmarc=fail (p=none dis=none) d=microchip.com X-IronPort-AV: E=Sophos;i="5.63,558,1557212400"; d="scan'208";a="36174409" Received: from unknown (HELO smtp.microsemi.com) ([208.19.100.22]) by esa2.microchip.iphmx.com with ESMTP/TLS/AES256-SHA256; 06 Jun 2019 00:09:59 -0700 Received: from AVMBX3.microsemi.net (10.100.34.33) by AVMBX2.microsemi.net (10.100.34.32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 6 Jun 2019 00:09:58 -0700 Received: from AVMBX3.microsemi.net (10.100.34.33) by AVMBX3.microsemi.net (10.100.34.33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Thu, 6 Jun 2019 00:09:58 -0700 Received: from NTB-Peer.microsemi.net (10.188.116.183) by avmbx3.microsemi.net (10.100.34.33) with Microsoft SMTP Server id 15.1.1713.5 via Frontend Transport; Thu, 6 Jun 2019 00:09:55 -0700 From: Kelvin Cao To: , , , , , , , CC: , Subject: [PATCH 3/3] ntb_hw_switchtec: Fix setup MW with failure bug Date: Thu, 6 Jun 2019 15:09:44 +0800 Message-ID: <1559804984-24698-4-git-send-email-kelvin.cao@microchip.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559804984-24698-1-git-send-email-kelvin.cao@microchip.com> References: <1559804984-24698-1-git-send-email-kelvin.cao@microchip.com> MIME-Version: 1.0 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Joey Zhang Switchtec does not support setting multiple MWs simultaneously. The driver takes a hardware lock to ensure that two peers are not doing this simultaneously and it fails if someone else takes the lock. In most cases, this is fine as clients only setup the MWs once on one side of the link. However, there's a race condition when a re-initialization is caused by a link event. The driver will re-setup the shared memory window asynchronously and this races with the client setting up it's memory windows on the link up event. To fix this we ensure do the entire initialization in a work queue and signal the client once it's done. Signed-off-by: Joey Zhang Signed-off-by: Wesley Sheng --- drivers/ntb/hw/mscc/ntb_hw_switchtec.c | 66 ++++++++++++++++++++-------------- 1 file changed, 39 insertions(+), 27 deletions(-) diff --git a/drivers/ntb/hw/mscc/ntb_hw_switchtec.c b/drivers/ntb/hw/mscc/ntb_hw_switchtec.c index 6cf15c18..fffff9a 100644 --- a/drivers/ntb/hw/mscc/ntb_hw_switchtec.c +++ b/drivers/ntb/hw/mscc/ntb_hw_switchtec.c @@ -95,7 +95,8 @@ struct switchtec_ntb { bool link_is_up; enum ntb_speed link_speed; enum ntb_width link_width; - struct work_struct link_reinit_work; + struct work_struct check_link_status_work; + bool link_force_down; }; static struct switchtec_ntb *ntb_sndev(struct ntb_dev *ntb) @@ -494,33 +495,11 @@ enum switchtec_msg { static int switchtec_ntb_reinit_peer(struct switchtec_ntb *sndev); -static void link_reinit_work(struct work_struct *work) -{ - struct switchtec_ntb *sndev; - - sndev = container_of(work, struct switchtec_ntb, link_reinit_work); - - switchtec_ntb_reinit_peer(sndev); -} - -static void switchtec_ntb_check_link(struct switchtec_ntb *sndev, - enum switchtec_msg msg) +static void switchtec_ntb_link_status_update(struct switchtec_ntb *sndev) { int link_sta; int old = sndev->link_is_up; - if (msg == MSG_LINK_FORCE_DOWN) { - schedule_work(&sndev->link_reinit_work); - - if (sndev->link_is_up) { - sndev->link_is_up = 0; - ntb_link_event(&sndev->ntb); - dev_info(&sndev->stdev->dev, "ntb link forced down\n"); - } - - return; - } - link_sta = sndev->self_shared->link_sta; if (link_sta) { u64 peer = ioread64(&sndev->peer_shared->magic); @@ -545,6 +524,38 @@ static void switchtec_ntb_check_link(struct switchtec_ntb *sndev, } } +static void check_link_status_work(struct work_struct *work) +{ + struct switchtec_ntb *sndev; + + sndev = container_of(work, struct switchtec_ntb, + check_link_status_work); + + if (sndev->link_force_down) { + sndev->link_force_down = false; + switchtec_ntb_reinit_peer(sndev); + + if (sndev->link_is_up) { + sndev->link_is_up = 0; + ntb_link_event(&sndev->ntb); + dev_info(&sndev->stdev->dev, "ntb link forced down\n"); + } + + return; + } + + switchtec_ntb_link_status_update(sndev); +} + +static void switchtec_ntb_check_link(struct switchtec_ntb *sndev, + enum switchtec_msg msg) +{ + if (msg == MSG_LINK_FORCE_DOWN) + sndev->link_force_down = true; + + schedule_work(&sndev->check_link_status_work); +} + static void switchtec_ntb_link_notification(struct switchtec_dev *stdev) { struct switchtec_ntb *sndev = stdev->sndev; @@ -577,7 +588,7 @@ static int switchtec_ntb_link_enable(struct ntb_dev *ntb, sndev->self_shared->link_sta = 1; switchtec_ntb_send_msg(sndev, LINK_MESSAGE, MSG_LINK_UP); - switchtec_ntb_check_link(sndev, MSG_CHECK_LINK); + switchtec_ntb_link_status_update(sndev); return 0; } @@ -591,7 +602,7 @@ static int switchtec_ntb_link_disable(struct ntb_dev *ntb) sndev->self_shared->link_sta = 0; switchtec_ntb_send_msg(sndev, LINK_MESSAGE, MSG_LINK_DOWN); - switchtec_ntb_check_link(sndev, MSG_CHECK_LINK); + switchtec_ntb_link_status_update(sndev); return 0; } @@ -844,7 +855,8 @@ static int switchtec_ntb_init_sndev(struct switchtec_ntb *sndev) sndev->ntb.topo = NTB_TOPO_SWITCH; sndev->ntb.ops = &switchtec_ntb_ops; - INIT_WORK(&sndev->link_reinit_work, link_reinit_work); + INIT_WORK(&sndev->check_link_status_work, check_link_status_work); + sndev->link_force_down = false; sndev->self_partition = sndev->stdev->partition;