From patchwork Mon Feb 4 15:35:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795945 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4B0CC922 for ; Mon, 4 Feb 2019 15:36:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3998B2ACAD for ; Mon, 4 Feb 2019 15:36:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2D7952B720; Mon, 4 Feb 2019 15:36:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D41082ACAD for ; Mon, 4 Feb 2019 15:36:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729696AbfBDPgP (ORCPT ); Mon, 4 Feb 2019 10:36:15 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49238 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729608AbfBDPgO (ORCPT ); Mon, 4 Feb 2019 10:36:14 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id A1BDD4195C; Mon, 4 Feb 2019 15:36:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294572; x=1551108973; bh=+FVDTVp6st3ouriHPebIuQPjLAMXm4IFW/O h4uBcbQ4=; b=kHnWZl9F1PXP0nPw82h1vk2isq0oiBJGawpiErHfIYoW5baIAAw eaqQDWl1050STIoZ4eAXXmbhejKd1dmWEVxYVwGDnWQsLN8V1HNoje5dchTG6Udp lj05o+VMKXo3lOgOA2+bTw80rDKOiUrPI08aMuwxHsnjZ7cYNxcwWrXQ= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 32MsPoFy_s4X; Mon, 4 Feb 2019 18:36:12 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 771D4418BF; Mon, 4 Feb 2019 18:36:10 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:09 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko Subject: [PATCH RFC v3 01/21] PCI: Fix writing invalid BARs during pci_restore_state() Date: Mon, 4 Feb 2019 18:35:41 +0300 Message-ID: <20190204153601.7576-2-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If BAR movement has happened (due to PCIe hotplug) after pci_save_state(), the saved addresses will become outdated. Restore them the most recently calculated values, not the ones stored in an arbitrary moment. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/pci.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index c25acace7d91..e1fc93c9eea1 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -1337,7 +1337,7 @@ static void pci_restore_config_space(struct pci_dev *pdev) if (pdev->hdr_type == PCI_HEADER_TYPE_NORMAL) { pci_restore_config_space_range(pdev, 10, 15, 0, false); /* Restore BARs before the command register. */ - pci_restore_config_space_range(pdev, 4, 9, 10, false); + pci_restore_bars(pdev); pci_restore_config_space_range(pdev, 0, 3, 0, false); } else if (pdev->hdr_type == PCI_HEADER_TYPE_BRIDGE) { pci_restore_config_space_range(pdev, 12, 15, 0, false); From patchwork Mon Feb 4 15:35:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795949 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A0A7914E1 for ; Mon, 4 Feb 2019 15:36:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8E44D2ACAD for ; Mon, 4 Feb 2019 15:36:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 81E032B71A; Mon, 4 Feb 2019 15:36:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0DC032ACAD for ; Mon, 4 Feb 2019 15:36:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730595AbfBDPgQ (ORCPT ); Mon, 4 Feb 2019 10:36:16 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49256 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726565AbfBDPgQ (ORCPT ); Mon, 4 Feb 2019 10:36:16 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id AE3A941965; Mon, 4 Feb 2019 15:36:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294573; x=1551108974; bh=OR7+kHmdLXiuucrl8doe382h27BVql/saF5 X7nTlQ/w=; b=Tx35XaCPqZrYvfTN/N64bIHJTg5LSEexAhnqb/GT/vXCv0HJT0k aX0TPiy7N99DJmvccb9k3/LUFmLdppgbC2wcPHgfALrQrWBDOlla3lqtu85YHCaw /2syD3LdzdqlgsqDTyqvjgLVg/RSx8VmK9ccycVsCWPMcjcYwQvQSKxM= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x8JMTZgsNevE; Mon, 4 Feb 2019 18:36:13 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 7AA8A41908; Mon, 4 Feb 2019 18:36:10 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:09 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko , , Benjamin Herrenschmidt , Oliver , Sam Bobroff Subject: [PATCH RFC v3 02/21] PCI: Fix race condition in pci_enable/disable_device() Date: Mon, 4 Feb 2019 18:35:42 +0300 Message-ID: <20190204153601.7576-3-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP CPU0 CPU1 pci_enable_device_mem() pci_enable_device_mem() pci_enable_bridge() pci_enable_bridge() pci_is_enabled() return false; atomic_inc_return(enable_cnt) Start actual enabling the bridge ... pci_is_enabled() ... return true; ... Start memory requests <-- FAIL ... Set the PCI_COMMAND_MEMORY bit <-- Must wait for this This patch protects the pci_enable/disable_device() and pci_enable_bridge() with mutexes. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/pci.c | 26 ++++++++++++++++++++++---- drivers/pci/probe.c | 1 + include/linux/pci.h | 1 + 3 files changed, 24 insertions(+), 4 deletions(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index e1fc93c9eea1..3a83e05f8363 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -1571,6 +1571,8 @@ static void pci_enable_bridge(struct pci_dev *dev) struct pci_dev *bridge; int retval; + mutex_lock(&dev->enable_mutex); + bridge = pci_upstream_bridge(dev); if (bridge) pci_enable_bridge(bridge); @@ -1578,6 +1580,7 @@ static void pci_enable_bridge(struct pci_dev *dev) if (pci_is_enabled(dev)) { if (!dev->is_busmaster) pci_set_master(dev); + mutex_unlock(&dev->enable_mutex); return; } @@ -1586,11 +1589,14 @@ static void pci_enable_bridge(struct pci_dev *dev) pci_err(dev, "Error enabling bridge (%d), continuing\n", retval); pci_set_master(dev); + mutex_unlock(&dev->enable_mutex); } static int pci_enable_device_flags(struct pci_dev *dev, unsigned long flags) { struct pci_dev *bridge; + /* Enable-locking of bridges is performed within the pci_enable_bridge() */ + bool need_lock = !dev->subordinate; int err; int i, bars = 0; @@ -1606,8 +1612,13 @@ static int pci_enable_device_flags(struct pci_dev *dev, unsigned long flags) dev->current_state = (pmcsr & PCI_PM_CTRL_STATE_MASK); } - if (atomic_inc_return(&dev->enable_cnt) > 1) + if (need_lock) + mutex_lock(&dev->enable_mutex); + if (pci_is_enabled(dev)) { + if (need_lock) + mutex_unlock(&dev->enable_mutex); return 0; /* already enabled */ + } bridge = pci_upstream_bridge(dev); if (bridge) @@ -1622,8 +1633,10 @@ static int pci_enable_device_flags(struct pci_dev *dev, unsigned long flags) bars |= (1 << i); err = do_pci_enable_device(dev, bars); - if (err < 0) - atomic_dec(&dev->enable_cnt); + if (err >= 0) + atomic_inc(&dev->enable_cnt); + if (need_lock) + mutex_unlock(&dev->enable_mutex); return err; } @@ -1866,15 +1879,20 @@ void pci_disable_device(struct pci_dev *dev) if (dr) dr->enabled = 0; + mutex_lock(&dev->enable_mutex); dev_WARN_ONCE(&dev->dev, atomic_read(&dev->enable_cnt) <= 0, "disabling already-disabled device"); - if (atomic_dec_return(&dev->enable_cnt) != 0) + if (atomic_dec_return(&dev->enable_cnt) != 0) { + mutex_unlock(&dev->enable_mutex); return; + } do_pci_disable_device(dev); dev->is_busmaster = 0; + + mutex_unlock(&dev->enable_mutex); } EXPORT_SYMBOL(pci_disable_device); diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 257b9f6f2ebb..bbc12934f041 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -2191,6 +2191,7 @@ struct pci_dev *pci_alloc_dev(struct pci_bus *bus) INIT_LIST_HEAD(&dev->bus_list); dev->dev.type = &pci_dev_type; dev->bus = pci_bus_get(bus); + mutex_init(&dev->enable_mutex); return dev; } diff --git a/include/linux/pci.h b/include/linux/pci.h index 65f1d8c2f082..28fecfdd598d 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -416,6 +416,7 @@ struct pci_dev { unsigned int no_vf_scan:1; /* Don't scan for VFs after IOV enablement */ pci_dev_flags_t dev_flags; atomic_t enable_cnt; /* pci_enable_device has been called */ + struct mutex enable_mutex; u32 saved_config_space[16]; /* Config space saved at suspend time */ struct hlist_head saved_cap_space; From patchwork Mon Feb 4 15:35:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795951 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0675F1805 for ; Mon, 4 Feb 2019 15:36:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E8EBA2ACAD for ; Mon, 4 Feb 2019 15:36:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DD8052AD42; Mon, 4 Feb 2019 15:36:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 964912B720 for ; Mon, 4 Feb 2019 15:36:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730763AbfBDPgQ (ORCPT ); Mon, 4 Feb 2019 10:36:16 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49270 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730135AbfBDPgQ (ORCPT ); Mon, 4 Feb 2019 10:36:16 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 89642411D9; Mon, 4 Feb 2019 15:36:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294574; x=1551108975; bh=WN6RYdU7ay5bXObl2dwS5zrLgkTPSgA1efE zWil4ZO8=; b=eOTjI4HsYjpHsdKi/l27myEAuJaiE/2v9E8MW34No654bTUAGB2 zL0DrR/+b08VlvVHs1B79n8IVjjaRuH4Mynu9d6rz0badc3+aGC/P8abbalgBz5+ wdHcgaRiuDxG8M30bYRCwzM7FWQCukm+5SOnEojtrzDovNwEvL9g21Jw= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Dw2xFiNMzHTb; Mon, 4 Feb 2019 18:36:14 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 9540541911; Mon, 4 Feb 2019 18:36:10 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:09 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko Subject: [PATCH RFC v3 03/21] PCI: Enable bridge's I/O and MEM access for hotplugged devices Date: Mon, 4 Feb 2019 18:35:43 +0300 Message-ID: <20190204153601.7576-4-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP After updating the bridge window resources, the PCI_COMMAND_IO and PCI_COMMAND_MEMORY bits of the bridge must be addressed as well. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/pci.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 3a83e05f8363..55cf18389c15 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -1578,6 +1578,14 @@ static void pci_enable_bridge(struct pci_dev *dev) pci_enable_bridge(bridge); if (pci_is_enabled(dev)) { + int i, bars = 0; + + for (i = PCI_BRIDGE_RESOURCES; i < DEVICE_COUNT_RESOURCE; i++) { + if (dev->resource[i].flags & (IORESOURCE_MEM | IORESOURCE_IO)) + bars |= (1 << i); + } + do_pci_enable_device(dev, bars); + if (!dev->is_busmaster) pci_set_master(dev); mutex_unlock(&dev->enable_mutex); From patchwork Mon Feb 4 15:35:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795953 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E4D77922 for ; Mon, 4 Feb 2019 15:36:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D3AC22ACAD for ; Mon, 4 Feb 2019 15:36:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C83D12B720; Mon, 4 Feb 2019 15:36:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6F1D52ACAD for ; Mon, 4 Feb 2019 15:36:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729298AbfBDPgS (ORCPT ); Mon, 4 Feb 2019 10:36:18 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49282 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726565AbfBDPgS (ORCPT ); Mon, 4 Feb 2019 10:36:18 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 3310441911; Mon, 4 Feb 2019 15:36:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294575; x=1551108976; bh=ZFSpeWXC0G6p3BEjk1YvsGQE5boQjBUMxO0 HRmXCPoU=; b=YJy5Vp6BGkXWZ6z7H4JsavNrPhnh130b9PCKUY+zQj/UaoD5q2p 923xIkIbPPdAemGZ/5pb6nZMWCWZUpiXotk0J0+9Foa/kid+dpncCbKz2O7bb8pp 2pN16+72D0+wLkMWNWr5pY0UwghFeYL+8zURBDhti34RriCHmd/rmmx4= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id T-SJkHKIRIRw; Mon, 4 Feb 2019 18:36:15 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 9935241925; Mon, 4 Feb 2019 18:36:10 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:10 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko Subject: [PATCH RFC v3 04/21] PCI: Define PCI-specific version of the release_child_resources() Date: Mon, 4 Feb 2019 18:35:44 +0300 Message-ID: <20190204153601.7576-5-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Make the released resources of a bridge valid for later re-assignment: clear the STARTALIGN flag. Resources marked with PCI_FIXED must preserve their offset and size. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/setup-bus.c | 47 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 46 insertions(+), 1 deletion(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index ed960436df5e..cca23c393e29 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1520,6 +1520,51 @@ static void __pci_bridge_assign_resources(const struct pci_dev *bridge, (IORESOURCE_IO | IORESOURCE_MEM | IORESOURCE_PREFETCH |\ IORESOURCE_MEM_64) +/* + * Similar to release_child_resources(), but aware of PCI_FIXED and STARTALIGN flags + */ +static void pci_release_child_resources(struct resource *r) +{ + struct resource *tmp, *p; + + if (!r) + return; + + if (r->flags & IORESOURCE_PCI_FIXED) + return; + + p = r->child; + r->child = NULL; + while (p) { + resource_size_t size = resource_size(p); + + tmp = p; + p = p->sibling; + + tmp->parent = NULL; + tmp->sibling = NULL; + pci_release_child_resources(tmp); + + if (!tmp->flags) + continue; + + if (tmp->flags & IORESOURCE_PCI_FIXED) { + pr_debug("PCI: release fixed %pR (%s), keep its flags, base and size\n", + tmp, tmp->name); + continue; + } + + pr_debug("PCI: release %pR (%s)\n", tmp, tmp->name); + + /* need to restore size, and keep all the flags but STARTALIGN */ + tmp->start = 0; + tmp->end = size - 1; + + tmp->flags &= ~IORESOURCE_STARTALIGN; + tmp->flags |= IORESOURCE_SIZEALIGN; + } +} + static void pci_bridge_release_resources(struct pci_bus *bus, unsigned long type) { @@ -1565,7 +1610,7 @@ static void pci_bridge_release_resources(struct pci_bus *bus, * if there are children under that, we should release them * all */ - release_child_resources(r); + pci_release_child_resources(r); if (!release_resource(r)) { type = old_flags = r->flags & PCI_RES_TYPE_MASK; pci_printk(KERN_DEBUG, dev, "resource %d %pR released\n", From patchwork Mon Feb 4 15:35:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795955 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2BAF31805 for ; Mon, 4 Feb 2019 15:36:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 14FE32ACAD for ; Mon, 4 Feb 2019 15:36:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 08EB22B71A; Mon, 4 Feb 2019 15:36:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 909CD2AD42 for ; Mon, 4 Feb 2019 15:36:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731098AbfBDPgS (ORCPT ); Mon, 4 Feb 2019 10:36:18 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49298 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729298AbfBDPgS (ORCPT ); Mon, 4 Feb 2019 10:36:18 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 9F19941925; Mon, 4 Feb 2019 15:36:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294575; x=1551108976; bh=fBZVvrDKnTqEuLszPLOAjv6NzdQSIKXVafV onFmNfH8=; b=oJz9sOq/wMFvnQcZp/+Yp/+tA6G8pd6wYD8uXlVeb+VqORzFzl6 fROcT2rXhaxXA/5cpOfbLVGvrtUjp0wCVGhmALdPL9i8yl50wZ+kRLKkRqbzcXVu uo0o3f7rSekM98B1xlWDBIePOevxc0JCzcbRtienO2g242fzzdkjYIzg= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZSN9cGbhrC5R; Mon, 4 Feb 2019 18:36:15 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id C3A4C41927; Mon, 4 Feb 2019 18:36:10 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:10 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko , , Benjamin Herrenschmidt , Oliver , Sam Bobroff , Lukas Wunner Subject: [PATCH RFC v3 05/21] PCI: hotplug: Add a flag for the movable BARs feature Date: Mon, 4 Feb 2019 18:35:45 +0300 Message-ID: <20190204153601.7576-6-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If a new PCIe device has been hot-plugged between the two active ones without big enough gap between their BARs, these BARs should be moved if their drivers support this feature. The drivers should be notified and paused during the procedure: 1) dev 8 (new) | v .. | dev 3 | dev 3 | dev 5 | dev 7 | .. | BAR 0 | BAR 1 | BAR 0 | BAR 0 | 2) dev 8 | v .. | dev 3 | dev 3 | --> --> | dev 5 | dev 7 | .. | BAR 0 | BAR 1 | --> --> | BAR 0 | BAR 0 | 3) .. | dev 3 | dev 3 | dev 8 | dev 8 | dev 5 | dev 7 | .. | BAR 0 | BAR 1 | BAR 0 | BAR 1 | BAR 0 | BAR 0 | Thus, prior reservation of memory regions by BIOS/bootloader/firmware is not required anymore for the PCIe hotplug. The PCI_MOVABLE_BARS flag is set by the platform is this feature is supported and tested, but can be overridden by the following command line option: pcie_movable_bars={ off | force } Signed-off-by: Sergey Miroshnichenko --- .../admin-guide/kernel-parameters.txt | 7 ++++++ drivers/pci/pci.c | 24 +++++++++++++++++++ include/linux/pci.h | 2 ++ 3 files changed, 33 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index b799bcf67d7b..2165c4b5aea6 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3387,6 +3387,13 @@ nomsi Do not use MSI for native PCIe PME signaling (this makes all PCIe root ports use INTx for all services). + pcie_movable_bars=[PCIE] + Override the movable BARs support detection: + off + Disable even if supported by the platform + force + Enable even if not explicitly declared as supported + pcmv= [HW,PCMCIA] BadgePAD 4 pd_ignore_unused diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 55cf18389c15..096413f9ee67 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -139,6 +139,30 @@ static int __init pcie_port_pm_setup(char *str) } __setup("pcie_port_pm=", pcie_port_pm_setup); +static bool pcie_movable_bars_off; +static bool pcie_movable_bars_force; +static int __init pcie_movable_bars_setup(char *str) +{ + if (!strcmp(str, "off")) + pcie_movable_bars_off = true; + else if (!strcmp(str, "force")) + pcie_movable_bars_force = true; + return 1; +} +__setup("pcie_movable_bars=", pcie_movable_bars_setup); + +bool pci_movable_bars_enabled(void) +{ + if (pcie_movable_bars_off) + return false; + + if (pcie_movable_bars_force) + return true; + + return pci_has_flag(PCI_MOVABLE_BARS); +} +EXPORT_SYMBOL(pci_movable_bars_enabled); + /* Time to wait after a reset for device to become responsive */ #define PCIE_RESET_READY_POLL_MS 60000 diff --git a/include/linux/pci.h b/include/linux/pci.h index 28fecfdd598d..ba0b1d0ea2d2 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -863,6 +863,7 @@ enum { PCI_ENABLE_PROC_DOMAINS = 0x00000010, /* Enable domains in /proc */ PCI_COMPAT_DOMAIN_0 = 0x00000020, /* ... except domain 0 */ PCI_SCAN_ALL_PCIE_DEVS = 0x00000040, /* Scan all, not just dev 0 */ + PCI_MOVABLE_BARS = 0x00000080, /* Runtime BAR reassign after hotplug */ }; /* These external functions are only available when PCI support is enabled */ @@ -1342,6 +1343,7 @@ unsigned char pci_bus_max_busnr(struct pci_bus *bus); void pci_setup_bridge(struct pci_bus *bus); resource_size_t pcibios_window_alignment(struct pci_bus *bus, unsigned long type); +bool pci_movable_bars_enabled(void); #define PCI_VGA_STATE_CHANGE_BRIDGE (1 << 0) #define PCI_VGA_STATE_CHANGE_DECODES (1 << 1) From patchwork Mon Feb 4 15:35:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795965 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 665991805 for ; Mon, 4 Feb 2019 15:36:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 53F922ACAD for ; Mon, 4 Feb 2019 15:36:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 482CD2B72F; Mon, 4 Feb 2019 15:36:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DDDF02B71A for ; Mon, 4 Feb 2019 15:36:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731344AbfBDPgZ (ORCPT ); Mon, 4 Feb 2019 10:36:25 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49298 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731323AbfBDPgY (ORCPT ); Mon, 4 Feb 2019 10:36:24 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id EF92F4196C; Mon, 4 Feb 2019 15:36:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294576; x=1551108977; bh=Y7bqz4DPDetczRyG4vWuemSZRvQz7iYuxkl Dbk+E/kM=; b=PCKufqSgo50tvCbks3cUJZosbl8zGShYkqUfV8zA2OExgV+8Mjg t5jQg1zUvqcZcWd/OpHS3niDZc7WN8PgMSMLGaOaumh7Il44Qxfji22fI+1hLYAY gZBSrME1hfJk3iXTnSVd4ZG9zg8wuUJ508+SDzgqXEV5Im83YWuakcLQ= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GWVQY8ui9u28; Mon, 4 Feb 2019 18:36:16 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 2342641931; Mon, 4 Feb 2019 18:36:11 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:10 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko , , Benjamin Herrenschmidt , Oliver , Sam Bobroff , Lukas Wunner Subject: [PATCH RFC v3 06/21] PCI: Pause the devices with movable BARs during rescan Date: Mon, 4 Feb 2019 18:35:46 +0300 Message-ID: <20190204153601.7576-7-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Drivers indicate their support of movable BARs by implementing the new rescan_prepare() and rescan_done() hooks in the struct pci_driver. All device's activity must be stopped during a rescan, and iounmap() +ioremap() must be applied to every used BAR. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/probe.c | 51 +++++++++++++++++++++++++++++++++++++++++++-- include/linux/pci.h | 2 ++ 2 files changed, 51 insertions(+), 2 deletions(-) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index bbc12934f041..e18d07996cf3 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -3172,6 +3172,38 @@ unsigned int pci_rescan_bus_bridge_resize(struct pci_dev *bridge) return max; } +static void pci_bus_rescan_prepare(struct pci_bus *bus) +{ + struct pci_dev *dev; + + list_for_each_entry(dev, &bus->devices, bus_list) { + struct pci_bus *child = dev->subordinate; + + if (child) { + pci_bus_rescan_prepare(child); + } else if (dev->driver && + dev->driver->rescan_prepare) { + dev->driver->rescan_prepare(dev); + } + } +} + +static void pci_bus_rescan_done(struct pci_bus *bus) +{ + struct pci_dev *dev; + + list_for_each_entry(dev, &bus->devices, bus_list) { + struct pci_bus *child = dev->subordinate; + + if (child) { + pci_bus_rescan_done(child); + } else if (dev->driver && + dev->driver->rescan_done) { + dev->driver->rescan_done(dev); + } + } +} + /** * pci_rescan_bus - Scan a PCI bus for devices * @bus: PCI bus to scan @@ -3185,8 +3217,23 @@ unsigned int pci_rescan_bus(struct pci_bus *bus) { unsigned int max; - max = pci_scan_child_bus(bus); - pci_assign_unassigned_bus_resources(bus); + if (pci_movable_bars_enabled()) { + struct pci_bus *root = bus; + + while (!pci_is_root_bus(root)) + root = root->parent; + + pci_bus_rescan_prepare(root); + + max = pci_scan_child_bus(root); + pci_assign_unassigned_root_bus_resources(root); + + pci_bus_rescan_done(root); + } else { + max = pci_scan_child_bus(bus); + pci_assign_unassigned_bus_resources(bus); + } + pci_bus_add_devices(bus); return max; diff --git a/include/linux/pci.h b/include/linux/pci.h index ba0b1d0ea2d2..5cd534b6631b 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -777,6 +777,8 @@ struct pci_driver { int (*resume)(struct pci_dev *dev); /* Device woken up */ void (*shutdown)(struct pci_dev *dev); int (*sriov_configure)(struct pci_dev *dev, int num_vfs); /* On PF */ + void (*rescan_prepare)(struct pci_dev *dev); + void (*rescan_done)(struct pci_dev *dev); const struct pci_error_handlers *err_handler; const struct attribute_group **groups; struct device_driver driver; From patchwork Mon Feb 4 15:35:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795957 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8C2111823 for ; Mon, 4 Feb 2019 15:36:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7750B2ACAD for ; Mon, 4 Feb 2019 15:36:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6BB6C2B71A; Mon, 4 Feb 2019 15:36:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 25BE72AD42 for ; Mon, 4 Feb 2019 15:36:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731301AbfBDPgT (ORCPT ); Mon, 4 Feb 2019 10:36:19 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49270 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730135AbfBDPgT (ORCPT ); Mon, 4 Feb 2019 10:36:19 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 853C241967; Mon, 4 Feb 2019 15:36:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294576; x=1551108977; bh=HpciNbmk+sNxY45aMdCPoyx/0SeyWnSlTYO UPzVN1nY=; b=bu/jyjH37JfvKhlXaV0m3Y9xR98SL6ARKTfUJLhzaqXQp0dvQMW NrZJrqjlMRGygDyOnXnSnY45b2QyBEenV9P4tYoBEa/aBighNYpH8tZ4ivsV0BGw qXg2bWpXpgfgV5MJeQIhyZp6d57hOgFLjRpxzz2708mFHcCx07e7Inug= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ts_wZlMgiMPT; Mon, 4 Feb 2019 18:36:16 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 467D94193A; Mon, 4 Feb 2019 18:36:11 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:10 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko Subject: [PATCH RFC v3 07/21] PCI: Wake up bridges during rescan when movable BARs enabled Date: Mon, 4 Feb 2019 18:35:47 +0300 Message-ID: <20190204153601.7576-8-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Use the PM runtime methods to wake up the bridges before accessing their config space. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/probe.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index e18d07996cf3..7fe68b4f04bf 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -3176,6 +3176,8 @@ static void pci_bus_rescan_prepare(struct pci_bus *bus) { struct pci_dev *dev; + pm_runtime_get_sync(&bus->dev); + list_for_each_entry(dev, &bus->devices, bus_list) { struct pci_bus *child = dev->subordinate; @@ -3202,6 +3204,8 @@ static void pci_bus_rescan_done(struct pci_bus *bus) dev->driver->rescan_done(dev); } } + + pm_runtime_put(&bus->dev); } /** From patchwork Mon Feb 4 15:35:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795959 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A3F18922 for ; Mon, 4 Feb 2019 15:36:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 933492ACAD for ; Mon, 4 Feb 2019 15:36:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 879CC2B71A; Mon, 4 Feb 2019 15:36:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2BE2D2ACAD for ; Mon, 4 Feb 2019 15:36:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731267AbfBDPgU (ORCPT ); Mon, 4 Feb 2019 10:36:20 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49298 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726565AbfBDPgU (ORCPT ); Mon, 4 Feb 2019 10:36:20 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 84B584193A; Mon, 4 Feb 2019 15:36:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294577; x=1551108978; bh=hDnKF7LYovCYjlPjeBctcYJ37UhnG2dTJdt hA73Me48=; b=rn0w5X+cez+7mJNFSBcVi4p45yHjN++AcJf+nBbeQ5wcHsNjzuQ pTegcR4V0X/IdhCdG4+7HnDe7tmrz6Af5+n4STp0t/tp5OzVc40QETgF2Ab8LXLX HzkhI+n1F3yYZ1lPTyKlOyYENaGcY9OTjtj6/Vqb6CEF1QDblFtxachI= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hE88EV9a7pQk; Mon, 4 Feb 2019 18:36:17 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 79F6F4193E; Mon, 4 Feb 2019 18:36:11 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:11 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko Subject: [PATCH RFC v3 08/21] nvme-pci: Handle movable BARs Date: Mon, 4 Feb 2019 18:35:48 +0300 Message-ID: <20190204153601.7576-9-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Hotplugged devices can affect the existing ones by moving their BARs. PCI subsystem will inform the NVME driver about this by invoking reset_prepare()+reset_done(), then iounmap()+ioremap() must be called. Signed-off-by: Sergey Miroshnichenko --- drivers/nvme/host/pci.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 9bc585415d9b..60d5e97fc878 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -114,6 +114,7 @@ struct nvme_dev { unsigned int num_vecs; int q_depth; u32 db_stride; + resource_size_t current_phys_bar; void __iomem *bar; unsigned long bar_mapped_size; struct work_struct remove_work; @@ -1678,13 +1679,16 @@ static int nvme_remap_bar(struct nvme_dev *dev, unsigned long size) { struct pci_dev *pdev = to_pci_dev(dev->dev); - if (size <= dev->bar_mapped_size) + if (dev->bar && + dev->current_phys_bar == pci_resource_start(pdev, 0) && + size <= dev->bar_mapped_size) return 0; if (size > pci_resource_len(pdev, 0)) return -ENOMEM; if (dev->bar) iounmap(dev->bar); - dev->bar = ioremap(pci_resource_start(pdev, 0), size); + dev->current_phys_bar = pci_resource_start(pdev, 0); + dev->bar = ioremap(dev->current_phys_bar, size); if (!dev->bar) { dev->bar_mapped_size = 0; return -ENOMEM; @@ -2550,6 +2554,8 @@ static void nvme_reset_work(struct work_struct *work) if (WARN_ON(dev->ctrl.state != NVME_CTRL_RESETTING)) goto out; + nvme_remap_bar(dev, db_bar_size(dev, 0)); + /* * If we're called to reset a live controller first shut it down before * moving on. @@ -2952,6 +2958,23 @@ static void nvme_error_resume(struct pci_dev *pdev) flush_work(&dev->ctrl.reset_work); } +void nvme_rescan_prepare(struct pci_dev *pdev) +{ + struct nvme_dev *dev = pci_get_drvdata(pdev); + + nvme_dev_disable(dev, false); + nvme_dev_unmap(dev); + dev->bar = NULL; +} + +void nvme_rescan_done(struct pci_dev *pdev) +{ + struct nvme_dev *dev = pci_get_drvdata(pdev); + + nvme_dev_map(dev); + nvme_reset_ctrl_sync(&dev->ctrl); +} + static const struct pci_error_handlers nvme_err_handler = { .error_detected = nvme_error_detected, .slot_reset = nvme_slot_reset, @@ -3016,6 +3039,8 @@ static struct pci_driver nvme_driver = { }, .sriov_configure = pci_sriov_configure_simple, .err_handler = &nvme_err_handler, + .rescan_prepare = nvme_rescan_prepare, + .rescan_done = nvme_rescan_done, }; static int __init nvme_init(void) From patchwork Mon Feb 4 15:35:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795961 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DAFEE1805 for ; Mon, 4 Feb 2019 15:36:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C92CD2ACAD for ; Mon, 4 Feb 2019 15:36:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BD8D82B71A; Mon, 4 Feb 2019 15:36:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5CE262B720 for ; Mon, 4 Feb 2019 15:36:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731254AbfBDPgV (ORCPT ); Mon, 4 Feb 2019 10:36:21 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49282 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730135AbfBDPgV (ORCPT ); Mon, 4 Feb 2019 10:36:21 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 65DB241969; Mon, 4 Feb 2019 15:36:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294578; x=1551108979; bh=jwH0P9Xag/Shas342g+4t1xNKBCTKGrg818 /YDwT5ow=; b=RvKBOAtfHDU2AK6iq2KyidYbTzkcBoNsS6qZJ6g5WWDYeAL1JPR Y6ie+vvJ6Jlt7PU3plNmPSuuckZo388fhXMGy8jgpvfvQYBMMoIHb1vTZiL/rVEg hgxgUP/kbb0Ae/FG1ogf6LNk1/IVGmGOciM6le5Al7IBrp8BO/NSyKMM= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AHYuLdBfHVnP; Mon, 4 Feb 2019 18:36:18 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id B11AF41258; Mon, 4 Feb 2019 18:36:11 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:11 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko Subject: [PATCH RFC v3 09/21] PCI: Mark immovable BARs with PCI_FIXED Date: Mon, 4 Feb 2019 18:35:49 +0300 Message-ID: <20190204153601.7576-10-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If a PCIe device driver doesn't yet have support for movable BARs, mark device's BARs with IORESOURCE_PCI_FIXED. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/probe.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 7fe68b4f04bf..1f50eb556d28 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -3186,6 +3186,21 @@ static void pci_bus_rescan_prepare(struct pci_bus *bus) } else if (dev->driver && dev->driver->rescan_prepare) { dev->driver->rescan_prepare(dev); + } else if (dev->driver || ((dev->class >> 8) == PCI_CLASS_DISPLAY_VGA)) { + int i; + + for (i = 0; i < PCI_NUM_RESOURCES; i++) { + struct resource *r = &dev->resource[i]; + + if (!r->flags || !r->parent || + (r->flags & IORESOURCE_UNSET) || + (r->flags & IORESOURCE_PCI_FIXED)) + continue; + + r->flags |= IORESOURCE_PCI_FIXED; + pci_warn(dev, "%s: no support for movable BARs, mark BAR %d (%pR) as fixed\n", + __func__, i, r); + } } } } From patchwork Mon Feb 4 15:35:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795967 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EF4121823 for ; Mon, 4 Feb 2019 15:36:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DEB032ACAD for ; Mon, 4 Feb 2019 15:36:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D328B2B723; Mon, 4 Feb 2019 15:36:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 897982ACAD for ; Mon, 4 Feb 2019 15:36:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731303AbfBDPg0 (ORCPT ); Mon, 4 Feb 2019 10:36:26 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49270 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731278AbfBDPgV (ORCPT ); Mon, 4 Feb 2019 10:36:21 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 443B441258; Mon, 4 Feb 2019 15:36:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294579; x=1551108980; bh=EjdM6COXzW6GuE8nkSgjoOyWzb8QUfdJh1s wwJrBvsg=; b=IWzTK0XLkUf8hX8agko7+VJNjq5lYAnTUIM/vYb7OdXz+WJoXSe dMI1vAwzCA3yQzP7pZusl0wF2WkgvynyLcahBKVylwjzQdqzcrX6LBaGNaFTE1a9 HDHX6hqpv+MAqhxZ7QWnREHQrMp7tk3H7UP5ZaXeMF/yDPGeXtazeKLg= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id D-1VySBUUZVO; Mon, 4 Feb 2019 18:36:19 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id E61E04194E; Mon, 4 Feb 2019 18:36:11 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:11 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko , David Daney Subject: [PATCH RFC v3 10/21] PCI: Fix assigning of fixed prefetchable resources Date: Mon, 4 Feb 2019 18:35:50 +0300 Message-ID: <20190204153601.7576-11-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Allow matching them to non-prefetchable windows, as it is done for movable resources. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/setup-bus.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index cca23c393e29..b1eec758d2c1 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1338,15 +1338,20 @@ static void assign_fixed_resource_on_bus(struct pci_bus *b, struct resource *r) { int i; struct resource *parent_r; - unsigned long mask = IORESOURCE_IO | IORESOURCE_MEM | - IORESOURCE_PREFETCH; + unsigned long mask = IORESOURCE_TYPE_BITS; pci_bus_for_each_resource(b, parent_r, i) { if (!parent_r) continue; - if ((r->flags & mask) == (parent_r->flags & mask) && - resource_contains(parent_r, r)) + if ((r->flags & mask) != (parent_r->flags & mask)) + continue; + + if (parent_r->flags & IORESOURCE_PREFETCH && + !(r->flags & IORESOURCE_PREFETCH)) + continue; + + if (resource_contains(parent_r, r)) request_resource(parent_r, r); } } From patchwork Mon Feb 4 15:35:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795963 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4FE0F922 for ; Mon, 4 Feb 2019 15:36:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3C5412B720 for ; Mon, 4 Feb 2019 15:36:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 301992B723; Mon, 4 Feb 2019 15:36:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD22A2ACAD for ; Mon, 4 Feb 2019 15:36:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731323AbfBDPgZ (ORCPT ); Mon, 4 Feb 2019 10:36:25 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49282 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731303AbfBDPgY (ORCPT ); Mon, 4 Feb 2019 10:36:24 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 4BE0841931; Mon, 4 Feb 2019 15:36:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294580; x=1551108981; bh=LfYwMoeGy/M1ZEnk8iKUjneOEQTHSScnchE 56SOIn5A=; b=b2RO3nD4R+kbv4Y2zdysWXmMixx02kMz220kgEygrl6s6H9Oy1I mXklmfTQIR8ZsreJ7nexPDDDTwlA8jqwS7o0cr8bHeHShSZV7Be2Cb3QDi78TEEA zhOE3ZgpoxjLPCSdrCifLX7o56SZLBi5n796j3CldpbEdIGfBZLvB1G8= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i79Tz5L8_5L0; Mon, 4 Feb 2019 18:36:20 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 76C0641949; Mon, 4 Feb 2019 18:36:12 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:11 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko Subject: [PATCH RFC v3 11/21] PCI: Release and reassign the root bridge resources during rescan Date: Mon, 4 Feb 2019 18:35:51 +0300 Message-ID: <20190204153601.7576-12-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When the movable BARs feature is enabled, don't rely on the memory gaps reserved by the BIOS/bootloader/firmware, but instead rearrange the BARs and bridge windows starting from the root. Endpoint device's BARs, after being released, are resorted and written back by the pci_assign_unassigned_root_bus_resources(). The last step of writing the recalculated windows to the bridges is done by the new pci_setup_bridges() function. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/pci.h | 1 + drivers/pci/probe.c | 22 ++++++++++++++++++++++ drivers/pci/setup-bus.c | 11 ++++++++++- 3 files changed, 33 insertions(+), 1 deletion(-) diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 224d88634115..e06e8692a7b1 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -248,6 +248,7 @@ void __pci_bus_assign_resources(const struct pci_bus *bus, struct list_head *realloc_head, struct list_head *fail_head); bool pci_bus_clip_resource(struct pci_dev *dev, int idx); +void pci_bus_release_root_bridge_resources(struct pci_bus *bus); void pci_reassigndev_resource_alignment(struct pci_dev *dev); void pci_disable_bridge_window(struct pci_dev *dev); diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index 1f50eb556d28..de808ca28797 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -3223,6 +3223,25 @@ static void pci_bus_rescan_done(struct pci_bus *bus) pm_runtime_put(&bus->dev); } +static void pci_setup_bridges(struct pci_bus *bus) +{ + struct pci_dev *dev; + + list_for_each_entry(dev, &bus->devices, bus_list) { + struct pci_bus *child; + + if (!pci_dev_is_added(dev) || pci_dev_is_ignored(dev)) + continue; + + child = dev->subordinate; + if (child) + pci_setup_bridges(child); + } + + if (bus->self) + pci_setup_bridge(bus); +} + /** * pci_rescan_bus - Scan a PCI bus for devices * @bus: PCI bus to scan @@ -3245,8 +3264,11 @@ unsigned int pci_rescan_bus(struct pci_bus *bus) pci_bus_rescan_prepare(root); max = pci_scan_child_bus(root); + + pci_bus_release_root_bridge_resources(root); pci_assign_unassigned_root_bus_resources(root); + pci_setup_bridges(root); pci_bus_rescan_done(root); } else { max = pci_scan_child_bus(bus); diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index b1eec758d2c1..07c6d9cc209f 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1621,7 +1621,7 @@ static void pci_bridge_release_resources(struct pci_bus *bus, pci_printk(KERN_DEBUG, dev, "resource %d %pR released\n", PCI_BRIDGE_RESOURCES + idx, r); /* keep the old size */ - r->end = resource_size(r) - 1; + r->end = pci_movable_bars_enabled() ? 0 : (resource_size(r) - 1); r->start = 0; r->flags = 0; @@ -1674,6 +1674,15 @@ static void pci_bus_release_bridge_resources(struct pci_bus *bus, pci_bridge_release_resources(bus, type); } +void pci_bus_release_root_bridge_resources(struct pci_bus *root_bus) +{ + pci_bus_release_bridge_resources(root_bus, IORESOURCE_IO, whole_subtree); + pci_bus_release_bridge_resources(root_bus, IORESOURCE_MEM, whole_subtree); + pci_bus_release_bridge_resources(root_bus, + IORESOURCE_MEM_64 | IORESOURCE_PREFETCH, + whole_subtree); +} + static void pci_bus_dump_res(struct pci_bus *bus) { struct resource *res; From patchwork Mon Feb 4 15:35:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795969 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6770F922 for ; Mon, 4 Feb 2019 15:36:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 553082ACAD for ; Mon, 4 Feb 2019 15:36:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4912C2B723; Mon, 4 Feb 2019 15:36:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AA7D42B71A for ; Mon, 4 Feb 2019 15:36:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731345AbfBDPg0 (ORCPT ); Mon, 4 Feb 2019 10:36:26 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49368 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731333AbfBDPg0 (ORCPT ); Mon, 4 Feb 2019 10:36:26 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 3B94F41970; Mon, 4 Feb 2019 15:36:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294582; x=1551108983; bh=lJzogISPsFoGC95zGQUmPj073aSvDOTpsbw wHFGEgy8=; b=ZDsK5hrBndQqh1Sg+cwM0txozcoCH2p45ZrSsbkUsvadh5ibL1P pL+wUJkxP0Ss8zqCEuFDBBr5RhANxtRNxiL9segg/zzVz0xfsrMX48n6LIQXBgv8 yXmkZeae/2nfrbNF2jEeK42rVz3w3R8iM2Nv2REasG0hF247dAvoMFQw= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KwahzMoQP7AP; Mon, 4 Feb 2019 18:36:22 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id AEEF641953; Mon, 4 Feb 2019 18:36:12 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:12 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko Subject: [PATCH RFC v3 12/21] PCI: Don't allow hotplugged devices to steal resources Date: Mon, 4 Feb 2019 18:35:52 +0300 Message-ID: <20190204153601.7576-13-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When movable BARs are enabled, the PCI subsystem at first releases all the bridge windows and then performs an attempt to assign new requested resources and re-assign the existing ones. If a hotplugged device gets its resources first, there could be no space left to re-assign resources of already working devices, which is unacceptable. If this happens, this patch marks one of the new devices with the new introduced flag PCI_DEV_IGNORE and retries the resource assignment. This patch adds a new res_mask bitmask to the struct pci_dev for storing the indices of assigned resources. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/bus.c | 5 ++ drivers/pci/pci.h | 11 +++++ drivers/pci/probe.c | 100 +++++++++++++++++++++++++++++++++++++++- drivers/pci/setup-bus.c | 15 ++++++ include/linux/pci.h | 1 + 5 files changed, 130 insertions(+), 2 deletions(-) diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c index 5cb40b2518f9..a9784144d6f2 100644 --- a/drivers/pci/bus.c +++ b/drivers/pci/bus.c @@ -311,6 +311,11 @@ void pci_bus_add_device(struct pci_dev *dev) { int retval; + if (pci_dev_is_ignored(dev)) { + pci_warn(dev, "%s: don't enable the ignored device\n", __func__); + return; + } + /* * Can not put in pci_device_add yet because resources * are not assigned yet for some devices. diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index e06e8692a7b1..56b905068ac5 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -366,6 +366,7 @@ static inline bool pci_dev_is_disconnected(const struct pci_dev *dev) /* pci_dev priv_flags */ #define PCI_DEV_ADDED 0 +#define PCI_DEV_IGNORE 1 static inline void pci_dev_assign_added(struct pci_dev *dev, bool added) { @@ -377,6 +378,16 @@ static inline bool pci_dev_is_added(const struct pci_dev *dev) return test_bit(PCI_DEV_ADDED, &dev->priv_flags); } +static inline void pci_dev_ignore(struct pci_dev *dev, bool ignore) +{ + assign_bit(PCI_DEV_IGNORE, &dev->priv_flags, ignore); +} + +static inline bool pci_dev_is_ignored(const struct pci_dev *dev) +{ + return test_bit(PCI_DEV_IGNORE, &dev->priv_flags); +} + #ifdef CONFIG_PCIEAER #include diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index de808ca28797..c1d88721697e 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -3172,6 +3172,23 @@ unsigned int pci_rescan_bus_bridge_resize(struct pci_dev *bridge) return max; } +static unsigned int pci_dev_res_mask(struct pci_dev *dev) +{ + unsigned int res_mask = 0; + int i; + + for (i = 0; i < PCI_BRIDGE_RESOURCES; i++) { + struct resource *r = &dev->resource[i]; + + if (!r->flags || (r->flags & IORESOURCE_UNSET) || !r->parent) + continue; + + res_mask |= (1 << i); + } + + return res_mask; +} + static void pci_bus_rescan_prepare(struct pci_bus *bus) { struct pci_dev *dev; @@ -3181,6 +3198,8 @@ static void pci_bus_rescan_prepare(struct pci_bus *bus) list_for_each_entry(dev, &bus->devices, bus_list) { struct pci_bus *child = dev->subordinate; + dev->res_mask = pci_dev_res_mask(dev); + if (child) { pci_bus_rescan_prepare(child); } else if (dev->driver && @@ -3242,6 +3261,84 @@ static void pci_setup_bridges(struct pci_bus *bus) pci_setup_bridge(bus); } +static struct pci_dev *pci_find_next_new_device(struct pci_bus *bus) +{ + struct pci_dev *dev; + + if (!bus) + return NULL; + + list_for_each_entry(dev, &bus->devices, bus_list) { + struct pci_bus *child_bus = dev->subordinate; + + if (!pci_dev_is_added(dev) && !pci_dev_is_ignored(dev)) + return dev; + + if (child_bus) { + struct pci_dev *next_new_dev; + + next_new_dev = pci_find_next_new_device(child_bus); + if (next_new_dev) + return next_new_dev; + } + } + + return NULL; +} + +static bool pci_bus_validate_resources(struct pci_bus *bus) +{ + struct pci_dev *dev; + bool ret = true; + + if (!bus) + return false; + + list_for_each_entry(dev, &bus->devices, bus_list) { + struct pci_bus *child = dev->subordinate; + unsigned int res_mask = pci_dev_res_mask(dev); + + if (pci_dev_is_ignored(dev)) + continue; + + if (dev->res_mask & ~res_mask) { + pci_err(dev, "%s: Non-re-enabled resources found: 0x%x -> 0x%x\n", + __func__, dev->res_mask, res_mask); + ret = false; + } + + if (child && !pci_bus_validate_resources(child)) + ret = false; + } + + return ret; +} + +static void pci_reassign_root_bus_resources(struct pci_bus *root) +{ + do { + struct pci_dev *next_new_dev; + + pci_bus_release_root_bridge_resources(root); + pci_assign_unassigned_root_bus_resources(root); + + if (pci_bus_validate_resources(root)) + break; + + next_new_dev = pci_find_next_new_device(root); + if (!next_new_dev) { + dev_err(&root->dev, "%s: failed to re-assign resources even after ignoring all the hotplugged devices\n", + __func__); + break; + } + + dev_warn(&root->dev, "%s: failed to re-assign resources, disable the next hotplugged device %s and retry\n", + __func__, dev_name(&next_new_dev->dev)); + + pci_dev_ignore(next_new_dev, true); + } while (true); +} + /** * pci_rescan_bus - Scan a PCI bus for devices * @bus: PCI bus to scan @@ -3265,8 +3362,7 @@ unsigned int pci_rescan_bus(struct pci_bus *bus) max = pci_scan_child_bus(root); - pci_bus_release_root_bridge_resources(root); - pci_assign_unassigned_root_bus_resources(root); + pci_reassign_root_bus_resources(root); pci_setup_bridges(root); pci_bus_rescan_done(root); diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index 07c6d9cc209f..92ce22d32f76 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -131,6 +131,9 @@ static void pdev_sort_resources(struct pci_dev *dev, struct list_head *head) { int i; + if (pci_dev_is_ignored(dev)) + return; + for (i = 0; i < PCI_NUM_RESOURCES; i++) { struct resource *r; struct pci_dev_resource *dev_res, *tmp; @@ -181,6 +184,9 @@ static void __dev_sort_resources(struct pci_dev *dev, { u16 class = dev->class >> 8; + if (pci_dev_is_ignored(dev)) + return; + /* Don't touch classless devices or host bridges or ioapics. */ if (class == PCI_CLASS_NOT_DEFINED || class == PCI_CLASS_BRIDGE_HOST) return; @@ -284,6 +290,9 @@ static void assign_requested_resources_sorted(struct list_head *head, int idx; list_for_each_entry(dev_res, head, list) { + if (pci_dev_is_ignored(dev_res->dev)) + continue; + res = dev_res->res; idx = res - &dev_res->dev->resource[0]; if (resource_size(res) && @@ -1028,6 +1037,9 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask, list_for_each_entry(dev, &bus->devices, bus_list) { int i; + if (pci_dev_is_ignored(dev)) + continue; + for (i = 0; i < PCI_NUM_RESOURCES; i++) { struct resource *r = &dev->resource[i]; resource_size_t r_size; @@ -1390,6 +1402,9 @@ void __pci_bus_assign_resources(const struct pci_bus *bus, pbus_assign_resources_sorted(bus, realloc_head, fail_head); list_for_each_entry(dev, &bus->devices, bus_list) { + if (pci_dev_is_ignored(dev)) + continue; + pdev_assign_fixed_resources(dev); b = dev->subordinate; diff --git a/include/linux/pci.h b/include/linux/pci.h index 5cd534b6631b..5c739bf0c94f 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -369,6 +369,7 @@ struct pci_dev { */ unsigned int irq; struct resource resource[DEVICE_COUNT_RESOURCE]; /* I/O and memory regions + expansion ROMs */ + unsigned int res_mask; /* Bitmask of assigned resources */ bool match_driver; /* Skip attaching driver */ From patchwork Mon Feb 4 15:35:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795983 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 039121823 for ; Mon, 4 Feb 2019 15:36:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E5C1C2ACAD for ; Mon, 4 Feb 2019 15:36:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DA2D72B71A; Mon, 4 Feb 2019 15:36:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8BAFE2B720 for ; Mon, 4 Feb 2019 15:36:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731337AbfBDPgZ (ORCPT ); Mon, 4 Feb 2019 10:36:25 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49270 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731330AbfBDPgY (ORCPT ); Mon, 4 Feb 2019 10:36:24 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 388DB41949; Mon, 4 Feb 2019 15:36:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294582; x=1551108983; bh=NPALoNSPN/ZGR/JOuhWP/ys3Y1RWMqEvoA5 Em/9/5wo=; b=vO7UiyYmZ21FFx2cMvN5NXbFYjhY+wAAALbKTPSm8OGGE+p+0Tq mBMFP+vH9nbfSewXigllwK+8TwGVmOBOc+K6E9HMDlzsxpevHz8LU5o+zpthJE5a /1n6PGpnp4H1GYEViIMDTGuRcIfQeZr2gdtVAt58fZSg8Br2JZgFZN94= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7SKxBz0YRsX3; Mon, 4 Feb 2019 18:36:22 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id D50C041958; Mon, 4 Feb 2019 18:36:12 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:12 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko Subject: [PATCH RFC v3 13/21] PCI: Include fixed BARs into the bus size calculating Date: Mon, 4 Feb 2019 18:35:53 +0300 Message-ID: <20190204153601.7576-14-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The only difference between the fixed and movable BARs is an offset preservation during the release+reassign procedure on PCIe rescan. When fixed BARs are included into the result of pbus_size_mem(), these BARs can be restricted: assign them to direct parents only. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/setup-bus.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index 92ce22d32f76..e6fba514e940 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1044,12 +1044,20 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask, struct resource *r = &dev->resource[i]; resource_size_t r_size; - if (r->parent || (r->flags & IORESOURCE_PCI_FIXED) || + if (r->parent || ((r->flags & mask) != type && (r->flags & mask) != type2 && (r->flags & mask) != type3)) continue; r_size = resource_size(r); + + if (r->flags & IORESOURCE_PCI_FIXED) { + if (pci_movable_bars_enabled()) + size += r_size; + + continue; + } + #ifdef CONFIG_PCI_IOV /* put SRIOV requested res to the optional list */ if (realloc_head && i >= PCI_IOV_RESOURCES && @@ -1388,6 +1396,8 @@ static void pdev_assign_fixed_resources(struct pci_dev *dev) while (b && !r->parent) { assign_fixed_resource_on_bus(b, r); b = b->parent; + if (!r->parent && pci_movable_bars_enabled()) + break; } } } From patchwork Mon Feb 4 15:35:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795973 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CC9DC1823 for ; Mon, 4 Feb 2019 15:36:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BB4842ACAD for ; Mon, 4 Feb 2019 15:36:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AF7A32B72F; Mon, 4 Feb 2019 15:36:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 69C842ACAD for ; Mon, 4 Feb 2019 15:36:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731330AbfBDPg1 (ORCPT ); Mon, 4 Feb 2019 10:36:27 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49376 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731335AbfBDPg0 (ORCPT ); Mon, 4 Feb 2019 10:36:26 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 1E9C541958; Mon, 4 Feb 2019 15:36:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294583; x=1551108984; bh=V+U4g+m7Dx+/3LbhG7nwC0CKK1VGcPBjyaZ UdQSSvIg=; b=F8dvKUArjDDaVJQKf3MFonLDPXGVZd8ggKfCsdhwcNBi9JUhcuc T3hSu0sQ3EAZcCsyfO51HIx9Ast+PJySLKALpbKrBl5FGzYGEzilVFK1tVLw8BOv 9o5Jlfnd6AMzNLL4jSn8xVhvCEttTF7ugrCXwaqsUqRLaLjDcPynY3yc= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Jlz6SIgFI5CB; Mon, 4 Feb 2019 18:36:23 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 10EA04122E; Mon, 4 Feb 2019 18:36:13 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:12 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko Subject: [PATCH RFC v3 14/21] PCI: Don't reserve memory for hotplug when enabled movable BARs Date: Mon, 4 Feb 2019 18:35:54 +0300 Message-ID: <20190204153601.7576-15-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP pbus_size_mem() returns a precise amount of memory required to fit all the requested BARs and windows of children bridges. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/setup-bus.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index e6fba514e940..f73bd3782120 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1266,7 +1266,7 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head) case PCI_CLASS_BRIDGE_PCI: pci_bridge_check_ranges(bus); - if (bus->self->is_hotplug_bridge) { + if (bus->self->is_hotplug_bridge && !pci_movable_bars_enabled()) { additional_io_size = pci_hotplug_io_size; additional_mem_size = pci_hotplug_mem_size; } From patchwork Mon Feb 4 15:35:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795971 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B43E61805 for ; Mon, 4 Feb 2019 15:36:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A2FF92B720 for ; Mon, 4 Feb 2019 15:36:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 97E462B723; Mon, 4 Feb 2019 15:36:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 48CD12B720 for ; Mon, 4 Feb 2019 15:36:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731346AbfBDPg1 (ORCPT ); Mon, 4 Feb 2019 10:36:27 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49386 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731336AbfBDPgZ (ORCPT ); Mon, 4 Feb 2019 10:36:25 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 234684122E; Mon, 4 Feb 2019 15:36:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294583; x=1551108984; bh=1pUHi71yZaE4qRYLCeuACunOQNmRND1Xn8j iL/pVcg0=; b=ED7rO7SxAWdxUI1Msl+6wpNh9Fr8W85/POMtGXgEBf0t93/6gBc VIuQuv7zGHjEMhSu0qAYbGaPAeFtPEjD8CxKfu8wCsmhmdWpGCxLuxATuDh3OHny 6P1uO4uXnP2BT2KB7yWRQL8S+pMrufVeAcSzxJGzQEaberXMlgqkFZFk= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WSbgzdR-Oeep; Mon, 4 Feb 2019 18:36:23 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 41BCE41957; Mon, 4 Feb 2019 18:36:13 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:12 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko Subject: [PATCH RFC v3 15/21] PCI: Allow the failed resources to be reassigned later Date: Mon, 4 Feb 2019 18:35:55 +0300 Message-ID: <20190204153601.7576-16-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Don't lose the size of the requested EP's BAR if it can't be fit in a current trial, so this can be retried. But a failed bridge window must be dropped and recalculated in the next trial. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/setup-bus.c | 3 ++- drivers/pci/setup-res.c | 12 ++++++++++++ 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index f73bd3782120..a1b1b1cda3e8 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -309,7 +309,8 @@ static void assign_requested_resources_sorted(struct list_head *head, 0 /* don't care */, 0 /* don't care */); } - reset_resource(res); + if (!pci_movable_bars_enabled()) + reset_resource(res); } } } diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c index d8ca40a97693..732d18f60f1b 100644 --- a/drivers/pci/setup-res.c +++ b/drivers/pci/setup-res.c @@ -298,6 +298,18 @@ static int _pci_assign_resource(struct pci_dev *dev, int resno, bus = dev->bus; while ((ret = __pci_assign_resource(bus, dev, resno, size, min_align))) { + if (pci_movable_bars_enabled()) { + if (resno >= PCI_BRIDGE_RESOURCES && + resno <= PCI_BRIDGE_RESOURCE_END) { + struct resource *res = dev->resource + resno; + + res->start = 0; + res->end = 0; + res->flags = 0; + } + break; + } + if (!bus->parent || !bus->self->transparent) break; bus = bus->parent; From patchwork Mon Feb 4 15:35:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795981 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ADECF14E1 for ; Mon, 4 Feb 2019 15:36:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9CE602ACAD for ; Mon, 4 Feb 2019 15:36:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 918502B723; Mon, 4 Feb 2019 15:36:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E780A2B71A for ; Mon, 4 Feb 2019 15:36:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731340AbfBDPg2 (ORCPT ); Mon, 4 Feb 2019 10:36:28 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49270 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731343AbfBDPg1 (ORCPT ); Mon, 4 Feb 2019 10:36:27 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 4EE37418BF; Mon, 4 Feb 2019 15:36:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294584; x=1551108985; bh=lnObJzqSWpLmiOuecnBxX8dHIT4V7sqiszw 829Ke50s=; b=qg/CSOhfC3HoOvT8lmyp2yyOUqdB5zK6cSgQ2Vdnp7AQCokvM9p BToN+/mwEIBqVS6GdNTDOCCi0LbBaVJOVoWqZ5Wsdw59MJ9pzS+zav5GNxlj5EV1 +8GEsKyQIq3FdvGhS8lI/8F0rauBxavsooKL+HiRkHXNnK+Q3k3Kq8vM= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rbfhejt_fIt3; Mon, 4 Feb 2019 18:36:24 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 738A04195A; Mon, 4 Feb 2019 18:36:13 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:13 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko Subject: [PATCH RFC v3 16/21] PCI: Calculate fixed areas of bridge windows based on fixed BARs Date: Mon, 4 Feb 2019 18:35:56 +0300 Message-ID: <20190204153601.7576-17-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP For every (IO, MEM, MEM64) bridge window, count the fixed resources of its children endpoints and children bridge windows: | <- BAR -> | | <- child bus fixed_range_hard -> | | <- fixed BAR -> | | <- bus's fixed_range_hard -> | | <- bus's bridge window -> | These ranges will be later used to arrange bridge windows in a way which covers every immovable BAR as well as the movable ones during hotplug. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/pci.h | 14 +++++++ drivers/pci/probe.c | 82 +++++++++++++++++++++++++++++++++++++++++ drivers/pci/setup-bus.c | 17 +++++++++ include/linux/pci.h | 6 +++ 4 files changed, 119 insertions(+) diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h index 56b905068ac5..14e3ebe68010 100644 --- a/drivers/pci/pci.h +++ b/drivers/pci/pci.h @@ -364,6 +364,20 @@ static inline bool pci_dev_is_disconnected(const struct pci_dev *dev) return dev->error_state == pci_channel_io_perm_failure; } +static inline int pci_get_bridge_resource_idx(struct resource *r) +{ + int idx = 1; + + if (r->flags & IORESOURCE_IO) + idx = 0; + else if (!(r->flags & IORESOURCE_PREFETCH)) + idx = 1; + else if (r->flags & IORESOURCE_MEM_64) + idx = 2; + + return idx; +} + /* pci_dev priv_flags */ #define PCI_DEV_ADDED 0 #define PCI_DEV_IGNORE 1 diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index c1d88721697e..cc302a5798bf 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -500,6 +500,7 @@ void pci_read_bridge_bases(struct pci_bus *child) static struct pci_bus *pci_alloc_bus(struct pci_bus *parent) { struct pci_bus *b; + int idx; b = kzalloc(sizeof(*b), GFP_KERNEL); if (!b) @@ -516,6 +517,11 @@ static struct pci_bus *pci_alloc_bus(struct pci_bus *parent) if (parent) b->domain_nr = parent->domain_nr; #endif + for (idx = 0; idx < PCI_BRIDGE_RESOURCE_NUM; ++idx) { + b->fixed_range_hard[idx].start = (resource_size_t)-1; + b->fixed_range_hard[idx].end = 0; + } + return b; } @@ -3261,6 +3267,81 @@ static void pci_setup_bridges(struct pci_bus *bus) pci_setup_bridge(bus); } +static void pci_bus_update_fixed_range_hard(struct pci_bus *bus) +{ + struct pci_dev *dev; + int idx; + resource_size_t start, end; + + for (idx = 0; idx < PCI_BRIDGE_RESOURCE_NUM; ++idx) { + bus->fixed_range_hard[idx].start = (resource_size_t)-1; + bus->fixed_range_hard[idx].end = 0; + } + + list_for_each_entry(dev, &bus->devices, bus_list) + if (dev->subordinate) + pci_bus_update_fixed_range_hard(dev->subordinate); + + list_for_each_entry(dev, &bus->devices, bus_list) { + int i; + + for (i = 0; i < PCI_BRIDGE_RESOURCES; ++i) { + struct resource *r = &dev->resource[i]; + + if (!r->flags || (r->flags & IORESOURCE_UNSET) || !r->parent) + continue; + + if (r->flags & IORESOURCE_PCI_FIXED) { + idx = pci_get_bridge_resource_idx(r); + start = bus->fixed_range_hard[idx].start; + end = bus->fixed_range_hard[idx].end; + + if (start > r->start) + start = r->start; + if (end < r->end) + end = r->end; + + if (bus->fixed_range_hard[idx].start != start || + bus->fixed_range_hard[idx].end != end) { + dev_dbg(&bus->dev, "%s: Found fixed 0x%llx-0x%llx in %s, expand the fixed bridge window %d to 0x%llx-0x%llx\n", + __func__, + (unsigned long long)r->start, + (unsigned long long)r->end, + dev_name(&dev->dev), idx, + (unsigned long long)start, + (unsigned long long)end); + bus->fixed_range_hard[idx].start = start; + bus->fixed_range_hard[idx].end = end; + } + } + } + + if (dev->subordinate) { + struct pci_bus *child = dev->subordinate; + + for (idx = 0; idx < PCI_BRIDGE_RESOURCE_NUM; ++idx) { + start = bus->fixed_range_hard[idx].start; + end = bus->fixed_range_hard[idx].end; + + if (start > child->fixed_range_hard[idx].start) + start = child->fixed_range_hard[idx].start; + if (end < child->fixed_range_hard[idx].end) + end = child->fixed_range_hard[idx].end; + + if (start < bus->fixed_range_hard[idx].start || + end > bus->fixed_range_hard[idx].end) { + dev_dbg(&bus->dev, "%s: Expand the fixed bridge window %d from %s to 0x%llx-0x%llx\n", + __func__, idx, dev_name(&child->dev), + (unsigned long long)start, + (unsigned long long)end); + bus->fixed_range_hard[idx].start = start; + bus->fixed_range_hard[idx].end = end; + } + } + } + } +} + static struct pci_dev *pci_find_next_new_device(struct pci_bus *bus) { struct pci_dev *dev; @@ -3361,6 +3442,7 @@ unsigned int pci_rescan_bus(struct pci_bus *bus) pci_bus_rescan_prepare(root); max = pci_scan_child_bus(root); + pci_bus_update_fixed_range_hard(root); pci_reassign_root_bus_resources(root); diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index a1b1b1cda3e8..59af85029869 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -916,9 +916,17 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size, resource_size_t children_add_size = 0; resource_size_t min_align, align; + resource_size_t fixed_start = bus->fixed_range_hard[0].start; + resource_size_t fixed_end = bus->fixed_range_hard[0].end; + resource_size_t fixed_size = (fixed_start < fixed_end) ? + (fixed_end - fixed_start + 1) : 0; + if (!b_res) return; + if (min_size < fixed_size) + min_size = fixed_size; + min_align = window_alignment(bus, IORESOURCE_IO); list_for_each_entry(dev, &bus->devices, bus_list) { int i; @@ -1027,6 +1035,15 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask, resource_size_t children_add_size = 0; resource_size_t children_add_align = 0; resource_size_t add_align = 0; + bool is_mem64 = (mask & IORESOURCE_MEM_64); + + resource_size_t fixed_start = bus->fixed_range_hard[is_mem64 ? 2 : 1].start; + resource_size_t fixed_end = bus->fixed_range_hard[is_mem64 ? 2 : 1].end; + resource_size_t fixed_size = (fixed_start < fixed_end) ? + (fixed_end - fixed_start + 1) : 0; + + if (min_size < fixed_size) + min_size = fixed_size; if (!b_res) return -ENOSPC; diff --git a/include/linux/pci.h b/include/linux/pci.h index 5c739bf0c94f..e2b4742c477a 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -569,6 +569,12 @@ struct pci_bus { struct list_head resources; /* Address space routed to this bus */ struct resource busn_res; /* Bus numbers routed to this bus */ + /* + * If there are fixed resources in the bridge window, the hard range + * contains the lowest and the highest addresses of them. + */ + struct resource fixed_range_hard[PCI_BRIDGE_RESOURCE_NUM]; + struct pci_ops *ops; /* Configuration access functions */ struct msi_controller *msi; /* MSI controller */ void *sysdata; /* Hook for sys-specific extension */ From patchwork Mon Feb 4 15:35:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795977 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D89A114E1 for ; Mon, 4 Feb 2019 15:36:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C5B512B720 for ; Mon, 4 Feb 2019 15:36:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BA5222B723; Mon, 4 Feb 2019 15:36:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 545D32ACAD for ; Mon, 4 Feb 2019 15:36:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731333AbfBDPg1 (ORCPT ); Mon, 4 Feb 2019 10:36:27 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49396 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731330AbfBDPg0 (ORCPT ); Mon, 4 Feb 2019 10:36:26 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 2A82B41957; Mon, 4 Feb 2019 15:36:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294584; x=1551108985; bh=Iht5/V7FVxHsASr0t5ZmGU1zlKkIy70F5Le ldIDJduo=; b=ETHMorzfeJaW5iopcBxM3p0i/gG6h/PUK0yILoX51s30/SeQ4Dl qt4HkG3bxbrIbMB9xrY713dxAO2KtV/ew5ykHRIieIYF8p5BeeSIOGSMQCQ/qzgy fxZ/VgBFGMWMiqyj3XDQdBqHuB5gV+N6uCGq/gCT8RfxpdfCev7jgkCg= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id adglySAQPIJ4; Mon, 4 Feb 2019 18:36:24 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id AC12F418BF; Mon, 4 Feb 2019 18:36:13 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:13 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko Subject: [PATCH RFC v3 17/21] PCI: Calculate boundaries for bridge windows Date: Mon, 4 Feb 2019 18:35:57 +0300 Message-ID: <20190204153601.7576-18-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If a bridge window contains fixed areas (there are PCIe devices with immovable BARs located on this bus), this window must be allocated within the bound memory area, limited by windows size and by address range of fixed resources, calculated as follows: | <-- bus's fixed_range_hard --> | | <-- fixed_range_hard.end - window size --> | | <-- fixed_range_hard.start + window size --> | | <-- bus's fixed_range_soft --> | Signed-off-by: Sergey Miroshnichenko --- drivers/pci/setup-bus.c | 56 +++++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 4 ++- 2 files changed, 59 insertions(+), 1 deletion(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index 59af85029869..be6c64d9bf33 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1846,6 +1846,61 @@ static enum enable_type pci_realloc_detect(struct pci_bus *bus, } #endif +static void pci_bus_update_fixed_range_soft(struct pci_bus *bus) +{ + struct pci_dev *dev; + struct pci_bus *parent = bus->parent; + int idx; + + list_for_each_entry(dev, &bus->devices, bus_list) + if (dev->subordinate) + pci_bus_update_fixed_range_soft(dev->subordinate); + + if (!parent || !bus->self) + return; + + for (idx = 0; idx < ARRAY_SIZE(bus->fixed_range_hard); ++idx) { + struct resource *r; + resource_size_t soft_start, soft_end; + resource_size_t hard_start = bus->fixed_range_hard[idx].start; + resource_size_t hard_end = bus->fixed_range_hard[idx].end; + + if (hard_start > hard_end) + continue; + + r = bus->resource[idx]; + + soft_start = hard_end - resource_size(r) + 1; + soft_end = hard_start + resource_size(r) - 1; + + if (soft_start > hard_start) + soft_start = hard_start; + + if (soft_end < hard_end) + soft_end = hard_end; + + list_for_each_entry(dev, &parent->devices, bus_list) { + struct pci_bus *sibling = dev->subordinate; + resource_size_t s_start, s_end; + + if (!sibling || sibling == bus) + continue; + + s_start = sibling->fixed_range_hard[idx].start; + s_end = sibling->fixed_range_hard[idx].end; + + if (s_start > s_end) + continue; + + if (s_end < hard_start && s_end > soft_start) + soft_start = s_end; + } + + bus->fixed_range_soft[idx].start = soft_start; + bus->fixed_range_soft[idx].end = soft_end; + } +} + /* * first try will not touch pci bridge res * second and later try will clear small leaf bridge res @@ -1884,6 +1939,7 @@ void pci_assign_unassigned_root_bus_resources(struct pci_bus *bus) /* Depth first, calculate sizes and alignments of all subordinate buses. */ __pci_bus_size_bridges(bus, add_list); + pci_bus_update_fixed_range_soft(bus); /* Depth last, allocate resources and update the hardware. */ __pci_bus_assign_resources(bus, add_list, &fail_head); diff --git a/include/linux/pci.h b/include/linux/pci.h index e2b4742c477a..311d51d0a59a 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -571,9 +571,11 @@ struct pci_bus { /* * If there are fixed resources in the bridge window, the hard range - * contains the lowest and the highest addresses of them. + * contains the lowest and the highest addresses of them, and this + * bridge window must reside within the soft range. */ struct resource fixed_range_hard[PCI_BRIDGE_RESOURCE_NUM]; + struct resource fixed_range_soft[PCI_BRIDGE_RESOURCE_NUM]; struct pci_ops *ops; /* Configuration access functions */ struct msi_controller *msi; /* MSI controller */ From patchwork Mon Feb 4 15:35:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795975 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E7ECA1805 for ; Mon, 4 Feb 2019 15:36:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D66582ACAD for ; Mon, 4 Feb 2019 15:36:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D3C7F2B72F; Mon, 4 Feb 2019 15:36:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 74CA02B71A for ; Mon, 4 Feb 2019 15:36:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731336AbfBDPg2 (ORCPT ); Mon, 4 Feb 2019 10:36:28 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49282 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731340AbfBDPg2 (ORCPT ); Mon, 4 Feb 2019 10:36:28 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 2B6094195A; Mon, 4 Feb 2019 15:36:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294585; x=1551108986; bh=ES4IHYHawY+rGNAmCsSDSgYU25ke2e/UjSw 9HcjfDXE=; b=P1Cichr7yw9/3ggUl1eSYtH3CCgSB/q53Nl4TIlELVWenZoM6Ye 45e6EOzYwVlvzYgIrhFNCrTdQsR4/M3c/nDxqtt7r0pqj/HypgOMlMUhFmyzSaXG xZRzDIOZ/CNuzTNpYPG2jIQxWD6/Bf/jQBYbL4sM/TqzEs1jjzuVuaq0= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uXIebw9GMRtv; Mon, 4 Feb 2019 18:36:25 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id DB89941960; Mon, 4 Feb 2019 18:36:13 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:13 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko Subject: [PATCH RFC v3 18/21] PCI: Make sure bridge windows include their fixed BARs Date: Mon, 4 Feb 2019 18:35:58 +0300 Message-ID: <20190204153601.7576-19-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Consider previously calculated boundaries when allocating a bridge window, setting the lowest allowed address and checking the result. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/bus.c | 2 +- drivers/pci/setup-res.c | 31 +++++++++++++++++++++++++++++-- 2 files changed, 30 insertions(+), 3 deletions(-) diff --git a/drivers/pci/bus.c b/drivers/pci/bus.c index a9784144d6f2..ce2d2aeedbd3 100644 --- a/drivers/pci/bus.c +++ b/drivers/pci/bus.c @@ -192,7 +192,7 @@ static int pci_bus_alloc_from_region(struct pci_bus *bus, struct resource *res, * this is an already-configured bridge window, its start * overrides "min". */ - if (avail.start) + if (min_used < avail.start) min_used = avail.start; max = avail.end; diff --git a/drivers/pci/setup-res.c b/drivers/pci/setup-res.c index 732d18f60f1b..04442339548d 100644 --- a/drivers/pci/setup-res.c +++ b/drivers/pci/setup-res.c @@ -248,9 +248,22 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev, struct resource *res = dev->resource + resno; resource_size_t min; int ret; + resource_size_t start = (resource_size_t)-1; + resource_size_t end = 0; min = (res->flags & IORESOURCE_IO) ? PCIBIOS_MIN_IO : PCIBIOS_MIN_MEM; + if (dev->subordinate && resno >= PCI_BRIDGE_RESOURCES) { + struct pci_bus *child_bus = dev->subordinate; + int b_resno = resno - PCI_BRIDGE_RESOURCES; + resource_size_t soft_start = child_bus->fixed_range_soft[b_resno].start; + + start = child_bus->fixed_range_hard[b_resno].start; + end = child_bus->fixed_range_hard[b_resno].end; + if (start < end) + min = soft_start; + } + /* * First, try exact prefetching match. Even if a 64-bit * prefetchable bridge window is below 4GB, we can't put a 32-bit @@ -262,7 +275,7 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev, IORESOURCE_PREFETCH | IORESOURCE_MEM_64, pcibios_align_resource, dev); if (ret == 0) - return 0; + goto check_fixed; /* * If the prefetchable window is only 32 bits wide, we can put @@ -274,7 +287,7 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev, IORESOURCE_PREFETCH, pcibios_align_resource, dev); if (ret == 0) - return 0; + goto check_fixed; } /* @@ -287,6 +300,20 @@ static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev, ret = pci_bus_alloc_resource(bus, res, size, align, min, 0, pcibios_align_resource, dev); +check_fixed: + if (ret == 0 && start < end) { + if (res->start > start || res->end < end) { + dev_err(&bus->dev, "%s: fixed area 0x%llx-0x%llx for %s doesn't fit in the allocated %pR (0x%llx-0x%llx)", + __func__, + (unsigned long long)start, (unsigned long long)end, + dev_name(&dev->dev), + res, (unsigned long long)res->start, + (unsigned long long)res->end); + release_resource(res); + return -1; + } + } + return ret; } From patchwork Mon Feb 4 15:35:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795985 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8609D14E1 for ; Mon, 4 Feb 2019 15:36:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 73FDF2B664 for ; Mon, 4 Feb 2019 15:36:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 684A62B720; Mon, 4 Feb 2019 15:36:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 036812B664 for ; Mon, 4 Feb 2019 15:36:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731351AbfBDPgd (ORCPT ); Mon, 4 Feb 2019 10:36:33 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49422 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730241AbfBDPgc (ORCPT ); Mon, 4 Feb 2019 10:36:32 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 1EB5441977; Mon, 4 Feb 2019 15:36:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294585; x=1551108986; bh=hZEDWvWqXBUqrszNs1idDsr9iYfPXn76G0M RQG+8EUo=; b=ojW7+VTq+Hv6nmHqhB3zJmfywtUkxaBaFP1XxJB8/gBw9neA4UZ naD+WqAZqmgF8ZzAWPhrtp9YbwhM5kzZ481r4dCHcmO+hmSDW7rsakMfZ1AdHdBC 9qqdHjLEqCSWrUEqU5FMdfHMXTlDnR9ReRINattT0Tdx2LGM0VO7m1Yc= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5pIpXL6tc9Qp; Mon, 4 Feb 2019 18:36:25 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 1AC0341961; Mon, 4 Feb 2019 18:36:14 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:13 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko Subject: [PATCH RFC v3 19/21] PCI: Prioritize fixed BAR assigning over the movable ones Date: Mon, 4 Feb 2019 18:35:59 +0300 Message-ID: <20190204153601.7576-20-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The allocated bridge windows are big enough to house all the children bridges and BARs, but the fixed resources must be assigned first, so the movable ones later divide the rest of the window. That's the assignment order: 1. Bridge windows with fixed areas; 2. Fixed BARs; 3. The rest of BARs and bridge windows. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/setup-bus.c | 69 ++++++++++++++++++++++++++++++++--------- 1 file changed, 55 insertions(+), 14 deletions(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index be6c64d9bf33..010911a0e458 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -272,31 +272,54 @@ static void reassign_resources_sorted(struct list_head *realloc_head, } } -/** - * assign_requested_resources_sorted() - satisfy resource requests - * - * @head : head of the list tracking requests for resources - * @fail_head : head of the list tracking requests that could - * not be allocated - * - * Satisfy resource requests of each element in the list. Add - * requests that could not satisfied to the failed_list. - */ -static void assign_requested_resources_sorted(struct list_head *head, - struct list_head *fail_head) +enum assign_step { + assign_fixed_bridge_windows, + assign_fixed_resources, + assign_float_resources, +}; + +static void _assign_requested_resources_sorted(struct list_head *head, + struct list_head *fail_head, + enum assign_step step) { struct resource *res; struct pci_dev_resource *dev_res; int idx; list_for_each_entry(dev_res, head, list) { + bool is_fixed; + bool is_fixed_bridge; + bool is_bridge; + if (pci_dev_is_ignored(dev_res->dev)) continue; res = dev_res->res; + if (!resource_size(res)) + continue; + idx = res - &dev_res->dev->resource[0]; - if (resource_size(res) && - pci_assign_resource(dev_res->dev, idx)) { + is_fixed = res->flags & IORESOURCE_PCI_FIXED; + is_bridge = dev_res->dev->subordinate && idx >= PCI_BRIDGE_RESOURCES; + + if (is_bridge) { + struct pci_bus *child = dev_res->dev->subordinate; + int b_res_idx = pci_get_bridge_resource_idx(res); + struct resource *fixed_res = &child->fixed_range_hard[b_res_idx]; + + is_fixed_bridge = fixed_res->start < fixed_res->end; + } else { + is_fixed_bridge = false; + } + + if (assign_fixed_bridge_windows == step && !is_fixed_bridge) + continue; + else if (assign_fixed_resources == step && (!is_fixed || is_bridge)) + continue; + else if (assign_float_resources == step && (is_fixed || is_fixed_bridge)) + continue; + + if (pci_assign_resource(dev_res->dev, idx)) { if (fail_head) { /* * if the failed res is for ROM BAR, and it will @@ -315,6 +338,24 @@ static void assign_requested_resources_sorted(struct list_head *head, } } +/** + * assign_requested_resources_sorted() - satisfy resource requests + * + * @head : head of the list tracking requests for resources + * @fail_head : head of the list tracking requests that could + * not be allocated + * + * Satisfy resource requests of each element in the list. Add + * requests that could not satisfied to the failed_list. + */ +static void assign_requested_resources_sorted(struct list_head *head, + struct list_head *fail_head) +{ + _assign_requested_resources_sorted(head, fail_head, assign_fixed_bridge_windows); + _assign_requested_resources_sorted(head, fail_head, assign_fixed_resources); + _assign_requested_resources_sorted(head, fail_head, assign_float_resources); +} + static unsigned long pci_fail_res_type_mask(struct list_head *fail_head) { struct pci_dev_resource *fail_res; From patchwork Mon Feb 4 15:36:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795979 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8352D922 for ; Mon, 4 Feb 2019 15:36:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 70EEE2ACAD for ; Mon, 4 Feb 2019 15:36:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 656CD2B723; Mon, 4 Feb 2019 15:36:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1E6382ACAD for ; Mon, 4 Feb 2019 15:36:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731278AbfBDPg3 (ORCPT ); Mon, 4 Feb 2019 10:36:29 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49368 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731335AbfBDPg1 (ORCPT ); Mon, 4 Feb 2019 10:36:27 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 0C78541973; Mon, 4 Feb 2019 15:36:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294586; x=1551108987; bh=OgLo7/CRt0OwDQd8jzrs/7oiC/sWradzhSM CAbKv7SA=; b=ftASL8qHgo7+MdJaCUEgA8JQP2VPhJPjVwbY94D0WWYo4ZqaJhn ITgnnoJhaBWyWa9Hpd2QzHccK9EnHTJ3Vk7G9YXQ7KtaK3zlyp3ROGiSke0gWyEa YzOlWeQlJrxHrzxL1n1xboCcPf7TE4TJKPGpOv3vrLhEfRkjOfOPGBp0= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id f5AM_CeScNXy; Mon, 4 Feb 2019 18:36:26 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 4953141962; Mon, 4 Feb 2019 18:36:14 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:13 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko Subject: [PATCH RFC v3 20/21] PCI: pciehp: Add support for the movable BARs feature Date: Mon, 4 Feb 2019 18:36:00 +0300 Message-ID: <20190204153601.7576-21-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP With movable BARs, adding a hotplugged device may affect all the PCIe domain starting from the root, so use a pci_rescan_bus() function which handles the rearrangement of existing BARs and bridge windows. Signed-off-by: Sergey Miroshnichenko --- drivers/pci/hotplug/pciehp_pci.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/drivers/pci/hotplug/pciehp_pci.c b/drivers/pci/hotplug/pciehp_pci.c index b9c1396db6fe..7c0871db5bae 100644 --- a/drivers/pci/hotplug/pciehp_pci.c +++ b/drivers/pci/hotplug/pciehp_pci.c @@ -56,12 +56,16 @@ int pciehp_configure_device(struct controller *ctrl) goto out; } - for_each_pci_bridge(dev, parent) - pci_hp_add_bridge(dev); + if (pci_movable_bars_enabled()) { + pci_rescan_bus(parent); + } else { + for_each_pci_bridge(dev, parent) + pci_hp_add_bridge(dev); - pci_assign_unassigned_bridge_resources(bridge); - pcie_bus_configure_settings(parent); - pci_bus_add_devices(parent); + pci_assign_unassigned_bridge_resources(bridge); + pcie_bus_configure_settings(parent); + pci_bus_add_devices(parent); + } out: pci_unlock_rescan_remove(); From patchwork Mon Feb 4 15:36:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sergei Miroshnichenko X-Patchwork-Id: 10795987 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1F08C922 for ; Mon, 4 Feb 2019 15:36:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0D2F22B664 for ; Mon, 4 Feb 2019 15:36:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 018D12B720; Mon, 4 Feb 2019 15:36:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AC7172B664 for ; Mon, 4 Feb 2019 15:36:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730241AbfBDPge (ORCPT ); Mon, 4 Feb 2019 10:36:34 -0500 Received: from mta-01.yadro.com ([89.207.88.251]:49428 "EHLO mta-01.yadro.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731343AbfBDPgd (ORCPT ); Mon, 4 Feb 2019 10:36:33 -0500 Received: from localhost (unknown [127.0.0.1]) by mta-01.yadro.com (Postfix) with ESMTP id 9CCC241961; Mon, 4 Feb 2019 15:36:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yadro.com; h= content-type:content-type:content-transfer-encoding:mime-version :references:in-reply-to:x-mailer:message-id:date:date:subject :subject:from:from:received:received:received; s=mta-01; t= 1549294587; x=1551108988; bh=z8sl9oc5HN9be07G2Rla1odORYqrXxkuLW/ tx5e5wbY=; b=GuqoSq+x4TchnIw3mmZHWzK1waXlNdtujnf05YeoLlKXtN5dfH2 WokKSnnEAg4vnN2gnexzTOl4/S8BV7HwFyQ4NTLb6bC6zPV5jOkEfDXPFSebxvGI c80esED1+8dEJnS8PAwtWgIEdrNEpH/MFlXLVtMaMZKuwNFkLyV3yGGw= X-Virus-Scanned: amavisd-new at yadro.com Received: from mta-01.yadro.com ([127.0.0.1]) by localhost (mta-01.yadro.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2mK_Pc1wXbU6; Mon, 4 Feb 2019 18:36:27 +0300 (MSK) Received: from T-EXCH-02.corp.yadro.com (t-exch-02.corp.yadro.com [172.17.10.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by mta-01.yadro.com (Postfix) with ESMTPS id 7F05341963; Mon, 4 Feb 2019 18:36:14 +0300 (MSK) Received: from NB-148.yadro.com (172.17.15.60) by T-EXCH-02.corp.yadro.com (172.17.10.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.669.32; Mon, 4 Feb 2019 18:36:14 +0300 From: Sergey Miroshnichenko To: CC: Bjorn Helgaas , Rajat Jain , , Sergey Miroshnichenko , , Benjamin Herrenschmidt , Oliver , Sam Bobroff Subject: [PATCH RFC v3 21/21] powerpc/pci: Fix crash with enabled movable BARs Date: Mon, 4 Feb 2019 18:36:01 +0300 Message-ID: <20190204153601.7576-22-s.miroshnichenko@yadro.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190204153601.7576-1-s.miroshnichenko@yadro.com> References: <20190204153601.7576-1-s.miroshnichenko@yadro.com> MIME-Version: 1.0 X-Originating-IP: [172.17.15.60] X-ClientProxiedBy: T-EXCH-01.corp.yadro.com (172.17.10.101) To T-EXCH-02.corp.yadro.com (172.17.10.102) Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Check a resource for the UNSET flags. Signed-off-by: Sergey Miroshnichenko --- arch/powerpc/platforms/powernv/pci-ioda.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c index 7db3119f8a5b..5354dfdf1028 100644 --- a/arch/powerpc/platforms/powernv/pci-ioda.c +++ b/arch/powerpc/platforms/powernv/pci-ioda.c @@ -3038,7 +3038,8 @@ static void pnv_ioda_setup_pe_res(struct pnv_ioda_pe *pe, int index; int64_t rc; - if (!res || !res->flags || res->start > res->end) + if (!res || !res->flags || res->start > res->end || + (res->flags & IORESOURCE_UNSET)) return; if (res->flags & IORESOURCE_IO) {