From patchwork Mon Jul 1 14:22:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Johnson X-Patchwork-Id: 11026119 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 63544138D for ; Mon, 1 Jul 2019 14:22:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 515D028600 for ; Mon, 1 Jul 2019 14:22:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 43F8128680; Mon, 1 Jul 2019 14:22:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1CF6C28600 for ; Mon, 1 Jul 2019 14:22:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729536AbfGAOWu convert rfc822-to-8bit (ORCPT ); Mon, 1 Jul 2019 10:22:50 -0400 Received: from mail-oln040092253101.outbound.protection.outlook.com ([40.92.253.101]:29710 "EHLO APC01-SG2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727064AbfGAOWu (ORCPT ); Mon, 1 Jul 2019 10:22:50 -0400 Received: from PU1APC01FT027.eop-APC01.prod.protection.outlook.com (10.152.252.59) by PU1APC01HT071.eop-APC01.prod.protection.outlook.com (10.152.253.141) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2032.15; Mon, 1 Jul 2019 14:22:43 +0000 Received: from SL2P216MB0187.KORP216.PROD.OUTLOOK.COM (10.152.252.60) by PU1APC01FT027.mail.protection.outlook.com (10.152.252.232) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2032.15 via Frontend Transport; Mon, 1 Jul 2019 14:22:43 +0000 Received: from SL2P216MB0187.KORP216.PROD.OUTLOOK.COM ([fe80::9d2d:391f:5f49:c806]) by SL2P216MB0187.KORP216.PROD.OUTLOOK.COM ([fe80::9d2d:391f:5f49:c806%6]) with mapi id 15.20.2032.019; Mon, 1 Jul 2019 14:22:43 +0000 From: Nicholas Johnson To: "linux-kernel@vger.kernel.org" CC: "linux-pci@vger.kernel.org" , "bhelgaas@google.com" , "mika.westerberg@linux.intel.com" , "corbet@lwn.net" , "benh@kernel.crashing.org" , "logang@deltatee.com" Subject: [PATCH v7 1/8] PCI: Simplify pci_bus_distribute_available_resources() Thread-Topic: [PATCH v7 1/8] PCI: Simplify pci_bus_distribute_available_resources() Thread-Index: AQHVMBhy/j20QAW14Ea8AXyV6YBqSg== Date: Mon, 1 Jul 2019 14:22:43 +0000 Message-ID: Accept-Language: en-AU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: SYXPR01CA0145.ausprd01.prod.outlook.com (2603:10c6:0:30::30) To SL2P216MB0187.KORP216.PROD.OUTLOOK.COM (2603:1096:100:22::19) x-incomingtopheadermarker: OriginalChecksum:B230098E27C8EDA92CC7E2DBE1387CD9CC8D631EA0F50958B205DE44B3456819;UpperCasedChecksum:41E9E1A05564A214069764DF0052BE4F90A6BD480842D23F50CB52839FD60D45;SizeAsReceived:7687;Count:47 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [kdAE7IwH8mYXH30fIrStVqN88iK95+Oa20R+Wx5L8ge+FEYn+KAqewpZZqMQ3NLDOWOKNAC7nXI=] x-microsoft-original-message-id: <20190701142227.GA5156@nicholas-usb> x-ms-publictraffictype: Email x-incomingheadercount: 47 x-eopattributedmessage: 0 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(5050001)(7020095)(20181119110)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031322404)(2017031323274)(2017031324274)(1601125500)(1603101475)(1701031045);SRVR:PU1APC01HT071; x-ms-traffictypediagnostic: PU1APC01HT071: x-ms-exchange-purlcount: 1 x-microsoft-antispam-message-info: hYHyYsxG0xfjs6MVFuZN62th1RBTazy/a9++SSayg/dhx2bfZSY6J0H5rz0pNB+siFmDMRQlthFD1M5F4LaJ3Wf/pgqJrta9f3axUTyjnjBs1WOkSk/JX3Kj711F6gGEuHd+eTnJTnEDurRWUoshPdwkKWNPfkY28xDwXHnKj21YrNk5tLDxa/tftoltGAMH Content-ID: <058E6F501B00A346BF0D296F2597AF77@KORP216.PROD.OUTLOOK.COM> MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 82559e89-cf13-4106-de24-08d6fe2f948a X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jul 2019 14:22:43.1713 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: PU1APC01HT071 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Reorder pci_bus_distribute_available_resources() to group related code together. No functional change intended. Link: https://lore.kernel.org/r/PS2P216MB0642C7A485649D2D787A1C6F80000@PS2P216MB0642.KORP216.PROD.OUTLOOK.COM Based-on-patch-by: Nicholas Johnson [bhelgaas: extracted from larger patch] Signed-off-by: Nicholas Johnson --- drivers/pci/setup-bus.c | 50 ++++++++++++++++++++--------------------- 1 file changed, 25 insertions(+), 25 deletions(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index 0cdd5ff38..af28af898 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1860,16 +1860,6 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus, extend_bridge_window(bridge, mmio_pref_res, add_list, available_mmio_pref); - /* - * Calculate the total amount of extra resource space we can - * pass to bridges below this one. This is basically the - * extra space reduced by the minimal required space for the - * non-hotplug bridges. - */ - remaining_io = available_io; - remaining_mmio = available_mmio; - remaining_mmio_pref = available_mmio_pref; - /* * Calculate how many hotplug bridges and normal bridges there * are on this bus. We will distribute the additional available @@ -1882,6 +1872,31 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus, normal_bridges++; } + /* + * There is only one bridge on the bus so it gets all available + * resources which it can then distribute to the possible hotplug + * bridges below. + */ + if (hotplug_bridges + normal_bridges == 1) { + dev = list_first_entry(&bus->devices, struct pci_dev, bus_list); + if (dev->subordinate) { + pci_bus_distribute_available_resources(dev->subordinate, + add_list, available_io, available_mmio, + available_mmio_pref); + } + return; + } + + /* + * Calculate the total amount of extra resource space we can + * pass to bridges below this one. This is basically the + * extra space reduced by the minimal required space for the + * non-hotplug bridges. + */ + remaining_io = available_io; + remaining_mmio = available_mmio; + remaining_mmio_pref = available_mmio_pref; + for_each_pci_bridge(dev, bus) { const struct resource *res; @@ -1905,21 +1920,6 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus, remaining_mmio_pref -= resource_size(res); } - /* - * There is only one bridge on the bus so it gets all available - * resources which it can then distribute to the possible hotplug - * bridges below. - */ - if (hotplug_bridges + normal_bridges == 1) { - dev = list_first_entry(&bus->devices, struct pci_dev, bus_list); - if (dev->subordinate) { - pci_bus_distribute_available_resources(dev->subordinate, - add_list, available_io, available_mmio, - available_mmio_pref); - } - return; - } - /* * Go over devices on this bus and distribute the remaining * resource space between hotplug bridges. From patchwork Mon Jul 1 14:23:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Johnson X-Patchwork-Id: 11026125 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 07780138D for ; Mon, 1 Jul 2019 14:23:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EC21428600 for ; Mon, 1 Jul 2019 14:23:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DE82A28680; Mon, 1 Jul 2019 14:23:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8489C28600 for ; Mon, 1 Jul 2019 14:23:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729535AbfGAOXg convert rfc822-to-8bit (ORCPT ); Mon, 1 Jul 2019 10:23:36 -0400 Received: from mail-oln040092254109.outbound.protection.outlook.com ([40.92.254.109]:14571 "EHLO APC01-PU1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728129AbfGAOXg (ORCPT ); Mon, 1 Jul 2019 10:23:36 -0400 Received: from PU1APC01FT027.eop-APC01.prod.protection.outlook.com (10.152.252.57) by PU1APC01HT089.eop-APC01.prod.protection.outlook.com (10.152.253.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2032.15; Mon, 1 Jul 2019 14:23:32 +0000 Received: from SL2P216MB0187.KORP216.PROD.OUTLOOK.COM (10.152.252.60) by PU1APC01FT027.mail.protection.outlook.com (10.152.252.232) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2032.15 via Frontend Transport; Mon, 1 Jul 2019 14:23:32 +0000 Received: from SL2P216MB0187.KORP216.PROD.OUTLOOK.COM ([fe80::9d2d:391f:5f49:c806]) by SL2P216MB0187.KORP216.PROD.OUTLOOK.COM ([fe80::9d2d:391f:5f49:c806%6]) with mapi id 15.20.2032.019; Mon, 1 Jul 2019 14:23:32 +0000 From: Nicholas Johnson To: "linux-kernel@vger.kernel.org" CC: "linux-pci@vger.kernel.org" , "bhelgaas@google.com" , "mika.westerberg@linux.intel.com" , "corbet@lwn.net" , "benh@kernel.crashing.org" , "logang@deltatee.com" Subject: [PATCH v7 2/8] PCI: Skip resource distribution when no hotplug bridges Thread-Topic: [PATCH v7 2/8] PCI: Skip resource distribution when no hotplug bridges Thread-Index: AQHVMBiP5bDg6T7Bu0SPG+HDJyh4Cw== Date: Mon, 1 Jul 2019 14:23:32 +0000 Message-ID: Accept-Language: en-AU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: SYBPR01CA0182.ausprd01.prod.outlook.com (2603:10c6:10:52::26) To SL2P216MB0187.KORP216.PROD.OUTLOOK.COM (2603:1096:100:22::19) x-incomingtopheadermarker: OriginalChecksum:7E4747C3D9BF8D3B8D4313F0B691BF67281AFE05A8C03827655A689AE334C134;UpperCasedChecksum:9E143170F26BA325A71C9F50700EB9153257DCE0AAF7BA0944A9B49B30E053F5;SizeAsReceived:7696;Count:47 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [QNw58P8eezGIQg1bREdWRCZcMV+0/IjsjvUvREBKuWdRTNv/ODR0hNWoMttjATb64hXHXluKyfg=] x-microsoft-original-message-id: <20190701142316.GA5201@nicholas-usb> x-ms-publictraffictype: Email x-incomingheadercount: 47 x-eopattributedmessage: 0 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(5050001)(7020095)(20181119110)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031322404)(2017031323274)(2017031324274)(1601125500)(1603101475)(1701031045);SRVR:PU1APC01HT089; x-ms-traffictypediagnostic: PU1APC01HT089: x-microsoft-antispam-message-info: YuQZyMB/0v1jQQihjQLw6sHpnD8+3F50hZtAfGQendM2yXCxgnjRXH8GL+29tzKhgxQEC20Q/EnEG8gT8F755DWeJVHklqFH8GbA9IyhVBnai+gSZ3vtF+iNkFKhgHYURVoH3Lfnj0ssdBZNaMXLbkyynElUn8zjRmgFhtwtvmCJTbDMIdYKDXup/iwjS+We Content-ID: <7E2897BBBFD585449B559074D51FAD9A@KORP216.PROD.OUTLOOK.COM> MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: a3b34ab7-6744-4208-039f-08d6fe2fb1d7 X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jul 2019 14:23:32.3292 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: PU1APC01HT089 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If "hotplug_bridges == 0", "!dev->is_hotplug_bridge" is always true, so the loop that divides the remaining resources among hotplug-capable bridges does nothing. Check for "hotplug_bridges == 0" earlier, so we don't even have to compute the amount of remaining resources. No functional change intended. Signed-off-by: Nicholas Johnson --- drivers/pci/setup-bus.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index af28af898..04adeebe8 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1887,6 +1887,9 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus, return; } + if (hotplug_bridges == 0) + return; + /* * Calculate the total amount of extra resource space we can * pass to bridges below this one. This is basically the @@ -1936,8 +1939,6 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus, * Distribute available extra resources equally between * hotplug-capable downstream ports taking alignment into * account. - * - * Here hotplug_bridges is always != 0. */ align = pci_resource_alignment(bridge, io_res); io = div64_ul(available_io, hotplug_bridges); From patchwork Mon Jul 1 14:23:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Johnson X-Patchwork-Id: 11026127 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 799FF138D for ; Mon, 1 Jul 2019 14:24:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6707C28600 for ; Mon, 1 Jul 2019 14:24:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5B09528680; Mon, 1 Jul 2019 14:24:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5F20428600 for ; Mon, 1 Jul 2019 14:24:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727571AbfGAOYH convert rfc822-to-8bit (ORCPT ); Mon, 1 Jul 2019 10:24:07 -0400 Received: from mail-oln040092253015.outbound.protection.outlook.com ([40.92.253.15]:6049 "EHLO APC01-SG2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728129AbfGAOYH (ORCPT ); Mon, 1 Jul 2019 10:24:07 -0400 Received: from PU1APC01FT027.eop-APC01.prod.protection.outlook.com (10.152.252.54) by PU1APC01HT043.eop-APC01.prod.protection.outlook.com (10.152.253.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2032.15; Mon, 1 Jul 2019 14:24:00 +0000 Received: from SL2P216MB0187.KORP216.PROD.OUTLOOK.COM (10.152.252.60) by PU1APC01FT027.mail.protection.outlook.com (10.152.252.232) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2032.15 via Frontend Transport; Mon, 1 Jul 2019 14:23:59 +0000 Received: from SL2P216MB0187.KORP216.PROD.OUTLOOK.COM ([fe80::9d2d:391f:5f49:c806]) by SL2P216MB0187.KORP216.PROD.OUTLOOK.COM ([fe80::9d2d:391f:5f49:c806%6]) with mapi id 15.20.2032.019; Mon, 1 Jul 2019 14:23:58 +0000 From: Nicholas Johnson To: "linux-kernel@vger.kernel.org" CC: "linux-pci@vger.kernel.org" , "bhelgaas@google.com" , "mika.westerberg@linux.intel.com" , "corbet@lwn.net" , "benh@kernel.crashing.org" , "logang@deltatee.com" Subject: [PATCH v7 3/8] PCI: Consider alignment of hot-added bridges when distributing available resources Thread-Topic: [PATCH v7 3/8] PCI: Consider alignment of hot-added bridges when distributing available resources Thread-Index: AQHVMBieC8Grv/1EnkOx4VczNSDx2g== Date: Mon, 1 Jul 2019 14:23:57 +0000 Message-ID: Accept-Language: en-AU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: SYBPR01CA0104.ausprd01.prod.outlook.com (2603:10c6:10:1::20) To SL2P216MB0187.KORP216.PROD.OUTLOOK.COM (2603:1096:100:22::19) x-incomingtopheadermarker: OriginalChecksum:1D76EC3B72827235804F281AD00B8A222EDEB6B1BF473C52C1A9B225E92B4168;UpperCasedChecksum:3E12A8D272F3678E200AFE8D16B5BB1E15B91891A8C9179D709A26E1DC3A02D3;SizeAsReceived:7746;Count:47 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [O0o0hVExPGW/9OJjRFJI1EoPoviYTBaIrGQ12svwM4S6MHTudLXyQN30QB9FkXucPjI+qjMvnGU=] x-microsoft-original-message-id: <20190701142342.GA5232@nicholas-usb> x-ms-publictraffictype: Email x-incomingheadercount: 47 x-eopattributedmessage: 0 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(5050001)(7020095)(20181119110)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031322404)(2017031323274)(2017031324274)(1601125500)(1603101475)(1701031045);SRVR:PU1APC01HT043; x-ms-traffictypediagnostic: PU1APC01HT043: x-ms-exchange-purlcount: 1 x-microsoft-antispam-message-info: uIW0kEQs9I9x8OtAjQ2KIl9qZohRCdcXjIBIR8w3mG6gjFV7Cb0EDYTc95jknyDkIKctKklsJJTd5bIyfTa5+RTClHdE9HbOxiticDh7xbTTSI7OZIEvykyaAM+JtXvndkY3CuTR/0UxwpO3llQnpviGGNK9uFbq22hvi3gLUU5zWrrMm53OgPEs2eADTsUG Content-ID: <91FD1686E01FF5468DE2AA64BA8C428F@KORP216.PROD.OUTLOOK.COM> MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: dd23f92c-d416-497f-d8e0-08d6fe2fc10f X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jul 2019 14:23:57.9756 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: PU1APC01HT043 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Rewrite pci_bus_distribute_available_resources to better handle bridges with different resource alignment requirements. Pass more details arguments recursively to track the resource start and end addresses relative to the initial hotplug bridge. This is especially useful for Thunderbolt with native PCI enumeration, enabling external graphics cards and other devices with bridge alignment higher than 1MB. Change extend_bridge_window to resize the actual resource, rather than using add_list and dev_res->add_size. If an additional resource entry exists for the given resource, zero out the add_size field to avoid it interfering. Because add_size is considered optional when allocating, using add_size could cause issues in some cases, because successful resource distribution requires sizes to be guaranteed. Such cases include hot-adding nested hotplug bridges in one enumeration, and potentially others which are yet to be encountered. Solves bug report: https://bugzilla.kernel.org/show_bug.cgi?id=199581 Reported-by: Mika Westerberg Signed-off-by: Nicholas Johnson --- drivers/pci/setup-bus.c | 148 +++++++++++++++++++--------------------- 1 file changed, 71 insertions(+), 77 deletions(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index 04adeebe8..898fea00c 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1835,12 +1835,10 @@ static void extend_bridge_window(struct pci_dev *bridge, struct resource *res, } static void pci_bus_distribute_available_resources(struct pci_bus *bus, - struct list_head *add_list, - resource_size_t available_io, - resource_size_t available_mmio, - resource_size_t available_mmio_pref) + struct list_head *add_list, struct resource io, + struct resource mmio, struct resource mmio_pref) { - resource_size_t remaining_io, remaining_mmio, remaining_mmio_pref; + resource_size_t io_per_hp, mmio_per_hp, mmio_pref_per_hp, align; unsigned int normal_bridges = 0, hotplug_bridges = 0; struct resource *io_res, *mmio_res, *mmio_pref_res; struct pci_dev *dev, *bridge = bus->self; @@ -1850,15 +1848,29 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus, mmio_pref_res = &bridge->resource[PCI_BRIDGE_RESOURCES + 2]; /* - * Update additional resource list (add_list) to fill all the - * extra resource space available for this port except the space - * calculated in __pci_bus_size_bridges() which covers all the - * devices currently connected to the port and below. + * The alignment of this bridge is yet to be considered, hence it must + * be done now before extending its bridge window. */ - extend_bridge_window(bridge, io_res, add_list, available_io); - extend_bridge_window(bridge, mmio_res, add_list, available_mmio); + align = pci_resource_alignment(bridge, io_res); + if (!io_res->parent && align) + io.start = ALIGN(io.start, align); + + align = pci_resource_alignment(bridge, mmio_res); + if (!mmio_res->parent && align) + mmio.start = ALIGN(mmio.start, align); + + align = pci_resource_alignment(bridge, mmio_pref_res); + if (!mmio_pref_res->parent && align) + mmio_pref.start = ALIGN(mmio_pref.start, align); + + /* + * Update the resources to fill as much remaining resource space in the + * parent bridge as possible, while considering alignment. + */ + extend_bridge_window(bridge, io_res, add_list, resource_size(&io)); + extend_bridge_window(bridge, mmio_res, add_list, resource_size(&mmio)); extend_bridge_window(bridge, mmio_pref_res, add_list, - available_mmio_pref); + resource_size(&mmio_pref)); /* * Calculate how many hotplug bridges and normal bridges there @@ -1879,108 +1891,90 @@ static void pci_bus_distribute_available_resources(struct pci_bus *bus, */ if (hotplug_bridges + normal_bridges == 1) { dev = list_first_entry(&bus->devices, struct pci_dev, bus_list); - if (dev->subordinate) { + if (dev->subordinate) pci_bus_distribute_available_resources(dev->subordinate, - add_list, available_io, available_mmio, - available_mmio_pref); - } + add_list, io, mmio, mmio_pref); return; } - if (hotplug_bridges == 0) - return; - /* - * Calculate the total amount of extra resource space we can - * pass to bridges below this one. This is basically the - * extra space reduced by the minimal required space for the - * non-hotplug bridges. + * Reduce the available resource space by what the + * bridge and devices below it occupy. */ - remaining_io = available_io; - remaining_mmio = available_mmio; - remaining_mmio_pref = available_mmio_pref; - for_each_pci_bridge(dev, bus) { - const struct resource *res; + struct resource *res; + resource_size_t used_size; if (dev->is_hotplug_bridge) continue; - /* - * Reduce the available resource space by what the - * bridge and devices below it occupy. - */ res = &dev->resource[PCI_BRIDGE_RESOURCES + 0]; - if (!res->parent && available_io > resource_size(res)) - remaining_io -= resource_size(res); + align = pci_resource_alignment(dev, res); + align = align ? ALIGN(io.start, align) - io.start : 0; + used_size = align + resource_size(res); + if (!res->parent && used_size <= resource_size(&io)) + io.start += used_size; res = &dev->resource[PCI_BRIDGE_RESOURCES + 1]; - if (!res->parent && available_mmio > resource_size(res)) - remaining_mmio -= resource_size(res); + align = pci_resource_alignment(dev, res); + align = align ? ALIGN(mmio.start, align) - mmio.start : 0; + used_size = align + resource_size(res); + if (!res->parent && used_size <= resource_size(&mmio)) + mmio.start += used_size; res = &dev->resource[PCI_BRIDGE_RESOURCES + 2]; - if (!res->parent && available_mmio_pref > resource_size(res)) - remaining_mmio_pref -= resource_size(res); + align = pci_resource_alignment(dev, res); + align = align ? ALIGN(mmio_pref.start, align) - + mmio_pref.start : 0; + used_size = align + resource_size(res); + if (!res->parent && used_size <= resource_size(&mmio_pref)) + mmio_pref.start += used_size; } + if (!hotplug_bridges) + return; + /* - * Go over devices on this bus and distribute the remaining - * resource space between hotplug bridges. + * Distribute any remaining resources equally between + * the hotplug-capable downstream ports. */ - for_each_pci_bridge(dev, bus) { - resource_size_t align, io, mmio, mmio_pref; - struct pci_bus *b; + io_per_hp = div64_ul(resource_size(&io), hotplug_bridges); + mmio_per_hp = div64_ul(resource_size(&mmio), hotplug_bridges); + mmio_pref_per_hp = div64_ul(resource_size(&mmio_pref), + hotplug_bridges); - b = dev->subordinate; - if (!b || !dev->is_hotplug_bridge) + for_each_pci_bridge(dev, bus) { + if (!dev->subordinate || !dev->is_hotplug_bridge) continue; - /* - * Distribute available extra resources equally between - * hotplug-capable downstream ports taking alignment into - * account. - */ - align = pci_resource_alignment(bridge, io_res); - io = div64_ul(available_io, hotplug_bridges); - io = min(ALIGN(io, align), remaining_io); - remaining_io -= io; - - align = pci_resource_alignment(bridge, mmio_res); - mmio = div64_ul(available_mmio, hotplug_bridges); - mmio = min(ALIGN(mmio, align), remaining_mmio); - remaining_mmio -= mmio; + io.end = io.start + io_per_hp - 1; + mmio.end = mmio.start + mmio_per_hp - 1; + mmio_pref.end = mmio_pref.start + mmio_pref_per_hp - 1; - align = pci_resource_alignment(bridge, mmio_pref_res); - mmio_pref = div64_ul(available_mmio_pref, hotplug_bridges); - mmio_pref = min(ALIGN(mmio_pref, align), remaining_mmio_pref); - remaining_mmio_pref -= mmio_pref; + pci_bus_distribute_available_resources(dev->subordinate, + add_list, io, mmio, mmio_pref); - pci_bus_distribute_available_resources(b, add_list, io, mmio, - mmio_pref); + io.start = io.end + 1; + mmio.start = mmio.end + 1; + mmio_pref.start = mmio_pref.end + 1; } } static void pci_bridge_distribute_available_resources(struct pci_dev *bridge, struct list_head *add_list) { - resource_size_t available_io, available_mmio, available_mmio_pref; - const struct resource *res; + struct resource io, mmio, mmio_pref; if (!bridge->is_hotplug_bridge) return; /* Take the initial extra resources from the hotplug port */ - res = &bridge->resource[PCI_BRIDGE_RESOURCES + 0]; - available_io = resource_size(res); - res = &bridge->resource[PCI_BRIDGE_RESOURCES + 1]; - available_mmio = resource_size(res); - res = &bridge->resource[PCI_BRIDGE_RESOURCES + 2]; - available_mmio_pref = resource_size(res); + io = bridge->resource[PCI_BRIDGE_RESOURCES + 0]; + mmio = bridge->resource[PCI_BRIDGE_RESOURCES + 1]; + mmio_pref = bridge->resource[PCI_BRIDGE_RESOURCES + 2]; - pci_bus_distribute_available_resources(bridge->subordinate, - add_list, available_io, - available_mmio, - available_mmio_pref); + pci_bus_distribute_available_resources(bridge->subordinate, add_list, + io, mmio, mmio_pref); } void pci_assign_unassigned_bridge_resources(struct pci_dev *bridge) From patchwork Mon Jul 1 14:24:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Johnson X-Patchwork-Id: 11026129 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 61FCD13BD for ; Mon, 1 Jul 2019 14:24:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 528A728600 for ; Mon, 1 Jul 2019 14:24:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 46F6C28686; Mon, 1 Jul 2019 14:24:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 20DA728600 for ; Mon, 1 Jul 2019 14:24:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727904AbfGAOYb convert rfc822-to-8bit (ORCPT ); Mon, 1 Jul 2019 10:24:31 -0400 Received: from mail-oln040092253055.outbound.protection.outlook.com ([40.92.253.55]:18752 "EHLO APC01-SG2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727687AbfGAOYb (ORCPT ); Mon, 1 Jul 2019 10:24:31 -0400 Received: from PU1APC01FT027.eop-APC01.prod.protection.outlook.com (10.152.252.59) by PU1APC01HT128.eop-APC01.prod.protection.outlook.com (10.152.253.57) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2032.15; Mon, 1 Jul 2019 14:24:25 +0000 Received: from SL2P216MB0187.KORP216.PROD.OUTLOOK.COM (10.152.252.60) by PU1APC01FT027.mail.protection.outlook.com (10.152.252.232) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2032.15 via Frontend Transport; Mon, 1 Jul 2019 14:24:25 +0000 Received: from SL2P216MB0187.KORP216.PROD.OUTLOOK.COM ([fe80::9d2d:391f:5f49:c806]) by SL2P216MB0187.KORP216.PROD.OUTLOOK.COM ([fe80::9d2d:391f:5f49:c806%6]) with mapi id 15.20.2032.019; Mon, 1 Jul 2019 14:24:25 +0000 From: Nicholas Johnson To: "linux-kernel@vger.kernel.org" CC: "linux-pci@vger.kernel.org" , "bhelgaas@google.com" , "mika.westerberg@linux.intel.com" , "corbet@lwn.net" , "benh@kernel.crashing.org" , "logang@deltatee.com" Subject: [PATCH v7 4/8] PCI: In extend_bridge_window() change available to new_size Thread-Topic: [PATCH v7 4/8] PCI: In extend_bridge_window() change available to new_size Thread-Index: AQHVMBivrx8FzDF0kkyzpGLK7TEXIw== Date: Mon, 1 Jul 2019 14:24:25 +0000 Message-ID: Accept-Language: en-AU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: SY3PR01CA0113.ausprd01.prod.outlook.com (2603:10c6:0:1a::22) To SL2P216MB0187.KORP216.PROD.OUTLOOK.COM (2603:1096:100:22::19) x-incomingtopheadermarker: OriginalChecksum:BBB382CFBDC955B73815AA804033FB78135BFBAC0FBB8FE7057E5462180B3774;UpperCasedChecksum:9F13726519B9E7DC163DC282B90BE7D48B32CF243C0D8BD86E2230F76B2D69ED;SizeAsReceived:7700;Count:47 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [gsg8Q7+rdyw3T6QQk+Jh1jsTc3tOjxDhvUmT0zgkslQEPWQbS1qlYB4gYM8uWCWoi7W8qzeJYxY=] x-microsoft-original-message-id: <20190701142410.GA5251@nicholas-usb> x-ms-publictraffictype: Email x-incomingheadercount: 47 x-eopattributedmessage: 0 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(5050001)(7020095)(20181119110)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031322404)(2017031323274)(2017031324274)(1601125500)(1603101475)(1701031045);SRVR:PU1APC01HT128; x-ms-traffictypediagnostic: PU1APC01HT128: x-microsoft-antispam-message-info: lmaoXnGwhPQEOtDUxTuXwyHALOAgDFLWgTt2N7XTa4/tRdkuP1/2X4ypn1xCXur6eWxd2Bd3ZxyqFSrgczcAaZ2v2fP2KywcbzrR0EsX3wOY4GQiK8KRzOuqh2d5lAX8KJCoMayllBMXwn9zSkRPG3i/eVpl1UwR+YvDQRaXhkXD+cZc3ScwH18maT8eCY2c Content-ID: MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: cf89e8fd-4aef-4196-5756-08d6fe2fd184 X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jul 2019 14:24:25.4445 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: PU1APC01HT128 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In extend_bridge_window() change "available" parameter name to new_size. This makes more sense as this parameter represents the new size for the window. Signed-off-by: Nicholas Johnson --- drivers/pci/setup-bus.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index 898fea00c..add1dc373 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1811,14 +1811,14 @@ void __init pci_assign_unassigned_resources(void) static void extend_bridge_window(struct pci_dev *bridge, struct resource *res, struct list_head *add_list, - resource_size_t available) + resource_size_t new_size) { struct pci_dev_resource *dev_res; if (res->parent) return; - if (resource_size(res) >= available) + if (resource_size(res) >= new_size) return; dev_res = res_to_dev_res(add_list, res); @@ -1826,10 +1826,10 @@ static void extend_bridge_window(struct pci_dev *bridge, struct resource *res, return; /* Is there room to extend the window? */ - if (available - resource_size(res) <= dev_res->add_size) + if (new_size - resource_size(res) <= dev_res->add_size) return; - dev_res->add_size = available - resource_size(res); + dev_res->add_size = new_size - resource_size(res); pci_dbg(bridge, "bridge window %pR extended by %pa\n", res, &dev_res->add_size); } From patchwork Mon Jul 1 14:24:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Johnson X-Patchwork-Id: 11026131 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3E983138D for ; Mon, 1 Jul 2019 14:25:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2CFE828600 for ; Mon, 1 Jul 2019 14:25:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1E7D428680; Mon, 1 Jul 2019 14:25:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AA45A28600 for ; Mon, 1 Jul 2019 14:25:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727817AbfGAOZF convert rfc822-to-8bit (ORCPT ); Mon, 1 Jul 2019 10:25:05 -0400 Received: from mail-oln040092255094.outbound.protection.outlook.com ([40.92.255.94]:36304 "EHLO APC01-HK2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727687AbfGAOZE (ORCPT ); Mon, 1 Jul 2019 10:25:04 -0400 Received: from PU1APC01FT027.eop-APC01.prod.protection.outlook.com (10.152.252.54) by PU1APC01HT090.eop-APC01.prod.protection.outlook.com (10.152.253.94) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2032.15; Mon, 1 Jul 2019 14:24:59 +0000 Received: from SL2P216MB0187.KORP216.PROD.OUTLOOK.COM (10.152.252.60) by PU1APC01FT027.mail.protection.outlook.com (10.152.252.232) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2032.15 via Frontend Transport; Mon, 1 Jul 2019 14:24:59 +0000 Received: from SL2P216MB0187.KORP216.PROD.OUTLOOK.COM ([fe80::9d2d:391f:5f49:c806]) by SL2P216MB0187.KORP216.PROD.OUTLOOK.COM ([fe80::9d2d:391f:5f49:c806%6]) with mapi id 15.20.2032.019; Mon, 1 Jul 2019 14:24:59 +0000 From: Nicholas Johnson To: "linux-kernel@vger.kernel.org" CC: "linux-pci@vger.kernel.org" , "bhelgaas@google.com" , "mika.westerberg@linux.intel.com" , "corbet@lwn.net" , "benh@kernel.crashing.org" , "logang@deltatee.com" Subject: [PATCH v7 5/8] PCI: Change extend_bridge_window() to set resource size directly Thread-Topic: [PATCH v7 5/8] PCI: Change extend_bridge_window() to set resource size directly Thread-Index: AQHVMBjDD8e6mskNZEOJPe9aWx8fNg== Date: Mon, 1 Jul 2019 14:24:59 +0000 Message-ID: Accept-Language: en-AU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: SYAPR01CA0024.ausprd01.prod.outlook.com (2603:10c6:1::36) To SL2P216MB0187.KORP216.PROD.OUTLOOK.COM (2603:1096:100:22::19) x-incomingtopheadermarker: OriginalChecksum:18A4005DFC9385174F9490232FD98C2678707A963C301EE78DC9D480C69DD8DF;UpperCasedChecksum:33AF3B687EA8AB3E677547488FD863CF843B0464329DFDB84C0DDA3D538367FF;SizeAsReceived:7689;Count:47 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [/YKWdA81hOgnLIDQ02OMCJ9w1V4wthNHC+EK10wypdF1DPTM89EcXueFCi/U0FIwOpTkwQ4Xnhc=] x-microsoft-original-message-id: <20190701142438.GA5276@nicholas-usb> x-ms-publictraffictype: Email x-incomingheadercount: 47 x-eopattributedmessage: 0 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(5050001)(7020095)(20181119110)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031322404)(2017031323274)(2017031324274)(1601125500)(1603101475)(1701031045);SRVR:PU1APC01HT090; x-ms-traffictypediagnostic: PU1APC01HT090: x-microsoft-antispam-message-info: d3KkN/qNby0171dtNwXNg7l2+2VDA9j3t0RHZE6oQ+fGusZMmAPLSbKNlRGBngssB5WMr9LCo6P6DgM/W3zkSNmM/nhNKV1p4KMD+aeA7A0PLbYfPfmC2MVtg5ejzhn+FsqFYlSPPplsSd5uBRLGkqn50W/j7q/1PWlZYtM++6nuXzP8a5pT3+BJqti1BmA6 Content-ID: <5F9F280FE5E5CF4DBE24FDB409D18676@KORP216.PROD.OUTLOOK.COM> MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: c6dbf5e8-43b0-4338-c513-08d6fe2fe5c7 X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jul 2019 14:24:59.4468 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: PU1APC01HT090 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Change extend_bridge_window() to set resource size directly instead of using additional resource lists. Because additional resource lists are optional resources, any algorithm that requires guaranteed allocation that uses them cannot be guaranteed to work. Remove the resource from add_list. If it is set to zero size and left, it can cause significant problems when it comes to assigning resources. Signed-off-by: Nicholas Johnson --- drivers/pci/setup-bus.c | 17 +++++------------ 1 file changed, 5 insertions(+), 12 deletions(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index add1dc373..d65982438 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1813,7 +1813,7 @@ static void extend_bridge_window(struct pci_dev *bridge, struct resource *res, struct list_head *add_list, resource_size_t new_size) { - struct pci_dev_resource *dev_res; + resource_size_t add_size; if (res->parent) return; @@ -1821,17 +1821,10 @@ static void extend_bridge_window(struct pci_dev *bridge, struct resource *res, if (resource_size(res) >= new_size) return; - dev_res = res_to_dev_res(add_list, res); - if (!dev_res) - return; - - /* Is there room to extend the window? */ - if (new_size - resource_size(res) <= dev_res->add_size) - return; - - dev_res->add_size = new_size - resource_size(res); - pci_dbg(bridge, "bridge window %pR extended by %pa\n", res, - &dev_res->add_size); + add_size = new_size - resource_size(res); + pci_dbg(bridge, "bridge window %pR extended by %pa\n", res, &add_size); + res->end = res->start + new_size - 1; + remove_from_list(add_list, res); } static void pci_bus_distribute_available_resources(struct pci_bus *bus, From patchwork Mon Jul 1 14:25:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Johnson X-Patchwork-Id: 11026133 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B05C138D for ; Mon, 1 Jul 2019 14:25:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 279F61FF40 for ; Mon, 1 Jul 2019 14:25:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1884E28680; Mon, 1 Jul 2019 14:25:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AD15828600 for ; Mon, 1 Jul 2019 14:25:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728159AbfGAOZ3 convert rfc822-to-8bit (ORCPT ); Mon, 1 Jul 2019 10:25:29 -0400 Received: from mail-oln040092254061.outbound.protection.outlook.com ([40.92.254.61]:22656 "EHLO APC01-PU1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727687AbfGAOZ2 (ORCPT ); Mon, 1 Jul 2019 10:25:28 -0400 Received: from PU1APC01FT027.eop-APC01.prod.protection.outlook.com (10.152.252.60) by PU1APC01HT051.eop-APC01.prod.protection.outlook.com (10.152.253.35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2032.15; Mon, 1 Jul 2019 14:25:24 +0000 Received: from SL2P216MB0187.KORP216.PROD.OUTLOOK.COM (10.152.252.60) by PU1APC01FT027.mail.protection.outlook.com (10.152.252.232) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2032.15 via Frontend Transport; Mon, 1 Jul 2019 14:25:24 +0000 Received: from SL2P216MB0187.KORP216.PROD.OUTLOOK.COM ([fe80::9d2d:391f:5f49:c806]) by SL2P216MB0187.KORP216.PROD.OUTLOOK.COM ([fe80::9d2d:391f:5f49:c806%6]) with mapi id 15.20.2032.019; Mon, 1 Jul 2019 14:25:24 +0000 From: Nicholas Johnson To: "linux-kernel@vger.kernel.org" CC: "linux-pci@vger.kernel.org" , "bhelgaas@google.com" , "mika.westerberg@linux.intel.com" , "corbet@lwn.net" , "benh@kernel.crashing.org" , "logang@deltatee.com" Subject: [PATCH v7 6/8] PCI: Allow extend_bridge_window() to shrink resource if necessary Thread-Topic: [PATCH v7 6/8] PCI: Allow extend_bridge_window() to shrink resource if necessary Thread-Index: AQHVMBjSFu30TOdswEyjNEb8yk6Glw== Date: Mon, 1 Jul 2019 14:25:24 +0000 Message-ID: Accept-Language: en-AU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: SY3PR01CA0097.ausprd01.prod.outlook.com (2603:10c6:0:19::30) To SL2P216MB0187.KORP216.PROD.OUTLOOK.COM (2603:1096:100:22::19) x-incomingtopheadermarker: OriginalChecksum:78B69A352EE2D48EE966902C7BE31E9D2695E685278EE4228767415B952D10DA;UpperCasedChecksum:3709DA5CBC05F479832E249D673D1A94F83452C4D7CEB6C0132E9E56FCDF4C3E;SizeAsReceived:7703;Count:47 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [Jy811Bkp+qp8vQM4tGhjmijTLwXczcgQn0IWdPFEsgjVuqDkfxyuAuYakF6urMCJ1fHws5/17hs=] x-microsoft-original-message-id: <20190701142508.GA5301@nicholas-usb> x-ms-publictraffictype: Email x-incomingheadercount: 47 x-eopattributedmessage: 0 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(5050001)(7020095)(20181119110)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031323274)(2017031324274)(2017031322404)(1601125500)(1603101475)(1701031045);SRVR:PU1APC01HT051; x-ms-traffictypediagnostic: PU1APC01HT051: x-microsoft-antispam-message-info: VOTRK/O2oW/4VL9oE/Y7hOE0wOJMIn2ZCZMNMLDndu8SjKE/0amgHTJscJ2FVb+K3N87cdCf6/Iyh9uH6lssC3uEgophYqN+PJqvaPs2F8yS+OftuZsfMy5lf7AUN1K8IYVV92mXnPD/7Cwz+jwpanm3k8l7Q1B9GCtQ2bpzu2Un/eLL0PIL7vepc39mCFQi Content-ID: MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: eed832f3-fab8-4ef1-4d1c-08d6fe2ff46e X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jul 2019 14:25:24.0313 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: PU1APC01HT051 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Remove checks for resource size in extend_bridge_window(). This is necessary to allow the pci_bus_distribute_available_resources() to function when the kernel parameter pci=hpmemsize=nn[KMG] is used to allocate resources. Because the kernel parameter sets the size of all hotplug bridges to be the same, there are problems when nested hotplug bridges are encountered. Fitting a downstream hotplug bridge with size X and normal bridges with size Y into parent hotplug bridge with size X is impossible, and hence the downstream hotplug bridge needs to shrink to fit into its parent. Add check for if bridge is extended or shrunken and adjust pci_dbg to reflect this. Reset the resource if its new size is zero (if we have run out of a bridge window resource). If it is set to zero size and left, it can cause significant problems when it comes to enabling devices. Signed-off-by: Nicholas Johnson --- drivers/pci/setup-bus.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index d65982438..9064fd964 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1818,13 +1818,19 @@ static void extend_bridge_window(struct pci_dev *bridge, struct resource *res, if (res->parent) return; - if (resource_size(res) >= new_size) - return; - - add_size = new_size - resource_size(res); - pci_dbg(bridge, "bridge window %pR extended by %pa\n", res, &add_size); + if (new_size > resource_size(res)) { + add_size = new_size - resource_size(res); + pci_dbg(bridge, "bridge window %pR extended by %pa\n", res, + &add_size); + } else if (new_size < resource_size(res)) { + add_size = resource_size(res) - new_size; + pci_dbg(bridge, "bridge window %pR shrunken by %pa\n", res, + &add_size); + } res->end = res->start + new_size - 1; remove_from_list(add_list, res); + if (!new_size) + reset_resource(res); } static void pci_bus_distribute_available_resources(struct pci_bus *bus, From patchwork Mon Jul 1 14:25:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Johnson X-Patchwork-Id: 11026135 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 041EB1510 for ; Mon, 1 Jul 2019 14:25:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E73252018E for ; Mon, 1 Jul 2019 14:25:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D8DCD2624B; Mon, 1 Jul 2019 14:25:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3748322B26 for ; Mon, 1 Jul 2019 14:25:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729625AbfGAOZz convert rfc822-to-8bit (ORCPT ); Mon, 1 Jul 2019 10:25:55 -0400 Received: from mail-oln040092253106.outbound.protection.outlook.com ([40.92.253.106]:34721 "EHLO APC01-SG2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727687AbfGAOZy (ORCPT ); Mon, 1 Jul 2019 10:25:54 -0400 Received: from PU1APC01FT027.eop-APC01.prod.protection.outlook.com (10.152.252.53) by PU1APC01HT121.eop-APC01.prod.protection.outlook.com (10.152.253.60) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2032.15; Mon, 1 Jul 2019 14:25:48 +0000 Received: from SL2P216MB0187.KORP216.PROD.OUTLOOK.COM (10.152.252.60) by PU1APC01FT027.mail.protection.outlook.com (10.152.252.232) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2032.15 via Frontend Transport; Mon, 1 Jul 2019 14:25:48 +0000 Received: from SL2P216MB0187.KORP216.PROD.OUTLOOK.COM ([fe80::9d2d:391f:5f49:c806]) by SL2P216MB0187.KORP216.PROD.OUTLOOK.COM ([fe80::9d2d:391f:5f49:c806%6]) with mapi id 15.20.2032.019; Mon, 1 Jul 2019 14:25:48 +0000 From: Nicholas Johnson To: "linux-kernel@vger.kernel.org" CC: "linux-pci@vger.kernel.org" , "bhelgaas@google.com" , "mika.westerberg@linux.intel.com" , "corbet@lwn.net" , "benh@kernel.crashing.org" , "logang@deltatee.com" Subject: [PATCH v7 7/8] PCI: Add hp_mmio_size and hp_mmio_pref_size parameters Thread-Topic: [PATCH v7 7/8] PCI: Add hp_mmio_size and hp_mmio_pref_size parameters Thread-Index: AQHVMBjgPnEXmKRAdkyaQ9Bk0tjoOA== Date: Mon, 1 Jul 2019 14:25:48 +0000 Message-ID: Accept-Language: en-AU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: SY2PR01CA0023.ausprd01.prod.outlook.com (2603:10c6:1:14::35) To SL2P216MB0187.KORP216.PROD.OUTLOOK.COM (2603:1096:100:22::19) x-incomingtopheadermarker: OriginalChecksum:AE65CB97516F9CA21DE0B73C9B05FE76D3F4FA9FCB658CC5D90B5E881BEC7E44;UpperCasedChecksum:042D3BECB0325DD5AAA83456F8ADD61F76D8F27987928FBE04025CDB037C82AD;SizeAsReceived:7674;Count:47 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [UsL19qhXSguCqt73BFCi48a1BfmMagVwg6Mu0jhLNObxiLKYBm95gG0AwyqxUeD2] x-microsoft-original-message-id: <20190701142533.GA5319@nicholas-usb> x-ms-publictraffictype: Email x-incomingheadercount: 47 x-eopattributedmessage: 0 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(5050001)(7020095)(20181119110)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031322404)(2017031323274)(2017031324274)(1601125500)(1603101475)(1701031045);SRVR:PU1APC01HT121; x-ms-traffictypediagnostic: PU1APC01HT121: x-microsoft-antispam-message-info: FQ7PuA0lGESrGAahg+nSWdPkRMgzaxX39fzbbj9fli0lek/DJu+r5kvSCOIHHiZ/QvQhrW6wPKKqAuJI0c+XeHnTQHNODW3PKZdLivl8R1WWSCpvxXV7pJPb09JRKY1Aw4Q48Cyfu4CLECSfzkjRdZ51bXnj0rEWPF+ccnuDjb8FT3YFGAqs7Xt5i+m4wuEO Content-ID: <268416731237D64B906E823ADB7DE961@KORP216.PROD.OUTLOOK.COM> MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 538fa49c-ccc9-4dae-df53-08d6fe300319 X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jul 2019 14:25:48.7039 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: PU1APC01HT121 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add kernel parameter pci=hpmmiosize=nn[KMG] to set MMIO bridge window size for hotplug bridges. Add kernel parameter pci=hpmmioprefsize=nn[KMG] to set MMIO_PREF bridge window size for hotplug bridges. Leave pci=hpmemsize=nn[KMG] unchanged, to prevent disruptions to existing users. This sets both MMIO and MMIO_PREF to the same size. The two new parameters conform to the style of pci=hpiosize=nn[KMG]. Signed-off-by: Nicholas Johnson --- .../admin-guide/kernel-parameters.txt | 9 ++++++- drivers/pci/pci.c | 17 ++++++++++--- drivers/pci/setup-bus.c | 25 +++++++++++-------- include/linux/pci.h | 3 ++- 4 files changed, 38 insertions(+), 16 deletions(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 138f6664b..f3cda0b07 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3438,8 +3438,15 @@ hpiosize=nn[KMG] The fixed amount of bus space which is reserved for hotplug bridge's IO window. Default size is 256 bytes. + hpmmiosize=nn[KMG] The fixed amount of bus space which is + reserved for hotplug bridge's MMIO window. + Default size is 2 megabytes. + hpmmioprefsize=nn[KMG] The fixed amount of bus space which is + reserved for hotplug bridge's MMIO_PREF window. + Default size is 2 megabytes. hpmemsize=nn[KMG] The fixed amount of bus space which is - reserved for hotplug bridge's memory window. + reserved for hotplug bridge's MMIO and + MMIO_PREF window. Default size is 2 megabytes. hpbussize=nn The minimum amount of additional bus numbers reserved for buses below a hotplug bridge. diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 8abc843b1..4ee1aaf5b 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -85,10 +85,12 @@ unsigned long pci_cardbus_io_size = DEFAULT_CARDBUS_IO_SIZE; unsigned long pci_cardbus_mem_size = DEFAULT_CARDBUS_MEM_SIZE; #define DEFAULT_HOTPLUG_IO_SIZE (256) -#define DEFAULT_HOTPLUG_MEM_SIZE (2*1024*1024) +#define DEFAULT_HOTPLUG_MMIO_SIZE (2*1024*1024) +#define DEFAULT_HOTPLUG_MMIO_PREF_SIZE (2*1024*1024) /* pci=hpmemsize=nnM,hpiosize=nn can override this */ unsigned long pci_hotplug_io_size = DEFAULT_HOTPLUG_IO_SIZE; -unsigned long pci_hotplug_mem_size = DEFAULT_HOTPLUG_MEM_SIZE; +unsigned long pci_hotplug_mmio_size = DEFAULT_HOTPLUG_MMIO_SIZE; +unsigned long pci_hotplug_mmio_pref_size = DEFAULT_HOTPLUG_MMIO_PREF_SIZE; #define DEFAULT_HOTPLUG_BUS_SIZE 1 unsigned long pci_hotplug_bus_size = DEFAULT_HOTPLUG_BUS_SIZE; @@ -6231,8 +6233,17 @@ static int __init pci_setup(char *str) pcie_ecrc_get_policy(str + 5); } else if (!strncmp(str, "hpiosize=", 9)) { pci_hotplug_io_size = memparse(str + 9, &str); + } else if (!strncmp(str, "hpmmiosize=", 11)) { + pci_hotplug_mmio_size = + memparse(str + 11, &str); + } else if (!strncmp(str, "hpmmioprefsize=", 15)) { + pci_hotplug_mmio_pref_size = + memparse(str + 15, &str); } else if (!strncmp(str, "hpmemsize=", 10)) { - pci_hotplug_mem_size = memparse(str + 10, &str); + pci_hotplug_mmio_size = + memparse(str + 10, &str); + pci_hotplug_mmio_pref_size = + memparse(str + 10, &str); } else if (!strncmp(str, "hpbussize=", 10)) { pci_hotplug_bus_size = simple_strtoul(str + 10, &str, 0); diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index 9064fd964..873df5482 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -1178,7 +1178,8 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head) { struct pci_dev *dev; unsigned long mask, prefmask, type2 = 0, type3 = 0; - resource_size_t additional_mem_size = 0, additional_io_size = 0; + resource_size_t additional_io_size = 0, additional_mmio_size = 0, + additional_mmio_pref_size = 0; struct resource *b_res; int ret; @@ -1212,7 +1213,8 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head) pci_bridge_check_ranges(bus); if (bus->self->is_hotplug_bridge) { additional_io_size = pci_hotplug_io_size; - additional_mem_size = pci_hotplug_mem_size; + additional_mmio_size = pci_hotplug_mmio_size; + additional_mmio_pref_size = pci_hotplug_mmio_pref_size; } /* Fall through */ default: @@ -1230,9 +1232,9 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head) if (b_res[2].flags & IORESOURCE_MEM_64) { prefmask |= IORESOURCE_MEM_64; ret = pbus_size_mem(bus, prefmask, prefmask, - prefmask, prefmask, - realloc_head ? 0 : additional_mem_size, - additional_mem_size, realloc_head); + prefmask, prefmask, + realloc_head ? 0 : additional_mmio_pref_size, + additional_mmio_pref_size, realloc_head); /* * If successful, all non-prefetchable resources @@ -1254,9 +1256,9 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head) if (!type2) { prefmask &= ~IORESOURCE_MEM_64; ret = pbus_size_mem(bus, prefmask, prefmask, - prefmask, prefmask, - realloc_head ? 0 : additional_mem_size, - additional_mem_size, realloc_head); + prefmask, prefmask, + realloc_head ? 0 : additional_mmio_pref_size, + additional_mmio_pref_size, realloc_head); /* * If successful, only non-prefetchable resources @@ -1265,7 +1267,8 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head) if (ret == 0) mask = prefmask; else - additional_mem_size += additional_mem_size; + additional_mmio_size += + additional_mmio_pref_size; type2 = type3 = IORESOURCE_MEM; } @@ -1285,8 +1288,8 @@ void __pci_bus_size_bridges(struct pci_bus *bus, struct list_head *realloc_head) * prefetchable resource in a 64-bit prefetchable window. */ pbus_size_mem(bus, mask, IORESOURCE_MEM, type2, type3, - realloc_head ? 0 : additional_mem_size, - additional_mem_size, realloc_head); + realloc_head ? 0 : additional_mmio_size, + additional_mmio_size, realloc_head); break; } } diff --git a/include/linux/pci.h b/include/linux/pci.h index dd436da7e..e9641a127 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1973,7 +1973,8 @@ extern u8 pci_dfl_cache_line_size; extern u8 pci_cache_line_size; extern unsigned long pci_hotplug_io_size; -extern unsigned long pci_hotplug_mem_size; +extern unsigned long pci_hotplug_mmio_size; +extern unsigned long pci_hotplug_mmio_pref_size; extern unsigned long pci_hotplug_bus_size; /* Architecture-specific versions may override these (weak) */ From patchwork Mon Jul 1 14:26:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Johnson X-Patchwork-Id: 11026137 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0D44113BD for ; Mon, 1 Jul 2019 14:26:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F17A92018E for ; Mon, 1 Jul 2019 14:26:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E59FC28686; Mon, 1 Jul 2019 14:26:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4E4D92018E for ; Mon, 1 Jul 2019 14:26:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728165AbfGAO0U convert rfc822-to-8bit (ORCPT ); Mon, 1 Jul 2019 10:26:20 -0400 Received: from mail-oln040092254057.outbound.protection.outlook.com ([40.92.254.57]:39968 "EHLO APC01-PU1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728090AbfGAO0U (ORCPT ); Mon, 1 Jul 2019 10:26:20 -0400 Received: from PU1APC01FT027.eop-APC01.prod.protection.outlook.com (10.152.252.53) by PU1APC01HT229.eop-APC01.prod.protection.outlook.com (10.152.252.198) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2032.15; Mon, 1 Jul 2019 14:26:13 +0000 Received: from SL2P216MB0187.KORP216.PROD.OUTLOOK.COM (10.152.252.60) by PU1APC01FT027.mail.protection.outlook.com (10.152.252.232) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.2032.15 via Frontend Transport; Mon, 1 Jul 2019 14:26:13 +0000 Received: from SL2P216MB0187.KORP216.PROD.OUTLOOK.COM ([fe80::9d2d:391f:5f49:c806]) by SL2P216MB0187.KORP216.PROD.OUTLOOK.COM ([fe80::9d2d:391f:5f49:c806%6]) with mapi id 15.20.2032.019; Mon, 1 Jul 2019 14:26:13 +0000 From: Nicholas Johnson To: "linux-kernel@vger.kernel.org" CC: "linux-pci@vger.kernel.org" , "bhelgaas@google.com" , "mika.westerberg@linux.intel.com" , "corbet@lwn.net" , "benh@kernel.crashing.org" , "logang@deltatee.com" Subject: [PATCH v7 8/8] PCI: Fix bug resulting in double hpmemsize being assigned to MMIO window Thread-Topic: [PATCH v7 8/8] PCI: Fix bug resulting in double hpmemsize being assigned to MMIO window Thread-Index: AQHVMBjvqV0RKtw8Ak6fj/QX5/LnSQ== Date: Mon, 1 Jul 2019 14:26:13 +0000 Message-ID: Accept-Language: en-AU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: SYBPR01CA0114.ausprd01.prod.outlook.com (2603:10c6:10:1::30) To SL2P216MB0187.KORP216.PROD.OUTLOOK.COM (2603:1096:100:22::19) x-incomingtopheadermarker: OriginalChecksum:0491EC7E998529029537C9166345947E22827050F8FE0A7ABA42EE12E0661AB2;UpperCasedChecksum:7F834409EF0D15F526E804CCEF51FAA354189B661292B6015E0A66B2237F2177;SizeAsReceived:7740;Count:47 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [qOTMwuQhe05QBO2JJaiCjQbv6eU213CbqdrqmtJJ+k2hCE8Gal/GVr64Sjyi0r9KFq4wsC+pcL4=] x-microsoft-original-message-id: <20190701142558.GA5338@nicholas-usb> x-ms-publictraffictype: Email x-incomingheadercount: 47 x-eopattributedmessage: 0 x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(5050001)(7020095)(20181119110)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031322404)(2017031323274)(2017031324274)(1601125500)(1603101475)(1701031045);SRVR:PU1APC01HT229; x-ms-traffictypediagnostic: PU1APC01HT229: x-ms-exchange-purlcount: 2 x-microsoft-antispam-message-info: ytnBVmlXWhAwbeHYbRAvaR/da1tEG0nSV388QAd6mwmywHdL3WzqEeq/aAvfiLjRwwVBa3p18VOEWgzimb0HKj8uu2YiDXdwAqjANTI1SAL+R02/nbzH0xmhI7LTWU9KXz4WwN0sgXoBQd5ow1QvXIkUKavQ/aWb4AvERRa/ZGN+Sm1sbydJ24OIMqQysukA Content-ID: <955A3DC6DF62EE4E8C686282AA5D1281@KORP216.PROD.OUTLOOK.COM> MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: 156990db-d3ff-45aa-22a7-08d6fe30120e X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jul 2019 14:26:13.7738 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: PU1APC01HT229 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Background ========================================================================== Currently, the kernel can sometimes assign the MMIO_PREF window additional size into the MMIO window, resulting in double the MMIO additional size, even if the MMIO_PREF window was successful. This happens if in the first pass, the MMIO_PREF succeeds but the MMIO fails. In the next pass, because MMIO_PREF is already assigned, the attempt to assign MMIO_PREF returns an error code instead of success (nothing more to do, already allocated). Example of problem (more context can be found in the bug report URL): Mainline kernel: pci 0000:06:01.0: BAR 14: assigned [mem 0x90100000-0xa00fffff] = 256M pci 0000:06:04.0: BAR 14: assigned [mem 0xa0200000-0xb01fffff] = 256M Patched kernel: pci 0000:06:01.0: BAR 14: assigned [mem 0x90100000-0x980fffff] = 128M pci 0000:06:04.0: BAR 14: assigned [mem 0x98200000-0xa01fffff] = 128M This was using pci=realloc,hpmemsize=128M,nocrs - on the same machine with the same configuration, with a Ubuntu mainline kernel and a kernel patched with this patch series. This patch is vital for the next patch in the series. The next patch allows the user to specify MMIO and MMIO_PREF independently. If the MMIO_PREF is set to be very large, this bug will end up more than doubling the MMIO size. The bug results in the MMIO_PREF being added to the MMIO window, which means doubling if MMIO_PREF size == MMIO size. With a large MMIO_PREF, without this patch, the MMIO window will likely fail to be assigned altogether due to lack of 32-bit address space. Patch notes ========================================================================== Change find_free_bus_resource() to not skip assigned resources with non-null parent. Add checks in pbus_size_io() and pbus_size_mem() to return success if resource returned from find_free_bus_resource() is already allocated. This avoids pbus_size_io() and pbus_size_mem() returning error code to __pci_bus_size_bridges() when a resource has been successfully assigned in a previous pass. This fixes the existing behaviour where space for a resource could be reserved multiple times in different parent bridge windows. This also greatly reduces the number of failed BAR messages in dmesg when Linux assigns resources. See related from Logan Gunthorpe (same problem, different solution): https://lore.kernel.org/lkml/20190531171216.20532-2-logang@deltatee.com/T/#u Solves bug report: https://bugzilla.kernel.org/show_bug.cgi?id=203243 Reported-by: Kit Chow Reported-by: Nicholas Johnson Signed-off-by: Nicholas Johnson --- drivers/pci/setup-bus.c | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index 873df5482..df4bf43b5 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -752,13 +752,18 @@ static void pci_bridge_check_ranges(struct pci_bus *bus) } /* - * Helper function for sizing routines: find first available bus resource - * of a given type. Note: we intentionally skip the bus resources which - * have already been assigned (that is, have non-NULL parent resource). + * Helper function for sizing routines: find first bus resource of a given + * type. Note: we do not skip the bus resources which have already been + * assigned (r->parent != NULL). This is because a resource that is already + * assigned (nothing more to be done) will be indistinguishable from one that + * failed due to lack of space if we skip assigned resources. If the caller + * function cannot tell the difference then it might try to place the + * resources in a different window, doubling up on resources or causing + * unforeseeable issues. */ -static struct resource *find_free_bus_resource(struct pci_bus *bus, - unsigned long type_mask, - unsigned long type) +static struct resource *find_bus_resource_of_type(struct pci_bus *bus, + unsigned long type_mask, + unsigned long type) { int i; struct resource *r; @@ -766,7 +771,7 @@ static struct resource *find_free_bus_resource(struct pci_bus *bus, pci_bus_for_each_resource(bus, r, i) { if (r == &ioport_resource || r == &iomem_resource) continue; - if (r && (r->flags & type_mask) == type && !r->parent) + if (r && (r->flags & type_mask) == type) return r; } return NULL; @@ -866,14 +871,16 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size, struct list_head *realloc_head) { struct pci_dev *dev; - struct resource *b_res = find_free_bus_resource(bus, IORESOURCE_IO, - IORESOURCE_IO); + struct resource *b_res = find_bus_resource_of_type(bus, IORESOURCE_IO, + IORESOURCE_IO); resource_size_t size = 0, size0 = 0, size1 = 0; resource_size_t children_add_size = 0; resource_size_t min_align, align; if (!b_res) return; + if (b_res->parent) + return; min_align = window_alignment(bus, IORESOURCE_IO); list_for_each_entry(dev, &bus->devices, bus_list) { @@ -978,7 +985,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask, resource_size_t min_align, align, size, size0, size1; resource_size_t aligns[18]; /* Alignments from 1MB to 128GB */ int order, max_order; - struct resource *b_res = find_free_bus_resource(bus, + struct resource *b_res = find_bus_resource_of_type(bus, mask | IORESOURCE_PREFETCH, type); resource_size_t children_add_size = 0; resource_size_t children_add_align = 0; @@ -986,6 +993,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask, if (!b_res) return -ENOSPC; + if (b_res->parent) + return 0; memset(aligns, 0, sizeof(aligns)); max_order = 0;