From patchwork Wed May 22 14:31:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Johnson X-Patchwork-Id: 10955945 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D8BA26C5 for ; Wed, 22 May 2019 14:31:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C86EF28BEC for ; Wed, 22 May 2019 14:31:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BC53226247; Wed, 22 May 2019 14:31:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1A86528BCC for ; Wed, 22 May 2019 14:31:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729471AbfEVObS (ORCPT ); Wed, 22 May 2019 10:31:18 -0400 Received: from mail-oln040092255048.outbound.protection.outlook.com ([40.92.255.48]:8931 "EHLO APC01-HK2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1728835AbfEVObS (ORCPT ); Wed, 22 May 2019 10:31:18 -0400 Received: from HK2APC01FT049.eop-APC01.prod.protection.outlook.com (10.152.248.59) by HK2APC01HT023.eop-APC01.prod.protection.outlook.com (10.152.249.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1900.16; Wed, 22 May 2019 14:31:11 +0000 Received: from PS2P216MB0642.KORP216.PROD.OUTLOOK.COM (10.152.248.53) by HK2APC01FT049.mail.protection.outlook.com (10.152.249.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1922.16 via Frontend Transport; Wed, 22 May 2019 14:31:11 +0000 Received: from PS2P216MB0642.KORP216.PROD.OUTLOOK.COM ([fe80::1c5e:7ea0:b90c:65c9]) by PS2P216MB0642.KORP216.PROD.OUTLOOK.COM ([fe80::1c5e:7ea0:b90c:65c9%10]) with mapi id 15.20.1922.016; Wed, 22 May 2019 14:31:11 +0000 From: Nicholas Johnson To: "linux-kernel@vger.kernel.org" CC: "linux-pci@vger.kernel.org" , "bhelgaas@google.com" , "mika.westerberg@linux.intel.com" , "corbet@lwn.net" , Nicholas Johnson Subject: [PATCH v6 3/4] PCI: Fix bug resulting in double hpmemsize being assigned to MMIO window Thread-Topic: [PATCH v6 3/4] PCI: Fix bug resulting in double hpmemsize being assigned to MMIO window Thread-Index: AQHVEKsAhMcuxcifSUWwjbAdzRxIBw== Date: Wed, 22 May 2019 14:31:11 +0000 Message-ID: References: <20190522222928.2964-1-nicholas.johnson-opensource@outlook.com.au> In-Reply-To: <20190522222928.2964-1-nicholas.johnson-opensource@outlook.com.au> Accept-Language: en-AU, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: SYBPR01CA0017.ausprd01.prod.outlook.com (2603:10c6:10::29) To PS2P216MB0642.KORP216.PROD.OUTLOOK.COM (2603:1096:300:1c::16) x-incomingtopheadermarker: OriginalChecksum:A3BF70338224C9F78FAF69B51F58D475ACCB1CE9984241A780134E3BD4621F53;UpperCasedChecksum:BC87D8C032667453E8BEF6058CEC6CDCB5818BF70E8235A8A67DA52BA9CC21B6;SizeAsReceived:7903;Count:50 x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.19.1 x-tmn: [qF877ri9MeBJPWMFZGJIgeN2XGdtbu3uJFb2mgp4eIZQe3RRQvfNK9YyuuqFzaPU6bWbcxVDKec=] x-microsoft-original-message-id: <20190522222928.2964-4-nicholas.johnson-opensource@outlook.com.au> x-ms-publictraffictype: Email x-incomingheadercount: 50 x-eopattributedmessage: 0 x-ms-exchange-slblob-mailprops: inBjZRGSCZlLxRoZX+b2aZrkZpb++rBSk3dhAeQuM+AqVfYpfycuKhwT/rQlZ5EuiUcX4clSwvmSFaXV5JbqrK2388cybQHFKZnC6lXNCAkaDgk2LrrUVK5aztbzFych41geZ/+M3bLT2UlBTPV9KNwJbtj8cBk1mLicE8DfpoGAZ6p3jr6yQjsW8pglh8u33/tG+XndjegQBHEggRH2aWbvGEtilghLYl3kypsZbeLKrtmb7wvV2bTNF0aOo44y1IkCk8Zo36piCrUus7TAB7JqJfepUB6JnrbH+2pPsNZRgF/m07zKFAo/CfigGPSviFDCvVaxTrmk3vq7XQ8l4VvWgqQlXlFqSNifpDm+rpCfwEeVCCXdQNIh4R1cKTc/5cFpP5OZuWchasnBd7Eoe7BMKkni9ib+G3lJMLGkxfNhYaOqsOD204Hz52h0NG2DoSURjpfNp2lvOiUnHqO4CLvAbbf3oPiZmcMQv415IPntbTAbmDoQKVC8NreCjtOuZO2VOMY0TSNMthTmuwgbJMNzekiuV3IehaMG+xVEfbNTlV38cI5IwVRKkrdottqLyL40+PnWFFgE/qCL/PHyXa+akVnndnrpOAZp99VpaCj/jfw/Yn9eHzYc5dWag9RF+l069gixo6rdV2UvJk2RoV4ESFq6NBRsNG+kCCP21ioRENIwDO3wrAb7SHirFiO3v5yCdQ9AjV/ZaoP8MXp9Eh5GBlR/VDDLDqcJ97p4aRXcVgfeTbscj8A76GUl3fNnQmsc21WOVR94iYP2y5G+mLkodLr3m/QzYAnhaBv/RA2e5gKBPr7QYqAFzk3UZVqmWHIFpPfCMW0IA0VOWUssgSRnFDi7+mzW21BRDXTbC57518dCpwY8fog4IP/jZvs/HTd60cKzpz8u0Ujh/Xgm4XOvjMlA+2nBl7QKBn57OlJHr4EQVWMKxhVL8WzqYd 2dHBrekK8nSImBSZDxD/aNZA== x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(5050001)(7020095)(20181119110)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(201702181274)(2017031323274)(2017031324274)(2017031322404)(1601125500)(1603101475)(1701031045);SRVR:HK2APC01HT023; x-ms-traffictypediagnostic: HK2APC01HT023: x-ms-exchange-purlcount: 1 x-microsoft-antispam-message-info: D0jMsHpww4Cn5hpw+IEs3C8Q6YZb3EYXkEb+sQ0GUXheGRbgh3dHwAPWPNUEyWjFlXWF/RcDNEq+xU5QVJMTO7h52a579WnBHOByXInC5HU4bBvx0zOtNTpDMGQS/6BUedZ+h2t/MA0e5VdlSZG8QCHxVKRbYDDzp+cEK5mzre0tICkd5IWHl/XHiZrIzJ+B MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: e9d71dd2-6d6e-4be8-d3ff-08d6dec22333 X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-originalarrivaltime: 22 May 2019 14:31:11.8077 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: HK2APC01HT023 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Background ========================================================================== Solve bug report: https://bugzilla.kernel.org/show_bug.cgi?id=203243 Currently, the kernel can sometimes assign the MMIO_PREF window additional size into the MMIO window, resulting in double the MMIO additional size, even if the MMIO_PREF window was successful. This happens if in the first pass, the MMIO_PREF succeeds but the MMIO fails. In the next pass, because MMIO_PREF is already assigned, the attempt to assign MMIO_PREF returns an error code instead of success (nothing more to do, already allocated). Example of problem (more context can be found in the bug report URL): Mainline kernel: pci 0000:06:01.0: BAR 14: assigned [mem 0x90100000-0xa00fffff] = 256M pci 0000:06:04.0: BAR 14: assigned [mem 0xa0200000-0xb01fffff] = 256M Patched kernel: pci 0000:06:01.0: BAR 14: assigned [mem 0x90100000-0x980fffff] = 128M pci 0000:06:04.0: BAR 14: assigned [mem 0x98200000-0xa01fffff] = 128M This was using pci=realloc,hpmemsize=128M,nocrs - on the same machine with the same configuration, with a Ubuntu mainline kernel and a kernel patched with this patch series. This patch is vital for the next patch in the series. The next patch allows the user to specify MMIO and MMIO_PREF independently. If the MMIO_PREF is set to be very large, this bug will end up more than doubling the MMIO size. The bug results in the MMIO_PREF being added to the MMIO window, which means doubling if MMIO_PREF size == MMIO size. With a large MMIO_PREF, without this patch, the MMIO window will likely fail to be assigned altogether due to lack of 32-bit address space. Patch notes ========================================================================== Change find_free_bus_resource() to not skip assigned resources with non-null parent. Add checks in pbus_size_io() and pbus_size_mem() to return success if resource returned from find_free_bus_resource() is already allocated. This avoids pbus_size_io() and pbus_size_mem() returning error code to __pci_bus_size_bridges() when a resource has been successfully assigned in a previous pass. This fixes the existing behaviour where space for a resource could be reserved multiple times in different parent bridge windows. This also greatly reduces the number of failed BAR messages in dmesg when Linux assigns resources. Signed-off-by: Nicholas Johnson --- drivers/pci/setup-bus.c | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/drivers/pci/setup-bus.c b/drivers/pci/setup-bus.c index 5675254fa..8e1bc7ee7 100644 --- a/drivers/pci/setup-bus.c +++ b/drivers/pci/setup-bus.c @@ -752,13 +752,18 @@ static void pci_bridge_check_ranges(struct pci_bus *bus) } /* - * Helper function for sizing routines: find first available bus resource - * of a given type. Note: we intentionally skip the bus resources which - * have already been assigned (that is, have non-NULL parent resource). + * Helper function for sizing routines: find first bus resource of a given + * type. Note: we do not skip the bus resources which have already been + * assigned (r->parent != NULL). This is because a resource that is already + * assigned (nothing more to be done) will be indistinguishable from one that + * failed due to lack of space if we skip assigned resources. If the caller + * function cannot tell the difference then it might try to place the + * resources in a different window, doubling up on resources or causing + * unforeseeable issues. */ -static struct resource *find_free_bus_resource(struct pci_bus *bus, - unsigned long type_mask, - unsigned long type) +struct resource *find_bus_resource_of_type(struct pci_bus *bus, + unsigned long type_mask, + unsigned long type) { int i; struct resource *r; @@ -766,7 +771,7 @@ static struct resource *find_free_bus_resource(struct pci_bus *bus, pci_bus_for_each_resource(bus, r, i) { if (r == &ioport_resource || r == &iomem_resource) continue; - if (r && (r->flags & type_mask) == type && !r->parent) + if (r && (r->flags & type_mask) == type) return r; } return NULL; @@ -866,14 +871,16 @@ static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size, struct list_head *realloc_head) { struct pci_dev *dev; - struct resource *b_res = find_free_bus_resource(bus, IORESOURCE_IO, - IORESOURCE_IO); + struct resource *b_res = find_bus_resource_of_type(bus, IORESOURCE_IO, + IORESOURCE_IO); resource_size_t size = 0, size0 = 0, size1 = 0; resource_size_t children_add_size = 0; resource_size_t min_align, align; if (!b_res) return; + if (b_res->parent) + return; min_align = window_alignment(bus, IORESOURCE_IO); list_for_each_entry(dev, &bus->devices, bus_list) { @@ -978,7 +985,7 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask, resource_size_t min_align, align, size, size0, size1; resource_size_t aligns[18]; /* Alignments from 1MB to 128GB */ int order, max_order; - struct resource *b_res = find_free_bus_resource(bus, + struct resource *b_res = find_bus_resource_of_type(bus, mask | IORESOURCE_PREFETCH, type); resource_size_t children_add_size = 0; resource_size_t children_add_align = 0; @@ -986,6 +993,8 @@ static int pbus_size_mem(struct pci_bus *bus, unsigned long mask, if (!b_res) return -ENOSPC; + if (b_res->parent) + return 0; memset(aligns, 0, sizeof(aligns)); max_order = 0;