From patchwork Thu Aug 16 18:45:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg Kroah-Hartman X-Patchwork-Id: 10568025 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 99FCD13B4 for ; Thu, 16 Aug 2018 18:47:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 989F428C13 for ; Thu, 16 Aug 2018 18:47:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8C6BF2A9CD; Thu, 16 Aug 2018 18:47:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1237228C13 for ; Thu, 16 Aug 2018 18:47:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2392D6B032B; Thu, 16 Aug 2018 14:47:41 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1E8AB6B032D; Thu, 16 Aug 2018 14:47:41 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0D84E6B032E; Thu, 16 Aug 2018 14:47:41 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) by kanga.kvack.org (Postfix) with ESMTP id C051C6B032B for ; Thu, 16 Aug 2018 14:47:40 -0400 (EDT) Received: by mail-pf1-f200.google.com with SMTP id q21-v6so2471622pff.21 for ; Thu, 16 Aug 2018 11:47:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:user-agent :mime-version; bh=69U65qnpJWSCcihjZSI4J/uGYRNeTlo0DkH4NAVujYU=; b=TSU6DKyoPoru6TdjK9bs+7DWM/ZMuPaftSo0lq6tPsKBPYkXQ/K3q53oq8ZsyqJN6M LTD/tXGKkXckIxB4WOo6mdYukdSNjzafKwg2a5HDrcFnz3AS4eITEAmN2L3Jde1W8v/C e4X8jlf6/Knw7vja9eNwobeYPrLFwX4wUMdjnDyWqzKkpX+QkpzNs2/oonkdstYkzuU7 e75SEeMOvlyTsU/mai/32QjrvlT83pHMdlW9nfb9lr3DmVwBsl4l4j+OdIKiJs3LW5cK pN+Qg58kgNJOaSMgjgvsqlcgaYh3yB1yDDnL0kZPoWNfO92NQiZxbzAUhrGxzWb3fCNC cWbw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org X-Gm-Message-State: AOUpUlErPSdVahnd6YimO45l+ZLoYxHbSoBztQ2saYimMWm3IN06kzp8 3rsj+x2UAP99H5VKoKs3VwcjEiCJlnVoDRPZBuVxTuWlIMNRxQlgHnyZqEP+isMB49FEghnTb2U ATWLLFAmGut2awDbngWBOroXXZliFJ49XpADp1S2dznXNBfwhBJWXHsOkmkcUmZnJzQ== X-Received: by 2002:a63:c60:: with SMTP id 32-v6mr30235623pgm.155.1534445260464; Thu, 16 Aug 2018 11:47:40 -0700 (PDT) X-Google-Smtp-Source: AA+uWPzJTCUiGs0DI3iC0SMhJOqhCzEp7DeYegoRvRTLT84POpStuyMZmZ2YKjRtY5ITCcA56uBP X-Received: by 2002:a63:c60:: with SMTP id 32-v6mr30235591pgm.155.1534445259845; Thu, 16 Aug 2018 11:47:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534445259; cv=none; d=google.com; s=arc-20160816; b=CoMFrphS+1gKNxMVDu8g9CLJGlbMPm5GTksCjRwkq34ReVcH6mj2RiSpvz6UWpUocQ HiSWnYvR7HhmAc3oZqnii1JfeJ6/Ln4hjwB2sz4eE0zvWFwc7Axy4Z4L02+NNzD4rZ6G XTwo3PA/vqNR172fSZ5kMNjZrZskydoIgl5rEoybVT/tmoLaS0aN850wAKoj0TFyQ2Qz ZspWP7d26/zzlxP6p0oRQ7E5P1TWu2r6WTlAlzsFkSUOyR1pXmz5t56sXIWMknFanfXo p2mwubc0caG9cngVB36fNr5LtO3auR8LEt8+5coB9kfEDHHjTAyyPF1LdBuRIXi6fdsA i+rw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=69U65qnpJWSCcihjZSI4J/uGYRNeTlo0DkH4NAVujYU=; b=r2+ellPd6e53oyigbOHSxm3sv9oMLUodku9MD9miwzgg59xtiU5wUGwsAklVSKjVFq xxZ08aHv00iER+L+caNbGZGTt2zLwHQD05FmcMVb0ToLjfmpvAiBgMZun+b3eIWRRLhF hsHoqp/G9aSJJxuh7YrQ1NUwQEq0FPKMNsc7d3aKmbHof+5ccoVzht1lmWwqOBnD5RaC BbpJ40b3UkpgWE6c44glwtjPUG7Q+7/PBxJpp2npcbgNuk+NlCJimQG5CrVUJE7rMLzn uYEUQwyIo2ngV4O3/kfIKVqec+z0vMqtmMoZ+1smwdm+fa2sptR34d2w7dqkjYXqTTIa 9hwg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org Received: from mail.linuxfoundation.org (mail.linuxfoundation.org. [140.211.169.12]) by mx.google.com with ESMTPS id v34-v6si29864plg.491.2018.08.16.11.47.39 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 16 Aug 2018 11:47:39 -0700 (PDT) Received-SPF: pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) client-ip=140.211.169.12; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org Received: from localhost (unknown [194.244.16.108]) by mail.linuxfoundation.org (Postfix) with ESMTPSA id 015D2D3B; Thu, 16 Aug 2018 18:47:39 +0000 (UTC) From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Joerg Roedel , Toshi Kani , Thomas Gleixner , mhocko@suse.com, akpm@linux-foundation.org, hpa@zytor.com, cpandya@codeaurora.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH 4.17 06/21] x86/mm: Disable ioremap free page handling on x86-PAE Date: Thu, 16 Aug 2018 20:45:16 +0200 Message-Id: <20180816171612.669302985@linuxfoundation.org> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180816171612.136242278@linuxfoundation.org> References: <20180816171612.136242278@linuxfoundation.org> User-Agent: quilt/0.65 X-stable: review MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP 4.17-stable review patch. If anyone has any objections, please let me know. ------------------ From: Toshi Kani commit f967db0b9ed44ec3057a28f3b28efc51df51b835 upstream. ioremap() supports pmd mappings on x86-PAE. However, kernel's pmd tables are not shared among processes on x86-PAE. Therefore, any update to sync'd pmd entries need re-syncing. Freeing a pte page also leads to a vmalloc fault and hits the BUG_ON in vmalloc_sync_one(). Disable free page handling on x86-PAE. pud_free_pmd_page() and pmd_free_pte_page() simply return 0 if a given pud/pmd entry is present. This assures that ioremap() does not update sync'd pmd entries at the cost of falling back to pte mappings. Fixes: 28ee90fe6048 ("x86/mm: implement free pmd/pte page interfaces") Reported-by: Joerg Roedel Signed-off-by: Toshi Kani Signed-off-by: Thomas Gleixner Cc: mhocko@suse.com Cc: akpm@linux-foundation.org Cc: hpa@zytor.com Cc: cpandya@codeaurora.org Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: stable@vger.kernel.org Cc: Andrew Morton Cc: Michal Hocko Cc: "H. Peter Anvin" Cc: Link: https://lkml.kernel.org/r/20180627141348.21777-2-toshi.kani@hpe.com Signed-off-by: Greg Kroah-Hartman --- arch/x86/mm/pgtable.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -715,6 +715,7 @@ int pmd_clear_huge(pmd_t *pmd) return 0; } +#ifdef CONFIG_X86_64 /** * pud_free_pmd_page - Clear pud entry and free pmd page. * @pud: Pointer to a PUD. @@ -762,4 +763,22 @@ int pmd_free_pte_page(pmd_t *pmd) return 1; } + +#else /* !CONFIG_X86_64 */ + +int pud_free_pmd_page(pud_t *pud) +{ + return pud_none(*pud); +} + +/* + * Disable free page handling on x86-PAE. This assures that ioremap() + * does not update sync'd pmd entries. See vmalloc_sync_one(). + */ +int pmd_free_pte_page(pmd_t *pmd) +{ + return pmd_none(*pmd); +} + +#endif /* CONFIG_X86_64 */ #endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */ From patchwork Thu Aug 16 18:45:30 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg Kroah-Hartman X-Patchwork-Id: 10568019 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B0FA61669 for ; Thu, 16 Aug 2018 18:47:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B04D728C13 for ; Thu, 16 Aug 2018 18:47:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A3AAD2A9CD; Thu, 16 Aug 2018 18:47:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DB08228C13 for ; Thu, 16 Aug 2018 18:47:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1282E6B0326; Thu, 16 Aug 2018 14:47:26 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0D79B6B0328; Thu, 16 Aug 2018 14:47:26 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F099E6B0329; Thu, 16 Aug 2018 14:47:25 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) by kanga.kvack.org (Postfix) with ESMTP id ACB906B0326 for ; Thu, 16 Aug 2018 14:47:25 -0400 (EDT) Received: by mail-pg1-f198.google.com with SMTP id r2-v6so2431171pgp.3 for ; Thu, 16 Aug 2018 11:47:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:user-agent :mime-version; bh=vKwtofEr6cgDDhcS4qqBgPW7aOEGOclfIL1oqSRRij4=; b=J8lGFVb6NGkxzVv536d3uzXp41MqgYeBK0usZrDmm2+m5W1HG7+LsUIepD9bU9OSq+ E4F4wxkzmuknCLksLnTrYde7n14RExhFDcg7nGEaCKEWizmEh/Axz8wxiJSbff/H9FLh U2HbSE+jr3hxUZVaOmlK56j25ZIofHpRpR/pr7++/JmXyjWpj2wqJHQT3JOVavD7cOWu QIo96zkYqKTNzhR0ZuufsZum+bIMe8MD++aDBMA8EGyqgqqQ38uMopswXXvW2RNLGXCf 7X+jQ/bmmbgz1iNaxS41xcr3daeWDEpx3v5dUkjXzyICh7twnDuAXUI0NaUPqhyDUu7S gN6A== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org X-Gm-Message-State: AOUpUlEvln9nQ3r/0ceSgsgPMKxpoyZisc/8D8oa3G0Io6xkNPLGS5xQ 8j6S8IzHkgQijHgh8tdyzEFFKzFLdy2TxUBMP7L5Ne43f+og0//GCvU0d3wgcxDRpTIxKG6NFfP lYo39McICJirbos6NlBZv8vGJiphoqclhlmoLE52amJkaccALANSStXyMjg+NMPUbdA== X-Received: by 2002:a62:8559:: with SMTP id u86-v6mr33740900pfd.32.1534445245367; Thu, 16 Aug 2018 11:47:25 -0700 (PDT) X-Google-Smtp-Source: AA+uWPwZvdYJn0VnKEmqgUCkNWWAGl+403YVPAcocHPoOfetchK9NKzB1jnpRTtFZiJBWKbgNRnW X-Received: by 2002:a62:8559:: with SMTP id u86-v6mr33740856pfd.32.1534445244683; Thu, 16 Aug 2018 11:47:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534445244; cv=none; d=google.com; s=arc-20160816; b=Z/LIiNVG0Xgkql7MAHP1D7b8j1uGemFbsmBnUmnBu03mHHYJCfS7M+ZH/ManRD/lmb nyQe1Oqr+3F6fTFL6xSzBSedAgGI19RLSshsEPm2P0SK+vv8ZUjjdU1t/AK8I7eoR9mw yoSDtj8zvLNNXrUawH9KKOGJR7OzWb4xEtamlmXYlRBaka1F8CTAHP8IgEYBprkCyb5h pmH/YZwr7h8zrpxWkmWmpzfM3tWIWlsgmGMPn3OHbKR4OohYdCZYiTD7s/HJ7Iw1tga8 5TCQE9k1Dv7lmAMZELSfHgdfklRjXBPaWVFOX0ElpOGHhiLbPrtT72oQAH6j8wACRmhK zpgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=vKwtofEr6cgDDhcS4qqBgPW7aOEGOclfIL1oqSRRij4=; b=wmF632SBNFFrmFwWbX1tsSRvIcmxfR/2IAGMuWG6wRtsXyoDkUJ0T57jUtNdchaQJl Nqodo0yIlNy86LRqkaNXutlMltllCxSw5wYNufbtcNFJQVr96eW5bk2pnKSKdDHkBMCH 2jen2f8RyN4xmxSS5MfX/F9AoP6TpN4EPquEeRsITfSVaZw20jSYS7Rzg8fWEDU5x4rN zADgxCL/k0WUCfccBdOjbBG/8tMW989y72Q6/Ed8qic5xBD1yxcrCSoei02bB6W8TsjZ hB/JpPwtNBUKxgCH3J7J3EXG4SAqMqcFBqvLpxi/1atL7Eqhnq4GxwCKSlf6jMhIDE2H 3fYQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org Received: from mail.linuxfoundation.org (mail.linuxfoundation.org. [140.211.169.12]) by mx.google.com with ESMTPS id f4-v6si48225plr.213.2018.08.16.11.47.24 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 16 Aug 2018 11:47:24 -0700 (PDT) Received-SPF: pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) client-ip=140.211.169.12; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org Received: from localhost (unknown [194.244.16.108]) by mail.linuxfoundation.org (Postfix) with ESMTPSA id C88FACC4; Thu, 16 Aug 2018 18:47:23 +0000 (UTC) From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Chintan Pandya , Toshi Kani , Thomas Gleixner , mhocko@suse.com, akpm@linux-foundation.org, hpa@zytor.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Will Deacon , Joerg Roedel Subject: [PATCH 4.17 20/21] ioremap: Update pgtable free interfaces with addr Date: Thu, 16 Aug 2018 20:45:30 +0200 Message-Id: <20180816171614.184181311@linuxfoundation.org> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180816171612.136242278@linuxfoundation.org> References: <20180816171612.136242278@linuxfoundation.org> User-Agent: quilt/0.65 X-stable: review MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP 4.17-stable review patch. If anyone has any objections, please let me know. ------------------ From: Chintan Pandya commit 785a19f9d1dd8a4ab2d0633be4656653bd3de1fc upstream. The following kernel panic was observed on ARM64 platform due to a stale TLB entry. 1. ioremap with 4K size, a valid pte page table is set. 2. iounmap it, its pte entry is set to 0. 3. ioremap the same address with 2M size, update its pmd entry with a new value. 4. CPU may hit an exception because the old pmd entry is still in TLB, which leads to a kernel panic. Commit b6bdb7517c3d ("mm/vmalloc: add interfaces to free unmapped page table") has addressed this panic by falling to pte mappings in the above case on ARM64. To support pmd mappings in all cases, TLB purge needs to be performed in this case on ARM64. Add a new arg, 'addr', to pud_free_pmd_page() and pmd_free_pte_page() so that TLB purge can be added later in seprate patches. [toshi.kani@hpe.com: merge changes, rewrite patch description] Fixes: 28ee90fe6048 ("x86/mm: implement free pmd/pte page interfaces") Signed-off-by: Chintan Pandya Signed-off-by: Toshi Kani Signed-off-by: Thomas Gleixner Cc: mhocko@suse.com Cc: akpm@linux-foundation.org Cc: hpa@zytor.com Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: Will Deacon Cc: Joerg Roedel Cc: stable@vger.kernel.org Cc: Andrew Morton Cc: Michal Hocko Cc: "H. Peter Anvin" Cc: Link: https://lkml.kernel.org/r/20180627141348.21777-3-toshi.kani@hpe.com Signed-off-by: Greg Kroah-Hartman --- arch/arm64/mm/mmu.c | 4 ++-- arch/x86/mm/pgtable.c | 12 +++++++----- include/asm-generic/pgtable.h | 8 ++++---- lib/ioremap.c | 4 ++-- 4 files changed, 15 insertions(+), 13 deletions(-) --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -977,12 +977,12 @@ int pmd_clear_huge(pmd_t *pmdp) return 1; } -int pud_free_pmd_page(pud_t *pud) +int pud_free_pmd_page(pud_t *pud, unsigned long addr) { return pud_none(*pud); } -int pmd_free_pte_page(pmd_t *pmd) +int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) { return pmd_none(*pmd); } --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -719,11 +719,12 @@ int pmd_clear_huge(pmd_t *pmd) /** * pud_free_pmd_page - Clear pud entry and free pmd page. * @pud: Pointer to a PUD. + * @addr: Virtual address associated with pud. * * Context: The pud range has been unmaped and TLB purged. * Return: 1 if clearing the entry succeeded. 0 otherwise. */ -int pud_free_pmd_page(pud_t *pud) +int pud_free_pmd_page(pud_t *pud, unsigned long addr) { pmd_t *pmd; int i; @@ -734,7 +735,7 @@ int pud_free_pmd_page(pud_t *pud) pmd = (pmd_t *)pud_page_vaddr(*pud); for (i = 0; i < PTRS_PER_PMD; i++) - if (!pmd_free_pte_page(&pmd[i])) + if (!pmd_free_pte_page(&pmd[i], addr + (i * PMD_SIZE))) return 0; pud_clear(pud); @@ -746,11 +747,12 @@ int pud_free_pmd_page(pud_t *pud) /** * pmd_free_pte_page - Clear pmd entry and free pte page. * @pmd: Pointer to a PMD. + * @addr: Virtual address associated with pmd. * * Context: The pmd range has been unmaped and TLB purged. * Return: 1 if clearing the entry succeeded. 0 otherwise. */ -int pmd_free_pte_page(pmd_t *pmd) +int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) { pte_t *pte; @@ -766,7 +768,7 @@ int pmd_free_pte_page(pmd_t *pmd) #else /* !CONFIG_X86_64 */ -int pud_free_pmd_page(pud_t *pud) +int pud_free_pmd_page(pud_t *pud, unsigned long addr) { return pud_none(*pud); } @@ -775,7 +777,7 @@ int pud_free_pmd_page(pud_t *pud) * Disable free page handling on x86-PAE. This assures that ioremap() * does not update sync'd pmd entries. See vmalloc_sync_one(). */ -int pmd_free_pte_page(pmd_t *pmd) +int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) { return pmd_none(*pmd); } --- a/include/asm-generic/pgtable.h +++ b/include/asm-generic/pgtable.h @@ -1019,8 +1019,8 @@ int pud_set_huge(pud_t *pud, phys_addr_t int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot); int pud_clear_huge(pud_t *pud); int pmd_clear_huge(pmd_t *pmd); -int pud_free_pmd_page(pud_t *pud); -int pmd_free_pte_page(pmd_t *pmd); +int pud_free_pmd_page(pud_t *pud, unsigned long addr); +int pmd_free_pte_page(pmd_t *pmd, unsigned long addr); #else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */ static inline int p4d_set_huge(p4d_t *p4d, phys_addr_t addr, pgprot_t prot) { @@ -1046,11 +1046,11 @@ static inline int pmd_clear_huge(pmd_t * { return 0; } -static inline int pud_free_pmd_page(pud_t *pud) +static inline int pud_free_pmd_page(pud_t *pud, unsigned long addr) { return 0; } -static inline int pmd_free_pte_page(pmd_t *pmd) +static inline int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) { return 0; } --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -92,7 +92,7 @@ static inline int ioremap_pmd_range(pud_ if (ioremap_pmd_enabled() && ((next - addr) == PMD_SIZE) && IS_ALIGNED(phys_addr + addr, PMD_SIZE) && - pmd_free_pte_page(pmd)) { + pmd_free_pte_page(pmd, addr)) { if (pmd_set_huge(pmd, phys_addr + addr, prot)) continue; } @@ -119,7 +119,7 @@ static inline int ioremap_pud_range(p4d_ if (ioremap_pud_enabled() && ((next - addr) == PUD_SIZE) && IS_ALIGNED(phys_addr + addr, PUD_SIZE) && - pud_free_pmd_page(pud)) { + pud_free_pmd_page(pud, addr)) { if (pud_set_huge(pud, phys_addr + addr, prot)) continue; } From patchwork Thu Aug 16 18:45:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg Kroah-Hartman X-Patchwork-Id: 10568021 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D4CB31575 for ; Thu, 16 Aug 2018 18:47:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D374028C13 for ; Thu, 16 Aug 2018 18:47:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C79862A9CD; Thu, 16 Aug 2018 18:47:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3BFE528C13 for ; Thu, 16 Aug 2018 18:47:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 125FE6B0329; Thu, 16 Aug 2018 14:47:29 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0D7986B032A; Thu, 16 Aug 2018 14:47:29 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E93CF6B032C; Thu, 16 Aug 2018 14:47:28 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id A750C6B0329 for ; Thu, 16 Aug 2018 14:47:28 -0400 (EDT) Received: by mail-pf1-f198.google.com with SMTP id p5-v6so2467391pfh.11 for ; Thu, 16 Aug 2018 11:47:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:user-agent :mime-version; bh=EaKK2AyYk/I6iOBzhhYAbrMpX3D9fTWGz7qHLYUQG0M=; b=K4wCst2Ar9EI8H4iCAbQI3mqsX2VA7XLiRr7dDvxTJIaheFbReIwNun0lX1iHrjpc8 E/S3gAkAYXLBCfmdrALZhR92GKiH861a+8TCFS+oBU7ATqUmUReem44D7xhIVkG5fDlP dwfp0iVZ+839WJWYjQWeHSYydsyDk4cJAAjHPsmLpqwSjksAWnCLdS2rqCwenbCAMoCx wwsttu7zb+EyjRQ+jDySk7ofLpdUWZseAZq+n4urJL+QSZS/HUbvsqpnzQ+JcqQ5xqdK iI1MTYMyhB/YgWRXY7XYQuTA9kua/2DxA8jIacFG0qFtUekn8Av2jcEjotb7XI5ujR/1 kwjw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org X-Gm-Message-State: AOUpUlG9tHd9YzQnqAPyuJQAfjAR1gQhTEM50tRIanwJptfcax8xdNLr BsTb3WWZvy1wjdQ5CDYiobzZdKovBVmJgQeeG0L4vYv0TIuuJIDJ8cfaGGQLNqDi9NBWtxnKcIg ZeeGNU/WTNlXw49m/ihQGcq0MtOIqCmRmp7H2ncpxf70Nqut5PkzJ5ReYCUqWZZEPzA== X-Received: by 2002:a63:b605:: with SMTP id j5-v6mr438914pgf.437.1534445248338; Thu, 16 Aug 2018 11:47:28 -0700 (PDT) X-Google-Smtp-Source: AA+uWPyPfHuKOCS8Wmm7ZiT+AQJxVt8b3Rt/NeQU2yWlIIujCeGYZaU5/JRt55ePTzb5bI6jespt X-Received: by 2002:a63:b605:: with SMTP id j5-v6mr438890pgf.437.1534445247727; Thu, 16 Aug 2018 11:47:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534445247; cv=none; d=google.com; s=arc-20160816; b=GlTA+yHT7qhy9GjLwZLFdo+EPkyGtTHCl1qyZ3VSshWK+6jLc7GJ4qwQ4GnIKZ6B6m lqXIh2wAvGTObHHO2J56YTtZDqAhaaYkHr/cP4Fqd0Gcv8EnwfiZLg9QZVBfI1HjXg5d 79Klj671qoFgHchusXmvLdyD5Plaw5UkqjWGJRgCXXrarojB2bqhtk61DxJGyTZpDbIf r+phG3HfNUR46O/QGDxRZaSMRmgvsLJhIf78X2GwKqEFu6HZAaZJGzHoVqmomE/RtxIu 4ydG1DmD9eZpIH5K+Qojl3ycg9gS9N+QWU62vUm8mWYoMuwspXKQrMbaZA3YS78Ar7bL r/vA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=EaKK2AyYk/I6iOBzhhYAbrMpX3D9fTWGz7qHLYUQG0M=; b=Dh8r3V/5N/DwUmG3QHECb4LTfwnj6FnGY4+sjKG/LVs5R9AKb/yLBWulP40RENgQ+B 3vj4SREdB9fjGu+iH1sTWpOHbPFcfkv6HvUwLF1W+0zw8IMwlHyUPJ1FUt+0a4P9toIB ypqwCVptmskBnpVhZXVajg1mxvad/BUCQS3V4dsFWP7RDJyldV/ZeNaE0XF3kathpNGf EbyvTtjuEU3uJJ9sLUaVRXYcnFEWij2JX7cIxw+Jhlw9ym9oB9/5VjXvHBsjxHevvTko nSeofOOHvxaOlmnnDZuMYGI2SwQVG/IfjLdHUxpGv3YtrNu8rjjlIYQ+e717kL2mAXCr zfUA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org Received: from mail.linuxfoundation.org (mail.linuxfoundation.org. [140.211.169.12]) by mx.google.com with ESMTPS id e14-v6si47236pgj.413.2018.08.16.11.47.27 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 16 Aug 2018 11:47:27 -0700 (PDT) Received-SPF: pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) client-ip=140.211.169.12; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org Received: from localhost (unknown [194.244.16.108]) by mail.linuxfoundation.org (Postfix) with ESMTPSA id D6F89CE8; Thu, 16 Aug 2018 18:47:26 +0000 (UTC) From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Toshi Kani , Thomas Gleixner , mhocko@suse.com, akpm@linux-foundation.org, hpa@zytor.com, cpandya@codeaurora.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Joerg Roedel Subject: [PATCH 4.17 21/21] x86/mm: Add TLB purge to free pmd/pte page interfaces Date: Thu, 16 Aug 2018 20:45:31 +0200 Message-Id: <20180816171614.266666560@linuxfoundation.org> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180816171612.136242278@linuxfoundation.org> References: <20180816171612.136242278@linuxfoundation.org> User-Agent: quilt/0.65 X-stable: review MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP 4.17-stable review patch. If anyone has any objections, please let me know. ------------------ From: Toshi Kani commit 5e0fb5df2ee871b841f96f9cb6a7f2784e96aa4e upstream. ioremap() calls pud_free_pmd_page() / pmd_free_pte_page() when it creates a pud / pmd map. The following preconditions are met at their entry. - All pte entries for a target pud/pmd address range have been cleared. - System-wide TLB purges have been peformed for a target pud/pmd address range. The preconditions assure that there is no stale TLB entry for the range. Speculation may not cache TLB entries since it requires all levels of page entries, including ptes, to have P & A-bits set for an associated address. However, speculation may cache pud/pmd entries (paging-structure caches) when they have P-bit set. Add a system-wide TLB purge (INVLPG) to a single page after clearing pud/pmd entry's P-bit. SDM 4.10.4.1, Operation that Invalidate TLBs and Paging-Structure Caches, states that: INVLPG invalidates all paging-structure caches associated with the current PCID regardless of the liner addresses to which they correspond. Fixes: 28ee90fe6048 ("x86/mm: implement free pmd/pte page interfaces") Signed-off-by: Toshi Kani Signed-off-by: Thomas Gleixner Cc: mhocko@suse.com Cc: akpm@linux-foundation.org Cc: hpa@zytor.com Cc: cpandya@codeaurora.org Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: Joerg Roedel Cc: stable@vger.kernel.org Cc: Andrew Morton Cc: Michal Hocko Cc: "H. Peter Anvin" Cc: Link: https://lkml.kernel.org/r/20180627141348.21777-4-toshi.kani@hpe.com Signed-off-by: Greg Kroah-Hartman --- arch/x86/mm/pgtable.c | 38 +++++++++++++++++++++++++++++++------- 1 file changed, 31 insertions(+), 7 deletions(-) --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -721,24 +721,44 @@ int pmd_clear_huge(pmd_t *pmd) * @pud: Pointer to a PUD. * @addr: Virtual address associated with pud. * - * Context: The pud range has been unmaped and TLB purged. + * Context: The pud range has been unmapped and TLB purged. * Return: 1 if clearing the entry succeeded. 0 otherwise. + * + * NOTE: Callers must allow a single page allocation. */ int pud_free_pmd_page(pud_t *pud, unsigned long addr) { - pmd_t *pmd; + pmd_t *pmd, *pmd_sv; + pte_t *pte; int i; if (pud_none(*pud)) return 1; pmd = (pmd_t *)pud_page_vaddr(*pud); - - for (i = 0; i < PTRS_PER_PMD; i++) - if (!pmd_free_pte_page(&pmd[i], addr + (i * PMD_SIZE))) - return 0; + pmd_sv = (pmd_t *)__get_free_page(GFP_KERNEL); + if (!pmd_sv) + return 0; + + for (i = 0; i < PTRS_PER_PMD; i++) { + pmd_sv[i] = pmd[i]; + if (!pmd_none(pmd[i])) + pmd_clear(&pmd[i]); + } pud_clear(pud); + + /* INVLPG to clear all paging-structure caches */ + flush_tlb_kernel_range(addr, addr + PAGE_SIZE-1); + + for (i = 0; i < PTRS_PER_PMD; i++) { + if (!pmd_none(pmd_sv[i])) { + pte = (pte_t *)pmd_page_vaddr(pmd_sv[i]); + free_page((unsigned long)pte); + } + } + + free_page((unsigned long)pmd_sv); free_page((unsigned long)pmd); return 1; @@ -749,7 +769,7 @@ int pud_free_pmd_page(pud_t *pud, unsign * @pmd: Pointer to a PMD. * @addr: Virtual address associated with pmd. * - * Context: The pmd range has been unmaped and TLB purged. + * Context: The pmd range has been unmapped and TLB purged. * Return: 1 if clearing the entry succeeded. 0 otherwise. */ int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) @@ -761,6 +781,10 @@ int pmd_free_pte_page(pmd_t *pmd, unsign pte = (pte_t *)pmd_page_vaddr(*pmd); pmd_clear(pmd); + + /* INVLPG to clear all paging-structure caches */ + flush_tlb_kernel_range(addr, addr + PAGE_SIZE-1); + free_page((unsigned long)pte); return 1;