From patchwork Thu Aug 16 18:45:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg KH X-Patchwork-Id: 10568009 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1E690921 for ; Thu, 16 Aug 2018 18:46:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1DC5F28C13 for ; Thu, 16 Aug 2018 18:46:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 11B562A9A9; Thu, 16 Aug 2018 18:46:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6A1F228C13 for ; Thu, 16 Aug 2018 18:46:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 484F06B0322; Thu, 16 Aug 2018 14:46:20 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 45AFA6B0323; Thu, 16 Aug 2018 14:46:20 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 34C076B0324; Thu, 16 Aug 2018 14:46:20 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl0-f72.google.com (mail-pl0-f72.google.com [209.85.160.72]) by kanga.kvack.org (Postfix) with ESMTP id DFD276B0322 for ; Thu, 16 Aug 2018 14:46:19 -0400 (EDT) Received: by mail-pl0-f72.google.com with SMTP id bg5-v6so231555plb.20 for ; Thu, 16 Aug 2018 11:46:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:user-agent :mime-version; bh=VB1Gmhdd0KcKpopNysnZ8/1B2sNWYxAx0mhaX+a5c9o=; b=opL2JyYXGm/q/gWTni4b0zZW8oFpNXlXLUAy7L0FnzcsC0uqeL7/10PaG1uxycZ7RM DfPZEVAGgmCifsAafcuBYM8p+Dg+D6yXlQxv0zLCWdCv21fLEE+afIX/LqjneAJZnKs8 SKaJfcOpxfFlPqy5J9sPEnBV6FTy6xAEpQZ7Ol9P1VG+WgN0aFGxCoy/25XcGKBcTmIK x1G4SYNa+5FddjBU2d/bKlW5FeBPriP0OdFCl6H0fNd6l5HDW2rSI7leQ2h66I4zdhwv 8c30+hPoyGvbvPcdHBbFUocQZbes9ZIyhzZiwQw/hPgyA07T7EGDES9Z1ZLZVtz9LcDj 1Y6A== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org X-Gm-Message-State: AOUpUlEH7o8sPNSOZdOximTfivMnSEyRhvLtKPoRlpHdXOMY3/DNLk32 iVks82/FDJvZqhX2aTsmWnAiJyQqBwLeP6BMJVA2/KvSPCS2u2zeB7tqCsR7ExVJ09wSYsAaDV7 bVhpxMt48z9d6Sx44qbgk8/OAUocahWa/poU1qA3QX9dv1pOdS5FdZMLi1z2Tq2awFA== X-Received: by 2002:a17:902:1101:: with SMTP id d1-v6mr29875290pla.131.1534445179594; Thu, 16 Aug 2018 11:46:19 -0700 (PDT) X-Google-Smtp-Source: AA+uWPznfXRXJ8+vMumWaC3tZdehGSrtdAh66+aL8Hu8O7frZDCy3cVEXixwA2hh7TPBfwosjAuV X-Received: by 2002:a17:902:1101:: with SMTP id d1-v6mr29875257pla.131.1534445178931; Thu, 16 Aug 2018 11:46:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534445178; cv=none; d=google.com; s=arc-20160816; b=Htz318Dlli8HNbXUIbAS/+BT9o0PFC6w9b0dwK/YoIyLhpBbLA4G0Q4if4ATIt4zes hdKsOcz335uuYdx24i3wvIsBFtapct3Vho72F0ocuVLbTktDnCte2jonb+qHxq7wOu+j KdtvGxDFpkZ3IaETsaHzt0+X978IGIym3JnWrqGBpUHD6Ts8nV2ftx0WYqoNefuVtdKx XeBqPD+/mVjCdR1XrV76tlc0Nr5coySX0emsNEt6LhdjDboP/zioR4XPHXWLnTCdKI+n 9vIfIJnpOrKcI/UItr4ecTHj5EWHv5ab5sfSqt/UkdJ6wgwASwZkSzEdHlRHb0iG/U6K JIwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=VB1Gmhdd0KcKpopNysnZ8/1B2sNWYxAx0mhaX+a5c9o=; b=etbQFGNCiMGV+YTjdSbktsVDwKzWlWmXG/H3Z4Oh6Mws+VkvcC6k7UnblTdAbx5cO1 BJVvtuoynLSxYnEEL70UdypTIcn1awtyW7la0TSBiwST9Dpdd0720BdIzjHS+iUpa9DY hRd+t52ny+Aet3hJQEP54sJvomxuur/nIT0LdfM1MfMStB+luR+fUDO00SI4LX28dstL TOItuAFyJV11hzIHbMe5zw6ZZtfgsmnA3MIpE+us69bqyc1s0iJLs8VSNBI5yK6Bb3dG XLE1/XNQHCd0tw2CAI20QIytS8LjrCnCr576Qj+QJhNmgPBynu4opdls2QxnkiL5cGRW cKcQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org Received: from mail.linuxfoundation.org (mail.linuxfoundation.org. [140.211.169.12]) by mx.google.com with ESMTPS id p17-v6si53005plo.186.2018.08.16.11.46.18 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 16 Aug 2018 11:46:18 -0700 (PDT) Received-SPF: pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) client-ip=140.211.169.12; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org Received: from localhost (unknown [194.244.16.108]) by mail.linuxfoundation.org (Postfix) with ESMTPSA id 161A8D02; Thu, 16 Aug 2018 18:46:18 +0000 (UTC) From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Toshi Kani , Thomas Gleixner , mhocko@suse.com, akpm@linux-foundation.org, hpa@zytor.com, cpandya@codeaurora.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, Joerg Roedel Subject: [PATCH 4.14 22/22] x86/mm: Add TLB purge to free pmd/pte page interfaces Date: Thu, 16 Aug 2018 20:45:23 +0200 Message-Id: <20180816171625.887193919@linuxfoundation.org> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180816171622.831964729@linuxfoundation.org> References: <20180816171622.831964729@linuxfoundation.org> User-Agent: quilt/0.65 X-stable: review MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP 4.14-stable review patch. If anyone has any objections, please let me know. ------------------ From: Toshi Kani commit 5e0fb5df2ee871b841f96f9cb6a7f2784e96aa4e upstream. ioremap() calls pud_free_pmd_page() / pmd_free_pte_page() when it creates a pud / pmd map. The following preconditions are met at their entry. - All pte entries for a target pud/pmd address range have been cleared. - System-wide TLB purges have been peformed for a target pud/pmd address range. The preconditions assure that there is no stale TLB entry for the range. Speculation may not cache TLB entries since it requires all levels of page entries, including ptes, to have P & A-bits set for an associated address. However, speculation may cache pud/pmd entries (paging-structure caches) when they have P-bit set. Add a system-wide TLB purge (INVLPG) to a single page after clearing pud/pmd entry's P-bit. SDM 4.10.4.1, Operation that Invalidate TLBs and Paging-Structure Caches, states that: INVLPG invalidates all paging-structure caches associated with the current PCID regardless of the liner addresses to which they correspond. Fixes: 28ee90fe6048 ("x86/mm: implement free pmd/pte page interfaces") Signed-off-by: Toshi Kani Signed-off-by: Thomas Gleixner Cc: mhocko@suse.com Cc: akpm@linux-foundation.org Cc: hpa@zytor.com Cc: cpandya@codeaurora.org Cc: linux-mm@kvack.org Cc: linux-arm-kernel@lists.infradead.org Cc: Joerg Roedel Cc: stable@vger.kernel.org Cc: Andrew Morton Cc: Michal Hocko Cc: "H. Peter Anvin" Cc: Link: https://lkml.kernel.org/r/20180627141348.21777-4-toshi.kani@hpe.com Signed-off-by: Greg Kroah-Hartman --- arch/x86/mm/pgtable.c | 38 +++++++++++++++++++++++++++++++------- 1 file changed, 31 insertions(+), 7 deletions(-) --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -718,24 +718,44 @@ int pmd_clear_huge(pmd_t *pmd) * @pud: Pointer to a PUD. * @addr: Virtual address associated with pud. * - * Context: The pud range has been unmaped and TLB purged. + * Context: The pud range has been unmapped and TLB purged. * Return: 1 if clearing the entry succeeded. 0 otherwise. + * + * NOTE: Callers must allow a single page allocation. */ int pud_free_pmd_page(pud_t *pud, unsigned long addr) { - pmd_t *pmd; + pmd_t *pmd, *pmd_sv; + pte_t *pte; int i; if (pud_none(*pud)) return 1; pmd = (pmd_t *)pud_page_vaddr(*pud); - - for (i = 0; i < PTRS_PER_PMD; i++) - if (!pmd_free_pte_page(&pmd[i], addr + (i * PMD_SIZE))) - return 0; + pmd_sv = (pmd_t *)__get_free_page(GFP_KERNEL); + if (!pmd_sv) + return 0; + + for (i = 0; i < PTRS_PER_PMD; i++) { + pmd_sv[i] = pmd[i]; + if (!pmd_none(pmd[i])) + pmd_clear(&pmd[i]); + } pud_clear(pud); + + /* INVLPG to clear all paging-structure caches */ + flush_tlb_kernel_range(addr, addr + PAGE_SIZE-1); + + for (i = 0; i < PTRS_PER_PMD; i++) { + if (!pmd_none(pmd_sv[i])) { + pte = (pte_t *)pmd_page_vaddr(pmd_sv[i]); + free_page((unsigned long)pte); + } + } + + free_page((unsigned long)pmd_sv); free_page((unsigned long)pmd); return 1; @@ -746,7 +766,7 @@ int pud_free_pmd_page(pud_t *pud, unsign * @pmd: Pointer to a PMD. * @addr: Virtual address associated with pmd. * - * Context: The pmd range has been unmaped and TLB purged. + * Context: The pmd range has been unmapped and TLB purged. * Return: 1 if clearing the entry succeeded. 0 otherwise. */ int pmd_free_pte_page(pmd_t *pmd, unsigned long addr) @@ -758,6 +778,10 @@ int pmd_free_pte_page(pmd_t *pmd, unsign pte = (pte_t *)pmd_page_vaddr(*pmd); pmd_clear(pmd); + + /* INVLPG to clear all paging-structure caches */ + flush_tlb_kernel_range(addr, addr + PAGE_SIZE-1); + free_page((unsigned long)pte); return 1;