From patchwork Thu Feb 21 11:34:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10823827 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 344CA6C2 for ; Thu, 21 Feb 2019 11:35:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3412D284AF for ; Thu, 21 Feb 2019 11:35:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 27E7A30634; Thu, 21 Feb 2019 11:35:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ACE1A284AF for ; Thu, 21 Feb 2019 11:35:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 792068E0073; Thu, 21 Feb 2019 06:35:22 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 73DFF8E0002; Thu, 21 Feb 2019 06:35:22 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E11A8E0073; Thu, 21 Feb 2019 06:35:22 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id EEFCA8E0002 for ; Thu, 21 Feb 2019 06:35:21 -0500 (EST) Received: by mail-ed1-f72.google.com with SMTP id x15so11459788edd.2 for ; Thu, 21 Feb 2019 03:35:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=MCQE4kh5SXgiU/bXz8lLVd48nfz6SGnOiJwrDNu6rZg=; b=sJZhZlJnOUrbreV4AQIethBwT1PJ1EIIi3j2Du7ENcGNcK9t42zJ7sjBBIFlfwULiq X+P/IxzKgD7BCseYU21B4Kjf8sMed3YKcSkwy8tEKLDCef6SOQ3sOyF14jJmMuDo110n sIi35B+FRjDSlaPbqtspD3Tq54v8orMGSDwqu3BhwTX++q+qskDxKzKx9DzBiocU6JNq av41yaGomd9oeEkIMc8OYpL5Ap9aZqdvchjTgFF4muAv661stEvQcno2YqpHCT1Fow63 9HOp2zOQgPsvWi7cpS1rzl3msHSc+fMVlItUfFoVQ2xEAgtOwXjKouAApQ7bTGatGrtH PVPA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAubYAF0YxEc/LhXeEb5pJviSDvXwy47+RKiZkw4VnmOpEW7F8ERv x8xUloePna1eD796rwwc4svzynmNBWXgMNmwRR3lKHb2vtmrNyC+/W47Wo58lGCLlqsjIZMmb+F aDElwrps7mf3Rz/+6s6Dw5dgnKsgM2yHfSf8rTgqoln+YoAJEOkIL2kD8oXelf+ywvA== X-Received: by 2002:a50:b1cd:: with SMTP id n13mr11738784edd.224.1550748921419; Thu, 21 Feb 2019 03:35:21 -0800 (PST) X-Google-Smtp-Source: AHgI3IaeYtmWbiY5Z5CwZi8HfputQzQO1snHvttuWPu0Z+QTUr+uchwP39gOtCfrlSn7k01StoCn X-Received: by 2002:a50:b1cd:: with SMTP id n13mr11738722edd.224.1550748920401; Thu, 21 Feb 2019 03:35:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550748920; cv=none; d=google.com; s=arc-20160816; b=KaWbUvj9kbBvjb1jWHlkC0qcXxP0XxWdKFzeZmZIwHffdC8MtFDOU/UxnM2JZ28UDb Gu+rAooAASqoHGVXizJtz31TFBvyT7QXVneNiNtQg0KPlKrNja7Om37Uvg77K73L92Qg rrTXtx/yhHoWH8ZD3TLfI2hz3XRsG7TS7begvTk1sXmYZsbUI9Tr2RiRB1WkTIdQ3Jq3 48QSyzc1wRmsPBWIKJdSs2pVmMslfBi6UpkCpwIvd6OjTXh4fj3jfeFn1BOKLAaYHyzP kM2jEB+TNR3kuXm7tnuBbCjsqFRxMpYQdiq3v2YIGkY5zgOAAVNplWrdkjILPgPSyfU7 q8rw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=MCQE4kh5SXgiU/bXz8lLVd48nfz6SGnOiJwrDNu6rZg=; b=aFcdFDIowHAAp8q6OAHfgYF9vD1vPhRD+ccMndf3LzO0bSQVEVV0uI1cdeJUBf5QMV pYa1p5JGxWoYEvZVYuA6iHyRp6S/jrRymZuVEgKo+zrstz/GXen1JNZEESpxBho6xobu wWzZI9Zwi8q7kpXyqSxdKegAt/hGN9gSRNTeibafAEuLN9v9RlD1skaQsCJVM5nogAw4 782r4Umc+IGqmyp6HwNpt0YoQLZXM4ca+L2po8wRbyAy7N9cGHFQR1kJQ2oo09PC3Srf Z5ed1PJxEt+iNR1LjPjCtSchm2LXE5QSSZBbXI6VeKrWJr+W2gVQdnoYy2soyEK+OWak xo2w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id k20si773696ede.343.2019.02.21.03.35.20 for ; Thu, 21 Feb 2019 03:35:20 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 64CE215AB; Thu, 21 Feb 2019 03:35:19 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D93133F5C1; Thu, 21 Feb 2019 03:35:15 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v2 01/13] arm64: mm: Add p?d_large() definitions Date: Thu, 21 Feb 2019 11:34:50 +0000 Message-Id: <20190221113502.54153-2-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190221113502.54153-1-steven.price@arm.com> References: <20190221113502.54153-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: James Morse Exposing the pud/pgd levels of the page tables to walk_page_range() means we may come across the exotic large mappings that come with large areas of contiguous memory (such as the kernel's linear map). Expose p?d_large() from each architecture to detect these large mappings. arm64 already has these macros defined, but with a different name. p?d_large() is used by s390, sparc and x86. Only arm/arm64 use p?d_sect(). Add a macro to allow both names. Signed-off-by: James Morse Signed-off-by: Steven Price Acked-by: Mark Rutland --- arch/arm64/include/asm/pgtable.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index de70c1eabf33..09d308921625 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -428,6 +428,7 @@ extern pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, PMD_TYPE_TABLE) #define pmd_sect(pmd) ((pmd_val(pmd) & PMD_TYPE_MASK) == \ PMD_TYPE_SECT) +#define pmd_large(x) pmd_sect(x) #if defined(CONFIG_ARM64_64K_PAGES) || CONFIG_PGTABLE_LEVELS < 3 #define pud_sect(pud) (0) @@ -435,6 +436,7 @@ extern pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, #else #define pud_sect(pud) ((pud_val(pud) & PUD_TYPE_MASK) == \ PUD_TYPE_SECT) +#define pud_large(x) pud_sect(x) #define pud_table(pud) ((pud_val(pud) & PUD_TYPE_MASK) == \ PUD_TYPE_TABLE) #endif From patchwork Thu Feb 21 11:34:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10823829 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 36F891399 for ; Thu, 21 Feb 2019 11:35:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 24B6830716 for ; Thu, 21 Feb 2019 11:35:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 183FD30736; Thu, 21 Feb 2019 11:35:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9D27430716 for ; Thu, 21 Feb 2019 11:35:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6F4398E0074; Thu, 21 Feb 2019 06:35:26 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6A1A78E0002; Thu, 21 Feb 2019 06:35:26 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 56E828E0074; Thu, 21 Feb 2019 06:35:26 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id E8B768E0002 for ; Thu, 21 Feb 2019 06:35:25 -0500 (EST) Received: by mail-ed1-f71.google.com with SMTP id v1so3443060eds.7 for ; Thu, 21 Feb 2019 03:35:25 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=mPd1aoVGFQvbKXoXf93LahtlYNZ3Q5UYsU8YHKpiyp0=; b=V17RdcwvgnCgU5AGjaQI3yiD2bgJ+jFcGgaUFGSa+svYtK8T0U+RBFFx97pxKYRiT6 iRbl+AskqApeSMmARe1lOiNGCQnRfujTg4m5R12Rv9krMivG300N7TwXIkpnPl/hiJPv 5DLXW0/qJP/2bw2o4CnjDl155ndtMnGF4bbVWONyHgVXtY8gAsrrrUU/x1cFiSZt8oNp G+TVnJ9bWDJyIDsZ5VVxVgR3H9fu1ydWIbBmKy2phKh5dxfoH7voICJOVBS/ne1JPj+K cTutnrgrpSqmg1rXQ26L/08M/MGm+Kk5taOw16cQup7HMaTvb6ojw05kuox6+/a4KnVV 928Q== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAubGluXOM+XNtP2YJQbMOZogfeejhEMF/Gt6r60KAzMaHKMUI91k TDbWHbfuaKcbhF8A0wttrv/F1mVw5VdMnRd9tUDtJEKvF8IUU6O1zKIo88QuUDz1SCU/++Ys35O 641GwDP17QtlXb92uJbjep5HGNJEbnNJpDBIHP7VkMpFYoKWYnO2jvZDhwab0gIwpkA== X-Received: by 2002:a50:b0e5:: with SMTP id j92mr29280626edd.188.1550748925419; Thu, 21 Feb 2019 03:35:25 -0800 (PST) X-Google-Smtp-Source: AHgI3IaICk3bSKyZyj71ozG2lDuZqKMBL1NKsv1OIle/XYsBu/pDSys183Ht2zOVnh6BmSD0XoOu X-Received: by 2002:a50:b0e5:: with SMTP id j92mr29280571edd.188.1550748924337; Thu, 21 Feb 2019 03:35:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550748924; cv=none; d=google.com; s=arc-20160816; b=d2hkCWyQpmW8m8yuX7aOQ6FSzpWyAQ88Tqc2V9cXrX3aeAsbhjaQaamUetv8h8rJP0 kPIfkrHdEgzB0quMF7ektbzwfJz8DFNvNqHw0K06qPj9jMn/WIz1NR51oMpCgkf0Cplo kdTZ1BX9XAOdvroule4vuvkOY1AnbRoqdF6wloP2kQLXxmvExrsEvUG+EKDXPc9jEYzl k8S9cd1CJ2ETxzUlFK1k8V8G/IECxSqxV0qBjIUXEsH+rT9EmlHVJk/CNQBmMaxaxcV+ 5kitG4M5hiWh1pQWCOu0LCj3jsEF5UoKvnLoVWpWJogU0ixNpq/vk6ntcCirQZiSf9TC 89gg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=mPd1aoVGFQvbKXoXf93LahtlYNZ3Q5UYsU8YHKpiyp0=; b=GDCV+6Q+XIiE/OZMEjoq8jCZdlT0mM/Pll+Y2eU/peEHIpejqg+gIOXncvV8V8A/j0 xsIB9XqCtaiV6rcUrJlel7BEUIUeswph4FnGeptzC4EoZx6SnDPk1pjqDbCqq5eUJNad y33eEn27MLlytp93iJ6QLWgPB+H9bMgJOCctbYzQPZSlYTQq5BaiGY5nFiS5lcZDH/r4 bfYOSMAbSz61z+B/ClN8ln15IhcuS43EAaO8ThYEa81hlDxyIQGxLhLlmfoQSqhdQAoP 44PupypLtUORBEpdPt7dADfsX+e0TUXtSFkaeZp+XmVUM2XRE2mpORTDzsjOpKOUsPAZ fTMw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id b16-v6si1782945ejp.242.2019.02.21.03.35.23 for ; Thu, 21 Feb 2019 03:35:24 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 33748165C; Thu, 21 Feb 2019 03:35:23 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A693F3F5C1; Thu, 21 Feb 2019 03:35:19 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v2 02/13] x86/mm: Add p?d_large() definitions Date: Thu, 21 Feb 2019 11:34:51 +0000 Message-Id: <20190221113502.54153-3-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190221113502.54153-1-steven.price@arm.com> References: <20190221113502.54153-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Exposing the pud/pgd levels of the page tables to walk_page_range() means we may come across the exotic large mappings that come with large areas of contiguous memory (such as the kernel's linear map). Expose p?d_large() from each architecture to detect these large mappings. x86 already has these defined as inline functions, add a macro of the same name so we don't end up with the generic version too. Signed-off-by: James Morse Signed-off-by: Steven Price --- arch/x86/include/asm/pgtable.h | 3 +++ arch/x86/mm/dump_pagetables.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 2779ace16d23..3695f6acb6af 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -234,6 +234,7 @@ static inline int pmd_large(pmd_t pte) { return pmd_flags(pte) & _PAGE_PSE; } +#define pmd_large(x) pmd_large(x) #ifdef CONFIG_TRANSPARENT_HUGEPAGE static inline int pmd_trans_huge(pmd_t pmd) @@ -873,6 +874,7 @@ static inline int pud_large(pud_t pud) return 0; } #endif /* CONFIG_PGTABLE_LEVELS > 2 */ +#define pud_large(x) pud_large(x) static inline unsigned long pud_index(unsigned long address) { @@ -1214,6 +1216,7 @@ static inline bool pgdp_maps_userspace(void *__ptr) } static inline int pgd_large(pgd_t pgd) { return 0; } +#define pgd_large(x) pgd_large(x) #ifdef CONFIG_PAGE_TABLE_ISOLATION /* diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index e3cdc85ce5b6..cf37abc0f58a 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -432,6 +432,7 @@ static void walk_pmd_level(struct seq_file *m, struct pg_state *st, pud_t addr, #else #define walk_pmd_level(m,s,a,e,p) walk_pte_level(m,s,__pmd(pud_val(a)),e,p) +#undef pud_large #define pud_large(a) pmd_large(__pmd(pud_val(a))) #define pud_none(a) pmd_none(__pmd(pud_val(a))) #endif @@ -469,6 +470,7 @@ static void walk_pud_level(struct seq_file *m, struct pg_state *st, p4d_t addr, #else #define walk_pud_level(m,s,a,e,p) walk_pmd_level(m,s,__pud(p4d_val(a)),e,p) +#undef p4d_large #define p4d_large(a) pud_large(__pud(p4d_val(a))) #define p4d_none(a) pud_none(__pud(p4d_val(a))) #endif @@ -503,6 +505,7 @@ static void walk_p4d_level(struct seq_file *m, struct pg_state *st, pgd_t addr, } } +#undef pgd_large #define pgd_large(a) (pgtable_l5_enabled() ? pgd_large(a) : p4d_large(__p4d(pgd_val(a)))) #define pgd_none(a) (pgtable_l5_enabled() ? pgd_none(a) : p4d_none(__p4d(pgd_val(a)))) From patchwork Thu Feb 21 11:34:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10823839 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 56E3A6C2 for ; Thu, 21 Feb 2019 11:35:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4585830717 for ; Thu, 21 Feb 2019 11:35:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 39CE330764; Thu, 21 Feb 2019 11:35:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 654D930717 for ; Thu, 21 Feb 2019 11:35:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 062298E0076; Thu, 21 Feb 2019 06:35:30 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 011878E0075; Thu, 21 Feb 2019 06:35:29 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E1C4A8E0076; Thu, 21 Feb 2019 06:35:29 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by kanga.kvack.org (Postfix) with ESMTP id 8542B8E0075 for ; Thu, 21 Feb 2019 06:35:29 -0500 (EST) Received: by mail-ed1-f70.google.com with SMTP id c18so11179632edt.23 for ; Thu, 21 Feb 2019 03:35:29 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=CXiwyxpsYbtktJw1JiQrFKlxs5Eu1bgqDSTeXz8tQMY=; b=URCacJ2xd9CRVm9iGtg7g9xP7TES/xD0Kr/P8Yk04gpszkJzcX1HXIQVBNNxtouAMC eTTVr2ZBf745aas44h2BIgmqAHm2SLdUPlpgKD3lnHAd88H+I2fyX465lNnQPBHjU5Su 6GcA4aFeaMQKoJxEoFQIDJk8YCeoYGjHQIXPk8HYGCu2s8bmD7eBWGzzaeSiG+WWEKzR cOq0LrcEO1f+VuuTmJdrW/tBpCQSCxBkKjEEfAri8Nt3RmEhpwWcBGL2hXVaHTDHzqwB yhSrpxQpa6DV1JJIblOT6POGbRn6Hrs02wabpYaYpFmZrS+kddA9yaSbfI1IUdOPFQYw bSIQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAuahxXkAC8FtyWtulf/6oi7XjPI0LWtQk9s8NI1oTBLOhC4/jyCs wH0f3mW/6QMME2iCb0FrJv73WvCbEDif/+mDHBGciLB8u7nmL6KSJK2aKTzlo6hB8d88XOr4CGp 3/6lWUBmomxmZem9wrIpX0BtaXl4P+hQn1aaCaRYirDkSMSmhVKFBuaxJ99c8p61WbA== X-Received: by 2002:a50:f5ea:: with SMTP id x39mr28548977edm.154.1550748929040; Thu, 21 Feb 2019 03:35:29 -0800 (PST) X-Google-Smtp-Source: AHgI3IayJYreM9rZSt5CGDg38IVAPURNqdK5Qbhu6oU9c9NX1xHVirgNlg9UectJ7pJtn30GeSoO X-Received: by 2002:a50:f5ea:: with SMTP id x39mr28548920edm.154.1550748928047; Thu, 21 Feb 2019 03:35:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550748928; cv=none; d=google.com; s=arc-20160816; b=fCpbO+GcrbiIxi5GETKl8pLJjgSz6RIwPEawtaswGgrnFlbGt9o0PfF91JMQNKvIFk JzEiHeo/dj5ZnEtNAsqj2vtXw/mIebOSzES4aT6BV+4qT3F8RHGJJLV3ssiNlCkvnT5V Xqq7e9kgNEG1pqzH5bhxtnPcG6qH7X2kM3Dzz8bFf90SJ3YLYX/RO49pQV3CTr5AqoVL LNdcJ0qIWsE5CLNjCtSlRk02TFA/D641DuCzppLZR6O51p1PNHK6LvU8Na+WYc8fz3KS 9Ov7FoivT6EbbhZWvEUsIA87/2ds3H46wBEU4ULDUeMY32YRjMfrwOhgli1FFZegip7a uRCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=CXiwyxpsYbtktJw1JiQrFKlxs5Eu1bgqDSTeXz8tQMY=; b=DRFg6lrTvEKExyyxjUdzowBuxAzgn5eu23RRKUlAGZoZ6TGfZ7PoVdGIvbdYDS3Rp6 PJq2d4XdffJHJUMMHdIo7uqswxDuKFVy4rAhjlkYoGpAAl+5HSjhBAglRNgLRZic4iZ8 P6Qb74w/bUVoTGtw3ekXsM+A5BTIG/60xGZvgb7zcgK5OtXwqi17nuiYYiWiu7KqQqUG 2MybcZnZmbQlbZgtmOTrJfLO1EKtDV9qSPlktcIsggISCHh/AZ7yL/fvTXGLxI9kYQXW SA6f3SkR2ep595A6WhFP6+ISxlqFbcNkk0h7OhGYvHOlU7GURrHbZgxapFhXIs8e032d loUQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id m27si460642eja.92.2019.02.21.03.35.27 for ; Thu, 21 Feb 2019 03:35:28 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0263315AB; Thu, 21 Feb 2019 03:35:27 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 75F723F5C1; Thu, 21 Feb 2019 03:35:23 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v2 03/13] mm: Add generic p?d_large() macros Date: Thu, 21 Feb 2019 11:34:52 +0000 Message-Id: <20190221113502.54153-4-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190221113502.54153-1-steven.price@arm.com> References: <20190221113502.54153-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: James Morse Exposing the pud/pgd levels of the page tables to walk_page_range() means we may come across the exotic large mappings that come with large areas of contiguous memory (such as the kernel's linear map). For architectures that don't provide p?d_large() macros, provided a does nothing default. Signed-off-by: James Morse Signed-off-by: Steven Price Acked-by: Mark Rutland --- include/asm-generic/pgtable.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h index 05e61e6c843f..f0de24100ac6 100644 --- a/include/asm-generic/pgtable.h +++ b/include/asm-generic/pgtable.h @@ -1186,4 +1186,23 @@ static inline bool arch_has_pfn_modify_check(void) #define mm_pmd_folded(mm) __is_defined(__PAGETABLE_PMD_FOLDED) #endif +/* + * p?d_large() - true if this entry is a final mapping to a physical address. + * This differs from p?d_huge() by the fact that they are always available (if + * the architecture supports large pages at the appropriate level) even + * if CONFIG_HUGETLB_PAGE is not defined. + */ +#ifndef pgd_large +#define pgd_large(x) 0 +#endif +#ifndef p4d_large +#define p4d_large(x) 0 +#endif +#ifndef pud_large +#define pud_large(x) 0 +#endif +#ifndef pmd_large +#define pmd_large(x) 0 +#endif + #endif /* _ASM_GENERIC_PGTABLE_H */ From patchwork Thu Feb 21 11:34:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10823835 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 759DF1399 for ; Thu, 21 Feb 2019 11:35:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 63DE130737 for ; Thu, 21 Feb 2019 11:35:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 57F933075F; Thu, 21 Feb 2019 11:35:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B159C30764 for ; Thu, 21 Feb 2019 11:35:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 128AF8E0077; Thu, 21 Feb 2019 06:35:34 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0D7BC8E0075; Thu, 21 Feb 2019 06:35:34 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6EAA8E0077; Thu, 21 Feb 2019 06:35:33 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id 87D988E0075 for ; Thu, 21 Feb 2019 06:35:33 -0500 (EST) Received: by mail-ed1-f69.google.com with SMTP id o27so944230edc.14 for ; Thu, 21 Feb 2019 03:35:33 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=HrdRugGr2uKHHo/TiR13vk3LQ/wnulW6YNAfJqdWS6g=; b=gAd2dQiK5S/Cji2eJ+YbjThNN7IlcdesHOzdeLpKS5M68v4e7c/G6XLDJNHBEF+sNb eC1rXvAwqgm2mST9b1G6RuQnkFQbwy2oLH7nGatt12ZfWlx0y4SeJYJGLtfbSiPIfjRk 5/Vd+aan7EEiVVM8SWY8jLM3cLDoCXa1SCUvkvp/cpUGHPo5icQNsyWaSCGrviXJxnJ2 wRAy/teSbOhY6/tq/mt3rzR7Muj5+atcpmO2hG+4l6mWZ27VydN+0zzYJSFsRC48LPAs ibJxovLwybuhVJRwkjsYVCotRicwPYOcWcUTvMBeWDMbKGHieAcLRtWFJU1jayX1/dBb 3d1Q== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAuYBDngdByqB+2onQmlqKsaQ2oAa3a2C9BoBVMDQmmeJqJCqQu/O 47YEEhD8uyaMeLfxReedkqtFW1wQhLeEUcizhEsczqnEdvyOik+omuHwdhyF5+7v+gExrc7WHye z7893rTxTTcX6RCYnduu6gsUG03duHWfjYvgA6XKh3EVJg8SRA/ZUBgBuVRuyG64iFg== X-Received: by 2002:a50:ae8a:: with SMTP id e10mr15746040edd.24.1550748933057; Thu, 21 Feb 2019 03:35:33 -0800 (PST) X-Google-Smtp-Source: AHgI3IaWYkrWK0l9fucY2kSlhM1zOh2Nt1wb2R4ZH+TRATnjS3Vl5h69eTGJutnJs9OePBlmZLy3 X-Received: by 2002:a50:ae8a:: with SMTP id e10mr15745978edd.24.1550748931964; Thu, 21 Feb 2019 03:35:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550748931; cv=none; d=google.com; s=arc-20160816; b=AYcfUOpZqWyMD+0gtZQTv16WTbrw723KYG2A0fxUgZV6l65fsdoyPJ6EZqCil4tbuO 4UeEKCnzbNwza1Ns5mH35vOVV+2TwArOA2SpfNcR59rDBJMqX8kv0eW0OHN0/VnCR9Zu PJdwrs414nJboUx8YO8NNAspooVLBjRDXqGN1bpAaB1buJPxdL5U/x90QbbcR5/T3oae WYf7dUyo9AWTihQWhPpXJFvmcAtZUT7UIetXvDL5WOCUwyN8jvFiDml/lqyU6ppgXvmf hiVxJKT4cv2zYPechJu3NBKCjZg1bfMyiGJYUGeO+HRR6aa3ofGLp9CVsp8Jui1mk+C6 24QQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=HrdRugGr2uKHHo/TiR13vk3LQ/wnulW6YNAfJqdWS6g=; b=mWOLyUlf+Jp26OsKIz1Fe4FRABsrA2R4urK+jXOw8oqkLDN3GP8/KkcsjUp5AAcSO5 tkG5ohE9h5X48dld0t6JCbzMuT8GJewiofAB5ywD6qkEoe9MpzxiyO2z60kJOEv18Ktu V38E+m8d7ENl4/qL3j/+HS9PzBWQ4lbACzORg7fZKV+J/ymYHxbf005ThbeMd4qJx16X 19AFg0dCQFar25R+OgLbD1OIw4ao+IymymvyUa8AfkqVoFWN64VQ9Odxo0y1NFZNdK7v hEizxau0+oLMkH6ttWx1T7++a8nN45+aTLW2Szz8OOrdQqsRmIH0xdzfHCGDCHZ9Xpg6 u53w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id i14si6372834edj.46.2019.02.21.03.35.31 for ; Thu, 21 Feb 2019 03:35:31 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C4CC780D; Thu, 21 Feb 2019 03:35:30 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 446643F5C1; Thu, 21 Feb 2019 03:35:27 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v2 04/13] mm: pagewalk: Add p4d_entry() and pgd_entry() Date: Thu, 21 Feb 2019 11:34:53 +0000 Message-Id: <20190221113502.54153-5-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190221113502.54153-1-steven.price@arm.com> References: <20190221113502.54153-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP pgd_entry() and pud_entry() were removed by commit 0b1fbfe50006c410 ("mm/pagewalk: remove pgd_entry() and pud_entry()") because there were no users. We're about to add users so reintroduce them, along with p4d_entry() as we now have 5 levels of tables. Note that commit a00cc7d9dd93d66a ("mm, x86: add support for PUD-sized transparent hugepages") already re-added pud_entry() but with different semantics to the other callbacks. Since there have never been upstream users of this, revert the semantics back to match the other callbacks. This means pud_entry() is called for all entries, not just transparent huge pages. Signed-off-by: Steven Price --- include/linux/mm.h | 9 ++++++--- mm/pagewalk.c | 27 ++++++++++++++++----------- 2 files changed, 22 insertions(+), 14 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 80bb6408fe73..1a4b1615d012 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1412,10 +1412,9 @@ void unmap_vmas(struct mmu_gather *tlb, struct vm_area_struct *start_vma, /** * mm_walk - callbacks for walk_page_range + * @pgd_entry: if set, called for each non-empty PGD (top-level) entry + * @p4d_entry: if set, called for each non-empty P4D (1st-level) entry * @pud_entry: if set, called for each non-empty PUD (2nd-level) entry - * this handler should only handle pud_trans_huge() puds. - * the pmd_entry or pte_entry callbacks will be used for - * regular PUDs. * @pmd_entry: if set, called for each non-empty PMD (3rd-level) entry * this handler is required to be able to handle * pmd_trans_huge() pmds. They may simply choose to @@ -1435,6 +1434,10 @@ void unmap_vmas(struct mmu_gather *tlb, struct vm_area_struct *start_vma, * (see the comment on walk_page_range() for more details) */ struct mm_walk { + int (*pgd_entry)(pgd_t *pgd, unsigned long addr, + unsigned long next, struct mm_walk *walk); + int (*p4d_entry)(p4d_t *p4d, unsigned long addr, + unsigned long next, struct mm_walk *walk); int (*pud_entry)(pud_t *pud, unsigned long addr, unsigned long next, struct mm_walk *walk); int (*pmd_entry)(pmd_t *pmd, unsigned long addr, diff --git a/mm/pagewalk.c b/mm/pagewalk.c index c3084ff2569d..98373a9f88b8 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -90,15 +90,9 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, } if (walk->pud_entry) { - spinlock_t *ptl = pud_trans_huge_lock(pud, walk->vma); - - if (ptl) { - err = walk->pud_entry(pud, addr, next, walk); - spin_unlock(ptl); - if (err) - break; - continue; - } + err = walk->pud_entry(pud, addr, next, walk); + if (err) + break; } split_huge_pud(walk->vma, pud, addr); @@ -131,7 +125,12 @@ static int walk_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, break; continue; } - if (walk->pmd_entry || walk->pte_entry) + if (walk->p4d_entry) { + err = walk->p4d_entry(p4d, addr, next, walk); + if (err) + break; + } + if (walk->pud_entry || walk->pmd_entry || walk->pte_entry) err = walk_pud_range(p4d, addr, next, walk); if (err) break; @@ -157,7 +156,13 @@ static int walk_pgd_range(unsigned long addr, unsigned long end, break; continue; } - if (walk->pmd_entry || walk->pte_entry) + if (walk->pgd_entry) { + err = walk->pgd_entry(pgd, addr, next, walk); + if (err) + break; + } + if (walk->p4d_entry || walk->pud_entry || walk->pmd_entry || + walk->pte_entry) err = walk_p4d_range(pgd, addr, next, walk); if (err) break; From patchwork Thu Feb 21 11:34:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10823831 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2DEF11399 for ; Thu, 21 Feb 2019 11:35:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1589330768 for ; Thu, 21 Feb 2019 11:35:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 08C0E30821; Thu, 21 Feb 2019 11:35:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7C14930768 for ; Thu, 21 Feb 2019 11:35:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 71AD88E0078; Thu, 21 Feb 2019 06:35:37 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6EFCD8E0075; Thu, 21 Feb 2019 06:35:37 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 593098E0078; Thu, 21 Feb 2019 06:35:37 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id EE72F8E0075 for ; Thu, 21 Feb 2019 06:35:36 -0500 (EST) Received: by mail-ed1-f71.google.com with SMTP id 29so1903280eds.12 for ; Thu, 21 Feb 2019 03:35:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=NXbLxkVEjkUC98nDqbwMu9XwBnfr8Fy+W9xUz87hsi8=; b=dhAridJa5RC9XXe4KwJw8YVGyjtEqlig6nrFwcv+O6Hyhz/v5rvJuwLAdiM7qYV+3T 6jX5IBAwsUDtGwMDOQ9MDpOjVKt6qThQakuQoGhsZLbKzlHYTmVBvtSb3d6Uzz7eDnNI 1chwQYvNHebj7Fr1ukCLRSkPD/YTPB38q0KGQztGCPuMu4OUrVB6lXDQZ/8VWXCFWF/Q 0rHNiXozIZ9xEl2nlEyBGlf89NmUmLVh62TnrgRHrlDOPva9ZUHBq6HkaZ1P5MvkG89X iwY6EgoY/jPcDhIeIjQBjdJcownEIYmE2MhpYDGArJ+Em/XMdWTayfeoGzaTxqlN/+0K bmdA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAub+65qiLD9kj45Lx8u/8mehQoCgtzdPmeDcirZxoLi2oXR64tEA cx1Srap9D/pHVt/zAemHTWuX+q0f6fI4zKqUmL+2L9ydPrBTg42lBOlhrqAXYdTNv+bvgXnHgTf kSwmbsMI5Yh05JaS7541t3A+jsoKjxSMxiLTDfgpSODQaBb5tazxp0dss4z7tfC8eKA== X-Received: by 2002:a17:906:4f15:: with SMTP id t21mr7075387eju.179.1550748936475; Thu, 21 Feb 2019 03:35:36 -0800 (PST) X-Google-Smtp-Source: AHgI3Iba6dIKtD7muF5BLXUJvdeWUmyzHHs+agSy7RUrZwZKk1tg9x42dYXYMkKVOJ9EHIuIVaV0 X-Received: by 2002:a17:906:4f15:: with SMTP id t21mr7075342eju.179.1550748935515; Thu, 21 Feb 2019 03:35:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550748935; cv=none; d=google.com; s=arc-20160816; b=nZkM1WlB+L2bFtWj1xVPOGHof70PWX5YAnfT63HwU8Fu/Ilbykl8ktyznDYFeRf6Sx rOtkybyl8chftHPaOs+jvJ0i8ab3vlCHx2Z6aAvZjMsWCgHm0richllZNvPZjq4zkIO5 8gLNEIZAcfh8d39s2YNTewePtrRznMocAtXV/ND56dn/8DN6E5k2Zmquzj5lPZkhNK8S KLbeR8y2qFjuFaK0N4H9Hglkpxo+oxY5h6jbv7XLr1OCG839cEaMJ+CliALTN1EuCzCg wgc4f/UfiIN7KCHYKuojYPEEMSlBBenrQi4t51/3VpskTUdVu8tyjgvMWrpnyQqovhJE 2F+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=NXbLxkVEjkUC98nDqbwMu9XwBnfr8Fy+W9xUz87hsi8=; b=Zf+mmTTJQ1qTV6t8WxwfYbw0EhZNc3zuH3ukhc9yIUq2NJFge0Cq9acpqyhNTRo3FE EghxY+LijaDdT8uttRvoHSz/kvmcmWCN/YrcoEhApylrIR+jt7R3UkmhO/SwS1/KDY14 sk1nBcAoF8Qqg4LkSdLq0Zb79ppQnrU3GBvZ1TkWjwSKfh7CUZwlsZwFiUED1TtC8AEn egL6o4a0qy+eRyXznsOLVp7jdM1KJfXsm2Juc3Kpp6UFIeBjuS3ZRNir7DJ1PWuPtoMJ qcdjbCXYbWo4agnI5e/CBDlDbaQfMKd5VciEffxDU+KYiF4CQvufY9TXsPs2HFoPd5b9 PiDg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id o21si80779edc.54.2019.02.21.03.35.35 for ; Thu, 21 Feb 2019 03:35:35 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 93A1C1682; Thu, 21 Feb 2019 03:35:34 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 131053F5C1; Thu, 21 Feb 2019 03:35:30 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v2 05/13] mm: pagewalk: Allow walking without vma Date: Thu, 21 Feb 2019 11:34:54 +0000 Message-Id: <20190221113502.54153-6-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190221113502.54153-1-steven.price@arm.com> References: <20190221113502.54153-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Since 48684a65b4e3: "mm: pagewalk: fix misbehavior of walk_page_range for vma(VM_PFNMAP)", page_table_walk() will report any kernel area as a hole, because it lacks a vma. This means each arch has re-implemented page table walking when needed, for example in the per-arch ptdump walker. Remove the requirement to have a vma except when trying to split huge pages. Signed-off-by: Steven Price --- mm/pagewalk.c | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/mm/pagewalk.c b/mm/pagewalk.c index 98373a9f88b8..dac0c848b458 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -36,7 +36,7 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, do { again: next = pmd_addr_end(addr, end); - if (pmd_none(*pmd) || !walk->vma) { + if (pmd_none(*pmd)) { if (walk->pte_hole) err = walk->pte_hole(addr, next, walk); if (err) @@ -59,9 +59,14 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, if (!walk->pte_entry) continue; - split_huge_pmd(walk->vma, pmd, addr); - if (pmd_trans_unstable(pmd)) - goto again; + if (walk->vma) { + split_huge_pmd(walk->vma, pmd, addr); + if (pmd_trans_unstable(pmd)) + goto again; + } else if (pmd_large(*pmd)) { + continue; + } + err = walk_pte_range(pmd, addr, next, walk); if (err) break; @@ -81,7 +86,7 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, do { again: next = pud_addr_end(addr, end); - if (pud_none(*pud) || !walk->vma) { + if (pud_none(*pud)) { if (walk->pte_hole) err = walk->pte_hole(addr, next, walk); if (err) @@ -95,9 +100,13 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, break; } - split_huge_pud(walk->vma, pud, addr); - if (pud_none(*pud)) - goto again; + if (walk->vma) { + split_huge_pud(walk->vma, pud, addr); + if (pud_none(*pud)) + goto again; + } else if (pud_large(*pud)) { + continue; + } if (walk->pmd_entry || walk->pte_entry) err = walk_pmd_range(pud, addr, next, walk); From patchwork Thu Feb 21 11:34:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10823841 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9DE89184E for ; Thu, 21 Feb 2019 11:35:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8BE7F3062D for ; Thu, 21 Feb 2019 11:35:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7FCB630736; Thu, 21 Feb 2019 11:35:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BB3313062D for ; Thu, 21 Feb 2019 11:35:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BFB658E0079; Thu, 21 Feb 2019 06:35:41 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BAB3B8E0075; Thu, 21 Feb 2019 06:35:41 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A9A028E0079; Thu, 21 Feb 2019 06:35:41 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id 4AA4B8E0075 for ; Thu, 21 Feb 2019 06:35:41 -0500 (EST) Received: by mail-ed1-f71.google.com with SMTP id 29so1903380eds.12 for ; Thu, 21 Feb 2019 03:35:41 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=zJK9tQmzH2wt8ZgtuBGoRD5SE3Dtp+m3gB1hPWai3es=; b=Y53z4OTMxQFd/JIUOL8FgtvCmRwdN0Mjz/PIUy7XTkVM202K2SoDiefzF2vMxMdDKQ QK9mNx2JSvfWVs2KP/ndQ4UKkTLuqSaZwKjI9Jdaup720VlNDkeF3y6Lxgsv3z5Q9SPq OI8X8cTBiIUM+U+qA88qQsQ5NkB+xeY0WIC6z3KlHzN6Ra4omNl84sRvjZsxDmFzQYhl SUGd4K8PSqB+tr84uxpIPIxExBzJYfTjW2WLBV47uqWLk63WSYcfbtfpf99J3qk0xIUh 2N2UeC/cR09SZ/V8uDA8qwkPFa4/iag+hrhyfPWyInBfBEe7LMMrPUs/B5S8+4Foupm9 yD1Q== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAuad+fuCSZmtzQXNQFP1XR+Q8ljKuR/C+J7aHq71N9KJx6/HOOyX qdXpTZE0tIXNxEYz101ZAd1zh/e+AiDukd+NEhOBlsGVZoMqIpdtkJQK1iCYLv0eL/LLlaww0Zx EsD5Er44zCrCaAyeqiL7V63aNCwvbZr6SgiRlMCNSRtTROb+2aPhk9VGunp/n/1ZevQ== X-Received: by 2002:a50:d55a:: with SMTP id f26mr15214039edj.292.1550748940781; Thu, 21 Feb 2019 03:35:40 -0800 (PST) X-Google-Smtp-Source: AHgI3IZPdb1esJA2PJYlCysk85ZwkWrApAk3SVY9cu0iaQ753RsGACzwCHSBSogUIA2x450PzxIg X-Received: by 2002:a50:d55a:: with SMTP id f26mr15213956edj.292.1550748939439; Thu, 21 Feb 2019 03:35:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550748939; cv=none; d=google.com; s=arc-20160816; b=K0zfiSNTA2mXNf4y1vsb3Dm+2px/C0wXKHBQq21chDV2ZTMrwNgRlDk4R/x1id8vIN fXQwvw2CQ0YBZcKjmPhqT66BHE03+rEF06zCDTjvEYQel23Um1ezdn1MKvwiShYlPZsd FSvf6PIa2cnUN9Gf/nH66kUw+ib/EDlDdMihOAl6TZf43OftyDoxAZ6VmH2O5g6thH5Q 5TBZb0iAct8RIl6xSVLkiF2FkIWFzgnHKUvEeZ8ZenSZCLxOUW5mktr+csP6sOujA4VE jB+rfSQ0ikrjmpr7oMONySAolkWNqJIijbmo6gaV6i91mZGGeNj/7Q3bX4SYMcmdBKMb Tdkw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=zJK9tQmzH2wt8ZgtuBGoRD5SE3Dtp+m3gB1hPWai3es=; b=WgiCoCjYBMlkXmUwU+pLgB8MAXe09E4qTBoxcJmMERCl3eeXNxEK+zwqufXP4j/U/K YstuVYWrMkTGGhe2YffBasRVZwfHmc4OZ8ZY6/kuVxnIbhIbu/sv5Mt4dcDdvB5IT5cX HhHMoEEcSSzBqCB4qTB4VhoeU/p8W85sBIVVj6xz0bloRWhuJ/fUjNAT1aLY2rYto7I1 F3d4ux2I/afoOCwHztvvfXD96RBzpZNB6rDsf8g943iezbN013uEl6EplWZv9whyCaE0 jwvtOF8AN4XTeKm+ZBRCpTC65e3tNp9wHhmZIOuoAKMhvDuJ7XrsbgnTFum8eiZOPv0i +mcw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id z48si40329edb.159.2019.02.21.03.35.39 for ; Thu, 21 Feb 2019 03:35:39 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6450D165C; Thu, 21 Feb 2019 03:35:38 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D5F223F5C1; Thu, 21 Feb 2019 03:35:34 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v2 06/13] mm: pagewalk: Add 'depth' parameter to pte_hole Date: Thu, 21 Feb 2019 11:34:55 +0000 Message-Id: <20190221113502.54153-7-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190221113502.54153-1-steven.price@arm.com> References: <20190221113502.54153-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The pte_hole() callback is called at multiple levels of the page tables. Code dumping the kernel page tables needs to know what at what depth the missing entry is. Add this is an extra parameter to pte_hole(). When the depth isn't know (e.g. processing a vma) then -1 is passed. The depth that is reported is the actual level where the entry is missing (ignoring any folding that is in place), i.e. any levels where PTRS_PER_P?D is set to 1 are ignored. Note that depth starts at 0 for a PGD so that PUD/PMD/PTE retain their natural numbers as levels 2/3/4. Signed-off-by: Steven Price --- fs/proc/task_mmu.c | 4 ++-- include/linux/mm.h | 6 ++++-- mm/hmm.c | 2 +- mm/migrate.c | 1 + mm/mincore.c | 1 + mm/pagewalk.c | 31 +++++++++++++++++++++++++------ 6 files changed, 34 insertions(+), 11 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index f0ec9edab2f3..91131cd4e9e0 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -474,7 +474,7 @@ static void smaps_account(struct mem_size_stats *mss, struct page *page, #ifdef CONFIG_SHMEM static int smaps_pte_hole(unsigned long addr, unsigned long end, - struct mm_walk *walk) + __always_unused int depth, struct mm_walk *walk) { struct mem_size_stats *mss = walk->private; @@ -1203,7 +1203,7 @@ static int add_to_pagemap(unsigned long addr, pagemap_entry_t *pme, } static int pagemap_pte_hole(unsigned long start, unsigned long end, - struct mm_walk *walk) + __always_unused int depth, struct mm_walk *walk) { struct pagemapread *pm = walk->private; unsigned long addr = start; diff --git a/include/linux/mm.h b/include/linux/mm.h index 1a4b1615d012..4ae3634a9118 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1420,7 +1420,9 @@ void unmap_vmas(struct mmu_gather *tlb, struct vm_area_struct *start_vma, * pmd_trans_huge() pmds. They may simply choose to * split_huge_page() instead of handling it explicitly. * @pte_entry: if set, called for each non-empty PTE (4th-level) entry - * @pte_hole: if set, called for each hole at all levels + * @pte_hole: if set, called for each hole at all levels, + * depth is -1 if not known, 0:PGD, 1:P4D, 2:PUD, 3:PMD, 4:PTE + * any depths where PTRS_PER_P?D is equal to 1 are skipped * @hugetlb_entry: if set, called for each hugetlb entry * @test_walk: caller specific callback function to determine whether * we walk over the current vma or not. Returning 0 @@ -1445,7 +1447,7 @@ struct mm_walk { int (*pte_entry)(pte_t *pte, unsigned long addr, unsigned long next, struct mm_walk *walk); int (*pte_hole)(unsigned long addr, unsigned long next, - struct mm_walk *walk); + int depth, struct mm_walk *walk); int (*hugetlb_entry)(pte_t *pte, unsigned long hmask, unsigned long addr, unsigned long next, struct mm_walk *walk); diff --git a/mm/hmm.c b/mm/hmm.c index a04e4b810610..e3e6b8fda437 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -440,7 +440,7 @@ static void hmm_range_need_fault(const struct hmm_vma_walk *hmm_vma_walk, } static int hmm_vma_walk_hole(unsigned long addr, unsigned long end, - struct mm_walk *walk) + __always_unused int depth, struct mm_walk *walk) { struct hmm_vma_walk *hmm_vma_walk = walk->private; struct hmm_range *range = hmm_vma_walk->range; diff --git a/mm/migrate.c b/mm/migrate.c index d4fd680be3b0..8b62a9fecb5c 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2121,6 +2121,7 @@ struct migrate_vma { static int migrate_vma_collect_hole(unsigned long start, unsigned long end, + __always_unused int depth, struct mm_walk *walk) { struct migrate_vma *migrate = walk->private; diff --git a/mm/mincore.c b/mm/mincore.c index 218099b5ed31..c4edbc688241 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -104,6 +104,7 @@ static int __mincore_unmapped_range(unsigned long addr, unsigned long end, } static int mincore_unmapped_range(unsigned long addr, unsigned long end, + __always_unused int depth, struct mm_walk *walk) { walk->private += __mincore_unmapped_range(addr, end, diff --git a/mm/pagewalk.c b/mm/pagewalk.c index dac0c848b458..57946bcd810c 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -4,6 +4,22 @@ #include #include +/* + * We want to know the real level where a entry is located ignoring any + * folding of levels which may be happening. For example if p4d is folded then + * a missing entry found at level 1 (p4d) is actually at level 0 (pgd). + */ +static int real_depth(int depth) +{ + if (depth == 3 && PTRS_PER_PMD == 1) + depth = 2; + if (depth == 2 && PTRS_PER_PUD == 1) + depth = 1; + if (depth == 1 && PTRS_PER_P4D == 1) + depth = 0; + return depth; +} + static int walk_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, struct mm_walk *walk) { @@ -31,6 +47,7 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, pmd_t *pmd; unsigned long next; int err = 0; + int depth = real_depth(3); pmd = pmd_offset(pud, addr); do { @@ -38,7 +55,7 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, next = pmd_addr_end(addr, end); if (pmd_none(*pmd)) { if (walk->pte_hole) - err = walk->pte_hole(addr, next, walk); + err = walk->pte_hole(addr, next, depth, walk); if (err) break; continue; @@ -81,6 +98,7 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, pud_t *pud; unsigned long next; int err = 0; + int depth = real_depth(2); pud = pud_offset(p4d, addr); do { @@ -88,7 +106,7 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, next = pud_addr_end(addr, end); if (pud_none(*pud)) { if (walk->pte_hole) - err = walk->pte_hole(addr, next, walk); + err = walk->pte_hole(addr, next, depth, walk); if (err) break; continue; @@ -123,13 +141,14 @@ static int walk_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, p4d_t *p4d; unsigned long next; int err = 0; + int depth = real_depth(1); p4d = p4d_offset(pgd, addr); do { next = p4d_addr_end(addr, end); if (p4d_none_or_clear_bad(p4d)) { if (walk->pte_hole) - err = walk->pte_hole(addr, next, walk); + err = walk->pte_hole(addr, next, depth, walk); if (err) break; continue; @@ -160,7 +179,7 @@ static int walk_pgd_range(unsigned long addr, unsigned long end, next = pgd_addr_end(addr, end); if (pgd_none_or_clear_bad(pgd)) { if (walk->pte_hole) - err = walk->pte_hole(addr, next, walk); + err = walk->pte_hole(addr, next, 0, walk); if (err) break; continue; @@ -206,7 +225,7 @@ static int walk_hugetlb_range(unsigned long addr, unsigned long end, if (pte) err = walk->hugetlb_entry(pte, hmask, addr, next, walk); else if (walk->pte_hole) - err = walk->pte_hole(addr, next, walk); + err = walk->pte_hole(addr, next, -1, walk); if (err) break; @@ -249,7 +268,7 @@ static int walk_page_test(unsigned long start, unsigned long end, if (vma->vm_flags & VM_PFNMAP) { int err = 1; if (walk->pte_hole) - err = walk->pte_hole(start, end, walk); + err = walk->pte_hole(start, end, -1, walk); return err ? err : 1; } return 0; From patchwork Thu Feb 21 11:34:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10823847 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5CD6E1399 for ; Thu, 21 Feb 2019 11:35:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4BB843062D for ; Thu, 21 Feb 2019 11:35:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3FFFC30717; Thu, 21 Feb 2019 11:35:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C1DA030737 for ; Thu, 21 Feb 2019 11:35:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7B89D8E007A; Thu, 21 Feb 2019 06:35:45 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 741258E0075; Thu, 21 Feb 2019 06:35:45 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E1D78E007A; Thu, 21 Feb 2019 06:35:45 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by kanga.kvack.org (Postfix) with ESMTP id 015BB8E0075 for ; Thu, 21 Feb 2019 06:35:45 -0500 (EST) Received: by mail-ed1-f70.google.com with SMTP id o9so4885835edh.10 for ; Thu, 21 Feb 2019 03:35:44 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=5+qDYU4THjLPvNOaZc1aA0s7EjXEEdRbPiGSrygYMH0=; b=UXqGMbrF7Coz7sqKrhD7lTYd/onQmkBiuEKTA7c5ykP/4UWNOeCbFs4SdwDFzIDoYk vSVjyAR8tJZaXEVuulWgzYNxeXPU9PAgygTbHSluJNPe6epAKBtZikSp1eouny/3s+bY ymUQwQLrAJnM+sewx1ATL1qo48SyCq65UbuhS6T44FX0z3GDmsjddFy0huwUQRfl72Fs xnL2ekAxt71dr+h0Lbs9arNRqI2aN1uRZaXv0OXR3VQKFdkn1bljuwmr/8KddOUZLr6a 5UqxIL27QlcvOG6DG+Bzi9cbIQagXFDBXqEFBIPr5bJVFxxuQo+qbMN9F/9+qqUFCqWE +Ckw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAuanxGgVKOPyD7lZQUefWntCyhgDJKcoQhN7ifng/zqXa2NX4IFl g/LNbobJr5iefX9wX1MI+TNPfKNUwiNMAnzx6/9UpO/gwVxeSqt/JKqNJYqt6plnWAqCL7o6Xmn y6wjNxZdegU8zHFozoDMggBzF2HWrVN/oyxgMOPKugOcyk2S3JfZjB2njB3oLGLdmig== X-Received: by 2002:a17:906:6c8f:: with SMTP id s15mr17791236ejr.133.1550748944476; Thu, 21 Feb 2019 03:35:44 -0800 (PST) X-Google-Smtp-Source: AHgI3IZYmsSNPnMTVOalFQ0KHq9rYn77ezujSl2cyMVA0aeJZAdU+xff1luqI7iFqBhnh9OLSEKW X-Received: by 2002:a17:906:6c8f:: with SMTP id s15mr17791187ejr.133.1550748943376; Thu, 21 Feb 2019 03:35:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550748943; cv=none; d=google.com; s=arc-20160816; b=jLVoYWmugEfEBDWvYz7+u1rTNSd2lfz7mmYwCXVqS9q57puVRILhfA5r56GpjxPRhu xc2YvizI96xAM64d2VWMk7Orym8SrpAe6St1xZSTeP25YshIwFKhgB/Jey8CAhwFpjtE ITL56M+UuswiIBRuGFEtPxnYagTPFGWppyxNRzkgFMSCgswF28ld7NMzbj/DsHhF+bRj Q1IUo6lyweGg5LJAPUijzmo4AzovWDm6YzZetbwIMHH79TsPSSyjNHJSgSlrw/0AI7rs nPRZrv32thVl26vGMnmwJ84Ysg3rmJ94/5Fmd421fN9AJFRWS4Nm9XKJjBbSMIW1J23u aSdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=5+qDYU4THjLPvNOaZc1aA0s7EjXEEdRbPiGSrygYMH0=; b=lHoetEquyiMFV1gJKNmt+XDQjAsnuTX99uYOb3/xOZszRgo5zdTA5Tr6/6YvDdDQVx SsnhUJvgn30ZCqpohQF/bE6hA+NeTfQMfF3xDggo1vMwMK8N3/W42Jbj2kW9OpwwStGt /BIgSukg0mDXdAxbHsG5bwTYvszGnDV4LBbKVYwHej7fa2a71R12S9FH6XJl5zdBhEq4 R76cO1F8N/iIRG9gW2m+ZeUsUYBct5o0NpOunpew28LU1V0xgADd/d1f2bYwjP0kkFMS +04x/ma+JG2H41Fgy6kGZ3EqCi11yKAkHEbNhXnyhr4lR+Uz5zte1jtP6xRPCMAXs//4 uuCw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id y26si3082726edc.439.2019.02.21.03.35.42 for ; Thu, 21 Feb 2019 03:35:43 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 328B780D; Thu, 21 Feb 2019 03:35:42 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A5CB33F5C1; Thu, 21 Feb 2019 03:35:38 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v2 07/13] mm: pagewalk: Add test_p?d callbacks Date: Thu, 21 Feb 2019 11:34:56 +0000 Message-Id: <20190221113502.54153-8-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190221113502.54153-1-steven.price@arm.com> References: <20190221113502.54153-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP It is useful to be able to skip parts of the page table tree even when walking without VMAs. Add test_p?d callbacks similar to test_walk but which are called just before a table at that level is walked. If the callback returns non-zero then the entire table is skipped. Signed-off-by: Steven Price --- include/linux/mm.h | 11 +++++++++++ mm/pagewalk.c | 24 ++++++++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 4ae3634a9118..581f31c6b6d9 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1429,6 +1429,11 @@ void unmap_vmas(struct mmu_gather *tlb, struct vm_area_struct *start_vma, * value means "do page table walk over the current vma," * and a negative one means "abort current page table walk * right now." 1 means "skip the current vma." + * @test_pmd: similar to test_walk(), but called for every pmd. + * @test_pud: similar to test_walk(), but called for every pud. + * @test_p4d: similar to test_walk(), but called for every p4d. + * Returning 0 means walk this part of the page tables, + * returning 1 means to skip this range. * @mm: mm_struct representing the target process of page table walk * @vma: vma currently walked (NULL if walking outside vmas) * @private: private data for callbacks' usage @@ -1453,6 +1458,12 @@ struct mm_walk { struct mm_walk *walk); int (*test_walk)(unsigned long addr, unsigned long next, struct mm_walk *walk); + int (*test_pmd)(unsigned long addr, unsigned long next, + pmd_t *pmd_start, struct mm_walk *walk); + int (*test_pud)(unsigned long addr, unsigned long next, + pud_t *pud_start, struct mm_walk *walk); + int (*test_p4d)(unsigned long addr, unsigned long next, + p4d_t *p4d_start, struct mm_walk *walk); struct mm_struct *mm; struct vm_area_struct *vma; void *private; diff --git a/mm/pagewalk.c b/mm/pagewalk.c index 57946bcd810c..ff2fc8490435 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -49,6 +49,14 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, int err = 0; int depth = real_depth(3); + if (walk->test_pmd) { + err = walk->test_pmd(addr, end, pmd_offset(pud, 0), walk); + if (err < 0) + return err; + if (err > 0) + return 0; + } + pmd = pmd_offset(pud, addr); do { again: @@ -100,6 +108,14 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, int err = 0; int depth = real_depth(2); + if (walk->test_pud) { + err = walk->test_pud(addr, end, pud_offset(p4d, 0), walk); + if (err < 0) + return err; + if (err > 0) + return 0; + } + pud = pud_offset(p4d, addr); do { again: @@ -143,6 +159,14 @@ static int walk_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, int err = 0; int depth = real_depth(1); + if (walk->test_p4d) { + err = walk->test_p4d(addr, end, p4d_offset(pgd, 0), walk); + if (err < 0) + return err; + if (err > 0) + return 0; + } + p4d = p4d_offset(pgd, addr); do { next = p4d_addr_end(addr, end); From patchwork Thu Feb 21 11:34:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10823853 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 604896C2 for ; Thu, 21 Feb 2019 11:35:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4ED173062D for ; Thu, 21 Feb 2019 11:35:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4314930736; Thu, 21 Feb 2019 11:35:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AF2B03062D for ; Thu, 21 Feb 2019 11:35:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5985A8E007B; Thu, 21 Feb 2019 06:35:49 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 51F2E8E0075; Thu, 21 Feb 2019 06:35:49 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3BED78E007B; Thu, 21 Feb 2019 06:35:49 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id D1CB38E0075 for ; Thu, 21 Feb 2019 06:35:48 -0500 (EST) Received: by mail-ed1-f72.google.com with SMTP id u12so3238414edo.5 for ; Thu, 21 Feb 2019 03:35:48 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+F2lxaeFDvQ8eJHjFbD9vB/FESYGJ6cBcX+b4/bW6NY=; b=cThZjvjSQDAj3t5CRLMoD5mEOSJsaQzKjtGKKtfN83++Ggsn1NG83SLw3emlSk+7kK zcamY6rz08HiMrSPJ4IGMIgoOLJpSOfOtXf+8UwIH7T5LbsiM4oje1AA4yxB9Rw7C/+f 6Mm8DfS9jzDR9EHzCukLDztIvcE0tFKJxdYAte7xeRRhDT9Jx0Y5X7xUdMoceOcwlWbs LOHfYcO6P4oH1oq+T8NMVDtRR0JUvUenCg9/JxeBnCO4TXh7wSqdM2+mb3tLWXN1fIpS h0Vhk+5lt2Y490hWW/oUXvo4QklQgTBj80FrjtnxuBViHWxkMcCd5UKZjjc4Xu18+j6f k92A== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAuYRwx5ZM8GV8OT6GkBm/TwPsL6Q1Ut3hsqWqxMttJ/ABCwRt3J/ 1CXbKAaPzr0csDaRo1npIgtY7MVMu/n85tlQtVf8WMO+30SLCrBw9aw9u+6bmYkaiWeFR2ru90C ut73zpL2BXW2w6Np3BzUz5QOVTR/FBvoqH2MteioxtQIyO2Ug9aQBwoVlXGgp8pqU1A== X-Received: by 2002:a50:90e8:: with SMTP id d37mr29250880eda.252.1550748948353; Thu, 21 Feb 2019 03:35:48 -0800 (PST) X-Google-Smtp-Source: AHgI3IaCnRAhqsw6nHjZFpbrN8nxxlRM4Ele/Qk8g1e9RGLVYmOw30Vyoj/mSTjNgIR6S6/lx3cA X-Received: by 2002:a50:90e8:: with SMTP id d37mr29250816eda.252.1550748947212; Thu, 21 Feb 2019 03:35:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550748947; cv=none; d=google.com; s=arc-20160816; b=oA8o+wlmHTlwngAXXj6Vfl1mctApSMXQJr68N01xAuEoSuMM59+3QcKXeyvKJno/El PtPSdWGvqfyblGAQL+TAiH/seOwx12jTyeTNIWLilHM2MgxGWqvVJdbitZKuTWduFfDK BHGZHFEaH4nNpG+Rx6eAsTBXBMk/hp67gyKqm8P6XmbGGeVAytKtYYmc1MmMIXWU9HZx qUhpak8TD8NSV4II84r8YjlUC18yUgVLGVUHk9GgXkVn83zZeNI4LJAhJqtlJsENs06n GYTYKFv1ightOViSJVI+jnSQSO0lNUeBqXuN4no4YaHBI/y27mGk9nnau0vgcpwCVBbz 7EeA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=+F2lxaeFDvQ8eJHjFbD9vB/FESYGJ6cBcX+b4/bW6NY=; b=MY7hheHYiZB4kxO8q7EUb3x46QV64iAEEBIBiwxZZ3rFs0K59Vzkz8dz+mnuI9YcvE 791e8ukGcuYkrztMsGGuSjbVvRIHr50oUHCHZwbLjgTnWVQ9wi3OtyXe2GR6aAywFxlK azx/LnoHv08vWLNd6C3k5HyDSLuPICDV4Se/16hewnJh+UeTFP2x0J2soxWyesn2hVdR 4dzhp40Xp86CK7xeQbV0weUaBNwYi/0iTkfhkI/SloNHgCpuz2Nc4tlvv0kvnhgc8+qi 737jwjsIBaRENmKyvmnZQnwjnZVC4vMGEuCA9du0/I/ttUi4CYsbTwKuE3JKyFe+L50Y 5bOw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id i12si8730832ede.179.2019.02.21.03.35.46 for ; Thu, 21 Feb 2019 03:35:47 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 027DD1688; Thu, 21 Feb 2019 03:35:46 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 750E03F5C1; Thu, 21 Feb 2019 03:35:42 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v2 08/13] arm64: mm: Convert mm/dump.c to use walk_page_range() Date: Thu, 21 Feb 2019 11:34:57 +0000 Message-Id: <20190221113502.54153-9-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190221113502.54153-1-steven.price@arm.com> References: <20190221113502.54153-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Now walk_page_range() can walk kernel page tables, we can switch the arm64 ptdump code over to using it, simplifying the code. Signed-off-by: Steven Price --- arch/arm64/mm/dump.c | 108 +++++++++++++++++++++---------------------- 1 file changed, 53 insertions(+), 55 deletions(-) diff --git a/arch/arm64/mm/dump.c b/arch/arm64/mm/dump.c index 99bb8facb5cb..ee0bc1441dd0 100644 --- a/arch/arm64/mm/dump.c +++ b/arch/arm64/mm/dump.c @@ -286,73 +286,71 @@ static void note_page(struct pg_state *st, unsigned long addr, unsigned level, } -static void walk_pte(struct pg_state *st, pmd_t *pmdp, unsigned long start, - unsigned long end) +static int pud_entry(pud_t *pud, unsigned long addr, + unsigned long next, struct mm_walk *walk) { - unsigned long addr = start; - pte_t *ptep = pte_offset_kernel(pmdp, start); + struct pg_state *st = walk->private; + pud_t val = READ_ONCE(*pud); - do { - note_page(st, addr, 4, READ_ONCE(pte_val(*ptep))); - } while (ptep++, addr += PAGE_SIZE, addr != end); + if (pud_table(val)) + return 0; + + note_page(st, addr, 2, pud_val(val)); + + return 0; } -static void walk_pmd(struct pg_state *st, pud_t *pudp, unsigned long start, - unsigned long end) +static int pmd_entry(pmd_t *pmd, unsigned long addr, + unsigned long next, struct mm_walk *walk) { - unsigned long next, addr = start; - pmd_t *pmdp = pmd_offset(pudp, start); - - do { - pmd_t pmd = READ_ONCE(*pmdp); - next = pmd_addr_end(addr, end); - - if (pmd_none(pmd) || pmd_sect(pmd)) { - note_page(st, addr, 3, pmd_val(pmd)); - } else { - BUG_ON(pmd_bad(pmd)); - walk_pte(st, pmdp, addr, next); - } - } while (pmdp++, addr = next, addr != end); + struct pg_state *st = walk->private; + pmd_t val = READ_ONCE(*pmd); + + if (pmd_table(val)) + return 0; + + note_page(st, addr, 3, pmd_val(val)); + + return 0; } -static void walk_pud(struct pg_state *st, pgd_t *pgdp, unsigned long start, - unsigned long end) +static int pte_entry(pte_t *pte, unsigned long addr, + unsigned long next, struct mm_walk *walk) { - unsigned long next, addr = start; - pud_t *pudp = pud_offset(pgdp, start); - - do { - pud_t pud = READ_ONCE(*pudp); - next = pud_addr_end(addr, end); - - if (pud_none(pud) || pud_sect(pud)) { - note_page(st, addr, 2, pud_val(pud)); - } else { - BUG_ON(pud_bad(pud)); - walk_pmd(st, pudp, addr, next); - } - } while (pudp++, addr = next, addr != end); + struct pg_state *st = walk->private; + pte_t val = READ_ONCE(*pte); + + note_page(st, addr, 4, pte_val(val)); + + return 0; +} + +static int pte_hole(unsigned long addr, unsigned long next, int depth, + struct mm_walk *walk) +{ + struct pg_state *st = walk->private; + + note_page(st, addr, depth+1, 0); + + return 0; } static void walk_pgd(struct pg_state *st, struct mm_struct *mm, - unsigned long start) + unsigned long start) { - unsigned long end = (start < TASK_SIZE_64) ? TASK_SIZE_64 : 0; - unsigned long next, addr = start; - pgd_t *pgdp = pgd_offset(mm, start); - - do { - pgd_t pgd = READ_ONCE(*pgdp); - next = pgd_addr_end(addr, end); - - if (pgd_none(pgd)) { - note_page(st, addr, 1, pgd_val(pgd)); - } else { - BUG_ON(pgd_bad(pgd)); - walk_pud(st, pgdp, addr, next); - } - } while (pgdp++, addr = next, addr != end); + struct mm_walk walk = { + .mm = mm, + .private = st, + .pud_entry = pud_entry, + .pmd_entry = pmd_entry, + .pte_entry = pte_entry, + .pte_hole = pte_hole + }; + down_read(&mm->mmap_sem); + walk_page_range(start, start | (((unsigned long)PTRS_PER_PGD << + PGDIR_SHIFT) - 1), + &walk); + up_read(&mm->mmap_sem); } void ptdump_walk_pgd(struct seq_file *m, struct ptdump_info *info) From patchwork Thu Feb 21 11:34:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10823855 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C53FD1399 for ; Thu, 21 Feb 2019 11:35:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B142C30737 for ; Thu, 21 Feb 2019 11:35:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A543F30764; Thu, 21 Feb 2019 11:35:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E074E3075F for ; Thu, 21 Feb 2019 11:35:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4D28A8E007C; Thu, 21 Feb 2019 06:35:53 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 433FD8E0075; Thu, 21 Feb 2019 06:35:53 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2FAB78E007C; Thu, 21 Feb 2019 06:35:53 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by kanga.kvack.org (Postfix) with ESMTP id C32B28E0075 for ; Thu, 21 Feb 2019 06:35:52 -0500 (EST) Received: by mail-ed1-f70.google.com with SMTP id s50so11309923edd.11 for ; Thu, 21 Feb 2019 03:35:52 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=GFC0FYBY+JUrvKT/vb2dapYcOoZxpui37OtBzYO/OGU=; b=ZgPfLnt0u9qL8JFMg8Bde5CEowXtBaLfgNh1xHcsAu0j0xTveiFKNJfkxzMwPtQflM oXJZA9hCiNhZj+fO98Br07v6wb+F4ukR0bI2Y5G3WLp+GalpoYGetD0l9icNjyqTF3h5 gxaUk+3LU3g6B740NaNx4BBdIUWQPfWAtKnjd7muHHvHR2O+uuVmURpDzd6atB3zMWpT /Bj5tfWM87MtyRQmCJn0IN0/Qsb1icxdD94zC+D9tPiaL4KWAolYmV4/1cuLY6V9sdQm RfMDDBdT9znSnxumyUygPTEXX1HK7myAxaJr8Bj+1Tn+Em0JE6J6GvYVW2N2vZE3qJsx wJjw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAub7Fc7/BXKSggZnj26WfXBJr6k1RMi6qzBwPJ+WdcZv97ACjCGo AkdqaseDdCZKrFru8qqzjefxfyyz4Rfgl15f762q62btrS2P6T7XEQkSCcTDY1NOaIKjEAifKkA sH1cokBugQ32CSpyJhVc3pcFoS3oFBdt9TLGODdmODUhayTyoTSP2HDEjfX2ZjOagGA== X-Received: by 2002:a50:9150:: with SMTP id f16mr22604436eda.96.1550748952279; Thu, 21 Feb 2019 03:35:52 -0800 (PST) X-Google-Smtp-Source: AHgI3IYle5aaBBYf06GYWD/zi19lw/y420DvSNHjTm4eSI2itzLeUZQSL2EQBY6kyHiS/cI5+1h8 X-Received: by 2002:a50:9150:: with SMTP id f16mr22604361eda.96.1550748950980; Thu, 21 Feb 2019 03:35:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550748950; cv=none; d=google.com; s=arc-20160816; b=k6Zabc0d1ph4emGc0NiWxl03oojT3W7lFwt+zLji2tA+RxjWHQPOno3jTdQua60cOD lfThGYI4moWI299rCWHfgbhMwcMEK1W7OTnPN5Y+QfWUUvgF2zEj2w2dVq1NCheng0wA gTri/4RHv1wP2WtlY1TAfBFm3o0NnlFXDp17hwFBvQw/ylyBfX2pylpvFBX30WSwNjxM dIrNRXapINeRsH15yiAywPwPqMjvfod1EgcBfzhEaBoCZ1MECCBbJdqt6O/vYfJS9r5y EvqiAP7gVpsXbv+pAleJKTSgzrSkRJj+/MQr7mPoppMNZMD6AwGONTJ8tDfsVsMHborB 3tiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=GFC0FYBY+JUrvKT/vb2dapYcOoZxpui37OtBzYO/OGU=; b=sBLBVcHx/MVyNzFxlGJvkD94rfRgMaxvRp+qOlEdlSTH/AwrJF/59MMwrJYUXT8gFz 4RYNoUQT31/zuZLRBvYSqwaD1HmRrvM4zMWIZy+IsK1z1Aio8t3vhWUO8VlaSfELlff5 BnuJ+mTPgRc8jNA/Fi46HBaZaayaO9eZo8xXoSdTDHViM40xd98aOr6PnkvN9Rf7ZM76 PL71w5Y60rnEshvCllgG4Q6thGqaTToDKja7GxfvJ9SDMWcVyfuRzXw8qfTdS/ICjlVj KiOebSDVfscrtcVs68fOT8Y9eibaPBpsVCKpERz1ZcgC/A9v3poU/13FCXrcb6f1XAVc 8NXw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id l2si1469454ejs.241.2019.02.21.03.35.50 for ; Thu, 21 Feb 2019 03:35:50 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id EBD6A16A3; Thu, 21 Feb 2019 03:35:49 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 451F73F5C1; Thu, 21 Feb 2019 03:35:46 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v2 09/13] x86/mm: Point to struct seq_file from struct pg_state Date: Thu, 21 Feb 2019 11:34:58 +0000 Message-Id: <20190221113502.54153-10-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190221113502.54153-1-steven.price@arm.com> References: <20190221113502.54153-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP mm/dump_pagetables.c passes both struct seq_file and struct pg_state down the chain of walk_*_level() functions to be passed to note_page(). Instead place the struct seq_file in struct pg_state and access it from struct pg_state (which is private to this file) in note_page(). Signed-off-by: Steven Price --- arch/x86/mm/dump_pagetables.c | 69 ++++++++++++++++++----------------- 1 file changed, 35 insertions(+), 34 deletions(-) diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index cf37abc0f58a..b1b86157d456 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -40,6 +40,7 @@ struct pg_state { bool to_dmesg; bool check_wx; unsigned long wx_pages; + struct seq_file *seq; }; struct addr_marker { @@ -268,11 +269,12 @@ static void note_wx(struct pg_state *st) * of PTE entries; the next one is different so we need to * print what we collected so far. */ -static void note_page(struct seq_file *m, struct pg_state *st, - pgprot_t new_prot, pgprotval_t new_eff, int level) +static void note_page(struct pg_state *st, pgprot_t new_prot, + pgprotval_t new_eff, int level) { pgprotval_t prot, cur, eff; static const char units[] = "BKMGTPE"; + struct seq_file *m = st->seq; /* * If we have a "break" in the series, we need to flush the state that @@ -357,8 +359,8 @@ static inline pgprotval_t effective_prot(pgprotval_t prot1, pgprotval_t prot2) ((prot1 | prot2) & _PAGE_NX); } -static void walk_pte_level(struct seq_file *m, struct pg_state *st, pmd_t addr, - pgprotval_t eff_in, unsigned long P) +static void walk_pte_level(struct pg_state *st, pmd_t addr, pgprotval_t eff_in, + unsigned long P) { int i; pte_t *pte; @@ -369,7 +371,7 @@ static void walk_pte_level(struct seq_file *m, struct pg_state *st, pmd_t addr, pte = pte_offset_map(&addr, st->current_address); prot = pte_flags(*pte); eff = effective_prot(eff_in, prot); - note_page(m, st, __pgprot(prot), eff, 5); + note_page(st, __pgprot(prot), eff, 5); pte_unmap(pte); } } @@ -382,22 +384,20 @@ static void walk_pte_level(struct seq_file *m, struct pg_state *st, pmd_t addr, * us dozens of seconds (minutes for 5-level config) while checking for * W+X mapping or reading kernel_page_tables debugfs file. */ -static inline bool kasan_page_table(struct seq_file *m, struct pg_state *st, - void *pt) +static inline bool kasan_page_table(struct pg_state *st, void *pt) { if (__pa(pt) == __pa(kasan_early_shadow_pmd) || (pgtable_l5_enabled() && __pa(pt) == __pa(kasan_early_shadow_p4d)) || __pa(pt) == __pa(kasan_early_shadow_pud)) { pgprotval_t prot = pte_flags(kasan_early_shadow_pte[0]); - note_page(m, st, __pgprot(prot), 0, 5); + note_page(st, __pgprot(prot), 0, 5); return true; } return false; } #else -static inline bool kasan_page_table(struct seq_file *m, struct pg_state *st, - void *pt) +static inline bool kasan_page_table(struct pg_state *st, void *pt) { return false; } @@ -405,7 +405,7 @@ static inline bool kasan_page_table(struct seq_file *m, struct pg_state *st, #if PTRS_PER_PMD > 1 -static void walk_pmd_level(struct seq_file *m, struct pg_state *st, pud_t addr, +static void walk_pmd_level(struct pg_state *st, pud_t addr, pgprotval_t eff_in, unsigned long P) { int i; @@ -419,19 +419,19 @@ static void walk_pmd_level(struct seq_file *m, struct pg_state *st, pud_t addr, prot = pmd_flags(*start); eff = effective_prot(eff_in, prot); if (pmd_large(*start) || !pmd_present(*start)) { - note_page(m, st, __pgprot(prot), eff, 4); - } else if (!kasan_page_table(m, st, pmd_start)) { - walk_pte_level(m, st, *start, eff, + note_page(st, __pgprot(prot), eff, 4); + } else if (!kasan_page_table(st, pmd_start)) { + walk_pte_level(st, *start, eff, P + i * PMD_LEVEL_MULT); } } else - note_page(m, st, __pgprot(0), 0, 4); + note_page(st, __pgprot(0), 0, 4); start++; } } #else -#define walk_pmd_level(m,s,a,e,p) walk_pte_level(m,s,__pmd(pud_val(a)),e,p) +#define walk_pmd_level(s,a,e,p) walk_pte_level(s,__pmd(pud_val(a)),e,p) #undef pud_large #define pud_large(a) pmd_large(__pmd(pud_val(a))) #define pud_none(a) pmd_none(__pmd(pud_val(a))) @@ -439,8 +439,8 @@ static void walk_pmd_level(struct seq_file *m, struct pg_state *st, pud_t addr, #if PTRS_PER_PUD > 1 -static void walk_pud_level(struct seq_file *m, struct pg_state *st, p4d_t addr, - pgprotval_t eff_in, unsigned long P) +static void walk_pud_level(struct pg_state *st, p4d_t addr, pgprotval_t eff_in, + unsigned long P) { int i; pud_t *start, *pud_start; @@ -455,13 +455,13 @@ static void walk_pud_level(struct seq_file *m, struct pg_state *st, p4d_t addr, prot = pud_flags(*start); eff = effective_prot(eff_in, prot); if (pud_large(*start) || !pud_present(*start)) { - note_page(m, st, __pgprot(prot), eff, 3); - } else if (!kasan_page_table(m, st, pud_start)) { - walk_pmd_level(m, st, *start, eff, + note_page(st, __pgprot(prot), eff, 3); + } else if (!kasan_page_table(st, pud_start)) { + walk_pmd_level(st, *start, eff, P + i * PUD_LEVEL_MULT); } } else - note_page(m, st, __pgprot(0), 0, 3); + note_page(st, __pgprot(0), 0, 3); prev_pud = start; start++; @@ -469,21 +469,21 @@ static void walk_pud_level(struct seq_file *m, struct pg_state *st, p4d_t addr, } #else -#define walk_pud_level(m,s,a,e,p) walk_pmd_level(m,s,__pud(p4d_val(a)),e,p) +#define walk_pud_level(s,a,e,p) walk_pmd_level(s,__pud(p4d_val(a)),e,p) #undef p4d_large #define p4d_large(a) pud_large(__pud(p4d_val(a))) #define p4d_none(a) pud_none(__pud(p4d_val(a))) #endif -static void walk_p4d_level(struct seq_file *m, struct pg_state *st, pgd_t addr, - pgprotval_t eff_in, unsigned long P) +static void walk_p4d_level(struct pg_state *st, pgd_t addr, pgprotval_t eff_in, + unsigned long P) { int i; p4d_t *start, *p4d_start; pgprotval_t prot, eff; if (PTRS_PER_P4D == 1) - return walk_pud_level(m, st, __p4d(pgd_val(addr)), eff_in, P); + return walk_pud_level(st, __p4d(pgd_val(addr)), eff_in, P); p4d_start = start = (p4d_t *)pgd_page_vaddr(addr); @@ -493,13 +493,13 @@ static void walk_p4d_level(struct seq_file *m, struct pg_state *st, pgd_t addr, prot = p4d_flags(*start); eff = effective_prot(eff_in, prot); if (p4d_large(*start) || !p4d_present(*start)) { - note_page(m, st, __pgprot(prot), eff, 2); - } else if (!kasan_page_table(m, st, p4d_start)) { - walk_pud_level(m, st, *start, eff, + note_page(st, __pgprot(prot), eff, 2); + } else if (!kasan_page_table(st, p4d_start)) { + walk_pud_level(st, *start, eff, P + i * P4D_LEVEL_MULT); } } else - note_page(m, st, __pgprot(0), 0, 2); + note_page(st, __pgprot(0), 0, 2); start++; } @@ -537,6 +537,7 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, } st.check_wx = checkwx; + st.seq = m; if (checkwx) st.wx_pages = 0; @@ -550,13 +551,13 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, eff = prot; #endif if (pgd_large(*start) || !pgd_present(*start)) { - note_page(m, &st, __pgprot(prot), eff, 1); + note_page(&st, __pgprot(prot), eff, 1); } else { - walk_p4d_level(m, &st, *start, eff, + walk_p4d_level(&st, *start, eff, i * PGD_LEVEL_MULT); } } else - note_page(m, &st, __pgprot(0), 0, 1); + note_page(&st, __pgprot(0), 0, 1); cond_resched(); start++; @@ -564,7 +565,7 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, /* Flush out the last page */ st.current_address = normalize_addr(PTRS_PER_PGD*PGD_LEVEL_MULT); - note_page(m, &st, __pgprot(0), 0, 0); + note_page(&st, __pgprot(0), 0, 0); if (!checkwx) return; if (st.wx_pages) From patchwork Thu Feb 21 11:34:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10823857 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B10626C2 for ; Thu, 21 Feb 2019 11:36:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A0B153062D for ; Thu, 21 Feb 2019 11:36:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 943B130736; Thu, 21 Feb 2019 11:36:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 117583062D for ; Thu, 21 Feb 2019 11:36:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D80268E007D; Thu, 21 Feb 2019 06:35:56 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D326D8E0075; Thu, 21 Feb 2019 06:35:56 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD92E8E007D; Thu, 21 Feb 2019 06:35:56 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id 591448E0075 for ; Thu, 21 Feb 2019 06:35:56 -0500 (EST) Received: by mail-ed1-f69.google.com with SMTP id d31so7281079eda.1 for ; Thu, 21 Feb 2019 03:35:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=A6FWLpEKbC8B3Rr+irQKv2sIC2Ggbgddq6ZimVEhPUk=; b=lp2OVFMBXyWdQ/6dNHC68UEX14m8y+IIebh+jEGqzBp4MZv7L5l16k2u+jn4gwdhzt V59PpT1PgtVFFVI8YtUPc/sPk18148F6pmIi8iMuqh2CxsHFcHjg6cY2Tc1Ef3AHMb+Z yyWjzlVWyAAaUAlYhIJjN2+eZX22dbZ+/HO+nhZjkzMEW3ruPv/sFlQMagtoagIvr6al LvpqM9U+x5/5rTsBcX4FvMcL6LKKIfCfu55bP6cSHWYn58Pk8RtoQHX22nmk1HTHomEh hdDKrCQhbLylsdXxT8vWDKs19fuiPvW7YwYvejgeK7QjmcMLZBmEItj1znZkWzP+85SW 0USQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAuZvocH3IShmcn7xpV89ZE6j0t8DgnBolzYB8zCuFqCkW+dHlkIR I2+0hcqlbKO2zM1D3qb4MrUIyogrHBKD7Ln/mwol/BZt/uoa/13ee8yh5LRj+MKmW9mtxVe4fck fB2mxYRiFvM0zDQXHyvDuycbxG0DnW9zDbDhPIrbc5Qypy45J0Ox+sH9dlFZi+SJtrw== X-Received: by 2002:a50:a741:: with SMTP id h59mr29446003edc.106.1550748955850; Thu, 21 Feb 2019 03:35:55 -0800 (PST) X-Google-Smtp-Source: AHgI3Ia1pYi9MPjlHuHlg/t1TTQv2hvyhJ5B6TKAZnZ7FkQBMObsYLdhPHLxy05iT5ZQ52r+ybdZ X-Received: by 2002:a50:a741:: with SMTP id h59mr29445953edc.106.1550748954835; Thu, 21 Feb 2019 03:35:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550748954; cv=none; d=google.com; s=arc-20160816; b=do5n5kMm38os2YjOzQf5olm4vgij0TvBBzN+MCIp3zI+l8MmhaVZeusfGjAZoWb0an c/53z7CCSSaYk/8CB46QGWrMkf4B13OwZ8ndMj/vpXj3dSQnhaoByusJrehJlrzjJf02 8hgfr4y4YWkGeprAsFScEdXgURm5uUeAhDdet+chfPW4ljxOZaTPKqjdG9GafWC/YRxx ak81JflYLuJMqFaHLYvzXon6153G3DRAwNFu7IYKEIxEcDUTz2FyI2F7Cb/RN7jadu9W LgWr5+hHQ+Di2YSkuTslW0uo4KGZoJgUDc6TXU8Hff5LuXhadqal9wquoX+Pch1fRpvv OFBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=A6FWLpEKbC8B3Rr+irQKv2sIC2Ggbgddq6ZimVEhPUk=; b=jQ3IuK/0DuI8Ea2iLvdi8Y+8lUmKWAUwlztj9ezoI6KifKI8zboxPWjW+/f2naj3lt RyP2YfGvByk430L5aYZGXoM3e3Tnar+47UmZmavQgjIEHEtL+gosPGAya5dLg81MYn5Q zbOWQH12HYbozHPvjElwUoSFOR60xvwVD/ztB7X+8tPHJQFDj3D2L1ENbst8bKoMdWA/ WhAPipPbZfrGvWQ75cWH8kzUmLoOLmTdb34oLB5r26i4bFJdVuiKgUuRtT5L9HjCldAh fjEU4pJ588iR6QWgmxCMRNOJPTq30agmAf60GyQGwFGboKlbIHlk2D+KoMXgm3G0GgKl gC+Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id by17si7098953ejb.40.2019.02.21.03.35.54 for ; Thu, 21 Feb 2019 03:35:54 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B9D1A1715; Thu, 21 Feb 2019 03:35:53 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 397C03F5C1; Thu, 21 Feb 2019 03:35:50 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v2 10/13] x86/mm+efi: Convert ptdump_walk_pgd_level() to take a mm_struct Date: Thu, 21 Feb 2019 11:34:59 +0000 Message-Id: <20190221113502.54153-11-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190221113502.54153-1-steven.price@arm.com> References: <20190221113502.54153-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP To enable x86 to use the generic walk_page_range() function, the callers of ptdump_walk_pgd_level() need to pass an mm_struct rather than the raw pgd_t pointer. Luckily since commit 7e904a91bf60 ("efi: Use efi_mm in x86 as well as ARM") we now have an mm_struct for EFI on x86. Signed-off-by: Steven Price --- arch/x86/include/asm/pgtable.h | 2 +- arch/x86/mm/dump_pagetables.c | 4 ++-- arch/x86/platform/efi/efi_32.c | 2 +- arch/x86/platform/efi/efi_64.c | 4 ++-- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 3695f6acb6af..371901283d5f 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -27,7 +27,7 @@ extern pgd_t early_top_pgt[PTRS_PER_PGD]; int __init __early_make_pgtable(unsigned long address, pmdval_t pmd); -void ptdump_walk_pgd_level(struct seq_file *m, pgd_t *pgd); +void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm); void ptdump_walk_pgd_level_debugfs(struct seq_file *m, pgd_t *pgd, bool user); void ptdump_walk_pgd_level_checkwx(void); void ptdump_walk_user_pgd_level_checkwx(void); diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index b1b86157d456..07f62d5517da 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -575,9 +575,9 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, pr_info("x86/mm: Checked W+X mappings: passed, no W+X pages found.\n"); } -void ptdump_walk_pgd_level(struct seq_file *m, pgd_t *pgd) +void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm) { - ptdump_walk_pgd_level_core(m, pgd, false, true); + ptdump_walk_pgd_level_core(m, mm->pgd, false, true); } void ptdump_walk_pgd_level_debugfs(struct seq_file *m, pgd_t *pgd, bool user) diff --git a/arch/x86/platform/efi/efi_32.c b/arch/x86/platform/efi/efi_32.c index 9959657127f4..9175ceaa6e72 100644 --- a/arch/x86/platform/efi/efi_32.c +++ b/arch/x86/platform/efi/efi_32.c @@ -49,7 +49,7 @@ void efi_sync_low_kernel_mappings(void) {} void __init efi_dump_pagetable(void) { #ifdef CONFIG_EFI_PGT_DUMP - ptdump_walk_pgd_level(NULL, swapper_pg_dir); + ptdump_walk_pgd_level(NULL, init_mm); #endif } diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c index cf0347f61b21..a2e0f9800190 100644 --- a/arch/x86/platform/efi/efi_64.c +++ b/arch/x86/platform/efi/efi_64.c @@ -611,9 +611,9 @@ void __init efi_dump_pagetable(void) { #ifdef CONFIG_EFI_PGT_DUMP if (efi_enabled(EFI_OLD_MEMMAP)) - ptdump_walk_pgd_level(NULL, swapper_pg_dir); + ptdump_walk_pgd_level(NULL, init_mm); else - ptdump_walk_pgd_level(NULL, efi_mm.pgd); + ptdump_walk_pgd_level(NULL, efi_mm); #endif } From patchwork Thu Feb 21 11:35:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10823859 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 93B576C2 for ; Thu, 21 Feb 2019 11:36:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 830E43062D for ; Thu, 21 Feb 2019 11:36:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 76EF130736; Thu, 21 Feb 2019 11:36:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DF01F3062D for ; Thu, 21 Feb 2019 11:36:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B37668E007E; Thu, 21 Feb 2019 06:36:00 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AE8B28E0075; Thu, 21 Feb 2019 06:36:00 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 966418E007E; Thu, 21 Feb 2019 06:36:00 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id 36F888E0075 for ; Thu, 21 Feb 2019 06:36:00 -0500 (EST) Received: by mail-ed1-f69.google.com with SMTP id a21so10232812eda.3 for ; Thu, 21 Feb 2019 03:36:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=1HaF7HdEQu5nvTk+zAQqC41ZLRHJB1dniFbJzkE+I4g=; b=D02KGzoWSXpQ26d7aP5ufMJnxSw28stbWekmZhlnv37eiT/JS86cb1lnfHtTDObS4p ul6VNiS8R8SHnYta5+ahnGQ5VDRzluZwVpfHmCQ7GQOC2e83QqmbInH0Z4DEHzVp7ei4 cnbG2XgM1SJ/Abo3QHujGvFRwW3AqVsukWLERRR2zvsZgV7xkU3W0Z8wtWj7qGtH23iM wk6CeAeuR0oME6N0EPDAQOIhspRAYuPcjfwy2evaBfxaLgZjHv5stKdXR30k10Xnltqy XfNkueFNGSYO/t2ID5u1HhPeLQlweNuIbKE7dCn0AqchyoL7+j9n9t9PAyzn+HTPP1dV MwJw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAuZWpCsOMLUcM7V5jV7Rh/b4G8Av5+WD4JZnhdJEVksW2l9LL4Ch 3vHifP3ddnc1tecGvJWMD710w4gMmjvNsgVgOyCxaXvoZ2Lf6Q+T0jLCF5zTV3lrAXwQH57tq1b ceoUAsIyHxa8hy2dLuNqM+TD2q2+E1z/LmA2/UdsVvvQLeIslc9Mw3d8EuODThNn0qg== X-Received: by 2002:a50:910c:: with SMTP id e12mr21680594eda.259.1550748959713; Thu, 21 Feb 2019 03:35:59 -0800 (PST) X-Google-Smtp-Source: AHgI3IYK53pUlbsxzCHC2I5TxEZwc6uKM19yndtBSuZRMI7AHcphuCVNa3aKjKyalY9eWd7Wt4P3 X-Received: by 2002:a50:910c:: with SMTP id e12mr21680539eda.259.1550748958679; Thu, 21 Feb 2019 03:35:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550748958; cv=none; d=google.com; s=arc-20160816; b=bW0PybuACR8QPrPrMHOuD/IQyf3NWBXAfFeeEnGL0CquR4fXuLfdHmoVuz0NnCN5ol V6iE71IIWdmsDq9DyH1FUfVwAYzhDkdumYjZMnuhBx/IuWU3MsHy/C5A/KQViHs5m7rZ SVzjL/TuEZdmnKejKBikLvItlRS3fxsERmpc1NnmroLHI0m8Y5oq0k7oRwufxYVlbm4q J/ZVTB2KxDtpwXGsNBGtIcTKVUypEBFmgJoHyAxalcK7dIOvwZOW8lRrllqFdYoeIGH+ 9RHjosq6DTwSDmVNaT11l7YiKjND8hM98Q/4hpfwnsco3etM9WPd5QHq2QGWnkwQZC6c xwUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=1HaF7HdEQu5nvTk+zAQqC41ZLRHJB1dniFbJzkE+I4g=; b=fTirzOGXMRSz4jYZzQNWF0X6TuHBMp1MS1MBZ3o/QnVwBj4bw3zS2lwrUUzFjHn+4e XSnkeJ2jRBdrGNxfjrRGtW/JZ5wcczynX2qvXipaD237GqtduHdqYN+RY81PlHtaaP8B XLb2qr0WHWSTtibBEIF56+0/oQKR3P1/LwWV9y9wqjFnoJ/83HHuMNzyo99Vnddytmqy vemkcASgHLKqc0f8qARTPO73q10QpTSnCYJiYgBttl0xSUKURKZ3QLkUeWGLvti8DGEK fcbiMdo52ZmRcW/vkaQiE89koCKTBi9arsvyx3twjqJcwPzsFvB3IDIGcduLaAGz7obf aErw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id a3si5582760edi.17.2019.02.21.03.35.58 for ; Thu, 21 Feb 2019 03:35:58 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 88DFD15AB; Thu, 21 Feb 2019 03:35:57 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 07E943F5C1; Thu, 21 Feb 2019 03:35:53 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v2 11/13] x86/mm: Convert ptdump_walk_pgd_level_debugfs() to take an mm_struct Date: Thu, 21 Feb 2019 11:35:00 +0000 Message-Id: <20190221113502.54153-12-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190221113502.54153-1-steven.price@arm.com> References: <20190221113502.54153-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP To enable x86 to use the generic walk_page_range() function, the callers of ptdump_walk_pgd_level_debugfs() need to pass in the mm_struct. This means that ptdump_walk_pgd_level_core() is now always passed a valid pgd, so drop the support for pgd==NULL. Signed-off-by: Steven Price --- arch/x86/include/asm/pgtable.h | 3 ++- arch/x86/mm/debug_pagetables.c | 8 ++++---- arch/x86/mm/dump_pagetables.c | 14 ++++++-------- 3 files changed, 12 insertions(+), 13 deletions(-) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 371901283d5f..ab2aa3eb05e9 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -28,7 +28,8 @@ extern pgd_t early_top_pgt[PTRS_PER_PGD]; int __init __early_make_pgtable(unsigned long address, pmdval_t pmd); void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm); -void ptdump_walk_pgd_level_debugfs(struct seq_file *m, pgd_t *pgd, bool user); +void ptdump_walk_pgd_level_debugfs(struct seq_file *m, struct mm_struct *mm, + bool user); void ptdump_walk_pgd_level_checkwx(void); void ptdump_walk_user_pgd_level_checkwx(void); diff --git a/arch/x86/mm/debug_pagetables.c b/arch/x86/mm/debug_pagetables.c index cd84f067e41d..824131052574 100644 --- a/arch/x86/mm/debug_pagetables.c +++ b/arch/x86/mm/debug_pagetables.c @@ -6,7 +6,7 @@ static int ptdump_show(struct seq_file *m, void *v) { - ptdump_walk_pgd_level_debugfs(m, NULL, false); + ptdump_walk_pgd_level_debugfs(m, &init_mm, false); return 0; } @@ -16,7 +16,7 @@ static int ptdump_curknl_show(struct seq_file *m, void *v) { if (current->mm->pgd) { down_read(¤t->mm->mmap_sem); - ptdump_walk_pgd_level_debugfs(m, current->mm->pgd, false); + ptdump_walk_pgd_level_debugfs(m, current->mm, false); up_read(¤t->mm->mmap_sem); } return 0; @@ -31,7 +31,7 @@ static int ptdump_curusr_show(struct seq_file *m, void *v) { if (current->mm->pgd) { down_read(¤t->mm->mmap_sem); - ptdump_walk_pgd_level_debugfs(m, current->mm->pgd, true); + ptdump_walk_pgd_level_debugfs(m, current->mm, true); up_read(¤t->mm->mmap_sem); } return 0; @@ -46,7 +46,7 @@ static struct dentry *pe_efi; static int ptdump_efi_show(struct seq_file *m, void *v) { if (efi_mm.pgd) - ptdump_walk_pgd_level_debugfs(m, efi_mm.pgd, false); + ptdump_walk_pgd_level_debugfs(m, &efi_mm, false); return 0; } diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index 07f62d5517da..8b457a65ad8e 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -526,16 +526,12 @@ static inline bool is_hypervisor_range(int idx) static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, bool checkwx, bool dmesg) { - pgd_t *start = INIT_PGD; + pgd_t *start = pgd; pgprotval_t prot, eff; int i; struct pg_state st = {}; - if (pgd) { - start = pgd; - st.to_dmesg = dmesg; - } - + st.to_dmesg = dmesg; st.check_wx = checkwx; st.seq = m; if (checkwx) @@ -580,8 +576,10 @@ void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm) ptdump_walk_pgd_level_core(m, mm->pgd, false, true); } -void ptdump_walk_pgd_level_debugfs(struct seq_file *m, pgd_t *pgd, bool user) +void ptdump_walk_pgd_level_debugfs(struct seq_file *m, struct mm_struct *mm, + bool user) { + pgd_t *pgd = mm->pgd; #ifdef CONFIG_PAGE_TABLE_ISOLATION if (user && static_cpu_has(X86_FEATURE_PTI)) pgd = kernel_to_user_pgdp(pgd); @@ -607,7 +605,7 @@ void ptdump_walk_user_pgd_level_checkwx(void) void ptdump_walk_pgd_level_checkwx(void) { - ptdump_walk_pgd_level_core(NULL, NULL, true, false); + ptdump_walk_pgd_level_core(NULL, INIT_PGD, true, false); } static int __init pt_dump_init(void) From patchwork Thu Feb 21 11:35:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10823861 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B894A6C2 for ; Thu, 21 Feb 2019 11:36:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A6D513062D for ; Thu, 21 Feb 2019 11:36:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9A24630736; Thu, 21 Feb 2019 11:36:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 22AB43062D for ; Thu, 21 Feb 2019 11:36:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8F2F58E007F; Thu, 21 Feb 2019 06:36:04 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 89E768E0075; Thu, 21 Feb 2019 06:36:04 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F13B8E007F; Thu, 21 Feb 2019 06:36:04 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by kanga.kvack.org (Postfix) with ESMTP id 127C18E0075 for ; Thu, 21 Feb 2019 06:36:04 -0500 (EST) Received: by mail-ed1-f70.google.com with SMTP id e46so1997148ede.9 for ; Thu, 21 Feb 2019 03:36:04 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=htZ43xbxBik8G+1SZkvefzMNP0baYE6/YcLJdQXc4CE=; b=R3Fv0yxNXHW6KLXXGONjVmTi2e3FdWaBkeKfSPA9T1AXuoImYKzdkxlh7dQ1TrRf2N Bg+HP2vXQGwaBR7eLS8/1cZtf3+n/CMJpiODMYQ5ZSX2PtgyZi9IP1Uzlx7V0GlqdUWr RtqBaOdpsblBNHKkvo9XPifDAMlA3zNmvPRbMPMwnNYJ34anN579A9aQvbkc3e90pei+ tdRehWWeG4cZENWHIK44C8McYE/1rLj3aTyxPXrWQ82wMhlzMM35gmGQ7Lkc0BsO/TUM WOtILkONXoA8wE3hma1ZXUCu7W2JNPGW0w1mKui7RwaTP787DbiK5MxgbnKStfuuZm4t nEmw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAub3/TjwP4q9A0K6/ZhRXLKjy8jtUKvYVqH5OsV8JzHqfnyjTQeb LQii5GQ8esiuM3b+yecpxCkFAsMNqTQ+p0NcLI6oHKXSe4m+vDvUzehuDf3Vh55MgodN6brDdlN 0O7tnENXfkUrVkmc7Cj4UKcNcYMkYq54jX16M8IbKIqYNmhZ+SBCi5YQRK29f7I9YGg== X-Received: by 2002:a17:906:ee1:: with SMTP id x1mr27091264eji.85.1550748963560; Thu, 21 Feb 2019 03:36:03 -0800 (PST) X-Google-Smtp-Source: AHgI3IaVRWF5vSfg6TFU2jZ41BCbgRNGNHempPD3bHlq4qVrEBOQ5uFR9PrwIaTfPG0nujXJ8ZVC X-Received: by 2002:a17:906:ee1:: with SMTP id x1mr27091208eji.85.1550748962443; Thu, 21 Feb 2019 03:36:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550748962; cv=none; d=google.com; s=arc-20160816; b=LU1frJm30cBboNHcRlKX+H08Jg37pb7BPkFBB2BvgYbvMbVvnEiKGtpU7Mk1/NE6lS IfRAL8k8eE60l1BmETchwzIWiLWasT0MOfapHCU2Sv1Z/Rt7PM8t+DPkE/rYK/9KJLDh v/qcGk+XOBmarXhxgW5s2tuBVkCg8houMZ6K2Cvm+5dAC9Oppg2yIfTSU7iWB6KR0VOF aiGLZ7AGcZ3qfQ6A0xdrj1GMwiyl0tzNO/krRcTrJChn+VGVkk2FjV+JH9RNZvak481J BcYIUmI1B8YkiGDhTImT8x1q3XSZ1W4UPwsuI4GLjir8ZcIpsnN2TG0+3L1hQAt1g/ms gEVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=htZ43xbxBik8G+1SZkvefzMNP0baYE6/YcLJdQXc4CE=; b=BCblbhlYx/O33/yk+mu41+SLq0A37lhwVwBeQA2S6W/+BB/KMhxA5RoY+f15PKNxBM 3yLWhoG9CfUUYMpHGvN0emlBZk7AdFpuo0pV6WC92s9OUq/Oel7gUEO8RPS5mCI5n0ty jeK2aGbgdakRlX7SaG8qRzfRIA4dTXmYDRO2HYbqrWDifR0sItpOto+6p7BsO0G3qZ7a zVYX87K75THkAVQIP7Dd+K+rVQ+Al6qQtlARS/mVsbLgNBHkRRp+QW6Uak69CyiEon3Y JCkRYlLdc1zX+bwlhFWeAQl+xULKB+eYJEFbBfeotpkel2KIBfKAtqpGd0JHUuxUHGkd FRDg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id a3si2082015edl.25.2019.02.21.03.36.01 for ; Thu, 21 Feb 2019 03:36:02 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 58B7480D; Thu, 21 Feb 2019 03:36:01 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id CB4BF3F5C1; Thu, 21 Feb 2019 03:35:57 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v2 12/13] x86/mm: Convert ptdump_walk_pgd_level_core() to take an mm_struct Date: Thu, 21 Feb 2019 11:35:01 +0000 Message-Id: <20190221113502.54153-13-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190221113502.54153-1-steven.price@arm.com> References: <20190221113502.54153-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP An mm_struct is needed to enable x86 to use of the generic walk_page_range() function. In the case of walking the user page tables (when CONFIG_PAGE_TABLE_ISOLATION is enabled), it is necessary to create a fake_mm structure because there isn't an mm_struct with a pointer to the pgd of the user page tables. This fake_mm structure is initialised with the minimum necessary for the generic page walk code. Signed-off-by: Steven Price --- arch/x86/mm/dump_pagetables.c | 36 ++++++++++++++++++++--------------- 1 file changed, 21 insertions(+), 15 deletions(-) diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index 8b457a65ad8e..0953bf44d792 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -111,8 +111,6 @@ static struct addr_marker address_markers[] = { [END_OF_SPACE_NR] = { -1, NULL } }; -#define INIT_PGD ((pgd_t *) &init_top_pgt) - #else /* CONFIG_X86_64 */ enum address_markers_idx { @@ -147,8 +145,6 @@ static struct addr_marker address_markers[] = { [END_OF_SPACE_NR] = { -1, NULL } }; -#define INIT_PGD (swapper_pg_dir) - #endif /* !CONFIG_X86_64 */ /* Multipliers for offsets within the PTEs */ @@ -523,10 +519,10 @@ static inline bool is_hypervisor_range(int idx) #endif } -static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, +static void ptdump_walk_pgd_level_core(struct seq_file *m, struct mm_struct *mm, bool checkwx, bool dmesg) { - pgd_t *start = pgd; + pgd_t *start = mm->pgd; pgprotval_t prot, eff; int i; struct pg_state st = {}; @@ -573,39 +569,49 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm) { - ptdump_walk_pgd_level_core(m, mm->pgd, false, true); + ptdump_walk_pgd_level_core(m, mm, false, true); } +#ifdef CONFIG_PAGE_TABLE_ISOLATION +static void ptdump_walk_pgd_level_user_core(struct seq_file *m, + struct mm_struct *mm, + bool checkwx, bool dmesg) +{ + struct mm_struct fake_mm = { + .pgd = kernel_to_user_pgdp(mm->pgd) + }; + init_rwsem(&fake_mm.mmap_sem); + ptdump_walk_pgd_level_core(m, &fake_mm, checkwx, dmesg); +} +#endif + void ptdump_walk_pgd_level_debugfs(struct seq_file *m, struct mm_struct *mm, bool user) { - pgd_t *pgd = mm->pgd; #ifdef CONFIG_PAGE_TABLE_ISOLATION if (user && static_cpu_has(X86_FEATURE_PTI)) - pgd = kernel_to_user_pgdp(pgd); + ptdump_walk_pgd_level_user_core(m, mm, false, false); + else #endif - ptdump_walk_pgd_level_core(m, pgd, false, false); + ptdump_walk_pgd_level_core(m, mm, false, false); } EXPORT_SYMBOL_GPL(ptdump_walk_pgd_level_debugfs); void ptdump_walk_user_pgd_level_checkwx(void) { #ifdef CONFIG_PAGE_TABLE_ISOLATION - pgd_t *pgd = INIT_PGD; - if (!(__supported_pte_mask & _PAGE_NX) || !static_cpu_has(X86_FEATURE_PTI)) return; pr_info("x86/mm: Checking user space page tables\n"); - pgd = kernel_to_user_pgdp(pgd); - ptdump_walk_pgd_level_core(NULL, pgd, true, false); + ptdump_walk_pgd_level_user_core(NULL, &init_mm, true, false); #endif } void ptdump_walk_pgd_level_checkwx(void) { - ptdump_walk_pgd_level_core(NULL, INIT_PGD, true, false); + ptdump_walk_pgd_level_core(NULL, &init_mm, true, false); } static int __init pt_dump_init(void) From patchwork Thu Feb 21 11:35:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10823865 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3BE341399 for ; Thu, 21 Feb 2019 11:36:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 27F1B3062D for ; Thu, 21 Feb 2019 11:36:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1BA3B30737; Thu, 21 Feb 2019 11:36:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 430EC3062D for ; Thu, 21 Feb 2019 11:36:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 87A298E0080; Thu, 21 Feb 2019 06:36:08 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 828B48E0075; Thu, 21 Feb 2019 06:36:08 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6A5878E0080; Thu, 21 Feb 2019 06:36:08 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id 052D18E0075 for ; Thu, 21 Feb 2019 06:36:08 -0500 (EST) Received: by mail-ed1-f69.google.com with SMTP id i22so5632237eds.20 for ; Thu, 21 Feb 2019 03:36:07 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=NqO2orxBM0NrjJiuXU9Xz8In8V4AR0Skp7YUe6D+Fac=; b=b6iNGPka82tcUu11K7FW3eXf6R/uVmK3rj4I74Skq8DzobZhGtwGMyewYfouI0VaBs GOHV1UhS3YiztK+Bxfdho2YVURrhOys4hguQFZzQ1AECA/A14Uh2px0DpbY9lej8c55P 5S/HItc+VdRTNcoQw51NWtkFJWan7u18PB67cOlkHM9R1ueah7fRmgPEdjBNx0++TVE/ 6VDpZD58gRCfvCM1SxG2pn/iTMZEfLiNMW/3f+TG+22awQz8FdYLw5svaw45MT0RqE13 Y+ALSXzAumNMNhCs2frau7YIlbWALDXxqVyqcvySBYzweEjzy0jj75gw7wwn8vzrmGwP 43MQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAubKzbD9UxNyM/kJHz9RysPrCtYdCk0ykxnCCzVg4hWrJlQZzUZE jPN/0H0PgiaN8n7n+3+Ai1eavnD1c3v03g29YxJ+Ve4AGsaPBb1QMpmyfoLELv0muzirfpCM1AJ oOnrPn757wXm7YxwiKhlLBvqfqKI12tNQYyBWjUL+dmovJ5omg4pQhMTKC/E0ehjIdA== X-Received: by 2002:a50:ad31:: with SMTP id y46mr30303970edc.97.1550748967479; Thu, 21 Feb 2019 03:36:07 -0800 (PST) X-Google-Smtp-Source: AHgI3Ib6eOJHTLC57IRupXey/AbX4DV0uzr8bWemaCMLddwljP4qqS8w4HB1xn2uaZNbtqi+IqcU X-Received: by 2002:a50:ad31:: with SMTP id y46mr30303895edc.97.1550748966221; Thu, 21 Feb 2019 03:36:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550748966; cv=none; d=google.com; s=arc-20160816; b=ubfLcxpl5zGSc7EGXyeSqPuiRb4zS9Idlj9RJqj6+472JCcKL0mbXYrUBZ4C68xxi1 T/Cworw5NVwCAYRVMKe5mfqdZYoJdD4o9ovk7eziNOPW1qSMuudjSnR0lZnuNcArqXg1 1fp1Znkz1sl6zOKTcpABDCD/SuEl90AWs5f6hRb5OTUw+rHZ8ncvG6zlPZ28WtkVixLa wgUrUXM9FG5wcjX9bYfYKh2uO65oZJYtf901b43hebxmGeWUIpfCvQqe2W8mkAwHn9Hu i3o4F+SynMiGMr4odMNxrrFAIGCfoDSi5t9zflAgF3208tQoQB2TMOlEt/rz1P9l+ov4 Ad5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=NqO2orxBM0NrjJiuXU9Xz8In8V4AR0Skp7YUe6D+Fac=; b=fiZbK1MStl6F/bxAPOSavJNMFoS907RGdo/NlvpevgQzcaD8Go4493Fax005uqAFyn +HlaVJ1q1ZzqasA3LjYQO1sfoKVb6BRJ3aFzvX7Bl2sn0Ya9SeG+SwUiXv4xrMkaDRhF QeLUOPNaOssizgWK/y2HZ00xw+HbZARKJhFVGh+cwZ00GqTG9nVw9OHs+qgY2NPavEiE CUdlzOGmoFlEhOiHB+ECHUz+44TxllQhLAiHS4fWrpWhGsSRTK1Eyya8EUAEbWjUZPKz U/pNvFv+DKymV1qGS4c3UuPBMJm0H9XNdqWVItuxHvd9x4b7JntvXtKMWCsO7Wt+lo5u sg1A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id q1si2342314ejz.59.2019.02.21.03.36.05 for ; Thu, 21 Feb 2019 03:36:06 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 281A0165C; Thu, 21 Feb 2019 03:36:05 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9B3C43F5C1; Thu, 21 Feb 2019 03:36:01 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v2 13/13] x86: mm: Convert dump_pagetables to use walk_page_range Date: Thu, 21 Feb 2019 11:35:02 +0000 Message-Id: <20190221113502.54153-14-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190221113502.54153-1-steven.price@arm.com> References: <20190221113502.54153-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Make use of the new functionality in walk_page_range to remove the arch page walking code and use the generic code to walk the page tables. The effective permissions are passed down the chain using new fields in struct pg_state. The KASAN optimisation is implemented by including test_p?d callbacks which can decide to skip an entire tree of entries Signed-off-by: Steven Price --- arch/x86/mm/dump_pagetables.c | 282 ++++++++++++++++++---------------- 1 file changed, 146 insertions(+), 136 deletions(-) diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index 0953bf44d792..64d1619493a4 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -33,6 +33,10 @@ struct pg_state { int level; pgprot_t current_prot; pgprotval_t effective_prot; + pgprotval_t effective_prot_pgd; + pgprotval_t effective_prot_p4d; + pgprotval_t effective_prot_pud; + pgprotval_t effective_prot_pmd; unsigned long start_address; unsigned long current_address; const struct addr_marker *marker; @@ -355,22 +359,21 @@ static inline pgprotval_t effective_prot(pgprotval_t prot1, pgprotval_t prot2) ((prot1 | prot2) & _PAGE_NX); } -static void walk_pte_level(struct pg_state *st, pmd_t addr, pgprotval_t eff_in, - unsigned long P) +static int ptdump_pte_entry(pte_t *pte, unsigned long addr, + unsigned long next, struct mm_walk *walk) { - int i; - pte_t *pte; - pgprotval_t prot, eff; - - for (i = 0; i < PTRS_PER_PTE; i++) { - st->current_address = normalize_addr(P + i * PTE_LEVEL_MULT); - pte = pte_offset_map(&addr, st->current_address); - prot = pte_flags(*pte); - eff = effective_prot(eff_in, prot); - note_page(st, __pgprot(prot), eff, 5); - pte_unmap(pte); - } + struct pg_state *st = walk->private; + pgprotval_t eff, prot; + + st->current_address = normalize_addr(addr); + + prot = pte_flags(*pte); + eff = effective_prot(st->effective_prot_pmd, prot); + note_page(st, __pgprot(prot), eff, 5); + + return 0; } + #ifdef CONFIG_KASAN /* @@ -399,133 +402,152 @@ static inline bool kasan_page_table(struct pg_state *st, void *pt) } #endif -#if PTRS_PER_PMD > 1 - -static void walk_pmd_level(struct pg_state *st, pud_t addr, - pgprotval_t eff_in, unsigned long P) +static int ptdump_test_pmd(unsigned long addr, unsigned long next, + pmd_t *pmd, struct mm_walk *walk) { - int i; - pmd_t *start, *pmd_start; - pgprotval_t prot, eff; - - pmd_start = start = (pmd_t *)pud_page_vaddr(addr); - for (i = 0; i < PTRS_PER_PMD; i++) { - st->current_address = normalize_addr(P + i * PMD_LEVEL_MULT); - if (!pmd_none(*start)) { - prot = pmd_flags(*start); - eff = effective_prot(eff_in, prot); - if (pmd_large(*start) || !pmd_present(*start)) { - note_page(st, __pgprot(prot), eff, 4); - } else if (!kasan_page_table(st, pmd_start)) { - walk_pte_level(st, *start, eff, - P + i * PMD_LEVEL_MULT); - } - } else - note_page(st, __pgprot(0), 0, 4); - start++; - } + struct pg_state *st = walk->private; + + st->current_address = normalize_addr(addr); + + if (kasan_page_table(st, pmd)) + return 1; + return 0; } -#else -#define walk_pmd_level(s,a,e,p) walk_pte_level(s,__pmd(pud_val(a)),e,p) -#undef pud_large -#define pud_large(a) pmd_large(__pmd(pud_val(a))) -#define pud_none(a) pmd_none(__pmd(pud_val(a))) -#endif +static int ptdump_pmd_entry(pmd_t *pmd, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + struct pg_state *st = walk->private; + pgprotval_t eff, prot; + + prot = pmd_flags(*pmd); + eff = effective_prot(st->effective_prot_pud, prot); + + st->current_address = normalize_addr(addr); + + if (pmd_large(*pmd)) + note_page(st, __pgprot(prot), eff, 4); -#if PTRS_PER_PUD > 1 + st->effective_prot_pmd = eff; -static void walk_pud_level(struct pg_state *st, p4d_t addr, pgprotval_t eff_in, - unsigned long P) + return 0; +} + +static int ptdump_test_pud(unsigned long addr, unsigned long next, + pud_t *pud, struct mm_walk *walk) { - int i; - pud_t *start, *pud_start; - pgprotval_t prot, eff; - pud_t *prev_pud = NULL; - - pud_start = start = (pud_t *)p4d_page_vaddr(addr); - - for (i = 0; i < PTRS_PER_PUD; i++) { - st->current_address = normalize_addr(P + i * PUD_LEVEL_MULT); - if (!pud_none(*start)) { - prot = pud_flags(*start); - eff = effective_prot(eff_in, prot); - if (pud_large(*start) || !pud_present(*start)) { - note_page(st, __pgprot(prot), eff, 3); - } else if (!kasan_page_table(st, pud_start)) { - walk_pmd_level(st, *start, eff, - P + i * PUD_LEVEL_MULT); - } - } else - note_page(st, __pgprot(0), 0, 3); + struct pg_state *st = walk->private; - prev_pud = start; - start++; - } + st->current_address = normalize_addr(addr); + + if (kasan_page_table(st, pud)) + return 1; + return 0; } -#else -#define walk_pud_level(s,a,e,p) walk_pmd_level(s,__pud(p4d_val(a)),e,p) -#undef p4d_large -#define p4d_large(a) pud_large(__pud(p4d_val(a))) -#define p4d_none(a) pud_none(__pud(p4d_val(a))) -#endif +static int ptdump_pud_entry(pud_t *pud, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + struct pg_state *st = walk->private; + pgprotval_t eff, prot; + + prot = pud_flags(*pud); + eff = effective_prot(st->effective_prot_p4d, prot); + + st->current_address = normalize_addr(addr); + + if (pud_large(*pud)) + note_page(st, __pgprot(prot), eff, 3); + + st->effective_prot_pud = eff; -static void walk_p4d_level(struct pg_state *st, pgd_t addr, pgprotval_t eff_in, - unsigned long P) + return 0; +} + +static int ptdump_test_p4d(unsigned long addr, unsigned long next, + p4d_t *p4d, struct mm_walk *walk) { - int i; - p4d_t *start, *p4d_start; - pgprotval_t prot, eff; - - if (PTRS_PER_P4D == 1) - return walk_pud_level(st, __p4d(pgd_val(addr)), eff_in, P); - - p4d_start = start = (p4d_t *)pgd_page_vaddr(addr); - - for (i = 0; i < PTRS_PER_P4D; i++) { - st->current_address = normalize_addr(P + i * P4D_LEVEL_MULT); - if (!p4d_none(*start)) { - prot = p4d_flags(*start); - eff = effective_prot(eff_in, prot); - if (p4d_large(*start) || !p4d_present(*start)) { - note_page(st, __pgprot(prot), eff, 2); - } else if (!kasan_page_table(st, p4d_start)) { - walk_pud_level(st, *start, eff, - P + i * P4D_LEVEL_MULT); - } - } else - note_page(st, __pgprot(0), 0, 2); + struct pg_state *st = walk->private; - start++; - } + st->current_address = normalize_addr(addr); + + if (kasan_page_table(st, p4d)) + return 1; + return 0; } -#undef pgd_large -#define pgd_large(a) (pgtable_l5_enabled() ? pgd_large(a) : p4d_large(__p4d(pgd_val(a)))) -#define pgd_none(a) (pgtable_l5_enabled() ? pgd_none(a) : p4d_none(__p4d(pgd_val(a)))) +static int ptdump_p4d_entry(p4d_t *p4d, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + struct pg_state *st = walk->private; + pgprotval_t eff, prot; + + prot = p4d_flags(*p4d); + eff = effective_prot(st->effective_prot_pgd, prot); + + st->current_address = normalize_addr(addr); + + if (p4d_large(*p4d)) + note_page(st, __pgprot(prot), eff, 2); + + st->effective_prot_p4d = eff; + + return 0; +} -static inline bool is_hypervisor_range(int idx) +static int ptdump_pgd_entry(pgd_t *pgd, unsigned long addr, + unsigned long next, struct mm_walk *walk) { -#ifdef CONFIG_X86_64 - /* - * A hole in the beginning of kernel address space reserved - * for a hypervisor. - */ - return (idx >= pgd_index(GUARD_HOLE_BASE_ADDR)) && - (idx < pgd_index(GUARD_HOLE_END_ADDR)); + struct pg_state *st = walk->private; + pgprotval_t eff, prot; + + prot = pgd_flags(*pgd); + +#ifdef CONFIG_X86_PAE + eff = _PAGE_USER | _PAGE_RW; #else - return false; + eff = prot; #endif + + st->current_address = normalize_addr(addr); + + if (pgd_large(*pgd)) + note_page(st, __pgprot(prot), eff, 1); + + st->effective_prot_pgd = eff; + + return 0; +} + +static int ptdump_hole(unsigned long addr, unsigned long next, int depth, + struct mm_walk *walk) +{ + struct pg_state *st = walk->private; + + st->current_address = normalize_addr(addr); + + note_page(st, __pgprot(0), 0, depth + 1); + + return 0; } static void ptdump_walk_pgd_level_core(struct seq_file *m, struct mm_struct *mm, bool checkwx, bool dmesg) { - pgd_t *start = mm->pgd; - pgprotval_t prot, eff; - int i; struct pg_state st = {}; + struct mm_walk walk = { + .mm = mm, + .pgd_entry = ptdump_pgd_entry, + .p4d_entry = ptdump_p4d_entry, + .pud_entry = ptdump_pud_entry, + .pmd_entry = ptdump_pmd_entry, + .pte_entry = ptdump_pte_entry, + .test_p4d = ptdump_test_p4d, + .test_pud = ptdump_test_pud, + .test_pmd = ptdump_test_pmd, + .pte_hole = ptdump_hole, + .private = &st + }; st.to_dmesg = dmesg; st.check_wx = checkwx; @@ -533,27 +555,15 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, struct mm_struct *mm, if (checkwx) st.wx_pages = 0; - for (i = 0; i < PTRS_PER_PGD; i++) { - st.current_address = normalize_addr(i * PGD_LEVEL_MULT); - if (!pgd_none(*start) && !is_hypervisor_range(i)) { - prot = pgd_flags(*start); -#ifdef CONFIG_X86_PAE - eff = _PAGE_USER | _PAGE_RW; + down_read(&mm->mmap_sem); +#ifdef CONFIG_X86_64 + walk_page_range(0, PTRS_PER_PGD*PGD_LEVEL_MULT/2, &walk); + walk_page_range(normalize_addr(PTRS_PER_PGD*PGD_LEVEL_MULT/2), ~0, + &walk); #else - eff = prot; + walk_page_range(0, ~0, &walk); #endif - if (pgd_large(*start) || !pgd_present(*start)) { - note_page(&st, __pgprot(prot), eff, 1); - } else { - walk_p4d_level(&st, *start, eff, - i * PGD_LEVEL_MULT); - } - } else - note_page(&st, __pgprot(0), 0, 1); - - cond_resched(); - start++; - } + up_read(&mm->mmap_sem); /* Flush out the last page */ st.current_address = normalize_addr(PTRS_PER_PGD*PGD_LEVEL_MULT);