From patchwork Fri Feb 15 17:02:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10815463 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9764B922 for ; Fri, 15 Feb 2019 17:03:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 862A92FBD5 for ; Fri, 15 Feb 2019 17:03:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 79CEB2FC1C; Fri, 15 Feb 2019 17:03:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 07BDF2FBD5 for ; Fri, 15 Feb 2019 17:03:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0CACE8E0004; Fri, 15 Feb 2019 12:03:12 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 078DE8E0001; Fri, 15 Feb 2019 12:03:12 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EA7C08E0004; Fri, 15 Feb 2019 12:03:11 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by kanga.kvack.org (Postfix) with ESMTP id 953A28E0001 for ; Fri, 15 Feb 2019 12:03:11 -0500 (EST) Received: by mail-ed1-f70.google.com with SMTP id x15so4251018edd.2 for ; Fri, 15 Feb 2019 09:03:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=3FSyeGjDNZIADKBQFr5V22ox8XwqOpOZQRYVkJgSrP0=; b=oE9D7IhT3kBGEkzrX/ajsqvLPJsBOLBiaSTbnrmHCI2D8V/V0SsiSS4BjChABGUqxc 9UMLHpOHuKOfkYPiW+sJW3Qx6Xf7VUW56QVd0jh7d1wJGLFh632GfAx3Uuk8OiKGl1vb sI5MyzA7Ar0HEmMnApUiu5waoYJLEh3x+TAGhho4WUEa/pbT3qa3O5d1vRUZtr56D6L+ uLdQJxJqhXP65XQoqohZHZ/PD1otGltytRRKx5ZkVfMwMuK4leql5FyQuOuLC7B/CQEc nAVxbvPIe9lTPdbS+AaQM8bbo6wI/k6PfNsT2C2qsy9FCZkNYyXvkTejqmLNSNwCKc/Z VEgw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAuZsIRaBd8pkebIZv+QVRUKdzk/Rjsq/8xMtcdTJ2xY8JEQXFMtd nKFxJ0hyxUQ/5ZJxQWPCxdEu9mKCjJO551TY2gdolNluhvy1MjJNed4ZsCyPdQ68Uzb1H1kUcAI O5B+a5q6QsvLkB2Uz3Zrr7prT72H75VvQecKLbOZjkvJGqoxFX/y5Qw4oS2n/wxseRw== X-Received: by 2002:a17:906:4a53:: with SMTP id a19mr7245743ejv.229.1550250191091; Fri, 15 Feb 2019 09:03:11 -0800 (PST) X-Google-Smtp-Source: AHgI3IYI8KwL1zFsylxBtU+j9jnuNApN3AN7XbQtW6p/39X9XfwsAGYsFr4Ucz1s/l5B5Pa34v7U X-Received: by 2002:a17:906:4a53:: with SMTP id a19mr7245685ejv.229.1550250190055; Fri, 15 Feb 2019 09:03:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550250190; cv=none; d=google.com; s=arc-20160816; b=tz8WbNCXaoU4eEe9x4hBHYZZNWN2GgwYTh19NiqYeTvvpQvlLPd5Ma98CfHpn2OXYB 8qGw0mqLVmMsb9XtqaQVpAXPcGEyjvzGqEcV9/ovwS3Wkq8HxkRbEiZ6UQbkGP3R487i 4xo8ecGEcuwDUouvVkI+i+XLIVsFf3qk59iWGhsJ/VseJ1nxBhY/3A2JXe27Qmem8BUe Ag8hBdNCFK/TBz/AdFKzqexF5sA5TSE3t2vJ2oh/yKSPFSNbihY2LXFY2Pj87o1jSZDv X6J14nyIAedKMRVH4ItIVRUKg2qROZrn1BUUWlkpaDdP/3Eih6Vm91R1DFXaVsil24i9 SG0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=3FSyeGjDNZIADKBQFr5V22ox8XwqOpOZQRYVkJgSrP0=; b=z1icTpBOW+D7lSsoa4teeZCwl/R2MavDX11OvSZONloul6/gS7aiH1e/+M52SgNu4/ yMTF8zUZMAvSW1qgSm1fdQRCK1gG5uek//KG8QbVg4z69htt3gZ14mADzHo/OmY6TRFX 0pIBx3kg1e5JbIoJB5A/AP2kU9az90vGiRwpA6vMMZw3aHfysiP9Kp2N7jf6t2BIijlq IJ4R4n1rjSx54q2tmqvkQCPtkdf9ciXOHFc1SWdxI7kUme0YavrI/mIUHTDFQjRuVZuP W219s3kZcRcMJjRO41eZB4k/Csmt+8B/Vo6v4M5H/pEpZRzLgFosEvFqbQw4oGuqdeCU nZLg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id k10si2156756ejd.85.2019.02.15.09.03.09 for ; Fri, 15 Feb 2019 09:03:10 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0AE541596; Fri, 15 Feb 2019 09:03:09 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 172FC3F557; Fri, 15 Feb 2019 09:03:05 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 01/13] arm64: mm: Add p?d_large() definitions Date: Fri, 15 Feb 2019 17:02:22 +0000 Message-Id: <20190215170235.23360-2-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190215170235.23360-1-steven.price@arm.com> References: <20190215170235.23360-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: James Morse Exposing the pud/pgd levels of the page tables to walk_page_range() means we may come across the exotic large mappings that come with large areas of contiguous memory (such as the kernel's linear map). Expose p?d_large() from each architecture to detect these large mappings. arm64 already has these macros defined, but with a different name. p?d_large() is used by s390, sparc and x86. Only arm/arm64 use p?d_sect(). Add a macro to allow both names. By not providing a pgd_large(), we get the generic version that always returns 0. Signed-off-by: James Morse Signed-off-by: Steven Price --- arch/arm64/include/asm/pgtable.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index de70c1eabf33..09d308921625 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -428,6 +428,7 @@ extern pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, PMD_TYPE_TABLE) #define pmd_sect(pmd) ((pmd_val(pmd) & PMD_TYPE_MASK) == \ PMD_TYPE_SECT) +#define pmd_large(x) pmd_sect(x) #if defined(CONFIG_ARM64_64K_PAGES) || CONFIG_PGTABLE_LEVELS < 3 #define pud_sect(pud) (0) @@ -435,6 +436,7 @@ extern pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, #else #define pud_sect(pud) ((pud_val(pud) & PUD_TYPE_MASK) == \ PUD_TYPE_SECT) +#define pud_large(x) pud_sect(x) #define pud_table(pud) ((pud_val(pud) & PUD_TYPE_MASK) == \ PUD_TYPE_TABLE) #endif From patchwork Fri Feb 15 17:02:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10815465 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 36B54922 for ; Fri, 15 Feb 2019 17:03:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 255B52FBD5 for ; Fri, 15 Feb 2019 17:03:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 190972FC08; Fri, 15 Feb 2019 17:03:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9FC132FBD5 for ; Fri, 15 Feb 2019 17:03:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5D0FA8E0005; Fri, 15 Feb 2019 12:03:15 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5A7668E0001; Fri, 15 Feb 2019 12:03:15 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 495BF8E0005; Fri, 15 Feb 2019 12:03:15 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id E60158E0001 for ; Fri, 15 Feb 2019 12:03:14 -0500 (EST) Received: by mail-ed1-f69.google.com with SMTP id s50so4185309edd.11 for ; Fri, 15 Feb 2019 09:03:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=fNP6+ygM8w6cTGyyHjAGfLv0kdsZRzrguRwibVqIzvI=; b=V/WHxIkEpBrkkZwX5+2aKy38r2fFkgj8OLVliUZJAPtsWHf7wEgh6gF9voTW3IPBq6 BcPSuyO98rikD5Ot17q5oR+qveEzLQ6phdMpsTqI56/EbsmThDxulMepDB/98uePc8JS XbAg048wUf5Rx2Wj2eWFuM7rjZARyRGXloAfh8c4GWTmz+RpTly2C/ZJdmfMy8xTzreq eWMf2rbNUQlDZQTk2w6jN4pg3HjbczqwtQJIsYjayu6d2Fh7f9TyKeOe8RLlSUF0a/MB tTcstaHjWy1gzIyxAUTBLpDRQM+SDjFY7fJQNH/i3UWWm2IT2Y2qNEDgeTNJnkuGrrGB RPmQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAuZAR6slaci40VXP1zfvZ8kTqE+ZygS4PpaW6zxfbRvb3kZEPp5H VP9Hq+zZ7rJqq00Ra5omozttek8CG9J4uKdF6YZ5rDx9z+wFHmxUbLJyTZd946brokssAw0/BN8 LOJBg7pp+BTizLls6KtmZKcQD7lDLYwC2URySvVrb+BKz4lvJjRuGprcJPONNyayD5g== X-Received: by 2002:a05:6402:1852:: with SMTP id v18mr8066718edy.198.1550250194397; Fri, 15 Feb 2019 09:03:14 -0800 (PST) X-Google-Smtp-Source: AHgI3IZmjLGoOBwlpqMIRKma1V65qGE1y8TUWImYVDQsQl4ImytotuXktEtPEE/rwnV3imLr6v+S X-Received: by 2002:a05:6402:1852:: with SMTP id v18mr8066648edy.198.1550250193267; Fri, 15 Feb 2019 09:03:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550250193; cv=none; d=google.com; s=arc-20160816; b=yhTsqUV/0lbwximTWNeU8zFyJXY0CgusjQIs6jmZ2CsyGo1J5a7v/TgYuvAqPUdTGh Q9Peg7Pbr2r+xCehqjCttjKnLOipK6exNl77pIEq8wQDrPiKyZBlzuS/PpVP5KPxCxlr 2PMTslR7uU/IJ4urL0c+rJBb0/fbwsxB4z9i+d22K0ytSj60/qLPOEI+I959NSDMey0j cchxoY+we3xIVdt1rFIAq77mrL8h52bHbMhkh2z1hvJmERBBZ88VyHOWvlY0n9iA4zDJ 78iWyaYMCtdzPOlxKlkK+j2UxTQ/2zzGl7JnlUfVRxaVRC/q4WxJTn78iLBmdshmjtYm BNRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=fNP6+ygM8w6cTGyyHjAGfLv0kdsZRzrguRwibVqIzvI=; b=ITTyltyn92ImRqfwrBZRqVfXo3bTzrTBKIbf7ZY7IOFIv+yKI5eXnEG3C3qg5kspkr JZqZsPCjf3JG0ISL8OwevEA9NS3aXh9t6z3wml0SIvVWdfanWf9+rxY7buOxyGllPy1y 0CoPoBaYN44AzRC0ZOgGfcoJULPlE07x887GguZROpu2P5xLfqT1YnK+iJtFcv+h6+Ds kFqNEeU5eLOi50ybuZWhmMzeM6sF8V0uYCJEmLMnO0Iei1z8MBeZvgl8G3H5wA4VYCrL 25CLm5aNRtmLYED7afTeRfRSd6W/2RpQke0RTnSdxCU79+muIdUm006Um9o/4CRn75GS KVRw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id m19si215869ejz.322.2019.02.15.09.03.12 for ; Fri, 15 Feb 2019 09:03:13 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3E46015BF; Fri, 15 Feb 2019 09:03:12 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4A48D3F557; Fri, 15 Feb 2019 09:03:09 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 02/13] x86/mm: Add p?d_large() definitions Date: Fri, 15 Feb 2019 17:02:23 +0000 Message-Id: <20190215170235.23360-3-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190215170235.23360-1-steven.price@arm.com> References: <20190215170235.23360-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Exposing the pud/pgd levels of the page tables to walk_page_range() means we may come across the exotic large mappings that come with large areas of contiguous memory (such as the kernel's linear map). Expose p?d_large() from each architecture to detect these large mappings. x86 already has these defined as inline functions, add a macro of the same name so we don't end up with the generic version too. Signed-off-by: James Morse Signed-off-by: Steven Price --- arch/x86/include/asm/pgtable.h | 3 +++ arch/x86/mm/dump_pagetables.c | 2 ++ 2 files changed, 5 insertions(+) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 2779ace16d23..3695f6acb6af 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -234,6 +234,7 @@ static inline int pmd_large(pmd_t pte) { return pmd_flags(pte) & _PAGE_PSE; } +#define pmd_large(x) pmd_large(x) #ifdef CONFIG_TRANSPARENT_HUGEPAGE static inline int pmd_trans_huge(pmd_t pmd) @@ -873,6 +874,7 @@ static inline int pud_large(pud_t pud) return 0; } #endif /* CONFIG_PGTABLE_LEVELS > 2 */ +#define pud_large(x) pud_large(x) static inline unsigned long pud_index(unsigned long address) { @@ -1214,6 +1216,7 @@ static inline bool pgdp_maps_userspace(void *__ptr) } static inline int pgd_large(pgd_t pgd) { return 0; } +#define pgd_large(x) pgd_large(x) #ifdef CONFIG_PAGE_TABLE_ISOLATION /* diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index e3cdc85ce5b6..695647dc9cb9 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -432,6 +432,7 @@ static void walk_pmd_level(struct seq_file *m, struct pg_state *st, pud_t addr, #else #define walk_pmd_level(m,s,a,e,p) walk_pte_level(m,s,__pmd(pud_val(a)),e,p) +#undef pud_large #define pud_large(a) pmd_large(__pmd(pud_val(a))) #define pud_none(a) pmd_none(__pmd(pud_val(a))) #endif @@ -503,6 +504,7 @@ static void walk_p4d_level(struct seq_file *m, struct pg_state *st, pgd_t addr, } } +#undef pgd_large #define pgd_large(a) (pgtable_l5_enabled() ? pgd_large(a) : p4d_large(__p4d(pgd_val(a)))) #define pgd_none(a) (pgtable_l5_enabled() ? pgd_none(a) : p4d_none(__p4d(pgd_val(a)))) From patchwork Fri Feb 15 17:02:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10815467 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1DC601390 for ; Fri, 15 Feb 2019 17:03:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0C7D72FC08 for ; Fri, 15 Feb 2019 17:03:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F2D442FC2F; Fri, 15 Feb 2019 17:03:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 87DA52FC08 for ; Fri, 15 Feb 2019 17:03:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8DF4E8E0006; Fri, 15 Feb 2019 12:03:18 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 88CB58E0001; Fri, 15 Feb 2019 12:03:18 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A44E8E0006; Fri, 15 Feb 2019 12:03:18 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id 25AAE8E0001 for ; Fri, 15 Feb 2019 12:03:18 -0500 (EST) Received: by mail-ed1-f69.google.com with SMTP id x15so4251183edd.2 for ; Fri, 15 Feb 2019 09:03:18 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=cgmVGKBRQKtEZdJyjA1gIFBtbXN/G8O1rrSlP5LxrbQ=; b=TVKs8Pd0nui+pmydpOMLMV5MWc6HUaxw814MBEuGGb1rdPwd261eCijcNom+86yH3t NEN2TJiWfaggMXSZPi1hqljHM/rENec43BRcwAT4FiJ0XFW9RHN3X032sxMfw+2sCp6R b4ErV2vD4hJr8ROfWvpd8/boeBRelE+9nPAZd/fG+brQZrqN8W32st0GtPSlC7zR6tcZ SU586ezxJ6FUaH8CWtIy0TsDk0KZTPDeK60ec7oe8FkYU6iLxdPPG7NSv609hWjZfZwQ 7PJGgdxZ31AF+z2+DPhcZk9CTYBbFhOdt5mPeQURv/Aj5iep6KxTmspNyXwLKBuC1uwj 8b8w== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAuYGdgAlhojg7AdqUUIjMx0U2TPPEelHXlNJkuD3fBuHAoMo4aB6 TBk4T/O8+/hFH2W5q/mckxRkVwwqEFN1xFc3t14NwAh3UVeK4vAyAsaf+lPLt7v3MlzNh7dd4+x 7METoiA3sBgETJfC+Od6sw7pLt1TemgiyADSY4g3G4STUb3A8onMJeXS8dYBMxbkQmg== X-Received: by 2002:a17:906:a3c3:: with SMTP id ca3mr7345751ejb.25.1550250197649; Fri, 15 Feb 2019 09:03:17 -0800 (PST) X-Google-Smtp-Source: AHgI3IaLag4HHIUgW/LTxlTrWys+if6hbmvRKCsMQCK8hAJXySe3xpHkDtGmiPVrcdn+YYK/qVD0 X-Received: by 2002:a17:906:a3c3:: with SMTP id ca3mr7345681ejb.25.1550250196408; Fri, 15 Feb 2019 09:03:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550250196; cv=none; d=google.com; s=arc-20160816; b=CjsfsvxQrLzDxo8fXitoXJpIFPsPIwXDiIMO/LeQlvc/Qd2YftWimAvcHbaoVonJAI f+PwQQYJjWUbZUZkjTirF8lwj90Y0XuMRgcQEzuuvSfPTKDhbQ+qjv9vbY4L4wNLQ46X bQMDxCbqN4AEaldiFrUzW9kuLzNaAd8sW4epcOUMNosNOuB3zd1iO+B9R15HghG7OlH7 X+USfystK5jKthhnaTQs42lX1XQLHUXPYHju1G7u434shL9JRg3nA3Z52tdlaTTk+h1T fZTjJxIeNgR8DzkXtbObIv6qxnVPqXfsLfVhtvdjon0wgHDQQh1gy7YgSLL7b49GfJ81 aqhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=cgmVGKBRQKtEZdJyjA1gIFBtbXN/G8O1rrSlP5LxrbQ=; b=eWf011aQJMS3iydDesn0idKd7TQQnCOjvyq+bB24jM3MQmn7B4HViU+WvbEVnPcr1k VuwyISsQ1VJZJc/h6DAjUl7PCWe6fcY8q5LJ9Zlc8Z8HpLltvOHzDepF6CJDNCdt/j5/ TjuC6N35vliRxTZ3c8N0Pb+LZ3M7UXyWDi3FqN8nZx0HAZvUxDCjZV+GIVGaKnR4aBOO fg1DABmUYIYuA08eckO/NSfU60kbjkr+xxtxnHJAZx+mIIG3XHnVFyztZeAL6s4U+Fve HXASyE+D2hP5K+MJ4fKsEzQu3EwFjkU+PeqNDYIHDgX36zDY7f5RPApqz9XcWYudmqXp dJZg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id p22si410809ejr.147.2019.02.15.09.03.16 for ; Fri, 15 Feb 2019 09:03:16 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 71631EBD; Fri, 15 Feb 2019 09:03:15 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7D4693F557; Fri, 15 Feb 2019 09:03:12 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 03/13] mm: Add generic p?d_large() macros Date: Fri, 15 Feb 2019 17:02:24 +0000 Message-Id: <20190215170235.23360-4-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190215170235.23360-1-steven.price@arm.com> References: <20190215170235.23360-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: James Morse Exposing the pud/pgd levels of the page tables to walk_page_range() means we may come across the exotic large mappings that come with large areas of contiguous memory (such as the kernel's linear map). For architectures that don't provide p?d_large() macros, provided a does nothing default. Signed-off-by: James Morse Signed-off-by: Steven Price --- include/asm-generic/pgtable.h | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h index 05e61e6c843f..7630d663cd51 100644 --- a/include/asm-generic/pgtable.h +++ b/include/asm-generic/pgtable.h @@ -1186,4 +1186,14 @@ static inline bool arch_has_pfn_modify_check(void) #define mm_pmd_folded(mm) __is_defined(__PAGETABLE_PMD_FOLDED) #endif +#ifndef pgd_large +#define pgd_large(x) 0 +#endif +#ifndef pud_large +#define pud_large(x) 0 +#endif +#ifndef pmd_large +#define pmd_large(x) 0 +#endif + #endif /* _ASM_GENERIC_PGTABLE_H */ From patchwork Fri Feb 15 17:02:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10815469 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CBB211390 for ; Fri, 15 Feb 2019 17:03:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B9CD72FC08 for ; Fri, 15 Feb 2019 17:03:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ADB632FC2F; Fri, 15 Feb 2019 17:03:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 32E902FC08 for ; Fri, 15 Feb 2019 17:03:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 55DF78E0007; Fri, 15 Feb 2019 12:03:22 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 50A6B8E0001; Fri, 15 Feb 2019 12:03:22 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F9998E0007; Fri, 15 Feb 2019 12:03:22 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id DBC038E0001 for ; Fri, 15 Feb 2019 12:03:21 -0500 (EST) Received: by mail-ed1-f71.google.com with SMTP id f11so4241824edi.5 for ; Fri, 15 Feb 2019 09:03:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=HrdRugGr2uKHHo/TiR13vk3LQ/wnulW6YNAfJqdWS6g=; b=MzqF2dqOIKIYvc6wHsXsbvlRYCBpS90JZx+zXhYB6FwC+uRlJyHpPg/vf5q11/guSB JABt9F46330VhpZP7isN6mnQAqfHiYUwh/NbSim9QxuRVradTONXIShPiYt18mjAYkKE iLthzo24mJs8Ylr3vReBfvaB1M8yAaa0WVSUDrzTxM28BcpvExSSgL3QPFrHoLJEUgXQ dmJ/AQHTOhT/7R49Iu4lq8hV0U6n74txkdeKFFxKP2j9peO6cYWGjAsWoP+aCXoRO857 h6HfRPLf2FZLbiU6koXrG5kNlEWcFDJ1ld34hmGk/3jmDCLZ/YDWsDjZfGESYEtsw45e wGMw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAubiB8ESNCGc4SpycBP4D162L7b70UkVrCq0pv/5z+LWmJREIK64 yxXg3GNa5y1RlIAes1z2mtS9uI812YZ4RHnOduMZ/YuIl/vsfvYeCK9FfogJzY0WJ0sI8GIp560 hQ4i61gvwLRent98fiooX23JCPmd9VWy+bBzPpbBvIuEMIecN3N0jyBuHTJa529Xe0w== X-Received: by 2002:a17:906:ee1:: with SMTP id x1mr7281284eji.85.1550250201377; Fri, 15 Feb 2019 09:03:21 -0800 (PST) X-Google-Smtp-Source: AHgI3IbWB2FCaQ8MET2ZoWm6lQxyL8Z58ALfHvYDprzy1ly2FA/3/XiFSGeDw5/vPVp6VXRxHV5K X-Received: by 2002:a17:906:ee1:: with SMTP id x1mr7281194eji.85.1550250199898; Fri, 15 Feb 2019 09:03:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550250199; cv=none; d=google.com; s=arc-20160816; b=XjWFY+H/o0veg5uTQ7CiKqaznfrHBo7cs1gsUGv62PPG3LFdB5vB63sIYP/lWqv4ID Um2fLMjGxGAlYAXxsJHlqqFWifCylNDb0J6IASt2SffpwfIiyY7A/v9azFH8VDB+Q+wx PJ1Z1mtmDB3oxgsged3JELeF2O98Q6a2T8GEVD9QwKhwnREcTtknDqkFPIzM8aIBt61x bJj6sa4VqVXoQ90VvqYHCUf7/SLNWeNzDhk/UbeZ6SUGm4v2elQiXjFfM5TF/IyFnKLh Q8/Si/lnhxsq+e516JDQlN08XTxTLhFSPlvaLEiB7LYMWrpgp9YqoFR0VFmwfxwLM7oS mwww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=HrdRugGr2uKHHo/TiR13vk3LQ/wnulW6YNAfJqdWS6g=; b=bxG74RvXv7hInV/M7ywrQVPGBFURaSyqy5FvlkqFoDCm3FqJ7N4DjlG81mXk2YhGph nYsY3IgAIXrQZ77DNRl3X7SCdQvuJ1oZeDUJr1sSw85TrHogVsnfF7iZrpIWaEUzgySg GR8p4wZu0Jqzuneejdi7x5uKh+rGkVQc8RhsGOST4Du4rlYvs8apn6f6e1dgVvECcwJ2 ca8076ynMRUjkFDWSfUQVfweASricguvwDcfu3NDMURlHK4V83QmT0vGAM+96Ej97nzb +bdYWWGbylGuErED+qNrEgETqBPsJ6dZtgsjlYrvCPWtEc5nc7ranDkdcP6cKPvkEUFI ccBg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id r26si1066670edb.333.2019.02.15.09.03.19 for ; Fri, 15 Feb 2019 09:03:19 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A4D441596; Fri, 15 Feb 2019 09:03:18 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B0C343F557; Fri, 15 Feb 2019 09:03:15 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 04/13] mm: pagewalk: Add p4d_entry() and pgd_entry() Date: Fri, 15 Feb 2019 17:02:25 +0000 Message-Id: <20190215170235.23360-5-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190215170235.23360-1-steven.price@arm.com> References: <20190215170235.23360-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP pgd_entry() and pud_entry() were removed by commit 0b1fbfe50006c410 ("mm/pagewalk: remove pgd_entry() and pud_entry()") because there were no users. We're about to add users so reintroduce them, along with p4d_entry() as we now have 5 levels of tables. Note that commit a00cc7d9dd93d66a ("mm, x86: add support for PUD-sized transparent hugepages") already re-added pud_entry() but with different semantics to the other callbacks. Since there have never been upstream users of this, revert the semantics back to match the other callbacks. This means pud_entry() is called for all entries, not just transparent huge pages. Signed-off-by: Steven Price --- include/linux/mm.h | 9 ++++++--- mm/pagewalk.c | 27 ++++++++++++++++----------- 2 files changed, 22 insertions(+), 14 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 80bb6408fe73..1a4b1615d012 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1412,10 +1412,9 @@ void unmap_vmas(struct mmu_gather *tlb, struct vm_area_struct *start_vma, /** * mm_walk - callbacks for walk_page_range + * @pgd_entry: if set, called for each non-empty PGD (top-level) entry + * @p4d_entry: if set, called for each non-empty P4D (1st-level) entry * @pud_entry: if set, called for each non-empty PUD (2nd-level) entry - * this handler should only handle pud_trans_huge() puds. - * the pmd_entry or pte_entry callbacks will be used for - * regular PUDs. * @pmd_entry: if set, called for each non-empty PMD (3rd-level) entry * this handler is required to be able to handle * pmd_trans_huge() pmds. They may simply choose to @@ -1435,6 +1434,10 @@ void unmap_vmas(struct mmu_gather *tlb, struct vm_area_struct *start_vma, * (see the comment on walk_page_range() for more details) */ struct mm_walk { + int (*pgd_entry)(pgd_t *pgd, unsigned long addr, + unsigned long next, struct mm_walk *walk); + int (*p4d_entry)(p4d_t *p4d, unsigned long addr, + unsigned long next, struct mm_walk *walk); int (*pud_entry)(pud_t *pud, unsigned long addr, unsigned long next, struct mm_walk *walk); int (*pmd_entry)(pmd_t *pmd, unsigned long addr, diff --git a/mm/pagewalk.c b/mm/pagewalk.c index c3084ff2569d..98373a9f88b8 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -90,15 +90,9 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, } if (walk->pud_entry) { - spinlock_t *ptl = pud_trans_huge_lock(pud, walk->vma); - - if (ptl) { - err = walk->pud_entry(pud, addr, next, walk); - spin_unlock(ptl); - if (err) - break; - continue; - } + err = walk->pud_entry(pud, addr, next, walk); + if (err) + break; } split_huge_pud(walk->vma, pud, addr); @@ -131,7 +125,12 @@ static int walk_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, break; continue; } - if (walk->pmd_entry || walk->pte_entry) + if (walk->p4d_entry) { + err = walk->p4d_entry(p4d, addr, next, walk); + if (err) + break; + } + if (walk->pud_entry || walk->pmd_entry || walk->pte_entry) err = walk_pud_range(p4d, addr, next, walk); if (err) break; @@ -157,7 +156,13 @@ static int walk_pgd_range(unsigned long addr, unsigned long end, break; continue; } - if (walk->pmd_entry || walk->pte_entry) + if (walk->pgd_entry) { + err = walk->pgd_entry(pgd, addr, next, walk); + if (err) + break; + } + if (walk->p4d_entry || walk->pud_entry || walk->pmd_entry || + walk->pte_entry) err = walk_p4d_range(pgd, addr, next, walk); if (err) break; From patchwork Fri Feb 15 17:02:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10815473 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2FC5217D5 for ; Fri, 15 Feb 2019 17:03:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1EC342FC08 for ; Fri, 15 Feb 2019 17:03:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 127162FC2F; Fri, 15 Feb 2019 17:03:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A44252FC08 for ; Fri, 15 Feb 2019 17:03:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D73FD8E0008; Fri, 15 Feb 2019 12:03:24 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CFC3C8E0001; Fri, 15 Feb 2019 12:03:24 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BEA1E8E0008; Fri, 15 Feb 2019 12:03:24 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id 69E7F8E0001 for ; Fri, 15 Feb 2019 12:03:24 -0500 (EST) Received: by mail-ed1-f72.google.com with SMTP id a21so2959166eda.3 for ; Fri, 15 Feb 2019 09:03:24 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=NXbLxkVEjkUC98nDqbwMu9XwBnfr8Fy+W9xUz87hsi8=; b=aWlJcoWIFJXMTKe9/JgiNjW04A9sSNuohtUluBSQWCuHWkTNytrXUlCxEya8SrRjCv ysNOkEvlk4ENuXbGCwbd1KFUvKppl6UDLsfmodnJvE3P05zla9FEoYtDKPnaafj03rre HonBY5OO5etaYNK3JCIP6NuIzqVVMHKUP8fpWdAxFWZeVA0j7KjS3dfjr3oifZk8IplT DD5KyX7rmHwURu5p/LBuXiaXWhu8tz9yx8LI27uN2aBTH+FpRlW7ZDtWWuTfqKpZ1fv9 P2gYjZssM83FkgxnRUpDjdhTzfnUMi0AM5SXpLqWPIbyrg0K9z40donQFpBkvF62EDe3 8IXQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAuYTjXFJ0yCwVh3RDQyeWya9wBj1rCkqAEmIAYUcl/c4EdeLZFx1 PFBAebOmHBV143XbJnH2ZJHeGP9bbjnGW2VariaSOZ8jvXFNyGITzJFG1qQrMXGDyquXO8P/E5a IXa0QYXWtboHp4uxaC+Thgm3KoBV6l87A3Hw7c8LWjx+zS93wG+9UBs7T8FYdtgwCDw== X-Received: by 2002:a50:8b26:: with SMTP id l35mr8043349edl.146.1550250203912; Fri, 15 Feb 2019 09:03:23 -0800 (PST) X-Google-Smtp-Source: AHgI3IZfAeFieHoUj7KrANvljbNoWb614R3s+Nh8Rz+d362XN54KrVQC8CtjD4zxYE3coIC+oDBd X-Received: by 2002:a50:8b26:: with SMTP id l35mr8043287edl.146.1550250202791; Fri, 15 Feb 2019 09:03:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550250202; cv=none; d=google.com; s=arc-20160816; b=waIlE0tQbn4S7ZLbQM/q6rak5nlqemSTOJ5iWaBBIQV7f1tJMQ/zaMcaV6OF8wHa6t QsBUVUUa+/39JgreZmRRXAurrZeahzrVPNqxOKucXWmOYSadsg3/XOYby6dY2N1uKpfi b02hkuWsbPGgCSi9hwA9T4EnweC/H+8xxIPbhtag5HfNXRljfxUj0FrPCfY+TLOEOPiN Vewbk8UobxD5O5yuQCDi4gDbIHxg3K3uIG0cmWZijy8aER5LQj7m+M5yuR1q0y9h/lHZ GZgt7iUf74HP8drulx2ueahJqkV2oTKcG115d1oOsNItJvUNwXF8M722vh17cIqJMXLP 7ItQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=NXbLxkVEjkUC98nDqbwMu9XwBnfr8Fy+W9xUz87hsi8=; b=F3RYZds8BVbuBKTnCrJGSZXKV7qWnnOEKq+qry3hHyGFOuidix5tET5cDTAYf3Rik7 SJqafpVHaG7yPET8I4aHjoG6Jqv6ML1/iIfTwFMjUiIXAIsMTlnKjOUqSMP88gWEC5Pj MhXWLnHno0G7dHpFZffhSkOuerusNAgfpjJD7NaTCMY0+VR1N+FdoxzZ5dX+a576wR2e HfjfISyxsSrmMJC7ZMpGWYCIAKKtFrSWrAxZNgN2fkFE45ejwh23etkwvyIM+MQfoP9k uelyRhgQHWAIhduoA3e2b+7ILFAQFjaTph64b1IRwUf4joOuzEcAgI8aiHpmvlASmZDx KaRA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id y59si433330edy.301.2019.02.15.09.03.22 for ; Fri, 15 Feb 2019 09:03:22 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D806115BF; Fri, 15 Feb 2019 09:03:21 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id E40033F557; Fri, 15 Feb 2019 09:03:18 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 05/13] mm: pagewalk: Allow walking without vma Date: Fri, 15 Feb 2019 17:02:26 +0000 Message-Id: <20190215170235.23360-6-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190215170235.23360-1-steven.price@arm.com> References: <20190215170235.23360-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Since 48684a65b4e3: "mm: pagewalk: fix misbehavior of walk_page_range for vma(VM_PFNMAP)", page_table_walk() will report any kernel area as a hole, because it lacks a vma. This means each arch has re-implemented page table walking when needed, for example in the per-arch ptdump walker. Remove the requirement to have a vma except when trying to split huge pages. Signed-off-by: Steven Price --- mm/pagewalk.c | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/mm/pagewalk.c b/mm/pagewalk.c index 98373a9f88b8..dac0c848b458 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -36,7 +36,7 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, do { again: next = pmd_addr_end(addr, end); - if (pmd_none(*pmd) || !walk->vma) { + if (pmd_none(*pmd)) { if (walk->pte_hole) err = walk->pte_hole(addr, next, walk); if (err) @@ -59,9 +59,14 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, if (!walk->pte_entry) continue; - split_huge_pmd(walk->vma, pmd, addr); - if (pmd_trans_unstable(pmd)) - goto again; + if (walk->vma) { + split_huge_pmd(walk->vma, pmd, addr); + if (pmd_trans_unstable(pmd)) + goto again; + } else if (pmd_large(*pmd)) { + continue; + } + err = walk_pte_range(pmd, addr, next, walk); if (err) break; @@ -81,7 +86,7 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, do { again: next = pud_addr_end(addr, end); - if (pud_none(*pud) || !walk->vma) { + if (pud_none(*pud)) { if (walk->pte_hole) err = walk->pte_hole(addr, next, walk); if (err) @@ -95,9 +100,13 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, break; } - split_huge_pud(walk->vma, pud, addr); - if (pud_none(*pud)) - goto again; + if (walk->vma) { + split_huge_pud(walk->vma, pud, addr); + if (pud_none(*pud)) + goto again; + } else if (pud_large(*pud)) { + continue; + } if (walk->pmd_entry || walk->pte_entry) err = walk_pmd_range(pud, addr, next, walk); From patchwork Fri Feb 15 17:02:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10815475 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 512B3922 for ; Fri, 15 Feb 2019 17:03:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3E7D42FC08 for ; Fri, 15 Feb 2019 17:03:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 326F32FC2F; Fri, 15 Feb 2019 17:03:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7EC282FC08 for ; Fri, 15 Feb 2019 17:03:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 819C08E0009; Fri, 15 Feb 2019 12:03:28 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7CA928E0001; Fri, 15 Feb 2019 12:03:28 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 692458E0009; Fri, 15 Feb 2019 12:03:28 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id 130F38E0001 for ; Fri, 15 Feb 2019 12:03:28 -0500 (EST) Received: by mail-ed1-f69.google.com with SMTP id f11so4241998edi.5 for ; Fri, 15 Feb 2019 09:03:28 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=igub7qrpJgjRv+2v35QNpXcDRvocVxsP5pFwSPlipmo=; b=E4h/Oz/bgeCZI4rhfYwQqmFoNN92mvBdlCL4I4I63qWs54nlEXhb3W24mq6Aff1kBx cK0npFrWJ2fgGikglcPRcLTAggpYj2zXTXR3luVUHITogERzAIPpHEkFnL6jzOJ806iw vVMROt55oL8VXhe6JAKUoACSt27QKzdLYm/JoMZHWYdY0z4eTSpFINQ7Z7stRM2wuPLd 2G7ME/T/9mtEyAni8HVmjq7Vr6feOaqbDBBYaKemVBm2SCXzg8m/JriqMClBg6yt6uJF +RWAz19cJj1LRcQx4iXTCHZEPC8yiq3nHtiZ8AONzrNvnEq73B8BCKcPx/sG2H/FvX/+ dG+Q== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAubVGporOQ8YMQyU5sgEp7dKavRBY+cwfWsu+4P7mDQPT85tC7Ah vnqRSUF+rw3gIvTBmsNLktazx+id/hVMb8qQNRfqKq9imwzRqzHcdkElwNorvWTgWvfsKSCyQvf 8TUhg73+uPJicpXpYwl2VDE12aNP4+5OdRm+7YB6gNEVtoUHz2piNuyZhtFnrrpgXEQ== X-Received: by 2002:a17:906:3b8e:: with SMTP id u14mr7406390ejf.130.1550250207548; Fri, 15 Feb 2019 09:03:27 -0800 (PST) X-Google-Smtp-Source: AHgI3IaDBAJ0q5YL4mKmCwdufgp9UR3oZyaE28PCbVB4Z0ctwjZFyCujhCgA044GX5GUf9sT2KTE X-Received: by 2002:a17:906:3b8e:: with SMTP id u14mr7406320ejf.130.1550250206218; Fri, 15 Feb 2019 09:03:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550250206; cv=none; d=google.com; s=arc-20160816; b=ZXpB4Zid8Z458YH7a1Vj4OIpOgkpQJqpyPsz4TQzGnYlRFUTZDlOLSE7/CNDH/Dt2s usxil/gaQ54Lb9mFxA1E6kN++eq+fUZlGGNGUbqnAd61/IJhGsSqmhYzCZANTxoKefu9 CrKAj5tzmP7oZ+4iq9ddFG70g3ZCH2cuFfrN6q7P9v3lgDKoj+jrRIxxDh22G+HLlKiu YUa2a0powlawFcCp4avFlx072dNTwaV6wtiXLSbjCqN1xJrHgEitD5y5sZrNatda86jp seUKqrfMS/+ToE5eQlHEEiqGfaPMdlXu5wvu6cyTgzvAxQX5o2b7HFfL90gEiWHLuXs1 mWVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=igub7qrpJgjRv+2v35QNpXcDRvocVxsP5pFwSPlipmo=; b=nnXQmDTQyo52zr+TXpt2xIU5mBA5Nj/6L5pvVECEDwIV0uxSXg+rmsp6KhlBitdIh1 CQUy/bcr5tNYI2RUbD3gupp99H17rcoEZyVHpjq2SVBeR5s5JI4nkKv+fKMqEdcdDoGv AlyV+zc1RHSJx4P4s/qmqdxOfBle0ZexjF0q+0CQ5nq4WLHxNrEmDa8ubkMq5anHGZkH kkbvP4/t4IETrXHobFWixXaXGxDh1pNW9Xf3grzTYdEjBP+SNl9Yn91XxL8euoTWPlhZ VhESq2hU7gtETKs13SBAggmBmlXPvjey+Er1o65mxOdtTlTqvUTk2fEYaDX7a3HYSK22 mkUw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id s7si689087eju.171.2019.02.15.09.03.25 for ; Fri, 15 Feb 2019 09:03:26 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1763AEBD; Fri, 15 Feb 2019 09:03:25 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 234B13F557; Fri, 15 Feb 2019 09:03:21 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 06/13] mm: pagewalk: Add 'depth' parameter to pte_hole Date: Fri, 15 Feb 2019 17:02:27 +0000 Message-Id: <20190215170235.23360-7-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190215170235.23360-1-steven.price@arm.com> References: <20190215170235.23360-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The pte_hole() callback is called at multiple levels of the page tables. Code dumping the kernel page tables needs to know what at what depth the missing entry is. Add this is an extra parameter to pte_hole(). When the depth isn't know (e.g. processing a vma) then -1 is passed. Note that depth starts at 0 for a PGD so that PUD/PMD/PTE retain their natural numbers as levels 2/3/4. Signed-off-by: Steven Price --- fs/proc/task_mmu.c | 4 ++-- include/linux/mm.h | 5 +++-- mm/hmm.c | 2 +- mm/migrate.c | 1 + mm/mincore.c | 1 + mm/pagewalk.c | 16 ++++++++++------ 6 files changed, 18 insertions(+), 11 deletions(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index f0ec9edab2f3..91131cd4e9e0 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -474,7 +474,7 @@ static void smaps_account(struct mem_size_stats *mss, struct page *page, #ifdef CONFIG_SHMEM static int smaps_pte_hole(unsigned long addr, unsigned long end, - struct mm_walk *walk) + __always_unused int depth, struct mm_walk *walk) { struct mem_size_stats *mss = walk->private; @@ -1203,7 +1203,7 @@ static int add_to_pagemap(unsigned long addr, pagemap_entry_t *pme, } static int pagemap_pte_hole(unsigned long start, unsigned long end, - struct mm_walk *walk) + __always_unused int depth, struct mm_walk *walk) { struct pagemapread *pm = walk->private; unsigned long addr = start; diff --git a/include/linux/mm.h b/include/linux/mm.h index 1a4b1615d012..0418a018d7b3 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1420,7 +1420,8 @@ void unmap_vmas(struct mmu_gather *tlb, struct vm_area_struct *start_vma, * pmd_trans_huge() pmds. They may simply choose to * split_huge_page() instead of handling it explicitly. * @pte_entry: if set, called for each non-empty PTE (4th-level) entry - * @pte_hole: if set, called for each hole at all levels + * @pte_hole: if set, called for each hole at all levels, + * depth is -1 if not known * @hugetlb_entry: if set, called for each hugetlb entry * @test_walk: caller specific callback function to determine whether * we walk over the current vma or not. Returning 0 @@ -1445,7 +1446,7 @@ struct mm_walk { int (*pte_entry)(pte_t *pte, unsigned long addr, unsigned long next, struct mm_walk *walk); int (*pte_hole)(unsigned long addr, unsigned long next, - struct mm_walk *walk); + int depth, struct mm_walk *walk); int (*hugetlb_entry)(pte_t *pte, unsigned long hmask, unsigned long addr, unsigned long next, struct mm_walk *walk); diff --git a/mm/hmm.c b/mm/hmm.c index a04e4b810610..e3e6b8fda437 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -440,7 +440,7 @@ static void hmm_range_need_fault(const struct hmm_vma_walk *hmm_vma_walk, } static int hmm_vma_walk_hole(unsigned long addr, unsigned long end, - struct mm_walk *walk) + __always_unused int depth, struct mm_walk *walk) { struct hmm_vma_walk *hmm_vma_walk = walk->private; struct hmm_range *range = hmm_vma_walk->range; diff --git a/mm/migrate.c b/mm/migrate.c index d4fd680be3b0..8b62a9fecb5c 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2121,6 +2121,7 @@ struct migrate_vma { static int migrate_vma_collect_hole(unsigned long start, unsigned long end, + __always_unused int depth, struct mm_walk *walk) { struct migrate_vma *migrate = walk->private; diff --git a/mm/mincore.c b/mm/mincore.c index 218099b5ed31..c4edbc688241 100644 --- a/mm/mincore.c +++ b/mm/mincore.c @@ -104,6 +104,7 @@ static int __mincore_unmapped_range(unsigned long addr, unsigned long end, } static int mincore_unmapped_range(unsigned long addr, unsigned long end, + __always_unused int depth, struct mm_walk *walk) { walk->private += __mincore_unmapped_range(addr, end, diff --git a/mm/pagewalk.c b/mm/pagewalk.c index dac0c848b458..b8038f852f06 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -38,7 +38,7 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, next = pmd_addr_end(addr, end); if (pmd_none(*pmd)) { if (walk->pte_hole) - err = walk->pte_hole(addr, next, walk); + err = walk->pte_hole(addr, next, 3, walk); if (err) break; continue; @@ -88,7 +88,7 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, next = pud_addr_end(addr, end); if (pud_none(*pud)) { if (walk->pte_hole) - err = walk->pte_hole(addr, next, walk); + err = walk->pte_hole(addr, next, 2, walk); if (err) break; continue; @@ -123,13 +123,17 @@ static int walk_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, p4d_t *p4d; unsigned long next; int err = 0; + /* If the p4ds are actually just pgds then we should report a depth + * of 0 not 1 (as a missing entry is really a missing pgd + */ + int depth = (PTRS_PER_P4D == 1)?0:1; p4d = p4d_offset(pgd, addr); do { next = p4d_addr_end(addr, end); if (p4d_none_or_clear_bad(p4d)) { if (walk->pte_hole) - err = walk->pte_hole(addr, next, walk); + err = walk->pte_hole(addr, next, depth, walk); if (err) break; continue; @@ -160,7 +164,7 @@ static int walk_pgd_range(unsigned long addr, unsigned long end, next = pgd_addr_end(addr, end); if (pgd_none_or_clear_bad(pgd)) { if (walk->pte_hole) - err = walk->pte_hole(addr, next, walk); + err = walk->pte_hole(addr, next, 0, walk); if (err) break; continue; @@ -206,7 +210,7 @@ static int walk_hugetlb_range(unsigned long addr, unsigned long end, if (pte) err = walk->hugetlb_entry(pte, hmask, addr, next, walk); else if (walk->pte_hole) - err = walk->pte_hole(addr, next, walk); + err = walk->pte_hole(addr, next, -1, walk); if (err) break; @@ -249,7 +253,7 @@ static int walk_page_test(unsigned long start, unsigned long end, if (vma->vm_flags & VM_PFNMAP) { int err = 1; if (walk->pte_hole) - err = walk->pte_hole(start, end, walk); + err = walk->pte_hole(start, end, -1, walk); return err ? err : 1; } return 0; From patchwork Fri Feb 15 17:02:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10815477 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4D3C4922 for ; Fri, 15 Feb 2019 17:03:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3B8FF2FC1C for ; Fri, 15 Feb 2019 17:03:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2BD5A2FC34; Fri, 15 Feb 2019 17:03:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A840D2FC1C for ; Fri, 15 Feb 2019 17:03:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A65418E000A; Fri, 15 Feb 2019 12:03:31 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9F0278E0001; Fri, 15 Feb 2019 12:03:31 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9046C8E000A; Fri, 15 Feb 2019 12:03:31 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id 38E2A8E0001 for ; Fri, 15 Feb 2019 12:03:31 -0500 (EST) Received: by mail-ed1-f71.google.com with SMTP id c53so4274898edc.9 for ; Fri, 15 Feb 2019 09:03:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=haQZE3AEpmV1qh2WAim70N1byJoD0iBoN23JNUzPnMs=; b=O7oW39fegNFOqzXBcIXBHVJbb2OAZVWFFoSl2NIAcqW70o5ud9xjn0pCHALeQesjN9 UuwqVHi3g3r9orVw0ZtdCQQyUkoMhAQpoGzpb9oDYGmOyHM7QdkytSm29br73z7BU3ch CT0hSKmV01F3cGqUAZmKiyZyreD5CC1ThjoJNkHyAQSXA8v7xlL8i464oMofw7kKCKbw SIJ0AzAsb4zEmvQDhr/bcLYINi+ZnPxnwcvUl7d1EU/tqY6+iw0rjfbBQJvnvjqz8u9o 9yJhgqt+R0sKOOw3+YfFx0rSi36Rh8/POtWFXMXaBqYOzXoLiUDvcUK+yKy+csdqKGQw RjmA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAua6C6hFz051GOs91g1B3j8eBpHZhYxqUoG79Xs1Ynh+ys1ULg5d fgcZTheyBjX+styjvyXcNPnvsSA2l4Z6ekaO0/Vl2s6NEgZFnocNZVXITrnVisRKcfih/qnDJpK MkBhSokbrU5Kpcn0vPqKfth+BF5bFgSeosofU+OxBvjg45IiwbR6h2geYCdvltRvpLA== X-Received: by 2002:aa7:dc51:: with SMTP id g17mr2427239edu.240.1550250210700; Fri, 15 Feb 2019 09:03:30 -0800 (PST) X-Google-Smtp-Source: AHgI3IZFl9XJkb0sWCML8KuW9VUyl2Atg46viYysT03D9i9R8lurPfwibTSII0oUBQ9uJ5zoy6QT X-Received: by 2002:aa7:dc51:: with SMTP id g17mr2427152edu.240.1550250209363; Fri, 15 Feb 2019 09:03:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550250209; cv=none; d=google.com; s=arc-20160816; b=P9o2g+kFsez0ITG13/gsdt8Ki7yutdDexjePxErCW7k6wZKofhkw//OSq1PlyBl24p NI0KJ14oT74lK9mtK0lysGgPONmUSgNwp5zs+7p/0LPTG+jVOCo0lpuVQOqIKtQhwMgD 0ZWbrH730EyTb/GhMQ00ucbeg0A3s1BNkgQ/+wmgvoGKqg8HgJ5r7CY/DTOKvT/FvXRE 1t2PlCAjqQ96nYWl63wNl3SSYh8VHKDLlLLbDzkO1huu1aW3yriM9HZrxEwILbjcVlvN cZkgaU3gSRqwe8JQLcdbxadtEZtYxh9zO6BiaPQzRMqhqrlQMNBbEUzcowsNL+CEI3ue mPVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=haQZE3AEpmV1qh2WAim70N1byJoD0iBoN23JNUzPnMs=; b=EvAkbuFKqraRvpjdQI9r5OwVL3SNKZxmTbro9hqSl/yUNQ9KisJJfajS/cqA192+IE llDcOigRip7pkIqoamH6Y1REDF2EYMsdlwqDMXyKRUTPXIGYJ0CIX0XOMCwkXVTT5LrR O/9zMi8pN4Arv6PJHXaIqirlKSEgp0qMCFNpI+iEMWcEJ/EzteCX6VBcUiE3MWOBtzwq P6NZF5+4j+Mjl4hPINGaK2NOJrMVmfueOihuagiiqCzhrhHBt0PuqXSqVM5rTqLe7AiL t+YL39eDaP52zoI5YcTBrerlaNWaxULvAWDjl4wl8q6XKNGQVcN5+av+Nh+SUbuHj4RI hRsA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id gv4si88050ejb.226.2019.02.15.09.03.28 for ; Fri, 15 Feb 2019 09:03:29 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4A4EC1596; Fri, 15 Feb 2019 09:03:28 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 568A13F557; Fri, 15 Feb 2019 09:03:25 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 07/13] mm: pagewalk: Add test_p?d callbacks Date: Fri, 15 Feb 2019 17:02:28 +0000 Message-Id: <20190215170235.23360-8-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190215170235.23360-1-steven.price@arm.com> References: <20190215170235.23360-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP It is useful to be able to skip parts of the page table tree even when walking without VMAs. Add test_p?d callbacks similar to test_walk but which are called just before a table at that level is walked. If the callback returns non-zero then the entire table is skipped. Signed-off-by: Steven Price --- include/linux/mm.h | 11 +++++++++++ mm/pagewalk.c | 24 ++++++++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 0418a018d7b3..0e4c9b6b554d 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1428,6 +1428,11 @@ void unmap_vmas(struct mmu_gather *tlb, struct vm_area_struct *start_vma, * value means "do page table walk over the current vma," * and a negative one means "abort current page table walk * right now." 1 means "skip the current vma." + * @test_pmd: similar to test_walk(), but called for every pmd. + * @test_pud: similar to test_walk(), but called for every pud. + * @test_p4d: similar to test_walk(), but called for every p4d. + * Returning 0 means walk this part of the page tables, + * returning 1 means to skip this range. * @mm: mm_struct representing the target process of page table walk * @vma: vma currently walked (NULL if walking outside vmas) * @private: private data for callbacks' usage @@ -1452,6 +1457,12 @@ struct mm_walk { struct mm_walk *walk); int (*test_walk)(unsigned long addr, unsigned long next, struct mm_walk *walk); + int (*test_pmd)(unsigned long addr, unsigned long next, + pmd_t *pmd_start, struct mm_walk *walk); + int (*test_pud)(unsigned long addr, unsigned long next, + pud_t *pud_start, struct mm_walk *walk); + int (*test_p4d)(unsigned long addr, unsigned long next, + p4d_t *p4d_start, struct mm_walk *walk); struct mm_struct *mm; struct vm_area_struct *vma; void *private; diff --git a/mm/pagewalk.c b/mm/pagewalk.c index b8038f852f06..bce7571c6563 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -32,6 +32,14 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, unsigned long next; int err = 0; + if (walk->test_pmd) { + err = walk->test_pmd(addr, end, pmd_offset(pud, 0), walk); + if (err < 0) + return err; + if (err > 0) + return 0; + } + pmd = pmd_offset(pud, addr); do { again: @@ -82,6 +90,14 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, unsigned long next; int err = 0; + if (walk->test_pud) { + err = walk->test_pud(addr, end, pud_offset(p4d, 0), walk); + if (err < 0) + return err; + if (err > 0) + return 0; + } + pud = pud_offset(p4d, addr); do { again: @@ -128,6 +144,14 @@ static int walk_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, */ int depth = (PTRS_PER_P4D == 1)?0:1; + if (walk->test_p4d) { + err = walk->test_p4d(addr, end, p4d_offset(pgd, 0), walk); + if (err < 0) + return err; + if (err > 0) + return 0; + } + p4d = p4d_offset(pgd, addr); do { next = p4d_addr_end(addr, end); From patchwork Fri Feb 15 17:02:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10815479 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EB462922 for ; Fri, 15 Feb 2019 17:03:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D95162FC1C for ; Fri, 15 Feb 2019 17:03:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CD30D2FC34; Fri, 15 Feb 2019 17:03:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 513BB2FC1C for ; Fri, 15 Feb 2019 17:03:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6E0D68E000B; Fri, 15 Feb 2019 12:03:35 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 691BC8E0001; Fri, 15 Feb 2019 12:03:35 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 580778E000B; Fri, 15 Feb 2019 12:03:35 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id F267D8E0001 for ; Fri, 15 Feb 2019 12:03:34 -0500 (EST) Received: by mail-ed1-f69.google.com with SMTP id u19so4147355eds.12 for ; Fri, 15 Feb 2019 09:03:34 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=+F2lxaeFDvQ8eJHjFbD9vB/FESYGJ6cBcX+b4/bW6NY=; b=nl8WQ59ihu9OAD0C3BsMSN5GwbwkhV8Jh9JisyWoPjNqRlIJ3myC/EsE36GXaKUS2U HztxWqYi7F4YUscudjr+4CVC4RnP1QmfQAvEsdTV89LPFzu2uUqzk8PLxo1lQtgQaZ38 zL4U+jN3sB/P7pFszPPuevxaCVrNwevH2x8NS6W1i1gPax3Slq5RusPiXasCR2NrCoqx qHsemtPKbUtCyFRUTHhn+eeA4bwtxzKJmDhoF0b6je2HsCh1b0LZ7iv9FlrpuIJ4oPNn RDKenaKOeJQhldKqlfPRzngba3oxoMk8cXfw/oQInQnXV59V72p+iz1RNddSHWhsOYXK iSBA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAubK/spHCC1xUnCTd5OEpmceYtZ587/85kfvhjfP4J4U/w/jM74C 8GnPk6xio+yeK8Cf+lhi1ZZ9aETPMHEbgib9cv8QJ26hdIE4KnIlfbph9LPBe5wVb7maGRmLuyQ LkTPWaYOFZ5Xr9Q9sU9PdizEs3//r/fdG8SpVccUgJfe88sHsMB2XEUcD7NRhGlY9VA== X-Received: by 2002:a17:906:3d69:: with SMTP id r9mr3216687ejf.92.1550250214464; Fri, 15 Feb 2019 09:03:34 -0800 (PST) X-Google-Smtp-Source: AHgI3IbKSv5CYlx2/ZuJdiDkb7Irk76FDUM6kdNc4z8zQLjFYGR8jVkTadP6MkJDU6UrBIFYmxyS X-Received: by 2002:a17:906:3d69:: with SMTP id r9mr3216602ejf.92.1550250212788; Fri, 15 Feb 2019 09:03:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550250212; cv=none; d=google.com; s=arc-20160816; b=lUAY9tcbJtRJlZQq05WKDkszA6oRcXsf4zWJ7r9LfmhBk124CfG/AcyjvjYpr7xV27 Y9frp0CMvjmVfmRkOub7Zav2bNiDEt6l0Bp2veR4Lv2d8ta48DYklSn/SnH8ud6dODhF zJR9Q/jDBa87LU/h6Z+MDQ9TYYMOh7re7yE0aLS3331GIsAskeOg8q7Mho4KP1JoG7B8 jyE+sfkaVZHcmuULkf9NwHZc6IdTGwteTiqEDhlqEEsDKBfO9Ww2JR+YnOSWjlOS9vjj H/pNIFg6JMn2djxTPhZoybGWyu61O0MU6+nNia6gthDHLUb4QGXtRPx6pztDLvDJ2+Rl U9zA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=+F2lxaeFDvQ8eJHjFbD9vB/FESYGJ6cBcX+b4/bW6NY=; b=L1fYX39wJ+4BbOoK/jYSTqVSWfjvpKibp5jt89TEC4SoLiv9hkzybBhUojEGSVu+eF xYiRYUlleS1a2XVh7NcWeCbQFFXjYz9XKd9AHwPH1XXT69dsmW5vyqRF2valRvsRjHGo kjSMB3ThuLJoyCWB9+zkNyaqtKMlQdKmCGZHuRyDl4nyf+0b+2jDOy2hgQ8D8DSRRwaE LrOdZYFDrUjFMW9BE/MWDT0r4I6xT8cYGepaYded0YtKaCIXmwM5ibRPIMzYM7FGfKbm A7IR9eIrwwDiMe6NQ0wDyCCpzpHfHnF1y8BzSOaevyV0FoyRRRI2tUL2NwsV+85dQAcN EWGg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id r53si236380edd.365.2019.02.15.09.03.32 for ; Fri, 15 Feb 2019 09:03:32 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7D43A1650; Fri, 15 Feb 2019 09:03:31 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 89BB73F557; Fri, 15 Feb 2019 09:03:28 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 08/13] arm64: mm: Convert mm/dump.c to use walk_page_range() Date: Fri, 15 Feb 2019 17:02:29 +0000 Message-Id: <20190215170235.23360-9-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190215170235.23360-1-steven.price@arm.com> References: <20190215170235.23360-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Now walk_page_range() can walk kernel page tables, we can switch the arm64 ptdump code over to using it, simplifying the code. Signed-off-by: Steven Price --- arch/arm64/mm/dump.c | 108 +++++++++++++++++++++---------------------- 1 file changed, 53 insertions(+), 55 deletions(-) diff --git a/arch/arm64/mm/dump.c b/arch/arm64/mm/dump.c index 99bb8facb5cb..ee0bc1441dd0 100644 --- a/arch/arm64/mm/dump.c +++ b/arch/arm64/mm/dump.c @@ -286,73 +286,71 @@ static void note_page(struct pg_state *st, unsigned long addr, unsigned level, } -static void walk_pte(struct pg_state *st, pmd_t *pmdp, unsigned long start, - unsigned long end) +static int pud_entry(pud_t *pud, unsigned long addr, + unsigned long next, struct mm_walk *walk) { - unsigned long addr = start; - pte_t *ptep = pte_offset_kernel(pmdp, start); + struct pg_state *st = walk->private; + pud_t val = READ_ONCE(*pud); - do { - note_page(st, addr, 4, READ_ONCE(pte_val(*ptep))); - } while (ptep++, addr += PAGE_SIZE, addr != end); + if (pud_table(val)) + return 0; + + note_page(st, addr, 2, pud_val(val)); + + return 0; } -static void walk_pmd(struct pg_state *st, pud_t *pudp, unsigned long start, - unsigned long end) +static int pmd_entry(pmd_t *pmd, unsigned long addr, + unsigned long next, struct mm_walk *walk) { - unsigned long next, addr = start; - pmd_t *pmdp = pmd_offset(pudp, start); - - do { - pmd_t pmd = READ_ONCE(*pmdp); - next = pmd_addr_end(addr, end); - - if (pmd_none(pmd) || pmd_sect(pmd)) { - note_page(st, addr, 3, pmd_val(pmd)); - } else { - BUG_ON(pmd_bad(pmd)); - walk_pte(st, pmdp, addr, next); - } - } while (pmdp++, addr = next, addr != end); + struct pg_state *st = walk->private; + pmd_t val = READ_ONCE(*pmd); + + if (pmd_table(val)) + return 0; + + note_page(st, addr, 3, pmd_val(val)); + + return 0; } -static void walk_pud(struct pg_state *st, pgd_t *pgdp, unsigned long start, - unsigned long end) +static int pte_entry(pte_t *pte, unsigned long addr, + unsigned long next, struct mm_walk *walk) { - unsigned long next, addr = start; - pud_t *pudp = pud_offset(pgdp, start); - - do { - pud_t pud = READ_ONCE(*pudp); - next = pud_addr_end(addr, end); - - if (pud_none(pud) || pud_sect(pud)) { - note_page(st, addr, 2, pud_val(pud)); - } else { - BUG_ON(pud_bad(pud)); - walk_pmd(st, pudp, addr, next); - } - } while (pudp++, addr = next, addr != end); + struct pg_state *st = walk->private; + pte_t val = READ_ONCE(*pte); + + note_page(st, addr, 4, pte_val(val)); + + return 0; +} + +static int pte_hole(unsigned long addr, unsigned long next, int depth, + struct mm_walk *walk) +{ + struct pg_state *st = walk->private; + + note_page(st, addr, depth+1, 0); + + return 0; } static void walk_pgd(struct pg_state *st, struct mm_struct *mm, - unsigned long start) + unsigned long start) { - unsigned long end = (start < TASK_SIZE_64) ? TASK_SIZE_64 : 0; - unsigned long next, addr = start; - pgd_t *pgdp = pgd_offset(mm, start); - - do { - pgd_t pgd = READ_ONCE(*pgdp); - next = pgd_addr_end(addr, end); - - if (pgd_none(pgd)) { - note_page(st, addr, 1, pgd_val(pgd)); - } else { - BUG_ON(pgd_bad(pgd)); - walk_pud(st, pgdp, addr, next); - } - } while (pgdp++, addr = next, addr != end); + struct mm_walk walk = { + .mm = mm, + .private = st, + .pud_entry = pud_entry, + .pmd_entry = pmd_entry, + .pte_entry = pte_entry, + .pte_hole = pte_hole + }; + down_read(&mm->mmap_sem); + walk_page_range(start, start | (((unsigned long)PTRS_PER_PGD << + PGDIR_SHIFT) - 1), + &walk); + up_read(&mm->mmap_sem); } void ptdump_walk_pgd(struct seq_file *m, struct ptdump_info *info) From patchwork Fri Feb 15 17:02:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10815481 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F1C631390 for ; Fri, 15 Feb 2019 17:03:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E06182FC1C for ; Fri, 15 Feb 2019 17:03:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D44842FC34; Fri, 15 Feb 2019 17:03:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1BD182FC2F for ; Fri, 15 Feb 2019 17:03:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C9FE8E000C; Fri, 15 Feb 2019 12:03:38 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 99EC28E0001; Fri, 15 Feb 2019 12:03:38 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 890BF8E000C; Fri, 15 Feb 2019 12:03:38 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id 290D98E0001 for ; Fri, 15 Feb 2019 12:03:38 -0500 (EST) Received: by mail-ed1-f72.google.com with SMTP id y91so4216215edy.21 for ; Fri, 15 Feb 2019 09:03:38 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=RIFopvQ+pFnZsqu/pJF26vPVtmQkZ8IgAGItIiM7xcI=; b=jZva/Bgjh8PVVaYzsZBb8Pl6Khbaz9XUPbX0MDTstIgZWOsbFeLaeeDKjOf71XaKI2 A1LptGmbTCvke1BB208I8EI37QoJrBiyhO9I/EY/TbEpd8O4Do/kyvMcrdiFZBSFeWuP hP/xK7VBqp5hvWg4Tj82blCwvIKgeMM+Hb0Uh/v5MG1w8VbPe62sXhgMSYAqLbLxul8Q 95K2bpbfTgONxP7fFfhO+0C/uDy6V1ty29f4KvY4vVdFzLeycMsgElwsl129pS1VkKVU IyzPlR1p22eVNTfFyMQ1W4vb9v630zSHEcxhyewlwPxQPWtl2ljlIBIiy9EBsNJkNhk2 jhqQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAuahgLdFjcx6A8Lnaoy6/6rAyAnKgwF+G6VbXGyJ6lbx20yikMmb Ce9z66mu6DTYLXPkJvWRRN+0FKmtxpf72Cr47QmlaGZTJ4jsDQq33fPwLuIedtBmaC5LFxg9dum ND9Ckq0icj5GNsabvAx2wqZdjD8G5W7An1SvZFXlzeIi5nu+nV+UsXbtZxt/bvsG09g== X-Received: by 2002:a17:906:4bc8:: with SMTP id x8mr7478870ejv.6.1550250217613; Fri, 15 Feb 2019 09:03:37 -0800 (PST) X-Google-Smtp-Source: AHgI3IaHUe2S4WahG3CuYPWDu5uAF7GWuDXlmRT8mfKrahurci6AL4PK3ysPmNNYanCNNu61fCiF X-Received: by 2002:a17:906:4bc8:: with SMTP id x8mr7478790ejv.6.1550250215954; Fri, 15 Feb 2019 09:03:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550250215; cv=none; d=google.com; s=arc-20160816; b=FaXck4JyhOMbn4zaJYzvvDMRbTPh4tVtj/YWiDC8flFje3/DqdSyo1x7h7uxMEgEB6 /1H/5WjECNdK9NZgdVsKo/Fy0No9BbcYZ6HqLH8CpwK/wMH6D24UiExUTuh4tMZoMbvH J6P73crtcU7UiooHAxKATofOJQADeDj6IzWuTd0iy8dKM/zGsokzXNEBOfExhW0/cBSQ IZeBv8sI3cUdMe1iLsPvtjnkwa6pECxdiD229+jJdw4OGdI/689g8n1Zj5S9JmRhEtQj Dz7eP6qjXab/Mc87yAqHipUkjZU/vXLI8+WqSQIirvkgMIUnoyX9TQYXbu2KMdlwrrZa DsrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=RIFopvQ+pFnZsqu/pJF26vPVtmQkZ8IgAGItIiM7xcI=; b=ADo7468rwgVqfE3Jzx2XnEB4EKTwOYmooaEcqn8sT9EEMcwVwhhKWlvHchyYKrOOyw Nn6PMtepZzBTG4HCkNRaPeRC+t9+EYlABNuWQKiL/pUajemMM3alsIwgijMxLmlSoJla EPX2oo1flIZDZWYnltVKH0S3A89rxTloCYa+tyU4d2hIzYCDA8e/4Dngkb6sNBlN9Kfh Ntjq3/wiln8kgubUr6hvLyeG13OgthIr7mUtxiJV2w1oDpxG6pfsLEIij3fxW8yzcqgL P0+w9MToMkHLuVNTUzEci4Yb+SS2SIPS0xocq9plWXTvvTaG+9AG/Ho/4D2heYF27ron +58g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id i16si2559128ejr.10.2019.02.15.09.03.35 for ; Fri, 15 Feb 2019 09:03:35 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D3D2215BF; Fri, 15 Feb 2019 09:03:34 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BCBF33F557; Fri, 15 Feb 2019 09:03:31 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 09/13] x86/mm: Point to struct seq_file from struct pg_state Date: Fri, 15 Feb 2019 17:02:30 +0000 Message-Id: <20190215170235.23360-10-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190215170235.23360-1-steven.price@arm.com> References: <20190215170235.23360-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP mm/dump_pagetables.c passes both struct seq_file and struct pg_state down the chain of walk_*_level() functions to be passed to note_page(). Instead place the struct seq_file in struct pg_state and access it from struct pg_state (which is private to this file) in note_page(). Signed-off-by: Steven Price --- arch/x86/mm/dump_pagetables.c | 69 ++++++++++++++++++----------------- 1 file changed, 35 insertions(+), 34 deletions(-) diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index 695647dc9cb9..ad6986c9e8e7 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -40,6 +40,7 @@ struct pg_state { bool to_dmesg; bool check_wx; unsigned long wx_pages; + struct seq_file *seq; }; struct addr_marker { @@ -268,11 +269,12 @@ static void note_wx(struct pg_state *st) * of PTE entries; the next one is different so we need to * print what we collected so far. */ -static void note_page(struct seq_file *m, struct pg_state *st, - pgprot_t new_prot, pgprotval_t new_eff, int level) +static void note_page(struct pg_state *st, pgprot_t new_prot, + pgprotval_t new_eff, int level) { pgprotval_t prot, cur, eff; static const char units[] = "BKMGTPE"; + struct seq_file *m = st->seq; /* * If we have a "break" in the series, we need to flush the state that @@ -357,8 +359,8 @@ static inline pgprotval_t effective_prot(pgprotval_t prot1, pgprotval_t prot2) ((prot1 | prot2) & _PAGE_NX); } -static void walk_pte_level(struct seq_file *m, struct pg_state *st, pmd_t addr, - pgprotval_t eff_in, unsigned long P) +static void walk_pte_level(struct pg_state *st, pmd_t addr, pgprotval_t eff_in, + unsigned long P) { int i; pte_t *pte; @@ -369,7 +371,7 @@ static void walk_pte_level(struct seq_file *m, struct pg_state *st, pmd_t addr, pte = pte_offset_map(&addr, st->current_address); prot = pte_flags(*pte); eff = effective_prot(eff_in, prot); - note_page(m, st, __pgprot(prot), eff, 5); + note_page(st, __pgprot(prot), eff, 5); pte_unmap(pte); } } @@ -382,22 +384,20 @@ static void walk_pte_level(struct seq_file *m, struct pg_state *st, pmd_t addr, * us dozens of seconds (minutes for 5-level config) while checking for * W+X mapping or reading kernel_page_tables debugfs file. */ -static inline bool kasan_page_table(struct seq_file *m, struct pg_state *st, - void *pt) +static inline bool kasan_page_table(struct pg_state *st, void *pt) { if (__pa(pt) == __pa(kasan_early_shadow_pmd) || (pgtable_l5_enabled() && __pa(pt) == __pa(kasan_early_shadow_p4d)) || __pa(pt) == __pa(kasan_early_shadow_pud)) { pgprotval_t prot = pte_flags(kasan_early_shadow_pte[0]); - note_page(m, st, __pgprot(prot), 0, 5); + note_page(st, __pgprot(prot), 0, 5); return true; } return false; } #else -static inline bool kasan_page_table(struct seq_file *m, struct pg_state *st, - void *pt) +static inline bool kasan_page_table(struct pg_state *st, void *pt) { return false; } @@ -405,7 +405,7 @@ static inline bool kasan_page_table(struct seq_file *m, struct pg_state *st, #if PTRS_PER_PMD > 1 -static void walk_pmd_level(struct seq_file *m, struct pg_state *st, pud_t addr, +static void walk_pmd_level(struct pg_state *st, pud_t addr, pgprotval_t eff_in, unsigned long P) { int i; @@ -419,19 +419,19 @@ static void walk_pmd_level(struct seq_file *m, struct pg_state *st, pud_t addr, prot = pmd_flags(*start); eff = effective_prot(eff_in, prot); if (pmd_large(*start) || !pmd_present(*start)) { - note_page(m, st, __pgprot(prot), eff, 4); - } else if (!kasan_page_table(m, st, pmd_start)) { - walk_pte_level(m, st, *start, eff, + note_page(st, __pgprot(prot), eff, 4); + } else if (!kasan_page_table(st, pmd_start)) { + walk_pte_level(st, *start, eff, P + i * PMD_LEVEL_MULT); } } else - note_page(m, st, __pgprot(0), 0, 4); + note_page(st, __pgprot(0), 0, 4); start++; } } #else -#define walk_pmd_level(m,s,a,e,p) walk_pte_level(m,s,__pmd(pud_val(a)),e,p) +#define walk_pmd_level(s,a,e,p) walk_pte_level(s,__pmd(pud_val(a)),e,p) #undef pud_large #define pud_large(a) pmd_large(__pmd(pud_val(a))) #define pud_none(a) pmd_none(__pmd(pud_val(a))) @@ -439,8 +439,8 @@ static void walk_pmd_level(struct seq_file *m, struct pg_state *st, pud_t addr, #if PTRS_PER_PUD > 1 -static void walk_pud_level(struct seq_file *m, struct pg_state *st, p4d_t addr, - pgprotval_t eff_in, unsigned long P) +static void walk_pud_level(struct pg_state *st, p4d_t addr, pgprotval_t eff_in, + unsigned long P) { int i; pud_t *start, *pud_start; @@ -455,13 +455,13 @@ static void walk_pud_level(struct seq_file *m, struct pg_state *st, p4d_t addr, prot = pud_flags(*start); eff = effective_prot(eff_in, prot); if (pud_large(*start) || !pud_present(*start)) { - note_page(m, st, __pgprot(prot), eff, 3); - } else if (!kasan_page_table(m, st, pud_start)) { - walk_pmd_level(m, st, *start, eff, + note_page(st, __pgprot(prot), eff, 3); + } else if (!kasan_page_table(st, pud_start)) { + walk_pmd_level(st, *start, eff, P + i * PUD_LEVEL_MULT); } } else - note_page(m, st, __pgprot(0), 0, 3); + note_page(st, __pgprot(0), 0, 3); prev_pud = start; start++; @@ -469,20 +469,20 @@ static void walk_pud_level(struct seq_file *m, struct pg_state *st, p4d_t addr, } #else -#define walk_pud_level(m,s,a,e,p) walk_pmd_level(m,s,__pud(p4d_val(a)),e,p) +#define walk_pud_level(s,a,e,p) walk_pmd_level(s,__pud(p4d_val(a)),e,p) #define p4d_large(a) pud_large(__pud(p4d_val(a))) #define p4d_none(a) pud_none(__pud(p4d_val(a))) #endif -static void walk_p4d_level(struct seq_file *m, struct pg_state *st, pgd_t addr, - pgprotval_t eff_in, unsigned long P) +static void walk_p4d_level(struct pg_state *st, pgd_t addr, pgprotval_t eff_in, + unsigned long P) { int i; p4d_t *start, *p4d_start; pgprotval_t prot, eff; if (PTRS_PER_P4D == 1) - return walk_pud_level(m, st, __p4d(pgd_val(addr)), eff_in, P); + return walk_pud_level(st, __p4d(pgd_val(addr)), eff_in, P); p4d_start = start = (p4d_t *)pgd_page_vaddr(addr); @@ -492,13 +492,13 @@ static void walk_p4d_level(struct seq_file *m, struct pg_state *st, pgd_t addr, prot = p4d_flags(*start); eff = effective_prot(eff_in, prot); if (p4d_large(*start) || !p4d_present(*start)) { - note_page(m, st, __pgprot(prot), eff, 2); - } else if (!kasan_page_table(m, st, p4d_start)) { - walk_pud_level(m, st, *start, eff, + note_page(st, __pgprot(prot), eff, 2); + } else if (!kasan_page_table(st, p4d_start)) { + walk_pud_level(st, *start, eff, P + i * P4D_LEVEL_MULT); } } else - note_page(m, st, __pgprot(0), 0, 2); + note_page(st, __pgprot(0), 0, 2); start++; } @@ -536,6 +536,7 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, } st.check_wx = checkwx; + st.seq = m; if (checkwx) st.wx_pages = 0; @@ -549,13 +550,13 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, eff = prot; #endif if (pgd_large(*start) || !pgd_present(*start)) { - note_page(m, &st, __pgprot(prot), eff, 1); + note_page(&st, __pgprot(prot), eff, 1); } else { - walk_p4d_level(m, &st, *start, eff, + walk_p4d_level(&st, *start, eff, i * PGD_LEVEL_MULT); } } else - note_page(m, &st, __pgprot(0), 0, 1); + note_page(&st, __pgprot(0), 0, 1); cond_resched(); start++; @@ -563,7 +564,7 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, /* Flush out the last page */ st.current_address = normalize_addr(PTRS_PER_PGD*PGD_LEVEL_MULT); - note_page(m, &st, __pgprot(0), 0, 0); + note_page(&st, __pgprot(0), 0, 0); if (!checkwx) return; if (st.wx_pages) From patchwork Fri Feb 15 17:02:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10815485 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5823C922 for ; Fri, 15 Feb 2019 17:03:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 460522FC1C for ; Fri, 15 Feb 2019 17:03:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3A3AB2FC34; Fri, 15 Feb 2019 17:03:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B8E9A2FC2F for ; Fri, 15 Feb 2019 17:03:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 50BD78E000D; Fri, 15 Feb 2019 12:03:41 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4BD4A8E0001; Fri, 15 Feb 2019 12:03:41 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3B1658E000D; Fri, 15 Feb 2019 12:03:41 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id BE1BF8E0001 for ; Fri, 15 Feb 2019 12:03:40 -0500 (EST) Received: by mail-ed1-f69.google.com with SMTP id u19so4147488eds.12 for ; Fri, 15 Feb 2019 09:03:40 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ZxfOscDKXutNwaZpM2VXWTmUvZzMiqbo2UDKLtcN8fs=; b=c1O2MNuFgdR4s93iWs4HkVyf7kK/66lqTOj24QWSuMTEIaFrP402eHIKkSy4yjKeMo RiGNOXOV8dDrP5C4LNLzy6/DL1PC6OX6s72IFZinmW7ksmiqMxzhqItxclV6rrC67YBo t3f58SZpUh2jPWh7okde+3Vgw7I1NSRLtDZTpHrVPyhZ5GtlSK94qz+mrRGGB3OKaf9s jPL1FH2YjJPqMJEoQh+WDemnuvNeaulHFSEAdxjdwIMOHP3azsYOEDhnFwss+b+zZoMy eDG2AGPVDu09JddGnXN7zoENeQNT55DAfqcHkAcpdXGK8NQlXkdK0PQKVSzdtzwiSQcQ sH3A== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAuZKduCEaTcmV9i85P1HPG0OQ9kxwP4M6xPOD2r+GfKPEnv4FNQw GLIhP8fGMi7eK13W48CMImLPfdd3I5kC4bWe+sv3iPpmJoPNnD/7MJndRWSh7E+yZynXTAMi4re H5uqUxWLLRBB+sS0m1mOhXj2byqFFbiIx848nzR8iQ5u45m9ZwJ5z3sEYp66xkxzReg== X-Received: by 2002:a17:906:14c9:: with SMTP id y9mr7378313ejc.182.1550250220252; Fri, 15 Feb 2019 09:03:40 -0800 (PST) X-Google-Smtp-Source: AHgI3Ib3/jlVymjyuu3xQEeo/DhlIJmkS0CMGpfZeD2JQeONde1vR2fPep6vVsBgnqUAxd9pe5md X-Received: by 2002:a17:906:14c9:: with SMTP id y9mr7378262ejc.182.1550250219210; Fri, 15 Feb 2019 09:03:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550250219; cv=none; d=google.com; s=arc-20160816; b=AN9qS2BTYQbtknyG7sOJaSa0+Y0TjIzoFw2O6397cbS3TCKmKvWP+Azy86c01tZob6 zt6+bQJKCRECSJcZIrVYK9wtzXJ615vK7qdPQqggXyuyGhdKi77mTzepKZjFJGet8DbC 9+YQHtzZ0ILw9HAn/2BUiNOKwfpBy2a6s9G1LP2V/SwEuXyJ3F1cu4NDLN6b7RiAhR5S RKyyVdrPUxnb6LOSd+WpY7kQMCNamSq/bi7/JSSnSW+9lCY5yBEG+x3od+cic8yH2DGH QcugdH46Wq4Re6aqvJYJMaImmutXZ3UTN4C0nL+aZLscXkPUlEXolVEWuCyjS/mN8sFu mqsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=ZxfOscDKXutNwaZpM2VXWTmUvZzMiqbo2UDKLtcN8fs=; b=Fi8z76iFnov1gD0HvFjJJoznkLA+E2Q+/OA92lasGHLstss7c/G8f/cq0DyRdvlxCs ZpJNZ/Mxs1B1BOZBkh/Q2HexvSbMr5L5KpJpLtTmduNxrSrfh9L/ZABxn5nHO2Ry6ncI wSib5ngpWee1/XcGiYTvcYvhDx6ySSEvoYK9Ocejdi7Yg5j1qRsExSARkt/8YEdklOpa q+ay+j6uj2rbeS5wgUP5M7a3TKsqtiYo6S87RjKyX0XjmiouRL1Bst0qgBsZFuCkfeGv krs9I56oh3XrC9biAFkx0d21EzB21/9lazU1GRg5uhmw6iXrH9DuEtzFHc3qiLxWBC5a WpyA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id ca20si2409135ejb.146.2019.02.15.09.03.38 for ; Fri, 15 Feb 2019 09:03:39 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 12BAE1596; Fri, 15 Feb 2019 09:03:38 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1F3883F557; Fri, 15 Feb 2019 09:03:34 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 10/13] x86/mm+efi: Convert ptdump_walk_pgd_level() to take a mm_struct Date: Fri, 15 Feb 2019 17:02:31 +0000 Message-Id: <20190215170235.23360-11-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190215170235.23360-1-steven.price@arm.com> References: <20190215170235.23360-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP To enable x86 to use the generic walk_page_range() function, the callers of ptdump_walk_pgd_level() need to pass an mm_struct rather than the raw pgd_t pointer. Luckily since commit 7e904a91bf60 ("efi: Use efi_mm in x86 as well as ARM") we now have an mm_struct for EFI on x86. Signed-off-by: Steven Price --- arch/x86/include/asm/pgtable.h | 2 +- arch/x86/mm/dump_pagetables.c | 4 ++-- arch/x86/platform/efi/efi_32.c | 2 +- arch/x86/platform/efi/efi_64.c | 4 ++-- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 3695f6acb6af..371901283d5f 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -27,7 +27,7 @@ extern pgd_t early_top_pgt[PTRS_PER_PGD]; int __init __early_make_pgtable(unsigned long address, pmdval_t pmd); -void ptdump_walk_pgd_level(struct seq_file *m, pgd_t *pgd); +void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm); void ptdump_walk_pgd_level_debugfs(struct seq_file *m, pgd_t *pgd, bool user); void ptdump_walk_pgd_level_checkwx(void); void ptdump_walk_user_pgd_level_checkwx(void); diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index ad6986c9e8e7..1a4a03e3a6bd 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -574,9 +574,9 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, pr_info("x86/mm: Checked W+X mappings: passed, no W+X pages found.\n"); } -void ptdump_walk_pgd_level(struct seq_file *m, pgd_t *pgd) +void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm) { - ptdump_walk_pgd_level_core(m, pgd, false, true); + ptdump_walk_pgd_level_core(m, mm->pgd, false, true); } void ptdump_walk_pgd_level_debugfs(struct seq_file *m, pgd_t *pgd, bool user) diff --git a/arch/x86/platform/efi/efi_32.c b/arch/x86/platform/efi/efi_32.c index 9959657127f4..9175ceaa6e72 100644 --- a/arch/x86/platform/efi/efi_32.c +++ b/arch/x86/platform/efi/efi_32.c @@ -49,7 +49,7 @@ void efi_sync_low_kernel_mappings(void) {} void __init efi_dump_pagetable(void) { #ifdef CONFIG_EFI_PGT_DUMP - ptdump_walk_pgd_level(NULL, swapper_pg_dir); + ptdump_walk_pgd_level(NULL, init_mm); #endif } diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c index cf0347f61b21..a2e0f9800190 100644 --- a/arch/x86/platform/efi/efi_64.c +++ b/arch/x86/platform/efi/efi_64.c @@ -611,9 +611,9 @@ void __init efi_dump_pagetable(void) { #ifdef CONFIG_EFI_PGT_DUMP if (efi_enabled(EFI_OLD_MEMMAP)) - ptdump_walk_pgd_level(NULL, swapper_pg_dir); + ptdump_walk_pgd_level(NULL, init_mm); else - ptdump_walk_pgd_level(NULL, efi_mm.pgd); + ptdump_walk_pgd_level(NULL, efi_mm); #endif } From patchwork Fri Feb 15 17:02:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10815487 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9DB811390 for ; Fri, 15 Feb 2019 17:03:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8942E2ED13 for ; Fri, 15 Feb 2019 17:03:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 797942ED1D; Fri, 15 Feb 2019 17:03:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E3D602FBED for ; Fri, 15 Feb 2019 17:03:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 583888E000E; Fri, 15 Feb 2019 12:03:44 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 531CF8E0001; Fri, 15 Feb 2019 12:03:44 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4481C8E000E; Fri, 15 Feb 2019 12:03:44 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id E2FD58E0001 for ; Fri, 15 Feb 2019 12:03:43 -0500 (EST) Received: by mail-ed1-f71.google.com with SMTP id p52so4145993eda.18 for ; Fri, 15 Feb 2019 09:03:43 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ctfQAjdhDywt5nLYxqKiK2RzJwkOOVzCEeLiTsCnJr0=; b=SjkblWy9HEfGCYxGjCKZrSiqEWJy9Mr79i/NG31dYC5bPep9IwLkGMvCg0p6xZKDVt Tgqx//ctH39Fe5Xalvg6S5THgTAnRlJZ+qDMOXg3AIj0MT1g6JCWLxu6xhLrSXZomreT GcXpHf3vCq3+cWJG0YR/6U+t9VxmXAcNFgmG0nk81uwvTFjNJRm2TkQS3NaWVCfcw2Bp 0GRt7tHHCx/32ocJIhc+kaXrvQ7GmOCZktyp9CwKmP+edBakmfFORp9veMWfIrnumAeE 3fxDYQbnuvm7eR1RmGQahvPR7KOk/vD+k7MxcWfOn5fRAo3ZBbPBGCLGPS3wqb1gqlBu wdgg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAuakeZsHlypynNvC9gfmkgP3NDU+t8h7GmnE4Bcfuadwjonn5DSc gvWrh9yWO6ZWM8XGHLNN9Z5T0m/sYEu4x79Vj2w4OB9BpWCp22ro+vlgwl3c8DSjZttJNRrSL6y lT4SOKkMp7KUJzIzOV0X5I6UQlVHnFKBPz0hU1yGHdGbpf5Eu/lUhvM221IZijICK/Q== X-Received: by 2002:aa7:ca41:: with SMTP id j1mr8399561edt.34.1550250223418; Fri, 15 Feb 2019 09:03:43 -0800 (PST) X-Google-Smtp-Source: AHgI3IYKPmxAbL22CRcIZbq9LjIwufV57isNz5EK59vKmzzZxTVIQImrfRE+hM1Mrq8fgEEMt6LN X-Received: by 2002:aa7:ca41:: with SMTP id j1mr8399499edt.34.1550250222330; Fri, 15 Feb 2019 09:03:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550250222; cv=none; d=google.com; s=arc-20160816; b=JlZAUffrFy0llUNI2Pa+zL9FqNDc+lYxtzP4B0o47xQDkyOLbHl6CLc3vjP68O6zlV Mavfn8gAGGSPEeYQI65iGH2hkgyaQcZPOzcIJCnUK9IVsGe2NLe/sxZWX8wu5u/ZmH8U F3auwrlwZ1EBcZpXLJ9GyBPAkW6IDjt2SoY5UOEUo3r4zqm56i7aRJIjiigGfK+ytSqk Ae8nTg7ZNsm5jAkaj5+gYkBRpERMsKLQdQSQJyErYhDjG4TS6HD/fSYwE/mIFNGqVM2m SXZ+8hnGTdYXkNuYoOIQ9F7YqMzgvvp0JJ7nlZ/m/BwIYquqM5KR386wWwaSRyWpLjc6 5bdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=ctfQAjdhDywt5nLYxqKiK2RzJwkOOVzCEeLiTsCnJr0=; b=eIGraqPc8eTeyQBHBrrfMvE+aSCSOHf55CnFmhxfG7Aqb2KYACT7p2hsJdbvClnxOU DCoYSg9s8JfiPk5J9HqQ2CxCFlkIuWbcEvpJ8I7briD3S25skihv+Rj9Hi4toZLITKzC gEEpK8Y5t4hhXE9Eur1LkIxyLFmTqBDB3BFuRT6mZpSboVh8EOpCKxQzCzHZHOVwqu2o LEi70cjRtzrVN0wgYBZyR4rlZes0vOdt1cr87O1na0QTzW7CcBv5YNeF48UWoTtyvvYk 2hUBWbD9vEbvvyS/AwQRqKkzhTOHGY3O+OJ8aAncjXyZVN9DBzLOByjPQuAT+Rsf4Aig JAQA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id f26si810897ejb.21.2019.02.15.09.03.41 for ; Fri, 15 Feb 2019 09:03:42 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 45C4B1682; Fri, 15 Feb 2019 09:03:41 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 521F63F557; Fri, 15 Feb 2019 09:03:38 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 11/13] x86/mm: Convert ptdump_walk_pgd_level_debugfs() to take an mm_struct Date: Fri, 15 Feb 2019 17:02:32 +0000 Message-Id: <20190215170235.23360-12-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190215170235.23360-1-steven.price@arm.com> References: <20190215170235.23360-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP To enable x86 to use the generic walk_page_range() function, the callers of ptdump_walk_pgd_level_debugfs() need to pass in the mm_struct. This means that ptdump_walk_pgd_level_core() is now always passed a valid pgd, so drop the support for pgd==NULL. Signed-off-by: Steven Price --- arch/x86/include/asm/pgtable.h | 3 ++- arch/x86/mm/debug_pagetables.c | 8 ++++---- arch/x86/mm/dump_pagetables.c | 14 ++++++-------- 3 files changed, 12 insertions(+), 13 deletions(-) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 371901283d5f..ab2aa3eb05e9 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -28,7 +28,8 @@ extern pgd_t early_top_pgt[PTRS_PER_PGD]; int __init __early_make_pgtable(unsigned long address, pmdval_t pmd); void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm); -void ptdump_walk_pgd_level_debugfs(struct seq_file *m, pgd_t *pgd, bool user); +void ptdump_walk_pgd_level_debugfs(struct seq_file *m, struct mm_struct *mm, + bool user); void ptdump_walk_pgd_level_checkwx(void); void ptdump_walk_user_pgd_level_checkwx(void); diff --git a/arch/x86/mm/debug_pagetables.c b/arch/x86/mm/debug_pagetables.c index cd84f067e41d..824131052574 100644 --- a/arch/x86/mm/debug_pagetables.c +++ b/arch/x86/mm/debug_pagetables.c @@ -6,7 +6,7 @@ static int ptdump_show(struct seq_file *m, void *v) { - ptdump_walk_pgd_level_debugfs(m, NULL, false); + ptdump_walk_pgd_level_debugfs(m, &init_mm, false); return 0; } @@ -16,7 +16,7 @@ static int ptdump_curknl_show(struct seq_file *m, void *v) { if (current->mm->pgd) { down_read(¤t->mm->mmap_sem); - ptdump_walk_pgd_level_debugfs(m, current->mm->pgd, false); + ptdump_walk_pgd_level_debugfs(m, current->mm, false); up_read(¤t->mm->mmap_sem); } return 0; @@ -31,7 +31,7 @@ static int ptdump_curusr_show(struct seq_file *m, void *v) { if (current->mm->pgd) { down_read(¤t->mm->mmap_sem); - ptdump_walk_pgd_level_debugfs(m, current->mm->pgd, true); + ptdump_walk_pgd_level_debugfs(m, current->mm, true); up_read(¤t->mm->mmap_sem); } return 0; @@ -46,7 +46,7 @@ static struct dentry *pe_efi; static int ptdump_efi_show(struct seq_file *m, void *v) { if (efi_mm.pgd) - ptdump_walk_pgd_level_debugfs(m, efi_mm.pgd, false); + ptdump_walk_pgd_level_debugfs(m, &efi_mm, false); return 0; } diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index 1a4a03e3a6bd..18fb6193311f 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -525,16 +525,12 @@ static inline bool is_hypervisor_range(int idx) static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, bool checkwx, bool dmesg) { - pgd_t *start = INIT_PGD; + pgd_t *start = pgd; pgprotval_t prot, eff; int i; struct pg_state st = {}; - if (pgd) { - start = pgd; - st.to_dmesg = dmesg; - } - + st.to_dmesg = dmesg; st.check_wx = checkwx; st.seq = m; if (checkwx) @@ -579,8 +575,10 @@ void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm) ptdump_walk_pgd_level_core(m, mm->pgd, false, true); } -void ptdump_walk_pgd_level_debugfs(struct seq_file *m, pgd_t *pgd, bool user) +void ptdump_walk_pgd_level_debugfs(struct seq_file *m, struct mm_struct *mm, + bool user) { + pgd_t *pgd = mm->pgd; #ifdef CONFIG_PAGE_TABLE_ISOLATION if (user && static_cpu_has(X86_FEATURE_PTI)) pgd = kernel_to_user_pgdp(pgd); @@ -606,7 +604,7 @@ void ptdump_walk_user_pgd_level_checkwx(void) void ptdump_walk_pgd_level_checkwx(void) { - ptdump_walk_pgd_level_core(NULL, NULL, true, false); + ptdump_walk_pgd_level_core(NULL, INIT_PGD, true, false); } static int __init pt_dump_init(void) From patchwork Fri Feb 15 17:02:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10815489 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1BB371390 for ; Fri, 15 Feb 2019 17:03:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 09C892ED13 for ; Fri, 15 Feb 2019 17:03:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F195D2ED5D; Fri, 15 Feb 2019 17:03:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7CB4C2ED13 for ; Fri, 15 Feb 2019 17:03:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BC5BA8E000F; Fri, 15 Feb 2019 12:03:47 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B76768E0001; Fri, 15 Feb 2019 12:03:47 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A3B558E000F; Fri, 15 Feb 2019 12:03:47 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by kanga.kvack.org (Postfix) with ESMTP id 4CA4D8E0001 for ; Fri, 15 Feb 2019 12:03:47 -0500 (EST) Received: by mail-ed1-f70.google.com with SMTP id w51so4195625edw.7 for ; Fri, 15 Feb 2019 09:03:47 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=p4xRgAKusZHkTC7J5q8NQBkr68Ska5Iak032oO+4xKM=; b=TsE4X3UUGHLGom0dMRvJ40y0sltdLwDiv+s7N5ex3zq6N0bYgmQSC0xhtfiIaUJe08 n4ZIHdrRrrenvRZGERMMRjOdYZaIJ8JUhZ7g90E/jIQEwF6tWEZzcA2fB7nJLKua1D9a g36t/961eHs/8ymvhWm1zNxs05sZFscFRVruMR2LKxRJ8K/vMEs7+KqmlvbIi4gRRP+I 2KQEnAMLtgCWHWs+BY2CLmeaVnyNMzR7v2mLPEazjVntDIMj8szx2EWOHYMqzYyJb1Nj XusaVSqBkV/ewuHR+UxQ1L1nV8Lm0lu5PkL2GQrh3SZHeWaiLx/Km0E6Knzu1aEFPXOj WavA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAuaLVip2MhSgNc1Yof+n/mDjvrKDr+4HG7pLag7yPSg6nBFlDThR dY6qkzQE/Aifz5m88lbOi4rWhG1+ysgzg2WqXyIhWuOQfBL/F1uLnr3yWndFenGuRYZklyyCvXb 9WqBqTR1RrBwZ50Jc5UCuxgyRA3DnUIoFKE3SZ9Rm9lJ/m0xhblIW5lVUcixMfCuAVw== X-Received: by 2002:aa7:c5d0:: with SMTP id h16mr8308276eds.107.1550250226762; Fri, 15 Feb 2019 09:03:46 -0800 (PST) X-Google-Smtp-Source: AHgI3Iaux92P3muc9T5MheL0oybHsiV7QrZnUKngU81sBAsej1p4Ixvvr5ynS9ofT6gus5L+26uJ X-Received: by 2002:aa7:c5d0:: with SMTP id h16mr8308210eds.107.1550250225586; Fri, 15 Feb 2019 09:03:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550250225; cv=none; d=google.com; s=arc-20160816; b=CoR7zqYGACccnhYwz/jRTRdeb3dZhh4ZSqtB72X5SMHem3Ti1RQg5Sd+iqenZlkIB5 /o+1CGE9Laplfejoj6Kp7fBC/Yc0Ejj2OjDtq7l7L9UVfKFmduR2swelNFarujF/hKNj VhvmSXwk3qlMV7weP26Wh0EUEJyKP0rjeO246K6duLc6bH28xnZReJMNnQAeqlnDgMzH XYU6XvStq3LE+WH9+KDyN/IbHIkq8Jyo6KeNaXJhABfgjvRHkaBMJess3FlCoCRHli17 9nPtizsIkums68x7QOeJ4Pb/1b+jdofYEODwKBFb6jlTgn5e/G/iYPugqgqGVPiqXcEU kEeQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=p4xRgAKusZHkTC7J5q8NQBkr68Ska5Iak032oO+4xKM=; b=wGR2ObXaq1Du72V7KyxVYEz0PkOh8AWKnVnq+DP8W7C3wa6eQ3Hw2n+Q1o5Gne2kZY pNfQMV5nkyab+fQ/wPnAYuCQL+Q/+/I2hTZwHbXH3y9jdTHt1Pu67vxUZwDgbFzQPbsT 4bnD/auw6tt7aiZSRoCHblTZayGd8Ec6tnIFBr4E4KZpZ/UacWWopUhypyOdKD8pV3Y4 CF6awKn54CYvxN9tkM57NCTrfYJLYha7JZVYs8jotD+E0flF/6QAFRsFgsVS4hSz6D6R JrqdljjDqROP3hK4RggUSS88xLQfmZEHYzo75n+Ub5MlIRReWmCvMh1gOi2LauI0G5m8 OfUA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id r15si2789729edd.226.2019.02.15.09.03.45 for ; Fri, 15 Feb 2019 09:03:45 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 78711169E; Fri, 15 Feb 2019 09:03:44 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 84F443F557; Fri, 15 Feb 2019 09:03:41 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 12/13] x86/mm: Convert ptdump_walk_pgd_level_core() to take an mm_struct Date: Fri, 15 Feb 2019 17:02:33 +0000 Message-Id: <20190215170235.23360-13-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190215170235.23360-1-steven.price@arm.com> References: <20190215170235.23360-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP An mm_struct is needed to enable x86 to use of the generic walk_page_range() function. In the case of walking the user page tables (when CONFIG_PAGE_TABLE_ISOLATION is enabled), it is necessary to create a fake_mm structure because there isn't an mm_struct with a pointer to the pgd of the user page tables. This fake_mm structure is initialised with the minimum necessary for the generic page walk code. Signed-off-by: Steven Price --- arch/x86/mm/dump_pagetables.c | 36 ++++++++++++++++++++--------------- 1 file changed, 21 insertions(+), 15 deletions(-) diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index 18fb6193311f..ddb22e7f81b3 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -111,8 +111,6 @@ static struct addr_marker address_markers[] = { [END_OF_SPACE_NR] = { -1, NULL } }; -#define INIT_PGD ((pgd_t *) &init_top_pgt) - #else /* CONFIG_X86_64 */ enum address_markers_idx { @@ -147,8 +145,6 @@ static struct addr_marker address_markers[] = { [END_OF_SPACE_NR] = { -1, NULL } }; -#define INIT_PGD (swapper_pg_dir) - #endif /* !CONFIG_X86_64 */ /* Multipliers for offsets within the PTEs */ @@ -522,10 +518,10 @@ static inline bool is_hypervisor_range(int idx) #endif } -static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, +static void ptdump_walk_pgd_level_core(struct seq_file *m, struct mm_struct *mm, bool checkwx, bool dmesg) { - pgd_t *start = pgd; + pgd_t *start = mm->pgd; pgprotval_t prot, eff; int i; struct pg_state st = {}; @@ -572,39 +568,49 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm) { - ptdump_walk_pgd_level_core(m, mm->pgd, false, true); + ptdump_walk_pgd_level_core(m, mm, false, true); } +#ifdef CONFIG_PAGE_TABLE_ISOLATION +static void ptdump_walk_pgd_level_user_core(struct seq_file *m, + struct mm_struct *mm, + bool checkwx, bool dmesg) +{ + struct mm_struct fake_mm = { + .pgd = kernel_to_user_pgdp(mm->pgd) + }; + init_rwsem(&fake_mm.mmap_sem); + ptdump_walk_pgd_level_core(m, &fake_mm, checkwx, dmesg); +} +#endif + void ptdump_walk_pgd_level_debugfs(struct seq_file *m, struct mm_struct *mm, bool user) { - pgd_t *pgd = mm->pgd; #ifdef CONFIG_PAGE_TABLE_ISOLATION if (user && static_cpu_has(X86_FEATURE_PTI)) - pgd = kernel_to_user_pgdp(pgd); + ptdump_walk_pgd_level_user_core(m, mm, false, false); + else #endif - ptdump_walk_pgd_level_core(m, pgd, false, false); + ptdump_walk_pgd_level_core(m, mm, false, false); } EXPORT_SYMBOL_GPL(ptdump_walk_pgd_level_debugfs); void ptdump_walk_user_pgd_level_checkwx(void) { #ifdef CONFIG_PAGE_TABLE_ISOLATION - pgd_t *pgd = INIT_PGD; - if (!(__supported_pte_mask & _PAGE_NX) || !static_cpu_has(X86_FEATURE_PTI)) return; pr_info("x86/mm: Checking user space page tables\n"); - pgd = kernel_to_user_pgdp(pgd); - ptdump_walk_pgd_level_core(NULL, pgd, true, false); + ptdump_walk_pgd_level_user_core(NULL, &init_mm, true, false); #endif } void ptdump_walk_pgd_level_checkwx(void) { - ptdump_walk_pgd_level_core(NULL, INIT_PGD, true, false); + ptdump_walk_pgd_level_core(NULL, &init_mm, true, false); } static int __init pt_dump_init(void) From patchwork Fri Feb 15 17:02:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10815491 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 34A34922 for ; Fri, 15 Feb 2019 17:03:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 21B172ED13 for ; Fri, 15 Feb 2019 17:03:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 14ED42ED5D; Fri, 15 Feb 2019 17:03:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3BC9E2ED13 for ; Fri, 15 Feb 2019 17:03:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 647098E0010; Fri, 15 Feb 2019 12:03:51 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5CE778E0001; Fri, 15 Feb 2019 12:03:51 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4E8A58E0010; Fri, 15 Feb 2019 12:03:51 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id E1A258E0001 for ; Fri, 15 Feb 2019 12:03:50 -0500 (EST) Received: by mail-ed1-f72.google.com with SMTP id d62so4267044edd.19 for ; Fri, 15 Feb 2019 09:03:50 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=ZRIIbvl66a2UpZAkFzyq5lze2+obrbp24oaaXHkFAyQ=; b=i+HFfPJkrogpdiAoxYR0L2tDBGZfnOu6xwXcjGvJ7dIufww7Sq2SZA++DK0RwWqHVZ aL5j6yQwCHH/fECuNuERz4QXoAiF9d632LoR4XDdmd5UDm7S9a0v2fmdKCTWph7HPly6 6QFYjKnagmJmq1shGxSCNEnWNTeiHibQh+h3qfQmA7N4ukYaS6w9n3if+u8aMshX85Sa VQ9pxslwRFaAiFW+KHj3ohTQxnKt7CUvAwqrh7i/fXvyXqcZkX341o3Uqped60xg4gJE VsIFJsQ7BG1XLkNtXU8USdkE81Ku5kPBRsuAJ6gftnQgT1gCRu92Jxybnxy7Z+wPcMlR pTGw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: AHQUAuY7sJKKyFdPBBzUkXv8HTnYkv5FcDBbgrjfEBVpZndxTwipJq5j XOMQOZQ+4MklButphS/Y1XJ/o3mzWMI2doF6pSpEzXcxY/BtZ5o6i1sBNa3A3K6BWHR83cEjuvu AdZQiO5LTpjnxBjCAg6OWiIGBvlhKHFZqNs+DQ4DtUlmatFRd3jV+hqv+ZlXjGyIwTw== X-Received: by 2002:a50:ad57:: with SMTP id z23mr8085894edc.223.1550250230374; Fri, 15 Feb 2019 09:03:50 -0800 (PST) X-Google-Smtp-Source: AHgI3IZ32iZhv7npII2Xbqdsts2rzQX3skSTJ+U/TOQXiFCowdriRk2/OfadxxPugsE2VBGxF70l X-Received: by 2002:a50:ad57:: with SMTP id z23mr8085819edc.223.1550250228991; Fri, 15 Feb 2019 09:03:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550250228; cv=none; d=google.com; s=arc-20160816; b=kmdsw4q62qxUkcpZ9rrWk4ZrL1gfF+9amsxKsYy2SJO5VrTRKMPVrivvL5QQ76wgcR ylchsE4vwLNOatdGDvS+jFgqI6ywdajPOrloXk5Nr6fSzu8XuiFShmrHcEE98daC3sAi Vlfm86PXU253lzS6qYEwC0zXn6WXG/uZve/US48N0sYV3QBC/RDAufUSTU1u385SDDVq +85smhwAgaeVLR3U4UF1p/gRSoBOPLDuX2nw+fjLiUgBj3iRD9kU8SKFSD8Ak9npRSZ7 B692shg3gEfZd8ShhinibVufxN9CvfJjCb/A8aN61h54O6sUjqDSoW+Dl4SbrNSY6cSX UcRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=ZRIIbvl66a2UpZAkFzyq5lze2+obrbp24oaaXHkFAyQ=; b=aqz8NwqccmQTdRYjqwJD4IPZ9vBA24s/zhy3CdyoOM4ZeCTrMB5H6ftTHPRcoQ2s1H QY0wACwW5stUFtXAgsFU+Zv9nG+F9UT1AfO5dcq33kU2PF6ZVsE9wn5w6tTz9evCd22q pErAcHYCAzkJ7yFfhHNl8wL8oK0P2mh0fVBTmDmdAdAMx7hlwr5pHa2A81SHQFYZznUv WHRMXSkiM1cDZnTnWtKLQ6/g9/phHog7aHrvyukX56LJJUFEdAj34T2KGAcup1R4+QOr r6shTITjIeJcSd22GknlyGCCgDHk9RHEjVyzfIWilKR2KJDeo0lzz5L/TY4+3lBteq4r BCug== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id p21si196203eda.281.2019.02.15.09.03.48 for ; Fri, 15 Feb 2019 09:03:48 -0800 (PST) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AB8F51596; Fri, 15 Feb 2019 09:03:47 -0800 (PST) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B82183F557; Fri, 15 Feb 2019 09:03:44 -0800 (PST) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 13/13] x86: mm: Convert dump_pagetables to use walk_page_range Date: Fri, 15 Feb 2019 17:02:34 +0000 Message-Id: <20190215170235.23360-14-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190215170235.23360-1-steven.price@arm.com> References: <20190215170235.23360-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Make use of the new functionality in walk_page_range to remove the arch page walking code and use the generic code to walk the page tables. The effective permissions are passed down the chain using new fields in struct pg_state. The KASAN optimisation is implemented by including test_p?d callbacks which can decide to skip an entire tree of entries Signed-off-by: Steven Price --- arch/x86/mm/dump_pagetables.c | 281 ++++++++++++++++++---------------- 1 file changed, 146 insertions(+), 135 deletions(-) diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index ddb22e7f81b3..64d1619493a4 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -33,6 +33,10 @@ struct pg_state { int level; pgprot_t current_prot; pgprotval_t effective_prot; + pgprotval_t effective_prot_pgd; + pgprotval_t effective_prot_p4d; + pgprotval_t effective_prot_pud; + pgprotval_t effective_prot_pmd; unsigned long start_address; unsigned long current_address; const struct addr_marker *marker; @@ -355,22 +359,21 @@ static inline pgprotval_t effective_prot(pgprotval_t prot1, pgprotval_t prot2) ((prot1 | prot2) & _PAGE_NX); } -static void walk_pte_level(struct pg_state *st, pmd_t addr, pgprotval_t eff_in, - unsigned long P) +static int ptdump_pte_entry(pte_t *pte, unsigned long addr, + unsigned long next, struct mm_walk *walk) { - int i; - pte_t *pte; - pgprotval_t prot, eff; - - for (i = 0; i < PTRS_PER_PTE; i++) { - st->current_address = normalize_addr(P + i * PTE_LEVEL_MULT); - pte = pte_offset_map(&addr, st->current_address); - prot = pte_flags(*pte); - eff = effective_prot(eff_in, prot); - note_page(st, __pgprot(prot), eff, 5); - pte_unmap(pte); - } + struct pg_state *st = walk->private; + pgprotval_t eff, prot; + + st->current_address = normalize_addr(addr); + + prot = pte_flags(*pte); + eff = effective_prot(st->effective_prot_pmd, prot); + note_page(st, __pgprot(prot), eff, 5); + + return 0; } + #ifdef CONFIG_KASAN /* @@ -399,132 +402,152 @@ static inline bool kasan_page_table(struct pg_state *st, void *pt) } #endif -#if PTRS_PER_PMD > 1 - -static void walk_pmd_level(struct pg_state *st, pud_t addr, - pgprotval_t eff_in, unsigned long P) +static int ptdump_test_pmd(unsigned long addr, unsigned long next, + pmd_t *pmd, struct mm_walk *walk) { - int i; - pmd_t *start, *pmd_start; - pgprotval_t prot, eff; - - pmd_start = start = (pmd_t *)pud_page_vaddr(addr); - for (i = 0; i < PTRS_PER_PMD; i++) { - st->current_address = normalize_addr(P + i * PMD_LEVEL_MULT); - if (!pmd_none(*start)) { - prot = pmd_flags(*start); - eff = effective_prot(eff_in, prot); - if (pmd_large(*start) || !pmd_present(*start)) { - note_page(st, __pgprot(prot), eff, 4); - } else if (!kasan_page_table(st, pmd_start)) { - walk_pte_level(st, *start, eff, - P + i * PMD_LEVEL_MULT); - } - } else - note_page(st, __pgprot(0), 0, 4); - start++; - } + struct pg_state *st = walk->private; + + st->current_address = normalize_addr(addr); + + if (kasan_page_table(st, pmd)) + return 1; + return 0; } -#else -#define walk_pmd_level(s,a,e,p) walk_pte_level(s,__pmd(pud_val(a)),e,p) -#undef pud_large -#define pud_large(a) pmd_large(__pmd(pud_val(a))) -#define pud_none(a) pmd_none(__pmd(pud_val(a))) -#endif +static int ptdump_pmd_entry(pmd_t *pmd, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + struct pg_state *st = walk->private; + pgprotval_t eff, prot; + + prot = pmd_flags(*pmd); + eff = effective_prot(st->effective_prot_pud, prot); + + st->current_address = normalize_addr(addr); + + if (pmd_large(*pmd)) + note_page(st, __pgprot(prot), eff, 4); -#if PTRS_PER_PUD > 1 + st->effective_prot_pmd = eff; -static void walk_pud_level(struct pg_state *st, p4d_t addr, pgprotval_t eff_in, - unsigned long P) + return 0; +} + +static int ptdump_test_pud(unsigned long addr, unsigned long next, + pud_t *pud, struct mm_walk *walk) { - int i; - pud_t *start, *pud_start; - pgprotval_t prot, eff; - pud_t *prev_pud = NULL; - - pud_start = start = (pud_t *)p4d_page_vaddr(addr); - - for (i = 0; i < PTRS_PER_PUD; i++) { - st->current_address = normalize_addr(P + i * PUD_LEVEL_MULT); - if (!pud_none(*start)) { - prot = pud_flags(*start); - eff = effective_prot(eff_in, prot); - if (pud_large(*start) || !pud_present(*start)) { - note_page(st, __pgprot(prot), eff, 3); - } else if (!kasan_page_table(st, pud_start)) { - walk_pmd_level(st, *start, eff, - P + i * PUD_LEVEL_MULT); - } - } else - note_page(st, __pgprot(0), 0, 3); + struct pg_state *st = walk->private; - prev_pud = start; - start++; - } + st->current_address = normalize_addr(addr); + + if (kasan_page_table(st, pud)) + return 1; + return 0; } -#else -#define walk_pud_level(s,a,e,p) walk_pmd_level(s,__pud(p4d_val(a)),e,p) -#define p4d_large(a) pud_large(__pud(p4d_val(a))) -#define p4d_none(a) pud_none(__pud(p4d_val(a))) -#endif +static int ptdump_pud_entry(pud_t *pud, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + struct pg_state *st = walk->private; + pgprotval_t eff, prot; + + prot = pud_flags(*pud); + eff = effective_prot(st->effective_prot_p4d, prot); + + st->current_address = normalize_addr(addr); + + if (pud_large(*pud)) + note_page(st, __pgprot(prot), eff, 3); + + st->effective_prot_pud = eff; -static void walk_p4d_level(struct pg_state *st, pgd_t addr, pgprotval_t eff_in, - unsigned long P) + return 0; +} + +static int ptdump_test_p4d(unsigned long addr, unsigned long next, + p4d_t *p4d, struct mm_walk *walk) { - int i; - p4d_t *start, *p4d_start; - pgprotval_t prot, eff; - - if (PTRS_PER_P4D == 1) - return walk_pud_level(st, __p4d(pgd_val(addr)), eff_in, P); - - p4d_start = start = (p4d_t *)pgd_page_vaddr(addr); - - for (i = 0; i < PTRS_PER_P4D; i++) { - st->current_address = normalize_addr(P + i * P4D_LEVEL_MULT); - if (!p4d_none(*start)) { - prot = p4d_flags(*start); - eff = effective_prot(eff_in, prot); - if (p4d_large(*start) || !p4d_present(*start)) { - note_page(st, __pgprot(prot), eff, 2); - } else if (!kasan_page_table(st, p4d_start)) { - walk_pud_level(st, *start, eff, - P + i * P4D_LEVEL_MULT); - } - } else - note_page(st, __pgprot(0), 0, 2); + struct pg_state *st = walk->private; - start++; - } + st->current_address = normalize_addr(addr); + + if (kasan_page_table(st, p4d)) + return 1; + return 0; } -#undef pgd_large -#define pgd_large(a) (pgtable_l5_enabled() ? pgd_large(a) : p4d_large(__p4d(pgd_val(a)))) -#define pgd_none(a) (pgtable_l5_enabled() ? pgd_none(a) : p4d_none(__p4d(pgd_val(a)))) +static int ptdump_p4d_entry(p4d_t *p4d, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + struct pg_state *st = walk->private; + pgprotval_t eff, prot; + + prot = p4d_flags(*p4d); + eff = effective_prot(st->effective_prot_pgd, prot); + + st->current_address = normalize_addr(addr); + + if (p4d_large(*p4d)) + note_page(st, __pgprot(prot), eff, 2); + + st->effective_prot_p4d = eff; + + return 0; +} -static inline bool is_hypervisor_range(int idx) +static int ptdump_pgd_entry(pgd_t *pgd, unsigned long addr, + unsigned long next, struct mm_walk *walk) { -#ifdef CONFIG_X86_64 - /* - * A hole in the beginning of kernel address space reserved - * for a hypervisor. - */ - return (idx >= pgd_index(GUARD_HOLE_BASE_ADDR)) && - (idx < pgd_index(GUARD_HOLE_END_ADDR)); + struct pg_state *st = walk->private; + pgprotval_t eff, prot; + + prot = pgd_flags(*pgd); + +#ifdef CONFIG_X86_PAE + eff = _PAGE_USER | _PAGE_RW; #else - return false; + eff = prot; #endif + + st->current_address = normalize_addr(addr); + + if (pgd_large(*pgd)) + note_page(st, __pgprot(prot), eff, 1); + + st->effective_prot_pgd = eff; + + return 0; +} + +static int ptdump_hole(unsigned long addr, unsigned long next, int depth, + struct mm_walk *walk) +{ + struct pg_state *st = walk->private; + + st->current_address = normalize_addr(addr); + + note_page(st, __pgprot(0), 0, depth + 1); + + return 0; } static void ptdump_walk_pgd_level_core(struct seq_file *m, struct mm_struct *mm, bool checkwx, bool dmesg) { - pgd_t *start = mm->pgd; - pgprotval_t prot, eff; - int i; struct pg_state st = {}; + struct mm_walk walk = { + .mm = mm, + .pgd_entry = ptdump_pgd_entry, + .p4d_entry = ptdump_p4d_entry, + .pud_entry = ptdump_pud_entry, + .pmd_entry = ptdump_pmd_entry, + .pte_entry = ptdump_pte_entry, + .test_p4d = ptdump_test_p4d, + .test_pud = ptdump_test_pud, + .test_pmd = ptdump_test_pmd, + .pte_hole = ptdump_hole, + .private = &st + }; st.to_dmesg = dmesg; st.check_wx = checkwx; @@ -532,27 +555,15 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, struct mm_struct *mm, if (checkwx) st.wx_pages = 0; - for (i = 0; i < PTRS_PER_PGD; i++) { - st.current_address = normalize_addr(i * PGD_LEVEL_MULT); - if (!pgd_none(*start) && !is_hypervisor_range(i)) { - prot = pgd_flags(*start); -#ifdef CONFIG_X86_PAE - eff = _PAGE_USER | _PAGE_RW; + down_read(&mm->mmap_sem); +#ifdef CONFIG_X86_64 + walk_page_range(0, PTRS_PER_PGD*PGD_LEVEL_MULT/2, &walk); + walk_page_range(normalize_addr(PTRS_PER_PGD*PGD_LEVEL_MULT/2), ~0, + &walk); #else - eff = prot; + walk_page_range(0, ~0, &walk); #endif - if (pgd_large(*start) || !pgd_present(*start)) { - note_page(&st, __pgprot(prot), eff, 1); - } else { - walk_p4d_level(&st, *start, eff, - i * PGD_LEVEL_MULT); - } - } else - note_page(&st, __pgprot(0), 0, 1); - - cond_resched(); - start++; - } + up_read(&mm->mmap_sem); /* Flush out the last page */ st.current_address = normalize_addr(PTRS_PER_PGD*PGD_LEVEL_MULT);