From patchwork Thu Mar 21 14:19:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863609 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F13C61922 for ; Thu, 21 Mar 2019 14:20:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D2D4C2A2A7 for ; Thu, 21 Mar 2019 14:20:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D0AD62A17A; Thu, 21 Mar 2019 14:20:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 313462A2AD for ; Thu, 21 Mar 2019 14:20:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 81F606B0007; Thu, 21 Mar 2019 10:20:12 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7CEEA6B000C; Thu, 21 Mar 2019 10:20:12 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6BFBC6B000D; Thu, 21 Mar 2019 10:20:12 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id 204686B0007 for ; Thu, 21 Mar 2019 10:20:12 -0400 (EDT) Received: by mail-ed1-f72.google.com with SMTP id x29so2273799edb.17 for ; Thu, 21 Mar 2019 07:20:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=3GzvBNszJ2F9qzyq8uNQC4ioxjUcI3EqpQkesdmXCx4=; b=ArTxfEe91Pup0vvlmVWj5vsj8Ox30CogQokc1gnIL5gnZwZJ/pbzofaYkFdn+/hK/T K9gSqVPoxu0rNeFsRcRIzSw5sKpo32y0WKUPtj0Ozjv7aiLwHXCg9/g/kKN6ZTpp3vW8 npcSQhSpUcW/JnGqgTt+SaqPDkjlNgtwkq+Gy8F0V7R53QzQMinHNvIDepRwMy/zUcpx RhbhH+sJTJF8SPrflWr0AvMTS0yS8QKpoYCWTdlM8Y5cnEXzpyhInxgp4U+EuLsHD+Kr MZ0NnllW+NmeYPXwNNTYFGhwgqN3yY0ouHigFOXTjaje+76pMp0gZB2xX7nJqw2hQL0x pU+A== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAUcWLjisyd8paFWXmWx9nYnb/knA6+qIRnEecJRTi7ZJbG6X4tZ WB/KooejqdvHGrc/GYLOG8OjVArL/uYX0lRpXRxQlwm1FCUSlrrisBE6usw4YcWh7w2CrBguzT7 zsFUXbBT4/wWvd0+5pX5xens/skeGPAj9zMMMveeQESFGF9kM5rt13XhUYUVEVNqiqA== X-Received: by 2002:a17:906:278a:: with SMTP id j10mr2475288ejc.12.1553178011665; Thu, 21 Mar 2019 07:20:11 -0700 (PDT) X-Google-Smtp-Source: APXvYqwCqfwMDqbJIAcawGEBR7p/BeRKn8bKU3wrUIA+ZbrDf6HZp1dLAjXirKJoReWXdNkNLwJq X-Received: by 2002:a17:906:278a:: with SMTP id j10mr2475250ejc.12.1553178010813; Thu, 21 Mar 2019 07:20:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178010; cv=none; d=google.com; s=arc-20160816; b=QM1dYvdrZWHqKazIrkg2EGg6EwLYjwuxSqpKc7vZCggs53GVOImkWOhVLvy+t2BQk2 3VabhxXxmvrPSgolZAAuO+VtOUGZso+2Xd5IwXsMiCCad/4HS29rh8grj0nVMU3og/Ec 5PSxfPjHkqwUhCzPeonT/iGvfcWON28Em8p/2EsN+rrcesDpy8O2KnZ6tQ2jHCf/vT1v YI3sgL7RWglD56ebNFLqb18b5Tb1jRhkubscIhBmtpLOa83UukZLRuPFBlv2Ab7PhuIJ jQfotMyunK2D3FTENylMHQFH6+/1MtHmiA0X5+3fe66Xf60labUwRJTfj+VhZVJqypDC yGwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=3GzvBNszJ2F9qzyq8uNQC4ioxjUcI3EqpQkesdmXCx4=; b=uDBzc6lE/3EETcX1AE+DYJm2cvK7bE7ZByEkD8vRJ2R4HRuBHF3StiKvo0kzoZ6oxN ku2Fevou3hR8mUa2nnsocGlGMrLKKQ/t8OBVxQlhnlGaS2LZIekcEAXyTaTKO4xcxu1G ZN7qI6u3Km2SjFTzWNgjthoJ/0TcV23zOVU1PAy4QMImYcJXPOnG0D6o2Kq+5dtDIyWT 7x/PlxRX+pDXhnkucB0peDZDwIXbiAfcQUAEDjaT7yY9wUWD7juQP89KohI7+k/T8qqH TzYM3ws2kcETF+jEyq+B3qSbrN6IiK5X9XvXmHB5Hl4pwDjOwLJqia3y86obeFwFOCZT 0uAQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id d13si2081850edx.162.2019.03.21.07.20.10 for ; Thu, 21 Mar 2019 07:20:10 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BBFCBEBD; Thu, 21 Mar 2019 07:20:09 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3A4E93F575; Thu, 21 Mar 2019 07:20:06 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" , Vineet Gupta , linux-snps-arc@lists.infradead.org Subject: [PATCH v5 01/19] arc: mm: Add p?d_large() definitions Date: Thu, 21 Mar 2019 14:19:35 +0000 Message-Id: <20190321141953.31960-2-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP walk_page_range() is going to be allowed to walk page tables other than those of user space. For this it needs to know when it has reached a 'leaf' entry in the page tables. This information will be provided by the p?d_large() functions/macros. For arc, we only have two levels, so only pmd_large() is needed. CC: Vineet Gupta CC: linux-snps-arc@lists.infradead.org Signed-off-by: Steven Price Acked-by: Vineet Gupta --- arch/arc/include/asm/pgtable.h | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arc/include/asm/pgtable.h b/arch/arc/include/asm/pgtable.h index cf4be70d5892..0edd27bc7018 100644 --- a/arch/arc/include/asm/pgtable.h +++ b/arch/arc/include/asm/pgtable.h @@ -277,6 +277,7 @@ static inline void pmd_set(pmd_t *pmdp, pte_t *ptep) #define pmd_none(x) (!pmd_val(x)) #define pmd_bad(x) ((pmd_val(x) & ~PAGE_MASK)) #define pmd_present(x) (pmd_val(x)) +#define pmd_large(x) (pmd_val(pmd) & _PAGE_HW_SZ) #define pmd_clear(xp) do { pmd_val(*(xp)) = 0; } while (0) #define pte_page(pte) pfn_to_page(pte_pfn(pte)) From patchwork Thu Mar 21 14:19:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863601 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B25801515 for ; Thu, 21 Mar 2019 14:20:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 931FE2889C for ; Thu, 21 Mar 2019 14:20:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 86A062A2AE; Thu, 21 Mar 2019 14:20:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 98ECB2A297 for ; Thu, 21 Mar 2019 14:20:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F04566B000C; Thu, 21 Mar 2019 10:20:15 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DE93C6B000D; Thu, 21 Mar 2019 10:20:15 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C8A8A6B000E; Thu, 21 Mar 2019 10:20:15 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id 7D6166B000C for ; Thu, 21 Mar 2019 10:20:15 -0400 (EDT) Received: by mail-ed1-f72.google.com with SMTP id n12so2297513edo.5 for ; Thu, 21 Mar 2019 07:20:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=BRz1fLaVHJrWps04LeU/9kcyNHUo1yfdaodtNUtdXHA=; b=qxeETPs2KPdoDY3HyBQKhZJeky/OHceUrG8NFiTKcOOW+mscZuU602L5TQjzKQlgak vAilgOfB6F8HMN9alfJwCGAq8xZI8e7+XogS32lcjoWn5E9GFuSkcauOj2yWNY8bVHbg /e4yLUZTEQ+JO0Vl13nM8KqcB2f2bLGu1SUe4xGQaVdYSDM0NSQcHN0qfpVpeixNd3QY gQtJTvAd957yfUOv4d4W2cYu2tcEpNg5KDlq3QR1Fx6qtceGQJequ4TP+KW0PrjLBSHq yZZbToUWm53UfsH/x47rUrAvIdJlQd+fgvWOme8g6RE/ot4fiGxuNrEIGMxhYS+KRHt9 nDgg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAUi0Akrw5b4F3R2cSQmw/fnYtu4PybfqcCSGY0lsMhFSI6ByAHR SCdOPEUSRIyeIKZvcVgNxbAc3aTAKfOBbsKlU2ZVImQO52WR/N5yHjmZBTJHoz1aDdU1CpwUHT8 neEkEXMpFwhLnFxeWM3d+88jBr9YQG03m3dQOCLahtEBJwVFdClMR5AXDxfYNx2nNpw== X-Received: by 2002:a17:906:a85a:: with SMTP id dx26mr2386153ejb.206.1553178015036; Thu, 21 Mar 2019 07:20:15 -0700 (PDT) X-Google-Smtp-Source: APXvYqzLEZ6E+Y+Y7EtyrNmqHgfCM32p/zDw0AkW/qPT4TFUFayFVB6E/+zC8EraQz+hRjnQ9+Y1 X-Received: by 2002:a17:906:a85a:: with SMTP id dx26mr2386114ejb.206.1553178014213; Thu, 21 Mar 2019 07:20:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178014; cv=none; d=google.com; s=arc-20160816; b=wYWxIR58DRhZFeXuMDKoz8asxLu8nkrK5JgCi6JkUNoq0dMc/NtMsCaatVU7YAn1Kh iIgw+SZymqvycT+KVM70LMxFiumE/iRndtJWxv2OPhogyvVoXtrbcyJTOFv3SJJSur4R X+K/Wa82soprOqYE5aemzS5yPZVdmZQ3tHKzVzP128p3+5veB4RNzp4KnkLh9tHIroxR rbTv+GctIcUadofAagkfT+2sD0NitAzAXfAk/h5efmNeTQXmcyvSWLsW8LWQHTmSg2Ur 12iuK7Nebn5YRxcdBy8Ixz7rDQ9SaHQJepI0Q5dAgNhed+fLmwT9NtA9tLKAPFrm8koZ 0WkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=BRz1fLaVHJrWps04LeU/9kcyNHUo1yfdaodtNUtdXHA=; b=bx+stsjy+hMkxPQByDhupgg/j/E/xp4Zn6D0MqP5lqQ5rg1rQTlI6QPAk1/8el00uh oWa8q1/2oe1/Bc9rx/OdM0w7JCqJUYSqbIhEAicqDeOYcBwmyLaZF6ICw3M0GX2Xk57K e7ZdKF103htvwT9Wgj9aREAmQeY5UWBlU3BPoC8o7oZ7jWyuFJ9Tyy/nwooTDE29mSvs UNiQTyc8u5CEbZSEBy8iZGpxW/M38AbHLwQIhE/KsRJUhkOXw925romILzg+RrHgPycN bNwMS1SAz3z9KwrvyD9n7iORNyXg969lLa2C0sDi8xNtwhvy+h3thtpItUnzgVIyB+7z KA0A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id n23si1527317ejk.268.2019.03.21.07.20.13 for ; Thu, 21 Mar 2019 07:20:14 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 44B84165C; Thu, 21 Mar 2019 07:20:13 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 079C73F575; Thu, 21 Mar 2019 07:20:09 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v5 02/19] arm64: mm: Add p?d_large() definitions Date: Thu, 21 Mar 2019 14:19:36 +0000 Message-Id: <20190321141953.31960-3-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP walk_page_range() is going to be allowed to walk page tables other than those of user space. For this it needs to know when it has reached a 'leaf' entry in the page tables. This information will be provided by the p?d_large() functions/macros. For arm64, we already have p?d_sect() macros which we can reuse for p?d_large(). CC: Catalin Marinas CC: Will Deacon Signed-off-by: Steven Price --- arch/arm64/include/asm/pgtable.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h index de70c1eabf33..6eef345dbaf4 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -428,6 +428,7 @@ extern pgprot_t phys_mem_access_prot(struct file *file, unsigned long pfn, PMD_TYPE_TABLE) #define pmd_sect(pmd) ((pmd_val(pmd) & PMD_TYPE_MASK) == \ PMD_TYPE_SECT) +#define pmd_large(pmd) pmd_sect(pmd) #if defined(CONFIG_ARM64_64K_PAGES) || CONFIG_PGTABLE_LEVELS < 3 #define pud_sect(pud) (0) @@ -511,6 +512,7 @@ static inline phys_addr_t pmd_page_paddr(pmd_t pmd) #define pud_none(pud) (!pud_val(pud)) #define pud_bad(pud) (!(pud_val(pud) & PUD_TABLE_BIT)) #define pud_present(pud) pte_present(pud_pte(pud)) +#define pud_large(pud) pud_sect(pud) #define pud_valid(pud) pte_valid(pud_pte(pud)) static inline void set_pud(pud_t *pudp, pud_t pud) From patchwork Thu Mar 21 14:19:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863603 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6331818EC for ; Thu, 21 Mar 2019 14:20:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 45A332A2AE for ; Thu, 21 Mar 2019 14:20:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3668D2A2AF; Thu, 21 Mar 2019 14:20:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 98DD32A17A for ; Thu, 21 Mar 2019 14:20:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2A1CB6B000D; Thu, 21 Mar 2019 10:20:20 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2514E6B000E; Thu, 21 Mar 2019 10:20:20 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 142CB6B0010; Thu, 21 Mar 2019 10:20:20 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id B33386B000D for ; Thu, 21 Mar 2019 10:20:19 -0400 (EDT) Received: by mail-ed1-f71.google.com with SMTP id t4so2300433eds.1 for ; Thu, 21 Mar 2019 07:20:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=MqMqukGILHQS5vjuNGdSQFWysz8AgSB4keSNgmoOKiU=; b=UxiSczopN3UepZAddtjQxdlXzpDlGXg3u9owE1Rg0uW0SLdAaIVDdEFaub9Vi16No9 vr844AkHb+mz9rStJQfmvqkP7R071l01KYoWSwvQs6jDZhZxlMhMW0FhryDxlnr6khSo fpHuF2s5Btsfdzru50Xcj3YPCGUU6S0FXkxgii2lyfeJvY/WPjfonOaO1miEHu4zycI0 1o1EWdDJQxTYEJjRx5SSyLFJOLkKPm91boILXxlldYULKS8qaP1jfdJdmL4ZaObEfx1h fEZxTYH0iAwY/QbZduF18Tdr3tYPF/uQnrubP70DhgFCUzkjJFDw49Nj8vyy5Bc4zHD/ 4TPg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAUZCbEHpmtjIg+TPyQB8S8F9fm9Tc0IaffUJtGPS8MzP1hxuI61 WeO21oHRq76HJ1syGDihJ6irCPn8KfDBvECd4qCI4pIAx3nAn6uh7M3XXDbj0fmDVKhIMRsXEfF Zv1EXp2D2lGPRfT+3J6Wvv0SpcbOOxR7pKkMcX7L/tDCKtcMEUqAjSmcozGy3Ejvx1Q== X-Received: by 2002:a50:8ed1:: with SMTP id x17mr2627388edx.168.1553178019255; Thu, 21 Mar 2019 07:20:19 -0700 (PDT) X-Google-Smtp-Source: APXvYqyXn0M2spWIw9+Bhyc3nvCYusy0dX8xyvrlN0Od1ogogG980QYivZWWMVI1TI/CI1GcFHto X-Received: by 2002:a50:8ed1:: with SMTP id x17mr2627337edx.168.1553178018308; Thu, 21 Mar 2019 07:20:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178018; cv=none; d=google.com; s=arc-20160816; b=mn7spcl9ETJCcyfXJheSFP1Xn9VDA381xQUYTzRbCXrRqw7SXpPf3RwgJqbivCkDrA IQ08VK3ocZu34/hu4Loi9da4bGBr+XMsVsb3lj1UuVE0+yoQ/bIMPZlM7e7pfTnUJl/m vZUHeFsX2ML9m8ieqSy5GTx/SudqGODXdFok3WKtQpgYJggj2F0b/rCppZ74bZ04c0EY /02frRcowOnXxVZcdllaGv9iCWwPtC7OgoS4mk1Q3iSaisWSgFr2jjuNf/D2KcldOLrf HPWZhfPDEN6TBf7Wr5KCC2X2I4UGQGCdcT4haIPUjYoWhjpAl5qdxhxlTtA2gnxZmYIB jHiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=MqMqukGILHQS5vjuNGdSQFWysz8AgSB4keSNgmoOKiU=; b=nq2ra2uMG8HOQgxX1m+XcIVBjRQ2ltP6Gji4DxtaKqYmAVTrWYiHq89N1mxxOytukn mSOrSaSgdcWlfBBkgbGuGavthKHckBfo5esVDbzJ5uflPEJ80gorwSg49nSvN+0Tr1Wc lGTZ4Q59w7nrXtprhHkA1r25S8WYM8eS7B299Zmi4j4fAh7TYGtoMU5wTjnkVKL69ASj UhywC72XbBkODHAjS2bXkXK7YhlsL4ZlWvBH8wCD99IDK9BKPYNNzUBFsSnMcKcAocMc qrFshmYCJz56G9puQUIDZnnbVjdC2FSDYGvg4MKFIAS0GG7bXLz702WrIuI1DXAtqvN9 vzCA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id g24si125209eje.214.2019.03.21.07.20.18 for ; Thu, 21 Mar 2019 07:20:18 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5957680D; Thu, 21 Mar 2019 07:20:17 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 84C153F575; Thu, 21 Mar 2019 07:20:13 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" , Ralf Baechle , Paul Burton , James Hogan , linux-mips@vger.kernel.org Subject: [PATCH v5 03/19] mips: mm: Add p?d_large() definitions Date: Thu, 21 Mar 2019 14:19:37 +0000 Message-Id: <20190321141953.31960-4-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP walk_page_range() is going to be allowed to walk page tables other than those of user space. For this it needs to know when it has reached a 'leaf' entry in the page tables. This information is provided by the p?d_large() functions/macros. For mips, we only support large pages on 64 bit. For 64 bit if _PAGE_HUGE is defined we can simply look for it. When not defined we can be confident that there are no large pages in existence and fall back on the generic implementation (added in a later patch) which returns 0. CC: Ralf Baechle CC: Paul Burton CC: James Hogan CC: linux-mips@vger.kernel.org Signed-off-by: Steven Price Acked-by: Paul Burton --- arch/mips/include/asm/pgtable-64.h | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/arch/mips/include/asm/pgtable-64.h b/arch/mips/include/asm/pgtable-64.h index 93a9dce31f25..42162877ac62 100644 --- a/arch/mips/include/asm/pgtable-64.h +++ b/arch/mips/include/asm/pgtable-64.h @@ -273,6 +273,10 @@ static inline int pmd_present(pmd_t pmd) return pmd_val(pmd) != (unsigned long) invalid_pte_table; } +#ifdef _PAGE_HUGE +#define pmd_large(pmd) ((pmd_val(pmd) & _PAGE_HUGE) != 0) +#endif + static inline void pmd_clear(pmd_t *pmdp) { pmd_val(*pmdp) = ((unsigned long) invalid_pte_table); @@ -297,6 +301,10 @@ static inline int pud_present(pud_t pud) return pud_val(pud) != (unsigned long) invalid_pmd_table; } +#ifdef _PAGE_HUGE +#define pud_large(pud) ((pud_val(pud) & _PAGE_HUGE) != 0) +#endif + static inline void pud_clear(pud_t *pudp) { pud_val(*pudp) = ((unsigned long) invalid_pmd_table); From patchwork Thu Mar 21 14:19:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863611 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 15BD41908 for ; Thu, 21 Mar 2019 14:20:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ED4882A179 for ; Thu, 21 Mar 2019 14:20:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EB3E02A2AD; Thu, 21 Mar 2019 14:20:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 992BC2A29B for ; Thu, 21 Mar 2019 14:20:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2A4036B000E; Thu, 21 Mar 2019 10:20:25 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 229BB6B0010; Thu, 21 Mar 2019 10:20:25 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0F3136B0266; Thu, 21 Mar 2019 10:20:25 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id AF5A46B000E for ; Thu, 21 Mar 2019 10:20:24 -0400 (EDT) Received: by mail-ed1-f71.google.com with SMTP id o9so2275364edh.10 for ; Thu, 21 Mar 2019 07:20:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=af5cRJh4Tq+8o3hmoF/y3vADwLOSPWNyyODfZV8nBh8=; b=tgkF7hdQpqsBoGJ9jnpkjGSLxvlJ+CgjQrIYX/Y+Y5bzdhguz7yiUGzc8t7sBomNvk HuMRydERYUQ8H4vp9gG80ruk58GetRLGQRl2NJn0U1y/x4X8dKtIfoYoytHgl8CON6fY j1/5DV8u8kYWx/ecmz7eoUZjjddMF8LlWHCs83vtCzYlYvMFINIAjSTvYBmz1mMWIAwj SHqpof+CxP+iMj5VdpizBLvc3N2uJsZTIowql8vkDVbVt9ENKsbczOrFQmyFkeR8Rjhj ekEkYxwBqa1KKtShjwn8iC41gMGUS+gCDQM6XYAYvL5HnlBGkAdavpcV660pfmFusL0G 8wLA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAUOgrR4sYGQK0NQT70vB9dmkm6JjCoKyE1e2YVckg1q2ikpz9/Z 3edYoFiHzUGpxllq9i+PIp4RPU+LpXC8dt2vpiBDLUzjZpmCN+OGPSzutOAcxeeP/btUAR8Uop+ 8o1Co5B+5e0Cy9lfoWZrFPDM4nCxqN9oLWjqhM0P/W0hz2GqBlV7Vb9iYWXABk5B31w== X-Received: by 2002:a50:b3ad:: with SMTP id s42mr2735935edd.142.1553178024231; Thu, 21 Mar 2019 07:20:24 -0700 (PDT) X-Google-Smtp-Source: APXvYqxZ1fcwVydW8ibxqc92amlEZSvRWV9pqZhvMnzU+VTJPGKdBL/i843Hto4FYCYhHJRBwxj2 X-Received: by 2002:a50:b3ad:: with SMTP id s42mr2735861edd.142.1553178022662; Thu, 21 Mar 2019 07:20:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178022; cv=none; d=google.com; s=arc-20160816; b=FfA+V8EZUqijX4Gwo+fwhW+qXv657wIS75eCRhw96viUsGyfCyHM9tAY0OgYbJPE1P 78ZkgOne70s8XCDXlRSlgUatOaBb/M/jtlDdq3dAiG7g0q5dI5SzvtQgTh0Rz3/MFFgc qS9WfgIgGCB8pE4Tc5zD260MrLoRz3QASJzVOBUc/Ed6jGrqLf8ITsGDV7MpQ8a/hhan 30sQqguYQWpUxCtiv1bI9HOJ7E402CKSrLZw3KQdhrl4tpGV/oqM4D4bIyMgjnXPLfi0 1mcGBHI8eSus6kyofl8phPF9gNKNWXucSyFR6A+3Md8SMPMd+X0RZ9wQpXu1LEx2xxFt zgJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=af5cRJh4Tq+8o3hmoF/y3vADwLOSPWNyyODfZV8nBh8=; b=NoDjWeyP/WPQWw+q7JHhaDru41haXSuW/Yzjqk3D2ZONoyS/tmfC7LabSRTaoCeVpg 7zDl77RWNvLDCm/LBfvV3JIXVx+CPUDERNzgx9kR4yOuJ0RaRAvwtMyPeKLGprzqt/dK h2p+WafIXPOUyIn7XK7Tn+D5VvPscmEgFl1506AcJjljlAqyVGCWkm1Jzv5mJEkQNcvp CSp1AW29MvIBJ/EQLTlE8GkFuX1GgJRnViDj6VG9s+za4YvaASmtMW1NSe4LYdKMsvZq bSjqG2MUIv+kSLRlTH7aEsn/M3kFCvxRTpdguSH8CCKlpueSENH82mXKlBSidJGtrKmU N5NA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id a51si476428edc.78.2019.03.21.07.20.22 for ; Thu, 21 Mar 2019 07:20:22 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9283CEBD; Thu, 21 Mar 2019 07:20:21 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 99F713F575; Thu, 21 Mar 2019 07:20:17 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , linuxppc-dev@lists.ozlabs.org, kvm-ppc@vger.kernel.org Subject: [PATCH v5 04/19] powerpc: mm: Add p?d_large() definitions Date: Thu, 21 Mar 2019 14:19:38 +0000 Message-Id: <20190321141953.31960-5-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP walk_page_range() is going to be allowed to walk page tables other than those of user space. For this it needs to know when it has reached a 'leaf' entry in the page tables. This information is provided by the p?d_large() functions/macros. For powerpc pmd_large() was already implemented, so hoist it out of the CONFIG_TRANSPARENT_HUGEPAGE condition and implement the other levels. Also since we now have a pmd_large always implemented we can drop the pmd_is_leaf() function. CC: Benjamin Herrenschmidt CC: Paul Mackerras CC: Michael Ellerman CC: linuxppc-dev@lists.ozlabs.org CC: kvm-ppc@vger.kernel.org Signed-off-by: Steven Price --- arch/powerpc/include/asm/book3s/64/pgtable.h | 30 ++++++++++++++------ arch/powerpc/kvm/book3s_64_mmu_radix.c | 12 ++------ 2 files changed, 24 insertions(+), 18 deletions(-) diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index 581f91be9dd4..f6d1ac8b832e 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -897,6 +897,12 @@ static inline int pud_present(pud_t pud) return !!(pud_raw(pud) & cpu_to_be64(_PAGE_PRESENT)); } +#define pud_large pud_large +static inline int pud_large(pud_t pud) +{ + return !!(pud_raw(pud) & cpu_to_be64(_PAGE_PTE)); +} + extern struct page *pud_page(pud_t pud); extern struct page *pmd_page(pmd_t pmd); static inline pte_t pud_pte(pud_t pud) @@ -940,6 +946,12 @@ static inline int pgd_present(pgd_t pgd) return !!(pgd_raw(pgd) & cpu_to_be64(_PAGE_PRESENT)); } +#define pgd_large pgd_large +static inline int pgd_large(pgd_t pgd) +{ + return !!(pgd_raw(pgd) & cpu_to_be64(_PAGE_PTE)); +} + static inline pte_t pgd_pte(pgd_t pgd) { return __pte_raw(pgd_raw(pgd)); @@ -1093,6 +1105,15 @@ static inline bool pmd_access_permitted(pmd_t pmd, bool write) return pte_access_permitted(pmd_pte(pmd), write); } +#define pmd_large pmd_large +/* + * returns true for pmd migration entries, THP, devmap, hugetlb + */ +static inline int pmd_large(pmd_t pmd) +{ + return !!(pmd_raw(pmd) & cpu_to_be64(_PAGE_PTE)); +} + #ifdef CONFIG_TRANSPARENT_HUGEPAGE extern pmd_t pfn_pmd(unsigned long pfn, pgprot_t pgprot); extern pmd_t mk_pmd(struct page *page, pgprot_t pgprot); @@ -1119,15 +1140,6 @@ pmd_hugepage_update(struct mm_struct *mm, unsigned long addr, pmd_t *pmdp, return hash__pmd_hugepage_update(mm, addr, pmdp, clr, set); } -/* - * returns true for pmd migration entries, THP, devmap, hugetlb - * But compile time dependent on THP config - */ -static inline int pmd_large(pmd_t pmd) -{ - return !!(pmd_raw(pmd) & cpu_to_be64(_PAGE_PTE)); -} - static inline pmd_t pmd_mknotpresent(pmd_t pmd) { return __pmd(pmd_val(pmd) & ~_PAGE_PRESENT); diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index f55ef071883f..1b57b4e3f819 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -363,12 +363,6 @@ static void kvmppc_pte_free(pte_t *ptep) kmem_cache_free(kvm_pte_cache, ptep); } -/* Like pmd_huge() and pmd_large(), but works regardless of config options */ -static inline int pmd_is_leaf(pmd_t pmd) -{ - return !!(pmd_val(pmd) & _PAGE_PTE); -} - static pmd_t *kvmppc_pmd_alloc(void) { return kmem_cache_alloc(kvm_pmd_cache, GFP_KERNEL); @@ -460,7 +454,7 @@ static void kvmppc_unmap_free_pmd(struct kvm *kvm, pmd_t *pmd, bool full, for (im = 0; im < PTRS_PER_PMD; ++im, ++p) { if (!pmd_present(*p)) continue; - if (pmd_is_leaf(*p)) { + if (pmd_large(*p)) { if (full) { pmd_clear(p); } else { @@ -593,7 +587,7 @@ int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte, else if (level <= 1) new_pmd = kvmppc_pmd_alloc(); - if (level == 0 && !(pmd && pmd_present(*pmd) && !pmd_is_leaf(*pmd))) + if (level == 0 && !(pmd && pmd_present(*pmd) && !pmd_large(*pmd))) new_ptep = kvmppc_pte_alloc(); /* Check if we might have been invalidated; let the guest retry if so */ @@ -662,7 +656,7 @@ int kvmppc_create_pte(struct kvm *kvm, pgd_t *pgtable, pte_t pte, new_pmd = NULL; } pmd = pmd_offset(pud, gpa); - if (pmd_is_leaf(*pmd)) { + if (pmd_large(*pmd)) { unsigned long lgpa = gpa & PMD_MASK; /* Check if we raced and someone else has set the same thing */ From patchwork Thu Mar 21 14:19:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863613 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A5D6F13B5 for ; Thu, 21 Mar 2019 14:20:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 86FF22A2AA for ; Thu, 21 Mar 2019 14:20:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 84AE22A2AD; Thu, 21 Mar 2019 14:20:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 00DE12A2AA for ; Thu, 21 Mar 2019 14:20:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2A3456B0010; Thu, 21 Mar 2019 10:20:28 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 253DD6B0266; Thu, 21 Mar 2019 10:20:28 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 141EA6B0269; Thu, 21 Mar 2019 10:20:28 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id BE1626B0010 for ; Thu, 21 Mar 2019 10:20:27 -0400 (EDT) Received: by mail-ed1-f71.google.com with SMTP id s27so2274550eda.16 for ; Thu, 21 Mar 2019 07:20:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=1lP5uxd4lkeHOs2R/letx90Hs4gQDKi/XVW6tvSp3kc=; b=BKt7wiSEw+NgGnYz8JOIufRMgko3c22NZOTJty1cVuusxrtTlvYKpjqjRkQ6Ol2PIA d3rMD0IiJTODVRkd/r8Qg8p6PWKxLmsXiSVlwejxHXLZCa/Ue31X25/rk33rY+GjxnXX Ytatc3bNPg6WMqYqLRX8rDnM7p/kyymoHpLX+hDI465TIH3dqZHRGHLqA3Fe51q05NL5 uU+b2ujv/xCGZPKRCHVDGnVCaccs3E9XWi94CbkFzZfMw4FxLk7g3YPqriIdjW/YUo6a K40B6r0Upqq2gXeEzdPJAN+hBbXS9WuwGrIoGEXQ/LxBNTMGlpFxls2KJqZVlNg+Kaii LbgQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAVCtVL+gf65WRDlMqgoFbjf+HMhTasFKM0IL5nGJf1F5w8TOS/U ls1qIPOs5hp3CmeYFqIgXiiyG0w7TwE6cce1J+rRA3Yr+qVkVWDcMZOAbF0EpsoMg78Cygw03Pi mR+LM7cdMExIqCH8OpOozQV2wnfG5SpmqOzi7c/VnX+1pGqbFnDYtP0o4RhBazgXkZg== X-Received: by 2002:a17:906:2983:: with SMTP id x3mr1673449eje.130.1553178027304; Thu, 21 Mar 2019 07:20:27 -0700 (PDT) X-Google-Smtp-Source: APXvYqz9K+XE6GDrTOhKYie9Ky4hlliU49d8q1Stw6vFZzQwyOe9GMUBe9K8nM7d6eAJHS/f4+hH X-Received: by 2002:a17:906:2983:: with SMTP id x3mr1673411eje.130.1553178026520; Thu, 21 Mar 2019 07:20:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178026; cv=none; d=google.com; s=arc-20160816; b=GNO3gudEZY6DNAHZUhZp7B9fzdkVNIuybIaDdZRVmSjVhiWJnvQYe3hZGTJfS+6EcY 54rbB9en2FhXdpQBcRLMNhdDtRaU2VDs+h62/+mX17FkyDH2yYjN8KVyVho/Yta8bhdC N7c29Oxb8YRRdTAg4tdCm/cA9BgDFtbhYtR9dWs5HUGQLRb97Mcv0KEqarhVSF3xWh+2 ViS69k1vgu69O2aBRae8XhLc5sHJfLFdJSW3KYTMUh8DEmG4RkEPrfVpbnI4bfPzohkV hSpDyCiiyD1FbQHDDWasK1tTGgLuygG4zi678zBZNuSDt8XC/UyazAkrRBQV5qTNkd3Y tE5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=1lP5uxd4lkeHOs2R/letx90Hs4gQDKi/XVW6tvSp3kc=; b=aEcz2Lrlqu2TdPJElvPFY5voiJJ4NtrZ83zEdHNXKX6dm8DIRKKhAd6ELzx9b16dm7 HPy4NdADZW937ArWpq+327kY2WbQ2519pvM5xkBL9t/pr781//jjUNY9CbO/0bXfao8x EFIN1ADHKuudYKx6lqvdLPjhVS6k0n3V9diBwxI1BL7clgDI6jJsUqdIDr1buqlsk3OJ JyGK1eHGDwiB8LDBBTRivlWNuFcvaQOi+UUsjMw41BgwtbFenGF74hNWIAl4dogSwMQj gFttu83JmuRjw6kr4iAHweZXkm7R1cW86zj57PYXHdeORtfKg7+2JIngZlRglnxZdMDR fWIQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id e3si2150143edd.270.2019.03.21.07.20.26 for ; Thu, 21 Mar 2019 07:20:26 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 853FD168F; Thu, 21 Mar 2019 07:20:25 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D266F3F575; Thu, 21 Mar 2019 07:20:21 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" , Palmer Dabbelt , Albert Ou , linux-riscv@lists.infradead.org Subject: [PATCH v5 05/19] riscv: mm: Add p?d_large() definitions Date: Thu, 21 Mar 2019 14:19:39 +0000 Message-Id: <20190321141953.31960-6-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP walk_page_range() is going to be allowed to walk page tables other than those of user space. For this it needs to know when it has reached a 'leaf' entry in the page tables. This information is provided by the p?d_large() functions/macros. For riscv a page is large when it has a read, write or execute bit set on it. CC: Palmer Dabbelt CC: Albert Ou CC: linux-riscv@lists.infradead.org Signed-off-by: Steven Price --- arch/riscv/include/asm/pgtable-64.h | 7 +++++++ arch/riscv/include/asm/pgtable.h | 7 +++++++ 2 files changed, 14 insertions(+) diff --git a/arch/riscv/include/asm/pgtable-64.h b/arch/riscv/include/asm/pgtable-64.h index 7aa0ea9bd8bb..73747d9d7c66 100644 --- a/arch/riscv/include/asm/pgtable-64.h +++ b/arch/riscv/include/asm/pgtable-64.h @@ -51,6 +51,13 @@ static inline int pud_bad(pud_t pud) return !pud_present(pud); } +#define pud_large pud_large +static inline int pud_large(pud_t pud) +{ + return pud_present(pud) + && (pud_val(pud) & (_PAGE_READ | _PAGE_WRITE | _PAGE_EXEC)); +} + static inline void set_pud(pud_t *pudp, pud_t pud) { *pudp = pud; diff --git a/arch/riscv/include/asm/pgtable.h b/arch/riscv/include/asm/pgtable.h index 1141364d990e..9570883c79e7 100644 --- a/arch/riscv/include/asm/pgtable.h +++ b/arch/riscv/include/asm/pgtable.h @@ -111,6 +111,13 @@ static inline int pmd_bad(pmd_t pmd) return !pmd_present(pmd); } +#define pmd_large pmd_large +static inline int pmd_large(pmd_t pmd) +{ + return pmd_present(pmd) + && (pmd_val(pmd) & (_PAGE_READ | _PAGE_WRITE | _PAGE_EXEC)); +} + static inline void set_pmd(pmd_t *pmdp, pmd_t pmd) { *pmdp = pmd; From patchwork Thu Mar 21 14:19:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863615 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0328C1390 for ; Thu, 21 Mar 2019 14:20:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DBD23292FA for ; Thu, 21 Mar 2019 14:20:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CF30B2A29B; Thu, 21 Mar 2019 14:20:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6BAF7292FA for ; Thu, 21 Mar 2019 14:20:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 31BB76B0266; Thu, 21 Mar 2019 10:20:32 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 2A0A76B0269; Thu, 21 Mar 2019 10:20:32 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 168D56B026A; Thu, 21 Mar 2019 10:20:32 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id B56676B0266 for ; Thu, 21 Mar 2019 10:20:31 -0400 (EDT) Received: by mail-ed1-f72.google.com with SMTP id v26so2244219edr.23 for ; Thu, 21 Mar 2019 07:20:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=EIMxzs3KcJ4jmgX2iikj+tJ0Q4+Nen0h2D0kdVtq1V0=; b=BnUL4HKd1GxO3txNz8pb1jfqzKxiEv6jRFOgVoxnaeaoRV3hsJpcOKfeiXJ33PDY+K Zc15ud+zTCMP0d/AbrNjzPfbkz1V61ZYf7ZFhWzP7s8xl7t2Z91mfBOTQcNwJa63xwRg fbeR9jSqCG6efCW7FqLpHIkFKRxbFmMRyiKCm6xWUdVqvAzeGsYOcC+7KMrXDRd5nYZz K6VC0MlEnxC6KoqMqclXHKyk4HeFSBchGYh54RJD6JjIAfaH10yXyBbgJPgcuRCwC4gh 910DXFciv/IhZKLJLn38zg3cPng8ah9ZPpG4tIQLvq5FyDgZWkE31C3jSQ3CxsKjd9Hv qDng== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAV2qXPvhP55h+uPV6q4mOsDMyRZ10zDXCOMe2Gisx/hZABFrpZ/ TypwSAcfnNUnZfhBv5nY90L/MApDNs65ULB0lx17YAKnIghPNJ+Bkwy880niony6PblXFtwhl1l 8mckb6c/FYWbRtJaNa3kV2ZEyDUv/OIwEwI/uIuleq7DVScAVlCJAF3BiaWI++Bx0Mg== X-Received: by 2002:a17:906:1942:: with SMTP id b2mr2505077eje.5.1553178031276; Thu, 21 Mar 2019 07:20:31 -0700 (PDT) X-Google-Smtp-Source: APXvYqz1ARIHJQlqUjM+cATkP9zGJpekhQDJtmJfGyl4BQHMSHlDLtTcjVwVs0SR2LbmzU8tygS0 X-Received: by 2002:a17:906:1942:: with SMTP id b2mr2505042eje.5.1553178030385; Thu, 21 Mar 2019 07:20:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178030; cv=none; d=google.com; s=arc-20160816; b=EByZdCVVvztJxZldlWgZ9DnzbXyN2xO3hIZaBvKnz2JyGPuGXqM3dt4wW86NLX/vqf Dq0tn9tKsXAjDgLNIQ4Jq8DZbwvncyzs+LAznAC9JQAyqouJeUHbhkQ/XNxyZlQZc6eN IC4iXJ2bIEjkq0kxBlA8ffxfPx0PXsKPIzcZcwcvAll4y1O1IPz87/VSQdodQ8DjVURU 20oUDEpuKd0YRINgXm5UrYat1nlF1YmMHDws1gg8pQDo/W6/Z6+s2H4IquF9FKqfSvbw KQaipno4+sBOm+RjweHUb0NhM1qN8YTuzI0eMB+tzAhj/gDiw9RlxXArKdomyzsGCGrA Qh8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=EIMxzs3KcJ4jmgX2iikj+tJ0Q4+Nen0h2D0kdVtq1V0=; b=k4ITfEhZ2eyvtKlgXTa2MipOYASg6tNFwCi0muCU65yI7B13J52vS1dgq+6xjdAThU zP5cBwBtrtcNsG1m3XLmJWsx7ATG+wHxScYNM1svAzsPzjuBqO4CauysF6fcZYGujdAz YB53FSnWQIlzAbGB9zHW1hIlgsKZwYhb2GZJ6P5cSVg4u9XRiRdzANCMKe05pfTk9ixD 7r/5wmp1tIywghVOtXeCmkyUdD3IgQTjQ+5vx6JRcQp7e5XcQuS/SE5dwhADKJz8N7+v e9X4Cl9XDYWf4fDgxmizPKuqimn+eZ/eE6G61H2v/BKv0xSaAOyNG/z+Gh86St995zRV zstQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id p24si1732630ejg.203.2019.03.21.07.20.30 for ; Thu, 21 Mar 2019 07:20:30 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 770BE80D; Thu, 21 Mar 2019 07:20:29 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C5D203F575; Thu, 21 Mar 2019 07:20:25 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" , Martin Schwidefsky , Heiko Carstens , linux-s390@vger.kernel.org Subject: [PATCH v5 06/19] s390: mm: Add p?d_large() definitions Date: Thu, 21 Mar 2019 14:19:40 +0000 Message-Id: <20190321141953.31960-7-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP walk_page_range() is going to be allowed to walk page tables other than those of user space. For this it needs to know when it has reached a 'leaf' entry in the page tables. This information is provided by the p?d_large() functions/macros. For s390, pud_large() and pmd_large() are already implemented as static inline functions. Add a #define so we don't pick up the generic version introduced in a later patch. CC: Martin Schwidefsky CC: Heiko Carstens CC: linux-s390@vger.kernel.org Signed-off-by: Steven Price --- arch/s390/include/asm/pgtable.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h index 76dc344edb8c..3ad4c69e1f2d 100644 --- a/arch/s390/include/asm/pgtable.h +++ b/arch/s390/include/asm/pgtable.h @@ -679,6 +679,7 @@ static inline int pud_none(pud_t pud) return pud_val(pud) == _REGION3_ENTRY_EMPTY; } +#define pud_large pud_large static inline int pud_large(pud_t pud) { if ((pud_val(pud) & _REGION_ENTRY_TYPE_MASK) != _REGION_ENTRY_TYPE_R3) @@ -696,6 +697,7 @@ static inline unsigned long pud_pfn(pud_t pud) return (pud_val(pud) & origin_mask) >> PAGE_SHIFT; } +#define pmd_large pmd_large static inline int pmd_large(pmd_t pmd) { return (pmd_val(pmd) & _SEGMENT_ENTRY_LARGE) != 0; From patchwork Thu Mar 21 14:19:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863619 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CA73A14DE for ; Thu, 21 Mar 2019 14:20:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B030228CBD for ; Thu, 21 Mar 2019 14:20:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A46242A2A6; Thu, 21 Mar 2019 14:20:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7CED228CBD for ; Thu, 21 Mar 2019 14:20:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 11E686B0269; Thu, 21 Mar 2019 10:20:36 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0A5126B026A; Thu, 21 Mar 2019 10:20:36 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EAD486B026B; Thu, 21 Mar 2019 10:20:35 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by kanga.kvack.org (Postfix) with ESMTP id 9CB9D6B0269 for ; Thu, 21 Mar 2019 10:20:35 -0400 (EDT) Received: by mail-ed1-f70.google.com with SMTP id m31so2296359edm.4 for ; Thu, 21 Mar 2019 07:20:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=PlduaFV1LjO5scXRf9kwcI+ukcg7zXNoRrXpWadVZPc=; b=N0Ep4EHLkLhXkL0NcjheJidPAdtipuUG8iMqIDeHzuw96iNHFCf3eNbyyZZ/6Nj/sF jJZNh4y+T/7uw15zuqssAP20HpNyud2cGUWAKdC8wIuMKzewN26VNRetJSLxA1YhBEav XQaqYgeDr8t0YGh3ysJp0+Ixll3GHkUEjHQomnOOUZYtDXKsEHr50vbrKHSi+oIzSbZf 4Dq+1MKDQjG9By7l4/ZKOxugvK57UkOUDSQKpiosMWcZyz6qENs0ovFolDAhY6pvtCv7 KzxlGYXw6awLetQZtYhDRRLp9hWi5dbbKARK9cVx41om+UVsPybqvkZmLZMGfLc5TJY3 PGEA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAVDph4mNvSn9dU0BoP/pXYuUXGjHOWObDOeh/alo1R3tnsZKw/B SEXvVqlQgAIM6gSpvWgmTFsolp8skfIUZk3jzZ+7mgiceloKP+RN/fLQ6qcRB8j10e6R/4LK/6W EU1164eSlrPvART5HqK1y/UD1wfEfR95bx3+k4oZLx2Iypi4X6NHWCs1HG1dYV7AAJQ== X-Received: by 2002:a17:906:4453:: with SMTP id i19mr2471635ejp.39.1553178035114; Thu, 21 Mar 2019 07:20:35 -0700 (PDT) X-Google-Smtp-Source: APXvYqxh1CutLRerQM19q8hwnttsZNEouKSVvR52ngB5hcfs8JbXSdW3ydnk1Vh6QyrxjEEUPpq/ X-Received: by 2002:a17:906:4453:: with SMTP id i19mr2471593ejp.39.1553178034294; Thu, 21 Mar 2019 07:20:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178034; cv=none; d=google.com; s=arc-20160816; b=I9BllptGDmRfEOcyLdKBytFh1XQqfOOn1qGTIMrn4FRqIZienyY8Pc3nhN+y2qt+bC bbMuSJKXob+SQeiw4WEXNRuyf90Ce8UKEMvbqR9ilVzbcJvMuCtStY2TI4okX2Gdq+Id cwCENdDkJRhVn/P14Vsgt3+IjzmRawyHNMmqrnNTPNj+SpMvP7srvtSjs6KpQ3XhR2oS 2qGa7IW1shYzEeMcOmdv30Hwn3eVVUQSaOYFSosEGIcCV3Ezw5U78JxzHO8hpiEdt5cT A4vF5/APu9yaZLZgoGWfLgnNu68yCK9q6Ej5Bn7+w228G3sDbv3EE9LgLPFYmcqFilKr dQJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=PlduaFV1LjO5scXRf9kwcI+ukcg7zXNoRrXpWadVZPc=; b=r1yJI44aPNJ605WoZWPepc0x+BolHWAVlt5N8YKFdhyms/WfH1M+8x6QTzK6kIPNDy kiDkBBEg65XXwVTSLZnjVnhay9NFhzP0BkTwd/hDdHpzcTKqM/oSDF17NYpjul/Y9Clc PlIWrKeXjTNsrBG8fLKHuxEHjSwWVCJnCIts/NXjZv35w4AlaD/14ZS12151AGTiNYZu f/aaI1tmEIX2A1T2OGAYaTS/s6bZdoQWQqPQiSgtPFrPYf/zM0eUgmpHliDLTKYW2Hqy i+UeLz+LlCtlf4GLrPgm+QW30/oqI44dnacU9CwHrRbL3yGViywaVcW0kfsWimeGT7bB J9Ow== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id g18si1218846edh.169.2019.03.21.07.20.33 for ; Thu, 21 Mar 2019 07:20:34 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4437D19BF; Thu, 21 Mar 2019 07:20:33 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B70D93F575; Thu, 21 Mar 2019 07:20:29 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" , "David S. Miller" , sparclinux@vger.kernel.org Subject: [PATCH v5 07/19] sparc: mm: Add p?d_large() definitions Date: Thu, 21 Mar 2019 14:19:41 +0000 Message-Id: <20190321141953.31960-8-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP walk_page_range() is going to be allowed to walk page tables other than those of user space. For this it needs to know when it has reached a 'leaf' entry in the page tables. This information is provided by the p?d_large() functions/macros. For sparc 64 bit, pmd_large() and pud_large() are already provided, so add #defines to prevent the generic versions (added in a later patch) from being used. CC: "David S. Miller" CC: sparclinux@vger.kernel.org Signed-off-by: Steven Price Acked-by: David S. Miller --- arch/sparc/include/asm/pgtable_64.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/sparc/include/asm/pgtable_64.h b/arch/sparc/include/asm/pgtable_64.h index 1393a8ac596b..f502e937c8fe 100644 --- a/arch/sparc/include/asm/pgtable_64.h +++ b/arch/sparc/include/asm/pgtable_64.h @@ -713,6 +713,7 @@ static inline unsigned long pte_special(pte_t pte) return pte_val(pte) & _PAGE_SPECIAL; } +#define pmd_large pmd_large static inline unsigned long pmd_large(pmd_t pmd) { pte_t pte = __pte(pmd_val(pmd)); @@ -894,6 +895,7 @@ static inline unsigned long pud_page_vaddr(pud_t pud) #define pgd_present(pgd) (pgd_val(pgd) != 0U) #define pgd_clear(pgdp) (pgd_val(*(pgdp)) = 0UL) +#define pud_large pud_large static inline unsigned long pud_large(pud_t pud) { pte_t pte = __pte(pud_val(pud)); From patchwork Thu Mar 21 14:19:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863621 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0CBC314DE for ; Thu, 21 Mar 2019 14:20:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E5FCF2A179 for ; Thu, 21 Mar 2019 14:20:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DA2EC2A2A3; Thu, 21 Mar 2019 14:20:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 619822A29B for ; Thu, 21 Mar 2019 14:20:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E89556B026A; Thu, 21 Mar 2019 10:20:39 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E37A66B026B; Thu, 21 Mar 2019 10:20:39 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D26EC6B026C; Thu, 21 Mar 2019 10:20:39 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id 874246B026A for ; Thu, 21 Mar 2019 10:20:39 -0400 (EDT) Received: by mail-ed1-f71.google.com with SMTP id 41so2302325edr.19 for ; Thu, 21 Mar 2019 07:20:39 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=3QT6OWyC9JAMeDMFDYFJBPhytDtBHqA6u+Xz2reowKo=; b=dWTEO26Ws/1gW0+ftOYPAMALovBymNgFNzQrenMCtY9fx5rbJD9rDhKnoiNviHUcQ+ CQZNDCcTrrXIQfOknyIo+Y5yd3AYx1v8j5dSjvAZCwf3pJATa3/PxXM2kDP8G8cICN7P 7x6gzSVr5ybN64mVW5fp1Xfl58oJBfxO+nhkT7npcVjxqSnZln13dTHVZ/glaKms9HLz a2tMA5Gix3BypNZlBjpyVuSgKZEo+K31RuI4+GMdyQ3FAgBCp4/M2r9jHkRPQVKeVBno GT023aS/GkEi3TRD4p7CgNT4RvpfNRcL0wLMw06ub+kAv02Ptp6hQNH2lqps85cSrGV2 7seg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAVpIOTuraqxRSVBrJv9AfcIND/BQZheL2mI2PMb8Fi6KvpC5HdP WPGD+5bVP+6qS48p5MEjN5VmAl9TAbfVmozg4UBeXGLUK0wsJFjCrrfQMLtdZ5URBZVFCYHNKyB 2Z0qh0S19OLxm1lwNL10A5nzcqh+iJxelyS9Ko4Z1IEewRnandaOTR5nPL6tE6pTouQ== X-Received: by 2002:a17:906:c827:: with SMTP id dd7mr2490495ejb.100.1553178039045; Thu, 21 Mar 2019 07:20:39 -0700 (PDT) X-Google-Smtp-Source: APXvYqwu1pNsoVVmejTIumheUL+5w0oiN/nFTq3Qq5yUtXsUfFWJWwP3c/dgck3/AFtYMPCKe0LE X-Received: by 2002:a17:906:c827:: with SMTP id dd7mr2490438ejb.100.1553178037960; Thu, 21 Mar 2019 07:20:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178037; cv=none; d=google.com; s=arc-20160816; b=BpJGfHaxFwgxhdvSgy8jtwYM4/EOCT9SNzzi1RftEbdsW/RnQrzPt3n+PUHQGBmUnb LjUwT1SiEBiiqzoFIpyGWrikghmwFtv4clcvhsUQaZ8+nNljR8OoQpyXY3TD2BBY2xo2 TGcvzppcaDohOXeI8VA2Xi81msWzkWuNdbQpUTk+SGX61i7VjhZFdPyzRVDYbNVGlBWa 6G3HTYeQ6FhPHq+O9gouKCPHgdEJPa8hsCjTJJQgDtd919bUNzwCnokJK5E6Az7vykNy x4uNIHSNMeyodUgfZcQQkYd2fNYhAFZXq4cazfrilDm5J/8DVr9X9odSn8wpQLd245X2 +q7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=3QT6OWyC9JAMeDMFDYFJBPhytDtBHqA6u+Xz2reowKo=; b=vrzJmQf3uQhgpTB9dFunOLqId2JRWdgW8h48V//va1x3SWywuN6thFXK40gIIvrf/a Z6MAo3ie4DSfncvGB2C/FbO2a0/7WVsFDn2aKtZHj9hoSnn0OBDGTIduXS8YQBFacTGE jFgHUM3TwuVO9bAPspp9IFKvKMz4vss0rrsKIpN7PB1bsYnUq6OfN0l3swkjJo/eRzcs E8BGsg9wVL6tSZ/Y1jMtxW9dYG+08J7Fvm8+UYVyT2rutmz95nF8Vo7kYCVeazEH1JDj ONNza8rSI7sPWSq6ZBgOQJYMOdYk7PLeVEYxf1SiYOJpKl1dEOQrD6UHgI+fm2eWQcuf 1ceA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id o46si1609372edc.36.2019.03.21.07.20.37 for ; Thu, 21 Mar 2019 07:20:37 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BF884EBD; Thu, 21 Mar 2019 07:20:36 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 83A1A3F575; Thu, 21 Mar 2019 07:20:33 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v5 08/19] x86: mm: Add p?d_large() definitions Date: Thu, 21 Mar 2019 14:19:42 +0000 Message-Id: <20190321141953.31960-9-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP walk_page_range() is going to be allowed to walk page tables other than those of user space. For this it needs to know when it has reached a 'leaf' entry in the page tables. This information is provided by the p?d_large() functions/macros. For x86 we already have static inline functions, so simply add #defines to prevent the generic versions (added in a later patch) from being picked up. We also need to add corresponding #undefs in dump_pagetables.c. This code will be removed when x86 is switched over to using the generic pagewalk code in a later patch. Signed-off-by: Steven Price --- arch/x86/include/asm/pgtable.h | 5 +++++ arch/x86/mm/dump_pagetables.c | 3 +++ 2 files changed, 8 insertions(+) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 2779ace16d23..0dd04cf6ebeb 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -222,6 +222,7 @@ static inline unsigned long pgd_pfn(pgd_t pgd) return (pgd_val(pgd) & PTE_PFN_MASK) >> PAGE_SHIFT; } +#define p4d_large p4d_large static inline int p4d_large(p4d_t p4d) { /* No 512 GiB pages yet */ @@ -230,6 +231,7 @@ static inline int p4d_large(p4d_t p4d) #define pte_page(pte) pfn_to_page(pte_pfn(pte)) +#define pmd_large pmd_large static inline int pmd_large(pmd_t pte) { return pmd_flags(pte) & _PAGE_PSE; @@ -857,6 +859,7 @@ static inline pmd_t *pmd_offset(pud_t *pud, unsigned long address) return (pmd_t *)pud_page_vaddr(*pud) + pmd_index(address); } +#define pud_large pud_large static inline int pud_large(pud_t pud) { return (pud_val(pud) & (_PAGE_PSE | _PAGE_PRESENT)) == @@ -868,6 +871,7 @@ static inline int pud_bad(pud_t pud) return (pud_flags(pud) & ~(_KERNPG_TABLE | _PAGE_USER)) != 0; } #else +#define pud_large pud_large static inline int pud_large(pud_t pud) { return 0; @@ -1213,6 +1217,7 @@ static inline bool pgdp_maps_userspace(void *__ptr) return (((ptr & ~PAGE_MASK) / sizeof(pgd_t)) < PGD_KERNEL_START); } +#define pgd_large pgd_large static inline int pgd_large(pgd_t pgd) { return 0; } #ifdef CONFIG_PAGE_TABLE_ISOLATION diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index ee8f8ab46941..ca270fb00805 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -432,6 +432,7 @@ static void walk_pmd_level(struct seq_file *m, struct pg_state *st, pud_t addr, #else #define walk_pmd_level(m,s,a,e,p) walk_pte_level(m,s,__pmd(pud_val(a)),e,p) +#undef pud_large #define pud_large(a) pmd_large(__pmd(pud_val(a))) #define pud_none(a) pmd_none(__pmd(pud_val(a))) #endif @@ -467,6 +468,7 @@ static void walk_pud_level(struct seq_file *m, struct pg_state *st, p4d_t addr, #else #define walk_pud_level(m,s,a,e,p) walk_pmd_level(m,s,__pud(p4d_val(a)),e,p) +#undef p4d_large #define p4d_large(a) pud_large(__pud(p4d_val(a))) #define p4d_none(a) pud_none(__pud(p4d_val(a))) #endif @@ -501,6 +503,7 @@ static void walk_p4d_level(struct seq_file *m, struct pg_state *st, pgd_t addr, } } +#undef pgd_large #define pgd_large(a) (pgtable_l5_enabled() ? pgd_large(a) : p4d_large(__p4d(pgd_val(a)))) #define pgd_none(a) (pgtable_l5_enabled() ? pgd_none(a) : p4d_none(__p4d(pgd_val(a)))) From patchwork Thu Mar 21 14:19:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863623 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3B3BD1390 for ; Thu, 21 Mar 2019 14:20:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1F6C729199 for ; Thu, 21 Mar 2019 14:20:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 13A862A2A5; Thu, 21 Mar 2019 14:20:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9C77629199 for ; Thu, 21 Mar 2019 14:20:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E8E666B026B; Thu, 21 Mar 2019 10:20:42 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DE8936B026C; Thu, 21 Mar 2019 10:20:42 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD86C6B026D; Thu, 21 Mar 2019 10:20:42 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by kanga.kvack.org (Postfix) with ESMTP id 7AD446B026B for ; Thu, 21 Mar 2019 10:20:42 -0400 (EDT) Received: by mail-ed1-f70.google.com with SMTP id t4so2300942eds.1 for ; Thu, 21 Mar 2019 07:20:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=U1aR140X7RnjOrD7ClczvI1VvyAi5tuJM37h24SqYlk=; b=FY0+XuOJmiLwjxPn8lX3I8K+0uflS/d+IvJXkxvnjMKtTOz8LS+S+cTM9/i87ryrWp 70p0uXsQENULWzPc4cAH0KrTnRsByeNXeN8Sg8JeKVPVY2w7rDXzV7VasW2jX86Y8d/1 kqb7gehgIyJ0/kK6m9dpP/TZAWTRUHn94NlYL/ulHSPKXI28PRKPr4kg9RZvNFGURd34 iRnc+Jmd1XaCDGtpmQ8V0qtgwC4hdWQCvQXziv8SjagBkot5n0JQDXzZrJgfeyCZwtIB +zckrREuLrg2+BldzAiwUohhAb2+NaLaLqrffthc6vnCPXAHuhQCs8gjLAQD9ExzGAVb O8YQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAXY1e3tNtfDdV//zvACNW3z37cPHqjcQN1ky6jnuawCALCIGPBi YVv+i+qP6eui/zO2Kx+ylvtsBrsSsRnCcvGAtsmgBNMtcgE7rpD3dbrJWcCC1Sq7j8w3lObj483 BKuNkgA+f6agm00d9sg1ow0+IyCwVzq42KMiPMydHOdbyrljQtWbkwMMd4DNh8VW31w== X-Received: by 2002:a50:976d:: with SMTP id d42mr2650833edb.5.1553178042047; Thu, 21 Mar 2019 07:20:42 -0700 (PDT) X-Google-Smtp-Source: APXvYqzDpTaKxml7LL0sNo3mje4/WTwT2TAKlfZo7vhANcsYZPZRbc3xFlwyackY5Ts6hK6OKngq X-Received: by 2002:a50:976d:: with SMTP id d42mr2650785edb.5.1553178041110; Thu, 21 Mar 2019 07:20:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178041; cv=none; d=google.com; s=arc-20160816; b=cuApSO61b/X3GR07tQZ8cQiv4+f3qZC88Qilzy5Hbu0c1LAWsGvhr8MnBYyAIbbdzN HAIWqvXvf1nV4ApcW9aFjPdT0mZBuXQ7g0XTC3yIQCoos/n/t3N8xyKFVsIQ6vlzmDlk 90xPVLrUD2x5q/+8MTDtHDmEmnupjymO4iTyuSACtHUcUkSqCWxHF+0uK3UIfOXIlbWb Xv6kE39dKNEC7ZyqQMI5IOuStkJJlZTPOkYvclQ8rkhrL3TrNWHi4TfOiuyzwtqI0PqQ NJZfI787d/O4jk6jIvxkdr8uvYaYZCqW+hUPiNnJmTFxTgkVLk81dtSIEMhtbDddZFRz vZjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=U1aR140X7RnjOrD7ClczvI1VvyAi5tuJM37h24SqYlk=; b=N/Njm9xrIhlG21t1+SUm4nqCnIKZh8kLRylNj5rldjkx1Le9NONPb+nnS75OMoeQ0W Qq90FrMiEvlVfpOEbV9Xoa2HCnvxfq1lJ/M5F2hsJh4591npMhh+goiNlzTNap+nsR5o Ft2DW0Ty0S1WMiPY5vAuTgi9NUt0W4slNK0LjNJcnG5h4xG3bV21iCnA78+qB/I8wZhl uldlvi8WObv0JaefLTRGIZdYZw2voNPYufCE8pjOD+XaLlTZY4H6bydQ6SuVoEHmLo+n VA4Uh1mYKD7ZX5TfJDdJ1ELTArytj6CHITN6CsNRAlTJLyBfRv4EtC5go5PkfA0OnvEn IG0g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id q1si952057ejs.275.2019.03.21.07.20.40 for ; Thu, 21 Mar 2019 07:20:41 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 45337168F; Thu, 21 Mar 2019 07:20:40 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0A8C33F575; Thu, 21 Mar 2019 07:20:36 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v5 09/19] mm: Add generic p?d_large() macros Date: Thu, 21 Mar 2019 14:19:43 +0000 Message-Id: <20190321141953.31960-10-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Exposing the pud/pgd levels of the page tables to walk_page_range() means we may come across the exotic large mappings that come with large areas of contiguous memory (such as the kernel's linear map). For architectures that don't provide p?d_large() macros, provide generic does nothing defaults. Signed-off-by: Steven Price --- include/asm-generic/pgtable.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h index fa782fba51ee..9c5d0f73db67 100644 --- a/include/asm-generic/pgtable.h +++ b/include/asm-generic/pgtable.h @@ -1186,4 +1186,23 @@ static inline bool arch_has_pfn_modify_check(void) #define mm_pmd_folded(mm) __is_defined(__PAGETABLE_PMD_FOLDED) #endif +/* + * p?d_large() - true if this entry is a final mapping to a physical address. + * This differs from p?d_huge() by the fact that they are always available (if + * the architecture supports large pages at the appropriate level) even + * if CONFIG_HUGETLB_PAGE is not defined. + */ +#ifndef pgd_large +#define pgd_large(x) 0 +#endif +#ifndef p4d_large +#define p4d_large(x) 0 +#endif +#ifndef pud_large +#define pud_large(x) 0 +#endif +#ifndef pmd_large +#define pmd_large(x) 0 +#endif + #endif /* _ASM_GENERIC_PGTABLE_H */ From patchwork Thu Mar 21 14:19:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863625 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E0CB71390 for ; Thu, 21 Mar 2019 14:20:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C552F2A2A4 for ; Thu, 21 Mar 2019 14:20:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B96822A2A5; Thu, 21 Mar 2019 14:20:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 309B6297C6 for ; Thu, 21 Mar 2019 14:20:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D299F6B026C; Thu, 21 Mar 2019 10:20:46 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CD9FD6B026D; Thu, 21 Mar 2019 10:20:46 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B02C06B026E; Thu, 21 Mar 2019 10:20:46 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id 5B4436B026C for ; Thu, 21 Mar 2019 10:20:46 -0400 (EDT) Received: by mail-ed1-f69.google.com with SMTP id p4so2295612edd.0 for ; Thu, 21 Mar 2019 07:20:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=pZnxgzhKJR9uuFgij05eVtjeQSd9JABLgi39/Q74xo8=; b=szXF4FYey1HDRrwFnFwJELoQhLvvPqSgo/GNd/9Rr+bsXQaUEOct2QX6xngTBH5fUZ QKyKVCfVUr/qLd57DUqeAexk/zR8qUc3cd69oWBNR1f6dB7JkMmtNeCeCczRCJvKxa+n V4l5cRBEcEjmpImY7oZxKGa4thWZNteYjC4f5ctPs7Kd9Aq/kGE9V41LY/Ze/uPs3AqL KLAvEuM+uZq1W3a8Q/8dSf6xw5kxoS7rdwZUtiaqsbmcjo5n4Wr1LadWQxO5qUhp0BnM IKTFdRwXoJ/LmFbT2pMzw/5qjOrkIsvHPfewT/o9gzjEXexHKeBVzmef8lnOcSDs8M4E 0TUg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAXVXY9741yKwHK5zjaw+RRJzCmcq8Xazrur85au34XJdKTkdROi wHp49+cKmM8rPJxjaeq1eABCKF/6foWvImjA0y4oKAEZJV1oZ2czVPCazvFjW7S/hhTNwrBfXqO 8TWf5GiKzRp55s6KC1gEgtoeopR+guCBbD6R4mJnYM/oqHQNv4EGP19YsPpuErCsmwA== X-Received: by 2002:a17:906:27c9:: with SMTP id k9mr2494722ejc.176.1553178045889; Thu, 21 Mar 2019 07:20:45 -0700 (PDT) X-Google-Smtp-Source: APXvYqyLvds/fhywfIdj72By5gTqWzv9KWN4d1tAO4xxAyteEon/hVMtzQ7Pc8u5gp8xtSEwXhdy X-Received: by 2002:a17:906:27c9:: with SMTP id k9mr2494676ejc.176.1553178044956; Thu, 21 Mar 2019 07:20:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178044; cv=none; d=google.com; s=arc-20160816; b=VWZSJbCGccdL+ZdlZjspNeXtZBxvdhOv/zBi9+UIeQkw+CGhWJSjA+1fPGGFdyndgL +Bp2t1cFsRugW6tFeXHv8rKaPTqL0XEy9PLeqf5PsJXNA6Br2cRLRj7atGQ54flbV/sx LL6EUcrCq1MswXEIMVrKTlp7LDSly5i1YmLVCw1YdHnZ9cOwfU3NJ2hHJjavkD+4yWsE 4QIv1g5ymtHjMuGIVlrfAraAxlbhsxrpIsVwsrDHbkmPfjBuHkwfsCZ25xTPEvhSPfPH 7/ehgETELk/lRthD47n2dR8wfQCtha3X+2cLsdJrv8atsz/0n6zBay3kKv+Q6DN5FlAP nrzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=pZnxgzhKJR9uuFgij05eVtjeQSd9JABLgi39/Q74xo8=; b=QHaTakOQavjPp31+Q3lJtB9B5wklImde7jy+CiPBlrUITwb1bMcYMttmkXITVnMjXK LlbW+TU+WcYF8uvJ1eaf9hHnHNdCpyLL2EmpXbW3qWmwCDh670q5/B+CmcA4/BJFjpVU 3kCeB1f0whaw6y2dI/ZYsm+8lHRm9Vh2xrB91ZvFPc5Dx1pjGwiDUeZ1dO6RHvMPGB2T uiCAUAmt6xgfq8MuYR6RMSbCK4/oo6nrjUFB/xKzf2r8gb6+Xc5hwOu7BEIAJ2218pvN iVswrIHzAW1PeKW9Lx7vC+ZUM70z2sq11/RH4oIYifSgKDkjJcNX6rgF8fzlCvwGn26e 1n0w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id k11si1717979eja.151.2019.03.21.07.20.44 for ; Thu, 21 Mar 2019 07:20:44 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C0638165C; Thu, 21 Mar 2019 07:20:43 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8488E3F575; Thu, 21 Mar 2019 07:20:40 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v5 10/19] mm: pagewalk: Add p4d_entry() and pgd_entry() Date: Thu, 21 Mar 2019 14:19:44 +0000 Message-Id: <20190321141953.31960-11-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP pgd_entry() and pud_entry() were removed by commit 0b1fbfe50006c410 ("mm/pagewalk: remove pgd_entry() and pud_entry()") because there were no users. We're about to add users so reintroduce them, along with p4d_entry() as we now have 5 levels of tables. Note that commit a00cc7d9dd93d66a ("mm, x86: add support for PUD-sized transparent hugepages") already re-added pud_entry() but with different semantics to the other callbacks. Since there have never been upstream users of this, revert the semantics back to match the other callbacks. This means pud_entry() is called for all entries, not just transparent huge pages. Signed-off-by: Steven Price --- include/linux/mm.h | 9 ++++++--- mm/pagewalk.c | 27 ++++++++++++++++----------- 2 files changed, 22 insertions(+), 14 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 76769749b5a5..2983f2396a72 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1367,10 +1367,9 @@ void unmap_vmas(struct mmu_gather *tlb, struct vm_area_struct *start_vma, /** * mm_walk - callbacks for walk_page_range + * @pgd_entry: if set, called for each non-empty PGD (top-level) entry + * @p4d_entry: if set, called for each non-empty P4D (1st-level) entry * @pud_entry: if set, called for each non-empty PUD (2nd-level) entry - * this handler should only handle pud_trans_huge() puds. - * the pmd_entry or pte_entry callbacks will be used for - * regular PUDs. * @pmd_entry: if set, called for each non-empty PMD (3rd-level) entry * this handler is required to be able to handle * pmd_trans_huge() pmds. They may simply choose to @@ -1390,6 +1389,10 @@ void unmap_vmas(struct mmu_gather *tlb, struct vm_area_struct *start_vma, * (see the comment on walk_page_range() for more details) */ struct mm_walk { + int (*pgd_entry)(pgd_t *pgd, unsigned long addr, + unsigned long next, struct mm_walk *walk); + int (*p4d_entry)(p4d_t *p4d, unsigned long addr, + unsigned long next, struct mm_walk *walk); int (*pud_entry)(pud_t *pud, unsigned long addr, unsigned long next, struct mm_walk *walk); int (*pmd_entry)(pmd_t *pmd, unsigned long addr, diff --git a/mm/pagewalk.c b/mm/pagewalk.c index c3084ff2569d..98373a9f88b8 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -90,15 +90,9 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, } if (walk->pud_entry) { - spinlock_t *ptl = pud_trans_huge_lock(pud, walk->vma); - - if (ptl) { - err = walk->pud_entry(pud, addr, next, walk); - spin_unlock(ptl); - if (err) - break; - continue; - } + err = walk->pud_entry(pud, addr, next, walk); + if (err) + break; } split_huge_pud(walk->vma, pud, addr); @@ -131,7 +125,12 @@ static int walk_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, break; continue; } - if (walk->pmd_entry || walk->pte_entry) + if (walk->p4d_entry) { + err = walk->p4d_entry(p4d, addr, next, walk); + if (err) + break; + } + if (walk->pud_entry || walk->pmd_entry || walk->pte_entry) err = walk_pud_range(p4d, addr, next, walk); if (err) break; @@ -157,7 +156,13 @@ static int walk_pgd_range(unsigned long addr, unsigned long end, break; continue; } - if (walk->pmd_entry || walk->pte_entry) + if (walk->pgd_entry) { + err = walk->pgd_entry(pgd, addr, next, walk); + if (err) + break; + } + if (walk->p4d_entry || walk->pud_entry || walk->pmd_entry || + walk->pte_entry) err = walk_p4d_range(pgd, addr, next, walk); if (err) break; From patchwork Thu Mar 21 14:19:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863627 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2D2431390 for ; Thu, 21 Mar 2019 14:21:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 11F8D2A2A6 for ; Thu, 21 Mar 2019 14:21:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 067E62A2AE; Thu, 21 Mar 2019 14:21:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7960D2A2A5 for ; Thu, 21 Mar 2019 14:21:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DC65C6B026E; Thu, 21 Mar 2019 10:20:49 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D74FE6B026F; Thu, 21 Mar 2019 10:20:49 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C62CD6B0270; Thu, 21 Mar 2019 10:20:49 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id 743E26B026E for ; Thu, 21 Mar 2019 10:20:49 -0400 (EDT) Received: by mail-ed1-f72.google.com with SMTP id t13so2305839edw.13 for ; Thu, 21 Mar 2019 07:20:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=NXbLxkVEjkUC98nDqbwMu9XwBnfr8Fy+W9xUz87hsi8=; b=nC4ayWdsfWt4QYRwGB90LCN64ylmeuFsPHTgnpHN0o+WAkp6hxuY8A9xj00TQr0lfm 4RL2ARhX42qbMAuMp2GrdyEK2oNw8DvFtvXj94OZv6t8g9pM1nERWsg+ghdjtFRbzgYH 4Q+kNnBIu+DF1Yx/0KPobjp1tsTB8BTXIM2Tof23TE19x79Jkq6VfszJpGT0JA06KQUp svEm3C6O7BISk+Z9Aa6U3YVMMX6ppi7skQAh25R8yCYudWsxNB0BGM0oNIxfTFwGxZd/ 8MR3FoLAitP4LkWID0Rh1xBcBJSjQh+mSqfixZgY0RSBYpmTb0oFpljjkJL1Uy6Ugnlo hbnQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAVp3QZ22/TDKENPDu3mxPBMixYo5PqOEGbBuYY3IDAR2BabK0nE mAbnicss/6Zr8BLNFXhGWpd3Eg173B32fFx/SiKh6kHoMPlupnWY6iKIP0Ti+iZHNo5obRMMuZJ uu8u3N2uN+ZropQ1oNoj7F3QdmXhzHNf3OuSCplY8vjSYLLEqY+x6GHb2sMtsRV41Lg== X-Received: by 2002:a50:eb0c:: with SMTP id y12mr2533240edp.237.1553178049011; Thu, 21 Mar 2019 07:20:49 -0700 (PDT) X-Google-Smtp-Source: APXvYqxySE6hr8eYm10y6ahBOgpEXpDaj8n3iKGmeR51mfeUxvO/oD4dBSbKLJ50qjaZ8O61dWRs X-Received: by 2002:a50:eb0c:: with SMTP id y12mr2533194edp.237.1553178048129; Thu, 21 Mar 2019 07:20:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178048; cv=none; d=google.com; s=arc-20160816; b=Ay5aqmhG2BtQFNEcLwTQnWgoYtG1w7nyQD0IdcftJS3aURhWQbp82r4ewLkgOdlKqM 9rn1oHjqMI7/KHtc7A3VjNu82EbAEl4CHdqsr3sbJ5GxnQFWGTafvOuxRAMUxdzqDUlC sQ77KDJ5/ybKOIi83qNSYO81vm5xFBGUa1piSuXjjZdDye/RdLsEewPCaYb/isc03N3w K0VEZ0xReB4IwPl8NYkBAWokDYFKXF7VbSB5drQ8KRB8NJ2S/VTGcawtN9jPOM1bCzqA uFizYl9k0SHI5Rtr206eh5Yn61H59WBkByw4LhxwMN6c6ycRk7s61zR7wgvfrOhW4vYc GHmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=NXbLxkVEjkUC98nDqbwMu9XwBnfr8Fy+W9xUz87hsi8=; b=hVkrZ/mNvNsKEJHbqC+gUmzh8+oxScB+5WJtU0AKhBSMgTp5QBcS8ODCMPTu6u9Lzz 83RvXVgNbu5ZHGYhxUumIJ/ySnhGu8+w/AsHHfURdiDUpwRsHZBx0rITV3DaW042uR7a YJpHcANcKbR2ckOuhHgwrm4FeFsWna/hF7XfEczBrejJ8XN9TvcDszH4EzHCFm3p3vnQ zVxQ7nsphp9aH5mnlcU0duvbVYyaiGTwuVfqG+NnzaRTgJI2MLRuTmpiVC2XE4tRfZI2 mYeRIySaQBo11AevCG5zKAthJmrPaLbkcQr3tGz5z2FFWHojnf/6Zd/uW8VKqyLjwZfs +UqQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id 90si1175545edq.392.2019.03.21.07.20.47 for ; Thu, 21 Mar 2019 07:20:48 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 45F781A25; Thu, 21 Mar 2019 07:20:47 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0B5563F575; Thu, 21 Mar 2019 07:20:43 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v5 11/19] mm: pagewalk: Allow walking without vma Date: Thu, 21 Mar 2019 14:19:45 +0000 Message-Id: <20190321141953.31960-12-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Since 48684a65b4e3: "mm: pagewalk: fix misbehavior of walk_page_range for vma(VM_PFNMAP)", page_table_walk() will report any kernel area as a hole, because it lacks a vma. This means each arch has re-implemented page table walking when needed, for example in the per-arch ptdump walker. Remove the requirement to have a vma except when trying to split huge pages. Signed-off-by: Steven Price --- mm/pagewalk.c | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/mm/pagewalk.c b/mm/pagewalk.c index 98373a9f88b8..dac0c848b458 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -36,7 +36,7 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, do { again: next = pmd_addr_end(addr, end); - if (pmd_none(*pmd) || !walk->vma) { + if (pmd_none(*pmd)) { if (walk->pte_hole) err = walk->pte_hole(addr, next, walk); if (err) @@ -59,9 +59,14 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, if (!walk->pte_entry) continue; - split_huge_pmd(walk->vma, pmd, addr); - if (pmd_trans_unstable(pmd)) - goto again; + if (walk->vma) { + split_huge_pmd(walk->vma, pmd, addr); + if (pmd_trans_unstable(pmd)) + goto again; + } else if (pmd_large(*pmd)) { + continue; + } + err = walk_pte_range(pmd, addr, next, walk); if (err) break; @@ -81,7 +86,7 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, do { again: next = pud_addr_end(addr, end); - if (pud_none(*pud) || !walk->vma) { + if (pud_none(*pud)) { if (walk->pte_hole) err = walk->pte_hole(addr, next, walk); if (err) @@ -95,9 +100,13 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, break; } - split_huge_pud(walk->vma, pud, addr); - if (pud_none(*pud)) - goto again; + if (walk->vma) { + split_huge_pud(walk->vma, pud, addr); + if (pud_none(*pud)) + goto again; + } else if (pud_large(*pud)) { + continue; + } if (walk->pmd_entry || walk->pte_entry) err = walk_pmd_range(pud, addr, next, walk); From patchwork Thu Mar 21 14:19:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863633 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 616E41515 for ; Thu, 21 Mar 2019 14:21:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 46CD72A29B for ; Thu, 21 Mar 2019 14:21:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3AE5A2A2A6; Thu, 21 Mar 2019 14:21:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AD65D2A2A6 for ; Thu, 21 Mar 2019 14:21:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DC4656B026F; Thu, 21 Mar 2019 10:20:53 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D9C3F6B0270; Thu, 21 Mar 2019 10:20:53 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C64766B0271; Thu, 21 Mar 2019 10:20:53 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id 7226B6B026F for ; Thu, 21 Mar 2019 10:20:53 -0400 (EDT) Received: by mail-ed1-f71.google.com with SMTP id o27so2279478edc.14 for ; Thu, 21 Mar 2019 07:20:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=iuMdLE8+u1uoSAbvOzwF5G8gMH8Pp5jo2G4hJiPSSFc=; b=UFwiCOndqDPPd2lFamNyB7ERUtGe93gubWyAWZgHLxZfepp621ShO8boJd0cpkHsoY vqmF+3o/5Fhw6jGylP3wcxc2nWG6yMKWl0TbChgvUwcbfdtsAIb+lUKfh4kUzChjug4E dhOMPI/AzEaAfETOUsVDC4BPwpYZgb6kvfUVjRL3VT+BtLtSgUz9Ma5Tew3T8hmsaw5G U+0c71wz5n12dOgbXMXgGHd/E2U4LWTrWWZqkBj7eDMzdaEBRN3r9x5K98OYb5CXELhz q9oqJlmQ0FRjOqKgQeSKRMg1jZ1/dWFKqvXu9ezQAU5kLUqXWUE4MnI7MQGupgEWEPki 500g== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAV70qaCNJpviEistMNBKssjJZcFTX2HtlThwkcteCRWxsvAFiQ7 WZhaWetzcQ9sbPGZvWSlEErquaLdkiWtkTnmuIuBGi8LgxwV2CsFAQTwT+B+xANK5QyAQ79bkbL LHLBQbzJEFblxnsYYx3k10HMlFNWzgWFLmzRrMVHQPnm4BlUUl0t9b+2OSjdl26gFqQ== X-Received: by 2002:a17:906:1d0f:: with SMTP id n15mr2519233ejh.102.1553178052986; Thu, 21 Mar 2019 07:20:52 -0700 (PDT) X-Google-Smtp-Source: APXvYqz6lZdAZvmcrlQm8c2K5Ooh1tG3DK0lTs3Tc12SCSoCN4C8QEefrRESC5Sz3hlwh3kmcRnH X-Received: by 2002:a17:906:1d0f:: with SMTP id n15mr2519173ejh.102.1553178051843; Thu, 21 Mar 2019 07:20:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178051; cv=none; d=google.com; s=arc-20160816; b=wpNUrkbFQpvWUIpBWx6SsnmypmcE5z/YRxaosWLdMC0AI1DlruMixEurFMES++KlTt gkPzxLJexZDMluvj7vomunt39wCHv9YmuTzBS0VAIP62/0pi6MRsnYyV05PFXN9Osen6 zh03F9MmQNhw0xat3t+3VHU7Xlyy8RdC1JaM5fV0dga2XN5PybOzdQckwuU8/8FX0aRd B746LUWYe6G5RWIPKYUXy8G8EdPGDkFp78JMbaiB1k1kjMkiPPTIBVYGRdqsvCmdZBQj OAQjOVrSWYSRNoFr1a79bE/JLqtvzXB3CLhWWbgVn1p10qhWNxMQ1SJ5fEbqkrhMp8Lm cgHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=iuMdLE8+u1uoSAbvOzwF5G8gMH8Pp5jo2G4hJiPSSFc=; b=Rr1KqYaGDO647Q+fEuGzKDIxgdRs7/aV5MpMk71fmvNtIgyT2x46O/BvOOwaVGdK+v iWXVBeCEQzshoAX+K7h2k+ceXM5YJjmfDn708fR815MdGOrHDF4wyDZUDTQL1UODXkaa 6+X0dPV7Y2jT+HC7Nw2SU2AKa2cK4sQzlkXAaBU0Etdy+pQ66e+FQ8mRyWzxiuErBEKp 4TifvcrR+uuPZ8IVEQEczs4TnhmmyvBkJhnFNgrRtXc6NklEUbLQXjYWJq7g6fv5BbPD KEIZrDujNB+osPmjoGGNyZSuu+IxDBdtAObGMxeWjgWX/IyPRUawTikK8dWN8y00ft27 fwvg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id e14si2086882edb.137.2019.03.21.07.20.51 for ; Thu, 21 Mar 2019 07:20:51 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C04AB80D; Thu, 21 Mar 2019 07:20:50 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8596F3F575; Thu, 21 Mar 2019 07:20:47 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v5 12/19] mm: pagewalk: Add test_p?d callbacks Date: Thu, 21 Mar 2019 14:19:46 +0000 Message-Id: <20190321141953.31960-13-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP It is useful to be able to skip parts of the page table tree even when walking without VMAs. Add test_p?d callbacks similar to test_walk but which are called just before a table at that level is walked. If the callback returns non-zero then the entire table is skipped. Signed-off-by: Steven Price --- include/linux/mm.h | 11 +++++++++++ mm/pagewalk.c | 24 ++++++++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 2983f2396a72..49a04cc9ee84 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1382,6 +1382,11 @@ void unmap_vmas(struct mmu_gather *tlb, struct vm_area_struct *start_vma, * value means "do page table walk over the current vma," * and a negative one means "abort current page table walk * right now." 1 means "skip the current vma." + * @test_pmd: similar to test_walk(), but called for every pmd. + * @test_pud: similar to test_walk(), but called for every pud. + * @test_p4d: similar to test_walk(), but called for every p4d. + * Returning 0 means walk this part of the page tables, + * returning 1 means to skip this range. * @mm: mm_struct representing the target process of page table walk * @vma: vma currently walked (NULL if walking outside vmas) * @private: private data for callbacks' usage @@ -1406,6 +1411,12 @@ struct mm_walk { struct mm_walk *walk); int (*test_walk)(unsigned long addr, unsigned long next, struct mm_walk *walk); + int (*test_pmd)(unsigned long addr, unsigned long next, + pmd_t *pmd_start, struct mm_walk *walk); + int (*test_pud)(unsigned long addr, unsigned long next, + pud_t *pud_start, struct mm_walk *walk); + int (*test_p4d)(unsigned long addr, unsigned long next, + p4d_t *p4d_start, struct mm_walk *walk); struct mm_struct *mm; struct vm_area_struct *vma; void *private; diff --git a/mm/pagewalk.c b/mm/pagewalk.c index dac0c848b458..231655db1295 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -32,6 +32,14 @@ static int walk_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, unsigned long next; int err = 0; + if (walk->test_pmd) { + err = walk->test_pmd(addr, end, pmd_offset(pud, 0), walk); + if (err < 0) + return err; + if (err > 0) + return 0; + } + pmd = pmd_offset(pud, addr); do { again: @@ -82,6 +90,14 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, unsigned long next; int err = 0; + if (walk->test_pud) { + err = walk->test_pud(addr, end, pud_offset(p4d, 0), walk); + if (err < 0) + return err; + if (err > 0) + return 0; + } + pud = pud_offset(p4d, addr); do { again: @@ -124,6 +140,14 @@ static int walk_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, unsigned long next; int err = 0; + if (walk->test_p4d) { + err = walk->test_p4d(addr, end, p4d_offset(pgd, 0), walk); + if (err < 0) + return err; + if (err > 0) + return 0; + } + p4d = p4d_offset(pgd, addr); do { next = p4d_addr_end(addr, end); From patchwork Thu Mar 21 14:19:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863635 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 230B514DE for ; Thu, 21 Mar 2019 14:21:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 03F942A2AD for ; Thu, 21 Mar 2019 14:21:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EABBA2A2A5; Thu, 21 Mar 2019 14:21:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5A04B2A2AD for ; Thu, 21 Mar 2019 14:21:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B7A126B0270; Thu, 21 Mar 2019 10:20:57 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B2BD86B0271; Thu, 21 Mar 2019 10:20:57 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F19B6B0272; Thu, 21 Mar 2019 10:20:57 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by kanga.kvack.org (Postfix) with ESMTP id 543466B0270 for ; Thu, 21 Mar 2019 10:20:57 -0400 (EDT) Received: by mail-ed1-f69.google.com with SMTP id n12so2298499edo.5 for ; Thu, 21 Mar 2019 07:20:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=Z5ii9WWPo9WVfhue9NtkIQuQCcbK0pxcqGqIbiUCEeU=; b=b/pkVHE/A2DoH+jshInyt1vnK2rgs3ieEcRtIuoB5ZV+V/qOrgx2ECz2c8Xjg9Iw/C 6tpNj400fK8H5gV0uR16DPqY3N8YLZ7Htw3Gn7hDKkpDEEJ5MfdmoWgMVR/nMzbarv/N SinIdIy+jjQkqifVld117M1ZxDCoFAIBRpLN4VC6WF1/r57W7VUV7vpk2ihB1g6RLe/8 Rop5ULkfVrHOOexqoHu3hCaZvQHGWQ6lyMumR/HCc5C+BLopTblvAxvcm4A3Ewv1hwAq vtWqSkJtzHwyFdzgNbSbKxA74lWM0Qh/lpKbbacfEzR2Sa4sWnrTnn8BL4nv2liaTNSC 95gw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAVpmjYD1Gfghi/MxM9i6wmWtOe3FXGU18pYSQRkPek0I0Io9ltq UtzqcwcCGd5783UAfCdOFZnQ+biuJtL4zZdTujSj51jBJjkNoTcvKhSwQjcgbUNSioAkmWWBN+S cC8vlDc96UONhjUPykTTvziliNT/rcMtMqR6+E6KIzvaHAPcgcRemhkivb0pcsHDAXg== X-Received: by 2002:a50:ba8c:: with SMTP id x12mr2744411ede.230.1553178056862; Thu, 21 Mar 2019 07:20:56 -0700 (PDT) X-Google-Smtp-Source: APXvYqykLvvcgP+znzUc7XxjvWbX3sGceMG4bxaEsv2IRqhNM+0a0kHWzEApbZjyFzWDnfyDX0/A X-Received: by 2002:a50:ba8c:: with SMTP id x12mr2744336ede.230.1553178055374; Thu, 21 Mar 2019 07:20:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178055; cv=none; d=google.com; s=arc-20160816; b=t+b/7udIWvq19dwa2UdYQFPAUy1GFxIPSaMJk4oITdiWKgP3wa6PtJAXUzD9fovdND tUJVzGuPweNybDVls4hqjVgfhjcaq5DaA1lU5Q11PNCcpkFgn7VbMhmp8K7hw9rX5BdN CIzblCjQPievn9EtajIXQ+/iDhpMzOYMO2dFt/D+1TZkHHPFGJTOp6jg2sjlyu3Wn9aw /plqul3T9wy/dRqhlNhXsrhKSGik1U2fyN6Nz/rbk5FyzudoN+gynyADKgmboUldIg+r Y/qyR77zCIJvKV35v1pnMghbexOslYUqnzYoOT5HWkb3GM5DhM89HxjsVhpKC2UqB1qp VGtQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=Z5ii9WWPo9WVfhue9NtkIQuQCcbK0pxcqGqIbiUCEeU=; b=XI3m1Hk3VF4Dge5hN0AH6aHuGEauTmxzoGm+MIpCWTlF+A+WWp89dD0m3gIcy66Mya 7i6A3iJpkbKWqxbORZqCbpFBdui0vLjvNKPhjNLvXkWmAtaYVbuWzjB99aYyOsBReTUo DGU0EdDfa2cJJvoCaCoJyn87/dZ4ope/d84tAiuVnW9+N+5RPsFKEHVl1f57C1lsBdcw tGz7DLHW5iFIk/q2/Z2uqB6VvbLSAVdZbIuwoHx1fU/13RtQkiYfIK8C/7EQOr3oQtpW yA+gf6196ClJzdE60OfaH5ihzQMWEkRWnqXXYB7IHz50IvOZRtl13I8uk8ybvKlkmM4+ lGQw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id g21si1324884eda.45.2019.03.21.07.20.54 for ; Thu, 21 Mar 2019 07:20:55 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 463FE1AC1; Thu, 21 Mar 2019 07:20:54 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0BBD83F575; Thu, 21 Mar 2019 07:20:50 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v5 13/19] arm64: mm: Convert mm/dump.c to use walk_page_range() Date: Thu, 21 Mar 2019 14:19:47 +0000 Message-Id: <20190321141953.31960-14-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Now walk_page_range() can walk kernel page tables, we can switch the arm64 ptdump code over to using it, simplifying the code. Signed-off-by: Steven Price --- arch/arm64/mm/dump.c | 117 ++++++++++++++++++++++--------------------- 1 file changed, 59 insertions(+), 58 deletions(-) diff --git a/arch/arm64/mm/dump.c b/arch/arm64/mm/dump.c index 14fe23cd5932..ea20c1213498 100644 --- a/arch/arm64/mm/dump.c +++ b/arch/arm64/mm/dump.c @@ -72,7 +72,7 @@ struct pg_state { struct seq_file *seq; const struct addr_marker *marker; unsigned long start_address; - unsigned level; + int level; u64 current_prot; bool check_wx; unsigned long wx_pages; @@ -234,11 +234,14 @@ static void note_prot_wx(struct pg_state *st, unsigned long addr) st->wx_pages += (addr - st->start_address) / PAGE_SIZE; } -static void note_page(struct pg_state *st, unsigned long addr, unsigned level, +static void note_page(struct pg_state *st, unsigned long addr, int level, u64 val) { static const char units[] = "KMGTPE"; - u64 prot = val & pg_level[level].mask; + u64 prot = 0; + + if (level >= 0) + prot = val & pg_level[level].mask; if (!st->level) { st->level = level; @@ -286,73 +289,71 @@ static void note_page(struct pg_state *st, unsigned long addr, unsigned level, } -static void walk_pte(struct pg_state *st, pmd_t *pmdp, unsigned long start, - unsigned long end) +static int pud_entry(pud_t *pud, unsigned long addr, + unsigned long next, struct mm_walk *walk) { - unsigned long addr = start; - pte_t *ptep = pte_offset_kernel(pmdp, start); + struct pg_state *st = walk->private; + pud_t val = READ_ONCE(*pud); + + if (pud_table(val)) + return 0; + + note_page(st, addr, 2, pud_val(val)); - do { - note_page(st, addr, 4, READ_ONCE(pte_val(*ptep))); - } while (ptep++, addr += PAGE_SIZE, addr != end); + return 0; } -static void walk_pmd(struct pg_state *st, pud_t *pudp, unsigned long start, - unsigned long end) +static int pmd_entry(pmd_t *pmd, unsigned long addr, + unsigned long next, struct mm_walk *walk) { - unsigned long next, addr = start; - pmd_t *pmdp = pmd_offset(pudp, start); - - do { - pmd_t pmd = READ_ONCE(*pmdp); - next = pmd_addr_end(addr, end); - - if (pmd_none(pmd) || pmd_sect(pmd)) { - note_page(st, addr, 3, pmd_val(pmd)); - } else { - BUG_ON(pmd_bad(pmd)); - walk_pte(st, pmdp, addr, next); - } - } while (pmdp++, addr = next, addr != end); + struct pg_state *st = walk->private; + pmd_t val = READ_ONCE(*pmd); + + if (pmd_table(val)) + return 0; + + note_page(st, addr, 3, pmd_val(val)); + + return 0; } -static void walk_pud(struct pg_state *st, pgd_t *pgdp, unsigned long start, - unsigned long end) +static int pte_entry(pte_t *pte, unsigned long addr, + unsigned long next, struct mm_walk *walk) { - unsigned long next, addr = start; - pud_t *pudp = pud_offset(pgdp, start); - - do { - pud_t pud = READ_ONCE(*pudp); - next = pud_addr_end(addr, end); - - if (pud_none(pud) || pud_sect(pud)) { - note_page(st, addr, 2, pud_val(pud)); - } else { - BUG_ON(pud_bad(pud)); - walk_pmd(st, pudp, addr, next); - } - } while (pudp++, addr = next, addr != end); + struct pg_state *st = walk->private; + pte_t val = READ_ONCE(*pte); + + note_page(st, addr, 4, pte_val(val)); + + return 0; +} + +static int pte_hole(unsigned long addr, unsigned long next, + struct mm_walk *walk) +{ + struct pg_state *st = walk->private; + + note_page(st, addr, -1, 0); + + return 0; } static void walk_pgd(struct pg_state *st, struct mm_struct *mm, - unsigned long start) + unsigned long start) { - unsigned long end = (start < TASK_SIZE_64) ? TASK_SIZE_64 : 0; - unsigned long next, addr = start; - pgd_t *pgdp = pgd_offset(mm, start); - - do { - pgd_t pgd = READ_ONCE(*pgdp); - next = pgd_addr_end(addr, end); - - if (pgd_none(pgd)) { - note_page(st, addr, 1, pgd_val(pgd)); - } else { - BUG_ON(pgd_bad(pgd)); - walk_pud(st, pgdp, addr, next); - } - } while (pgdp++, addr = next, addr != end); + struct mm_walk walk = { + .mm = mm, + .private = st, + .pud_entry = pud_entry, + .pmd_entry = pmd_entry, + .pte_entry = pte_entry, + .pte_hole = pte_hole + }; + down_read(&mm->mmap_sem); + walk_page_range(start, start | (((unsigned long)PTRS_PER_PGD << + PGDIR_SHIFT) - 1), + &walk); + up_read(&mm->mmap_sem); } void ptdump_walk_pgd(struct seq_file *m, struct ptdump_info *info) From patchwork Thu Mar 21 14:19:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863639 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0EEED1390 for ; Thu, 21 Mar 2019 14:21:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E5D102A2B1 for ; Thu, 21 Mar 2019 14:21:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DA0B62A2AE; Thu, 21 Mar 2019 14:21:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 757682889C for ; Thu, 21 Mar 2019 14:21:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5864D6B0272; Thu, 21 Mar 2019 10:21:00 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 536DE6B0273; Thu, 21 Mar 2019 10:21:00 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4260F6B0274; Thu, 21 Mar 2019 10:21:00 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id E1A276B0272 for ; Thu, 21 Mar 2019 10:20:59 -0400 (EDT) Received: by mail-ed1-f71.google.com with SMTP id i59so2280297edi.15 for ; Thu, 21 Mar 2019 07:20:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=VefPUADSnZrha76TxVHSwbw6LrT1e+Tmwp23y0+XUXU=; b=mJZKSYTPIFqZmbE+Mf6mcB8AzturfbjbCmJoL05yeoH6BkGnnR9dh5c2Wffh2rYG3R 7p0RoOyqZnsB7JS2TNDs/uUTbzabwN5bNEhOxNZsm2t5/qzI/IrGs7sfU50Fxj59BzMW VgEv5TQK77NpcpNbYfxxReHJ6obu4NnY3VNYyU5QlkirqsjG3bU5VTQFl7btAqlD3bYy wHf081b4wLJy1GG3xhDx4iWTt/R7dr6qRpyTt+3me8kW0/pcc/8qtFd9kEpup3Wu5lk4 1/GOapmTAHNDPMDMEV4r/nmkPvrIYKWOIiGbE25F9M/D1dS4AAAhrfosxa/kJv6zta3V YQYw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAWshK7XZIrtfezkzFJMP0rVbse7O8uxhYtESVNGwOZj3hyOzbmO t128beuCkWDTDU9eBt8ZzUjNQt0d67HVk0/RpxyRuhJDLY9BpKkKwzCQ81JLh1Fn11L8M+ln7qZ GgZ9Xdvh05pcmGxdEgteQyCa4uSa3eBMXnOBBsMNn/Hg+0XRXsExUnClvO6xLiJELRg== X-Received: by 2002:a50:9826:: with SMTP id g35mr2550262edb.247.1553178059457; Thu, 21 Mar 2019 07:20:59 -0700 (PDT) X-Google-Smtp-Source: APXvYqxuABwLTsCotftEhoeI/JfPtb2aUBDHrWPEYAqSOZEiMnc8/LuhPixOkJ8rayptZNLmWSv/ X-Received: by 2002:a50:9826:: with SMTP id g35mr2550221edb.247.1553178058654; Thu, 21 Mar 2019 07:20:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178058; cv=none; d=google.com; s=arc-20160816; b=ZxSscDdfCD91r8xoPjoMb9M33s3Go9ofHarG/A83ndz5JC6QU7wKoLd75YVYAYw6Dw 01eF9D38KpigYH1X9Y0N81U0vvu0gSNOEHcHxUYxeIJFT+lyH0oSyPB7o43i1Se3HQAW A8X0HdL/WziY4oQ64rlDfu6nksX/XDfd/Zgrn72Cw2Dh4BkK2+pftnieLc9CpvCkv2uT F2WvFDGgVXurzsIU9r1u5AiQR7isBxY5802pZA9ZD7pDY7z0XFR2Cpdd6qSvelK86J0P rRNrpoH8GKIjRpqQ0mhwT2vxOPvP29JsC77HcWSMkUSHfdmYZ0byHio9rcd70CLGy7Mi fCDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=VefPUADSnZrha76TxVHSwbw6LrT1e+Tmwp23y0+XUXU=; b=Kzbpby2gy8U2BRnze4eiSlgNp+0aISwdg8h/D2o7OOj0ZX4waUj+ECbgByz5+9wdTf CnKNmwl3BFevg+tQWvzvAdxRDI4E8IV9IyetFU9iLhHtRM8WlitDANGIXpC3gvafIoau Tc3o20hhm6dbtYGp9Gy0qMoSXNbgretNB8wJwWGWqcnd+INGDqkxKDQx1TgjEnkcYreQ ikrRLY3rAYd/+b5DiCXmrPHgKIQfbpDNdd0p+t0BeM3aWMgXdvasYi1jC+kXMdHmfA7I dOyu7/TX45DrX9+HfdWZiXlQ91wSRubqEDSM7axLKt18tKx4T5UZjjnz3HajMZCkPWZY bnDg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id f10si108898edf.247.2019.03.21.07.20.58 for ; Thu, 21 Mar 2019 07:20:58 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C274880D; Thu, 21 Mar 2019 07:20:57 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 85B413F575; Thu, 21 Mar 2019 07:20:54 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v5 14/19] x86: mm: Don't display pages which aren't present in debugfs Date: Thu, 21 Mar 2019 14:19:48 +0000 Message-Id: <20190321141953.31960-15-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP For the /sys/kernel/debug/page_tables/ files, rather than outputing a mostly empty line when a block of memory isn't present just skip the line. This keeps the output shorter and will help with a future change switching to using the generic page walk code as we no longer care about the 'level' that the page table holes are at. Signed-off-by: Steven Price --- arch/x86/mm/dump_pagetables.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index ca270fb00805..e2b53db92c34 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -304,8 +304,8 @@ static void note_page(struct seq_file *m, struct pg_state *st, /* * Now print the actual finished series */ - if (!st->marker->max_lines || - st->lines < st->marker->max_lines) { + if ((cur & _PAGE_PRESENT) && (!st->marker->max_lines || + st->lines < st->marker->max_lines)) { pt_dump_seq_printf(m, st->to_dmesg, "0x%0*lx-0x%0*lx ", width, st->start_address, @@ -321,7 +321,8 @@ static void note_page(struct seq_file *m, struct pg_state *st, printk_prot(m, st->current_prot, st->level, st->to_dmesg); } - st->lines++; + if (cur & _PAGE_PRESENT) + st->lines++; /* * We print markers for special areas of address space, From patchwork Thu Mar 21 14:19:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863643 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AD50114DE for ; Thu, 21 Mar 2019 14:21:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8CF692A2AE for ; Thu, 21 Mar 2019 14:21:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 817402A2A9; Thu, 21 Mar 2019 14:21:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AE8662A2B0 for ; Thu, 21 Mar 2019 14:21:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 908EF6B0273; Thu, 21 Mar 2019 10:21:04 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 8B95F6B0274; Thu, 21 Mar 2019 10:21:04 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7CF2C6B0275; Thu, 21 Mar 2019 10:21:04 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id 440F36B0273 for ; Thu, 21 Mar 2019 10:21:04 -0400 (EDT) Received: by mail-ed1-f71.google.com with SMTP id l19so2257653edr.12 for ; Thu, 21 Mar 2019 07:21:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=EdmI+0I92GKFAd3iMq11M1xz2Gn4jOCXE9cUiL18SLA=; b=nkHHkE+zVQlW8mR7ncwTCiTtDrEo4YCC7mevKHc4nu7kSTvql7kw4pyIfJCNzItP2Q t2kFiBfMpkVWQdMg9f6R0NSW29dnuix3UGtWpUugxWtnGhEI6D+Ro29FHk85nVsCc/47 CbzaFo5OxHVVVF4GaEhPXhWrxGB2YZNO5aIPRnRreIlpukSqCqpzYir4m2Ky2ddy7Br+ hCW8gtMBBT+bzsyz4MnY3uBGVzF4R+mlK0yaB3d4dek7U/R9niLhHFvTm1oPHkk7n0bK 7hgki29WhEof+JnJ1j4AZuxlkViZflhSU1spHuZaTY+zcc3iXNXsCePZ2GoKsRPdoYCi TG7A== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAXpa/tNsjS8ZiTRGhwyGGKG4+YGfM30HEsy0Ce9tZZpfKFXNrRh BpDyMotFW5Q/BQqOQv3z7xobrl8fTo1EBbwz8mE7sV+c9fO6EpMEnQERXYoBC4uXwHjh83/W+hj ehwCwgw/ikJWNFqEKu7U7IBgOTmwVPtJJrzPZBPRFduIPehCR1WWtW1UZFvzC9VVPuw== X-Received: by 2002:a17:906:5949:: with SMTP id g9mr2491971ejr.15.1553178063663; Thu, 21 Mar 2019 07:21:03 -0700 (PDT) X-Google-Smtp-Source: APXvYqyQ6i9suOZO09+OGmz4BPUDstDVNxIRjXGyh+i37VGLGS290OExKTRvtXQNHYwsRVXJKXqi X-Received: by 2002:a17:906:5949:: with SMTP id g9mr2491914ejr.15.1553178062500; Thu, 21 Mar 2019 07:21:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178062; cv=none; d=google.com; s=arc-20160816; b=GhrcTp/6sQH+l1uwUHCNwOEujjn/1sKxjWik5N5+AbKgO5dHEHiiuAflEgXumeO1v9 7zZlE4xxHYersScG2Xu8NLGv2ojIKTMfROyfaQt8lRRWqOWLW03PLd2sLTlyNNzTn6+W OarfvrrEn19ZV41G+TOKLNF5qlcambQ6JOW3Qx3s7WNrhGPPONsX98RcCRA1XsF7Ly79 iHjqbBipYQYF30Amo4SQr3vK9gpCxth6yUvHWJWu1b0X05YA/+ZKgV74FfqkJYWLrSHh RifynzopLl8FPPnf6F/DkYHAy0jT9DQ6gwq8/1mknXFFxnFMc8iKiKz6IrdYJ3KwDMT5 aGNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=EdmI+0I92GKFAd3iMq11M1xz2Gn4jOCXE9cUiL18SLA=; b=Cscg/5RfHVHuUT4pkWqLWWSkcY1glvygu25UAOrbMQT9bTyF9xb4mT5Wq04E9MS1OO HBblnbhtlqlUEucT5cJENwOkCXdn5bhnImNpll3Ya1LHeUjL0baPipJICe3OPQqTmxet ZIxH4yP99viaksrtLBMPR8s1F9B4vXOFq9iV9za5+va+ChJ4nFCvrL3iD6xy1jjx7jno sdqrfPir8euNGM9d2WVs12XYzzCpunoqidIJDvdnDo2mWyah/Mp/FM2XSPkZox6b3uoi u1bZYdPosO6fys7iZf5axMJRiybIq92zcrcNNvzxPYl3rsk+oKARnltI49nmrLKOtfQR Knqg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id e21si2081917edd.287.2019.03.21.07.21.02 for ; Thu, 21 Mar 2019 07:21:02 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6C34280D; Thu, 21 Mar 2019 07:21:01 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0DBAF3F575; Thu, 21 Mar 2019 07:20:57 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v5 15/19] x86: mm: Point to struct seq_file from struct pg_state Date: Thu, 21 Mar 2019 14:19:49 +0000 Message-Id: <20190321141953.31960-16-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP mm/dump_pagetables.c passes both struct seq_file and struct pg_state down the chain of walk_*_level() functions to be passed to note_page(). Instead place the struct seq_file in struct pg_state and access it from struct pg_state (which is private to this file) in note_page(). Signed-off-by: Steven Price --- arch/x86/mm/dump_pagetables.c | 69 ++++++++++++++++++----------------- 1 file changed, 35 insertions(+), 34 deletions(-) diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index e2b53db92c34..3d12ac031144 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -40,6 +40,7 @@ struct pg_state { bool to_dmesg; bool check_wx; unsigned long wx_pages; + struct seq_file *seq; }; struct addr_marker { @@ -268,11 +269,12 @@ static void note_wx(struct pg_state *st) * of PTE entries; the next one is different so we need to * print what we collected so far. */ -static void note_page(struct seq_file *m, struct pg_state *st, - pgprot_t new_prot, pgprotval_t new_eff, int level) +static void note_page(struct pg_state *st, pgprot_t new_prot, + pgprotval_t new_eff, int level) { pgprotval_t prot, cur, eff; static const char units[] = "BKMGTPE"; + struct seq_file *m = st->seq; /* * If we have a "break" in the series, we need to flush the state that @@ -358,8 +360,8 @@ static inline pgprotval_t effective_prot(pgprotval_t prot1, pgprotval_t prot2) ((prot1 | prot2) & _PAGE_NX); } -static void walk_pte_level(struct seq_file *m, struct pg_state *st, pmd_t addr, - pgprotval_t eff_in, unsigned long P) +static void walk_pte_level(struct pg_state *st, pmd_t addr, pgprotval_t eff_in, + unsigned long P) { int i; pte_t *pte; @@ -370,7 +372,7 @@ static void walk_pte_level(struct seq_file *m, struct pg_state *st, pmd_t addr, pte = pte_offset_map(&addr, st->current_address); prot = pte_flags(*pte); eff = effective_prot(eff_in, prot); - note_page(m, st, __pgprot(prot), eff, 5); + note_page(st, __pgprot(prot), eff, 5); pte_unmap(pte); } } @@ -383,22 +385,20 @@ static void walk_pte_level(struct seq_file *m, struct pg_state *st, pmd_t addr, * us dozens of seconds (minutes for 5-level config) while checking for * W+X mapping or reading kernel_page_tables debugfs file. */ -static inline bool kasan_page_table(struct seq_file *m, struct pg_state *st, - void *pt) +static inline bool kasan_page_table(struct pg_state *st, void *pt) { if (__pa(pt) == __pa(kasan_early_shadow_pmd) || (pgtable_l5_enabled() && __pa(pt) == __pa(kasan_early_shadow_p4d)) || __pa(pt) == __pa(kasan_early_shadow_pud)) { pgprotval_t prot = pte_flags(kasan_early_shadow_pte[0]); - note_page(m, st, __pgprot(prot), 0, 5); + note_page(st, __pgprot(prot), 0, 5); return true; } return false; } #else -static inline bool kasan_page_table(struct seq_file *m, struct pg_state *st, - void *pt) +static inline bool kasan_page_table(struct pg_state *st, void *pt) { return false; } @@ -406,7 +406,7 @@ static inline bool kasan_page_table(struct seq_file *m, struct pg_state *st, #if PTRS_PER_PMD > 1 -static void walk_pmd_level(struct seq_file *m, struct pg_state *st, pud_t addr, +static void walk_pmd_level(struct pg_state *st, pud_t addr, pgprotval_t eff_in, unsigned long P) { int i; @@ -420,19 +420,19 @@ static void walk_pmd_level(struct seq_file *m, struct pg_state *st, pud_t addr, prot = pmd_flags(*start); eff = effective_prot(eff_in, prot); if (pmd_large(*start) || !pmd_present(*start)) { - note_page(m, st, __pgprot(prot), eff, 4); - } else if (!kasan_page_table(m, st, pmd_start)) { - walk_pte_level(m, st, *start, eff, + note_page(st, __pgprot(prot), eff, 4); + } else if (!kasan_page_table(st, pmd_start)) { + walk_pte_level(st, *start, eff, P + i * PMD_LEVEL_MULT); } } else - note_page(m, st, __pgprot(0), 0, 4); + note_page(st, __pgprot(0), 0, 4); start++; } } #else -#define walk_pmd_level(m,s,a,e,p) walk_pte_level(m,s,__pmd(pud_val(a)),e,p) +#define walk_pmd_level(s,a,e,p) walk_pte_level(s,__pmd(pud_val(a)),e,p) #undef pud_large #define pud_large(a) pmd_large(__pmd(pud_val(a))) #define pud_none(a) pmd_none(__pmd(pud_val(a))) @@ -440,8 +440,8 @@ static void walk_pmd_level(struct seq_file *m, struct pg_state *st, pud_t addr, #if PTRS_PER_PUD > 1 -static void walk_pud_level(struct seq_file *m, struct pg_state *st, p4d_t addr, - pgprotval_t eff_in, unsigned long P) +static void walk_pud_level(struct pg_state *st, p4d_t addr, pgprotval_t eff_in, + unsigned long P) { int i; pud_t *start, *pud_start; @@ -455,34 +455,34 @@ static void walk_pud_level(struct seq_file *m, struct pg_state *st, p4d_t addr, prot = pud_flags(*start); eff = effective_prot(eff_in, prot); if (pud_large(*start) || !pud_present(*start)) { - note_page(m, st, __pgprot(prot), eff, 3); - } else if (!kasan_page_table(m, st, pud_start)) { - walk_pmd_level(m, st, *start, eff, + note_page(st, __pgprot(prot), eff, 3); + } else if (!kasan_page_table(st, pud_start)) { + walk_pmd_level(st, *start, eff, P + i * PUD_LEVEL_MULT); } } else - note_page(m, st, __pgprot(0), 0, 3); + note_page(st, __pgprot(0), 0, 3); start++; } } #else -#define walk_pud_level(m,s,a,e,p) walk_pmd_level(m,s,__pud(p4d_val(a)),e,p) +#define walk_pud_level(s,a,e,p) walk_pmd_level(s,__pud(p4d_val(a)),e,p) #undef p4d_large #define p4d_large(a) pud_large(__pud(p4d_val(a))) #define p4d_none(a) pud_none(__pud(p4d_val(a))) #endif -static void walk_p4d_level(struct seq_file *m, struct pg_state *st, pgd_t addr, - pgprotval_t eff_in, unsigned long P) +static void walk_p4d_level(struct pg_state *st, pgd_t addr, pgprotval_t eff_in, + unsigned long P) { int i; p4d_t *start, *p4d_start; pgprotval_t prot, eff; if (PTRS_PER_P4D == 1) - return walk_pud_level(m, st, __p4d(pgd_val(addr)), eff_in, P); + return walk_pud_level(st, __p4d(pgd_val(addr)), eff_in, P); p4d_start = start = (p4d_t *)pgd_page_vaddr(addr); @@ -492,13 +492,13 @@ static void walk_p4d_level(struct seq_file *m, struct pg_state *st, pgd_t addr, prot = p4d_flags(*start); eff = effective_prot(eff_in, prot); if (p4d_large(*start) || !p4d_present(*start)) { - note_page(m, st, __pgprot(prot), eff, 2); - } else if (!kasan_page_table(m, st, p4d_start)) { - walk_pud_level(m, st, *start, eff, + note_page(st, __pgprot(prot), eff, 2); + } else if (!kasan_page_table(st, p4d_start)) { + walk_pud_level(st, *start, eff, P + i * P4D_LEVEL_MULT); } } else - note_page(m, st, __pgprot(0), 0, 2); + note_page(st, __pgprot(0), 0, 2); start++; } @@ -536,6 +536,7 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, } st.check_wx = checkwx; + st.seq = m; if (checkwx) st.wx_pages = 0; @@ -549,13 +550,13 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, eff = prot; #endif if (pgd_large(*start) || !pgd_present(*start)) { - note_page(m, &st, __pgprot(prot), eff, 1); + note_page(&st, __pgprot(prot), eff, 1); } else { - walk_p4d_level(m, &st, *start, eff, + walk_p4d_level(&st, *start, eff, i * PGD_LEVEL_MULT); } } else - note_page(m, &st, __pgprot(0), 0, 1); + note_page(&st, __pgprot(0), 0, 1); cond_resched(); start++; @@ -563,7 +564,7 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, /* Flush out the last page */ st.current_address = normalize_addr(PTRS_PER_PGD*PGD_LEVEL_MULT); - note_page(m, &st, __pgprot(0), 0, 0); + note_page(&st, __pgprot(0), 0, 0); if (!checkwx) return; if (st.wx_pages) From patchwork Thu Mar 21 14:19:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863647 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 32C2E1390 for ; Thu, 21 Mar 2019 14:21:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 17A9D2A06B for ; Thu, 21 Mar 2019 14:21:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 157092A2A9; Thu, 21 Mar 2019 14:21:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7E0FF2A06B for ; Thu, 21 Mar 2019 14:21:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A690C6B0275; Thu, 21 Mar 2019 10:21:07 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A18B96B0276; Thu, 21 Mar 2019 10:21:07 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 92E436B0277; Thu, 21 Mar 2019 10:21:07 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id 454486B0275 for ; Thu, 21 Mar 2019 10:21:07 -0400 (EDT) Received: by mail-ed1-f71.google.com with SMTP id n12so2298701edo.5 for ; Thu, 21 Mar 2019 07:21:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=82GVaLwkCHLAJuw0cg7cs7mz13Kyr233/Gpjb9qS/I0=; b=pX/PB4n2IeBS7sjZhWU1hfRYsJGKweBLwoVKBOHKWGQsCoqtuQG9bF/iOeSghyo7Bg HUFXuqRqMYTVWtkgzozT8mnj0PHrBZt7OZJ7SGu6GxwNAUe3gjxSlFWUJe4HbVDkFph5 AGwFsPbqTiRzlm8APwLsNb/Tdumxv3Lu4aFdqwN5tB/3xkHQc5yAvyNDsoyBq8BWTfRX mBAUUHBBZlGQ4TBGkdc7HkW74h0lOMmlbzpyLyo2rqAN+GVZkF1SAW0jxSvpRCLWrCIg +hXgrqFjLxolIw3jPPd7xSakTyzckp0rMrLMMfcIoEPbBRm9sNvdNC5g7qf3FxfZ/q7+ 9+kQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAV57+ezVYF/wsY6Ze0VJ2I6twwkLKAPCQ4x0lY5sGngEW+6whpX oDt0p2dDriDaP3Zvhde97xp9xZHw0MywYulAT7c5+NIg61oz/YdbeGG6FNWBp5mJIpn4QXB4+bu UxZSjL4i/KyGxZW42Gvw9tjH9n/LmIOqQgv93cAQjGiXo4D5gpIcmXtDF/vxbYNI82Q== X-Received: by 2002:a50:fe14:: with SMTP id f20mr2610472edt.187.1553178066818; Thu, 21 Mar 2019 07:21:06 -0700 (PDT) X-Google-Smtp-Source: APXvYqyXzurXw7Q4bu/3YCXHYlrtBq+G3uJ++P4j5gQ1trZT8D+OF78rz5lNdfgYkkaNw/uIkMsg X-Received: by 2002:a50:fe14:: with SMTP id f20mr2610435edt.187.1553178065996; Thu, 21 Mar 2019 07:21:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178065; cv=none; d=google.com; s=arc-20160816; b=NqWePeW6BAJUGsCxIyB8hmvY83n8Z8UGwwb5dMX1NudcbDy3BFcdR4WR6TPkMmyKnU ejJ8jyUpIEsOEZfQ6pTWs6jmu9q0gwP1kjpsbIbsFLuhPm1j3IFOJ3nq0D13GlYQujU8 S/g5Q0NNsMmGn7fUs3tyepyFxdxxOiAa7lEmcrmvS24//L60MuarY6QFGEA+sul1ofW2 b+OhS9cdCRt/NyOj8exv3Sn9juEbl631wqpx9/vGQH56t8jI56ya9uwOjV4opr481+WT bQeGacwonexaZQi5BcojGXxAKouxq0ptNUnP0MGSwAp3rZK+6fiYTQeoQz7dokMFteId /82Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=82GVaLwkCHLAJuw0cg7cs7mz13Kyr233/Gpjb9qS/I0=; b=qJZpZzQgvblT4kSu1EvSWGm+NdyA3dalpfc0vX+OX4APEj2dhkeQlCrN9vv48nd3xy /rCyv96s1G90t+bugjF5KWmSeCG1qrMQxwUnYwe258jeHoO1yovDcLb77GOpdJQtlEpM rChX4/tnCwmXvYeqtF62okI6F9qQ+bo3zBYqS2AlwPKHQjusFrld32lLNzHhxDzSolRS 5eKAYiVsUPJcKV7bVE9ELYfjsChzWf+lwqRUNTrRyIStI1cH4YJ+qD1EdmkGNYEOvXns Y6Xt3hV7At0gPkfg+COWf74ZfZRgpj0j4alqgK++8ZXV/cF5D308eikUWh++a1w30yu2 5fiQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id t17si507611edf.305.2019.03.21.07.21.05 for ; Thu, 21 Mar 2019 07:21:05 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E637BEBD; Thu, 21 Mar 2019 07:21:04 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id AB8203F575; Thu, 21 Mar 2019 07:21:01 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v5 16/19] x86: mm+efi: Convert ptdump_walk_pgd_level() to take a mm_struct Date: Thu, 21 Mar 2019 14:19:50 +0000 Message-Id: <20190321141953.31960-17-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP To enable x86 to use the generic walk_page_range() function, the callers of ptdump_walk_pgd_level() need to pass an mm_struct rather than the raw pgd_t pointer. Luckily since commit 7e904a91bf60 ("efi: Use efi_mm in x86 as well as ARM") we now have an mm_struct for EFI on x86. Signed-off-by: Steven Price --- arch/x86/include/asm/pgtable.h | 2 +- arch/x86/mm/dump_pagetables.c | 4 ++-- arch/x86/platform/efi/efi_32.c | 2 +- arch/x86/platform/efi/efi_64.c | 4 ++-- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 0dd04cf6ebeb..579959750f34 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -27,7 +27,7 @@ extern pgd_t early_top_pgt[PTRS_PER_PGD]; int __init __early_make_pgtable(unsigned long address, pmdval_t pmd); -void ptdump_walk_pgd_level(struct seq_file *m, pgd_t *pgd); +void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm); void ptdump_walk_pgd_level_debugfs(struct seq_file *m, pgd_t *pgd, bool user); void ptdump_walk_pgd_level_checkwx(void); void ptdump_walk_user_pgd_level_checkwx(void); diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index 3d12ac031144..ddf8ea6b059d 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -574,9 +574,9 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, pr_info("x86/mm: Checked W+X mappings: passed, no W+X pages found.\n"); } -void ptdump_walk_pgd_level(struct seq_file *m, pgd_t *pgd) +void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm) { - ptdump_walk_pgd_level_core(m, pgd, false, true); + ptdump_walk_pgd_level_core(m, mm->pgd, false, true); } void ptdump_walk_pgd_level_debugfs(struct seq_file *m, pgd_t *pgd, bool user) diff --git a/arch/x86/platform/efi/efi_32.c b/arch/x86/platform/efi/efi_32.c index 9959657127f4..9175ceaa6e72 100644 --- a/arch/x86/platform/efi/efi_32.c +++ b/arch/x86/platform/efi/efi_32.c @@ -49,7 +49,7 @@ void efi_sync_low_kernel_mappings(void) {} void __init efi_dump_pagetable(void) { #ifdef CONFIG_EFI_PGT_DUMP - ptdump_walk_pgd_level(NULL, swapper_pg_dir); + ptdump_walk_pgd_level(NULL, init_mm); #endif } diff --git a/arch/x86/platform/efi/efi_64.c b/arch/x86/platform/efi/efi_64.c index cf0347f61b21..a2e0f9800190 100644 --- a/arch/x86/platform/efi/efi_64.c +++ b/arch/x86/platform/efi/efi_64.c @@ -611,9 +611,9 @@ void __init efi_dump_pagetable(void) { #ifdef CONFIG_EFI_PGT_DUMP if (efi_enabled(EFI_OLD_MEMMAP)) - ptdump_walk_pgd_level(NULL, swapper_pg_dir); + ptdump_walk_pgd_level(NULL, init_mm); else - ptdump_walk_pgd_level(NULL, efi_mm.pgd); + ptdump_walk_pgd_level(NULL, efi_mm); #endif } From patchwork Thu Mar 21 14:19:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863649 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D1F791515 for ; Thu, 21 Mar 2019 14:21:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B4D1A2A17A for ; Thu, 21 Mar 2019 14:21:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A91F32A2B1; Thu, 21 Mar 2019 14:21:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1A4832A17A for ; Thu, 21 Mar 2019 14:21:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 586AA6B0277; Thu, 21 Mar 2019 10:21:11 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5363F6B0278; Thu, 21 Mar 2019 10:21:11 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 451446B0279; Thu, 21 Mar 2019 10:21:11 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by kanga.kvack.org (Postfix) with ESMTP id E172A6B0277 for ; Thu, 21 Mar 2019 10:21:10 -0400 (EDT) Received: by mail-ed1-f70.google.com with SMTP id o27so2279817edc.14 for ; Thu, 21 Mar 2019 07:21:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=QI1ft8uG/6LmqX3YnTkHhfrqxc9+RDV6RJGxqI2lX8w=; b=GSIuQ7cd8bfNPfqwdPKqFPs5U6lkSv2P7HoHWJxUQ/fQRbojBVmJ9HAvRhqCffNZUK WEYJDJOdl5EZexPS95Myiy7HU4KgF2Kf5rcGPuOjRO918w9DVPZOCJCJCbFZMCYxCXj9 OE6fLrlwIAKTKpMrr2Ca3k6pI4rfsZyaq2gJQlBjIEs0F9i3qe+DmDsxT+VjjPvaXvq2 3Xes3qmQDs1GAl1rusv1O4SJGwOQNOfQJ5K4x1v90OTqr95+nfHNE9sNVmIizMcjy5rO ylAJYCClifjIsBz29GkZQEIzvVv4ahTp/a6wTeqenHqblsKkFGHQoKvA6KJM2n/8vZ8G 5IFw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAWdxcpAR00COwGFRw8di+MoUPKl91RICryqhDLSIElM+dnMHxoQ aj+gvDBsK8h/Wc2p50REWrdki2vv/4NODHidNNfcTzAdgpyBmr1htvT8vlxTSTCStFcTsICe8YI qRdMjkfT4OfbByKKRJ6BEIe4QHLKEFVXnluutI8KgiuUpcXhccjMhJc7mWyF0weyqxA== X-Received: by 2002:a17:906:708d:: with SMTP id b13mr2363072ejk.120.1553178070441; Thu, 21 Mar 2019 07:21:10 -0700 (PDT) X-Google-Smtp-Source: APXvYqxgs1x9XcDLpCnhRPVb0+GB0NmeZ6dm7X4+SO4Vpmh8cK3RDzo52S4MRqThtOMu/mJBeBm5 X-Received: by 2002:a17:906:708d:: with SMTP id b13mr2363033ejk.120.1553178069449; Thu, 21 Mar 2019 07:21:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178069; cv=none; d=google.com; s=arc-20160816; b=Wd2S2baUihNObolVNvar1Q9FZaLx4GV1dtzde8Qo9Z84EUOI/ZQC2E3omL7RO9mc8N Rir6KyeebOE+hlql1i9oJIaKZqFcnoUESMBWhtLK7/Nl3BHBeF6alV7WDjivNF1ZBeF2 BV6h2c9NiPvj6qosZKd0uW9Ym9yKjbZC0ek98YVaTXRo1EQQuNX2/iphvWx5N5UWuXeB pGLbxs36n4Jr9e+5gFBSr62hh2X1QiiMpOoTPNN9QMFQ4Xi8h3hgOKhfrsdzYt4ug69P jB1bIqElQ2g9p8bK4pkp8VY7xr5BWPwlAHceM9NTq2wJaZxmWTZv1kMdp2cMoapdg8Q3 6RfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=QI1ft8uG/6LmqX3YnTkHhfrqxc9+RDV6RJGxqI2lX8w=; b=rRzp+M5+bFJ1PeSgX+tc0kIZGUfqo6M9BR2LSiPZaFwDJGEMIGKr7hsvXI2NmKwvuq qm/4pba5L9Cu3kERJMsaAjsSr++GPK7s80Kyv4LBWOg/7WvVo11Q0ib2HJd7IVmRfgJT o8YzgDL99JMguGPb93MEMfi8P0B4zUJqBJIUbzYFMhTPcvogZBrqmrLRBkFl1AUy/bsC gdGtQ6zZZDraiXZYgwcSnwT+IV/QsEcgOTWff0flWufGmIX9xnbZ42sHRpm94rpnR+uH lNerLUF9bD5OC2NXNS7tBfnxsdyZ3eJDDIPxWbvIsNDrUB53e67EVbSBJAldyjND92rA EFCQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id e3si2150862edd.270.2019.03.21.07.21.09 for ; Thu, 21 Mar 2019 07:21:09 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6EB6F165C; Thu, 21 Mar 2019 07:21:08 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 31C9C3F575; Thu, 21 Mar 2019 07:21:05 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v5 17/19] x86: mm: Convert ptdump_walk_pgd_level_debugfs() to take an mm_struct Date: Thu, 21 Mar 2019 14:19:51 +0000 Message-Id: <20190321141953.31960-18-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP To enable x86 to use the generic walk_page_range() function, the callers of ptdump_walk_pgd_level_debugfs() need to pass in the mm_struct. This means that ptdump_walk_pgd_level_core() is now always passed a valid pgd, so drop the support for pgd==NULL. Signed-off-by: Steven Price --- arch/x86/include/asm/pgtable.h | 3 ++- arch/x86/mm/debug_pagetables.c | 8 ++++---- arch/x86/mm/dump_pagetables.c | 14 ++++++-------- 3 files changed, 12 insertions(+), 13 deletions(-) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 579959750f34..5abf693dc9b2 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -28,7 +28,8 @@ extern pgd_t early_top_pgt[PTRS_PER_PGD]; int __init __early_make_pgtable(unsigned long address, pmdval_t pmd); void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm); -void ptdump_walk_pgd_level_debugfs(struct seq_file *m, pgd_t *pgd, bool user); +void ptdump_walk_pgd_level_debugfs(struct seq_file *m, struct mm_struct *mm, + bool user); void ptdump_walk_pgd_level_checkwx(void); void ptdump_walk_user_pgd_level_checkwx(void); diff --git a/arch/x86/mm/debug_pagetables.c b/arch/x86/mm/debug_pagetables.c index cd84f067e41d..824131052574 100644 --- a/arch/x86/mm/debug_pagetables.c +++ b/arch/x86/mm/debug_pagetables.c @@ -6,7 +6,7 @@ static int ptdump_show(struct seq_file *m, void *v) { - ptdump_walk_pgd_level_debugfs(m, NULL, false); + ptdump_walk_pgd_level_debugfs(m, &init_mm, false); return 0; } @@ -16,7 +16,7 @@ static int ptdump_curknl_show(struct seq_file *m, void *v) { if (current->mm->pgd) { down_read(¤t->mm->mmap_sem); - ptdump_walk_pgd_level_debugfs(m, current->mm->pgd, false); + ptdump_walk_pgd_level_debugfs(m, current->mm, false); up_read(¤t->mm->mmap_sem); } return 0; @@ -31,7 +31,7 @@ static int ptdump_curusr_show(struct seq_file *m, void *v) { if (current->mm->pgd) { down_read(¤t->mm->mmap_sem); - ptdump_walk_pgd_level_debugfs(m, current->mm->pgd, true); + ptdump_walk_pgd_level_debugfs(m, current->mm, true); up_read(¤t->mm->mmap_sem); } return 0; @@ -46,7 +46,7 @@ static struct dentry *pe_efi; static int ptdump_efi_show(struct seq_file *m, void *v) { if (efi_mm.pgd) - ptdump_walk_pgd_level_debugfs(m, efi_mm.pgd, false); + ptdump_walk_pgd_level_debugfs(m, &efi_mm, false); return 0; } diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index ddf8ea6b059d..40b3f1da6e15 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -525,16 +525,12 @@ static inline bool is_hypervisor_range(int idx) static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, bool checkwx, bool dmesg) { - pgd_t *start = INIT_PGD; + pgd_t *start = pgd; pgprotval_t prot, eff; int i; struct pg_state st = {}; - if (pgd) { - start = pgd; - st.to_dmesg = dmesg; - } - + st.to_dmesg = dmesg; st.check_wx = checkwx; st.seq = m; if (checkwx) @@ -579,8 +575,10 @@ void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm) ptdump_walk_pgd_level_core(m, mm->pgd, false, true); } -void ptdump_walk_pgd_level_debugfs(struct seq_file *m, pgd_t *pgd, bool user) +void ptdump_walk_pgd_level_debugfs(struct seq_file *m, struct mm_struct *mm, + bool user) { + pgd_t *pgd = mm->pgd; #ifdef CONFIG_PAGE_TABLE_ISOLATION if (user && static_cpu_has(X86_FEATURE_PTI)) pgd = kernel_to_user_pgdp(pgd); @@ -606,7 +604,7 @@ void ptdump_walk_user_pgd_level_checkwx(void) void ptdump_walk_pgd_level_checkwx(void) { - ptdump_walk_pgd_level_core(NULL, NULL, true, false); + ptdump_walk_pgd_level_core(NULL, INIT_PGD, true, false); } static int __init pt_dump_init(void) From patchwork Thu Mar 21 14:19:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863653 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 767491390 for ; Thu, 21 Mar 2019 14:21:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 57E2E292FA for ; Thu, 21 Mar 2019 14:21:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4BFAC2A2AC; Thu, 21 Mar 2019 14:21:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B79E7292FA for ; Thu, 21 Mar 2019 14:21:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C3D1D6B0279; Thu, 21 Mar 2019 10:21:14 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BED0B6B027A; Thu, 21 Mar 2019 10:21:14 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B02986B027B; Thu, 21 Mar 2019 10:21:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by kanga.kvack.org (Postfix) with ESMTP id 61C086B0279 for ; Thu, 21 Mar 2019 10:21:14 -0400 (EDT) Received: by mail-ed1-f71.google.com with SMTP id h27so2261258eda.8 for ; Thu, 21 Mar 2019 07:21:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=0Qu5DM7t7O6eLZ930vwdwyGGSyJREtb45OisMEetzTk=; b=NW+H9YJ0EBMaSTkG46Kdq0/ErGBO4hOVohMx58YfIFTgEaujxVCGWEdkr/wf2wRO0K hH5lxkfF1tidb6ztAh/kZY3To/mc7wzWrLXsekzpmuU88dV+Qvmbvxkgl/9IM37h/WYr LzQkGDJbFvVB2ovd2wWKBBNXpyZucxH+q7Gf1t5NGMYHL7sHinEs1Ek6MikGvgNuyJzE LdwpKPUDwR7O1gOc7lUvgzw5vkAB5fHLx0CiPlCllAXNonuCjJJZ4idwgrg4Z7VhVwOn TDZquD+pYaTFKqsWcerk6Sq79eI70cY/A+/4sph0LUBoChm2RoXRgdBVYOjtQDgBgI/R s2hw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAW+c5EafexHTEP95XeLshWewWe9qQGS1Lw9oMSuc2FDXuIzRKu+ VL0oXQM+15DKbybnw6MtUnj99QZe/NtQSzOp6ZI7qw8/0z+S1f2zxwO99L/izxuurkFfUVytn4f BUMhMzMhbGPimHaut2ntWRccX44eZq7uLVESJTHKq8PU21KtLDzCxxgV/nTQ82s/SgA== X-Received: by 2002:a17:906:77c1:: with SMTP id m1mr2183370ejn.148.1553178073917; Thu, 21 Mar 2019 07:21:13 -0700 (PDT) X-Google-Smtp-Source: APXvYqzGIlmz/Yms0NzHwFuqCH3jp2gOf+ttdlWH/Yt8/NMwwT/qGE6zDqeF+nfSxrpYTDggqPs5 X-Received: by 2002:a17:906:77c1:: with SMTP id m1mr2183336ejn.148.1553178073005; Thu, 21 Mar 2019 07:21:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178073; cv=none; d=google.com; s=arc-20160816; b=Ht91OfLmPcs8kfN/6adMpRo0E70UcU2a3ZdotlB+QxTrMAoe4EYtZkdO4rzm6wstZQ z+paVy3mqzl7Rkhhpo111+kcAcD5Qo+T7th5QU/LCFJ6nBb0jhZAmgdqw+gmJ72co+Jg EPR34wy53kjz8Svbcn+rPt1Ck9u/7AB2z7YzxAKR4pGXyBsBbbQckVE/BGnL2yMaKm4v p2QaFHiiLlddKhRjxcaZLQJhzm412f5xyJXuHI7XLECqhIZSxXApOVxnZnih/eRabEJ6 60m571YdaEpCUxLozUZ8mmOYbuBKWZuDdXcmg5lrmcCZOfbnhGrNHy4BsezXaDfZE5Xg RF0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=0Qu5DM7t7O6eLZ930vwdwyGGSyJREtb45OisMEetzTk=; b=eC81z+GFvsv9b1btcgeeJE6wBctHyOtx/ZT7xwiUuCGjDSa713yt6kcz5QlU/qsGWt kldt4iNzVKlrpX/gxACVAQ5XnVOuXAZZ95VzWFpV4l/F9fsoVZ2DoUCVEm5oaLpDkLIw Mh6+xu5ILCteyx//6Wu9rrykUp6ucdZIXNx11hhPFDDt2kRcZcWqEAOWHsyEZIBV9PGJ bMDj20VVj6KpuqnbBl9nBAIUHBI0ND5vQynLo8+JeXKRYbCXlXDEMxXWaywfBUDlxH8B 4VpkpIOGuLmRw++445ASnxqJMSM0RdkmZJIWTz+XK/rbOxqTnCCeIOUvNdh4Z+dJjW8s GaJQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id m29si2132203edm.213.2019.03.21.07.21.12 for ; Thu, 21 Mar 2019 07:21:12 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E8ECB168F; Thu, 21 Mar 2019 07:21:11 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id AE3743F575; Thu, 21 Mar 2019 07:21:08 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v5 18/19] x86: mm: Convert ptdump_walk_pgd_level_core() to take an mm_struct Date: Thu, 21 Mar 2019 14:19:52 +0000 Message-Id: <20190321141953.31960-19-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP An mm_struct is needed to enable x86 to use of the generic walk_page_range() function. In the case of walking the user page tables (when CONFIG_PAGE_TABLE_ISOLATION is enabled), it is necessary to create a fake_mm structure because there isn't an mm_struct with a pointer to the pgd of the user page tables. This fake_mm structure is initialised with the minimum necessary for the generic page walk code. Signed-off-by: Steven Price --- arch/x86/mm/dump_pagetables.c | 36 ++++++++++++++++++++--------------- 1 file changed, 21 insertions(+), 15 deletions(-) diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index 40b3f1da6e15..c0fbb9e5a790 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -111,8 +111,6 @@ static struct addr_marker address_markers[] = { [END_OF_SPACE_NR] = { -1, NULL } }; -#define INIT_PGD ((pgd_t *) &init_top_pgt) - #else /* CONFIG_X86_64 */ enum address_markers_idx { @@ -147,8 +145,6 @@ static struct addr_marker address_markers[] = { [END_OF_SPACE_NR] = { -1, NULL } }; -#define INIT_PGD (swapper_pg_dir) - #endif /* !CONFIG_X86_64 */ /* Multipliers for offsets within the PTEs */ @@ -522,10 +518,10 @@ static inline bool is_hypervisor_range(int idx) #endif } -static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, +static void ptdump_walk_pgd_level_core(struct seq_file *m, struct mm_struct *mm, bool checkwx, bool dmesg) { - pgd_t *start = pgd; + pgd_t *start = mm->pgd; pgprotval_t prot, eff; int i; struct pg_state st = {}; @@ -572,39 +568,49 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, pgd_t *pgd, void ptdump_walk_pgd_level(struct seq_file *m, struct mm_struct *mm) { - ptdump_walk_pgd_level_core(m, mm->pgd, false, true); + ptdump_walk_pgd_level_core(m, mm, false, true); } +#ifdef CONFIG_PAGE_TABLE_ISOLATION +static void ptdump_walk_pgd_level_user_core(struct seq_file *m, + struct mm_struct *mm, + bool checkwx, bool dmesg) +{ + struct mm_struct fake_mm = { + .pgd = kernel_to_user_pgdp(mm->pgd) + }; + init_rwsem(&fake_mm.mmap_sem); + ptdump_walk_pgd_level_core(m, &fake_mm, checkwx, dmesg); +} +#endif + void ptdump_walk_pgd_level_debugfs(struct seq_file *m, struct mm_struct *mm, bool user) { - pgd_t *pgd = mm->pgd; #ifdef CONFIG_PAGE_TABLE_ISOLATION if (user && static_cpu_has(X86_FEATURE_PTI)) - pgd = kernel_to_user_pgdp(pgd); + ptdump_walk_pgd_level_user_core(m, mm, false, false); + else #endif - ptdump_walk_pgd_level_core(m, pgd, false, false); + ptdump_walk_pgd_level_core(m, mm, false, false); } EXPORT_SYMBOL_GPL(ptdump_walk_pgd_level_debugfs); void ptdump_walk_user_pgd_level_checkwx(void) { #ifdef CONFIG_PAGE_TABLE_ISOLATION - pgd_t *pgd = INIT_PGD; - if (!(__supported_pte_mask & _PAGE_NX) || !static_cpu_has(X86_FEATURE_PTI)) return; pr_info("x86/mm: Checking user space page tables\n"); - pgd = kernel_to_user_pgdp(pgd); - ptdump_walk_pgd_level_core(NULL, pgd, true, false); + ptdump_walk_pgd_level_user_core(NULL, &init_mm, true, false); #endif } void ptdump_walk_pgd_level_checkwx(void) { - ptdump_walk_pgd_level_core(NULL, INIT_PGD, true, false); + ptdump_walk_pgd_level_core(NULL, &init_mm, true, false); } static int __init pt_dump_init(void) From patchwork Thu Mar 21 14:19:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Price X-Patchwork-Id: 10863655 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 75D3014DE for ; Thu, 21 Mar 2019 14:21:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 50C1E292FA for ; Thu, 21 Mar 2019 14:21:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 44D132A2AA; Thu, 21 Mar 2019 14:21:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 079222A2AF for ; Thu, 21 Mar 2019 14:21:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 97E746B027A; Thu, 21 Mar 2019 10:21:18 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 908C56B027B; Thu, 21 Mar 2019 10:21:18 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81DAB6B027C; Thu, 21 Mar 2019 10:21:18 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by kanga.kvack.org (Postfix) with ESMTP id 294506B027A for ; Thu, 21 Mar 2019 10:21:18 -0400 (EDT) Received: by mail-ed1-f72.google.com with SMTP id p4so2296268edd.0 for ; Thu, 21 Mar 2019 07:21:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=kX4dLJraJo092n4y125DLKk3n4IU7HsLifEATF3quag=; b=b3RU9or3VH5TjZNLagAzSnoQg2jDHmtehXBtFnLRRapzP47g0wy0s31YQyynt6TzXu KNLxW6Pnr0VShCrlI/fN2Dkdq/ZScb5sO7WwqfOWRmzH5/qTkEF0B5kQ8q/5fUucHJFO 6GhC8hk1dwg0DUi1SLpt9A0lSwiKd6cF5RlPy9WUTF4AUkfC0B+DSNXJsH+OCikb2MTu oMtqdlOeAPQpIvaYDGBqoYriJrcB0VND+Q/lqobRT8i4IqyACjSCrZ4naRJhmKBXsZ7i qZfJIoHR7CvVqsTPuOyN2ijHcA6IxS5B0lcS4q6DsiIeXJOhnf0bc7vry3q9/lOFB4qe fBdg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com X-Gm-Message-State: APjAAAWjbfbI2RkQjJ7IGgJ+/aNRE0uXNYHAGQwLR2EL+KjcjkEIQgXE rkmmaHANpEMntZWA6ks14Iqen/cDJ4Rbn3OV0t/s14ZMfKbCQOwSQSartPZMXbwt+bbKp6dZ70m G3m0w8w3pmJ3f9GPXnQsjobKN08WTTrY6qEtKim1BbsmSld+gn/+A/trfsKl+OdoFTg== X-Received: by 2002:a17:906:5e0d:: with SMTP id n13mr2512778eju.37.1553178077652; Thu, 21 Mar 2019 07:21:17 -0700 (PDT) X-Google-Smtp-Source: APXvYqyIQ6uss0ykqykZjPwbI/OCEOtvyUcSsProXOkc5mF46cZ9mVwjO3mM4Ip1sdmeXZMPiBkZ X-Received: by 2002:a17:906:5e0d:: with SMTP id n13mr2512717eju.37.1553178076417; Thu, 21 Mar 2019 07:21:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553178076; cv=none; d=google.com; s=arc-20160816; b=x6warzKE9+CaYUSlBn3Cz5hr5JeK4K3LW5ShgN5BkjgC4IqMSnZmbRJhFNpRw+hgxN fYHfPpCOYslF5LHT6uLx4fHpHWZ458y7N60QjsxpBdcEAP5pwUEciWzmOzEU9NBLcrtQ +dSpV5N9NDiVZ8EyzAX8aaLAg4fILwfJES/Ev043zQ2PhRVLQg7OZpIwjyi8ayJ1PUT4 OkiNln72bu6FE7C7OskrP4RsMMgs9aF2PIwnFjgK4pIOF1TSiF+WDQfkkCcZLFBUbvVz fF6USk3yxSvnx29f5xleAmdR4s2qZmrVbF1xk7iAOAmqts0vAZCs0VrUQ8FRRxkWADpB 82Yw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=kX4dLJraJo092n4y125DLKk3n4IU7HsLifEATF3quag=; b=sOe7rBVl2LcIw3p9OQLBf3Q+xmE+dXDJK+bjZp+6MxndHRCcgg89O0/Uo8AOrE3cHN gumcJQahRNm++nwiiC9CE/2n2Wmqk0P4+t/Y87whMYMsLSMk+Xv8i3G6zwTsOO+zppjl FZ8LlB0OoTEXc+lj7G4bkJWyRUXDdyKh3/W/u7R101yWNfkocabLmuW2IHRTiln+P5cB yt9m3XVjN/JTfzqaWFvw4VV0pfHDkCGg6Qxc0Q5s0g1kYCuh3tNfOKzDDgMXkfIxeo2f cOjduGmtj9W8RsRNk+KPCwh30TR+fZXjgEq09NaVz/IPWVBHbO0syDZ8uxKqxxEuCLry CSnw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id i8si1860453edj.297.2019.03.21.07.21.16 for ; Thu, 21 Mar 2019 07:21:16 -0700 (PDT) Received-SPF: pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of steven.price@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=steven.price@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6EE6C1A25; Thu, 21 Mar 2019 07:21:15 -0700 (PDT) Received: from e112269-lin.arm.com (e112269-lin.cambridge.arm.com [10.1.196.69]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 345D83F575; Thu, 21 Mar 2019 07:21:12 -0700 (PDT) From: Steven Price To: linux-mm@kvack.org Cc: Steven Price , Andy Lutomirski , Ard Biesheuvel , Arnd Bergmann , Borislav Petkov , Catalin Marinas , Dave Hansen , Ingo Molnar , James Morse , =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , Peter Zijlstra , Thomas Gleixner , Will Deacon , x86@kernel.org, "H. Peter Anvin" , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Rutland , "Liang, Kan" Subject: [PATCH v5 19/19] x86: mm: Convert dump_pagetables to use walk_page_range Date: Thu, 21 Mar 2019 14:19:53 +0000 Message-Id: <20190321141953.31960-20-steven.price@arm.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190321141953.31960-1-steven.price@arm.com> References: <20190321141953.31960-1-steven.price@arm.com> MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Make use of the new functionality in walk_page_range to remove the arch page walking code and use the generic code to walk the page tables. The effective permissions are passed down the chain using new fields in struct pg_state. The KASAN optimisation is implemented by including test_p?d callbacks which can decide to skip an entire tree of entries Signed-off-by: Steven Price --- arch/x86/mm/dump_pagetables.c | 280 ++++++++++++++++++---------------- 1 file changed, 146 insertions(+), 134 deletions(-) diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c index c0fbb9e5a790..f6b814aaddf7 100644 --- a/arch/x86/mm/dump_pagetables.c +++ b/arch/x86/mm/dump_pagetables.c @@ -33,6 +33,10 @@ struct pg_state { int level; pgprot_t current_prot; pgprotval_t effective_prot; + pgprotval_t effective_prot_pgd; + pgprotval_t effective_prot_p4d; + pgprotval_t effective_prot_pud; + pgprotval_t effective_prot_pmd; unsigned long start_address; unsigned long current_address; const struct addr_marker *marker; @@ -356,22 +360,21 @@ static inline pgprotval_t effective_prot(pgprotval_t prot1, pgprotval_t prot2) ((prot1 | prot2) & _PAGE_NX); } -static void walk_pte_level(struct pg_state *st, pmd_t addr, pgprotval_t eff_in, - unsigned long P) +static int ptdump_pte_entry(pte_t *pte, unsigned long addr, + unsigned long next, struct mm_walk *walk) { - int i; - pte_t *pte; - pgprotval_t prot, eff; - - for (i = 0; i < PTRS_PER_PTE; i++) { - st->current_address = normalize_addr(P + i * PTE_LEVEL_MULT); - pte = pte_offset_map(&addr, st->current_address); - prot = pte_flags(*pte); - eff = effective_prot(eff_in, prot); - note_page(st, __pgprot(prot), eff, 5); - pte_unmap(pte); - } + struct pg_state *st = walk->private; + pgprotval_t eff, prot; + + st->current_address = normalize_addr(addr); + + prot = pte_flags(*pte); + eff = effective_prot(st->effective_prot_pmd, prot); + note_page(st, __pgprot(prot), eff, 5); + + return 0; } + #ifdef CONFIG_KASAN /* @@ -400,131 +403,152 @@ static inline bool kasan_page_table(struct pg_state *st, void *pt) } #endif -#if PTRS_PER_PMD > 1 - -static void walk_pmd_level(struct pg_state *st, pud_t addr, - pgprotval_t eff_in, unsigned long P) +static int ptdump_test_pmd(unsigned long addr, unsigned long next, + pmd_t *pmd, struct mm_walk *walk) { - int i; - pmd_t *start, *pmd_start; - pgprotval_t prot, eff; - - pmd_start = start = (pmd_t *)pud_page_vaddr(addr); - for (i = 0; i < PTRS_PER_PMD; i++) { - st->current_address = normalize_addr(P + i * PMD_LEVEL_MULT); - if (!pmd_none(*start)) { - prot = pmd_flags(*start); - eff = effective_prot(eff_in, prot); - if (pmd_large(*start) || !pmd_present(*start)) { - note_page(st, __pgprot(prot), eff, 4); - } else if (!kasan_page_table(st, pmd_start)) { - walk_pte_level(st, *start, eff, - P + i * PMD_LEVEL_MULT); - } - } else - note_page(st, __pgprot(0), 0, 4); - start++; - } + struct pg_state *st = walk->private; + + st->current_address = normalize_addr(addr); + + if (kasan_page_table(st, pmd)) + return 1; + return 0; } -#else -#define walk_pmd_level(s,a,e,p) walk_pte_level(s,__pmd(pud_val(a)),e,p) -#undef pud_large -#define pud_large(a) pmd_large(__pmd(pud_val(a))) -#define pud_none(a) pmd_none(__pmd(pud_val(a))) -#endif +static int ptdump_pmd_entry(pmd_t *pmd, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + struct pg_state *st = walk->private; + pgprotval_t eff, prot; + + prot = pmd_flags(*pmd); + eff = effective_prot(st->effective_prot_pud, prot); + + st->current_address = normalize_addr(addr); + + if (pmd_large(*pmd)) + note_page(st, __pgprot(prot), eff, 4); -#if PTRS_PER_PUD > 1 + st->effective_prot_pmd = eff; -static void walk_pud_level(struct pg_state *st, p4d_t addr, pgprotval_t eff_in, - unsigned long P) + return 0; +} + +static int ptdump_test_pud(unsigned long addr, unsigned long next, + pud_t *pud, struct mm_walk *walk) { - int i; - pud_t *start, *pud_start; - pgprotval_t prot, eff; - - pud_start = start = (pud_t *)p4d_page_vaddr(addr); - - for (i = 0; i < PTRS_PER_PUD; i++) { - st->current_address = normalize_addr(P + i * PUD_LEVEL_MULT); - if (!pud_none(*start)) { - prot = pud_flags(*start); - eff = effective_prot(eff_in, prot); - if (pud_large(*start) || !pud_present(*start)) { - note_page(st, __pgprot(prot), eff, 3); - } else if (!kasan_page_table(st, pud_start)) { - walk_pmd_level(st, *start, eff, - P + i * PUD_LEVEL_MULT); - } - } else - note_page(st, __pgprot(0), 0, 3); + struct pg_state *st = walk->private; - start++; - } + st->current_address = normalize_addr(addr); + + if (kasan_page_table(st, pud)) + return 1; + return 0; } -#else -#define walk_pud_level(s,a,e,p) walk_pmd_level(s,__pud(p4d_val(a)),e,p) -#undef p4d_large -#define p4d_large(a) pud_large(__pud(p4d_val(a))) -#define p4d_none(a) pud_none(__pud(p4d_val(a))) -#endif +static int ptdump_pud_entry(pud_t *pud, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + struct pg_state *st = walk->private; + pgprotval_t eff, prot; + + prot = pud_flags(*pud); + eff = effective_prot(st->effective_prot_p4d, prot); + + st->current_address = normalize_addr(addr); + + if (pud_large(*pud)) + note_page(st, __pgprot(prot), eff, 3); + + st->effective_prot_pud = eff; -static void walk_p4d_level(struct pg_state *st, pgd_t addr, pgprotval_t eff_in, - unsigned long P) + return 0; +} + +static int ptdump_test_p4d(unsigned long addr, unsigned long next, + p4d_t *p4d, struct mm_walk *walk) { - int i; - p4d_t *start, *p4d_start; - pgprotval_t prot, eff; - - if (PTRS_PER_P4D == 1) - return walk_pud_level(st, __p4d(pgd_val(addr)), eff_in, P); - - p4d_start = start = (p4d_t *)pgd_page_vaddr(addr); - - for (i = 0; i < PTRS_PER_P4D; i++) { - st->current_address = normalize_addr(P + i * P4D_LEVEL_MULT); - if (!p4d_none(*start)) { - prot = p4d_flags(*start); - eff = effective_prot(eff_in, prot); - if (p4d_large(*start) || !p4d_present(*start)) { - note_page(st, __pgprot(prot), eff, 2); - } else if (!kasan_page_table(st, p4d_start)) { - walk_pud_level(st, *start, eff, - P + i * P4D_LEVEL_MULT); - } - } else - note_page(st, __pgprot(0), 0, 2); + struct pg_state *st = walk->private; - start++; - } + st->current_address = normalize_addr(addr); + + if (kasan_page_table(st, p4d)) + return 1; + return 0; } -#undef pgd_large -#define pgd_large(a) (pgtable_l5_enabled() ? pgd_large(a) : p4d_large(__p4d(pgd_val(a)))) -#define pgd_none(a) (pgtable_l5_enabled() ? pgd_none(a) : p4d_none(__p4d(pgd_val(a)))) +static int ptdump_p4d_entry(p4d_t *p4d, unsigned long addr, + unsigned long next, struct mm_walk *walk) +{ + struct pg_state *st = walk->private; + pgprotval_t eff, prot; + + prot = p4d_flags(*p4d); + eff = effective_prot(st->effective_prot_pgd, prot); + + st->current_address = normalize_addr(addr); + + if (p4d_large(*p4d)) + note_page(st, __pgprot(prot), eff, 2); + + st->effective_prot_p4d = eff; + + return 0; +} -static inline bool is_hypervisor_range(int idx) +static int ptdump_pgd_entry(pgd_t *pgd, unsigned long addr, + unsigned long next, struct mm_walk *walk) { -#ifdef CONFIG_X86_64 - /* - * A hole in the beginning of kernel address space reserved - * for a hypervisor. - */ - return (idx >= pgd_index(GUARD_HOLE_BASE_ADDR)) && - (idx < pgd_index(GUARD_HOLE_END_ADDR)); + struct pg_state *st = walk->private; + pgprotval_t eff, prot; + + prot = pgd_flags(*pgd); + +#ifdef CONFIG_X86_PAE + eff = _PAGE_USER | _PAGE_RW; #else - return false; + eff = prot; #endif + + st->current_address = normalize_addr(addr); + + if (pgd_large(*pgd)) + note_page(st, __pgprot(prot), eff, 1); + + st->effective_prot_pgd = eff; + + return 0; +} + +static int ptdump_hole(unsigned long addr, unsigned long next, + struct mm_walk *walk) +{ + struct pg_state *st = walk->private; + + st->current_address = normalize_addr(addr); + + note_page(st, __pgprot(0), 0, -1); + + return 0; } static void ptdump_walk_pgd_level_core(struct seq_file *m, struct mm_struct *mm, bool checkwx, bool dmesg) { - pgd_t *start = mm->pgd; - pgprotval_t prot, eff; - int i; struct pg_state st = {}; + struct mm_walk walk = { + .mm = mm, + .pgd_entry = ptdump_pgd_entry, + .p4d_entry = ptdump_p4d_entry, + .pud_entry = ptdump_pud_entry, + .pmd_entry = ptdump_pmd_entry, + .pte_entry = ptdump_pte_entry, + .test_p4d = ptdump_test_p4d, + .test_pud = ptdump_test_pud, + .test_pmd = ptdump_test_pmd, + .pte_hole = ptdump_hole, + .private = &st + }; st.to_dmesg = dmesg; st.check_wx = checkwx; @@ -532,27 +556,15 @@ static void ptdump_walk_pgd_level_core(struct seq_file *m, struct mm_struct *mm, if (checkwx) st.wx_pages = 0; - for (i = 0; i < PTRS_PER_PGD; i++) { - st.current_address = normalize_addr(i * PGD_LEVEL_MULT); - if (!pgd_none(*start) && !is_hypervisor_range(i)) { - prot = pgd_flags(*start); -#ifdef CONFIG_X86_PAE - eff = _PAGE_USER | _PAGE_RW; + down_read(&mm->mmap_sem); +#ifdef CONFIG_X86_64 + walk_page_range(0, PTRS_PER_PGD*PGD_LEVEL_MULT/2, &walk); + walk_page_range(normalize_addr(PTRS_PER_PGD*PGD_LEVEL_MULT/2), ~0, + &walk); #else - eff = prot; + walk_page_range(0, ~0, &walk); #endif - if (pgd_large(*start) || !pgd_present(*start)) { - note_page(&st, __pgprot(prot), eff, 1); - } else { - walk_p4d_level(&st, *start, eff, - i * PGD_LEVEL_MULT); - } - } else - note_page(&st, __pgprot(0), 0, 1); - - cond_resched(); - start++; - } + up_read(&mm->mmap_sem); /* Flush out the last page */ st.current_address = normalize_addr(PTRS_PER_PGD*PGD_LEVEL_MULT);