From patchwork Fri Apr 30 14:55:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12233605 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11B6FC433ED for ; Fri, 30 Apr 2021 14:57:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 665C661409 for ; Fri, 30 Apr 2021 14:57:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 665C661409 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9DACB6B006E; Fri, 30 Apr 2021 10:57:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9B1DC6B0070; Fri, 30 Apr 2021 10:57:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 804656B0071; Fri, 30 Apr 2021 10:57:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0254.hostedemail.com [216.40.44.254]) by kanga.kvack.org (Postfix) with ESMTP id 613566B006E for ; Fri, 30 Apr 2021 10:57:23 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 1951E45C1 for ; Fri, 30 Apr 2021 14:57:23 +0000 (UTC) X-FDA: 78089336766.10.3381EB3 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf02.hostedemail.com (Postfix) with ESMTP id D473A40002E3 for ; Fri, 30 Apr 2021 14:56:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=hBpQR/0D3JnfgszdRdi1IhKjYmrqMJy8CCih3IT3ePo=; b=DnaOMTf2IOjbQYLGQRuwVzpEE7 EHoiZ/VYbSfGzjV9ZlyCoCthz6FNuYmt4jJs6jsl6vmmgI8jFy+4nAzd+V+fiyEoCdu0LQMnv43f7 PmABrkYV5bsRFaxnO83t4GpMpcTygXu4HUpBEjaJwu4z+HXGchQZNe7L/lPldajmc5sYmkzFBt/ll SG8Q0wA4KSzqDobey/Yqe540pHRz4QY8Fn47/z779AD7NBOFg7INb+YSTl+e5/pylne612RfjO8hF oL6kdaZMZD9GXOwhVEkKQxls4Rn3NhvakTH+RSBENdcqjjjHaaINu21UtBOLzhXvw8WlB1f2eCiDp UMNGh+Dw==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1lcUZ1-00BAdP-Ge; Fri, 30 Apr 2021 14:56:26 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, akpm@linux-foundation.org Cc: "Matthew Wilcox (Oracle)" , Christoph Hellwig , David Hildenbrand , Zi Yan Subject: [PATCH 1/8] mm: Optimise nth_page for contiguous memmap Date: Fri, 30 Apr 2021 15:55:42 +0100 Message-Id: <20210430145549.2662354-2-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210430145549.2662354-1-willy@infradead.org> References: <20210430145549.2662354-1-willy@infradead.org> MIME-Version: 1.0 Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=DnaOMTf2; dmarc=none; spf=none (imf02.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Rspamd-Server: rspam03 X-Stat-Signature: scoexcr1bcfaj9ez787spcrcj8gc8dfx X-Rspamd-Queue-Id: D473A40002E3 Received-SPF: none (infradead.org>: No applicable sender policy available) receiver=imf02; identity=mailfrom; envelope-from=""; helo=casper.infradead.org; client-ip=90.155.50.34 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619794611-409441 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If the memmap is virtually contiguous (either because we're using a virtually mapped memmap or because we don't support a discontig memmap at all), then we can implement nth_page() by simple addition. Contrary to popular belief, the compiler is not able to optimise this itself for a vmemmap configuration. This reduces one example user (sg.c) by four instructions: struct page *page = nth_page(rsv_schp->pages[k], offset >> PAGE_SHIFT); before: 49 8b 45 70 mov 0x70(%r13),%rax 48 63 c9 movslq %ecx,%rcx 48 c1 eb 0c shr $0xc,%rbx 48 8b 04 c8 mov (%rax,%rcx,8),%rax 48 2b 05 00 00 00 00 sub 0x0(%rip),%rax R_X86_64_PC32 vmemmap_base-0x4 48 c1 f8 06 sar $0x6,%rax 48 01 d8 add %rbx,%rax 48 c1 e0 06 shl $0x6,%rax 48 03 05 00 00 00 00 add 0x0(%rip),%rax R_X86_64_PC32 vmemmap_base-0x4 after: 49 8b 45 70 mov 0x70(%r13),%rax 48 63 c9 movslq %ecx,%rcx 48 c1 eb 0c shr $0xc,%rbx 48 c1 e3 06 shl $0x6,%rbx 48 03 1c c8 add (%rax,%rcx,8),%rbx Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig Reviewed-by: David Hildenbrand Reviewed-by: Zi Yan --- include/linux/mm.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 25b9041f9925..2327f99b121f 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -234,7 +234,11 @@ int overcommit_policy_handler(struct ctl_table *, int, void *, size_t *, int __add_to_page_cache_locked(struct page *page, struct address_space *mapping, pgoff_t index, gfp_t gfp, void **shadowp); +#if defined(CONFIG_SPARSEMEM) && !defined(CONFIG_SPARSEMEM_VMEMMAP) #define nth_page(page,n) pfn_to_page(page_to_pfn((page)) + (n)) +#else +#define nth_page(page,n) ((page) + (n)) +#endif /* to align the pointer to the (next) page boundary */ #define PAGE_ALIGN(addr) ALIGN(addr, PAGE_SIZE) From patchwork Fri Apr 30 14:55:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12233611 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CC69C433B4 for ; Fri, 30 Apr 2021 14:59:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A6A1961409 for ; Fri, 30 Apr 2021 14:59:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A6A1961409 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 284706B006C; Fri, 30 Apr 2021 10:59:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 25BA36B006E; Fri, 30 Apr 2021 10:59:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 14B0E6B0070; Fri, 30 Apr 2021 10:59:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0174.hostedemail.com [216.40.44.174]) by kanga.kvack.org (Postfix) with ESMTP id EE6D36B006C for ; Fri, 30 Apr 2021 10:59:39 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 3342B4821 for ; Fri, 30 Apr 2021 14:59:39 +0000 (UTC) X-FDA: 78089342478.07.1570A62 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf14.hostedemail.com (Postfix) with ESMTP id 03046C0007F2 for ; Fri, 30 Apr 2021 14:59:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=46uKbRZFXcZtWinMwdAPkEZSqlzYBs3FzrcvKBHNtps=; b=EOB5qy5DBrXMdfJWsa8UWFiFGM Hfqj42uCsZ1vupFU/OT9LzjH7PMJ11d/W06bejDViBT4ovTNSIcHjISD8/34ccOrDAkPHAft/sLgl 747bNVbdk1x739Wu8W6Wghd3ESUdtqIcw24Pw6RcL56oNkscirjMPyc36xk4Xms6xYCFHTdW+j+i4 4SjPS0ect8osbS4NKYHFhcxA7V230E3r1n0qT6W9axp4ejDkTLXPEEN3sNN0Kb2TTq6GQQK5MkZtP 8WUDZEjlfWJAfhBpTMzCQ+Ail8cs/tRm0yC/aq7Vj+vqr5JIN+y+VhPc+itk+T0vCTXBD43ukE3L2 Nq9YJ1uw==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1lcUZa-00BAf0-BU; Fri, 30 Apr 2021 14:57:04 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, akpm@linux-foundation.org Cc: "Matthew Wilcox (Oracle)" , William Kucharski , Vlastimil Babka , Anshuman Khandual Subject: [PATCH 2/8] mm: Make __dump_page static Date: Fri, 30 Apr 2021 15:55:43 +0100 Message-Id: <20210430145549.2662354-3-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210430145549.2662354-1-willy@infradead.org> References: <20210430145549.2662354-1-willy@infradead.org> MIME-Version: 1.0 Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=EOB5qy5D; dmarc=none; spf=none (imf14.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Stat-Signature: e9zm37hj5cq4a1i7g6gurktq5rh8q8wy X-Rspamd-Queue-Id: 03046C0007F2 X-Rspamd-Server: rspam05 Received-SPF: none (infradead.org>: No applicable sender policy available) receiver=imf14; identity=mailfrom; envelope-from=""; helo=casper.infradead.org; client-ip=90.155.50.34 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619794760-264819 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The only caller of __dump_page() now opencodes dump_page(), so remove it as an externally visible symbol. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: William Kucharski Reviewed-by: Vlastimil Babka Reviewed-by: Anshuman Khandual --- include/linux/mmdebug.h | 3 +-- mm/debug.c | 2 +- mm/page_alloc.c | 3 +-- 3 files changed, 3 insertions(+), 5 deletions(-) diff --git a/include/linux/mmdebug.h b/include/linux/mmdebug.h index 5d0767cb424a..1935d4c72d10 100644 --- a/include/linux/mmdebug.h +++ b/include/linux/mmdebug.h @@ -9,8 +9,7 @@ struct page; struct vm_area_struct; struct mm_struct; -extern void dump_page(struct page *page, const char *reason); -extern void __dump_page(struct page *page, const char *reason); +void dump_page(struct page *page, const char *reason); void dump_vma(const struct vm_area_struct *vma); void dump_mm(const struct mm_struct *mm); diff --git a/mm/debug.c b/mm/debug.c index 0bdda8407f71..84cdcd0f7bd3 100644 --- a/mm/debug.c +++ b/mm/debug.c @@ -42,7 +42,7 @@ const struct trace_print_flags vmaflag_names[] = { {0, NULL} }; -void __dump_page(struct page *page, const char *reason) +static void __dump_page(struct page *page, const char *reason) { struct page *head = compound_head(page); struct address_space *mapping; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index a2fe714aed93..f23702e7c564 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -658,8 +658,7 @@ static void bad_page(struct page *page, const char *reason) pr_alert("BUG: Bad page state in process %s pfn:%05lx\n", current->comm, page_to_pfn(page)); - __dump_page(page, reason); - dump_page_owner(page); + dump_page(page, reason); print_modules(); dump_stack(); From patchwork Fri Apr 30 14:55:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12233665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70E53C433B4 for ; Fri, 30 Apr 2021 15:19:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DFC436143D for ; Fri, 30 Apr 2021 15:19:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DFC436143D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5A39A6B006C; Fri, 30 Apr 2021 11:19:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 579CB6B006E; Fri, 30 Apr 2021 11:19:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 443336B0070; Fri, 30 Apr 2021 11:19:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0084.hostedemail.com [216.40.44.84]) by kanga.kvack.org (Postfix) with ESMTP id 270156B006C for ; Fri, 30 Apr 2021 11:19:30 -0400 (EDT) Received: from smtpin36.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id D80DB181AEF3F for ; Fri, 30 Apr 2021 15:19:29 +0000 (UTC) X-FDA: 78089392458.36.0FB9858 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf11.hostedemail.com (Postfix) with ESMTP id 9F2102000385 for ; Fri, 30 Apr 2021 15:19:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=TW6D4gLjvFkVSJeHS9vsTuEGdTwutQJI9gyKTOBGklQ=; b=Gm+T+ZSKq6PFZuKksGAGCnGJkB z9CX8JGPyabWOeqtFptgiDekaVhsYTlKuz09/RcdR5PX9OJb3lOebFNSPegDkIg32HAfqYAA/amLm 0ngPvxjNwJn4UXvwE4nll1ES0ft2ZG+KrO8Qmo/cdDAPemIS8Fy7p2PT36SiB8NXx3vM5Goq8KYmg ue/sqltcJV1/ihtlgDwjKdb4Z/agHOGFHMza1nif9ddYaMzVc9TMWQy0+g50D9T4WWtd/7L1F0JvO RpKBN/sBote10IYQW08C++5dCbGcUIRJ92Q2NiTFC2F2u/pUIkoJ6j5yvjUUxvtL32IG8nzk55/SQ ZR7Qtrzw==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1lcUah-00BAhx-4Z; Fri, 30 Apr 2021 14:58:47 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, akpm@linux-foundation.org Cc: "Matthew Wilcox (Oracle)" , William Kucharski , Vlastimil Babka , Anshuman Khandual Subject: [PATCH 3/8] mm/debug: Factor PagePoisoned out of __dump_page Date: Fri, 30 Apr 2021 15:55:44 +0100 Message-Id: <20210430145549.2662354-4-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210430145549.2662354-1-willy@infradead.org> References: <20210430145549.2662354-1-willy@infradead.org> MIME-Version: 1.0 Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Gm+T+ZSK; dmarc=none; spf=none (imf11.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Rspamd-Server: rspam03 X-Stat-Signature: 3ssgyu9karnx68agyxxgarx58pemybfg X-Rspamd-Queue-Id: 9F2102000385 Received-SPF: none (infradead.org>: No applicable sender policy available) receiver=imf11; identity=mailfrom; envelope-from=""; helo=casper.infradead.org; client-ip=90.155.50.34 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619795953-672920 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Move the PagePoisoned test into dump_page(). Skip the hex print for poisoned pages -- we know they're full of ffffffff. Move the reason printing from __dump_page() to dump_page(). Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: William Kucharski Reviewed-by: Vlastimil Babka Reviewed-by: Anshuman Khandual --- mm/debug.c | 25 +++++++------------------ 1 file changed, 7 insertions(+), 18 deletions(-) diff --git a/mm/debug.c b/mm/debug.c index 84cdcd0f7bd3..e73fe0a8ec3d 100644 --- a/mm/debug.c +++ b/mm/debug.c @@ -42,11 +42,10 @@ const struct trace_print_flags vmaflag_names[] = { {0, NULL} }; -static void __dump_page(struct page *page, const char *reason) +static void __dump_page(struct page *page) { struct page *head = compound_head(page); struct address_space *mapping; - bool page_poisoned = PagePoisoned(page); bool compound = PageCompound(page); /* * Accessing the pageblock without the zone lock. It could change to @@ -58,16 +57,6 @@ static void __dump_page(struct page *page, const char *reason) int mapcount; char *type = ""; - /* - * If struct page is poisoned don't access Page*() functions as that - * leads to recursive loop. Page*() check for poisoned pages, and calls - * dump_page() when detected. - */ - if (page_poisoned) { - pr_warn("page:%px is uninitialized and poisoned", page); - goto hex_only; - } - if (page < head || (page >= head + MAX_ORDER_NR_PAGES)) { /* * Corrupt page, so we cannot call page_mapping. Instead, do a @@ -173,8 +162,6 @@ static void __dump_page(struct page *page, const char *reason) pr_warn("%sflags: %#lx(%pGp)%s\n", type, head->flags, &head->flags, page_cma ? " CMA" : ""); - -hex_only: print_hex_dump(KERN_WARNING, "raw: ", DUMP_PREFIX_NONE, 32, sizeof(unsigned long), page, sizeof(struct page), false); @@ -182,14 +169,16 @@ static void __dump_page(struct page *page, const char *reason) print_hex_dump(KERN_WARNING, "head: ", DUMP_PREFIX_NONE, 32, sizeof(unsigned long), head, sizeof(struct page), false); - - if (reason) - pr_warn("page dumped because: %s\n", reason); } void dump_page(struct page *page, const char *reason) { - __dump_page(page, reason); + if (PagePoisoned(page)) + pr_warn("page:%p is uninitialized and poisoned", page); + else + __dump_page(page); + if (reason) + pr_warn("page dumped because: %s\n", reason); dump_page_owner(page); } EXPORT_SYMBOL(dump_page); From patchwork Fri Apr 30 14:55:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12233623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 293A9C43460 for ; Fri, 30 Apr 2021 15:02:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 92AC76143A for ; Fri, 30 Apr 2021 15:02:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 92AC76143A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8C2B26B006C; Fri, 30 Apr 2021 11:02:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8996F6B006E; Fri, 30 Apr 2021 11:02:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7750C6B0070; Fri, 30 Apr 2021 11:02:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0234.hostedemail.com [216.40.44.234]) by kanga.kvack.org (Postfix) with ESMTP id 5B93E6B006C for ; Fri, 30 Apr 2021 11:02:56 -0400 (EDT) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 09BBA181AEF39 for ; Fri, 30 Apr 2021 15:02:56 +0000 (UTC) X-FDA: 78089350752.05.E203896 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf27.hostedemail.com (Postfix) with ESMTP id 66D28801A835 for ; Fri, 30 Apr 2021 15:02:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=fyQDBoLRjVEIc34oTX/DhVq3Ck4tbF5ZYnx/FCHbDo8=; b=fszMhBpTACAkkQgZG4tyh5FsCF uVa0gtuljHPnTZ17Ib69X8Vm0LznNs2hH+biVMUGjHW09gf/PchBxjc1V4aiAa6mul/T+x6GJvveu umtuTMHDKxYErM9ciHDEQDm4Blu7NTZpk3xIthuZctmiX47yuIKk/25jmoX7W0HA4txsE4XqohCe2 dU3n+gejT6VHYMI60Sl8hj51zuJWJLh+lpwSXnwdrJnQA5WOinCzTmQ4CyQHSPcIkCZV5NpmtYnKB dBjVBeGG3tw2n1oeYOd3Qft5IdTjNJfYTsbfX97+5yK4TNJCLn0sZOAc0xmKqsakyD6k9pkhm9RRv ahrDUAbg==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1lcUdL-00BAw1-8Q; Fri, 30 Apr 2021 15:01:05 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, akpm@linux-foundation.org Cc: "Matthew Wilcox (Oracle)" , William Kucharski , Vlastimil Babka , Anshuman Khandual Subject: [PATCH 4/8] mm/page_owner: Constify dump_page_owner Date: Fri, 30 Apr 2021 15:55:45 +0100 Message-Id: <20210430145549.2662354-5-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210430145549.2662354-1-willy@infradead.org> References: <20210430145549.2662354-1-willy@infradead.org> MIME-Version: 1.0 Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=fszMhBpT; spf=none (imf27.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 66D28801A835 X-Stat-Signature: ca7ffkzeujgbjb9n9kr8qx63698knfdt Received-SPF: none (infradead.org>: No applicable sender policy available) receiver=imf27; identity=mailfrom; envelope-from=""; helo=casper.infradead.org; client-ip=90.155.50.34 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619794950-450535 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: dump_page_owner() only uses struct page to find the page_ext, and lookup_page_ext() already takes a const argument. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: William Kucharski Reviewed-by: Vlastimil Babka Reviewed-by: Anshuman Khandual --- include/linux/page_owner.h | 6 +++--- mm/page_owner.c | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/include/linux/page_owner.h b/include/linux/page_owner.h index 3468794f83d2..719bfe5108c5 100644 --- a/include/linux/page_owner.h +++ b/include/linux/page_owner.h @@ -14,7 +14,7 @@ extern void __set_page_owner(struct page *page, extern void __split_page_owner(struct page *page, unsigned int nr); extern void __copy_page_owner(struct page *oldpage, struct page *newpage); extern void __set_page_owner_migrate_reason(struct page *page, int reason); -extern void __dump_page_owner(struct page *page); +extern void __dump_page_owner(const struct page *page); extern void pagetypeinfo_showmixedcount_print(struct seq_file *m, pg_data_t *pgdat, struct zone *zone); @@ -46,7 +46,7 @@ static inline void set_page_owner_migrate_reason(struct page *page, int reason) if (static_branch_unlikely(&page_owner_inited)) __set_page_owner_migrate_reason(page, reason); } -static inline void dump_page_owner(struct page *page) +static inline void dump_page_owner(const struct page *page) { if (static_branch_unlikely(&page_owner_inited)) __dump_page_owner(page); @@ -69,7 +69,7 @@ static inline void copy_page_owner(struct page *oldpage, struct page *newpage) static inline void set_page_owner_migrate_reason(struct page *page, int reason) { } -static inline void dump_page_owner(struct page *page) +static inline void dump_page_owner(const struct page *page) { } #endif /* CONFIG_PAGE_OWNER */ diff --git a/mm/page_owner.c b/mm/page_owner.c index adfabb560eb9..f51a57e92aa3 100644 --- a/mm/page_owner.c +++ b/mm/page_owner.c @@ -392,7 +392,7 @@ print_page_owner(char __user *buf, size_t count, unsigned long pfn, return -ENOMEM; } -void __dump_page_owner(struct page *page) +void __dump_page_owner(const struct page *page) { struct page_ext *page_ext = lookup_page_ext(page); struct page_owner *page_owner; From patchwork Fri Apr 30 14:55:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12233625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9CC1C433B4 for ; Fri, 30 Apr 2021 15:04:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0E81961409 for ; Fri, 30 Apr 2021 15:04:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0E81961409 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 89A936B006C; Fri, 30 Apr 2021 11:04:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 84A056B006E; Fri, 30 Apr 2021 11:04:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 713086B0070; Fri, 30 Apr 2021 11:04:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0009.hostedemail.com [216.40.44.9]) by kanga.kvack.org (Postfix) with ESMTP id 507396B006C for ; Fri, 30 Apr 2021 11:04:02 -0400 (EDT) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 0CECE485A for ; Fri, 30 Apr 2021 15:04:02 +0000 (UTC) X-FDA: 78089353524.02.7490C22 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf05.hostedemail.com (Postfix) with ESMTP id 7FC7AE005F11 for ; Fri, 30 Apr 2021 15:03:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=lOmqEPnS7Hw2fLegrIjeF+PaasHglD3GyP6d0HUc9GQ=; b=CsSgrvoY1Chojmg4apEUjTeFyV jcSU8yhuHSGNZQB8SlqPojoAg78sGXhlsa6UD2fxyb0rmtQBNeEm2Vg6Uv5DLevvjs7IxBjCMvBgz H02+A28vy6QIN9e93lrg1V9yTjJEhsmHsILCQhAIdJksaCYT8WQrklDc4DqMiu2l+XPcyoJe2+Kzu /ML1zjlk4MtAzJ8JXCloFc8sOmN45vhp8EGOuiuEjWuOtDd6CmzVcsvyFCJmclW8jHUDs/+i5G+VS NCItqz0H7s4Pv87DmYqOWBmBalraS9FflB3gyd2rYVJW2XS8M0WxcmD0lhktAnoHHoI+pJakXT3hu 33G87KJw==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1lcUeR-00BB4J-7a; Fri, 30 Apr 2021 15:02:36 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, akpm@linux-foundation.org Cc: "Matthew Wilcox (Oracle)" , William Kucharski , Vlastimil Babka , Anshuman Khandual Subject: [PATCH 5/8] mm: Make compound_head const-preserving Date: Fri, 30 Apr 2021 15:55:46 +0100 Message-Id: <20210430145549.2662354-6-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210430145549.2662354-1-willy@infradead.org> References: <20210430145549.2662354-1-willy@infradead.org> MIME-Version: 1.0 X-Stat-Signature: sxisnbiwgm4xazcnj1b9ipjrwhpeczo5 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 7FC7AE005F11 Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=CsSgrvoY; spf=none (imf05.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none Received-SPF: none (infradead.org>: No applicable sender policy available) receiver=imf05; identity=mailfrom; envelope-from=""; helo=casper.infradead.org; client-ip=90.155.50.34 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619795037-499338 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: If you pass a const pointer to compound_head(), you get a const pointer back; if you pass a mutable pointer, you get a mutable pointer back. Also remove an unnecessary forward definition of struct page; we're about to dereference page->compound_head, so it must already have been defined. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: William Kucharski Reviewed-by: Vlastimil Babka Reviewed-by: Anshuman Khandual --- include/linux/page-flags.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 04a34c08e0a6..d8e26243db25 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -177,17 +177,17 @@ enum pageflags { #ifndef __GENERATING_BOUNDS_H -struct page; /* forward declaration */ - -static inline struct page *compound_head(struct page *page) +static inline unsigned long _compound_head(const struct page *page) { unsigned long head = READ_ONCE(page->compound_head); if (unlikely(head & 1)) - return (struct page *) (head - 1); - return page; + return head - 1; + return (unsigned long)page; } +#define compound_head(page) ((typeof(page))_compound_head(page)) + static __always_inline int PageTail(struct page *page) { return READ_ONCE(page->compound_head) & 1; From patchwork Fri Apr 30 14:55:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12233627 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC04EC433ED for ; Fri, 30 Apr 2021 15:04:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4E5BF61165 for ; Fri, 30 Apr 2021 15:04:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4E5BF61165 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C91B18D0002; Fri, 30 Apr 2021 11:04:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C68656B0070; Fri, 30 Apr 2021 11:04:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B33318D0002; Fri, 30 Apr 2021 11:04:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0042.hostedemail.com [216.40.44.42]) by kanga.kvack.org (Postfix) with ESMTP id 94EDA6B006E for ; Fri, 30 Apr 2021 11:04:42 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 50EB1180ACF1F for ; Fri, 30 Apr 2021 15:04:42 +0000 (UTC) X-FDA: 78089355204.21.8898590 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf08.hostedemail.com (Postfix) with ESMTP id C0D1680192F3 for ; Fri, 30 Apr 2021 15:04:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=ANmp9Os/LHR/T8LANBKOkml2dwgYRoABl1lrJ/0d2NQ=; b=CUBUN6oI0fK7XGKP0dLAM1UhYA iJF1+2Hem8T8ezJX6Ihu5zuA7mValTrJUAPCUXNWLbKLPSPnnelcDLtX5CAJ/llyX5mVTmPzJOuoq DRJc6oynkDxwasrGDllNuhL4NG+tLam6OIIzEqes/85yn1v8E04IJjkviJKthYy1zRnR2Bv4k+bw7 SpnrXON1V4LvIdaA7V+xbF5Q663ztFhl/UIQ7y3yruf/Tv+ZVYwefW9l7bwPxfp8iA5wlFxP/5I4c zhsKTXMR89WIGoHDItM1XVK8sjFZXE0H5v1qy0Hg4POzkDSXLWw5umN/IcYbZi/q8J2chS9ojOFfv LoUog4Bg==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1lcUfe-00BBB4-Tv; Fri, 30 Apr 2021 15:03:36 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, akpm@linux-foundation.org Cc: "Matthew Wilcox (Oracle)" , William Kucharski , Vlastimil Babka , Anshuman Khandual Subject: [PATCH 6/8] mm: Constify get_pfnblock_flags_mask and get_pfnblock_migratetype Date: Fri, 30 Apr 2021 15:55:47 +0100 Message-Id: <20210430145549.2662354-7-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210430145549.2662354-1-willy@infradead.org> References: <20210430145549.2662354-1-willy@infradead.org> MIME-Version: 1.0 Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=CUBUN6oI; spf=none (imf08.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: C0D1680192F3 X-Stat-Signature: 4qkxs8s331rtkh15zq3b96ebdfkug1sk Received-SPF: none (infradead.org>: No applicable sender policy available) receiver=imf08; identity=mailfrom; envelope-from=""; helo=casper.infradead.org; client-ip=90.155.50.34 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619795058-201211 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The struct page is not modified by these routines, so it can be marked const. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: William Kucharski Reviewed-by: Vlastimil Babka Reviewed-by: Anshuman Khandual --- include/linux/pageblock-flags.h | 2 +- mm/page_alloc.c | 13 +++++++------ 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/include/linux/pageblock-flags.h b/include/linux/pageblock-flags.h index fff52ad370c1..973fd731a520 100644 --- a/include/linux/pageblock-flags.h +++ b/include/linux/pageblock-flags.h @@ -54,7 +54,7 @@ extern unsigned int pageblock_order; /* Forward declaration */ struct page; -unsigned long get_pfnblock_flags_mask(struct page *page, +unsigned long get_pfnblock_flags_mask(const struct page *page, unsigned long pfn, unsigned long mask); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index f23702e7c564..5a1e5b624594 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -474,7 +474,7 @@ static inline bool defer_init(int nid, unsigned long pfn, unsigned long end_pfn) #endif /* Return a pointer to the bitmap storing bits affecting a block of pages */ -static inline unsigned long *get_pageblock_bitmap(struct page *page, +static inline unsigned long *get_pageblock_bitmap(const struct page *page, unsigned long pfn) { #ifdef CONFIG_SPARSEMEM @@ -484,7 +484,7 @@ static inline unsigned long *get_pageblock_bitmap(struct page *page, #endif /* CONFIG_SPARSEMEM */ } -static inline int pfn_to_bitidx(struct page *page, unsigned long pfn) +static inline int pfn_to_bitidx(const struct page *page, unsigned long pfn) { #ifdef CONFIG_SPARSEMEM pfn &= (PAGES_PER_SECTION-1); @@ -495,7 +495,7 @@ static inline int pfn_to_bitidx(struct page *page, unsigned long pfn) } static __always_inline -unsigned long __get_pfnblock_flags_mask(struct page *page, +unsigned long __get_pfnblock_flags_mask(const struct page *page, unsigned long pfn, unsigned long mask) { @@ -520,13 +520,14 @@ unsigned long __get_pfnblock_flags_mask(struct page *page, * * Return: pageblock_bits flags */ -unsigned long get_pfnblock_flags_mask(struct page *page, unsigned long pfn, - unsigned long mask) +unsigned long get_pfnblock_flags_mask(const struct page *page, + unsigned long pfn, unsigned long mask) { return __get_pfnblock_flags_mask(page, pfn, mask); } -static __always_inline int get_pfnblock_migratetype(struct page *page, unsigned long pfn) +static __always_inline int get_pfnblock_migratetype(const struct page *page, + unsigned long pfn) { return __get_pfnblock_flags_mask(page, pfn, MIGRATETYPE_MASK); } From patchwork Fri Apr 30 14:55:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12233629 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D127C433B4 for ; Fri, 30 Apr 2021 15:06:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E735B6142A for ; Fri, 30 Apr 2021 15:06:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E735B6142A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 59DEF6B006C; Fri, 30 Apr 2021 11:06:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 561D56B006E; Fri, 30 Apr 2021 11:06:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 42A0B6B0070; Fri, 30 Apr 2021 11:06:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0110.hostedemail.com [216.40.44.110]) by kanga.kvack.org (Postfix) with ESMTP id 27CDA6B006C for ; Fri, 30 Apr 2021 11:06:05 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id D5BD312E7 for ; Fri, 30 Apr 2021 15:06:04 +0000 (UTC) X-FDA: 78089358648.27.90255DC Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf26.hostedemail.com (Postfix) with ESMTP id DA57540002D7 for ; Fri, 30 Apr 2021 15:05:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=27fvFe0ryVXLSHC2zpPkvk9Wh4j4iIi9y7tCdwHH8bI=; b=bCgiIdKZfp8hXijO5K4VjhQ28z tTQSVyYOncTCLiLNpnXnX0OzkI2qxbG1JPGV+qDLPEc4LrJ6cOdEK94+EsWsz5JWlsrqLj3XNZvT9 mzlI0+Gy6xogrBpja/O3M0wCbnYTtJctrJeVjI3Puocw4WDH5aCeNFM0B3xiN8QODLE/TrU31165p J5zFbfQ4BB9ssEVO5AIx00ppKIMBoKzUTfYTA4Go7oVEDyBqixdxB+LX0o6dhyFuy5DnyhG0bUPZB Qyb31kio9TMyimzvJwWzTc+XKZXd0SZktx4Z5i1H/ztGpLDCBYXrzDDs5u/9eAkHLDtENd+FKkpmF byJqO0iQ==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1lcUgc-00BBFh-KU; Fri, 30 Apr 2021 15:04:40 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, akpm@linux-foundation.org Cc: "Matthew Wilcox (Oracle)" , William Kucharski , Vlastimil Babka , Anshuman Khandual Subject: [PATCH 7/8] mm: Constify page_count and page_ref_count Date: Fri, 30 Apr 2021 15:55:48 +0100 Message-Id: <20210430145549.2662354-8-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210430145549.2662354-1-willy@infradead.org> References: <20210430145549.2662354-1-willy@infradead.org> MIME-Version: 1.0 X-Stat-Signature: y3zhprjcs3ic7cpi6h5dzsosbkcnf8yp X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: DA57540002D7 Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=bCgiIdKZ; spf=none (imf26.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none Received-SPF: none (infradead.org>: No applicable sender policy available) receiver=imf26; identity=mailfrom; envelope-from=""; helo=casper.infradead.org; client-ip=90.155.50.34 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619795154-154367 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now that compound_head() accepts a const struct page pointer, these two functions can be marked as not modifying the page pointer they are passed. Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: William Kucharski Reviewed-by: Vlastimil Babka Reviewed-by: Anshuman Khandual --- include/linux/page_ref.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/page_ref.h b/include/linux/page_ref.h index f3318f34fc54..7ad46f45df39 100644 --- a/include/linux/page_ref.h +++ b/include/linux/page_ref.h @@ -62,12 +62,12 @@ static inline void __page_ref_unfreeze(struct page *page, int v) #endif -static inline int page_ref_count(struct page *page) +static inline int page_ref_count(const struct page *page) { return atomic_read(&page->_refcount); } -static inline int page_count(struct page *page) +static inline int page_count(const struct page *page) { return atomic_read(&compound_head(page)->_refcount); } From patchwork Fri Apr 30 14:55:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Matthew Wilcox X-Patchwork-Id: 12233631 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22209C433ED for ; Fri, 30 Apr 2021 15:06:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 96A7761185 for ; Fri, 30 Apr 2021 15:06:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 96A7761185 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=infradead.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1E0A36B006E; Fri, 30 Apr 2021 11:06:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1B8C58D0002; Fri, 30 Apr 2021 11:06:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 00B636B0071; Fri, 30 Apr 2021 11:06:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0035.hostedemail.com [216.40.44.35]) by kanga.kvack.org (Postfix) with ESMTP id D85626B006E for ; Fri, 30 Apr 2021 11:06:33 -0400 (EDT) Received: from smtpin34.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 988963636 for ; Fri, 30 Apr 2021 15:06:33 +0000 (UTC) X-FDA: 78089359866.34.04583A4 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf29.hostedemail.com (Postfix) with ESMTP id BD79B3DB for ; Fri, 30 Apr 2021 15:06:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=IgD6Z/R8KKTP/Q+2YwElx93aowViVL+rNNVE6FqBr6U=; b=kfRDms8usgMzMruu4mkIOcuFZm Dmc0Wi4xZEH7EQ1XC3k9RM9/PH9CQadh0FYJ9FrGHn2fQVjN0BL2c017tAUQWm9225qJgYDGwEg1Y R7u4nab6+rjOSTS1eyPDqAtyrHJ4rajm40DKed6bTKYrcTzW18CM1ahBUZcuutrycdY+En7lbBaSc pdL25ZE1bimMpnRq9g9q1QqNtaYBd5Dzcv+Uijc6M0uitTFl7U1GOXcCEiB668797EH7nQMc5jCES S/dcjL6cR1JguflSP3JNYDLu35Jv6jy68LlSnNnl/o/SgLL+L0zWbEVeIkBEdKZLTibLCbGiy3mOF K2TS9Qrg==; Received: from willy by casper.infradead.org with local (Exim 4.94 #2 (Red Hat Linux)) id 1lcUho-00BBPM-Jk; Fri, 30 Apr 2021 15:05:48 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, akpm@linux-foundation.org Cc: "Matthew Wilcox (Oracle)" , Ilias Apalodimas , Jesper Dangaard Brouer Subject: [PATCH 8/8] mm: Fix struct page layout on 32-bit systems Date: Fri, 30 Apr 2021 15:55:49 +0100 Message-Id: <20210430145549.2662354-9-willy@infradead.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210430145549.2662354-1-willy@infradead.org> References: <20210430145549.2662354-1-willy@infradead.org> MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: BD79B3DB Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=kfRDms8u; dmarc=none; spf=none (imf29.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org X-Stat-Signature: jt6acow4iikscioicfxs7txz1rgqbhhp Received-SPF: none (infradead.org>: No applicable sender policy available) receiver=imf29; identity=mailfrom; envelope-from=""; helo=casper.infradead.org; client-ip=90.155.50.34 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619795188-273653 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: 32-bit architectures which expect 8-byte alignment for 8-byte integers and need 64-bit DMA addresses (arm, mips, ppc) had their struct page inadvertently expanded in 2019. When the dma_addr_t was added, it forced the alignment of the union to 8 bytes, which inserted a 4 byte gap between 'flags' and the union. Fix this by storing the dma_addr_t in one or two adjacent unsigned longs. This restores the alignment to that of an unsigned long. We always store the low bits in the first word to prevent the PageTail bit from being inadvertently set on a big endian platform. If that happened, get_user_pages_fast() racing against a page which was freed and reallocated to the page_pool could dereference a bogus compound_head(), which would be hard to trace back to this cause. Fixes: c25fff7171be ("mm: add dma_addr_t to struct page") Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Ilias Apalodimas Acked-by: Jesper Dangaard Brouer --- include/linux/mm_types.h | 4 ++-- include/net/page_pool.h | 12 +++++++++++- net/core/page_pool.c | 12 +++++++----- 3 files changed, 20 insertions(+), 8 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 6613b26a8894..5aacc1c10a45 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -97,10 +97,10 @@ struct page { }; struct { /* page_pool used by netstack */ /** - * @dma_addr: might require a 64-bit value even on + * @dma_addr: might require a 64-bit value on * 32-bit architectures. */ - dma_addr_t dma_addr; + unsigned long dma_addr[2]; }; struct { /* slab, slob and slub */ union { diff --git a/include/net/page_pool.h b/include/net/page_pool.h index 6d517a37c18b..b4b6de909c93 100644 --- a/include/net/page_pool.h +++ b/include/net/page_pool.h @@ -198,7 +198,17 @@ static inline void page_pool_recycle_direct(struct page_pool *pool, static inline dma_addr_t page_pool_get_dma_addr(struct page *page) { - return page->dma_addr; + dma_addr_t ret = page->dma_addr[0]; + if (sizeof(dma_addr_t) > sizeof(unsigned long)) + ret |= (dma_addr_t)page->dma_addr[1] << 16 << 16; + return ret; +} + +static inline void page_pool_set_dma_addr(struct page *page, dma_addr_t addr) +{ + page->dma_addr[0] = addr; + if (sizeof(dma_addr_t) > sizeof(unsigned long)) + page->dma_addr[1] = upper_32_bits(addr); } static inline bool is_page_pool_compiled_in(void) diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 9ec1aa9640ad..3c4c4c7a0402 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -174,8 +174,10 @@ static void page_pool_dma_sync_for_device(struct page_pool *pool, struct page *page, unsigned int dma_sync_size) { + dma_addr_t dma_addr = page_pool_get_dma_addr(page); + dma_sync_size = min(dma_sync_size, pool->p.max_len); - dma_sync_single_range_for_device(pool->p.dev, page->dma_addr, + dma_sync_single_range_for_device(pool->p.dev, dma_addr, pool->p.offset, dma_sync_size, pool->p.dma_dir); } @@ -195,7 +197,7 @@ static bool page_pool_dma_map(struct page_pool *pool, struct page *page) if (dma_mapping_error(pool->p.dev, dma)) return false; - page->dma_addr = dma; + page_pool_set_dma_addr(page, dma); if (pool->p.flags & PP_FLAG_DMA_SYNC_DEV) page_pool_dma_sync_for_device(pool, page, pool->p.max_len); @@ -331,13 +333,13 @@ void page_pool_release_page(struct page_pool *pool, struct page *page) */ goto skip_dma_unmap; - dma = page->dma_addr; + dma = page_pool_get_dma_addr(page); - /* When page is unmapped, it cannot be returned our pool */ + /* When page is unmapped, it cannot be returned to our pool */ dma_unmap_page_attrs(pool->p.dev, dma, PAGE_SIZE << pool->p.order, pool->p.dma_dir, DMA_ATTR_SKIP_CPU_SYNC); - page->dma_addr = 0; + page_pool_set_dma_addr(page, 0); skip_dma_unmap: /* This may be the last page returned, releasing the pool, so * it is not safe to reference pool afterwards.