From patchwork Wed Feb 24 20:00:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102415 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 131C6C433DB for ; Wed, 24 Feb 2021 20:00:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9DCFB64F08 for ; Wed, 24 Feb 2021 20:00:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9DCFB64F08 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2F15E6B0005; Wed, 24 Feb 2021 15:00:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 278FB6B0006; Wed, 24 Feb 2021 15:00:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 141A26B006C; Wed, 24 Feb 2021 15:00:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0202.hostedemail.com [216.40.44.202]) by kanga.kvack.org (Postfix) with ESMTP id EE4866B0005 for ; Wed, 24 Feb 2021 15:00:14 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id AA9DE18503B45 for ; Wed, 24 Feb 2021 20:00:14 +0000 (UTC) X-FDA: 77854227948.24.552AC5F Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id 6D6FC2BD0 for ; Wed, 24 Feb 2021 20:00:08 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id DE4F164EDD; Wed, 24 Feb 2021 20:00:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196813; bh=KhBZu/x9Y9iz441us0rEONmOakIrn2jhUmCr6tbzEwk=; h=Date:From:To:Subject:In-Reply-To:From; b=pCs3RvokqPXI35YcZF0w7hlyz154yeLcXyV4mkg4u7uJWKRacBykegJidH5PRKthJ DYEgzabiw5RyPkzxXS8NmJNqsjEl9VAC/yrCtNf7Ai2NHB4enVyBt4nHPySQe8pwfO 6hzzckTkojwj1VpLVO1lZWn2NUe+6K8Hnw2RYqSg= Date: Wed, 24 Feb 2021 12:00:12 -0800 From: Andrew Morton To: akpm@linux-foundation.org, bcain@codeaurora.org, gregkh@linuxfoundation.org, keescook@chromium.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, rdunlap@infradead.org, torvalds@linux-foundation.org Subject: [patch 001/173] hexagon: remove CONFIG_EXPERIMENTAL from defconfigs Message-ID: <20210224200012.M4mVg_RSL%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 6D6FC2BD0 X-Stat-Signature: zauohz5ajk5h8hgiyb5wpjdaxtcqifrn Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf12; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196808-453005 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Randy Dunlap Subject: hexagon: remove CONFIG_EXPERIMENTAL from defconfigs Since CONFIG_EXPERIMENTAL was removed in 2013, go ahead and drop it from any defconfig files. Link: https://lkml.kernel.org/r/20210115010011.29483-1-rdunlap@infradead.org Fixes: 3d374d09f16f ("final removal of CONFIG_EXPERIMENTAL") Signed-off-by: Randy Dunlap Cc: Kees Cook Cc: Greg Kroah-Hartman Cc: Brian Cain Signed-off-by: Andrew Morton --- arch/hexagon/configs/comet_defconfig | 1 - 1 file changed, 1 deletion(-) --- a/arch/hexagon/configs/comet_defconfig~hexagon-remove-config_experimental-from-defconfigs +++ a/arch/hexagon/configs/comet_defconfig @@ -1,7 +1,6 @@ CONFIG_SMP=y CONFIG_DEFAULT_MMAP_MIN_ADDR=0 CONFIG_HZ_100=y -CONFIG_EXPERIMENTAL=y CONFIG_CROSS_COMPILE="hexagon-" CONFIG_LOCALVERSION="-smp" # CONFIG_LOCALVERSION_AUTO is not set From patchwork Wed Feb 24 20:00:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102417 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DEC53C433E0 for ; Wed, 24 Feb 2021 20:00:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 81F2264E2B for ; Wed, 24 Feb 2021 20:00:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 81F2264E2B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 142EB6B0006; Wed, 24 Feb 2021 15:00:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0CE096B006C; Wed, 24 Feb 2021 15:00:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EFDC56B0070; Wed, 24 Feb 2021 15:00:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0150.hostedemail.com [216.40.44.150]) by kanga.kvack.org (Postfix) with ESMTP id D5DF06B0006 for ; Wed, 24 Feb 2021 15:00:17 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id A2706F01E for ; Wed, 24 Feb 2021 20:00:17 +0000 (UTC) X-FDA: 77854228074.29.86B18CB Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf16.hostedemail.com (Postfix) with ESMTP id 88839801A827 for ; Wed, 24 Feb 2021 20:00:16 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id EE66E64F0B; Wed, 24 Feb 2021 20:00:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196816; bh=AyrAnzFlz+24Nks6FyCV8kwiw//Kf6WeFWFFanccBP4=; h=Date:From:To:Subject:In-Reply-To:From; b=nCOyC3XVxwOuIp80Ad25iFmkolnWltKVIWa/Y8AuEH+RbzYFPMMl4AX2s67ohzZ9/ 38ickzpqcSjcZxqaGmGY8CXfi/dTZTYhGWT5eGGUb1Uh1ElNpgIK7tJSxFkx8zCCuJ oYJg5sQyFxX5H82NdITV7FK8+tKH3OKn1YPvR040= Date: Wed, 24 Feb 2021 12:00:15 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, tangchunyou@yulong.com, torvalds@linux-foundation.org, zhangwen@yulong.com Subject: [patch 002/173] scripts/spelling.txt: increase error-prone spell checking Message-ID: <20210224200015.HbMroA039%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 88839801A827 X-Stat-Signature: j9uxuyfi4bx4wet5zadwx6ca7btex6ew Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196816-24438 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: tangchunyou Subject: scripts/spelling.txt: increase error-prone spell checking Increase maping spelling error check. Link: https://lkml.kernel.org/r/20210121092125.2663-1-tangchunyou@163.com Signed-off-by: WenZhang Signed-off-by: Andrew Morton --- scripts/spelling.txt | 1 + 1 file changed, 1 insertion(+) --- a/scripts/spelling.txt~scripts-spellingtxt-increase-error-prone-spell-checking +++ a/scripts/spelling.txt @@ -875,6 +875,7 @@ manger||manager manoeuvering||maneuvering manufaucturing||manufacturing mappping||mapping +maping||mapping matchs||matches mathimatical||mathematical mathimatic||mathematic From patchwork Wed Feb 24 20:00:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102419 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A663FC433E0 for ; Wed, 24 Feb 2021 20:00:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 25EBD64DA3 for ; Wed, 24 Feb 2021 20:00:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 25EBD64DA3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AB81D6B006C; Wed, 24 Feb 2021 15:00:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A410C6B0070; Wed, 24 Feb 2021 15:00:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 92F9E6B0071; Wed, 24 Feb 2021 15:00:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0155.hostedemail.com [216.40.44.155]) by kanga.kvack.org (Postfix) with ESMTP id 7C6EA6B006C for ; Wed, 24 Feb 2021 15:00:21 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 4321E943A for ; Wed, 24 Feb 2021 20:00:21 +0000 (UTC) X-FDA: 77854228242.06.DF57557 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf21.hostedemail.com (Postfix) with ESMTP id E3F3CE005F08 for ; Wed, 24 Feb 2021 20:00:16 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 087EF64E2B; Wed, 24 Feb 2021 20:00:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196819; bh=ReO0lxwlyE3MFnpfHhj39/n4lOBBeambyL5c9OIfQKs=; h=Date:From:To:Subject:In-Reply-To:From; b=2rPDiNNS6upRZK69P78UDFiLeAfjlQsDpyUFlgazsFAyrTQ8pqfEomSh9IkOJm2fm 6bpELYUPJrQwogszTjeMvrhlgrXSwZQLuesbi5poSRDUcCgTeqAW63MERq/ssbwu2b kptMdv0V5LBbRICPlxt3s0tDLJScVdrHoAsvp6j8= Date: Wed, 24 Feb 2021 12:00:18 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, zuoqilin@yulong.com Subject: [patch 003/173] scripts/spelling.txt: check for "exeeds" Message-ID: <20210224200018.kFGvkPr92%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: E3F3CE005F08 X-Stat-Signature: zchy4797xdus1g16z84mxzoeggz9rqt7 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf21; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196816-770521 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: zuoqilin Subject: scripts/spelling.txt: check for "exeeds" Increase exeeds spelling error check. Link: https://lkml.kernel.org/r/20210127060049.915-1-zuoqilin1@163.com Signed-off-by: zuoqilin Signed-off-by: Andrew Morton --- scripts/spelling.txt | 1 + 1 file changed, 1 insertion(+) --- a/scripts/spelling.txt~scripts-spellingtxt-increase-error-prone-spell-checking-2 +++ a/scripts/spelling.txt @@ -566,6 +566,7 @@ estbalishment||establishment etsablishment||establishment etsbalishment||establishment evalution||evaluation +exeeds||exceeds excecutable||executable exceded||exceeded exceds||exceeds From patchwork Wed Feb 24 20:00:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102421 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3755FC433E0 for ; Wed, 24 Feb 2021 20:00:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C555F64DA3 for ; Wed, 24 Feb 2021 20:00:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C555F64DA3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 53A3D6B0070; Wed, 24 Feb 2021 15:00:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4EA546B0071; Wed, 24 Feb 2021 15:00:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 38B2A6B0072; Wed, 24 Feb 2021 15:00:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0249.hostedemail.com [216.40.44.249]) by kanga.kvack.org (Postfix) with ESMTP id 2362E6B0070 for ; Wed, 24 Feb 2021 15:00:24 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id DBF718083CD3 for ; Wed, 24 Feb 2021 20:00:23 +0000 (UTC) X-FDA: 77854228326.14.D679C28 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf10.hostedemail.com (Postfix) with ESMTP id E5C944080F47 for ; Wed, 24 Feb 2021 20:00:17 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 063FD64E6C; Wed, 24 Feb 2021 20:00:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196822; bh=I2VSNTV4crk2ImvnlU5ZAPKjEZLebQlg7USVoOG/bqg=; h=Date:From:To:Subject:In-Reply-To:From; b=a2MF/nE2n0D8kythNJpBB1YI6zavFccFrQtck9FHCuoNMnNV6R93syjJK1wnPneXx O3PRopLHkPlZ+W/xmfTZYjqECqmRYdSXlLltLVBRS3jknjlnX1qItE44uPNI1myd5v OP8zA8MKOkwqreTb9AxbVnbcDF8LWCBxNCZIL5hY= Date: Wed, 24 Feb 2021 12:00:21 -0800 From: Andrew Morton To: akpm@linux-foundation.org, dingsenjie@yulong.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 004/173] scripts/spelling.txt: add "allocted" and "exeeds" typo Message-ID: <20210224200021.P8tnAV7L0%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: E5C944080F47 X-Stat-Signature: ef6794x3hrq9d9e7q967dd44ap6dtip3 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf10; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196817-598967 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: dingsenjie Subject: scripts/spelling.txt: add "allocted" and "exeeds" typo Increase "allocted" and "exeeds" spelling error check. Link: https://lkml.kernel.org/r/20210127081919.1928-1-dingsenjie@163.com Signed-off-by: dingsenjie Signed-off-by: Andrew Morton --- scripts/spelling.txt | 2 ++ 1 file changed, 2 insertions(+) --- a/scripts/spelling.txt~scripts-spellingtxt-add-allocted-and-exeeds-typo +++ a/scripts/spelling.txt @@ -103,6 +103,7 @@ alloated||allocated allocatote||allocate allocatrd||allocated allocte||allocate +allocted||allocated allpication||application alocate||allocate alogirhtms||algorithms @@ -575,6 +576,7 @@ excellant||excellent execeeded||exceeded execeeds||exceeds exeed||exceed +exeeds||exceeds exeuction||execution existance||existence existant||existent From patchwork Wed Feb 24 20:00:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102423 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AC93C433DB for ; Wed, 24 Feb 2021 20:00:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E58DF64DA3 for ; Wed, 24 Feb 2021 20:00:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E58DF64DA3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6C1176B0071; Wed, 24 Feb 2021 15:00:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 672486B0072; Wed, 24 Feb 2021 15:00:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 539426B0073; Wed, 24 Feb 2021 15:00:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0177.hostedemail.com [216.40.44.177]) by kanga.kvack.org (Postfix) with ESMTP id 3848F6B0071 for ; Wed, 24 Feb 2021 15:00:27 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id EE1A868BF for ; Wed, 24 Feb 2021 20:00:26 +0000 (UTC) X-FDA: 77854228452.23.392657D Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf13.hostedemail.com (Postfix) with ESMTP id EEBB2E007A44 for ; Wed, 24 Feb 2021 20:00:23 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 24FA064EFA; Wed, 24 Feb 2021 20:00:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196825; bh=XM1rSyHaqGeVb6D5Rj22HMP0r//ZU/7lsoBhkKMOenc=; h=Date:From:To:Subject:In-Reply-To:From; b=vFINtJm0OCESWltA4qIZccRLvptA894/DTsaRyupeez1Bdx2GLR+9T3G03xDV65pW 3pfGnw+UF02YBn1K232fUjCvrNynR/8z7xzgkWolJK0H/PNrZXIAZOPG/G4tI66DfV CfAgDpyvppcV+Nqc+9/GZUYOJqFlku0wPZ4IBXqU= Date: Wed, 24 Feb 2021 12:00:24 -0800 From: Andrew Morton To: akpm@linux-foundation.org, colin.king@canonical.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 005/173] scripts/spelling.txt: add more spellings to spelling.txt Message-ID: <20210224200024.mBnciPmnu%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: EEBB2E007A44 X-Stat-Signature: 7ch7rkiip9bbdgk8xysupceh64i6hno5 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf13; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196823-610783 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Colin Ian King Subject: scripts/spelling.txt: add more spellings to spelling.txt Here are some of the more common spelling mistakes and typos that I've found while fixing up spelling mistakes in the kernel since September 2020 Link: https://lkml.kernel.org/r/20210210124318.55082-1-colin.king@canonical.com Signed-off-by: Colin Ian King Signed-off-by: Andrew Morton --- scripts/spelling.txt | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) --- a/scripts/spelling.txt~scripts-spellingtxt-add-more-spellings-to-spellingtxt +++ a/scripts/spelling.txt @@ -340,6 +340,7 @@ comppatible||compatible compres||compress compresion||compression comression||compression +comunicate||communicate comunication||communication conbination||combination conditionaly||conditionally @@ -467,6 +468,7 @@ developpment||development deveolpment||development devided||divided deviece||device +devision||division diable||disable dicline||decline dictionnary||dictionary @@ -480,6 +482,7 @@ difinition||definition digial||digital dimention||dimension dimesions||dimensions +diconnected||disconnected disgest||digest dispalying||displaying diplay||display @@ -519,6 +522,7 @@ downlads||downloads droped||dropped droput||dropout druing||during +dyanmic||dynamic dynmaic||dynamic eanable||enable eanble||enable @@ -543,6 +547,7 @@ encrupted||encrypted encrypiton||encryption encryptio||encryption endianess||endianness +enpoint||endpoint enhaced||enhanced enlightnment||enlightenment enqueing||enqueuing @@ -644,6 +649,7 @@ forwardig||forwarding frambuffer||framebuffer framming||framing framwork||framework +frequence||frequency frequncy||frequency frequancy||frequency frome||from @@ -686,10 +692,12 @@ handfull||handful hanlde||handle hanled||handled happend||happened +hardare||hardware harware||hardware havind||having heirarchically||hierarchically helpfull||helpful +heterogenous||heterogeneous hexdecimal||hexadecimal hybernate||hibernate hierachy||hierarchy @@ -734,6 +742,7 @@ inconsistant||inconsistent increas||increase incremeted||incremented incrment||increment +incuding||including inculde||include indendation||indentation indended||intended @@ -744,6 +753,7 @@ indiate||indicate indicat||indicate inexpect||inexpected inferface||interface +infinit||infinite infomation||information informatiom||information informations||information @@ -774,6 +784,7 @@ instace||instance instal||install instanciate||instantiate instanciated||instantiated +instuments||instruments insufficent||insufficient inteface||interface integreated||integrated @@ -872,6 +883,7 @@ mailformed||malformed malplaced||misplaced malplace||misplace managable||manageable +managament||management managment||management mangement||management manger||manager @@ -890,6 +902,7 @@ meetign||meeting memeory||memory memmber||member memoery||memory +memroy||memory ment||meant mergable||mergeable mesage||message @@ -1003,6 +1016,7 @@ overlaping||overlapping overide||override overrided||overridden overriden||overridden +overrrun||overrun overun||overrun overwritting||overwriting overwriten||overwritten @@ -1039,6 +1053,7 @@ peforming||performing peice||piece pendantic||pedantic peprocessor||preprocessor +perfomance||performance perfoming||performing perfomring||performing periperal||peripheral @@ -1104,6 +1119,7 @@ prodecure||procedure progamming||programming progams||programs progess||progress +programable||programmable programers||programmers programm||program programms||programs @@ -1148,6 +1164,7 @@ recieved||received recieve||receive reciever||receiver recieves||receives +recieving||receiving recogniced||recognised recognizeable||recognizable recommanded||recommended @@ -1251,6 +1268,7 @@ searchs||searches secquence||sequence secund||second segement||segment +seleted||selected semaphone||semaphore senario||scenario senarios||scenarios @@ -1267,6 +1285,7 @@ seqeunce||sequence seqeuncer||sequencer seqeuencer||sequencer sequece||sequence +sequemce||sequence sequencial||sequential serivce||service serveral||several @@ -1337,6 +1356,7 @@ suble||subtle substract||subtract submited||submitted submition||submission +succeded||succeeded suceed||succeed succesfully||successfully succesful||successful @@ -1357,6 +1377,7 @@ supportin||supporting suppoted||supported suppported||supported suppport||support +supprot||support supress||suppress surpressed||suppressed surpresses||suppresses @@ -1405,6 +1426,7 @@ thresold||threshold throught||through trackling||tracking troughput||throughput +trys||tries thses||these tiggers||triggers tiggered||triggered @@ -1418,7 +1440,9 @@ traking||tracking tramsmitted||transmitted tramsmit||transmit tranasction||transaction +tranceiver||transceiver tranfer||transfer +tranmission||transmission transcevier||transceiver transciever||transceiver transferd||transferred @@ -1472,6 +1496,7 @@ unnecesary||unnecessary unneedingly||unnecessarily unnsupported||unsupported unmached||unmatched +unprecise||imprecise unregester||unregister unresgister||unregister unrgesiter||unregister @@ -1507,6 +1532,7 @@ varient||variant vaule||value verbse||verbose veify||verify +veriosn||version verisons||versions verison||version verson||version From patchwork Wed Feb 24 20:00:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102425 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E00D5C433E0 for ; Wed, 24 Feb 2021 20:00:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8CC2764DA3 for ; Wed, 24 Feb 2021 20:00:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8CC2764DA3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1B8746B0072; Wed, 24 Feb 2021 15:00:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 148BE6B0073; Wed, 24 Feb 2021 15:00:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F02A16B0074; Wed, 24 Feb 2021 15:00:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0166.hostedemail.com [216.40.44.166]) by kanga.kvack.org (Postfix) with ESMTP id D8B906B0072 for ; Wed, 24 Feb 2021 15:00:29 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A80B48083CCC for ; Wed, 24 Feb 2021 20:00:29 +0000 (UTC) X-FDA: 77854228578.17.7CB53F5 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP id 559A09001444 for ; Wed, 24 Feb 2021 20:00:25 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 35DEE64F07; Wed, 24 Feb 2021 20:00:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196828; bh=/9aIYVJWZbcNr6F7gti99XLsAT0JIhwvvWlZzB7xumM=; h=Date:From:To:Subject:In-Reply-To:From; b=jmn1jr8f4YtbG7IS3/0a1F/oDTF6wmZ0Dc1WWtrAyvjnMvjh40A0zK/Sb0hI/mhKs 0/DmM4avcA0M1eCOy0gkHmWlDkpK88ReXvZ0WO+CNvG+GJKGVjjSGNhNHr9/7iQzKI sy27DvpXpDcOpgbrqngLg5Kh7DHz1p8M4WmV9mG0= Date: Wed, 24 Feb 2021 12:00:27 -0800 From: Andrew Morton To: akpm@linux-foundation.org, anton@tuxera.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rdunlap@infradead.org, torvalds@linux-foundation.org Subject: [patch 006/173] ntfs: layout.h: delete duplicated words Message-ID: <20210224200027.a9TRFzIfq%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 559A09001444 X-Stat-Signature: 4mcrj63wxq7g1powzkhtmeyqhnps6o3b Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf19; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196825-387978 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000168, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Randy Dunlap Subject: ntfs: layout.h: delete duplicated words Drop the repeated words "the" and "in" in comments. Link: https://lkml.kernel.org/r/20210125194937.24627-1-rdunlap@infradead.org Signed-off-by: Randy Dunlap Acked-by: Anton Altaparmakov Signed-off-by: Andrew Morton --- fs/ntfs/layout.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/fs/ntfs/layout.h~ntfs-layouth-delete-duplicated-words +++ a/fs/ntfs/layout.h @@ -703,7 +703,7 @@ typedef struct { /* 14*/ le16 instance; /* The instance of this attribute record. This number is unique within this mft record (see MFT_RECORD/next_attribute_instance notes in - in mft.h for more details). */ + mft.h for more details). */ /* 16*/ union { /* Resident attributes. */ struct { @@ -1838,7 +1838,7 @@ typedef struct { * Also, each security descriptor is stored twice in the $SDS stream with a * fixed offset of 0x40000 bytes (256kib, the Windows cache manager's max size) * between them; i.e. if a SDS_ENTRY specifies an offset of 0x51d0, then the - * the first copy of the security descriptor will be at offset 0x51d0 in the + * first copy of the security descriptor will be at offset 0x51d0 in the * $SDS data stream and the second copy will be at offset 0x451d0. */ typedef struct { From patchwork Wed Feb 24 20:00:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102429 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4B2DC433E0 for ; Wed, 24 Feb 2021 20:00:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6E11C64F0D for ; Wed, 24 Feb 2021 20:00:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6E11C64F0D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id ED9DD6B0074; Wed, 24 Feb 2021 15:00:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E38496B0075; Wed, 24 Feb 2021 15:00:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB34E6B0078; Wed, 24 Feb 2021 15:00:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0080.hostedemail.com [216.40.44.80]) by kanga.kvack.org (Postfix) with ESMTP id B47BC6B0074 for ; Wed, 24 Feb 2021 15:00:38 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 7B4C3184E6C9E for ; Wed, 24 Feb 2021 20:00:38 +0000 (UTC) X-FDA: 77854228956.14.791A212 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP id F0D7FC0042CC for ; Wed, 24 Feb 2021 20:00:34 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B13FF64F08; Wed, 24 Feb 2021 20:00:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196832; bh=lgDhecVyjLcC8CX2QsFWfn7GWsa4VUqLSi3Jc9aifws=; h=Date:From:To:Subject:In-Reply-To:From; b=HsgIJOJKi0RhtDiBvgO9IlUsy/qvs9dild4oSVsfCTPAAOkapyl1h/kh+1MMfNE+M EoC44MGoL6+qCb3P/PqQvMjB72z1oKdmARXOJiCZyWEmw27s0FrRxD7VRPFvmoE6G1 LixUS8uPgKHGqBvR5nXfHploTd+IRFqiqf//Y5S8= Date: Wed, 24 Feb 2021 12:00:30 -0800 From: Andrew Morton To: akpm@linux-foundation.org, anton@tuxera.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rkovhaev@gmail.com, stable@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 007/173] ntfs: check for valid standard information attribute Message-ID: <20210224200030.F8x-RJnAx%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: eprichfui8befcqpd5h7dacahptg3eet X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: F0D7FC0042CC Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf03; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196834-996526 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Rustam Kovhaev Subject: ntfs: check for valid standard information attribute Mounting a corrupted filesystem with NTFS resulted in a kernel crash. We should check for valid STANDARD_INFORMATION attribute offset and length before trying to access it Link: https://lkml.kernel.org/r/20210217155930.1506815-1-rkovhaev@gmail.com Link: https://syzkaller.appspot.com/bug?extid=c584225dabdea2f71969 Signed-off-by: Rustam Kovhaev Reported-by: syzbot+c584225dabdea2f71969@syzkaller.appspotmail.com Tested-by: syzbot+c584225dabdea2f71969@syzkaller.appspotmail.com Acked-by: Anton Altaparmakov Cc: Signed-off-by: Andrew Morton --- fs/ntfs/inode.c | 6 ++++++ 1 file changed, 6 insertions(+) --- a/fs/ntfs/inode.c~ntfs-check-for-valid-standard-information-attribute +++ a/fs/ntfs/inode.c @@ -629,6 +629,12 @@ static int ntfs_read_locked_inode(struct } a = ctx->attr; /* Get the standard information attribute value. */ + if ((u8 *)a + le16_to_cpu(a->data.resident.value_offset) + + le32_to_cpu(a->data.resident.value_length) > + (u8 *)ctx->mrec + vol->mft_record_size) { + ntfs_error(vi->i_sb, "Corrupt standard information attribute in inode."); + goto unm_err_out; + } si = (STANDARD_INFORMATION*)((u8*)a + le16_to_cpu(a->data.resident.value_offset)); From patchwork Wed Feb 24 20:00:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102427 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A158C433DB for ; Wed, 24 Feb 2021 20:00:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BA6E964F09 for ; Wed, 24 Feb 2021 20:00:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BA6E964F09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5606A6B0073; Wed, 24 Feb 2021 15:00:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E8CE6B0074; Wed, 24 Feb 2021 15:00:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3FED36B0075; Wed, 24 Feb 2021 15:00:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0158.hostedemail.com [216.40.44.158]) by kanga.kvack.org (Postfix) with ESMTP id 25CC66B0073 for ; Wed, 24 Feb 2021 15:00:37 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id DD06118503B49 for ; Wed, 24 Feb 2021 20:00:36 +0000 (UTC) X-FDA: 77854228872.23.0FE38E6 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id A19293745 for ; Wed, 24 Feb 2021 20:00:30 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 085E364EF1; Wed, 24 Feb 2021 20:00:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196835; bh=+o1F19wGgMUvmhOnRYeoH2ndZPYN3vtY3RMcAPJB/ek=; h=Date:From:To:Subject:In-Reply-To:From; b=dLVBJ9JiZtF0RFcNnQtHYTCKh6DDNA5l3/fJ1AA4cqgDkeDW86xcbKSI3f27NX0WL O7C7WBIByqnzBF278BMMQTFyaaSGaDWrN8aOVKyxA0B4VgUT+sgSOxQeR056WK1WK6 nZueeUMS0+WCX8TJqdVWG3YeA+fsgItOphhjjx/s= Date: Wed, 24 Feb 2021 12:00:34 -0800 From: Andrew Morton To: akpm@linux-foundation.org, gechangwei@live.cn, ghe@suse.com, jlbec@evilplan.org, joseph.qi@linux.alibaba.com, junxiao.bi@oracle.com, linux-mm@kvack.org, mark@fasheh.com, mm-commits@vger.kernel.org, piaojun@huawei.com, torvalds@linux-foundation.org, yili@winhong.com Subject: [patch 008/173] ocfs2: remove redundant conditional before iput Message-ID: <20210224200034.HBgw5B2Gy%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: frtij9jq1wk7c8w38yoq3hk41woaa7du X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: A19293745 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf12; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196830-436403 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yi Li Subject: ocfs2: remove redundant conditional before iput iput handles NULL pointers gracefully, so there's no need to check the pointer before the call. Link: https://lkml.kernel.org/r/20201231040535.4091761-1-yili@winhong.com Signed-off-by: Yi Li Acked-by: Joseph Qi Cc: Mark Fasheh Cc: Joel Becker Cc: Junxiao Bi Cc: Changwei Ge Cc: Gang He Cc: Jun Piao Signed-off-by: Andrew Morton --- fs/ocfs2/super.c | 2 -- 1 file changed, 2 deletions(-) --- a/fs/ocfs2/super.c~ocfs2-remove-redundant-conditional-before-iput +++ a/fs/ocfs2/super.c @@ -973,8 +973,6 @@ static void ocfs2_disable_quotas(struct * quota files */ dquot_disable(sb, type, DQUOT_USAGE_ENABLED | DQUOT_LIMITS_ENABLED); - if (!inode) - continue; iput(inode); } } From patchwork Wed Feb 24 20:00:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102431 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC531C433E0 for ; Wed, 24 Feb 2021 20:00:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 44B7164F0F for ; Wed, 24 Feb 2021 20:00:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 44B7164F0F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CB4766B0075; Wed, 24 Feb 2021 15:00:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C639D6B0078; Wed, 24 Feb 2021 15:00:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B2ED36B007B; Wed, 24 Feb 2021 15:00:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0122.hostedemail.com [216.40.44.122]) by kanga.kvack.org (Postfix) with ESMTP id 944B56B0075 for ; Wed, 24 Feb 2021 15:00:40 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 5F43F1808F55C for ; Wed, 24 Feb 2021 20:00:40 +0000 (UTC) X-FDA: 77854229040.14.696C776 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf06.hostedemail.com (Postfix) with ESMTP id 7DC27C0007D3 for ; Wed, 24 Feb 2021 20:00:41 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 99CE564F09; Wed, 24 Feb 2021 20:00:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196839; bh=5SLsoieTj/NszhC297QywX3bjC/l54Nq9Xzmhrozaow=; h=Date:From:To:Subject:In-Reply-To:From; b=JuDP6vPl2Jyv3SNNgZOdzS8lel2aURWY84bNZkYZoCHh4cV+6SHevexaly5Wgtx67 Idgg/ugUj4iM/Wxi/5/abeApXBuAN+nxf5rV56/jKVhsjSch6RBShgRDSoMfDAckBo ENngPcu2lNd6dGI3IFQGwt3UJZKELyl43IoOXIJM= Date: Wed, 24 Feb 2021 12:00:38 -0800 From: Andrew Morton To: akpm@linux-foundation.org, gechangwei@live.cn, ghe@suse.com, guozh88@chinatelecom.cn, jlbec@evilplan.org, joseph.qi@linux.alibaba.com, junxiao.bi@oracle.com, linux-mm@kvack.org, mark@fasheh.com, mm-commits@vger.kernel.org, piaojun@huawei.com, torvalds@linux-foundation.org Subject: [patch 009/173] ocfs2: clean up some definitions which are not used any more Message-ID: <20210224200038.8DNzWkTSw%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: nc5gb4wd9do1odw5uouowqtfi6pr1xy6 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 7DC27C0007D3 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf06; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196841-37912 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: guozh Subject: ocfs2: clean up some definitions which are not used any more There are some definitions which is not used anymore in OCFS2 module, so as to be removed. Link: https://lkml.kernel.org/r/2021011916182284700534@chinatelecom.cn Signed-off-by: Guozhonghua Acked-by: Joseph Qi Cc: Mark Fasheh Cc: Joel Becker Cc: Junxiao Bi Cc: Changwei Ge Cc: Gang He Cc: Jun Piao Signed-off-by: Andrew Morton --- fs/ocfs2/dlm/dlmast.c | 10 ---------- fs/ocfs2/dlm/dlmcommon.h | 4 ---- 2 files changed, 14 deletions(-) --- a/fs/ocfs2/dlm/dlmast.c~ocfs2-cleanup-some-definitions-which-is-not-used-anymore +++ a/fs/ocfs2/dlm/dlmast.c @@ -165,16 +165,6 @@ void __dlm_queue_bast(struct dlm_ctxt *d spin_unlock(&lock->spinlock); } -void dlm_queue_bast(struct dlm_ctxt *dlm, struct dlm_lock *lock) -{ - BUG_ON(!dlm); - BUG_ON(!lock); - - spin_lock(&dlm->ast_lock); - __dlm_queue_bast(dlm, lock); - spin_unlock(&dlm->ast_lock); -} - static void dlm_update_lvb(struct dlm_ctxt *dlm, struct dlm_lock_resource *res, struct dlm_lock *lock) { --- a/fs/ocfs2/dlm/dlmcommon.h~ocfs2-cleanup-some-definitions-which-is-not-used-anymore +++ a/fs/ocfs2/dlm/dlmcommon.h @@ -17,10 +17,7 @@ #define DLM_LOCKID_NAME_MAX 32 -#define DLM_DOMAIN_NAME_MAX_LEN 255 #define DLM_LOCK_RES_OWNER_UNKNOWN O2NM_MAX_NODES -#define DLM_THREAD_SHUFFLE_INTERVAL 5 // flush everything every 5 passes -#define DLM_THREAD_MS 200 // flush at least every 200 ms #define DLM_HASH_SIZE_DEFAULT (1 << 17) #if DLM_HASH_SIZE_DEFAULT < PAGE_SIZE @@ -902,7 +899,6 @@ void __dlm_lockres_grab_inflight_worker( struct dlm_lock_resource *res); void dlm_queue_ast(struct dlm_ctxt *dlm, struct dlm_lock *lock); -void dlm_queue_bast(struct dlm_ctxt *dlm, struct dlm_lock *lock); void __dlm_queue_ast(struct dlm_ctxt *dlm, struct dlm_lock *lock); void __dlm_queue_bast(struct dlm_ctxt *dlm, struct dlm_lock *lock); void dlm_do_local_ast(struct dlm_ctxt *dlm, From patchwork Wed Feb 24 20:00:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61D62C433DB for ; Wed, 24 Feb 2021 20:00:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 063AF64F13 for ; Wed, 24 Feb 2021 20:00:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 063AF64F13 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9F3DE6B0078; Wed, 24 Feb 2021 15:00:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9A3B96B007B; Wed, 24 Feb 2021 15:00:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8BB246B007D; Wed, 24 Feb 2021 15:00:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0212.hostedemail.com [216.40.44.212]) by kanga.kvack.org (Postfix) with ESMTP id 72ADA6B0078 for ; Wed, 24 Feb 2021 15:00:44 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 33E8A8083CDB for ; Wed, 24 Feb 2021 20:00:44 +0000 (UTC) X-FDA: 77854229208.29.359276D Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf13.hostedemail.com (Postfix) with ESMTP id 13006E005F12 for ; Wed, 24 Feb 2021 20:00:40 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 12C5664F0D; Wed, 24 Feb 2021 20:00:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196842; bh=tFLvzF/05wfF40cbbsEzhmNPBeTsfhCRhMW9E521+Z8=; h=Date:From:To:Subject:In-Reply-To:From; b=A2v4M5TiFJyLKT8uyHrcGl7hfTClrl6u1PwGWkqcRH77yZMxJoNWBrnO+PMnfEtiy P+8EmSzX2HGyOBwhA4E7K3I/eexUM2c0B3/RPf1BRkQyxV/KHZPwEkr2Xka7fuitzA D27QhMjWawJKB2Ev82MOnpKyJi/34TfVXpQwsGOo= Date: Wed, 24 Feb 2021 12:00:41 -0800 From: Andrew Morton To: akpm@linux-foundation.org, dan.carpenter@oracle.com, gechangwei@live.cn, ghe@suse.com, jlbec@evilplan.org, joseph.qi@linux.alibaba.com, junxiao.bi@oracle.com, linux-mm@kvack.org, mark@fasheh.com, mm-commits@vger.kernel.org, piaojun@huawei.com, torvalds@linux-foundation.org Subject: [patch 010/173] ocfs2: fix a use after free on error Message-ID: <20210224200041.4n4v6wlIZ%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: zn8uite7intsg7tjc918umycee8j8s46 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 13006E005F12 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf13; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196840-668473 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Dan Carpenter Subject: ocfs2: fix a use after free on error The error handling in this function frees "reg" but it is still on the "o2hb_all_regions" list so it will lead to a use after freew. Joseph Qi points out that we need to clear the bit in the "o2hb_region_bitmap" as well Link: https://lkml.kernel.org/r/YBk4M6HUG8jB/jc7@mwanda Fixes: 1cf257f51191 ("ocfs2: fix memory leak") Signed-off-by: Dan Carpenter Reviewed-by: Joseph Qi Cc: Mark Fasheh Cc: Joel Becker Cc: Junxiao Bi Cc: Changwei Ge Cc: Gang He Cc: Jun Piao Signed-off-by: Andrew Morton --- fs/ocfs2/cluster/heartbeat.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) --- a/fs/ocfs2/cluster/heartbeat.c~ocfs2-fix-a-use-after-free-on-error +++ a/fs/ocfs2/cluster/heartbeat.c @@ -2042,7 +2042,7 @@ static struct config_item *o2hb_heartbea o2hb_nego_timeout_handler, reg, NULL, ®->hr_handler_list); if (ret) - goto free; + goto remove_item; ret = o2net_register_handler(O2HB_NEGO_APPROVE_MSG, reg->hr_key, sizeof(struct o2hb_nego_msg), @@ -2057,6 +2057,12 @@ static struct config_item *o2hb_heartbea unregister_handler: o2net_unregister_handler_list(®->hr_handler_list); +remove_item: + spin_lock(&o2hb_live_lock); + list_del(®->hr_all_item); + if (o2hb_global_heartbeat_active()) + clear_bit(reg->hr_region_num, o2hb_region_bitmap); + spin_unlock(&o2hb_live_lock); free: kfree(reg); return ERR_PTR(ret); From patchwork Wed Feb 24 20:00:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102435 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81ADBC433DB for ; Wed, 24 Feb 2021 20:00:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 235A264F1A for ; Wed, 24 Feb 2021 20:00:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 235A264F1A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AB7C56B007B; Wed, 24 Feb 2021 15:00:47 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A66636B007D; Wed, 24 Feb 2021 15:00:47 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 958696B007E; Wed, 24 Feb 2021 15:00:47 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0121.hostedemail.com [216.40.44.121]) by kanga.kvack.org (Postfix) with ESMTP id 7BAE06B007B for ; Wed, 24 Feb 2021 15:00:47 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 4A244180428E4 for ; Wed, 24 Feb 2021 20:00:47 +0000 (UTC) X-FDA: 77854229334.24.8AD6F42 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf02.hostedemail.com (Postfix) with ESMTP id 8710E40F8C3B for ; Wed, 24 Feb 2021 20:00:33 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8733764F13; Wed, 24 Feb 2021 20:00:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196846; bh=bPZ87oWzslRoFI8kO/hepKZB4OjSgr0BiEmXwDLybUs=; h=Date:From:To:Subject:In-Reply-To:From; b=eyYoNo8cVNdOYefNCQNJ64lgDM69Awt2l+J4a+j9gUvgdeaia32Xk4xprlNfiY94E FJQTM1Ndwyy0tHaSjou9gMYhLBxM6hwZ3Aac4rOUDOlTIRmFF5S0w0m69bWUUmI24M fRJ/4ZTtvJ9bhtqr0mVUtKZ3Ppic631zyTD3m9AY= Date: Wed, 24 Feb 2021 12:00:45 -0800 From: Andrew Morton To: abaci@linux.alibaba.com, akpm@linux-foundation.org, gechangwei@live.cn, ghe@suse.com, jiapeng.chong@linux.alibaba.com, jlbec@evilplan.org, joseph.qi@linux.alibaba.com, junxiao.bi@oracle.com, linux-mm@kvack.org, mark@fasheh.com, mm-commits@vger.kernel.org, piaojun@huawei.com, torvalds@linux-foundation.org Subject: [patch 011/173] ocfs2: simplify the calculation of variables Message-ID: <20210224200045.tBGNb2xJQ%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 8710E40F8C3B X-Stat-Signature: 313my4bf6zs5dgzsx3wpisp1ng6jxbp6 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf02; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196833-66628 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jiapeng Chong Subject: ocfs2: simplify the calculation of variables Fix the following coccicheck warnings: ./fs/ocfs2/refcounttree.c:981:16-18: WARNING !A || A && B is equivalent to !A || B. Link: https://lkml.kernel.org/r/1612235424-80367-1-git-send-email-jiapeng.chong@linux.alibaba.com Signed-off-by: Jiapeng Chong Reported-by: Abaci Robot Reviewed-by: Joseph Qi Cc: Mark Fasheh Cc: Joel Becker Cc: Junxiao Bi Cc: Changwei Ge Cc: Gang He Cc: Jun Piao Signed-off-by: Andrew Morton --- fs/ocfs2/refcounttree.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/fs/ocfs2/refcounttree.c~ocfs2-simplify-the-calculation-of-variables +++ a/fs/ocfs2/refcounttree.c @@ -978,7 +978,7 @@ static int ocfs2_get_refcount_cpos_end(s return 0; } - if (!eb || (eb && !eb->h_next_leaf_blk)) { + if (!eb || !eb->h_next_leaf_blk) { /* * We are the last extent rec, so any high cpos should * be stored in this leaf refcount block. From patchwork Wed Feb 24 20:00:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5697DC433E0 for ; Wed, 24 Feb 2021 20:00:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D1D7764F06 for ; Wed, 24 Feb 2021 20:00:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D1D7764F06 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 64BC46B007D; Wed, 24 Feb 2021 15:00:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D3E46B007E; Wed, 24 Feb 2021 15:00:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4C1D76B0080; Wed, 24 Feb 2021 15:00:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0112.hostedemail.com [216.40.44.112]) by kanga.kvack.org (Postfix) with ESMTP id 363B06B007D for ; Wed, 24 Feb 2021 15:00:51 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id E63A0184B5914 for ; Wed, 24 Feb 2021 20:00:50 +0000 (UTC) X-FDA: 77854229460.09.A225DFC Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf15.hostedemail.com (Postfix) with ESMTP id 3BBF4A004CEC for ; Wed, 24 Feb 2021 20:00:49 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0608D64F18; Wed, 24 Feb 2021 20:00:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196849; bh=89S/3DP/jXqKVG70oQMz27dwJmzmZuhNqPwHhZvwuFk=; h=Date:From:To:Subject:In-Reply-To:From; b=ZnHn4RkM4NwMBpCs3LCs++zBAIGErFQcHgb0oZBhOtxOWnRJX/nAs2KzjiTcsjtYa 57Gyyf4VeruoVfjlHIhS+vjLDnozO7T3v0D6aSt+beRpVHPQdylub1F63dqAFG3kXb WClN09sTMRS02HiKr+j/aEs1ZU5uG88UuanLsVso= Date: Wed, 24 Feb 2021 12:00:48 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, rdunlap@infradead.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, willy@infradead.org Subject: [patch 012/173] fs: delete repeated words in comments Message-ID: <20210224200048.4ZoIcq1Pf%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: swg15n1mxz9o4xk5zfzg9iybxnxbwifx X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 3BBF4A004CEC Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf15; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196849-792305 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Randy Dunlap Subject: fs: delete repeated words in comments Delete duplicate words in fs/*.c. The doubled words that are being dropped are: that, be, the, in, and, for Link: https://lkml.kernel.org/r/20201224052810.25315-1-rdunlap@infradead.org Signed-off-by: Randy Dunlap Reviewed-by: Matthew Wilcox (Oracle) Cc: Alexander Viro Signed-off-by: Andrew Morton --- fs/block_dev.c | 2 +- fs/dcache.c | 4 ++-- fs/direct-io.c | 4 ++-- fs/exec.c | 4 ++-- fs/fhandle.c | 2 +- fs/pipe.c | 2 +- 6 files changed, 9 insertions(+), 9 deletions(-) --- a/fs/block_dev.c~fs-delete-repeated-words-in-comments +++ a/fs/block_dev.c @@ -1270,7 +1270,7 @@ rescan: return ret; } /* - * Only exported for for loop and dasd for historic reasons. Don't use in new + * Only exported for loop and dasd for historic reasons. Don't use in new * code! */ EXPORT_SYMBOL_GPL(bdev_disk_changed); --- a/fs/dcache.c~fs-delete-repeated-words-in-comments +++ a/fs/dcache.c @@ -2176,8 +2176,8 @@ EXPORT_SYMBOL(d_obtain_root); * same inode, only the actual correct case is stored in the dcache for * case-insensitive filesystems. * - * For a case-insensitive lookup match and if the the case-exact dentry - * already exists in in the dcache, use it and return it. + * For a case-insensitive lookup match and if the case-exact dentry + * already exists in the dcache, use it and return it. * * If no entry exists with the exact case name, allocate new dentry with * the exact case, and return the spliced entry. --- a/fs/direct-io.c~fs-delete-repeated-words-in-comments +++ a/fs/direct-io.c @@ -462,7 +462,7 @@ static inline void dio_cleanup(struct di * Wait for the next BIO to complete. Remove it and return it. NULL is * returned once all BIOs have been completed. This must only be called once * all bios have been issued so that dio->refcount can only decrease. This - * requires that that the caller hold a reference on the dio. + * requires that the caller hold a reference on the dio. */ static struct bio *dio_await_one(struct dio *dio) { @@ -1279,7 +1279,7 @@ do_blockdev_direct_IO(struct kiocb *iocb if (retval == -ENOTBLK) { /* * The remaining part of the request will be - * be handled by buffered I/O when we return + * handled by buffered I/O when we return */ retval = 0; } --- a/fs/exec.c~fs-delete-repeated-words-in-comments +++ a/fs/exec.c @@ -1455,7 +1455,7 @@ EXPORT_SYMBOL(finalize_exec); /* * Prepare credentials and lock ->cred_guard_mutex. * setup_new_exec() commits the new creds and drops the lock. - * Or, if exec fails before, free_bprm() should release ->cred and + * Or, if exec fails before, free_bprm() should release ->cred * and unlock. */ static int prepare_bprm_creds(struct linux_binprm *bprm) @@ -1841,7 +1841,7 @@ static int bprm_execve(struct linux_binp out: /* - * If past the point of no return ensure the the code never + * If past the point of no return ensure the code never * returns to the userspace process. Use an existing fatal * signal if present otherwise terminate the process with * SIGSEGV. --- a/fs/fhandle.c~fs-delete-repeated-words-in-comments +++ a/fs/fhandle.c @@ -173,7 +173,7 @@ static int handle_to_path(int mountdirfd /* * With handle we don't look at the execute bit on the - * the directory. Ideally we would like CAP_DAC_SEARCH. + * directory. Ideally we would like CAP_DAC_SEARCH. * But we don't have that */ if (!capable(CAP_DAC_READ_SEARCH)) { --- a/fs/pipe.c~fs-delete-repeated-words-in-comments +++ a/fs/pipe.c @@ -171,7 +171,7 @@ EXPORT_SYMBOL(generic_pipe_buf_try_steal * * Description: * This function grabs an extra reference to @buf. It's used in - * in the tee() system call, when we duplicate the buffers in one + * the tee() system call, when we duplicate the buffers in one * pipe into another. */ bool generic_pipe_buf_get(struct pipe_inode_info *pipe, struct pipe_buffer *buf) From patchwork Wed Feb 24 20:00:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102439 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7607DC433DB for ; Wed, 24 Feb 2021 20:00:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1821364F22 for ; Wed, 24 Feb 2021 20:00:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1821364F22 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B08046B007E; Wed, 24 Feb 2021 15:00:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AB55F6B0080; Wed, 24 Feb 2021 15:00:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9CC3E6B0081; Wed, 24 Feb 2021 15:00:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0137.hostedemail.com [216.40.44.137]) by kanga.kvack.org (Postfix) with ESMTP id 8753A6B007E for ; Wed, 24 Feb 2021 15:00:55 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 53C692497 for ; Wed, 24 Feb 2021 20:00:55 +0000 (UTC) X-FDA: 77854229670.07.3CD827F Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP id 8B03BA07C for ; Wed, 24 Feb 2021 20:00:53 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 56CC264F20; Wed, 24 Feb 2021 20:00:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196852; bh=BG36qJtyKsDqyW3kruv9FcwEBVvj/yyLc2TZ80f2/7Q=; h=Date:From:To:Subject:In-Reply-To:From; b=xYNu2gUAXGavgLOrj7KZ/TWpJyWBlmjMjlm/5rTor+aa7QD9C2QrozZBZN1EFSy2X 2+SMttSq5sYX9BBG/Kw0u+/+2Y0FVKuJoQAUlR/KnBbC5P8A/haSDaamIKO2coTAu+ FBS0QHohgY9mgFatyg5MDWX2vCuBmpln//fhD5LA= Date: Wed, 24 Feb 2021 12:00:51 -0800 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, christian.brauner@ubuntu.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 013/173] ramfs: support O_TMPFILE Message-ID: <20210224200051.MRGrUqyzj%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 8B03BA07C X-Stat-Signature: 3ypce5ebnuw8w4h5ocyu99ox8ucsfqo6 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf20; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196853-968303 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alexey Dobriyan Subject: ramfs: support O_TMPFILE [akpm@linux-foundation.org: update inode_operations.tmpfile] Link: http://lkml.kernel.org/r/20190206073349.GA15311@avx2 Signed-off-by: Alexey Dobriyan Cc: Christian Brauner Cc: Al Viro Signed-off-by: Andrew Morton Acked-by: Christian Brauner --- fs/ramfs/inode.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) --- a/fs/ramfs/inode.c~ramfs-support-o_tmpfile +++ a/fs/ramfs/inode.c @@ -151,6 +151,18 @@ static int ramfs_symlink(struct user_nam return error; } +static int ramfs_tmpfile(struct user_namespace *mnt_userns, + struct inode *dir, struct dentry *dentry, umode_t mode) +{ + struct inode *inode; + + inode = ramfs_get_inode(dir->i_sb, dir, mode, 0); + if (!inode) + return -ENOSPC; + d_tmpfile(dentry, inode); + return 0; +} + static const struct inode_operations ramfs_dir_inode_operations = { .create = ramfs_create, .lookup = simple_lookup, @@ -161,6 +173,7 @@ static const struct inode_operations ram .rmdir = simple_rmdir, .mknod = ramfs_mknod, .rename = simple_rename, + .tmpfile = ramfs_tmpfile, }; /* From patchwork Wed Feb 24 20:00:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102475 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D54C7C433DB for ; Wed, 24 Feb 2021 20:01:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 73D5564E90 for ; Wed, 24 Feb 2021 20:01:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 73D5564E90 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 16C468D0005; Wed, 24 Feb 2021 15:01:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0C90E8D0001; Wed, 24 Feb 2021 15:01:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EACE28D0005; Wed, 24 Feb 2021 15:01:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0155.hostedemail.com [216.40.44.155]) by kanga.kvack.org (Postfix) with ESMTP id D0E2C8D0001 for ; Wed, 24 Feb 2021 15:01:57 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 9B7C6184956D8 for ; Wed, 24 Feb 2021 20:01:57 +0000 (UTC) X-FDA: 77854232274.08.395B669 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf06.hostedemail.com (Postfix) with ESMTP id 7AC0BC0042F6 for ; Wed, 24 Feb 2021 20:00:58 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A5D2364F1A; Wed, 24 Feb 2021 20:00:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196856; bh=bWj03YS4dUlN+riW8Sff77uR4XGuT6t7mCUH0Zzaf24=; h=Date:From:To:Subject:In-Reply-To:From; b=tiiQFko54DO0Ya883VTM/xN++Arg7dq1aVoOrxBVhn1b5OAOMbnNd+OBx0jZWxl/n 8/idO4T8Rb0DPrOsVfwJgJ2bdgojlrLSQj1hCgMGEzSPLs4posqmEBBVmyOx9XVpLN X6x+SBfEyuPEKa/vmW/KVnSZM2kbrpr/sijXa1I0= Date: Wed, 24 Feb 2021 12:00:55 -0800 From: Andrew Morton To: akpm@linux-foundation.org, cl@linux.com, iamjoonsoo.kim@lge.com, jian.w.wen@oracle.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, paulmck@linux.vnet.ibm.com, penberg@kernel.org, rientjes@google.com, rostedt@goodmis.org, torvalds@linux-foundation.org Subject: [patch 014/173] mm, tracing: record slab name for kmem_cache_free() Message-ID: <20210224200055.U7Xz47kX5%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 6ibub95qpt9drntdbqfta6cx3akb4s5c X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 7AC0BC0042F6 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf06; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196858-787087 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jacob Wen Subject: mm, tracing: record slab name for kmem_cache_free() Currently, a trace record generated by the RCU core is as below. ... kmem_cache_free: call_site=rcu_core+0x1fd/0x610 ptr=00000000f3b49a66 It doesn't tell us what the RCU core has freed. This patch adds the slab name to trace_kmem_cache_free(). The new format is as follows. ... kmem_cache_free: call_site=rcu_core+0x1fd/0x610 ptr=0000000037f79c8d name=dentry ... kmem_cache_free: call_site=rcu_core+0x1fd/0x610 ptr=00000000f78cb7b5 name=sock_inode_cache ... kmem_cache_free: call_site=rcu_core+0x1fd/0x610 ptr=0000000018768985 name=pool_workqueue ... kmem_cache_free: call_site=rcu_core+0x1fd/0x610 ptr=000000006a6cb484 name=radix_tree_node We can use it to understand what the RCU core is going to free. For example, some users maybe interested in when the RCU core starts freeing reclaimable slabs like dentry to reduce memory pressure. Link: https://lkml.kernel.org/r/20201216072804.8838-1-jian.w.wen@oracle.com Signed-off-by: Jacob Wen Cc: Christoph Lameter Cc: Pekka Enberg Cc: David Rientjes Cc: Joonsoo Kim Cc: Steven Rostedt Cc: "Paul E. McKenney" Signed-off-by: Andrew Morton --- include/trace/events/kmem.h | 24 ++++++++++++++++-------- mm/slab.c | 2 +- mm/slob.c | 2 +- mm/slub.c | 2 +- 4 files changed, 19 insertions(+), 11 deletions(-) --- a/include/trace/events/kmem.h~mm-tracing-record-slab-name-for-kmem_cache_free +++ a/include/trace/events/kmem.h @@ -115,7 +115,7 @@ DEFINE_EVENT(kmem_alloc_node, kmem_cache TP_ARGS(call_site, ptr, bytes_req, bytes_alloc, gfp_flags, node) ); -DECLARE_EVENT_CLASS(kmem_free, +TRACE_EVENT(kfree, TP_PROTO(unsigned long call_site, const void *ptr), @@ -135,18 +135,26 @@ DECLARE_EVENT_CLASS(kmem_free, (void *)__entry->call_site, __entry->ptr) ); -DEFINE_EVENT(kmem_free, kfree, +TRACE_EVENT(kmem_cache_free, - TP_PROTO(unsigned long call_site, const void *ptr), + TP_PROTO(unsigned long call_site, const void *ptr, const char *name), - TP_ARGS(call_site, ptr) -); + TP_ARGS(call_site, ptr, name), -DEFINE_EVENT(kmem_free, kmem_cache_free, + TP_STRUCT__entry( + __field( unsigned long, call_site ) + __field( const void *, ptr ) + __field( const char *, name ) + ), - TP_PROTO(unsigned long call_site, const void *ptr), + TP_fast_assign( + __entry->call_site = call_site; + __entry->ptr = ptr; + __entry->name = name; + ), - TP_ARGS(call_site, ptr) + TP_printk("call_site=%pS ptr=%p name=%s", + (void *)__entry->call_site, __entry->ptr, __entry->name) ); TRACE_EVENT(mm_page_free, --- a/mm/slab.c~mm-tracing-record-slab-name-for-kmem_cache_free +++ a/mm/slab.c @@ -3717,7 +3717,7 @@ void kmem_cache_free(struct kmem_cache * __cache_free(cachep, objp, _RET_IP_); local_irq_restore(flags); - trace_kmem_cache_free(_RET_IP_, objp); + trace_kmem_cache_free(_RET_IP_, objp, cachep->name); } EXPORT_SYMBOL(kmem_cache_free); --- a/mm/slob.c~mm-tracing-record-slab-name-for-kmem_cache_free +++ a/mm/slob.c @@ -673,7 +673,7 @@ void kmem_cache_free(struct kmem_cache * __kmem_cache_free(b, c->size); } - trace_kmem_cache_free(_RET_IP_, b); + trace_kmem_cache_free(_RET_IP_, b, c->name); } EXPORT_SYMBOL(kmem_cache_free); --- a/mm/slub.c~mm-tracing-record-slab-name-for-kmem_cache_free +++ a/mm/slub.c @@ -3157,7 +3157,7 @@ void kmem_cache_free(struct kmem_cache * if (!s) return; slab_free(s, virt_to_head_page(x), x, NULL, 1, _RET_IP_); - trace_kmem_cache_free(_RET_IP_, x); + trace_kmem_cache_free(_RET_IP_, x, s->name); } EXPORT_SYMBOL(kmem_cache_free); From patchwork Wed Feb 24 20:00:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102441 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CA09C433E0 for ; Wed, 24 Feb 2021 20:01:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D70DA64F24 for ; Wed, 24 Feb 2021 20:01:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D70DA64F24 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6718F6B0080; Wed, 24 Feb 2021 15:01:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6205E6B0081; Wed, 24 Feb 2021 15:01:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 538956B0082; Wed, 24 Feb 2021 15:01:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0192.hostedemail.com [216.40.44.192]) by kanga.kvack.org (Postfix) with ESMTP id 394AA6B0080 for ; Wed, 24 Feb 2021 15:01:01 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 0178E184B5463 for ; Wed, 24 Feb 2021 20:01:00 +0000 (UTC) X-FDA: 77854229922.25.CC39D01 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf13.hostedemail.com (Postfix) with ESMTP id 287DEE016168 for ; Wed, 24 Feb 2021 20:00:58 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id EF1FA64F22; Wed, 24 Feb 2021 20:00:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196859; bh=EjHlAJ8vM/EnGMmH1KGhEVeYOLDsTpdEZUyGnWAmZ3k=; h=Date:From:To:Subject:In-Reply-To:From; b=o80CueqjMmPHb89VDuwsQ+ni/s8CeUGB1YVt+ErrNDi9CZX8btM9nyOaf1NPg61hR 1FZA9X2bPmv9x2+q+3atGGEQ9bBHAr50cDAc0/YsWNi4V3H7WGN+RXKdW9sGHeieVw JziG+PvycGxDL4PE/O45qsxMzCbBYRptutWejZhQ= Date: Wed, 24 Feb 2021 12:00:58 -0800 From: Andrew Morton To: akpm@linux-foundation.org, cl@linux.com, iamjoonsoo.kim@lge.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, nborisov@suse.com, penberg@kernel.org, rientjes@google.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 015/173] mm/sl?b.c: remove ctor argument from kmem_cache_flags Message-ID: <20210224200058.TONZYffhK%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 287DEE016168 X-Stat-Signature: ad56iimt6bfuwkg5q8pan6nzrumebd6f Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf13; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196858-408585 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Nikolay Borisov Subject: mm/sl?b.c: remove ctor argument from kmem_cache_flags This argument hasn't been used since e153362a50a3 ("slub: Remove objsize check in kmem_cache_flags()") so simply remove it. Link: https://lkml.kernel.org/r/20210126095733.974665-1-nborisov@suse.com Signed-off-by: Nikolay Borisov Reviewed-by: Miaohe Lin Reviewed-by: Vlastimil Babka Acked-by: Christoph Lameter Acked-by: David Rientjes Cc: Pekka Enberg Cc: Joonsoo Kim Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- mm/slab.c | 3 +-- mm/slab.h | 6 ++---- mm/slab_common.c | 2 +- mm/slub.c | 9 +++------ 4 files changed, 7 insertions(+), 13 deletions(-) --- a/mm/slab.c~mm-remove-ctor-argument-from-kmem_cache_flags +++ a/mm/slab.c @@ -1790,8 +1790,7 @@ static int __ref setup_cpu_cache(struct } slab_flags_t kmem_cache_flags(unsigned int object_size, - slab_flags_t flags, const char *name, - void (*ctor)(void *)) + slab_flags_t flags, const char *name) { return flags; } --- a/mm/slab_common.c~mm-remove-ctor-argument-from-kmem_cache_flags +++ a/mm/slab_common.c @@ -197,7 +197,7 @@ struct kmem_cache *find_mergeable(unsign size = ALIGN(size, sizeof(void *)); align = calculate_alignment(flags, align, size); size = ALIGN(size, align); - flags = kmem_cache_flags(size, flags, name, NULL); + flags = kmem_cache_flags(size, flags, name); if (flags & SLAB_NEVER_MERGE) return NULL; --- a/mm/slab.h~mm-remove-ctor-argument-from-kmem_cache_flags +++ a/mm/slab.h @@ -110,8 +110,7 @@ __kmem_cache_alias(const char *name, uns slab_flags_t flags, void (*ctor)(void *)); slab_flags_t kmem_cache_flags(unsigned int object_size, - slab_flags_t flags, const char *name, - void (*ctor)(void *)); + slab_flags_t flags, const char *name); #else static inline struct kmem_cache * __kmem_cache_alias(const char *name, unsigned int size, unsigned int align, @@ -119,8 +118,7 @@ __kmem_cache_alias(const char *name, uns { return NULL; } static inline slab_flags_t kmem_cache_flags(unsigned int object_size, - slab_flags_t flags, const char *name, - void (*ctor)(void *)) + slab_flags_t flags, const char *name) { return flags; } --- a/mm/slub.c~mm-remove-ctor-argument-from-kmem_cache_flags +++ a/mm/slub.c @@ -1400,7 +1400,6 @@ __setup("slub_debug", setup_slub_debug); * @object_size: the size of an object without meta data * @flags: flags to set * @name: name of the cache - * @ctor: constructor function * * Debug option(s) are applied to @flags. In addition to the debug * option(s), if a slab name (or multiple) is specified i.e. @@ -1408,8 +1407,7 @@ __setup("slub_debug", setup_slub_debug); * then only the select slabs will receive the debug option(s). */ slab_flags_t kmem_cache_flags(unsigned int object_size, - slab_flags_t flags, const char *name, - void (*ctor)(void *)) + slab_flags_t flags, const char *name) { char *iter; size_t len; @@ -1474,8 +1472,7 @@ static inline void add_full(struct kmem_ static inline void remove_full(struct kmem_cache *s, struct kmem_cache_node *n, struct page *page) {} slab_flags_t kmem_cache_flags(unsigned int object_size, - slab_flags_t flags, const char *name, - void (*ctor)(void *)) + slab_flags_t flags, const char *name) { return flags; } @@ -3797,7 +3794,7 @@ static int calculate_sizes(struct kmem_c static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags) { - s->flags = kmem_cache_flags(s->size, flags, s->name, s->ctor); + s->flags = kmem_cache_flags(s->size, flags, s->name); #ifdef CONFIG_SLAB_FREELIST_HARDENED s->random = get_random_long(); #endif From patchwork Wed Feb 24 20:01:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102443 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 37CBFC433E0 for ; Wed, 24 Feb 2021 20:01:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CE51564F25 for ; Wed, 24 Feb 2021 20:01:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CE51564F25 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5FC976B0081; Wed, 24 Feb 2021 15:01:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 583E96B0082; Wed, 24 Feb 2021 15:01:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 475496B0083; Wed, 24 Feb 2021 15:01:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0243.hostedemail.com [216.40.44.243]) by kanga.kvack.org (Postfix) with ESMTP id 2E12A6B0081 for ; Wed, 24 Feb 2021 15:01:04 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id EC7BBB9F0 for ; Wed, 24 Feb 2021 20:01:03 +0000 (UTC) X-FDA: 77854230006.17.C83DC12 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id C69D6204C588 for ; Wed, 24 Feb 2021 20:00:53 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 49FA564F24; Wed, 24 Feb 2021 20:01:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196862; bh=GqC5KjqXTiGz469Xck+nzFaTNFWpRG1Prz9qgE1+Dgo=; h=Date:From:To:Subject:In-Reply-To:From; b=WiGbVbeFG5Oo+f3Qe1lCAnz7GyRNAhpeIK9Lcj/WD9yQiyn5YsvK9XIRNf69CfSZO d4kpvFtUpPtaeINZ+75R+bucjp5o6/4nraKZERXI/smlFWolMBhvW2xe9Vezq6otQB SYLdRc4ZIMig4Ij0e1F9E8AnYextXI8FO86v6k4s= Date: Wed, 24 Feb 2021 12:01:01 -0800 From: Andrew Morton To: akpm@linux-foundation.org, daizhiyuan@phytium.com.cn, linux-mm@kvack.org, mm-commits@vger.kernel.org, rientjes@google.com, torvalds@linux-foundation.org Subject: [patch 016/173] mm/slab: minor coding style tweaks Message-ID: <20210224200101.IcBvdfJKS%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: C69D6204C588 X-Stat-Signature: oyj8mqokc66qhhqdcqoychzx7uqs6pjf Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf11; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196853-81365 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zhiyuan Dai Subject: mm/slab: minor coding style tweaks Fix some coding style issues, improve code reading. Adds whitespace to clearly separate the parameters. Link: https://lkml.kernel.org/r/1612841499-32166-1-git-send-email-daizhiyuan@phytium.com.cn Signed-off-by: Zhiyuan Dai Acked-by: David Rientjes Signed-off-by: Andrew Morton --- mm/slab.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) --- a/mm/slab.c~mm-slab-minor-coding-style-tweaks +++ a/mm/slab.c @@ -272,7 +272,7 @@ static void kmem_cache_node_init(struct #define STATS_DEC_ACTIVE(x) ((x)->num_active--) #define STATS_INC_ALLOCED(x) ((x)->num_allocations++) #define STATS_INC_GROWN(x) ((x)->grown++) -#define STATS_ADD_REAPED(x,y) ((x)->reaped += (y)) +#define STATS_ADD_REAPED(x, y) ((x)->reaped += (y)) #define STATS_SET_HIGH(x) \ do { \ if ((x)->num_active > (x)->high_mark) \ @@ -296,7 +296,7 @@ static void kmem_cache_node_init(struct #define STATS_DEC_ACTIVE(x) do { } while (0) #define STATS_INC_ALLOCED(x) do { } while (0) #define STATS_INC_GROWN(x) do { } while (0) -#define STATS_ADD_REAPED(x,y) do { (void)(y); } while (0) +#define STATS_ADD_REAPED(x, y) do { (void)(y); } while (0) #define STATS_SET_HIGH(x) do { } while (0) #define STATS_INC_ERR(x) do { } while (0) #define STATS_INC_NODEALLOCS(x) do { } while (0) @@ -332,7 +332,7 @@ static int obj_offset(struct kmem_cache static unsigned long long *dbg_redzone1(struct kmem_cache *cachep, void *objp) { BUG_ON(!(cachep->flags & SLAB_RED_ZONE)); - return (unsigned long long*) (objp + obj_offset(cachep) - + return (unsigned long long *) (objp + obj_offset(cachep) - sizeof(unsigned long long)); } @@ -580,7 +580,7 @@ static int transfer_objects(struct array if (!nr) return 0; - memcpy(to->entry + to->avail, from->entry + from->avail -nr, + memcpy(to->entry + to->avail, from->entry + from->avail - nr, sizeof(void *) *nr); from->avail -= nr; @@ -2737,7 +2737,7 @@ static void *cache_free_debugcheck(struc #else #define kfree_debugcheck(x) do { } while(0) -#define cache_free_debugcheck(x,objp,z) (objp) +#define cache_free_debugcheck(x, objp, z) (objp) #endif static inline void fixup_objfreelist_debug(struct kmem_cache *cachep, @@ -3024,7 +3024,7 @@ static void *cache_alloc_debugcheck_afte return objp; } #else -#define cache_alloc_debugcheck_after(a,b,objp,d) (objp) +#define cache_alloc_debugcheck_after(a, b, objp, d) (objp) #endif static inline void *____cache_alloc(struct kmem_cache *cachep, gfp_t flags) From patchwork Wed Feb 24 20:01:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102463 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27AE3C433E6 for ; Wed, 24 Feb 2021 20:01:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D619864E6B for ; Wed, 24 Feb 2021 20:01:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D619864E6B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E39FC6B0088; Wed, 24 Feb 2021 15:01:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DC3806B0089; Wed, 24 Feb 2021 15:01:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C15EF6B008A; Wed, 24 Feb 2021 15:01:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0126.hostedemail.com [216.40.44.126]) by kanga.kvack.org (Postfix) with ESMTP id AD38E6B0088 for ; Wed, 24 Feb 2021 15:01:39 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 7C58D82C49D0 for ; Wed, 24 Feb 2021 20:01:39 +0000 (UTC) X-FDA: 77854231518.11.85F572A Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id C2527FE for ; Wed, 24 Feb 2021 20:01:32 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 770CF64F21; Wed, 24 Feb 2021 20:01:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196866; bh=/3W9GpPijcVwD4pFLLD3dvWkaLEuJAkhG4fAflBESJc=; h=Date:From:To:Subject:In-Reply-To:From; b=Fvanrqz/WJFFSMTVp9c8j/3AG2MbUux0m+9UWOUoAFUVkFGu0OmjgA/60nLlHSwK8 Py28uKcOlpi0RbAiu1m1lUwDELN6hrkrqDZFjZrIvC/qlBhG/xjAEF2KQzcf3TljBv Q1Zs/LwGBkIK0eQCLhvoIukL+ZEAd5zIh0Rc3xR8= Date: Wed, 24 Feb 2021 12:01:04 -0800 From: Andrew Morton To: akpm@linux-foundation.org, catalin.marinas@arm.com, cl@linux.com, iamjoonsoo.kim@lge.com, johannes.berg@intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, penberg@kernel.org, rientjes@google.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 017/173] mm/slub: disable user tracing for kmemleak caches by default Message-ID: <20210224200104.pbo4Kya9g%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: ynirxsu866qeia441f5z4gm5udw8zj6u X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: C2527FE Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf12; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196892-405836 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Berg Subject: mm/slub: disable user tracing for kmemleak caches by default If kmemleak is enabled, it uses a kmem cache for its own objects. These objects are used to hold information kmemleak uses, including a stack trace. If slub_debug is also turned on, each of them has *another* stack trace, so the overhead adds up, and on my tests (on ARCH=um, admittedly) 2/3rds of the allocations end up being doing the stack tracing. Turn off SLAB_STORE_USER if SLAB_NOLEAKTRACE was given, to avoid storing the essentially same data twice. Link: https://lkml.kernel.org/r/20210113215114.d94efa13ba30.I117b6764e725b3192318bbcf4269b13b709539ae@changeid Signed-off-by: Johannes Berg Acked-by: David Rientjes Acked-by: Catalin Marinas Acked-by: Vlastimil Babka Cc: Christoph Lameter Cc: Pekka Enberg Cc: Joonsoo Kim Signed-off-by: Andrew Morton --- mm/slub.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) --- a/mm/slub.c~mm-slub-disable-user-tracing-for-kmemleak-caches-by-default +++ a/mm/slub.c @@ -1413,6 +1413,15 @@ slab_flags_t kmem_cache_flags(unsigned i size_t len; char *next_block; slab_flags_t block_flags; + slab_flags_t slub_debug_local = slub_debug; + + /* + * If the slab cache is for debugging (e.g. kmemleak) then + * don't store user (stack trace) information by default, + * but let the user enable it via the command line below. + */ + if (flags & SLAB_NOLEAKTRACE) + slub_debug_local &= ~SLAB_STORE_USER; len = strlen(name); next_block = slub_debug_string; @@ -1447,7 +1456,7 @@ slab_flags_t kmem_cache_flags(unsigned i } } - return flags | slub_debug; + return flags | slub_debug_local; } #else /* !CONFIG_SLUB_DEBUG */ static inline void setup_object_debug(struct kmem_cache *s, From patchwork Wed Feb 24 20:01:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8BA18C433E0 for ; Wed, 24 Feb 2021 20:01:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2936C64F28 for ; Wed, 24 Feb 2021 20:01:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2936C64F28 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B05A56B0082; Wed, 24 Feb 2021 15:01:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AB4936B0083; Wed, 24 Feb 2021 15:01:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9CC006B0085; Wed, 24 Feb 2021 15:01:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0073.hostedemail.com [216.40.44.73]) by kanga.kvack.org (Postfix) with ESMTP id 815BB6B0082 for ; Wed, 24 Feb 2021 15:01:11 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 4B570184C0D66 for ; Wed, 24 Feb 2021 20:01:11 +0000 (UTC) X-FDA: 77854230342.21.366994C Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf23.hostedemail.com (Postfix) with ESMTP id 58A00A014F4D for ; Wed, 24 Feb 2021 20:01:10 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 1D9DE64F26; Wed, 24 Feb 2021 20:01:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196869; bh=H7QzZJOVMSLdatthA7i2NQjXwAhb467dQPerhyWULIo=; h=Date:From:To:Subject:In-Reply-To:From; b=aPSK9Id0cKOXGyd+b3yfY01oXxUg2/Lg9McDBBtq0gfAzQcolq49Ko+PDkzl2sFvo ZVcC/tXbnmuTlt//oP4Ie1U5oo5PNpQdR3XiFbAFNm21OziZBVd0YWwxaHmdpAxElv X6+WAJhGhP+W45CHKqJOSOIsJRFbiPYYCsK4zbso= Date: Wed, 24 Feb 2021 12:01:08 -0800 From: Andrew Morton To: akpm@linux-foundation.org, cai@redhat.com, cl@linux.com, david@redhat.com, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, penberg@kernel.org, rientjes@google.com, torvalds@linux-foundation.org, vbabka@suse.cz, vdavydov.dev@gmail.com Subject: [patch 018/173] mm, slub: stop freeing kmem_cache_node structures on node offline Message-ID: <20210224200108.Ikw8wCI3v%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 6xpc1azrtub6amtd54kswhr1cy6uuhcq X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 58A00A014F4D Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf23; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196870-824272 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vlastimil Babka Subject: mm, slub: stop freeing kmem_cache_node structures on node offline Patch series "mm, slab, slub: remove cpu and memory hotplug locks". Some related work caused me to look at how we use get/put_mems_online() and get/put_online_cpus() during kmem cache creation/descruction/shrinking, and realize that it should be actually safe to remove all of that with rather small effort (as e.g. Michal Hocko suspected in some of the past discussions already). This has the benefit to avoid rather heavy locks that have caused locking order issues already in the past. So this is the result, Patches 2 and 3 remove memory hotplug and cpu hotplug locking, respectively. Patch 1 is due to realization that in fact some races exist despite the locks (even if not removed), but the most sane solution is not to introduce more of them, but rather accept some wasted memory in scenarios that should be rare anyway (full memory hot remove), as we do the same in other contexts already. This patch (of 3): Commit e4f8e513c3d3 ("mm/slub: fix a deadlock in show_slab_objects()") has fixed a problematic locking order by removing the memory hotplug lock get/put_online_mems() from show_slab_objects(). During the discussion, it was argued [1] that this is OK, because existing slabs on the node would prevent a hotremove to proceed. That's true, but per-node kmem_cache_node structures are not necessarily allocated on the same node and may exist even without actual slab pages on the same node. Any path that uses get_node() directly or via for_each_kmem_cache_node() (such as show_slab_objects()) can race with freeing of kmem_cache_node even with the !NULL check, resulting in use-after-free. To that end, commit e4f8e513c3d3 argues in a comment that: * We don't really need mem_hotplug_lock (to hold off * slab_mem_going_offline_callback) here because slab's memory hot * unplug code doesn't destroy the kmem_cache->node[] data. While it's true that slab_mem_going_offline_callback() doesn't free the kmem_cache_node, the later callback slab_mem_offline_callback() actually does, so the race and use-after-free exists. Not just for show_slab_objects() after commit e4f8e513c3d3, but also many other places that are not under slab_mutex. And adding slab_mutex locking or other synchronization to SLUB paths such as get_any_partial() would be bad for performance and error-prone. The easiest solution is therefore to make the abovementioned comment true and stop freeing the kmem_cache_node structures, accepting some wasted memory in the full memory node removal scenario. Analogically we also don't free hotremoved pgdat as mentioned in [1], nor the similar per-node structures in SLAB. Importantly this approach will not block the hotremove, as generally such nodes should be movable in order to succeed hotremove in the first place, and thus the GFP_KERNEL allocated kmem_cache_node will come from elsewhere. [1] https://lore.kernel.org/linux-mm/20190924151147.GB23050@dhcp22.suse.cz/ Link: https://lkml.kernel.org/r/20210113131634.3671-1-vbabka@suse.cz Link: https://lkml.kernel.org/r/20210113131634.3671-2-vbabka@suse.cz Signed-off-by: Vlastimil Babka Cc: Christoph Lameter Cc: Pekka Enberg Cc: David Rientjes Cc: Joonsoo Kim Cc: Vladimir Davydov Cc: Qian Cai Cc: David Hildenbrand Cc: Michal Hocko Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- mm/slub.c | 28 +++++++++++----------------- 1 file changed, 11 insertions(+), 17 deletions(-) --- a/mm/slub.c~mm-slub-stop-freeing-kmem_cache_node-structures-on-node-offline +++ a/mm/slub.c @@ -4273,8 +4273,6 @@ static int slab_mem_going_offline_callba static void slab_mem_offline_callback(void *arg) { - struct kmem_cache_node *n; - struct kmem_cache *s; struct memory_notify *marg = arg; int offline_node; @@ -4288,21 +4286,11 @@ static void slab_mem_offline_callback(vo return; mutex_lock(&slab_mutex); - list_for_each_entry(s, &slab_caches, list) { - n = get_node(s, offline_node); - if (n) { - /* - * if n->nr_slabs > 0, slabs still exist on the node - * that is going down. We were unable to free them, - * and offline_pages() function shouldn't call this - * callback. So, we must fail. - */ - BUG_ON(slabs_node(s, offline_node)); - - s->node[offline_node] = NULL; - kmem_cache_free(kmem_cache_node, n); - } - } + /* + * We no longer free kmem_cache_node structures here, as it would be + * racy with all get_node() users, and infeasible to protect them with + * slab_mutex. + */ mutex_unlock(&slab_mutex); } @@ -4329,6 +4317,12 @@ static int slab_mem_going_online_callbac mutex_lock(&slab_mutex); list_for_each_entry(s, &slab_caches, list) { /* + * The structure may already exist if the node was previously + * onlined and offlined. + */ + if (get_node(s, nid)) + continue; + /* * XXX: kmem_cache_alloc_node will fallback to other nodes * since memory is not yet available from the node that * is brought up. From patchwork Wed Feb 24 20:01:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F85CC433DB for ; Wed, 24 Feb 2021 20:01:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E9E6264F29 for ; Wed, 24 Feb 2021 20:01:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E9E6264F29 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7332B6B0083; Wed, 24 Feb 2021 15:01:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6BF646B0085; Wed, 24 Feb 2021 15:01:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 564346B0087; Wed, 24 Feb 2021 15:01:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0053.hostedemail.com [216.40.44.53]) by kanga.kvack.org (Postfix) with ESMTP id 3BB7E6B0083 for ; Wed, 24 Feb 2021 15:01:15 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id F077945BB for ; Wed, 24 Feb 2021 20:01:14 +0000 (UTC) X-FDA: 77854230468.05.C29D5E4 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf28.hostedemail.com (Postfix) with ESMTP id 4A85D2000399 for ; Wed, 24 Feb 2021 20:01:14 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B575564F25; Wed, 24 Feb 2021 20:01:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196873; bh=rItnaG2cH6d6XPeZ6bkSwlBLsFnuNmEyhSPvslokdxs=; h=Date:From:To:Subject:In-Reply-To:From; b=XCB7c09YqxUX+8JahP3DKX7kv5RFcRZo4w4s6dp1pzHddZCL9MPf7dhJyjPaZ5hO7 1yJy8rbtIyWygniG/RHRIUfWsh4eEODWMEsM6H3rgW6LWwkIdonHxU+48A2dLURwkQ IZKVIqUq3WT6rTQ/b26CBrjrMajm7CV0wIBaw56M= Date: Wed, 24 Feb 2021 12:01:12 -0800 From: Andrew Morton To: akpm@linux-foundation.org, cai@redhat.com, cl@linux.com, david@redhat.com, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, penberg@kernel.org, rientjes@google.com, torvalds@linux-foundation.org, vbabka@suse.cz, vdavydov.dev@gmail.com Subject: [patch 019/173] mm, slab, slub: stop taking memory hotplug lock Message-ID: <20210224200112.lvAMjFt4C%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: feusbc9g176orbr4zae3bfn1snkybizo X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 4A85D2000399 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf28; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196874-512530 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vlastimil Babka Subject: mm, slab, slub: stop taking memory hotplug lock Since commit 03afc0e25f7f ("slab: get_online_mems for kmem_cache_{create,destroy,shrink}") we are taking memory hotplug lock for SLAB and SLUB when creating, destroying or shrinking a cache. It is quite a heavy lock and it's best to avoid it if possible, as we had several issues with lockdep complaining about ordering in the past, see e.g. e4f8e513c3d3 ("mm/slub: fix a deadlock in show_slab_objects()"). The problem scenario in 03afc0e25f7f (solved by the memory hotplug lock) can be summarized as follows: while there's slab_mutex synchronizing new kmem cache creation and SLUB's MEM_GOING_ONLINE callback slab_mem_going_online_callback(), we may miss creation of kmem_cache_node for the hotplugged node in the new kmem cache, because the hotplug callback doesn't yet see the new cache, and cache creation in init_kmem_cache_nodes() only inits kmem_cache_node for nodes in the N_NORMAL_MEMORY nodemask, which however may not yet include the new node, as that happens only later after the MEM_GOING_ONLINE callback. Instead of using get/put_online_mems(), the problem can be solved by SLUB maintaining its own nodemask of nodes for which it has allocated the per-node kmem_cache_node structures. This nodemask would generally mirror the N_NORMAL_MEMORY nodemask, but would be updated only in under SLUB's control in its memory hotplug callbacks under the slab_mutex. This patch adds such nodemask and its handling. Commit 03afc0e25f7f mentiones "issues like [the one above]", but there don't appear to be further issues. All the paths (shared for SLAB and SLUB) taking the memory hotplug locks are also taking the slab_mutex, except kmem_cache_shrink() where 03afc0e25f7f replaced slab_mutex with get/put_online_mems(). We however cannot simply restore slab_mutex in kmem_cache_shrink(), as SLUB can enters the function from a write to sysfs 'shrink' file, thus holding kernfs lock, and in kmem_cache_create() the kernfs lock is nested within slab_mutex. But on closer inspection we don't actually need to protect kmem_cache_shrink() from hotplug callbacks: While SLUB's __kmem_cache_shrink() does for_each_kmem_cache_node(), missing a new node added in parallel hotplug is not fatal, and parallel hotremove does not free kmem_cache_node's anymore after the previous patch, so use-after free cannot happen. The per-node shrinking itself is protected by n->list_lock. Same is true for SLAB, and SLOB is no-op. SLAB also doesn't need the memory hotplug locking, which it only gained by 03afc0e25f7f through the shared paths in slab_common.c. Its memory hotplug callbacks are also protected by slab_mutex against races with these paths. The problem of SLUB relying on N_NORMAL_MEMORY doesn't apply to SLAB, as its setup_kmem_cache_nodes relies on N_ONLINE, and the new node is already set there during the MEM_GOING_ONLINE callback, so no special care is needed for SLAB. As such, this patch removes all get/put_online_mems() usage by the slab subsystem. Link: https://lkml.kernel.org/r/20210113131634.3671-3-vbabka@suse.cz Signed-off-by: Vlastimil Babka Cc: Christoph Lameter Cc: David Hildenbrand Cc: David Rientjes Cc: Joonsoo Kim Cc: Michal Hocko Cc: Pekka Enberg Cc: Qian Cai Cc: Vladimir Davydov Signed-off-by: Andrew Morton --- mm/slab_common.c | 8 ++------ mm/slub.c | 28 +++++++++++++++++++++++++--- 2 files changed, 27 insertions(+), 9 deletions(-) --- a/mm/slab_common.c~mm-slab-slub-stop-taking-memory-hotplug-lock +++ a/mm/slab_common.c @@ -310,7 +310,6 @@ kmem_cache_create_usercopy(const char *n int err; get_online_cpus(); - get_online_mems(); mutex_lock(&slab_mutex); @@ -360,7 +359,6 @@ kmem_cache_create_usercopy(const char *n out_unlock: mutex_unlock(&slab_mutex); - put_online_mems(); put_online_cpus(); if (err) { @@ -487,7 +485,6 @@ void kmem_cache_destroy(struct kmem_cach return; get_online_cpus(); - get_online_mems(); mutex_lock(&slab_mutex); @@ -504,7 +501,6 @@ void kmem_cache_destroy(struct kmem_cach out_unlock: mutex_unlock(&slab_mutex); - put_online_mems(); put_online_cpus(); } EXPORT_SYMBOL(kmem_cache_destroy); @@ -523,10 +519,10 @@ int kmem_cache_shrink(struct kmem_cache int ret; get_online_cpus(); - get_online_mems(); + kasan_cache_shrink(cachep); ret = __kmem_cache_shrink(cachep); - put_online_mems(); + put_online_cpus(); return ret; } --- a/mm/slub.c~mm-slab-slub-stop-taking-memory-hotplug-lock +++ a/mm/slub.c @@ -235,6 +235,14 @@ static inline void stat(const struct kme #endif } +/* + * Tracks for which NUMA nodes we have kmem_cache_nodes allocated. + * Corresponds to node_state[N_NORMAL_MEMORY], but can temporarily + * differ during memory hotplug/hotremove operations. + * Protected by slab_mutex. + */ +static nodemask_t slab_nodes; + /******************************************************************** * Core slab cache functions *******************************************************************/ @@ -2678,7 +2686,7 @@ static void *___slab_alloc(struct kmem_c * ignore the node constraint */ if (unlikely(node != NUMA_NO_NODE && - !node_state(node, N_NORMAL_MEMORY))) + !node_isset(node, slab_nodes))) node = NUMA_NO_NODE; goto new_slab; } @@ -2689,7 +2697,7 @@ redo: * same as above but node_match() being false already * implies node != NUMA_NO_NODE */ - if (!node_state(node, N_NORMAL_MEMORY)) { + if (!node_isset(node, slab_nodes)) { node = NUMA_NO_NODE; goto redo; } else { @@ -3592,7 +3600,7 @@ static int init_kmem_cache_nodes(struct { int node; - for_each_node_state(node, N_NORMAL_MEMORY) { + for_each_node_mask(node, slab_nodes) { struct kmem_cache_node *n; if (slab_state == DOWN) { @@ -4286,6 +4294,7 @@ static void slab_mem_offline_callback(vo return; mutex_lock(&slab_mutex); + node_clear(offline_node, slab_nodes); /* * We no longer free kmem_cache_node structures here, as it would be * racy with all get_node() users, and infeasible to protect them with @@ -4335,6 +4344,11 @@ static int slab_mem_going_online_callbac init_kmem_cache_node(n); s->node[nid] = n; } + /* + * Any cache created after this point will also have kmem_cache_node + * initialized for the new node. + */ + node_set(nid, slab_nodes); out: mutex_unlock(&slab_mutex); return ret; @@ -4415,6 +4429,7 @@ void __init kmem_cache_init(void) { static __initdata struct kmem_cache boot_kmem_cache, boot_kmem_cache_node; + int node; if (debug_guardpage_minorder()) slub_max_order = 0; @@ -4422,6 +4437,13 @@ void __init kmem_cache_init(void) kmem_cache_node = &boot_kmem_cache_node; kmem_cache = &boot_kmem_cache; + /* + * Initialize the nodemask for which we will allocate per node + * structures. Here we don't need taking slab_mutex yet. + */ + for_each_node_state(node, N_NORMAL_MEMORY) + node_set(node, slab_nodes); + create_boot_cache(kmem_cache_node, "kmem_cache_node", sizeof(struct kmem_cache_node), SLAB_HWCACHE_ALIGN, 0, 0); From patchwork Wed Feb 24 20:01:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102449 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82CDCC433E0 for ; Wed, 24 Feb 2021 20:01:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3B44864F28 for ; Wed, 24 Feb 2021 20:01:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3B44864F28 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CD7B46B0005; Wed, 24 Feb 2021 15:01:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C613B6B0085; Wed, 24 Feb 2021 15:01:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B77726B0087; Wed, 24 Feb 2021 15:01:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0159.hostedemail.com [216.40.44.159]) by kanga.kvack.org (Postfix) with ESMTP id A281A6B0005 for ; Wed, 24 Feb 2021 15:01:18 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 6953982D08AA for ; Wed, 24 Feb 2021 20:01:18 +0000 (UTC) X-FDA: 77854230636.30.4A5D7BD Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id 3AB5E2000396 for ; Wed, 24 Feb 2021 20:01:08 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 57F9664F23; Wed, 24 Feb 2021 20:01:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196876; bh=sUsO3EhZIzAc5iCkRXCgeaT0vg9qFPJbgpzN4SykNpk=; h=Date:From:To:Subject:In-Reply-To:From; b=ik8LH3+0Gw/ORMtmV4I3olNvPpSs++/WKVjHKEApDG7NypUQTcGFqCBb9+xQxOFAG aspsFWyZyCQ3pmw71jyHId4lMWycIzm1TNE83WBGCUbjAY0PnkxeZjrOAgdiiq2lkD ZjGp3DQxWdjGotePAtLRxYJyF9k1+MjCLnfFiPWo= Date: Wed, 24 Feb 2021 12:01:15 -0800 From: Andrew Morton To: akpm@linux-foundation.org, cai@redhat.com, cl@linux.com, david@redhat.com, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, penberg@kernel.org, rientjes@google.com, torvalds@linux-foundation.org, vbabka@suse.cz, vdavydov.dev@gmail.com Subject: [patch 020/173] mm, slab, slub: stop taking cpu hotplug lock Message-ID: <20210224200115.Sn4XTbiPC%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 3AB5E2000396 X-Stat-Signature: 6iufu9k8ydahrfifbqqqs15q51zow7ei Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf11; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196868-594422 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vlastimil Babka Subject: mm, slab, slub: stop taking cpu hotplug lock SLAB has been using get/put_online_cpus() around creating, destroying and shrinking kmem caches since 95402b382901 ("cpu-hotplug: replace per-subsystem mutexes with get_online_cpus()") in 2008, which is supposed to be replacing a private mutex (cache_chain_mutex, called slab_mutex today) with system-wide mechanism, but in case of SLAB it's in fact used in addition to the existing mutex, without explanation why. SLUB appears to have avoided the cpu hotplug lock initially, but gained it due to common code unification, such as 20cea9683ecc ("mm, sl[aou]b: Move kmem_cache_create mutex handling to common code"). Regardless of the history, checking if the hotplug lock is actually needed today suggests that it's not, and therefore it's better to avoid this system-wide lock and the ordering this imposes wrt other locks (such as slab_mutex). Specifically, in SLAB we have for_each_online_cpu() in do_tune_cpucache() protected by slab_mutex, and cpu hotplug callbacks that also take the slab_mutex, which is also taken by the common slab function that currently also take the hotplug lock. Thus the slab_mutex protection should be sufficient. Also per-cpu array caches are allocated for each possible cpu, so not affected by their online/offline state. In SLUB we have for_each_online_cpu() in functions that show statistics and are already unprotected today, as racing with hotplug is not harmful. Otherwise SLUB relies on percpu allocator. The slub_cpu_dead() hotplug callback takes the slab_mutex. To sum up, this patch removes get/put_online_cpus() calls from slab as it should be safe without further adjustments. Link: https://lkml.kernel.org/r/20210113131634.3671-4-vbabka@suse.cz Signed-off-by: Vlastimil Babka Cc: Christoph Lameter Cc: David Hildenbrand Cc: David Rientjes Cc: Joonsoo Kim Cc: Michal Hocko Cc: Pekka Enberg Cc: Qian Cai Cc: Vladimir Davydov Signed-off-by: Andrew Morton --- mm/slab_common.c | 10 ---------- 1 file changed, 10 deletions(-) --- a/mm/slab_common.c~mm-slab-slub-stop-taking-cpu-hotplug-lock +++ a/mm/slab_common.c @@ -309,8 +309,6 @@ kmem_cache_create_usercopy(const char *n const char *cache_name; int err; - get_online_cpus(); - mutex_lock(&slab_mutex); err = kmem_cache_sanity_check(name, size); @@ -359,8 +357,6 @@ kmem_cache_create_usercopy(const char *n out_unlock: mutex_unlock(&slab_mutex); - put_online_cpus(); - if (err) { if (flags & SLAB_PANIC) panic("kmem_cache_create: Failed to create slab '%s'. Error %d\n", @@ -484,8 +480,6 @@ void kmem_cache_destroy(struct kmem_cach if (unlikely(!s)) return; - get_online_cpus(); - mutex_lock(&slab_mutex); s->refcount--; @@ -500,8 +494,6 @@ void kmem_cache_destroy(struct kmem_cach } out_unlock: mutex_unlock(&slab_mutex); - - put_online_cpus(); } EXPORT_SYMBOL(kmem_cache_destroy); @@ -518,12 +510,10 @@ int kmem_cache_shrink(struct kmem_cache { int ret; - get_online_cpus(); kasan_cache_shrink(cachep); ret = __kmem_cache_shrink(cachep); - put_online_cpus(); return ret; } EXPORT_SYMBOL(kmem_cache_shrink); From patchwork Wed Feb 24 20:01:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3EA43C433E9 for ; Wed, 24 Feb 2021 20:01:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C669C64DF3 for ; Wed, 24 Feb 2021 20:01:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C669C64DF3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5FAB06B006C; Wed, 24 Feb 2021 15:01:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 587696B0070; Wed, 24 Feb 2021 15:01:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3FEBB6B0085; Wed, 24 Feb 2021 15:01:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0182.hostedemail.com [216.40.44.182]) by kanga.kvack.org (Postfix) with ESMTP id 244756B006C for ; Wed, 24 Feb 2021 15:01:27 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id E1FDC82B176A for ; Wed, 24 Feb 2021 20:01:26 +0000 (UTC) X-FDA: 77854230972.04.CB7DB9C Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP id 3E89290009F8 for ; Wed, 24 Feb 2021 20:01:22 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C946D64F27; Wed, 24 Feb 2021 20:01:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196880; bh=jJmBw70Q4ZmcE00zu4Pf1un0MifT4w1jTt1CpNzF+Ew=; h=Date:From:To:Subject:In-Reply-To:From; b=xWu6QaUQY9OnoibF1/q20w8jpMApfD41tmLHFMUR2+tfLAJIxlHa8jJw4ZnJ3jCME CGty2LpA4YB9OBM4O5aDJHQWJaORuj2c1cMHTQs0fEYgJxT7dqb0iZvVsyvhr/vOrs AOxaCYd7qFCmVpj9a7DqhWtWhF4MqTRL/eskSULg= Date: Wed, 24 Feb 2021 12:01:19 -0800 From: Andrew Morton To: akpm@linux-foundation.org, cl@linux.com, iamjoonsoo.kim@lge.com, jannh@google.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, penberg@kernel.org, rientjes@google.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 021/173] mm, slub: splice cpu and page freelists in deactivate_slab() Message-ID: <20210224200119.Ytx7rYOAU%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 7up78k8ziup5kwsrq7nbam31yme4micc X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 3E89290009F8 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf19; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196882-548383 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vlastimil Babka Subject: mm, slub: splice cpu and page freelists in deactivate_slab() In deactivate_slab() we currently move all but one objects on the cpu freelist to the page freelist one by one using the costly cmpxchg_double() operation. Then we unfreeze the page while moving the last object on page freelist, with a final cmpxchg_double(). This can be optimized to avoid the cmpxchg_double() per object. Just count the objects on cpu freelist (to adjust page->inuse properly) and also remember the last object in the chain. Then splice page->freelist to the last object and effectively add the whole cpu freelist to page->freelist while unfreezing the page, with a single cmpxchg_double(). Link: https://lkml.kernel.org/r/20210115183543.15097-1-vbabka@suse.cz Signed-off-by: Vlastimil Babka Reviewed-by: Jann Horn Cc: Christoph Lameter Cc: Pekka Enberg Cc: David Rientjes Cc: Joonsoo Kim Signed-off-by: Andrew Morton --- mm/slub.c | 59 +++++++++++++++++++++------------------------------- 1 file changed, 24 insertions(+), 35 deletions(-) --- a/mm/slub.c~mm-slub-splice-cpu-and-page-freelists-in-deactivate_slab +++ a/mm/slub.c @@ -2167,9 +2167,9 @@ static void deactivate_slab(struct kmem_ { enum slab_modes { M_NONE, M_PARTIAL, M_FULL, M_FREE }; struct kmem_cache_node *n = get_node(s, page_to_nid(page)); - int lock = 0; + int lock = 0, free_delta = 0; enum slab_modes l = M_NONE, m = M_NONE; - void *nextfree; + void *nextfree, *freelist_iter, *freelist_tail; int tail = DEACTIVATE_TO_HEAD; struct page new; struct page old; @@ -2180,45 +2180,34 @@ static void deactivate_slab(struct kmem_ } /* - * Stage one: Free all available per cpu objects back - * to the page freelist while it is still frozen. Leave the - * last one. - * - * There is no need to take the list->lock because the page - * is still frozen. + * Stage one: Count the objects on cpu's freelist as free_delta and + * remember the last object in freelist_tail for later splicing. */ - while (freelist && (nextfree = get_freepointer(s, freelist))) { - void *prior; - unsigned long counters; + freelist_tail = NULL; + freelist_iter = freelist; + while (freelist_iter) { + nextfree = get_freepointer(s, freelist_iter); /* * If 'nextfree' is invalid, it is possible that the object at - * 'freelist' is already corrupted. So isolate all objects - * starting at 'freelist'. + * 'freelist_iter' is already corrupted. So isolate all objects + * starting at 'freelist_iter' by skipping them. */ - if (freelist_corrupted(s, page, &freelist, nextfree)) + if (freelist_corrupted(s, page, &freelist_iter, nextfree)) break; - do { - prior = page->freelist; - counters = page->counters; - set_freepointer(s, freelist, prior); - new.counters = counters; - new.inuse--; - VM_BUG_ON(!new.frozen); - - } while (!__cmpxchg_double_slab(s, page, - prior, counters, - freelist, new.counters, - "drain percpu freelist")); + freelist_tail = freelist_iter; + free_delta++; - freelist = nextfree; + freelist_iter = nextfree; } /* - * Stage two: Ensure that the page is unfrozen while the - * list presence reflects the actual number of objects - * during unfreeze. + * Stage two: Unfreeze the page while splicing the per-cpu + * freelist to the head of page's freelist. + * + * Ensure that the page is unfrozen while the list presence + * reflects the actual number of objects during unfreeze. * * We setup the list membership and then perform a cmpxchg * with the count. If there is a mismatch then the page @@ -2231,15 +2220,15 @@ static void deactivate_slab(struct kmem_ */ redo: - old.freelist = page->freelist; - old.counters = page->counters; + old.freelist = READ_ONCE(page->freelist); + old.counters = READ_ONCE(page->counters); VM_BUG_ON(!old.frozen); /* Determine target state of the slab */ new.counters = old.counters; - if (freelist) { - new.inuse--; - set_freepointer(s, freelist, old.freelist); + if (freelist_tail) { + new.inuse -= free_delta; + set_freepointer(s, freelist_tail, old.freelist); new.freelist = freelist; } else new.freelist = old.freelist; From patchwork Wed Feb 24 20:01:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E492C433E0 for ; Wed, 24 Feb 2021 20:01:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 15B3A64F2C for ; Wed, 24 Feb 2021 20:01:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 15B3A64F2C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AB63D6B0006; Wed, 24 Feb 2021 15:01:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A65666B006C; Wed, 24 Feb 2021 15:01:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 92DE06B0085; Wed, 24 Feb 2021 15:01:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0039.hostedemail.com [216.40.44.39]) by kanga.kvack.org (Postfix) with ESMTP id 7D66A6B0006 for ; Wed, 24 Feb 2021 15:01:25 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 3D22BEFFB for ; Wed, 24 Feb 2021 20:01:25 +0000 (UTC) X-FDA: 77854230930.25.6AFFF34 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id CA4C83517 for ; Wed, 24 Feb 2021 20:01:18 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 40E4A64F2A; Wed, 24 Feb 2021 20:01:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196883; bh=O+gcSV7IoWq9TpjxG246UONGWhXkk6udUtmKMd8G5mE=; h=Date:From:To:Subject:In-Reply-To:From; b=pw/WQOneQ45Jmyk684TlbPAcIHY5FnsfrFvVbPgyfm0/DakFPllSDnSyUEKyfC+iz yiLM5fyQUWV55oaYV86cBsyOIFdYnHkXGZniwRhc+uclWeJH1OHItYqix1hbjh/dMD 3eDphOOmiwpkdVMxvwjeEVjeeMWDEVYWDc3ho+rQ= Date: Wed, 24 Feb 2021 12:01:22 -0800 From: Andrew Morton To: akpm@linux-foundation.org, cl@linux.com, david@redhat.com, guro@fb.com, iamjoonsoo.kim@lge.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, penberg@kernel.org, rientjes@google.com, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 022/173] mm, slub: remove slub_memcg_sysfs boot param and CONFIG_SLUB_MEMCG_SYSFS_ON Message-ID: <20210224200122.rWHC1lQt0%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: okhhnupgn8m9iqek3r43xiinhrxj34kn X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: CA4C83517 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf12; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196878-291822 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vlastimil Babka Subject: mm, slub: remove slub_memcg_sysfs boot param and CONFIG_SLUB_MEMCG_SYSFS_ON The boot param and config determine the value of memcg_sysfs_enabled, which is unused since commit 10befea91b61 ("mm: memcg/slab: use a single set of kmem_caches for all allocations") as there are no per-memcg kmem caches anymore. Link: https://lkml.kernel.org/r/20210127124745.7928-1-vbabka@suse.cz Signed-off-by: Vlastimil Babka Reviewed-by: David Hildenbrand Acked-by: Roman Gushchin Acked-by: David Rientjes Reviewed-by: Miaohe Lin Cc: Christoph Lameter Cc: Pekka Enberg Cc: Joonsoo Kim Signed-off-by: Andrew Morton --- Documentation/admin-guide/kernel-parameters.txt | 8 ------- init/Kconfig | 14 ------------ mm/slub.c | 16 -------------- 3 files changed, 38 deletions(-) --- a/Documentation/admin-guide/kernel-parameters.txt~mm-slub-remove-slub_memcg_sysfs-boot-param-and-config_slub_memcg_sysfs_on +++ a/Documentation/admin-guide/kernel-parameters.txt @@ -4899,14 +4899,6 @@ last alloc / free. For more information see Documentation/vm/slub.rst. - slub_memcg_sysfs= [MM, SLUB] - Determines whether to enable sysfs directories for - memory cgroup sub-caches. 1 to enable, 0 to disable. - The default is determined by CONFIG_SLUB_MEMCG_SYSFS_ON. - Enabling this can lead to a very high number of debug - directories and files being created under - /sys/kernel/slub. - slub_max_order= [MM, SLUB] Determines the maximum allowed order for slabs. A high setting may cause OOMs due to memory --- a/init/Kconfig~mm-slub-remove-slub_memcg_sysfs-boot-param-and-config_slub_memcg_sysfs_on +++ a/init/Kconfig @@ -1861,20 +1861,6 @@ config SLUB_DEBUG SLUB sysfs support. /sys/slab will not exist and there will be no support for cache validation etc. -config SLUB_MEMCG_SYSFS_ON - default n - bool "Enable memcg SLUB sysfs support by default" if EXPERT - depends on SLUB && SYSFS && MEMCG - help - SLUB creates a directory under /sys/kernel/slab for each - allocation cache to host info and debug files. If memory - cgroup is enabled, each cache can have per memory cgroup - caches. SLUB can create the same sysfs directories for these - caches under /sys/kernel/slab/CACHE/cgroup but it can lead - to a very high number of debug files being created. This is - controlled by slub_memcg_sysfs boot parameter and this - config option determines the parameter's default value. - config COMPAT_BRK bool "Disable heap randomization" default y --- a/mm/slub.c~mm-slub-remove-slub_memcg_sysfs-boot-param-and-config_slub_memcg_sysfs_on +++ a/mm/slub.c @@ -4943,22 +4943,6 @@ enum slab_stat_type { #define SO_OBJECTS (1 << SL_OBJECTS) #define SO_TOTAL (1 << SL_TOTAL) -#ifdef CONFIG_MEMCG -static bool memcg_sysfs_enabled = IS_ENABLED(CONFIG_SLUB_MEMCG_SYSFS_ON); - -static int __init setup_slub_memcg_sysfs(char *str) -{ - int v; - - if (get_option(&str, &v) > 0) - memcg_sysfs_enabled = v; - - return 1; -} - -__setup("slub_memcg_sysfs=", setup_slub_memcg_sysfs); -#endif - static ssize_t show_slab_objects(struct kmem_cache *s, char *buf, unsigned long flags) { From patchwork Wed Feb 24 20:01:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 823F5C433E0 for ; Wed, 24 Feb 2021 20:01:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1FA4464E2B for ; Wed, 24 Feb 2021 20:01:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1FA4464E2B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BC9106B0070; Wed, 24 Feb 2021 15:01:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B76CC6B0085; Wed, 24 Feb 2021 15:01:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A41E56B0087; Wed, 24 Feb 2021 15:01:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0157.hostedemail.com [216.40.44.157]) by kanga.kvack.org (Postfix) with ESMTP id 8DFAD6B0070 for ; Wed, 24 Feb 2021 15:01:28 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 3866AF004 for ; Wed, 24 Feb 2021 20:01:28 +0000 (UTC) X-FDA: 77854231056.20.22061C2 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf10.hostedemail.com (Postfix) with ESMTP id 7A70940B8CDF for ; Wed, 24 Feb 2021 20:01:22 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 833DA64F2B; Wed, 24 Feb 2021 20:01:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196886; bh=N2W9qplJMuNdg0y15XwwMPC2Ssu4JtA+NY3o9xSBvoA=; h=Date:From:To:Subject:In-Reply-To:From; b=q957SYxiQ0X2k8dtvnLMNgdP1YwqynVQ6ZnGxhBm5lHpk8oRKAigO5uuuO0tVqbYI v3JFObP6bjMZxp8bYfPD2nzGIANryESHD0dcmrTbiy+JjoL/lCFh8WAmfW4qlyvuNv IKAMUfI2NOAw5yhQxYMEAMLQXcosGfkDhhMtHcOY= Date: Wed, 24 Feb 2021 12:01:26 -0800 From: Andrew Morton To: akpm@linux-foundation.org, daizhiyuan@phytium.com.cn, linux-mm@kvack.org, mm-commits@vger.kernel.org, rientjes@google.com, torvalds@linux-foundation.org Subject: [patch 023/173] mm/slub: minor coding style tweaks Message-ID: <20210224200126.NFkZDiLL2%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7A70940B8CDF X-Stat-Signature: rbajx9i3mdyebh8hpkh9pmex7b7g99jt Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf10; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196882-421244 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zhiyuan Dai Subject: mm/slub: minor coding style tweaks Add whitespace to fix coding style issues, improve code reading. Link: https://lkml.kernel.org/r/1612847403-5594-1-git-send-email-daizhiyuan@phytium.com.cn Signed-off-by: Zhiyuan Dai Acked-by: David Rientjes Signed-off-by: Andrew Morton --- mm/slub.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/slub.c~mm-slub-minor-coding-style-tweaks +++ a/mm/slub.c @@ -3269,7 +3269,7 @@ void kmem_cache_free_bulk(struct kmem_ca if (!df.page) continue; - slab_free(df.s, df.page, df.freelist, df.tail, df.cnt,_RET_IP_); + slab_free(df.s, df.page, df.freelist, df.tail, df.cnt, _RET_IP_); } while (likely(size)); } EXPORT_SYMBOL(kmem_cache_free_bulk); From patchwork Wed Feb 24 20:01:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C2CCC433E0 for ; Wed, 24 Feb 2021 20:01:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id ABC9B64E09 for ; Wed, 24 Feb 2021 20:01:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ABC9B64E09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D75316B0071; Wed, 24 Feb 2021 15:01:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D24856B0085; Wed, 24 Feb 2021 15:01:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC5FE6B0087; Wed, 24 Feb 2021 15:01:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0083.hostedemail.com [216.40.44.83]) by kanga.kvack.org (Postfix) with ESMTP id A44EE6B0071 for ; Wed, 24 Feb 2021 15:01:31 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 6D337F017 for ; Wed, 24 Feb 2021 20:01:31 +0000 (UTC) X-FDA: 77854231182.15.D05A4D3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf21.hostedemail.com (Postfix) with ESMTP id A9AE7E0011DD for ; Wed, 24 Feb 2021 20:01:27 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id AE9B864E20; Wed, 24 Feb 2021 20:01:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196890; bh=toh4JbE/mFziqaT8vghw6ctptenKDfbrWau1XAqpAkk=; h=Date:From:To:Subject:In-Reply-To:From; b=1I5XAof1vqyjwy2NSLCMOnqiYWzLbsdTts2qjLc95SSguL9cuLiAEIaM8tj63RKF7 bfpy5lOy77laJpmTfVH+VfeCc1BH+e5xztBmpOp8ppoywy+yJ141Jst439lPFAa96A Jh87IYRAlUDJhbEiLkfz1cD6Bd++kkelGOUnAcho= Date: Wed, 24 Feb 2021 12:01:29 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org, ziy@nvidia.com Subject: [patch 024/173] mm/debug: improve memcg debugging Message-ID: <20210224200129.GAA7qjsYK%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: A9AE7E0011DD X-Stat-Signature: ifc8fogkt9ig4a3chh6qsz1u74kzt48f Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf21; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196887-111265 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/debug: improve memcg debugging The memcg_data is only valid on the head page, not the tail pages. Change the format and location of the printout within the dump to match the other parts of struct page better. Link: https://lkml.kernel.org/r/20210114190200.1894484-1-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Zi Yan Cc: Michal Hocko Cc: Johannes Weiner Signed-off-by: Andrew Morton --- mm/debug.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) --- a/mm/debug.c~mm-debug-improve-memcg-debugging +++ a/mm/debug.c @@ -110,6 +110,11 @@ void __dump_page(struct page *page, cons head_compound_mapcount(head)); } } + +#ifdef CONFIG_MEMCG + if (head->memcg_data) + pr_warn("memcg:%lx\n", head->memcg_data); +#endif if (PageKsm(page)) type = "ksm "; else if (PageAnon(page)) @@ -180,11 +185,6 @@ hex_only: if (reason) pr_warn("page dumped because: %s\n", reason); - -#ifdef CONFIG_MEMCG - if (!page_poisoned && page->memcg_data) - pr_warn("pages's memcg:%lx\n", page->memcg_data); -#endif } void dump_page(struct page *page, const char *reason) From patchwork Wed Feb 24 20:01:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 686FDC433DB for ; Wed, 24 Feb 2021 20:01:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 053F564E2B for ; Wed, 24 Feb 2021 20:01:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 053F564E2B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 86A566B0085; Wed, 24 Feb 2021 15:01:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7FA966B0087; Wed, 24 Feb 2021 15:01:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 699F86B0088; Wed, 24 Feb 2021 15:01:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0079.hostedemail.com [216.40.44.79]) by kanga.kvack.org (Postfix) with ESMTP id 4FA4D6B0085 for ; Wed, 24 Feb 2021 15:01:35 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 11894185044AB for ; Wed, 24 Feb 2021 20:01:35 +0000 (UTC) X-FDA: 77854231350.24.2C8E2B0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf15.hostedemail.com (Postfix) with ESMTP id 5D1C2A000534 for ; Wed, 24 Feb 2021 20:01:33 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0F87564E24; Wed, 24 Feb 2021 20:01:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196893; bh=zDnrIdeajgeCMj/o2TWxtTmXqzrAj4a9Lw3xt/MpVDc=; h=Date:From:To:Subject:In-Reply-To:From; b=oyhzIKXqqK4D4XpCLmw/qA6Mm5yKo9bbUs+A8ATj7CRXf5xqpmWMIULqUpgDkJg6b X09Xh/tONmmUsQ9Y4R8tyQg6KLtlfosmc8PgLTyE/uSFaHiXZmmyOU1/e3VGGs7Yy2 N9Ut3Z78p6lKwUWWLTZH+VV3HxGsU+6WC/adIZss= Date: Wed, 24 Feb 2021 12:01:32 -0800 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, catalin.marinas@arm.com, christophe.leroy@csgroup.eu, gerald.schaefer@de.ibm.com, gerald.schaefer@linux.ibm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, paul.walmsley@sifive.com, steven.price@arm.com, torvalds@linux-foundation.org, vgupta@synopsys.com Subject: [patch 025/173] mm/debug_vm_pgtable/basic: add validation for dirtiness after write protect Message-ID: <20210224200132.9dJyRZOkT%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 5D1C2A000534 X-Stat-Signature: xkjyachwd4eouhyhpgu8zjm1a7txpim7 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf15; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196893-450666 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Anshuman Khandual Subject: mm/debug_vm_pgtable/basic: add validation for dirtiness after write protect Patch series "mm/debug_vm_pgtable: Some minor updates", v3. This series contains some cleanups and new test suggestions from Catalin from an earlier discussion. https://lore.kernel.org/linux-mm/20201123142237.GF17833@gaia/ This patch (of 2): This adds validation tests for dirtiness after write protect conversion for each page table level. There are two new separate test types involved here. The first test ensures that a given page table entry does not become dirty after pxx_wrprotect(). This is important for platforms like arm64 which transfers and drops the hardware dirty bit (!PTE_RDONLY) to the software dirty bit while making it an write protected one. This test ensures that no fresh page table entry could be created with hardware dirty bit set. The second test ensures that a given page table entry always preserve the dirty information across pxx_wrprotect(). This adds two previously missing PUD level basic tests and while here fixes pxx_wrprotect() related typos in the documentation file. Link: https://lkml.kernel.org/r/1611137241-26220-1-git-send-email-anshuman.khandual@arm.com Link: https://lkml.kernel.org/r/1611137241-26220-2-git-send-email-anshuman.khandual@arm.com Signed-off-by: Anshuman Khandual Suggested-by: Catalin Marinas Tested-by: Gerald Schaefer [s390] Cc: Christophe Leroy Cc: Gerald Schaefer Cc: Vineet Gupta Cc: Paul Walmsley Cc: Steven Price Signed-off-by: Andrew Morton --- Documentation/vm/arch_pgtable_helpers.rst | 8 ++-- mm/debug_vm_pgtable.c | 39 ++++++++++++++++++++ 2 files changed, 43 insertions(+), 4 deletions(-) --- a/Documentation/vm/arch_pgtable_helpers.rst~mm-debug_vm_pgtable-basic-add-validation-for-dirtiness-after-write-protect +++ a/Documentation/vm/arch_pgtable_helpers.rst @@ -50,7 +50,7 @@ PTE Page Table Helpers +---------------------------+--------------------------------------------------+ | pte_mkwrite | Creates a writable PTE | +---------------------------+--------------------------------------------------+ -| pte_mkwrprotect | Creates a write protected PTE | +| pte_wrprotect | Creates a write protected PTE | +---------------------------+--------------------------------------------------+ | pte_mkspecial | Creates a special PTE | +---------------------------+--------------------------------------------------+ @@ -120,7 +120,7 @@ PMD Page Table Helpers +---------------------------+--------------------------------------------------+ | pmd_mkwrite | Creates a writable PMD | +---------------------------+--------------------------------------------------+ -| pmd_mkwrprotect | Creates a write protected PMD | +| pmd_wrprotect | Creates a write protected PMD | +---------------------------+--------------------------------------------------+ | pmd_mkspecial | Creates a special PMD | +---------------------------+--------------------------------------------------+ @@ -186,7 +186,7 @@ PUD Page Table Helpers +---------------------------+--------------------------------------------------+ | pud_mkwrite | Creates a writable PUD | +---------------------------+--------------------------------------------------+ -| pud_mkwrprotect | Creates a write protected PUD | +| pud_wrprotect | Creates a write protected PUD | +---------------------------+--------------------------------------------------+ | pud_mkdevmap | Creates a ZONE_DEVICE mapped PUD | +---------------------------+--------------------------------------------------+ @@ -224,7 +224,7 @@ HugeTLB Page Table Helpers +---------------------------+--------------------------------------------------+ | huge_pte_mkwrite | Creates a writable HugeTLB | +---------------------------+--------------------------------------------------+ -| huge_pte_mkwrprotect | Creates a write protected HugeTLB | +| huge_pte_wrprotect | Creates a write protected HugeTLB | +---------------------------+--------------------------------------------------+ | huge_ptep_get_and_clear | Clears a HugeTLB | +---------------------------+--------------------------------------------------+ --- a/mm/debug_vm_pgtable.c~mm-debug_vm_pgtable-basic-add-validation-for-dirtiness-after-write-protect +++ a/mm/debug_vm_pgtable.c @@ -63,6 +63,16 @@ static void __init pte_basic_tests(unsig pte_t pte = pfn_pte(pfn, prot); pr_debug("Validating PTE basic\n"); + + /* + * This test needs to be executed after the given page table entry + * is created with pfn_pte() to make sure that protection_map[idx] + * does not have the dirty bit enabled from the beginning. This is + * important for platforms like arm64 where (!PTE_RDONLY) indicate + * dirty bit being set. + */ + WARN_ON(pte_dirty(pte_wrprotect(pte))); + WARN_ON(!pte_same(pte, pte)); WARN_ON(!pte_young(pte_mkyoung(pte_mkold(pte)))); WARN_ON(!pte_dirty(pte_mkdirty(pte_mkclean(pte)))); @@ -70,6 +80,8 @@ static void __init pte_basic_tests(unsig WARN_ON(pte_young(pte_mkold(pte_mkyoung(pte)))); WARN_ON(pte_dirty(pte_mkclean(pte_mkdirty(pte)))); WARN_ON(pte_write(pte_wrprotect(pte_mkwrite(pte)))); + WARN_ON(pte_dirty(pte_wrprotect(pte_mkclean(pte)))); + WARN_ON(!pte_dirty(pte_wrprotect(pte_mkdirty(pte)))); } static void __init pte_advanced_tests(struct mm_struct *mm, @@ -137,6 +149,17 @@ static void __init pmd_basic_tests(unsig return; pr_debug("Validating PMD basic\n"); + + /* + * This test needs to be executed after the given page table entry + * is created with pfn_pmd() to make sure that protection_map[idx] + * does not have the dirty bit enabled from the beginning. This is + * important for platforms like arm64 where (!PTE_RDONLY) indicate + * dirty bit being set. + */ + WARN_ON(pmd_dirty(pmd_wrprotect(pmd))); + + WARN_ON(!pmd_same(pmd, pmd)); WARN_ON(!pmd_young(pmd_mkyoung(pmd_mkold(pmd)))); WARN_ON(!pmd_dirty(pmd_mkdirty(pmd_mkclean(pmd)))); @@ -144,6 +167,8 @@ static void __init pmd_basic_tests(unsig WARN_ON(pmd_young(pmd_mkold(pmd_mkyoung(pmd)))); WARN_ON(pmd_dirty(pmd_mkclean(pmd_mkdirty(pmd)))); WARN_ON(pmd_write(pmd_wrprotect(pmd_mkwrite(pmd)))); + WARN_ON(pmd_dirty(pmd_wrprotect(pmd_mkclean(pmd)))); + WARN_ON(!pmd_dirty(pmd_wrprotect(pmd_mkdirty(pmd)))); /* * A huge page does not point to next level page table * entry. Hence this must qualify as pmd_bad(). @@ -257,11 +282,25 @@ static void __init pud_basic_tests(unsig return; pr_debug("Validating PUD basic\n"); + + /* + * This test needs to be executed after the given page table entry + * is created with pfn_pud() to make sure that protection_map[idx] + * does not have the dirty bit enabled from the beginning. This is + * important for platforms like arm64 where (!PTE_RDONLY) indicate + * dirty bit being set. + */ + WARN_ON(pud_dirty(pud_wrprotect(pud))); + WARN_ON(!pud_same(pud, pud)); WARN_ON(!pud_young(pud_mkyoung(pud_mkold(pud)))); + WARN_ON(!pud_dirty(pud_mkdirty(pud_mkclean(pud)))); + WARN_ON(pud_dirty(pud_mkclean(pud_mkdirty(pud)))); WARN_ON(!pud_write(pud_mkwrite(pud_wrprotect(pud)))); WARN_ON(pud_write(pud_wrprotect(pud_mkwrite(pud)))); WARN_ON(pud_young(pud_mkold(pud_mkyoung(pud)))); + WARN_ON(pud_dirty(pud_wrprotect(pud_mkclean(pud)))); + WARN_ON(!pud_dirty(pud_wrprotect(pud_mkdirty(pud)))); if (mm_pmd_folded(mm)) return; From patchwork Wed Feb 24 20:01:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102461 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E2B1C433E0 for ; Wed, 24 Feb 2021 20:01:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A66F764E60 for ; Wed, 24 Feb 2021 20:01:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A66F764E60 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3B0696B0087; Wed, 24 Feb 2021 15:01:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 339246B0088; Wed, 24 Feb 2021 15:01:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 200F56B0089; Wed, 24 Feb 2021 15:01:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0071.hostedemail.com [216.40.44.71]) by kanga.kvack.org (Postfix) with ESMTP id 0A0DA6B0087 for ; Wed, 24 Feb 2021 15:01:39 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id C8FA4F007 for ; Wed, 24 Feb 2021 20:01:38 +0000 (UTC) X-FDA: 77854231476.08.EE05FCD Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP id 79185A000503 for ; Wed, 24 Feb 2021 20:01:37 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8E33564E2B; Wed, 24 Feb 2021 20:01:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196897; bh=iyqlcazRZhVYI57v2Zx9aYSd+L+Iv187sM8HOlA8XAY=; h=Date:From:To:Subject:In-Reply-To:From; b=PhbPM7szZXniPW2hv7OwpQiAotztIUjM+68K6mmIbiS34AUi/sr+1R7XxU114o/EV m3rduAMduArjGqX1xZXBxUl9hU1DZ4SmSDt0x4KOx1ktmnXr/d6WXHNeWlCpLhVD16 pMoILaK+6mJDmoVS+xKegdzl6bcAWb/N8AawvPFs= Date: Wed, 24 Feb 2021 12:01:36 -0800 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, catalin.marinas@arm.com, christophe.leroy@csgroup.eu, gerald.schaefer@de.ibm.com, gerald.schaefer@linux.ibm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, paul.walmsley@sifive.com, steven.price@arm.com, torvalds@linux-foundation.org, vgupta@synopsys.com Subject: [patch 026/173] mm/debug_vm_pgtable/basic: iterate over entire protection_map[] Message-ID: <20210224200136.SOwEQBEbR%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: r1ah9j3szor5jph6a3j6kowdkr7ptbmj X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 79185A000503 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf07; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196897-189496 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Anshuman Khandual Subject: mm/debug_vm_pgtable/basic: iterate over entire protection_map[] Currently the basic tests just validate various page table transformations after starting with vm_get_page_prot(VM_READ|VM_WRITE|VM_EXEC) protection. Instead scan over the entire protection_map[] for better coverage. It also makes sure that all these basic page table tranformations checks hold true irrespective of the starting protection value for the page table entry. There is also a slight change in the debug print format for basic tests to capture the protection value it is being tested with. The modified output looks something like [pte_basic_tests ]: Validating PTE basic () [pte_basic_tests ]: Validating PTE basic (read) [pte_basic_tests ]: Validating PTE basic (write) [pte_basic_tests ]: Validating PTE basic (read|write) [pte_basic_tests ]: Validating PTE basic (exec) [pte_basic_tests ]: Validating PTE basic (read|exec) [pte_basic_tests ]: Validating PTE basic (write|exec) [pte_basic_tests ]: Validating PTE basic (read|write|exec) [pte_basic_tests ]: Validating PTE basic (shared) [pte_basic_tests ]: Validating PTE basic (read|shared) [pte_basic_tests ]: Validating PTE basic (write|shared) [pte_basic_tests ]: Validating PTE basic (read|write|shared) [pte_basic_tests ]: Validating PTE basic (exec|shared) [pte_basic_tests ]: Validating PTE basic (read|exec|shared) [pte_basic_tests ]: Validating PTE basic (write|exec|shared) [pte_basic_tests ]: Validating PTE basic (read|write|exec|shared) This adds a missing argument 'struct mm_struct *' in pud_basic_tests() test . This never got exposed before as PUD based THP is available only on X86 platform where mm_pmd_folded(mm) call gets macro replaced without requiring the mm_struct i.e __is_defined(__PAGETABLE_PMD_FOLDED). Link: https://lkml.kernel.org/r/1611137241-26220-3-git-send-email-anshuman.khandual@arm.com Signed-off-by: Anshuman Khandual Tested-by: Gerald Schaefer [s390] Reviewed-by: Steven Price Suggested-by: Catalin Marinas Cc: Christophe Leroy Cc: Gerald Schaefer Cc: Paul Walmsley Cc: Vineet Gupta Signed-off-by: Andrew Morton --- mm/debug_vm_pgtable.c | 47 +++++++++++++++++++++++++++++----------- 1 file changed, 35 insertions(+), 12 deletions(-) --- a/mm/debug_vm_pgtable.c~mm-debug_vm_pgtable-basic-iterate-over-entire-protection_map +++ a/mm/debug_vm_pgtable.c @@ -58,11 +58,13 @@ #define RANDOM_ORVALUE (GENMASK(BITS_PER_LONG - 1, 0) & ~ARCH_SKIP_MASK) #define RANDOM_NZVALUE GENMASK(7, 0) -static void __init pte_basic_tests(unsigned long pfn, pgprot_t prot) +static void __init pte_basic_tests(unsigned long pfn, int idx) { + pgprot_t prot = protection_map[idx]; pte_t pte = pfn_pte(pfn, prot); + unsigned long val = idx, *ptr = &val; - pr_debug("Validating PTE basic\n"); + pr_debug("Validating PTE basic (%pGv)\n", ptr); /* * This test needs to be executed after the given page table entry @@ -141,14 +143,16 @@ static void __init pte_savedwrite_tests( } #ifdef CONFIG_TRANSPARENT_HUGEPAGE -static void __init pmd_basic_tests(unsigned long pfn, pgprot_t prot) +static void __init pmd_basic_tests(unsigned long pfn, int idx) { + pgprot_t prot = protection_map[idx]; pmd_t pmd = pfn_pmd(pfn, prot); + unsigned long val = idx, *ptr = &val; if (!has_transparent_hugepage()) return; - pr_debug("Validating PMD basic\n"); + pr_debug("Validating PMD basic (%pGv)\n", ptr); /* * This test needs to be executed after the given page table entry @@ -274,14 +278,16 @@ static void __init pmd_savedwrite_tests( } #ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD -static void __init pud_basic_tests(unsigned long pfn, pgprot_t prot) +static void __init pud_basic_tests(struct mm_struct *mm, unsigned long pfn, int idx) { + pgprot_t prot = protection_map[idx]; pud_t pud = pfn_pud(pfn, prot); + unsigned long val = idx, *ptr = &val; if (!has_transparent_hugepage()) return; - pr_debug("Validating PUD basic\n"); + pr_debug("Validating PUD basic (%pGv)\n", ptr); /* * This test needs to be executed after the given page table entry @@ -398,7 +404,7 @@ static void __init pud_huge_tests(pud_t #endif /* !CONFIG_HAVE_ARCH_HUGE_VMAP */ #else /* !CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */ -static void __init pud_basic_tests(unsigned long pfn, pgprot_t prot) { } +static void __init pud_basic_tests(struct mm_struct *mm, unsigned long pfn, int idx) { } static void __init pud_advanced_tests(struct mm_struct *mm, struct vm_area_struct *vma, pud_t *pudp, unsigned long pfn, unsigned long vaddr, @@ -411,8 +417,8 @@ static void __init pud_huge_tests(pud_t } #endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */ #else /* !CONFIG_TRANSPARENT_HUGEPAGE */ -static void __init pmd_basic_tests(unsigned long pfn, pgprot_t prot) { } -static void __init pud_basic_tests(unsigned long pfn, pgprot_t prot) { } +static void __init pmd_basic_tests(unsigned long pfn, int idx) { } +static void __init pud_basic_tests(struct mm_struct *mm, unsigned long pfn, int idx) { } static void __init pmd_advanced_tests(struct mm_struct *mm, struct vm_area_struct *vma, pmd_t *pmdp, unsigned long pfn, unsigned long vaddr, @@ -938,6 +944,7 @@ static int __init debug_vm_pgtable(void) unsigned long vaddr, pte_aligned, pmd_aligned; unsigned long pud_aligned, p4d_aligned, pgd_aligned; spinlock_t *ptl = NULL; + int idx; pr_info("Validating architecture page table helpers\n"); prot = vm_get_page_prot(VMFLAGS); @@ -1002,9 +1009,25 @@ static int __init debug_vm_pgtable(void) saved_pmdp = pmd_offset(pudp, 0UL); saved_ptep = pmd_pgtable(pmd); - pte_basic_tests(pte_aligned, prot); - pmd_basic_tests(pmd_aligned, prot); - pud_basic_tests(pud_aligned, prot); + /* + * Iterate over the protection_map[] to make sure that all + * the basic page table transformation validations just hold + * true irrespective of the starting protection value for a + * given page table entry. + */ + for (idx = 0; idx < ARRAY_SIZE(protection_map); idx++) { + pte_basic_tests(pte_aligned, idx); + pmd_basic_tests(pmd_aligned, idx); + pud_basic_tests(mm, pud_aligned, idx); + } + + /* + * Both P4D and PGD level tests are very basic which do not + * involve creating page table entries from the protection + * value and the given pfn. Hence just keep them out from + * the above iteration for now to save some test execution + * time. + */ p4d_basic_tests(p4d_aligned, prot); pgd_basic_tests(pgd_aligned, prot); From patchwork Wed Feb 24 20:01:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102465 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA093C433DB for ; Wed, 24 Feb 2021 20:01:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 70D2E64E7A for ; Wed, 24 Feb 2021 20:01:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 70D2E64E7A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 13FD26B0089; Wed, 24 Feb 2021 15:01:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0EF5E6B008A; Wed, 24 Feb 2021 15:01:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F226A6B008C; Wed, 24 Feb 2021 15:01:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0095.hostedemail.com [216.40.44.95]) by kanga.kvack.org (Postfix) with ESMTP id D34BB6B0089 for ; Wed, 24 Feb 2021 15:01:41 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 70141EFF2 for ; Wed, 24 Feb 2021 20:01:41 +0000 (UTC) X-FDA: 77854231602.29.296BD80 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id 56C652000DA5 for ; Wed, 24 Feb 2021 20:01:31 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id E812C64E09; Wed, 24 Feb 2021 20:01:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196900; bh=HztGbotrskxQl5YEUxl1erJfoM7EIYD5NNsM7KtS8xY=; h=Date:From:To:Subject:In-Reply-To:From; b=2r7H+2/4FpJpBFhJDHIWauRQ01dJvKczCP4jqVCZRF12mYCD9sfimyf6y9f9q17W9 DeOvJuojHTyaDiOj4B5nwhBZpzJfo4ev2lJUmQ0FkFvawz4Mnq4CgHYTcYE9Iqmfgl KlG0EEFBINis18cPiq0clQz7+0/Z4a7aAZ0QrCMo= Date: Wed, 24 Feb 2021 12:01:39 -0800 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 027/173] mm/page_owner: use helper function zone_end_pfn() to get end_pfn Message-ID: <20210224200139.gHZVOOVcn%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: wkrr5hph57ad77m4e7y4efjohjpy7gfg X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 56C652000DA5 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf11; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196891-13684 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/page_owner: use helper function zone_end_pfn() to get end_pfn Commit 108bcc96ef70 ("mm: add & use zone_end_pfn() and zone_spans_pfn()") introduced the helper zone_end_pfn() to calculate the zone end pfn. But pagetypeinfo_showmixedcount_print forgot to use it. And the initialization of local variable pfn is duplicated, remove one. Link: https://lkml.kernel.org/r/20210123070538.5861-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: Andrew Morton Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- mm/page_owner.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/mm/page_owner.c~mm-page_owner-use-helper-function-zone_end_pfn-to-get-end_pfn +++ a/mm/page_owner.c @@ -263,8 +263,8 @@ void pagetypeinfo_showmixedcount_print(s struct page *page; struct page_ext *page_ext; struct page_owner *page_owner; - unsigned long pfn = zone->zone_start_pfn, block_end_pfn; - unsigned long end_pfn = pfn + zone->spanned_pages; + unsigned long pfn, block_end_pfn; + unsigned long end_pfn = zone_end_pfn(zone); unsigned long count[MIGRATE_TYPES] = { 0, }; int pageblock_mt, page_mt; int i; From patchwork Wed Feb 24 20:01:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102467 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 333AEC433E6 for ; Wed, 24 Feb 2021 20:01:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CC44364E6C for ; Wed, 24 Feb 2021 20:01:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CC44364E6C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 33BDA6B008A; Wed, 24 Feb 2021 15:01:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2EBA76B008C; Wed, 24 Feb 2021 15:01:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 18F9E6B0092; Wed, 24 Feb 2021 15:01:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0126.hostedemail.com [216.40.44.126]) by kanga.kvack.org (Postfix) with ESMTP id EF6A46B008A for ; Wed, 24 Feb 2021 15:01:45 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id B957082C6222 for ; Wed, 24 Feb 2021 20:01:45 +0000 (UTC) X-FDA: 77854231770.20.1A33C95 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id B5E252000396 for ; Wed, 24 Feb 2021 20:01:34 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 1618764E60; Wed, 24 Feb 2021 20:01:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196903; bh=Gx2UYBXkLKplZFlSpc9Xu7tBsp3g7Fq+GrNge51Cqsg=; h=Date:From:To:Subject:In-Reply-To:From; b=Z85b3vtFhKgUpbwj5fuw71BjQtykRpL+418oZL5X5gFFGL0xNEKb7gArTSWofk5sv rBnt9RZ3sti89Bm0Aqyoz9Oc2yiuPY7zls3uj/mDgvMyhhcRgjJ03Dcnayf8HN+Btf 0sVfLORhQws6aNfKFwwzwfIq+QqXuml0EgkT3IVo= Date: Wed, 24 Feb 2021 12:01:42 -0800 From: Andrew Morton To: akpm@linux-foundation.org, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, miklos@szeredi.hu, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 028/173] mm/filemap: remove unused parameter and change to void type for replace_page_cache_page() Message-ID: <20210224200142.Kn7XyNig0%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: ucdrfoid34swu6w9qimdrcjpwej4owtq X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: B5E252000396 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf11; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196894-69212 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Baolin Wang Subject: mm/filemap: remove unused parameter and change to void type for replace_page_cache_page() Since commit 74d609585d8b ("page cache: Add and replace pages using the XArray") was merged, the replace_page_cache_page() can not fail and always return 0, we can remove the redundant return value and void it. Moreover remove the unused gfp_mask. Link: https://lkml.kernel.org/r/609c30e5274ba15d8b90c872fd0d8ac437a9b2bb.1610071401.git.baolin.wang@linux.alibaba.com Signed-off-by: Baolin Wang Cc: Matthew Wilcox Cc: Miklos Szeredi Signed-off-by: Andrew Morton --- fs/fuse/dev.c | 6 +----- include/linux/pagemap.h | 2 +- mm/filemap.c | 7 +------ 3 files changed, 3 insertions(+), 12 deletions(-) --- a/fs/fuse/dev.c~mm-filemap-remove-unused-parameter-and-change-to-void-type-for-replace_page_cache_page +++ a/fs/fuse/dev.c @@ -844,11 +844,7 @@ static int fuse_try_move_page(struct fus if (WARN_ON(PageMlocked(oldpage))) goto out_fallback_unlock; - err = replace_page_cache_page(oldpage, newpage, GFP_KERNEL); - if (err) { - unlock_page(newpage); - goto out_put_old; - } + replace_page_cache_page(oldpage, newpage); get_page(newpage); --- a/include/linux/pagemap.h~mm-filemap-remove-unused-parameter-and-change-to-void-type-for-replace_page_cache_page +++ a/include/linux/pagemap.h @@ -757,7 +757,7 @@ int add_to_page_cache_lru(struct page *p pgoff_t index, gfp_t gfp_mask); extern void delete_from_page_cache(struct page *page); extern void __delete_from_page_cache(struct page *page, void *shadow); -int replace_page_cache_page(struct page *old, struct page *new, gfp_t gfp_mask); +void replace_page_cache_page(struct page *old, struct page *new); void delete_from_page_cache_batch(struct address_space *mapping, struct pagevec *pvec); --- a/mm/filemap.c~mm-filemap-remove-unused-parameter-and-change-to-void-type-for-replace_page_cache_page +++ a/mm/filemap.c @@ -777,7 +777,6 @@ EXPORT_SYMBOL(file_write_and_wait_range) * replace_page_cache_page - replace a pagecache page with a new one * @old: page to be replaced * @new: page to replace with - * @gfp_mask: allocation mode * * This function replaces a page in the pagecache with a new one. On * success it acquires the pagecache reference for the new page and @@ -786,10 +785,8 @@ EXPORT_SYMBOL(file_write_and_wait_range) * caller must do that. * * The remove + add is atomic. This function cannot fail. - * - * Return: %0 */ -int replace_page_cache_page(struct page *old, struct page *new, gfp_t gfp_mask) +void replace_page_cache_page(struct page *old, struct page *new) { struct address_space *mapping = old->mapping; void (*freepage)(struct page *) = mapping->a_ops->freepage; @@ -824,8 +821,6 @@ int replace_page_cache_page(struct page if (freepage) freepage(old); put_page(old); - - return 0; } EXPORT_SYMBOL_GPL(replace_page_cache_page); From patchwork Wed Feb 24 20:01:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102469 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8EC0EC4332B for ; Wed, 24 Feb 2021 20:01:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3AB1864EC4 for ; Wed, 24 Feb 2021 20:01:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3AB1864EC4 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 955906B008C; Wed, 24 Feb 2021 15:01:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8EF9F8D0001; Wed, 24 Feb 2021 15:01:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5FB656B0093; Wed, 24 Feb 2021 15:01:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0151.hostedemail.com [216.40.44.151]) by kanga.kvack.org (Postfix) with ESMTP id 46B596B008C for ; Wed, 24 Feb 2021 15:01:48 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 06A33185044AA for ; Wed, 24 Feb 2021 20:01:48 +0000 (UTC) X-FDA: 77854231896.15.73C1604 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf05.hostedemail.com (Postfix) with ESMTP id 89869E000106 for ; Wed, 24 Feb 2021 20:01:46 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 4132D64E7A; Wed, 24 Feb 2021 20:01:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196906; bh=G/hO3Rnc1ew2BsKhMRd+zRTTHSdeEe5zj0NmD4Ixw64=; h=Date:From:To:Subject:In-Reply-To:From; b=EQYjTzH+iKQGk53pf7Zsh36awevfIq5t1FcXGf6MmQMD0ybpgBHfLFwtRl2LU4Tv0 cad4VzJnxt+ALywiAqXycpdT92itcCh7z0nBpuTS4kcXYsPNKCTfoCyPcwM/8NJ1+Z sa2p3yiTIIaoOF9G16lRd3JYtCwvfiesDWdGSdUU= Date: Wed, 24 Feb 2021 12:01:45 -0800 From: Andrew Morton To: akpm@linux-foundation.org, asml.silence@gmail.com, axboe@kernel.dk, hch@lst.de, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk Subject: [patch 029/173] mm/filemap: don't revert iter on -EIOCBQUEUED Message-ID: <20210224200145.tzDYWtTYF%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: oszq579cdm86qpbu8a1znctx7dwcnihi X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 89869E000106 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf05; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196906-960447 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Pavel Begunkov Subject: mm/filemap: don't revert iter on -EIOCBQUEUED Currently, if I/O is enqueued for async execution direct paths of generic_file_{read,write}_iter() will always revert the iter. There are no users expecting that, and that is also costly. Leave iterators as is on -EIOCBQUEUED. Link: https://lkml.kernel.org/r/f5247b60e7abbd2ff850cd108491f53a2e0c501a.1610207781.git.asml.silence@gmail.com Signed-off-by: Pavel Begunkov Reviewed-by: Christoph Hellwig Cc: Al Viro Cc: Jens Axboe Signed-off-by: Andrew Morton --- mm/filemap.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) --- a/mm/filemap.c~mm-filemap-dont-revert-iter-on-eiocbqueued +++ a/mm/filemap.c @@ -2615,7 +2615,8 @@ generic_file_read_iter(struct kiocb *ioc iocb->ki_pos += retval; count -= retval; } - iov_iter_revert(iter, count - iov_iter_count(iter)); + if (retval != -EIOCBQUEUED) + iov_iter_revert(iter, count - iov_iter_count(iter)); /* * Btrfs can have a short DIO read if we encounter @@ -3426,7 +3427,8 @@ generic_file_direct_write(struct kiocb * } iocb->ki_pos = pos; } - iov_iter_revert(from, write_len - iov_iter_count(from)); + if (written != -EIOCBQUEUED) + iov_iter_revert(from, write_len - iov_iter_count(from)); out: return written; } From patchwork Wed Feb 24 20:01:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102471 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C03C9C433DB for ; Wed, 24 Feb 2021 20:01:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 591E264E6C for ; Wed, 24 Feb 2021 20:01:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 591E264E6C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 008838D0002; Wed, 24 Feb 2021 15:01:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id ED5058D0001; Wed, 24 Feb 2021 15:01:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D99C38D0002; Wed, 24 Feb 2021 15:01:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0056.hostedemail.com [216.40.44.56]) by kanga.kvack.org (Postfix) with ESMTP id C07508D0001 for ; Wed, 24 Feb 2021 15:01:51 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 878A118017BF0 for ; Wed, 24 Feb 2021 20:01:51 +0000 (UTC) X-FDA: 77854232022.13.C544A60 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf01.hostedemail.com (Postfix) with ESMTP id 49D1C2000390 for ; Wed, 24 Feb 2021 20:01:50 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id AB9F864EC4; Wed, 24 Feb 2021 20:01:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196910; bh=2hPNHFAWK4SArcZOrl8XmoD8oHkK0U679kOXky30KlM=; h=Date:From:To:Subject:In-Reply-To:From; b=D+PvGpxneGSFeMozTOlaz69CIVKeZFK8bTqXhEzEPnzuF5h6Fovbl6PjdUlpABdMc /Q2YfOADjGRMGtbxr9kzeTnVj1DlC1xPj+GMtjGoTX9LVHrDCW8/264FYyGI7NJx/Y 65CufEydGrTT7wwwk6KClJ83S9S7f6qYhpQ2+o/8= Date: Wed, 24 Feb 2021 12:01:49 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 030/173] mm/filemap: rename generic_file_buffered_read subfunctions Message-ID: <20210224200149.q7jigblJl%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 49D1C2000390 X-Stat-Signature: jnzxdc6nco9sxqggndjwerewfowkwqp7 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf01; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196910-990698 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/filemap: rename generic_file_buffered_read subfunctions Patch series "Refactor generic_file_buffered_read", v5. This is a combination of Christoph's work to refactor generic_file_buffered_read() and some of my large-page support which was disrupted by Kent's refactoring of generic_file_buffered_read. This patch (of 18): The recent split of generic_file_buffered_read() created some very long function names which are hard to distinguish from each other. Rename as follows: generic_file_buffered_read_readpage -> filemap_read_page generic_file_buffered_read_pagenotuptodate -> filemap_update_page generic_file_buffered_read_no_cached_page -> filemap_create_page generic_file_buffered_read_get_pages -> filemap_get_pages Link: https://lkml.kernel.org/r/20210122160140.223228-1-willy@infradead.org Link: https://lkml.kernel.org/r/20210122160140.223228-2-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig Reviewed-by: Miaohe Lin Signed-off-by: Andrew Morton --- mm/filemap.c | 44 +++++++++++++++----------------------------- 1 file changed, 15 insertions(+), 29 deletions(-) --- a/mm/filemap.c~mm-filemap-rename-generic_file_buffered_read-subfunctions +++ a/mm/filemap.c @@ -2178,11 +2178,8 @@ static int lock_page_for_iocb(struct kio return lock_page_killable(page); } -static struct page * -generic_file_buffered_read_readpage(struct kiocb *iocb, - struct file *filp, - struct address_space *mapping, - struct page *page) +static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, + struct address_space *mapping, struct page *page) { struct file_ra_state *ra = &filp->f_ra; int error; @@ -2233,12 +2230,9 @@ generic_file_buffered_read_readpage(stru return page; } -static struct page * -generic_file_buffered_read_pagenotuptodate(struct kiocb *iocb, - struct file *filp, - struct iov_iter *iter, - struct page *page, - loff_t pos, loff_t count) +static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, + struct iov_iter *iter, struct page *page, loff_t pos, + loff_t count) { struct address_space *mapping = filp->f_mapping; struct inode *inode = mapping->host; @@ -2301,12 +2295,11 @@ page_not_up_to_date_locked: return page; } - return generic_file_buffered_read_readpage(iocb, filp, mapping, page); + return filemap_read_page(iocb, filp, mapping, page); } -static struct page * -generic_file_buffered_read_no_cached_page(struct kiocb *iocb, - struct iov_iter *iter) +static struct page *filemap_create_page(struct kiocb *iocb, + struct iov_iter *iter) { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; @@ -2317,10 +2310,6 @@ generic_file_buffered_read_no_cached_pag if (iocb->ki_flags & IOCB_NOIO) return ERR_PTR(-EAGAIN); - /* - * Ok, it wasn't cached, so we need to create a new - * page.. - */ page = page_cache_alloc(mapping); if (!page) return ERR_PTR(-ENOMEM); @@ -2332,13 +2321,11 @@ generic_file_buffered_read_no_cached_pag return error != -EEXIST ? ERR_PTR(error) : NULL; } - return generic_file_buffered_read_readpage(iocb, filp, mapping, page); + return filemap_read_page(iocb, filp, mapping, page); } -static int generic_file_buffered_read_get_pages(struct kiocb *iocb, - struct iov_iter *iter, - struct page **pages, - unsigned int nr) +static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, + struct page **pages, unsigned int nr) { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; @@ -2365,7 +2352,7 @@ find_page: if (nr_got) goto got_pages; - pages[0] = generic_file_buffered_read_no_cached_page(iocb, iter); + pages[0] = filemap_create_page(iocb, iter); err = PTR_ERR_OR_ZERO(pages[0]); if (!IS_ERR_OR_NULL(pages[0])) nr_got = 1; @@ -2399,8 +2386,8 @@ got_pages: break; } - page = generic_file_buffered_read_pagenotuptodate(iocb, - filp, iter, page, pg_pos, pg_count); + page = filemap_update_page(iocb, filp, iter, page, + pg_pos, pg_count); if (IS_ERR_OR_NULL(page)) { for (j = i + 1; j < nr_got; j++) put_page(pages[j]); @@ -2479,8 +2466,7 @@ ssize_t generic_file_buffered_read(struc iocb->ki_flags |= IOCB_NOWAIT; i = 0; - pg_nr = generic_file_buffered_read_get_pages(iocb, iter, - pages, nr_pages); + pg_nr = filemap_get_pages(iocb, iter, pages, nr_pages); if (pg_nr < 0) { error = pg_nr; break; From patchwork Wed Feb 24 20:01:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102473 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6237C433E0 for ; Wed, 24 Feb 2021 20:01:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 90D9064E6B for ; Wed, 24 Feb 2021 20:01:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 90D9064E6B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 365638D0003; Wed, 24 Feb 2021 15:01:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 310ED8D0001; Wed, 24 Feb 2021 15:01:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24E8E8D0003; Wed, 24 Feb 2021 15:01:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0132.hostedemail.com [216.40.44.132]) by kanga.kvack.org (Postfix) with ESMTP id 103CE8D0001 for ; Wed, 24 Feb 2021 15:01:55 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id CED1A8083CE6 for ; Wed, 24 Feb 2021 20:01:54 +0000 (UTC) X-FDA: 77854232148.16.3F8E73A Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP id 2E650E000108 for ; Wed, 24 Feb 2021 20:01:54 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0DC9264E83; Wed, 24 Feb 2021 20:01:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196913; bh=ZJh8e4jsUNE/0SFEbgId4dpUQgEPbalDIanXSj7b92I=; h=Date:From:To:Subject:In-Reply-To:From; b=Y92M9F8CLjrxrWhIFwnor0e47wk4EE3fgS8zam6yfVbnaV6yDrAWRr0Qu5ALT72MD TSM0cPfQf3IgZEJf06ymxQv0T9KJluDWvaUf3WiG/Wyh2mkrsbKNDtT/7rObfRwOAx up+IOdv/QPc2ptukQNEgol3FFJx8zTMakcS6zFvU= Date: Wed, 24 Feb 2021 12:01:52 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 031/173] mm/filemap: remove dynamically allocated array from filemap_read Message-ID: <20210224200152.sbTamoMyG%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 2E650E000108 X-Stat-Signature: 9z38c6kkg8oakbui97gjuyh6843c36y1 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf30; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196914-678057 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/filemap: remove dynamically allocated array from filemap_read Increasing the batch size runs into diminishing returns. It's probably better to make, eg, three calls to filemap_get_pages() than it is to call into kmalloc(). Link: https://lkml.kernel.org/r/20210122160140.223228-3-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig Reviewed-by: Miaohe Lin Cc: Kent Overstreet Signed-off-by: Andrew Morton --- mm/filemap.c | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-) --- a/mm/filemap.c~mm-filemap-remove-dynamically-allocated-array-from-filemap_read +++ a/mm/filemap.c @@ -2431,8 +2431,8 @@ ssize_t generic_file_buffered_read(struc struct file_ra_state *ra = &filp->f_ra; struct address_space *mapping = filp->f_mapping; struct inode *inode = mapping->host; - struct page *pages_onstack[PAGEVEC_SIZE], **pages = NULL; - unsigned int nr_pages = min_t(unsigned int, 512, + struct page *pages[PAGEVEC_SIZE]; + unsigned int nr_pages = min_t(unsigned int, PAGEVEC_SIZE, ((iocb->ki_pos + iter->count + PAGE_SIZE - 1) >> PAGE_SHIFT) - (iocb->ki_pos >> PAGE_SHIFT)); int i, pg_nr, error = 0; @@ -2446,14 +2446,6 @@ ssize_t generic_file_buffered_read(struc iov_iter_truncate(iter, inode->i_sb->s_maxbytes); - if (nr_pages > ARRAY_SIZE(pages_onstack)) - pages = kmalloc_array(nr_pages, sizeof(void *), GFP_KERNEL); - - if (!pages) { - pages = pages_onstack; - nr_pages = min_t(unsigned int, nr_pages, ARRAY_SIZE(pages_onstack)); - } - do { cond_resched(); @@ -2538,9 +2530,6 @@ put_pages: file_accessed(filp); - if (pages != pages_onstack) - kfree(pages); - return written ? written : error; } EXPORT_SYMBOL_GPL(generic_file_buffered_read); From patchwork Wed Feb 24 20:01:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102477 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFA7EC433E6 for ; Wed, 24 Feb 2021 20:02:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6658464F16 for ; Wed, 24 Feb 2021 20:02:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6658464F16 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C42718D0006; Wed, 24 Feb 2021 15:01:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BA1F58D0001; Wed, 24 Feb 2021 15:01:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A17C48D0006; Wed, 24 Feb 2021 15:01:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0105.hostedemail.com [216.40.44.105]) by kanga.kvack.org (Postfix) with ESMTP id 8CC8F8D0001 for ; Wed, 24 Feb 2021 15:01:59 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 520971808F578 for ; Wed, 24 Feb 2021 20:01:59 +0000 (UTC) X-FDA: 77854232358.07.F315ADF Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP id 69C86C0001FE for ; Wed, 24 Feb 2021 20:01:54 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 71C0764E6C; Wed, 24 Feb 2021 20:01:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196916; bh=7Uu81tCcjTvVcqhbc4Nl8dv5hbWGMlI+LW/sneiUIbY=; h=Date:From:To:Subject:In-Reply-To:From; b=izZtGdj4S3YIDD9hBOmM27WzmLlmbINd9UsCoqwS34HS3wzAQ7d4Aqb3V0vHVFluT zewYOwgut5AQrirLPBLa1U2wMUrWA0vpWm/lifNwpxnm/r08julOrHSc5XtZtwJvwc /vxTTOvhB5hXM2j9d2lg09z00e26lKTzuWBILHGg= Date: Wed, 24 Feb 2021 12:01:55 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 032/173] mm/filemap: convert filemap_get_pages to take a pagevec Message-ID: <20210224200155.pgDlQeaHP%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: xfx8arp466qfw3oh69sx1g7kd5u8yym6 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 69C86C0001FE Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf22; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196914-451435 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/filemap: convert filemap_get_pages to take a pagevec Using a pagevec lets us keep the pages and the number of pages together which simplifies a lot of the calling conventions. Link: https://lkml.kernel.org/r/20210122160140.223228-4-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Christoph Hellwig Cc: Kent Overstreet Cc: Miaohe Lin Signed-off-by: Andrew Morton --- mm/filemap.c | 82 ++++++++++++++++++++++--------------------------- 1 file changed, 38 insertions(+), 44 deletions(-) --- a/mm/filemap.c~mm-filemap-convert-filemap_get_pages-to-take-a-pagevec +++ a/mm/filemap.c @@ -2325,22 +2325,22 @@ static struct page *filemap_create_page( } static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, - struct page **pages, unsigned int nr) + struct pagevec *pvec) { struct file *filp = iocb->ki_filp; struct address_space *mapping = filp->f_mapping; struct file_ra_state *ra = &filp->f_ra; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; pgoff_t last_index = (iocb->ki_pos + iter->count + PAGE_SIZE-1) >> PAGE_SHIFT; - int i, j, nr_got, err = 0; + unsigned int nr = min_t(unsigned long, last_index - index, PAGEVEC_SIZE); + int i, j, err = 0; - nr = min_t(unsigned long, last_index - index, nr); find_page: if (fatal_signal_pending(current)) return -EINTR; - nr_got = find_get_pages_contig(mapping, index, nr, pages); - if (nr_got) + pvec->nr = find_get_pages_contig(mapping, index, nr, pvec->pages); + if (pvec->nr) goto got_pages; if (iocb->ki_flags & IOCB_NOIO) @@ -2348,17 +2348,17 @@ find_page: page_cache_sync_readahead(mapping, ra, filp, index, last_index - index); - nr_got = find_get_pages_contig(mapping, index, nr, pages); - if (nr_got) + pvec->nr = find_get_pages_contig(mapping, index, nr, pvec->pages); + if (pvec->nr) goto got_pages; - pages[0] = filemap_create_page(iocb, iter); - err = PTR_ERR_OR_ZERO(pages[0]); - if (!IS_ERR_OR_NULL(pages[0])) - nr_got = 1; + pvec->pages[0] = filemap_create_page(iocb, iter); + err = PTR_ERR_OR_ZERO(pvec->pages[0]); + if (!IS_ERR_OR_NULL(pvec->pages[0])) + pvec->nr = 1; got_pages: - for (i = 0; i < nr_got; i++) { - struct page *page = pages[i]; + for (i = 0; i < pvec->nr; i++) { + struct page *page = pvec->pages[i]; pgoff_t pg_index = index + i; loff_t pg_pos = max(iocb->ki_pos, (loff_t) pg_index << PAGE_SHIFT); @@ -2366,9 +2366,9 @@ got_pages: if (PageReadahead(page)) { if (iocb->ki_flags & IOCB_NOIO) { - for (j = i; j < nr_got; j++) - put_page(pages[j]); - nr_got = i; + for (j = i; j < pvec->nr; j++) + put_page(pvec->pages[j]); + pvec->nr = i; err = -EAGAIN; break; } @@ -2379,9 +2379,9 @@ got_pages: if (!PageUptodate(page)) { if ((iocb->ki_flags & IOCB_NOWAIT) || ((iocb->ki_flags & IOCB_WAITQ) && i)) { - for (j = i; j < nr_got; j++) - put_page(pages[j]); - nr_got = i; + for (j = i; j < pvec->nr; j++) + put_page(pvec->pages[j]); + pvec->nr = i; err = -EAGAIN; break; } @@ -2389,17 +2389,17 @@ got_pages: page = filemap_update_page(iocb, filp, iter, page, pg_pos, pg_count); if (IS_ERR_OR_NULL(page)) { - for (j = i + 1; j < nr_got; j++) - put_page(pages[j]); - nr_got = i; + for (j = i + 1; j < pvec->nr; j++) + put_page(pvec->pages[j]); + pvec->nr = i; err = PTR_ERR_OR_ZERO(page); break; } } } - if (likely(nr_got)) - return nr_got; + if (likely(pvec->nr)) + return 0; if (err) return err; /* @@ -2431,11 +2431,8 @@ ssize_t generic_file_buffered_read(struc struct file_ra_state *ra = &filp->f_ra; struct address_space *mapping = filp->f_mapping; struct inode *inode = mapping->host; - struct page *pages[PAGEVEC_SIZE]; - unsigned int nr_pages = min_t(unsigned int, PAGEVEC_SIZE, - ((iocb->ki_pos + iter->count + PAGE_SIZE - 1) >> PAGE_SHIFT) - - (iocb->ki_pos >> PAGE_SHIFT)); - int i, pg_nr, error = 0; + struct pagevec pvec; + int i, error = 0; bool writably_mapped; loff_t isize, end_offset; @@ -2457,12 +2454,9 @@ ssize_t generic_file_buffered_read(struc if ((iocb->ki_flags & IOCB_WAITQ) && written) iocb->ki_flags |= IOCB_NOWAIT; - i = 0; - pg_nr = filemap_get_pages(iocb, iter, pages, nr_pages); - if (pg_nr < 0) { - error = pg_nr; + error = filemap_get_pages(iocb, iter, &pvec); + if (error < 0) break; - } /* * i_size must be checked after we know the pages are Uptodate. @@ -2478,9 +2472,9 @@ ssize_t generic_file_buffered_read(struc end_offset = min_t(loff_t, isize, iocb->ki_pos + iter->count); - while ((iocb->ki_pos >> PAGE_SHIFT) + pg_nr > + while ((iocb->ki_pos >> PAGE_SHIFT) + pvec.nr > (end_offset + PAGE_SIZE - 1) >> PAGE_SHIFT) - put_page(pages[--pg_nr]); + put_page(pvec.pages[--pvec.nr]); /* * Once we start copying data, we don't want to be touching any @@ -2494,11 +2488,11 @@ ssize_t generic_file_buffered_read(struc */ if (iocb->ki_pos >> PAGE_SHIFT != ra->prev_pos >> PAGE_SHIFT) - mark_page_accessed(pages[0]); - for (i = 1; i < pg_nr; i++) - mark_page_accessed(pages[i]); + mark_page_accessed(pvec.pages[0]); + for (i = 1; i < pagevec_count(&pvec); i++) + mark_page_accessed(pvec.pages[i]); - for (i = 0; i < pg_nr; i++) { + for (i = 0; i < pagevec_count(&pvec); i++) { unsigned int offset = iocb->ki_pos & ~PAGE_MASK; unsigned int bytes = min_t(loff_t, end_offset - iocb->ki_pos, PAGE_SIZE - offset); @@ -2510,9 +2504,9 @@ ssize_t generic_file_buffered_read(struc * before reading the page on the kernel side. */ if (writably_mapped) - flush_dcache_page(pages[i]); + flush_dcache_page(pvec.pages[i]); - copied = copy_page_to_iter(pages[i], offset, bytes, iter); + copied = copy_page_to_iter(pvec.pages[i], offset, bytes, iter); written += copied; iocb->ki_pos += copied; @@ -2524,8 +2518,8 @@ ssize_t generic_file_buffered_read(struc } } put_pages: - for (i = 0; i < pg_nr; i++) - put_page(pages[i]); + for (i = 0; i < pagevec_count(&pvec); i++) + put_page(pvec.pages[i]); } while (iov_iter_count(iter) && iocb->ki_pos < isize && !error); file_accessed(filp); From patchwork Wed Feb 24 20:01:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102479 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5514EC433E0 for ; Wed, 24 Feb 2021 20:02:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F158964F1D for ; Wed, 24 Feb 2021 20:02:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F158964F1D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E85788D0007; Wed, 24 Feb 2021 15:02:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E0DF68D0001; Wed, 24 Feb 2021 15:02:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CFE6D8D0007; Wed, 24 Feb 2021 15:02:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0179.hostedemail.com [216.40.44.179]) by kanga.kvack.org (Postfix) with ESMTP id B00498D0001 for ; Wed, 24 Feb 2021 15:02:01 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 7756FEFEE for ; Wed, 24 Feb 2021 20:02:01 +0000 (UTC) X-FDA: 77854232442.13.D824CCA Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP id D8E4EE00010F for ; Wed, 24 Feb 2021 20:02:00 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C29BB64F0E; Wed, 24 Feb 2021 20:01:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196920; bh=aCK0dNgwGkv90of6nPlfiG1AVyeLOiO5Ok0TXQU0XMc=; h=Date:From:To:Subject:In-Reply-To:From; b=D5IzzQK/qajMCdfut204yMk9o/QPSWaK/gcOBWiYEh3t7WQPbYAF0ht3XjcaJ5LMm cb46X6COJ+VNPBrQZq1j6OVsKgLKYaYn0OBbK+Asxkq2GjuNqRCq0iN2U6sXFDmQI7 1550nZBo0PmZlZafnawguYrx7VerSdxv1e/nyCKM= Date: Wed, 24 Feb 2021 12:01:59 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 033/173] mm/filemap: use head pages in generic_file_buffered_read Message-ID: <20210224200159.h_ITNzOVN%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: D8E4EE00010F X-Stat-Signature: n3ikhocfysu79gd4fa9ubj1qacqeio5y Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf30; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196920-745510 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/filemap: use head pages in generic_file_buffered_read Add filemap_get_read_batch() which returns the head pages which represent a contiguous array of bytes in the file. It also stops when encountering a page marked as Readahead or !Uptodate (but does return that page) so it can be handled appropriately by filemap_get_pages(). That lets us remove the loop in filemap_get_pages() and check only the last page. Link: https://lkml.kernel.org/r/20210122160140.223228-5-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig Cc: Miaohe Lin Signed-off-by: Andrew Morton --- mm/filemap.c | 122 ++++++++++++++++++++++++++++++++++--------------- 1 file changed, 85 insertions(+), 37 deletions(-) --- a/mm/filemap.c~mm-filemap-use-head-pages-in-generic_file_buffered_read +++ a/mm/filemap.c @@ -2178,6 +2178,51 @@ static int lock_page_for_iocb(struct kio return lock_page_killable(page); } +/* + * filemap_get_read_batch - Get a batch of pages for read + * + * Get a batch of pages which represent a contiguous range of bytes + * in the file. No tail pages will be returned. If @index is in the + * middle of a THP, the entire THP will be returned. The last page in + * the batch may have Readahead set or be not Uptodate so that the + * caller can take the appropriate action. + */ +static void filemap_get_read_batch(struct address_space *mapping, + pgoff_t index, pgoff_t max, struct pagevec *pvec) +{ + XA_STATE(xas, &mapping->i_pages, index); + struct page *head; + + rcu_read_lock(); + for (head = xas_load(&xas); head; head = xas_next(&xas)) { + if (xas_retry(&xas, head)) + continue; + if (xas.xa_index > max || xa_is_value(head)) + break; + if (!page_cache_get_speculative(head)) + goto retry; + + /* Has the page moved or been split? */ + if (unlikely(head != xas_reload(&xas))) + goto put_page; + + if (!pagevec_add(pvec, head)) + break; + if (!PageUptodate(head)) + break; + if (PageReadahead(head)) + break; + xas.xa_index = head->index + thp_nr_pages(head) - 1; + xas.xa_offset = (xas.xa_index >> xas.xa_shift) & XA_CHUNK_MASK; + continue; +put_page: + put_page(head); +retry: + xas_reset(&xas); + } + rcu_read_unlock(); +} + static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, struct address_space *mapping, struct page *page) { @@ -2331,15 +2376,15 @@ static int filemap_get_pages(struct kioc struct address_space *mapping = filp->f_mapping; struct file_ra_state *ra = &filp->f_ra; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; - pgoff_t last_index = (iocb->ki_pos + iter->count + PAGE_SIZE-1) >> PAGE_SHIFT; - unsigned int nr = min_t(unsigned long, last_index - index, PAGEVEC_SIZE); - int i, j, err = 0; + pgoff_t last_index; + int err = 0; + last_index = DIV_ROUND_UP(iocb->ki_pos + iter->count, PAGE_SIZE); find_page: if (fatal_signal_pending(current)) return -EINTR; - pvec->nr = find_get_pages_contig(mapping, index, nr, pvec->pages); + filemap_get_read_batch(mapping, index, last_index, pvec); if (pvec->nr) goto got_pages; @@ -2348,29 +2393,30 @@ find_page: page_cache_sync_readahead(mapping, ra, filp, index, last_index - index); - pvec->nr = find_get_pages_contig(mapping, index, nr, pvec->pages); + filemap_get_read_batch(mapping, index, last_index, pvec); if (pvec->nr) goto got_pages; pvec->pages[0] = filemap_create_page(iocb, iter); err = PTR_ERR_OR_ZERO(pvec->pages[0]); - if (!IS_ERR_OR_NULL(pvec->pages[0])) - pvec->nr = 1; + if (IS_ERR_OR_NULL(pvec->pages[0])) + goto err; + pvec->nr = 1; + return 0; got_pages: - for (i = 0; i < pvec->nr; i++) { - struct page *page = pvec->pages[i]; - pgoff_t pg_index = index + i; + { + struct page *page = pvec->pages[pvec->nr - 1]; + pgoff_t pg_index = page->index; loff_t pg_pos = max(iocb->ki_pos, (loff_t) pg_index << PAGE_SHIFT); loff_t pg_count = iocb->ki_pos + iter->count - pg_pos; if (PageReadahead(page)) { if (iocb->ki_flags & IOCB_NOIO) { - for (j = i; j < pvec->nr; j++) - put_page(pvec->pages[j]); - pvec->nr = i; + put_page(page); + pvec->nr--; err = -EAGAIN; - break; + goto err; } page_cache_async_readahead(mapping, ra, filp, page, pg_index, last_index - pg_index); @@ -2378,26 +2424,23 @@ got_pages: if (!PageUptodate(page)) { if ((iocb->ki_flags & IOCB_NOWAIT) || - ((iocb->ki_flags & IOCB_WAITQ) && i)) { - for (j = i; j < pvec->nr; j++) - put_page(pvec->pages[j]); - pvec->nr = i; + ((iocb->ki_flags & IOCB_WAITQ) && pvec->nr > 1)) { + put_page(page); + pvec->nr--; err = -EAGAIN; - break; + goto err; } page = filemap_update_page(iocb, filp, iter, page, pg_pos, pg_count); if (IS_ERR_OR_NULL(page)) { - for (j = i + 1; j < pvec->nr; j++) - put_page(pvec->pages[j]); - pvec->nr = i; + pvec->nr--; err = PTR_ERR_OR_ZERO(page); - break; } } } +err: if (likely(pvec->nr)) return 0; if (err) @@ -2442,6 +2485,7 @@ ssize_t generic_file_buffered_read(struc return 0; iov_iter_truncate(iter, inode->i_sb->s_maxbytes); + pagevec_init(&pvec); do { cond_resched(); @@ -2469,13 +2513,8 @@ ssize_t generic_file_buffered_read(struc isize = i_size_read(inode); if (unlikely(iocb->ki_pos >= isize)) goto put_pages; - end_offset = min_t(loff_t, isize, iocb->ki_pos + iter->count); - while ((iocb->ki_pos >> PAGE_SHIFT) + pvec.nr > - (end_offset + PAGE_SIZE - 1) >> PAGE_SHIFT) - put_page(pvec.pages[--pvec.nr]); - /* * Once we start copying data, we don't want to be touching any * cachelines that might be contended: @@ -2489,24 +2528,32 @@ ssize_t generic_file_buffered_read(struc if (iocb->ki_pos >> PAGE_SHIFT != ra->prev_pos >> PAGE_SHIFT) mark_page_accessed(pvec.pages[0]); - for (i = 1; i < pagevec_count(&pvec); i++) - mark_page_accessed(pvec.pages[i]); for (i = 0; i < pagevec_count(&pvec); i++) { - unsigned int offset = iocb->ki_pos & ~PAGE_MASK; - unsigned int bytes = min_t(loff_t, end_offset - iocb->ki_pos, - PAGE_SIZE - offset); - unsigned int copied; + struct page *page = pvec.pages[i]; + size_t page_size = thp_size(page); + size_t offset = iocb->ki_pos & (page_size - 1); + size_t bytes = min_t(loff_t, end_offset - iocb->ki_pos, + page_size - offset); + size_t copied; + if (end_offset < page_offset(page)) + break; + if (i > 0) + mark_page_accessed(page); /* * If users can be writing to this page using arbitrary * virtual addresses, take care about potential aliasing * before reading the page on the kernel side. */ - if (writably_mapped) - flush_dcache_page(pvec.pages[i]); + if (writably_mapped) { + int j; + + for (j = 0; j < thp_nr_pages(page); j++) + flush_dcache_page(page + j); + } - copied = copy_page_to_iter(pvec.pages[i], offset, bytes, iter); + copied = copy_page_to_iter(page, offset, bytes, iter); written += copied; iocb->ki_pos += copied; @@ -2520,6 +2567,7 @@ ssize_t generic_file_buffered_read(struc put_pages: for (i = 0; i < pagevec_count(&pvec); i++) put_page(pvec.pages[i]); + pagevec_reinit(&pvec); } while (iov_iter_count(iter) && iocb->ki_pos < isize && !error); file_accessed(filp); From patchwork Wed Feb 24 20:02:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102481 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B084C433E0 for ; Wed, 24 Feb 2021 20:02:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B815964F19 for ; Wed, 24 Feb 2021 20:02:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B815964F19 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 55E478D0008; Wed, 24 Feb 2021 15:02:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E56B8D0001; Wed, 24 Feb 2021 15:02:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3D4F18D0008; Wed, 24 Feb 2021 15:02:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0156.hostedemail.com [216.40.44.156]) by kanga.kvack.org (Postfix) with ESMTP id 2211F8D0001 for ; Wed, 24 Feb 2021 15:02:05 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id CBC75EFEE for ; Wed, 24 Feb 2021 20:02:04 +0000 (UTC) X-FDA: 77854232568.21.8DA8D45 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf05.hostedemail.com (Postfix) with ESMTP id 46A99E0011D9 for ; Wed, 24 Feb 2021 20:02:03 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0076164F28; Wed, 24 Feb 2021 20:02:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196923; bh=cm6mdEuoH//0Y6Q7axT1Aq8f1W47cBae5a1t7577Krs=; h=Date:From:To:Subject:In-Reply-To:From; b=q0NuLO6N6GC7auWubzEgj/x/lUomTWEfg36WnunhQ2OijXlUCuNaJnZkRvXuelQIO kd++hTzzcbkRIBDcnAoWNBZlA7Eju+PWgPVLvFty68Yn8vnEndbhr7NzxY1CFPRtGr nwbGVbkTZ+1Pz9X2uM5wzCuVqTVv3oQXhjPCWyV0= Date: Wed, 24 Feb 2021 12:02:02 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 034/173] mm/filemap: pass a sleep state to put_and_wait_on_page_locked Message-ID: <20210224200202.wo6eFwQ-w%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: oji9kbqhjycb1nteuqssrxtop8ss5i5f X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 46A99E0011D9 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf05; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196923-838587 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/filemap: pass a sleep state to put_and_wait_on_page_locked This is prep work for the next patch, but I think at least one of the current callers would prefer a killable sleep to an uninterruptible one. Link: https://lkml.kernel.org/r/20210122160140.223228-6-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig Cc: Miaohe Lin Signed-off-by: Andrew Morton --- include/linux/pagemap.h | 3 +-- mm/filemap.c | 7 +++++-- mm/huge_memory.c | 4 ++-- mm/migrate.c | 4 ++-- 4 files changed, 10 insertions(+), 8 deletions(-) --- a/include/linux/pagemap.h~mm-filemap-pass-a-sleep-state-to-put_and_wait_on_page_locked +++ a/include/linux/pagemap.h @@ -681,8 +681,7 @@ static inline int wait_on_page_locked_ki return wait_on_page_bit_killable(compound_head(page), PG_locked); } -extern void put_and_wait_on_page_locked(struct page *page); - +int put_and_wait_on_page_locked(struct page *page, int state); void wait_on_page_writeback(struct page *page); extern void end_page_writeback(struct page *page); void wait_for_stable_page(struct page *page); --- a/mm/filemap.c~mm-filemap-pass-a-sleep-state-to-put_and_wait_on_page_locked +++ a/mm/filemap.c @@ -1384,20 +1384,23 @@ static int wait_on_page_locked_async(str /** * put_and_wait_on_page_locked - Drop a reference and wait for it to be unlocked * @page: The page to wait for. + * @state: The sleep state (TASK_KILLABLE, TASK_UNINTERRUPTIBLE, etc). * * The caller should hold a reference on @page. They expect the page to * become unlocked relatively soon, but do not wish to hold up migration * (for example) by holding the reference while waiting for the page to * come unlocked. After this function returns, the caller should not * dereference @page. + * + * Return: 0 if the page was unlocked or -EINTR if interrupted by a signal. */ -void put_and_wait_on_page_locked(struct page *page) +int put_and_wait_on_page_locked(struct page *page, int state) { wait_queue_head_t *q; page = compound_head(page); q = page_waitqueue(page); - wait_on_page_bit_common(q, page, PG_locked, TASK_UNINTERRUPTIBLE, DROP); + return wait_on_page_bit_common(q, page, PG_locked, state, DROP); } /** --- a/mm/huge_memory.c~mm-filemap-pass-a-sleep-state-to-put_and_wait_on_page_locked +++ a/mm/huge_memory.c @@ -1439,7 +1439,7 @@ vm_fault_t do_huge_pmd_numa_page(struct if (!get_page_unless_zero(page)) goto out_unlock; spin_unlock(vmf->ptl); - put_and_wait_on_page_locked(page); + put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); goto out; } @@ -1475,7 +1475,7 @@ vm_fault_t do_huge_pmd_numa_page(struct if (!get_page_unless_zero(page)) goto out_unlock; spin_unlock(vmf->ptl); - put_and_wait_on_page_locked(page); + put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); goto out; } --- a/mm/migrate.c~mm-filemap-pass-a-sleep-state-to-put_and_wait_on_page_locked +++ a/mm/migrate.c @@ -331,7 +331,7 @@ void __migration_entry_wait(struct mm_st if (!get_page_unless_zero(page)) goto out; pte_unmap_unlock(ptep, ptl); - put_and_wait_on_page_locked(page); + put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); return; out: pte_unmap_unlock(ptep, ptl); @@ -365,7 +365,7 @@ void pmd_migration_entry_wait(struct mm_ if (!get_page_unless_zero(page)) goto unlock; spin_unlock(ptl); - put_and_wait_on_page_locked(page); + put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE); return; unlock: spin_unlock(ptl); From patchwork Wed Feb 24 20:02:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102483 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62408C433E0 for ; Wed, 24 Feb 2021 20:02:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 01CAE64F1D for ; Wed, 24 Feb 2021 20:02:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 01CAE64F1D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9071D8D0009; Wed, 24 Feb 2021 15:02:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B6868D0001; Wed, 24 Feb 2021 15:02:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A6138D0009; Wed, 24 Feb 2021 15:02:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0248.hostedemail.com [216.40.44.248]) by kanga.kvack.org (Postfix) with ESMTP id 644478D0001 for ; Wed, 24 Feb 2021 15:02:08 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 30A7218503B59 for ; Wed, 24 Feb 2021 20:02:08 +0000 (UTC) X-FDA: 77854232736.21.24D1B5C Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP id C49272000D81 for ; Wed, 24 Feb 2021 20:02:08 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 344C264F19; Wed, 24 Feb 2021 20:02:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196926; bh=JmLv10yizEZE0o5tZ++EnEfeyETnEYZZ84sJDcFCmkg=; h=Date:From:To:Subject:In-Reply-To:From; b=MljHdnSe/nEmGPUdTr1q6Icma8WmvQaCcNisrNPApwYmTweuVkHQJjSSxAAQ+3YRw Ld1ROpuBZtuvmcfd7gJPXb4efq1EIc9pGS9H4Bb5MvCVRk9XRO4nTuaF5ZxqvLsTih T3FMT42BbpQ87JmxIg0jgNn3eX2mkPP5jAdXi7rk= Date: Wed, 24 Feb 2021 12:02:05 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 035/173] mm/filemap: support readpage splitting a page Message-ID: <20210224200205.TfDHCg_Gm%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: C49272000D81 X-Stat-Signature: qm1qcj4ukfae6n88xjjhew1s7oyw13bb Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196928-374487 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/filemap: support readpage splitting a page For page splitting to succeed, the thread asking to split the page has to be the only one with a reference to the page. Calling wait_on_page_locked() while holding a reference to the page will effectively prevent this from happening with sufficient threads waiting on the same page. Use put_and_wait_on_page_locked() to sleep without holding a reference to the page, then retry the page lookup after the page is unlocked. Since we now get the page lock a little earlier in filemap_update_page(), we can eliminate a number of duplicate checks. The original intent (commit ebded02788b5 ("avoid unnecessary calls to lock_page when waiting for IO to complete during a read")) behind getting the page lock later was to avoid re-locking the page after it has been brought uptodate by another thread. We still avoid that because we go through the normal lookup path again after the winning thread has brought the page uptodate. Link: https://lkml.kernel.org/r/20210122160140.223228-7-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig Cc: Miaohe Lin Signed-off-by: Andrew Morton --- mm/filemap.c | 76 ++++++++++++++----------------------------------- 1 file changed, 23 insertions(+), 53 deletions(-) --- a/mm/filemap.c~mm-filemap-support-readpage-splitting-a-page +++ a/mm/filemap.c @@ -1373,14 +1373,6 @@ static int __wait_on_page_locked_async(s return ret; } -static int wait_on_page_locked_async(struct page *page, - struct wait_page_queue *wait) -{ - if (!PageLocked(page)) - return 0; - return __wait_on_page_locked_async(compound_head(page), wait, false); -} - /** * put_and_wait_on_page_locked - Drop a reference and wait for it to be unlocked * @page: The page to wait for. @@ -2286,64 +2278,42 @@ static struct page *filemap_update_page( struct inode *inode = mapping->host; int error; - /* - * See comment in do_read_cache_page on why - * wait_on_page_locked is used to avoid unnecessarily - * serialisations and why it's safe. - */ if (iocb->ki_flags & IOCB_WAITQ) { - error = wait_on_page_locked_async(page, - iocb->ki_waitq); + error = lock_page_async(page, iocb->ki_waitq); + if (error) { + put_page(page); + return ERR_PTR(error); + } } else { - error = wait_on_page_locked_killable(page); - } - if (unlikely(error)) { - put_page(page); - return ERR_PTR(error); + if (!trylock_page(page)) { + put_and_wait_on_page_locked(page, TASK_KILLABLE); + return NULL; + } } - if (PageUptodate(page)) - return page; + if (!page->mapping) + goto truncated; + if (PageUptodate(page)) + goto uptodate; if (inode->i_blkbits == PAGE_SHIFT || !mapping->a_ops->is_partially_uptodate) - goto page_not_up_to_date; + goto readpage; /* pipes can't handle partially uptodate pages */ if (unlikely(iov_iter_is_pipe(iter))) - goto page_not_up_to_date; - if (!trylock_page(page)) - goto page_not_up_to_date; - /* Did it get truncated before we got the lock? */ - if (!page->mapping) - goto page_not_up_to_date_locked; + goto readpage; if (!mapping->a_ops->is_partially_uptodate(page, - pos & ~PAGE_MASK, count)) - goto page_not_up_to_date_locked; + pos & (thp_size(page) - 1), count)) + goto readpage; +uptodate: unlock_page(page); return page; -page_not_up_to_date: - /* Get exclusive access to the page ... */ - error = lock_page_for_iocb(iocb, page); - if (unlikely(error)) { - put_page(page); - return ERR_PTR(error); - } - -page_not_up_to_date_locked: - /* Did it get truncated before we got the lock? */ - if (!page->mapping) { - unlock_page(page); - put_page(page); - return NULL; - } - - /* Did somebody else fill it already? */ - if (PageUptodate(page)) { - unlock_page(page); - return page; - } - +readpage: return filemap_read_page(iocb, filp, mapping, page); +truncated: + unlock_page(page); + put_page(page); + return NULL; } static struct page *filemap_create_page(struct kiocb *iocb, From patchwork Wed Feb 24 20:02:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102485 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C90CC433E6 for ; Wed, 24 Feb 2021 20:02:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F15E964F2E for ; Wed, 24 Feb 2021 20:02:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F15E964F2E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 955768D000A; Wed, 24 Feb 2021 15:02:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 92B678D0001; Wed, 24 Feb 2021 15:02:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 819B88D000A; Wed, 24 Feb 2021 15:02:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0156.hostedemail.com [216.40.44.156]) by kanga.kvack.org (Postfix) with ESMTP id 69DA48D0001 for ; Wed, 24 Feb 2021 15:02:11 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 406A8F007 for ; Wed, 24 Feb 2021 20:02:11 +0000 (UTC) X-FDA: 77854232862.08.1BCBE38 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP id 62BE780192E9 for ; Wed, 24 Feb 2021 20:02:04 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7B3C364F1D; Wed, 24 Feb 2021 20:02:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196929; bh=eCrVsLRflRDFs0Z2a3p4+SDwrGDzmtEh/uivsQxMBOc=; h=Date:From:To:Subject:In-Reply-To:From; b=deePSUpORCI9BSGx6pWqTfF94Z6ltWvBGqP1vF+bYJfD+oEleKC1ZEJwyQQ1b8T9K /BFQYv0Wn+YUSfToQtOecMFU7hscAmaKprYeUtouptC8A7FG3Yds4g6PYdgK5lsESA NY/83ISQuSWH3kslU0e1qdvlAgsMt9gyCizRc0BI= Date: Wed, 24 Feb 2021 12:02:09 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 036/173] mm/filemap: inline __wait_on_page_locked_async into caller Message-ID: <20210224200209.lnQgQ3a5R%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 62BE780192E9 X-Stat-Signature: 3zsfnwgzm18ft1xep6tfdqrfp91r94dj Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf27; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196924-649074 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/filemap: inline __wait_on_page_locked_async into caller The previous patch removed wait_on_page_locked_async(), so inline __wait_on_page_locked_async into __lock_page_async(). Link: https://lkml.kernel.org/r/20210122160140.223228-8-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig Cc: Miaohe Lin Signed-off-by: Andrew Morton --- mm/filemap.c | 53 ++++++++++++++++++++----------------------------- 1 file changed, 22 insertions(+), 31 deletions(-) --- a/mm/filemap.c~mm-filemap-inline-__wait_on_page_locked_async-into-caller +++ a/mm/filemap.c @@ -1343,36 +1343,6 @@ int wait_on_page_bit_killable(struct pag } EXPORT_SYMBOL(wait_on_page_bit_killable); -static int __wait_on_page_locked_async(struct page *page, - struct wait_page_queue *wait, bool set) -{ - struct wait_queue_head *q = page_waitqueue(page); - int ret = 0; - - wait->page = page; - wait->bit_nr = PG_locked; - - spin_lock_irq(&q->lock); - __add_wait_queue_entry_tail(q, &wait->wait); - SetPageWaiters(page); - if (set) - ret = !trylock_page(page); - else - ret = PageLocked(page); - /* - * If we were successful now, we know we're still on the - * waitqueue as we're still under the lock. This means it's - * safe to remove and return success, we know the callback - * isn't going to trigger. - */ - if (!ret) - __remove_wait_queue(q, &wait->wait); - else - ret = -EIOCBQUEUED; - spin_unlock_irq(&q->lock); - return ret; -} - /** * put_and_wait_on_page_locked - Drop a reference and wait for it to be unlocked * @page: The page to wait for. @@ -1548,7 +1518,28 @@ EXPORT_SYMBOL_GPL(__lock_page_killable); int __lock_page_async(struct page *page, struct wait_page_queue *wait) { - return __wait_on_page_locked_async(page, wait, true); + struct wait_queue_head *q = page_waitqueue(page); + int ret = 0; + + wait->page = page; + wait->bit_nr = PG_locked; + + spin_lock_irq(&q->lock); + __add_wait_queue_entry_tail(q, &wait->wait); + SetPageWaiters(page); + ret = !trylock_page(page); + /* + * If we were successful now, we know we're still on the + * waitqueue as we're still under the lock. This means it's + * safe to remove and return success, we know the callback + * isn't going to trigger. + */ + if (!ret) + __remove_wait_queue(q, &wait->wait); + else + ret = -EIOCBQUEUED; + spin_unlock_irq(&q->lock); + return ret; } /* From patchwork Wed Feb 24 20:02:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102487 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F1CCC43381 for ; Wed, 24 Feb 2021 20:02:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1293C64F14 for ; Wed, 24 Feb 2021 20:02:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1293C64F14 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9A2A38D000B; Wed, 24 Feb 2021 15:02:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9573C8D0001; Wed, 24 Feb 2021 15:02:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 846648D000B; Wed, 24 Feb 2021 15:02:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0127.hostedemail.com [216.40.44.127]) by kanga.kvack.org (Postfix) with ESMTP id 6D2408D0001 for ; Wed, 24 Feb 2021 15:02:14 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 35C432497 for ; Wed, 24 Feb 2021 20:02:14 +0000 (UTC) X-FDA: 77854232988.21.DB820CA Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf02.hostedemail.com (Postfix) with ESMTP id 590334080F56 for ; Wed, 24 Feb 2021 20:02:00 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B045D64F0D; Wed, 24 Feb 2021 20:02:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196933; bh=mv3sydC8iwPhnDe9IKvFJj2EroiDNW/Ic3ZTZGyBEC8=; h=Date:From:To:Subject:In-Reply-To:From; b=mbSBc69aFp6+S1Hl+gyzr1x/qyQMJVsc8m438HAdBWJ4ZKgPFG+az9xd2A9gu6ELp wP/9Mh0iQpK/QYbjYBuNPjrd07eqV8b8dQDl27/QyWuwIEjo0mSUO/Az6FzfqL03ER W1l4akPlxYAN5FRp3yZBnysleVLRZkjTpxdoPzNM= Date: Wed, 24 Feb 2021 12:02:12 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 037/173] mm/filemap: don't call ->readpage if IOCB_WAITQ is set Message-ID: <20210224200212.It57cmiS5%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 590334080F56 X-Stat-Signature: g94xfqukc8xxp5rpef5uuhsfjandjm6h Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf02; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196920-461406 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/filemap: don't call ->readpage if IOCB_WAITQ is set The readpage operation can block in many (most?) filesystems, so we should punt to a work queue instead of calling it. This was the last caller of lock_page_for_iocb(), so remove it. Link: https://lkml.kernel.org/r/20210122160140.223228-9-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig Cc: Miaohe Lin Signed-off-by: Andrew Morton --- mm/filemap.c | 14 ++------------ 1 file changed, 2 insertions(+), 12 deletions(-) --- a/mm/filemap.c~mm-filemap-dont-call-readpage-if-iocb_waitq-is-set +++ a/mm/filemap.c @@ -2154,16 +2154,6 @@ static void shrink_readahead_size_eio(st ra->ra_pages /= 4; } -static int lock_page_for_iocb(struct kiocb *iocb, struct page *page) -{ - if (iocb->ki_flags & IOCB_WAITQ) - return lock_page_async(page, iocb->ki_waitq); - else if (iocb->ki_flags & IOCB_NOWAIT) - return trylock_page(page) ? 0 : -EAGAIN; - else - return lock_page_killable(page); -} - /* * filemap_get_read_batch - Get a batch of pages for read * @@ -2215,7 +2205,7 @@ static struct page *filemap_read_page(st struct file_ra_state *ra = &filp->f_ra; int error; - if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT)) { + if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { unlock_page(page); put_page(page); return ERR_PTR(-EAGAIN); @@ -2236,7 +2226,7 @@ static struct page *filemap_read_page(st } if (!PageUptodate(page)) { - error = lock_page_for_iocb(iocb, page); + error = lock_page_killable(page); if (unlikely(error)) { put_page(page); return ERR_PTR(error); From patchwork Wed Feb 24 20:02:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102489 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56FB7C433E0 for ; Wed, 24 Feb 2021 20:02:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 01A1A64F09 for ; Wed, 24 Feb 2021 20:02:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 01A1A64F09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8E0408D000C; Wed, 24 Feb 2021 15:02:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B5918D0001; Wed, 24 Feb 2021 15:02:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F2878D000C; Wed, 24 Feb 2021 15:02:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0201.hostedemail.com [216.40.44.201]) by kanga.kvack.org (Postfix) with ESMTP id 657F28D0001 for ; Wed, 24 Feb 2021 15:02:18 -0500 (EST) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 294002497 for ; Wed, 24 Feb 2021 20:02:18 +0000 (UTC) X-FDA: 77854233156.12.8480D04 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf05.hostedemail.com (Postfix) with ESMTP id 20BB0E00011F for ; Wed, 24 Feb 2021 20:02:16 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id E89A364F0C; Wed, 24 Feb 2021 20:02:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196936; bh=NpsEh2E0z0A6UvwS4s+vZoh8U0eKngMLANkAqbqfZgk=; h=Date:From:To:Subject:In-Reply-To:From; b=AwtzEHDLgPnp92bJVZ8By9JDe3jrDR5QWWxynSZaNYk1OniKvhXPWV8aPCEUsYaLu yPg0GoGm/V0hQHXNuRKDVZagV3HuCTkDJNVmAAcDbKBd1fZJ7ZLLV8ZmwFoRqCSKH5 aR2VqgoHAWokKm0vBxLYBFZQCdE67reEf4KUv7JE= Date: Wed, 24 Feb 2021 12:02:15 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 038/173] mm/filemap: change filemap_read_page calling conventions Message-ID: <20210224200215.LkSqq8XQQ%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 20BB0E00011F X-Stat-Signature: 4u4on5eohk1e79iwdj6w1xdbiuk47rs5 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf05; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196936-111548 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/filemap: change filemap_read_page calling conventions Make this function more generic by passing the file instead of the iocb. Check in the callers whether we should call readpage or not. Also make it return an errno / 0 / AOP_TRUNCATED_PAGE, and make calling put_page() the caller's responsibility. Link: https://lkml.kernel.org/r/20210122160140.223228-10-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig Cc: Miaohe Lin Signed-off-by: Andrew Morton --- mm/filemap.c | 89 +++++++++++++++++++++++-------------------------- 1 file changed, 42 insertions(+), 47 deletions(-) --- a/mm/filemap.c~mm-filemap-change-filemap_read_page-calling-conventions +++ a/mm/filemap.c @@ -2199,56 +2199,38 @@ retry: rcu_read_unlock(); } -static struct page *filemap_read_page(struct kiocb *iocb, struct file *filp, - struct address_space *mapping, struct page *page) +static int filemap_read_page(struct file *file, struct address_space *mapping, + struct page *page) { - struct file_ra_state *ra = &filp->f_ra; int error; - if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { - unlock_page(page); - put_page(page); - return ERR_PTR(-EAGAIN); - } - /* - * A previous I/O error may have been due to temporary - * failures, eg. multipath errors. - * PG_error will be set again if readpage fails. + * A previous I/O error may have been due to temporary failures, + * eg. multipath errors. PG_error will be set again if readpage + * fails. */ ClearPageError(page); /* Start the actual read. The read will unlock the page. */ - error = mapping->a_ops->readpage(filp, page); - - if (unlikely(error)) { - put_page(page); - return error != AOP_TRUNCATED_PAGE ? ERR_PTR(error) : NULL; - } + error = mapping->a_ops->readpage(file, page); + if (error) + return error; + if (PageUptodate(page)) + return 0; + error = lock_page_killable(page); + if (error) + return error; if (!PageUptodate(page)) { - error = lock_page_killable(page); - if (unlikely(error)) { - put_page(page); - return ERR_PTR(error); - } - if (!PageUptodate(page)) { - if (page->mapping == NULL) { - /* - * invalidate_mapping_pages got it - */ - unlock_page(page); - put_page(page); - return NULL; - } - unlock_page(page); - shrink_readahead_size_eio(ra); - put_page(page); - return ERR_PTR(-EIO); + if (page->mapping == NULL) { + /* page truncated */ + error = AOP_TRUNCATED_PAGE; + } else { + shrink_readahead_size_eio(&file->f_ra); + error = -EIO; } - unlock_page(page); } - - return page; + unlock_page(page); + return error; } static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, @@ -2290,7 +2272,18 @@ uptodate: return page; readpage: - return filemap_read_page(iocb, filp, mapping, page); + if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { + unlock_page(page); + put_page(page); + return ERR_PTR(-EAGAIN); + } + error = filemap_read_page(iocb->ki_filp, mapping, page); + if (!error) + return page; + put_page(page); + if (error == AOP_TRUNCATED_PAGE) + return NULL; + return ERR_PTR(error); truncated: unlock_page(page); put_page(page); @@ -2306,7 +2299,7 @@ static struct page *filemap_create_page( struct page *page; int error; - if (iocb->ki_flags & IOCB_NOIO) + if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) return ERR_PTR(-EAGAIN); page = page_cache_alloc(mapping); @@ -2315,12 +2308,14 @@ static struct page *filemap_create_page( error = add_to_page_cache_lru(page, mapping, index, mapping_gfp_constraint(mapping, GFP_KERNEL)); - if (error) { - put_page(page); - return error != -EEXIST ? ERR_PTR(error) : NULL; - } - - return filemap_read_page(iocb, filp, mapping, page); + if (!error) + error = filemap_read_page(iocb->ki_filp, mapping, page); + if (!error) + return page; + put_page(page); + if (error == -EEXIST || error == AOP_TRUNCATED_PAGE) + return NULL; + return ERR_PTR(error); } static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, From patchwork Wed Feb 24 20:02:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102491 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 115F9C433E0 for ; Wed, 24 Feb 2021 20:02:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9F53364F0F for ; Wed, 24 Feb 2021 20:02:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9F53364F0F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 361D36B0070; Wed, 24 Feb 2021 15:02:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 315346B0071; Wed, 24 Feb 2021 15:02:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24D518D0001; Wed, 24 Feb 2021 15:02:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0129.hostedemail.com [216.40.44.129]) by kanga.kvack.org (Postfix) with ESMTP id 0F5ED6B0070 for ; Wed, 24 Feb 2021 15:02:21 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id CE7968083CD1 for ; Wed, 24 Feb 2021 20:02:20 +0000 (UTC) X-FDA: 77854233240.17.604AC5D Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf13.hostedemail.com (Postfix) with ESMTP id 15D13E00012A for ; Wed, 24 Feb 2021 20:02:17 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 3222C64F08; Wed, 24 Feb 2021 20:02:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196939; bh=CdXgBHefHzbdNfMiTx+xiAkE87/5qlsEwwhQUWVm3yA=; h=Date:From:To:Subject:In-Reply-To:From; b=X6DQ+bydTn1344ece3FBlJ7+1YJlSA2olm02i7d+cipmofYHNU/0tvuUqd9H/aiFj MN95DjIBi8MVbkWYYg6eLO4WPl1wxYTjNDqZdkm2bwvoX7WcJIhl+nj96edHMm6ibZ fFOTgewVsFsqRuIBu1vLcmT9zplKOQXiEW+zIMg4= Date: Wed, 24 Feb 2021 12:02:18 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 039/173] mm/filemap: change filemap_create_page calling conventions Message-ID: <20210224200218.GOP7HnWLS%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 15D13E00012A X-Stat-Signature: 565ti7whm8mxjpriqyb95sfh4qzqoe65 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf13; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196937-585099 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/filemap: change filemap_create_page calling conventions By moving the iocb flag checks to the caller, we can pass the file and the page index instead of the iocb. It never needed the iter. By passing the pagevec, we can return an errno (or AOP_TRUNCATED_PAGE) instead of an ERR_PTR. Link: https://lkml.kernel.org/r/20210122160140.223228-11-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig Cc: Miaohe Lin Signed-off-by: Andrew Morton --- mm/filemap.c | 53 ++++++++++++++++++++++++------------------------- 1 file changed, 27 insertions(+), 26 deletions(-) --- a/mm/filemap.c~mm-filemap-change-filemap_create_page-calling-conventions +++ a/mm/filemap.c @@ -2290,32 +2290,33 @@ truncated: return NULL; } -static struct page *filemap_create_page(struct kiocb *iocb, - struct iov_iter *iter) +static int filemap_create_page(struct file *file, + struct address_space *mapping, pgoff_t index, + struct pagevec *pvec) { - struct file *filp = iocb->ki_filp; - struct address_space *mapping = filp->f_mapping; - pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; struct page *page; int error; - if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) - return ERR_PTR(-EAGAIN); - page = page_cache_alloc(mapping); if (!page) - return ERR_PTR(-ENOMEM); + return -ENOMEM; error = add_to_page_cache_lru(page, mapping, index, - mapping_gfp_constraint(mapping, GFP_KERNEL)); - if (!error) - error = filemap_read_page(iocb->ki_filp, mapping, page); - if (!error) - return page; + mapping_gfp_constraint(mapping, GFP_KERNEL)); + if (error == -EEXIST) + error = AOP_TRUNCATED_PAGE; + if (error) + goto error; + + error = filemap_read_page(file, mapping, page); + if (error) + goto error; + + pagevec_add(pvec, page); + return 0; +error: put_page(page); - if (error == -EEXIST || error == AOP_TRUNCATED_PAGE) - return NULL; - return ERR_PTR(error); + return error; } static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, @@ -2343,15 +2344,15 @@ find_page: page_cache_sync_readahead(mapping, ra, filp, index, last_index - index); filemap_get_read_batch(mapping, index, last_index, pvec); - if (pvec->nr) - goto got_pages; - - pvec->pages[0] = filemap_create_page(iocb, iter); - err = PTR_ERR_OR_ZERO(pvec->pages[0]); - if (IS_ERR_OR_NULL(pvec->pages[0])) - goto err; - pvec->nr = 1; - return 0; + if (!pagevec_count(pvec)) { + if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) + return -EAGAIN; + err = filemap_create_page(filp, mapping, + iocb->ki_pos >> PAGE_SHIFT, pvec); + if (err == AOP_TRUNCATED_PAGE) + goto find_page; + return err; + } got_pages: { struct page *page = pvec->pages[pvec->nr - 1]; From patchwork Wed Feb 24 20:02:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102493 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CCAB0C433E0 for ; Wed, 24 Feb 2021 20:02:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6A00564EC3 for ; Wed, 24 Feb 2021 20:02:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6A00564EC3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 02D826B0071; Wed, 24 Feb 2021 15:02:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EF77D6B0072; Wed, 24 Feb 2021 15:02:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E0EC68D0001; Wed, 24 Feb 2021 15:02:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0073.hostedemail.com [216.40.44.73]) by kanga.kvack.org (Postfix) with ESMTP id C84386B0071 for ; Wed, 24 Feb 2021 15:02:24 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 89137185044B4 for ; Wed, 24 Feb 2021 20:02:24 +0000 (UTC) X-FDA: 77854233408.29.61545EC Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP id D9B9FC0007C7 for ; Wed, 24 Feb 2021 20:02:16 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8306964F09; Wed, 24 Feb 2021 20:02:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196943; bh=0hLm6k8UHSFQXHvK/O2/yipiT2IcubYRnrUJk3x4j9w=; h=Date:From:To:Subject:In-Reply-To:From; b=iPhKYVP6BnX90nSy51BX5HA/RSJ1JNblKUfcLJAPDxYgCMHuwP9EOVUI+XTKBsuun x1XiYLb80ZIPNpYTvak16VjC7PZQIUIq7ittYvxn4Fp0K2jVkFhGgjqp2L9OepngVt sG1wwWOujRhhRrjhck+IkC8WlmOOii7+PZN9ZSdg= Date: Wed, 24 Feb 2021 12:02:22 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 040/173] mm/filemap: convert filemap_update_page to return an errno Message-ID: <20210224200222.XcdktnE5J%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: D9B9FC0007C7 X-Stat-Signature: rhq3mwzxxugy8wyrhd8sdii7tc9wp65m Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf14; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196936-635046 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/filemap: convert filemap_update_page to return an errno Use AOP_TRUNCATED_PAGE to indicate that no error occurred, but the page we looked up is no longer valid. In this case, the reference to the page will have been removed; if we hit any other error, the caller will release the reference. Link: https://lkml.kernel.org/r/20210122160140.223228-12-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig Cc: Miaohe Lin Signed-off-by: Andrew Morton --- mm/filemap.c | 38 +++++++++++++++++--------------------- 1 file changed, 17 insertions(+), 21 deletions(-) --- a/mm/filemap.c~mm-filemap-convert-filemap_update_page-to-return-an-errno +++ a/mm/filemap.c @@ -2233,24 +2233,21 @@ static int filemap_read_page(struct file return error; } -static struct page *filemap_update_page(struct kiocb *iocb, struct file *filp, - struct iov_iter *iter, struct page *page, loff_t pos, - loff_t count) +static int filemap_update_page(struct kiocb *iocb, + struct address_space *mapping, struct iov_iter *iter, + struct page *page, loff_t pos, loff_t count) { - struct address_space *mapping = filp->f_mapping; struct inode *inode = mapping->host; int error; if (iocb->ki_flags & IOCB_WAITQ) { error = lock_page_async(page, iocb->ki_waitq); - if (error) { - put_page(page); - return ERR_PTR(error); - } + if (error) + return error; } else { if (!trylock_page(page)) { put_and_wait_on_page_locked(page, TASK_KILLABLE); - return NULL; + return AOP_TRUNCATED_PAGE; } } @@ -2269,25 +2266,21 @@ static struct page *filemap_update_page( goto readpage; uptodate: unlock_page(page); - return page; + return 0; readpage: if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { unlock_page(page); - put_page(page); - return ERR_PTR(-EAGAIN); + return -EAGAIN; } error = filemap_read_page(iocb->ki_filp, mapping, page); - if (!error) - return page; - put_page(page); if (error == AOP_TRUNCATED_PAGE) - return NULL; - return ERR_PTR(error); + put_page(page); + return error; truncated: unlock_page(page); put_page(page); - return NULL; + return AOP_TRUNCATED_PAGE; } static int filemap_create_page(struct file *file, @@ -2381,11 +2374,12 @@ got_pages: goto err; } - page = filemap_update_page(iocb, filp, iter, page, + err = filemap_update_page(iocb, mapping, iter, page, pg_pos, pg_count); - if (IS_ERR_OR_NULL(page)) { + if (err) { + if (err < 0) + put_page(page); pvec->nr--; - err = PTR_ERR_OR_ZERO(page); } } } @@ -2393,6 +2387,8 @@ got_pages: err: if (likely(pvec->nr)) return 0; + if (err == AOP_TRUNCATED_PAGE) + goto find_page; if (err) return err; /* From patchwork Wed Feb 24 20:02:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102495 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 551DEC433DB for ; Wed, 24 Feb 2021 20:02:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E205464F11 for ; Wed, 24 Feb 2021 20:02:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E205464F11 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6E30A6B0072; Wed, 24 Feb 2021 15:02:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 695AA6B0073; Wed, 24 Feb 2021 15:02:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A8976B0074; Wed, 24 Feb 2021 15:02:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0187.hostedemail.com [216.40.44.187]) by kanga.kvack.org (Postfix) with ESMTP id 42E9B6B0072 for ; Wed, 24 Feb 2021 15:02:28 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 0DFA1F042 for ; Wed, 24 Feb 2021 20:02:28 +0000 (UTC) X-FDA: 77854233576.19.862CB74 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP id D75D8130 for ; Wed, 24 Feb 2021 20:02:25 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0925164F0F; Wed, 24 Feb 2021 20:02:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196946; bh=50RIy6gSVGR+5nbDqH9C0gXkkbUIElU5fJL77nzYvfw=; h=Date:From:To:Subject:In-Reply-To:From; b=myw2uQA8sBOF5+XPNimNk9n/hQcCGChZz6qld3KilPpUFsrnTcm6I2YEIs/KPMmBC /g8aL1SfEGHauHXNZTKdhUDJyorFoSdKRSENeAtFV0b0eVfFR3z6hjhnPvFFCKsoaj dnvr4UUUwpuZ84MnhS9nmeAkEsKS9CMnHyRB3Nx0= Date: Wed, 24 Feb 2021 12:02:25 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 041/173] mm/filemap: move the iocb checks into filemap_update_page Message-ID: <20210224200225.5JimbKIaH%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: D75D8130 X-Stat-Signature: 1qurkhirowepxdq5qzy1w5z6cbhahtpd Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf04; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196945-60632 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/filemap: move the iocb checks into filemap_update_page We don't need to give up when a non-blocking request sees a !Uptodate page. We may be able to satisfy the read from a partially-uptodate page. Link: https://lkml.kernel.org/r/20210122160140.223228-13-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig Cc: Miaohe Lin Signed-off-by: Andrew Morton --- mm/filemap.c | 24 ++++++++++-------------- 1 file changed, 10 insertions(+), 14 deletions(-) --- a/mm/filemap.c~mm-filemap-move-the-iocb-checks-into-filemap_update_page +++ a/mm/filemap.c @@ -2240,15 +2240,16 @@ static int filemap_update_page(struct ki struct inode *inode = mapping->host; int error; - if (iocb->ki_flags & IOCB_WAITQ) { - error = lock_page_async(page, iocb->ki_waitq); - if (error) - return error; - } else { - if (!trylock_page(page)) { + if (!trylock_page(page)) { + if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_NOIO)) + return -EAGAIN; + if (!(iocb->ki_flags & IOCB_WAITQ)) { put_and_wait_on_page_locked(page, TASK_KILLABLE); return AOP_TRUNCATED_PAGE; } + error = __lock_page_async(page, iocb->ki_waitq); + if (error) + return error; } if (!page->mapping) @@ -2366,14 +2367,9 @@ got_pages: } if (!PageUptodate(page)) { - if ((iocb->ki_flags & IOCB_NOWAIT) || - ((iocb->ki_flags & IOCB_WAITQ) && pvec->nr > 1)) { - put_page(page); - pvec->nr--; - err = -EAGAIN; - goto err; - } - + if ((iocb->ki_flags & IOCB_WAITQ) && + pagevec_count(pvec) > 1) + iocb->ki_flags |= IOCB_NOWAIT; err = filemap_update_page(iocb, mapping, iter, page, pg_pos, pg_count); if (err) { From patchwork Wed Feb 24 20:02:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102499 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FA17C433E6 for ; Wed, 24 Feb 2021 20:02:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E1F2E64F1B for ; Wed, 24 Feb 2021 20:02:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E1F2E64F1B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7CE1A6B006C; Wed, 24 Feb 2021 15:02:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 780746B0073; Wed, 24 Feb 2021 15:02:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 648E58D0001; Wed, 24 Feb 2021 15:02:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0091.hostedemail.com [216.40.44.91]) by kanga.kvack.org (Postfix) with ESMTP id 4FCA76B006C for ; Wed, 24 Feb 2021 15:02:36 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 1662218500A2D for ; Wed, 24 Feb 2021 20:02:36 +0000 (UTC) X-FDA: 77854233912.03.590749C Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP id 43383C0007CA for ; Wed, 24 Feb 2021 20:02:28 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5049964F14; Wed, 24 Feb 2021 20:02:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196949; bh=cl43UZV3YQvoxhUQicgJiuTx+VGzf0FHHHIdTHt3ITE=; h=Date:From:To:Subject:In-Reply-To:From; b=JXwasE6KUBHAK8feAsAOgc2EI3qkrb/h57+3VOg7yzii4SvHb2qJjH9tTLJEMCxAy uvR3Y7GNudPOD6ecREabcZSviG4xPVEY9XOAB3InzPifw/WF4JNjG+hWN9b06fF0jf Kvm8b9LKQsX6TDM9GbvIYSiKtphOUdJvG+sr+6Ak= Date: Wed, 24 Feb 2021 12:02:28 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 042/173] mm/filemap: add filemap_range_uptodate Message-ID: <20210224200228.uUlzvUdtO%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 43383C0007CA X-Stat-Signature: aza4oqxpqd77946t8tgymfpkyz7hj4rx Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf14; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196948-792236 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/filemap: add filemap_range_uptodate Move the complicated condition and the calculations out of filemap_update_page() into its own function. [willy@infradead.org: unlock page before dropping its refcount] Link: https://lkml.kernel.org/r/20210201125229.GO308988@casper.infradead.org Link: https://lkml.kernel.org/r/20210122160140.223228-14-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig Cc: Miaohe Lin Signed-off-by: Andrew Morton --- mm/filemap.c | 65 +++++++++++++++++++++++++++++-------------------- 1 file changed, 39 insertions(+), 26 deletions(-) --- a/mm/filemap.c~mm-filemap-add-filemap_range_uptodate +++ a/mm/filemap.c @@ -2233,11 +2233,36 @@ static int filemap_read_page(struct file return error; } +static bool filemap_range_uptodate(struct address_space *mapping, + loff_t pos, struct iov_iter *iter, struct page *page) +{ + int count; + + if (PageUptodate(page)) + return true; + /* pipes can't handle partially uptodate pages */ + if (iov_iter_is_pipe(iter)) + return false; + if (!mapping->a_ops->is_partially_uptodate) + return false; + if (mapping->host->i_blkbits >= (PAGE_SHIFT + thp_order(page))) + return false; + + count = iter->count; + if (page_offset(page) > pos) { + count -= page_offset(page) - pos; + pos = 0; + } else { + pos -= page_offset(page); + } + + return mapping->a_ops->is_partially_uptodate(page, pos, count); +} + static int filemap_update_page(struct kiocb *iocb, struct address_space *mapping, struct iov_iter *iter, - struct page *page, loff_t pos, loff_t count) + struct page *page) { - struct inode *inode = mapping->host; int error; if (!trylock_page(page)) { @@ -2254,26 +2279,15 @@ static int filemap_update_page(struct ki if (!page->mapping) goto truncated; - if (PageUptodate(page)) - goto uptodate; - if (inode->i_blkbits == PAGE_SHIFT || - !mapping->a_ops->is_partially_uptodate) - goto readpage; - /* pipes can't handle partially uptodate pages */ - if (unlikely(iov_iter_is_pipe(iter))) - goto readpage; - if (!mapping->a_ops->is_partially_uptodate(page, - pos & (thp_size(page) - 1), count)) - goto readpage; -uptodate: - unlock_page(page); - return 0; -readpage: - if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) { - unlock_page(page); - return -EAGAIN; - } + error = 0; + if (filemap_range_uptodate(mapping, iocb->ki_pos, iter, page)) + goto unlock; + + error = -EAGAIN; + if (iocb->ki_flags & (IOCB_NOIO | IOCB_NOWAIT | IOCB_WAITQ)) + goto unlock; + error = filemap_read_page(iocb->ki_filp, mapping, page); if (error == AOP_TRUNCATED_PAGE) put_page(page); @@ -2282,6 +2296,9 @@ truncated: unlock_page(page); put_page(page); return AOP_TRUNCATED_PAGE; +unlock: + unlock_page(page); + return error; } static int filemap_create_page(struct file *file, @@ -2351,9 +2368,6 @@ got_pages: { struct page *page = pvec->pages[pvec->nr - 1]; pgoff_t pg_index = page->index; - loff_t pg_pos = max(iocb->ki_pos, - (loff_t) pg_index << PAGE_SHIFT); - loff_t pg_count = iocb->ki_pos + iter->count - pg_pos; if (PageReadahead(page)) { if (iocb->ki_flags & IOCB_NOIO) { @@ -2370,8 +2384,7 @@ got_pages: if ((iocb->ki_flags & IOCB_WAITQ) && pagevec_count(pvec) > 1) iocb->ki_flags |= IOCB_NOWAIT; - err = filemap_update_page(iocb, mapping, iter, page, - pg_pos, pg_count); + err = filemap_update_page(iocb, mapping, iter, page); if (err) { if (err < 0) put_page(page); From patchwork Wed Feb 24 20:02:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102497 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB68BC433DB for ; Wed, 24 Feb 2021 20:02:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6AB9B64F1B for ; Wed, 24 Feb 2021 20:02:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6AB9B64F1B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 02B9F6B0006; Wed, 24 Feb 2021 15:02:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F1E516B006C; Wed, 24 Feb 2021 15:02:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E5C8B6B0073; Wed, 24 Feb 2021 15:02:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0104.hostedemail.com [216.40.44.104]) by kanga.kvack.org (Postfix) with ESMTP id CE6886B0006 for ; Wed, 24 Feb 2021 15:02:34 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 91203ABEB for ; Wed, 24 Feb 2021 20:02:34 +0000 (UTC) X-FDA: 77854233828.08.FD99125 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf02.hostedemail.com (Postfix) with ESMTP id A4C41407F8DF for ; Wed, 24 Feb 2021 20:02:20 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8844164F11; Wed, 24 Feb 2021 20:02:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196953; bh=ZJP+b3ad9NR5VUxyODSnTAxkn39JSQ/wkrFSj1+3kz0=; h=Date:From:To:Subject:In-Reply-To:From; b=fEwUSy3WJ9Wjg4GITnWwqk5g6QCNQ2oz7ufFKjm/kXPCgi8LBbsNYYL0vOWdXIVzT dGsSvVPWGC8lVBJpVDN0W6xMOIQMQwDZt99cR/bv6IC54I/KoPBaFd0WPVCKCsu2Pl P9tEiOM8XofP4vDuqW53XRlh4Ax3JWhUhpXCx0tQ= Date: Wed, 24 Feb 2021 12:02:32 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 043/173] mm/filemap: split filemap_readahead out of filemap_get_pages Message-ID: <20210224200232.DQJ_71Mup%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: A4C41407F8DF X-Stat-Signature: 51yybjqsha9ots47bsn5i613njuigoak Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf02; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196940-77236 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/filemap: split filemap_readahead out of filemap_get_pages This simplifies the error handling. Link: https://lkml.kernel.org/r/20210122160140.223228-15-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig Cc: Miaohe Lin Signed-off-by: Andrew Morton --- mm/filemap.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) --- a/mm/filemap.c~mm-filemap-split-filemap_readahead-out-of-filemap_get_pages +++ a/mm/filemap.c @@ -2330,6 +2330,17 @@ error: return error; } +static int filemap_readahead(struct kiocb *iocb, struct file *file, + struct address_space *mapping, struct page *page, + pgoff_t last_index) +{ + if (iocb->ki_flags & IOCB_NOIO) + return -EAGAIN; + page_cache_async_readahead(mapping, &file->f_ra, file, page, + page->index, last_index - page->index); + return 0; +} + static int filemap_get_pages(struct kiocb *iocb, struct iov_iter *iter, struct pagevec *pvec) { @@ -2367,17 +2378,15 @@ find_page: got_pages: { struct page *page = pvec->pages[pvec->nr - 1]; - pgoff_t pg_index = page->index; if (PageReadahead(page)) { - if (iocb->ki_flags & IOCB_NOIO) { + err = filemap_readahead(iocb, filp, mapping, page, + last_index); + if (err) { put_page(page); pvec->nr--; - err = -EAGAIN; goto err; } - page_cache_async_readahead(mapping, ra, filp, page, - pg_index, last_index - pg_index); } if (!PageUptodate(page)) { From patchwork Wed Feb 24 20:02:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102501 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAB99C433DB for ; Wed, 24 Feb 2021 20:02:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5450B64F1F for ; Wed, 24 Feb 2021 20:02:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5450B64F1F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 66FDF8D0001; Wed, 24 Feb 2021 15:02:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 61CD96B0074; Wed, 24 Feb 2021 15:02:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5348D8D0001; Wed, 24 Feb 2021 15:02:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0078.hostedemail.com [216.40.44.78]) by kanga.kvack.org (Postfix) with ESMTP id 37E886B0073 for ; Wed, 24 Feb 2021 15:02:38 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 0153CF04F for ; Wed, 24 Feb 2021 20:02:37 +0000 (UTC) X-FDA: 77854233996.20.F2DE03F Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf06.hostedemail.com (Postfix) with ESMTP id B8FABC0007D2 for ; Wed, 24 Feb 2021 20:02:38 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id D7CE164EC3; Wed, 24 Feb 2021 20:02:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196956; bh=IZFR+1MgXfbwAerHWVzCbYbdrlC2nLdj5MY1wrhuZhQ=; h=Date:From:To:Subject:In-Reply-To:From; b=UmpHEu8GFFXroVVuozOB6qTgKj8aof9L5snAEEGJGj/Pq/rR5nrsqQ3YTdN0Y6Taj ccqKd7eLj8MiKb5Ttwca0HqiCsKyfYVWUBQpzH1w0KJxQOFdV9eBa+rLPFlmOmNNlO ZinVEnrpZSHNp9i4/10/ZrzYUHkNWpz8UnnuScPY= Date: Wed, 24 Feb 2021 12:02:35 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 044/173] mm/filemap: restructure filemap_get_pages Message-ID: <20210224200235.QaAy8bCLA%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: e7uyp8wj1npicj3qzhkk58gkhskdeiij X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: B8FABC0007D2 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf06; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196958-567213 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/filemap: restructure filemap_get_pages Remove the got_pages label, remove indentation, rename find_page to retry, simplify error handling. Link: https://lkml.kernel.org/r/20210122160140.223228-16-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig Cc: Miaohe Lin Signed-off-by: Andrew Morton --- mm/filemap.c | 71 +++++++++++++++++++------------------------------ 1 file changed, 28 insertions(+), 43 deletions(-) --- a/mm/filemap.c~mm-filemap-restructure-filemap_get_pages +++ a/mm/filemap.c @@ -2349,70 +2349,55 @@ static int filemap_get_pages(struct kioc struct file_ra_state *ra = &filp->f_ra; pgoff_t index = iocb->ki_pos >> PAGE_SHIFT; pgoff_t last_index; + struct page *page; int err = 0; last_index = DIV_ROUND_UP(iocb->ki_pos + iter->count, PAGE_SIZE); -find_page: +retry: if (fatal_signal_pending(current)) return -EINTR; filemap_get_read_batch(mapping, index, last_index, pvec); - if (pvec->nr) - goto got_pages; - - if (iocb->ki_flags & IOCB_NOIO) - return -EAGAIN; - - page_cache_sync_readahead(mapping, ra, filp, index, last_index - index); - - filemap_get_read_batch(mapping, index, last_index, pvec); + if (!pagevec_count(pvec)) { + if (iocb->ki_flags & IOCB_NOIO) + return -EAGAIN; + page_cache_sync_readahead(mapping, ra, filp, index, + last_index - index); + filemap_get_read_batch(mapping, index, last_index, pvec); + } if (!pagevec_count(pvec)) { if (iocb->ki_flags & (IOCB_NOWAIT | IOCB_WAITQ)) return -EAGAIN; err = filemap_create_page(filp, mapping, iocb->ki_pos >> PAGE_SHIFT, pvec); if (err == AOP_TRUNCATED_PAGE) - goto find_page; + goto retry; return err; } -got_pages: - { - struct page *page = pvec->pages[pvec->nr - 1]; - - if (PageReadahead(page)) { - err = filemap_readahead(iocb, filp, mapping, page, - last_index); - if (err) { - put_page(page); - pvec->nr--; - goto err; - } - } - if (!PageUptodate(page)) { - if ((iocb->ki_flags & IOCB_WAITQ) && - pagevec_count(pvec) > 1) - iocb->ki_flags |= IOCB_NOWAIT; - err = filemap_update_page(iocb, mapping, iter, page); - if (err) { - if (err < 0) - put_page(page); - pvec->nr--; - } - } + page = pvec->pages[pagevec_count(pvec) - 1]; + if (PageReadahead(page)) { + err = filemap_readahead(iocb, filp, mapping, page, last_index); + if (err) + goto err; + } + if (!PageUptodate(page)) { + if ((iocb->ki_flags & IOCB_WAITQ) && pagevec_count(pvec) > 1) + iocb->ki_flags |= IOCB_NOWAIT; + err = filemap_update_page(iocb, mapping, iter, page); + if (err) + goto err; } + return 0; err: - if (likely(pvec->nr)) + if (err < 0) + put_page(page); + if (likely(--pvec->nr)) return 0; if (err == AOP_TRUNCATED_PAGE) - goto find_page; - if (err) - return err; - /* - * No pages and no error means we raced and should retry: - */ - goto find_page; + goto retry; + return err; } /** From patchwork Wed Feb 24 20:02:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102503 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74ACDC433DB for ; Wed, 24 Feb 2021 20:02:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0EDE164F1F for ; Wed, 24 Feb 2021 20:02:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0EDE164F1F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 92B658D0003; Wed, 24 Feb 2021 15:02:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8DC126B0073; Wed, 24 Feb 2021 15:02:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81B288D0003; Wed, 24 Feb 2021 15:02:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0094.hostedemail.com [216.40.44.94]) by kanga.kvack.org (Postfix) with ESMTP id 6B6646B0071 for ; Wed, 24 Feb 2021 15:02:41 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 39A15807D9E4 for ; Wed, 24 Feb 2021 20:02:41 +0000 (UTC) X-FDA: 77854234122.04.4E1724E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf28.hostedemail.com (Postfix) with ESMTP id A3CE2200039F for ; Wed, 24 Feb 2021 20:02:40 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6B9BD64F29; Wed, 24 Feb 2021 20:02:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196959; bh=ExvvksIdO4A/g3Doeu+xtkJ20UGN+WKkXDwwypdpzgk=; h=Date:From:To:Subject:In-Reply-To:From; b=ykLFk8/VuJylHJ+e77s8jaqnXXIVOmRpaGTfJoC/Yf8aBZuwdpmvgcvqOuIvU464a ZouQCUsMfqx7rEIFeSwqwaxH3oUEGxXMqI/yoITa8pisSTERJEXQHt3L5WcyCOn86Y MWzQ0S07OaU+Hn3z73T60rxL9P5Pi09LT0QokvyE= Date: Wed, 24 Feb 2021 12:02:38 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 045/173] mm/filemap: don't relock the page after calling readpage Message-ID: <20210224200238.BXxudVT6w%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: igk5hkodnrn6kc8t33phaw5eis4dnn53 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: A3CE2200039F Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf28; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196960-270171 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/filemap: don't relock the page after calling readpage We don't need to get the page lock again; we just need to wait for the I/O to finish, so use wait_on_page_locked_killable() like the other callers of ->readpage. Link: https://lkml.kernel.org/r/20210122160140.223228-17-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Reviewed-by: Christoph Hellwig Cc: Miaohe Lin Signed-off-by: Andrew Morton --- mm/filemap.c | 21 +++++++-------------- 1 file changed, 7 insertions(+), 14 deletions(-) --- a/mm/filemap.c~mm-filemap-dont-relock-the-page-after-calling-readpage +++ a/mm/filemap.c @@ -2214,23 +2214,16 @@ static int filemap_read_page(struct file error = mapping->a_ops->readpage(file, page); if (error) return error; - if (PageUptodate(page)) - return 0; - error = lock_page_killable(page); + error = wait_on_page_locked_killable(page); if (error) return error; - if (!PageUptodate(page)) { - if (page->mapping == NULL) { - /* page truncated */ - error = AOP_TRUNCATED_PAGE; - } else { - shrink_readahead_size_eio(&file->f_ra); - error = -EIO; - } - } - unlock_page(page); - return error; + if (PageUptodate(page)) + return 0; + if (!page->mapping) /* page truncated */ + return AOP_TRUNCATED_PAGE; + shrink_readahead_size_eio(&file->f_ra); + return -EIO; } static bool filemap_range_uptodate(struct address_space *mapping, From patchwork Wed Feb 24 20:02:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102505 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F7A4C433E0 for ; Wed, 24 Feb 2021 20:02:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E6E1D64F24 for ; Wed, 24 Feb 2021 20:02:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E6E1D64F24 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9045A8D0005; Wed, 24 Feb 2021 15:02:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B5E16B0073; Wed, 24 Feb 2021 15:02:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7C9EF8D0005; Wed, 24 Feb 2021 15:02:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0102.hostedemail.com [216.40.44.102]) by kanga.kvack.org (Postfix) with ESMTP id 67F996B0071 for ; Wed, 24 Feb 2021 15:02:46 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 2BABB184E6507 for ; Wed, 24 Feb 2021 20:02:46 +0000 (UTC) X-FDA: 77854234332.21.CBE3D50 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf15.hostedemail.com (Postfix) with ESMTP id 0B954A0000FA for ; Wed, 24 Feb 2021 20:02:42 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C719364F1B; Wed, 24 Feb 2021 20:02:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196963; bh=3407BZQKaLL3VUVCUT6HrkwmWmQupFU/Ni4YH9UKCwc=; h=Date:From:To:Subject:In-Reply-To:From; b=Q4z7P4XckNZF50qdLV65r069/jyf41Gert83mSOxbj6vEDQoJ2+GNMIpyIEEegOwQ sHZgxq6yXNV1VAFNvFO0fpsLKLYaAync4uvvNgP7iE/ky789DDYC54Osh1GLaWFwnF KkCzGt3aTEu8TV5R8nFlJJm4EVteIx/CkMlI/vQA= Date: Wed, 24 Feb 2021 12:02:42 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 046/173] mm/filemap: rename generic_file_buffered_read to filemap_read Message-ID: <20210224200242.DUC-5p9cw%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 0B954A0000FA X-Stat-Signature: yqzg8hk88d31juo6rab16syuwme7bhjs Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf15; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196962-333296 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Subject: mm/filemap: rename generic_file_buffered_read to filemap_read Rename generic_file_buffered_read to match the naming of filemap_fault, also update the written parameter to a more descriptive name and improve the kerneldoc comment. Link: https://lkml.kernel.org/r/20210122160140.223228-18-willy@infradead.org Signed-off-by: Christoph Hellwig Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Cc: Miaohe Lin Signed-off-by: Andrew Morton --- fs/btrfs/file.c | 2 +- include/linux/fs.h | 4 ++-- mm/filemap.c | 39 ++++++++++++++++++--------------------- 3 files changed, 21 insertions(+), 24 deletions(-) --- a/fs/btrfs/file.c~mm-filemap-rename-generic_file_buffered_read-to-filemap_read +++ a/fs/btrfs/file.c @@ -3634,7 +3634,7 @@ static ssize_t btrfs_file_read_iter(stru return ret; } - return generic_file_buffered_read(iocb, to, ret); + return filemap_read(iocb, to, ret); } const struct file_operations btrfs_file_operations = { --- a/include/linux/fs.h~mm-filemap-rename-generic_file_buffered_read-to-filemap_read +++ a/include/linux/fs.h @@ -3080,8 +3080,8 @@ extern ssize_t generic_write_checks(stru extern int generic_write_check_limits(struct file *file, loff_t pos, loff_t *count); extern int generic_file_rw_checks(struct file *file_in, struct file *file_out); -extern ssize_t generic_file_buffered_read(struct kiocb *iocb, - struct iov_iter *to, ssize_t already_read); +ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *to, + ssize_t already_read); extern ssize_t generic_file_read_iter(struct kiocb *, struct iov_iter *); extern ssize_t __generic_file_write_iter(struct kiocb *, struct iov_iter *); extern ssize_t generic_file_write_iter(struct kiocb *, struct iov_iter *); --- a/mm/filemap.c~mm-filemap-rename-generic_file_buffered_read-to-filemap_read +++ a/mm/filemap.c @@ -2394,23 +2394,20 @@ err: } /** - * generic_file_buffered_read - generic file read routine - * @iocb: the iocb to read - * @iter: data destination - * @written: already copied - * - * This is a generic file read routine, and uses the - * mapping->a_ops->readpage() function for the actual low-level stuff. - * - * This is really ugly. But the goto's actually try to clarify some - * of the logic when it comes to error handling etc. - * - * Return: - * * total number of bytes copied, including those the were already @written - * * negative error code if nothing was copied + * filemap_read - Read data from the page cache. + * @iocb: The iocb to read. + * @iter: Destination for the data. + * @already_read: Number of bytes already read by the caller. + * + * Copies data from the page cache. If the data is not currently present, + * uses the readahead and readpage address_space operations to fetch it. + * + * Return: Total number of bytes copied, including those already read by + * the caller. If an error happens before any bytes are copied, returns + * a negative error number. */ -ssize_t generic_file_buffered_read(struct kiocb *iocb, - struct iov_iter *iter, ssize_t written) +ssize_t filemap_read(struct kiocb *iocb, struct iov_iter *iter, + ssize_t already_read) { struct file *filp = iocb->ki_filp; struct file_ra_state *ra = &filp->f_ra; @@ -2437,7 +2434,7 @@ ssize_t generic_file_buffered_read(struc * can no longer safely return -EIOCBQUEUED. Hence mark * an async read NOWAIT at that point. */ - if ((iocb->ki_flags & IOCB_WAITQ) && written) + if ((iocb->ki_flags & IOCB_WAITQ) && already_read) iocb->ki_flags |= IOCB_NOWAIT; error = filemap_get_pages(iocb, iter, &pvec); @@ -2497,7 +2494,7 @@ ssize_t generic_file_buffered_read(struc copied = copy_page_to_iter(page, offset, bytes, iter); - written += copied; + already_read += copied; iocb->ki_pos += copied; ra->prev_pos = iocb->ki_pos; @@ -2514,9 +2511,9 @@ put_pages: file_accessed(filp); - return written ? written : error; + return already_read ? already_read : error; } -EXPORT_SYMBOL_GPL(generic_file_buffered_read); +EXPORT_SYMBOL_GPL(filemap_read); /** * generic_file_read_iter - generic filesystem read routine @@ -2591,7 +2588,7 @@ generic_file_read_iter(struct kiocb *ioc goto out; } - retval = generic_file_buffered_read(iocb, iter, retval); + retval = filemap_read(iocb, iter, retval); out: return retval; } From patchwork Wed Feb 24 20:02:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102507 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B7A3C433DB for ; Wed, 24 Feb 2021 20:02:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 03C3964F20 for ; Wed, 24 Feb 2021 20:02:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 03C3964F20 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 88E746B0071; Wed, 24 Feb 2021 15:02:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 840508D0007; Wed, 24 Feb 2021 15:02:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 731CE6B0074; Wed, 24 Feb 2021 15:02:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0238.hostedemail.com [216.40.44.238]) by kanga.kvack.org (Postfix) with ESMTP id 5D1586B0071 for ; Wed, 24 Feb 2021 15:02:48 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 19C1AFB60 for ; Wed, 24 Feb 2021 20:02:48 +0000 (UTC) X-FDA: 77854234416.11.E9A05FE Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf08.hostedemail.com (Postfix) with ESMTP id ECA2580192C6 for ; Wed, 24 Feb 2021 20:02:35 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 2480D64F1F; Wed, 24 Feb 2021 20:02:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196966; bh=AqNbbROry6vyOab/G1XkUcbNgncY1pD+FBLjyRYrMJI=; h=Date:From:To:Subject:In-Reply-To:From; b=RkbuNlgcJhCawzGExEhirDoZ1gmw5tf2B4jHVl00A9PYYH3a4Jk9QTCbm5NR6+z6W E8ymh/vomdXWa9/1q4u26WbSK5iRyS650G+ypvBnVGqPVQK4u4fYQOA+vWsw/f7wkM yWRMFVro4YAWagIKtxEcxbWid5OKez0QW3Qqx8Ag= Date: Wed, 24 Feb 2021 12:02:45 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hch@lst.de, kent.overstreet@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 047/173] mm/filemap: simplify generic_file_read_iter Message-ID: <20210224200245.GVV1PLlJl%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: f84oariowi63fjpb6yfmcz8zmot1qpb6 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: ECA2580192C6 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf08; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196955-601820 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Christoph Hellwig Subject: mm/filemap: simplify generic_file_read_iter Avoid the pointless goto out just for returning retval. Link: https://lkml.kernel.org/r/20210122160140.223228-19-willy@infradead.org Signed-off-by: Christoph Hellwig Signed-off-by: Matthew Wilcox (Oracle) Reviewed-by: Kent Overstreet Cc: Miaohe Lin Signed-off-by: Andrew Morton --- mm/filemap.c | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) --- a/mm/filemap.c~mm-filemap-simplify-generic_file_read_iter +++ a/mm/filemap.c @@ -2543,7 +2543,7 @@ generic_file_read_iter(struct kiocb *ioc ssize_t retval = 0; if (!count) - goto out; /* skip atime */ + return 0; /* skip atime */ if (iocb->ki_flags & IOCB_DIRECT) { struct file *file = iocb->ki_filp; @@ -2561,7 +2561,7 @@ generic_file_read_iter(struct kiocb *ioc iocb->ki_pos, iocb->ki_pos + count - 1); if (retval < 0) - goto out; + return retval; } file_accessed(file); @@ -2585,12 +2585,10 @@ generic_file_read_iter(struct kiocb *ioc */ if (retval < 0 || !count || iocb->ki_pos >= size || IS_DAX(inode)) - goto out; + return retval; } - retval = filemap_read(iocb, iter, retval); -out: - return retval; + return filemap_read(iocb, iter, retval); } EXPORT_SYMBOL(generic_file_read_iter); From patchwork Wed Feb 24 20:02:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102509 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E600C433E0 for ; Wed, 24 Feb 2021 20:02:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0E77A64E09 for ; Wed, 24 Feb 2021 20:02:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0E77A64E09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A1B168D000A; Wed, 24 Feb 2021 15:02:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 99EE78D0007; Wed, 24 Feb 2021 15:02:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 88DD08D000A; Wed, 24 Feb 2021 15:02:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0119.hostedemail.com [216.40.44.119]) by kanga.kvack.org (Postfix) with ESMTP id 6C39C8D0007 for ; Wed, 24 Feb 2021 15:02:51 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 34414185044A8 for ; Wed, 24 Feb 2021 20:02:51 +0000 (UTC) X-FDA: 77854234542.16.6B55999 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id 032952BC7 for ; Wed, 24 Feb 2021 20:02:44 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6B95D64F20; Wed, 24 Feb 2021 20:02:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196969; bh=aN4ApV2pMzQ/PNBdet6SPu7L+unhlAHyg8vhrUrR6sg=; h=Date:From:To:Subject:In-Reply-To:From; b=UrE/NdWpXL5rQp3U9EZioVPHE2XOZu8aY8D2TSkINtWY9QMmpFMEynO4ggDAjviya 2KnnefqIWEFg9MLr5nYZpdOCnrpVRWa5aPYjbu3DBz90sLdmGOzcTP+AigEIjZYZJ3 x0c5tD6TUvUC/NhE7zLnFCACE8MlbENSolbZWqys= Date: Wed, 24 Feb 2021 12:02:48 -0800 From: Andrew Morton To: akpm@linux-foundation.org, guoyang2@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, npiggin@suse.de, torvalds@linux-foundation.org, viro@zeniv.linux.org.uk, zhangshaokun@hisilicon.com Subject: [patch 048/173] fs/buffer.c: add checking buffer head stat before clear Message-ID: <20210224200248.86pqR4gin%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: bxqopwifqyta6sjgouw9ahi7zq1hgok5 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 032952BC7 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf12; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196964-158478 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Guo Subject: fs/buffer.c: add checking buffer head stat before clear clear_buffer_new() is used to clear buffer new stat. When PAGE_SIZE is 64K, most buffer heads in the list are not needed to clear. clear_buffer_new() has an enpensive atomic modification operation, Let's add checking buffer head before clear it as __block_write_begin_int does which is good for performance. Link: https://lkml.kernel.org/r/1612332890-57918-1-git-send-email-zhangshaokun@hisilicon.com Signed-off-by: Yang Guo Signed-off-by: Shaokun Zhang Cc: Alexander Viro Cc: Nick Piggin Signed-off-by: Andrew Morton --- fs/buffer.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) --- a/fs/buffer.c~fs-bufferc-add-checking-buffer-head-stat-before-clear +++ a/fs/buffer.c @@ -2083,7 +2083,8 @@ static int __block_commit_write(struct i set_buffer_uptodate(bh); mark_buffer_dirty(bh); } - clear_buffer_new(bh); + if (buffer_new(bh)) + clear_buffer_new(bh); block_start = block_end; bh = bh->b_this_page; From patchwork Wed Feb 24 20:02:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102511 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B610BC433E6 for ; Wed, 24 Feb 2021 20:02:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 588C764E2B for ; Wed, 24 Feb 2021 20:02:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 588C764E2B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E61498D000C; Wed, 24 Feb 2021 15:02:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DE4558D0007; Wed, 24 Feb 2021 15:02:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CFF168D000C; Wed, 24 Feb 2021 15:02:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0089.hostedemail.com [216.40.44.89]) by kanga.kvack.org (Postfix) with ESMTP id B24318D0007 for ; Wed, 24 Feb 2021 15:02:54 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 81A1118017BF0 for ; Wed, 24 Feb 2021 20:02:54 +0000 (UTC) X-FDA: 77854234668.01.74FE475 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP id 2A33690009E2 for ; Wed, 24 Feb 2021 20:02:50 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id CE2DE64E24; Wed, 24 Feb 2021 20:02:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196973; bh=HCPjiNwzWrwoCDwlL5BQXz4M25hz0CXJAsyLIbdAKII=; h=Date:From:To:Subject:In-Reply-To:From; b=2rErF54w8GZj5N9Ke90U/gF5Pd9DU2lmKSeDEddXOul43pfwcKZEm5j/tw79n8tym Hi6s965Kl3PXSQch2ZPJDVI8kD4EwiQQyHIBP0i8ks9JePRq8FbLCqOBm3HGrZwPPh 2QMhKDdM5JaNjUj4JwynEzUAMlapwbf9tQGaQ+dY= Date: Wed, 24 Feb 2021 12:02:52 -0800 From: Andrew Morton To: akpm@linux-foundation.org, baolin.wang@linux.alibaba.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 049/173] mm: backing-dev: Remove duplicated macro definition Message-ID: <20210224200252.0fTpxR4AC%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 9x8quutgzhaio7p53j3y54hx365bjbcd X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 2A33690009E2 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf19; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196970-92298 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Baolin Wang Subject: mm: backing-dev: Remove duplicated macro definition Move the K() macro a little forward to remove the same macro definition. Link: https://lkml.kernel.org/r/d1ccdf2d3116dce9814f2bcc1f0415ecb4c76ea5.1612862230.git.baolin.wang@linux.alibaba.com Signed-off-by: Baolin Wang Reviewed-by: Andrew Morton Signed-off-by: Andrew Morton --- mm/backing-dev.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) --- a/mm/backing-dev.c~mm-backing-dev-remove-duplicated-macro-definition +++ a/mm/backing-dev.c @@ -32,6 +32,8 @@ LIST_HEAD(bdi_list); /* bdi_wq serves all asynchronous writeback tasks */ struct workqueue_struct *bdi_wq; +#define K(x) ((x) << (PAGE_SHIFT - 10)) + #ifdef CONFIG_DEBUG_FS #include #include @@ -69,7 +71,6 @@ static int bdi_debug_stats_show(struct s global_dirty_limits(&background_thresh, &dirty_thresh); wb_thresh = wb_calc_thresh(wb, dirty_thresh); -#define K(x) ((x) << (PAGE_SHIFT - 10)) seq_printf(m, "BdiWriteback: %10lu kB\n" "BdiReclaimable: %10lu kB\n" @@ -98,7 +99,6 @@ static int bdi_debug_stats_show(struct s nr_more_io, nr_dirty_time, !list_empty(&bdi->bdi_list), bdi->wb.state); -#undef K return 0; } @@ -146,8 +146,6 @@ static ssize_t read_ahead_kb_store(struc return count; } -#define K(pages) ((pages) << (PAGE_SHIFT - 10)) - #define BDI_SHOW(name, expr) \ static ssize_t name##_show(struct device *dev, \ struct device_attribute *attr, char *buf) \ From patchwork Wed Feb 24 20:02:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102513 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F02FDC433DB for ; Wed, 24 Feb 2021 20:02:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9644964E60 for ; Wed, 24 Feb 2021 20:02:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9644964E60 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 259418D000D; Wed, 24 Feb 2021 15:02:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1FF078D0007; Wed, 24 Feb 2021 15:02:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 13B108D000D; Wed, 24 Feb 2021 15:02:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0155.hostedemail.com [216.40.44.155]) by kanga.kvack.org (Postfix) with ESMTP id E833D8D0007 for ; Wed, 24 Feb 2021 15:02:57 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id A78D0F045 for ; Wed, 24 Feb 2021 20:02:57 +0000 (UTC) X-FDA: 77854234794.18.6B70FC3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP id 0897CC0007DA for ; Wed, 24 Feb 2021 20:02:49 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 16D7564E09; Wed, 24 Feb 2021 20:02:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196976; bh=T85Xr9DPB97rqTmjlllDlIAKZVTOHYa0asJFqBT5srk=; h=Date:From:To:Subject:In-Reply-To:From; b=q/AFx7eU6VRssTE7HU2EtPoLJq8H9Pf4MSSLyVqwUsPAIDkYyaKK9RNvGuyJuy+Li 80A4Pf7CGtbKaMh1A9NHBfTA0M7p+dCHLJZn/hlrvkHkvBrlLSwCpqjtAl3EaRtGII Vn7/POCjuooG51grwbwzN8jNOEjNsLevplIkDnjs= Date: Wed, 24 Feb 2021 12:02:55 -0800 From: Andrew Morton To: abaci-bugfix@linux.alibaba.com, abaci@linux.alibaba.com, akpm@linux-foundation.org, david@redhat.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 050/173] mm/swap_slots.c: remove redundant NULL check Message-ID: <20210224200255.cTufYVXJh%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: z9crfh6gk4jttubjewga8xupkgkfxsmc X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 0897CC0007DA Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf14; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196969-596077 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Li Subject: mm/swap_slots.c: remove redundant NULL check Fix below warnings reported by coccicheck: ./mm/swap_slots.c:197:3-9: WARNING: NULL check before some freeing functions is not needed. Link: https://lkml.kernel.org/r/1611214229-11225-1-git-send-email-abaci-bugfix@linux.alibaba.com Signed-off-by: Yang Li Reported-by: Abaci Robot Reviewed-by: Andrew Morton Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- mm/swap_slots.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) --- a/mm/swap_slots.c~mm-swap_slotsc-remove-redundant-null-check +++ a/mm/swap_slots.c @@ -193,8 +193,7 @@ static void drain_slots_cache_cpu(unsign cache->slots_ret = NULL; } spin_unlock_irq(&cache->free_lock); - if (slots) - kvfree(slots); + kvfree(slots); } } From patchwork Wed Feb 24 20:02:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102515 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9823C433E6 for ; Wed, 24 Feb 2021 20:03:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A5E9764E60 for ; Wed, 24 Feb 2021 20:03:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A5E9764E60 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 428028D000E; Wed, 24 Feb 2021 15:03:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3AFC18D0007; Wed, 24 Feb 2021 15:03:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2E9958D000E; Wed, 24 Feb 2021 15:03:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0186.hostedemail.com [216.40.44.186]) by kanga.kvack.org (Postfix) with ESMTP id 0EEA38D0007 for ; Wed, 24 Feb 2021 15:03:01 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id D035EF048 for ; Wed, 24 Feb 2021 20:03:00 +0000 (UTC) X-FDA: 77854234920.09.D0297F8 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id A5BAFFE for ; Wed, 24 Feb 2021 20:02:54 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 3135864E7A; Wed, 24 Feb 2021 20:02:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196979; bh=CVIEiTchpZw0XsVoJdhriHeLgGDcugYxey5h859uyyc=; h=Date:From:To:Subject:In-Reply-To:From; b=K7n42VJVjAyXutN8ZX5cs4h64oOxmEJVvlE3siRF3FtLJo2C+YXVBvcZ1DWUBw8CO AIzURrB1ximB2O4Rwjg00PsXngAjghrfBZ1dlQCdnF5zlgweqZkeDOBqgZAcSsUEuE viAXbbOeberfKGT8BjTxKnIrNa90NlhnZN5ahepE= Date: Wed, 24 Feb 2021 12:02:58 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, stephenzhangzsd@gmail.com, torvalds@linux-foundation.org Subject: [patch 051/173] mm/swapfile.c: fix debugging information problem Message-ID: <20210224200258.WYqSGoEu3%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: A5BAFFE X-Stat-Signature: d1gosmtezsbgoj4zc8wamdsb755uqx4e Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf12; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196974-698158 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Stephen Zhang Subject: mm/swapfile.c: fix debugging information problem Once the function name is changed, it may be easy to forget to modify the corresponding code here. Link: https://lkml.kernel.org/r/1611369120-2276-1-git-send-email-stephenzhangzsd@gmail.com Signed-off-by: Stephen Zhang Signed-off-by: Andrew Morton --- mm/swapfile.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) --- a/mm/swapfile.c~mm-swapfilec-fix-debugging-information-problem +++ a/mm/swapfile.c @@ -1157,13 +1157,13 @@ static struct swap_info_struct *__swap_i return p; bad_offset: - pr_err("swap_info_get: %s%08lx\n", Bad_offset, entry.val); + pr_err("%s: %s%08lx\n", __func__, Bad_offset, entry.val); goto out; bad_device: - pr_err("swap_info_get: %s%08lx\n", Unused_file, entry.val); + pr_err("%s: %s%08lx\n", __func__, Unused_file, entry.val); goto out; bad_nofile: - pr_err("swap_info_get: %s%08lx\n", Bad_file, entry.val); + pr_err("%s: %s%08lx\n", __func__, Bad_file, entry.val); out: return NULL; } @@ -1180,7 +1180,7 @@ static struct swap_info_struct *_swap_in return p; bad_free: - pr_err("swap_info_get: %s%08lx\n", Unused_offset, entry.val); + pr_err("%s: %s%08lx\n", __func__, Unused_offset, entry.val); out: return NULL; } From patchwork Wed Feb 24 20:03:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102517 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CF4DC433E0 for ; Wed, 24 Feb 2021 20:03:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D828164F23 for ; Wed, 24 Feb 2021 20:03:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D828164F23 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 61DD98D000F; Wed, 24 Feb 2021 15:03:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D0458D0007; Wed, 24 Feb 2021 15:03:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4E8D48D000F; Wed, 24 Feb 2021 15:03:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0181.hostedemail.com [216.40.44.181]) by kanga.kvack.org (Postfix) with ESMTP id 319818D0007 for ; Wed, 24 Feb 2021 15:03:04 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id F00DDF049 for ; Wed, 24 Feb 2021 20:03:03 +0000 (UTC) X-FDA: 77854235046.28.A6ECAA2 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf06.hostedemail.com (Postfix) with ESMTP id 0B963C0007CB for ; Wed, 24 Feb 2021 20:03:04 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6D42B64E60; Wed, 24 Feb 2021 20:03:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196982; bh=EMCDtPTrNWcTg4yKqS51VPz73gYHPSpsGa5jQGbYHF4=; h=Date:From:To:Subject:In-Reply-To:From; b=fVOSBhGXC2/zbOjc9Nxl+thAVd02hDPDlzG2HgtbNL3gWH+FEsakmlhD5UMeCEfEx 1UAWSiwKLKSWu3UbklKqrn2hJzYeUCsYm46udHXHJyTvm8ZXXtsbTQLSfBL/TJ5s2e REslQvhJmaspN10o8LNI0dBPRfuHKsX8kN5Oq+3E= Date: Wed, 24 Feb 2021 12:03:01 -0800 From: Andrew Morton To: akpm@linux-foundation.org, georgi.djakov@linaro.org, linmiaohe@huawei.com, linux-mm@kvack.org, minchan@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 052/173] mm/page_io: use pr_alert_ratelimited for swap read/write errors Message-ID: <20210224200301._0q-FlWXk%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 0B963C0007CB X-Stat-Signature: q1yrh6cfsxuef1h45f57wrdyjg117y13 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf06; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196984-752151 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Georgi Djakov Subject: mm/page_io: use pr_alert_ratelimited for swap read/write errors If there are errors during swap read or write, they can easily fill the log buffer and remove any previous messages that might be useful for debugging, especially on systems that rely for logging only on the kernel ring-buffer. For example, on a systems using zram as swap, we are more likely to see any page allocation errors preceding the swap write errors if the alerts are ratelimited. Link: https://lkml.kernel.org/r/20210201142055.29068-1-georgi.djakov@linaro.org Signed-off-by: Georgi Djakov Acked-by: Minchan Kim Reviewed-by: Miaohe Lin Signed-off-by: Andrew Morton --- mm/page_io.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) --- a/mm/page_io.c~mm-page_io-use-pr_alert_ratelimited-for-swap-read-write-errors +++ a/mm/page_io.c @@ -41,9 +41,9 @@ void end_swap_bio_write(struct bio *bio) * Also clear PG_reclaim to avoid rotate_reclaimable_page() */ set_page_dirty(page); - pr_alert("Write-error on swap-device (%u:%u:%llu)\n", - MAJOR(bio_dev(bio)), MINOR(bio_dev(bio)), - (unsigned long long)bio->bi_iter.bi_sector); + pr_alert_ratelimited("Write-error on swap-device (%u:%u:%llu)\n", + MAJOR(bio_dev(bio)), MINOR(bio_dev(bio)), + (unsigned long long)bio->bi_iter.bi_sector); ClearPageReclaim(page); } end_page_writeback(page); @@ -106,9 +106,9 @@ static void end_swap_bio_read(struct bio if (bio->bi_status) { SetPageError(page); ClearPageUptodate(page); - pr_alert("Read-error on swap-device (%u:%u:%llu)\n", - MAJOR(bio_dev(bio)), MINOR(bio_dev(bio)), - (unsigned long long)bio->bi_iter.bi_sector); + pr_alert_ratelimited("Read-error on swap-device (%u:%u:%llu)\n", + MAJOR(bio_dev(bio)), MINOR(bio_dev(bio)), + (unsigned long long)bio->bi_iter.bi_sector); goto out; } From patchwork Wed Feb 24 20:03:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102519 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E970C433E6 for ; Wed, 24 Feb 2021 20:03:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EFB2964F19 for ; Wed, 24 Feb 2021 20:03:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EFB2964F19 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 88DFA8D0010; Wed, 24 Feb 2021 15:03:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 818A88D0007; Wed, 24 Feb 2021 15:03:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 731278D0010; Wed, 24 Feb 2021 15:03:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0135.hostedemail.com [216.40.44.135]) by kanga.kvack.org (Postfix) with ESMTP id 5B2E38D0007 for ; Wed, 24 Feb 2021 15:03:07 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 2A41318020C25 for ; Wed, 24 Feb 2021 20:03:07 +0000 (UTC) X-FDA: 77854235214.23.D30EDB9 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP id 87E90C0007EB for ; Wed, 24 Feb 2021 20:02:59 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 95C8A64F0E; Wed, 24 Feb 2021 20:03:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196985; bh=5ensjn87mwrhvVw9AWrG1AEYC9m7pNc5DwXvSR53iok=; h=Date:From:To:Subject:In-Reply-To:From; b=g7+uOkmIc4/PHUOjStnTjfbRllasW4w48W8KA7sKszdq5UmIrmwY3KqGcqw/TKxne KcrafOSDyQmVLaMuF5yavSTZ4FBP2MHtmM/bSkZfFhIYdyaaubGwN33DmCap7uxw4H Dm76YcHDkzMPV5miwUoVxPaM4IrMhnEl30cMIzA4= Date: Wed, 24 Feb 2021 12:03:05 -0800 From: Andrew Morton To: akpm@linux-foundation.org, enbyamy@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rikard.falkeborn@gmail.com, torvalds@linux-foundation.org, ying.huang@intel.com Subject: [patch 053/173] mm/swap_state: constify static struct attribute_group Message-ID: <20210224200305.LTTJ0SaIm%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: ifujzhmuq9dboetj39frs4r8hxewjuhm X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 87E90C0007EB Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf14; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196979-907019 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Rikard Falkeborn Subject: mm/swap_state: constify static struct attribute_group The only usage of swap_attr_group is to pass its address to sysfs_create_group() which takes a pointer to const attribute_group. Make it const to allow the compiler to put it in read-only memory. Link: https://lkml.kernel.org/r/20210201233254.91809-1-rikard.falkeborn@gmail.com Signed-off-by: Rikard Falkeborn Reviewed-by: Amy Parker Acked-by: "Huang, Ying" Reviewed-by: Miaohe Lin Signed-off-by: Andrew Morton --- mm/swap_state.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/swap_state.c~mm-swap_state-constify-static-struct-attribute_group +++ a/mm/swap_state.c @@ -927,7 +927,7 @@ static struct attribute *swap_attrs[] = NULL, }; -static struct attribute_group swap_attr_group = { +static const struct attribute_group swap_attr_group = { .attrs = swap_attrs, }; From patchwork Wed Feb 24 20:03:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102521 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E64D8C433E0 for ; Wed, 24 Feb 2021 20:03:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8449864F1D for ; Wed, 24 Feb 2021 20:03:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8449864F1D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 116AB8D0011; Wed, 24 Feb 2021 15:03:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 077A38D0007; Wed, 24 Feb 2021 15:03:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF5E08D0011; Wed, 24 Feb 2021 15:03:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0026.hostedemail.com [216.40.44.26]) by kanga.kvack.org (Postfix) with ESMTP id D91208D0007 for ; Wed, 24 Feb 2021 15:03:10 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 9C3A88017CCB for ; Wed, 24 Feb 2021 20:03:10 +0000 (UTC) X-FDA: 77854235340.05.B66A857 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP id 1348BE000126 for ; Wed, 24 Feb 2021 20:03:09 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id DF54964F0D; Wed, 24 Feb 2021 20:03:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196989; bh=gRejEKoFluiBxWJ6U1UkCg86trl6uLiF3N4Bv/ikOiY=; h=Date:From:To:Subject:In-Reply-To:From; b=0c+NRm+GaXwfSjlN2hmXJJrElBvtZWLoF9Bm8Ee461wybC6QV+vaKIgFtABNLJGrZ zC67pA5D0zXH76j+QJ4z7+f23H7sj5Pjl0g8Aeda/oRFAHqSy7Jb1QPAROvvXHG/HH fE3nWPhNHlzqspJalSAnFXLHTdYGtyg0na31bsJI= Date: Wed, 24 Feb 2021 12:03:08 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hannes@cmpxchg.org, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, yuzhao@google.com Subject: [patch 054/173] mm/swap: don't SetPageWorkingset unconditionally during swapin Message-ID: <20210224200308.c29_ITugV%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 3kzq9hsquey8tgigs59y7ya8xrjoapbh X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 1348BE000126 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf30; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196989-911695 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu Zhao Subject: mm/swap: don't SetPageWorkingset unconditionally during swapin We are capable of SetPageWorkingset based on refault distances after commit aae466b0052e ("mm/swap: implement workingset detection for anonymous LRU"). This is done by workingset_refault(), which is right above the unconditional SetPageWorkingset deleted by this patch. The unconditional SetPageWorkingset miscategorizes pages that are read ahead or never belonged to the working set (e.g., tmpfs pages accessed only once by fd). When those pages are swapped in (after they were swapped out) for the first time, they skew PSI (when using async swap). When this happens again, depending on their refault distances, they might skew workingset_restore_anon counter in addition to PSI because their shadows indicate they were part of the working set. Historically, SetPageWorkingset was added as part of the PSI series, and Johannes said: "It was meant to mark incoming pages under IO with SetPageWorkingset when waiting for them constituted a memory stall. On the page cache side, because we HAVE workingset detection, this was specific to recently evicted pages that had been active in their previous life. On the anon side, the aging algorithm had no distinction between workingset and sporadically used pages. Given the choice between a) no swapin stalls are pressure and b) all swapin stalls are pressure, I went with the latter in order to detect swap storms. The false positive case - high rate of swapin without severe memory pressure - was relatively unlikely, because we tried to avoid swapping until everything was completely on fire in the first place." Link: https://lkml.kernel.org/r/20201209012400.1771150-1-yuzhao@google.com Link: https://lkml.kernel.org/r/20201214231253.62313-1-yuzhao@google.com Signed-off-by: Yu Zhao Acked-by: Vlastimil Babka Acked-by: Johannes Weiner Acked-by: Joonsoo Kim Acked-by: Michal Hocko Signed-off-by: Andrew Morton --- mm/swap_state.c | 1 - 1 file changed, 1 deletion(-) --- a/mm/swap_state.c~mm-swap-dont-setpageworkingset-unconditionally-during-swapin +++ a/mm/swap_state.c @@ -537,7 +537,6 @@ struct page *__read_swap_cache_async(swp workingset_refault(page, shadow); /* Caller will initiate read into locked page */ - SetPageWorkingset(page); lru_cache_add(page); *new_page_allocated = true; return page; From patchwork Wed Feb 24 20:03:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102523 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65F93C433DB for ; Wed, 24 Feb 2021 20:03:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 060D864F24 for ; Wed, 24 Feb 2021 20:03:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 060D864F24 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8B6588D0012; Wed, 24 Feb 2021 15:03:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 86B9D8D0007; Wed, 24 Feb 2021 15:03:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 757658D0012; Wed, 24 Feb 2021 15:03:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0151.hostedemail.com [216.40.44.151]) by kanga.kvack.org (Postfix) with ESMTP id 5B7078D0007 for ; Wed, 24 Feb 2021 15:03:14 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 280A21801E6BA for ; Wed, 24 Feb 2021 20:03:14 +0000 (UTC) X-FDA: 77854235508.13.0726A06 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP id 92FE060024A0 for ; Wed, 24 Feb 2021 20:03:09 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 2F61F64F19; Wed, 24 Feb 2021 20:03:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196992; bh=wnv2gPunmDBgfhIz6H6tZTejPLxGw4Ic4evpBnqFYQ4=; h=Date:From:To:Subject:In-Reply-To:From; b=lECEGzugYoFjLz5CPiSxnlK8SHx4bX46GTaX+NuCW5TqpejmzF6ZoG/ApqGPunilF 2H1UtIz9Ue3glyOTOZA7lsmKpoIck0HT2KWROOrCifuw3CHvex0x2GtEik74VXHfWl shmPHZV13WbFz9NSeplbvaZv6ZTjWrTLzDd8Z2Pw= Date: Wed, 24 Feb 2021 12:03:11 -0800 From: Andrew Morton To: akpm@linux-foundation.org, cl@linux.com, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 055/173] mm: memcg/slab: pre-allocate obj_cgroups for slab caches with SLAB_ACCOUNT Message-ID: <20210224200311.FvZgQBV11%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: eqothuts7dge9hskrme8hd8ie95zmsye X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 92FE060024A0 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf09; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196989-24045 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Roman Gushchin Subject: mm: memcg/slab: pre-allocate obj_cgroups for slab caches with SLAB_ACCOUNT In general it's unknown in advance if a slab page will contain accounted objects or not. In order to avoid memory waste, an obj_cgroup vector is allocated dynamically when a need to account of a new object arises. Such approach is memory efficient, but requires an expensive cmpxchg() to set up the memcg/objcgs pointer, because an allocation can race with a different allocation on another cpu. But in some common cases it's known for sure that a slab page will contain accounted objects: if the page belongs to a slab cache with a SLAB_ACCOUNT flag set. It includes such popular objects like vm_area_struct, anon_vma, task_struct, etc. In such cases we can pre-allocate the objcgs vector and simple assign it to the page without any atomic operations, because at this early stage the page is not visible to anyone else. A very simplistic benchmark (allocating 10000000 64-bytes objects in a row) shows ~15% win. In the real life it seems that most workloads are not very sensitive to the speed of (accounted) slab allocations. [guro@fb.com: open-code set_page_objcgs() and add some comments, by Johannes] Link: https://lkml.kernel.org/r/20201113001926.GA2934489@carbon.dhcp.thefacebook.com [akpm@linux-foundation.org: fix it for mm-slub-call-account_slab_page-after-slab-page-initialization-fix.patch] Link: https://lkml.kernel.org/r/20201110195753.530157-2-guro@fb.com Signed-off-by: Roman Gushchin Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Cc: Michal Hocko Cc: Christoph Lameter Signed-off-by: Andrew Morton --- include/linux/memcontrol.h | 19 ------------------- mm/memcontrol.c | 23 +++++++++++++++++++---- mm/slab.c | 2 +- mm/slab.h | 14 ++++++++++---- mm/slub.c | 2 +- 5 files changed, 31 insertions(+), 29 deletions(-) --- a/include/linux/memcontrol.h~mm-memcg-slab-pre-allocate-obj_cgroups-for-slab-caches-with-slab_account +++ a/include/linux/memcontrol.h @@ -475,19 +475,6 @@ static inline struct obj_cgroup **page_o return (struct obj_cgroup **)(memcg_data & ~MEMCG_DATA_FLAGS_MASK); } -/* - * set_page_objcgs - associate a page with a object cgroups vector - * @page: a pointer to the page struct - * @objcgs: a pointer to the object cgroups vector - * - * Atomically associates a page with a vector of object cgroups. - */ -static inline bool set_page_objcgs(struct page *page, - struct obj_cgroup **objcgs) -{ - return !cmpxchg(&page->memcg_data, 0, (unsigned long)objcgs | - MEMCG_DATA_OBJCGS); -} #else static inline struct obj_cgroup **page_objcgs(struct page *page) { @@ -498,12 +485,6 @@ static inline struct obj_cgroup **page_o { return NULL; } - -static inline bool set_page_objcgs(struct page *page, - struct obj_cgroup **objcgs) -{ - return true; -} #endif static __always_inline bool memcg_stat_item_in_bytes(int idx) --- a/mm/memcontrol.c~mm-memcg-slab-pre-allocate-obj_cgroups-for-slab-caches-with-slab_account +++ a/mm/memcontrol.c @@ -2935,9 +2935,10 @@ static void commit_charge(struct page *p #ifdef CONFIG_MEMCG_KMEM int memcg_alloc_page_obj_cgroups(struct page *page, struct kmem_cache *s, - gfp_t gfp) + gfp_t gfp, bool new_page) { unsigned int objects = objs_per_slab_page(s, page); + unsigned long memcg_data; void *vec; vec = kcalloc_node(objects, sizeof(struct obj_cgroup *), gfp, @@ -2945,11 +2946,25 @@ int memcg_alloc_page_obj_cgroups(struct if (!vec) return -ENOMEM; - if (!set_page_objcgs(page, vec)) + memcg_data = (unsigned long) vec | MEMCG_DATA_OBJCGS; + if (new_page) { + /* + * If the slab page is brand new and nobody can yet access + * it's memcg_data, no synchronization is required and + * memcg_data can be simply assigned. + */ + page->memcg_data = memcg_data; + } else if (cmpxchg(&page->memcg_data, 0, memcg_data)) { + /* + * If the slab page is already in use, somebody can allocate + * and assign obj_cgroups in parallel. In this case the existing + * objcg vector should be reused. + */ kfree(vec); - else - kmemleak_not_leak(vec); + return 0; + } + kmemleak_not_leak(vec); return 0; } --- a/mm/slab.c~mm-memcg-slab-pre-allocate-obj_cgroups-for-slab-caches-with-slab_account +++ a/mm/slab.c @@ -1379,7 +1379,7 @@ static struct page *kmem_getpages(struct return NULL; } - account_slab_page(page, cachep->gfporder, cachep); + account_slab_page(page, cachep->gfporder, cachep, flags); __SetPageSlab(page); /* Record if ALLOC_NO_WATERMARKS was set when allocating the slab */ if (sk_memalloc_socks() && page_is_pfmemalloc(page)) --- a/mm/slab.h~mm-memcg-slab-pre-allocate-obj_cgroups-for-slab-caches-with-slab_account +++ a/mm/slab.h @@ -238,7 +238,7 @@ static inline bool kmem_cache_debug_flag #ifdef CONFIG_MEMCG_KMEM int memcg_alloc_page_obj_cgroups(struct page *page, struct kmem_cache *s, - gfp_t gfp); + gfp_t gfp, bool new_page); static inline void memcg_free_page_obj_cgroups(struct page *page) { @@ -315,7 +315,8 @@ static inline void memcg_slab_post_alloc page = virt_to_head_page(p[i]); if (!page_objcgs(page) && - memcg_alloc_page_obj_cgroups(page, s, flags)) { + memcg_alloc_page_obj_cgroups(page, s, flags, + false)) { obj_cgroup_uncharge(objcg, obj_full_size(s)); continue; } @@ -379,7 +380,8 @@ static inline struct mem_cgroup *memcg_f } static inline int memcg_alloc_page_obj_cgroups(struct page *page, - struct kmem_cache *s, gfp_t gfp) + struct kmem_cache *s, gfp_t gfp, + bool new_page) { return 0; } @@ -420,8 +422,12 @@ static inline struct kmem_cache *virt_to } static __always_inline void account_slab_page(struct page *page, int order, - struct kmem_cache *s) + struct kmem_cache *s, + gfp_t gfp) { + if (memcg_kmem_enabled() && (s->flags & SLAB_ACCOUNT)) + memcg_alloc_page_obj_cgroups(page, s, gfp, true); + mod_node_page_state(page_pgdat(page), cache_vmstat_idx(s), PAGE_SIZE << order); } --- a/mm/slub.c~mm-memcg-slab-pre-allocate-obj_cgroups-for-slab-caches-with-slab_account +++ a/mm/slub.c @@ -1785,7 +1785,7 @@ static struct page *allocate_slab(struct page->objects = oo_objects(oo); - account_slab_page(page, oo_order(oo), s); + account_slab_page(page, oo_order(oo), s, flags); page->slab_cache = s; __SetPageSlab(page); From patchwork Wed Feb 24 20:03:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102525 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B6E2C433E6 for ; Wed, 24 Feb 2021 20:03:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CB8F364F1D for ; Wed, 24 Feb 2021 20:03:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CB8F364F1D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5D10F8D0013; Wed, 24 Feb 2021 15:03:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5801F8D0007; Wed, 24 Feb 2021 15:03:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 46F4F8D0013; Wed, 24 Feb 2021 15:03:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0197.hostedemail.com [216.40.44.197]) by kanga.kvack.org (Postfix) with ESMTP id 300E38D0007 for ; Wed, 24 Feb 2021 15:03:18 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id E60E618022010 for ; Wed, 24 Feb 2021 20:03:17 +0000 (UTC) X-FDA: 77854235634.05.6DBAC2C Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf21.hostedemail.com (Postfix) with ESMTP id F179AE000116 for ; Wed, 24 Feb 2021 20:03:13 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id AF6AF64F23; Wed, 24 Feb 2021 20:03:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614196996; bh=3x4m3lmkAA65F08g8yrHU6ZnNa8VHA5bEtx+8Pta33E=; h=Date:From:To:Subject:In-Reply-To:From; b=nzZi2ydsGEFIkeOxO9z1qvD7VU13vqzkuY3GZCJ7WK6Rp+3wbcOZ+NEyAHbGwXMuu lpaUrcz8zkwSy/tuiOzQyGRlQwiqbELw3QSCI9eh97/M2JX0St7OG22fJiY1WUwbt9 DUA1cpv8TGZhrwo0SKvhKLO8Lc1C5N2RXRjTgUe8= Date: Wed, 24 Feb 2021 12:03:15 -0800 From: Andrew Morton To: akpm@linux-foundation.org, chris@chrisdown.name, guro@fb.com, hannes@cmpxchg.org, laoar.shao@gmail.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, richard.weiyang@gmail.com, sfr@canb.auug.org.au, shakeelb@google.com, songmuchun@bytedance.com, torvalds@linux-foundation.org, vdavydov.dev@gmail.com Subject: [patch 056/173] mm: memcontrol: optimize per-lruvec stats counter memory usage Message-ID: <20210224200315.yvHU1H9Am%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: F179AE000116 X-Stat-Signature: 7ssoh3pt3n6qk6abm3muk73gzuxhq96h Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf21; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196993-572564 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: memcontrol: optimize per-lruvec stats counter memory usage The vmstat threshold is 32 (MEMCG_CHARGE_BATCH), Actually the threshold can be as big as MEMCG_CHARGE_BATCH * PAGE_SIZE. It still fits into s32. So introduce struct batched_lruvec_stat to optimize memory usage. The size of struct lruvec_stat is 304 bytes on 64 bit systems. As it is a per-cpu structure. So with this patch, we can save 304 / 2 * ncpu bytes per-memcg per-node where ncpu is the number of the possible CPU. If there are c memory cgroup (include dying cgroup) and n NUMA node in the system. Finally, we can save (152 * ncpu * c * n) bytes. [akpm@linux-foundation.org: fix typo in comment] Link: https://lkml.kernel.org/r/20201210042121.39665-1-songmuchun@bytedance.com Signed-off-by: Muchun Song Reviewed-by: Shakeel Butt Cc: Johannes Weiner Cc: Michal Hocko Cc: Vladimir Davydov Cc: Shakeel Butt Cc: Roman Gushchin Cc: Stephen Rothwell Cc: Chris Down Cc: Yafang Shao Cc: Wei Yang Signed-off-by: Andrew Morton --- include/linux/memcontrol.h | 14 ++++++++++++-- mm/memcontrol.c | 10 +++++++++- 2 files changed, 21 insertions(+), 3 deletions(-) --- a/include/linux/memcontrol.h~mm-memcontrol-optimize-per-lruvec-stats-counter-memory-usage +++ a/include/linux/memcontrol.h @@ -92,6 +92,10 @@ struct lruvec_stat { long count[NR_VM_NODE_STAT_ITEMS]; }; +struct batched_lruvec_stat { + s32 count[NR_VM_NODE_STAT_ITEMS]; +}; + /* * Bitmap of shrinker::id corresponding to memcg-aware shrinkers, * which have elements charged to this memcg. @@ -107,11 +111,17 @@ struct memcg_shrinker_map { struct mem_cgroup_per_node { struct lruvec lruvec; - /* Legacy local VM stats */ + /* + * Legacy local VM stats. This should be struct lruvec_stat and + * cannot be optimized to struct batched_lruvec_stat. Because + * the threshold of the lruvec_stat_cpu can be as big as + * MEMCG_CHARGE_BATCH * PAGE_SIZE. It can fit into s32. But this + * filed has no upper limit. + */ struct lruvec_stat __percpu *lruvec_stat_local; /* Subtree VM stats (batched updates) */ - struct lruvec_stat __percpu *lruvec_stat_cpu; + struct batched_lruvec_stat __percpu *lruvec_stat_cpu; atomic_long_t lruvec_stat[NR_VM_NODE_STAT_ITEMS]; unsigned long lru_zone_size[MAX_NR_ZONES][NR_LRU_LISTS]; --- a/mm/memcontrol.c~mm-memcontrol-optimize-per-lruvec-stats-counter-memory-usage +++ a/mm/memcontrol.c @@ -5208,7 +5208,7 @@ static int alloc_mem_cgroup_per_node_inf return 1; } - pn->lruvec_stat_cpu = alloc_percpu_gfp(struct lruvec_stat, + pn->lruvec_stat_cpu = alloc_percpu_gfp(struct batched_lruvec_stat, GFP_KERNEL_ACCOUNT); if (!pn->lruvec_stat_cpu) { free_percpu(pn->lruvec_stat_local); @@ -7093,6 +7093,14 @@ static int __init mem_cgroup_init(void) { int cpu, node; + /* + * Currently s32 type (can refer to struct batched_lruvec_stat) is + * used for per-memcg-per-cpu caching of per-node statistics. In order + * to work fine, we should make sure that the overfill threshold can't + * exceed S32_MAX / PAGE_SIZE. + */ + BUILD_BUG_ON(MEMCG_CHARGE_BATCH > S32_MAX / PAGE_SIZE); + cpuhp_setup_state_nocalls(CPUHP_MM_MEMCQ_DEAD, "mm/memctrl:dead", NULL, memcg_hotplug_cpu_dead); From patchwork Wed Feb 24 20:03:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102527 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 242E1C4332B for ; Wed, 24 Feb 2021 20:03:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BB30764F1D for ; Wed, 24 Feb 2021 20:03:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BB30764F1D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5D6C88D0002; Wed, 24 Feb 2021 15:03:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5581D6B0074; Wed, 24 Feb 2021 15:03:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4965F8D0002; Wed, 24 Feb 2021 15:03:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0103.hostedemail.com [216.40.44.103]) by kanga.kvack.org (Postfix) with ESMTP id 31A8B6B0073 for ; Wed, 24 Feb 2021 15:03:22 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id EF9311801AB52 for ; Wed, 24 Feb 2021 20:03:21 +0000 (UTC) X-FDA: 77854235802.28.229BE68 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf08.hostedemail.com (Postfix) with ESMTP id E49EB80192E9 for ; Wed, 24 Feb 2021 20:03:09 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 9557B64F25; Wed, 24 Feb 2021 20:03:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197000; bh=QbJ3YkkYDTvv4Gjc+7xdeNRdcykXss4NT5t1rXlDarc=; h=Date:From:To:Subject:In-Reply-To:From; b=pLq4hV5A7vEkHiF/euckUdd5jVV+pmj2A7zuyLwni4UTqq2URMxcKdCfN8IZZRrgr WncEjrkm3je5pbGOqfAavQ46HAcfoOnaszgd/XYA1XdyUB40QIqT6IPyHNCgptSNrr ks+aPuRtqwXb9FeGJ4fJbxYoEpAzfB278EkIQCmo= Date: Wed, 24 Feb 2021 12:03:19 -0800 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, feng.tang@intel.com, gregkh@linuxfoundation.org, guro@fb.com, hannes@cmpxchg.org, hughd@google.com, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, neilb@suse.de, pankaj.gupta@cloud.ionos.com, rafael@kernel.org, rdunlap@infradead.org, samitolvanen@google.com, shakeelb@google.com, songmuchun@bytedance.com, torvalds@linux-foundation.org, vdavydov.dev@gmail.com Subject: [patch 057/173] mm: memcontrol: fix NR_ANON_THPS accounting in charge moving Message-ID: <20210224200319.BJzkKisOt%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 8cu6n6opnuesmih8pf4c9sszt9ut8c5o X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: E49EB80192E9 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf08; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614196989-421827 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: memcontrol: fix NR_ANON_THPS accounting in charge moving Patch series "Convert all THP vmstat counters to pages", v6. This patch series is aimed to convert all THP vmstat counters to pages. The unit of some vmstat counters are pages, some are bytes, some are HPAGE_PMD_NR, and some are KiB. When we want to expose these vmstat counters to the userspace, we have to know the unit of the vmstat counters is which one. When the unit is bytes or kB, both clearly distinguishable by the B/KB suffix. But for the THP vmstat counters, we may make mistakes. For example, the below is some bug fix for the THP vmstat counters: - 7de2e9f195b9 ("mm: memcontrol: correct the NR_ANON_THPS counter of hierarchical memcg") - The first commit in this series ("fix NR_ANON_THPS accounting in charge moving") This patch series can make the code clear. And make all the unit of the THP vmstat counters in pages. Finally, the unit of the vmstat counters are pages, kB and bytes. The B/KB suffix can tell us that the unit is bytes or kB. The rest which is without suffix are pages. In this series, I changed the following vmstat counters unit from HPAGE_PMD_NR to pages. However, there is no change to the print format of output to user space. - NR_ANON_THPS - NR_FILE_THPS - NR_SHMEM_THPS - NR_SHMEM_PMDMAPPED - NR_FILE_PMDMAPPED Doing this also can make the statistics more accuracy for the THP vmstat counters. This series is consistent with 8f182270dfec ("mm/swap.c: flush lru pvecs on compound page arrival"). Because we use struct per_cpu_nodestat to cache the vmstat counters, which leads to inaccurate statistics especially THP vmstat counters. In the systems with hundreds of processors it can be GBs of memory. For example, for a 96 CPUs system, the threshold is the maximum number of 125. And the per cpu counters can cache 23.4375 GB in total. The THP page is already a form of batched addition (it will add 512 worth of memory in one go) so skipping the batching seems like sensible. Although every THP stats update overflows the per-cpu counter, resorting to atomic global updates. But it can make the statistics more accuracy for the THP vmstat counters. From this point of view, I think that do this converting is reasonable. Thanks Hugh for mentioning this. This was inspired by Johannes and Roman. Thanks to them. This patch (of 7): The unit of NR_ANON_THPS is HPAGE_PMD_NR already. So it should inc/dec by one rather than nr_pages. Link: https://lkml.kernel.org/r/20201228164110.2838-1-songmuchun@bytedance.com Link: https://lkml.kernel.org/r/20201228164110.2838-2-songmuchun@bytedance.com Fixes: 468c398233da ("mm: memcontrol: switch to native NR_ANON_THPS counter") Signed-off-by: Muchun Song Acked-by: Michal Hocko Acked-by: Johannes Weiner Acked-by: Pankaj Gupta Reviewed-by: Roman Gushchin Reviewed-by: Shakeel Butt Cc: Alexey Dobriyan Cc: Feng Tang Cc: Greg Kroah-Hartman Cc: Hugh Dickins Cc: Joonsoo Kim Cc: NeilBrown Cc: Rafael. J. Wysocki Cc: Randy Dunlap Cc: Sami Tolvanen Cc: Vladimir Davydov Signed-off-by: Andrew Morton --- mm/memcontrol.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) --- a/mm/memcontrol.c~mm-memcontrol-fix-nr_anon_thps-accounting-in-charge-moving +++ a/mm/memcontrol.c @@ -5652,10 +5652,8 @@ static int mem_cgroup_move_account(struc __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages); __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages); if (PageTransHuge(page)) { - __mod_lruvec_state(from_vec, NR_ANON_THPS, - -nr_pages); - __mod_lruvec_state(to_vec, NR_ANON_THPS, - nr_pages); + __dec_lruvec_state(from_vec, NR_ANON_THPS); + __inc_lruvec_state(to_vec, NR_ANON_THPS); } } From patchwork Wed Feb 24 20:03:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102545 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31CF3C433E6 for ; Wed, 24 Feb 2021 20:04:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BF9F264F09 for ; Wed, 24 Feb 2021 20:04:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BF9F264F09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4BDB66B008A; Wed, 24 Feb 2021 15:04:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 471CF8D0003; Wed, 24 Feb 2021 15:04:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 311058D0001; Wed, 24 Feb 2021 15:04:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0130.hostedemail.com [216.40.44.130]) by kanga.kvack.org (Postfix) with ESMTP id 19A1B6B008A for ; Wed, 24 Feb 2021 15:04:00 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id D3B1B1802ED75 for ; Wed, 24 Feb 2021 20:03:59 +0000 (UTC) X-FDA: 77854237398.24.635E793 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id A350C200039B for ; Wed, 24 Feb 2021 20:03:49 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A3FE864EC4; Wed, 24 Feb 2021 20:03:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197004; bh=M1Gn7Dj+t+KJBU+acHHEte0g2HJKcoyAty8oF30LUW4=; h=Date:From:To:Subject:In-Reply-To:From; b=arMops0b5/Q1k9CiAvYb+8F5xr9A/6Y+gku61JQ8o0YB+6UuVdwZ55d6XEZmDJvyz PH2T/9TcBrKMNblnDszs6qrsXIWB/TlZ5cQjRIxI9pqV25tcbZIiTwteHkZKyJF1NR CVKiteHDG4FTXHmKXrmaNcwhooE9YM85YrkTZVSk= Date: Wed, 24 Feb 2021 12:03:23 -0800 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, feng.tang@intel.com, gregkh@linuxfoundation.org, guro@fb.com, hannes@cmpxchg.org, hughd@google.com, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, neilb@suse.de, pankaj.gupta@cloud.ionos.com, rafael@kernel.org, rdunlap@infradead.org, samitolvanen@google.com, shakeelb@google.com, songmuchun@bytedance.com, torvalds@linux-foundation.org, vdavydov.dev@gmail.com Subject: [patch 058/173] mm: memcontrol: convert NR_ANON_THPS account to pages Message-ID: <20210224200323.A_ab8u0-S%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 9c9utngj4rpokkitsqt7syojspfpm1gq X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: A350C200039B Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf11; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197029-78677 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: memcontrol: convert NR_ANON_THPS account to pages Currently we use struct per_cpu_nodestat to cache the vmstat counters, which leads to inaccurate statistics especially THP vmstat counters. In the systems with hundreds of processors it can be GBs of memory. For example, for a 96 CPUs system, the threshold is the maximum number of 125. And the per cpu counters can cache 23.4375 GB in total. The THP page is already a form of batched addition (it will add 512 worth of memory in one go) so skipping the batching seems like sensible. Although every THP stats update overflows the per-cpu counter, resorting to atomic global updates. But it can make the statistics more accuracy for the THP vmstat counters. So we convert the NR_ANON_THPS account to pages. This patch is consistent with 8f182270dfec ("mm/swap.c: flush lru pvecs on compound page arrival"). Doing this also can make the unit of vmstat counters more unified. Finally, the unit of the vmstat counters are pages, kB and bytes. The B/KB suffix can tell us that the unit is bytes or kB. The rest which is without suffix are pages. Link: https://lkml.kernel.org/r/20201228164110.2838-3-songmuchun@bytedance.com Signed-off-by: Muchun Song Cc: Greg Kroah-Hartman Cc: Rafael. J. Wysocki Cc: Alexey Dobriyan Cc: Johannes Weiner Cc: Vladimir Davydov Cc: Hugh Dickins Cc: Shakeel Butt Cc: Roman Gushchin Cc: Sami Tolvanen Cc: Feng Tang Cc: NeilBrown Cc: Joonsoo Kim Cc: Randy Dunlap Cc: Michal Hocko Cc: Pankaj Gupta Signed-off-by: Andrew Morton --- drivers/base/node.c | 15 +++++++++------ fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 13 +++++++++++++ mm/huge_memory.c | 3 ++- mm/memcontrol.c | 20 ++++++-------------- mm/page_alloc.c | 2 +- mm/rmap.c | 6 +++--- mm/vmstat.c | 11 +++++++++-- 8 files changed, 44 insertions(+), 28 deletions(-) --- a/drivers/base/node.c~mm-memcontrol-convert-nr_anon_thps-account-to-pages +++ a/drivers/base/node.c @@ -461,8 +461,7 @@ static ssize_t node_read_meminfo(struct nid, K(sunreclaimable) #ifdef CONFIG_TRANSPARENT_HUGEPAGE , - nid, K(node_page_state(pgdat, NR_ANON_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_ANON_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * @@ -519,10 +518,14 @@ static ssize_t node_read_vmstat(struct d sum_zone_numa_state(nid, i)); #endif - for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) - len += sysfs_emit_at(buf, len, "%s %lu\n", - node_stat_name(i), - node_page_state_pages(pgdat, i)); + for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { + unsigned long pages = node_page_state_pages(pgdat, i); + + if (vmstat_item_print_in_thp(i)) + pages /= HPAGE_PMD_NR; + len += sysfs_emit_at(buf, len, "%s %lu\n", node_stat_name(i), + pages); + } return len; } --- a/fs/proc/meminfo.c~mm-memcontrol-convert-nr_anon_thps-account-to-pages +++ a/fs/proc/meminfo.c @@ -129,7 +129,7 @@ static int meminfo_proc_show(struct seq_ #ifdef CONFIG_TRANSPARENT_HUGEPAGE show_val_kb(m, "AnonHugePages: ", - global_node_page_state(NR_ANON_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_ANON_THPS)); show_val_kb(m, "ShmemHugePages: ", global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); show_val_kb(m, "ShmemPmdMapped: ", --- a/include/linux/mmzone.h~mm-memcontrol-convert-nr_anon_thps-account-to-pages +++ a/include/linux/mmzone.h @@ -210,6 +210,19 @@ enum node_stat_item { }; /* + * Returns true if the item should be printed in THPs (/proc/vmstat + * currently prints number of anon, file and shmem THPs. But the item + * is charged in pages). + */ +static __always_inline bool vmstat_item_print_in_thp(enum node_stat_item item) +{ + if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) + return false; + + return item == NR_ANON_THPS; +} + +/* * Returns true if the value is measured in bytes (most vmstat values are * measured in pages). This defines the API part, the internal representation * might be different. --- a/mm/huge_memory.c~mm-memcontrol-convert-nr_anon_thps-account-to-pages +++ a/mm/huge_memory.c @@ -2176,7 +2176,8 @@ static void __split_huge_pmd_locked(stru lock_page_memcg(page); if (atomic_add_negative(-1, compound_mapcount_ptr(page))) { /* Last compound_mapcount is gone. */ - __dec_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, + -HPAGE_PMD_NR); if (TestClearPageDoubleMap(page)) { /* No need in mapcount reference anymore */ for (i = 0; i < HPAGE_PMD_NR; i++) --- a/mm/memcontrol.c~mm-memcontrol-convert-nr_anon_thps-account-to-pages +++ a/mm/memcontrol.c @@ -1533,7 +1533,7 @@ static struct memory_stat memory_stats[] * on some architectures, the macro of HPAGE_PMD_SIZE is not * constant(e.g. powerpc). */ - { "anon_thp", 0, NR_ANON_THPS }, + { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, { "file_thp", 0, NR_FILE_THPS }, { "shmem_thp", 0, NR_SHMEM_THPS }, #endif @@ -1566,8 +1566,7 @@ static int __init memory_stats_init(void for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { #ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_ANON_THPS || - memory_stats[i].idx == NR_FILE_THPS || + if (memory_stats[i].idx == NR_FILE_THPS || memory_stats[i].idx == NR_SHMEM_THPS) memory_stats[i].ratio = HPAGE_PMD_SIZE; #endif @@ -4087,10 +4086,6 @@ static int memcg_stat_show(struct seq_fi if (memcg1_stats[i] == MEMCG_SWAP && !do_memsw_account()) continue; nr = memcg_page_state_local(memcg, memcg1_stats[i]); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memcg1_stats[i] == NR_ANON_THPS) - nr *= HPAGE_PMD_NR; -#endif seq_printf(m, "%s %lu\n", memcg1_stat_names[i], nr * PAGE_SIZE); } @@ -4121,10 +4116,6 @@ static int memcg_stat_show(struct seq_fi if (memcg1_stats[i] == MEMCG_SWAP && !do_memsw_account()) continue; nr = memcg_page_state(memcg, memcg1_stats[i]); -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memcg1_stats[i] == NR_ANON_THPS) - nr *= HPAGE_PMD_NR; -#endif seq_printf(m, "total_%s %llu\n", memcg1_stat_names[i], (u64)nr * PAGE_SIZE); } @@ -5652,10 +5643,11 @@ static int mem_cgroup_move_account(struc __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages); __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages); if (PageTransHuge(page)) { - __dec_lruvec_state(from_vec, NR_ANON_THPS); - __inc_lruvec_state(to_vec, NR_ANON_THPS); + __mod_lruvec_state(from_vec, NR_ANON_THPS, + -nr_pages); + __mod_lruvec_state(to_vec, NR_ANON_THPS, + nr_pages); } - } } else { __mod_lruvec_state(from_vec, NR_FILE_PAGES, -nr_pages); --- a/mm/page_alloc.c~mm-memcontrol-convert-nr_anon_thps-account-to-pages +++ a/mm/page_alloc.c @@ -5587,7 +5587,7 @@ void show_free_areas(unsigned int filter K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), - K(node_page_state(pgdat, NR_ANON_THPS) * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), node_page_state(pgdat, NR_KERNEL_STACK_KB), --- a/mm/rmap.c~mm-memcontrol-convert-nr_anon_thps-account-to-pages +++ a/mm/rmap.c @@ -1144,7 +1144,7 @@ void do_page_add_anon_rmap(struct page * * disabled. */ if (compound) - __inc_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, nr); __mod_lruvec_page_state(page, NR_ANON_MAPPED, nr); } @@ -1186,7 +1186,7 @@ void page_add_new_anon_rmap(struct page if (hpage_pincount_available(page)) atomic_set(compound_pincount_ptr(page), 0); - __inc_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, nr); } else { /* Anon THP always mapped first with PMD */ VM_BUG_ON_PAGE(PageTransCompound(page), page); @@ -1292,7 +1292,7 @@ static void page_remove_anon_compound_rm if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) return; - __dec_lruvec_page_state(page, NR_ANON_THPS); + __mod_lruvec_page_state(page, NR_ANON_THPS, -thp_nr_pages(page)); if (TestClearPageDoubleMap(page)) { /* --- a/mm/vmstat.c~mm-memcontrol-convert-nr_anon_thps-account-to-pages +++ a/mm/vmstat.c @@ -1619,8 +1619,12 @@ static void zoneinfo_show_print(struct s if (is_zone_first_populated(pgdat, zone)) { seq_printf(m, "\n per-node stats"); for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { + unsigned long pages = node_page_state_pages(pgdat, i); + + if (vmstat_item_print_in_thp(i)) + pages /= HPAGE_PMD_NR; seq_printf(m, "\n %-12s %lu", node_stat_name(i), - node_page_state_pages(pgdat, i)); + pages); } } seq_printf(m, @@ -1740,8 +1744,11 @@ static void *vmstat_start(struct seq_fil v += NR_VM_NUMA_STAT_ITEMS; #endif - for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) + for (i = 0; i < NR_VM_NODE_STAT_ITEMS; i++) { v[i] = global_node_page_state_pages(i); + if (vmstat_item_print_in_thp(i)) + v[i] /= HPAGE_PMD_NR; + } v += NR_VM_NODE_STAT_ITEMS; global_dirty_limits(v + NR_DIRTY_BG_THRESHOLD, From patchwork Wed Feb 24 20:03:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102529 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45A22C433E6 for ; Wed, 24 Feb 2021 20:03:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D4A1D64F24 for ; Wed, 24 Feb 2021 20:03:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D4A1D64F24 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6E24D6B0072; Wed, 24 Feb 2021 15:03:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6B8BF8D0007; Wed, 24 Feb 2021 15:03:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5CF5B8D0006; Wed, 24 Feb 2021 15:03:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0116.hostedemail.com [216.40.44.116]) by kanga.kvack.org (Postfix) with ESMTP id 46CC66B0072 for ; Wed, 24 Feb 2021 15:03:30 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 151DFF01B for ; Wed, 24 Feb 2021 20:03:30 +0000 (UTC) X-FDA: 77854236180.02.F7303C1 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP id 5BDAEE000115 for ; Wed, 24 Feb 2021 20:03:29 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B040B64F1D; Wed, 24 Feb 2021 20:03:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197008; bh=kwho/Ra8rKQRvPpWm38RH82MxQgAjdzU1WudoyYMLvY=; h=Date:From:To:Subject:In-Reply-To:From; b=zDlzbciHaorhkfhVQM0fgjcINmSOXdmbH6QBwUufoFzfeKxlxnEf276V8pmNo4uFt mQVmFdit1NoT9+s4oy6cK97zfH8R8QhWk/avZSw4zww8ANzeUJY8XX6wMNKXTbkaLu H3aU0r4tl6f/tBG/GPC5wpZQxALCSoDE1o5R2j0E= Date: Wed, 24 Feb 2021 12:03:27 -0800 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, feng.tang@intel.com, gregkh@linuxfoundation.org, guro@fb.com, hannes@cmpxchg.org, hughd@google.com, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, neilb@suse.de, pankaj.gupta@cloud.ionos.com, rafael@kernel.org, rdunlap@infradead.org, samitolvanen@google.com, shakeelb@google.com, songmuchun@bytedance.com, torvalds@linux-foundation.org, vdavydov.dev@gmail.com Subject: [patch 059/173] mm: memcontrol: convert NR_FILE_THPS account to pages Message-ID: <20210224200327.2tXW4nUtb%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 5BDAEE000115 X-Stat-Signature: box54sugsdx9cpkmnynspr49szzdbffa Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf30; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197009-804781 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: memcontrol: convert NR_FILE_THPS account to pages Currently we use struct per_cpu_nodestat to cache the vmstat counters, which leads to inaccurate statistics especially THP vmstat counters. In the systems with if hundreds of processors it can be GBs of memory. For example, for a 96 CPUs system, the threshold is the maximum number of 125. And the per cpu counters can cache 23.4375 GB in total. The THP page is already a form of batched addition (it will add 512 worth of memory in one go) so skipping the batching seems like sensible. Although every THP stats update overflows the per-cpu counter, resorting to atomic global updates. But it can make the statistics more accuracy for the THP vmstat counters. So we convert the NR_FILE_THPS account to pages. This patch is consistent with 8f182270dfec ("mm/swap.c: flush lru pvecs on compound page arrival"). Doing this also can make the unit of vmstat counters more unified. Finally, the unit of the vmstat counters are pages, kB and bytes. The B/KB suffix can tell us that the unit is bytes or kB. The rest which is without suffix are pages. Link: https://lkml.kernel.org/r/20201228164110.2838-4-songmuchun@bytedance.com Signed-off-by: Muchun Song Cc: Alexey Dobriyan Cc: Feng Tang Cc: Greg Kroah-Hartman Cc: Hugh Dickins Cc: Johannes Weiner Cc: Joonsoo Kim Cc: Michal Hocko Cc: NeilBrown Cc: Pankaj Gupta Cc: Rafael. J. Wysocki Cc: Randy Dunlap Cc: Roman Gushchin Cc: Sami Tolvanen Cc: Shakeel Butt Cc: Vladimir Davydov Signed-off-by: Andrew Morton --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 3 ++- mm/filemap.c | 2 +- mm/huge_memory.c | 5 ++++- mm/khugepaged.c | 4 +++- mm/memcontrol.c | 5 ++--- 7 files changed, 14 insertions(+), 10 deletions(-) --- a/drivers/base/node.c~mm-memcontrol-convert-nr_file_thps-account-to-pages +++ a/drivers/base/node.c @@ -466,8 +466,7 @@ static ssize_t node_read_meminfo(struct HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), - nid, K(node_page_state(pgdat, NR_FILE_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_FILE_THPS)), nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) #endif --- a/fs/proc/meminfo.c~mm-memcontrol-convert-nr_file_thps-account-to-pages +++ a/fs/proc/meminfo.c @@ -135,7 +135,7 @@ static int meminfo_proc_show(struct seq_ show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); show_val_kb(m, "FileHugePages: ", - global_node_page_state(NR_FILE_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); #endif --- a/include/linux/mmzone.h~mm-memcontrol-convert-nr_file_thps-account-to-pages +++ a/include/linux/mmzone.h @@ -219,7 +219,8 @@ static __always_inline bool vmstat_item_ if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) return false; - return item == NR_ANON_THPS; + return item == NR_ANON_THPS || + item == NR_FILE_THPS; } /* --- a/mm/filemap.c~mm-memcontrol-convert-nr_file_thps-account-to-pages +++ a/mm/filemap.c @@ -208,7 +208,7 @@ static void unaccount_page_cache_page(st if (PageTransHuge(page)) __dec_lruvec_page_state(page, NR_SHMEM_THPS); } else if (PageTransHuge(page)) { - __dec_lruvec_page_state(page, NR_FILE_THPS); + __mod_lruvec_page_state(page, NR_FILE_THPS, -nr); filemap_nr_thps_dec(mapping); } --- a/mm/huge_memory.c~mm-memcontrol-convert-nr_file_thps-account-to-pages +++ a/mm/huge_memory.c @@ -2752,10 +2752,13 @@ int split_huge_page_to_list(struct page } spin_unlock(&ds_queue->split_queue_lock); if (mapping) { + int nr = thp_nr_pages(head); + if (PageSwapBacked(head)) __dec_lruvec_page_state(head, NR_SHMEM_THPS); else - __dec_lruvec_page_state(head, NR_FILE_THPS); + __mod_lruvec_page_state(head, NR_FILE_THPS, + -nr); } __split_huge_page(page, list, end); --- a/mm/khugepaged.c~mm-memcontrol-convert-nr_file_thps-account-to-pages +++ a/mm/khugepaged.c @@ -1643,6 +1643,7 @@ static void collapse_file(struct mm_stru XA_STATE_ORDER(xas, &mapping->i_pages, start, HPAGE_PMD_ORDER); int nr_none = 0, result = SCAN_SUCCEED; bool is_shmem = shmem_file(file); + int nr; VM_BUG_ON(!IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && !is_shmem); VM_BUG_ON(start & (HPAGE_PMD_NR - 1)); @@ -1854,11 +1855,12 @@ out_unlock: put_page(page); goto xa_unlocked; } + nr = thp_nr_pages(new_page); if (is_shmem) __inc_lruvec_page_state(new_page, NR_SHMEM_THPS); else { - __inc_lruvec_page_state(new_page, NR_FILE_THPS); + __mod_lruvec_page_state(new_page, NR_FILE_THPS, nr); filemap_nr_thps_inc(mapping); } --- a/mm/memcontrol.c~mm-memcontrol-convert-nr_file_thps-account-to-pages +++ a/mm/memcontrol.c @@ -1534,7 +1534,7 @@ static struct memory_stat memory_stats[] * constant(e.g. powerpc). */ { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, - { "file_thp", 0, NR_FILE_THPS }, + { "file_thp", PAGE_SIZE, NR_FILE_THPS }, { "shmem_thp", 0, NR_SHMEM_THPS }, #endif { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, @@ -1566,8 +1566,7 @@ static int __init memory_stats_init(void for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { #ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_FILE_THPS || - memory_stats[i].idx == NR_SHMEM_THPS) + if (memory_stats[i].idx == NR_SHMEM_THPS) memory_stats[i].ratio = HPAGE_PMD_SIZE; #endif VM_BUG_ON(!memory_stats[i].ratio); From patchwork Wed Feb 24 20:03:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102531 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E1A3C433DB for ; Wed, 24 Feb 2021 20:03:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B67D864E6C for ; Wed, 24 Feb 2021 20:03:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B67D864E6C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 496566B0075; Wed, 24 Feb 2021 15:03:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 46F0A8D0007; Wed, 24 Feb 2021 15:03:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 336828D0006; Wed, 24 Feb 2021 15:03:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0198.hostedemail.com [216.40.44.198]) by kanga.kvack.org (Postfix) with ESMTP id 1E29C6B0075 for ; Wed, 24 Feb 2021 15:03:35 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id D6A9E824C448 for ; Wed, 24 Feb 2021 20:03:34 +0000 (UTC) X-FDA: 77854236348.22.8B6EE23 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf13.hostedemail.com (Postfix) with ESMTP id AFCF2E000112 for ; Wed, 24 Feb 2021 20:03:31 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0C88764F24; Wed, 24 Feb 2021 20:03:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197013; bh=jZTHcxv/5SzSsCDU3fViTdSXnALMKQaB98xFlmh9jmw=; h=Date:From:To:Subject:In-Reply-To:From; b=OUGdOsDyGkHLAt0p31fyv5MZlFnZqUwZHWWwoK257R6PkSxbYwpFZlCOxIbyzYjsw OdxslnYWn55AzIYBmAV+yk7onogVp3RQedH77KArNxUbHcB5IxyfHN41RPpg1tMaUa yZ4wLNfNncwesfK2dggWbRNZJ6w/auHnSHjk/ubU= Date: Wed, 24 Feb 2021 12:03:31 -0800 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, feng.tang@intel.com, gregkh@linuxfoundation.org, guro@fb.com, hannes@cmpxchg.org, hughd@google.com, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, neilb@suse.de, pankaj.gupta@cloud.ionos.com, rafael@kernel.org, rdunlap@infradead.org, samitolvanen@google.com, shakeelb@google.com, songmuchun@bytedance.com, torvalds@linux-foundation.org, vdavydov.dev@gmail.com Subject: [patch 060/173] mm: memcontrol: convert NR_SHMEM_THPS account to pages Message-ID: <20210224200331.zTTNbuNoQ%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: AFCF2E000112 X-Stat-Signature: bgmgf4osg17mf8jjpdswwtio81hctfoz Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf13; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197011-934551 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: memcontrol: convert NR_SHMEM_THPS account to pages Currently we use struct per_cpu_nodestat to cache the vmstat counters, which leads to inaccurate statistics especially THP vmstat counters. In the systems with hundreds of processors it can be GBs of memory. For example, for a 96 CPUs system, the threshold is the maximum number of 125. And the per cpu counters can cache 23.4375 GB in total. The THP page is already a form of batched addition (it will add 512 worth of memory in one go) so skipping the batching seems like sensible. Although every THP stats update overflows the per-cpu counter, resorting to atomic global updates. But it can make the statistics more accuracy for the THP vmstat counters. So we convert the NR_SHMEM_THPS account to pages. This patch is consistent with 8f182270dfec ("mm/swap.c: flush lru pvecs on compound page arrival"). Doing this also can make the unit of vmstat counters more unified. Finally, the unit of the vmstat counters are pages, kB and bytes. The B/KB suffix can tell us that the unit is bytes or kB. The rest which is without suffix are pages. Link: https://lkml.kernel.org/r/20201228164110.2838-5-songmuchun@bytedance.com Signed-off-by: Muchun Song Cc: Alexey Dobriyan Cc: Feng Tang Cc: Greg Kroah-Hartman Cc: Hugh Dickins Cc: Johannes Weiner Cc: Joonsoo Kim Cc: Michal Hocko Cc: NeilBrown Cc: Pankaj Gupta Cc: Rafael. J. Wysocki Cc: Randy Dunlap Cc: Roman Gushchin Cc: Sami Tolvanen Cc: Shakeel Butt Cc: Vladimir Davydov Signed-off-by: Andrew Morton --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 3 ++- mm/filemap.c | 2 +- mm/huge_memory.c | 3 ++- mm/khugepaged.c | 2 +- mm/memcontrol.c | 26 ++------------------------ mm/page_alloc.c | 2 +- mm/shmem.c | 2 +- 9 files changed, 12 insertions(+), 33 deletions(-) --- a/drivers/base/node.c~mm-memcontrol-convert-nr_shmem_thps-account-to-pages +++ a/drivers/base/node.c @@ -462,8 +462,7 @@ static ssize_t node_read_meminfo(struct #ifdef CONFIG_TRANSPARENT_HUGEPAGE , nid, K(node_page_state(pgdat, NR_ANON_THPS)), - nid, K(node_page_state(pgdat, NR_SHMEM_THPS) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), nid, K(node_page_state(pgdat, NR_FILE_THPS)), --- a/fs/proc/meminfo.c~mm-memcontrol-convert-nr_shmem_thps-account-to-pages +++ a/fs/proc/meminfo.c @@ -131,7 +131,7 @@ static int meminfo_proc_show(struct seq_ show_val_kb(m, "AnonHugePages: ", global_node_page_state(NR_ANON_THPS)); show_val_kb(m, "ShmemHugePages: ", - global_node_page_state(NR_SHMEM_THPS) * HPAGE_PMD_NR); + global_node_page_state(NR_SHMEM_THPS)); show_val_kb(m, "ShmemPmdMapped: ", global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); show_val_kb(m, "FileHugePages: ", --- a/include/linux/mmzone.h~mm-memcontrol-convert-nr_shmem_thps-account-to-pages +++ a/include/linux/mmzone.h @@ -220,7 +220,8 @@ static __always_inline bool vmstat_item_ return false; return item == NR_ANON_THPS || - item == NR_FILE_THPS; + item == NR_FILE_THPS || + item == NR_SHMEM_THPS; } /* --- a/mm/filemap.c~mm-memcontrol-convert-nr_shmem_thps-account-to-pages +++ a/mm/filemap.c @@ -206,7 +206,7 @@ static void unaccount_page_cache_page(st if (PageSwapBacked(page)) { __mod_lruvec_page_state(page, NR_SHMEM, -nr); if (PageTransHuge(page)) - __dec_lruvec_page_state(page, NR_SHMEM_THPS); + __mod_lruvec_page_state(page, NR_SHMEM_THPS, -nr); } else if (PageTransHuge(page)) { __mod_lruvec_page_state(page, NR_FILE_THPS, -nr); filemap_nr_thps_dec(mapping); --- a/mm/huge_memory.c~mm-memcontrol-convert-nr_shmem_thps-account-to-pages +++ a/mm/huge_memory.c @@ -2755,7 +2755,8 @@ int split_huge_page_to_list(struct page int nr = thp_nr_pages(head); if (PageSwapBacked(head)) - __dec_lruvec_page_state(head, NR_SHMEM_THPS); + __mod_lruvec_page_state(head, NR_SHMEM_THPS, + -nr); else __mod_lruvec_page_state(head, NR_FILE_THPS, -nr); --- a/mm/khugepaged.c~mm-memcontrol-convert-nr_shmem_thps-account-to-pages +++ a/mm/khugepaged.c @@ -1858,7 +1858,7 @@ out_unlock: nr = thp_nr_pages(new_page); if (is_shmem) - __inc_lruvec_page_state(new_page, NR_SHMEM_THPS); + __mod_lruvec_page_state(new_page, NR_SHMEM_THPS, nr); else { __mod_lruvec_page_state(new_page, NR_FILE_THPS, nr); filemap_nr_thps_inc(mapping); --- a/mm/memcontrol.c~mm-memcontrol-convert-nr_shmem_thps-account-to-pages +++ a/mm/memcontrol.c @@ -1516,7 +1516,7 @@ struct memory_stat { unsigned int idx; }; -static struct memory_stat memory_stats[] = { +static const struct memory_stat memory_stats[] = { { "anon", PAGE_SIZE, NR_ANON_MAPPED }, { "file", PAGE_SIZE, NR_FILE_PAGES }, { "kernel_stack", 1024, NR_KERNEL_STACK_KB }, @@ -1528,14 +1528,9 @@ static struct memory_stat memory_stats[] { "file_dirty", PAGE_SIZE, NR_FILE_DIRTY }, { "file_writeback", PAGE_SIZE, NR_WRITEBACK }, #ifdef CONFIG_TRANSPARENT_HUGEPAGE - /* - * The ratio will be initialized in memory_stats_init(). Because - * on some architectures, the macro of HPAGE_PMD_SIZE is not - * constant(e.g. powerpc). - */ { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, { "file_thp", PAGE_SIZE, NR_FILE_THPS }, - { "shmem_thp", 0, NR_SHMEM_THPS }, + { "shmem_thp", PAGE_SIZE, NR_SHMEM_THPS }, #endif { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, { "active_anon", PAGE_SIZE, NR_ACTIVE_ANON }, @@ -1560,23 +1555,6 @@ static struct memory_stat memory_stats[] { "workingset_nodereclaim", 1, WORKINGSET_NODERECLAIM }, }; -static int __init memory_stats_init(void) -{ - int i; - - for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - if (memory_stats[i].idx == NR_SHMEM_THPS) - memory_stats[i].ratio = HPAGE_PMD_SIZE; -#endif - VM_BUG_ON(!memory_stats[i].ratio); - VM_BUG_ON(memory_stats[i].idx >= MEMCG_NR_STAT); - } - - return 0; -} -pure_initcall(memory_stats_init); - static char *memory_stat_format(struct mem_cgroup *memcg) { struct seq_buf s; --- a/mm/page_alloc.c~mm-memcontrol-convert-nr_shmem_thps-account-to-pages +++ a/mm/page_alloc.c @@ -5584,7 +5584,7 @@ void show_free_areas(unsigned int filter K(node_page_state(pgdat, NR_WRITEBACK)), K(node_page_state(pgdat, NR_SHMEM)), #ifdef CONFIG_TRANSPARENT_HUGEPAGE - K(node_page_state(pgdat, NR_SHMEM_THPS) * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_SHMEM_THPS)), K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR), K(node_page_state(pgdat, NR_ANON_THPS)), --- a/mm/shmem.c~mm-memcontrol-convert-nr_shmem_thps-account-to-pages +++ a/mm/shmem.c @@ -713,7 +713,7 @@ next: } if (PageTransHuge(page)) { count_vm_event(THP_FILE_ALLOC); - __inc_lruvec_page_state(page, NR_SHMEM_THPS); + __mod_lruvec_page_state(page, NR_SHMEM_THPS, nr); } mapping->nrpages += nr; __mod_lruvec_page_state(page, NR_FILE_PAGES, nr); From patchwork Wed Feb 24 20:03:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102533 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D4080C433DB for ; Wed, 24 Feb 2021 20:03:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6DE7F64F27 for ; Wed, 24 Feb 2021 20:03:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6DE7F64F27 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EF6476B0006; Wed, 24 Feb 2021 15:03:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EA74D6B007B; Wed, 24 Feb 2021 15:03:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DBDD28D0006; Wed, 24 Feb 2021 15:03:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0098.hostedemail.com [216.40.44.98]) by kanga.kvack.org (Postfix) with ESMTP id C3DD16B0006 for ; Wed, 24 Feb 2021 15:03:39 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 92AA9F006 for ; Wed, 24 Feb 2021 20:03:39 +0000 (UTC) X-FDA: 77854236558.04.8E78529 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP id 98C472BC6 for ; Wed, 24 Feb 2021 20:03:36 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 3053264F26; Wed, 24 Feb 2021 20:03:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197017; bh=AFxRM4kCInF5En0Aet2ikFxdWfUXoU3bNgDO7Rr+GAc=; h=Date:From:To:Subject:In-Reply-To:From; b=QrAgp6k5EiRQgL2JrGZm1gueidEhTexULU2KUGXc++oQ5DxbNkxXGaLiQyrY/R28I /C9Wz6Dw/cUAHX5aXuW9lSIWtp43t/XwjoBfI4mRuwYL0nFll+SALxMgs1pVR0NtnW 9+WPIcZZCjg/4WId8Bf4WzU1WE0Pxszu+JbOlcI4= Date: Wed, 24 Feb 2021 12:03:35 -0800 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, feng.tang@intel.com, gregkh@linuxfoundation.org, guro@fb.com, hannes@cmpxchg.org, hughd@google.com, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, neilb@suse.de, pankaj.gupta@cloud.ionos.com, rafael@kernel.org, rdunlap@infradead.org, samitolvanen@google.com, shakeelb@google.com, songmuchun@bytedance.com, torvalds@linux-foundation.org, vdavydov.dev@gmail.com Subject: [patch 061/173] mm: memcontrol: convert NR_SHMEM_PMDMAPPED account to pages Message-ID: <20210224200335.i3uyijHmX%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 98C472BC6 X-Stat-Signature: 8fwxy1b7kju73o8crsf3xcohd7obfnpy Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf04; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197016-498313 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: memcontrol: convert NR_SHMEM_PMDMAPPED account to pages Currently we use struct per_cpu_nodestat to cache the vmstat counters, which leads to inaccurate statistics especially THP vmstat counters. In the systems with hundreds of processors it can be GBs of memory. For example, for a 96 CPUs system, the threshold is the maximum number of 125. And the per cpu counters can cache 23.4375 GB in total. The THP page is already a form of batched addition (it will add 512 worth of memory in one go) so skipping the batching seems like sensible. Although every THP stats update overflows the per-cpu counter, resorting to atomic global updates. But it can make the statistics more accuracy for the THP vmstat counters. So we convert the NR_SHMEM_PMDMAPPED account to pages. This patch is consistent with 8f182270dfec ("mm/swap.c: flush lru pvecs on compound page arrival"). Doing this also can make the unit of vmstat counters more unified. Finally, the unit of the vmstat counters are pages, kB and bytes. The B/KB suffix can tell us that the unit is bytes or kB. The rest which is without suffix are pages. Link: https://lkml.kernel.org/r/20201228164110.2838-6-songmuchun@bytedance.com Signed-off-by: Muchun Song Cc: Alexey Dobriyan Cc: Feng Tang Cc: Greg Kroah-Hartman Cc: Hugh Dickins Cc: Johannes Weiner Cc: Joonsoo Kim Cc: Michal Hocko Cc: NeilBrown Cc: Pankaj Gupta Cc: Rafael. J. Wysocki Cc: Randy Dunlap Cc: Roman Gushchin Cc: Sami Tolvanen Cc: Shakeel Butt Cc: Vladimir Davydov Signed-off-by: Andrew Morton --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 3 ++- mm/page_alloc.c | 3 +-- mm/rmap.c | 14 ++++++++++---- 5 files changed, 15 insertions(+), 10 deletions(-) --- a/drivers/base/node.c~mm-memcontrol-convert-nr_shmem_pmdmapped-account-to-pages +++ a/drivers/base/node.c @@ -463,8 +463,7 @@ static ssize_t node_read_meminfo(struct , nid, K(node_page_state(pgdat, NR_ANON_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), - nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) * - HPAGE_PMD_NR), + nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), nid, K(node_page_state(pgdat, NR_FILE_THPS)), nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * HPAGE_PMD_NR) --- a/fs/proc/meminfo.c~mm-memcontrol-convert-nr_shmem_pmdmapped-account-to-pages +++ a/fs/proc/meminfo.c @@ -133,7 +133,7 @@ static int meminfo_proc_show(struct seq_ show_val_kb(m, "ShmemHugePages: ", global_node_page_state(NR_SHMEM_THPS)); show_val_kb(m, "ShmemPmdMapped: ", - global_node_page_state(NR_SHMEM_PMDMAPPED) * HPAGE_PMD_NR); + global_node_page_state(NR_SHMEM_PMDMAPPED)); show_val_kb(m, "FileHugePages: ", global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", --- a/include/linux/mmzone.h~mm-memcontrol-convert-nr_shmem_pmdmapped-account-to-pages +++ a/include/linux/mmzone.h @@ -221,7 +221,8 @@ static __always_inline bool vmstat_item_ return item == NR_ANON_THPS || item == NR_FILE_THPS || - item == NR_SHMEM_THPS; + item == NR_SHMEM_THPS || + item == NR_SHMEM_PMDMAPPED; } /* --- a/mm/page_alloc.c~mm-memcontrol-convert-nr_shmem_pmdmapped-account-to-pages +++ a/mm/page_alloc.c @@ -5585,8 +5585,7 @@ void show_free_areas(unsigned int filter K(node_page_state(pgdat, NR_SHMEM)), #ifdef CONFIG_TRANSPARENT_HUGEPAGE K(node_page_state(pgdat, NR_SHMEM_THPS)), - K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED) - * HPAGE_PMD_NR), + K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), K(node_page_state(pgdat, NR_ANON_THPS)), #endif K(node_page_state(pgdat, NR_WRITEBACK_TEMP)), --- a/mm/rmap.c~mm-memcontrol-convert-nr_shmem_pmdmapped-account-to-pages +++ a/mm/rmap.c @@ -1211,14 +1211,17 @@ void page_add_file_rmap(struct page *pag VM_BUG_ON_PAGE(compound && !PageTransHuge(page), page); lock_page_memcg(page); if (compound && PageTransHuge(page)) { - for (i = 0, nr = 0; i < thp_nr_pages(page); i++) { + int nr_pages = thp_nr_pages(page); + + for (i = 0, nr = 0; i < nr_pages; i++) { if (atomic_inc_and_test(&page[i]._mapcount)) nr++; } if (!atomic_inc_and_test(compound_mapcount_ptr(page))) goto out; if (PageSwapBacked(page)) - __inc_node_page_state(page, NR_SHMEM_PMDMAPPED); + __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, + nr_pages); else __inc_node_page_state(page, NR_FILE_PMDMAPPED); } else { @@ -1252,14 +1255,17 @@ static void page_remove_file_rmap(struct /* page still mapped by someone else? */ if (compound && PageTransHuge(page)) { - for (i = 0, nr = 0; i < thp_nr_pages(page); i++) { + int nr_pages = thp_nr_pages(page); + + for (i = 0, nr = 0; i < nr_pages; i++) { if (atomic_add_negative(-1, &page[i]._mapcount)) nr++; } if (!atomic_add_negative(-1, compound_mapcount_ptr(page))) return; if (PageSwapBacked(page)) - __dec_node_page_state(page, NR_SHMEM_PMDMAPPED); + __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, + -nr_pages); else __dec_node_page_state(page, NR_FILE_PMDMAPPED); } else { From patchwork Wed Feb 24 20:03:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102535 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABE21C433E0 for ; Wed, 24 Feb 2021 20:03:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6479A64F27 for ; Wed, 24 Feb 2021 20:03:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6479A64F27 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EB57B6B007B; Wed, 24 Feb 2021 15:03:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E16856B007E; Wed, 24 Feb 2021 15:03:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD5318D0006; Wed, 24 Feb 2021 15:03:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0155.hostedemail.com [216.40.44.155]) by kanga.kvack.org (Postfix) with ESMTP id B57426B007B for ; Wed, 24 Feb 2021 15:03:42 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 86F07181AF5E1 for ; Wed, 24 Feb 2021 20:03:42 +0000 (UTC) X-FDA: 77854236684.25.E008534 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf02.hostedemail.com (Postfix) with ESMTP id 9AB354080F51 for ; Wed, 24 Feb 2021 20:03:28 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5879D64F2A; Wed, 24 Feb 2021 20:03:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197021; bh=H1SDug3eSX3RmBv6Fgm5dqLyOpZRkCjKCItKkzO0aaw=; h=Date:From:To:Subject:In-Reply-To:From; b=Q0qgQy0Oy7rQ8lvexS5AZB19BTH/rwx8yRvJxaTnaDUCwT4MveM5DpBeUBD9GwKrj o0JJnUBdmTcbGA4tPBOMTsIs/M4mTb29RG4xcfkz8o0ZrP0QuJz5PMZ+H6+c86c8Dz Xoejaj18ZkijEXNGvo4NrC8B0j9flKq5+y5MYAgU= Date: Wed, 24 Feb 2021 12:03:39 -0800 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, feng.tang@intel.com, gregkh@linuxfoundation.org, guro@fb.com, hannes@cmpxchg.org, hughd@google.com, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, neilb@suse.de, pankaj.gupta@cloud.ionos.com, rafael@kernel.org, rdunlap@infradead.org, samitolvanen@google.com, shakeelb@google.com, songmuchun@bytedance.com, torvalds@linux-foundation.org, vdavydov.dev@gmail.com Subject: [patch 062/173] mm: memcontrol: convert NR_FILE_PMDMAPPED account to pages Message-ID: <20210224200339.bFOkRzUbu%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 6suskr7yjykh7gcp8j73tawef45y6w6y X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 9AB354080F51 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf02; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197008-371564 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: memcontrol: convert NR_FILE_PMDMAPPED account to pages Currently we use struct per_cpu_nodestat to cache the vmstat counters, which leads to inaccurate statistics especially THP vmstat counters. In the systems with hundreds of processors it can be GBs of memory. For example, for a 96 CPUs system, the threshold is the maximum number of 125. And the per cpu counters can cache 23.4375 GB in total. The THP page is already a form of batched addition (it will add 512 worth of memory in one go) so skipping the batching seems like sensible. Although every THP stats update overflows the per-cpu counter, resorting to atomic global updates. But it can make the statistics more accuracy for the THP vmstat counters. So we convert the NR_FILE_PMDMAPPED account to pages. This patch is consistent with 8f182270dfec ("mm/swap.c: flush lru pvecs on compound page arrival"). Doing this also can make the unit of vmstat counters more unified. Finally, the unit of the vmstat counters are pages, kB and bytes. The B/KB suffix can tell us that the unit is bytes or kB. The rest which is without suffix are pages. Link: https://lkml.kernel.org/r/20201228164110.2838-7-songmuchun@bytedance.com Signed-off-by: Muchun Song Cc: Alexey Dobriyan Cc: Feng Tang Cc: Greg Kroah-Hartman Cc: Hugh Dickins Cc: Johannes Weiner Cc: Joonsoo Kim Cc: Michal Hocko Cc: NeilBrown Cc: Pankaj Gupta Cc: Rafael. J. Wysocki Cc: Randy Dunlap Cc: Roman Gushchin Cc: Sami Tolvanen Cc: Shakeel Butt Cc: Vladimir Davydov Signed-off-by: Andrew Morton --- drivers/base/node.c | 3 +-- fs/proc/meminfo.c | 2 +- include/linux/mmzone.h | 3 ++- mm/rmap.c | 6 ++++-- 4 files changed, 8 insertions(+), 6 deletions(-) --- a/drivers/base/node.c~mm-memcontrol-convert-nr_file_pmdmapped-account-to-pages +++ a/drivers/base/node.c @@ -465,8 +465,7 @@ static ssize_t node_read_meminfo(struct nid, K(node_page_state(pgdat, NR_SHMEM_THPS)), nid, K(node_page_state(pgdat, NR_SHMEM_PMDMAPPED)), nid, K(node_page_state(pgdat, NR_FILE_THPS)), - nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED) * - HPAGE_PMD_NR) + nid, K(node_page_state(pgdat, NR_FILE_PMDMAPPED)) #endif ); len += hugetlb_report_node_meminfo(buf, len, nid); --- a/fs/proc/meminfo.c~mm-memcontrol-convert-nr_file_pmdmapped-account-to-pages +++ a/fs/proc/meminfo.c @@ -137,7 +137,7 @@ static int meminfo_proc_show(struct seq_ show_val_kb(m, "FileHugePages: ", global_node_page_state(NR_FILE_THPS)); show_val_kb(m, "FilePmdMapped: ", - global_node_page_state(NR_FILE_PMDMAPPED) * HPAGE_PMD_NR); + global_node_page_state(NR_FILE_PMDMAPPED)); #endif #ifdef CONFIG_CMA --- a/include/linux/mmzone.h~mm-memcontrol-convert-nr_file_pmdmapped-account-to-pages +++ a/include/linux/mmzone.h @@ -222,7 +222,8 @@ static __always_inline bool vmstat_item_ return item == NR_ANON_THPS || item == NR_FILE_THPS || item == NR_SHMEM_THPS || - item == NR_SHMEM_PMDMAPPED; + item == NR_SHMEM_PMDMAPPED || + item == NR_FILE_PMDMAPPED; } /* --- a/mm/rmap.c~mm-memcontrol-convert-nr_file_pmdmapped-account-to-pages +++ a/mm/rmap.c @@ -1223,7 +1223,8 @@ void page_add_file_rmap(struct page *pag __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, nr_pages); else - __inc_node_page_state(page, NR_FILE_PMDMAPPED); + __mod_lruvec_page_state(page, NR_FILE_PMDMAPPED, + nr_pages); } else { if (PageTransCompound(page) && page_mapping(page)) { VM_WARN_ON_ONCE(!PageLocked(page)); @@ -1267,7 +1268,8 @@ static void page_remove_file_rmap(struct __mod_lruvec_page_state(page, NR_SHMEM_PMDMAPPED, -nr_pages); else - __dec_node_page_state(page, NR_FILE_PMDMAPPED); + __mod_lruvec_page_state(page, NR_FILE_PMDMAPPED, + -nr_pages); } else { if (!atomic_add_negative(-1, &page->_mapcount)) return; From patchwork Wed Feb 24 20:03:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102537 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BB3DC433DB for ; Wed, 24 Feb 2021 20:03:48 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E1ECC64F27 for ; Wed, 24 Feb 2021 20:03:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E1ECC64F27 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 759526B0081; Wed, 24 Feb 2021 15:03:47 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 730EB8D0003; Wed, 24 Feb 2021 15:03:47 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5CDED8D0001; Wed, 24 Feb 2021 15:03:47 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0242.hostedemail.com [216.40.44.242]) by kanga.kvack.org (Postfix) with ESMTP id 471C96B0081 for ; Wed, 24 Feb 2021 15:03:47 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 0E052F042 for ; Wed, 24 Feb 2021 20:03:47 +0000 (UTC) X-FDA: 77854236894.10.278153E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id BD8142000395 for ; Wed, 24 Feb 2021 20:03:36 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8188464E6C; Wed, 24 Feb 2021 20:03:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197025; bh=4wlivdGNqC0NnsKVU4fGd/bbq7ZarQlOGThmJ9RTiSA=; h=Date:From:To:Subject:In-Reply-To:From; b=TUr3M9j1sVNHzmr1V+nAqxLUekm8Q/fgYoSc62bf9i/00sdoNja3RGPMBcw9HTA7M +ANWdzNJ2CEBC1qO0gfaWQoXrVYjrmKFfD2p/fz7UEgAwFsQenEqU/pgHVKzs9Udmd Emy22TtQDY+QPBDpOKn8Rl8uIUUl5P3ubcY4rNYM= Date: Wed, 24 Feb 2021 12:03:43 -0800 From: Andrew Morton To: adobriyan@gmail.com, akpm@linux-foundation.org, feng.tang@intel.com, gregkh@linuxfoundation.org, guro@fb.com, hannes@cmpxchg.org, hughd@google.com, iamjoonsoo.kim@lge.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, neilb@suse.de, pankaj.gupta@cloud.ionos.com, rafael@kernel.org, rdunlap@infradead.org, samitolvanen@google.com, shakeelb@google.com, songmuchun@bytedance.com, torvalds@linux-foundation.org, vdavydov.dev@gmail.com Subject: [patch 063/173] mm: memcontrol: make the slab calculation consistent Message-ID: <20210224200343.eA4DvW2aL%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 1m53tn4pczbmsn88syqprbw8cxz4c3ou X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: BD8142000395 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf11; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197016-610402 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: memcontrol: make the slab calculation consistent Although the ratio of the slab is one, we also should read the ratio from the related memory_stats instead of hard-coding. And the local variable of size is already the value of slab_unreclaimable. So we do not need to read again. To do this we need some code like below: if (unlikely(memory_stats[i].idx == NR_SLAB_UNRECLAIMABLE_B)) { - size = memcg_page_state(memcg, NR_SLAB_RECLAIMABLE_B) + - memcg_page_state(memcg, NR_SLAB_UNRECLAIMABLE_B); + VM_BUG_ON(i < 1); + VM_BUG_ON(memory_stats[i - 1].idx != NR_SLAB_RECLAIMABLE_B); + size += memcg_page_state(memcg, memory_stats[i - 1].idx) * + memory_stats[i - 1].ratio; It requires a series of VM_BUG_ONs or comments to ensure these two items are actually adjacent and in the right order. So it would probably be easier to implement this using a wrapper that has a big switch() for unit conversion. More details about this discussion can refer to: https://lore.kernel.org/patchwork/patch/1348611/ This would fix the ratio inconsistency and get rid of the order guarantee. Link: https://lkml.kernel.org/r/20201228164110.2838-8-songmuchun@bytedance.com Signed-off-by: Muchun Song Cc: Alexey Dobriyan Cc: Feng Tang Cc: Greg Kroah-Hartman Cc: Hugh Dickins Cc: Johannes Weiner Cc: Joonsoo Kim Cc: Michal Hocko Cc: NeilBrown Cc: Pankaj Gupta Cc: Rafael. J. Wysocki Cc: Randy Dunlap Cc: Roman Gushchin Cc: Sami Tolvanen Cc: Shakeel Butt Cc: Vladimir Davydov Signed-off-by: Andrew Morton --- mm/memcontrol.c | 105 ++++++++++++++++++++++++++++------------------ 1 file changed, 66 insertions(+), 39 deletions(-) --- a/mm/memcontrol.c~mm-memcontrol-make-the-slab-calculation-consistent +++ a/mm/memcontrol.c @@ -1512,49 +1512,71 @@ static bool mem_cgroup_wait_acct_move(st struct memory_stat { const char *name; - unsigned int ratio; unsigned int idx; }; static const struct memory_stat memory_stats[] = { - { "anon", PAGE_SIZE, NR_ANON_MAPPED }, - { "file", PAGE_SIZE, NR_FILE_PAGES }, - { "kernel_stack", 1024, NR_KERNEL_STACK_KB }, - { "pagetables", PAGE_SIZE, NR_PAGETABLE }, - { "percpu", 1, MEMCG_PERCPU_B }, - { "sock", PAGE_SIZE, MEMCG_SOCK }, - { "shmem", PAGE_SIZE, NR_SHMEM }, - { "file_mapped", PAGE_SIZE, NR_FILE_MAPPED }, - { "file_dirty", PAGE_SIZE, NR_FILE_DIRTY }, - { "file_writeback", PAGE_SIZE, NR_WRITEBACK }, + { "anon", NR_ANON_MAPPED }, + { "file", NR_FILE_PAGES }, + { "kernel_stack", NR_KERNEL_STACK_KB }, + { "pagetables", NR_PAGETABLE }, + { "percpu", MEMCG_PERCPU_B }, + { "sock", MEMCG_SOCK }, + { "shmem", NR_SHMEM }, + { "file_mapped", NR_FILE_MAPPED }, + { "file_dirty", NR_FILE_DIRTY }, + { "file_writeback", NR_WRITEBACK }, #ifdef CONFIG_TRANSPARENT_HUGEPAGE - { "anon_thp", PAGE_SIZE, NR_ANON_THPS }, - { "file_thp", PAGE_SIZE, NR_FILE_THPS }, - { "shmem_thp", PAGE_SIZE, NR_SHMEM_THPS }, + { "anon_thp", NR_ANON_THPS }, + { "file_thp", NR_FILE_THPS }, + { "shmem_thp", NR_SHMEM_THPS }, #endif - { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, - { "active_anon", PAGE_SIZE, NR_ACTIVE_ANON }, - { "inactive_file", PAGE_SIZE, NR_INACTIVE_FILE }, - { "active_file", PAGE_SIZE, NR_ACTIVE_FILE }, - { "unevictable", PAGE_SIZE, NR_UNEVICTABLE }, - - /* - * Note: The slab_reclaimable and slab_unreclaimable must be - * together and slab_reclaimable must be in front. - */ - { "slab_reclaimable", 1, NR_SLAB_RECLAIMABLE_B }, - { "slab_unreclaimable", 1, NR_SLAB_UNRECLAIMABLE_B }, + { "inactive_anon", NR_INACTIVE_ANON }, + { "active_anon", NR_ACTIVE_ANON }, + { "inactive_file", NR_INACTIVE_FILE }, + { "active_file", NR_ACTIVE_FILE }, + { "unevictable", NR_UNEVICTABLE }, + { "slab_reclaimable", NR_SLAB_RECLAIMABLE_B }, + { "slab_unreclaimable", NR_SLAB_UNRECLAIMABLE_B }, /* The memory events */ - { "workingset_refault_anon", 1, WORKINGSET_REFAULT_ANON }, - { "workingset_refault_file", 1, WORKINGSET_REFAULT_FILE }, - { "workingset_activate_anon", 1, WORKINGSET_ACTIVATE_ANON }, - { "workingset_activate_file", 1, WORKINGSET_ACTIVATE_FILE }, - { "workingset_restore_anon", 1, WORKINGSET_RESTORE_ANON }, - { "workingset_restore_file", 1, WORKINGSET_RESTORE_FILE }, - { "workingset_nodereclaim", 1, WORKINGSET_NODERECLAIM }, + { "workingset_refault_anon", WORKINGSET_REFAULT_ANON }, + { "workingset_refault_file", WORKINGSET_REFAULT_FILE }, + { "workingset_activate_anon", WORKINGSET_ACTIVATE_ANON }, + { "workingset_activate_file", WORKINGSET_ACTIVATE_FILE }, + { "workingset_restore_anon", WORKINGSET_RESTORE_ANON }, + { "workingset_restore_file", WORKINGSET_RESTORE_FILE }, + { "workingset_nodereclaim", WORKINGSET_NODERECLAIM }, }; +/* Translate stat items to the correct unit for memory.stat output */ +static int memcg_page_state_unit(int item) +{ + switch (item) { + case MEMCG_PERCPU_B: + case NR_SLAB_RECLAIMABLE_B: + case NR_SLAB_UNRECLAIMABLE_B: + case WORKINGSET_REFAULT_ANON: + case WORKINGSET_REFAULT_FILE: + case WORKINGSET_ACTIVATE_ANON: + case WORKINGSET_ACTIVATE_FILE: + case WORKINGSET_RESTORE_ANON: + case WORKINGSET_RESTORE_FILE: + case WORKINGSET_NODERECLAIM: + return 1; + case NR_KERNEL_STACK_KB: + return SZ_1K; + default: + return PAGE_SIZE; + } +} + +static inline unsigned long memcg_page_state_output(struct mem_cgroup *memcg, + int item) +{ + return memcg_page_state(memcg, item) * memcg_page_state_unit(item); +} + static char *memory_stat_format(struct mem_cgroup *memcg) { struct seq_buf s; @@ -1578,13 +1600,12 @@ static char *memory_stat_format(struct m for (i = 0; i < ARRAY_SIZE(memory_stats); i++) { u64 size; - size = memcg_page_state(memcg, memory_stats[i].idx); - size *= memory_stats[i].ratio; + size = memcg_page_state_output(memcg, memory_stats[i].idx); seq_buf_printf(&s, "%s %llu\n", memory_stats[i].name, size); if (unlikely(memory_stats[i].idx == NR_SLAB_UNRECLAIMABLE_B)) { - size = memcg_page_state(memcg, NR_SLAB_RECLAIMABLE_B) + - memcg_page_state(memcg, NR_SLAB_UNRECLAIMABLE_B); + size += memcg_page_state_output(memcg, + NR_SLAB_RECLAIMABLE_B); seq_buf_printf(&s, "slab %llu\n", size); } } @@ -6375,6 +6396,12 @@ static int memory_stat_show(struct seq_f } #ifdef CONFIG_NUMA +static inline unsigned long lruvec_page_state_output(struct lruvec *lruvec, + int item) +{ + return lruvec_page_state(lruvec, item) * memcg_page_state_unit(item); +} + static int memory_numa_stat_show(struct seq_file *m, void *v) { int i; @@ -6392,8 +6419,8 @@ static int memory_numa_stat_show(struct struct lruvec *lruvec; lruvec = mem_cgroup_lruvec(memcg, NODE_DATA(nid)); - size = lruvec_page_state(lruvec, memory_stats[i].idx); - size *= memory_stats[i].ratio; + size = lruvec_page_state_output(lruvec, + memory_stats[i].idx); seq_printf(m, " N%d=%llu", nid, size); } seq_putc(m, '\n'); From patchwork Wed Feb 24 20:03:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102539 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3AA82C433DB for ; Wed, 24 Feb 2021 20:03:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E100064F27 for ; Wed, 24 Feb 2021 20:03:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E100064F27 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7CB0C6B0083; Wed, 24 Feb 2021 15:03:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 77D6B8D0003; Wed, 24 Feb 2021 15:03:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 66ACB8D0001; Wed, 24 Feb 2021 15:03:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0034.hostedemail.com [216.40.44.34]) by kanga.kvack.org (Postfix) with ESMTP id 4B8396B0083 for ; Wed, 24 Feb 2021 15:03:50 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 1B57CF05F for ; Wed, 24 Feb 2021 20:03:50 +0000 (UTC) X-FDA: 77854237020.29.901341C Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP id 465AEDE for ; Wed, 24 Feb 2021 20:03:48 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5FD4464F2C; Wed, 24 Feb 2021 20:03:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197028; bh=JFEu0I+ART4jv4GXhslXzIUBsR5SPSsSdsZ4TblWBIQ=; h=Date:From:To:Subject:In-Reply-To:From; b=fiF8C6S8GwYhwaz3YONmtICNcvF/g2lyeqjFsYMnMlX6MjxmaKaZqteMKLgt4k36D 9j1v3weph1FV7utPwMER650ZDTQgFfcC1RBqV5jr8tHGtGpWJuJBvNhQtLRe03Tgv/ F21COc250RAUb6LLZinsva47CQ/9JjEGRWIcxmJU= Date: Wed, 24 Feb 2021 12:03:47 -0800 From: Andrew Morton To: akpm@linux-foundation.org, alex.shi@linux.alibaba.com, hannes@cmpxchg.org, hughd@google.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vdavydov.dev@gmail.com Subject: [patch 064/173] mm/memcg: revise the using condition of lock_page_lruvec function series Message-ID: <20210224200347.wvBiCvR8A%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 465AEDE X-Stat-Signature: zs89wad7sn7nzckg6cnbi3jwen8x4qt1 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf04; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197028-436882 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alex Shi Subject: mm/memcg: revise the using condition of lock_page_lruvec function series lock_page_lruvec() and its variants are safe to use under the same conditions as commit_charge(): add lock_page_memcg() to the comment. Polished with Hugh Dickins' suggestions, thanks! Link: https://lkml.kernel.org/r/1608614453-10739-1-git-send-email-alex.shi@linux.alibaba.com Signed-off-by: Alex Shi Acked-by: Hugh Dickins Cc: Hugh Dickins Cc: Johannes Weiner Cc: Michal Hocko Cc: Vladimir Davydov Signed-off-by: Andrew Morton --- mm/memcontrol.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) --- a/mm/memcontrol.c~mm-memcg-revise-the-using-condition-of-lock_page_lruvec-function-series +++ a/mm/memcontrol.c @@ -1346,10 +1346,11 @@ void lruvec_memcg_debug(struct lruvec *l * lock_page_lruvec - lock and return lruvec for a given page. * @page: the page * - * This series functions should be used in either conditions: - * PageLRU is cleared or unset - * or page->_refcount is zero - * or page is locked. + * These functions are safe to use under any of the following conditions: + * - page locked + * - PageLRU cleared + * - lock_page_memcg() + * - page->_refcount is zero */ struct lruvec *lock_page_lruvec(struct page *page) { From patchwork Wed Feb 24 20:03:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102541 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45547C433E0 for ; Wed, 24 Feb 2021 20:03:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D717464F30 for ; Wed, 24 Feb 2021 20:03:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D717464F30 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7555F6B0087; Wed, 24 Feb 2021 15:03:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 706D48D0003; Wed, 24 Feb 2021 15:03:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 619F88D0001; Wed, 24 Feb 2021 15:03:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0216.hostedemail.com [216.40.44.216]) by kanga.kvack.org (Postfix) with ESMTP id 4A9156B0087 for ; Wed, 24 Feb 2021 15:03:54 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 109E1824C448 for ; Wed, 24 Feb 2021 20:03:54 +0000 (UTC) X-FDA: 77854237188.20.336BA65 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP id F07BCA000500 for ; Wed, 24 Feb 2021 20:03:52 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 413C064F2E; Wed, 24 Feb 2021 20:03:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197032; bh=MdbD5dS61DyOcD0SHAgsXk29hGrzLzSJMLlZwZMUUbI=; h=Date:From:To:Subject:In-Reply-To:From; b=u8DxfA1fn/Q9EmdbwDmwTB7PY6D4XozEJPFqpiSPhHYRwOTvA8402z5zO3oJb3G9j KwYNg5cZ16SyMJALepmISOXxTQJOdoWLaH3ufMGbbD1iJOykMRTr//WZCOhDvScldu GAzjJF4iZQt+cyUn0AmvhrY3/v+y/dt4axlkymJg= Date: Wed, 24 Feb 2021 12:03:51 -0800 From: Andrew Morton To: akpm@linux-foundation.org, alex.shi@linux.alibaba.com, hannes@cmpxchg.org, hughd@google.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vdavydov.dev@gmail.com Subject: [patch 065/173] mm/memcg: remove rcu locking for lock_page_lruvec function series Message-ID: <20210224200351.MIWbYvcmP%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: F07BCA000500 X-Stat-Signature: gknhpg8c15m5orpw1gbreyc7wcxk6ae5 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf07; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197032-118405 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alex Shi Subject: mm/memcg: remove rcu locking for lock_page_lruvec function series lock_page_lruvec() and its variants used rcu_read_lock() with the intention of safeguarding against the mem_cgroup being destroyed concurrently; but so long as they are called under the specified conditions (as they are), there is no way for the page's mem_cgroup to be destroyed. Delete the unnecessary rcu_read_lock() and _unlock(). Hugh Dickins polished the commit log. Thanks a lot! Link: https://lkml.kernel.org/r/1608614453-10739-2-git-send-email-alex.shi@linux.alibaba.com Signed-off-by: Alex Shi Acked-by: Hugh Dickins Cc: Hugh Dickins Cc: Johannes Weiner Cc: Michal Hocko Cc: Vladimir Davydov Signed-off-by: Andrew Morton --- mm/memcontrol.c | 6 ------ 1 file changed, 6 deletions(-) --- a/mm/memcontrol.c~mm-memcg-remove-rcu-locking-for-lock_page_lruvec-function-series +++ a/mm/memcontrol.c @@ -1357,10 +1357,8 @@ struct lruvec *lock_page_lruvec(struct p struct lruvec *lruvec; struct pglist_data *pgdat = page_pgdat(page); - rcu_read_lock(); lruvec = mem_cgroup_page_lruvec(page, pgdat); spin_lock(&lruvec->lru_lock); - rcu_read_unlock(); lruvec_memcg_debug(lruvec, page); @@ -1372,10 +1370,8 @@ struct lruvec *lock_page_lruvec_irq(stru struct lruvec *lruvec; struct pglist_data *pgdat = page_pgdat(page); - rcu_read_lock(); lruvec = mem_cgroup_page_lruvec(page, pgdat); spin_lock_irq(&lruvec->lru_lock); - rcu_read_unlock(); lruvec_memcg_debug(lruvec, page); @@ -1387,10 +1383,8 @@ struct lruvec *lock_page_lruvec_irqsave( struct lruvec *lruvec; struct pglist_data *pgdat = page_pgdat(page); - rcu_read_lock(); lruvec = mem_cgroup_page_lruvec(page, pgdat); spin_lock_irqsave(&lruvec->lru_lock, *flags); - rcu_read_unlock(); lruvec_memcg_debug(lruvec, page); From patchwork Wed Feb 24 20:03:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102543 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC344C433E0 for ; Wed, 24 Feb 2021 20:03:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 69E6B64F2D for ; Wed, 24 Feb 2021 20:03:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 69E6B64F2D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E80D36B0089; Wed, 24 Feb 2021 15:03:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E32998D0001; Wed, 24 Feb 2021 15:03:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D48076B008C; Wed, 24 Feb 2021 15:03:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0024.hostedemail.com [216.40.44.24]) by kanga.kvack.org (Postfix) with ESMTP id BDA586B0089 for ; Wed, 24 Feb 2021 15:03:57 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 84E8C181AF5E1 for ; Wed, 24 Feb 2021 20:03:57 +0000 (UTC) X-FDA: 77854237314.02.157D2EB Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP id E2C48DE for ; Wed, 24 Feb 2021 20:03:56 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8FD7A64F27; Wed, 24 Feb 2021 20:03:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197036; bh=Y6AsmoIeR0LGl6rs2OMLJycdv1qY2mn+0uMI1ay016M=; h=Date:From:To:Subject:In-Reply-To:From; b=AxFTGxls9o8pt7gtSir67s370mnxqCHAZiOnhxNusw2TIUT4/hnY7KWkz0GQFbRf7 9YN5qijJsyaSjMU8bLfIrEH1f/Jc/6DvpYhc3Qjhu23OW2XybpS68iJR+YNoK2zrvU JJ6ORGaSwBDx9TJvrtirNkAhq0OapW0zLNp6U0O4= Date: Wed, 24 Feb 2021 12:03:55 -0800 From: Andrew Morton To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, shakeelb@google.com, shy828301@gmail.com, songmuchun@bytedance.com, torvalds@linux-foundation.org Subject: [patch 066/173] mm: memcg: add swapcache stat for memcg v2 Message-ID: <20210224200355.nG8Kq29Hu%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: E2C48DE X-Stat-Signature: mxsjt1jhq4fjzt5t47ep1ay78uh39r79 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf29; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197036-291334 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Shakeel Butt Subject: mm: memcg: add swapcache stat for memcg v2 This patch adds swapcache stat for the cgroup v2. The swapcache represents the memory that is accounted against both the memory and the swap limit of the cgroup. The main motivation behind exposing the swapcache stat is for enabling users to gracefully migrate from cgroup v1's memsw counter to cgroup v2's memory and swap counters. Cgroup v1's memsw limit allows users to limit the memory+swap usage of a workload but without control on the exact proportion of memory and swap. Cgroup v2 provides separate limits for memory and swap which enables more control on the exact usage of memory and swap individually for the workload. With some little subtleties, the v1's memsw limit can be switched with the sum of the v2's memory and swap limits. However the alternative for memsw usage is not yet available in cgroup v2. Exposing per-cgroup swapcache stat enables that alternative. Adding the memory usage and swap usage and subtracting the swapcache will approximate the memsw usage. This will help in the transparent migration of the workloads depending on memsw usage and limit to v2' memory and swap counters. The reasons these applications are still interested in this approximate memsw usage are: (1) these applications are not really interested in two separate memory and swap usage metrics. A single usage metric is more simple to use and reason about for them. (2) The memsw usage metric hides the underlying system's swap setup from the applications. Applications with multiple instances running in a datacenter with heterogeneous systems (some have swap and some don't) will keep seeing a consistent view of their usage. [akpm@linux-foundation.org: fix CONFIG_SWAP=n build] Link: https://lkml.kernel.org/r/20210108155813.2914586-3-shakeelb@google.com Signed-off-by: Shakeel Butt Acked-by: Michal Hocko Reviewed-by: Roman Gushchin Cc: Johannes Weiner Cc: Muchun Song Cc: Yang Shi Signed-off-by: Andrew Morton --- Documentation/admin-guide/cgroup-v2.rst | 4 +++ drivers/base/node.c | 6 ++++ include/linux/mmzone.h | 3 ++ include/linux/swap.h | 6 +++- mm/memcontrol.c | 3 ++ mm/migrate.c | 6 ++++ mm/swap_state.c | 28 +--------------------- mm/vmstat.c | 3 ++ 8 files changed, 32 insertions(+), 27 deletions(-) --- a/Documentation/admin-guide/cgroup-v2.rst~mm-memcg-add-swapcache-stat-for-memcg-v2 +++ a/Documentation/admin-guide/cgroup-v2.rst @@ -1299,6 +1299,10 @@ PAGE_SIZE multiple when read back. Amount of cached filesystem data that was modified and is currently being written back to disk + swapcached + Amount of swap cached in memory. The swapcache is accounted + against both memory and swap usage. + anon_thp Amount of memory used in anonymous mappings backed by transparent hugepages --- a/drivers/base/node.c~mm-memcg-add-swapcache-stat-for-memcg-v2 +++ a/drivers/base/node.c @@ -372,14 +372,19 @@ static ssize_t node_read_meminfo(struct struct pglist_data *pgdat = NODE_DATA(nid); struct sysinfo i; unsigned long sreclaimable, sunreclaimable; + unsigned long swapcached = 0; si_meminfo_node(&i, nid); sreclaimable = node_page_state_pages(pgdat, NR_SLAB_RECLAIMABLE_B); sunreclaimable = node_page_state_pages(pgdat, NR_SLAB_UNRECLAIMABLE_B); +#ifdef CONFIG_SWAP + swapcached = node_page_state_pages(pgdat, NR_SWAPCACHE); +#endif len = sysfs_emit_at(buf, len, "Node %d MemTotal: %8lu kB\n" "Node %d MemFree: %8lu kB\n" "Node %d MemUsed: %8lu kB\n" + "Node %d SwapCached: %8lu kB\n" "Node %d Active: %8lu kB\n" "Node %d Inactive: %8lu kB\n" "Node %d Active(anon): %8lu kB\n" @@ -391,6 +396,7 @@ static ssize_t node_read_meminfo(struct nid, K(i.totalram), nid, K(i.freeram), nid, K(i.totalram - i.freeram), + nid, K(swapcached), nid, K(node_page_state(pgdat, NR_ACTIVE_ANON) + node_page_state(pgdat, NR_ACTIVE_FILE)), nid, K(node_page_state(pgdat, NR_INACTIVE_ANON) + --- a/include/linux/mmzone.h~mm-memcg-add-swapcache-stat-for-memcg-v2 +++ a/include/linux/mmzone.h @@ -206,6 +206,9 @@ enum node_stat_item { NR_KERNEL_SCS_KB, /* measured in KiB */ #endif NR_PAGETABLE, /* used for pagetables */ +#ifdef CONFIG_SWAP + NR_SWAPCACHE, +#endif NR_VM_NODE_STAT_ITEMS }; --- a/include/linux/swap.h~mm-memcg-add-swapcache-stat-for-memcg-v2 +++ a/include/linux/swap.h @@ -408,7 +408,11 @@ extern struct address_space *swapper_spa #define swap_address_space(entry) \ (&swapper_spaces[swp_type(entry)][swp_offset(entry) \ >> SWAP_ADDRESS_SPACE_SHIFT]) -extern unsigned long total_swapcache_pages(void); +static inline unsigned long total_swapcache_pages(void) +{ + return global_node_page_state(NR_SWAPCACHE); +} + extern void show_swap_cache_info(void); extern int add_to_swap(struct page *page); extern void *get_shadow_from_swap_cache(swp_entry_t entry); --- a/mm/memcontrol.c~mm-memcg-add-swapcache-stat-for-memcg-v2 +++ a/mm/memcontrol.c @@ -1521,6 +1521,9 @@ static const struct memory_stat memory_s { "file_mapped", NR_FILE_MAPPED }, { "file_dirty", NR_FILE_DIRTY }, { "file_writeback", NR_WRITEBACK }, +#ifdef CONFIG_SWAP + { "swapcached", NR_SWAPCACHE }, +#endif #ifdef CONFIG_TRANSPARENT_HUGEPAGE { "anon_thp", NR_ANON_THPS }, { "file_thp", NR_FILE_THPS }, --- a/mm/migrate.c~mm-memcg-add-swapcache-stat-for-memcg-v2 +++ a/mm/migrate.c @@ -500,6 +500,12 @@ int migrate_page_move_mapping(struct add __mod_lruvec_state(old_lruvec, NR_SHMEM, -nr); __mod_lruvec_state(new_lruvec, NR_SHMEM, nr); } +#ifdef CONFIG_SWAP + if (PageSwapCache(page)) { + __mod_lruvec_state(old_lruvec, NR_SWAPCACHE, -nr); + __mod_lruvec_state(new_lruvec, NR_SWAPCACHE, nr); + } +#endif if (dirty && mapping_can_writeback(mapping)) { __mod_lruvec_state(old_lruvec, NR_FILE_DIRTY, -nr); __mod_zone_page_state(oldzone, NR_ZONE_WRITE_PENDING, -nr); --- a/mm/swap_state.c~mm-memcg-add-swapcache-stat-for-memcg-v2 +++ a/mm/swap_state.c @@ -68,32 +68,6 @@ static struct { unsigned long find_total; } swap_cache_info; -unsigned long total_swapcache_pages(void) -{ - unsigned int i, j, nr; - unsigned long ret = 0; - struct address_space *spaces; - struct swap_info_struct *si; - - for (i = 0; i < MAX_SWAPFILES; i++) { - swp_entry_t entry = swp_entry(i, 1); - - /* Avoid get_swap_device() to warn for bad swap entry */ - if (!swp_swap_info(entry)) - continue; - /* Prevent swapoff to free swapper_spaces */ - si = get_swap_device(entry); - if (!si) - continue; - nr = nr_swapper_spaces[i]; - spaces = swapper_spaces[i]; - for (j = 0; j < nr; j++) - ret += spaces[j].nrpages; - put_swap_device(si); - } - return ret; -} - static atomic_t swapin_readahead_hits = ATOMIC_INIT(4); void show_swap_cache_info(void) @@ -163,6 +137,7 @@ int add_to_swap_cache(struct page *page, address_space->nrexceptional -= nr_shadows; address_space->nrpages += nr; __mod_node_page_state(page_pgdat(page), NR_FILE_PAGES, nr); + __mod_lruvec_page_state(page, NR_SWAPCACHE, nr); ADD_CACHE_INFO(add_total, nr); unlock: xas_unlock_irq(&xas); @@ -203,6 +178,7 @@ void __delete_from_swap_cache(struct pag address_space->nrexceptional += nr; address_space->nrpages -= nr; __mod_node_page_state(page_pgdat(page), NR_FILE_PAGES, -nr); + __mod_lruvec_page_state(page, NR_SWAPCACHE, -nr); ADD_CACHE_INFO(del_total, nr); } --- a/mm/vmstat.c~mm-memcg-add-swapcache-stat-for-memcg-v2 +++ a/mm/vmstat.c @@ -1215,6 +1215,9 @@ const char * const vmstat_text[] = { "nr_shadow_call_stack", #endif "nr_page_table_pages", +#ifdef CONFIG_SWAP + "nr_swapcached", +#endif /* enum writeback_stat_item counters */ "nr_dirty_threshold", From patchwork Wed Feb 24 20:03:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102547 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78AD0C433E0 for ; Wed, 24 Feb 2021 20:04:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F30C664F09 for ; Wed, 24 Feb 2021 20:04:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F30C664F09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 251936B008C; Wed, 24 Feb 2021 15:04:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1D9D88D0003; Wed, 24 Feb 2021 15:04:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F41808D0001; Wed, 24 Feb 2021 15:04:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0148.hostedemail.com [216.40.44.148]) by kanga.kvack.org (Postfix) with ESMTP id DED136B008C for ; Wed, 24 Feb 2021 15:04:00 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id A4C1EFB62 for ; Wed, 24 Feb 2021 20:04:00 +0000 (UTC) X-FDA: 77854237440.25.3DD9C17 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf16.hostedemail.com (Postfix) with ESMTP id 8CD0380192C0 for ; Wed, 24 Feb 2021 20:03:59 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id E65B264F2F; Wed, 24 Feb 2021 20:03:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197039; bh=QqTMeJTbGJz/dCouHrhTiGyDXHQTRy1K8qh2winai+g=; h=Date:From:To:Subject:In-Reply-To:From; b=ciRXpoESzD+p9ZztBT1g0MSZArxQlGEmnhiU1WjUpvuymDEzfXp2GBdKDFpdRIAVc PnesOG00K0r1DdCe4XvlhPuSR37iEMd0kFNL/jBiMzjZtBw6ewhXSkZknFGJiSx3Q0 EMAZ2w8NAsiQRoCyD86lm2n8V8drO1ZD+tAM+cz4= Date: Wed, 24 Feb 2021 12:03:58 -0800 From: Andrew Morton To: akpm@linux-foundation.org, guro@fb.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 067/173] mm: kmem: make __memcg_kmem_(un)charge static Message-ID: <20210224200358.hE1s6mymp%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 4t1s37dnfspo13tqn7izbb9eutabsmuk X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 8CD0380192C0 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197039-196946 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Roman Gushchin Subject: mm: kmem: make __memcg_kmem_(un)charge static I've noticed that __memcg_kmem_charge() and __memcg_kmem_uncharge() are not used anywhere except memcontrol.c. Yet they are not declared as non-static and are declared in memcontrol.h. This patch makes them static. Link: https://lkml.kernel.org/r/20210108020332.4096911-1-guro@fb.com Signed-off-by: Roman Gushchin Reviewed-by: Shakeel Butt Signed-off-by: Andrew Morton --- include/linux/memcontrol.h | 3 --- mm/memcontrol.c | 11 ++++++++--- 2 files changed, 8 insertions(+), 6 deletions(-) --- a/include/linux/memcontrol.h~mm-kmem-make-__memcg_kmem_uncharge-static +++ a/include/linux/memcontrol.h @@ -1592,9 +1592,6 @@ static inline void memcg_set_shrinker_bi #endif #ifdef CONFIG_MEMCG_KMEM -int __memcg_kmem_charge(struct mem_cgroup *memcg, gfp_t gfp, - unsigned int nr_pages); -void __memcg_kmem_uncharge(struct mem_cgroup *memcg, unsigned int nr_pages); int __memcg_kmem_charge_page(struct page *page, gfp_t gfp, int order); void __memcg_kmem_uncharge_page(struct page *page, int order); --- a/mm/memcontrol.c~mm-kmem-make-__memcg_kmem_uncharge-static +++ a/mm/memcontrol.c @@ -255,6 +255,11 @@ struct cgroup_subsys_state *vmpressure_t #ifdef CONFIG_MEMCG_KMEM extern spinlock_t css_set_lock; +static int __memcg_kmem_charge(struct mem_cgroup *memcg, gfp_t gfp, + unsigned int nr_pages); +static void __memcg_kmem_uncharge(struct mem_cgroup *memcg, + unsigned int nr_pages); + static void obj_cgroup_release(struct percpu_ref *ref) { struct obj_cgroup *objcg = container_of(ref, struct obj_cgroup, refcnt); @@ -3087,8 +3092,8 @@ static void memcg_free_cache_id(int id) * * Returns 0 on success, an error code on failure. */ -int __memcg_kmem_charge(struct mem_cgroup *memcg, gfp_t gfp, - unsigned int nr_pages) +static int __memcg_kmem_charge(struct mem_cgroup *memcg, gfp_t gfp, + unsigned int nr_pages) { struct page_counter *counter; int ret; @@ -3120,7 +3125,7 @@ int __memcg_kmem_charge(struct mem_cgrou * @memcg: memcg to uncharge * @nr_pages: number of pages to uncharge */ -void __memcg_kmem_uncharge(struct mem_cgroup *memcg, unsigned int nr_pages) +static void __memcg_kmem_uncharge(struct mem_cgroup *memcg, unsigned int nr_pages) { if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) page_counter_uncharge(&memcg->kmem, nr_pages); From patchwork Wed Feb 24 20:04:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102549 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBB55C433E0 for ; Wed, 24 Feb 2021 20:04:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7695A64F28 for ; Wed, 24 Feb 2021 20:04:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7695A64F28 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AFF206B0095; Wed, 24 Feb 2021 15:04:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A65018D0003; Wed, 24 Feb 2021 15:04:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 901DB8D0001; Wed, 24 Feb 2021 15:04:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0097.hostedemail.com [216.40.44.97]) by kanga.kvack.org (Postfix) with ESMTP id 799686B0095 for ; Wed, 24 Feb 2021 15:04:04 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 3CBF9F042 for ; Wed, 24 Feb 2021 20:04:04 +0000 (UTC) X-FDA: 77854237608.08.FC29C3C Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf08.hostedemail.com (Postfix) with ESMTP id 3D3CC80192E5 for ; Wed, 24 Feb 2021 20:03:52 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 57D0764F0C; Wed, 24 Feb 2021 20:04:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197042; bh=ChBiOWWFlV55hgQd9As/5MlsMtGmosNljD20kyHR1Xk=; h=Date:From:To:Subject:In-Reply-To:From; b=nHBNmRJufE8X4VcOw5Ym8jxe0xMIi9KpOvhRa4xmDJcZzlugHBti2tMPRchLF3Pat 9SscKTlPvoVeU+KecH2Y/Esl3T7VMABxEIigV1mKIT3QTAWtsFie0FYttGJNWQ2++I D/+WUTjvJ69+TeIBOUH8DgXAUidc1YiHpQK2iNRk= Date: Wed, 24 Feb 2021 12:04:01 -0800 From: Andrew Morton To: akpm@linux-foundation.org, feng.tang@intel.com, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org Subject: [patch 068/173] mm: page_counter: re-layout structure to reduce false sharing Message-ID: <20210224200401._Hfko2JIY%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 MIME-Version: 1.0 X-Stat-Signature: 9mec4jfk7caagxpy7pdaewn65z8xf8t9 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 3D3CC80192E5 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf08; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197032-58415 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Feng Tang Subject: mm: page_counter: re-layout structure to reduce false sharing When checking a memory cgroup related performance regression [1], from the perf c2c profiling data, we found high false sharing for accessing 'usage' and 'parent'. On 64 bit system, the 'usage' and 'parent' are close to each other, and easy to be in one cacheline (for cacheline size == 64+ B). 'usage' is usally written, while 'parent' is usually read as the cgroup's hierarchical counting nature. So move the 'parent' to the end of the structure to make sure they are in different cache lines. Following are some performance data with the patch, against v5.11-rc1. [ In the data, A means a platform with 2 sockets 48C/96T, B is a platform of 4 sockests 72C/144T, and if a %stddev will be shown bigger than 2%, P100/P50 means number of test tasks equals to 100%/50% of nr_cpu] will-it-scale/malloc1 --------------------- v5.11-rc1 v5.11-rc1+patch A-P100 15782 ± 2% -0.1% 15765 ± 3% will-it-scale.per_process_ops A-P50 21511 +8.9% 23432 will-it-scale.per_process_ops B-P100 9155 +2.2% 9357 will-it-scale.per_process_ops B-P50 10967 +7.1% 11751 ± 2% will-it-scale.per_process_ops will-it-scale/pagefault2 ------------------------ v5.11-rc1 v5.11-rc1+patch A-P100 79028 +3.0% 81411 will-it-scale.per_process_ops A-P50 183960 ± 2% +4.4% 192078 ± 2% will-it-scale.per_process_ops B-P100 85966 +9.9% 94467 ± 3% will-it-scale.per_process_ops B-P50 198195 +9.8% 217526 will-it-scale.per_process_ops fio (4k/1M is block size) ------------------------- v5.11-rc1 v5.11-rc1+patch A-P50-r-4k 16881 ± 2% +1.2% 17081 ± 2% fio.read_bw_MBps A-P50-w-4k 3931 +4.5% 4111 ± 2% fio.write_bw_MBps A-P50-r-1M 15178 -0.2% 15154 fio.read_bw_MBps A-P50-w-1M 3924 +0.1% 3929 fio.write_bw_MBps [1].https://lore.kernel.org/lkml/20201102091543.GM31092@shao2-debian/ Link: https://lkml.kernel.org/r/1611040814-33449-1-git-send-email-feng.tang@intel.com Signed-off-by: Feng Tang Reviewed-by: Roman Gushchin Reviewed-by: Shakeel Butt Acked-by: Johannes Weiner Acked-by: Michal Hocko Signed-off-by: Andrew Morton --- include/linux/page_counter.h | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) --- a/include/linux/page_counter.h~mm-page_counter-relayout-structure-to-reduce-false-sharing +++ a/include/linux/page_counter.h @@ -12,7 +12,6 @@ struct page_counter { unsigned long low; unsigned long high; unsigned long max; - struct page_counter *parent; /* effective memory.min and memory.min usage tracking */ unsigned long emin; @@ -27,6 +26,14 @@ struct page_counter { /* legacy */ unsigned long watermark; unsigned long failcnt; + + /* + * 'parent' is placed here to be far from 'usage' to reduce + * cache false sharing, as 'usage' is written mostly while + * parent is frequently read for cgroup's hierarchical + * counting nature. + */ + struct page_counter *parent; }; #if BITS_PER_LONG == 32 From patchwork Wed Feb 24 20:04:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102551 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA251C433DB for ; Wed, 24 Feb 2021 20:04:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 62E2E64F09 for ; Wed, 24 Feb 2021 20:04:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 62E2E64F09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id ECF478D0001; Wed, 24 Feb 2021 15:04:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E803A6B0099; Wed, 24 Feb 2021 15:04:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D4C238D0001; Wed, 24 Feb 2021 15:04:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0116.hostedemail.com [216.40.44.116]) by kanga.kvack.org (Postfix) with ESMTP id BCDD06B0098 for ; Wed, 24 Feb 2021 15:04:07 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 865E9FB63 for ; Wed, 24 Feb 2021 20:04:07 +0000 (UTC) X-FDA: 77854237734.23.BEEFA4D Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf01.hostedemail.com (Postfix) with ESMTP id 75C642000382 for ; Wed, 24 Feb 2021 20:04:06 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id CBBF264F2B; Wed, 24 Feb 2021 20:04:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197046; bh=AW6VtjcalHNtIglQEhdtRWcN5XijATaQZwWBHKishGU=; h=Date:From:To:Subject:In-Reply-To:From; b=WOwYQg0f0aShgkmdreL9Hx32w6yw0L7F8+ZyE5kQ4jXvQweMy+p1zHpAGYTSv/S3e 2b0FqLI898ujScdYYPTdc8AvX2W4wXiCLr0JBnyCmrHgxyUQvb3dKIrSHsSkQg5iXO 1KMwRIAqWTbiOya19RwIMc0QfF+QX1SjfFQ2P1SM= Date: Wed, 24 Feb 2021 12:04:05 -0800 From: Andrew Morton To: abaci-bugfix@linux.alibaba.com, abaci@linux.alibaba.com, akpm@linux-foundation.org, david@redhat.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 069/173] mm/memcontrol: remove redundant NULL check Message-ID: <20210224200405.jNrV0hE5t%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: qqo5q5buz1tzity6i8sqpmjben11nt4i X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 75C642000382 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf01; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197046-474278 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Li Subject: mm/memcontrol: remove redundant NULL check Fix below warnings reported by coccicheck: ./mm/memcontrol.c:451:3-9: WARNING: NULL check before some freeing functions is not needed. Link: https://lkml.kernel.org/r/1611216029-34397-1-git-send-email-abaci-bugfix@linux.alibaba.com Signed-off-by: Yang Li Reported-by: Abaci Robot Reviewed-by: Andrew Morton Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- mm/memcontrol.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) --- a/mm/memcontrol.c~mm-memcontrol-remove-redundant-null-check +++ a/mm/memcontrol.c @@ -452,8 +452,7 @@ static void memcg_free_shrinker_maps(str for_each_node(nid) { pn = mem_cgroup_nodeinfo(memcg, nid); map = rcu_dereference_protected(pn->shrinker_map, true); - if (map) - kvfree(map); + kvfree(map); rcu_assign_pointer(pn->shrinker_map, NULL); } } From patchwork Wed Feb 24 20:04:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102553 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C180C433E9 for ; Wed, 24 Feb 2021 20:04:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BDAE564F09 for ; Wed, 24 Feb 2021 20:04:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BDAE564F09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 668B48D0003; Wed, 24 Feb 2021 15:04:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 641976B009A; Wed, 24 Feb 2021 15:04:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 55B648D0003; Wed, 24 Feb 2021 15:04:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0208.hostedemail.com [216.40.44.208]) by kanga.kvack.org (Postfix) with ESMTP id 3AB086B0099 for ; Wed, 24 Feb 2021 15:04:11 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 01FEF1801E8D1 for ; Wed, 24 Feb 2021 20:04:11 +0000 (UTC) X-FDA: 77854237902.10.D7BD4E0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf06.hostedemail.com (Postfix) with ESMTP id 0334FC0007EC for ; Wed, 24 Feb 2021 20:04:11 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 2AF9364F0F; Wed, 24 Feb 2021 20:04:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197049; bh=5c8za154E5ZZs1Pe0YPwFgF4ghXJ/O1bpaJjs3eUP5A=; h=Date:From:To:Subject:In-Reply-To:From; b=jxs6EAW4b50ev3OUu2zJH/HgrNQ2gHhIk6ZP92Df0amnoJfoACQC23RwLJAHI1/Fs KtGiuPbRk8HJtCnW90z+rzdWd8Kpt/j8OpI6duke/Ckny7ksTo3N4K+WBchFdhmt2b 7r5nP1b3THPM3huKc4uhmu5HGf0F72m0Qu35GMSA= Date: Wed, 24 Feb 2021 12:04:08 -0800 From: Andrew Morton To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org, linmiaohe@huawei.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, songmuchun@bytedance.com, torvalds@linux-foundation.org Subject: [patch 070/173] mm: memcontrol: replace the loop with a list_for_each_entry() Message-ID: <20210224200408.tuzvi2hBn%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 0334FC0007EC X-Stat-Signature: zbtf61drour4h3e6pq5x4o4adgurrqwi Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf06; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197051-901181 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: memcontrol: replace the loop with a list_for_each_entry() The rule of list walk has gone since: commit a9d5adeeb4b2 ("mm/memcontrol: allow to uncharge page without using page->lru field") So remove the strange comment and replace the loop with a list_for_each_entry(). There is only one caller of the uncharge_list(). So just fold it into mem_cgroup_uncharge_list() and remove it. Link: https://lkml.kernel.org/r/20210204163055.56080-1-songmuchun@bytedance.com Signed-off-by: Muchun Song Acked-by: Johannes Weiner Acked-by: Roman Gushchin Reviewed-by: Miaohe Lin Acked-by: Michal Hocko Signed-off-by: Andrew Morton --- mm/memcontrol.c | 35 ++++++++--------------------------- 1 file changed, 8 insertions(+), 27 deletions(-) --- a/mm/memcontrol.c~mm-memcontrol-replace-the-loop-with-a-list_for_each_entry +++ a/mm/memcontrol.c @@ -6862,31 +6862,6 @@ static void uncharge_page(struct page *p css_put(&ug->memcg->css); } -static void uncharge_list(struct list_head *page_list) -{ - struct uncharge_gather ug; - struct list_head *next; - - uncharge_gather_clear(&ug); - - /* - * Note that the list can be a single page->lru; hence the - * do-while loop instead of a simple list_for_each_entry(). - */ - next = page_list->next; - do { - struct page *page; - - page = list_entry(next, struct page, lru); - next = page->lru.next; - - uncharge_page(page, &ug); - } while (next != page_list); - - if (ug.memcg) - uncharge_batch(&ug); -} - /** * mem_cgroup_uncharge - uncharge a page * @page: page to uncharge @@ -6918,11 +6893,17 @@ void mem_cgroup_uncharge(struct page *pa */ void mem_cgroup_uncharge_list(struct list_head *page_list) { + struct uncharge_gather ug; + struct page *page; + if (mem_cgroup_disabled()) return; - if (!list_empty(page_list)) - uncharge_list(page_list); + uncharge_gather_clear(&ug); + list_for_each_entry(page, page_list, lru) + uncharge_page(page, &ug); + if (ug.memcg) + uncharge_batch(&ug); } /** From patchwork Wed Feb 24 20:04:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102555 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2CBEC433DB for ; Wed, 24 Feb 2021 20:04:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4117664F08 for ; Wed, 24 Feb 2021 20:04:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4117664F08 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CF8C48D0005; Wed, 24 Feb 2021 15:04:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CAA5A6B009B; Wed, 24 Feb 2021 15:04:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE78D8D0005; Wed, 24 Feb 2021 15:04:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0185.hostedemail.com [216.40.44.185]) by kanga.kvack.org (Postfix) with ESMTP id A63B16B009A for ; Wed, 24 Feb 2021 15:04:14 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5B50A1801E8D1 for ; Wed, 24 Feb 2021 20:04:14 +0000 (UTC) X-FDA: 77854238028.15.04D19E3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP id 4D5E8A0000FD for ; Wed, 24 Feb 2021 20:04:13 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A022964F09; Wed, 24 Feb 2021 20:04:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197053; bh=rOl2K5A6xz5E0IsjoSbi2nB1NiYRwOQUTHn35o9C8zo=; h=Date:From:To:Subject:In-Reply-To:From; b=WA0lklzXX+iW3uL9sFIliMjch6fpqyYriAG10Tsw0iMSt71BD4a97YNXBKtQXNEZT jdNz7VgHniVnZ5nPF/pYRuybtHguTYMBCjhUaJRk0IkuYTdKM6JIBfcS5QVECSMS69 2/7xTCwP6PLcBldJgo4zj3qbj4Okb5CGUxtxwCyw= Date: Wed, 24 Feb 2021 12:04:12 -0800 From: Andrew Morton To: akpm@linux-foundation.org, ktkhai@virtuozzo.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, shakeelb@google.com, torvalds@linux-foundation.org, urezki@gmail.com Subject: [patch 071/173] mm/list_lru.c: remove kvfree_rcu_local() Message-ID: <20210224200412.eSiorwyiz%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 4D5E8A0000FD X-Stat-Signature: 16i4cbgtw5iwggt9bxe3tqo6w4a8zjkh Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf07; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197053-388372 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Shakeel Butt Subject: mm/list_lru.c: remove kvfree_rcu_local() The list_lru file used to have local kvfree_rcu() which was renamed by commit e0feed08ab41 ("mm/list_lru.c: Rename kvfree_rcu() to local variant") to introduce the globally visible kvfree_rcu(). Now we have global kvfree_rcu(), so remove the local kvfree_rcu_local() and just use the global one. Link: https://lkml.kernel.org/r/20210207152148.1285842-1-shakeelb@google.com Signed-off-by: Shakeel Butt Reviewed-by: Uladzislau Rezki Reviewed-by: Kirill Tkhai Signed-off-by: Andrew Morton --- mm/list_lru.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) --- a/mm/list_lru.c~mm-list_lruc-remove-kvfree_rcu_local +++ a/mm/list_lru.c @@ -373,21 +373,13 @@ static void memcg_destroy_list_lru_node( struct list_lru_memcg *memcg_lrus; /* * This is called when shrinker has already been unregistered, - * and nobody can use it. So, there is no need to use kvfree_rcu_local(). + * and nobody can use it. So, there is no need to use kvfree_rcu(). */ memcg_lrus = rcu_dereference_protected(nlru->memcg_lrus, true); __memcg_destroy_list_lru_node(memcg_lrus, 0, memcg_nr_cache_ids); kvfree(memcg_lrus); } -static void kvfree_rcu_local(struct rcu_head *head) -{ - struct list_lru_memcg *mlru; - - mlru = container_of(head, struct list_lru_memcg, rcu); - kvfree(mlru); -} - static int memcg_update_list_lru_node(struct list_lru_node *nlru, int old_size, int new_size) { @@ -419,7 +411,7 @@ static int memcg_update_list_lru_node(st rcu_assign_pointer(nlru->memcg_lrus, new); spin_unlock_irq(&nlru->lock); - call_rcu(&old->rcu, kvfree_rcu_local); + kvfree_rcu(old, rcu); return 0; } From patchwork Wed Feb 24 20:04:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102557 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43FE3C433DB for ; Wed, 24 Feb 2021 20:04:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C4B2064F14 for ; Wed, 24 Feb 2021 20:04:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C4B2064F14 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 52F716B009B; Wed, 24 Feb 2021 15:04:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E14A8D0007; Wed, 24 Feb 2021 15:04:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F7CF8D0006; Wed, 24 Feb 2021 15:04:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0212.hostedemail.com [216.40.44.212]) by kanga.kvack.org (Postfix) with ESMTP id 2A7456B009B for ; Wed, 24 Feb 2021 15:04:18 -0500 (EST) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id E8862182861F7 for ; Wed, 24 Feb 2021 20:04:17 +0000 (UTC) X-FDA: 77854238154.15.927795C Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP id 49813C0007E9 for ; Wed, 24 Feb 2021 20:04:14 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0A5B364F08; Wed, 24 Feb 2021 20:04:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197056; bh=qpY4DySb6Qdg0r072i2zKud5TA2ga17Nkt0ocrLYpx4=; h=Date:From:To:Subject:In-Reply-To:From; b=PosR8Gnp70bvAUnt3yMyiJSKJ7/KDjCL+wpzAfG7mzl13WMpvDlK2XKh5k5Oa55Fj y7f+hCESUIeE+Rr1P9rClfevjxPgXMlw4ONI08oqV/1Jix3ZQ889mynw2d7MsStHiP hznNLx0MPNvRRtp8+M7mhOSdznGfP5sEqL8SPi4A= Date: Wed, 24 Feb 2021 12:04:15 -0800 From: Andrew Morton To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, shakeelb@google.com, songmuchun@bytedance.com, torvalds@linux-foundation.org Subject: [patch 072/173] fs: buffer: use raw page_memcg() on locked page Message-ID: <20210224200415.uBhXwogkS%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 49813C0007E9 X-Stat-Signature: csq69xaprsdjm7m16cgjxzi97jekphoj Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf03; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197054-528754 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Subject: fs: buffer: use raw page_memcg() on locked page alloc_page_buffers() currently uses get_mem_cgroup_from_page() for charging the buffers to the page owner, which does an rcu-protected page->memcg lookup and acquires a reference. But buffer allocation has the page lock held throughout, which pins the page to the memcg and thereby the memcg - neither rcu nor holding an extra reference during the allocation are necessary. Use a raw page_memcg() instead. This was the last user of get_mem_cgroup_from_page(), delete it. Link: https://lkml.kernel.org/r/20210209190126.97842-1-hannes@cmpxchg.org Signed-off-by: Johannes Weiner Reported-by: Muchun Song Reviewed-by: Shakeel Butt Acked-by: Roman Gushchin Acked-by: Michal Hocko Signed-off-by: Andrew Morton --- fs/buffer.c | 4 ++-- include/linux/memcontrol.h | 7 ------- mm/memcontrol.c | 23 ----------------------- 3 files changed, 2 insertions(+), 32 deletions(-) --- a/fs/buffer.c~fs-buffer-use-raw-page_memcg-on-locked-page +++ a/fs/buffer.c @@ -847,7 +847,8 @@ struct buffer_head *alloc_page_buffers(s if (retry) gfp |= __GFP_NOFAIL; - memcg = get_mem_cgroup_from_page(page); + /* The page lock pins the memcg */ + memcg = page_memcg(page); old_memcg = set_active_memcg(memcg); head = NULL; @@ -868,7 +869,6 @@ struct buffer_head *alloc_page_buffers(s } out: set_active_memcg(old_memcg); - mem_cgroup_put(memcg); return head; /* * In case anything failed, we just free everything we got. --- a/include/linux/memcontrol.h~fs-buffer-use-raw-page_memcg-on-locked-page +++ a/include/linux/memcontrol.h @@ -680,8 +680,6 @@ struct mem_cgroup *mem_cgroup_from_task( struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm); -struct mem_cgroup *get_mem_cgroup_from_page(struct page *page); - struct lruvec *lock_page_lruvec(struct page *page); struct lruvec *lock_page_lruvec_irq(struct page *page); struct lruvec *lock_page_lruvec_irqsave(struct page *page, @@ -1190,11 +1188,6 @@ static inline struct mem_cgroup *get_mem { return NULL; } - -static inline struct mem_cgroup *get_mem_cgroup_from_page(struct page *page) -{ - return NULL; -} static inline void mem_cgroup_put(struct mem_cgroup *memcg) { --- a/mm/memcontrol.c~fs-buffer-use-raw-page_memcg-on-locked-page +++ a/mm/memcontrol.c @@ -1047,29 +1047,6 @@ struct mem_cgroup *get_mem_cgroup_from_m } EXPORT_SYMBOL(get_mem_cgroup_from_mm); -/** - * get_mem_cgroup_from_page: Obtain a reference on given page's memcg. - * @page: page from which memcg should be extracted. - * - * Obtain a reference on page->memcg and returns it if successful. Otherwise - * root_mem_cgroup is returned. - */ -struct mem_cgroup *get_mem_cgroup_from_page(struct page *page) -{ - struct mem_cgroup *memcg = page_memcg(page); - - if (mem_cgroup_disabled()) - return NULL; - - rcu_read_lock(); - /* Page should not get uncharged and freed memcg under us. */ - if (!memcg || WARN_ON_ONCE(!css_tryget(&memcg->css))) - memcg = root_mem_cgroup; - rcu_read_unlock(); - return memcg; -} -EXPORT_SYMBOL(get_mem_cgroup_from_page); - static __always_inline struct mem_cgroup *active_memcg(void) { if (in_interrupt()) From patchwork Wed Feb 24 20:04:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9899C433DB for ; Wed, 24 Feb 2021 20:04:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 83DFB64F11 for ; Wed, 24 Feb 2021 20:04:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 83DFB64F11 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0C3566B009D; Wed, 24 Feb 2021 15:04:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0758B8D0007; Wed, 24 Feb 2021 15:04:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF4418D0006; Wed, 24 Feb 2021 15:04:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0215.hostedemail.com [216.40.44.215]) by kanga.kvack.org (Postfix) with ESMTP id DAA0C6B009D for ; Wed, 24 Feb 2021 15:04:21 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id A16A3182CA8FD for ; Wed, 24 Feb 2021 20:04:21 +0000 (UTC) X-FDA: 77854238322.19.D23CE5D Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf10.hostedemail.com (Postfix) with ESMTP id DC95F4080F4A for ; Wed, 24 Feb 2021 20:04:15 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 99A5B64F32; Wed, 24 Feb 2021 20:04:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197060; bh=b8z0jFYGhZJXgwfpm3qd1Qnj3LqjaHi5HU4vzHcmA/4=; h=Date:From:To:Subject:In-Reply-To:From; b=IKlOtRgJqR4hrwoFFiTiea8tUcTonCvTXKqzlR36HWl3pVc8oQDuxnFuarMG5wrVx wCvh/VP1XcTMCad9gSOmfnn0NJ1WjoKc3RviocJGf3x+S2R+EE2aT5PSjM+Z2yeXdh UvkGKUmMia0KYZKfwznkgiSKFRYtdqibrnJAdbXI= Date: Wed, 24 Feb 2021 12:04:19 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, shakeelb@google.com, songmuchun@bytedance.com, stable@vger.kernel.org, torvalds@linux-foundation.org, vdavydov.dev@gmail.com Subject: [patch 073/173] mm: memcontrol: fix swap undercounting in cgroup2 Message-ID: <20210224200419.SKAB6Vl7M%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 43c8s3ac1raeut4hx4tebiqseubykjya X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: DC95F4080F4A Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf10; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197055-308186 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: memcontrol: fix swap undercounting in cgroup2 When pages are swapped in, the VM may retain the swap copy to avoid repeated writes in the future. It's also retained if shared pages are faulted back in some processes, but not in others. During that time we have an in-memory copy of the page, as well as an on-swap copy. Cgroup1 and cgroup2 handle these overlapping lifetimes slightly differently due to the nature of how they account memory and swap: Cgroup1 has a unified memory+swap counter that tracks a data page regardless whether it's in-core or swapped out. On swapin, we transfer the charge from the swap entry to the newly allocated swapcache page, even though the swap entry might stick around for a while. That's why we have a mem_cgroup_uncharge_swap() call inside mem_cgroup_charge(). Cgroup2 tracks memory and swap as separate, independent resources and thus has split memory and swap counters. On swapin, we charge the newly allocated swapcache page as memory, while the swap slot in turn must remain charged to the swap counter as long as its allocated too. The cgroup2 logic was broken by commit 2d1c498072de ("mm: memcontrol: make swap tracking an integral part of memory control"), because it accidentally removed the do_memsw_account() check in the branch inside mem_cgroup_uncharge() that was supposed to tell the difference between the charge transfer in cgroup1 and the separate counters in cgroup2. As a result, cgroup2 currently undercounts retained swap to varying degrees: swap slots are cached up to 50% of the configured limit or total available swap space; partially faulted back shared pages are only limited by physical capacity. This in turn allows cgroups to significantly overconsume their alloted swap space. Add the do_memsw_account() check back to fix this problem. Link: https://lkml.kernel.org/r/20210217153237.92484-1-songmuchun@bytedance.com Fixes: 2d1c498072de ("mm: memcontrol: make swap tracking an integral part of memory control") Signed-off-by: Muchun Song Acked-by: Johannes Weiner Reviewed-by: Shakeel Butt Acked-by: Michal Hocko Cc: Vladimir Davydov Cc: [5.8+] Signed-off-by: Andrew Morton --- mm/memcontrol.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) --- a/mm/memcontrol.c~mm-memcontrol-fix-swap-undercounting-in-cgroup2 +++ a/mm/memcontrol.c @@ -6748,7 +6748,19 @@ int mem_cgroup_charge(struct page *page, memcg_check_events(memcg, page); local_irq_enable(); - if (PageSwapCache(page)) { + /* + * Cgroup1's unified memory+swap counter has been charged with the + * new swapcache page, finish the transfer by uncharging the swap + * slot. The swap slot would also get uncharged when it dies, but + * it can stick around indefinitely and we'd count the page twice + * the entire time. + * + * Cgroup2 has separate resource counters for memory and swap, + * so this is a non-issue here. Memory and swap charge lifetimes + * correspond 1:1 to page and swap slot lifetimes: we charge the + * page to memory here, and uncharge swap when the slot is freed. + */ + if (do_memsw_account() && PageSwapCache(page)) { swp_entry_t entry = { .val = page_private(page) }; /* * The swap entry might not get freed for a long time, From patchwork Wed Feb 24 20:04:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97D10C433DB for ; Wed, 24 Feb 2021 20:04:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2E30B64F28 for ; Wed, 24 Feb 2021 20:04:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2E30B64F28 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BC0D88D0006; Wed, 24 Feb 2021 15:04:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B738C6B00A0; Wed, 24 Feb 2021 15:04:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AAC6E8D0006; Wed, 24 Feb 2021 15:04:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0203.hostedemail.com [216.40.44.203]) by kanga.kvack.org (Postfix) with ESMTP id 94AC06B009F for ; Wed, 24 Feb 2021 15:04:25 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 3F087FB45 for ; Wed, 24 Feb 2021 20:04:25 +0000 (UTC) X-FDA: 77854238490.29.EAAB2C7 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf06.hostedemail.com (Postfix) with ESMTP id 56EDFC0007DF for ; Wed, 24 Feb 2021 20:04:26 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 31F4764F11; Wed, 24 Feb 2021 20:04:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197063; bh=G31EBYnGYWA+9ziOs13ui7hLFTA9Ds66VuzLUR1+Iv4=; h=Date:From:To:Subject:In-Reply-To:From; b=si5bUio5hFqsEjKRig7n4vmR60gzC42liNYSxQ6OQc3feQk003GyrD0TmiSURD3DK NqjymUnrfzPZHB1usvHOdEf13vscSZfoeV6hQINEKMuno0e5H0/KG4D1aFDvMsr/Te MmT+prNuBlA4QG+t3+FVRVtGntzQenx+rTuRsyWI= Date: Wed, 24 Feb 2021 12:04:22 -0800 From: Andrew Morton To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, shakeelb@google.com, songmuchun@bytedance.com, stable@vger.kernel.org, torvalds@linux-foundation.org, vdavydov.dev@gmail.com Subject: [patch 074/173] mm: memcontrol: fix get_active_memcg return value Message-ID: <20210224200422.mMBuJs4L_%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 77twxhejskxhpf6gfhda3p4jednb15go X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 56EDFC0007DF Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf06; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197066-197917 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: memcontrol: fix get_active_memcg return value We use a global percpu int_active_memcg variable to store the remote memcg when we are in the interrupt context. But get_active_memcg always return the current->active_memcg or root_mem_cgroup. The remote memcg (set in the interrupt context) is ignored. This is not what we want. So fix it. Link: https://lkml.kernel.org/r/20210223091101.42150-1-songmuchun@bytedance.com Fixes: 37d5985c003d ("mm: kmem: prepare remote memcg charging infra for interrupt contexts") Signed-off-by: Muchun Song Reviewed-by: Shakeel Butt Reviewed-by: Roman Gushchin Cc: Johannes Weiner Cc: Michal Hocko Cc: Vladimir Davydov Cc: Signed-off-by: Andrew Morton --- mm/memcontrol.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) --- a/mm/memcontrol.c~mm-memcontrol-fix-get_active_memcg-return-value +++ a/mm/memcontrol.c @@ -1061,13 +1061,9 @@ static __always_inline struct mem_cgroup rcu_read_lock(); memcg = active_memcg(); - if (memcg) { - /* current->active_memcg must hold a ref. */ - if (WARN_ON_ONCE(!css_tryget(&memcg->css))) - memcg = root_mem_cgroup; - else - memcg = current->active_memcg; - } + /* remote memcg must hold a ref. */ + if (memcg && WARN_ON_ONCE(!css_tryget(&memcg->css))) + memcg = root_mem_cgroup; rcu_read_unlock(); return memcg; From patchwork Wed Feb 24 20:04:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102563 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5917C433E0 for ; Wed, 24 Feb 2021 20:04:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7B84764F14 for ; Wed, 24 Feb 2021 20:04:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7B84764F14 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 113B38D0007; Wed, 24 Feb 2021 15:04:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0E9656B00A1; Wed, 24 Feb 2021 15:04:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F40FB8D0007; Wed, 24 Feb 2021 15:04:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0160.hostedemail.com [216.40.44.160]) by kanga.kvack.org (Postfix) with ESMTP id DFBA96B00A0 for ; Wed, 24 Feb 2021 15:04:28 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id A90C3184B40D3 for ; Wed, 24 Feb 2021 20:04:28 +0000 (UTC) X-FDA: 77854238616.24.A617E05 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf26.hostedemail.com (Postfix) with ESMTP id BE0314080F52 for ; Wed, 24 Feb 2021 20:04:23 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id CC04464F34; Wed, 24 Feb 2021 20:04:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197067; bh=ZM3i/IwR0/R6XZBVEpCe3w7JVAyigaLJFGrhMVLQt40=; h=Date:From:To:Subject:In-Reply-To:From; b=Y7O7f3i2GzIa4txrrDQj0QZunST22mFHG5bZbVk4fdMh/TfeUej+BBGkSklhh2f0+ sjCR+0a15xfOCbFFZA4pYeAPlFuzNVXQ7IasxXDcCn/IIy6REhPqPNdd3DSgHE7uZ2 zW4Prw9QwP8SUEVOI4ykLJls1yFFzQEJXllI0MZA= Date: Wed, 24 Feb 2021 12:04:26 -0800 From: Andrew Morton To: akpm@linux-foundation.org, guro@fb.com, hannes@cmpxchg.org, linux-mm@kvack.org, mhocko@kernel.org, mkoutny@suse.com, mm-commits@vger.kernel.org, shakeelb@google.com, songmuchun@bytedance.com, torvalds@linux-foundation.org, vdavydov.dev@gmail.com Subject: [patch 075/173] mm: memcontrol: fix slub memory accounting Message-ID: <20210224200426.vEeBLOjbH%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 MIME-Version: 1.0 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: BE0314080F52 X-Stat-Signature: wajxmrfuu1ytzc7crh3x55mzhoqtic8m Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf26; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197063-901726 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Muchun Song Subject: mm: memcontrol: fix slub memory accounting SLUB currently account kmalloc() and kmalloc_node() allocations larger than order-1 page per-node. But it forget to update the per-memcg vmstats. So it can lead to inaccurate statistics of "slab_unreclaimable" which is from memory.stat. Fix it by using mod_lruvec_page_state instead of mod_node_page_state. Link: https://lkml.kernel.org/r/20210223092423.42420-1-songmuchun@bytedance.com Fixes: 6a486c0ad4dc ("mm, sl[ou]b: improve memory accounting") Signed-off-by: Muchun Song Reviewed-by: Shakeel Butt Reviewed-by: Roman Gushchin Reviewed-by: Michal Koutný Cc: Johannes Weiner Cc: Michal Hocko Cc: Vladimir Davydov Signed-off-by: Andrew Morton --- mm/slab_common.c | 4 ++-- mm/slub.c | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-) --- a/mm/slab_common.c~mm-memcontrol-fix-slub-memory-accounting +++ a/mm/slab_common.c @@ -898,8 +898,8 @@ void *kmalloc_order(size_t size, gfp_t f page = alloc_pages(flags, order); if (likely(page)) { ret = page_address(page); - mod_node_page_state(page_pgdat(page), NR_SLAB_UNRECLAIMABLE_B, - PAGE_SIZE << order); + mod_lruvec_page_state(page, NR_SLAB_UNRECLAIMABLE_B, + PAGE_SIZE << order); } ret = kasan_kmalloc_large(ret, size, flags); /* As ret might get tagged, call kmemleak hook after KASAN. */ --- a/mm/slub.c~mm-memcontrol-fix-slub-memory-accounting +++ a/mm/slub.c @@ -4042,8 +4042,8 @@ static void *kmalloc_large_node(size_t s page = alloc_pages_node(node, flags, order); if (page) { ptr = page_address(page); - mod_node_page_state(page_pgdat(page), NR_SLAB_UNRECLAIMABLE_B, - PAGE_SIZE << order); + mod_lruvec_page_state(page, NR_SLAB_UNRECLAIMABLE_B, + PAGE_SIZE << order); } return kmalloc_large_node_hook(ptr, size, flags); @@ -4174,8 +4174,8 @@ void kfree(const void *x) BUG_ON(!PageCompound(page)); kfree_hook(object); - mod_node_page_state(page_pgdat(page), NR_SLAB_UNRECLAIMABLE_B, - -(PAGE_SIZE << order)); + mod_lruvec_page_state(page, NR_SLAB_UNRECLAIMABLE_B, + -(PAGE_SIZE << order)); __free_pages(page, order); return; } From patchwork Wed Feb 24 20:04:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102565 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3512FC433E0 for ; Wed, 24 Feb 2021 20:04:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AF95764F30 for ; Wed, 24 Feb 2021 20:04:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AF95764F30 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3F80F8D0008; Wed, 24 Feb 2021 15:04:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3A8926B00A2; Wed, 24 Feb 2021 15:04:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2EDD18D0008; Wed, 24 Feb 2021 15:04:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0058.hostedemail.com [216.40.44.58]) by kanga.kvack.org (Postfix) with ESMTP id 195AC6B00A1 for ; Wed, 24 Feb 2021 15:04:32 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id D789182E0272 for ; Wed, 24 Feb 2021 20:04:31 +0000 (UTC) X-FDA: 77854238742.26.0DD6BAC Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP id 668FE130 for ; Wed, 24 Feb 2021 20:04:31 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 3BF1C64F14; Wed, 24 Feb 2021 20:04:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197070; bh=GpUPPlkFhzr/eMBBQFJLDCF7K0yl2WxELSQlNxSfFuA=; h=Date:From:To:Subject:In-Reply-To:From; b=nYb66waLHLpFS9yg5VkpYOdoB+sm/HmP+cKOlotu4OFiV1OLyBh4FOT9oP04QPKTC VYDqiT3Kr1xqZEne7IteqjZKTzpjDYKAkCLkRZZwi0XLBz9EtQ5cxhU909Q2x2HpCw gTfZ26v+kzmj7qsK7Rj+p4L3R5+qatVx45XEfx+A= Date: Wed, 24 Feb 2021 12:04:29 -0800 From: Andrew Morton To: ahuang12@lenovo.com, akpm@linux-foundation.org, david@redhat.com, jrdr.linux@gmail.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 076/173] mm/mmap.c: remove unnecessary local variable Message-ID: <20210224200429.srHZRhLS0%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 1ob135cfng5neassysk1bsre4ss4ycwd X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 668FE130 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf29; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197071-131120 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Adrian Huang Subject: mm/mmap.c: remove unnecessary local variable The local variable 'retval' is assigned just for once in __do_sys_brk(), and the function returns the value of the local variable right after the assignment. Remove unnecessary assignment and local variable declaration. Link: https://lkml.kernel.org/r/20201222103249.30683-1-adrianhuang0701@gmail.com Signed-off-by: Adrian Huang Acked-by: Souptick Joarder Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- mm/mmap.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) --- a/mm/mmap.c~mm-mmap-remove-unnecessary-local-variable +++ a/mm/mmap.c @@ -189,7 +189,6 @@ static int do_brk_flags(unsigned long ad struct list_head *uf); SYSCALL_DEFINE1(brk, unsigned long, brk) { - unsigned long retval; unsigned long newbrk, oldbrk, origbrk; struct mm_struct *mm = current->mm; struct vm_area_struct *next; @@ -281,9 +280,8 @@ success: return brk; out: - retval = origbrk; mmap_write_unlock(mm); - return retval; + return origbrk; } static inline unsigned long vma_compute_gap(struct vm_area_struct *vma) From patchwork Wed Feb 24 20:04:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102567 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BADE2C43381 for ; Wed, 24 Feb 2021 20:04:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5C2F064F33 for ; Wed, 24 Feb 2021 20:04:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5C2F064F33 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E0AD38D0009; Wed, 24 Feb 2021 15:04:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DBA496B00A3; Wed, 24 Feb 2021 15:04:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CF68F8D0009; Wed, 24 Feb 2021 15:04:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0122.hostedemail.com [216.40.44.122]) by kanga.kvack.org (Postfix) with ESMTP id B9FD16B00A2 for ; Wed, 24 Feb 2021 15:04:35 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 859DCFB55 for ; Wed, 24 Feb 2021 20:04:35 +0000 (UTC) X-FDA: 77854238910.16.1BF29CF Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP id 5D2E8200038A for ; Wed, 24 Feb 2021 20:04:36 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id ABA4A64F28; Wed, 24 Feb 2021 20:04:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197074; bh=ZMX7Cyocnd5Q+RWAIfyBqoozo9BNvXR0pWfWkaU5ECc=; h=Date:From:To:Subject:In-Reply-To:From; b=Q4uIjkespeCCnpFojjEr203EFyBUxFVSV+NOJRgkPQ8emcv2Tb5pYEbW33IJqI+xS NMvfm47pNoK8EeN7rOQer0wzdD0c9aLJv5efeke97RiluBIAo9txWtma2/ceIXinQQ aV8NiAUvPhBRaCKpmfBbSvuerJ44GkGXV3zkN/pQ= Date: Wed, 24 Feb 2021 12:04:33 -0800 From: Andrew Morton To: ak@linux.intel.com, akpm@linux-foundation.org, dave.hansen@intel.com, jpoimboe@redhat.com, linmiaohe@huawei.com, linux-mm@kvack.org, louhongxiang@huawei.com, mm-commits@vger.kernel.org, tglx@linutronix.de, torvalds@linux-foundation.org Subject: [patch 077/173] mm/memory.c: fix potential pte_unmap_unlock pte error Message-ID: <20210224200433.NUH3KlBjC%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 5D2E8200038A X-Stat-Signature: pc6xi8ogmy8ozk97khxq454fjo8j4mx1 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197076-394790 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/memory.c: fix potential pte_unmap_unlock pte error Since commit 42e4089c7890 ("x86/speculation/l1tf: Disallow non privileged high MMIO PROT_NONE mappings"), when the first pfn modify is not allowed, we would break the loop with pte unchanged. Then the wrong pte - 1 would be passed to pte_unmap_unlock. Andi said: : While the fix is correct, I'm not sure if it actually is a real bug. Is : there any architecture that would do something else than unlocking the : underlying page? If it's just the underlying page then it should be : always the same page, so no bug. Link: https://lkml.kernel.org/r/20210109080118.20885-1-linmiaohe@huawei.com Fixes: 42e4089c789 ("x86/speculation/l1tf: Disallow non privileged high MMIO PROT_NONE mappings") Signed-off-by: Hongxiang Lou Signed-off-by: Miaohe Lin Cc: Thomas Gleixner Cc: Dave Hansen Cc: Andi Kleen Cc: Josh Poimboeuf Signed-off-by: Andrew Morton --- mm/memory.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) --- a/mm/memory.c~mm-fix-potential-pte_unmap_unlock-pte-error +++ a/mm/memory.c @@ -2177,11 +2177,11 @@ static int remap_pte_range(struct mm_str unsigned long addr, unsigned long end, unsigned long pfn, pgprot_t prot) { - pte_t *pte; + pte_t *pte, *mapped_pte; spinlock_t *ptl; int err = 0; - pte = pte_alloc_map_lock(mm, pmd, addr, &ptl); + mapped_pte = pte = pte_alloc_map_lock(mm, pmd, addr, &ptl); if (!pte) return -ENOMEM; arch_enter_lazy_mmu_mode(); @@ -2195,7 +2195,7 @@ static int remap_pte_range(struct mm_str pfn++; } while (pte++, addr += PAGE_SIZE, addr != end); arch_leave_lazy_mmu_mode(); - pte_unmap_unlock(pte - 1, ptl); + pte_unmap_unlock(mapped_pte, ptl); return err; } From patchwork Wed Feb 24 20:04:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102571 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53FAAC433E0 for ; Wed, 24 Feb 2021 20:04:45 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DA6D864F38 for ; Wed, 24 Feb 2021 20:04:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DA6D864F38 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 707CD6B00A5; Wed, 24 Feb 2021 15:04:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6B6CC6B00A6; Wed, 24 Feb 2021 15:04:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5CC1D8D000B; Wed, 24 Feb 2021 15:04:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0047.hostedemail.com [216.40.44.47]) by kanga.kvack.org (Postfix) with ESMTP id 432016B00A5 for ; Wed, 24 Feb 2021 15:04:44 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 10A71F02A for ; Wed, 24 Feb 2021 20:04:44 +0000 (UTC) X-FDA: 77854239288.24.4136551 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf26.hostedemail.com (Postfix) with ESMTP id D837F407F8F3 for ; Wed, 24 Feb 2021 20:04:38 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 096DD64F33; Wed, 24 Feb 2021 20:04:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197077; bh=0UgaiyCy4K6CLFeEvqJrQ7SfRWJaZcn/x8Ckb8PI/vs=; h=Date:From:To:Subject:In-Reply-To:From; b=uT9KRS9fEQ2txJ93EtfOzEiESaj5QJF4yxgmmMdG5AjaJQuJKZM0MxZgJX6iJwbZ3 X5e8lI2PcM/q/tSmY1Ao4myexl/LujSXtnZE21ndxRtbp+amvfwA2C33Rk3f+EG9Vm 5bcUqKC30aXEl31xer4Y89PtCR7T8wZgk2Ly+goc= Date: Wed, 24 Feb 2021 12:04:36 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 078/173] mm/pgtable-generic.c: simplify the VM_BUG_ON condition in pmdp_huge_clear_flush() Message-ID: <20210224200436.R2zPY2Es_%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: D837F407F8F3 X-Stat-Signature: j7w16og9pku5jpnemj9bu7q9nint8nsp Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf26; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197078-452180 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/pgtable-generic.c: simplify the VM_BUG_ON condition in pmdp_huge_clear_flush() The condition (A && !C && !D) || !A is equivalent to !A || (A && !C && !D) and can be further simplified to !A || (!C && !D). Link: https://lkml.kernel.org/r/20210201114319.34720-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: Andrew Morton Signed-off-by: Andrew Morton --- mm/pgtable-generic.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/mm/pgtable-generic.c~mm-simplify-the-vm_bug_on-condition-in-pmdp_huge_clear_flush +++ a/mm/pgtable-generic.c @@ -135,8 +135,8 @@ pmd_t pmdp_huge_clear_flush(struct vm_ar { pmd_t pmd; VM_BUG_ON(address & ~HPAGE_PMD_MASK); - VM_BUG_ON((pmd_present(*pmdp) && !pmd_trans_huge(*pmdp) && - !pmd_devmap(*pmdp)) || !pmd_present(*pmdp)); + VM_BUG_ON(!pmd_present(*pmdp) || (!pmd_trans_huge(*pmdp) && + !pmd_devmap(*pmdp))); pmd = pmdp_huge_get_and_clear(vma->vm_mm, address, pmdp); flush_pmd_tlb_range(vma, address, address + HPAGE_PMD_SIZE); return pmd; From patchwork Wed Feb 24 20:04:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102569 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A950C433E0 for ; Wed, 24 Feb 2021 20:04:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B5D8764F29 for ; Wed, 24 Feb 2021 20:04:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B5D8764F29 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 556366B00A3; Wed, 24 Feb 2021 15:04:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 508086B00A4; Wed, 24 Feb 2021 15:04:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41E288D000B; Wed, 24 Feb 2021 15:04:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0229.hostedemail.com [216.40.44.229]) by kanga.kvack.org (Postfix) with ESMTP id 2D2916B00A3 for ; Wed, 24 Feb 2021 15:04:42 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id E9CA3184C0752 for ; Wed, 24 Feb 2021 20:04:41 +0000 (UTC) X-FDA: 77854239162.05.7D56AC7 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP id 39843C0007D7 for ; Wed, 24 Feb 2021 20:04:34 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 3397364F2D; Wed, 24 Feb 2021 20:04:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197080; bh=gTjnI4BIkQ1GHRRNKlBFUJ5q0apX1+RAnlPsE3l1wrM=; h=Date:From:To:Subject:In-Reply-To:From; b=rxojsoa+cRx2XPRA9Tgq0RbmX4T5PVc6bvMRulNKxb0w/nPJQsLKp7Ult8N7TQXop Qt+mvrAgu50RxT+RJDHuvr1MhuMgHYO4eZZaAdg0JUbbE8X6m8enKLyhQJpBiN0Jrd CYKIz+y1sHvZGBFBe2zhvLpjLVA8OxlBIpFOER6A= Date: Wed, 24 Feb 2021 12:04:39 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 079/173] mm/pgtable-generic.c: optimize the VM_BUG_ON condition in pmdp_huge_clear_flush() Message-ID: <20210224200439.0k6rJzU8V%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 39843C0007D7 X-Stat-Signature: 8hr5y6jtkxdnc4om1denua6311ipf41f Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf14; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197074-671484 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/pgtable-generic.c: optimize the VM_BUG_ON condition in pmdp_huge_clear_flush() The developer will have trouble figuring out why the BUG actually triggered when there is a complex expression in the VM_BUG_ON. Because we can only identify the condition triggered BUG via line number provided by VM_BUG_ON. Optimize this by spliting such a complex expression into two simple conditions. Link: https://lkml.kernel.org/r/20210203084137.25522-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Suggested-by: Andrew Morton Signed-off-by: Andrew Morton --- mm/pgtable-generic.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) --- a/mm/pgtable-generic.c~mm-pgtable-genericc-optimize-the-vm_bug_on-condition-in-pmdp_huge_clear_flush +++ a/mm/pgtable-generic.c @@ -135,8 +135,9 @@ pmd_t pmdp_huge_clear_flush(struct vm_ar { pmd_t pmd; VM_BUG_ON(address & ~HPAGE_PMD_MASK); - VM_BUG_ON(!pmd_present(*pmdp) || (!pmd_trans_huge(*pmdp) && - !pmd_devmap(*pmdp))); + VM_BUG_ON(!pmd_present(*pmdp)); + /* Below assumes pmd_present() is true */ + VM_BUG_ON(!pmd_trans_huge(*pmdp) && !pmd_devmap(*pmdp)); pmd = pmdp_huge_get_and_clear(vma->vm_mm, address, pmdp); flush_pmd_tlb_range(vma, address, address + HPAGE_PMD_SIZE); return pmd; From patchwork Wed Feb 24 20:04:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102639 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C2C0C433DB for ; Wed, 24 Feb 2021 20:06:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id ACB0964EC3 for ; Wed, 24 Feb 2021 20:06:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ACB0964EC3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 354898D001D; Wed, 24 Feb 2021 15:06:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2DE516B00E0; Wed, 24 Feb 2021 15:06:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 17E178D001D; Wed, 24 Feb 2021 15:06:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0254.hostedemail.com [216.40.44.254]) by kanga.kvack.org (Postfix) with ESMTP id F0B4E6B00DE for ; Wed, 24 Feb 2021 15:06:56 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id BDA6582C950D for ; Wed, 24 Feb 2021 20:06:56 +0000 (UTC) X-FDA: 77854244832.16.1BA10B6 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP id 3B8C9E00010A for ; Wed, 24 Feb 2021 20:06:56 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7A1D164F29; Wed, 24 Feb 2021 20:04:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197083; bh=AkQdFROiB4kD2nW5q7VH4tUqdd3BV6k+AP0uR4ayswc=; h=Date:From:To:Subject:In-Reply-To:From; b=e9XDXRGhwP0WHk4ljHoV9sgAL1d2A9TvarTG13j1TYLoItgmwP50tMTRsdD7dPFwz 0g68dzJRBvX+iBTeOjS4tEIqW5rFT5YdvDhCxEdOkVE28KFzNb5g/JQvOeOkieTSKP Rn3xyBqUPXSrIO0o/XOfUCkQQW8IJNRaUtOF4wKI= Date: Wed, 24 Feb 2021 12:04:42 -0800 From: Andrew Morton To: akpm@linux-foundation.org, chrisw@sous-sol.org, ian.pratt@xensource.com, linmiaohe@huawei.com, linux-mm@kvack.org, mingo@elte.hu, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 080/173] mm/memory.c: fix potential pte_unmap_unlock pte error Message-ID: <20210224200442.Apg5eElaa%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 3B8C9E00010A X-Stat-Signature: c4zy8smq1uyphhk95674134mco1uwiph Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf30; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197216-917458 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/memory.c: fix potential pte_unmap_unlock pte error If all pte entry is none in 'non-create' case, we would break the loop with pte unchanged. Then the wrong pte - 1 would be passed to pte_unmap_unlock. This is a theoretical issue which may not be a real bug. So it's not worth cc stable. Link: https://lkml.kernel.org/r/20210205081925.59809-1-linmiaohe@huawei.com Fixes: aee16b3cee27 ("Add apply_to_page_range() which applies a function to a pte range") Signed-off-by: Miaohe Lin Reviewed-by: Andrew Morton Cc: Ingo Molnar Cc: Ian Pratt Cc: Chris Wright Signed-off-by: Andrew Morton --- mm/memory.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) --- a/mm/memory.c~mm-memoryc-fix-potential-pte_unmap_unlock-pte-error +++ a/mm/memory.c @@ -2394,18 +2394,18 @@ static int apply_to_pte_range(struct mm_ pte_fn_t fn, void *data, bool create, pgtbl_mod_mask *mask) { - pte_t *pte; + pte_t *pte, *mapped_pte; int err = 0; spinlock_t *ptl; if (create) { - pte = (mm == &init_mm) ? + mapped_pte = pte = (mm == &init_mm) ? pte_alloc_kernel_track(pmd, addr, mask) : pte_alloc_map_lock(mm, pmd, addr, &ptl); if (!pte) return -ENOMEM; } else { - pte = (mm == &init_mm) ? + mapped_pte = pte = (mm == &init_mm) ? pte_offset_kernel(pmd, addr) : pte_offset_map_lock(mm, pmd, addr, &ptl); } @@ -2428,7 +2428,7 @@ static int apply_to_pte_range(struct mm_ arch_leave_lazy_mmu_mode(); if (mm != &init_mm) - pte_unmap_unlock(pte-1, ptl); + pte_unmap_unlock(mapped_pte, ptl); return err; } From patchwork Wed Feb 24 20:04:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102573 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D096DC433E0 for ; Wed, 24 Feb 2021 20:04:49 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7AEE464F36 for ; Wed, 24 Feb 2021 20:04:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7AEE464F36 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 09B4C6B00A7; Wed, 24 Feb 2021 15:04:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 04BC98D0014; Wed, 24 Feb 2021 15:04:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ECA298D000B; Wed, 24 Feb 2021 15:04:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0077.hostedemail.com [216.40.44.77]) by kanga.kvack.org (Postfix) with ESMTP id D74B56B00A7 for ; Wed, 24 Feb 2021 15:04:48 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A056F82C27FE for ; Wed, 24 Feb 2021 20:04:48 +0000 (UTC) X-FDA: 77854239456.27.4FF1C98 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id 8F0E02000392 for ; Wed, 24 Feb 2021 20:04:38 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id D3EB964F31; Wed, 24 Feb 2021 20:04:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197087; bh=c04nelTSFATbEmttuyWFPL753KSZH3WH9JphC+q50wg=; h=Date:From:To:Subject:In-Reply-To:From; b=E8HNuA8A6+YYmOpYnWWmw6qupGzOBLphP1S4+0REheTgGu67/nCoZJth7s7YFslyy 7Do1jlQDsuqfEClFC155nuWEQBVWuXQvcWw8wFSFLx09eIsaXQU6UMg7d0qxyvY5B/ PBAHo9XmhaU7YAwWQoJbnnbRifRydV87MTsynQn8= Date: Wed, 24 Feb 2021 12:04:46 -0800 From: Andrew Morton To: akpm@linux-foundation.org, jarkko@kernel.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, tianjia.zhang@linux.alibaba.com, torvalds@linux-foundation.org, zhang.jia@linux.alibaba.com Subject: [patch 081/173] mm/mprotect.c: optimize error detection in do_mprotect_pkey() Message-ID: <20210224200446.uerlj0CaI%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 8F0E02000392 X-Stat-Signature: kyhtq19uwz6gifkw4u3sizw8chmm5zd7 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf11; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197078-97569 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Tianjia Zhang Subject: mm/mprotect.c: optimize error detection in do_mprotect_pkey() Obviously, the error variable detection of the if statement is for the mprotect callback function, so it is also put into the scope of calling callbck. This is a cleanup which makes this site consistent with the rest of this function's error handling. Link: https://lkml.kernel.org/r/20210118133310.98375-1-tianjia.zhang@linux.alibaba.com Signed-off-by: Tianjia Zhang Reported-by: Jia Zhang Reviewed-by: Andrew Morton Cc: Jarkko Sakkinen Signed-off-by: Andrew Morton --- mm/mprotect.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) --- a/mm/mprotect.c~mm-optimizing-error-condition-detection-in-do_mprotect_pkey +++ a/mm/mprotect.c @@ -617,10 +617,11 @@ static int do_mprotect_pkey(unsigned lon if (tmp > end) tmp = end; - if (vma->vm_ops && vma->vm_ops->mprotect) + if (vma->vm_ops && vma->vm_ops->mprotect) { error = vma->vm_ops->mprotect(vma, nstart, tmp, newflags); - if (error) - goto out; + if (error) + goto out; + } error = mprotect_fixup(vma, &prev, nstart, tmp, newflags); if (error) From patchwork Wed Feb 24 20:04:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102575 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A8B0EC433E0 for ; Wed, 24 Feb 2021 20:04:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 13E1264F35 for ; Wed, 24 Feb 2021 20:04:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 13E1264F35 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 99E0C6B00A9; Wed, 24 Feb 2021 15:04:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 94D2B8D0014; Wed, 24 Feb 2021 15:04:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 863468D000B; Wed, 24 Feb 2021 15:04:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0230.hostedemail.com [216.40.44.230]) by kanga.kvack.org (Postfix) with ESMTP id 71FFB6B00A9 for ; Wed, 24 Feb 2021 15:04:52 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 41E51184C0D6D for ; Wed, 24 Feb 2021 20:04:52 +0000 (UTC) X-FDA: 77854239624.18.18D2268 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf24.hostedemail.com (Postfix) with ESMTP id 609D7A000510 for ; Wed, 24 Feb 2021 20:04:46 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7E34164F36; Wed, 24 Feb 2021 20:04:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197090; bh=o5B2UwUoJBuubQ9hFq7S1qGARKUxOP/DrAwHAuIUB9s=; h=Date:From:To:Subject:In-Reply-To:From; b=oCvZTMNtDzZZMa6A+QSEMsvYKBuAbdcR1ITA0W++v4hXJfLUtlxY67kEX9CQJ29xd LKR3CgnGwgnAuNU5Y/R1kOoUaoojpydwZXr/Fearnp4OlAPuEAIPQiTKvxDgJLExFQ pNngKXLa4/hBrIJxzsKHa2T/1vkfJ+VBzBo4EsqQ= Date: Wed, 24 Feb 2021 12:04:49 -0800 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, bgeffon@google.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, lixinhai.lxh@gmail.com, lokeshgidra@google.com, minchan@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 082/173] mm: rmap: explicitly reset vma->anon_vma in unlink_anon_vmas() Message-ID: <20210224200449.hkU5GTEiH%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 609D7A000510 X-Stat-Signature: rbbf8459rsw5nkjytidrko1jwrpmjsb3 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf24; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197086-391718 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Li Xinhai Subject: mm: rmap: explicitly reset vma->anon_vma in unlink_anon_vmas() In case the vma will continue to be used after unlink its relevant anon_vma, we need to reset the vma->anon_vma pointer to NULL. So, later when fault happen within this vma again, a new anon_vma will be prepared. By this way, the vma will only be checked for reverse mapping of pages which been fault in after the unlink_anon_vmas call. Currently, the mremap with MREMAP_DONTUNMAP scenario will continue use the vma after moved its page table entries to a new vma. For other scenarios, the vma itself will be freed after call unlink_anon_vmas. Link: https://lkml.kernel.org/r/20210119075126.3513154-1-lixinhai.lxh@gmail.com Signed-off-by: Li Xinhai Cc: Andrea Arcangeli Cc: Brian Geffon Cc: Kirill A. Shutemov Cc: Lokesh Gidra Cc: Minchan Kim Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- mm/rmap.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) --- a/mm/rmap.c~mm-rmap-explicitly-reset-vma-anon_vma-in-unlink_anon_vmas +++ a/mm/rmap.c @@ -413,8 +413,15 @@ void unlink_anon_vmas(struct vm_area_str list_del(&avc->same_vma); anon_vma_chain_free(avc); } - if (vma->anon_vma) + if (vma->anon_vma) { vma->anon_vma->degree--; + + /* + * vma would still be needed after unlink, and anon_vma will be prepared + * when handle fault. + */ + vma->anon_vma = NULL; + } unlock_anon_vma_root(root); /* From patchwork Wed Feb 24 20:04:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102577 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33036C433DB for ; Wed, 24 Feb 2021 20:04:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C1DA064F35 for ; Wed, 24 Feb 2021 20:04:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C1DA064F35 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5575F6B00AB; Wed, 24 Feb 2021 15:04:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5078D8D0014; Wed, 24 Feb 2021 15:04:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3F5D18D000B; Wed, 24 Feb 2021 15:04:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0174.hostedemail.com [216.40.44.174]) by kanga.kvack.org (Postfix) with ESMTP id 2AB3E6B00AB for ; Wed, 24 Feb 2021 15:04:56 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id E5815184DD1ED for ; Wed, 24 Feb 2021 20:04:55 +0000 (UTC) X-FDA: 77854239750.06.C19353C Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP id 75823E4 for ; Wed, 24 Feb 2021 20:04:55 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id F3DC464F37; Wed, 24 Feb 2021 20:04:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197094; bh=k1hKdwrJHIO+A24G95yDbw/sYmpvr+nfWn01btySRwo=; h=Date:From:To:Subject:In-Reply-To:From; b=MwSpEj+WUQe55xi1uFZg0iBdxbBJ1ij+JM/q2AMWhhoa2/Aixeuz1itnlS2XjODHp UGBDf0Ng0UIdcKVkQkNyMaxDHonLdhoxWpWTnKl9t1cV1gSMMasmBNg16dHOR33Plt 6J8lDKopmtQvvRely11yqFZgBOt1L7/06Ta173w0= Date: Wed, 24 Feb 2021 12:04:53 -0800 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, bgeffon@google.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, lixinhai.lxh@gmail.com, lokeshgidra@google.com, minchan@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 083/173] mm: mremap: unlink anon_vmas when mremap with MREMAP_DONTUNMAP success Message-ID: <20210224200453.9k7nHP6Ig%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: rm5rta9w1wmgnj9j4ft3upmkk5x6m8m4 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 75823E4 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf29; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197095-609985 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Li Xinhai Subject: mm: mremap: unlink anon_vmas when mremap with MREMAP_DONTUNMAP success mremap with MREMAP_DONTUNMAP can move all page table entries to new vma, which means all pages allocated for the old vma are not relevant to it anymore, and the relevant anon_vma links needs to be unlinked, in nature the old vma is much like been freshly created and have no pages been fault in. But we should not do unlink, if the new vma has effectively merged with the old one. [lixinhai.lxh@gmail.com: v2] Link: https://lkml.kernel.org/r/20210127083917.309264-2-lixinhai.lxh@gmail.com Link: https://lkml.kernel.org/r/20210119075126.3513154-2-lixinhai.lxh@gmail.com Signed-off-by: Li Xinhai Cc: Brian Geffon Cc: Lokesh Gidra Cc: Minchan Kim Cc: Kirill A. Shutemov Cc: Vlastimil Babka Cc: Andrea Arcangeli Signed-off-by: Andrew Morton --- mm/mremap.c | 8 ++++++++ 1 file changed, 8 insertions(+) --- a/mm/mremap.c~mm-mremap-unlink-anon_vmas-when-mremap-with-mremap_dontunmap-success +++ a/mm/mremap.c @@ -593,6 +593,14 @@ static unsigned long move_vma(struct vm_ /* We always clear VM_LOCKED[ONFAULT] on the old vma */ vma->vm_flags &= VM_LOCKED_CLEAR_MASK; + /* + * anon_vma links of the old vma is no longer needed after its page + * table has been moved. + */ + if (new_vma != vma && vma->vm_start == old_addr && + vma->vm_end == (old_addr + old_len)) + unlink_anon_vmas(vma); + /* Because we won't unmap we don't need to touch locked_vm */ return new_addr; } From patchwork Wed Feb 24 20:04:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102579 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8D015C433DB for ; Wed, 24 Feb 2021 20:05:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1C62764E20 for ; Wed, 24 Feb 2021 20:05:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1C62764E20 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9764A6B00AD; Wed, 24 Feb 2021 15:04:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 926138D0014; Wed, 24 Feb 2021 15:04:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 83AC58D000B; Wed, 24 Feb 2021 15:04:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0176.hostedemail.com [216.40.44.176]) by kanga.kvack.org (Postfix) with ESMTP id 6C94F6B00AD for ; Wed, 24 Feb 2021 15:04:59 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 3915252C1 for ; Wed, 24 Feb 2021 20:04:59 +0000 (UTC) X-FDA: 77854239918.02.D72EF19 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf02.hostedemail.com (Postfix) with ESMTP id 7D128407F8F7 for ; Wed, 24 Feb 2021 20:04:45 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 940C364F35; Wed, 24 Feb 2021 20:04:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197098; bh=XQKP27moMFkKHciNc/q9f6Gqms+vt4sg+mpiFBlu66A=; h=Date:From:To:Subject:In-Reply-To:From; b=unRm6qyPrQozw7PKb0Xj7gUnEnmSgS739GJqoGn6KcwB1oo9HsTJzBl7yAmGLyy6e cJoFW0UsKH/FwyNXkhsPtRgWwK/ZPByWKHiAXafBSu1wTFr7bP58dlp5UVz6ZpNnvD QudmVwmcsJZzloC0oagjXXsdSi0kX9cRGUPG6Zvo= Date: Wed, 24 Feb 2021 12:04:57 -0800 From: Andrew Morton To: akpm@linux-foundation.org, alexander.h.duyck@linux.intel.com, david@redhat.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, sh_def@163.com, torvalds@linux-foundation.org Subject: [patch 084/173] mm/page_reporting: use list_entry_is_head() in page_reporting_cycle() Message-ID: <20210224200457.Ws5HbjiS0%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: dd1uhcucbb8eapebnfjhmeoowzuegcx9 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 7D128407F8F7 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf02; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197085-154441 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: sh Subject: mm/page_reporting: use list_entry_is_head() in page_reporting_cycle() Replace '&next->lru != list' with list_entry_is_head(). No functional change. Link: https://lkml.kernel.org/r/20201222182735.GA1257912@ubuntu-A520I-AC Signed-off-by: sh Reviewed-by: Andrew Morton Cc: Alexander Duyck Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- mm/page_reporting.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/page_reporting.c~mm-page_reporting-use-list_entry_is_head-in-page_reporting_cycle +++ a/mm/page_reporting.c @@ -211,7 +211,7 @@ page_reporting_cycle(struct page_reporti } /* Rotate any leftover pages to the head of the freelist */ - if (&next->lru != list && !list_is_first(&next->lru, list)) + if (!list_entry_is_head(next, list, lru) && !list_is_first(&next->lru, list)) list_rotate_to_front(&next->lru, list); spin_unlock_irq(&zone->lock); From patchwork Wed Feb 24 20:05:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102581 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BFC3C433E0 for ; Wed, 24 Feb 2021 20:05:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CFBC664E20 for ; Wed, 24 Feb 2021 20:05:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CFBC664E20 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6B4FC6B00AF; Wed, 24 Feb 2021 15:05:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 68E038D0014; Wed, 24 Feb 2021 15:05:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5A4D68D000B; Wed, 24 Feb 2021 15:05:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0204.hostedemail.com [216.40.44.204]) by kanga.kvack.org (Postfix) with ESMTP id 4576F6B00AF for ; Wed, 24 Feb 2021 15:05:04 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 169281730876 for ; Wed, 24 Feb 2021 20:05:04 +0000 (UTC) X-FDA: 77854240128.02.D2F5338 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf02.hostedemail.com (Postfix) with ESMTP id 1DB17407F8DC for ; Wed, 24 Feb 2021 20:04:50 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 15D7F64F20; Wed, 24 Feb 2021 20:05:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197102; bh=vFh4qvEWMqr1XNAa0CR0/f9E19one+5mBlwsfnK63do=; h=Date:From:To:Subject:In-Reply-To:From; b=meL1AX21BoUYPv4qlKtSqm6odGKPfuEoD82MDU2TScTniGMZwOr0pcsbPO237E+d5 iq1ur6bK0JZRaeMauMQsK/WRfqXyV28JeKKfqPIL5oXDZKPRL++LHVDOzz2CKZn3H/ KS9feUiChP4IAVZbnqA2JgCKXc6s7/AeglK5esw4= Date: Wed, 24 Feb 2021 12:05:00 -0800 From: Andrew Morton To: abaci-bugfix@linux.alibaba.com, abaci@linux.alibaba.com, adobriyan@gmail.com, akpm@linux-foundation.org, bhe@redhat.com, dyoung@redhat.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, urezki@gmail.com, vgoyal@redhat.com Subject: [patch 085/173] vmalloc: remove redundant NULL check Message-ID: <20210224200500.Nfm1N1293%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 1DB17407F8DC X-Stat-Signature: zrxbri53qeyz5qd37hp6cehsjks8p9zr Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf02; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197090-607353 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yang Li Subject: vmalloc: remove redundant NULL check Fix below warnings reported by coccicheck: ./fs/proc/vmcore.c:1503:2-7: WARNING: NULL check before some freeing functions is not needed. Link: https://lkml.kernel.org/r/1611216753-44598-1-git-send-email-abaci-bugfix@linux.alibaba.com Signed-off-by: Yang Li Reported-by: Abaci Robot Acked-by: Baoquan He Cc: Dave Young Cc: Vivek Goyal Cc: Alexey Dobriyan Cc: "Uladzislau Rezki (Sony)" Signed-off-by: Andrew Morton --- fs/proc/vmcore.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) --- a/fs/proc/vmcore.c~vmalloc-remove-redundant-null-check +++ a/fs/proc/vmcore.c @@ -1503,11 +1503,8 @@ int vmcore_add_device_dump(struct vmcore return 0; out_err: - if (buf) - vfree(buf); - - if (dump) - vfree(dump); + vfree(buf); + vfree(dump); return ret; } From patchwork Wed Feb 24 20:05:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102583 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFC69C433E9 for ; Wed, 24 Feb 2021 20:05:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4BBC864E20 for ; Wed, 24 Feb 2021 20:05:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4BBC864E20 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D46566B00B1; Wed, 24 Feb 2021 15:05:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D1B6E8D0014; Wed, 24 Feb 2021 15:05:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE5308D000B; Wed, 24 Feb 2021 15:05:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0110.hostedemail.com [216.40.44.110]) by kanga.kvack.org (Postfix) with ESMTP id A3E886B00B1 for ; Wed, 24 Feb 2021 15:05:08 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 62E82246E for ; Wed, 24 Feb 2021 20:05:08 +0000 (UTC) X-FDA: 77854240296.08.2719DE1 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP id 272B8F0 for ; Wed, 24 Feb 2021 20:05:06 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C082564F1B; Wed, 24 Feb 2021 20:05:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197106; bh=BU/7ZWgDCjiColg29WPClBoZh6IhcSt3ULrdfHykyl4=; h=Date:From:To:Subject:In-Reply-To:From; b=oGm+cwo28DuptWRVIGvwFzbFEF0ZoOvVuJG1f9qb8H+8A9PC8OW9Q7XPNg03xEo01 Ua0CQ9Bvy+bzFkrYoztv/WmNqtHqW6sEaMJo5e+Xnlz3YRDWigZCW3+GiUBa8MmMEj FOpCdpDW0LqOCUDttDTsldwF/roIRttoCYFsW6AY= Date: Wed, 24 Feb 2021 12:05:05 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, pcc@google.com, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 086/173] kasan: prefix global functions with kasan_ Message-ID: <20210224200505.oxPIcJ5Il%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 18gwkzwr9wsqwefu8qtuzd3n6k5mnwi8 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 272B8F0 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf04; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197106-883184 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: prefix global functions with kasan_ Patch series "kasan: HW_TAGS tests support and fixes", v4. This patchset adds support for running KASAN-KUnit tests with the hardware tag-based mode and also contains a few fixes. This patch (of 15): There's a number of internal KASAN functions that are used across multiple source code files and therefore aren't marked as static inline. To avoid littering the kernel function names list with generic function names, prefix all such KASAN functions with kasan_. As a part of this change: - Rename internal (un)poison_range() to kasan_(un)poison() (no _range) to avoid name collision with a public kasan_unpoison_range(). - Rename check_memory_region() to kasan_check_range(), as it's a more fitting name. Link: https://lkml.kernel.org/r/cover.1610733117.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/I719cc93483d4ba288a634dba80ee6b7f2809cd26 Link: https://lkml.kernel.org/r/13777aedf8d3ebbf35891136e1f2287e2f34aaba.1610733117.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Suggested-by: Marco Elver Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Cc: Catalin Marinas Cc: Vincenzo Frascino Cc: Dmitry Vyukov Cc: Will Deacon Cc: Andrey Ryabinin Cc: Peter Collingbourne Cc: Evgenii Stepanov Cc: Branislav Rankov Cc: Kevin Brodsky Signed-off-by: Andrew Morton --- mm/kasan/common.c | 47 +++++++++++++++++------------------ mm/kasan/generic.c | 36 +++++++++++++-------------- mm/kasan/kasan.h | 48 ++++++++++++++++++------------------ mm/kasan/quarantine.c | 22 ++++++++-------- mm/kasan/report.c | 13 +++++---- mm/kasan/report_generic.c | 8 +++--- mm/kasan/report_hw_tags.c | 8 +++--- mm/kasan/report_sw_tags.c | 8 +++--- mm/kasan/shadow.c | 26 +++++++++---------- mm/kasan/sw_tags.c | 16 ++++++------ tools/objtool/check.c | 2 - 11 files changed, 117 insertions(+), 117 deletions(-) --- a/mm/kasan/common.c~kasan-prefix-global-functions-with-kasan_ +++ a/mm/kasan/common.c @@ -60,7 +60,7 @@ void kasan_disable_current(void) void __kasan_unpoison_range(const void *address, size_t size) { - unpoison_range(address, size); + kasan_unpoison(address, size); } #if CONFIG_KASAN_STACK @@ -69,7 +69,7 @@ void kasan_unpoison_task_stack(struct ta { void *base = task_stack_page(task); - unpoison_range(base, THREAD_SIZE); + kasan_unpoison(base, THREAD_SIZE); } /* Unpoison the stack for the current task beyond a watermark sp value. */ @@ -82,7 +82,7 @@ asmlinkage void kasan_unpoison_task_stac */ void *base = (void *)((unsigned long)watermark & ~(THREAD_SIZE - 1)); - unpoison_range(base, watermark - base); + kasan_unpoison(base, watermark - base); } #endif /* CONFIG_KASAN_STACK */ @@ -105,18 +105,17 @@ void __kasan_alloc_pages(struct page *pa if (unlikely(PageHighMem(page))) return; - tag = random_tag(); + tag = kasan_random_tag(); for (i = 0; i < (1 << order); i++) page_kasan_tag_set(page + i, tag); - unpoison_range(page_address(page), PAGE_SIZE << order); + kasan_unpoison(page_address(page), PAGE_SIZE << order); } void __kasan_free_pages(struct page *page, unsigned int order) { if (likely(!PageHighMem(page))) - poison_range(page_address(page), - PAGE_SIZE << order, - KASAN_FREE_PAGE); + kasan_poison(page_address(page), PAGE_SIZE << order, + KASAN_FREE_PAGE); } /* @@ -246,18 +245,18 @@ void __kasan_poison_slab(struct page *pa for (i = 0; i < compound_nr(page); i++) page_kasan_tag_reset(page + i); - poison_range(page_address(page), page_size(page), + kasan_poison(page_address(page), page_size(page), KASAN_KMALLOC_REDZONE); } void __kasan_unpoison_object_data(struct kmem_cache *cache, void *object) { - unpoison_range(object, cache->object_size); + kasan_unpoison(object, cache->object_size); } void __kasan_poison_object_data(struct kmem_cache *cache, void *object) { - poison_range(object, cache->object_size, KASAN_KMALLOC_REDZONE); + kasan_poison(object, cache->object_size, KASAN_KMALLOC_REDZONE); } /* @@ -294,7 +293,7 @@ static u8 assign_tag(struct kmem_cache * * set, assign a tag when the object is being allocated (init == false). */ if (!cache->ctor && !(cache->flags & SLAB_TYPESAFE_BY_RCU)) - return init ? KASAN_TAG_KERNEL : random_tag(); + return init ? KASAN_TAG_KERNEL : kasan_random_tag(); /* For caches that either have a constructor or SLAB_TYPESAFE_BY_RCU: */ #ifdef CONFIG_SLAB @@ -305,7 +304,7 @@ static u8 assign_tag(struct kmem_cache * * For SLUB assign a random tag during slab creation, otherwise reuse * the already assigned tag. */ - return init ? random_tag() : get_tag(object); + return init ? kasan_random_tag() : get_tag(object); #endif } @@ -346,12 +345,12 @@ static bool ____kasan_slab_free(struct k if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) return false; - if (check_invalid_free(tagged_object)) { + if (kasan_check_invalid_free(tagged_object)) { kasan_report_invalid_free(tagged_object, ip); return true; } - poison_range(object, cache->object_size, KASAN_KMALLOC_FREE); + kasan_poison(object, cache->object_size, KASAN_KMALLOC_FREE); if (!kasan_stack_collection_enabled()) return false; @@ -361,7 +360,7 @@ static bool ____kasan_slab_free(struct k kasan_set_free_info(cache, object, tag); - return quarantine_put(cache, object); + return kasan_quarantine_put(cache, object); } bool __kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip) @@ -386,7 +385,7 @@ void __kasan_slab_free_mempool(void *ptr kasan_report_invalid_free(ptr, ip); return; } - poison_range(ptr, page_size(page), KASAN_FREE_PAGE); + kasan_poison(ptr, page_size(page), KASAN_FREE_PAGE); } else { ____kasan_slab_free(page->slab_cache, ptr, ip, false); } @@ -409,7 +408,7 @@ static void *____kasan_kmalloc(struct km u8 tag; if (gfpflags_allow_blocking(flags)) - quarantine_reduce(); + kasan_quarantine_reduce(); if (unlikely(object == NULL)) return NULL; @@ -421,9 +420,9 @@ static void *____kasan_kmalloc(struct km tag = assign_tag(cache, object, false, keep_tag); /* Tag is ignored in set_tag without CONFIG_KASAN_SW/HW_TAGS */ - unpoison_range(set_tag(object, tag), size); - poison_range((void *)redzone_start, redzone_end - redzone_start, - KASAN_KMALLOC_REDZONE); + kasan_unpoison(set_tag(object, tag), size); + kasan_poison((void *)redzone_start, redzone_end - redzone_start, + KASAN_KMALLOC_REDZONE); if (kasan_stack_collection_enabled()) set_alloc_info(cache, (void *)object, flags); @@ -452,7 +451,7 @@ void * __must_check __kasan_kmalloc_larg unsigned long redzone_end; if (gfpflags_allow_blocking(flags)) - quarantine_reduce(); + kasan_quarantine_reduce(); if (unlikely(ptr == NULL)) return NULL; @@ -462,8 +461,8 @@ void * __must_check __kasan_kmalloc_larg KASAN_GRANULE_SIZE); redzone_end = (unsigned long)ptr + page_size(page); - unpoison_range(ptr, size); - poison_range((void *)redzone_start, redzone_end - redzone_start, + kasan_unpoison(ptr, size); + kasan_poison((void *)redzone_start, redzone_end - redzone_start, KASAN_PAGE_REDZONE); return (void *)ptr; --- a/mm/kasan/generic.c~kasan-prefix-global-functions-with-kasan_ +++ a/mm/kasan/generic.c @@ -158,7 +158,7 @@ static __always_inline bool memory_is_po return memory_is_poisoned_n(addr, size); } -static __always_inline bool check_memory_region_inline(unsigned long addr, +static __always_inline bool check_region_inline(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { @@ -179,13 +179,13 @@ static __always_inline bool check_memory return !kasan_report(addr, size, write, ret_ip); } -bool check_memory_region(unsigned long addr, size_t size, bool write, - unsigned long ret_ip) +bool kasan_check_range(unsigned long addr, size_t size, bool write, + unsigned long ret_ip) { - return check_memory_region_inline(addr, size, write, ret_ip); + return check_region_inline(addr, size, write, ret_ip); } -bool check_invalid_free(void *addr) +bool kasan_check_invalid_free(void *addr) { s8 shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(addr)); @@ -194,22 +194,22 @@ bool check_invalid_free(void *addr) void kasan_cache_shrink(struct kmem_cache *cache) { - quarantine_remove_cache(cache); + kasan_quarantine_remove_cache(cache); } void kasan_cache_shutdown(struct kmem_cache *cache) { if (!__kmem_cache_empty(cache)) - quarantine_remove_cache(cache); + kasan_quarantine_remove_cache(cache); } static void register_global(struct kasan_global *global) { size_t aligned_size = round_up(global->size, KASAN_GRANULE_SIZE); - unpoison_range(global->beg, global->size); + kasan_unpoison(global->beg, global->size); - poison_range(global->beg + aligned_size, + kasan_poison(global->beg + aligned_size, global->size_with_redzone - aligned_size, KASAN_GLOBAL_REDZONE); } @@ -231,7 +231,7 @@ EXPORT_SYMBOL(__asan_unregister_globals) #define DEFINE_ASAN_LOAD_STORE(size) \ void __asan_load##size(unsigned long addr) \ { \ - check_memory_region_inline(addr, size, false, _RET_IP_);\ + check_region_inline(addr, size, false, _RET_IP_); \ } \ EXPORT_SYMBOL(__asan_load##size); \ __alias(__asan_load##size) \ @@ -239,7 +239,7 @@ EXPORT_SYMBOL(__asan_unregister_globals) EXPORT_SYMBOL(__asan_load##size##_noabort); \ void __asan_store##size(unsigned long addr) \ { \ - check_memory_region_inline(addr, size, true, _RET_IP_); \ + check_region_inline(addr, size, true, _RET_IP_); \ } \ EXPORT_SYMBOL(__asan_store##size); \ __alias(__asan_store##size) \ @@ -254,7 +254,7 @@ DEFINE_ASAN_LOAD_STORE(16); void __asan_loadN(unsigned long addr, size_t size) { - check_memory_region(addr, size, false, _RET_IP_); + kasan_check_range(addr, size, false, _RET_IP_); } EXPORT_SYMBOL(__asan_loadN); @@ -264,7 +264,7 @@ EXPORT_SYMBOL(__asan_loadN_noabort); void __asan_storeN(unsigned long addr, size_t size) { - check_memory_region(addr, size, true, _RET_IP_); + kasan_check_range(addr, size, true, _RET_IP_); } EXPORT_SYMBOL(__asan_storeN); @@ -290,11 +290,11 @@ void __asan_alloca_poison(unsigned long WARN_ON(!IS_ALIGNED(addr, KASAN_ALLOCA_REDZONE_SIZE)); - unpoison_range((const void *)(addr + rounded_down_size), - size - rounded_down_size); - poison_range(left_redzone, KASAN_ALLOCA_REDZONE_SIZE, + kasan_unpoison((const void *)(addr + rounded_down_size), + size - rounded_down_size); + kasan_poison(left_redzone, KASAN_ALLOCA_REDZONE_SIZE, KASAN_ALLOCA_LEFT); - poison_range(right_redzone, padding_size + KASAN_ALLOCA_REDZONE_SIZE, + kasan_poison(right_redzone, padding_size + KASAN_ALLOCA_REDZONE_SIZE, KASAN_ALLOCA_RIGHT); } EXPORT_SYMBOL(__asan_alloca_poison); @@ -305,7 +305,7 @@ void __asan_allocas_unpoison(const void if (unlikely(!stack_top || stack_top > stack_bottom)) return; - unpoison_range(stack_top, stack_bottom - stack_top); + kasan_unpoison(stack_top, stack_bottom - stack_top); } EXPORT_SYMBOL(__asan_allocas_unpoison); --- a/mm/kasan/kasan.h~kasan-prefix-global-functions-with-kasan_ +++ a/mm/kasan/kasan.h @@ -195,14 +195,14 @@ static inline bool addr_has_metadata(con } /** - * check_memory_region - Check memory region, and report if invalid access. + * kasan_check_range - Check memory region, and report if invalid access. * @addr: the accessed address * @size: the accessed size * @write: true if access is a write access * @ret_ip: return address * @return: true if access was valid, false if invalid */ -bool check_memory_region(unsigned long addr, size_t size, bool write, +bool kasan_check_range(unsigned long addr, size_t size, bool write, unsigned long ret_ip); #else /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ @@ -215,19 +215,19 @@ static inline bool addr_has_metadata(con #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) -void print_tags(u8 addr_tag, const void *addr); +void kasan_print_tags(u8 addr_tag, const void *addr); #else -static inline void print_tags(u8 addr_tag, const void *addr) { } +static inline void kasan_print_tags(u8 addr_tag, const void *addr) { } #endif -void *find_first_bad_addr(void *addr, size_t size); -const char *get_bug_type(struct kasan_access_info *info); -void metadata_fetch_row(char *buffer, void *row); +void *kasan_find_first_bad_addr(void *addr, size_t size); +const char *kasan_get_bug_type(struct kasan_access_info *info); +void kasan_metadata_fetch_row(char *buffer, void *row); #if defined(CONFIG_KASAN_GENERIC) && CONFIG_KASAN_STACK -void print_address_stack_frame(const void *addr); +void kasan_print_address_stack_frame(const void *addr); #else -static inline void print_address_stack_frame(const void *addr) { } +static inline void kasan_print_address_stack_frame(const void *addr) { } #endif bool kasan_report(unsigned long addr, size_t size, @@ -244,13 +244,13 @@ struct kasan_track *kasan_get_free_track #if defined(CONFIG_KASAN_GENERIC) && \ (defined(CONFIG_SLAB) || defined(CONFIG_SLUB)) -bool quarantine_put(struct kmem_cache *cache, void *object); -void quarantine_reduce(void); -void quarantine_remove_cache(struct kmem_cache *cache); +bool kasan_quarantine_put(struct kmem_cache *cache, void *object); +void kasan_quarantine_reduce(void); +void kasan_quarantine_remove_cache(struct kmem_cache *cache); #else -static inline bool quarantine_put(struct kmem_cache *cache, void *object) { return false; } -static inline void quarantine_reduce(void) { } -static inline void quarantine_remove_cache(struct kmem_cache *cache) { } +static inline bool kasan_quarantine_put(struct kmem_cache *cache, void *object) { return false; } +static inline void kasan_quarantine_reduce(void) { } +static inline void kasan_quarantine_remove_cache(struct kmem_cache *cache) { } #endif #ifndef arch_kasan_set_tag @@ -293,28 +293,28 @@ static inline const void *arch_kasan_set #endif /* CONFIG_KASAN_HW_TAGS */ #ifdef CONFIG_KASAN_SW_TAGS -u8 random_tag(void); +u8 kasan_random_tag(void); #elif defined(CONFIG_KASAN_HW_TAGS) -static inline u8 random_tag(void) { return hw_get_random_tag(); } +static inline u8 kasan_random_tag(void) { return hw_get_random_tag(); } #else -static inline u8 random_tag(void) { return 0; } +static inline u8 kasan_random_tag(void) { return 0; } #endif #ifdef CONFIG_KASAN_HW_TAGS -static inline void poison_range(const void *address, size_t size, u8 value) +static inline void kasan_poison(const void *address, size_t size, u8 value) { hw_set_mem_tag_range(kasan_reset_tag(address), round_up(size, KASAN_GRANULE_SIZE), value); } -static inline void unpoison_range(const void *address, size_t size) +static inline void kasan_unpoison(const void *address, size_t size) { hw_set_mem_tag_range(kasan_reset_tag(address), round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); } -static inline bool check_invalid_free(void *addr) +static inline bool kasan_check_invalid_free(void *addr) { u8 ptr_tag = get_tag(addr); u8 mem_tag = hw_get_mem_tag(addr); @@ -325,9 +325,9 @@ static inline bool check_invalid_free(vo #else /* CONFIG_KASAN_HW_TAGS */ -void poison_range(const void *address, size_t size, u8 value); -void unpoison_range(const void *address, size_t size); -bool check_invalid_free(void *addr); +void kasan_poison(const void *address, size_t size, u8 value); +void kasan_unpoison(const void *address, size_t size); +bool kasan_check_invalid_free(void *addr); #endif /* CONFIG_KASAN_HW_TAGS */ --- a/mm/kasan/quarantine.c~kasan-prefix-global-functions-with-kasan_ +++ a/mm/kasan/quarantine.c @@ -168,7 +168,7 @@ static void qlist_free_all(struct qlist_ qlist_init(q); } -bool quarantine_put(struct kmem_cache *cache, void *object) +bool kasan_quarantine_put(struct kmem_cache *cache, void *object) { unsigned long flags; struct qlist_head *q; @@ -184,11 +184,11 @@ bool quarantine_put(struct kmem_cache *c /* * Note: irq must be disabled until after we move the batch to the - * global quarantine. Otherwise quarantine_remove_cache() can miss - * some objects belonging to the cache if they are in our local temp - * list. quarantine_remove_cache() executes on_each_cpu() at the - * beginning which ensures that it either sees the objects in per-cpu - * lists or in the global quarantine. + * global quarantine. Otherwise kasan_quarantine_remove_cache() can + * miss some objects belonging to the cache if they are in our local + * temp list. kasan_quarantine_remove_cache() executes on_each_cpu() + * at the beginning which ensures that it either sees the objects in + * per-cpu lists or in the global quarantine. */ local_irq_save(flags); @@ -222,7 +222,7 @@ bool quarantine_put(struct kmem_cache *c return true; } -void quarantine_reduce(void) +void kasan_quarantine_reduce(void) { size_t total_size, new_quarantine_size, percpu_quarantines; unsigned long flags; @@ -234,7 +234,7 @@ void quarantine_reduce(void) return; /* - * srcu critical section ensures that quarantine_remove_cache() + * srcu critical section ensures that kasan_quarantine_remove_cache() * will not miss objects belonging to the cache while they are in our * local to_free list. srcu is chosen because (1) it gives us private * grace period domain that does not interfere with anything else, @@ -309,15 +309,15 @@ static void per_cpu_remove_cache(void *a } /* Free all quarantined objects belonging to cache. */ -void quarantine_remove_cache(struct kmem_cache *cache) +void kasan_quarantine_remove_cache(struct kmem_cache *cache) { unsigned long flags, i; struct qlist_head to_free = QLIST_INIT; /* * Must be careful to not miss any objects that are being moved from - * per-cpu list to the global quarantine in quarantine_put(), - * nor objects being freed in quarantine_reduce(). on_each_cpu() + * per-cpu list to the global quarantine in kasan_quarantine_put(), + * nor objects being freed in kasan_quarantine_reduce(). on_each_cpu() * achieves the first goal, while synchronize_srcu() achieves the * second. */ --- a/mm/kasan/report.c~kasan-prefix-global-functions-with-kasan_ +++ a/mm/kasan/report.c @@ -61,7 +61,7 @@ __setup("kasan_multi_shot", kasan_set_mu static void print_error_description(struct kasan_access_info *info) { pr_err("BUG: KASAN: %s in %pS\n", - get_bug_type(info), (void *)info->ip); + kasan_get_bug_type(info), (void *)info->ip); if (info->access_size) pr_err("%s of size %zu at addr %px by task %s/%d\n", info->is_write ? "Write" : "Read", info->access_size, @@ -247,7 +247,7 @@ static void print_address_description(vo dump_page(page, "kasan: bad access detected"); } - print_address_stack_frame(addr); + kasan_print_address_stack_frame(addr); } static bool meta_row_is_guilty(const void *row, const void *addr) @@ -293,7 +293,7 @@ static void print_memory_metadata(const * function, because generic functions may try to * access kasan mapping for the passed address. */ - metadata_fetch_row(&metadata[0], row); + kasan_metadata_fetch_row(&metadata[0], row); print_hex_dump(KERN_ERR, buffer, DUMP_PREFIX_NONE, META_BYTES_PER_ROW, 1, @@ -350,7 +350,7 @@ void kasan_report_invalid_free(void *obj start_report(&flags); pr_err("BUG: KASAN: double-free or invalid-free in %pS\n", (void *)ip); - print_tags(tag, object); + kasan_print_tags(tag, object); pr_err("\n"); print_address_description(object, tag); pr_err("\n"); @@ -378,7 +378,8 @@ static void __kasan_report(unsigned long info.access_addr = tagged_addr; if (addr_has_metadata(untagged_addr)) - info.first_bad_addr = find_first_bad_addr(tagged_addr, size); + info.first_bad_addr = + kasan_find_first_bad_addr(tagged_addr, size); else info.first_bad_addr = untagged_addr; info.access_size = size; @@ -389,7 +390,7 @@ static void __kasan_report(unsigned long print_error_description(&info); if (addr_has_metadata(untagged_addr)) - print_tags(get_tag(tagged_addr), info.first_bad_addr); + kasan_print_tags(get_tag(tagged_addr), info.first_bad_addr); pr_err("\n"); if (addr_has_metadata(untagged_addr)) { --- a/mm/kasan/report_generic.c~kasan-prefix-global-functions-with-kasan_ +++ a/mm/kasan/report_generic.c @@ -30,7 +30,7 @@ #include "kasan.h" #include "../slab.h" -void *find_first_bad_addr(void *addr, size_t size) +void *kasan_find_first_bad_addr(void *addr, size_t size) { void *p = addr; @@ -105,7 +105,7 @@ static const char *get_wild_bug_type(str return bug_type; } -const char *get_bug_type(struct kasan_access_info *info) +const char *kasan_get_bug_type(struct kasan_access_info *info) { /* * If access_size is a negative number, then it has reason to be @@ -123,7 +123,7 @@ const char *get_bug_type(struct kasan_ac return get_wild_bug_type(info); } -void metadata_fetch_row(char *buffer, void *row) +void kasan_metadata_fetch_row(char *buffer, void *row) { memcpy(buffer, kasan_mem_to_shadow(row), META_BYTES_PER_ROW); } @@ -263,7 +263,7 @@ static bool __must_check get_address_sta return true; } -void print_address_stack_frame(const void *addr) +void kasan_print_address_stack_frame(const void *addr) { unsigned long offset; const char *frame_descr; --- a/mm/kasan/report_hw_tags.c~kasan-prefix-global-functions-with-kasan_ +++ a/mm/kasan/report_hw_tags.c @@ -15,17 +15,17 @@ #include "kasan.h" -const char *get_bug_type(struct kasan_access_info *info) +const char *kasan_get_bug_type(struct kasan_access_info *info) { return "invalid-access"; } -void *find_first_bad_addr(void *addr, size_t size) +void *kasan_find_first_bad_addr(void *addr, size_t size) { return kasan_reset_tag(addr); } -void metadata_fetch_row(char *buffer, void *row) +void kasan_metadata_fetch_row(char *buffer, void *row) { int i; @@ -33,7 +33,7 @@ void metadata_fetch_row(char *buffer, vo buffer[i] = hw_get_mem_tag(row + i * KASAN_GRANULE_SIZE); } -void print_tags(u8 addr_tag, const void *addr) +void kasan_print_tags(u8 addr_tag, const void *addr) { u8 memory_tag = hw_get_mem_tag((void *)addr); --- a/mm/kasan/report_sw_tags.c~kasan-prefix-global-functions-with-kasan_ +++ a/mm/kasan/report_sw_tags.c @@ -29,7 +29,7 @@ #include "kasan.h" #include "../slab.h" -const char *get_bug_type(struct kasan_access_info *info) +const char *kasan_get_bug_type(struct kasan_access_info *info) { #ifdef CONFIG_KASAN_SW_TAGS_IDENTIFY struct kasan_alloc_meta *alloc_meta; @@ -72,7 +72,7 @@ const char *get_bug_type(struct kasan_ac return "invalid-access"; } -void *find_first_bad_addr(void *addr, size_t size) +void *kasan_find_first_bad_addr(void *addr, size_t size) { u8 tag = get_tag(addr); void *p = kasan_reset_tag(addr); @@ -83,12 +83,12 @@ void *find_first_bad_addr(void *addr, si return p; } -void metadata_fetch_row(char *buffer, void *row) +void kasan_metadata_fetch_row(char *buffer, void *row) { memcpy(buffer, kasan_mem_to_shadow(row), META_BYTES_PER_ROW); } -void print_tags(u8 addr_tag, const void *addr) +void kasan_print_tags(u8 addr_tag, const void *addr) { u8 *shadow = (u8 *)kasan_mem_to_shadow(addr); --- a/mm/kasan/shadow.c~kasan-prefix-global-functions-with-kasan_ +++ a/mm/kasan/shadow.c @@ -27,20 +27,20 @@ bool __kasan_check_read(const volatile void *p, unsigned int size) { - return check_memory_region((unsigned long)p, size, false, _RET_IP_); + return kasan_check_range((unsigned long)p, size, false, _RET_IP_); } EXPORT_SYMBOL(__kasan_check_read); bool __kasan_check_write(const volatile void *p, unsigned int size) { - return check_memory_region((unsigned long)p, size, true, _RET_IP_); + return kasan_check_range((unsigned long)p, size, true, _RET_IP_); } EXPORT_SYMBOL(__kasan_check_write); #undef memset void *memset(void *addr, int c, size_t len) { - if (!check_memory_region((unsigned long)addr, len, true, _RET_IP_)) + if (!kasan_check_range((unsigned long)addr, len, true, _RET_IP_)) return NULL; return __memset(addr, c, len); @@ -50,8 +50,8 @@ void *memset(void *addr, int c, size_t l #undef memmove void *memmove(void *dest, const void *src, size_t len) { - if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) || - !check_memory_region((unsigned long)dest, len, true, _RET_IP_)) + if (!kasan_check_range((unsigned long)src, len, false, _RET_IP_) || + !kasan_check_range((unsigned long)dest, len, true, _RET_IP_)) return NULL; return __memmove(dest, src, len); @@ -61,8 +61,8 @@ void *memmove(void *dest, const void *sr #undef memcpy void *memcpy(void *dest, const void *src, size_t len) { - if (!check_memory_region((unsigned long)src, len, false, _RET_IP_) || - !check_memory_region((unsigned long)dest, len, true, _RET_IP_)) + if (!kasan_check_range((unsigned long)src, len, false, _RET_IP_) || + !kasan_check_range((unsigned long)dest, len, true, _RET_IP_)) return NULL; return __memcpy(dest, src, len); @@ -72,7 +72,7 @@ void *memcpy(void *dest, const void *src * Poisons the shadow memory for 'size' bytes starting from 'addr'. * Memory addresses should be aligned to KASAN_GRANULE_SIZE. */ -void poison_range(const void *address, size_t size, u8 value) +void kasan_poison(const void *address, size_t size, u8 value) { void *shadow_start, *shadow_end; @@ -90,7 +90,7 @@ void poison_range(const void *address, s __memset(shadow_start, value, shadow_end - shadow_start); } -void unpoison_range(const void *address, size_t size) +void kasan_unpoison(const void *address, size_t size) { u8 tag = get_tag(address); @@ -101,7 +101,7 @@ void unpoison_range(const void *address, */ address = kasan_reset_tag(address); - poison_range(address, size, tag); + kasan_poison(address, size, tag); if (size & KASAN_GRANULE_MASK) { u8 *shadow = (u8 *)kasan_mem_to_shadow(address + size); @@ -286,7 +286,7 @@ int kasan_populate_vmalloc(unsigned long * // vmalloc() allocates memory * // let a = area->addr * // we reach kasan_populate_vmalloc - * // and call unpoison_range: + * // and call kasan_unpoison: * STORE shadow(a), unpoison_val * ... * STORE shadow(a+99), unpoison_val x = LOAD p @@ -321,7 +321,7 @@ void kasan_poison_vmalloc(const void *st return; size = round_up(size, KASAN_GRANULE_SIZE); - poison_range(start, size, KASAN_VMALLOC_INVALID); + kasan_poison(start, size, KASAN_VMALLOC_INVALID); } void kasan_unpoison_vmalloc(const void *start, unsigned long size) @@ -329,7 +329,7 @@ void kasan_unpoison_vmalloc(const void * if (!is_vmalloc_or_module_addr(start)) return; - unpoison_range(start, size); + kasan_unpoison(start, size); } static int kasan_depopulate_vmalloc_pte(pte_t *ptep, unsigned long addr, --- a/mm/kasan/sw_tags.c~kasan-prefix-global-functions-with-kasan_ +++ a/mm/kasan/sw_tags.c @@ -57,7 +57,7 @@ void __init kasan_init_sw_tags(void) * sequence has in fact positive effect, since interrupts that randomly skew * PRNG at unpredictable points do only good. */ -u8 random_tag(void) +u8 kasan_random_tag(void) { u32 state = this_cpu_read(prng_state); @@ -67,7 +67,7 @@ u8 random_tag(void) return (u8)(state % (KASAN_TAG_MAX + 1)); } -bool check_memory_region(unsigned long addr, size_t size, bool write, +bool kasan_check_range(unsigned long addr, size_t size, bool write, unsigned long ret_ip) { u8 tag; @@ -118,7 +118,7 @@ bool check_memory_region(unsigned long a return true; } -bool check_invalid_free(void *addr) +bool kasan_check_invalid_free(void *addr) { u8 tag = get_tag(addr); u8 shadow_byte = READ_ONCE(*(u8 *)kasan_mem_to_shadow(kasan_reset_tag(addr))); @@ -130,12 +130,12 @@ bool check_invalid_free(void *addr) #define DEFINE_HWASAN_LOAD_STORE(size) \ void __hwasan_load##size##_noabort(unsigned long addr) \ { \ - check_memory_region(addr, size, false, _RET_IP_); \ + kasan_check_range(addr, size, false, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_load##size##_noabort); \ void __hwasan_store##size##_noabort(unsigned long addr) \ { \ - check_memory_region(addr, size, true, _RET_IP_); \ + kasan_check_range(addr, size, true, _RET_IP_); \ } \ EXPORT_SYMBOL(__hwasan_store##size##_noabort) @@ -147,19 +147,19 @@ DEFINE_HWASAN_LOAD_STORE(16); void __hwasan_loadN_noabort(unsigned long addr, unsigned long size) { - check_memory_region(addr, size, false, _RET_IP_); + kasan_check_range(addr, size, false, _RET_IP_); } EXPORT_SYMBOL(__hwasan_loadN_noabort); void __hwasan_storeN_noabort(unsigned long addr, unsigned long size) { - check_memory_region(addr, size, true, _RET_IP_); + kasan_check_range(addr, size, true, _RET_IP_); } EXPORT_SYMBOL(__hwasan_storeN_noabort); void __hwasan_tag_memory(unsigned long addr, u8 tag, unsigned long size) { - poison_range((void *)addr, size, tag); + kasan_poison((void *)addr, size, tag); } EXPORT_SYMBOL(__hwasan_tag_memory); --- a/tools/objtool/check.c~kasan-prefix-global-functions-with-kasan_ +++ a/tools/objtool/check.c @@ -666,7 +666,7 @@ static void add_ignores(struct objtool_f static const char *uaccess_safe_builtin[] = { /* KASAN */ "kasan_report", - "check_memory_region", + "kasan_check_range", /* KASAN out-of-line */ "__asan_loadN_noabort", "__asan_load1_noabort", From patchwork Wed Feb 24 20:05:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102585 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A3BDC433E0 for ; Wed, 24 Feb 2021 20:05:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E094F64EC3 for ; Wed, 24 Feb 2021 20:05:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E094F64EC3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 79FC96B00B3; Wed, 24 Feb 2021 15:05:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 74F778D0014; Wed, 24 Feb 2021 15:05:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 63D848D000B; Wed, 24 Feb 2021 15:05:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0237.hostedemail.com [216.40.44.237]) by kanga.kvack.org (Postfix) with ESMTP id 4F1306B00B3 for ; Wed, 24 Feb 2021 15:05:12 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1456F184E4E53 for ; Wed, 24 Feb 2021 20:05:12 +0000 (UTC) X-FDA: 77854240464.02.0AF832B Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP id 26E5DDC for ; Wed, 24 Feb 2021 20:05:10 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C403A64E24; Wed, 24 Feb 2021 20:05:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197110; bh=u/ZZUFu5EdOZyWmDwDctH8QWsfT8Dm6bHKUUj49CqMY=; h=Date:From:To:Subject:In-Reply-To:From; b=n47u80MmqZdD5pHozDI53r/rsVKwVNnaUHXF+X3KXHFFCcY2f3mLpKhYHQ+EFTxV5 1e4VacEg3AmHT2yG1D80EVqXJcpMpd8f+ZFryGqbZZmlH/tOXgoCqcMkghOYL1U9BI PvvGH9M9F8XkxdCHZqYJXqlNVtLuXbcMp/CGqXLY= Date: Wed, 24 Feb 2021 12:05:09 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, pcc@google.com, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 087/173] kasan: clarify HW_TAGS impact on TBI Message-ID: <20210224200509.mPXocwY3C%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 9isd9spk7iqmnohamfm1r11y1jjfykfk X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 26E5DDC Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf04; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197110-551797 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: clarify HW_TAGS impact on TBI Mention in the documentation that enabling CONFIG_KASAN_HW_TAGS always results in in-kernel TBI (Top Byte Ignore) being enabled. Also do a few minor documentation cleanups. Link: https://linux-review.googlesource.com/id/Iba2a6697e3c6304cb53f89ec61dedc77fa29e3ae Link: https://lkml.kernel.org/r/3b4ea6875bb14d312092ad14ac55cb456c83c08e.1610733117.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Peter Collingbourne Cc: Vincenzo Frascino Cc: Will Deacon Signed-off-by: Andrew Morton --- Documentation/dev-tools/kasan.rst | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) --- a/Documentation/dev-tools/kasan.rst~kasan-clarify-hw_tags-impact-on-tbi +++ a/Documentation/dev-tools/kasan.rst @@ -147,15 +147,14 @@ negative values to distinguish between d like redzones or freed memory (see mm/kasan/kasan.h). In the report above the arrows point to the shadow byte 03, which means that -the accessed address is partially accessible. - -For tag-based KASAN this last report section shows the memory tags around the -accessed address (see `Implementation details`_ section). +the accessed address is partially accessible. For tag-based KASAN modes this +last report section shows the memory tags around the accessed address +(see the `Implementation details`_ section). Boot parameters ~~~~~~~~~~~~~~~ -Hardware tag-based KASAN mode (see the section about different mode below) is +Hardware tag-based KASAN mode (see the section about various modes below) is intended for use in production as a security mitigation. Therefore it supports boot parameters that allow to disable KASAN competely or otherwise control particular KASAN features. @@ -289,6 +288,13 @@ reserved to tag freed memory regions. Hardware tag-based KASAN currently only supports tagging of kmem_cache_alloc/kmalloc and page_alloc memory. +If the hardware doesn't support MTE (pre ARMv8.5), hardware tag-based KASAN +won't be enabled. In this case all boot parameters are ignored. + +Note, that enabling CONFIG_KASAN_HW_TAGS always results in in-kernel TBI being +enabled. Even when kasan.mode=off is provided, or when the hardware doesn't +support MTE (but supports TBI). + What memory accesses are sanitised by KASAN? -------------------------------------------- From patchwork Wed Feb 24 20:05:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102587 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C4DBC433DB for ; Wed, 24 Feb 2021 20:05:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E19BB64F1F for ; Wed, 24 Feb 2021 20:05:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E19BB64F1F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6B57F8D000B; Wed, 24 Feb 2021 15:05:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6665B6B00B6; Wed, 24 Feb 2021 15:05:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 57D548D000B; Wed, 24 Feb 2021 15:05:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 437B06B00B5 for ; Wed, 24 Feb 2021 15:05:16 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 095C7184DE0BC for ; Wed, 24 Feb 2021 20:05:16 +0000 (UTC) X-FDA: 77854240632.30.BD58929 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf05.hostedemail.com (Postfix) with ESMTP id 6A469E00010A for ; Wed, 24 Feb 2021 20:05:14 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C14F464EC3; Wed, 24 Feb 2021 20:05:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197114; bh=i5A/n84G6tQugekUKzrpdqWP1NQNeP29K8UiGpeGJ3w=; h=Date:From:To:Subject:In-Reply-To:From; b=e9gJ9CPXrFNg/hbjUJSLDu84ng4T7AAmEXVvBa+EL3iiUr6GQePDUFf5AQ9cuGr6Q gK0QPdkh9Vy6TBGQX2xGCGg5kCswYE3N9qte8TG1GaSmcFcJZmZAPnvZ0694Rx4N9/ kIlKrafWCWMqEfC6OkeDtEYF4vxEn/f/4PClvkAU= Date: Wed, 24 Feb 2021 12:05:13 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, pcc@google.com, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 088/173] kasan: clean up comments in tests Message-ID: <20210224200513.XuFNJsijR%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 6A469E00010A X-Stat-Signature: 8bgb1jzmdbiegbuf4re7mdnxiqqxyw6u Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf05; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197114-937117 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: clean up comments in tests Clarify and update comments in KASAN tests. Link: https://linux-review.googlesource.com/id/I6c816c51fa1e0eb7aa3dead6bda1f339d2af46c8 Link: https://lkml.kernel.org/r/ba6db104d53ae0e3796f80ef395f6873c1c1282f.1610733117.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Peter Collingbourne Cc: Vincenzo Frascino Cc: Will Deacon Signed-off-by: Andrew Morton --- lib/test_kasan.c | 59 +++++++++++++++++++++++--------------- lib/test_kasan_module.c | 5 +-- 2 files changed, 39 insertions(+), 25 deletions(-) --- a/lib/test_kasan.c~kasan-clean-up-comments-in-tests +++ a/lib/test_kasan.c @@ -28,10 +28,9 @@ #define OOB_TAG_OFF (IS_ENABLED(CONFIG_KASAN_GENERIC) ? 0 : KASAN_GRANULE_SIZE) /* - * We assign some test results to these globals to make sure the tests - * are not eliminated as dead code. + * Some tests use these global variables to store return values from function + * calls that could otherwise be eliminated by the compiler as dead code. */ - void *kasan_ptr_result; int kasan_int_result; @@ -39,14 +38,13 @@ static struct kunit_resource resource; static struct kunit_kasan_expectation fail_data; static bool multishot; +/* + * Temporarily enable multi-shot mode. Otherwise, KASAN would only report the + * first detected bug and panic the kernel if panic_on_warn is enabled. + */ static int kasan_test_init(struct kunit *test) { - /* - * Temporarily enable multi-shot mode and set panic_on_warn=0. - * Otherwise, we'd only get a report for the first case. - */ multishot = kasan_save_enable_multi_shot(); - return 0; } @@ -56,12 +54,12 @@ static void kasan_test_exit(struct kunit } /** - * KUNIT_EXPECT_KASAN_FAIL() - Causes a test failure when the expression does - * not cause a KASAN error. This uses a KUnit resource named "kasan_data." Do - * Do not use this name for a KUnit resource outside here. - * + * KUNIT_EXPECT_KASAN_FAIL() - check that the executed expression produces a + * KASAN report; causes a test failure otherwise. This relies on a KUnit + * resource named "kasan_data". Do not use this name for KUnit resources + * outside of KASAN tests. */ -#define KUNIT_EXPECT_KASAN_FAIL(test, condition) do { \ +#define KUNIT_EXPECT_KASAN_FAIL(test, expression) do { \ fail_data.report_expected = true; \ fail_data.report_found = false; \ kunit_add_named_resource(test, \ @@ -69,7 +67,7 @@ static void kasan_test_exit(struct kunit NULL, \ &resource, \ "kasan_data", &fail_data); \ - condition; \ + expression; \ KUNIT_EXPECT_EQ(test, \ fail_data.report_expected, \ fail_data.report_found); \ @@ -121,7 +119,8 @@ static void kmalloc_pagealloc_oob_right( return; } - /* Allocate a chunk that does not fit into a SLUB cache to trigger + /* + * Allocate a chunk that does not fit into a SLUB cache to trigger * the page allocator fallback. */ ptr = kmalloc(size, GFP_KERNEL); @@ -168,7 +167,9 @@ static void kmalloc_large_oob_right(stru { char *ptr; size_t size = KMALLOC_MAX_CACHE_SIZE - 256; - /* Allocate a chunk that is large enough, but still fits into a slab + + /* + * Allocate a chunk that is large enough, but still fits into a slab * and does not trigger the page allocator fallback in SLUB. */ ptr = kmalloc(size, GFP_KERNEL); @@ -469,10 +470,13 @@ static void ksize_unpoisons_memory(struc ptr = kmalloc(size, GFP_KERNEL); KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr); real_size = ksize(ptr); - /* This access doesn't trigger an error. */ + + /* This access shouldn't trigger a KASAN report. */ ptr[size] = 'x'; - /* This one does. */ + + /* This one must. */ KUNIT_EXPECT_KASAN_FAIL(test, ptr[real_size] = 'y'); + kfree(ptr); } @@ -568,7 +572,7 @@ static void kmem_cache_invalid_free(stru return; } - /* Trigger invalid free, the object doesn't get freed */ + /* Trigger invalid free, the object doesn't get freed. */ KUNIT_EXPECT_KASAN_FAIL(test, kmem_cache_free(cache, p + 1)); /* @@ -585,7 +589,10 @@ static void kasan_memchr(struct kunit *t char *ptr; size_t size = 24; - /* See https://bugzilla.kernel.org/show_bug.cgi?id=206337 */ + /* + * str* functions are not instrumented with CONFIG_AMD_MEM_ENCRYPT. + * See https://bugzilla.kernel.org/show_bug.cgi?id=206337 for details. + */ if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { kunit_info(test, "str* functions are not instrumented with CONFIG_AMD_MEM_ENCRYPT"); @@ -610,7 +617,10 @@ static void kasan_memcmp(struct kunit *t size_t size = 24; int arr[9]; - /* See https://bugzilla.kernel.org/show_bug.cgi?id=206337 */ + /* + * str* functions are not instrumented with CONFIG_AMD_MEM_ENCRYPT. + * See https://bugzilla.kernel.org/show_bug.cgi?id=206337 for details. + */ if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { kunit_info(test, "str* functions are not instrumented with CONFIG_AMD_MEM_ENCRYPT"); @@ -634,7 +644,10 @@ static void kasan_strings(struct kunit * char *ptr; size_t size = 24; - /* See https://bugzilla.kernel.org/show_bug.cgi?id=206337 */ + /* + * str* functions are not instrumented with CONFIG_AMD_MEM_ENCRYPT. + * See https://bugzilla.kernel.org/show_bug.cgi?id=206337 for details. + */ if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { kunit_info(test, "str* functions are not instrumented with CONFIG_AMD_MEM_ENCRYPT"); @@ -706,7 +719,7 @@ static void kasan_bitops_generic(struct } /* - * Allocate 1 more byte, which causes kzalloc to round up to 16-bytes; + * Allocate 1 more byte, which causes kzalloc to round up to 16 bytes; * this way we do not actually corrupt other memory. */ bits = kzalloc(sizeof(*bits) + 1, GFP_KERNEL); --- a/lib/test_kasan_module.c~kasan-clean-up-comments-in-tests +++ a/lib/test_kasan_module.c @@ -123,8 +123,9 @@ static noinline void __init kasan_workqu static int __init test_kasan_module_init(void) { /* - * Temporarily enable multi-shot mode. Otherwise, we'd only get a - * report for the first case. + * Temporarily enable multi-shot mode. Otherwise, KASAN would only + * report the first detected bug and panic the kernel if panic_on_warn + * is enabled. */ bool multishot = kasan_save_enable_multi_shot(); From patchwork Wed Feb 24 20:05:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102589 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56634C433DB for ; Wed, 24 Feb 2021 20:05:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DC55B64E20 for ; Wed, 24 Feb 2021 20:05:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DC55B64E20 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7785C6B00B6; Wed, 24 Feb 2021 15:05:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 72A186B00B7; Wed, 24 Feb 2021 15:05:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 640FD8D0014; Wed, 24 Feb 2021 15:05:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0236.hostedemail.com [216.40.44.236]) by kanga.kvack.org (Postfix) with ESMTP id 4F0356B00B6 for ; Wed, 24 Feb 2021 15:05:20 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 1A1F41730876 for ; Wed, 24 Feb 2021 20:05:20 +0000 (UTC) X-FDA: 77854240800.05.4AFF664 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP id 76FFCE00011A for ; Wed, 24 Feb 2021 20:05:19 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B76D064F38; Wed, 24 Feb 2021 20:05:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197118; bh=lKv1jnq85qrOt+QUT92XssPGkS3N2MFnhhedvJdORTc=; h=Date:From:To:Subject:In-Reply-To:From; b=fSebqP2tyetcjFazjINUdG+ox2GsNRkJqbECZk1U9PSuzeXIpiFKiY9AYoi+Lgo6d BWzbwn85s2Xmm50aIMTEhWLSXQRLixPx8r8WqlBoq/wTgjZr6W8lW+OG1KKiFBSd+Z w7BoYerR0sxp/XBaMBL+YgnpElrF2blQFpPE4IAs= Date: Wed, 24 Feb 2021 12:05:17 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, pcc@google.com, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 089/173] kasan: add macros to simplify checking test constraints Message-ID: <20210224200517.cxNN7b75E%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: ffsbnqc3swo1ajatfdjuydac5y1fqndu X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 76FFCE00011A Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf30; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197119-50687 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: add macros to simplify checking test constraints Some KASAN tests require specific kernel configs to be enabled. Instead of copy-pasting the checks for these configs add a few helper macros and use them. Link: https://linux-review.googlesource.com/id/I237484a7fddfedf4a4aae9cc61ecbcdbe85a0a63 Link: https://lkml.kernel.org/r/6a0fcdb9676b7e869cfc415893ede12d916c246c.1610733117.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Suggested-by: Alexander Potapenko Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Peter Collingbourne Cc: Vincenzo Frascino Cc: Will Deacon Signed-off-by: Andrew Morton --- lib/test_kasan.c | 101 +++++++++++++-------------------------------- 1 file changed, 31 insertions(+), 70 deletions(-) --- a/lib/test_kasan.c~kasan-add-macros-to-simplify-checking-test-constraints +++ a/lib/test_kasan.c @@ -73,6 +73,20 @@ static void kasan_test_exit(struct kunit fail_data.report_found); \ } while (0) +#define KASAN_TEST_NEEDS_CONFIG_ON(test, config) do { \ + if (!IS_ENABLED(config)) { \ + kunit_info((test), "skipping, " #config " required"); \ + return; \ + } \ +} while (0) + +#define KASAN_TEST_NEEDS_CONFIG_OFF(test, config) do { \ + if (IS_ENABLED(config)) { \ + kunit_info((test), "skipping, " #config " enabled"); \ + return; \ + } \ +} while (0) + static void kmalloc_oob_right(struct kunit *test) { char *ptr; @@ -114,10 +128,7 @@ static void kmalloc_pagealloc_oob_right( char *ptr; size_t size = KMALLOC_MAX_CACHE_SIZE + 10; - if (!IS_ENABLED(CONFIG_SLUB)) { - kunit_info(test, "CONFIG_SLUB is not enabled."); - return; - } + KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_SLUB); /* * Allocate a chunk that does not fit into a SLUB cache to trigger @@ -135,10 +146,7 @@ static void kmalloc_pagealloc_uaf(struct char *ptr; size_t size = KMALLOC_MAX_CACHE_SIZE + 10; - if (!IS_ENABLED(CONFIG_SLUB)) { - kunit_info(test, "CONFIG_SLUB is not enabled."); - return; - } + KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_SLUB); ptr = kmalloc(size, GFP_KERNEL); KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr); @@ -152,10 +160,7 @@ static void kmalloc_pagealloc_invalid_fr char *ptr; size_t size = KMALLOC_MAX_CACHE_SIZE + 10; - if (!IS_ENABLED(CONFIG_SLUB)) { - kunit_info(test, "CONFIG_SLUB is not enabled."); - return; - } + KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_SLUB); ptr = kmalloc(size, GFP_KERNEL); KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr); @@ -218,10 +223,7 @@ static void kmalloc_oob_16(struct kunit } *ptr1, *ptr2; /* This test is specifically crafted for the generic mode. */ - if (!IS_ENABLED(CONFIG_KASAN_GENERIC)) { - kunit_info(test, "CONFIG_KASAN_GENERIC required\n"); - return; - } + KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_KASAN_GENERIC); ptr1 = kmalloc(sizeof(*ptr1) - 3, GFP_KERNEL); KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr1); @@ -454,10 +456,7 @@ static void kasan_global_oob(struct kuni char *p = &global_array[ARRAY_SIZE(global_array) + i]; /* Only generic mode instruments globals. */ - if (!IS_ENABLED(CONFIG_KASAN_GENERIC)) { - kunit_info(test, "CONFIG_KASAN_GENERIC required"); - return; - } + KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_KASAN_GENERIC); KUNIT_EXPECT_KASAN_FAIL(test, *(volatile char *)p); } @@ -486,10 +485,7 @@ static void kasan_stack_oob(struct kunit volatile int i = OOB_TAG_OFF; char *p = &stack_array[ARRAY_SIZE(stack_array) + i]; - if (!IS_ENABLED(CONFIG_KASAN_STACK)) { - kunit_info(test, "CONFIG_KASAN_STACK is not enabled"); - return; - } + KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_KASAN_STACK); KUNIT_EXPECT_KASAN_FAIL(test, *(volatile char *)p); } @@ -501,15 +497,8 @@ static void kasan_alloca_oob_left(struct char *p = alloca_array - 1; /* Only generic mode instruments dynamic allocas. */ - if (!IS_ENABLED(CONFIG_KASAN_GENERIC)) { - kunit_info(test, "CONFIG_KASAN_GENERIC required"); - return; - } - - if (!IS_ENABLED(CONFIG_KASAN_STACK)) { - kunit_info(test, "CONFIG_KASAN_STACK is not enabled"); - return; - } + KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_KASAN_GENERIC); + KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_KASAN_STACK); KUNIT_EXPECT_KASAN_FAIL(test, *(volatile char *)p); } @@ -521,15 +510,8 @@ static void kasan_alloca_oob_right(struc char *p = alloca_array + i; /* Only generic mode instruments dynamic allocas. */ - if (!IS_ENABLED(CONFIG_KASAN_GENERIC)) { - kunit_info(test, "CONFIG_KASAN_GENERIC required"); - return; - } - - if (!IS_ENABLED(CONFIG_KASAN_STACK)) { - kunit_info(test, "CONFIG_KASAN_STACK is not enabled"); - return; - } + KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_KASAN_GENERIC); + KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_KASAN_STACK); KUNIT_EXPECT_KASAN_FAIL(test, *(volatile char *)p); } @@ -593,11 +575,7 @@ static void kasan_memchr(struct kunit *t * str* functions are not instrumented with CONFIG_AMD_MEM_ENCRYPT. * See https://bugzilla.kernel.org/show_bug.cgi?id=206337 for details. */ - if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { - kunit_info(test, - "str* functions are not instrumented with CONFIG_AMD_MEM_ENCRYPT"); - return; - } + KASAN_TEST_NEEDS_CONFIG_OFF(test, CONFIG_AMD_MEM_ENCRYPT); if (OOB_TAG_OFF) size = round_up(size, OOB_TAG_OFF); @@ -621,11 +599,7 @@ static void kasan_memcmp(struct kunit *t * str* functions are not instrumented with CONFIG_AMD_MEM_ENCRYPT. * See https://bugzilla.kernel.org/show_bug.cgi?id=206337 for details. */ - if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { - kunit_info(test, - "str* functions are not instrumented with CONFIG_AMD_MEM_ENCRYPT"); - return; - } + KASAN_TEST_NEEDS_CONFIG_OFF(test, CONFIG_AMD_MEM_ENCRYPT); if (OOB_TAG_OFF) size = round_up(size, OOB_TAG_OFF); @@ -648,11 +622,7 @@ static void kasan_strings(struct kunit * * str* functions are not instrumented with CONFIG_AMD_MEM_ENCRYPT. * See https://bugzilla.kernel.org/show_bug.cgi?id=206337 for details. */ - if (IS_ENABLED(CONFIG_AMD_MEM_ENCRYPT)) { - kunit_info(test, - "str* functions are not instrumented with CONFIG_AMD_MEM_ENCRYPT"); - return; - } + KASAN_TEST_NEEDS_CONFIG_OFF(test, CONFIG_AMD_MEM_ENCRYPT); ptr = kmalloc(size, GFP_KERNEL | __GFP_ZERO); KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr); @@ -713,10 +683,7 @@ static void kasan_bitops_generic(struct long *bits; /* This test is specifically crafted for the generic mode. */ - if (!IS_ENABLED(CONFIG_KASAN_GENERIC)) { - kunit_info(test, "CONFIG_KASAN_GENERIC required\n"); - return; - } + KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_KASAN_GENERIC); /* * Allocate 1 more byte, which causes kzalloc to round up to 16 bytes; @@ -744,11 +711,8 @@ static void kasan_bitops_tags(struct kun { long *bits; - /* This test is specifically crafted for the tag-based mode. */ - if (IS_ENABLED(CONFIG_KASAN_GENERIC)) { - kunit_info(test, "CONFIG_KASAN_SW_TAGS required\n"); - return; - } + /* This test is specifically crafted for tag-based modes. */ + KASAN_TEST_NEEDS_CONFIG_OFF(test, CONFIG_KASAN_GENERIC); /* Allocation size will be rounded to up granule size, which is 16. */ bits = kzalloc(sizeof(*bits), GFP_KERNEL); @@ -777,10 +741,7 @@ static void vmalloc_oob(struct kunit *te { void *area; - if (!IS_ENABLED(CONFIG_KASAN_VMALLOC)) { - kunit_info(test, "CONFIG_KASAN_VMALLOC is not enabled."); - return; - } + KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_KASAN_VMALLOC); /* * We have to be careful not to hit the guard page. From patchwork Wed Feb 24 20:05:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102591 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CA605C433E0 for ; Wed, 24 Feb 2021 20:05:27 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5EE1A64E7A for ; Wed, 24 Feb 2021 20:05:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5EE1A64E7A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E072D6B00B8; Wed, 24 Feb 2021 15:05:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D6AFE6B00B9; Wed, 24 Feb 2021 15:05:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C7FC18D0014; Wed, 24 Feb 2021 15:05:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0070.hostedemail.com [216.40.44.70]) by kanga.kvack.org (Postfix) with ESMTP id B084D6B00B8 for ; Wed, 24 Feb 2021 15:05:26 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 6A92E184F7329 for ; Wed, 24 Feb 2021 20:05:26 +0000 (UTC) X-FDA: 77854241052.26.7D57B8C Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id A58A9F7 for ; Wed, 24 Feb 2021 20:05:18 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8BF5964E60; Wed, 24 Feb 2021 20:05:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197123; bh=+FzytRtLq3YZHgN4z3guncsIOKjp0EfaPXnD+tn4AoQ=; h=Date:From:To:Subject:In-Reply-To:From; b=emnCzvMMwxpHZJ2YGoTqVoI45M6hNlAqqKGKsXUdCX2ICvpnx3hDWZ+UsrttJVIkt agXHYohcEkkBLZjow0YLXMusz053IEGGQI64KhH7DBRIS+IE287iS6KUbpeMfF9Wy/ CzDvr+YbH8xWJtO6XXFR+gMA+LtOj84+7safxhik= Date: Wed, 24 Feb 2021 12:05:21 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, arnd@arndb.de, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, pcc@google.com, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 090/173] kasan: add match-all tag tests Message-ID: <20210224200521.x30tGzcEG%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: A58A9F7 X-Stat-Signature: 4zzsqwkxwp9utpg51umqnbtgxi53uj4h Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf12; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197118-743837 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: add match-all tag tests Add 3 new tests for tag-based KASAN modes: 1. Check that match-all pointer tag is not assigned randomly. 2. Check that 0xff works as a match-all pointer tag. 3. Check that there are no match-all memory tags. Note, that test #3 causes a significant number (255) of KASAN reports to be printed during execution for the SW_TAGS mode. [arnd@arndb.de: export kasan_poison] Link: https://lkml.kernel.org/r/20210125112831.2156212-1-arnd@kernel.org [akpm@linux-foundation.org: s/EXPORT_SYMBOL_GPL/EXPORT_SYMBOL/, per Andrey] Link: https://linux-review.googlesource.com/id/I78f1375efafa162b37f3abcb2c5bc2f3955dfd8e Link: https://lkml.kernel.org/r/da841a5408e2204bf25f3b23f70540a65844e8a4.1610733117.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Signed-off-by: Arnd Bergmann Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Peter Collingbourne Cc: Vincenzo Frascino Cc: Will Deacon Signed-off-by: Andrew Morton --- lib/test_kasan.c | 92 ++++++++++++++++++++++++++++++++++++++++++++ mm/kasan/kasan.h | 6 ++ mm/kasan/shadow.c | 1 3 files changed, 99 insertions(+) --- a/lib/test_kasan.c~kasan-add-match-all-tag-tests +++ a/lib/test_kasan.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -754,6 +755,94 @@ static void vmalloc_oob(struct kunit *te vfree(area); } +/* + * Check that the assigned pointer tag falls within the [KASAN_TAG_MIN, + * KASAN_TAG_KERNEL) range (note: excluding the match-all tag) for tag-based + * modes. + */ +static void match_all_not_assigned(struct kunit *test) +{ + char *ptr; + struct page *pages; + int i, size, order; + + KASAN_TEST_NEEDS_CONFIG_OFF(test, CONFIG_KASAN_GENERIC); + + for (i = 0; i < 256; i++) { + size = (get_random_int() % 1024) + 1; + ptr = kmalloc(size, GFP_KERNEL); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr); + KUNIT_EXPECT_GE(test, (u8)get_tag(ptr), (u8)KASAN_TAG_MIN); + KUNIT_EXPECT_LT(test, (u8)get_tag(ptr), (u8)KASAN_TAG_KERNEL); + kfree(ptr); + } + + for (i = 0; i < 256; i++) { + order = (get_random_int() % 4) + 1; + pages = alloc_pages(GFP_KERNEL, order); + ptr = page_address(pages); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr); + KUNIT_EXPECT_GE(test, (u8)get_tag(ptr), (u8)KASAN_TAG_MIN); + KUNIT_EXPECT_LT(test, (u8)get_tag(ptr), (u8)KASAN_TAG_KERNEL); + free_pages((unsigned long)ptr, order); + } +} + +/* Check that 0xff works as a match-all pointer tag for tag-based modes. */ +static void match_all_ptr_tag(struct kunit *test) +{ + char *ptr; + u8 tag; + + KASAN_TEST_NEEDS_CONFIG_OFF(test, CONFIG_KASAN_GENERIC); + + ptr = kmalloc(128, GFP_KERNEL); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr); + + /* Backup the assigned tag. */ + tag = get_tag(ptr); + KUNIT_EXPECT_NE(test, tag, (u8)KASAN_TAG_KERNEL); + + /* Reset the tag to 0xff.*/ + ptr = set_tag(ptr, KASAN_TAG_KERNEL); + + /* This access shouldn't trigger a KASAN report. */ + *ptr = 0; + + /* Recover the pointer tag and free. */ + ptr = set_tag(ptr, tag); + kfree(ptr); +} + +/* Check that there are no match-all memory tags for tag-based modes. */ +static void match_all_mem_tag(struct kunit *test) +{ + char *ptr; + int tag; + + KASAN_TEST_NEEDS_CONFIG_OFF(test, CONFIG_KASAN_GENERIC); + + ptr = kmalloc(128, GFP_KERNEL); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr); + KUNIT_EXPECT_NE(test, (u8)get_tag(ptr), (u8)KASAN_TAG_KERNEL); + + /* For each possible tag value not matching the pointer tag. */ + for (tag = KASAN_TAG_MIN; tag <= KASAN_TAG_KERNEL; tag++) { + if (tag == get_tag(ptr)) + continue; + + /* Mark the first memory granule with the chosen memory tag. */ + kasan_poison(ptr, KASAN_GRANULE_SIZE, (u8)tag); + + /* This access must cause a KASAN report. */ + KUNIT_EXPECT_KASAN_FAIL(test, *ptr = 0); + } + + /* Recover the memory tag and free. */ + kasan_poison(ptr, KASAN_GRANULE_SIZE, get_tag(ptr)); + kfree(ptr); +} + static struct kunit_case kasan_kunit_test_cases[] = { KUNIT_CASE(kmalloc_oob_right), KUNIT_CASE(kmalloc_oob_left), @@ -793,6 +882,9 @@ static struct kunit_case kasan_kunit_tes KUNIT_CASE(kasan_bitops_tags), KUNIT_CASE(kmalloc_double_kzfree), KUNIT_CASE(vmalloc_oob), + KUNIT_CASE(match_all_not_assigned), + KUNIT_CASE(match_all_ptr_tag), + KUNIT_CASE(match_all_mem_tag), {} }; --- a/mm/kasan/kasan.h~kasan-add-match-all-tag-tests +++ a/mm/kasan/kasan.h @@ -36,6 +36,12 @@ extern bool kasan_flag_panic __ro_after_ #define KASAN_TAG_INVALID 0xFE /* inaccessible memory tag */ #define KASAN_TAG_MAX 0xFD /* maximum value for random tags */ +#ifdef CONFIG_KASAN_HW_TAGS +#define KASAN_TAG_MIN 0xF0 /* mimimum value for random tags */ +#else +#define KASAN_TAG_MIN 0x00 /* mimimum value for random tags */ +#endif + #ifdef CONFIG_KASAN_GENERIC #define KASAN_FREE_PAGE 0xFF /* page was freed */ #define KASAN_PAGE_REDZONE 0xFE /* redzone for kmalloc_large allocations */ --- a/mm/kasan/shadow.c~kasan-add-match-all-tag-tests +++ a/mm/kasan/shadow.c @@ -89,6 +89,7 @@ void kasan_poison(const void *address, s __memset(shadow_start, value, shadow_end - shadow_start); } +EXPORT_SYMBOL(kasan_poison); void kasan_unpoison(const void *address, size_t size) { From patchwork Wed Feb 24 20:05:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102593 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B588C433E0 for ; Wed, 24 Feb 2021 20:05:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E17CA64E7A for ; Wed, 24 Feb 2021 20:05:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E17CA64E7A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6DD516B00BA; Wed, 24 Feb 2021 15:05:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6B6FD6B00BB; Wed, 24 Feb 2021 15:05:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F1C58D0014; Wed, 24 Feb 2021 15:05:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0043.hostedemail.com [216.40.44.43]) by kanga.kvack.org (Postfix) with ESMTP id 48E166B00BA for ; Wed, 24 Feb 2021 15:05:29 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id CF2B918503B2C for ; Wed, 24 Feb 2021 20:05:28 +0000 (UTC) X-FDA: 77854241136.23.B79DB1B Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf16.hostedemail.com (Postfix) with ESMTP id A5DE580192ED for ; Wed, 24 Feb 2021 20:05:27 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 95BF364E09; Wed, 24 Feb 2021 20:05:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197127; bh=goSuvaeD6f2xCJ1ALHPL7tCZ7vOQaiFPZJ0++COoln8=; h=Date:From:To:Subject:In-Reply-To:From; b=JSnvZ97rb9bGzuc/7OdIaROep115GbC83A9WV0sbB04odeVS7d9ZNGvkjeNrptVQ5 7k11iJa/LVZH/dqW6NR2KX7ksXeda1TDgr0BFDM1Z8GIJtnV4xWlOwbqGu0PUXQj76 5n67JbV9L0UDujppxI8Wp32fiHk0ZtUfeK7w6fps= Date: Wed, 24 Feb 2021 12:05:26 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, pcc@google.com, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 091/173] kasan, arm64: allow using KUnit tests with HW_TAGS mode Message-ID: <20210224200526.338HIq6nE%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: qprg9xwmmps8cop39e5158hdiakpydfx X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: A5DE580192ED Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197127-370778 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan, arm64: allow using KUnit tests with HW_TAGS mode On a high level, this patch allows running KUnit KASAN tests with the hardware tag-based KASAN mode. Internally, this change reenables tag checking at the end of each KASAN test that triggers a tag fault and leads to tag checking being disabled. Also simplify is_write calculation in report_tag_fault. With this patch KASAN tests are still failing for the hardware tag-based mode; fixes come in the next few patches. [andreyknvl@google.com: export HW_TAGS symbols for KUnit tests] Link: https://lkml.kernel.org/r/e7eeb252da408b08f0c81b950a55fb852f92000b.1613155970.git.andreyknvl@google.com Link: https://linux-review.googlesource.com/id/Id94dc9eccd33b23cda4950be408c27f879e474c8 Link: https://lkml.kernel.org/r/51b23112cf3fd62b8f8e9df81026fa2b15870501.1610733117.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Reviewed-by: Catalin Marinas Reviewed-by: Vincenzo Frascino Cc: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Marco Elver Cc: Peter Collingbourne Cc: Will Deacon Signed-off-by: Andrew Morton --- arch/arm64/include/asm/memory.h | 1 arch/arm64/include/asm/mte-kasan.h | 12 +++++++ arch/arm64/kernel/mte.c | 12 +++++++ arch/arm64/mm/fault.c | 20 +++++++++--- lib/Kconfig.kasan | 4 +- lib/test_kasan.c | 42 ++++++++++++++++++--------- mm/kasan/hw_tags.c | 16 ++++++++++ mm/kasan/kasan.h | 21 +++++++++++++ 8 files changed, 107 insertions(+), 21 deletions(-) --- a/arch/arm64/include/asm/memory.h~kasan-arm64-allow-using-kunit-tests-with-hw_tags-mode +++ a/arch/arm64/include/asm/memory.h @@ -244,6 +244,7 @@ static inline const void *__tag_set(cons #ifdef CONFIG_KASAN_HW_TAGS #define arch_enable_tagging() mte_enable_kernel() +#define arch_set_tagging_report_once(state) mte_set_report_once(state) #define arch_init_tags(max_tag) mte_init_tags(max_tag) #define arch_get_random_tag() mte_get_random_tag() #define arch_get_mem_tag(addr) mte_get_mem_tag(addr) --- a/arch/arm64/include/asm/mte-kasan.h~kasan-arm64-allow-using-kunit-tests-with-hw_tags-mode +++ a/arch/arm64/include/asm/mte-kasan.h @@ -32,6 +32,9 @@ void *mte_set_mem_tag_range(void *addr, void mte_enable_kernel(void); void mte_init_tags(u64 max_tag); +void mte_set_report_once(bool state); +bool mte_report_once(void); + #else /* CONFIG_ARM64_MTE */ static inline u8 mte_get_ptr_tag(void *ptr) @@ -60,6 +63,15 @@ static inline void mte_init_tags(u64 max { } +static inline void mte_set_report_once(bool state) +{ +} + +static inline bool mte_report_once(void) +{ + return false; +} + #endif /* CONFIG_ARM64_MTE */ #endif /* __ASSEMBLY__ */ --- a/arch/arm64/kernel/mte.c~kasan-arm64-allow-using-kunit-tests-with-hw_tags-mode +++ a/arch/arm64/kernel/mte.c @@ -25,6 +25,8 @@ u64 gcr_kernel_excl __ro_after_init; +static bool report_fault_once = true; + static void mte_sync_page_tags(struct page *page, pte_t *ptep, bool check_swap) { pte_t old_pte = READ_ONCE(*ptep); @@ -158,6 +160,16 @@ void mte_enable_kernel(void) isb(); } +void mte_set_report_once(bool state) +{ + WRITE_ONCE(report_fault_once, state); +} + +bool mte_report_once(void) +{ + return READ_ONCE(report_fault_once); +} + static void update_sctlr_el1_tcf0(u64 tcf0) { /* ISB required for the kernel uaccess routines */ --- a/arch/arm64/mm/fault.c~kasan-arm64-allow-using-kunit-tests-with-hw_tags-mode +++ a/arch/arm64/mm/fault.c @@ -302,12 +302,24 @@ static void die_kernel_fault(const char static void report_tag_fault(unsigned long addr, unsigned int esr, struct pt_regs *regs) { - bool is_write = ((esr & ESR_ELx_WNR) >> ESR_ELx_WNR_SHIFT) != 0; + static bool reported; + bool is_write; + + if (READ_ONCE(reported)) + return; + + /* + * This is used for KASAN tests and assumes that no MTE faults + * happened before running the tests. + */ + if (mte_report_once()) + WRITE_ONCE(reported, true); /* * SAS bits aren't set for all faults reported in EL1, so we can't * find out access size. */ + is_write = !!(esr & ESR_ELx_WNR); kasan_report(addr, 0, is_write, regs->pc); } #else @@ -319,12 +331,8 @@ static inline void report_tag_fault(unsi static void do_tag_recovery(unsigned long addr, unsigned int esr, struct pt_regs *regs) { - static bool reported; - if (!READ_ONCE(reported)) { - report_tag_fault(addr, esr, regs); - WRITE_ONCE(reported, true); - } + report_tag_fault(addr, esr, regs); /* * Disable MTE Tag Checking on the local CPU for the current EL. --- a/lib/Kconfig.kasan~kasan-arm64-allow-using-kunit-tests-with-hw_tags-mode +++ a/lib/Kconfig.kasan @@ -190,11 +190,11 @@ config KASAN_KUNIT_TEST kernel debugging features like KASAN. For more information on KUnit and unit tests in general, please refer - to the KUnit documentation in Documentation/dev-tools/kunit + to the KUnit documentation in Documentation/dev-tools/kunit. config TEST_KASAN_MODULE tristate "KUnit-incompatible tests of KASAN bug detection capabilities" - depends on m && KASAN + depends on m && KASAN && !KASAN_HW_TAGS help This is a part of the KASAN test suite that is incompatible with KUnit. Currently includes tests that do bad copy_from/to_user --- a/lib/test_kasan.c~kasan-arm64-allow-using-kunit-tests-with-hw_tags-mode +++ a/lib/test_kasan.c @@ -41,16 +41,20 @@ static bool multishot; /* * Temporarily enable multi-shot mode. Otherwise, KASAN would only report the - * first detected bug and panic the kernel if panic_on_warn is enabled. + * first detected bug and panic the kernel if panic_on_warn is enabled. For + * hardware tag-based KASAN also allow tag checking to be reenabled for each + * test, see the comment for KUNIT_EXPECT_KASAN_FAIL(). */ static int kasan_test_init(struct kunit *test) { multishot = kasan_save_enable_multi_shot(); + kasan_set_tagging_report_once(false); return 0; } static void kasan_test_exit(struct kunit *test) { + kasan_set_tagging_report_once(true); kasan_restore_multi_shot(multishot); } @@ -59,19 +63,31 @@ static void kasan_test_exit(struct kunit * KASAN report; causes a test failure otherwise. This relies on a KUnit * resource named "kasan_data". Do not use this name for KUnit resources * outside of KASAN tests. + * + * For hardware tag-based KASAN, when a tag fault happens, tag checking is + * normally auto-disabled. When this happens, this test handler reenables + * tag checking. As tag checking can be only disabled or enabled per CPU, this + * handler disables migration (preemption). */ -#define KUNIT_EXPECT_KASAN_FAIL(test, expression) do { \ - fail_data.report_expected = true; \ - fail_data.report_found = false; \ - kunit_add_named_resource(test, \ - NULL, \ - NULL, \ - &resource, \ - "kasan_data", &fail_data); \ - expression; \ - KUNIT_EXPECT_EQ(test, \ - fail_data.report_expected, \ - fail_data.report_found); \ +#define KUNIT_EXPECT_KASAN_FAIL(test, expression) do { \ + if (IS_ENABLED(CONFIG_KASAN_HW_TAGS)) \ + migrate_disable(); \ + fail_data.report_expected = true; \ + fail_data.report_found = false; \ + kunit_add_named_resource(test, \ + NULL, \ + NULL, \ + &resource, \ + "kasan_data", &fail_data); \ + expression; \ + KUNIT_EXPECT_EQ(test, \ + fail_data.report_expected, \ + fail_data.report_found); \ + if (IS_ENABLED(CONFIG_KASAN_HW_TAGS)) { \ + if (fail_data.report_found) \ + kasan_enable_tagging(); \ + migrate_enable(); \ + } \ } while (0) #define KASAN_TEST_NEEDS_CONFIG_ON(test, config) do { \ --- a/mm/kasan/hw_tags.c~kasan-arm64-allow-using-kunit-tests-with-hw_tags-mode +++ a/mm/kasan/hw_tags.c @@ -185,3 +185,19 @@ struct kasan_track *kasan_get_free_track return &alloc_meta->free_track[0]; } + +#if IS_ENABLED(CONFIG_KASAN_KUNIT_TEST) + +void kasan_set_tagging_report_once(bool state) +{ + hw_set_tagging_report_once(state); +} +EXPORT_SYMBOL_GPL(kasan_set_tagging_report_once); + +void kasan_enable_tagging(void) +{ + hw_enable_tagging(); +} +EXPORT_SYMBOL_GPL(kasan_enable_tagging); + +#endif --- a/mm/kasan/kasan.h~kasan-arm64-allow-using-kunit-tests-with-hw_tags-mode +++ a/mm/kasan/kasan.h @@ -280,6 +280,9 @@ static inline const void *arch_kasan_set #ifndef arch_init_tags #define arch_init_tags(max_tag) #endif +#ifndef arch_set_tagging_report_once +#define arch_set_tagging_report_once(state) +#endif #ifndef arch_get_random_tag #define arch_get_random_tag() (0xFF) #endif @@ -292,12 +295,30 @@ static inline const void *arch_kasan_set #define hw_enable_tagging() arch_enable_tagging() #define hw_init_tags(max_tag) arch_init_tags(max_tag) +#define hw_set_tagging_report_once(state) arch_set_tagging_report_once(state) #define hw_get_random_tag() arch_get_random_tag() #define hw_get_mem_tag(addr) arch_get_mem_tag(addr) #define hw_set_mem_tag_range(addr, size, tag) arch_set_mem_tag_range((addr), (size), (tag)) +#else /* CONFIG_KASAN_HW_TAGS */ + +#define hw_enable_tagging() +#define hw_set_tagging_report_once(state) + #endif /* CONFIG_KASAN_HW_TAGS */ +#if defined(CONFIG_KASAN_HW_TAGS) && IS_ENABLED(CONFIG_KASAN_KUNIT_TEST) + +void kasan_set_tagging_report_once(bool state); +void kasan_enable_tagging(void); + +#else /* CONFIG_KASAN_HW_TAGS || CONFIG_KASAN_KUNIT_TEST */ + +static inline void kasan_set_tagging_report_once(bool state) { } +static inline void kasan_enable_tagging(void) { } + +#endif /* CONFIG_KASAN_HW_TAGS || CONFIG_KASAN_KUNIT_TEST */ + #ifdef CONFIG_KASAN_SW_TAGS u8 kasan_random_tag(void); #elif defined(CONFIG_KASAN_HW_TAGS) From patchwork Wed Feb 24 20:05:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102595 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F130FC433DB for ; Wed, 24 Feb 2021 20:05:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 966B764E7A for ; Wed, 24 Feb 2021 20:05:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 966B764E7A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2E3276B00BC; Wed, 24 Feb 2021 15:05:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2957F8D0015; Wed, 24 Feb 2021 15:05:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1D1508D0014; Wed, 24 Feb 2021 15:05:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0190.hostedemail.com [216.40.44.190]) by kanga.kvack.org (Postfix) with ESMTP id 089A86B00BC for ; Wed, 24 Feb 2021 15:05:33 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id CD06482F9DEF for ; Wed, 24 Feb 2021 20:05:32 +0000 (UTC) X-FDA: 77854241304.04.4441826 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id A4350200039C for ; Wed, 24 Feb 2021 20:05:22 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7993664E20; Wed, 24 Feb 2021 20:05:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197131; bh=+zKbQoNPgHNt3ErXvSR41yhnexPq2doV0RBlDWwlggE=; h=Date:From:To:Subject:In-Reply-To:From; b=typAWSZuoAnX2te227FAXN/g8+Nb4Ff1St3dNyeD8OyCgYkI3HjmcanFvLmMPG7iO uomCyYIsISSwQVMTDE/hMacOxX3l3g1XtH0L+CyWyJ2jzNQbj1C83ksp0VTVDUIdLJ o9UnA2OCjL76l+ZT4anxm0m8h8TWD7MlYnufm9BM= Date: Wed, 24 Feb 2021 12:05:29 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, pcc@google.com, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 092/173] kasan: rename CONFIG_TEST_KASAN_MODULE Message-ID: <20210224200529.HNKAZ1n1p%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: A4350200039C X-Stat-Signature: x3hfjsb6ymyd1qoompauwquatqnmhqdp Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf11; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197122-377928 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: rename CONFIG_TEST_KASAN_MODULE Rename CONFIG_TEST_KASAN_MODULE to CONFIG_KASAN_MODULE_TEST. This naming is more consistent with the existing CONFIG_KASAN_KUNIT_TEST. Link: https://linux-review.googlesource.com/id/Id347dfa5fe8788b7a1a189863e039f409da0ae5f Link: https://lkml.kernel.org/r/f08250246683981bcf8a094fbba7c361995624d2.1610733117.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Peter Collingbourne Cc: Vincenzo Frascino Cc: Will Deacon Signed-off-by: Andrew Morton --- Documentation/dev-tools/kasan.rst | 8 ++++---- lib/Kconfig.kasan | 2 +- lib/Makefile | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-) --- a/Documentation/dev-tools/kasan.rst~kasan-rename-config_test_kasan_module +++ a/Documentation/dev-tools/kasan.rst @@ -358,17 +358,17 @@ unmapped. This will require changes in a This allows ``VMAP_STACK`` support on x86, and can simplify support of architectures that do not have a fixed module region. -CONFIG_KASAN_KUNIT_TEST & CONFIG_TEST_KASAN_MODULE --------------------------------------------------- +CONFIG_KASAN_KUNIT_TEST and CONFIG_KASAN_MODULE_TEST +---------------------------------------------------- -KASAN tests consist on two parts: +KASAN tests consist of two parts: 1. Tests that are integrated with the KUnit Test Framework. Enabled with ``CONFIG_KASAN_KUNIT_TEST``. These tests can be run and partially verified automatically in a few different ways, see the instructions below. 2. Tests that are currently incompatible with KUnit. Enabled with -``CONFIG_TEST_KASAN_MODULE`` and can only be run as a module. These tests can +``CONFIG_KASAN_MODULE_TEST`` and can only be run as a module. These tests can only be verified manually, by loading the kernel module and inspecting the kernel log for KASAN reports. --- a/lib/Kconfig.kasan~kasan-rename-config_test_kasan_module +++ a/lib/Kconfig.kasan @@ -192,7 +192,7 @@ config KASAN_KUNIT_TEST For more information on KUnit and unit tests in general, please refer to the KUnit documentation in Documentation/dev-tools/kunit. -config TEST_KASAN_MODULE +config KASAN_MODULE_TEST tristate "KUnit-incompatible tests of KASAN bug detection capabilities" depends on m && KASAN && !KASAN_HW_TAGS help --- a/lib/Makefile~kasan-rename-config_test_kasan_module +++ a/lib/Makefile @@ -66,7 +66,7 @@ obj-$(CONFIG_TEST_IDA) += test_ida.o obj-$(CONFIG_KASAN_KUNIT_TEST) += test_kasan.o CFLAGS_test_kasan.o += -fno-builtin CFLAGS_test_kasan.o += $(call cc-disable-warning, vla) -obj-$(CONFIG_TEST_KASAN_MODULE) += test_kasan_module.o +obj-$(CONFIG_KASAN_MODULE_TEST) += test_kasan_module.o CFLAGS_test_kasan_module.o += -fno-builtin obj-$(CONFIG_TEST_UBSAN) += test_ubsan.o CFLAGS_test_ubsan.o += $(call cc-disable-warning, vla) From patchwork Wed Feb 24 20:05:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102597 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 056FDC433E6 for ; Wed, 24 Feb 2021 20:05:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8C41964F0E for ; Wed, 24 Feb 2021 20:05:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8C41964F0E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1ABA76B00BE; Wed, 24 Feb 2021 15:05:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 15CA58D0015; Wed, 24 Feb 2021 15:05:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 09A248D0014; Wed, 24 Feb 2021 15:05:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0058.hostedemail.com [216.40.44.58]) by kanga.kvack.org (Postfix) with ESMTP id E65A46B00BE for ; Wed, 24 Feb 2021 15:05:37 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id B749382FC25D for ; Wed, 24 Feb 2021 20:05:37 +0000 (UTC) X-FDA: 77854241514.24.8171EEF Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP id 7D2122000381 for ; Wed, 24 Feb 2021 20:05:38 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 64DE164F1F; Wed, 24 Feb 2021 20:05:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197136; bh=Qiyegy6GgJrFfih/Z9Yl0FqPnozPTJKh2V4KK6EnueU=; h=Date:From:To:Subject:In-Reply-To:From; b=sDLxUSzWaDgoAk6jkLytx6fL8LafAqjLPJJwgPb8U8aaNvHpvOEEA79FiNryQyp8D b49F7/tZJEVXVevhVl3OWAJTBYhGNl+9d/XDSxZA7BbUmIoqW6nvRRd//ro6sJB6VY 8ESm2ElurC4QfVW1/U2k3v0vg9Y9yNkaBw6OU9yM= Date: Wed, 24 Feb 2021 12:05:34 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, pcc@google.com, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 093/173] kasan: add compiler barriers to KUNIT_EXPECT_KASAN_FAIL Message-ID: <20210224200534.sK9xuqhpq%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: km6sk7inphp6ouzh5h7d3amo3ajz7stg X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 7D2122000381 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197138-539870 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: add compiler barriers to KUNIT_EXPECT_KASAN_FAIL It might not be obvious to the compiler that the expression must be executed between writing and reading to fail_data. In this case, the compiler might reorder or optimize away some of the accesses, and the tests will fail. Add compiler barriers around the expression in KUNIT_EXPECT_KASAN_FAIL and use READ/WRITE_ONCE() for accessing fail_data fields. Link: https://linux-review.googlesource.com/id/I046079f48641a1d36fe627fc8827a9249102fd50 Link: https://lkml.kernel.org/r/6f11596f367d8ae8f71d800351e9a5d91eda19f6.1610733117.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Peter Collingbourne Cc: Vincenzo Frascino Cc: Will Deacon Signed-off-by: Andrew Morton --- lib/test_kasan.c | 17 ++++++++++++----- mm/kasan/report.c | 2 +- 2 files changed, 13 insertions(+), 6 deletions(-) --- a/lib/test_kasan.c~kasan-add-compiler-barriers-to-kunit_expect_kasan_fail +++ a/lib/test_kasan.c @@ -68,23 +68,30 @@ static void kasan_test_exit(struct kunit * normally auto-disabled. When this happens, this test handler reenables * tag checking. As tag checking can be only disabled or enabled per CPU, this * handler disables migration (preemption). + * + * Since the compiler doesn't see that the expression can change the fail_data + * fields, it can reorder or optimize away the accesses to those fields. + * Use READ/WRITE_ONCE() for the accesses and compiler barriers around the + * expression to prevent that. */ #define KUNIT_EXPECT_KASAN_FAIL(test, expression) do { \ if (IS_ENABLED(CONFIG_KASAN_HW_TAGS)) \ migrate_disable(); \ - fail_data.report_expected = true; \ - fail_data.report_found = false; \ + WRITE_ONCE(fail_data.report_expected, true); \ + WRITE_ONCE(fail_data.report_found, false); \ kunit_add_named_resource(test, \ NULL, \ NULL, \ &resource, \ "kasan_data", &fail_data); \ + barrier(); \ expression; \ + barrier(); \ KUNIT_EXPECT_EQ(test, \ - fail_data.report_expected, \ - fail_data.report_found); \ + READ_ONCE(fail_data.report_expected), \ + READ_ONCE(fail_data.report_found)); \ if (IS_ENABLED(CONFIG_KASAN_HW_TAGS)) { \ - if (fail_data.report_found) \ + if (READ_ONCE(fail_data.report_found)) \ kasan_enable_tagging(); \ migrate_enable(); \ } \ --- a/mm/kasan/report.c~kasan-add-compiler-barriers-to-kunit_expect_kasan_fail +++ a/mm/kasan/report.c @@ -331,7 +331,7 @@ static void kasan_update_kunit_status(st } kasan_data = (struct kunit_kasan_expectation *)resource->data; - kasan_data->report_found = true; + WRITE_ONCE(kasan_data->report_found, true); kunit_put_resource(resource); } #endif /* IS_ENABLED(CONFIG_KUNIT) */ From patchwork Wed Feb 24 20:05:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102599 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA9C0C433DB for ; Wed, 24 Feb 2021 20:05:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4E39664E6C for ; Wed, 24 Feb 2021 20:05:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4E39664E6C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D1AAE6B00C0; Wed, 24 Feb 2021 15:05:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CCCBE8D0015; Wed, 24 Feb 2021 15:05:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C08618D0014; Wed, 24 Feb 2021 15:05:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0088.hostedemail.com [216.40.44.88]) by kanga.kvack.org (Postfix) with ESMTP id AA88C6B00C0 for ; Wed, 24 Feb 2021 15:05:41 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 7916EFB63 for ; Wed, 24 Feb 2021 20:05:41 +0000 (UTC) X-FDA: 77854241682.26.E7649DD Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP id AE57F80192D5 for ; Wed, 24 Feb 2021 20:05:34 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 414E064EC4; Wed, 24 Feb 2021 20:05:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197140; bh=F6+6qRl94RcbrU28MNSRfxeMUmURNJ5b/eD235BZS/c=; h=Date:From:To:Subject:In-Reply-To:From; b=asl209Up9XJu+mloGiNJCet+X2k0pcIUqeHKwvOdzU5iC0cZI7pD7XDgwrC0CamE8 3e3NfsFLU3a8Udx2q0qGGDfKBL0aTiEQ8eor+UztoXMUYJG2XXxfGZQU+OOD9hDmhK BFO2ITKFfYuFU/iPA0M1xdr9iccs4C43MakLUqb8= Date: Wed, 24 Feb 2021 12:05:38 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, pcc@google.com, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 094/173] kasan: adapt kmalloc_uaf2 test to HW_TAGS mode Message-ID: <20210224200538.lbNI_UzhS%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 35xbphxpsgbybibqoywpgwb43o4gbt6o X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: AE57F80192D5 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf27; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197134-512067 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: adapt kmalloc_uaf2 test to HW_TAGS mode In the kmalloc_uaf2() test, the pointers to the two allocated memory blocks might happen to be the same, and the test will fail. With the software tag-based mode, the probability of the that is 1/254, so it's hard to observe the failure. For the hardware tag-based mode though, the probablity is 1/14, which is quite noticable. Allow up to 16 attempts at generating different tags for the tag-based modes. Link: https://linux-review.googlesource.com/id/Ibfa458ef2804ff465d8eb07434a300bf36388d55 Link: https://lkml.kernel.org/r/9cd5cf2f633dcbf55cab801cd26845d2b075cec7.1610733117.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Peter Collingbourne Cc: Vincenzo Frascino Cc: Will Deacon Signed-off-by: Andrew Morton --- lib/test_kasan.c | 11 +++++++++++ 1 file changed, 11 insertions(+) --- a/lib/test_kasan.c~kasan-adapt-kmalloc_uaf2-test-to-hw_tags-mode +++ a/lib/test_kasan.c @@ -382,7 +382,9 @@ static void kmalloc_uaf2(struct kunit *t { char *ptr1, *ptr2; size_t size = 43; + int counter = 0; +again: ptr1 = kmalloc(size, GFP_KERNEL); KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr1); @@ -391,6 +393,15 @@ static void kmalloc_uaf2(struct kunit *t ptr2 = kmalloc(size, GFP_KERNEL); KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr2); + /* + * For tag-based KASAN ptr1 and ptr2 tags might happen to be the same. + * Allow up to 16 attempts at generating different tags. + */ + if (!IS_ENABLED(CONFIG_KASAN_GENERIC) && ptr1 == ptr2 && counter++ < 16) { + kfree(ptr2); + goto again; + } + KUNIT_EXPECT_KASAN_FAIL(test, ptr1[40] = 'x'); KUNIT_EXPECT_PTR_NE(test, ptr1, ptr2); From patchwork Wed Feb 24 20:05:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102601 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6830C433DB for ; Wed, 24 Feb 2021 20:05:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5AD8964E7A for ; Wed, 24 Feb 2021 20:05:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5AD8964E7A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D49286B00C2; Wed, 24 Feb 2021 15:05:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CF51B8D0015; Wed, 24 Feb 2021 15:05:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C2EAC8D0014; Wed, 24 Feb 2021 15:05:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0168.hostedemail.com [216.40.44.168]) by kanga.kvack.org (Postfix) with ESMTP id AD7686B00C2 for ; Wed, 24 Feb 2021 15:05:45 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 6F4EA246E for ; Wed, 24 Feb 2021 20:05:45 +0000 (UTC) X-FDA: 77854241850.05.E327EEA Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP id C8B17C0007CA for ; Wed, 24 Feb 2021 20:05:37 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 30E8A64E6C; Wed, 24 Feb 2021 20:05:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197144; bh=eHVlfq1Jp1E6GJdPOMUnPqvgGPEhYGHteAiNW+BMbFg=; h=Date:From:To:Subject:In-Reply-To:From; b=M5r5b4jRRK1WVcvhVSxN3vW3Je19QHFvgl18yiBcVo/rvXM6IfqXwC3M2o3gHqPXc h42ENBGoLdpc7HskDrQYy72C+/SwmHqL6075xw/kPs9H9QUht2VyJSfHIWgHQ96t1u KWge8eyjqc/iL1pqi+U9TzlRTXJM+tm5t9hR8fdg= Date: Wed, 24 Feb 2021 12:05:42 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, pcc@google.com, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 095/173] kasan: fix memory corruption in kasan_bitops_tags test Message-ID: <20210224200542.rVrGhQGZD%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 6tzzokzyq48f7ndtxysy38gdk9t9aoxu X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: C8B17C0007CA Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf14; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197137-610735 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: fix memory corruption in kasan_bitops_tags test Since the hardware tag-based KASAN mode might not have a redzone that comes after an allocated object (when kasan.mode=prod is enabled), the kasan_bitops_tags() test ends up corrupting the next object in memory. Change the test so it always accesses the redzone that lies within the allocated object's boundaries. Link: https://linux-review.googlesource.com/id/I67f51d1ee48f0a8d0fe2658c2a39e4879fe0832a Link: https://lkml.kernel.org/r/7d452ce4ae35bb1988d2c9244dfea56cf2cc9315.1610733117.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Peter Collingbourne Cc: Vincenzo Frascino Cc: Will Deacon Signed-off-by: Andrew Morton --- lib/test_kasan.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) --- a/lib/test_kasan.c~kasan-fix-memory-corruption-in-kasan_bitops_tags-test +++ a/lib/test_kasan.c @@ -749,13 +749,13 @@ static void kasan_bitops_tags(struct kun /* This test is specifically crafted for tag-based modes. */ KASAN_TEST_NEEDS_CONFIG_OFF(test, CONFIG_KASAN_GENERIC); - /* Allocation size will be rounded to up granule size, which is 16. */ - bits = kzalloc(sizeof(*bits), GFP_KERNEL); + /* kmalloc-64 cache will be used and the last 16 bytes will be the redzone. */ + bits = kzalloc(48, GFP_KERNEL); KUNIT_ASSERT_NOT_ERR_OR_NULL(test, bits); - /* Do the accesses past the 16 allocated bytes. */ - kasan_bitops_modify(test, BITS_PER_LONG, &bits[1]); - kasan_bitops_test_and_modify(test, BITS_PER_LONG + BITS_PER_BYTE, &bits[1]); + /* Do the accesses past the 48 allocated bytes, but within the redone. */ + kasan_bitops_modify(test, BITS_PER_LONG, (void *)bits + 48); + kasan_bitops_test_and_modify(test, BITS_PER_LONG + BITS_PER_BYTE, (void *)bits + 48); kfree(bits); } From patchwork Wed Feb 24 20:05:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E070AC433DB for ; Wed, 24 Feb 2021 20:05:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7108064F0D for ; Wed, 24 Feb 2021 20:05:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7108064F0D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F15E96B00C4; Wed, 24 Feb 2021 15:05:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EC8916B00C5; Wed, 24 Feb 2021 15:05:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E04488D0014; Wed, 24 Feb 2021 15:05:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0088.hostedemail.com [216.40.44.88]) by kanga.kvack.org (Postfix) with ESMTP id CB21A6B00C4 for ; Wed, 24 Feb 2021 15:05:49 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 981E9FB58 for ; Wed, 24 Feb 2021 20:05:49 +0000 (UTC) X-FDA: 77854242018.14.C6C3526 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP id 02D15DA for ; Wed, 24 Feb 2021 20:05:48 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 2D3EF64F19; Wed, 24 Feb 2021 20:05:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197148; bh=yJNQq+u/nTVdQILwEE03KxPhDBZIN0rTuiHt9AS+OD4=; h=Date:From:To:Subject:In-Reply-To:From; b=lP2N+BdEZF14ZWDWD2HONZVPzlBAnE3fmZf/KDaFTbdmdsQhp/dNEv6IRKjBwY3Ts AJT/bmhTzYKHrEdtFpdauJEwax7nmHP3i+0NtYyQwHsGC6Jpsga/HZNltKTJSQb8X6 wFdpTcnhuZzdTVQhrqoYrLK7utq0KBFlioUvmfEQ= Date: Wed, 24 Feb 2021 12:05:46 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, pcc@google.com, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 096/173] kasan: move _RET_IP_ to inline wrappers Message-ID: <20210224200546.sgHAsN8ST%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 02D15DA X-Stat-Signature: j5hiwhwdxyktmx64kbefb69f58ox3poh Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf20; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197148-510207 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: move _RET_IP_ to inline wrappers Generic mm functions that call KASAN annotations that might report a bug pass _RET_IP_ to them as an argument. This allows KASAN to include the name of the function that called the mm function in its report's header. Now that KASAN has inline wrappers for all of its annotations, move _RET_IP_ to those wrappers to simplify annotation call sites. Link: https://linux-review.googlesource.com/id/I8fb3c06d49671305ee184175a39591bc26647a67 Link: https://lkml.kernel.org/r/5c1490eddf20b436b8c4eeea83fce47687d5e4a4.1610733117.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Peter Collingbourne Cc: Vincenzo Frascino Cc: Will Deacon Signed-off-by: Andrew Morton --- include/linux/kasan.h | 20 +++++++++----------- mm/mempool.c | 2 +- mm/slab.c | 2 +- mm/slub.c | 4 ++-- 4 files changed, 13 insertions(+), 15 deletions(-) --- a/include/linux/kasan.h~kasan-move-_ret_ip_-to-inline-wrappers +++ a/include/linux/kasan.h @@ -185,19 +185,18 @@ static __always_inline void * __must_che } bool __kasan_slab_free(struct kmem_cache *s, void *object, unsigned long ip); -static __always_inline bool kasan_slab_free(struct kmem_cache *s, void *object, - unsigned long ip) +static __always_inline bool kasan_slab_free(struct kmem_cache *s, void *object) { if (kasan_enabled()) - return __kasan_slab_free(s, object, ip); + return __kasan_slab_free(s, object, _RET_IP_); return false; } void __kasan_slab_free_mempool(void *ptr, unsigned long ip); -static __always_inline void kasan_slab_free_mempool(void *ptr, unsigned long ip) +static __always_inline void kasan_slab_free_mempool(void *ptr) { if (kasan_enabled()) - __kasan_slab_free_mempool(ptr, ip); + __kasan_slab_free_mempool(ptr, _RET_IP_); } void * __must_check __kasan_slab_alloc(struct kmem_cache *s, @@ -241,10 +240,10 @@ static __always_inline void * __must_che } void __kasan_kfree_large(void *ptr, unsigned long ip); -static __always_inline void kasan_kfree_large(void *ptr, unsigned long ip) +static __always_inline void kasan_kfree_large(void *ptr) { if (kasan_enabled()) - __kasan_kfree_large(ptr, ip); + __kasan_kfree_large(ptr, _RET_IP_); } bool kasan_save_enable_multi_shot(void); @@ -277,12 +276,11 @@ static inline void *kasan_init_slab_obj( { return (void *)object; } -static inline bool kasan_slab_free(struct kmem_cache *s, void *object, - unsigned long ip) +static inline bool kasan_slab_free(struct kmem_cache *s, void *object) { return false; } -static inline void kasan_slab_free_mempool(void *ptr, unsigned long ip) {} +static inline void kasan_slab_free_mempool(void *ptr) {} static inline void *kasan_slab_alloc(struct kmem_cache *s, void *object, gfp_t flags) { @@ -302,7 +300,7 @@ static inline void *kasan_krealloc(const { return (void *)object; } -static inline void kasan_kfree_large(void *ptr, unsigned long ip) {} +static inline void kasan_kfree_large(void *ptr) {} #endif /* CONFIG_KASAN */ --- a/mm/mempool.c~kasan-move-_ret_ip_-to-inline-wrappers +++ a/mm/mempool.c @@ -104,7 +104,7 @@ static inline void poison_element(mempoo static __always_inline void kasan_poison_element(mempool_t *pool, void *element) { if (pool->alloc == mempool_alloc_slab || pool->alloc == mempool_kmalloc) - kasan_slab_free_mempool(element, _RET_IP_); + kasan_slab_free_mempool(element); else if (pool->alloc == mempool_alloc_pages) kasan_free_pages(element, (unsigned long)pool->pool_data); } --- a/mm/slab.c~kasan-move-_ret_ip_-to-inline-wrappers +++ a/mm/slab.c @@ -3420,7 +3420,7 @@ static __always_inline void __cache_free memset(objp, 0, cachep->object_size); /* Put the object into the quarantine, don't touch it for now. */ - if (kasan_slab_free(cachep, objp, _RET_IP_)) + if (kasan_slab_free(cachep, objp)) return; /* Use KCSAN to help debug racy use-after-free. */ --- a/mm/slub.c~kasan-move-_ret_ip_-to-inline-wrappers +++ a/mm/slub.c @@ -1528,7 +1528,7 @@ static inline void *kmalloc_large_node_h static __always_inline void kfree_hook(void *x) { kmemleak_free(x); - kasan_kfree_large(x, _RET_IP_); + kasan_kfree_large(x); } static __always_inline bool slab_free_hook(struct kmem_cache *s, void *x) @@ -1558,7 +1558,7 @@ static __always_inline bool slab_free_ho KCSAN_ACCESS_WRITE | KCSAN_ACCESS_ASSERT); /* KASAN might put x into memory quarantine, delaying its reuse */ - return kasan_slab_free(s, x, _RET_IP_); + return kasan_slab_free(s, x); } static inline bool slab_free_freelist_hook(struct kmem_cache *s, From patchwork Wed Feb 24 20:05:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102607 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 212C7C433E0 for ; Wed, 24 Feb 2021 20:06:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9C40664F09 for ; Wed, 24 Feb 2021 20:06:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9C40664F09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2E4508D0014; Wed, 24 Feb 2021 15:06:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1D2E56B00C9; Wed, 24 Feb 2021 15:06:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F3CB28D0014; Wed, 24 Feb 2021 15:06:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0007.hostedemail.com [216.40.44.7]) by kanga.kvack.org (Postfix) with ESMTP id D6C846B00C8 for ; Wed, 24 Feb 2021 15:06:00 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 9FEE075A0 for ; Wed, 24 Feb 2021 20:06:00 +0000 (UTC) X-FDA: 77854242480.18.39F6A08 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf23.hostedemail.com (Postfix) with ESMTP id B7372A00050F for ; Wed, 24 Feb 2021 20:05:54 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 3AA4F64E7A; Wed, 24 Feb 2021 20:05:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197152; bh=4BBsA7s1orAkJzdtjXgCVsfOs+8y692pDZ+TS9DXPmQ=; h=Date:From:To:Subject:In-Reply-To:From; b=rUgT42GWcy187uae6TswjUtiaEdpBMOjcoa833hzHEEJ63HHPoKpvmybKS+Cnqyox IcZQhOV9ChAHaXO2dYKfPp/PNbQx04kb95yjUWUo+zefMh/+NtqApVXKOziU/Lgday ElrwEVyy83V27lZHU0Pk9Kx4Hvh8z5eIh+ygLWOY= Date: Wed, 24 Feb 2021 12:05:50 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, pcc@google.com, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 097/173] kasan: fix bug detection via ksize for HW_TAGS mode Message-ID: <20210224200550.p9pDdaWub%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 6dx8ztbp1ukowqzopxb7ceggjherr8h1 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: B7372A00050F Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf23; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197154-133818 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: fix bug detection via ksize for HW_TAGS mode The currently existing kasan_check_read/write() annotations are intended to be used for kernel modules that have KASAN compiler instrumentation disabled. Thus, they are only relevant for the software KASAN modes that rely on compiler instrumentation. However there's another use case for these annotations: ksize() checks that the object passed to it is indeed accessible before unpoisoning the whole object. This is currently done via __kasan_check_read(), which is compiled away for the hardware tag-based mode that doesn't rely on compiler instrumentation. This leads to KASAN missing detecting some memory corruptions. Provide another annotation called kasan_check_byte() that is available for all KASAN modes. As the implementation rename and reuse kasan_check_invalid_free(). Use this new annotation in ksize(). To avoid having ksize() as the top frame in the reported stack trace pass _RET_IP_ to __kasan_check_byte(). Also add a new ksize_uaf() test that checks that a use-after-free is detected via ksize() itself, and via plain accesses that happen later. Link: https://linux-review.googlesource.com/id/Iaabf771881d0f9ce1b969f2a62938e99d3308ec5 Link: https://lkml.kernel.org/r/f32ad74a60b28d8402482a38476f02bb7600f620.1610733117.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Peter Collingbourne Cc: Vincenzo Frascino Cc: Will Deacon Signed-off-by: Andrew Morton --- include/linux/kasan-checks.h | 6 ++++++ include/linux/kasan.h | 17 +++++++++++++++++ lib/test_kasan.c | 20 ++++++++++++++++++++ mm/kasan/common.c | 11 ++++++++++- mm/kasan/generic.c | 4 ++-- mm/kasan/kasan.h | 10 +++++----- mm/kasan/sw_tags.c | 6 +++--- mm/slab_common.c | 16 +++++++++------- 8 files changed, 72 insertions(+), 18 deletions(-) --- a/include/linux/kasan-checks.h~kasan-fix-bug-detection-via-ksize-for-hw_tags-mode +++ a/include/linux/kasan-checks.h @@ -5,6 +5,12 @@ #include /* + * The annotations present in this file are only relevant for the software + * KASAN modes that rely on compiler instrumentation, and will be optimized + * away for the hardware tag-based KASAN mode. Use kasan_check_byte() instead. + */ + +/* * __kasan_check_*: Always available when KASAN is enabled. This may be used * even in compilation units that selectively disable KASAN, but must use KASAN * to validate access to an address. Never use these in header files! --- a/include/linux/kasan.h~kasan-fix-bug-detection-via-ksize-for-hw_tags-mode +++ a/include/linux/kasan.h @@ -246,6 +246,19 @@ static __always_inline void kasan_kfree_ __kasan_kfree_large(ptr, _RET_IP_); } +/* + * Unlike kasan_check_read/write(), kasan_check_byte() is performed even for + * the hardware tag-based mode that doesn't rely on compiler instrumentation. + */ +bool __kasan_check_byte(const void *addr, unsigned long ip); +static __always_inline bool kasan_check_byte(const void *addr) +{ + if (kasan_enabled()) + return __kasan_check_byte(addr, _RET_IP_); + return true; +} + + bool kasan_save_enable_multi_shot(void); void kasan_restore_multi_shot(bool enabled); @@ -301,6 +314,10 @@ static inline void *kasan_krealloc(const return (void *)object; } static inline void kasan_kfree_large(void *ptr) {} +static inline bool kasan_check_byte(const void *address) +{ + return true; +} #endif /* CONFIG_KASAN */ --- a/lib/test_kasan.c~kasan-fix-bug-detection-via-ksize-for-hw_tags-mode +++ a/lib/test_kasan.c @@ -496,6 +496,7 @@ static void kasan_global_oob(struct kuni KUNIT_EXPECT_KASAN_FAIL(test, *(volatile char *)p); } +/* Check that ksize() makes the whole object accessible. */ static void ksize_unpoisons_memory(struct kunit *test) { char *ptr; @@ -514,6 +515,24 @@ static void ksize_unpoisons_memory(struc kfree(ptr); } +/* + * Check that a use-after-free is detected by ksize() and via normal accesses + * after it. + */ +static void ksize_uaf(struct kunit *test) +{ + char *ptr; + int size = 128 - KASAN_GRANULE_SIZE; + + ptr = kmalloc(size, GFP_KERNEL); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr); + kfree(ptr); + + KUNIT_EXPECT_KASAN_FAIL(test, ksize(ptr)); + KUNIT_EXPECT_KASAN_FAIL(test, kasan_int_result = *ptr); + KUNIT_EXPECT_KASAN_FAIL(test, kasan_int_result = *(ptr + size)); +} + static void kasan_stack_oob(struct kunit *test) { char stack_array[10]; @@ -907,6 +926,7 @@ static struct kunit_case kasan_kunit_tes KUNIT_CASE(kasan_alloca_oob_left), KUNIT_CASE(kasan_alloca_oob_right), KUNIT_CASE(ksize_unpoisons_memory), + KUNIT_CASE(ksize_uaf), KUNIT_CASE(kmem_cache_double_free), KUNIT_CASE(kmem_cache_invalid_free), KUNIT_CASE(kasan_memchr), --- a/mm/kasan/common.c~kasan-fix-bug-detection-via-ksize-for-hw_tags-mode +++ a/mm/kasan/common.c @@ -345,7 +345,7 @@ static bool ____kasan_slab_free(struct k if (unlikely(cache->flags & SLAB_TYPESAFE_BY_RCU)) return false; - if (kasan_check_invalid_free(tagged_object)) { + if (!kasan_byte_accessible(tagged_object)) { kasan_report_invalid_free(tagged_object, ip); return true; } @@ -490,3 +490,12 @@ void __kasan_kfree_large(void *ptr, unsi kasan_report_invalid_free(ptr, ip); /* The object will be poisoned by kasan_free_pages(). */ } + +bool __kasan_check_byte(const void *address, unsigned long ip) +{ + if (!kasan_byte_accessible(address)) { + kasan_report((unsigned long)address, 1, false, ip); + return false; + } + return true; +} --- a/mm/kasan/generic.c~kasan-fix-bug-detection-via-ksize-for-hw_tags-mode +++ a/mm/kasan/generic.c @@ -185,11 +185,11 @@ bool kasan_check_range(unsigned long add return check_region_inline(addr, size, write, ret_ip); } -bool kasan_check_invalid_free(void *addr) +bool kasan_byte_accessible(const void *addr) { s8 shadow_byte = READ_ONCE(*(s8 *)kasan_mem_to_shadow(addr)); - return shadow_byte < 0 || shadow_byte >= KASAN_GRANULE_SIZE; + return shadow_byte >= 0 && shadow_byte < KASAN_GRANULE_SIZE; } void kasan_cache_shrink(struct kmem_cache *cache) --- a/mm/kasan/kasan.h~kasan-fix-bug-detection-via-ksize-for-hw_tags-mode +++ a/mm/kasan/kasan.h @@ -341,20 +341,20 @@ static inline void kasan_unpoison(const round_up(size, KASAN_GRANULE_SIZE), get_tag(address)); } -static inline bool kasan_check_invalid_free(void *addr) +static inline bool kasan_byte_accessible(const void *addr) { u8 ptr_tag = get_tag(addr); - u8 mem_tag = hw_get_mem_tag(addr); + u8 mem_tag = hw_get_mem_tag((void *)addr); - return (mem_tag == KASAN_TAG_INVALID) || - (ptr_tag != KASAN_TAG_KERNEL && ptr_tag != mem_tag); + return (mem_tag != KASAN_TAG_INVALID) && + (ptr_tag == KASAN_TAG_KERNEL || ptr_tag == mem_tag); } #else /* CONFIG_KASAN_HW_TAGS */ void kasan_poison(const void *address, size_t size, u8 value); void kasan_unpoison(const void *address, size_t size); -bool kasan_check_invalid_free(void *addr); +bool kasan_byte_accessible(const void *addr); #endif /* CONFIG_KASAN_HW_TAGS */ --- a/mm/kasan/sw_tags.c~kasan-fix-bug-detection-via-ksize-for-hw_tags-mode +++ a/mm/kasan/sw_tags.c @@ -118,13 +118,13 @@ bool kasan_check_range(unsigned long add return true; } -bool kasan_check_invalid_free(void *addr) +bool kasan_byte_accessible(const void *addr) { u8 tag = get_tag(addr); u8 shadow_byte = READ_ONCE(*(u8 *)kasan_mem_to_shadow(kasan_reset_tag(addr))); - return (shadow_byte == KASAN_TAG_INVALID) || - (tag != KASAN_TAG_KERNEL && tag != shadow_byte); + return (shadow_byte != KASAN_TAG_INVALID) && + (tag == KASAN_TAG_KERNEL || tag == shadow_byte); } #define DEFINE_HWASAN_LOAD_STORE(size) \ --- a/mm/slab_common.c~kasan-fix-bug-detection-via-ksize-for-hw_tags-mode +++ a/mm/slab_common.c @@ -1218,19 +1218,21 @@ size_t ksize(const void *objp) size_t size; /* - * We need to check that the pointed to object is valid, and only then - * unpoison the shadow memory below. We use __kasan_check_read(), to - * generate a more useful report at the time ksize() is called (rather - * than later where behaviour is undefined due to potential - * use-after-free or double-free). + * We need to first check that the pointer to the object is valid, and + * only then unpoison the memory. The report printed from ksize() is + * more useful, then when it's printed later when the behaviour could + * be undefined due to a potential use-after-free or double-free. * - * If the pointed to memory is invalid we return 0, to avoid users of + * We use kasan_check_byte(), which is supported for the hardware + * tag-based KASAN mode, unlike kasan_check_read/write(). + * + * If the pointed to memory is invalid, we return 0 to avoid users of * ksize() writing to and potentially corrupting the memory region. * * We want to perform the check before __ksize(), to avoid potentially * crashing in __ksize() due to accessing invalid metadata. */ - if (unlikely(ZERO_OR_NULL_PTR(objp)) || !__kasan_check_read(objp, 1)) + if (unlikely(ZERO_OR_NULL_PTR(objp)) || !kasan_check_byte(objp)) return 0; size = __ksize(objp); From patchwork Wed Feb 24 20:05:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102605 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39666C433E0 for ; Wed, 24 Feb 2021 20:05:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C216264F0D for ; Wed, 24 Feb 2021 20:05:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C216264F0D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 616416B00C6; Wed, 24 Feb 2021 15:05:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5ED426B00C7; Wed, 24 Feb 2021 15:05:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 52CCD6B00C8; Wed, 24 Feb 2021 15:05:58 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0082.hostedemail.com [216.40.44.82]) by kanga.kvack.org (Postfix) with ESMTP id 3EC4A6B00C6 for ; Wed, 24 Feb 2021 15:05:58 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 09BD41829200A for ; Wed, 24 Feb 2021 20:05:58 +0000 (UTC) X-FDA: 77854242396.07.D448DA6 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf26.hostedemail.com (Postfix) with ESMTP id 10CF5407F8DF for ; Wed, 24 Feb 2021 20:05:52 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A1A1664F0E; Wed, 24 Feb 2021 20:05:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197156; bh=iXxCAfUagaaNEkYZ2Acgf0LD3UD8IMv4eWVMhOt8pPM=; h=Date:From:To:Subject:In-Reply-To:From; b=o09mvRr+PulX4UxT/0HS6phu22cd6Uhe3+7yvEFo3NHv7ucRoy3WYvjvlTtdGV+W8 EtjaqpwXv88FUwboKKXsRmWOaw+FrsNoL4o1iJCOi2NpXDvFP7LuWL4e/EMx56oWlG jBn5dwusjpw9O8zdV0lqCfk4kQRyUuk3j9CVUJyo= Date: Wed, 24 Feb 2021 12:05:55 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, pcc@google.com, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 098/173] kasan: add proper page allocator tests Message-ID: <20210224200555.zQSVpXkxz%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 10CF5407F8DF X-Stat-Signature: 8xe1dw8c69ban5q15mod44p5rduf8h5d Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf26; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197152-133549 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: add proper page allocator tests The currently existing page allocator tests rely on kmalloc fallback with large sizes that is only present for SLUB. Add proper tests that use alloc/free_pages(). Link: https://linux-review.googlesource.com/id/Ia173d5a1b215fe6b2548d814ef0f4433cf983570 Link: https://lkml.kernel.org/r/a2648930e55ff75b8e700f2e0d905c2b55a67483.1610733117.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Peter Collingbourne Cc: Vincenzo Frascino Cc: Will Deacon Signed-off-by: Andrew Morton --- lib/test_kasan.c | 51 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 46 insertions(+), 5 deletions(-) --- a/lib/test_kasan.c~kasan-add-proper-page-allocator-tests +++ a/lib/test_kasan.c @@ -147,6 +147,12 @@ static void kmalloc_node_oob_right(struc kfree(ptr); } +/* + * These kmalloc_pagealloc_* tests try allocating a memory chunk that doesn't + * fit into a slab cache and therefore is allocated via the page allocator + * fallback. Since this kind of fallback is only implemented for SLUB, these + * tests are limited to that allocator. + */ static void kmalloc_pagealloc_oob_right(struct kunit *test) { char *ptr; @@ -154,14 +160,11 @@ static void kmalloc_pagealloc_oob_right( KASAN_TEST_NEEDS_CONFIG_ON(test, CONFIG_SLUB); - /* - * Allocate a chunk that does not fit into a SLUB cache to trigger - * the page allocator fallback. - */ ptr = kmalloc(size, GFP_KERNEL); KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr); KUNIT_EXPECT_KASAN_FAIL(test, ptr[size + OOB_TAG_OFF] = 0); + kfree(ptr); } @@ -174,8 +177,8 @@ static void kmalloc_pagealloc_uaf(struct ptr = kmalloc(size, GFP_KERNEL); KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr); - kfree(ptr); + KUNIT_EXPECT_KASAN_FAIL(test, ptr[0] = 0); } @@ -192,6 +195,42 @@ static void kmalloc_pagealloc_invalid_fr KUNIT_EXPECT_KASAN_FAIL(test, kfree(ptr + 1)); } +static void pagealloc_oob_right(struct kunit *test) +{ + char *ptr; + struct page *pages; + size_t order = 4; + size_t size = (1UL << (PAGE_SHIFT + order)); + + /* + * With generic KASAN page allocations have no redzones, thus + * out-of-bounds detection is not guaranteed. + * See https://bugzilla.kernel.org/show_bug.cgi?id=210503. + */ + KASAN_TEST_NEEDS_CONFIG_OFF(test, CONFIG_KASAN_GENERIC); + + pages = alloc_pages(GFP_KERNEL, order); + ptr = page_address(pages); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr); + + KUNIT_EXPECT_KASAN_FAIL(test, ptr[size] = 0); + free_pages((unsigned long)ptr, order); +} + +static void pagealloc_uaf(struct kunit *test) +{ + char *ptr; + struct page *pages; + size_t order = 4; + + pages = alloc_pages(GFP_KERNEL, order); + ptr = page_address(pages); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ptr); + free_pages((unsigned long)ptr, order); + + KUNIT_EXPECT_KASAN_FAIL(test, ptr[0] = 0); +} + static void kmalloc_large_oob_right(struct kunit *test) { char *ptr; @@ -903,6 +942,8 @@ static struct kunit_case kasan_kunit_tes KUNIT_CASE(kmalloc_pagealloc_oob_right), KUNIT_CASE(kmalloc_pagealloc_uaf), KUNIT_CASE(kmalloc_pagealloc_invalid_free), + KUNIT_CASE(pagealloc_oob_right), + KUNIT_CASE(pagealloc_uaf), KUNIT_CASE(kmalloc_large_oob_right), KUNIT_CASE(kmalloc_oob_krealloc_more), KUNIT_CASE(kmalloc_oob_krealloc_less), From patchwork Wed Feb 24 20:05:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102609 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.9 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, UNWANTED_LANGUAGE_BODY,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1569DC433DB for ; Wed, 24 Feb 2021 20:06:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id ACEA364F23 for ; Wed, 24 Feb 2021 20:06:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ACEA364F23 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 776C26B00C8; Wed, 24 Feb 2021 15:06:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6D8968D0016; Wed, 24 Feb 2021 15:06:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 505EA8D0015; Wed, 24 Feb 2021 15:06:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0139.hostedemail.com [216.40.44.139]) by kanga.kvack.org (Postfix) with ESMTP id 2DBF16B00C8 for ; Wed, 24 Feb 2021 15:06:02 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id CB041841F for ; Wed, 24 Feb 2021 20:06:01 +0000 (UTC) X-FDA: 77854242522.16.17BEEFC Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP id 3C271C0007CF for ; Wed, 24 Feb 2021 20:05:58 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 91DAD64F0D; Wed, 24 Feb 2021 20:05:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197160; bh=A0KJ2LuKmPONcOAAtAP2Ry7eDDPAfY7KJPBJ9DDrR1g=; h=Date:From:To:Subject:In-Reply-To:From; b=xvq9jup1WpRfys/tZaV7HgIK1E04kr8ZRaD0Pm1xipTJUNL5KiCkeyf0Y2kE3rXi6 hY0yDQ4iQeIGAQ5ZbJNQT/1ePSI0js4Xijb6YdSNq9/lLnlfVQXZzKm/kSCOSugIcs MJ9jgDhe5PTRIZjiNY1WoMFVcGIqsg/bvRC3dDgE= Date: Wed, 24 Feb 2021 12:05:59 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, pcc@google.com, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 099/173] kasan: add a test for kmem_cache_alloc/free_bulk Message-ID: <20210224200559.eIGHFvuuh%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 3C271C0007CF X-Stat-Signature: r66ndkjwmrtn1z37wutu3mho3ytt7ej1 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf03; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197158-298897 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: add a test for kmem_cache_alloc/free_bulk Add a test for kmem_cache_alloc/free_bulk to make sure there are no false-positives when these functions are used. Link: https://linux-review.googlesource.com/id/I2a8bf797aecf81baeac61380c567308f319e263d Link: https://lkml.kernel.org/r/418122ebe4600771ac81e9ca6eab6740cf8dcfa1.1610733117.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Peter Collingbourne Cc: Vincenzo Frascino Cc: Will Deacon Signed-off-by: Andrew Morton --- lib/test_kasan.c | 38 +++++++++++++++++++++++++++++++++----- 1 file changed, 33 insertions(+), 5 deletions(-) --- a/lib/test_kasan.c~kasan-add-a-test-for-kmem_cache_alloc-free_bulk +++ a/lib/test_kasan.c @@ -479,10 +479,11 @@ static void kmem_cache_oob(struct kunit { char *p; size_t size = 200; - struct kmem_cache *cache = kmem_cache_create("test_cache", - size, 0, - 0, NULL); + struct kmem_cache *cache; + + cache = kmem_cache_create("test_cache", size, 0, 0, NULL); KUNIT_ASSERT_NOT_ERR_OR_NULL(test, cache); + p = kmem_cache_alloc(cache, GFP_KERNEL); if (!p) { kunit_err(test, "Allocation failed: %s\n", __func__); @@ -491,11 +492,12 @@ static void kmem_cache_oob(struct kunit } KUNIT_EXPECT_KASAN_FAIL(test, *p = p[size + OOB_TAG_OFF]); + kmem_cache_free(cache, p); kmem_cache_destroy(cache); } -static void memcg_accounted_kmem_cache(struct kunit *test) +static void kmem_cache_accounted(struct kunit *test) { int i; char *p; @@ -522,6 +524,31 @@ free_cache: kmem_cache_destroy(cache); } +static void kmem_cache_bulk(struct kunit *test) +{ + struct kmem_cache *cache; + size_t size = 200; + char *p[10]; + bool ret; + int i; + + cache = kmem_cache_create("test_cache", size, 0, 0, NULL); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, cache); + + ret = kmem_cache_alloc_bulk(cache, GFP_KERNEL, ARRAY_SIZE(p), (void **)&p); + if (!ret) { + kunit_err(test, "Allocation failed: %s\n", __func__); + kmem_cache_destroy(cache); + return; + } + + for (i = 0; i < ARRAY_SIZE(p); i++) + p[i][0] = p[i][size - 1] = 42; + + kmem_cache_free_bulk(cache, ARRAY_SIZE(p), (void **)&p); + kmem_cache_destroy(cache); +} + static char global_array[10]; static void kasan_global_oob(struct kunit *test) @@ -961,7 +988,8 @@ static struct kunit_case kasan_kunit_tes KUNIT_CASE(kfree_via_page), KUNIT_CASE(kfree_via_phys), KUNIT_CASE(kmem_cache_oob), - KUNIT_CASE(memcg_accounted_kmem_cache), + KUNIT_CASE(kmem_cache_accounted), + KUNIT_CASE(kmem_cache_bulk), KUNIT_CASE(kasan_global_oob), KUNIT_CASE(kasan_stack_oob), KUNIT_CASE(kasan_alloca_oob_left), From patchwork Wed Feb 24 20:06:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102611 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1FC9BC433E0 for ; Wed, 24 Feb 2021 20:06:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A626E64F09 for ; Wed, 24 Feb 2021 20:06:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A626E64F09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2E5706B00CA; Wed, 24 Feb 2021 15:06:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 245558D0016; Wed, 24 Feb 2021 15:06:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 110A98D0015; Wed, 24 Feb 2021 15:06:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0178.hostedemail.com [216.40.44.178]) by kanga.kvack.org (Postfix) with ESMTP id E985B6B00CA for ; Wed, 24 Feb 2021 15:06:05 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id ABCAE82FC77F for ; Wed, 24 Feb 2021 20:06:05 +0000 (UTC) X-FDA: 77854242690.23.8011525 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP id 180A5C0001EE for ; Wed, 24 Feb 2021 20:06:01 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7BB6664F0C; Wed, 24 Feb 2021 20:06:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197164; bh=qw9ptL7Cnw4elSq3daOYS3InTUivIfyhGpsiItG/Ub4=; h=Date:From:To:Subject:In-Reply-To:From; b=nTIFO2KC6jeZYIbqNAvBnC08GwHIkzy/kbuUSxjTD9v2R1YYSP7N8T4X4b6+zEsWY VW4mtLTcJpVQLDP6YfeAfY+BatsBRJlOU3YNVCH4uz4P1YM/MjLeE13UtsoZnEHaTY xLP8nCd/XGQonaFLYeoJ9DKpWYYWpt3E8Vn8mFfo= Date: Wed, 24 Feb 2021 12:06:02 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, aryabinin@virtuozzo.com, Branislav.Rankov@arm.com, catalin.marinas@arm.com, dvyukov@google.com, elver@google.com, eugenis@google.com, glider@google.com, kevin.brodsky@arm.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, pcc@google.com, torvalds@linux-foundation.org, vincenzo.frascino@arm.com, will.deacon@arm.com Subject: [patch 100/173] kasan: don't run tests when KASAN is not enabled Message-ID: <20210224200602.U_Wshqs7c%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: zh6s71ryrauhty8msfazwpshw4siuuj1 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 180A5C0001EE Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf03; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197161-468014 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Andrey Konovalov Subject: kasan: don't run tests when KASAN is not enabled Don't run KASAN tests when it's disabled with kasan.mode=off to avoid corrupting kernel memory. Link: https://linux-review.googlesource.com/id/I6447af436a69a94bfc35477f6bf4e2122948355e Link: https://lkml.kernel.org/r/25bd4fb5cae7b421d806a1f33fb633edd313f0c7.1610733117.git.andreyknvl@google.com Signed-off-by: Andrey Konovalov Reviewed-by: Marco Elver Reviewed-by: Alexander Potapenko Cc: Andrey Ryabinin Cc: Branislav Rankov Cc: Catalin Marinas Cc: Dmitry Vyukov Cc: Evgenii Stepanov Cc: Kevin Brodsky Cc: Peter Collingbourne Cc: Vincenzo Frascino Cc: Will Deacon Signed-off-by: Andrew Morton --- lib/test_kasan.c | 5 +++++ 1 file changed, 5 insertions(+) --- a/lib/test_kasan.c~kasan-dont-run-tests-when-kasan-is-not-enabled +++ a/lib/test_kasan.c @@ -47,6 +47,11 @@ static bool multishot; */ static int kasan_test_init(struct kunit *test) { + if (!kasan_enabled()) { + kunit_err(test, "can't run KASAN tests with KASAN disabled"); + return -1; + } + multishot = kasan_save_enable_multi_shot(); kasan_set_tagging_report_once(false); return 0; From patchwork Wed Feb 24 20:06:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102613 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CAF40C433DB for ; Wed, 24 Feb 2021 20:06:10 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6C01964F09 for ; Wed, 24 Feb 2021 20:06:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6C01964F09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EEE866B00CC; Wed, 24 Feb 2021 15:06:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E7B1B8D0016; Wed, 24 Feb 2021 15:06:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D19C18D0015; Wed, 24 Feb 2021 15:06:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0154.hostedemail.com [216.40.44.154]) by kanga.kvack.org (Postfix) with ESMTP id B26776B00CC for ; Wed, 24 Feb 2021 15:06:09 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 78B1F83E2 for ; Wed, 24 Feb 2021 20:06:09 +0000 (UTC) X-FDA: 77854242858.21.6714899 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP id 3737D2000392 for ; Wed, 24 Feb 2021 20:06:10 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5BF8764F1D; Wed, 24 Feb 2021 20:06:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197168; bh=3ZNqSqqDplsX1y/ZFBZgkR9nQAYS0PdFf0g51fPa2Kc=; h=Date:From:To:Subject:In-Reply-To:From; b=flb6h3kYcOBSsBOreY0DfbP6KXrRayMoN/qcQQNFeD5klvNxOoNF/g9rmTE53jxhB ScawZ4bxs8zRgFoNFGZzG12b6qaH8rLfeLRcGfr1MobQanxn56ZrrUeFtqT10gf4co sQBGL33aXU3NI7bKD3lE6lcJ+pGOAAxfgjPk0yLQ= Date: Wed, 24 Feb 2021 12:06:06 -0800 From: Andrew Morton To: akpm@linux-foundation.org, andreyknvl@google.com, arnd@arndb.de, aryabinin@virtuozzo.com, dvyukov@google.com, glider@google.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, natechancellor@gmail.com, torvalds@linux-foundation.org, walter-zh.wu@mediatek.com Subject: [patch 101/173] kasan: remove redundant config option Message-ID: <20210224200606.fil4kbsxV%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 3737D2000392 X-Stat-Signature: e4mxk8bk8oc1p5rm8k7u83jipybxtikk Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197170-362260 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Walter Wu Subject: kasan: remove redundant config option CONFIG_KASAN_STACK and CONFIG_KASAN_STACK_ENABLE both enable KASAN stack instrumentation, but we should only need one config, so that we remove CONFIG_KASAN_STACK_ENABLE and make CONFIG_KASAN_STACK workable. see [1]. When enable KASAN stack instrumentation, then for gcc we could do no prompt and default value y, and for clang prompt and default value n. [1]: https://bugzilla.kernel.org/show_bug.cgi?id=210221 [walter-zh.wu@mediatek.com: fix remaining the pre-processor syntax] Link: https://lkml.kernel.org/r/20210108040940.1138-1-walter-zh.wu@mediatek.com Link: https://lkml.kernel.org/r/20210107075315.3482-1-walter-zh.wu@mediatek.com Signed-off-by: Walter Wu Suggested-by: Dmitry Vyukov Reviewed-by: Nathan Chancellor Acked-by: Arnd Bergmann Reviewed-by: Andrey Konovalov Cc: Andrey Ryabinin Cc: Dmitry Vyukov Cc: Alexander Potapenko Signed-off-by: Andrew Morton --- arch/arm64/kernel/sleep.S | 2 +- arch/x86/kernel/acpi/wakeup_64.S | 2 +- include/linux/kasan.h | 2 +- lib/Kconfig.kasan | 8 ++------ mm/kasan/common.c | 2 +- mm/kasan/kasan.h | 2 +- mm/kasan/report_generic.c | 2 +- scripts/Makefile.kasan | 10 ++++++++-- 8 files changed, 16 insertions(+), 14 deletions(-) --- a/arch/arm64/kernel/sleep.S~kasan-remove-redundant-config-option +++ a/arch/arm64/kernel/sleep.S @@ -134,7 +134,7 @@ SYM_FUNC_START(_cpu_resume) */ bl cpu_do_resume -#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK +#if defined(CONFIG_KASAN) && defined(CONFIG_KASAN_STACK) mov x0, sp bl kasan_unpoison_task_stack_below #endif --- a/arch/x86/kernel/acpi/wakeup_64.S~kasan-remove-redundant-config-option +++ a/arch/x86/kernel/acpi/wakeup_64.S @@ -115,7 +115,7 @@ SYM_FUNC_START(do_suspend_lowlevel) movq pt_regs_r14(%rax), %r14 movq pt_regs_r15(%rax), %r15 -#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK +#if defined(CONFIG_KASAN) && defined(CONFIG_KASAN_STACK) /* * The suspend path may have poisoned some areas deeper in the stack, * which we now need to unpoison. --- a/include/linux/kasan.h~kasan-remove-redundant-config-option +++ a/include/linux/kasan.h @@ -321,7 +321,7 @@ static inline bool kasan_check_byte(cons #endif /* CONFIG_KASAN */ -#if defined(CONFIG_KASAN) && CONFIG_KASAN_STACK +#if defined(CONFIG_KASAN) && defined(CONFIG_KASAN_STACK) void kasan_unpoison_task_stack(struct task_struct *task); #else static inline void kasan_unpoison_task_stack(struct task_struct *task) {} --- a/lib/Kconfig.kasan~kasan-remove-redundant-config-option +++ a/lib/Kconfig.kasan @@ -138,9 +138,10 @@ config KASAN_INLINE endchoice -config KASAN_STACK_ENABLE +config KASAN_STACK bool "Enable stack instrumentation (unsafe)" if CC_IS_CLANG && !COMPILE_TEST depends on KASAN_GENERIC || KASAN_SW_TAGS + default y if CC_IS_GCC help The LLVM stack address sanitizer has a know problem that causes excessive stack usage in a lot of functions, see @@ -154,11 +155,6 @@ config KASAN_STACK_ENABLE CONFIG_COMPILE_TEST. On gcc it is assumed to always be safe to use and enabled by default. -config KASAN_STACK - int - default 1 if KASAN_STACK_ENABLE || CC_IS_GCC - default 0 - config KASAN_SW_TAGS_IDENTIFY bool "Enable memory corruption identification" depends on KASAN_SW_TAGS --- a/mm/kasan/common.c~kasan-remove-redundant-config-option +++ a/mm/kasan/common.c @@ -63,7 +63,7 @@ void __kasan_unpoison_range(const void * kasan_unpoison(address, size); } -#if CONFIG_KASAN_STACK +#ifdef CONFIG_KASAN_STACK /* Unpoison the entire stack for a task. */ void kasan_unpoison_task_stack(struct task_struct *task) { --- a/mm/kasan/kasan.h~kasan-remove-redundant-config-option +++ a/mm/kasan/kasan.h @@ -230,7 +230,7 @@ void *kasan_find_first_bad_addr(void *ad const char *kasan_get_bug_type(struct kasan_access_info *info); void kasan_metadata_fetch_row(char *buffer, void *row); -#if defined(CONFIG_KASAN_GENERIC) && CONFIG_KASAN_STACK +#if defined(CONFIG_KASAN_GENERIC) && defined(CONFIG_KASAN_STACK) void kasan_print_address_stack_frame(const void *addr); #else static inline void kasan_print_address_stack_frame(const void *addr) { } --- a/mm/kasan/report_generic.c~kasan-remove-redundant-config-option +++ a/mm/kasan/report_generic.c @@ -128,7 +128,7 @@ void kasan_metadata_fetch_row(char *buff memcpy(buffer, kasan_mem_to_shadow(row), META_BYTES_PER_ROW); } -#if CONFIG_KASAN_STACK +#ifdef CONFIG_KASAN_STACK static bool __must_check tokenize_frame_descr(const char **frame_descr, char *token, size_t max_tok_len, unsigned long *value) --- a/scripts/Makefile.kasan~kasan-remove-redundant-config-option +++ a/scripts/Makefile.kasan @@ -2,6 +2,12 @@ CFLAGS_KASAN_NOSANITIZE := -fno-builtin KASAN_SHADOW_OFFSET ?= $(CONFIG_KASAN_SHADOW_OFFSET) +ifdef CONFIG_KASAN_STACK + stack_enable := 1 +else + stack_enable := 0 +endif + ifdef CONFIG_KASAN_GENERIC ifdef CONFIG_KASAN_INLINE @@ -27,7 +33,7 @@ else CFLAGS_KASAN := $(CFLAGS_KASAN_SHADOW) \ $(call cc-param,asan-globals=1) \ $(call cc-param,asan-instrumentation-with-call-threshold=$(call_threshold)) \ - $(call cc-param,asan-stack=$(CONFIG_KASAN_STACK)) \ + $(call cc-param,asan-stack=$(stack_enable)) \ $(call cc-param,asan-instrument-allocas=1) endif @@ -42,7 +48,7 @@ else endif CFLAGS_KASAN := -fsanitize=kernel-hwaddress \ - -mllvm -hwasan-instrument-stack=$(CONFIG_KASAN_STACK) \ + -mllvm -hwasan-instrument-stack=$(stack_enable) \ -mllvm -hwasan-use-short-granules=0 \ $(instrumentation_flags) From patchwork Wed Feb 24 20:06:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102615 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C642C4332B for ; Wed, 24 Feb 2021 20:06:14 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C00D864F23 for ; Wed, 24 Feb 2021 20:06:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C00D864F23 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4DD9A6B00CE; Wed, 24 Feb 2021 15:06:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4661A8D0016; Wed, 24 Feb 2021 15:06:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 37CA58D0015; Wed, 24 Feb 2021 15:06:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0038.hostedemail.com [216.40.44.38]) by kanga.kvack.org (Postfix) with ESMTP id 1CE5F6B00CE for ; Wed, 24 Feb 2021 15:06:13 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id E2A2383F9 for ; Wed, 24 Feb 2021 20:06:12 +0000 (UTC) X-FDA: 77854242984.18.6AC6C47 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP id 5906312E for ; Wed, 24 Feb 2021 20:06:12 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id F20C264F09; Wed, 24 Feb 2021 20:06:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197171; bh=gg4KEmsFG5PsVjT1oWzTkiBzMpx7mEfnkesuQBpNn6w=; h=Date:From:To:Subject:In-Reply-To:From; b=YMi9NJaVQBzwiAhOjTyNgezLmOIeUt6C9l7F2Oyv7YT9w4uOLe0DHT1NdhifcG93+ 3Xd8/sZCKzRpOE7/pK8ukSDmCbPFyD+rH9EWlgNLrJAZtcjgLakRKv/Xh9kWLAWhZl KcAp61yosVZ4v3IVpMd/9d8wrPkN0vzEpzT8qz8E= Date: Wed, 24 Feb 2021 12:06:10 -0800 From: Andrew Morton To: akpm@linux-foundation.org, bhe@redhat.com, david@redhat.com, linux-mm@kvack.org, lkp@intel.com, mm-commits@vger.kernel.org, rppt@linux.ibm.com, torvalds@linux-foundation.org Subject: [patch 102/173] mm: fix prototype warning from kernel test robot Message-ID: <20210224200610.xfm7vXrLJ%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: dyqg75toc96hh6wex8ef43jkdiyb4eag X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 5906312E Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf20; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197172-69086 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Baoquan He Subject: mm: fix prototype warning from kernel test robot Patch series "mm: clean up names and parameters of memmap_init_xxxx functions", v5. This patchset corrects inappropriate function names of memmap_init_xxx, and simplify parameters of functions in the code flow. And also fix a prototype warning reported by lkp. This patch (of 5); Kernel test robot calling make with 'W=1' is triggering warning like below for memmap_init_zone() function. mm/page_alloc.c:6259:23: warning: no previous prototype for 'memmap_init_zone' [-Wmissing-prototypes] 6259 | void __meminit __weak memmap_init_zone(unsigned long size, int nid, | ^~~~~~~~~~~~~~~~ Fix it by adding the function declaration in include/linux/mm.h. Since memmap_init_zone() has a generic version with '__weak', the declaratoin in ia64 header file can be simply removed. Link: https://lkml.kernel.org/r/20210122135956.5946-1-bhe@redhat.com Link: https://lkml.kernel.org/r/20210122135956.5946-2-bhe@redhat.com Signed-off-by: Baoquan He Reported-by: kernel test robot Reviewed-by: Mike Rapoport Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- arch/ia64/include/asm/pgtable.h | 6 ------ include/linux/mm.h | 2 ++ 2 files changed, 2 insertions(+), 6 deletions(-) --- a/arch/ia64/include/asm/pgtable.h~mm-fix-prototype-warning-from-kernel-test-robot +++ a/arch/ia64/include/asm/pgtable.h @@ -517,12 +517,6 @@ extern struct page *zero_page_memmap_ptr __changed; \ }) #endif - -# ifdef CONFIG_VIRTUAL_MEM_MAP - /* arch mem_map init routine is needed due to holes in a virtual mem_map */ - extern void memmap_init (unsigned long size, int nid, unsigned long zone, - unsigned long start_pfn); -# endif /* CONFIG_VIRTUAL_MEM_MAP */ # endif /* !__ASSEMBLY__ */ /* --- a/include/linux/mm.h~mm-fix-prototype-warning-from-kernel-test-robot +++ a/include/linux/mm.h @@ -2408,6 +2408,8 @@ extern void set_dma_reserve(unsigned lon extern void memmap_init_zone(unsigned long, int, unsigned long, unsigned long, unsigned long, enum meminit_context, struct vmem_altmap *, int migratetype); +extern void memmap_init(unsigned long size, int nid, + unsigned long zone, unsigned long range_start_pfn); extern void setup_per_zone_wmarks(void); extern int __meminit init_per_zone_wmark_min(void); extern void mem_init(void); From patchwork Wed Feb 24 20:06:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102617 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE82AC433DB for ; Wed, 24 Feb 2021 20:06:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8393F64F0F for ; Wed, 24 Feb 2021 20:06:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8393F64F0F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 10D756B00D0; Wed, 24 Feb 2021 15:06:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 06E148D0016; Wed, 24 Feb 2021 15:06:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E53A58D0015; Wed, 24 Feb 2021 15:06:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0199.hostedemail.com [216.40.44.199]) by kanga.kvack.org (Postfix) with ESMTP id CBF586B00D0 for ; Wed, 24 Feb 2021 15:06:16 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 92634185044BC for ; Wed, 24 Feb 2021 20:06:16 +0000 (UTC) X-FDA: 77854243152.06.F3C6631 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP id 10B12C0007E1 for ; Wed, 24 Feb 2021 20:06:12 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A984964F27; Wed, 24 Feb 2021 20:06:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197175; bh=uAp2YeH2AOCJhYps1y8zP5wSEtDhpWAfcFM3KrhDJOg=; h=Date:From:To:Subject:In-Reply-To:From; b=l9gOO7/fKw9BYkUbNtG9Ea9x5RyVCmalGNMo7bEo0TN9y76QIfXxBxTAY2/7Zuddb MvAuRc4uG6ufN+qwx3HE5T1glcTdm5HMrlERkOT4aBDHaRD8vXQ0xGwdwtAkBAf7s9 34RfTSACc46c0TUmTV+awu+nFgcW8hldPVqfPMFc= Date: Wed, 24 Feb 2021 12:06:14 -0800 From: Andrew Morton To: akpm@linux-foundation.org, bhe@redhat.com, david@redhat.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rppt@linux.ibm.com, torvalds@linux-foundation.org Subject: [patch 103/173] mm: rename memmap_init() and memmap_init_zone() Message-ID: <20210224200614.YSLkrcSIr%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 57k67aw415kdk95ephkxpkoittbsn5oj X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 10B12C0007E1 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf03; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197172-193880 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Baoquan He Subject: mm: rename memmap_init() and memmap_init_zone() The current memmap_init_zone() only handles memory region inside one zone, actually memmap_init() does the memmap init of one zone. So rename both of them accordingly. Link: https://lkml.kernel.org/r/20210122135956.5946-3-bhe@redhat.com Signed-off-by: Baoquan He Reviewed-by: Mike Rapoport Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- arch/ia64/mm/init.c | 6 +++--- include/linux/mm.h | 4 ++-- mm/memory_hotplug.c | 2 +- mm/page_alloc.c | 8 ++++---- 4 files changed, 10 insertions(+), 10 deletions(-) --- a/arch/ia64/mm/init.c~mm-rename-memmap_init-and-memmap_init_zone +++ a/arch/ia64/mm/init.c @@ -536,18 +536,18 @@ virtual_memmap_init(u64 start, u64 end, / sizeof(struct page)); if (map_start < map_end) - memmap_init_zone((unsigned long)(map_end - map_start), + memmap_init_range((unsigned long)(map_end - map_start), args->nid, args->zone, page_to_pfn(map_start), page_to_pfn(map_end), MEMINIT_EARLY, NULL, MIGRATE_MOVABLE); return 0; } void __meminit -memmap_init (unsigned long size, int nid, unsigned long zone, +memmap_init_zone(unsigned long size, int nid, unsigned long zone, unsigned long start_pfn) { if (!vmem_map) { - memmap_init_zone(size, nid, zone, start_pfn, start_pfn + size, + memmap_init_range(size, nid, zone, start_pfn, start_pfn + size, MEMINIT_EARLY, NULL, MIGRATE_MOVABLE); } else { struct page *start; --- a/include/linux/mm.h~mm-rename-memmap_init-and-memmap_init_zone +++ a/include/linux/mm.h @@ -2405,10 +2405,10 @@ extern int __meminit early_pfn_to_nid(un #endif extern void set_dma_reserve(unsigned long new_dma_reserve); -extern void memmap_init_zone(unsigned long, int, unsigned long, +extern void memmap_init_range(unsigned long, int, unsigned long, unsigned long, unsigned long, enum meminit_context, struct vmem_altmap *, int migratetype); -extern void memmap_init(unsigned long size, int nid, +extern void memmap_init_zone(unsigned long size, int nid, unsigned long zone, unsigned long range_start_pfn); extern void setup_per_zone_wmarks(void); extern int __meminit init_per_zone_wmark_min(void); --- a/mm/memory_hotplug.c~mm-rename-memmap_init-and-memmap_init_zone +++ a/mm/memory_hotplug.c @@ -713,7 +713,7 @@ void __ref move_pfn_range_to_zone(struct * expects the zone spans the pfn range. All the pages in the range * are reserved so nobody should be touching them so we should be safe */ - memmap_init_zone(nr_pages, nid, zone_idx(zone), start_pfn, 0, + memmap_init_range(nr_pages, nid, zone_idx(zone), start_pfn, 0, MEMINIT_HOTPLUG, altmap, migratetype); set_zone_contiguous(zone); --- a/mm/page_alloc.c~mm-rename-memmap_init-and-memmap_init_zone +++ a/mm/page_alloc.c @@ -6121,7 +6121,7 @@ overlap_memmap_init(unsigned long zone, * (usually MIGRATE_MOVABLE). Besides setting the migratetype, no related * zone stats (e.g., nr_isolate_pageblock) are touched. */ -void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone, +void __meminit memmap_init_range(unsigned long size, int nid, unsigned long zone, unsigned long start_pfn, unsigned long zone_end_pfn, enum meminit_context context, struct vmem_altmap *altmap, int migratetype) @@ -6258,7 +6258,7 @@ static void __meminit zone_init_free_lis } } -void __meminit __weak memmap_init(unsigned long size, int nid, +void __meminit __weak memmap_init_zone(unsigned long size, int nid, unsigned long zone, unsigned long range_start_pfn) { @@ -6272,7 +6272,7 @@ void __meminit __weak memmap_init(unsign if (end_pfn > start_pfn) { size = end_pfn - start_pfn; - memmap_init_zone(size, nid, zone, start_pfn, range_end_pfn, + memmap_init_range(size, nid, zone, start_pfn, range_end_pfn, MEMINIT_EARLY, NULL, MIGRATE_MOVABLE); } } @@ -6982,7 +6982,7 @@ static void __init free_area_init_core(s set_pageblock_order(); setup_usemap(pgdat, zone, zone_start_pfn, size); init_currently_empty_zone(zone, zone_start_pfn, size); - memmap_init(size, nid, j, zone_start_pfn); + memmap_init_zone(size, nid, j, zone_start_pfn); } } From patchwork Wed Feb 24 20:06:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102619 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1FFEAC433E9 for ; Wed, 24 Feb 2021 20:06:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B96CD64F08 for ; Wed, 24 Feb 2021 20:06:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B96CD64F08 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 48F016B00D2; Wed, 24 Feb 2021 15:06:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 418978D0016; Wed, 24 Feb 2021 15:06:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 306978D0015; Wed, 24 Feb 2021 15:06:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0126.hostedemail.com [216.40.44.126]) by kanga.kvack.org (Postfix) with ESMTP id 163156B00D2 for ; Wed, 24 Feb 2021 15:06:20 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id D3A6A185044C9 for ; Wed, 24 Feb 2021 20:06:19 +0000 (UTC) X-FDA: 77854243278.04.B78579F Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP id 10A5980192DA for ; Wed, 24 Feb 2021 20:06:12 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 2D2ED64F0F; Wed, 24 Feb 2021 20:06:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197178; bh=NkZ36CXExD/jG4Elx6ZSK/DhOKkW17k04iCAoMGaVlQ=; h=Date:From:To:Subject:In-Reply-To:From; b=CPT/Kp9sLndW2VjRIbABZ2bmDdiRchOZd52CmlJSVrdBe7clEySwdB2yHQMA1DkgP kstT4qWoG1mZ5WcHRB3Fu9cB1RxgSzCE1EhuoZUmqeKCh8m+1PLKWDvSpS9dwNiosg wCjdQEy8lKZpXLf0D0Oqw2c1QdK9mrx1H9DWhqqQ= Date: Wed, 24 Feb 2021 12:06:17 -0800 From: Andrew Morton To: akpm@linux-foundation.org, bhe@redhat.com, david@redhat.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rppt@linux.ibm.com, torvalds@linux-foundation.org Subject: [patch 104/173] mm: simplify parater of function memmap_init_zone() Message-ID: <20210224200617.MCRE7bWj9%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: z818xdjktdjiswa4inxrfakhm9q6momp X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 10A5980192DA Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf27; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197172-432948 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Baoquan He Subject: mm: simplify parater of function memmap_init_zone() As David suggested, simply passing 'struct zone *zone' is enough. We can get all needed information from 'struct zone*' easily. Link: https://lkml.kernel.org/r/20210122135956.5946-4-bhe@redhat.com Signed-off-by: Baoquan He Suggested-by: David Hildenbrand Reviewed-by: Mike Rapoport Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- arch/ia64/mm/init.c | 12 +++++++----- include/linux/mm.h | 3 +-- mm/page_alloc.c | 24 +++++++++++------------- 3 files changed, 19 insertions(+), 20 deletions(-) --- a/arch/ia64/mm/init.c~mm-simplify-parater-of-function-memmap_init_zone +++ a/arch/ia64/mm/init.c @@ -542,12 +542,14 @@ virtual_memmap_init(u64 start, u64 end, return 0; } -void __meminit -memmap_init_zone(unsigned long size, int nid, unsigned long zone, - unsigned long start_pfn) +void __meminit memmap_init_zone(struct zone *zone) { + int nid = zone_to_nid(zone), zone_id = zone_idx(zone); + unsigned long start_pfn = zone->zone_start_pfn; + unsigned long size = zone->spanned_pages; + if (!vmem_map) { - memmap_init_range(size, nid, zone, start_pfn, start_pfn + size, + memmap_init_range(size, nid, zone_id, start_pfn, start_pfn + size, MEMINIT_EARLY, NULL, MIGRATE_MOVABLE); } else { struct page *start; @@ -557,7 +559,7 @@ memmap_init_zone(unsigned long size, int args.start = start; args.end = start + size; args.nid = nid; - args.zone = zone; + args.zone = zone_id; efi_memmap_walk(virtual_memmap_init, &args); } --- a/include/linux/mm.h~mm-simplify-parater-of-function-memmap_init_zone +++ a/include/linux/mm.h @@ -2408,8 +2408,7 @@ extern void set_dma_reserve(unsigned lon extern void memmap_init_range(unsigned long, int, unsigned long, unsigned long, unsigned long, enum meminit_context, struct vmem_altmap *, int migratetype); -extern void memmap_init_zone(unsigned long size, int nid, - unsigned long zone, unsigned long range_start_pfn); +extern void memmap_init_zone(struct zone *zone); extern void setup_per_zone_wmarks(void); extern int __meminit init_per_zone_wmark_min(void); extern void mem_init(void); --- a/mm/page_alloc.c~mm-simplify-parater-of-function-memmap_init_zone +++ a/mm/page_alloc.c @@ -6258,23 +6258,21 @@ static void __meminit zone_init_free_lis } } -void __meminit __weak memmap_init_zone(unsigned long size, int nid, - unsigned long zone, - unsigned long range_start_pfn) +void __meminit __weak memmap_init_zone(struct zone *zone) { + unsigned long zone_start_pfn = zone->zone_start_pfn; + unsigned long zone_end_pfn = zone_start_pfn + zone->spanned_pages; + int i, nid = zone_to_nid(zone), zone_id = zone_idx(zone); unsigned long start_pfn, end_pfn; - unsigned long range_end_pfn = range_start_pfn + size; - int i; for_each_mem_pfn_range(i, nid, &start_pfn, &end_pfn, NULL) { - start_pfn = clamp(start_pfn, range_start_pfn, range_end_pfn); - end_pfn = clamp(end_pfn, range_start_pfn, range_end_pfn); + start_pfn = clamp(start_pfn, zone_start_pfn, zone_end_pfn); + end_pfn = clamp(end_pfn, zone_start_pfn, zone_end_pfn); - if (end_pfn > start_pfn) { - size = end_pfn - start_pfn; - memmap_init_range(size, nid, zone, start_pfn, range_end_pfn, - MEMINIT_EARLY, NULL, MIGRATE_MOVABLE); - } + if (end_pfn > start_pfn) + memmap_init_range(end_pfn - start_pfn, nid, + zone_id, start_pfn, zone_end_pfn, + MEMINIT_EARLY, NULL, MIGRATE_MOVABLE); } } @@ -6982,7 +6980,7 @@ static void __init free_area_init_core(s set_pageblock_order(); setup_usemap(pgdat, zone, zone_start_pfn, size); init_currently_empty_zone(zone, zone_start_pfn, size); - memmap_init_zone(size, nid, j, zone_start_pfn); + memmap_init_zone(zone); } } From patchwork Wed Feb 24 20:06:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102621 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02489C433E0 for ; Wed, 24 Feb 2021 20:06:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 924BE64F08 for ; Wed, 24 Feb 2021 20:06:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 924BE64F08 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 32F556B00D4; Wed, 24 Feb 2021 15:06:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 269A18D0016; Wed, 24 Feb 2021 15:06:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 17F0C8D0015; Wed, 24 Feb 2021 15:06:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0146.hostedemail.com [216.40.44.146]) by kanga.kvack.org (Postfix) with ESMTP id ECCBA6B00D4 for ; Wed, 24 Feb 2021 15:06:23 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id A10A0184D8FB3 for ; Wed, 24 Feb 2021 20:06:23 +0000 (UTC) X-FDA: 77854243446.16.612E14E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf01.hostedemail.com (Postfix) with ESMTP id 92B3E200038F for ; Wed, 24 Feb 2021 20:06:22 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0330A64F14; Wed, 24 Feb 2021 20:06:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197182; bh=1EFjS2w3Hgd8OC0s/XzvhRFBsK9utEtfBSJV2hJHbcw=; h=Date:From:To:Subject:In-Reply-To:From; b=mja8ATPF0iXrqFKUOVBcMCd2BIbVwxS86zBQtDgi4u3hw7+xvML9tlGGGUtcE2i4Z BOmosPTZxygtIH26RFvXO2rCX8X4q5GkwrBNh8rh53B841CFDaDoUOnhTBvQyKbLPm axScWdF6YlV13sgzXTseeEQ1EXcAWUciBwbRgtD8= Date: Wed, 24 Feb 2021 12:06:20 -0800 From: Andrew Morton To: akpm@linux-foundation.org, bhe@redhat.com, david@redhat.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rppt@linux.ibm.com, torvalds@linux-foundation.org Subject: [patch 105/173] mm: simplify parameter of setup_usemap() Message-ID: <20210224200620.gvrD3A1BS%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 92B3E200038F X-Stat-Signature: p4g83e8yununfmrf84o4rk1k1hffpc5c Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf01; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197182-753954 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Baoquan He Subject: mm: simplify parameter of setup_usemap() Parameter 'zone' has got needed information, let's remove other unnecessary parameters. Link: https://lkml.kernel.org/r/20210122135956.5946-5-bhe@redhat.com Signed-off-by: Baoquan He Reviewed-by: Mike Rapoport Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- mm/page_alloc.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-) --- a/mm/page_alloc.c~mm-simplify-parameter-of-setup_usemap +++ a/mm/page_alloc.c @@ -6765,25 +6765,22 @@ static unsigned long __init usemap_size( return usemapsize / 8; } -static void __ref setup_usemap(struct pglist_data *pgdat, - struct zone *zone, - unsigned long zone_start_pfn, - unsigned long zonesize) +static void __ref setup_usemap(struct zone *zone) { - unsigned long usemapsize = usemap_size(zone_start_pfn, zonesize); + unsigned long usemapsize = usemap_size(zone->zone_start_pfn, + zone->spanned_pages); zone->pageblock_flags = NULL; if (usemapsize) { zone->pageblock_flags = memblock_alloc_node(usemapsize, SMP_CACHE_BYTES, - pgdat->node_id); + zone_to_nid(zone)); if (!zone->pageblock_flags) panic("Failed to allocate %ld bytes for zone %s pageblock flags on node %d\n", - usemapsize, zone->name, pgdat->node_id); + usemapsize, zone->name, zone_to_nid(zone)); } } #else -static inline void setup_usemap(struct pglist_data *pgdat, struct zone *zone, - unsigned long zone_start_pfn, unsigned long zonesize) {} +static inline void setup_usemap(struct zone *zone) {} #endif /* CONFIG_SPARSEMEM */ #ifdef CONFIG_HUGETLB_PAGE_SIZE_VARIABLE @@ -6978,7 +6975,7 @@ static void __init free_area_init_core(s continue; set_pageblock_order(); - setup_usemap(pgdat, zone, zone_start_pfn, size); + setup_usemap(zone); init_currently_empty_zone(zone, zone_start_pfn, size); memmap_init_zone(zone); } From patchwork Wed Feb 24 20:06:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 548BCC433DB for ; Wed, 24 Feb 2021 20:06:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D9E4D64F25 for ; Wed, 24 Feb 2021 20:06:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D9E4D64F25 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6B0598D0015; Wed, 24 Feb 2021 15:06:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 63ACC6B00D7; Wed, 24 Feb 2021 15:06:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4DC458D0015; Wed, 24 Feb 2021 15:06:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0123.hostedemail.com [216.40.44.123]) by kanga.kvack.org (Postfix) with ESMTP id 320996B00D6 for ; Wed, 24 Feb 2021 15:06:27 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id DCD6D185044C9 for ; Wed, 24 Feb 2021 20:06:26 +0000 (UTC) X-FDA: 77854243572.23.B6BC2B1 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP id A0A2A90009D6 for ; Wed, 24 Feb 2021 20:06:22 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 442D764F08; Wed, 24 Feb 2021 20:06:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197185; bh=aI14ldqlQPq3LB4iu84l53EENUlUFQywOomiUM/t7KU=; h=Date:From:To:Subject:In-Reply-To:From; b=SRh3XejzM+L880q567IwSb/wyIhRmO5vGCU5q60/YydELoJumjXGOPeMrl4Uu0WLW F/FPzoBlls9pzVFX7GuNSP6mi4AJwhNj6ySkpZggnd35mroI8yvL2KCwD/ZjhSKyNE mJdYCO7HWG0T67BSY/w5/OLpVmshTrktcorm49qg= Date: Wed, 24 Feb 2021 12:06:24 -0800 From: Andrew Morton To: akpm@linux-foundation.org, bhe@redhat.com, david@redhat.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, rppt@linux.ibm.com, torvalds@linux-foundation.org Subject: [patch 106/173] mm: remove unneeded local variable in free_area_init_core Message-ID: <20210224200624.6O7uelaSh%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: fh73x3hdxxgsirdodafk1nxm8jjbdybk X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: A0A2A90009D6 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf19; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197182-626379 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Baoquan He Subject: mm: remove unneeded local variable in free_area_init_core Local variable 'zone_start_pfn' is not needed since there's only one call site in free_area_init_core(). Let's remove it and pass zone->zone_start_pfn directly to init_currently_empty_zone(). Link: https://lkml.kernel.org/r/20210122135956.5946-6-bhe@redhat.com Signed-off-by: Baoquan He Reviewed-by: Mike Rapoport Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- mm/page_alloc.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) --- a/mm/page_alloc.c~mm-remove-unneeded-local-variable-in-free_area_init_core +++ a/mm/page_alloc.c @@ -6927,7 +6927,6 @@ static void __init free_area_init_core(s for (j = 0; j < MAX_NR_ZONES; j++) { struct zone *zone = pgdat->node_zones + j; unsigned long size, freesize, memmap_pages; - unsigned long zone_start_pfn = zone->zone_start_pfn; size = zone->spanned_pages; freesize = zone->present_pages; @@ -6976,7 +6975,7 @@ static void __init free_area_init_core(s set_pageblock_order(); setup_usemap(zone); - init_currently_empty_zone(zone, zone_start_pfn, size); + init_currently_empty_zone(zone, zone->zone_start_pfn, size); memmap_init_zone(zone); } } From patchwork Wed Feb 24 20:06:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102625 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4EBAC433DB for ; Wed, 24 Feb 2021 20:06:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 44D9E64F25 for ; Wed, 24 Feb 2021 20:06:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 44D9E64F25 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C7D228D0016; Wed, 24 Feb 2021 15:06:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C077F6B00D8; Wed, 24 Feb 2021 15:06:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ACF5D8D0016; Wed, 24 Feb 2021 15:06:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0218.hostedemail.com [216.40.44.218]) by kanga.kvack.org (Postfix) with ESMTP id 92C5B6B00D7 for ; Wed, 24 Feb 2021 15:06:31 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 5CFA0D20A for ; Wed, 24 Feb 2021 20:06:31 +0000 (UTC) X-FDA: 77854243782.30.B050BBB Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP id 79EC32BC0 for ; Wed, 24 Feb 2021 20:06:30 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id D309464F2E; Wed, 24 Feb 2021 20:06:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197189; bh=2iYM7+rPPzb9zRWJHyALLlK17AlvPdA0VGoBKJqkNl0=; h=Date:From:To:Subject:In-Reply-To:From; b=sglWpnpvceFKaln0vF7TE+rMBorTO46C1sycoaEkHgeKUVMiVNbZyhS1Vs6BPSDnu w0/joEXelTJZkkseaLKpm5Ry/Y/rT0EztIdnEsbioJkpxV0Pmvr2snTqx2sSIreTzN FoDJXFfphM1hvNn9cmR5CGP4l1FGbLoVTyPxQT2s= Date: Wed, 24 Feb 2021 12:06:28 -0800 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, david@redhat.com, gustavoars@kernel.org, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, osalvador@suse.de, peterz@infradead.org, richard.weiyang@linux.alibaba.com, rppt@kernel.org, sam@ravnborg.org, tglx@linutronix.de, torvalds@linux-foundation.org Subject: [patch 107/173] video: fbdev: acornfb: remove free_unused_pages() Message-ID: <20210224200628.3noD6aLz6%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: r6j77ct43aajbozotqe4moufn6e9kx9i X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 79EC32BC0 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf29; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197190-872666 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: video: fbdev: acornfb: remove free_unused_pages() Patch series "mm: simplify free_highmem_page() and free_reserved_page()". Let's simplify and unify free_highmem_page() and free_reserved_page(). This patch (of 2): This function is never used and it is one of the last remaining user of __free_reserved_page(). Let's just drop it. Link: https://lkml.kernel.org/r/20210126182113.19892-1-david@redhat.com Link: https://lkml.kernel.org/r/20210126182113.19892-2-david@redhat.com Fixes: ffd29195ed720188 ("drivers/video/acornfb.c: remove dead code") Signed-off-by: David Hildenbrand Reviewed-by: Oscar Salvador Reviewed-by: Anshuman Khandual Cc: Thomas Gleixner Cc: "Peter Zijlstra (Intel)" Cc: Mike Rapoport Cc: Michal Hocko Cc: Wei Yang Cc: "Gustavo A. R. Silva" Cc: Sam Ravnborg Signed-off-by: Andrew Morton --- drivers/video/fbdev/acornfb.c | 34 -------------------------------- 1 file changed, 34 deletions(-) --- a/drivers/video/fbdev/acornfb.c~video-fbdev-acornfb-remove-free_unused_pages +++ a/drivers/video/fbdev/acornfb.c @@ -921,40 +921,6 @@ static int acornfb_detect_monitortype(vo return 4; } -/* - * This enables the unused memory to be freed on older Acorn machines. - * We are freeing memory on behalf of the architecture initialisation - * code here. - */ -static inline void -free_unused_pages(unsigned int virtual_start, unsigned int virtual_end) -{ - int mb_freed = 0; - - /* - * Align addresses - */ - virtual_start = PAGE_ALIGN(virtual_start); - virtual_end = PAGE_ALIGN(virtual_end); - - while (virtual_start < virtual_end) { - struct page *page; - - /* - * Clear page reserved bit, - * set count to 1, and free - * the page. - */ - page = virt_to_page(virtual_start); - __free_reserved_page(page); - - virtual_start += PAGE_SIZE; - mb_freed += PAGE_SIZE / 1024; - } - - printk("acornfb: freed %dK memory\n", mb_freed); -} - static int acornfb_probe(struct platform_device *dev) { unsigned long size; From patchwork Wed Feb 24 20:06:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102627 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8915BC433E0 for ; Wed, 24 Feb 2021 20:06:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 07B7A64F2C for ; Wed, 24 Feb 2021 20:06:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 07B7A64F2C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8395D8D0017; Wed, 24 Feb 2021 15:06:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E7C36B00D9; Wed, 24 Feb 2021 15:06:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6AF298D0017; Wed, 24 Feb 2021 15:06:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0128.hostedemail.com [216.40.44.128]) by kanga.kvack.org (Postfix) with ESMTP id 50C6B6B00D8 for ; Wed, 24 Feb 2021 15:06:35 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 1F64DB9EC for ; Wed, 24 Feb 2021 20:06:35 +0000 (UTC) X-FDA: 77854243950.05.AFA622D Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf13.hostedemail.com (Postfix) with ESMTP id 40869E000104 for ; Wed, 24 Feb 2021 20:06:31 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id ACA5564F26; Wed, 24 Feb 2021 20:06:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197193; bh=ZqorPdS+5o6z98fVqJGmBGjcGtgCA8dZsxbVd3l6lzE=; h=Date:From:To:Subject:In-Reply-To:From; b=iGlRr2b+sFDgca7zeL+fh9dO6rwbBghk5878IpMTVOwtVJi6ud9R3CxWiM1KaZ73L 2fMEPgVZDmJcGimhuzsFydbO7NfxOTe+MHHG5nXdDr1ukYGtdJT60hapgU/5z0BNAi bQ9xKr9D7QSXZUK/4iuDWsWzTfHUS11yE92yN59s= Date: Wed, 24 Feb 2021 12:06:32 -0800 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, david@redhat.com, gustavoars@kernel.org, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, osalvador@suse.de, peterz@infradead.org, richard.weiyang@linux.alibaba.com, rppt@kernel.org, sam@ravnborg.org, tglx@linutronix.de, torvalds@linux-foundation.org Subject: [patch 108/173] mm: simplify free_highmem_page() and free_reserved_page() Message-ID: <20210224200632.FywpPUWVq%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 40869E000104 X-Stat-Signature: t817t5nw9i5tsagqeu9aw4tgfuwdpi8t Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf13; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197191-606896 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: David Hildenbrand Subject: mm: simplify free_highmem_page() and free_reserved_page() adjust_managed_page_count() as called by free_reserved_page() properly handles pages in a highmem zone, so we can reuse it for free_highmem_page(). We can now get rid of totalhigh_pages_inc() and simplify free_reserved_page(). Link: https://lkml.kernel.org/r/20210126182113.19892-3-david@redhat.com Signed-off-by: David Hildenbrand Reviewed-by: Oscar Salvador Reviewed-by: Anshuman Khandual Cc: Thomas Gleixner Cc: "Peter Zijlstra (Intel)" Cc: Mike Rapoport Cc: Michal Hocko Cc: Wei Yang Cc: "Gustavo A. R. Silva" Cc: Sam Ravnborg Signed-off-by: Andrew Morton --- include/linux/highmem-internal.h | 5 ----- include/linux/mm.h | 16 ++-------------- mm/page_alloc.c | 11 ----------- 3 files changed, 2 insertions(+), 30 deletions(-) --- a/include/linux/highmem-internal.h~mm-simplify-free_highmem_page-and-free_reserved_page +++ a/include/linux/highmem-internal.h @@ -127,11 +127,6 @@ static inline unsigned long totalhigh_pa return (unsigned long)atomic_long_read(&_totalhigh_pages); } -static inline void totalhigh_pages_inc(void) -{ - atomic_long_inc(&_totalhigh_pages); -} - static inline void totalhigh_pages_add(long count) { atomic_long_add(count, &_totalhigh_pages); --- a/include/linux/mm.h~mm-simplify-free_highmem_page-and-free_reserved_page +++ a/include/linux/mm.h @@ -2310,32 +2310,20 @@ extern void free_initmem(void); extern unsigned long free_reserved_area(void *start, void *end, int poison, const char *s); -#ifdef CONFIG_HIGHMEM -/* - * Free a highmem page into the buddy system, adjusting totalhigh_pages - * and totalram_pages. - */ -extern void free_highmem_page(struct page *page); -#endif - extern void adjust_managed_page_count(struct page *page, long count); extern void mem_init_print_info(const char *str); extern void reserve_bootmem_region(phys_addr_t start, phys_addr_t end); /* Free the reserved page into the buddy system, so it gets managed. */ -static inline void __free_reserved_page(struct page *page) +static inline void free_reserved_page(struct page *page) { ClearPageReserved(page); init_page_count(page); __free_page(page); -} - -static inline void free_reserved_page(struct page *page) -{ - __free_reserved_page(page); adjust_managed_page_count(page, 1); } +#define free_highmem_page(page) free_reserved_page(page) static inline void mark_page_reserved(struct page *page) { --- a/mm/page_alloc.c~mm-simplify-free_highmem_page-and-free_reserved_page +++ a/mm/page_alloc.c @@ -7691,17 +7691,6 @@ unsigned long free_reserved_area(void *s return pages; } -#ifdef CONFIG_HIGHMEM -void free_highmem_page(struct page *page) -{ - __free_reserved_page(page); - totalram_pages_inc(); - atomic_long_inc(&page_zone(page)->managed_pages); - totalhigh_pages_inc(); -} -#endif - - void __init mem_init_print_info(const char *str) { unsigned long physpages, codesize, datasize, rosize, bss_size; From patchwork Wed Feb 24 20:06:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102629 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A0672C433E6 for ; Wed, 24 Feb 2021 20:06:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 34DD964F25 for ; Wed, 24 Feb 2021 20:06:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 34DD964F25 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C2E3E8D0018; Wed, 24 Feb 2021 15:06:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BB97B6B00DA; Wed, 24 Feb 2021 15:06:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA71B8D0018; Wed, 24 Feb 2021 15:06:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0143.hostedemail.com [216.40.44.143]) by kanga.kvack.org (Postfix) with ESMTP id 8D45C6B00D9 for ; Wed, 24 Feb 2021 15:06:38 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 51549184D8FB3 for ; Wed, 24 Feb 2021 20:06:38 +0000 (UTC) X-FDA: 77854244076.05.2BC2DE2 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id 20713FD for ; Wed, 24 Feb 2021 20:06:31 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 82C8364F3A; Wed, 24 Feb 2021 20:06:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197196; bh=S0oS6KcS8t0bMJvL3VnGBGEPzhOsu8h+HkMYXUqvW2Y=; h=Date:From:To:Subject:In-Reply-To:From; b=TE75duOwfU7grRNPC5L5rn88ZXXmNTHGlKwSw2RYcogd+nCWEY+ehZ84wp2h+5OuK 5HulX7QCutHP7ZAtOlk0qypcGFstnZIuK/vqDYUwgOuMm418CvOEV4qmZmPhyAqUVw NZazUqx6XOjZjVhjqUFLyFrr8+uOYBhyu8/due/s= Date: Wed, 24 Feb 2021 12:06:35 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, nathan@kernel.org, rppt@linux.ibm.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 109/173] mm/gfp: add kernel-doc for gfp_t Message-ID: <20210224200635.7ADCBJmnK%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 20713FD X-Stat-Signature: 5y43nz8o1pq9rtkzdgzh94xpcjw9gqxy Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf12; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197191-390966 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Matthew Wilcox (Oracle)" Subject: mm/gfp: add kernel-doc for gfp_t The generated html will link to the definition of the gfp_t automatically once we define it. Move the one-paragraph overview of GFP flags from the documentation directory into gfp.h and pull gfp.h into the documentation. This generates warnings with clang (https://lkml.kernel.org/r/20210219195509.GA59987@24bbad8f3778), so use a #if 0 to hide it from the compiler for now. Link: https://lkml.kernel.org/r/20210215204909.3824509-1-willy@infradead.org Link: https://lkml.kernel.org/r/20210220003049.GZ2858050@casper.infradead.org Signed-off-by: Matthew Wilcox (Oracle) Acked-by: Mike Rapoport Cc: Nathan Chancellor Signed-off-by: Andrew Morton --- Documentation/core-api/mm-api.rst | 7 ++----- include/linux/gfp.h | 14 ++++++++++++++ 2 files changed, 16 insertions(+), 5 deletions(-) --- a/Documentation/core-api/mm-api.rst~mm-gfp-add-kernel-doc-for-gfp_t +++ a/Documentation/core-api/mm-api.rst @@ -19,11 +19,8 @@ User Space Memory Access Memory Allocation Controls ========================== -Functions which need to allocate memory often use GFP flags to express -how that memory should be allocated. The GFP acronym stands for "get -free pages", the underlying memory allocation function. Not every GFP -flag is allowed to every function which may allocate memory. Most -users will want to use a plain ``GFP_KERNEL``. +.. kernel-doc:: include/linux/gfp.h + :internal: .. kernel-doc:: include/linux/gfp.h :doc: Page mobility and placement hints --- a/include/linux/gfp.h~mm-gfp-add-kernel-doc-for-gfp_t +++ a/include/linux/gfp.h @@ -8,6 +8,20 @@ #include #include +/* The typedef is in types.h but we want the documentation here */ +#if 0 +/** + * typedef gfp_t - Memory allocation flags. + * + * GFP flags are commonly used throughout Linux to indicate how memory + * should be allocated. The GFP acronym stands for get_free_pages(), + * the underlying memory allocation function. Not every GFP flag is + * supported by every function which may allocate memory. Most users + * will want to use a plain ``GFP_KERNEL``. + */ +typedef unsigned int __bitwise gfp_t; +#endif + struct vm_area_struct; /* From patchwork Wed Feb 24 20:06:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102631 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23283C433E0 for ; Wed, 24 Feb 2021 20:06:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9DA9164E24 for ; Wed, 24 Feb 2021 20:06:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9DA9164E24 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2DFEE8D0019; Wed, 24 Feb 2021 15:06:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 268926B00DB; Wed, 24 Feb 2021 15:06:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 130B28D0019; Wed, 24 Feb 2021 15:06:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0009.hostedemail.com [216.40.44.9]) by kanga.kvack.org (Postfix) with ESMTP id ED2336B00DA for ; Wed, 24 Feb 2021 15:06:41 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id B7667EFCA for ; Wed, 24 Feb 2021 20:06:41 +0000 (UTC) X-FDA: 77854244202.26.F42B461 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf16.hostedemail.com (Postfix) with ESMTP id 9F79B80192D4 for ; Wed, 24 Feb 2021 20:06:40 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id E1D4864F2C; Wed, 24 Feb 2021 20:06:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197200; bh=KmSwyPcLxnCFmzNRnp4bpCLTmQgNmwGXX4pXGDnpKmU=; h=Date:From:To:Subject:In-Reply-To:From; b=zcOddZuEJE0pUFIbfPh1Iv/+UI6OvR8DudN+LdO6qkibgyzI6cdZNfm1yX6BU5r+i k235VOmkrD5iEjoQGcUWCArPQ5Q1/EHSYqU4ZyWAQLImqfERDN5SP3446ipSukqHIn dQFkLtcwmynd6mJNsS/Scfcu2iINvYJLMAbGnb1w= Date: Wed, 24 Feb 2021 12:06:39 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, naoya.horiguchi@nec.com, osalvador@suse.de, torvalds@linux-foundation.org, yaoaili@kingsoft.com Subject: [patch 110/173] mm,hwpoison: send SIGBUS to PF_MCE_EARLY processes on action required events Message-ID: <20210224200639.x12MQxc9B%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 9F79B80192D4 X-Stat-Signature: jxhydoc8ttfsbk91r8ycztrb3pe6sqwo Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197200-54173 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Aili Yao Subject: mm,hwpoison: send SIGBUS to PF_MCE_EARLY processes on action required events When a memory uncorrected error is triggered by process who accessed the address with error, It's Action Required Case for only current process which triggered this; This Action Required case means Action optional to other process who share the same page. Usually killing current process will be sufficient, other processes sharing the same page will get be signaled when they really touch the poisoned page. But there is another scenario that other processes sharing the same page want to be signaled early with PF_MCE_EARLY set. In this case, we should get them into kill list and signal BUS_MCEERR_AO to them. So in this patch, task_early_kill will check current process if force_early is set, and if not current,the code will fallback to find_early_kill_thread() to check if there is PF_MCE_EARLY process who cares the error. In kill_proc(), BUS_MCEERR_AR is only send to current, other processes in kill list will be signaled with BUS_MCEERR_AO. Link: https://lkml.kernel.org/r/20210122132424.313c8f5f.yaoaili@kingsoft.com Signed-off-by: Aili Yao Reviewed-by: Oscar Salvador Acked-by: Naoya Horiguchi Signed-off-by: Andrew Morton --- mm/memory-failure.c | 34 +++++++++++++++++++--------------- 1 file changed, 19 insertions(+), 15 deletions(-) --- a/mm/memory-failure.c~mmhwpoison-send-sigbus-to-pf_mce_early-processes-on-action-required-events +++ a/mm/memory-failure.c @@ -243,9 +243,13 @@ static int kill_proc(struct to_kill *tk, pfn, t->comm, t->pid); if (flags & MF_ACTION_REQUIRED) { - WARN_ON_ONCE(t != current); - ret = force_sig_mceerr(BUS_MCEERR_AR, + if (t == current) + ret = force_sig_mceerr(BUS_MCEERR_AR, (void __user *)tk->addr, addr_lsb); + else + /* Signal other processes sharing the page if they have PF_MCE_EARLY set. */ + ret = send_sig_mceerr(BUS_MCEERR_AO, (void __user *)tk->addr, + addr_lsb, t); } else { /* * Don't use force here, it's convenient if the signal @@ -440,26 +444,26 @@ static struct task_struct *find_early_ki * Determine whether a given process is "early kill" process which expects * to be signaled when some page under the process is hwpoisoned. * Return task_struct of the dedicated thread (main thread unless explicitly - * specified) if the process is "early kill," and otherwise returns NULL. + * specified) if the process is "early kill" and otherwise returns NULL. * - * Note that the above is true for Action Optional case, but not for Action - * Required case where SIGBUS should sent only to the current thread. + * Note that the above is true for Action Optional case. For Action Required + * case, it's only meaningful to the current thread which need to be signaled + * with SIGBUS, this error is Action Optional for other non current + * processes sharing the same error page,if the process is "early kill", the + * task_struct of the dedicated thread will also be returned. */ static struct task_struct *task_early_kill(struct task_struct *tsk, int force_early) { if (!tsk->mm) return NULL; - if (force_early) { - /* - * Comparing ->mm here because current task might represent - * a subthread, while tsk always points to the main thread. - */ - if (tsk->mm == current->mm) - return current; - else - return NULL; - } + /* + * Comparing ->mm here because current task might represent + * a subthread, while tsk always points to the main thread. + */ + if (force_early && tsk->mm == current->mm) + return current; + return find_early_kill_thread(tsk); } From patchwork Wed Feb 24 20:06:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102633 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB9A7C433E6 for ; Wed, 24 Feb 2021 20:06:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 78E2464E6C for ; Wed, 24 Feb 2021 20:06:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 78E2464E6C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0E3648D001A; Wed, 24 Feb 2021 15:06:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 06C676B00DC; Wed, 24 Feb 2021 15:06:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E9DE68D001A; Wed, 24 Feb 2021 15:06:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0057.hostedemail.com [216.40.44.57]) by kanga.kvack.org (Postfix) with ESMTP id D06CD6B00DB for ; Wed, 24 Feb 2021 15:06:45 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 9F3B7EFBE for ; Wed, 24 Feb 2021 20:06:45 +0000 (UTC) X-FDA: 77854244370.13.65FFC00 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf16.hostedemail.com (Postfix) with ESMTP id 86CE780192E9 for ; Wed, 24 Feb 2021 20:06:44 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7510A64E24; Wed, 24 Feb 2021 20:06:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197204; bh=axrXK3Oqhx1hjzIewrX2nB/fM2JIWPXy9yjR0sOCWEk=; h=Date:From:To:Subject:In-Reply-To:From; b=uy1GrTPOnHm+QhRD2/mV45xZHGV99B6bfz0icWb1IMOcwMk6Chbap4/PnJJXoWzNW oOC+xRExfTGg5WODWecGVV1gppmcettqENiJTLlpheDALHiv0FID03oShASIQj5CH6 /KVg2Md6E7Ndwr/lsCKDQnNWTrTcrFfaSYYh57Zs= Date: Wed, 24 Feb 2021 12:06:42 -0800 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, dansilsby@gmail.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, maobibo@loongson.cn, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, paulburton@kernel.org, ralf@linux-mips.org, rppt@linux.ibm.com, torvalds@linux-foundation.org, tsbogend@alpha.franken.de Subject: [patch 111/173] mm/huge_memory.c: update tlb entry if pmd is changed Message-ID: <20210224200642.rrDUZyoaP%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 86CE780192E9 X-Stat-Signature: zwwy6p6umgdnpykppmtnhiwuxnwuqfuf Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197204-389139 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Bibo Mao Subject: mm/huge_memory.c: update tlb entry if pmd is changed When set_pmd_at is called in function do_huge_pmd_anonymous_page, new tlb entry can be added by software on MIPS platform. Here add update_mmu_cache_pmd when pmd entry is set, and update_mmu_cache_pmd is defined as empty excepts arc/mips platform. This patch has no negative effect on other platforms except arc/mips system. Link: http://lkml.kernel.org/r/1592990792-1923-2-git-send-email-maobibo@loongson.cn Signed-off-by: Bibo Mao Cc: Anshuman Khandual Cc: Daniel Silsby Cc: "Kirill A. Shutemov" Cc: Mike Kravetz Cc: Mike Rapoport Cc: Paul Burton Cc: Ralf Baechle Cc: Thomas Bogendoerfer Signed-off-by: Andrew Morton --- mm/huge_memory.c | 2 ++ 1 file changed, 2 insertions(+) --- a/mm/huge_memory.c~mm-huge_memoryc-update-tlb-entry-if-pmd-is-changed +++ a/mm/huge_memory.c @@ -636,6 +636,7 @@ static vm_fault_t __do_huge_pmd_anonymou lru_cache_add_inactive_or_unevictable(page, vma); pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, pgtable); set_pmd_at(vma->vm_mm, haddr, vmf->pmd, entry); + update_mmu_cache_pmd(vma, vmf->address, vmf->pmd); add_mm_counter(vma->vm_mm, MM_ANONPAGES, HPAGE_PMD_NR); mm_inc_nr_ptes(vma->vm_mm); spin_unlock(vmf->ptl); @@ -749,6 +750,7 @@ vm_fault_t do_huge_pmd_anonymous_page(st } else { set_huge_zero_page(pgtable, vma->vm_mm, vma, haddr, vmf->pmd, zero_page); + update_mmu_cache_pmd(vma, vmf->address, vmf->pmd); spin_unlock(vmf->ptl); } } else { From patchwork Wed Feb 24 20:06:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102635 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCA07C433E9 for ; Wed, 24 Feb 2021 20:06:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5646064E09 for ; Wed, 24 Feb 2021 20:06:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5646064E09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DDC0D8D001B; Wed, 24 Feb 2021 15:06:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D65036B00DD; Wed, 24 Feb 2021 15:06:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C2BC78D001B; Wed, 24 Feb 2021 15:06:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0121.hostedemail.com [216.40.44.121]) by kanga.kvack.org (Postfix) with ESMTP id A9FB56B00DC for ; Wed, 24 Feb 2021 15:06:49 -0500 (EST) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 7362918503B3F for ; Wed, 24 Feb 2021 20:06:49 +0000 (UTC) X-FDA: 77854244538.12.DB95E81 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP id C5D55E00011B for ; Wed, 24 Feb 2021 20:06:48 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 4F29D64E6C; Wed, 24 Feb 2021 20:06:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197207; bh=ExvbAqaH3jEwzhaUnfNFLwu8yOToxp3gruKqvmvQXso=; h=Date:From:To:Subject:In-Reply-To:From; b=Pa+TGNiTyxleaNzoer7qAKcCG+ZBTxGB673cHipDA7S0MFKkBuf/lEtNdiREylVmd WtA4FRWyjDmDYx0FjKnYV+kWpZSUWw+5/IiufYy3V5T9yz/M+XXD3FdUoRxsrS61Y8 rOOaRRsx6Jqqe7J7/s0crY3arPe2a6g0XkSooOao= Date: Wed, 24 Feb 2021 12:06:46 -0800 From: Andrew Morton To: akpm@linux-foundation.org, anshuman.khandual@arm.com, dansilsby@gmail.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, maobibo@loongson.cn, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, paulburton@kernel.org, ralf@linux-mips.org, rppt@linux.ibm.com, torvalds@linux-foundation.org, tsbogend@alpha.franken.de Subject: [patch 112/173] MIPS: do not call flush_tlb_all when setting pmd entry Message-ID: <20210224200646.TrJOnYm-B%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: C5D55E00011B X-Stat-Signature: pw5y8657d41nnsy9rfcdgkuxjr1co6yk Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf30; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197208-316495 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Bibo Mao Subject: MIPS: do not call flush_tlb_all when setting pmd entry Function set_pmd_at is to set pmd entry, if tlb entry need to be flushed, there exists pmdp_huge_clear_flush alike function before set_pmd_at is called. So it is not necessary to call flush_tlb_all in this function. In these scenarios, tlb for the pmd range needs to be flushed: 1. privilege degrade such as wrprotect is set on the pmd entry 2. pmd entry is cleared 3. there is exception if set_pmd_at is issued by dup_mmap, since flush_tlb_mm is called for parent process, it is not necessary to flush tlb in function copy_huge_pmd. Link: http://lkml.kernel.org/r/1592990792-1923-3-git-send-email-maobibo@loongson.cn Signed-off-by: Bibo Mao Cc: Anshuman Khandual Cc: Daniel Silsby Cc: "Kirill A. Shutemov" Cc: Mike Kravetz Cc: Mike Rapoport Cc: Paul Burton Cc: Ralf Baechle Cc: Thomas Bogendoerfer Signed-off-by: Andrew Morton --- arch/mips/mm/pgtable-32.c | 1 - arch/mips/mm/pgtable-64.c | 1 - 2 files changed, 2 deletions(-) --- a/arch/mips/mm/pgtable-32.c~mips-do-not-call-flush_tlb_all-when-setting-pmd-entry +++ a/arch/mips/mm/pgtable-32.c @@ -45,7 +45,6 @@ void set_pmd_at(struct mm_struct *mm, un pmd_t *pmdp, pmd_t pmd) { *pmdp = pmd; - flush_tlb_all(); } #endif /* defined(CONFIG_TRANSPARENT_HUGEPAGE) */ --- a/arch/mips/mm/pgtable-64.c~mips-do-not-call-flush_tlb_all-when-setting-pmd-entry +++ a/arch/mips/mm/pgtable-64.c @@ -100,7 +100,6 @@ void set_pmd_at(struct mm_struct *mm, un pmd_t *pmdp, pmd_t pmd) { *pmdp = pmd; - flush_tlb_all(); } void __init pagetable_init(void) From patchwork Wed Feb 24 20:06:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102637 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1CC5DC433DB for ; Wed, 24 Feb 2021 20:06:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A662064E60 for ; Wed, 24 Feb 2021 20:06:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A662064E60 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 353788D001C; Wed, 24 Feb 2021 15:06:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2DE046B00DE; Wed, 24 Feb 2021 15:06:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1F4D48D001C; Wed, 24 Feb 2021 15:06:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0189.hostedemail.com [216.40.44.189]) by kanga.kvack.org (Postfix) with ESMTP id 04BE46B00DD for ; Wed, 24 Feb 2021 15:06:53 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id BA10F185044C9 for ; Wed, 24 Feb 2021 20:06:52 +0000 (UTC) X-FDA: 77854244664.02.0B9FC6E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP id B40EAA000507 for ; Wed, 24 Feb 2021 20:06:51 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id E71EE64E09; Wed, 24 Feb 2021 20:06:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197211; bh=J4QGtsY8ILAfV/w1Blb+s7AsW7JGQBezZGKqktjAZhs=; h=Date:From:To:Subject:In-Reply-To:From; b=wSmCWyLEhWJWVdnzUfgqGegdt2RckQ3k4YGF/QtH5exNZCEG3BTtMrLhe2EmNLBLb zZC3E1N6jWKMx98BEKXbeTK7uCrZU8gQhHtgLz0qN2j7F0N8Xm0TWomhVsS7mTUvfA 40t0/n3PgaYeUtl4APaB+HFZIzYYLFnUVGfcPKhI= Date: Wed, 24 Feb 2021 12:06:50 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, smuchun@gmail.com, torvalds@linux-foundation.org Subject: [patch 113/173] mm/hugetlb: fix potential double free in hugetlb_register_node() error path Message-ID: <20210224200650.z-aOHWadP%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B40EAA000507 X-Stat-Signature: pfsdtws1ai7h4kny1mhy57ygowdaapot Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf07; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197211-218326 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/hugetlb: fix potential double free in hugetlb_register_node() error path In hugetlb_sysfs_add_hstate(), we would do kobject_put() on hstate_kobjs when failed to create sysfs group but forget to set hstate_kobjs to NULL. Then in hugetlb_register_node() error path, we may free it again via hugetlb_unregister_node(). Link: https://lkml.kernel.org/r/20210107123249.36964-1-linmiaohe@huawei.com Fixes: a3437870160c ("hugetlb: new sysfs interface") Signed-off-by: Miaohe Lin Reviewed-by: Mike Kravetz Reviewed-by: Muchun Song Signed-off-by: Andrew Morton --- mm/hugetlb.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) --- a/mm/hugetlb.c~mm-hugetlb-fix-potential-double-free-in-hugetlb_register_node-error-path +++ a/mm/hugetlb.c @@ -2988,8 +2988,10 @@ static int hugetlb_sysfs_add_hstate(stru return -ENOMEM; retval = sysfs_create_group(hstate_kobjs[hi], hstate_attr_group); - if (retval) + if (retval) { kobject_put(hstate_kobjs[hi]); + hstate_kobjs[hi] = NULL; + } return retval; } From patchwork Wed Feb 24 20:06:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102641 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9E6BC433E6 for ; Wed, 24 Feb 2021 20:06:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4943064F19 for ; Wed, 24 Feb 2021 20:06:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4943064F19 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 725FB6B00DE; Wed, 24 Feb 2021 15:06:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6AF766B00DF; Wed, 24 Feb 2021 15:06:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5304C8D001E; Wed, 24 Feb 2021 15:06:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0083.hostedemail.com [216.40.44.83]) by kanga.kvack.org (Postfix) with ESMTP id 1B0AB6B00DE for ; Wed, 24 Feb 2021 15:06:57 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id D6578EFBC for ; Wed, 24 Feb 2021 20:06:56 +0000 (UTC) X-FDA: 77854244832.23.D305A26 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP id 0E0FC80192C7 for ; Wed, 24 Feb 2021 20:06:49 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 1A59664E20; Wed, 24 Feb 2021 20:06:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197215; bh=J32eFDCcaWW6E+PJ6XnxGzyIKiT/onRMncTa6LqCYJU=; h=Date:From:To:Subject:In-Reply-To:From; b=MMCQZTSa3Ylo2UUgV2x95HqASRPIIcOmN8XP5qfVFlQjJIALEjARMLfEToxjlgtAz /V1LiIqNAAg9QzpzmmGlJcJsTg1lvqIWJfkoViWsFNgDceoZbXuWijoDGSMlUHcGAx Ii2o3aaJR15i8LSPYiPB7QZpwqsBvFqw0nUjyBFU= Date: Wed, 24 Feb 2021 12:06:54 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, lixinhai.lxh@gmail.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, peterx@redhat.com, torvalds@linux-foundation.org Subject: [patch 114/173] mm/hugetlb.c: fix unnecessary address expansion of pmd sharing Message-ID: <20210224200654.pgKWTyt9O%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: rc6thy6ewssn64ri8ne67atquszyunc1 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 0E0FC80192C7 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf27; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197209-741702 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Li Xinhai Subject: mm/hugetlb.c: fix unnecessary address expansion of pmd sharing The current code would unnecessarily expand the address range. Consider one example, (start, end) = (1G-2M, 3G+2M), and (vm_start, vm_end) = (1G-4M, 3G+4M), the expected adjustment should be keep (1G-2M, 3G+2M) without expand. But the current result will be (1G-4M, 3G+4M). Actually, the range (1G-4M, 1G) and (3G, 3G+4M) would never been involved in pmd sharing. After this patch, we will check that the vma span at least one PUD aligned size and the start,end range overlap the aligned range of vma. With above example, the aligned vma range is (1G, 3G), so if (start, end) range is within (1G-4M, 1G), or within (3G, 3G+4M), then no adjustment to both start and end. Otherwise, we will have chance to adjust start downwards or end upwards without exceeding (vm_start, vm_end). Mike: : The 'adjusted range' is used for calls to mmu notifiers and cache(tlb) : flushing. Since the current code unnecessarily expands the range in some : cases, more entries than necessary would be flushed. This would/could : result in performance degradation. However, this is highly dependent on : the user runtime. Is there a combination of vma layout and calls to : actually hit this issue? If the issue is hit, will those entries : unnecessarily flushed be used again and need to be unnecessarily reloaded? Link: https://lkml.kernel.org/r/20210104081631.2921415-1-lixinhai.lxh@gmail.com Fixes: 75802ca66354 ("mm/hugetlb: fix calculation of adjust_range_if_pmd_sharing_possible") Signed-off-by: Li Xinhai Suggested-by: Mike Kravetz Reviewed-by: Mike Kravetz Cc: Peter Xu Signed-off-by: Andrew Morton --- mm/hugetlb.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) --- a/mm/hugetlb.c~mm-hugetlbc-fix-unnecessary-address-expansion-of-pmd-sharing +++ a/mm/hugetlb.c @@ -5288,21 +5288,23 @@ static bool vma_shareable(struct vm_area void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma, unsigned long *start, unsigned long *end) { - unsigned long a_start, a_end; + unsigned long v_start = ALIGN(vma->vm_start, PUD_SIZE), + v_end = ALIGN_DOWN(vma->vm_end, PUD_SIZE); - if (!(vma->vm_flags & VM_MAYSHARE)) + /* + * vma need span at least one aligned PUD size and the start,end range + * must at least partialy within it. + */ + if (!(vma->vm_flags & VM_MAYSHARE) || !(v_end > v_start) || + (*end <= v_start) || (*start >= v_end)) return; /* Extend the range to be PUD aligned for a worst case scenario */ - a_start = ALIGN_DOWN(*start, PUD_SIZE); - a_end = ALIGN(*end, PUD_SIZE); + if (*start > v_start) + *start = ALIGN_DOWN(*start, PUD_SIZE); - /* - * Intersect the range with the vma range, since pmd sharing won't be - * across vma after all - */ - *start = max(vma->vm_start, a_start); - *end = min(vma->vm_end, a_end); + if (*end < v_end) + *end = ALIGN(*end, PUD_SIZE); } /* From patchwork Wed Feb 24 20:06:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102643 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5104C433DB for ; Wed, 24 Feb 2021 20:07:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 70B0464E60 for ; Wed, 24 Feb 2021 20:07:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 70B0464E60 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C580E8D001F; Wed, 24 Feb 2021 15:07:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B6CB28D001E; Wed, 24 Feb 2021 15:07:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E4758D001F; Wed, 24 Feb 2021 15:07:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0057.hostedemail.com [216.40.44.57]) by kanga.kvack.org (Postfix) with ESMTP id 7D5CD8D001E for ; Wed, 24 Feb 2021 15:07:00 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 4CD0D1730876 for ; Wed, 24 Feb 2021 20:07:00 +0000 (UTC) X-FDA: 77854245000.13.7F25DF3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf28.hostedemail.com (Postfix) with ESMTP id C3A922000382 for ; Wed, 24 Feb 2021 20:06:59 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 786B564EC4; Wed, 24 Feb 2021 20:06:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197218; bh=KpeBFR/dGOJBi1Oh10U6eRZSUVrBkCx1AFX7PM/uvEY=; h=Date:From:To:Subject:In-Reply-To:From; b=yKSHnlS+wXLplcNYbCfxIfoHXwf1k4bYdGNDebtR6+HZ//Q0EKj4zmbaxXPk8Y8qn pgnimL0UN378C/Tcijaxk+DlVTyhljlm/e++dNyG2LsoQipzy00kWsI5eivKEcVHcN fEVfUqjdEIbUksue+YfJqwk4ERKK8eyehYFdBn2c= Date: Wed, 24 Feb 2021 12:06:57 -0800 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, osalvador@suse.de, torvalds@linux-foundation.org Subject: [patch 115/173] mm/hugetlb: avoid unnecessary hugetlb_acct_memory() call Message-ID: <20210224200657.oU6qkHah9%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: C3A922000382 X-Stat-Signature: xg18gmmessr3ui7w77hk5anft3zxtdm3 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf28; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197219-221666 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/hugetlb: avoid unnecessary hugetlb_acct_memory() call When reservation accounting remains unchanged, hugetlb_acct_memory() will do nothing except holding and releasing hugetlb_lock. We should avoid this unnecessary hugetlb_lock lock/unlock cycle which is happening on 'most' hugetlb munmap operations by check delta against 0 at the beginning of hugetlb_acct_memory. Link: https://lkml.kernel.org/r/20210115092013.61012-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: David Hildenbrand Reviewed-by: Oscar Salvador Cc: Mike Kravetz Signed-off-by: Andrew Morton --- mm/hugetlb.c | 3 +++ 1 file changed, 3 insertions(+) --- a/mm/hugetlb.c~mm-hugetlb-avoid-unnecessary-hugetlb_acct_memory-call +++ a/mm/hugetlb.c @@ -3591,6 +3591,9 @@ static int hugetlb_acct_memory(struct hs { int ret = -ENOMEM; + if (!delta) + return 0; + spin_lock(&hugetlb_lock); /* * When cpuset is configured, it breaks the strict hugetlb page From patchwork Wed Feb 24 20:07:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102645 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01B64C433DB for ; Wed, 24 Feb 2021 20:07:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8AED464EC3 for ; Wed, 24 Feb 2021 20:07:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8AED464EC3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 17EE68D0020; Wed, 24 Feb 2021 15:07:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1085A8D001E; Wed, 24 Feb 2021 15:07:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F39B78D0020; Wed, 24 Feb 2021 15:07:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0047.hostedemail.com [216.40.44.47]) by kanga.kvack.org (Postfix) with ESMTP id D0B698D001E for ; Wed, 24 Feb 2021 15:07:04 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 9C64DEFDB for ; Wed, 24 Feb 2021 20:07:04 +0000 (UTC) X-FDA: 77854245168.09.4E8917D Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP id 800D3A000529 for ; Wed, 24 Feb 2021 20:07:03 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id D369F64F19; Wed, 24 Feb 2021 20:07:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197223; bh=XpOa6CoMVvrGtFIGWWAhgEEXehR2Y6aQUgckuO2DBxs=; h=Date:From:To:Subject:In-Reply-To:From; b=Se5s58OJfggXjYY+G359KjjrVAcC4tcTyTIP+1cL3TfmIJhpnJLqbk2E/0NAiNNIn kOm7us/coZz09p/vOZ+GRsHZzYbwAO1JmYI/0SHAQBLPD9ZoPP50QFXEfTEclnzO9P V1+K9XLZQO1MP1T16Zv3UewtUQHf4gsGEVSrXHss= Date: Wed, 24 Feb 2021 12:07:01 -0800 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 116/173] mm/hugetlb: use helper huge_page_order and pages_per_huge_page Message-ID: <20210224200701._da-PYkZ7%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: rkysfy39468yd7ox1b9njirg4t5qdabs X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 800D3A000529 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf07; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197223-285955 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/hugetlb: use helper huge_page_order and pages_per_huge_page Since commit a5516438959d ("hugetlb: modular state for hugetlb page size"), we can use huge_page_order to access hstate->order and pages_per_huge_page to fetch the pages per huge page. But gather_bootmem_prealloc() forgot to use it. Link: https://lkml.kernel.org/r/20210114114435.40075-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: David Hildenbrand Reviewed-by: Mike Kravetz Signed-off-by: Andrew Morton --- mm/hugetlb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/mm/hugetlb.c~mm-hugetlb-use-helper-huge_page_order-and-pages_per_huge_page +++ a/mm/hugetlb.c @@ -2476,7 +2476,7 @@ static void __init gather_bootmem_preall struct hstate *h = m->hstate; WARN_ON(page_count(page) != 1); - prep_compound_huge_page(page, h->order); + prep_compound_huge_page(page, huge_page_order(h)); WARN_ON(PageReserved(page)); prep_new_huge_page(h, page, page_to_nid(page)); put_page(page); /* free it into the hugepage allocator */ @@ -2488,7 +2488,7 @@ static void __init gather_bootmem_preall * side-effects, like CommitLimit going negative. */ if (hstate_is_gigantic(h)) - adjust_managed_page_count(page, 1 << h->order); + adjust_managed_page_count(page, pages_per_huge_page(h)); cond_resched(); } } From patchwork Wed Feb 24 20:07:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102647 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7BC4AC433E0 for ; Wed, 24 Feb 2021 20:07:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0AE4A64EC3 for ; Wed, 24 Feb 2021 20:07:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0AE4A64EC3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 885378D0021; Wed, 24 Feb 2021 15:07:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 80C568D001E; Wed, 24 Feb 2021 15:07:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D4CD8D0021; Wed, 24 Feb 2021 15:07:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0246.hostedemail.com [216.40.44.246]) by kanga.kvack.org (Postfix) with ESMTP id 53D378D001E for ; Wed, 24 Feb 2021 15:07:08 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 1EAFB185044C3 for ; Wed, 24 Feb 2021 20:07:08 +0000 (UTC) X-FDA: 77854245336.08.D2974F1 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP id 9AE71C0007C9 for ; Wed, 24 Feb 2021 20:07:04 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 4B16764E60; Wed, 24 Feb 2021 20:07:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197226; bh=U1XGcjm1zL4VDArObARHXrQYuQHUmmj/NdRZmLmtHbA=; h=Date:From:To:Subject:In-Reply-To:From; b=MWk1jijJ/85bjsjdSUeR0RYLYUpw9Gu06BeEbQHI94ruVpkkPSszUEzfSAIL+7tKv zanTGZBk0+w2SFhcrNBXrFIP61wxNfwxSG81jeA98L57yOX8R5kXMzuSt9RfbKpDsF wGo4W8AqQuiUBb00wy4WZhjwYfQNx+pAKbkbWxEw= Date: Wed, 24 Feb 2021 12:07:05 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linmiaohe@huawei.com, linux-mm@kvack.org, louhongxiang@huawei.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 117/173] mm/hugetlb: fix use after free when subpool max_hpages accounting is not enabled Message-ID: <20210224200705.o8J0hqp8s%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: t7gpuwy8msc5hrczmhzz86yrwmiayjzp X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 9AE71C0007C9 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf03; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197224-76173 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/hugetlb: fix use after free when subpool max_hpages accounting is not enabled If a hugetlbfs filesystem is created with the min_size option and without the size option, used_hpages is always 0 and might lead to release subpool prematurely because it indicates no pages are used now while there might be. In order to fix this issue, we should check used_hpages == 0 iff max_hpages accounting is enabled. As max_hpages accounting should be enabled in most common case, this is not worth a Cc stable. [mike.kravetz@oracle.com: new changelog] Link: https://lkml.kernel.org/r/20210126115510.53374-1-linmiaohe@huawei.com Signed-off-by: Hongxiang Lou Signed-off-by: Miaohe Lin Reviewed-by: Mike Kravetz Signed-off-by: Andrew Morton --- mm/hugetlb.c | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) --- a/mm/hugetlb.c~mm-hugetlb-fix-use-after-free-when-subpool-max_hpages-accounting-is-not-enabled +++ a/mm/hugetlb.c @@ -97,16 +97,26 @@ static inline void ClearPageHugeFreed(st /* Forward declaration */ static int hugetlb_acct_memory(struct hstate *h, long delta); -static inline void unlock_or_release_subpool(struct hugepage_subpool *spool) +static inline bool subpool_is_free(struct hugepage_subpool *spool) { - bool free = (spool->count == 0) && (spool->used_hpages == 0); + if (spool->count) + return false; + if (spool->max_hpages != -1) + return spool->used_hpages == 0; + if (spool->min_hpages != -1) + return spool->rsv_hpages == spool->min_hpages; + + return true; +} +static inline void unlock_or_release_subpool(struct hugepage_subpool *spool) +{ spin_unlock(&spool->lock); /* If no pages are used, and no other handles to the subpool * remain, give up any reservations based on minimum size and * free the subpool */ - if (free) { + if (subpool_is_free(spool)) { if (spool->min_hpages != -1) hugetlb_acct_memory(spool->hstate, -spool->min_hpages); From patchwork Wed Feb 24 20:07:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102649 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1A96C433E0 for ; Wed, 24 Feb 2021 20:07:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 708BA64F0C for ; Wed, 24 Feb 2021 20:07:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 708BA64F0C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F170F8D0022; Wed, 24 Feb 2021 15:07:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E75408D001E; Wed, 24 Feb 2021 15:07:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D65708D0022; Wed, 24 Feb 2021 15:07:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0185.hostedemail.com [216.40.44.185]) by kanga.kvack.org (Postfix) with ESMTP id B814E8D001E for ; Wed, 24 Feb 2021 15:07:11 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 8E3AB185044C9 for ; Wed, 24 Feb 2021 20:07:11 +0000 (UTC) X-FDA: 77854245462.24.DE262D4 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf07.hostedemail.com (Postfix) with ESMTP id 7CBFFA000520 for ; Wed, 24 Feb 2021 20:07:10 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B468A64F1F; Wed, 24 Feb 2021 20:07:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197230; bh=ME5TivSeHxBbPSdqMTMENP3aFGlgsqej1We2XP3Q4SI=; h=Date:From:To:Subject:In-Reply-To:From; b=KuYtW3Rkqldv6c9i/lEqmjcd8xq3UcI4vZqYqDzZNhAEu1LIG2dIDKNLEJ+ic8bk6 GBliOPWPq0ZTQRNrulgHeKroWz15WWz10Pd3Qs06383Eq6ZpaaPvheljbigcCgPIVu BekW0ZXahAFtukYAkd5uE7bMqMU1omc3KNggj9L0= Date: Wed, 24 Feb 2021 12:07:09 -0800 From: Andrew Morton To: abaci-bugfix@linux.alibaba.com, abaci@linux.alibaba.com, akpm@linux-foundation.org, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 118/173] mm/hugetlb: simplify the calculation of variables Message-ID: <20210224200709.BeWr9LbFg%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 1j1o87e6ryq39wt58j8e9mc1of9gofhz X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 7CBFFA000520 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf07; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197230-39529 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jiapeng Zhong Subject: mm/hugetlb: simplify the calculation of variables Fix the following coccicheck warnings: ./mm/hugetlb.c:3372:20-22: WARNING !A || A && B is equivalent to !A || B. Link: https://lkml.kernel.org/r/1611643468-52233-1-git-send-email-abaci-bugfix@linux.alibaba.com Signed-off-by: Jiapeng Zhong Reported-by: Abaci Robot Reviewed-by: Miaohe Lin Reviewed-by: Mike Kravetz Signed-off-by: Andrew Morton --- mm/hugetlb.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) --- a/mm/hugetlb.c~mm-hugetlb-simplify-the-calculation-of-variables +++ a/mm/hugetlb.c @@ -3420,8 +3420,7 @@ static unsigned int allowed_mems_nr(stru mpol_allowed = policy_nodemask_current(gfp_mask); for_each_node_mask(node, cpuset_current_mems_allowed) { - if (!mpol_allowed || - (mpol_allowed && node_isset(node, *mpol_allowed))) + if (!mpol_allowed || node_isset(node, *mpol_allowed)) nr += array[node]; } From patchwork Wed Feb 24 20:07:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102651 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F65CC433E0 for ; Wed, 24 Feb 2021 20:07:16 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E1DD164EC3 for ; Wed, 24 Feb 2021 20:07:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E1DD164EC3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 725A28D0023; Wed, 24 Feb 2021 15:07:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6D34E8D001E; Wed, 24 Feb 2021 15:07:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 59CCD8D0023; Wed, 24 Feb 2021 15:07:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0038.hostedemail.com [216.40.44.38]) by kanga.kvack.org (Postfix) with ESMTP id 403328D001E for ; Wed, 24 Feb 2021 15:07:15 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 145CC82FC4AB for ; Wed, 24 Feb 2021 20:07:15 +0000 (UTC) X-FDA: 77854245630.22.D45E0B0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP id 2AEC880192EA for ; Wed, 24 Feb 2021 20:07:08 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 419B964F0C; Wed, 24 Feb 2021 20:07:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197233; bh=xu3dQIUcuFjKfiYnfBSYI8L8Ak26a5TGLhjBocQIuXo=; h=Date:From:To:Subject:In-Reply-To:From; b=WG5n3xx/ZcIZ04y3c+lE1Hi5PtijhYR4kBQ2jxqep6VdWEXIcdaYsX5en3Kff2cjV AswbRKLKSgLpGX93SpnGmPXr6jTvNlLQs2neEJzOb9f77KBIN/hc5Qg52ELM/AU3OA S9g9CkVx8C991JKw1Q7IHiUSfeJ1/m75PzxU4G5g= Date: Wed, 24 Feb 2021 12:07:12 -0800 From: Andrew Morton To: akpm@linux-foundation.org, joao.m.martins@oracle.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 119/173] mm/hugetlb: grab head page refcount once for group of subpages Message-ID: <20210224200712.-Mz_odraQ%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 3gw4heagmmaa4atd5m7j54gh9iju4co8 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 2AEC880192EA Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf27; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197228-264449 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Joao Martins Subject: mm/hugetlb: grab head page refcount once for group of subpages Patch series "mm/hugetlb: follow_hugetlb_page() improvements", v2. While looking at ZONE_DEVICE struct page reuse particularly the last patch[0], I found two possible improvements for follow_hugetlb_page() which is solely used for get_user_pages()/pin_user_pages(). The first patch batches page refcount updates while the second tidies up storing the subpages/vmas. Both together bring the cost of slow variant of gup() cost from ~87.6k usecs to ~5.8k usecs. libhugetlbfs tests seem to pass as well gup_test benchmarks with hugetlbfs vmas. This patch (of 2): follow_hugetlb_page() once it locks the pmd/pud, checks all its N subpages in a huge page and grabs a reference for each one. Similar to gup-fast, have follow_hugetlb_page() grab the head page refcount only after counting all its subpages that are part of the just faulted huge page. Consequently we reduce the number of atomics necessary to pin said huge page, which improves non-fast gup() considerably: - 16G with 1G huge page size gup_test -f /mnt/huge/file -m 16384 -r 10 -L -S -n 512 -w PIN_LONGTERM_BENCHMARK: ~87.6k us -> ~12.8k us Link: https://lkml.kernel.org/r/20210128182632.24562-1-joao.m.martins@oracle.com Link: https://lkml.kernel.org/r/20210128182632.24562-2-joao.m.martins@oracle.com Signed-off-by: Joao Martins Reviewed-by: Mike Kravetz Signed-off-by: Andrew Morton --- include/linux/mm.h | 3 +++ mm/gup.c | 5 ++--- mm/hugetlb.c | 43 ++++++++++++++++++++++++------------------- 3 files changed, 29 insertions(+), 22 deletions(-) --- a/include/linux/mm.h~mm-hugetlb-grab-head-page-refcount-once-for-group-of-subpages +++ a/include/linux/mm.h @@ -1187,6 +1187,9 @@ static inline void get_page(struct page } bool __must_check try_grab_page(struct page *page, unsigned int flags); +__maybe_unused struct page *try_grab_compound_head(struct page *page, int refs, + unsigned int flags); + static inline __must_check bool try_get_page(struct page *page) { --- a/mm/gup.c~mm-hugetlb-grab-head-page-refcount-once-for-group-of-subpages +++ a/mm/gup.c @@ -78,9 +78,8 @@ static inline struct page *try_get_compo * considered failure, and furthermore, a likely bug in the caller, so a warning * is also emitted. */ -static __maybe_unused struct page *try_grab_compound_head(struct page *page, - int refs, - unsigned int flags) +__maybe_unused struct page *try_grab_compound_head(struct page *page, + int refs, unsigned int flags) { if (flags & FOLL_GET) return try_get_compound_head(page, refs); --- a/mm/hugetlb.c~mm-hugetlb-grab-head-page-refcount-once-for-group-of-subpages +++ a/mm/hugetlb.c @@ -4796,7 +4796,7 @@ long follow_hugetlb_page(struct mm_struc unsigned long vaddr = *position; unsigned long remainder = *nr_pages; struct hstate *h = hstate_vma(vma); - int err = -EFAULT; + int err = -EFAULT, refs; while (vaddr < vma->vm_end && remainder) { pte_t *pte; @@ -4916,26 +4916,11 @@ long follow_hugetlb_page(struct mm_struc continue; } + refs = 0; + same_page: - if (pages) { + if (pages) pages[i] = mem_map_offset(page, pfn_offset); - /* - * try_grab_page() should always succeed here, because: - * a) we hold the ptl lock, and b) we've just checked - * that the huge page is present in the page tables. If - * the huge page is present, then the tail pages must - * also be present. The ptl prevents the head page and - * tail pages from being rearranged in any way. So this - * page must be available at this point, unless the page - * refcount overflowed: - */ - if (WARN_ON_ONCE(!try_grab_page(pages[i], flags))) { - spin_unlock(ptl); - remainder = 0; - err = -ENOMEM; - break; - } - } if (vmas) vmas[i] = vma; @@ -4944,6 +4929,7 @@ same_page: ++pfn_offset; --remainder; ++i; + ++refs; if (vaddr < vma->vm_end && remainder && pfn_offset < pages_per_huge_page(h)) { /* @@ -4951,6 +4937,25 @@ same_page: * of this compound page. */ goto same_page; + } else if (pages) { + /* + * try_grab_compound_head() should always succeed here, + * because: a) we hold the ptl lock, and b) we've just + * checked that the huge page is present in the page + * tables. If the huge page is present, then the tail + * pages must also be present. The ptl prevents the + * head page and tail pages from being rearranged in + * any way. So this page must be available at this + * point, unless the page refcount overflowed: + */ + if (WARN_ON_ONCE(!try_grab_compound_head(pages[i-1], + refs, + flags))) { + spin_unlock(ptl); + remainder = 0; + err = -ENOMEM; + break; + } } spin_unlock(ptl); } From patchwork Wed Feb 24 20:07:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1156EC433E0 for ; Wed, 24 Feb 2021 20:07:20 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 91DB364F20 for ; Wed, 24 Feb 2021 20:07:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 91DB364F20 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 1A9A18D0024; Wed, 24 Feb 2021 15:07:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 107D78D001E; Wed, 24 Feb 2021 15:07:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F11BA8D0024; Wed, 24 Feb 2021 15:07:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0135.hostedemail.com [216.40.44.135]) by kanga.kvack.org (Postfix) with ESMTP id D80818D001E for ; Wed, 24 Feb 2021 15:07:18 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id A7EC8EFCA for ; Wed, 24 Feb 2021 20:07:18 +0000 (UTC) X-FDA: 77854245756.02.5EF5FB0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id 44F78200038B for ; Wed, 24 Feb 2021 20:07:08 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 98CEA64EC3; Wed, 24 Feb 2021 20:07:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197236; bh=oCoZ5ZuMn2MQBNBxn3RHBg0iGZjgl+ZqjzMn4M8xWWw=; h=Date:From:To:Subject:In-Reply-To:From; b=DHIIHjJaZK15cSmn8qiJYsUH6SP3zhBP5cUdOfxGDrifnT7ctF+dp3X7BCawH3AkN XgxtqbzWbtLcWnCK0G/NKxrluG1SSg5Si/y/JzSrTgT3NfqkGaQ2FGSP2bmo/q3ZET WmIC73gokbhd7gZa1Lxh68MUxNYt+Df1tbGnTWhA= Date: Wed, 24 Feb 2021 12:07:16 -0800 From: Andrew Morton To: akpm@linux-foundation.org, joao.m.martins@oracle.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 120/173] mm/hugetlb: refactor subpage recording Message-ID: <20210224200716.FuqvixbD6%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: y84cx3hhw8bwonpzfcn9u4ugxqx9woih X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 44F78200038B Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf11; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197228-470433 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Joao Martins Subject: mm/hugetlb: refactor subpage recording For a given hugepage backing a VA, there's a rather ineficient loop which is solely responsible for storing subpages in GUP @pages/@vmas array. For each subpage we check whether it's within range or size of @pages and keep increment @pfn_offset and a couple other variables per subpage iteration. Simplify this logic and minimize the cost of each iteration to just store the output page/vma. Instead of incrementing number of @refs iteratively, we do it through pre-calculation of @refs and only with a tight loop for storing pinned subpages/vmas. Additionally, retain existing behaviour with using mem_map_offset() when recording the subpages for configurations that don't have a contiguous mem_map. pinning consequently improves bringing us close to {pin,get}_user_pages_fast: - 16G with 1G huge page size gup_test -f /mnt/huge/file -m 16384 -r 30 -L -S -n 512 -w PIN_LONGTERM_BENCHMARK: ~12.8k us -> ~5.8k us PIN_FAST_BENCHMARK: ~3.7k us Link: https://lkml.kernel.org/r/20210128182632.24562-3-joao.m.martins@oracle.com Signed-off-by: Joao Martins Reviewed-by: Mike Kravetz Signed-off-by: Andrew Morton --- mm/hugetlb.c | 49 ++++++++++++++++++++++++++++--------------------- 1 file changed, 28 insertions(+), 21 deletions(-) --- a/mm/hugetlb.c~mm-hugetlb-refactor-subpage-recording +++ a/mm/hugetlb.c @@ -4787,6 +4787,20 @@ out_release_nounlock: goto out; } +static void record_subpages_vmas(struct page *page, struct vm_area_struct *vma, + int refs, struct page **pages, + struct vm_area_struct **vmas) +{ + int nr; + + for (nr = 0; nr < refs; nr++) { + if (likely(pages)) + pages[nr] = mem_map_offset(page, nr); + if (vmas) + vmas[nr] = vma; + } +} + long follow_hugetlb_page(struct mm_struct *mm, struct vm_area_struct *vma, struct page **pages, struct vm_area_struct **vmas, unsigned long *position, unsigned long *nr_pages, @@ -4916,28 +4930,16 @@ long follow_hugetlb_page(struct mm_struc continue; } - refs = 0; + refs = min3(pages_per_huge_page(h) - pfn_offset, + (vma->vm_end - vaddr) >> PAGE_SHIFT, remainder); -same_page: - if (pages) - pages[i] = mem_map_offset(page, pfn_offset); + if (pages || vmas) + record_subpages_vmas(mem_map_offset(page, pfn_offset), + vma, refs, + likely(pages) ? pages + i : NULL, + vmas ? vmas + i : NULL); - if (vmas) - vmas[i] = vma; - - vaddr += PAGE_SIZE; - ++pfn_offset; - --remainder; - ++i; - ++refs; - if (vaddr < vma->vm_end && remainder && - pfn_offset < pages_per_huge_page(h)) { - /* - * We use pfn_offset to avoid touching the pageframes - * of this compound page. - */ - goto same_page; - } else if (pages) { + if (pages) { /* * try_grab_compound_head() should always succeed here, * because: a) we hold the ptl lock, and b) we've just @@ -4948,7 +4950,7 @@ same_page: * any way. So this page must be available at this * point, unless the page refcount overflowed: */ - if (WARN_ON_ONCE(!try_grab_compound_head(pages[i-1], + if (WARN_ON_ONCE(!try_grab_compound_head(pages[i], refs, flags))) { spin_unlock(ptl); @@ -4957,6 +4959,11 @@ same_page: break; } } + + vaddr += (refs << PAGE_SHIFT); + remainder -= refs; + i += refs; + spin_unlock(ptl); } *nr_pages = remainder; From patchwork Wed Feb 24 20:07:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102655 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92741C433DB for ; Wed, 24 Feb 2021 20:07:23 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2D78264F20 for ; Wed, 24 Feb 2021 20:07:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2D78264F20 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id ACC8D8D0025; Wed, 24 Feb 2021 15:07:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A56628D001E; Wed, 24 Feb 2021 15:07:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8F68E8D0025; Wed, 24 Feb 2021 15:07:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0111.hostedemail.com [216.40.44.111]) by kanga.kvack.org (Postfix) with ESMTP id 753858D001E for ; Wed, 24 Feb 2021 15:07:22 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 39F1E1801A212 for ; Wed, 24 Feb 2021 20:07:22 +0000 (UTC) X-FDA: 77854245924.22.3956AB3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf10.hostedemail.com (Postfix) with ESMTP id 1F58A407F8FA for ; Wed, 24 Feb 2021 20:07:15 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id E752364F23; Wed, 24 Feb 2021 20:07:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197240; bh=uMWlgw9tOoEw/DXcTXlWnrGApna+wy9o5SEsUZ5XyA0=; h=Date:From:To:Subject:In-Reply-To:From; b=xcW88dNM/p/iDUnEr08O8Wm4pXIu7/YDZCWej8vDiQbh9zt8nTa7K6kkXlxVM+ATe A4g+FuJx+jGQrnG9HMmV3oqKTHMPkhYM34OjK21dlOoDWUi/Jd4C7aJecf4NqIqhlg OshZGVkdYyvH00Tsendeb9aDvep3YLxeErPgLhPc= Date: Wed, 24 Feb 2021 12:07:19 -0800 From: Andrew Morton To: akpm@linux-foundation.org, jrdr.linux@gmail.com, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 121/173] mm/hugetlb: fix some comment typos Message-ID: <20210224200719.KYC8uNy-0%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 1F58A407F8FA X-Stat-Signature: ms8t8c7n69sgymjky5cd8486nhg7fbpp Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf10; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197235-694999 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/hugetlb: fix some comment typos Fix typos sasitfy to satisfy, reservtion to reservation, hugegpage to hugepage and uniprocesor to uniprocessor in comments. Link: https://lkml.kernel.org/r/20210128112028.64831-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: Souptick Joarder Cc: Mike Kravetz Signed-off-by: Andrew Morton --- include/linux/hugetlb.h | 2 +- mm/hugetlb.c | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) --- a/include/linux/hugetlb.h~mm-hugetlb-fix-some-comment-typos +++ a/include/linux/hugetlb.h @@ -37,7 +37,7 @@ struct hugepage_subpool { struct hstate *hstate; long min_hpages; /* Minimum huge pages or -1 if no minimum. */ long rsv_hpages; /* Pages reserved against global pool to */ - /* sasitfy minimum size. */ + /* satisfy minimum size. */ }; struct resv_map { --- a/mm/hugetlb.c~mm-hugetlb-fix-some-comment-typos +++ a/mm/hugetlb.c @@ -1434,7 +1434,7 @@ static void __free_huge_page(struct page * reservation. If the page was associated with a subpool, there * would have been a page reserved in the subpool before allocation * via hugepage_subpool_get_pages(). Since we are 'restoring' the - * reservtion, do not call hugepage_subpool_put_pages() as this will + * reservation, do not call hugepage_subpool_put_pages() as this will * remove the reserved page from the subpool. */ if (!restore_reserve) { @@ -3707,7 +3707,7 @@ static unsigned long hugetlb_vm_op_pages /* * We cannot handle pagefaults against hugetlb pages at all. They cause * handle_mm_fault() to try to instantiate regular-sized pages in the - * hugegpage VMA. do_page_fault() is supposed to trap this, so BUG is we get + * hugepage VMA. do_page_fault() is supposed to trap this, so BUG is we get * this far. */ static vm_fault_t hugetlb_vm_op_fault(struct vm_fault *vmf) @@ -4491,7 +4491,7 @@ u32 hugetlb_fault_mutex_hash(struct addr } #else /* - * For uniprocesor systems we always use a single mutex, so just + * For uniprocessor systems we always use a single mutex, so just * return 0 and avoid the hashing overhead. */ u32 hugetlb_fault_mutex_hash(struct address_space *mapping, pgoff_t idx) From patchwork Wed Feb 24 20:07:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102657 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F87EC433E0 for ; Wed, 24 Feb 2021 20:07:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 068D564F25 for ; Wed, 24 Feb 2021 20:07:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 068D564F25 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8A9A28D0026; Wed, 24 Feb 2021 15:07:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 833B38D001E; Wed, 24 Feb 2021 15:07:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6AC718D0026; Wed, 24 Feb 2021 15:07:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0085.hostedemail.com [216.40.44.85]) by kanga.kvack.org (Postfix) with ESMTP id 506438D001E for ; Wed, 24 Feb 2021 15:07:25 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 1BE22EFDB for ; Wed, 24 Feb 2021 20:07:25 +0000 (UTC) X-FDA: 77854246050.04.CE70D3F Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP id 900B0E000109 for ; Wed, 24 Feb 2021 20:07:24 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6348664F1B; Wed, 24 Feb 2021 20:07:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197243; bh=MPMYyb7bAp0/VLqCjy8OzVun3Or16xOaDaXsakgvwx4=; h=Date:From:To:Subject:In-Reply-To:From; b=0G+D+NxM3hgQFO05WJx5IaRYaKNUa0wjTUk96gCp9jt6paaYX52wmkI+/FuoBHi1O 69mQrKvP5ohPrR68bS50S1i9xETKiVs8kUaGt4Mo0WQ+xUwrDrHtJ64/oRm9R6/dtN jz2vRRDT45WgCXozoB9oBKExIOWud8qn2SZdknew= Date: Wed, 24 Feb 2021 12:07:22 -0800 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, yanfei.xu@windriver.com Subject: [patch 122/173] mm/hugetlb: remove redundant check in preparing and destroying gigantic page Message-ID: <20210224200722.kWLDke_bB%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 8yykjnzgcrqe8fw8w99iq3kttnj1ts5a X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 900B0E000109 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf30; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197244-59881 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yanfei Xu Subject: mm/hugetlb: remove redundant check in preparing and destroying gigantic page Gigantic page is a compound page and its order is more than 1. Thus it must be available for hpage_pincount. Let's remove the redundant check for gigantic page. Link: https://lkml.kernel.org/r/20210202112002.73170-1-yanfei.xu@windriver.com Signed-off-by: Yanfei Xu Reviewed-by: Mike Kravetz Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- mm/hugetlb.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) --- a/mm/hugetlb.c~mm-hugetlb-remove-redundant-check-in-preparing-and-destroying-gigantic-page +++ a/mm/hugetlb.c @@ -1234,8 +1234,7 @@ static void destroy_compound_gigantic_pa struct page *p = page + 1; atomic_set(compound_mapcount_ptr(page), 0); - if (hpage_pincount_available(page)) - atomic_set(compound_pincount_ptr(page), 0); + atomic_set(compound_pincount_ptr(page), 0); for (i = 1; i < nr_pages; i++, p = mem_map_next(p, page, i)) { clear_compound_head(p); @@ -1563,9 +1562,7 @@ static void prep_compound_gigantic_page( set_compound_head(p, page); } atomic_set(compound_mapcount_ptr(page), -1); - - if (hpage_pincount_available(page)) - atomic_set(compound_pincount_ptr(page), 0); + atomic_set(compound_pincount_ptr(page), 0); } /* From patchwork Wed Feb 24 20:07:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102659 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7589C433DB for ; Wed, 24 Feb 2021 20:07:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5E5C464F28 for ; Wed, 24 Feb 2021 20:07:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5E5C464F28 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E4FB98D0027; Wed, 24 Feb 2021 15:07:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E09C28D001E; Wed, 24 Feb 2021 15:07:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC7728D0027; Wed, 24 Feb 2021 15:07:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0170.hostedemail.com [216.40.44.170]) by kanga.kvack.org (Postfix) with ESMTP id B2F758D001E for ; Wed, 24 Feb 2021 15:07:28 -0500 (EST) Received: from smtpin25.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 8194E83AF77F for ; Wed, 24 Feb 2021 20:07:28 +0000 (UTC) X-FDA: 77854246176.25.AF91B56 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP id F0EAAE000106 for ; Wed, 24 Feb 2021 20:07:27 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B1BE364F25; Wed, 24 Feb 2021 20:07:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197247; bh=sVHLziYvhxSq2bqIqKaw8dnFYh7i8OV7YiwoHz6hNdA=; h=Date:From:To:Subject:In-Reply-To:From; b=vfDx4b9OmUiupH/wE5E9hHFafDYavzRcZoWdYsPYneqp3cOCZ/lKHbdcrxyJsEoHc B07AbLGTa0Gm+Ej+PMSdAEgY35kvwcF6vSdP2nIiqxFcZ4Vh24K1Ny1feFw0W/tDeY s87wVsF2F40TjWgnIKzSXsa/gwQv5wkE/dRDPFHI= Date: Wed, 24 Feb 2021 12:07:26 -0800 From: Andrew Morton To: akpm@linux-foundation.org, daizhiyuan@phytium.com.cn, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 123/173] mm/hugetlb.c: fix typos in comments Message-ID: <20210224200726.aZkBl9bAn%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: F0EAAE000106 X-Stat-Signature: zxmnzkj1n13buwrmo5x4c9inppjk14fj Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf30; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197247-99834 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zhiyuan Dai Subject: mm/hugetlb.c: fix typos in comments Fix typo in comment. Link: https://lkml.kernel.org/r/1612256106-9436-1-git-send-email-daizhiyuan@phytium.com.cn Signed-off-by: Zhiyuan Dai Signed-off-by: Andrew Morton --- mm/hugetlb.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/hugetlb.c~mm-hugetlbc-fix-typos-in-comments +++ a/mm/hugetlb.c @@ -4028,7 +4028,7 @@ void unmap_hugepage_range(struct vm_area /* * This is called when the original mapper is failing to COW a MAP_PRIVATE - * mappping it owns the reserve page for. The intention is to unmap the page + * mapping it owns the reserve page for. The intention is to unmap the page * from other VMAs and let the children be SIGKILLed if they are faulting the * same region. */ From patchwork Wed Feb 24 20:07:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102661 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20589C433DB for ; Wed, 24 Feb 2021 20:07:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A2EC864F29 for ; Wed, 24 Feb 2021 20:07:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A2EC864F29 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2B7848D0028; Wed, 24 Feb 2021 15:07:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 218628D001E; Wed, 24 Feb 2021 15:07:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 107128D0028; Wed, 24 Feb 2021 15:07:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0185.hostedemail.com [216.40.44.185]) by kanga.kvack.org (Postfix) with ESMTP id EB1D88D001E for ; Wed, 24 Feb 2021 15:07:31 -0500 (EST) Received: from smtpin02.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id AA7FE185044DA for ; Wed, 24 Feb 2021 20:07:31 +0000 (UTC) X-FDA: 77854246302.02.5959895 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP id 4C1CF60024A9 for ; Wed, 24 Feb 2021 20:07:27 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0B5F864F20; Wed, 24 Feb 2021 20:07:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197250; bh=x4Kev3w/HWUO8XzagillSlgaP/LUtP4yRoEgl1VDGMU=; h=Date:From:To:Subject:In-Reply-To:From; b=mcY/GET/sAC05KhQd9j3+V4Nsnc9FiK9VaaT41VB5p+0DU+gHgaiyHZ+pKKvgd76F t9UKpig+eBQkdqLCI+VfCCILOf8V/++doBZpSzqBScIgocgy1sepAWgPAtONHJImrn kW375lDKK5sxh/64wXy2GnhtVdfka4Jl808gg18k= Date: Wed, 24 Feb 2021 12:07:29 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 124/173] mm/huge_memory.c: remove unused return value of set_huge_zero_page() Message-ID: <20210224200729._Bjv-Xo-g%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: b79z55kb4xizc6xrbk4m1ypudzk86mny X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 4C1CF60024A9 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf09; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197247-558309 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/huge_memory.c: remove unused return value of set_huge_zero_page() The return value of set_huge_zero_page() is always ignored. So we should drop such return value. Link: https://lkml.kernel.org/r/20210203084816.46307-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Cc: Mike Kravetz Signed-off-by: Andrew Morton --- mm/huge_memory.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) --- a/mm/huge_memory.c~mm-remove-unused-return-value-of-set_huge_zero_page +++ a/mm/huge_memory.c @@ -691,20 +691,19 @@ static inline gfp_t alloc_hugepage_direc } /* Caller must hold page table lock. */ -static bool set_huge_zero_page(pgtable_t pgtable, struct mm_struct *mm, +static void set_huge_zero_page(pgtable_t pgtable, struct mm_struct *mm, struct vm_area_struct *vma, unsigned long haddr, pmd_t *pmd, struct page *zero_page) { pmd_t entry; if (!pmd_none(*pmd)) - return false; + return; entry = mk_pmd(zero_page, vma->vm_page_prot); entry = pmd_mkhuge(entry); if (pgtable) pgtable_trans_huge_deposit(mm, pmd, pgtable); set_pmd_at(mm, haddr, pmd, entry); mm_inc_nr_ptes(mm); - return true; } vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) From patchwork Wed Feb 24 20:07:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102663 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6540C433DB for ; Wed, 24 Feb 2021 20:07:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 69EE664F29 for ; Wed, 24 Feb 2021 20:07:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 69EE664F29 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EED018D0029; Wed, 24 Feb 2021 15:07:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E74F98D001E; Wed, 24 Feb 2021 15:07:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D8A558D0029; Wed, 24 Feb 2021 15:07:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C036B8D001E for ; Wed, 24 Feb 2021 15:07:35 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 8E5D9EFE9 for ; Wed, 24 Feb 2021 20:07:35 +0000 (UTC) X-FDA: 77854246470.14.42B61C5 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf21.hostedemail.com (Postfix) with ESMTP id A3AE4E000102 for ; Wed, 24 Feb 2021 20:07:31 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7E1DA64F2A; Wed, 24 Feb 2021 20:07:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197254; bh=NpB9lmmRQyZZ/AG1rGQlsl6vLwQN94NP96W1Shi5m34=; h=Date:From:To:Subject:In-Reply-To:From; b=plaHmhh/zYKMaQLEIje9AAVGxXxB96eIPA9AC0KwwuItJhTU0wsAc6S2nVsUiIs6t Fgo6j8SSkEnsz2a+kvdKrTXinhitk+UFymbgcQ/Ik310HNhGiKeVHnqY4t3iJ3y9Lm Y5OAXwQDnGNS8eDcBSxbH9JYaLqNiSCVRtqge9Cs= Date: Wed, 24 Feb 2021 12:07:32 -0800 From: Andrew Morton To: akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com, dan.j.williams@intel.com, david@redhat.com, jack@suse.cz, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 125/173] mm/pmem: avoid inserting hugepage PTE entry with fsdax if hugepage support is disabled Message-ID: <20210224200732.t575kkZPb%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: A3AE4E000102 X-Stat-Signature: mjbj4ebfb4b5g1k8d56wqoyw1rofa5eb Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf21; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197251-718003 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Aneesh Kumar K.V" Subject: mm/pmem: avoid inserting hugepage PTE entry with fsdax if hugepage support is disabled Differentiate between hardware not supporting hugepages and user disabling THP via 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' For the devdax namespace, the kernel handles the above via the supported_alignment attribute and failing to initialize the namespace if the namespace align value is not supported on the platform. For the fsdax namespace, the kernel will continue to initialize the namespace. This can result in the kernel creating a huge pte entry even though the hardware don't support the same. We do want hugepage support with pmem even if the end-user disabled THP via sysfs file (/sys/kernel/mm/transparent_hugepage/enabled). Hence differentiate between hardware/firmware lacking support vs user-controlled disable of THP and prevent a huge fault if the hardware lacks hugepage support. Link: https://lkml.kernel.org/r/20210205023956.417587-1-aneesh.kumar@linux.ibm.com Signed-off-by: Aneesh Kumar K.V Reviewed-by: Dan Williams Cc: "Kirill A . Shutemov" Cc: Jan Kara Cc: David Hildenbrand Signed-off-by: Andrew Morton --- include/linux/huge_mm.h | 15 +++++++++------ mm/huge_memory.c | 6 +++++- 2 files changed, 14 insertions(+), 7 deletions(-) --- a/include/linux/huge_mm.h~mm-pmem-avoid-inserting-hugepage-pte-entry-with-fsdax-if-hugepage-support-is-disabled +++ a/include/linux/huge_mm.h @@ -78,6 +78,7 @@ static inline vm_fault_t vmf_insert_pfn_ } enum transparent_hugepage_flag { + TRANSPARENT_HUGEPAGE_NEVER_DAX, TRANSPARENT_HUGEPAGE_FLAG, TRANSPARENT_HUGEPAGE_REQ_MADV_FLAG, TRANSPARENT_HUGEPAGE_DEFRAG_DIRECT_FLAG, @@ -123,6 +124,13 @@ extern unsigned long transparent_hugepag */ static inline bool __transparent_hugepage_enabled(struct vm_area_struct *vma) { + + /* + * If the hardware/firmware marked hugepage support disabled. + */ + if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_NEVER_DAX)) + return false; + if (vma->vm_flags & VM_NOHUGEPAGE) return false; @@ -134,12 +142,7 @@ static inline bool __transparent_hugepag if (transparent_hugepage_flags & (1 << TRANSPARENT_HUGEPAGE_FLAG)) return true; - /* - * For dax vmas, try to always use hugepage mappings. If the kernel does - * not support hugepages, fsdax mappings will fallback to PAGE_SIZE - * mappings, and device-dax namespaces, that try to guarantee a given - * mapping size, will fail to enable - */ + if (vma_is_dax(vma)) return true; --- a/mm/huge_memory.c~mm-pmem-avoid-inserting-hugepage-pte-entry-with-fsdax-if-hugepage-support-is-disabled +++ a/mm/huge_memory.c @@ -386,7 +386,11 @@ static int __init hugepage_init(void) struct kobject *hugepage_kobj; if (!has_transparent_hugepage()) { - transparent_hugepage_flags = 0; + /* + * Hardware doesn't support hugepages, hence disable + * DAX PMD support. + */ + transparent_hugepage_flags = 1 << TRANSPARENT_HUGEPAGE_NEVER_DAX; return -EINVAL; } From patchwork Wed Feb 24 20:07:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 165A1C433E0 for ; Wed, 24 Feb 2021 20:07:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9A5FF64F2B for ; Wed, 24 Feb 2021 20:07:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9A5FF64F2B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2B51F8D002A; Wed, 24 Feb 2021 15:07:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 266278D001E; Wed, 24 Feb 2021 15:07:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A2A98D002A; Wed, 24 Feb 2021 15:07:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0239.hostedemail.com [216.40.44.239]) by kanga.kvack.org (Postfix) with ESMTP id 016488D001E for ; Wed, 24 Feb 2021 15:07:38 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id AD258A8D1 for ; Wed, 24 Feb 2021 20:07:38 +0000 (UTC) X-FDA: 77854246596.10.D787AA3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf10.hostedemail.com (Postfix) with ESMTP id 11157407F8E8 for ; Wed, 24 Feb 2021 20:07:32 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 03E8364F28; Wed, 24 Feb 2021 20:07:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197257; bh=TODsPkWC3h62fUhX/PuCRZMw/GoI2cyuofAu66TO1Ag=; h=Date:From:To:Subject:In-Reply-To:From; b=YRjX29Do/fMtGh0bvBYR18h4BMxQ3BWG3mEYwnS1Rtqn7Ctkxt3pilOImWyk75O3A B5AeM1NPVf1K+VGQPQYiztoYyItbISJHV4tz9py1VmHRuR57oWz/WTpTmemZ+dXmcm nrbCnpxOWV3+u7qOJWWxdvncE3DeNrVheHL6DUsU= Date: Wed, 24 Feb 2021 12:07:36 -0800 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 126/173] hugetlb_cgroup: use helper pages_per_huge_page() in hugetlb_cgroup Message-ID: <20210224200736.Bv6MD-sBr%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 11157407F8E8 X-Stat-Signature: uq9n5gp5w3rdhxcj4k5wme1dfwebkwd5 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf10; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197252-156931 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: hugetlb_cgroup: use helper pages_per_huge_page() in hugetlb_cgroup We could use helper function pages_per_huge_page() to get the number of pages in a hstate to simplify the code slightly. Link: https://lkml.kernel.org/r/20210205084513.29624-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- mm/hugetlb_cgroup.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) --- a/mm/hugetlb_cgroup.c~hugetlb_cgroup-use-helper-pages_per_huge_page-in-hugetlb_cgroup +++ a/mm/hugetlb_cgroup.c @@ -113,7 +113,7 @@ static void hugetlb_cgroup_init(struct h rsvd_parent); limit = round_down(PAGE_COUNTER_MAX, - 1 << huge_page_order(&hstates[idx])); + pages_per_huge_page(&hstates[idx])); ret = page_counter_set_max( hugetlb_cgroup_counter_from_cgroup(h_cgroup, idx), @@ -460,7 +460,7 @@ static int hugetlb_cgroup_read_u64_max(s counter = &h_cg->hugepage[idx]; limit = round_down(PAGE_COUNTER_MAX, - 1 << huge_page_order(&hstates[idx])); + pages_per_huge_page(&hstates[idx])); switch (MEMFILE_ATTR(cft->private)) { case RES_RSVD_USAGE: @@ -507,7 +507,7 @@ static ssize_t hugetlb_cgroup_write(stru return ret; idx = MEMFILE_IDX(of_cft(of)->private); - nr_pages = round_down(nr_pages, 1 << huge_page_order(&hstates[idx])); + nr_pages = round_down(nr_pages, pages_per_huge_page(&hstates[idx])); switch (MEMFILE_ATTR(of_cft(of)->private)) { case RES_RSVD_LIMIT: From patchwork Wed Feb 24 20:07:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102667 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 685CAC433DB for ; Wed, 24 Feb 2021 20:07:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F2C9F64E7A for ; Wed, 24 Feb 2021 20:07:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F2C9F64E7A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8339B8D002B; Wed, 24 Feb 2021 15:07:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E46B8D001E; Wed, 24 Feb 2021 15:07:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 687968D002B; Wed, 24 Feb 2021 15:07:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0174.hostedemail.com [216.40.44.174]) by kanga.kvack.org (Postfix) with ESMTP id 4C84A8D001E for ; Wed, 24 Feb 2021 15:07:42 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 174D7EFD6 for ; Wed, 24 Feb 2021 20:07:42 +0000 (UTC) X-FDA: 77854246764.03.C2285C4 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf24.hostedemail.com (Postfix) with ESMTP id 2D40AA000510 for ; Wed, 24 Feb 2021 20:07:36 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5040264F29; Wed, 24 Feb 2021 20:07:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197260; bh=mEtSQa/cE/LWdEp2XFe1XgeBpvKg8seZ00QYZRH2M2Q=; h=Date:From:To:Subject:In-Reply-To:From; b=H1ZOjJUlA16UgOzwt+Rmz04pwzmiox49nQilfujR4/9gvl5Y9JerH081UDSy5Omo1 n51ie4GzeCpWwQMVchyx+vM2NDZaW29AbLaSHbKN060q7kwn3HBxzxWtrawdYqYnpe lxnpIm8xupDWxckB8HpKxDt+B5U6wAOOexMa5E3g= Date: Wed, 24 Feb 2021 12:07:39 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 127/173] mm/hugetlb: use helper function range_in_vma() in page_table_shareable() Message-ID: <20210224200739.v_LULY55m%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: w9y9ird6woy9ujdu7ejxtcmr6hpnke34 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 2D40AA000510 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf24; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197256-79950 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/hugetlb: use helper function range_in_vma() in page_table_shareable() We could use helper function range_in_vma() to check whether the vma is in the desired range to simplify the code. Link: https://lkml.kernel.org/r/20210204112949.43051-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: Mike Kravetz Signed-off-by: Andrew Morton --- mm/hugetlb.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/hugetlb.c~mm-hugetlb-use-helper-function-range_in_vma-in-page_table_shareable +++ a/mm/hugetlb.c @@ -5282,7 +5282,7 @@ static unsigned long page_table_shareabl */ if (pmd_index(addr) != pmd_index(saddr) || vm_flags != svm_flags || - sbase < svma->vm_start || svma->vm_end < s_end) + !range_in_vma(svma, sbase, s_end)) return 0; return saddr; From patchwork Wed Feb 24 20:07:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102669 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B461C433DB for ; Wed, 24 Feb 2021 20:07:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2934464E7A for ; Wed, 24 Feb 2021 20:07:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2934464E7A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AA55A8D002C; Wed, 24 Feb 2021 15:07:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A2E568D001E; Wed, 24 Feb 2021 15:07:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9446C8D002C; Wed, 24 Feb 2021 15:07:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0037.hostedemail.com [216.40.44.37]) by kanga.kvack.org (Postfix) with ESMTP id 78CDE8D001E for ; Wed, 24 Feb 2021 15:07:45 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 49857184C0D6D for ; Wed, 24 Feb 2021 20:07:45 +0000 (UTC) X-FDA: 77854246890.21.77FA2CF Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf21.hostedemail.com (Postfix) with ESMTP id 94E4FE000103 for ; Wed, 24 Feb 2021 20:07:41 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 9C59764E24; Wed, 24 Feb 2021 20:07:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197264; bh=zr53lxw3Y/WxY3qjgQVBAoALDier3GA7yfrx6iT5MWY=; h=Date:From:To:Subject:In-Reply-To:From; b=b4FXhies5+XsKPT4ZZSQJo0IZPaRTTJljRQ62Esn6yCNGmLym8qbyEKh1alRvmsxZ LIOLA7ohMV9hhF6TLpVQVT/ViGGxPGE42zN6hOwSdAqbl/n6qCday60wDZO3Hu/WI5 7EbE/nRSO1uE5SERv/zqWEvXLdIJ7xKGdyrMthR4= Date: Wed, 24 Feb 2021 12:07:43 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, osalvador@suse.de, torvalds@linux-foundation.org Subject: [patch 128/173] mm/hugetlb: remove unnecessary VM_BUG_ON_PAGE on putback_active_hugepage() Message-ID: <20210224200743.jzLsuQCyf%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 45jz4pt33yrbw6mezt5pqgrze81y5u1y X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 94E4FE000103 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf21; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197261-104881 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/hugetlb: remove unnecessary VM_BUG_ON_PAGE on putback_active_hugepage() All callers know they are operating on a hugetlb head page. So this VM_BUG_ON_PAGE can not catch anything useful. Link: https://lkml.kernel.org/r/20210209071151.44731-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: Oscar Salvador Cc: Mike Kravetz Signed-off-by: Andrew Morton --- mm/hugetlb.c | 1 - 1 file changed, 1 deletion(-) --- a/mm/hugetlb.c~mm-hugetlb-remove-unnecessary-vm_bug_on_page-on-putback_active_hugepage +++ a/mm/hugetlb.c @@ -5622,7 +5622,6 @@ unlock: void putback_active_hugepage(struct page *page) { - VM_BUG_ON_PAGE(!PageHead(page), page); spin_lock(&hugetlb_lock); set_page_huge_active(page); list_move_tail(&page->lru, &(page_hstate(page))->hugepage_activelist); From patchwork Wed Feb 24 20:07:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18752C433E9 for ; Wed, 24 Feb 2021 20:07:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A2F7764E20 for ; Wed, 24 Feb 2021 20:07:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A2F7764E20 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 303E38D002D; Wed, 24 Feb 2021 15:07:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2B3D58D001E; Wed, 24 Feb 2021 15:07:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1C9F18D002D; Wed, 24 Feb 2021 15:07:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0163.hostedemail.com [216.40.44.163]) by kanga.kvack.org (Postfix) with ESMTP id 040158D001E for ; Wed, 24 Feb 2021 15:07:49 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id BE02883AF770 for ; Wed, 24 Feb 2021 20:07:48 +0000 (UTC) X-FDA: 77854247016.14.4D27B52 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP id E12FFE4 for ; Wed, 24 Feb 2021 20:07:46 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0525A64E7A; Wed, 24 Feb 2021 20:07:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197267; bh=kv32r65Ct5Ddk3L+ZOpeibQpQi2DRSWgw4px8T82zyA=; h=Date:From:To:Subject:In-Reply-To:From; b=nszIRVN1QPi+8LsVO9/3AzLo5xhXP5hPxS2hwHOYfabjc3uJBdwRF2Fvmdh/znWWK AJFlgMdEYzvvryvwfkLOLtQfHWSn5dNyug1bEVJSPI1p01BAlwYCUcLCqGElyXYW/s Il8DmKk9Z5pfHv9SFU2bwGEZ0Iq6vpeaV054vkD8= Date: Wed, 24 Feb 2021 12:07:46 -0800 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 129/173] mm/hugetlb: use helper huge_page_size() to get hugepage size Message-ID: <20210224200746.eYJNaWdIU%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: j81i7r8od7z1ud381rpfuezu9caktwiz X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: E12FFE4 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf04; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197266-841658 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/hugetlb: use helper huge_page_size() to get hugepage size We can use helper huge_page_size() to get the hugepage size directly to simplify the code slightly. [linmiaohe@huawei.com: use helper huge_page_size() to get hugepage size] Link: https://lkml.kernel.org/r/20210209021803.49211-1-linmiaohe@huawei.com Link: https://lkml.kernel.org/r/20210208082450.15716-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: Mike Kravetz Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- mm/hugetlb.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) --- a/mm/hugetlb.c~mm-hugetlb-use-helper-huge_page_size-to-get-hugepage-size +++ a/mm/hugetlb.c @@ -3248,7 +3248,7 @@ void __init hugetlb_add_hstate(unsigned BUG_ON(order == 0); h = &hstates[hugetlb_max_hstate++]; h->order = order; - h->mask = ~((1ULL << (order + PAGE_SHIFT)) - 1); + h->mask = ~(huge_page_size(h) - 1); for (i = 0; i < MAX_NUMNODES; ++i) INIT_LIST_HEAD(&h->hugepage_freelists[i]); INIT_LIST_HEAD(&h->hugepage_activelist); @@ -3523,7 +3523,7 @@ void hugetlb_report_meminfo(struct seq_f for_each_hstate(h) { unsigned long count = h->nr_huge_pages; - total += (PAGE_SIZE << huge_page_order(h)) * count; + total += huge_page_size(h) * count; if (h == &default_hstate) seq_printf(m, @@ -3536,10 +3536,10 @@ void hugetlb_report_meminfo(struct seq_f h->free_huge_pages, h->resv_huge_pages, h->surplus_huge_pages, - (PAGE_SIZE << huge_page_order(h)) / 1024); + huge_page_size(h) / SZ_1K); } - seq_printf(m, "Hugetlb: %8lu kB\n", total / 1024); + seq_printf(m, "Hugetlb: %8lu kB\n", total / SZ_1K); } int hugetlb_report_node_meminfo(char *buf, int len, int nid) @@ -3573,7 +3573,7 @@ void hugetlb_show_meminfo(void) h->nr_huge_pages_node[nid], h->free_huge_pages_node[nid], h->surplus_huge_pages_node[nid], - 1UL << (huge_page_order(h) + PAGE_SHIFT - 10)); + huge_page_size(h) / SZ_1K); } void hugetlb_report_usage(struct seq_file *m, struct mm_struct *mm) @@ -3696,9 +3696,7 @@ static int hugetlb_vm_op_split(struct vm static unsigned long hugetlb_vm_op_pagesize(struct vm_area_struct *vma) { - struct hstate *hstate = hstate_vma(vma); - - return 1UL << huge_page_shift(hstate); + return huge_page_size(hstate_vma(vma)); } /* From patchwork Wed Feb 24 20:07:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 002C0C433DB for ; Wed, 24 Feb 2021 20:07:53 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8FFA764E09 for ; Wed, 24 Feb 2021 20:07:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8FFA764E09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 23FA68D002E; Wed, 24 Feb 2021 15:07:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1EF978D001E; Wed, 24 Feb 2021 15:07:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0DF738D002E; Wed, 24 Feb 2021 15:07:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0135.hostedemail.com [216.40.44.135]) by kanga.kvack.org (Postfix) with ESMTP id E51C68D001E for ; Wed, 24 Feb 2021 15:07:52 -0500 (EST) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id B3CDD185044D8 for ; Wed, 24 Feb 2021 20:07:52 +0000 (UTC) X-FDA: 77854247184.10.4BD13E4 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP id 0460CC0007C4 for ; Wed, 24 Feb 2021 20:07:48 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 9964764E60; Wed, 24 Feb 2021 20:07:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197271; bh=/PowuA+liQXKosAi5ro+HSU9tErdU/1suZR4xdRManE=; h=Date:From:To:Subject:In-Reply-To:From; b=jvOjdcmzKa5aBOYZ+fyNyBW2hJVjNkQRdaXh/D8Ntd5ODyhnrYVOoJZCPIgYkJoDf FRxeEIVvTk/8EqZvTBtX1BG7fxOuBTsl2KR9+Q9jHRU8oWrLD3xK5z17aBkWM3CFt6 trWjNffznm0i3evCunxpUI93ScH6chRbhfWhMKjo= Date: Wed, 24 Feb 2021 12:07:50 -0800 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, dbueso@suse.de, joao.m.martins@oracle.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, osalvador@suse.de, stable@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org, ziy@nvidia.com Subject: [patch 130/173] hugetlb: fix update_and_free_page contig page struct assumption Message-ID: <20210224200750.Dfh-Co_Ux%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 MIME-Version: 1.0 X-Stat-Signature: q9wguq6aj5ozx563g7h1sfk6ahu49hxt X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 0460CC0007C4 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf22; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197268-3136 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Kravetz Subject: hugetlb: fix update_and_free_page contig page struct assumption page structs are not guaranteed to be contiguous for gigantic pages. The routine update_and_free_page can encounter a gigantic page, yet it assumes page structs are contiguous when setting page flags in subpages. If update_and_free_page encounters non-contiguous page structs, we can see “BUG: Bad page state in process …” errors. Non-contiguous page structs are generally not an issue. However, they can exist with a specific kernel configuration and hotplug operations. For example: Configure the kernel with CONFIG_SPARSEMEM and !CONFIG_SPARSEMEM_VMEMMAP. Then, hotplug add memory for the area where the gigantic page will be allocated. Zi Yan outlined steps to reproduce here [1]. [1] https://lore.kernel.org/linux-mm/16F7C58B-4D79-41C5-9B64-A1A1628F4AF2@nvidia.com/ Link: https://lkml.kernel.org/r/20210217184926.33567-1-mike.kravetz@oracle.com Fixes: 944d9fec8d7a ("hugetlb: add support for gigantic page allocation at runtime") Signed-off-by: Zi Yan Signed-off-by: Mike Kravetz Cc: Zi Yan Cc: Davidlohr Bueso Cc: "Kirill A . Shutemov" Cc: Andrea Arcangeli Cc: Matthew Wilcox Cc: Oscar Salvador Cc: Joao Martins Cc: Signed-off-by: Andrew Morton --- mm/hugetlb.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) --- a/mm/hugetlb.c~hugetlb-fix-update_and_free_page-contig-page-struct-assumption +++ a/mm/hugetlb.c @@ -1321,14 +1321,16 @@ static inline void destroy_compound_giga static void update_and_free_page(struct hstate *h, struct page *page) { int i; + struct page *subpage = page; if (hstate_is_gigantic(h) && !gigantic_page_runtime_supported()) return; h->nr_huge_pages--; h->nr_huge_pages_node[page_to_nid(page)]--; - for (i = 0; i < pages_per_huge_page(h); i++) { - page[i].flags &= ~(1 << PG_locked | 1 << PG_error | + for (i = 0; i < pages_per_huge_page(h); + i++, subpage = mem_map_next(subpage, page, i)) { + subpage->flags &= ~(1 << PG_locked | 1 << PG_error | 1 << PG_referenced | 1 << PG_dirty | 1 << PG_active | 1 << PG_private | 1 << PG_writeback); From patchwork Wed Feb 24 20:07:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102675 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A03F5C433DB for ; Wed, 24 Feb 2021 20:07:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 301BE64E6C for ; Wed, 24 Feb 2021 20:07:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 301BE64E6C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AF6358D002F; Wed, 24 Feb 2021 15:07:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AA2678D001E; Wed, 24 Feb 2021 15:07:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9B83A8D002F; Wed, 24 Feb 2021 15:07:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0238.hostedemail.com [216.40.44.238]) by kanga.kvack.org (Postfix) with ESMTP id 83A5F8D001E for ; Wed, 24 Feb 2021 15:07:57 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 50B39EFBC for ; Wed, 24 Feb 2021 20:07:57 +0000 (UTC) X-FDA: 77854247394.26.4633218 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf06.hostedemail.com (Postfix) with ESMTP id 5533CC0001EE for ; Wed, 24 Feb 2021 20:07:58 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 389E364E09; Wed, 24 Feb 2021 20:07:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197275; bh=br2SndbgWMiTdshaOJ96Bl/IjnRBOH6TI6vIRuk13hA=; h=Date:From:To:Subject:In-Reply-To:From; b=j4coavo/7wIDbnXSiGAKMsdOw27MuUSL3vUcUJzYlO/u6RGStE0PdlKVWTpfr31Vv ANtks2OJDILcCxp2ugUvB8JDW958pEt5FAaPluYVH8jto6wNkdiKq6bkQpCGsvsUJH E9SGLDvWoIGvr+THUgAo1a5adCKoUUbaiYU2eMzU= Date: Wed, 24 Feb 2021 12:07:54 -0800 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, dbueso@suse.de, joao.m.martins@oracle.com, kirill.shutemov@linux.intel.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, osalvador@suse.de, stable@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org, ziy@nvidia.com Subject: [patch 131/173] hugetlb: fix copy_huge_page_from_user contig page struct assumption Message-ID: <20210224200754.-vZcYjb2n%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 5533CC0001EE X-Stat-Signature: xja1xfkx14m4r4hdhq8fgm8nm7qssyz3 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf06; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197278-166480 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Kravetz Subject: hugetlb: fix copy_huge_page_from_user contig page struct assumption page structs are not guaranteed to be contiguous for gigantic pages. The routine copy_huge_page_from_user can encounter gigantic pages, yet it assumes page structs are contiguous when copying pages from user space. Since page structs for the target gigantic page are not contiguous, the data copied from user space could overwrite other pages not associated with the gigantic page and cause data corruption. Non-contiguous page structs are generally not an issue. However, they can exist with a specific kernel configuration and hotplug operations. For example: Configure the kernel with CONFIG_SPARSEMEM and !CONFIG_SPARSEMEM_VMEMMAP. Then, hotplug add memory for the area where the gigantic page will be allocated. Link: https://lkml.kernel.org/r/20210217184926.33567-2-mike.kravetz@oracle.com Fixes: 8fb5debc5fcd ("userfaultfd: hugetlbfs: add hugetlb_mcopy_atomic_pte for userfaultfd support") Signed-off-by: Mike Kravetz Cc: Zi Yan Cc: Davidlohr Bueso Cc: "Kirill A . Shutemov" Cc: Andrea Arcangeli Cc: Matthew Wilcox Cc: Oscar Salvador Cc: Joao Martins Cc: Signed-off-by: Andrew Morton --- mm/memory.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) --- a/mm/memory.c~hugetlb-fix-copy_huge_page_from_user-contig-page-struct-assumption +++ a/mm/memory.c @@ -5177,17 +5177,19 @@ long copy_huge_page_from_user(struct pag void *page_kaddr; unsigned long i, rc = 0; unsigned long ret_val = pages_per_huge_page * PAGE_SIZE; + struct page *subpage = dst_page; - for (i = 0; i < pages_per_huge_page; i++) { + for (i = 0; i < pages_per_huge_page; + i++, subpage = mem_map_next(subpage, dst_page, i)) { if (allow_pagefault) - page_kaddr = kmap(dst_page + i); + page_kaddr = kmap(subpage); else - page_kaddr = kmap_atomic(dst_page + i); + page_kaddr = kmap_atomic(subpage); rc = copy_from_user(page_kaddr, (const void __user *)(src + i * PAGE_SIZE), PAGE_SIZE); if (allow_pagefault) - kunmap(dst_page + i); + kunmap(subpage); else kunmap_atomic(page_kaddr); From patchwork Wed Feb 24 20:07:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102691 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF9E6C433E0 for ; Wed, 24 Feb 2021 20:08:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6040664E24 for ; Wed, 24 Feb 2021 20:08:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6040664E24 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DD6D28D0037; Wed, 24 Feb 2021 15:08:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D878C8D001E; Wed, 24 Feb 2021 15:08:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C4F628D0037; Wed, 24 Feb 2021 15:08:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0237.hostedemail.com [216.40.44.237]) by kanga.kvack.org (Postfix) with ESMTP id AA6E28D001E for ; Wed, 24 Feb 2021 15:08:31 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 809D1F056 for ; Wed, 24 Feb 2021 20:08:31 +0000 (UTC) X-FDA: 77854248822.18.7E62685 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf24.hostedemail.com (Postfix) with ESMTP id 9F4ACA000507 for ; Wed, 24 Feb 2021 20:08:25 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 011B964E6C; Wed, 24 Feb 2021 20:07:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197279; bh=layVAKYQgKq7CaKR9E20ubXvjiBD05MV/IWqpgzrmn4=; h=Date:From:To:Subject:In-Reply-To:From; b=ksFkAFkWPYdesOKuwBmM8TuRbZi25rlW5s5aqPpL6wBVT/8ZXr5+2sVCatUlpkTno 2U8V1MfbUSBjr+lKmXhwyRH5F/hhf/y3jNcOaVsIhHMfdYvM6myAkgavg5jtP8L7Tt 0FTr03BEz1o5TFfLpRha5gy5dPfyVwzbMr+zOuA0= Date: Wed, 24 Feb 2021 12:07:58 -0800 From: Andrew Morton To: akpm@linux-foundation.org, chenwandun@huawei.com, guro@fb.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 132/173] mm/hugetlb: suppress wrong warning info when alloc gigantic page Message-ID: <20210224200758.zTVwhq0yN%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 9379n3xkwuaajcxepoxe9eummge9eyz3 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 9F4ACA000507 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf24; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197305-187341 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Chen Wandun Subject: mm/hugetlb: suppress wrong warning info when alloc gigantic page If hugetlb_cma is enabled, it will skip boot time allocation when allocating gigantic page, that doesn't means allocation failure, so suppress this warning info. Link: https://lkml.kernel.org/r/20210219123909.13130-1-chenwandun@huawei.com Fixes: cf11e85fc08c ("mm: hugetlb: optionally allocate gigantic hugepages using cma") Signed-off-by: Chen Wandun Reviewed-by: Mike Kravetz Cc: Roman Gushchin Signed-off-by: Andrew Morton --- mm/hugetlb.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/mm/hugetlb.c~mm-hugetlb-suppress-wrong-warning-info-when-alloc-gigantic-page +++ a/mm/hugetlb.c @@ -2529,7 +2529,7 @@ static void __init hugetlb_hstate_alloc_ if (hstate_is_gigantic(h)) { if (hugetlb_cma_size) { pr_warn_once("HugeTLB: hugetlb_cma is enabled, skip boot time allocation\n"); - break; + goto free; } if (!alloc_bootmem_huge_page(h)) break; @@ -2547,7 +2547,7 @@ static void __init hugetlb_hstate_alloc_ h->max_huge_pages, buf, i); h->max_huge_pages = i; } - +free: kfree(node_alloc_noretry); } From patchwork Wed Feb 24 20:08:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102677 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A74F6C433E6 for ; Wed, 24 Feb 2021 20:08:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3536864F08 for ; Wed, 24 Feb 2021 20:08:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3536864F08 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B8EB08D0030; Wed, 24 Feb 2021 15:08:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B3EAE8D001E; Wed, 24 Feb 2021 15:08:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A54BC8D0030; Wed, 24 Feb 2021 15:08:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0197.hostedemail.com [216.40.44.197]) by kanga.kvack.org (Postfix) with ESMTP id 8BA898D001E for ; Wed, 24 Feb 2021 15:08:05 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 5526D180206DF for ; Wed, 24 Feb 2021 20:08:05 +0000 (UTC) X-FDA: 77854247730.20.A885317 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id EF624F7 for ; Wed, 24 Feb 2021 20:07:58 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 199E264EC3; Wed, 24 Feb 2021 20:08:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197283; bh=gVElk86W0iovkBG5MvbBrGnab76DsuQO3EEOO4hWJfM=; h=Date:From:To:Subject:In-Reply-To:From; b=vWxkBcUMJH4Xnkkfz1FRjD4bIN3o+7GLfmEBKFdUJpBA6HXrOmlpVbO6AgcJ0zzRQ E2pP3cnN/1uckK2DJ7bjq7eKa7Ku4nnH60FPcxvawM1akH+VDfGUBHt2o+s9vXB+rF +KD1pTnwCb5mkfpcoJIUH3sw6oY0q2zd4Xc2cpKQ= Date: Wed, 24 Feb 2021 12:08:01 -0800 From: Andrew Morton To: akpm@linux-foundation.org, alex.shi@linux.alibaba.com, hughd@google.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, willy@infradead.org, yuzhao@google.com Subject: [patch 133/173] mm/vmscan: __isolate_lru_page_prepare() cleanup Message-ID: <20210224200801.zJa7Eq7jv%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: EF624F7 X-Stat-Signature: fwqjqfwo9zzibz1taju43je3hwu4reoy Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf12; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197278-940935 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alex Shi Subject: mm/vmscan: __isolate_lru_page_prepare() cleanup The function just returns 2 results, so using a 'switch' to deal with its result is unnecessary. Also simplify it to a bool func as Vlastimil suggested. Also remove 'goto' by reusing list_move(), and take Matthew Wilcox's suggestion to update comments in function. Link: https://lkml.kernel.org/r/728874d7-2d93-4049-68c1-dcc3b2d52ccd@linux.alibaba.com Signed-off-by: Alex Shi Reviewed-by: Andrew Morton Acked-by: Vlastimil Babka Cc: Matthew Wilcox Cc: Hugh Dickins Cc: Yu Zhao Cc: Michal Hocko Signed-off-by: Andrew Morton --- include/linux/swap.h | 2 - mm/compaction.c | 2 - mm/vmscan.c | 68 ++++++++++++++++++----------------------- 3 files changed, 33 insertions(+), 39 deletions(-) --- a/include/linux/swap.h~mm-vmscan-__isolate_lru_page_prepare-clean-up +++ a/include/linux/swap.h @@ -356,7 +356,7 @@ extern void lru_cache_add_inactive_or_un extern unsigned long zone_reclaimable_pages(struct zone *zone); extern unsigned long try_to_free_pages(struct zonelist *zonelist, int order, gfp_t gfp_mask, nodemask_t *mask); -extern int __isolate_lru_page_prepare(struct page *page, isolate_mode_t mode); +extern bool __isolate_lru_page_prepare(struct page *page, isolate_mode_t mode); extern unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg, unsigned long nr_pages, gfp_t gfp_mask, --- a/mm/compaction.c~mm-vmscan-__isolate_lru_page_prepare-clean-up +++ a/mm/compaction.c @@ -988,7 +988,7 @@ isolate_migratepages_block(struct compac if (unlikely(!get_page_unless_zero(page))) goto isolate_fail; - if (__isolate_lru_page_prepare(page, isolate_mode) != 0) + if (!__isolate_lru_page_prepare(page, isolate_mode)) goto isolate_fail_put; /* Try isolate the page */ --- a/mm/vmscan.c~mm-vmscan-__isolate_lru_page_prepare-clean-up +++ a/mm/vmscan.c @@ -1539,19 +1539,17 @@ unsigned int reclaim_clean_pages_from_li * page: page to consider * mode: one of the LRU isolation modes defined above * - * returns 0 on success, -ve errno on failure. + * returns true on success, false on failure. */ -int __isolate_lru_page_prepare(struct page *page, isolate_mode_t mode) +bool __isolate_lru_page_prepare(struct page *page, isolate_mode_t mode) { - int ret = -EBUSY; - /* Only take pages on the LRU. */ if (!PageLRU(page)) - return ret; + return false; /* Compaction should not handle unevictable pages but CMA can do so */ if (PageUnevictable(page) && !(mode & ISOLATE_UNEVICTABLE)) - return ret; + return false; /* * To minimise LRU disruption, the caller can indicate that it only @@ -1564,7 +1562,7 @@ int __isolate_lru_page_prepare(struct pa if (mode & ISOLATE_ASYNC_MIGRATE) { /* All the caller can do on PageWriteback is block */ if (PageWriteback(page)) - return ret; + return false; if (PageDirty(page)) { struct address_space *mapping; @@ -1580,20 +1578,20 @@ int __isolate_lru_page_prepare(struct pa * from the page cache. */ if (!trylock_page(page)) - return ret; + return false; mapping = page_mapping(page); migrate_dirty = !mapping || mapping->a_ops->migratepage; unlock_page(page); if (!migrate_dirty) - return ret; + return false; } } if ((mode & ISOLATE_UNMAPPED) && page_mapped(page)) - return ret; + return false; - return 0; + return true; } /* @@ -1677,35 +1675,31 @@ static unsigned long isolate_lru_pages(u * only when the page is being freed somewhere else. */ scan += nr_pages; - switch (__isolate_lru_page_prepare(page, mode)) { - case 0: - /* - * Be careful not to clear PageLRU until after we're - * sure the page is not being freed elsewhere -- the - * page release code relies on it. - */ - if (unlikely(!get_page_unless_zero(page))) - goto busy; + if (!__isolate_lru_page_prepare(page, mode)) { + /* It is being freed elsewhere */ + list_move(&page->lru, src); + continue; + } + /* + * Be careful not to clear PageLRU until after we're + * sure the page is not being freed elsewhere -- the + * page release code relies on it. + */ + if (unlikely(!get_page_unless_zero(page))) { + list_move(&page->lru, src); + continue; + } - if (!TestClearPageLRU(page)) { - /* - * This page may in other isolation path, - * but we still hold lru_lock. - */ - put_page(page); - goto busy; - } - - nr_taken += nr_pages; - nr_zone_taken[page_zonenum(page)] += nr_pages; - list_move(&page->lru, dst); - break; - - default: -busy: - /* else it is being freed elsewhere */ + if (!TestClearPageLRU(page)) { + /* Another thread is already isolating this page */ + put_page(page); list_move(&page->lru, src); + continue; } + + nr_taken += nr_pages; + nr_zone_taken[page_zonenum(page)] += nr_pages; + list_move(&page->lru, dst); } /* From patchwork Wed Feb 24 20:08:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102679 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3F01C433E0 for ; Wed, 24 Feb 2021 20:08:09 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5971964E20 for ; Wed, 24 Feb 2021 20:08:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5971964E20 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D14C88D0031; Wed, 24 Feb 2021 15:08:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CC5E08D001E; Wed, 24 Feb 2021 15:08:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C01B68D0031; Wed, 24 Feb 2021 15:08:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0138.hostedemail.com [216.40.44.138]) by kanga.kvack.org (Postfix) with ESMTP id A53498D001E for ; Wed, 24 Feb 2021 15:08:08 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 622271730876 for ; Wed, 24 Feb 2021 20:08:08 +0000 (UTC) X-FDA: 77854247856.11.BE38262 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id 5F40820003A6 for ; Wed, 24 Feb 2021 20:07:58 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A84C464EC4; Wed, 24 Feb 2021 20:08:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197287; bh=aJq1EZ5tqLwxvE0hEHMV5cwVHEialTcL9lxlP2KRRHI=; h=Date:From:To:Subject:In-Reply-To:From; b=PoYoC1NOeuFitEuMNQmDHwLZeY4Ux6mEW3maY5AfAUk9kVdwrHpBXAZl/o6IPBfhG nM1ekyD+/yf4QkhG2XEGEhGPz6hd0b76+q2iFpJT6dxImWUX6URRBjBEfz5YYAse9+ OM/mAXzrIomKFgPTOaI1ZJmKnSOpP4OZJ/kR3df0= Date: Wed, 24 Feb 2021 12:08:06 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hannes@cmpxchg.org, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 134/173] mm/workingset.c: avoid unnecessary max_nodes estimation in count_shadow_nodes() Message-ID: <20210224200806.DDlsnNZ0R%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: abzyou7pd7wsazbbtzsjhwpri5xe93j9 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 5F40820003A6 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf11; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197278-624345 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/workingset.c: avoid unnecessary max_nodes estimation in count_shadow_nodes() If list_lru_shrink_count is 0, we always return SHRINK_EMPTY regardless of the value of max_nodes. So we can return early if nodes == 0 to save some cpu cycles of approximating a reasonable limit for the nodes. Link: https://lkml.kernel.org/r/20210123073825.46709-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: Andrew Morton Acked-by: Vlastimil Babka Cc: Johannes Weiner Signed-off-by: Andrew Morton --- mm/workingset.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) --- a/mm/workingset.c~mm-workingsetc-avoid-unnecessary-max_nodes-estimation-in-count_shadow_nodes +++ a/mm/workingset.c @@ -461,6 +461,8 @@ static unsigned long count_shadow_nodes( unsigned long pages; nodes = list_lru_shrink_count(&shadow_nodes, sc); + if (!nodes) + return SHRINK_EMPTY; /* * Approximate a reasonable limit for the nodes @@ -503,9 +505,6 @@ static unsigned long count_shadow_nodes( max_nodes = pages >> (XA_CHUNK_SHIFT - 3); - if (!nodes) - return SHRINK_EMPTY; - if (nodes <= max_nodes) return 0; return nodes - max_nodes; From patchwork Wed Feb 24 20:08:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102681 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB48CC433E6 for ; Wed, 24 Feb 2021 20:08:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3CE0F64F09 for ; Wed, 24 Feb 2021 20:08:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3CE0F64F09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BDCF48D0032; Wed, 24 Feb 2021 15:08:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B8DA78D001E; Wed, 24 Feb 2021 15:08:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA2798D0032; Wed, 24 Feb 2021 15:08:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0134.hostedemail.com [216.40.44.134]) by kanga.kvack.org (Postfix) with ESMTP id 91CF68D001E for ; Wed, 24 Feb 2021 15:08:12 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 5C3EE80196D7 for ; Wed, 24 Feb 2021 20:08:12 +0000 (UTC) X-FDA: 77854248024.22.93E7BF3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP id D7699E00010F for ; Wed, 24 Feb 2021 20:08:11 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 3B4FA64E20; Wed, 24 Feb 2021 20:08:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197291; bh=+Zoh4nGQnzlHZdjko/P99CODOn2oRmcFDH23R/7asjI=; h=Date:From:To:Subject:In-Reply-To:From; b=rDInu1UBWS+FClGI3ENNejvS1kd6HyXdYFxyu0wl73BaaedpFgTILJK8DiKzgFdq5 EKskMP6hQWOX6hGebx5VAhQYUpQ/RSGcpaZmfThVn28tFTzqV5elqWuLb6CeNZVy/c CLy4VH8j6zy53835UGC3Wl+V58jCJAMXQZGZF6Fg= Date: Wed, 24 Feb 2021 12:08:09 -0800 From: Andrew Morton To: akpm@linux-foundation.org, alex.shi@linux.alibaba.com, guro@fb.com, hannes@cmpxchg.org, hughd@google.com, linmiaohe@huawei.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, vdavydov.dev@gmail.com, willy@infradead.org, yuzhao@google.com Subject: [patch 135/173] mm/vmscan.c: use add_page_to_lru_list() Message-ID: <20210224200809.sGCYmo_d4%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: ppsfut3mpzh6j44kqram1dongba91kcd X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: D7699E00010F Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf30; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197291-802017 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu Zhao Subject: mm/vmscan.c: use add_page_to_lru_list() Patch series "mm: lru related cleanups", v2. The cleanups are intended to reduce the verbosity in lru list operations and make them less error-prone. A typical example would be how the patches change __activate_page(): static void __activate_page(struct page *page, struct lruvec *lruvec) { if (!PageActive(page) && !PageUnevictable(page)) { - int lru = page_lru_base_type(page); int nr_pages = thp_nr_pages(page); - del_page_from_lru_list(page, lruvec, lru); + del_page_from_lru_list(page, lruvec); SetPageActive(page); - lru += LRU_ACTIVE; - add_page_to_lru_list(page, lruvec, lru); + add_page_to_lru_list(page, lruvec); trace_mm_lru_activate(page); There are a few more places like __activate_page() and they are unnecessarily repetitive in terms of figuring out which list a page should be added onto or deleted from. And with the duplicated code removed, they are easier to read, IMO. Patch 1 to 5 basically cover the above. Patch 6 and 7 make code more robust by improving bug reporting. Patch 8, 9 and 10 take care of some dangling helpers left in header files. This patch (of 10): There is add_page_to_lru_list(), and move_pages_to_lru() should reuse it, not duplicate it. Link: https://lkml.kernel.org/r/20210122220600.906146-1-yuzhao@google.com Link: https://lore.kernel.org/linux-mm/20201207220949.830352-2-yuzhao@google.com/ Link: https://lkml.kernel.org/r/20210122220600.906146-2-yuzhao@google.com Signed-off-by: Yu Zhao Reviewed-by: Alex Shi Acked-by: Vlastimil Babka Reviewed-by: Miaohe Lin Cc: Hugh Dickins Cc: Michal Hocko Cc: Johannes Weiner Cc: Vladimir Davydov Cc: Roman Gushchin Cc: Matthew Wilcox Signed-off-by: Andrew Morton --- mm/vmscan.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) --- a/mm/vmscan.c~mm-use-add_page_to_lru_list +++ a/mm/vmscan.c @@ -1823,7 +1823,6 @@ static unsigned noinline_for_stack move_ int nr_pages, nr_moved = 0; LIST_HEAD(pages_to_free); struct page *page; - enum lru_list lru; while (!list_empty(list)) { page = lru_to_page(list); @@ -1868,11 +1867,8 @@ static unsigned noinline_for_stack move_ * inhibits memcg migration). */ VM_BUG_ON_PAGE(!lruvec_holds_page_lru_lock(page, lruvec), page); - lru = page_lru(page); + add_page_to_lru_list(page, lruvec, page_lru(page)); nr_pages = thp_nr_pages(page); - - update_lru_size(lruvec, lru, page_zonenum(page), nr_pages); - list_add(&page->lru, &lruvec->lists[lru]); nr_moved += nr_pages; if (PageActive(page)) workingset_age_nonresident(lruvec, nr_pages); From patchwork Wed Feb 24 20:08:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102683 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA9FCC433E6 for ; Wed, 24 Feb 2021 20:08:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 371A664F0D for ; Wed, 24 Feb 2021 20:08:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 371A664F0D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BB5BE8D0033; Wed, 24 Feb 2021 15:08:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B64308D001E; Wed, 24 Feb 2021 15:08:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA23C8D0033; Wed, 24 Feb 2021 15:08:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0049.hostedemail.com [216.40.44.49]) by kanga.kvack.org (Postfix) with ESMTP id 9116E8D001E for ; Wed, 24 Feb 2021 15:08:16 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 5047DEFBC for ; Wed, 24 Feb 2021 20:08:16 +0000 (UTC) X-FDA: 77854248192.29.C39AAAC Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP id 928F5C0007D4 for ; Wed, 24 Feb 2021 20:08:12 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 2619E64F08; Wed, 24 Feb 2021 20:08:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197294; bh=Ze2kz5wkXUpGaQd+JexYlvhuZCedOXyxG0IKush6dYY=; h=Date:From:To:Subject:In-Reply-To:From; b=XvsQK8316P2PFdaAR7obv0j5PqYFXjw0knyLMZah5IovFgkWvrLMpdYoI3iD2p3Su iDK89Cu07UsMm4QxbfU4YXiDckO1UpzYSduVy/8K3MIcVuv6DM0+jgPeATCQ5YVZVr 5QIcHojw+M0PpwFCH4UcXTyA4K78apqS53Sj3ts0= Date: Wed, 24 Feb 2021 12:08:13 -0800 From: Andrew Morton To: akpm@linux-foundation.org, alex.shi@linux.alibaba.com, guro@fb.com, hannes@cmpxchg.org, hughd@google.com, linmiaohe@huawei.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, vdavydov.dev@gmail.com, willy@infradead.org, yuzhao@google.com Subject: [patch 136/173] include/linux/mm_inline.h: shuffle lru list addition and deletion functions Message-ID: <20210224200813.841lGEZQx%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: f8abuw9tjdotqswkcmyxhq63exnuoe5t X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 928F5C0007D4 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf22; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197292-717714 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu Zhao Subject: include/linux/mm_inline.h: shuffle lru list addition and deletion functions These functions will call page_lru() in the following patches. Move them below page_lru() to avoid the forward declaration. Link: https://lore.kernel.org/linux-mm/20201207220949.830352-3-yuzhao@google.com/ Link: https://lkml.kernel.org/r/20210122220600.906146-3-yuzhao@google.com Signed-off-by: Yu Zhao Acked-by: Vlastimil Babka Reviewed-by: Miaohe Lin Cc: Alex Shi Cc: Hugh Dickins Cc: Johannes Weiner Cc: Matthew Wilcox Cc: Michal Hocko Cc: Roman Gushchin Cc: Vladimir Davydov Signed-off-by: Andrew Morton --- include/linux/mm_inline.h | 42 ++++++++++++++++++------------------ 1 file changed, 21 insertions(+), 21 deletions(-) --- a/include/linux/mm_inline.h~mm-shuffle-lru-list-addition-and-deletion-functions +++ a/include/linux/mm_inline.h @@ -45,27 +45,6 @@ static __always_inline void update_lru_s #endif } -static __always_inline void add_page_to_lru_list(struct page *page, - struct lruvec *lruvec, enum lru_list lru) -{ - update_lru_size(lruvec, lru, page_zonenum(page), thp_nr_pages(page)); - list_add(&page->lru, &lruvec->lists[lru]); -} - -static __always_inline void add_page_to_lru_list_tail(struct page *page, - struct lruvec *lruvec, enum lru_list lru) -{ - update_lru_size(lruvec, lru, page_zonenum(page), thp_nr_pages(page)); - list_add_tail(&page->lru, &lruvec->lists[lru]); -} - -static __always_inline void del_page_from_lru_list(struct page *page, - struct lruvec *lruvec, enum lru_list lru) -{ - list_del(&page->lru); - update_lru_size(lruvec, lru, page_zonenum(page), -thp_nr_pages(page)); -} - /** * page_lru_base_type - which LRU list type should a page be on? * @page: the page to test @@ -125,4 +104,25 @@ static __always_inline enum lru_list pag } return lru; } + +static __always_inline void add_page_to_lru_list(struct page *page, + struct lruvec *lruvec, enum lru_list lru) +{ + update_lru_size(lruvec, lru, page_zonenum(page), thp_nr_pages(page)); + list_add(&page->lru, &lruvec->lists[lru]); +} + +static __always_inline void add_page_to_lru_list_tail(struct page *page, + struct lruvec *lruvec, enum lru_list lru) +{ + update_lru_size(lruvec, lru, page_zonenum(page), thp_nr_pages(page)); + list_add_tail(&page->lru, &lruvec->lists[lru]); +} + +static __always_inline void del_page_from_lru_list(struct page *page, + struct lruvec *lruvec, enum lru_list lru) +{ + list_del(&page->lru); + update_lru_size(lruvec, lru, page_zonenum(page), -thp_nr_pages(page)); +} #endif From patchwork Wed Feb 24 20:08:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102685 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 839BEC433DB for ; Wed, 24 Feb 2021 20:08:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 17C5064F19 for ; Wed, 24 Feb 2021 20:08:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 17C5064F19 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9B9298D0034; Wed, 24 Feb 2021 15:08:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9906F8D001E; Wed, 24 Feb 2021 15:08:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 87FF18D0034; Wed, 24 Feb 2021 15:08:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0232.hostedemail.com [216.40.44.232]) by kanga.kvack.org (Postfix) with ESMTP id 6E8208D001E for ; Wed, 24 Feb 2021 15:08:20 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 38BFF18014C9F for ; Wed, 24 Feb 2021 20:08:20 +0000 (UTC) X-FDA: 77854248360.19.EB267F8 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf11.hostedemail.com (Postfix) with ESMTP id 1F7782000391 for ; Wed, 24 Feb 2021 20:08:09 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0D93964F0D; Wed, 24 Feb 2021 20:08:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197298; bh=9H1S/DN99tmFca0yESGx7ECCkIXsZWFLc1b1JEIkSRw=; h=Date:From:To:Subject:In-Reply-To:From; b=NGtfF1Cd9XmXQ9//NGt7KZMvdAT2n3G8Wi9wojo8v19jkpDgf9/pQ0L9pS3drjphq D1b96Maa2SKPFrszcW540HlzmYezgUIoAc1v62OhCD7qH/GCfHT/+2UsGEKG9HoylE 34jU0oiELmgs6VYBDGeiwu8ioZ8R4NNygwQPSzfo= Date: Wed, 24 Feb 2021 12:08:17 -0800 From: Andrew Morton To: akpm@linux-foundation.org, alex.shi@linux.alibaba.com, guro@fb.com, hannes@cmpxchg.org, hughd@google.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, vdavydov.dev@gmail.com, willy@infradead.org, yuzhao@google.com Subject: [patch 137/173] mm: don't pass "enum lru_list" to lru list addition functions Message-ID: <20210224200817.3EXVRnUYe%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 1F7782000391 X-Stat-Signature: hqrygpzq8bzeo7iqpirgxi7ff3pj1tc9 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf11; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197289-739648 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu Zhao Subject: mm: don't pass "enum lru_list" to lru list addition functions The "enum lru_list" parameter to add_page_to_lru_list() and add_page_to_lru_list_tail() is redundant in the sense that it can be extracted from the "struct page" parameter by page_lru(). A caveat is that we need to make sure PageActive() or PageUnevictable() is correctly set or cleared before calling these two functions. And they are indeed. Link: https://lore.kernel.org/linux-mm/20201207220949.830352-4-yuzhao@google.com/ Link: https://lkml.kernel.org/r/20210122220600.906146-4-yuzhao@google.com Signed-off-by: Yu Zhao Cc: Alex Shi Cc: Hugh Dickins Cc: Johannes Weiner Cc: Matthew Wilcox Cc: Michal Hocko Cc: Roman Gushchin Cc: Vladimir Davydov Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- include/linux/mm_inline.h | 8 ++++++-- mm/swap.c | 15 +++++++-------- mm/vmscan.c | 6 ++---- 3 files changed, 15 insertions(+), 14 deletions(-) --- a/include/linux/mm_inline.h~mm-dont-pass-enum-lru_list-to-lru-list-addition-functions +++ a/include/linux/mm_inline.h @@ -106,15 +106,19 @@ static __always_inline enum lru_list pag } static __always_inline void add_page_to_lru_list(struct page *page, - struct lruvec *lruvec, enum lru_list lru) + struct lruvec *lruvec) { + enum lru_list lru = page_lru(page); + update_lru_size(lruvec, lru, page_zonenum(page), thp_nr_pages(page)); list_add(&page->lru, &lruvec->lists[lru]); } static __always_inline void add_page_to_lru_list_tail(struct page *page, - struct lruvec *lruvec, enum lru_list lru) + struct lruvec *lruvec) { + enum lru_list lru = page_lru(page); + update_lru_size(lruvec, lru, page_zonenum(page), thp_nr_pages(page)); list_add_tail(&page->lru, &lruvec->lists[lru]); } --- a/mm/swap.c~mm-dont-pass-enum-lru_list-to-lru-list-addition-functions +++ a/mm/swap.c @@ -231,7 +231,7 @@ static void pagevec_move_tail_fn(struct if (!PageUnevictable(page)) { del_page_from_lru_list(page, lruvec, page_lru(page)); ClearPageActive(page); - add_page_to_lru_list_tail(page, lruvec, page_lru(page)); + add_page_to_lru_list_tail(page, lruvec); __count_vm_events(PGROTATED, thp_nr_pages(page)); } } @@ -313,8 +313,7 @@ static void __activate_page(struct page del_page_from_lru_list(page, lruvec, lru); SetPageActive(page); - lru += LRU_ACTIVE; - add_page_to_lru_list(page, lruvec, lru); + add_page_to_lru_list(page, lruvec); trace_mm_lru_activate(page); __count_vm_events(PGACTIVATE, nr_pages); @@ -543,14 +542,14 @@ static void lru_deactivate_file_fn(struc * It can make readahead confusing. But race window * is _really_ small and it's non-critical problem. */ - add_page_to_lru_list(page, lruvec, lru); + add_page_to_lru_list(page, lruvec); SetPageReclaim(page); } else { /* * The page's writeback ends up during pagevec * We moves tha page into tail of inactive. */ - add_page_to_lru_list_tail(page, lruvec, lru); + add_page_to_lru_list_tail(page, lruvec); __count_vm_events(PGROTATED, nr_pages); } @@ -570,7 +569,7 @@ static void lru_deactivate_fn(struct pag del_page_from_lru_list(page, lruvec, lru + LRU_ACTIVE); ClearPageActive(page); ClearPageReferenced(page); - add_page_to_lru_list(page, lruvec, lru); + add_page_to_lru_list(page, lruvec); __count_vm_events(PGDEACTIVATE, nr_pages); __count_memcg_events(lruvec_memcg(lruvec), PGDEACTIVATE, @@ -595,7 +594,7 @@ static void lru_lazyfree_fn(struct page * anonymous pages */ ClearPageSwapBacked(page); - add_page_to_lru_list(page, lruvec, LRU_INACTIVE_FILE); + add_page_to_lru_list(page, lruvec); __count_vm_events(PGLAZYFREE, nr_pages); __count_memcg_events(lruvec_memcg(lruvec), PGLAZYFREE, @@ -1005,7 +1004,7 @@ static void __pagevec_lru_add_fn(struct __count_vm_events(UNEVICTABLE_PGCULLED, nr_pages); } - add_page_to_lru_list(page, lruvec, lru); + add_page_to_lru_list(page, lruvec); trace_mm_lru_insertion(page, lru); } --- a/mm/vmscan.c~mm-dont-pass-enum-lru_list-to-lru-list-addition-functions +++ a/mm/vmscan.c @@ -1867,7 +1867,7 @@ static unsigned noinline_for_stack move_ * inhibits memcg migration). */ VM_BUG_ON_PAGE(!lruvec_holds_page_lru_lock(page, lruvec), page); - add_page_to_lru_list(page, lruvec, page_lru(page)); + add_page_to_lru_list(page, lruvec); nr_pages = thp_nr_pages(page); nr_moved += nr_pages; if (PageActive(page)) @@ -4282,12 +4282,10 @@ void check_move_unevictable_pages(struct lruvec = relock_page_lruvec_irq(page, lruvec); if (page_evictable(page) && PageUnevictable(page)) { - enum lru_list lru = page_lru_base_type(page); - VM_BUG_ON_PAGE(PageActive(page), page); ClearPageUnevictable(page); del_page_from_lru_list(page, lruvec, LRU_UNEVICTABLE); - add_page_to_lru_list(page, lruvec, lru); + add_page_to_lru_list(page, lruvec); pgrescued += nr_pages; } SetPageLRU(page); From patchwork Wed Feb 24 20:08:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102687 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B6A4C433E0 for ; Wed, 24 Feb 2021 20:08:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 119E664F1B for ; Wed, 24 Feb 2021 20:08:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 119E664F1B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 91C9E8D0035; Wed, 24 Feb 2021 15:08:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8CC018D001E; Wed, 24 Feb 2021 15:08:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E2728D0035; Wed, 24 Feb 2021 15:08:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0152.hostedemail.com [216.40.44.152]) by kanga.kvack.org (Postfix) with ESMTP id 660DF8D001E for ; Wed, 24 Feb 2021 15:08:25 -0500 (EST) Received: from smtpin04.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 331ABEFBC for ; Wed, 24 Feb 2021 20:08:25 +0000 (UTC) X-FDA: 77854248570.04.CB433CF Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP id C81F490009EB for ; Wed, 24 Feb 2021 20:08:19 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id E860564F09; Wed, 24 Feb 2021 20:08:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197302; bh=bRWqzn2KTS9tKTMlKKY2Uwp6eBB0yq+O+PeWay7ZNOQ=; h=Date:From:To:Subject:In-Reply-To:From; b=SxcmFi0riSsCpkrMYGazBCkgbyxenbKUslKbwVy/lPfKijWK5Ngo6/bV78N//2D4f UinRBVCcsNaVKkWo6IjpQT97xl5fKuZ9DMMvC+JF+5nnE3lUcpuA8K1DNbNXsxEMro dDuYq5urSOKjK5pKknaEPuJQIw9gee3ljHfOYwlI= Date: Wed, 24 Feb 2021 12:08:21 -0800 From: Andrew Morton To: akpm@linux-foundation.org, alex.shi@linux.alibaba.com, guro@fb.com, hannes@cmpxchg.org, hughd@google.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, vdavydov.dev@gmail.com, willy@infradead.org, yuzhao@google.com Subject: [patch 138/173] mm/swap.c: don't pass "enum lru_list" to trace_mm_lru_insertion() Message-ID: <20210224200821.Qm_Mk-Rah%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: C81F490009EB X-Stat-Signature: fmuxswgr678ec5xu5hmme1ge4chsz3ca Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf19; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197299-856962 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu Zhao Subject: mm/swap.c: don't pass "enum lru_list" to trace_mm_lru_insertion() The parameter is redundant in the sense that it can be extracted from the "struct page" parameter by page_lru() correctly. Link: https://lore.kernel.org/linux-mm/20201207220949.830352-5-yuzhao@google.com/ Link: https://lkml.kernel.org/r/20210122220600.906146-5-yuzhao@google.com Signed-off-by: Yu Zhao Reviewed-by: Alex Shi Cc: Hugh Dickins Cc: Johannes Weiner Cc: Matthew Wilcox Cc: Michal Hocko Cc: Roman Gushchin Cc: Vladimir Davydov Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- include/trace/events/pagemap.h | 11 ++++------- mm/swap.c | 5 +---- 2 files changed, 5 insertions(+), 11 deletions(-) --- a/include/trace/events/pagemap.h~mm-dont-pass-enum-lru_list-to-trace_mm_lru_insertion +++ a/include/trace/events/pagemap.h @@ -27,24 +27,21 @@ TRACE_EVENT(mm_lru_insertion, - TP_PROTO( - struct page *page, - int lru - ), + TP_PROTO(struct page *page), - TP_ARGS(page, lru), + TP_ARGS(page), TP_STRUCT__entry( __field(struct page *, page ) __field(unsigned long, pfn ) - __field(int, lru ) + __field(enum lru_list, lru ) __field(unsigned long, flags ) ), TP_fast_assign( __entry->page = page; __entry->pfn = page_to_pfn(page); - __entry->lru = lru; + __entry->lru = page_lru(page); __entry->flags = trace_pagemap_flags(page); ), --- a/mm/swap.c~mm-dont-pass-enum-lru_list-to-trace_mm_lru_insertion +++ a/mm/swap.c @@ -957,7 +957,6 @@ EXPORT_SYMBOL(__pagevec_release); static void __pagevec_lru_add_fn(struct page *page, struct lruvec *lruvec) { - enum lru_list lru; int was_unevictable = TestClearPageUnevictable(page); int nr_pages = thp_nr_pages(page); @@ -993,11 +992,9 @@ static void __pagevec_lru_add_fn(struct smp_mb__after_atomic(); if (page_evictable(page)) { - lru = page_lru(page); if (was_unevictable) __count_vm_events(UNEVICTABLE_PGRESCUED, nr_pages); } else { - lru = LRU_UNEVICTABLE; ClearPageActive(page); SetPageUnevictable(page); if (!was_unevictable) @@ -1005,7 +1002,7 @@ static void __pagevec_lru_add_fn(struct } add_page_to_lru_list(page, lruvec); - trace_mm_lru_insertion(page, lru); + trace_mm_lru_insertion(page); } /* From patchwork Wed Feb 24 20:08:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102689 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6702BC433E0 for ; Wed, 24 Feb 2021 20:08:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E105964E24 for ; Wed, 24 Feb 2021 20:08:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E105964E24 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 795178D0036; Wed, 24 Feb 2021 15:08:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 746448D001E; Wed, 24 Feb 2021 15:08:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 634DF8D0036; Wed, 24 Feb 2021 15:08:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0060.hostedemail.com [216.40.44.60]) by kanga.kvack.org (Postfix) with ESMTP id 474CB8D001E for ; Wed, 24 Feb 2021 15:08:28 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 02F3A181AF5C2 for ; Wed, 24 Feb 2021 20:08:28 +0000 (UTC) X-FDA: 77854248696.23.237FFDC Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP id 71D09600249E for ; Wed, 24 Feb 2021 20:08:23 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B902F64F0E; Wed, 24 Feb 2021 20:08:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197306; bh=Z1dPdMVUtCMcxzFTOSbKrbwASX4qh9o6g9mKPLE0Tw4=; h=Date:From:To:Subject:In-Reply-To:From; b=WXAhxOSgFWMjGsKQt+5aZvh0KcF71NuU7/YeOB4y0XNwxnFhP94VlQjd8MbZ7J8FC ATeKhdYIRqs3EnSsY1q7CQ4OLEDj8IxwCyn+2/DwpgCmK2Ba075wd6j76pZTKe1KXx MdY9/SGIWEWbUVLadNRIxjaSOZ8tq03/+aoE6dgk= Date: Wed, 24 Feb 2021 12:08:25 -0800 From: Andrew Morton To: akpm@linux-foundation.org, alex.shi@linux.alibaba.com, guro@fb.com, hannes@cmpxchg.org, hughd@google.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, vdavydov.dev@gmail.com, willy@infradead.org, yuzhao@google.com Subject: [patch 139/173] mm/swap.c: don't pass "enum lru_list" to del_page_from_lru_list() Message-ID: <20210224200825._eP-FQyg3%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: s9xkj7nuea3pmagws5hbu7jpbjan47j6 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 71D09600249E Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf09; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197303-236718 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu Zhao Subject: mm/swap.c: don't pass "enum lru_list" to del_page_from_lru_list() The parameter is redundant in the sense that it can be potentially extracted from the "struct page" parameter by page_lru(). We need to make sure that existing PageActive() or PageUnevictable() remains until the function returns. A few places don't conform, and simple reordering fixes them. This patch may have left page_off_lru() seemingly odd, and we'll take care of it in the next patch. Link: https://lore.kernel.org/linux-mm/20201207220949.830352-6-yuzhao@google.com/ Link: https://lkml.kernel.org/r/20210122220600.906146-6-yuzhao@google.com Signed-off-by: Yu Zhao Cc: Alex Shi Cc: Hugh Dickins Cc: Johannes Weiner Cc: Matthew Wilcox Cc: Michal Hocko Cc: Roman Gushchin Cc: Vladimir Davydov Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- include/linux/mm_inline.h | 5 +++-- mm/compaction.c | 2 +- mm/mlock.c | 3 +-- mm/swap.c | 26 ++++++++++---------------- mm/vmscan.c | 4 ++-- 5 files changed, 17 insertions(+), 23 deletions(-) --- a/include/linux/mm_inline.h~mm-dont-pass-enum-lru_list-to-del_page_from_lru_list +++ a/include/linux/mm_inline.h @@ -124,9 +124,10 @@ static __always_inline void add_page_to_ } static __always_inline void del_page_from_lru_list(struct page *page, - struct lruvec *lruvec, enum lru_list lru) + struct lruvec *lruvec) { list_del(&page->lru); - update_lru_size(lruvec, lru, page_zonenum(page), -thp_nr_pages(page)); + update_lru_size(lruvec, page_lru(page), page_zonenum(page), + -thp_nr_pages(page)); } #endif --- a/mm/compaction.c~mm-dont-pass-enum-lru_list-to-del_page_from_lru_list +++ a/mm/compaction.c @@ -1034,7 +1034,7 @@ isolate_migratepages_block(struct compac low_pfn += compound_nr(page) - 1; /* Successfully isolated */ - del_page_from_lru_list(page, lruvec, page_lru(page)); + del_page_from_lru_list(page, lruvec); mod_node_page_state(page_pgdat(page), NR_ISOLATED_ANON + page_is_file_lru(page), thp_nr_pages(page)); --- a/mm/mlock.c~mm-dont-pass-enum-lru_list-to-del_page_from_lru_list +++ a/mm/mlock.c @@ -278,8 +278,7 @@ static void __munlock_pagevec(struct pag */ if (TestClearPageLRU(page)) { lruvec = relock_page_lruvec_irq(page, lruvec); - del_page_from_lru_list(page, lruvec, - page_lru(page)); + del_page_from_lru_list(page, lruvec); continue; } else __munlock_isolation_failed(page); --- a/mm/swap.c~mm-dont-pass-enum-lru_list-to-del_page_from_lru_list +++ a/mm/swap.c @@ -85,7 +85,8 @@ static void __page_cache_release(struct lruvec = lock_page_lruvec_irqsave(page, &flags); VM_BUG_ON_PAGE(!PageLRU(page), page); __ClearPageLRU(page); - del_page_from_lru_list(page, lruvec, page_off_lru(page)); + del_page_from_lru_list(page, lruvec); + page_off_lru(page); unlock_page_lruvec_irqrestore(lruvec, flags); } __ClearPageWaiters(page); @@ -229,7 +230,7 @@ static void pagevec_lru_move_fn(struct p static void pagevec_move_tail_fn(struct page *page, struct lruvec *lruvec) { if (!PageUnevictable(page)) { - del_page_from_lru_list(page, lruvec, page_lru(page)); + del_page_from_lru_list(page, lruvec); ClearPageActive(page); add_page_to_lru_list_tail(page, lruvec); __count_vm_events(PGROTATED, thp_nr_pages(page)); @@ -308,10 +309,9 @@ void lru_note_cost_page(struct page *pag static void __activate_page(struct page *page, struct lruvec *lruvec) { if (!PageActive(page) && !PageUnevictable(page)) { - int lru = page_lru_base_type(page); int nr_pages = thp_nr_pages(page); - del_page_from_lru_list(page, lruvec, lru); + del_page_from_lru_list(page, lruvec); SetPageActive(page); add_page_to_lru_list(page, lruvec); trace_mm_lru_activate(page); @@ -518,8 +518,7 @@ void lru_cache_add_inactive_or_unevictab */ static void lru_deactivate_file_fn(struct page *page, struct lruvec *lruvec) { - int lru; - bool active; + bool active = PageActive(page); int nr_pages = thp_nr_pages(page); if (PageUnevictable(page)) @@ -529,10 +528,7 @@ static void lru_deactivate_file_fn(struc if (page_mapped(page)) return; - active = PageActive(page); - lru = page_lru_base_type(page); - - del_page_from_lru_list(page, lruvec, lru + active); + del_page_from_lru_list(page, lruvec); ClearPageActive(page); ClearPageReferenced(page); @@ -563,10 +559,9 @@ static void lru_deactivate_file_fn(struc static void lru_deactivate_fn(struct page *page, struct lruvec *lruvec) { if (PageActive(page) && !PageUnevictable(page)) { - int lru = page_lru_base_type(page); int nr_pages = thp_nr_pages(page); - del_page_from_lru_list(page, lruvec, lru + LRU_ACTIVE); + del_page_from_lru_list(page, lruvec); ClearPageActive(page); ClearPageReferenced(page); add_page_to_lru_list(page, lruvec); @@ -581,11 +576,9 @@ static void lru_lazyfree_fn(struct page { if (PageAnon(page) && PageSwapBacked(page) && !PageSwapCache(page) && !PageUnevictable(page)) { - bool active = PageActive(page); int nr_pages = thp_nr_pages(page); - del_page_from_lru_list(page, lruvec, - LRU_INACTIVE_ANON + active); + del_page_from_lru_list(page, lruvec); ClearPageActive(page); ClearPageReferenced(page); /* @@ -919,7 +912,8 @@ void release_pages(struct page **pages, VM_BUG_ON_PAGE(!PageLRU(page), page); __ClearPageLRU(page); - del_page_from_lru_list(page, lruvec, page_off_lru(page)); + del_page_from_lru_list(page, lruvec); + page_off_lru(page); } __ClearPageWaiters(page); --- a/mm/vmscan.c~mm-dont-pass-enum-lru_list-to-del_page_from_lru_list +++ a/mm/vmscan.c @@ -1766,7 +1766,7 @@ int isolate_lru_page(struct page *page) get_page(page); lruvec = lock_page_lruvec_irq(page); - del_page_from_lru_list(page, lruvec, page_lru(page)); + del_page_from_lru_list(page, lruvec); unlock_page_lruvec_irq(lruvec); ret = 0; } @@ -4283,8 +4283,8 @@ void check_move_unevictable_pages(struct lruvec = relock_page_lruvec_irq(page, lruvec); if (page_evictable(page) && PageUnevictable(page)) { VM_BUG_ON_PAGE(PageActive(page), page); + del_page_from_lru_list(page, lruvec); ClearPageUnevictable(page); - del_page_from_lru_list(page, lruvec, LRU_UNEVICTABLE); add_page_to_lru_list(page, lruvec); pgrescued += nr_pages; } From patchwork Wed Feb 24 20:08:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102693 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 923AEC433E0 for ; Wed, 24 Feb 2021 20:08:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1285B64F1B for ; Wed, 24 Feb 2021 20:08:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1285B64F1B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 10D738D0038; Wed, 24 Feb 2021 15:08:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 041CC8D001E; Wed, 24 Feb 2021 15:08:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E9A368D0038; Wed, 24 Feb 2021 15:08:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0236.hostedemail.com [216.40.44.236]) by kanga.kvack.org (Postfix) with ESMTP id CE3EC8D001E for ; Wed, 24 Feb 2021 15:08:32 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 906E4801C0FE for ; Wed, 24 Feb 2021 20:08:32 +0000 (UTC) X-FDA: 77854248864.03.04BCE49 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf30.hostedemail.com (Postfix) with ESMTP id 0FB73E000119 for ; Wed, 24 Feb 2021 20:08:30 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8176464F1D; Wed, 24 Feb 2021 20:08:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197310; bh=Y0+LHY/h97swDRw6Vb+0jE85/g2oXqbQ5XEZhkLKnok=; h=Date:From:To:Subject:In-Reply-To:From; b=If8bOR32/MNwHofCUdN07MzYq8bIuxZKKqY6mCUA9F9ots/zDe4zD+wzSASCr2VvI PckfHxCr7M+LQoMI9crhuul9mhgBGI52IWKAa8MBiXO8g14mqSvBrHQT3QVcrpd4d6 DnZvV9EWpq+UGmGAZi+WPupcRWR17Fhv7zkghlTc= Date: Wed, 24 Feb 2021 12:08:28 -0800 From: Andrew Morton To: akpm@linux-foundation.org, alex.shi@linux.alibaba.com, guro@fb.com, hannes@cmpxchg.org, hughd@google.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, vdavydov.dev@gmail.com, willy@infradead.org, yuzhao@google.com Subject: [patch 140/173] mm: add __clear_page_lru_flags() to replace page_off_lru() Message-ID: <20210224200828.TIcm2MbA8%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 0FB73E000119 X-Stat-Signature: 9fhup3s1gsokiq8z3ci5m7m5qoam6ug7 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf30; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197310-736749 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu Zhao Subject: mm: add __clear_page_lru_flags() to replace page_off_lru() Similar to page_off_lru(), the new function does non-atomic clearing of PageLRU() in addition to PageActive() and PageUnevictable(), on a page that has no references left. If PageActive() and PageUnevictable() are both set, refuse to clear either and leave them to bad_page(). This is a behavior change that is meant to help debug. Link: https://lore.kernel.org/linux-mm/20201207220949.830352-7-yuzhao@google.com/ Link: https://lkml.kernel.org/r/20210122220600.906146-7-yuzhao@google.com Signed-off-by: Yu Zhao Cc: Alex Shi Cc: Hugh Dickins Cc: Johannes Weiner Cc: Matthew Wilcox Cc: Michal Hocko Cc: Roman Gushchin Cc: Vladimir Davydov Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- include/linux/mm_inline.h | 28 ++++++++++------------------ mm/swap.c | 6 ++---- mm/vmscan.c | 3 +-- 3 files changed, 13 insertions(+), 24 deletions(-) --- a/include/linux/mm_inline.h~mm-add-__clear_page_lru_flags-to-replace-page_off_lru +++ a/include/linux/mm_inline.h @@ -61,27 +61,19 @@ static inline enum lru_list page_lru_bas } /** - * page_off_lru - which LRU list was page on? clearing its lru flags. - * @page: the page to test - * - * Returns the LRU list a page was on, as an index into the array of LRU - * lists; and clears its Unevictable or Active flags, ready for freeing. + * __clear_page_lru_flags - clear page lru flags before releasing a page + * @page: the page that was on lru and now has a zero reference */ -static __always_inline enum lru_list page_off_lru(struct page *page) +static __always_inline void __clear_page_lru_flags(struct page *page) { - enum lru_list lru; + __ClearPageLRU(page); - if (PageUnevictable(page)) { - __ClearPageUnevictable(page); - lru = LRU_UNEVICTABLE; - } else { - lru = page_lru_base_type(page); - if (PageActive(page)) { - __ClearPageActive(page); - lru += LRU_ACTIVE; - } - } - return lru; + /* this shouldn't happen, so leave the flags to bad_page() */ + if (PageActive(page) && PageUnevictable(page)) + return; + + __ClearPageActive(page); + __ClearPageUnevictable(page); } /** --- a/mm/swap.c~mm-add-__clear_page_lru_flags-to-replace-page_off_lru +++ a/mm/swap.c @@ -84,9 +84,8 @@ static void __page_cache_release(struct lruvec = lock_page_lruvec_irqsave(page, &flags); VM_BUG_ON_PAGE(!PageLRU(page), page); - __ClearPageLRU(page); del_page_from_lru_list(page, lruvec); - page_off_lru(page); + __clear_page_lru_flags(page); unlock_page_lruvec_irqrestore(lruvec, flags); } __ClearPageWaiters(page); @@ -911,9 +910,8 @@ void release_pages(struct page **pages, lock_batch = 0; VM_BUG_ON_PAGE(!PageLRU(page), page); - __ClearPageLRU(page); del_page_from_lru_list(page, lruvec); - page_off_lru(page); + __clear_page_lru_flags(page); } __ClearPageWaiters(page); --- a/mm/vmscan.c~mm-add-__clear_page_lru_flags-to-replace-page_off_lru +++ a/mm/vmscan.c @@ -1849,8 +1849,7 @@ static unsigned noinline_for_stack move_ SetPageLRU(page); if (unlikely(put_page_testzero(page))) { - __ClearPageLRU(page); - __ClearPageActive(page); + __clear_page_lru_flags(page); if (unlikely(PageCompound(page))) { spin_unlock_irq(&lruvec->lru_lock); From patchwork Wed Feb 24 20:08:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102695 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10CA1C433DB for ; Wed, 24 Feb 2021 20:08:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AC6BB64F0C for ; Wed, 24 Feb 2021 20:08:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AC6BB64F0C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 302BC8D003B; Wed, 24 Feb 2021 15:08:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 28B438D001E; Wed, 24 Feb 2021 15:08:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 17BD68D003B; Wed, 24 Feb 2021 15:08:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0083.hostedemail.com [216.40.44.83]) by kanga.kvack.org (Postfix) with ESMTP id E59B38D001E for ; Wed, 24 Feb 2021 15:08:35 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id B8509B9EC for ; Wed, 24 Feb 2021 20:08:35 +0000 (UTC) X-FDA: 77854248990.06.6D07FB8 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf13.hostedemail.com (Postfix) with ESMTP id 942EAE000113 for ; Wed, 24 Feb 2021 20:08:32 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 48C6F64E7A; Wed, 24 Feb 2021 20:08:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197314; bh=sylk1bvGCaTJ5Qua3xWqLf5tDYuM1TisyKVRzm9pXdo=; h=Date:From:To:Subject:In-Reply-To:From; b=TwSKAyX3rsJsdBfH1iN3WFEsc+IpsOhWyJyAQoQYD0ufkUFk8vY28lLmcSb9ZcfBq rEJanW8UETr4HgNU8N9aBSCbtMtYRIZLnFCPIVvuhSmUJnA8nacaOGWpmAqZ2O4v7W E7zRBjbFQ+YO+g/MaOjpRmu56dgZ3WjAX0//Xq2Q= Date: Wed, 24 Feb 2021 12:08:32 -0800 From: Andrew Morton To: akpm@linux-foundation.org, alex.shi@linux.alibaba.com, guro@fb.com, hannes@cmpxchg.org, hughd@google.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, vdavydov.dev@gmail.com, willy@infradead.org, yuzhao@google.com Subject: [patch 141/173] mm: VM_BUG_ON lru page flags Message-ID: <20210224200832.cfPceYDI1%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: oqgekc5p7qqsrbg8z4xoe9a43d8ykk38 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 942EAE000113 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf13; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197312-139114 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu Zhao Subject: mm: VM_BUG_ON lru page flags Move scattered VM_BUG_ONs to two essential places that cover all lru list additions and deletions. Link: https://lore.kernel.org/linux-mm/20201207220949.830352-8-yuzhao@google.com/ Link: https://lkml.kernel.org/r/20210122220600.906146-8-yuzhao@google.com Signed-off-by: Yu Zhao Cc: Alex Shi Cc: Hugh Dickins Cc: Johannes Weiner Cc: Matthew Wilcox Cc: Michal Hocko Cc: Roman Gushchin Cc: Vladimir Davydov Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- include/linux/mm_inline.h | 4 ++++ mm/swap.c | 2 -- mm/vmscan.c | 1 - 3 files changed, 4 insertions(+), 3 deletions(-) --- a/include/linux/mm_inline.h~mm-vm_bug_on-lru-page-flags +++ a/include/linux/mm_inline.h @@ -66,6 +66,8 @@ static inline enum lru_list page_lru_bas */ static __always_inline void __clear_page_lru_flags(struct page *page) { + VM_BUG_ON_PAGE(!PageLRU(page), page); + __ClearPageLRU(page); /* this shouldn't happen, so leave the flags to bad_page() */ @@ -87,6 +89,8 @@ static __always_inline enum lru_list pag { enum lru_list lru; + VM_BUG_ON_PAGE(PageActive(page) && PageUnevictable(page), page); + if (PageUnevictable(page)) lru = LRU_UNEVICTABLE; else { --- a/mm/swap.c~mm-vm_bug_on-lru-page-flags +++ a/mm/swap.c @@ -83,7 +83,6 @@ static void __page_cache_release(struct unsigned long flags; lruvec = lock_page_lruvec_irqsave(page, &flags); - VM_BUG_ON_PAGE(!PageLRU(page), page); del_page_from_lru_list(page, lruvec); __clear_page_lru_flags(page); unlock_page_lruvec_irqrestore(lruvec, flags); @@ -909,7 +908,6 @@ void release_pages(struct page **pages, if (prev_lruvec != lruvec) lock_batch = 0; - VM_BUG_ON_PAGE(!PageLRU(page), page); del_page_from_lru_list(page, lruvec); __clear_page_lru_flags(page); } --- a/mm/vmscan.c~mm-vm_bug_on-lru-page-flags +++ a/mm/vmscan.c @@ -4281,7 +4281,6 @@ void check_move_unevictable_pages(struct lruvec = relock_page_lruvec_irq(page, lruvec); if (page_evictable(page) && PageUnevictable(page)) { - VM_BUG_ON_PAGE(PageActive(page), page); del_page_from_lru_list(page, lruvec); ClearPageUnevictable(page); add_page_to_lru_list(page, lruvec); From patchwork Wed Feb 24 20:08:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102697 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99397C433E0 for ; Wed, 24 Feb 2021 20:08:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E50E464E24 for ; Wed, 24 Feb 2021 20:08:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E50E464E24 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 76E4E8D0040; Wed, 24 Feb 2021 15:08:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 71DCB8D001E; Wed, 24 Feb 2021 15:08:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 65D628D0040; Wed, 24 Feb 2021 15:08:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0247.hostedemail.com [216.40.44.247]) by kanga.kvack.org (Postfix) with ESMTP id 4C2448D001E for ; Wed, 24 Feb 2021 15:08:39 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 15551EFBC for ; Wed, 24 Feb 2021 20:08:39 +0000 (UTC) X-FDA: 77854249158.18.2FC239B Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf15.hostedemail.com (Postfix) with ESMTP id 8A70CA00051E for ; Wed, 24 Feb 2021 20:08:37 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0CF9264F19; Wed, 24 Feb 2021 20:08:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197317; bh=cHn69o82uKThLvMhITz2Vx+7K02tQd2WOr0j7Ajb9zs=; h=Date:From:To:Subject:In-Reply-To:From; b=laH0GWEN/PCjIef2AZYhkbYoGx6biS/96zg7CcxVqEB6VBE+iMmWXKCLRY7TEYrZ3 4hf+rVi93Tyrk9UPG8VDX+vct14VOhWSFIJayrygzBQLEg0R7f2aajrSfUKqqLhy1u ZeVC2Ox2OZdMKVIyX3Ens5O1HC0b8+oznhPul5Qo= Date: Wed, 24 Feb 2021 12:08:36 -0800 From: Andrew Morton To: akpm@linux-foundation.org, alex.shi@linux.alibaba.com, guro@fb.com, hannes@cmpxchg.org, hughd@google.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, vdavydov.dev@gmail.com, willy@infradead.org, yuzhao@google.com Subject: [patch 142/173] include/linux/mm_inline.h: fold page_lru_base_type() into its sole caller Message-ID: <20210224200836.7SCiOTMZW%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: efcc77g5j6ad4bgf8da8ke4a9czit9pd X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 8A70CA00051E Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf15; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197317-770753 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu Zhao Subject: include/linux/mm_inline.h: fold page_lru_base_type() into its sole caller We've removed all other references to this function. Link: https://lore.kernel.org/linux-mm/20201207220949.830352-9-yuzhao@google.com/ Link: https://lkml.kernel.org/r/20210122220600.906146-9-yuzhao@google.com Signed-off-by: Yu Zhao Reviewed-by: Alex Shi Cc: Hugh Dickins Cc: Johannes Weiner Cc: Matthew Wilcox Cc: Michal Hocko Cc: Roman Gushchin Cc: Vladimir Davydov Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- include/linux/mm_inline.h | 27 ++++++--------------------- 1 file changed, 6 insertions(+), 21 deletions(-) --- a/include/linux/mm_inline.h~mm-fold-page_lru_base_type-into-its-sole-caller +++ a/include/linux/mm_inline.h @@ -46,21 +46,6 @@ static __always_inline void update_lru_s } /** - * page_lru_base_type - which LRU list type should a page be on? - * @page: the page to test - * - * Used for LRU list index arithmetic. - * - * Returns the base LRU type - file or anon - @page should be on. - */ -static inline enum lru_list page_lru_base_type(struct page *page) -{ - if (page_is_file_lru(page)) - return LRU_INACTIVE_FILE; - return LRU_INACTIVE_ANON; -} - -/** * __clear_page_lru_flags - clear page lru flags before releasing a page * @page: the page that was on lru and now has a zero reference */ @@ -92,12 +77,12 @@ static __always_inline enum lru_list pag VM_BUG_ON_PAGE(PageActive(page) && PageUnevictable(page), page); if (PageUnevictable(page)) - lru = LRU_UNEVICTABLE; - else { - lru = page_lru_base_type(page); - if (PageActive(page)) - lru += LRU_ACTIVE; - } + return LRU_UNEVICTABLE; + + lru = page_is_file_lru(page) ? LRU_INACTIVE_FILE : LRU_INACTIVE_ANON; + if (PageActive(page)) + lru += LRU_ACTIVE; + return lru; } From patchwork Wed Feb 24 20:08:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102699 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D0F9C433E9 for ; Wed, 24 Feb 2021 20:08:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CC08E64F1B for ; Wed, 24 Feb 2021 20:08:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CC08E64F1B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 59A798D0041; Wed, 24 Feb 2021 15:08:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 56FE08D001E; Wed, 24 Feb 2021 15:08:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4AD948D0041; Wed, 24 Feb 2021 15:08:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0176.hostedemail.com [216.40.44.176]) by kanga.kvack.org (Postfix) with ESMTP id 3217A8D001E for ; Wed, 24 Feb 2021 15:08:43 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id F0A86180ACF0E for ; Wed, 24 Feb 2021 20:08:42 +0000 (UTC) X-FDA: 77854249284.01.A8904AB Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf08.hostedemail.com (Postfix) with ESMTP id 1047A80192D8 for ; Wed, 24 Feb 2021 20:08:30 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C793B64F0C; Wed, 24 Feb 2021 20:08:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197321; bh=B3j6aQ5k4nwbd/NDC8Jy9OtgrbJbOTgeQPo39FtirZE=; h=Date:From:To:Subject:In-Reply-To:From; b=Azm8Rfk1XsdfrdYh6f8kz02UHqm5Q8mULS5jXGVsMZDFioDh4Q3dddNmfA1Pg0n4e QhFzIIXhX0ZBXuorjyy5FVxb6+qa+eywBFiOFbHpL11r+bfgVF5DJy8JXajd4lKfmU nuoO33UaQ0SF60ccjju5BHH4tUfEk8o+bXczD3wM= Date: Wed, 24 Feb 2021 12:08:40 -0800 From: Andrew Morton To: akpm@linux-foundation.org, alex.shi@linux.alibaba.com, guro@fb.com, hannes@cmpxchg.org, hughd@google.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, vdavydov.dev@gmail.com, willy@infradead.org, yuzhao@google.com Subject: [patch 143/173] include/linux/mm_inline.h: fold __update_lru_size() into its sole caller Message-ID: <20210224200840.g4YLXIUDy%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 1047A80192D8 X-Stat-Signature: gi99u7eirn8n9egu1zrdn6tu5eauiun3 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf08; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197310-339318 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu Zhao Subject: include/linux/mm_inline.h: fold __update_lru_size() into its sole caller All other references to the function were removed after commit a892cb6b977f ("mm/vmscan.c: use update_lru_size() in update_lru_sizes()"). Link: https://lore.kernel.org/linux-mm/20201207220949.830352-10-yuzhao@google.com/ Link: https://lkml.kernel.org/r/20210122220600.906146-10-yuzhao@google.com Signed-off-by: Yu Zhao Reviewed-by: Alex Shi Cc: Hugh Dickins Cc: Johannes Weiner Cc: Matthew Wilcox Cc: Michal Hocko Cc: Roman Gushchin Cc: Vladimir Davydov Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- include/linux/mm_inline.h | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) --- a/include/linux/mm_inline.h~mm-fold-__update_lru_size-into-its-sole-caller +++ a/include/linux/mm_inline.h @@ -24,7 +24,7 @@ static inline int page_is_file_lru(struc return !PageSwapBacked(page); } -static __always_inline void __update_lru_size(struct lruvec *lruvec, +static __always_inline void update_lru_size(struct lruvec *lruvec, enum lru_list lru, enum zone_type zid, int nr_pages) { @@ -33,13 +33,6 @@ static __always_inline void __update_lru __mod_lruvec_state(lruvec, NR_LRU_BASE + lru, nr_pages); __mod_zone_page_state(&pgdat->node_zones[zid], NR_ZONE_LRU_BASE + lru, nr_pages); -} - -static __always_inline void update_lru_size(struct lruvec *lruvec, - enum lru_list lru, enum zone_type zid, - int nr_pages) -{ - __update_lru_size(lruvec, lru, zid, nr_pages); #ifdef CONFIG_MEMCG mem_cgroup_update_lru_size(lruvec, lru, zid, nr_pages); #endif From patchwork Wed Feb 24 20:08:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102701 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1AB3C433E6 for ; Wed, 24 Feb 2021 20:08:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6512164E24 for ; Wed, 24 Feb 2021 20:08:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6512164E24 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EC0BA8D0047; Wed, 24 Feb 2021 15:08:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E97F08D001E; Wed, 24 Feb 2021 15:08:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DAE678D0047; Wed, 24 Feb 2021 15:08:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0192.hostedemail.com [216.40.44.192]) by kanga.kvack.org (Postfix) with ESMTP id C23EE8D001E for ; Wed, 24 Feb 2021 15:08:46 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 911081802EAC2 for ; Wed, 24 Feb 2021 20:08:46 +0000 (UTC) X-FDA: 77854249452.11.5A49B2F Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP id 275A4E7 for ; Wed, 24 Feb 2021 20:08:46 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 96B9A64EC4; Wed, 24 Feb 2021 20:08:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197325; bh=5wYGO9GCOFBImmQiQViHlgsmfkGZss8yis5SW5Tk48g=; h=Date:From:To:Subject:In-Reply-To:From; b=XWUEQKa3o3aOockGyH5JwG69l8TK/eT5Ut/RT/Wzov5Ceq3DHSiQigmfSxIhO7nwx DuNtTm+x/D57ZqXKmoMtAp1P+mOz39/AXh48bdQo3yGOFxoIpfoBeZdWSZZEgGDVLE Q0cVDZgk3o3/G3ayE/wvdmOWJ/xzJoARJ4suCvLw= Date: Wed, 24 Feb 2021 12:08:44 -0800 From: Andrew Morton To: akpm@linux-foundation.org, alex.shi@linux.alibaba.com, guro@fb.com, hannes@cmpxchg.org, hughd@google.com, linux-mm@kvack.org, mhocko@kernel.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, vdavydov.dev@gmail.com, willy@infradead.org, yuzhao@google.com Subject: [patch 144/173] mm/vmscan.c: make lruvec_lru_size() static Message-ID: <20210224200844.NjYv68NcK%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: oxajhswryu7ixeazxri8i97uciybqtsn X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 275A4E7 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf20; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197326-422245 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu Zhao Subject: mm/vmscan.c: make lruvec_lru_size() static All other references to the function were removed after commit b910718a948a ("mm: vmscan: detect file thrashing at the reclaim root"). Link: https://lore.kernel.org/linux-mm/20201207220949.830352-11-yuzhao@google.com/ Link: https://lkml.kernel.org/r/20210122220600.906146-11-yuzhao@google.com Signed-off-by: Yu Zhao Reviewed-by: Alex Shi Cc: Hugh Dickins Cc: Johannes Weiner Cc: Matthew Wilcox Cc: Michal Hocko Cc: Roman Gushchin Cc: Vladimir Davydov Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- include/linux/mmzone.h | 2 -- mm/vmscan.c | 3 ++- 2 files changed, 2 insertions(+), 3 deletions(-) --- a/include/linux/mmzone.h~mm-make-lruvec_lru_size-static +++ a/include/linux/mmzone.h @@ -892,8 +892,6 @@ static inline struct pglist_data *lruvec #endif } -extern unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru, int zone_idx); - #ifdef CONFIG_HAVE_MEMORYLESS_NODES int local_memory_node(int node_id); #else --- a/mm/vmscan.c~mm-make-lruvec_lru_size-static +++ a/mm/vmscan.c @@ -310,7 +310,8 @@ unsigned long zone_reclaimable_pages(str * @lru: lru to use * @zone_idx: zones to consider (use MAX_NR_ZONES for the whole LRU list) */ -unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru, int zone_idx) +static unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru, + int zone_idx) { unsigned long size = 0; int zid; From patchwork Wed Feb 24 20:08:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102703 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D43AC433E0 for ; Wed, 24 Feb 2021 20:08:51 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0ED3864EC3 for ; Wed, 24 Feb 2021 20:08:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0ED3864EC3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8F43D8D0048; Wed, 24 Feb 2021 15:08:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8CB6D8D001E; Wed, 24 Feb 2021 15:08:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E1658D0048; Wed, 24 Feb 2021 15:08:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0223.hostedemail.com [216.40.44.223]) by kanga.kvack.org (Postfix) with ESMTP id 64E888D001E for ; Wed, 24 Feb 2021 15:08:50 -0500 (EST) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 35099180ACF0E for ; Wed, 24 Feb 2021 20:08:50 +0000 (UTC) X-FDA: 77854249620.01.559DD32 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf13.hostedemail.com (Postfix) with ESMTP id 4FFE4E000118 for ; Wed, 24 Feb 2021 20:08:47 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 5434A64F20; Wed, 24 Feb 2021 20:08:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197328; bh=787urpnHfsBgwCqx+7n1DXo2pCIOdY7aT9xYWCKKemQ=; h=Date:From:To:Subject:In-Reply-To:From; b=k3OOA/kGtXegfeV6+K0cAAaTqzkUK9YMGAoHfdagwXjJ/upZge71dn2m88bciVnLi qEQvDtOPsMQDVk677wh7zM+j8Gwe7YvwB9QII+iqzi/Hw1rD3wxMJsNmbgli8dxNNj CdLG57fMLdzZPWM0DBmAUDQSi8WcAjjlb62mTSpI= Date: Wed, 24 Feb 2021 12:08:47 -0800 From: Andrew Morton To: akpm@linux-foundation.org, hannes@cmpxchg.org, linux-mm@kvack.org, mm-commits@vger.kernel.org, osalvador@suse.de, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 145/173] mm: workingset: clarify eviction order and distance calculation Message-ID: <20210224200847.MgJboFbIP%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: q6fseboutqbyzhwt16hgjnft37cdh3t1 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 4FFE4E000118 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf13; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197327-489164 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Oscar Salvador Subject: mm: workingset: clarify eviction order and distance calculation The premise of the refault distance is that it can be seen as a deficit of the inactive list space, so that if the inactive list would have had (R - E) more slots, the page would not have been evicted but promoted to the active list instead. However, the way the code is ordered right now set us to be off by one, so the real number of slots would be (R - E) + 1. I stumbled upon this when trying to understand the code and it puzzled me that the comments did not match what the code did. This it not an issue at all since evictions and refaults tend to happen in a number large enough that being off-by-one does not have any impact - and since the compiler and CPUs are free to rearrange the execution sequence anyway. But as Johannes says, it is better to re-arrange the code in the proper order since otherwise would be misleading to somebody who is actively reading and trying to understand the logic of the code - like it happened to me. Link: https://lkml.kernel.org/r/20210201060651.3781-1-osalvador@suse.de Signed-off-by: Oscar Salvador Acked-by: Johannes Weiner Acked-by: Vlastimil Babka Signed-off-by: Andrew Morton --- mm/workingset.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/workingset.c~mm-workingset-clarify-eviction-order-and-distance-calculation +++ a/mm/workingset.c @@ -263,10 +263,10 @@ void *workingset_eviction(struct page *p VM_BUG_ON_PAGE(!PageLocked(page), page); lruvec = mem_cgroup_lruvec(target_memcg, pgdat); - workingset_age_nonresident(lruvec, thp_nr_pages(page)); /* XXX: target_memcg can be NULL, go through lruvec */ memcgid = mem_cgroup_id(lruvec_memcg(lruvec)); eviction = atomic_long_read(&lruvec->nonresident_age); + workingset_age_nonresident(lruvec, thp_nr_pages(page)); return pack_shadow(memcgid, pgdat, eviction, PageWorkingset(page)); } From patchwork Wed Feb 24 20:08:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102705 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 114C0C433E0 for ; Wed, 24 Feb 2021 20:08:57 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8554264EC3 for ; Wed, 24 Feb 2021 20:08:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8554264EC3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 153A38D0049; Wed, 24 Feb 2021 15:08:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 12BEA8D001E; Wed, 24 Feb 2021 15:08:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 019A58D0049; Wed, 24 Feb 2021 15:08:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0246.hostedemail.com [216.40.44.246]) by kanga.kvack.org (Postfix) with ESMTP id DE70F8D001E for ; Wed, 24 Feb 2021 15:08:55 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id AC310181355F9 for ; Wed, 24 Feb 2021 20:08:55 +0000 (UTC) X-FDA: 77854249830.05.37EE524 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf08.hostedemail.com (Postfix) with ESMTP id 8150980192EC for ; Wed, 24 Feb 2021 20:08:43 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6B86464E60; Wed, 24 Feb 2021 20:08:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197334; bh=iaNTttlUJYbOEEJfjHtUCVq9Rx8aoSHNXkMXtrtEYSA=; h=Date:From:To:Subject:In-Reply-To:From; b=U+9ieD+pxZPzwb4udYhJmAZTqbeGo0TXJPwCoqWVt06BhNbcHa+HMd3bpSGOA5e1T XkERglD5O1KuHkKLHNaoaFAp8WRzIPUQ+P8tjf/hFTOK1Tfnghp9quiQZRSioTgQ5g WwrNNuey6/ExgV2dJNxlwAEynfHBMmvgHPnoGZEk= Date: Wed, 24 Feb 2021 12:08:51 -0800 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linmiaohe@huawei.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, n-horiguchi@ah.jp.nec.com, osalvador@suse.de, songmuchun@bytedance.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 146/173] hugetlb: use page.private for hugetlb specific page flags Message-ID: <20210224200851.hPKZWjK2C%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 1eob5unggngtzxg8ebt18r74juppd8x6 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 8150980192EC Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf08; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197323-117085 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Kravetz Subject: hugetlb: use page.private for hugetlb specific page flags Patch series "create hugetlb flags to consolidate state", v3. While discussing a series of hugetlb fixes in [1], it became evident that the hugetlb specific page state information is stored in a somewhat haphazard manner. Code dealing with state information would be easier to read, understand and maintain if this information was stored in a consistent manner. This series uses page.private of the hugetlb head page for storing a set of hugetlb specific page flags. Routines are priovided for test, set and clear of the flags. [1] https://lore.kernel.org/r/20210106084739.63318-1-songmuchun@bytedance.com This patch (of 4): As hugetlbfs evolved, state information about hugetlb pages was added. One 'convenient' way of doing this was to use available fields in tail pages. Over time, it has become difficult to know the meaning or contents of fields simply by looking at a small bit of code. Sometimes, the naming is just confusing. For example: The PagePrivate flag indicates a huge page reservation was consumed and needs to be restored if an error is encountered and the page is freed before it is instantiated. The page.private field contains the pointer to a subpool if the page is associated with one. In an effort to make the code more readable, use page.private to contain hugetlb specific page flags. These flags will have test, set and clear functions similar to those used for 'normal' page flags. More importantly, an enum of flag values will be created with names that actually reflect their purpose. In this patch, - Create infrastructure for hugetlb specific page flag functions - Move subpool pointer to page[1].private to make way for flags Create routines with meaningful names to modify subpool field - Use new HPageRestoreReserve flag instead of PagePrivate Conversion of other state information will happen in subsequent patches. Link: https://lkml.kernel.org/r/20210122195231.324857-1-mike.kravetz@oracle.com Link: https://lkml.kernel.org/r/20210122195231.324857-2-mike.kravetz@oracle.com Signed-off-by: Mike Kravetz Reviewed-by: Oscar Salvador Acked-by: Michal Hocko Cc: Naoya Horiguchi Cc: Muchun Song Cc: David Hildenbrand Cc: Matthew Wilcox Cc: Miaohe Lin Signed-off-by: Andrew Morton --- fs/hugetlbfs/inode.c | 12 +----- include/linux/hugetlb.h | 68 ++++++++++++++++++++++++++++++++++++++ mm/hugetlb.c | 48 +++++++++++++------------- 3 files changed, 96 insertions(+), 32 deletions(-) --- a/fs/hugetlbfs/inode.c~hugetlb-use-pageprivate-for-hugetlb-specific-page-flags +++ a/fs/hugetlbfs/inode.c @@ -973,15 +973,9 @@ static int hugetlbfs_migrate_page(struct if (rc != MIGRATEPAGE_SUCCESS) return rc; - /* - * page_private is subpool pointer in hugetlb pages. Transfer to - * new page. PagePrivate is not associated with page_private for - * hugetlb pages and can not be set here as only page_huge_active - * pages can be migrated. - */ - if (page_private(page)) { - set_page_private(newpage, page_private(page)); - set_page_private(page, 0); + if (hugetlb_page_subpool(page)) { + hugetlb_set_page_subpool(newpage, hugetlb_page_subpool(page)); + hugetlb_set_page_subpool(page, NULL); } if (mode != MIGRATE_SYNC_NO_COPY) --- a/include/linux/hugetlb.h~hugetlb-use-pageprivate-for-hugetlb-specific-page-flags +++ a/include/linux/hugetlb.h @@ -472,6 +472,60 @@ unsigned long hugetlb_get_unmapped_area( unsigned long flags); #endif /* HAVE_ARCH_HUGETLB_UNMAPPED_AREA */ +/* + * huegtlb page specific state flags. These flags are located in page.private + * of the hugetlb head page. Functions created via the below macros should be + * used to manipulate these flags. + * + * HPG_restore_reserve - Set when a hugetlb page consumes a reservation at + * allocation time. Cleared when page is fully instantiated. Free + * routine checks flag to restore a reservation on error paths. + */ +enum hugetlb_page_flags { + HPG_restore_reserve = 0, + __NR_HPAGEFLAGS, +}; + +/* + * Macros to create test, set and clear function definitions for + * hugetlb specific page flags. + */ +#ifdef CONFIG_HUGETLB_PAGE +#define TESTHPAGEFLAG(uname, flname) \ +static inline int HPage##uname(struct page *page) \ + { return test_bit(HPG_##flname, &(page->private)); } + +#define SETHPAGEFLAG(uname, flname) \ +static inline void SetHPage##uname(struct page *page) \ + { set_bit(HPG_##flname, &(page->private)); } + +#define CLEARHPAGEFLAG(uname, flname) \ +static inline void ClearHPage##uname(struct page *page) \ + { clear_bit(HPG_##flname, &(page->private)); } +#else +#define TESTHPAGEFLAG(uname, flname) \ +static inline int HPage##uname(struct page *page) \ + { return 0; } + +#define SETHPAGEFLAG(uname, flname) \ +static inline void SetHPage##uname(struct page *page) \ + { } + +#define CLEARHPAGEFLAG(uname, flname) \ +static inline void ClearHPage##uname(struct page *page) \ + { } +#endif + +#define HPAGEFLAG(uname, flname) \ + TESTHPAGEFLAG(uname, flname) \ + SETHPAGEFLAG(uname, flname) \ + CLEARHPAGEFLAG(uname, flname) \ + +/* + * Create functions associated with hugetlb page flags + */ +HPAGEFLAG(RestoreReserve, restore_reserve) + #ifdef CONFIG_HUGETLB_PAGE #define HSTATE_NAME_LEN 32 @@ -531,6 +585,20 @@ extern unsigned int default_hstate_idx; #define default_hstate (hstates[default_hstate_idx]) +/* + * hugetlb page subpool pointer located in hpage[1].private + */ +static inline struct hugepage_subpool *hugetlb_page_subpool(struct page *hpage) +{ + return (struct hugepage_subpool *)(hpage+1)->private; +} + +static inline void hugetlb_set_page_subpool(struct page *hpage, + struct hugepage_subpool *subpool) +{ + set_page_private(hpage+1, (unsigned long)subpool); +} + static inline struct hstate *hstate_file(struct file *f) { return hstate_inode(file_inode(f)); --- a/mm/hugetlb.c~hugetlb-use-pageprivate-for-hugetlb-specific-page-flags +++ a/mm/hugetlb.c @@ -1143,7 +1143,7 @@ static struct page *dequeue_huge_page_vm nid = huge_node(vma, address, gfp_mask, &mpol, &nodemask); page = dequeue_huge_page_nodemask(h, gfp_mask, nid, nodemask); if (page && !avoid_reserve && vma_has_reserves(vma, chg)) { - SetPagePrivate(page); + SetHPageRestoreReserve(page); h->resv_huge_pages--; } @@ -1418,20 +1418,19 @@ static void __free_huge_page(struct page */ struct hstate *h = page_hstate(page); int nid = page_to_nid(page); - struct hugepage_subpool *spool = - (struct hugepage_subpool *)page_private(page); + struct hugepage_subpool *spool = hugetlb_page_subpool(page); bool restore_reserve; VM_BUG_ON_PAGE(page_count(page), page); VM_BUG_ON_PAGE(page_mapcount(page), page); - set_page_private(page, 0); + hugetlb_set_page_subpool(page, NULL); page->mapping = NULL; - restore_reserve = PagePrivate(page); - ClearPagePrivate(page); + restore_reserve = HPageRestoreReserve(page); + ClearHPageRestoreReserve(page); /* - * If PagePrivate() was set on page, page allocation consumed a + * If HPageRestoreReserve was set on page, page allocation consumed a * reservation. If the page was associated with a subpool, there * would have been a page reserved in the subpool before allocation * via hugepage_subpool_get_pages(). Since we are 'restoring' the @@ -2263,24 +2262,24 @@ static long vma_add_reservation(struct h * This routine is called to restore a reservation on error paths. In the * specific error paths, a huge page was allocated (via alloc_huge_page) * and is about to be freed. If a reservation for the page existed, - * alloc_huge_page would have consumed the reservation and set PagePrivate - * in the newly allocated page. When the page is freed via free_huge_page, - * the global reservation count will be incremented if PagePrivate is set. - * However, free_huge_page can not adjust the reserve map. Adjust the - * reserve map here to be consistent with global reserve count adjustments - * to be made by free_huge_page. + * alloc_huge_page would have consumed the reservation and set + * HPageRestoreReserve in the newly allocated page. When the page is freed + * via free_huge_page, the global reservation count will be incremented if + * HPageRestoreReserve is set. However, free_huge_page can not adjust the + * reserve map. Adjust the reserve map here to be consistent with global + * reserve count adjustments to be made by free_huge_page. */ static void restore_reserve_on_error(struct hstate *h, struct vm_area_struct *vma, unsigned long address, struct page *page) { - if (unlikely(PagePrivate(page))) { + if (unlikely(HPageRestoreReserve(page))) { long rc = vma_needs_reservation(h, vma, address); if (unlikely(rc < 0)) { /* * Rare out of memory condition in reserve map - * manipulation. Clear PagePrivate so that + * manipulation. Clear HPageRestoreReserve so that * global reserve count will not be incremented * by free_huge_page. This will make it appear * as though the reservation for this page was @@ -2289,7 +2288,7 @@ static void restore_reserve_on_error(str * is better than inconsistent global huge page * accounting of reserve counts. */ - ClearPagePrivate(page); + ClearHPageRestoreReserve(page); } else if (rc) { rc = vma_add_reservation(h, vma, address); if (unlikely(rc < 0)) @@ -2297,7 +2296,7 @@ static void restore_reserve_on_error(str * See above comment about rare out of * memory condition. */ - ClearPagePrivate(page); + ClearHPageRestoreReserve(page); } else vma_end_reservation(h, vma, address); } @@ -2378,7 +2377,7 @@ struct page *alloc_huge_page(struct vm_a if (!page) goto out_uncharge_cgroup; if (!avoid_reserve && vma_has_reserves(vma, gbl_chg)) { - SetPagePrivate(page); + SetHPageRestoreReserve(page); h->resv_huge_pages--; } spin_lock(&hugetlb_lock); @@ -2396,7 +2395,7 @@ struct page *alloc_huge_page(struct vm_a spin_unlock(&hugetlb_lock); - set_page_private(page, (unsigned long)spool); + hugetlb_set_page_subpool(page, spool); map_commit = vma_commit_reservation(h, vma, addr); if (unlikely(map_chg > map_commit)) { @@ -3170,6 +3169,9 @@ static int __init hugetlb_init(void) { int i; + BUILD_BUG_ON(sizeof_field(struct page, private) * BITS_PER_BYTE < + __NR_HPAGEFLAGS); + if (!hugepages_supported()) { if (hugetlb_max_hstate || default_hstate_max_huge_pages) pr_warn("HugeTLB: huge pages not supported, ignoring associated command-line parameters\n"); @@ -4207,7 +4209,7 @@ retry_avoidcopy: spin_lock(ptl); ptep = huge_pte_offset(mm, haddr, huge_page_size(h)); if (likely(ptep && pte_same(huge_ptep_get(ptep), pte))) { - ClearPagePrivate(new_page); + ClearHPageRestoreReserve(new_page); /* Break COW */ huge_ptep_clear_flush(vma, haddr, ptep); @@ -4274,7 +4276,7 @@ int huge_add_to_page_cache(struct page * if (err) return err; - ClearPagePrivate(page); + ClearHPageRestoreReserve(page); /* * set page dirty so that it will not be removed from cache/file @@ -4436,7 +4438,7 @@ retry: goto backout; if (anon_rmap) { - ClearPagePrivate(page); + ClearHPageRestoreReserve(page); hugepage_add_new_anon_rmap(page, vma, haddr); } else page_dup_rmap(page, true); @@ -4750,7 +4752,7 @@ int hugetlb_mcopy_atomic_pte(struct mm_s if (vm_shared) { page_dup_rmap(page, true); } else { - ClearPagePrivate(page); + ClearHPageRestoreReserve(page); hugepage_add_new_anon_rmap(page, dst_vma, dst_addr); } From patchwork Wed Feb 24 20:08:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102707 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05BD3C433DB for ; Wed, 24 Feb 2021 20:09:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7799E64E24 for ; Wed, 24 Feb 2021 20:09:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7799E64E24 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 091368D004A; Wed, 24 Feb 2021 15:09:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 067C38D001E; Wed, 24 Feb 2021 15:09:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EBEAD8D004A; Wed, 24 Feb 2021 15:08:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0172.hostedemail.com [216.40.44.172]) by kanga.kvack.org (Postfix) with ESMTP id D3BC18D001E for ; Wed, 24 Feb 2021 15:08:59 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 9B4F0181355F9 for ; Wed, 24 Feb 2021 20:08:59 +0000 (UTC) X-FDA: 77854249998.09.3875103 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf05.hostedemail.com (Postfix) with ESMTP id DBEC0E000117 for ; Wed, 24 Feb 2021 20:08:57 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6115364F1B; Wed, 24 Feb 2021 20:08:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197338; bh=X10FdWo7ZBgG9/LpPj+4BAPYv7HNjkhcf1mK8lNnWbE=; h=Date:From:To:Subject:In-Reply-To:From; b=Eapsrm5e5qvyoihuF1CQP3fZu/bQucj0h0k1Wu14goi/kTeDWMKtJoOv2TmuLciXf CXyJDd4EJj8EToL+Rh9sY/rkGvk0mXyLNtRBmoHOQgxOajlnQR2fynSC22ototuj/s DqB99WoVc4CDY0C7RQd5IzoN3XM436pwuBcbyC0s= Date: Wed, 24 Feb 2021 12:08:56 -0800 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linmiaohe@huawei.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, n-horiguchi@ah.jp.nec.com, osalvador@suse.de, songmuchun@bytedance.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 147/173] hugetlb: convert page_huge_active() HPageMigratable flag Message-ID: <20210224200856.v70i--a0K%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 3ccwo8xgnnouahf7xzbnnd7bqhjggg6e X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: DBEC0E000117 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf05; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197337-47809 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Kravetz Subject: hugetlb: convert page_huge_active() HPageMigratable flag Use the new hugetlb page specific flag HPageMigratable to replace the page_huge_active interfaces. By it's name, page_huge_active implied that a huge page was on the active list. However, that is not really what code checking the flag wanted to know. It really wanted to determine if the huge page could be migrated. This happens when the page is actually added to the page cache and/or task page table. This is the reasoning behind the name change. The VM_BUG_ON_PAGE() calls in the *_huge_active() interfaces are not really necessary as we KNOW the page is a hugetlb page. Therefore, they are removed. The routine page_huge_active checked for PageHeadHuge before testing the active bit. This is unnecessary in the case where we hold a reference or lock and know it is a hugetlb head page. page_huge_active is also called without holding a reference or lock (scan_movable_pages), and can race with code freeing the page. The extra check in page_huge_active shortened the race window, but did not prevent the race. Offline code calling scan_movable_pages already deals with these races, so removing the check is acceptable. Add comment to racy code. [songmuchun@bytedance.com: remove set_page_huge_active() declaration from include/linux/hugetlb.h] Link: https://lkml.kernel.org/r/CAMZfGtUda+KoAZscU0718TN61cSFwp4zy=y2oZ=+6Z2TAZZwng@mail.gmail.com Link: https://lkml.kernel.org/r/20210122195231.324857-3-mike.kravetz@oracle.com Signed-off-by: Mike Kravetz Reviewed-by: Oscar Salvador Reviewed-by: Muchun Song Reviewed-by: Miaohe Lin Acked-by: Michal Hocko Cc: David Hildenbrand Cc: Matthew Wilcox Cc: Naoya Horiguchi Signed-off-by: Andrew Morton --- fs/hugetlbfs/inode.c | 2 - include/linux/hugetlb.h | 7 +++-- include/linux/page-flags.h | 6 ---- mm/hugetlb.c | 45 ++++++++--------------------------- mm/memory_hotplug.c | 9 ++++++- 5 files changed, 25 insertions(+), 44 deletions(-) --- a/fs/hugetlbfs/inode.c~hugetlb-convert-page_huge_active-hpagemigratable-flag +++ a/fs/hugetlbfs/inode.c @@ -735,7 +735,7 @@ static long hugetlbfs_fallocate(struct f mutex_unlock(&hugetlb_fault_mutex_table[hash]); - set_page_huge_active(page); + SetHPageMigratable(page); /* * unlock_page because locked by add_to_page_cache() * put_page() due to reference from alloc_huge_page() --- a/include/linux/hugetlb.h~hugetlb-convert-page_huge_active-hpagemigratable-flag +++ a/include/linux/hugetlb.h @@ -480,9 +480,13 @@ unsigned long hugetlb_get_unmapped_area( * HPG_restore_reserve - Set when a hugetlb page consumes a reservation at * allocation time. Cleared when page is fully instantiated. Free * routine checks flag to restore a reservation on error paths. + * HPG_migratable - Set after a newly allocated page is added to the page + * cache and/or page tables. Indicates the page is a candidate for + * migration. */ enum hugetlb_page_flags { HPG_restore_reserve = 0, + HPG_migratable, __NR_HPAGEFLAGS, }; @@ -525,6 +529,7 @@ static inline void ClearHPage##uname(str * Create functions associated with hugetlb page flags */ HPAGEFLAG(RestoreReserve, restore_reserve) +HPAGEFLAG(Migratable, migratable) #ifdef CONFIG_HUGETLB_PAGE @@ -838,8 +843,6 @@ static inline void huge_ptep_modify_prot } #endif -void set_page_huge_active(struct page *page); - #else /* CONFIG_HUGETLB_PAGE */ struct hstate {}; --- a/include/linux/page-flags.h~hugetlb-convert-page_huge_active-hpagemigratable-flag +++ a/include/linux/page-flags.h @@ -592,15 +592,9 @@ static inline void ClearPageCompound(str #ifdef CONFIG_HUGETLB_PAGE int PageHuge(struct page *page); int PageHeadHuge(struct page *page); -bool page_huge_active(struct page *page); #else TESTPAGEFLAG_FALSE(Huge) TESTPAGEFLAG_FALSE(HeadHuge) - -static inline bool page_huge_active(struct page *page) -{ - return 0; -} #endif --- a/mm/hugetlb.c~hugetlb-convert-page_huge_active-hpagemigratable-flag +++ a/mm/hugetlb.c @@ -1365,30 +1365,6 @@ struct hstate *size_to_hstate(unsigned l } /* - * Test to determine whether the hugepage is "active/in-use" (i.e. being linked - * to hstate->hugepage_activelist.) - * - * This function can be called for tail pages, but never returns true for them. - */ -bool page_huge_active(struct page *page) -{ - return PageHeadHuge(page) && PagePrivate(&page[1]); -} - -/* never called for tail page */ -void set_page_huge_active(struct page *page) -{ - VM_BUG_ON_PAGE(!PageHeadHuge(page), page); - SetPagePrivate(&page[1]); -} - -static void clear_page_huge_active(struct page *page) -{ - VM_BUG_ON_PAGE(!PageHeadHuge(page), page); - ClearPagePrivate(&page[1]); -} - -/* * Internal hugetlb specific page flag. Do not use outside of the hugetlb * code */ @@ -1449,7 +1425,7 @@ static void __free_huge_page(struct page } spin_lock(&hugetlb_lock); - clear_page_huge_active(page); + ClearHPageMigratable(page); hugetlb_cgroup_uncharge_page(hstate_index(h), pages_per_huge_page(h), page); hugetlb_cgroup_uncharge_page_rsvd(hstate_index(h), @@ -4218,7 +4194,7 @@ retry_avoidcopy: make_huge_pte(vma, new_page, 1)); page_remove_rmap(old_page, true); hugepage_add_new_anon_rmap(new_page, vma, haddr); - set_page_huge_active(new_page); + SetHPageMigratable(new_page); /* Make the old page be freed below */ new_page = old_page; } @@ -4455,12 +4431,12 @@ retry: spin_unlock(ptl); /* - * Only make newly allocated pages active. Existing pages found - * in the pagecache could be !page_huge_active() if they have been - * isolated for migration. + * Only set HPageMigratable in newly allocated pages. Existing pages + * found in the pagecache may not have HPageMigratableset if they have + * been isolated for migration. */ if (new_page) - set_page_huge_active(page); + SetHPageMigratable(page); unlock_page(page); out: @@ -4771,7 +4747,7 @@ int hugetlb_mcopy_atomic_pte(struct mm_s update_mmu_cache(dst_vma, dst_addr, dst_pte); spin_unlock(ptl); - set_page_huge_active(page); + SetHPageMigratable(page); if (vm_shared) unlock_page(page); ret = 0; @@ -5610,12 +5586,13 @@ bool isolate_huge_page(struct page *page bool ret = true; spin_lock(&hugetlb_lock); - if (!PageHeadHuge(page) || !page_huge_active(page) || + if (!PageHeadHuge(page) || + !HPageMigratable(page) || !get_page_unless_zero(page)) { ret = false; goto unlock; } - clear_page_huge_active(page); + ClearHPageMigratable(page); list_move_tail(&page->lru, list); unlock: spin_unlock(&hugetlb_lock); @@ -5625,7 +5602,7 @@ unlock: void putback_active_hugepage(struct page *page) { spin_lock(&hugetlb_lock); - set_page_huge_active(page); + SetHPageMigratable(page); list_move_tail(&page->lru, &(page_hstate(page))->hugepage_activelist); spin_unlock(&hugetlb_lock); put_page(page); --- a/mm/memory_hotplug.c~hugetlb-convert-page_huge_active-hpagemigratable-flag +++ a/mm/memory_hotplug.c @@ -1260,7 +1260,14 @@ static int scan_movable_pages(unsigned l if (!PageHuge(page)) continue; head = compound_head(page); - if (page_huge_active(head)) + /* + * This test is racy as we hold no reference or lock. The + * hugetlb page could have been free'ed and head is no longer + * a hugetlb page before the following check. In such unlikely + * cases false positives and negatives are possible. Calling + * code must deal with these scenarios. + */ + if (HPageMigratable(head)) goto found; skip = compound_nr(head) - (page - head); pfn += skip - 1; From patchwork Wed Feb 24 20:09:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102709 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CD84C433E0 for ; Wed, 24 Feb 2021 20:09:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DABC064F1F for ; Wed, 24 Feb 2021 20:09:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DABC064F1F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5F1CB8D004B; Wed, 24 Feb 2021 15:09:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 596458D001E; Wed, 24 Feb 2021 15:09:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4AC9A8D004B; Wed, 24 Feb 2021 15:09:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0206.hostedemail.com [216.40.44.206]) by kanga.kvack.org (Postfix) with ESMTP id 346C68D001E for ; Wed, 24 Feb 2021 15:09:05 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 060D0B9EC for ; Wed, 24 Feb 2021 20:09:05 +0000 (UTC) X-FDA: 77854250250.09.EB408E7 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP id BC5242000388 for ; Wed, 24 Feb 2021 20:09:05 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0C3A464F25; Wed, 24 Feb 2021 20:09:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197342; bh=df+0qV+j7KcQUdsOCl5oTF2mMefbNZbXZqcH02cX53o=; h=Date:From:To:Subject:In-Reply-To:From; b=LVa/atVTsOfFbGudUgrKD2pftzm3OWSReTX0eym68t5EuPPpCeLsQ+NzyE9lnRqke HvOTc3by5R4gFmMqhFurO8IApOT3pQLh4v9J/QYxY7SfN4nuFhIDonajGGmRSPByR5 7GepXAZIBFqWDcm1lQ5hKGnTZSafhdHQ3wqXofBM= Date: Wed, 24 Feb 2021 12:09:00 -0800 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linmiaohe@huawei.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, n-horiguchi@ah.jp.nec.com, osalvador@suse.de, songmuchun@bytedance.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 148/173] hugetlb: convert PageHugeTemporary() to HPageTemporary flag Message-ID: <20210224200900.91raiWk2W%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: BC5242000388 X-Stat-Signature: 9mtkkhicyp9zqpx1ifr8zgztrksma7g5 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197345-812724 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Kravetz Subject: hugetlb: convert PageHugeTemporary() to HPageTemporary flag Use new hugetlb specific HPageTemporary flag to replace the PageHugeTemporary() interfaces. PageHugeTemporary does contain a PageHuge() check. However, this interface is only used within hugetlb code where we know we are dealing with a hugetlb page. Therefore, the check can be eliminated. Link: https://lkml.kernel.org/r/20210122195231.324857-5-mike.kravetz@oracle.com Signed-off-by: Mike Kravetz Reviewed-by: Oscar Salvador Reviewed-by: Muchun Song Acked-by: Michal Hocko Cc: David Hildenbrand Cc: Matthew Wilcox Cc: Miaohe Lin Cc: Naoya Horiguchi Signed-off-by: Andrew Morton --- include/linux/hugetlb.h | 6 ++++++ mm/hugetlb.c | 36 +++++++----------------------------- 2 files changed, 13 insertions(+), 29 deletions(-) --- a/include/linux/hugetlb.h~hugetlb-convert-pagehugetemporary-to-hpagetemporary-flag +++ a/include/linux/hugetlb.h @@ -483,10 +483,15 @@ unsigned long hugetlb_get_unmapped_area( * HPG_migratable - Set after a newly allocated page is added to the page * cache and/or page tables. Indicates the page is a candidate for * migration. + * HPG_temporary - - Set on a page that is temporarily allocated from the buddy + * allocator. Typically used for migration target pages when no pages + * are available in the pool. The hugetlb free page path will + * immediately free pages with this flag set to the buddy allocator. */ enum hugetlb_page_flags { HPG_restore_reserve = 0, HPG_migratable, + HPG_temporary, __NR_HPAGEFLAGS, }; @@ -530,6 +535,7 @@ static inline void ClearHPage##uname(str */ HPAGEFLAG(RestoreReserve, restore_reserve) HPAGEFLAG(Migratable, migratable) +HPAGEFLAG(Temporary, temporary) #ifdef CONFIG_HUGETLB_PAGE --- a/mm/hugetlb.c~hugetlb-convert-pagehugetemporary-to-hpagetemporary-flag +++ a/mm/hugetlb.c @@ -1364,28 +1364,6 @@ struct hstate *size_to_hstate(unsigned l return NULL; } -/* - * Internal hugetlb specific page flag. Do not use outside of the hugetlb - * code - */ -static inline bool PageHugeTemporary(struct page *page) -{ - if (!PageHuge(page)) - return false; - - return (unsigned long)page[2].mapping == -1U; -} - -static inline void SetPageHugeTemporary(struct page *page) -{ - page[2].mapping = (void *)-1U; -} - -static inline void ClearPageHugeTemporary(struct page *page) -{ - page[2].mapping = NULL; -} - static void __free_huge_page(struct page *page) { /* @@ -1433,9 +1411,9 @@ static void __free_huge_page(struct page if (restore_reserve) h->resv_huge_pages++; - if (PageHugeTemporary(page)) { + if (HPageTemporary(page)) { list_del(&page->lru); - ClearPageHugeTemporary(page); + ClearHPageTemporary(page); update_and_free_page(h, page); } else if (h->surplus_huge_pages_node[nid]) { /* remove the page from active list */ @@ -1869,7 +1847,7 @@ static struct page *alloc_surplus_huge_p * codeflow */ if (h->surplus_huge_pages >= h->nr_overcommit_huge_pages) { - SetPageHugeTemporary(page); + SetHPageTemporary(page); spin_unlock(&hugetlb_lock); put_page(page); return NULL; @@ -1900,7 +1878,7 @@ static struct page *alloc_migrate_huge_p * We do not account these pages as surplus because they are only * temporary and will be released properly on the last reference */ - SetPageHugeTemporary(page); + SetHPageTemporary(page); return page; } @@ -5625,12 +5603,12 @@ void move_hugetlb_state(struct page *old * here as well otherwise the global surplus count will not match * the per-node's. */ - if (PageHugeTemporary(newpage)) { + if (HPageTemporary(newpage)) { int old_nid = page_to_nid(oldpage); int new_nid = page_to_nid(newpage); - SetPageHugeTemporary(oldpage); - ClearPageHugeTemporary(newpage); + SetHPageTemporary(oldpage); + ClearHPageTemporary(newpage); spin_lock(&hugetlb_lock); if (h->surplus_huge_pages_node[old_nid]) { From patchwork Wed Feb 24 20:09:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102711 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 539ADC433E6 for ; Wed, 24 Feb 2021 20:09:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E842F64EC3 for ; Wed, 24 Feb 2021 20:09:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E842F64EC3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7C1248D004C; Wed, 24 Feb 2021 15:09:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 76B8E8D001E; Wed, 24 Feb 2021 15:09:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 682358D004C; Wed, 24 Feb 2021 15:09:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0097.hostedemail.com [216.40.44.97]) by kanga.kvack.org (Postfix) with ESMTP id 4BD4A8D001E for ; Wed, 24 Feb 2021 15:09:07 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 1309C1848A1F4 for ; Wed, 24 Feb 2021 20:09:07 +0000 (UTC) X-FDA: 77854250334.19.5AB435B Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf08.hostedemail.com (Postfix) with ESMTP id 1953180192E4 for ; Wed, 24 Feb 2021 20:08:54 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 0412964E24; Wed, 24 Feb 2021 20:09:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197345; bh=QC0AIZ6aUxbhZZgviX5mx6Iepg59j3OqAEvMZ48Jxcs=; h=Date:From:To:Subject:In-Reply-To:From; b=NMYp21prKFxsZPE19g44nbOMSNdYhaxiJAVU+bEtNwp0DCROfZWWw60fFpJVk/s90 rroG9i5Vc0mpMzu1YT8oxiObU8tlZGMebINdtlOrytwvBPBzn52DW0wIAPWCzmZcak qlBrxMa9BI6WGlZTPA6ErlodsB+XCTE11K5SEPko= Date: Wed, 24 Feb 2021 12:09:04 -0800 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linmiaohe@huawei.com, linux-mm@kvack.org, mhocko@kernel.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, n-horiguchi@ah.jp.nec.com, osalvador@suse.de, songmuchun@bytedance.com, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 149/173] hugetlb: convert PageHugeFreed to HPageFreed flag Message-ID: <20210224200904.e6JwA1acj%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: hn4wcqypj7yg5196ama6beidnine7i59 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 1953180192E4 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf08; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197334-516132 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Kravetz Subject: hugetlb: convert PageHugeFreed to HPageFreed flag Use new hugetlb specific HPageFreed flag to replace the PageHugeFreed interfaces. Link: https://lkml.kernel.org/r/20210122195231.324857-6-mike.kravetz@oracle.com Signed-off-by: Mike Kravetz Reviewed-by: Oscar Salvador Reviewed-by: Muchun Song Acked-by: Michal Hocko Cc: David Hildenbrand Cc: Matthew Wilcox Cc: Miaohe Lin Cc: Michal Hocko Cc: Naoya Horiguchi Signed-off-by: Andrew Morton --- include/linux/hugetlb.h | 3 +++ mm/hugetlb.c | 23 ++++------------------- 2 files changed, 7 insertions(+), 19 deletions(-) --- a/include/linux/hugetlb.h~hugetlb-convert-pagehugefreed-to-hpagefreed-flag +++ a/include/linux/hugetlb.h @@ -487,11 +487,13 @@ unsigned long hugetlb_get_unmapped_area( * allocator. Typically used for migration target pages when no pages * are available in the pool. The hugetlb free page path will * immediately free pages with this flag set to the buddy allocator. + * HPG_freed - Set when page is on the free lists. */ enum hugetlb_page_flags { HPG_restore_reserve = 0, HPG_migratable, HPG_temporary, + HPG_freed, __NR_HPAGEFLAGS, }; @@ -536,6 +538,7 @@ static inline void ClearHPage##uname(str HPAGEFLAG(RestoreReserve, restore_reserve) HPAGEFLAG(Migratable, migratable) HPAGEFLAG(Temporary, temporary) +HPAGEFLAG(Freed, freed) #ifdef CONFIG_HUGETLB_PAGE --- a/mm/hugetlb.c~hugetlb-convert-pagehugefreed-to-hpagefreed-flag +++ a/mm/hugetlb.c @@ -79,21 +79,6 @@ DEFINE_SPINLOCK(hugetlb_lock); static int num_fault_mutexes; struct mutex *hugetlb_fault_mutex_table ____cacheline_aligned_in_smp; -static inline bool PageHugeFreed(struct page *head) -{ - return page_private(head + 4) == -1UL; -} - -static inline void SetPageHugeFreed(struct page *head) -{ - set_page_private(head + 4, -1UL); -} - -static inline void ClearPageHugeFreed(struct page *head) -{ - set_page_private(head + 4, 0); -} - /* Forward declaration */ static int hugetlb_acct_memory(struct hstate *h, long delta); @@ -1053,7 +1038,7 @@ static void enqueue_huge_page(struct hst list_move(&page->lru, &h->hugepage_freelists[nid]); h->free_huge_pages++; h->free_huge_pages_node[nid]++; - SetPageHugeFreed(page); + SetHPageFreed(page); } static struct page *dequeue_huge_page_node_exact(struct hstate *h, int nid) @@ -1070,7 +1055,7 @@ static struct page *dequeue_huge_page_no list_move(&page->lru, &h->hugepage_activelist); set_page_refcounted(page); - ClearPageHugeFreed(page); + ClearHPageFreed(page); h->free_huge_pages--; h->free_huge_pages_node[nid]--; return page; @@ -1485,7 +1470,7 @@ static void prep_new_huge_page(struct hs spin_lock(&hugetlb_lock); h->nr_huge_pages++; h->nr_huge_pages_node[nid]++; - ClearPageHugeFreed(page); + ClearHPageFreed(page); spin_unlock(&hugetlb_lock); } @@ -1756,7 +1741,7 @@ retry: * We should make sure that the page is already on the free list * when it is dissolved. */ - if (unlikely(!PageHugeFreed(head))) { + if (unlikely(!HPageFreed(head))) { spin_unlock(&hugetlb_lock); cond_resched(); From patchwork Wed Feb 24 20:09:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102713 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6989C433E0 for ; Wed, 24 Feb 2021 20:09:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 453C964F23 for ; Wed, 24 Feb 2021 20:09:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 453C964F23 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C9CAE8D004D; Wed, 24 Feb 2021 15:09:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C4C8A8D001E; Wed, 24 Feb 2021 15:09:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B62F28D004D; Wed, 24 Feb 2021 15:09:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0175.hostedemail.com [216.40.44.175]) by kanga.kvack.org (Postfix) with ESMTP id 9DD648D001E for ; Wed, 24 Feb 2021 15:09:10 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 62046F045 for ; Wed, 24 Feb 2021 20:09:10 +0000 (UTC) X-FDA: 77854250460.29.126A1B6 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP id CCDCAF3 for ; Wed, 24 Feb 2021 20:09:09 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 9430364EC3; Wed, 24 Feb 2021 20:09:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197348; bh=lFbVt0vxQbWNFTZr7by1YJiXlS0bmT0HIfAulncMONc=; h=Date:From:To:Subject:In-Reply-To:From; b=aGXKVx2Wg9YzDKxelFFWD46mA357KjNkd3juLoGxhhEqalEYruJ+2yEz/52154tsz EyVIEW3Wp1o6alN8tKLgsdXmZ9s5t4HSyjAzJVF+j2y5Ct5AaVstTg41U7MbBGQj2m jpuwO20J7RhMAkp4pJ0SJ9SCrGVe5eFIJ0w3R81U= Date: Wed, 24 Feb 2021 12:09:08 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 150/173] include/linux/hugetlb.h: add synchronization information for new hugetlb specific flags Message-ID: <20210224200908.vZa9u-els%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: zpet1ihcac8y4qdd9k4gcqnrcx99ux7p X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: CCDCAF3 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf20; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197349-856023 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Kravetz Subject: include/linux/hugetlb.h: add synchronization information for new hugetlb specific flags Add comments, no functional change. Link: https://lkml.kernel.org/r/62a80585-2a73-10cc-4a2d-5721540d4ad2@oracle.com Signed-off-by: Mike Kravetz Acked-by: Michal Hocko Signed-off-by: Andrew Morton --- include/linux/hugetlb.h | 10 ++++++++++ 1 file changed, 10 insertions(+) --- a/include/linux/hugetlb.h~include-linux-hugetlbh-add-synchronization-information-for-new-hugetlb-specific-flags +++ a/include/linux/hugetlb.h @@ -480,14 +480,24 @@ unsigned long hugetlb_get_unmapped_area( * HPG_restore_reserve - Set when a hugetlb page consumes a reservation at * allocation time. Cleared when page is fully instantiated. Free * routine checks flag to restore a reservation on error paths. + * Synchronization: Examined or modified by code that knows it has + * the only reference to page. i.e. After allocation but before use + * or when the page is being freed. * HPG_migratable - Set after a newly allocated page is added to the page * cache and/or page tables. Indicates the page is a candidate for * migration. + * Synchronization: Initially set after new page allocation with no + * locking. When examined and modified during migration processing + * (isolate, migrate, putback) the hugetlb_lock is held. * HPG_temporary - - Set on a page that is temporarily allocated from the buddy * allocator. Typically used for migration target pages when no pages * are available in the pool. The hugetlb free page path will * immediately free pages with this flag set to the buddy allocator. + * Synchronization: Can be set after huge page allocation from buddy when + * code knows it has only reference. All other examinations and + * modifications require hugetlb_lock. * HPG_freed - Set when page is on the free lists. + * Synchronization: hugetlb_lock held for examination and modification. */ enum hugetlb_page_flags { HPG_restore_reserve = 0, From patchwork Wed Feb 24 20:09:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102715 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 550FBC433E0 for ; Wed, 24 Feb 2021 20:09:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id CDD8464F26 for ; Wed, 24 Feb 2021 20:09:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CDD8464F26 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 60D238D004E; Wed, 24 Feb 2021 15:09:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5BCA28D001E; Wed, 24 Feb 2021 15:09:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4F9738D004E; Wed, 24 Feb 2021 15:09:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0198.hostedemail.com [216.40.44.198]) by kanga.kvack.org (Postfix) with ESMTP id 34BC98D001E for ; Wed, 24 Feb 2021 15:09:14 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 0116282E86D0 for ; Wed, 24 Feb 2021 20:09:14 +0000 (UTC) X-FDA: 77854250628.28.B5B0C1E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP id 84B47ED for ; Wed, 24 Feb 2021 20:09:13 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 19B6F64F1F; Wed, 24 Feb 2021 20:09:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197352; bh=gB3hb8WuHHV/SNs+QFMV+iCPOAETqI6CRyNbgfNVZZA=; h=Date:From:To:Subject:In-Reply-To:From; b=ioQmKqMGtd7p7x2I/paqYKYpsLdZJnsm0pyysVvaYWNv0b3oCnvKmgVDwxDiAp6eH JwFf8eOn5nJ+pGEik/M0Ts9pdH0RuywNkmZebNGtkW0c/ZVmn4uJv1vtikChJRYZuZ 0kj9yIYHUUbOcJFEDNnqGUmh4ta5fN5hMhMPkCMA= Date: Wed, 24 Feb 2021 12:09:11 -0800 From: Andrew Morton To: akpm@linux-foundation.org, gerald.schaefer@linux.ibm.com, hca@linux.ibm.com, linux-mm@kvack.org, mhocko@suse.com, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, osalvador@suse.de, songmuchun@bytedance.com, svens@linux.ibm.com, torvalds@linux-foundation.org Subject: [patch 151/173] hugetlb: fix uninitialized subpool pointer Message-ID: <20210224200911.aP4lDAAYo%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 84B47ED X-Stat-Signature: q4ooo7fzptsp17nmmqxhhasfu7ydsagq Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf29; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197353-31470 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Kravetz Subject: hugetlb: fix uninitialized subpool pointer Gerald Schaefer reported a panic on s390 in hugepage_subpool_put_pages() with linux-next 5.12.0-20210222. Call trace: hugepage_subpool_put_pages.part.0+0x2c/0x138 __free_huge_page+0xce/0x310 alloc_pool_huge_page+0x102/0x120 set_max_huge_pages+0x13e/0x350 hugetlb_sysctl_handler_common+0xd8/0x110 hugetlb_sysctl_handler+0x48/0x58 proc_sys_call_handler+0x138/0x238 new_sync_write+0x10e/0x198 vfs_write.part.0+0x12c/0x238 ksys_write+0x68/0xf8 do_syscall+0x82/0xd0 __do_syscall+0xb4/0xc8 system_call+0x72/0x98 This is a result of the change which moved the hugetlb page subpool pointer from page->private to page[1]->private. When new pages are allocated from the buddy allocator, the private field of the head page will be cleared, but the private field of subpages is not modified. Therefore, old values may remain. Fix by initializing hugetlb page subpool pointer in prep_new_huge_page(). Link: https://lkml.kernel.org/r/20210223215544.313871-1-mike.kravetz@oracle.com Fixes: f1280272ae4d ("hugetlb: use page.private for hugetlb specific page flags") Signed-off-by: Mike Kravetz Reported-by: Gerald Schaefer Reviewed-by: Oscar Salvador Acked-by: Michal Hocko Cc: Gerald Schaefer Cc: Muchun Song Cc: Heiko Carstens Cc: Sven Schnelle Signed-off-by: Andrew Morton --- mm/hugetlb.c | 1 + 1 file changed, 1 insertion(+) --- a/mm/hugetlb.c~hugetlb-fix-uninitialized-subpool-pointer +++ a/mm/hugetlb.c @@ -1465,6 +1465,7 @@ static void prep_new_huge_page(struct hs { INIT_LIST_HEAD(&page->lru); set_compound_page_dtor(page, HUGETLB_PAGE_DTOR); + hugetlb_set_page_subpool(page, NULL); set_hugetlb_cgroup(page, NULL); set_hugetlb_cgroup_rsvd(page, NULL); spin_lock(&hugetlb_lock); From patchwork Wed Feb 24 20:09:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102717 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83CAEC433DB for ; Wed, 24 Feb 2021 20:09:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 1E5D764E7A for ; Wed, 24 Feb 2021 20:09:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1E5D764E7A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A79AB8D004F; Wed, 24 Feb 2021 15:09:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A293C8D001E; Wed, 24 Feb 2021 15:09:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9445F8D004F; Wed, 24 Feb 2021 15:09:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0238.hostedemail.com [216.40.44.238]) by kanga.kvack.org (Postfix) with ESMTP id 7819C8D001E for ; Wed, 24 Feb 2021 15:09:18 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 4D6021848A1F4 for ; Wed, 24 Feb 2021 20:09:18 +0000 (UTC) X-FDA: 77854250796.28.C0CD3A4 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf05.hostedemail.com (Postfix) with ESMTP id C775CE00010B for ; Wed, 24 Feb 2021 20:09:16 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id DDE8564F23; Wed, 24 Feb 2021 20:09:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197356; bh=6uZAIWo00IIlfca439cq6f0bs6LY/39RlcishU7qFJI=; h=Date:From:To:Subject:In-Reply-To:From; b=S9O5QbeZKjn956bTm527aEuDSv+5HUVYmBTRmRGq6PUGt892WCBjgHCNgiA/lFZtS 8hBD/I4ICCbHqovlrvcQAnEaHvIcNeA8BEQXD2xoTtmwkFl8Ls99ivazAKQOp06Qpt EQKJ39Uizh8yDk4GrrJIGU3Xsjfv+A+IptZe1PKU= Date: Wed, 24 Feb 2021 12:09:15 -0800 From: Andrew Morton To: akpm@linux-foundation.org, alex.shi@linux.alibaba.com, ben.widawsky@intel.com, cai@lca.pw, cl@linux.com, dan.j.williams@intel.com, dave.hansen@linux.intel.com, dwagner@suse.de, linux-mm@kvack.org, mm-commits@vger.kernel.org, osalvador@suse.de, rientjes@google.com, stable@vger.kernel.org, tobin@kernel.org, torvalds@linux-foundation.org, ying.huang@intel.com Subject: [patch 152/173] mm/vmscan: restore zone_reclaim_mode ABI Message-ID: <20210224200915.lASg4FukJ%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: C775CE00010B X-Stat-Signature: a11hgthwfijpbizbmhpuddoswjjk1fgh Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf05; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197356-653289 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Dave Hansen Subject: mm/vmscan: restore zone_reclaim_mode ABI I went to go add a new RECLAIM_* mode for the zone_reclaim_mode sysctl. Like a good kernel developer, I also went to go update the documentation. I noticed that the bits in the documentation didn't match the bits in the #defines. The VM never explicitly checks the RECLAIM_ZONE bit. The bit is, however implicitly checked when checking 'node_reclaim_mode==0'. The RECLAIM_ZONE #define was removed in a cleanup. That, by itself is fine. But, when the bit was removed (bit 0) the _other_ bit locations also got changed. That's not OK because the bit values are documented to mean one specific thing. Users surely do not expect the meaning to change from kernel to kernel. The end result is that if someone had a script that did: sysctl vm.zone_reclaim_mode=1 it would have gone from enabling node reclaim for clean unmapped pages to writing out pages during node reclaim after the commit in question. That's not great. Put the bits back the way they were and add a comment so something like this is a bit harder to do again. Update the documentation to make it clear that the first bit is ignored. Link: https://lkml.kernel.org/r/20210219172555.FF0CDF23@viggo.jf.intel.com Signed-off-by: Dave Hansen Fixes: 648b5cf368e0 ("mm/vmscan: remove unused RECLAIM_OFF/RECLAIM_ZONE") Reviewed-by: Ben Widawsky Reviewed-by: Oscar Salvador Acked-by: David Rientjes Acked-by: Christoph Lameter Cc: Alex Shi Cc: Daniel Wagner Cc: "Tobin C. Harding" Cc: Christoph Lameter Cc: Andrew Morton Cc: Huang Ying Cc: Dan Williams Cc: Qian Cai Cc: Signed-off-by: Andrew Morton --- Documentation/admin-guide/sysctl/vm.rst | 10 +++++----- mm/vmscan.c | 9 +++++++-- 2 files changed, 12 insertions(+), 7 deletions(-) --- a/Documentation/admin-guide/sysctl/vm.rst~mm-vmscan-restore-zone_reclaim_mode-abi +++ a/Documentation/admin-guide/sysctl/vm.rst @@ -983,11 +983,11 @@ that benefit from having their data cach left disabled as the caching effect is likely to be more important than data locality. -zone_reclaim may be enabled if it's known that the workload is partitioned -such that each partition fits within a NUMA node and that accessing remote -memory would cause a measurable performance reduction. The page allocator -will then reclaim easily reusable pages (those page cache pages that are -currently not used) before allocating off node pages. +Consider enabling one or more zone_reclaim mode bits if it's known that the +workload is partitioned such that each partition fits within a NUMA node +and that accessing remote memory would cause a measurable performance +reduction. The page allocator will take additional actions before +allocating off node pages. Allowing zone reclaim to write out pages stops processes that are writing large amounts of data from dirtying pages on other nodes. Zone --- a/mm/vmscan.c~mm-vmscan-restore-zone_reclaim_mode-abi +++ a/mm/vmscan.c @@ -4085,8 +4085,13 @@ module_init(kswapd_init) */ int node_reclaim_mode __read_mostly; -#define RECLAIM_WRITE (1<<0) /* Writeout pages during reclaim */ -#define RECLAIM_UNMAP (1<<1) /* Unmap pages during reclaim */ +/* + * These bit locations are exposed in the vm.zone_reclaim_mode sysctl + * ABI. New bits are OK, but existing bits can never change. + */ +#define RECLAIM_ZONE (1<<0) /* Run shrink_inactive_list on the zone */ +#define RECLAIM_WRITE (1<<1) /* Writeout pages during reclaim */ +#define RECLAIM_UNMAP (1<<2) /* Unmap pages during reclaim */ /* * Priority for NODE_RECLAIM. This determines the fraction of pages From patchwork Wed Feb 24 20:09:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102719 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AFA84C4332D for ; Wed, 24 Feb 2021 20:09:22 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3D34064E20 for ; Wed, 24 Feb 2021 20:09:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3D34064E20 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C4F4F8D0050; Wed, 24 Feb 2021 15:09:21 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BD7388D001E; Wed, 24 Feb 2021 15:09:21 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A77818D0050; Wed, 24 Feb 2021 15:09:21 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0171.hostedemail.com [216.40.44.171]) by kanga.kvack.org (Postfix) with ESMTP id 8CC448D001E for ; Wed, 24 Feb 2021 15:09:21 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 4EBFC184956D0 for ; Wed, 24 Feb 2021 20:09:21 +0000 (UTC) X-FDA: 77854250922.19.0C6D390 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf16.hostedemail.com (Postfix) with ESMTP id 400C080192EF for ; Wed, 24 Feb 2021 20:09:20 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B673064F29; Wed, 24 Feb 2021 20:09:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197360; bh=mGAFGdaR52V6BaIhJsDW3FXyc08uLOqLChpprmUAP84=; h=Date:From:To:Subject:In-Reply-To:From; b=PDQlptvDZAWHJbqr0flF2JsUFimYkU5hChqe9n+VHD+xYp3YtgAIgUAxZNgIJcLQ4 HooujY02gKUTNCgu5FXp/AgtS0fG002BK/FlKDKLYWBHqQYdrZivTOevKxaMJP3lEb SZMPJ0KpqxzjEgOkg+/0kgqPYsQpiPXdwWe9328Q= Date: Wed, 24 Feb 2021 12:09:19 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vitaly.wool@konsulko.com Subject: [patch 153/173] z3fold: remove unused attribute for release_z3fold_page Message-ID: <20210224200919.5JDaZHvvd%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: dxohitm8pq3uxzkjyedm5jcwomy41rb6 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 400C080192EF Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197360-714551 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: z3fold: remove unused attribute for release_z3fold_page Since commit dcf5aedb24f8 ("z3fold: stricter locking and more careful reclaim"), release_z3fold_page() is used again. So we can drop the unused attribute safely. Link: https://lkml.kernel.org/r/20210120084008.58432-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Cc: Vitaly Wool Signed-off-by: Andrew Morton --- mm/z3fold.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) --- a/mm/z3fold.c~z3fold-remove-unused-attribute-for-release_z3fold_page +++ a/mm/z3fold.c @@ -541,8 +541,7 @@ static void __release_z3fold_page(struct spin_unlock(&pool->stale_lock); } -static void __attribute__((__unused__)) - release_z3fold_page(struct kref *ref) +static void release_z3fold_page(struct kref *ref) { struct z3fold_header *zhdr = container_of(ref, struct z3fold_header, refcount); From patchwork Wed Feb 24 20:09:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102721 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29FBAC433DB for ; Wed, 24 Feb 2021 20:09:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AD16864E6C for ; Wed, 24 Feb 2021 20:09:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AD16864E6C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3752A8D0051; Wed, 24 Feb 2021 15:09:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 34B5A8D001E; Wed, 24 Feb 2021 15:09:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 212CC8D0051; Wed, 24 Feb 2021 15:09:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0045.hostedemail.com [216.40.44.45]) by kanga.kvack.org (Postfix) with ESMTP id 09CF38D001E for ; Wed, 24 Feb 2021 15:09:25 -0500 (EST) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id C8B84EFD6 for ; Wed, 24 Feb 2021 20:09:24 +0000 (UTC) X-FDA: 77854251048.21.1681CBD Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP id 45300C0007C4 for ; Wed, 24 Feb 2021 20:09:21 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 16BCB64E20; Wed, 24 Feb 2021 20:09:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197363; bh=cbWVhJb2ikl9MPOfzpsd3ykSmD86J00qxvIHN0u7wzY=; h=Date:From:To:Subject:In-Reply-To:From; b=LK3mBFc5d+lEAOnttoR9xNMFXxpPVEmmiyG9jv3Z0b+4uef/moQizfMf58c6n6/cG gOv/c+xUejgEILcPS7Uzi6tYXXSiep6ZZGplNC/+3fh6KWfQh0mTladuS6/dSHmllK 85pWuisGmufqy7Rj3EZk1b7uz55hYfIVyI/dAMV4= Date: Wed, 24 Feb 2021 12:09:22 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vitaly.wool@konsulko.com Subject: [patch 154/173] z3fold: simplify the zhdr initialization code in init_z3fold_page() Message-ID: <20210224200922.1vIVbQDbU%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: mcu7ijpj8w96ndc6d1jzwhans96w61c8 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 45300C0007C4 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf03; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197361-942260 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: z3fold: simplify the zhdr initialization code in init_z3fold_page() We can simplify the zhdr initialization by memset() the zhdr first instead of set struct member to zero one by one. This would also make code more compact and clear. Link: https://lkml.kernel.org/r/20210120085851.16159-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Cc: Vitaly Wool Signed-off-by: Andrew Morton --- mm/z3fold.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) --- a/mm/z3fold.c~z3fold-simplify-the-zhdr-initialization-code-in-init_z3fold_page +++ a/mm/z3fold.c @@ -413,16 +413,10 @@ static struct z3fold_header *init_z3fold if (!slots) return NULL; + memset(zhdr, 0, sizeof(*zhdr)); spin_lock_init(&zhdr->page_lock); kref_init(&zhdr->refcount); - zhdr->first_chunks = 0; - zhdr->middle_chunks = 0; - zhdr->last_chunks = 0; - zhdr->first_num = 0; - zhdr->start_middle = 0; zhdr->cpu = -1; - zhdr->foreign_handles = 0; - zhdr->mapped_count = 0; zhdr->slots = slots; zhdr->pool = pool; INIT_LIST_HEAD(&zhdr->buddy); From patchwork Wed Feb 24 20:09:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61DE1C433E6 for ; Wed, 24 Feb 2021 20:09:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E830464E6C for ; Wed, 24 Feb 2021 20:09:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E830464E6C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7BBD88D0052; Wed, 24 Feb 2021 15:09:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 76A5F8D001E; Wed, 24 Feb 2021 15:09:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 659D48D0052; Wed, 24 Feb 2021 15:09:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0234.hostedemail.com [216.40.44.234]) by kanga.kvack.org (Postfix) with ESMTP id 508F68D001E for ; Wed, 24 Feb 2021 15:09:28 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 213C018498763 for ; Wed, 24 Feb 2021 20:09:28 +0000 (UTC) X-FDA: 77854251216.18.AAA43E7 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf14.hostedemail.com (Postfix) with ESMTP id 6E04EC0007C3 for ; Wed, 24 Feb 2021 20:09:20 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 66EF164E7A; Wed, 24 Feb 2021 20:09:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197366; bh=uC0aIgTXfqlnX4HP+lCV4J82gG1VEkXDb/DDvmEK9/U=; h=Date:From:To:Subject:In-Reply-To:From; b=O8466gW7shCYnPZaCt9fNeH6374dckNJRUCYddXUHLIRxkvYHP5A/70IaDS3/pUko CzmX9D4zo5FZ6cQp4rxZ9xPjDc6RIHsDCVe7gadiTMoJOrkjGxO4R2DcqSK+hysRZK sc59B3PRmTiM22n5pAhy9gzSnEgcpaRMVKnsgXTc= Date: Wed, 24 Feb 2021 12:09:25 -0800 From: Andrew Morton To: akpm@linux-foundation.org, alex.shi@linux.alibaba.com, hannes@cmpxchg.org, hughd@google.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 155/173] mm/compaction: remove rcu_read_lock during page compaction Message-ID: <20210224200925.i6zoXNQy3%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: j4ps739hpjrrbdbyyuk97irbkh6awjpc X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 6E04EC0007C3 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf14; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197360-866902 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alex Shi Subject: mm/compaction: remove rcu_read_lock during page compaction isolate_migratepages_block() used rcu_read_lock() with the intention of safeguarding against the mem_cgroup being destroyed concurrently; but its TestClearPageLRU already protects against that. Delete the unnecessary rcu_read_lock() and _unlock(). Hugh Dickins helped on commit log polishing, Thanks! Link: https://lkml.kernel.org/r/1608614453-10739-3-git-send-email-alex.shi@linux.alibaba.com Signed-off-by: Alex Shi Acked-by: Hugh Dickins Cc: Hugh Dickins Cc: Johannes Weiner Signed-off-by: Andrew Morton --- mm/compaction.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) --- a/mm/compaction.c~mm-compaction-remove-rcu_read_lock-during-page-compaction +++ a/mm/compaction.c @@ -995,7 +995,6 @@ isolate_migratepages_block(struct compac if (!TestClearPageLRU(page)) goto isolate_fail_put; - rcu_read_lock(); lruvec = mem_cgroup_page_lruvec(page, pgdat); /* If we already hold the lock, we can skip some rechecking */ @@ -1005,7 +1004,6 @@ isolate_migratepages_block(struct compac compact_lock_irqsave(&lruvec->lru_lock, &flags, cc); locked = lruvec; - rcu_read_unlock(); lruvec_memcg_debug(lruvec, page); @@ -1026,8 +1024,7 @@ isolate_migratepages_block(struct compac SetPageLRU(page); goto isolate_fail_put; } - } else - rcu_read_unlock(); + } /* The whole page is taken off the LRU; skip the tail pages. */ if (PageCompound(page)) From patchwork Wed Feb 24 20:09:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97567C433E0 for ; Wed, 24 Feb 2021 20:09:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 381FA64F08 for ; Wed, 24 Feb 2021 20:09:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 381FA64F08 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C01778D0053; Wed, 24 Feb 2021 15:09:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BAFBA8D001E; Wed, 24 Feb 2021 15:09:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A9FE78D0053; Wed, 24 Feb 2021 15:09:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0020.hostedemail.com [216.40.44.20]) by kanga.kvack.org (Postfix) with ESMTP id 92BC18D001E for ; Wed, 24 Feb 2021 15:09:31 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 55DA1184E6F3F for ; Wed, 24 Feb 2021 20:09:31 +0000 (UTC) X-FDA: 77854251342.16.51768E6 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf24.hostedemail.com (Postfix) with ESMTP id 75F7CA000500 for ; Wed, 24 Feb 2021 20:09:25 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id AACCB64E09; Wed, 24 Feb 2021 20:09:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197370; bh=wR6AXF6Q1HrYX1lGQbY3h6eyj9AgwCkMouT0JLup6T0=; h=Date:From:To:Subject:In-Reply-To:From; b=r/9s3y6n6dVkTZacf5i0orv9gfvc/+MB+CCSlq1ugwfkg0mRvgbGSn6jJegeZ9+0p Tl3fdrPaL1nwZRISIZjsAzijIHz6/3tVqfyCH2dQ6pVwnFp0pNqmJAESNz1+3WqFoj KQ5G1O3NQevrGTDrxltwLHNG3lixnEIGEf8eYCoc= Date: Wed, 24 Feb 2021 12:09:29 -0800 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 156/173] mm/compaction: remove duplicated VM_BUG_ON_PAGE !PageLocked Message-ID: <20210224200929._jmZRUUuN%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 75F7CA000500 X-Stat-Signature: gttyncqcu3c3utxut4s6q4uimin5d1st Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf24; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197365-865929 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/compaction: remove duplicated VM_BUG_ON_PAGE !PageLocked The VM_BUG_ON_PAGE(!PageLocked(page), page) is also done in PageMovable. Remove this explicitly one. Link: https://lkml.kernel.org/r/20210109081420.46030-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- mm/compaction.c | 1 - 1 file changed, 1 deletion(-) --- a/mm/compaction.c~mm-compaction-remove-duplicated-vm_bug_on_page-pagelocked +++ a/mm/compaction.c @@ -137,7 +137,6 @@ EXPORT_SYMBOL(__SetPageMovable); void __ClearPageMovable(struct page *page) { - VM_BUG_ON_PAGE(!PageLocked(page), page); VM_BUG_ON_PAGE(!PageMovable(page), page); /* * Clear registered address_space val with keeping PAGE_MAPPING_MOVABLE From patchwork Wed Feb 24 20:09:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5D026C433DB for ; Wed, 24 Feb 2021 20:09:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E141C64F0D for ; Wed, 24 Feb 2021 20:09:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E141C64F0D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6F9208D0054; Wed, 24 Feb 2021 15:09:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6CE338D001E; Wed, 24 Feb 2021 15:09:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 60A8A8D0054; Wed, 24 Feb 2021 15:09:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0158.hostedemail.com [216.40.44.158]) by kanga.kvack.org (Postfix) with ESMTP id 47ED68D001E for ; Wed, 24 Feb 2021 15:09:35 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 17AA7B9EC for ; Wed, 24 Feb 2021 20:09:35 +0000 (UTC) X-FDA: 77854251510.17.E80CEBA Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP id 4D984C0001FA for ; Wed, 24 Feb 2021 20:09:31 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 1EC2B64E6C; Wed, 24 Feb 2021 20:09:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197373; bh=PQWvhvdBfHr4OhmnPEiXyLRvO2DsK9d75GRbw1D9ll4=; h=Date:From:To:Subject:In-Reply-To:From; b=elHPssemNdkXpsEaSzqZsK1lJUARt1u0vbQui5AsZaPNQ8TzFp1V0sZ+2kX6RvdY7 7zX2JQztr+5Yn7Ca235APQ/8gbjNxMHFJfEoTLD7vWAyRXhws/jXnMho4beqBjvrJA ueFHQDzlFkn0cqg27QzS/hiHRb1LuAyMqqDdy3Aw= Date: Wed, 24 Feb 2021 12:09:32 -0800 From: Andrew Morton To: akpm@linux-foundation.org, charante@codeaurora.org, khalid.aziz@oracle.com, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, ngupta@nitingupta.dev, rientjes@google.com, torvalds@linux-foundation.org, vbabka@suse.cz, vinmenon@codeaurora.org Subject: [patch 157/173] mm/compaction: correct deferral logic for proactive compaction Message-ID: <20210224200932.9R0ZuROdT%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: s5tgu8yat4hp79g3jxqt7ih9mmetjczx X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 4D984C0001FA Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf22; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197371-590357 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Charan Teja Reddy Subject: mm/compaction: correct deferral logic for proactive compaction should_proactive_compact_node() returns true when sum of the weighted fragmentation score of all the zones in the node is greater than the wmark_high of compaction, which then triggers the proactive compaction that operates on the individual zones of the node. But proactive compaction runs on the zone only when its weighted fragmentation score is greater than wmark_low(=wmark_high - 10). This means that the sum of the weighted fragmentation scores of all the zones can exceed the wmark_high but individual weighted fragmentation zone scores can still be less than wmark_low which makes the unnecessary trigger of the proactive compaction only to return doing nothing. Issue with the return of proactive compaction with out even trying is its deferral. It is simply deferred for 1 << COMPACT_MAX_DEFER_SHIFT if the scores across the proactive compaction is same, thinking that compaction didn't make any progress but in reality it didn't even try. With the delay between successive retries for proactive compaction is 500msec, it can result into the deferral for ~30sec with out even trying the proactive compaction. Test scenario is that: compaction_proactiveness=50 thus the wmark_low = 50 and wmark_high = 60. System have 2 zones(Normal and Movable) with sizes 5GB and 6GB respectively. After opening some apps on the android, the weighted fragmentation scores of these zones are 47 and 49 respectively. Since the sum of these fragmentation scores are above the wmark_high which triggers the proactive compaction and there since the individual zones weighted fragmentation scores are below wmark_low, it returns without trying the proactive compaction. As a result the weighted fragmentation scores of the zones are still 47 and 49 which makes the existing logic to defer the compaction thinking that noprogress is made across the compaction. Fix this by checking just zone fragmentation score, not the weighted, in __compact_finished() and use the zones weighted fragmentation score in fragmentation_score_node(). In the test case above, If the weighted average of is above wmark_high, then individual score (not adjusted) of atleast one zone has to be above wmark_high. Thus it avoids the unnecessary trigger and deferrals of the proactive compaction. Link: https://lkml.kernel.org/r/1610989938-31374-1-git-send-email-charante@codeaurora.org Signed-off-by: Charan Teja Reddy Suggested-by: Vlastimil Babka Acked-by: Vlastimil Babka Reviewed-by: Khalid Aziz Acked-by: David Rientjes Cc: Michal Hocko Cc: Nitin Gupta Cc: Vinayak Menon Signed-off-by: Andrew Morton --- mm/compaction.c | 20 ++++++++++++++------ 1 file changed, 14 insertions(+), 6 deletions(-) --- a/mm/compaction.c~mm-compaction-correct-deferral-logic-for-proactive-compaction +++ a/mm/compaction.c @@ -1922,20 +1922,28 @@ static bool kswapd_is_running(pg_data_t /* * A zone's fragmentation score is the external fragmentation wrt to the - * COMPACTION_HPAGE_ORDER scaled by the zone's size. It returns a value - * in the range [0, 100]. + * COMPACTION_HPAGE_ORDER. It returns a value in the range [0, 100]. + */ +static unsigned int fragmentation_score_zone(struct zone *zone) +{ + return extfrag_for_order(zone, COMPACTION_HPAGE_ORDER); +} + +/* + * A weighted zone's fragmentation score is the external fragmentation + * wrt to the COMPACTION_HPAGE_ORDER scaled by the zone's size. It + * returns a value in the range [0, 100]. * * The scaling factor ensures that proactive compaction focuses on larger * zones like ZONE_NORMAL, rather than smaller, specialized zones like * ZONE_DMA32. For smaller zones, the score value remains close to zero, * and thus never exceeds the high threshold for proactive compaction. */ -static unsigned int fragmentation_score_zone(struct zone *zone) +static unsigned int fragmentation_score_zone_weighted(struct zone *zone) { unsigned long score; - score = zone->present_pages * - extfrag_for_order(zone, COMPACTION_HPAGE_ORDER); + score = zone->present_pages * fragmentation_score_zone(zone); return div64_ul(score, zone->zone_pgdat->node_present_pages + 1); } @@ -1955,7 +1963,7 @@ static unsigned int fragmentation_score_ struct zone *zone; zone = &pgdat->node_zones[zoneid]; - score += fragmentation_score_zone(zone); + score += fragmentation_score_zone_weighted(zone); } return score; From patchwork Wed Feb 24 20:09:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102729 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C775C433DB for ; Wed, 24 Feb 2021 20:09:40 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7B6DD64F08 for ; Wed, 24 Feb 2021 20:09:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7B6DD64F08 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 08F4A8D0055; Wed, 24 Feb 2021 15:09:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 03EAC8D001E; Wed, 24 Feb 2021 15:09:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E95FB8D0055; Wed, 24 Feb 2021 15:09:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0239.hostedemail.com [216.40.44.239]) by kanga.kvack.org (Postfix) with ESMTP id D2DB68D001E for ; Wed, 24 Feb 2021 15:09:38 -0500 (EST) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 9D5D682F4072 for ; Wed, 24 Feb 2021 20:09:38 +0000 (UTC) X-FDA: 77854251636.06.0452A71 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf20.hostedemail.com (Postfix) with ESMTP id EDFD92BCF for ; Wed, 24 Feb 2021 20:09:37 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id A66E764F0D; Wed, 24 Feb 2021 20:09:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197377; bh=GSGlVlt9tDAHnmijpnHpxdcRPGZZnZy58WjHF9y9VNI=; h=Date:From:To:Subject:In-Reply-To:From; b=SDEZ1twNEaYS6NogeeNPgZqZge62MhcWJ/hDELpr9axCCQwenUFbMo3B5QSLGk0MK 1NZupWLqsabKUq4ARTBREVrXqqnRTNZw0M81/dxdDGtElyIciQY2WOLKf7+dw1fiVa r3+vG7n4jW9Agh0mPY4jJqB6CbcE6WhVkWlBUEjM= Date: Wed, 24 Feb 2021 12:09:36 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mgorman@techsingularity.net, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz, vvghjk1234@gmail.com Subject: [patch 158/173] mm/compaction: fix misbehaviors of fast_find_migrateblock() Message-ID: <20210224200936.2tiDMI299%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: EDFD92BCF X-Stat-Signature: gjibriifgeb3xy5ufjra55uo9euuexso Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf20; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197377-246320 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Wonhyuk Yang Subject: mm/compaction: fix misbehaviors of fast_find_migrateblock() In the fast_find_migrateblock(), it iterates ocer the freelist to find the proper pageblock. But there are some misbehaviors. First, if the page we found is equal to cc->migrate_pfn, it is considered that we didn't find a suitable pageblock. Secondly, if the loop was terminated because order is less than PAGE_ALLOC_COSTLY_ORDER, it could be considered that we found a suitable one. Thirdly, if the skip bit is set on the page block and we goto continue, it doesn't check nr_scanned. Fourthly, if the page block's skip bit is set, it checks that page block is the last of list, which is unnecessary. Link: https://lkml.kernel.org/r/20210128130411.6125-1-vvghjk1234@gmail.com Fixes: 70b44595eafe9 ("mm, compaction: use free lists to quickly locate a migration source") Signed-off-by: Wonhyuk Yang Acked-by: Vlastimil Babka Cc: Mel Gorman Signed-off-by: Andrew Morton --- mm/compaction.c | 27 ++++++++++++--------------- 1 file changed, 12 insertions(+), 15 deletions(-) --- a/mm/compaction.c~mm-compactoin-fix-misbehaviors-of-fast_find_migrateblock +++ a/mm/compaction.c @@ -1698,6 +1698,7 @@ static unsigned long fast_find_migratebl unsigned long pfn = cc->migrate_pfn; unsigned long high_pfn; int order; + bool found_block = false; /* Skip hints are relied on to avoid repeats on the fast search */ if (cc->ignore_skip_hint) @@ -1740,7 +1741,7 @@ static unsigned long fast_find_migratebl high_pfn = pageblock_start_pfn(cc->migrate_pfn + distance); for (order = cc->order - 1; - order >= PAGE_ALLOC_COSTLY_ORDER && pfn == cc->migrate_pfn && nr_scanned < limit; + order >= PAGE_ALLOC_COSTLY_ORDER && !found_block && nr_scanned < limit; order--) { struct free_area *area = &cc->zone->free_area[order]; struct list_head *freelist; @@ -1755,7 +1756,11 @@ static unsigned long fast_find_migratebl list_for_each_entry(freepage, freelist, lru) { unsigned long free_pfn; - nr_scanned++; + if (nr_scanned++ >= limit) { + move_freelist_tail(freelist, freepage); + break; + } + free_pfn = page_to_pfn(freepage); if (free_pfn < high_pfn) { /* @@ -1764,12 +1769,8 @@ static unsigned long fast_find_migratebl * the list assumes an entry is deleted, not * reordered. */ - if (get_pageblock_skip(freepage)) { - if (list_is_last(freelist, &freepage->lru)) - break; - + if (get_pageblock_skip(freepage)) continue; - } /* Reorder to so a future search skips recent pages */ move_freelist_tail(freelist, freepage); @@ -1777,15 +1778,10 @@ static unsigned long fast_find_migratebl update_fast_start_pfn(cc, free_pfn); pfn = pageblock_start_pfn(free_pfn); cc->fast_search_fail = 0; + found_block = true; set_pageblock_skip(freepage); break; } - - if (nr_scanned >= limit) { - cc->fast_search_fail++; - move_freelist_tail(freelist, freepage); - break; - } } spin_unlock_irqrestore(&cc->zone->lock, flags); } @@ -1796,9 +1792,10 @@ static unsigned long fast_find_migratebl * If fast scanning failed then use a cached entry for a page block * that had free pages as the basis for starting a linear scan. */ - if (pfn == cc->migrate_pfn) + if (!found_block) { + cc->fast_search_fail++; pfn = reinit_migrate_pfn(cc); - + } return pfn; } From patchwork Wed Feb 24 20:09:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102731 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87694C433DB for ; Wed, 24 Feb 2021 20:09:43 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 18DDE64F0E for ; Wed, 24 Feb 2021 20:09:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 18DDE64F0E Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A520C8D0056; Wed, 24 Feb 2021 15:09:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A016B8D001E; Wed, 24 Feb 2021 15:09:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 93F058D0056; Wed, 24 Feb 2021 15:09:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0004.hostedemail.com [216.40.44.4]) by kanga.kvack.org (Postfix) with ESMTP id 7E2238D001E for ; Wed, 24 Feb 2021 15:09:42 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 45BC8184FA880 for ; Wed, 24 Feb 2021 20:09:42 +0000 (UTC) X-FDA: 77854251804.05.9B23E46 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf23.hostedemail.com (Postfix) with ESMTP id 6BAB2A000505 for ; Wed, 24 Feb 2021 20:09:41 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 3626E64F08; Wed, 24 Feb 2021 20:09:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197380; bh=VV6ZulOy2kH1v4Md6bTtg1bw8UcpYGu+3f/5IWxm+AE=; h=Date:From:To:Subject:In-Reply-To:From; b=vP6hVi4+mA48JEBv/XbAVy3giNWWIgob9CiX0fd69Xjersrv61MOqia/yl3pJuNIL ov0ujnwWJevafn16RetSlB83DYQ9WUQDUDoEhFyp5bG5e/svxnI5FFqrPSqaBdhVkV IjUbInV6VzzHt2CJwuS0LuiFORhbebSny6ud0Z6o= Date: Wed, 24 Feb 2021 12:09:39 -0800 From: Andrew Morton To: aarcange@redhat.com, akpm@linux-foundation.org, david@redhat.com, linux-mm@kvack.org, mgorman@techsingularity.net, mhocko@kernel.org, mm-commits@vger.kernel.org, rientjes@google.com, rppt@kernel.org, stable@vger.kernel.org, torvalds@linux-foundation.org, vbabka@suse.cz Subject: [patch 159/173] mm, compaction: make fast_isolate_freepages() stay within zone Message-ID: <20210224200939.yLPGAqLE6%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: ezi51rqq66754wfedc1xe5cgy9e616ks X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 6BAB2A000505 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf23; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197381-486450 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Vlastimil Babka Subject: mm, compaction: make fast_isolate_freepages() stay within zone Compaction always operates on pages from a single given zone when isolating both pages to migrate and freepages. Pageblock boundaries are intersected with zone boundaries to be safe in case zone starts or ends in the middle of pageblock. The use of pageblock_pfn_to_page() protects against non-contiguous pageblocks. The functions fast_isolate_freepages() and fast_isolate_around() don't currently protect the fast freepage isolation thoroughly enough against these corner cases, and can result in freepage isolation operate outside of zone boundaries: - in fast_isolate_freepages() if we get a pfn from the first pageblock of a zone that starts in the middle of that pageblock, 'highest' can be a pfn outside of the zone. If we fail to isolate anything in this function, we may then call fast_isolate_around() on a pfn outside of the zone and there effectively do a set_pageblock_skip(page_to_pfn(highest)) which may currently hit a VM_BUG_ON() in some configurations - fast_isolate_around() checks only the zone end boundary and not beginning, nor that the pageblock is contiguous (with pageblock_pfn_to_page()) so it's possible that we end up calling isolate_freepages_block() on a range of pfn's from two different zones and end up e.g. isolating freepages under the wrong zone's lock. This patch should fix the above issues. Link: https://lkml.kernel.org/r/20210217173300.6394-1-vbabka@suse.cz Fixes: 5a811889de10 ("mm, compaction: use free lists to quickly locate a migration target") Signed-off-by: Vlastimil Babka Acked-by: David Rientjes Acked-by: Mel Gorman Cc: Andrea Arcangeli Cc: David Hildenbrand Cc: Michal Hocko Cc: Mike Rapoport Cc: Signed-off-by: Andrew Morton --- mm/compaction.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) --- a/mm/compaction.c~mm-compaction-make-fast_isolate_freepages-stay-within-zone +++ a/mm/compaction.c @@ -1284,7 +1284,7 @@ static void fast_isolate_around(struct compact_control *cc, unsigned long pfn, unsigned long nr_isolated) { unsigned long start_pfn, end_pfn; - struct page *page = pfn_to_page(pfn); + struct page *page; /* Do not search around if there are enough pages already */ if (cc->nr_freepages >= cc->nr_migratepages) @@ -1295,8 +1295,12 @@ fast_isolate_around(struct compact_contr return; /* Pageblock boundaries */ - start_pfn = pageblock_start_pfn(pfn); - end_pfn = min(pageblock_end_pfn(pfn), zone_end_pfn(cc->zone)) - 1; + start_pfn = max(pageblock_start_pfn(pfn), cc->zone->zone_start_pfn); + end_pfn = min(pageblock_end_pfn(pfn), zone_end_pfn(cc->zone)); + + page = pageblock_pfn_to_page(start_pfn, end_pfn, cc->zone); + if (!page) + return; /* Scan before */ if (start_pfn != pfn) { @@ -1398,7 +1402,8 @@ fast_isolate_freepages(struct compact_co pfn = page_to_pfn(freepage); if (pfn >= highest) - highest = pageblock_start_pfn(pfn); + highest = max(pageblock_start_pfn(pfn), + cc->zone->zone_start_pfn); if (pfn >= low_pfn) { cc->fast_search_fail = 0; @@ -1468,7 +1473,8 @@ fast_isolate_freepages(struct compact_co } else { if (cc->direct_compaction && pfn_valid(min_pfn)) { page = pageblock_pfn_to_page(min_pfn, - pageblock_end_pfn(min_pfn), + min(pageblock_end_pfn(min_pfn), + zone_end_pfn(cc->zone)), cc->zone); cc->free_pfn = min_pfn; } From patchwork Wed Feb 24 20:09:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102733 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A5A5FC433DB for ; Wed, 24 Feb 2021 20:09:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2DB1C64F26 for ; Wed, 24 Feb 2021 20:09:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2DB1C64F26 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id B8BF58D0057; Wed, 24 Feb 2021 15:09:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B62088D001E; Wed, 24 Feb 2021 15:09:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A9DB78D0057; Wed, 24 Feb 2021 15:09:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0038.hostedemail.com [216.40.44.38]) by kanga.kvack.org (Postfix) with ESMTP id 935728D001E for ; Wed, 24 Feb 2021 15:09:46 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 61039184EA71E for ; Wed, 24 Feb 2021 20:09:46 +0000 (UTC) X-FDA: 77854251972.03.E750DD0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP id 814FDC0007CA for ; Wed, 24 Feb 2021 20:09:42 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 1D5C964F0E; Wed, 24 Feb 2021 20:09:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197384; bh=jkS6LfSm4GMWa4nyL6DQ04X9mrUpPN/dSogoQT+I89E=; h=Date:From:To:Subject:In-Reply-To:From; b=N/O60CIOoQBwlRy321JJFmctNicfv3WVDiw4Oic84Xx3KZvXLch4cELZeRrfyFbca 9g+YNq0i8+q2x8TSsXK1T0sL+2nYVpOGciv20PQ7MlHg0pq80kPWfUlwTlbvyqn1y3 M5VgUqFKLrThzWugyTQZyiOJdtKGF8UR1vauGgZ0= Date: Wed, 24 Feb 2021 12:09:43 -0800 From: Andrew Morton To: ak@linux.intel.com, akpm@linux-foundation.org, dave.hansen@intel.com, hannes@cmpxchg.org, linux-mm@kvack.org, mgorman@suse.de, mhocko@suse.com, mingo@redhat.com, mm-commits@vger.kernel.org, peterz@infradead.org, riel@surriel.com, rientjes@google.com, torvalds@linux-foundation.org, willy@infradead.org, ying.huang@intel.com Subject: [patch 160/173] numa balancing: migrate on fault among multiple bound nodes Message-ID: <20210224200943.NOZOj1YCC%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 814FDC0007CA X-Stat-Signature: yk3edsdrnto5yd4oam4medrigieins73 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf22; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197382-716835 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Huang Ying Subject: numa balancing: migrate on fault among multiple bound nodes Now, NUMA balancing can only optimize the page placement among the NUMA nodes if the default memory policy is used. Because the memory policy specified explicitly should take precedence. But this seems too strict in some situations. For example, on a system with 4 NUMA nodes, if the memory of an application is bound to the node 0 and 1, NUMA balancing can potentially migrate the pages between the node 0 and 1 to reduce cross-node accessing without breaking the explicit memory binding policy. So in this patch, we add MPOL_F_NUMA_BALANCING mode flag to set_mempolicy() when mode is MPOL_BIND. With the flag specified, NUMA balancing will be enabled within the thread to optimize the page placement within the constrains of the specified memory binding policy. With the newly added flag, the NUMA balancing control mechanism becomes, - sysctl knob numa_balancing can enable/disable the NUMA balancing globally. - even if sysctl numa_balancing is enabled, the NUMA balancing will be disabled for the memory areas or applications with the explicit memory policy by default. - MPOL_F_NUMA_BALANCING can be used to enable the NUMA balancing for the applications when specifying the explicit memory policy (MPOL_BIND). Various page placement optimization based on the NUMA balancing can be done with these flags. As the first step, in this patch, if the memory of the application is bound to multiple nodes (MPOL_BIND), and in the hint page fault handler the accessing node are in the policy nodemask, the page will be tried to be migrated to the accessing node to reduce the cross-node accessing. If the newly added MPOL_F_NUMA_BALANCING flag is specified by an application on an old kernel version without its support, set_mempolicy() will return -1 and errno will be set to EINVAL. The application can use this behavior to run on both old and new kernel versions. And if the MPOL_F_NUMA_BALANCING flag is specified for the mode other than MPOL_BIND, set_mempolicy() will return -1 and errno will be set to EINVAL as before. Because we don't support optimization based on the NUMA balancing for these modes. In the previous version of the patch, we tried to reuse MPOL_MF_LAZY for mbind(). But that flag is tied to MPOL_MF_MOVE.*, so it seems not a good API/ABI for the purpose of the patch. And because it's not clear whether it's necessary to enable NUMA balancing for a specific memory area inside an application, so we only add the flag at the thread level (set_mempolicy()) instead of the memory area level (mbind()). We can do that when it become necessary. To test the patch, we run a test case as follows on a 4-node machine with 192 GB memory (48 GB per node). 1. Change pmbench memory accessing benchmark to call set_mempolicy() to bind its memory to node 1 and 3 and enable NUMA balancing. Some related code snippets are as follows, #include #include struct bitmask *bmp; int ret; bmp = numa_parse_nodestring("1,3"); ret = set_mempolicy(MPOL_BIND | MPOL_F_NUMA_BALANCING, bmp->maskp, bmp->size + 1); /* If MPOL_F_NUMA_BALANCING isn't supported, fall back to MPOL_BIND */ if (ret < 0 && errno == EINVAL) ret = set_mempolicy(MPOL_BIND, bmp->maskp, bmp->size + 1); if (ret < 0) { perror("Failed to call set_mempolicy"); exit(-1); } 2. Run a memory eater on node 3 to use 40 GB memory before running pmbench. 3. Run pmbench with 64 processes, the working-set size of each process is 640 MB, so the total working-set size is 64 * 640 MB = 40 GB. The CPU and the memory (as in step 1.) of all pmbench processes is bound to node 1 and 3. So, after CPU usage is balanced, some pmbench processes run on the CPUs of the node 3 will access the memory of the node 1. 4. After the pmbench processes run for 100 seconds, kill the memory eater. Now it's possible for some pmbench processes to migrate their pages from node 1 to node 3 to reduce cross-node accessing. Test results show that, with the patch, the pages can be migrated from node 1 to node 3 after killing the memory eater, and the pmbench score can increase about 17.5%. Link: https://lkml.kernel.org/r/20210120061235.148637-2-ying.huang@intel.com Signed-off-by: "Huang, Ying" Acked-by: Mel Gorman Cc: Peter Zijlstra Cc: Ingo Molnar Cc: Rik van Riel Cc: Johannes Weiner Cc: "Matthew Wilcox (Oracle)" Cc: Dave Hansen Cc: Andi Kleen Cc: Michal Hocko Cc: David Rientjes Signed-off-by: Andrew Morton --- include/uapi/linux/mempolicy.h | 4 +++- mm/mempolicy.c | 16 ++++++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-) --- a/include/uapi/linux/mempolicy.h~numa-balancing-migrate-on-fault-among-multiple-bound-nodes +++ a/include/uapi/linux/mempolicy.h @@ -28,12 +28,14 @@ enum { /* Flags for set_mempolicy */ #define MPOL_F_STATIC_NODES (1 << 15) #define MPOL_F_RELATIVE_NODES (1 << 14) +#define MPOL_F_NUMA_BALANCING (1 << 13) /* Optimize with NUMA balancing if possible */ /* * MPOL_MODE_FLAGS is the union of all possible optional mode flags passed to * either set_mempolicy() or mbind(). */ -#define MPOL_MODE_FLAGS (MPOL_F_STATIC_NODES | MPOL_F_RELATIVE_NODES) +#define MPOL_MODE_FLAGS \ + (MPOL_F_STATIC_NODES | MPOL_F_RELATIVE_NODES | MPOL_F_NUMA_BALANCING) /* Flags for get_mempolicy */ #define MPOL_F_NODE (1<<0) /* return next IL mode instead of node mask */ --- a/mm/mempolicy.c~numa-balancing-migrate-on-fault-among-multiple-bound-nodes +++ a/mm/mempolicy.c @@ -875,6 +875,16 @@ static long do_set_mempolicy(unsigned sh goto out; } + if (flags & MPOL_F_NUMA_BALANCING) { + if (new && new->mode == MPOL_BIND) { + new->flags |= (MPOL_F_MOF | MPOL_F_MORON); + } else { + ret = -EINVAL; + mpol_put(new); + goto out; + } + } + ret = mpol_set_nodemask(new, nodes, scratch); if (ret) { mpol_put(new); @@ -2486,6 +2496,12 @@ int mpol_misplaced(struct page *page, st break; case MPOL_BIND: + /* Optimize placement among multiple nodes via NUMA balancing */ + if (pol->flags & MPOL_F_MORON) { + if (node_isset(thisnid, pol->v.nodes)) + break; + goto out; + } /* * allows binding to multiple nodes. From patchwork Wed Feb 24 20:09:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102735 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DF03C433E6 for ; Wed, 24 Feb 2021 20:09:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5687E64F19 for ; Wed, 24 Feb 2021 20:09:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5687E64F19 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DFC3F8D0058; Wed, 24 Feb 2021 15:09:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DD1278D001E; Wed, 24 Feb 2021 15:09:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CC12E8D0058; Wed, 24 Feb 2021 15:09:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0092.hostedemail.com [216.40.44.92]) by kanga.kvack.org (Postfix) with ESMTP id B0CA58D001E for ; Wed, 24 Feb 2021 15:09:50 -0500 (EST) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 82CFA18028209 for ; Wed, 24 Feb 2021 20:09:50 +0000 (UTC) X-FDA: 77854252140.12.A84CA91 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP id 66EBC200038C for ; Wed, 24 Feb 2021 20:09:51 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id C96CD64F28; Wed, 24 Feb 2021 20:09:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197388; bh=kELzXRgMWhr84yS6vpamNA1MvvVXg1cqUXHnFSv7dMU=; h=Date:From:To:Subject:In-Reply-To:From; b=yi+9/Lh803MgLnPbL/joOHOTrqSX+H+eJDNgbCwdJf4HOQDC4WQgSl3GPpH4penhv 3jPzFhqUZIQt0aW0cdM1OmwkgoH8YKNkXlBMNO9VDSBX5nBEzzV35faZHfqod8pWMe l3/yRu7m/Lqm3WgqGw2Zt2b+zXaOJx9cpGUw5bZo= Date: Wed, 24 Feb 2021 12:09:47 -0800 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linmiaohe@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 161/173] mm/mempolicy: use helper range_in_vma() in queue_pages_test_walk() Message-ID: <20210224200947.1hilApkHE%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: n4qsokeo9dwrwkw5xj64soh1nrm4cyc8 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 66EBC200038C Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197391-219282 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: mm/mempolicy: use helper range_in_vma() in queue_pages_test_walk() The helper range_in_vma() is introduced via commit 017b1660df89 ("mm: migration: fix migration of huge PMD shared pages"). But we forgot to use it in queue_pages_test_walk(). Link: https://lkml.kernel.org/r/20210130091352.20220-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- mm/mempolicy.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/mm/mempolicy.c~mm-mempolicy-use-helper-range_in_vma-in-queue_pages_test_walk +++ a/mm/mempolicy.c @@ -677,7 +677,7 @@ static int queue_pages_test_walk(unsigne unsigned long flags = qp->flags; /* range check first */ - VM_BUG_ON_VMA((vma->vm_start > start) || (vma->vm_end < end), vma); + VM_BUG_ON_VMA(!range_in_vma(vma, start, end), vma); if (!qp->first) { qp->first = vma; From patchwork Wed Feb 24 20:09:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102737 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE50DC433DB for ; Wed, 24 Feb 2021 20:09:54 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 62DBC64F19 for ; Wed, 24 Feb 2021 20:09:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 62DBC64F19 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id EE7DD8D0059; Wed, 24 Feb 2021 15:09:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EBC698D001E; Wed, 24 Feb 2021 15:09:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DD26E8D0059; Wed, 24 Feb 2021 15:09:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0184.hostedemail.com [216.40.44.184]) by kanga.kvack.org (Postfix) with ESMTP id C32C78D001E for ; Wed, 24 Feb 2021 15:09:53 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 8C03F18028209 for ; Wed, 24 Feb 2021 20:09:53 +0000 (UTC) X-FDA: 77854252266.30.4BE6B8A Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf29.hostedemail.com (Postfix) with ESMTP id 0C5FEDC for ; Wed, 24 Feb 2021 20:09:52 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 2C2A964EC4; Wed, 24 Feb 2021 20:09:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197392; bh=eeixNsFXgZKGs1v3XLlSzeyRAoDd0cdPojtTzEEi/bY=; h=Date:From:To:Subject:In-Reply-To:From; b=snZ7Y5mJ4fcFA5ZWB0FCTHJ/SAjVZXTILJW9SAfnVRGtGckkrsfgim1BYhiLLd/9J OaNPUq4WORm7hjyVwP4nlDtVMaPS7dpdAijRrP526LoxpXqcYGedtuOjaoUMk7VTx/ vZ84gYLa/7b29yYXoE+BJR9ra8xeI4owsg8FeWeQ= Date: Wed, 24 Feb 2021 12:09:50 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linux-mm@kvack.org, mhocko@suse.com, mm-commits@vger.kernel.org, rientjes@google.com, tangyizhou@huawei.com, torvalds@linux-foundation.org Subject: [patch 162/173] mm, oom: fix a comment in dump_task() Message-ID: <20210224200950.wUwH8Cuz5%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: b9dug7qpywtkea7f6r3axb3gm1uphxsc X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 0C5FEDC Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf29; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197392-645911 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Tang Yizhou Subject: mm, oom: fix a comment in dump_task() If p is a kthread, it will be checked in oom_unkillable_task() so we can delete the corresponding comment. Link: https://lkml.kernel.org/r/20210125133006.7242-1-tangyizhou@huawei.com Signed-off-by: Tang Yizhou Acked-by: David Rientjes Acked-by: Michal Hocko Signed-off-by: Andrew Morton --- mm/oom_kill.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) --- a/mm/oom_kill.c~mm-oom-fix-a-comment-in-dump_task +++ a/mm/oom_kill.c @@ -395,9 +395,8 @@ static int dump_task(struct task_struct task = find_lock_task_mm(p); if (!task) { /* - * This is a kthread or all of p's threads have already - * detached their mm's. There's no need to report - * them; they can't be oom killed anyway. + * All of p's threads have already detached their mm's. There's + * no need to report them; they can't be oom killed anyway. */ return 0; } From patchwork Wed Feb 24 20:09:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102739 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6325CC433E0 for ; Wed, 24 Feb 2021 20:09:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E5FA464E60 for ; Wed, 24 Feb 2021 20:09:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E5FA464E60 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 809308D005A; Wed, 24 Feb 2021 15:09:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7DE738D001E; Wed, 24 Feb 2021 15:09:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F3798D005A; Wed, 24 Feb 2021 15:09:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0196.hostedemail.com [216.40.44.196]) by kanga.kvack.org (Postfix) with ESMTP id 5A1A58D001E for ; Wed, 24 Feb 2021 15:09:57 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 2D39DEFD6 for ; Wed, 24 Feb 2021 20:09:57 +0000 (UTC) X-FDA: 77854252434.11.AA2D575 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf21.hostedemail.com (Postfix) with ESMTP id 4F7EDE000114 for ; Wed, 24 Feb 2021 20:09:53 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 2B25B64F19; Wed, 24 Feb 2021 20:09:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197395; bh=ewtbKrz9EVTe5Ju1dtoA9cVJGYEdBfxK9cX1MhuWCtQ=; h=Date:From:To:Subject:In-Reply-To:From; b=qlZOmZxJB85P/xbqTO3hvUIp3sxQDxLfgVQ42XF1B4Y3K4hbEnXuDsspVbIVAzcQC cu67cIKmZKEyEPmBAWR4//3XRkdb21JyKCPpYcH83UnjO1uY6mSeQ1eVXXauXKl+iZ OQ0lGM69EwqS1DywdQnXALPX6Rl+OENAgCdzsbcs= Date: Wed, 24 Feb 2021 12:09:54 -0800 From: Andrew Morton To: akpm@linux-foundation.org, dan.carpenter@oracle.com, dave@stgolabs.net, david@redhat.com, linux-mm@kvack.org, mhocko@kernel.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 163/173] mm/hugetlb: change hugetlb_reserve_pages() to type bool Message-ID: <20210224200954.pCGSwj02-%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 4F7EDE000114 X-Stat-Signature: ezrqzw6s9byfmqsbnjawmis4xuxec6ob Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf21; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197393-376079 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Kravetz Subject: mm/hugetlb: change hugetlb_reserve_pages() to type bool While reviewing a bug in hugetlb_reserve_pages, it was noticed that all callers ignore the return value. Any failure is considered an ENOMEM error by the callers. Change the function to be of type bool. The function will return true if the reservation was successful, false otherwise. Callers currently assume a zero return code indicates success. Change the callers to look for true to indicate success. No functional change, only code cleanup. Link: https://lkml.kernel.org/r/20201221192542.15732-1-mike.kravetz@oracle.com Signed-off-by: Mike Kravetz Reviewed-by: Matthew Wilcox (Oracle) Cc: David Hildenbrand Cc: Dan Carpenter Cc: Michal Hocko Cc: Davidlohr Bueso Signed-off-by: Andrew Morton --- fs/hugetlbfs/inode.c | 4 ++-- include/linux/hugetlb.h | 2 +- mm/hugetlb.c | 37 ++++++++++++++----------------------- 3 files changed, 17 insertions(+), 26 deletions(-) --- a/fs/hugetlbfs/inode.c~mm-hugetlb-change-hugetlb_reserve_pages-to-type-bool +++ a/fs/hugetlbfs/inode.c @@ -171,7 +171,7 @@ static int hugetlbfs_file_mmap(struct fi file_accessed(file); ret = -ENOMEM; - if (hugetlb_reserve_pages(inode, + if (!hugetlb_reserve_pages(inode, vma->vm_pgoff >> huge_page_order(h), len >> huge_page_shift(h), vma, vma->vm_flags)) @@ -1493,7 +1493,7 @@ struct file *hugetlb_file_setup(const ch inode->i_size = size; clear_nlink(inode); - if (hugetlb_reserve_pages(inode, 0, + if (!hugetlb_reserve_pages(inode, 0, size >> huge_page_shift(hstate_inode(inode)), NULL, acctflag)) file = ERR_PTR(-ENOMEM); --- a/include/linux/hugetlb.h~mm-hugetlb-change-hugetlb_reserve_pages-to-type-bool +++ a/include/linux/hugetlb.h @@ -139,7 +139,7 @@ int hugetlb_mcopy_atomic_pte(struct mm_s unsigned long dst_addr, unsigned long src_addr, struct page **pagep); -int hugetlb_reserve_pages(struct inode *inode, long from, long to, +bool hugetlb_reserve_pages(struct inode *inode, long from, long to, struct vm_area_struct *vma, vm_flags_t vm_flags); long hugetlb_unreserve_pages(struct inode *inode, long start, long end, --- a/mm/hugetlb.c~mm-hugetlb-change-hugetlb_reserve_pages-to-type-bool +++ a/mm/hugetlb.c @@ -5016,12 +5016,13 @@ unsigned long hugetlb_change_protection( return pages << h->order; } -int hugetlb_reserve_pages(struct inode *inode, +/* Return true if reservation was successful, false otherwise. */ +bool hugetlb_reserve_pages(struct inode *inode, long from, long to, struct vm_area_struct *vma, vm_flags_t vm_flags) { - long ret, chg, add = -1; + long chg, add = -1; struct hstate *h = hstate_inode(inode); struct hugepage_subpool *spool = subpool_inode(inode); struct resv_map *resv_map; @@ -5031,7 +5032,7 @@ int hugetlb_reserve_pages(struct inode * /* This should never happen */ if (from > to) { VM_WARN(1, "%s called with a negative range\n", __func__); - return -EINVAL; + return false; } /* @@ -5040,7 +5041,7 @@ int hugetlb_reserve_pages(struct inode * * without using reserves */ if (vm_flags & VM_NORESERVE) - return 0; + return true; /* * Shared mappings base their reservation on the number of pages that @@ -5062,7 +5063,7 @@ int hugetlb_reserve_pages(struct inode * /* Private mapping. */ resv_map = resv_map_alloc(); if (!resv_map) - return -ENOMEM; + return false; chg = to - from; @@ -5070,18 +5071,12 @@ int hugetlb_reserve_pages(struct inode * set_vma_resv_flags(vma, HPAGE_RESV_OWNER); } - if (chg < 0) { - ret = chg; + if (chg < 0) goto out_err; - } - - ret = hugetlb_cgroup_charge_cgroup_rsvd( - hstate_index(h), chg * pages_per_huge_page(h), &h_cg); - if (ret < 0) { - ret = -ENOMEM; + if (hugetlb_cgroup_charge_cgroup_rsvd(hstate_index(h), + chg * pages_per_huge_page(h), &h_cg) < 0) goto out_err; - } if (vma && !(vma->vm_flags & VM_MAYSHARE) && h_cg) { /* For private mappings, the hugetlb_cgroup uncharge info hangs @@ -5096,19 +5091,15 @@ int hugetlb_reserve_pages(struct inode * * reservations already in place (gbl_reserve). */ gbl_reserve = hugepage_subpool_get_pages(spool, chg); - if (gbl_reserve < 0) { - ret = -ENOSPC; + if (gbl_reserve < 0) goto out_uncharge_cgroup; - } /* * Check enough hugepages are available for the reservation. * Hand the pages back to the subpool if there are not */ - ret = hugetlb_acct_memory(h, gbl_reserve); - if (ret < 0) { + if (hugetlb_acct_memory(h, gbl_reserve) < 0) goto out_put_pages; - } /* * Account for the reservations made. Shared mappings record regions @@ -5126,7 +5117,6 @@ int hugetlb_reserve_pages(struct inode * if (unlikely(add < 0)) { hugetlb_acct_memory(h, -gbl_reserve); - ret = add; goto out_put_pages; } else if (unlikely(chg > add)) { /* @@ -5147,7 +5137,8 @@ int hugetlb_reserve_pages(struct inode * hugetlb_acct_memory(h, -rsv_adjust); } } - return 0; + return true; + out_put_pages: /* put back original number of pages, chg */ (void)hugepage_subpool_put_pages(spool, chg); @@ -5163,7 +5154,7 @@ out_err: region_abort(resv_map, from, to, regions_needed); if (vma && is_vma_resv_set(vma, HPAGE_RESV_OWNER)) kref_put(&resv_map->refs, resv_map_release); - return ret; + return false; } long hugetlb_unreserve_pages(struct inode *inode, long start, long end, From patchwork Wed Feb 24 20:09:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102741 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8ACDC433DB for ; Wed, 24 Feb 2021 20:10:01 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 895A464F09 for ; Wed, 24 Feb 2021 20:10:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 895A464F09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 178CA8D005B; Wed, 24 Feb 2021 15:10:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 14E548D001E; Wed, 24 Feb 2021 15:10:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 067388D005B; Wed, 24 Feb 2021 15:10:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0040.hostedemail.com [216.40.44.40]) by kanga.kvack.org (Postfix) with ESMTP id DF4428D001E for ; Wed, 24 Feb 2021 15:10:00 -0500 (EST) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id A308B1826E867 for ; Wed, 24 Feb 2021 20:10:00 +0000 (UTC) X-FDA: 77854252560.17.B2AE46C Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP id 844102000381 for ; Wed, 24 Feb 2021 20:10:01 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id BA0BF64F27; Wed, 24 Feb 2021 20:09:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197399; bh=4MJ/NAWmYYe0nNVtU+J2jEVOC5245uE7UWy6odDvmxY=; h=Date:From:To:Subject:In-Reply-To:From; b=ZSGxZixPGLYWxGeGgrzyLUaOSIDxEaPVamXsmgvyt44UQstEJ9IpUBbqvNKARSWNl 6lqFahkY/9Eg/PEfoqs/R8BcIj06sWRWYfIMHvEU/XOcyo9OThAKJbPKTpnAZTcVdc JUuqE+jJ++M/MmFkHW9G2jX5PDwkgaRe/Q2iYLcs= Date: Wed, 24 Feb 2021 12:09:58 -0800 From: Andrew Morton To: akpm@linux-foundation.org, dan.carpenter@oracle.com, dave@stgolabs.net, david@redhat.com, linux-mm@kvack.org, mhocko@kernel.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org, willy@infradead.org Subject: [patch 164/173] hugetlbfs: remove special hugetlbfs_set_page_dirty() Message-ID: <20210224200958.LUMxEgoR8%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 5gs7qbem9ttc7scrhp4mou8gys1xc78b X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 844102000381 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197401-526218 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mike Kravetz Subject: hugetlbfs: remove special hugetlbfs_set_page_dirty() Matthew Wilcox noticed that hugetlbfs_set_page_dirty always returns 0. Instead, it should return 1 or 0 depending on the previous state of the dirty bit. In addition, the call to compound_head is redundant as it is also performed in calling routine set_page_dirty. Replace the hugetlbfs specific routine hugetlbfs_set_page_dirty with __set_page_dirty_no_writeback as it addresses both of these issues. Link: https://lkml.kernel.org/r/20201221192542.15732-2-mike.kravetz@oracle.com Signed-off-by: Mike Kravetz Suggested-by: Matthew Wilcox Cc: Dan Carpenter Cc: David Hildenbrand Cc: Davidlohr Bueso Cc: Michal Hocko Signed-off-by: Andrew Morton --- fs/hugetlbfs/inode.c | 13 +------------ 1 file changed, 1 insertion(+), 12 deletions(-) --- a/fs/hugetlbfs/inode.c~hugetlbfs-remove-special-hugetlbfs_set_page_dirty +++ a/fs/hugetlbfs/inode.c @@ -952,17 +952,6 @@ static int hugetlbfs_symlink(struct user return error; } -/* - * mark the head page dirty - */ -static int hugetlbfs_set_page_dirty(struct page *page) -{ - struct page *head = compound_head(page); - - SetPageDirty(head); - return 0; -} - static int hugetlbfs_migrate_page(struct address_space *mapping, struct page *newpage, struct page *page, enum migrate_mode mode) @@ -1150,7 +1139,7 @@ static void hugetlbfs_destroy_inode(stru static const struct address_space_operations hugetlbfs_aops = { .write_begin = hugetlbfs_write_begin, .write_end = hugetlbfs_write_end, - .set_page_dirty = hugetlbfs_set_page_dirty, + .set_page_dirty = __set_page_dirty_no_writeback, .migratepage = hugetlbfs_migrate_page, .error_remove_page = hugetlbfs_error_remove_page, }; From patchwork Wed Feb 24 20:10:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102743 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A152C433E0 for ; Wed, 24 Feb 2021 20:10:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9648E64F09 for ; Wed, 24 Feb 2021 20:10:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9648E64F09 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 260EC8D005C; Wed, 24 Feb 2021 15:10:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 211188D001E; Wed, 24 Feb 2021 15:10:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 128088D005C; Wed, 24 Feb 2021 15:10:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0023.hostedemail.com [216.40.44.23]) by kanga.kvack.org (Postfix) with ESMTP id EE4108D001E for ; Wed, 24 Feb 2021 15:10:03 -0500 (EST) Received: from smtpin19.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id B78A3184F4458 for ; Wed, 24 Feb 2021 20:10:03 +0000 (UTC) X-FDA: 77854252686.19.8FD3F33 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP id 55CBC600249E for ; Wed, 24 Feb 2021 20:09:59 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 261DC64E60; Wed, 24 Feb 2021 20:10:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197402; bh=VqA2PqwG7SReedcbqRT8YlSEi7xH2bQ+gG14gABYMBA=; h=Date:From:To:Subject:In-Reply-To:From; b=FgY9x3s8Xp6vTKa4XYsP0blDnFaHrSCGz1P4EuHANdrytHUyOiz57f5+3EOYmAGIY ETdSh5wjdE9lm8eGFDR+rqxVny51SWsajf9795ygyx+XEDZJoDZc53DFAdfgMhR7kV /1QI4e/4JMYe/mZOobPUZq6jDRIjVHcssWwjtXCE= Date: Wed, 24 Feb 2021 12:10:01 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 165/173] hugetlbfs: remove useless BUG_ON(!inode) in hugetlbfs_setattr() Message-ID: <20210224201001.nY1X0ah5G%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 55CBC600249E X-Stat-Signature: pd5p8yfymp7zqsae1j1gpbddbmxkf4h4 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf09; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197399-313346 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: hugetlbfs: remove useless BUG_ON(!inode) in hugetlbfs_setattr() When we reach here with inode = NULL, we should have crashed as inode has already been dereferenced via hstate_inode. So this BUG_ON(!inode) does not take effect and should be removed. Link: https://lkml.kernel.org/r/20210118110700.52506-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: Mike Kravetz Signed-off-by: Andrew Morton --- fs/hugetlbfs/inode.c | 2 -- 1 file changed, 2 deletions(-) --- a/fs/hugetlbfs/inode.c~hugetlbfs-remove-useless-bug_oninode-in-hugetlbfs_setattr +++ a/fs/hugetlbfs/inode.c @@ -761,8 +761,6 @@ static int hugetlbfs_setattr(struct user unsigned int ia_valid = attr->ia_valid; struct hugetlbfs_inode_info *info = HUGETLBFS_I(inode); - BUG_ON(!inode); - error = setattr_prepare(&init_user_ns, dentry, attr); if (error) return error; From patchwork Wed Feb 24 20:10:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102745 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69D50C433E0 for ; Wed, 24 Feb 2021 20:10:08 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id ED8A564F1B for ; Wed, 24 Feb 2021 20:10:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ED8A564F1B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 854A88D005D; Wed, 24 Feb 2021 15:10:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 805118D001E; Wed, 24 Feb 2021 15:10:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 71B698D005D; Wed, 24 Feb 2021 15:10:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0212.hostedemail.com [216.40.44.212]) by kanga.kvack.org (Postfix) with ESMTP id 59E068D001E for ; Wed, 24 Feb 2021 15:10:07 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 23EB175A7 for ; Wed, 24 Feb 2021 20:10:07 +0000 (UTC) X-FDA: 77854252854.03.FCCACC1 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf09.hostedemail.com (Postfix) with ESMTP id BF81C60024A1 for ; Wed, 24 Feb 2021 20:10:02 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 7FF8764F09; Wed, 24 Feb 2021 20:10:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197405; bh=89ua4NCxevzCS8WG7x5y58cgAeK8KEYSosUa8A8VxB8=; h=Date:From:To:Subject:In-Reply-To:From; b=ve+MVRp6i9cuedqmwDxqTHGzKBMn6zkIqjo/bbeL7tuo+85l+1TZUd1Yf+MtbXfN/ K/tICoMdTbdFZEsr9eF1AERrSva8cddBet5a7pT0+itDwaQExLElwW7AIrNm4c1UI9 yKGzy2NgBOudtJVgbKRr0b4FLG6XTI6fD0onzKLs= Date: Wed, 24 Feb 2021 12:10:04 -0800 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 166/173] hugetlbfs: use helper macro default_hstate in init_hugetlbfs_fs Message-ID: <20210224201004.oP8JoZock%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: BF81C60024A1 X-Stat-Signature: u4ye11q98sh1n8yxwoegp8uddrz61qji Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf09; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197402-91450 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: hugetlbfs: use helper macro default_hstate in init_hugetlbfs_fs Since commit e5ff215941d5 ("hugetlb: multiple hstates for multiple page sizes"), we can use macro default_hstate to get the struct hstate which we use by default. But init_hugetlbfs_fs() forgot to use it. Link: https://lkml.kernel.org/r/20210116091827.20982-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: David Hildenbrand Reviewed-by: Mike Kravetz Signed-off-by: Andrew Morton --- fs/hugetlbfs/inode.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/fs/hugetlbfs/inode.c~hugetlbfs-use-helper-macro-default_hstate-in-init_hugetlbfs_fs +++ a/fs/hugetlbfs/inode.c @@ -1543,7 +1543,7 @@ static int __init init_hugetlbfs_fs(void goto out_free; /* default hstate mount is required */ - mnt = mount_one_hugetlbfs(&hstates[default_hstate_idx]); + mnt = mount_one_hugetlbfs(&default_hstate); if (IS_ERR(mnt)) { error = PTR_ERR(mnt); goto out_unreg; From patchwork Wed Feb 24 20:10:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102747 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CADA2C433DB for ; Wed, 24 Feb 2021 20:10:11 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6240364F1B for ; Wed, 24 Feb 2021 20:10:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6240364F1B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E71308D005E; Wed, 24 Feb 2021 15:10:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E46598D001E; Wed, 24 Feb 2021 15:10:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D5BEE8D005E; Wed, 24 Feb 2021 15:10:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0234.hostedemail.com [216.40.44.234]) by kanga.kvack.org (Postfix) with ESMTP id BDE828D001E for ; Wed, 24 Feb 2021 15:10:10 -0500 (EST) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 8D0C718503639 for ; Wed, 24 Feb 2021 20:10:10 +0000 (UTC) X-FDA: 77854252980.29.CE2BC20 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf19.hostedemail.com (Postfix) with ESMTP id 3B86390009ED for ; Wed, 24 Feb 2021 20:10:06 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id CDB1164F20; Wed, 24 Feb 2021 20:10:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197409; bh=juL+5lyI9b1Hdl5lDh7SDbMIrdL7R4f4tvjKK++GYio=; h=Date:From:To:Subject:In-Reply-To:From; b=pMTliPgkYeGKgpA6nSmp1knXiP+BQ8OR5IV/teoWP8YuyJht43ncSfQbzVj7haVmb +e9F1kORNJbqFCz4d7SvanI+EkITQZGBEKC9C4ejKMBhOUFgrvfiENAiJkgCF/dVUJ NsuMuWnVHZ0JC8BwoMJlMafhZ8EX0D125sweFp5E= Date: Wed, 24 Feb 2021 12:10:08 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 167/173] hugetlbfs: correct obsolete function name in hugetlbfs_read_iter() Message-ID: <20210224201008.DyxiBZKNy%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 3B86390009ED X-Stat-Signature: y1jn8m18upzganco86xtfyoyxjfatjkr Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf19; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197406-627496 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: hugetlbfs: correct obsolete function name in hugetlbfs_read_iter() Since commit 36e789144267 ("kill do_generic_mapping_read"), the function do_generic_mapping_read() is renamed to do_generic_file_read(). And then commit 47c27bc46946 ("fs: pass iocb to do_generic_file_read") renamed it to generic_file_buffered_read(). So replace do_generic_mapping_read() with generic_file_buffered_read() to keep comment uptodate. Link: https://lkml.kernel.org/r/20210118063210.47118-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: Mike Kravetz Signed-off-by: Andrew Morton --- fs/hugetlbfs/inode.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/fs/hugetlbfs/inode.c~hugetlbfs-correct-obsolete-function-name-in-hugetlbfs_read_iter +++ a/fs/hugetlbfs/inode.c @@ -310,7 +310,7 @@ hugetlbfs_read_actor(struct page *page, /* * Support for read() - Find the page attached to f_mapping and copy out the - * data. Its *very* similar to do_generic_mapping_read(), we can't use that + * data. Its *very* similar to generic_file_buffered_read(), we can't use that * since it has PAGE_SIZE assumptions. */ static ssize_t hugetlbfs_read_iter(struct kiocb *iocb, struct iov_iter *to) From patchwork Wed Feb 24 20:10:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 320D7C433E0 for ; Wed, 24 Feb 2021 20:10:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B062C64F1B for ; Wed, 24 Feb 2021 20:10:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B062C64F1B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 436378D005F; Wed, 24 Feb 2021 15:10:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3E5568D001E; Wed, 24 Feb 2021 15:10:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2FBAA8D005F; Wed, 24 Feb 2021 15:10:14 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0142.hostedemail.com [216.40.44.142]) by kanga.kvack.org (Postfix) with ESMTP id 198338D001E for ; Wed, 24 Feb 2021 15:10:14 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id CE4C5184EA71E for ; Wed, 24 Feb 2021 20:10:13 +0000 (UTC) X-FDA: 77854253106.24.4F0DDFF Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf12.hostedemail.com (Postfix) with ESMTP id AA797F4 for ; Wed, 24 Feb 2021 20:10:07 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 235C064F0C; Wed, 24 Feb 2021 20:10:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197412; bh=M/TTcjgjOb3SjPpWj5Hs3N3IiOT6/aEzVseG2KL1nZE=; h=Date:From:To:Subject:In-Reply-To:From; b=sOHCBslAvpWXgGcvo5pdpxMYaJ0T9e1arwSrv3qWHTo8TWysr9d/uNOGP6WwbKTfr ibwzPMUIyIUpcVzmzV05kKy9I4PN8gJ+6iUBBOk5MANNKewkWON42+ZVTQGeAE0yEN tokXRI3prZebaZFuxuwFQVGGzrt9v6/v6tsyyAK0= Date: Wed, 24 Feb 2021 12:10:11 -0800 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 168/173] hugetlbfs: remove meaningless variable avoid_reserve Message-ID: <20210224201011.D0Jwjhk-w%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: AA797F4 X-Stat-Signature: ys5b7g51p7cciac3rdz8sw6cxkiphc9x Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf12; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197407-385401 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: hugetlbfs: remove meaningless variable avoid_reserve The variable avoid_reserve is meaningless because we never changed its value and just passed it to alloc_huge_page(). So remove it to make code more clear that in hugetlbfs_fallocate, we never avoid reserve when alloc hugepage yet. Also add a comment offered by Mike Kravetz to explain this. Link: https://lkml.kernel.org/r/20210120071508.9078-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: David Hildenbrand Reviewed-by: Mike Kravetz Signed-off-by: Andrew Morton --- fs/hugetlbfs/inode.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) --- a/fs/hugetlbfs/inode.c~hugetlbfs-remove-meaningless-variable-avoid_reserve +++ a/fs/hugetlbfs/inode.c @@ -680,7 +680,6 @@ static long hugetlbfs_fallocate(struct f */ struct page *page; unsigned long addr; - int avoid_reserve = 0; cond_resched(); @@ -716,8 +715,15 @@ static long hugetlbfs_fallocate(struct f continue; } - /* Allocate page and add to page cache */ - page = alloc_huge_page(&pseudo_vma, addr, avoid_reserve); + /* + * Allocate page without setting the avoid_reserve argument. + * There certainly are no reserves associated with the + * pseudo_vma. However, there could be shared mappings with + * reserves for the file at the inode level. If we fallocate + * pages in these areas, we need to consume the reserves + * to keep reservation accounting consistent. + */ + page = alloc_huge_page(&pseudo_vma, addr, 0); hugetlb_drop_vma_policy(&pseudo_vma); if (IS_ERR(page)) { mutex_unlock(&hugetlb_fault_mutex_table[hash]); From patchwork Wed Feb 24 20:10:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102751 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63CF3C433E0 for ; Wed, 24 Feb 2021 20:10:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E4E8E64F1B for ; Wed, 24 Feb 2021 20:10:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E4E8E64F1B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 8072B8D0060; Wed, 24 Feb 2021 15:10:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7DCEF8D001E; Wed, 24 Feb 2021 15:10:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F2298D0060; Wed, 24 Feb 2021 15:10:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0178.hostedemail.com [216.40.44.178]) by kanga.kvack.org (Postfix) with ESMTP id 53A368D001E for ; Wed, 24 Feb 2021 15:10:17 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 1EBC6EFD6 for ; Wed, 24 Feb 2021 20:10:17 +0000 (UTC) X-FDA: 77854253274.26.994069D Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf22.hostedemail.com (Postfix) with ESMTP id 5B522C0001F7 for ; Wed, 24 Feb 2021 20:10:13 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6E21D64F2A; Wed, 24 Feb 2021 20:10:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197415; bh=wwDyPiRX0SleMTwcSvM+kHwyYUxyk2ocyf48Jh8LQ0Y=; h=Date:From:To:Subject:In-Reply-To:From; b=w/ksMts0BfzPvK9ZEG29m0RcZntoykvx2TFFIfCAGAiPVB1zVY7RcjSYYa74UaMzo 2JxR7u7+94J75QvtuVP1LE0w7xiXIq2dxnbFEw91BUVLL4dd7fr2iS4twLLwqIT5PF gtmVJjfP2hDKE17gNCa36lyzIcZbdYconFIpAK00= Date: Wed, 24 Feb 2021 12:10:14 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 169/173] hugetlbfs: make hugepage size conversion more readable Message-ID: <20210224201014.gMolLNjG3%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 5B522C0001F7 X-Stat-Signature: 5mrgtghbutr8bmujtyjupdesp8yzchx5 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf22; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197413-378507 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: hugetlbfs: make hugepage size conversion more readable The calculation 1U << (h->order + PAGE_SHIFT - 10) is actually equal to (PAGE_SHIFT << (h->order)) >> 10. So we can make it more readable by replace it with huge_page_size(h) >> 10. Link: https://lkml.kernel.org/r/20210122083141.24548-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: Mike Kravetz Signed-off-by: Andrew Morton --- fs/hugetlbfs/inode.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/fs/hugetlbfs/inode.c~hugetlbfs-make-hugepage-size-conversion-more-readable +++ a/fs/hugetlbfs/inode.c @@ -1520,8 +1520,8 @@ static struct vfsmount *__init mount_one put_fs_context(fc); } if (IS_ERR(mnt)) - pr_err("Cannot mount internal hugetlbfs for page size %uK", - 1U << (h->order + PAGE_SHIFT - 10)); + pr_err("Cannot mount internal hugetlbfs for page size %luK", + huge_page_size(h) >> 10); return mnt; } From patchwork Wed Feb 24 20:10:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102753 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2E5DC433E0 for ; Wed, 24 Feb 2021 20:10:21 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3E3EE64F1D for ; Wed, 24 Feb 2021 20:10:21 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3E3EE64F1D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C25A88D0061; Wed, 24 Feb 2021 15:10:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BD4EE8D001E; Wed, 24 Feb 2021 15:10:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AEBA28D0061; Wed, 24 Feb 2021 15:10:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0163.hostedemail.com [216.40.44.163]) by kanga.kvack.org (Postfix) with ESMTP id 9568A8D001E for ; Wed, 24 Feb 2021 15:10:20 -0500 (EST) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 6507382FC492 for ; Wed, 24 Feb 2021 20:10:20 +0000 (UTC) X-FDA: 77854253400.20.3C09AE3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf27.hostedemail.com (Postfix) with ESMTP id A2A3B80192EA for ; Wed, 24 Feb 2021 20:10:13 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B2C5064F12; Wed, 24 Feb 2021 20:10:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197419; bh=tHWvyoZDisRvTn/EDMCKyTePaT1tbvgEM8wBRd/eKkk=; h=Date:From:To:Subject:In-Reply-To:From; b=UYujt1yS0Lv0EgsBCsTk3z1901w5siKfAThBWn8MGuW6jAWmrDqbIMKCA38HZbHbx htZ93kEvF4bwo9kwO8gUn6PBcWm/wwQ9AYtmYCVmKJzczqu7fHzWTmrmID/d8T2uAz AhJ3QYrwLBuxhkepyqpRziVT/O6E+6mnNvtKZJSY= Date: Wed, 24 Feb 2021 12:10:18 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 170/173] hugetlbfs: correct some obsolete comments about inode i_mutex Message-ID: <20210224201018.Sn23UYdQF%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: 63kknd9oqeupnpupgncryd6z5zzwer91 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: A2A3B80192EA Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf27; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197413-105558 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: hugetlbfs: correct some obsolete comments about inode i_mutex Since commit 9902af79c01a ("parallel lookups: actual switch to rwsem"), i_mutex of inode is converted to i_rwsem. So replace i_mutex with i_rwsem to make comments up to date. Link: https://lkml.kernel.org/r/20210127093111.36672-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Cc: Mike Kravetz Signed-off-by: Andrew Morton --- fs/hugetlbfs/inode.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --- a/fs/hugetlbfs/inode.c~hugetlbfs-correct-some-obsolete-comments-about-inode-i_mutex +++ a/fs/hugetlbfs/inode.c @@ -604,7 +604,7 @@ static long hugetlbfs_punch_hole(struct inode_lock(inode); - /* protected by i_mutex */ + /* protected by i_rwsem */ if (info->seals & (F_SEAL_WRITE | F_SEAL_FUTURE_WRITE)) { inode_unlock(inode); return -EPERM; @@ -777,7 +777,7 @@ static int hugetlbfs_setattr(struct user if (newsize & ~huge_page_mask(h)) return -EINVAL; - /* protected by i_mutex */ + /* protected by i_rwsem */ if ((newsize < oldsize && (info->seals & F_SEAL_SHRINK)) || (newsize > oldsize && (info->seals & F_SEAL_GROW))) return -EPERM; From patchwork Wed Feb 24 20:10:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102755 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80249C433E0 for ; Wed, 24 Feb 2021 20:10:25 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0FD9A64F1B for ; Wed, 24 Feb 2021 20:10:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0FD9A64F1B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 916F78D0062; Wed, 24 Feb 2021 15:10:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8C7628D001E; Wed, 24 Feb 2021 15:10:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7DDFC8D0062; Wed, 24 Feb 2021 15:10:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0248.hostedemail.com [216.40.44.248]) by kanga.kvack.org (Postfix) with ESMTP id 62FB08D001E for ; Wed, 24 Feb 2021 15:10:24 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 338C282FAD56 for ; Wed, 24 Feb 2021 20:10:24 +0000 (UTC) X-FDA: 77854253568.14.EBF320E Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf26.hostedemail.com (Postfix) with ESMTP id 4F7BA40001DE for ; Wed, 24 Feb 2021 20:10:19 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8022664F25; Wed, 24 Feb 2021 20:10:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197422; bh=yUN5ZcO0jc8n3cZluia1W28TtIUkTxd1OhTe483NuHw=; h=Date:From:To:Subject:In-Reply-To:From; b=YFcYL9n/0LZ2sQbT/npKdG23QuJykLGmLTBGLCPLf9CQlrcUeP9ooHXvtb5ZpLqzD 9nFvSPfZA1pcqjPGZKmW6zhTyUpbq9GBcRv1hGjnmjelg1LmaGME7QvYatoSGrPyTO kYc2dVLAUzjYK5z76wDfscXeKd5/+C5PAgSx38fQ= Date: Wed, 24 Feb 2021 12:10:21 -0800 From: Andrew Morton To: akpm@linux-foundation.org, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 171/173] hugetlbfs: fix some comment typos Message-ID: <20210224201021.2q1rLvHtB%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 4F7BA40001DE X-Stat-Signature: 5tf19krf3ixynj946igi69gaarsbbwnr Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf26; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197419-664978 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: hugetlbfs: fix some comment typos Fix typos reserv to reserve, minimim to minimum. No functional change intended. Link: https://lkml.kernel.org/r/20210130092351.28072-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Cc: Mike Kravetz Signed-off-by: Andrew Morton --- fs/hugetlbfs/inode.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) --- a/fs/hugetlbfs/inode.c~hugetlbfs-fix-some-comment-typos +++ a/fs/hugetlbfs/inode.c @@ -442,15 +442,15 @@ hugetlb_vmdelete_list(struct rb_root_cac * * truncation is indicated by end of range being LLONG_MAX * In this case, we first scan the range and release found pages. - * After releasing pages, hugetlb_unreserve_pages cleans up region/reserv + * After releasing pages, hugetlb_unreserve_pages cleans up region/reserve * maps and global counts. Page faults can not race with truncation * in this routine. hugetlb_no_page() holds i_mmap_rwsem and prevents * page faults in the truncated range by checking i_size. i_size is * modified while holding i_mmap_rwsem. * hole punch is indicated if end is not LLONG_MAX * In the hole punch case we scan the range and release found pages. - * Only when releasing a page is the associated region/reserv map - * deleted. The region/reserv map for ranges without associated + * Only when releasing a page is the associated region/reserve map + * deleted. The region/reserve map for ranges without associated * pages are not modified. Page faults can race with hole punch. * This is indicated if we find a mapped page. * Note: If the passed end of range value is beyond the end of file, but @@ -1343,7 +1343,7 @@ hugetlbfs_fill_super(struct super_block /* * Allocate and initialize subpool if maximum or minimum size is - * specified. Any needed reservations (for minimim size) are taken + * specified. Any needed reservations (for minimum size) are taken * taken when the subpool is created. */ if (ctx->max_hpages != -1 || ctx->min_hpages != -1) { From patchwork Wed Feb 24 20:10:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102757 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1194AC433DB for ; Wed, 24 Feb 2021 20:10:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A3D1964F2B for ; Wed, 24 Feb 2021 20:10:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A3D1964F2B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3744B8D001E; Wed, 24 Feb 2021 15:10:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 322618D0063; Wed, 24 Feb 2021 15:10:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 240078D001E; Wed, 24 Feb 2021 15:10:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0021.hostedemail.com [216.40.44.21]) by kanga.kvack.org (Postfix) with ESMTP id 0F2948D001E for ; Wed, 24 Feb 2021 15:10:28 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id CBB50EFF7 for ; Wed, 24 Feb 2021 20:10:27 +0000 (UTC) X-FDA: 77854253694.16.561E9BA Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf18.hostedemail.com (Postfix) with ESMTP id B3DDC2000399 for ; Wed, 24 Feb 2021 20:10:28 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 1AB6F64F1B; Wed, 24 Feb 2021 20:10:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197426; bh=rSO2luexVimrqg2JY/1Yf/LNpbLTIzH4DCqcom+dtr0=; h=Date:From:To:Subject:In-Reply-To:From; b=Cu7o2h/g86hUb4q1bRnpXkoglPgyOv6ipsvELNSB2f0vEobOh4ZWyFQGbrMa6RPXw X/gyBraw//j9Vhf8Ukt0TMMZGYLf6TNf9ILxcov9elshJPDt+C8vLNnGFC4j2SNaGX RSbXj5f/jTV8rt8WzNzkGVeTpFAnUp5I7v4nXUQg= Date: Wed, 24 Feb 2021 12:10:25 -0800 From: Andrew Morton To: akpm@linux-foundation.org, david@redhat.com, linmiaohe@huawei.com, linux-mm@kvack.org, mike.kravetz@oracle.com, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 172/173] hugetlbfs: remove unneeded return value of hugetlb_vmtruncate() Message-ID: <20210224201025.7aCgAbBpr%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Stat-Signature: h6dipksysbwsa9fe7w8e45mb8dkb1ght X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: B3DDC2000399 Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197428-608252 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Miaohe Lin Subject: hugetlbfs: remove unneeded return value of hugetlb_vmtruncate() The function hugetlb_vmtruncate() is guaranteed to always success since commit 7aa91e104028 ("hugetlb: allow extending ftruncate on hugetlbfs"). So we should remove the unneeded return value which is always 0. Link: https://lkml.kernel.org/r/20210208084637.47789-1-linmiaohe@huawei.com Signed-off-by: Miaohe Lin Reviewed-by: Mike Kravetz Reviewed-by: David Hildenbrand Signed-off-by: Andrew Morton --- fs/hugetlbfs/inode.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) --- a/fs/hugetlbfs/inode.c~hugetlbfs-remove-unneeded-return-value-of-hugetlb_vmtruncate +++ a/fs/hugetlbfs/inode.c @@ -567,7 +567,7 @@ static void hugetlbfs_evict_inode(struct clear_inode(inode); } -static int hugetlb_vmtruncate(struct inode *inode, loff_t offset) +static void hugetlb_vmtruncate(struct inode *inode, loff_t offset) { pgoff_t pgoff; struct address_space *mapping = inode->i_mapping; @@ -582,7 +582,6 @@ static int hugetlb_vmtruncate(struct ino hugetlb_vmdelete_list(&mapping->i_mmap, pgoff, 0); i_mmap_unlock_write(mapping); remove_inode_hugepages(inode, offset, LLONG_MAX); - return 0; } static long hugetlbfs_punch_hole(struct inode *inode, loff_t offset, loff_t len) @@ -781,9 +780,7 @@ static int hugetlbfs_setattr(struct user if ((newsize < oldsize && (info->seals & F_SEAL_SHRINK)) || (newsize > oldsize && (info->seals & F_SEAL_GROW))) return -EPERM; - error = hugetlb_vmtruncate(inode, newsize); - if (error) - return error; + hugetlb_vmtruncate(inode, newsize); } setattr_copy(&init_user_ns, inode, attr); From patchwork Wed Feb 24 20:10:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Morton X-Patchwork-Id: 12102759 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 582AAC433E0 for ; Wed, 24 Feb 2021 20:10:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E153464E24 for ; Wed, 24 Feb 2021 20:10:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E153464E24 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=linux-foundation.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6D9008D0064; Wed, 24 Feb 2021 15:10:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 67E8B8D0063; Wed, 24 Feb 2021 15:10:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5BA358D0064; Wed, 24 Feb 2021 15:10:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0180.hostedemail.com [216.40.44.180]) by kanga.kvack.org (Postfix) with ESMTP id 470A78D0063 for ; Wed, 24 Feb 2021 15:10:31 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 0F80482FAD56 for ; Wed, 24 Feb 2021 20:10:31 +0000 (UTC) X-FDA: 77854253862.27.2BF62F0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf04.hostedemail.com (Postfix) with ESMTP id 2E4F5EF for ; Wed, 24 Feb 2021 20:10:29 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 6F27264EF1; Wed, 24 Feb 2021 20:10:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1614197429; bh=LRl9CRg1xZG87/rb0dwunkFiHMY6fzrAYDEuE1k7b2w=; h=Date:From:To:Subject:In-Reply-To:From; b=oUMOiWmuhO/1j6Z6W9fM7W86sDv+tbzoCwwt8aYx9jo1SffF3eE9m7UkTUOqnaqa1 5om9fBWi60bm/6zQ3cB3PamFMp67oqCeAB7bwspCVV4pTMDW2iyW2EIVxe1dsg8FY2 ybrFDcBNxUeimN0uoZNq/5mY4xDDBKjJJHOGAMZQ= Date: Wed, 24 Feb 2021 12:10:28 -0800 From: Andrew Morton To: akpm@linux-foundation.org, cy.fan@huawei.com, linux-mm@kvack.org, mm-commits@vger.kernel.org, torvalds@linux-foundation.org Subject: [patch 173/173] mm/migrate: remove unneeded semicolons Message-ID: <20210224201028._WSYzsUf7%akpm@linux-foundation.org> In-Reply-To: <20210224115824.1e289a6895087f10c41dd8d6@linux-foundation.org> User-Agent: s-nail v14.8.16 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 2E4F5EF X-Stat-Signature: 16uaajo11567hkohtkw1igcda9nw5hhz Received-SPF: none (linux-foundation.org>: No applicable sender policy available) receiver=imf04; identity=mailfrom; envelope-from=""; helo=mail.kernel.org; client-ip=198.145.29.99 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1614197429-660296 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Chengyang Fan Subject: mm/migrate: remove unneeded semicolons Remove superfluous semicolons after function definitions. Link: https://lkml.kernel.org/r/20210115110131.2359683-1-cy.fan@huawei.com Signed-off-by: Chengyang Fan Signed-off-by: Andrew Morton --- include/linux/migrate.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- a/include/linux/migrate.h~mm-migrate-remove-unneeded-semicolons +++ a/include/linux/migrate.h @@ -89,7 +89,7 @@ extern int PageMovable(struct page *page extern void __SetPageMovable(struct page *page, struct address_space *mapping); extern void __ClearPageMovable(struct page *page); #else -static inline int PageMovable(struct page *page) { return 0; }; +static inline int PageMovable(struct page *page) { return 0; } static inline void __SetPageMovable(struct page *page, struct address_space *mapping) {