From patchwork Thu Aug 24 03:35:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13363408 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6020BC71145 for ; Thu, 24 Aug 2023 03:36:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D950B280062; Wed, 23 Aug 2023 23:36:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D1D6C8E0011; Wed, 23 Aug 2023 23:36:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B9859280062; Wed, 23 Aug 2023 23:36:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id A36A98E0011 for ; Wed, 23 Aug 2023 23:36:26 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 755D4B19BF for ; Thu, 24 Aug 2023 03:36:26 +0000 (UTC) X-FDA: 81157585572.28.2369135 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) by imf16.hostedemail.com (Postfix) with ESMTP id B1E4418000D for ; Thu, 24 Aug 2023 03:36:24 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=bi91mAkW; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf16.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692848184; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GrI/X5Yh7LcU5jmS4qIzMzIMVvFD9p+WXlgnh63DuQo=; b=j8TJ/yhRPiJBr22x7Ob0to5hW3LN5P/PIIkY/M4Zd+aC3Yf44ArUWa8/6bA0F1QbZPjd1M XRHs0AIuHL6vuM/EDRqGwYj5mT4cH6ps51ahrN/XjQzawfFGbEwmRaAinLHcMWzVlI9a0q AxJ/h/6qRYGBUT3C27Jby0uRRF2FqV8= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=bi91mAkW; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf16.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.171 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692848184; a=rsa-sha256; cv=none; b=n6CqPq04k0uolOCBWDELCH3WDfxQ37QTzAALS2eOFvsJPBOkI8U3n39693Ixkw26GHN3uJ ziUIFVm9ruLm61DYLXlWCX74/sZzV1yMArux+dEXN1rLEOE3cVOqBiwj1QzogBDOL4eS+G LTDVxUY0w3+U1AyD9iDrydx78eqAuIU= Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-68a56ed12c0so754618b3a.0 for ; Wed, 23 Aug 2023 20:36:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1692848183; x=1693452983; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GrI/X5Yh7LcU5jmS4qIzMzIMVvFD9p+WXlgnh63DuQo=; b=bi91mAkWfHF+ZGYrAGi8fSfP1rZgaQ9UJQma4R2GISMfzIMSepcMEEtmQfvpPwQ6sZ 52K3xo34HKG6b4Tb0EhiNBVsZ9PyIACSu80NiXODexlSiTfsepdidiPEL6ktcGMDBKkH CWpSVA/inH6bwziARO2vWCSnFe3pz89phQgHiYKVDAZf74s4T/6V3tF34JyKaqboDMmH SESxt5Qwuo7Pdz2vKABn2yS97A32aC1lsl9BQ3K0Wakrgi/UtYf3bF0wCw1wYDKP5wJG GOFGVNp3DQTv8FP9lay3DWqR5HarnFkL1KQcm6aePLHP9ZUkzjnAXDUEXu0QaA+19pee Q3Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692848183; x=1693452983; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GrI/X5Yh7LcU5jmS4qIzMzIMVvFD9p+WXlgnh63DuQo=; b=RjmaHyIexI9SuudmIBk2UEiCD3Eeq8c7i4srvPzPtDTKzzokHXxbKU8WWotL/2eJ9i 0QksRD2qI6CZ1im+i7vqrA3XKVLeUQqQIgS+z+JW7VePdqchTBx7oDEqPNh5Po0Mp5gt e8UJNUGCJ2ffhclUS/ysy7u3UiD+e/A0uI9fUoNdRmYFIFcqR4exTqp1O4VrlEXmT/9N AVOtQqK1+Ysu2ffIe1QpB4A7xYqMZyM/vuK5HIUTnNdgkzYj/u/F7Ah+zSsjZ6y73ysI guCNkwSGv/P66krQLJC2L5B+BibKPmCs+LRmyS8xMR4lvaa4b4P1UW7Aut4S0o6FKpJ9 saoA== X-Gm-Message-State: AOJu0YyA7KcVvaUqjZ5Y8ePRKPKi2cvD4/y1hWKZjP83AFQMhF6eypME ZaxZJmCCKyc316wYjW8Q6P7bdw== X-Google-Smtp-Source: AGHT+IHygFMFVjr3jXqgDVchpFeZd7tEZ1ZJUV56jgY+zNR9KGBBKA6XO9KnSRG3Wm4xupEdPrToWQ== X-Received: by 2002:a05:6a20:938d:b0:13c:bda3:79c3 with SMTP id x13-20020a056a20938d00b0013cbda379c3mr17537865pzh.4.1692848183684; Wed, 23 Aug 2023 20:36:23 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.146]) by smtp.gmail.com with ESMTPSA id p16-20020a62ab10000000b0068b6137d144sm2996570pff.30.2023.08.23.20.36.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Aug 2023 20:36:23 -0700 (PDT) From: Qi Zheng To: akpm@linux-foundation.org, david@fromorbit.com, tkhai@ya.ru, vbabka@suse.cz, roman.gushchin@linux.dev, djwong@kernel.org, brauner@kernel.org, paulmck@kernel.org, tytso@mit.edu, steven.price@arm.com, cel@kernel.org, senozhatsky@chromium.org, yujie.liu@intel.com, gregkh@linuxfoundation.org, muchun.song@linux.dev, joel@joelfernandes.org, christian.koenig@amd.com, daniel@ffwll.ch Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, dri-devel@lists.freedesktop.org, linux-fsdevel@vger.kernel.org, Qi Zheng , Muchun Song Subject: [PATCH v3 1/4] mm: move some shrinker-related function declarations to mm/internal.h Date: Thu, 24 Aug 2023 11:35:36 +0800 Message-Id: <20230824033539.34570-2-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20230824033539.34570-1-zhengqi.arch@bytedance.com> References: <20230824033539.34570-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: nhbq1jxikgcr5qt47aceihwrrqi99xxf X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: B1E4418000D X-HE-Tag: 1692848184-386827 X-HE-Meta: U2FsdGVkX19rpnwiux1LnDSrNmt8rMwi/sjacEuZ6UTHyela+ekG2qwJZzbofDVvqHb3bZZKypOb5/ZrFNcpxzf2KHdGnytnlRZjgEbkXcSfe9o2oQG8cnVOWZEg+KFqubeArAu+prU57/E8+Di+9wQRUgWK/8TbCCHPQYOx34jw54RGtreVh++G+g3ezbOfRm9sa+vJyOLKM2P3HBTMU9At2AZFo542vr5iy2LvZK/7eY8b9AQ0eTHR0FoPIUaDxMlFTauP+1MJRG/rsu9UeGzPFPxT//9RSgZl6c1PYoZLuJ7BmlJv7WYeYh7yCVJNXrpjSJtGB9XYIb5qOVMbMPqiwqn6EeDgTWEAATuAkct5xMtxxuRqOADjKu2ArvvqKX29JkXBByJQtbEgW1eflMEbspkZNWFIAV8sWwf8ebAUbSiF0w3tMG/6rcy8LBOtP86/00A5vWFXIhyeEDJ/QO4zosAEe3AqzPcAhXbDp4tpVr7CtzgY3X+aWe6ANeLzAbwdgkwo2MxJ9vhvKhioFuh+AFMT9ut2vzc1+oyZvAdYcH7JgNN7ruzwta4oiT3GTVDCy0Iwz8mc/UiRO74t5vWcDPGD9EPB9IfN1Iojt6vQeQDU2Vhh/vuEK8ZWtvi71yF+AsumRLG2VrFvWQLoezCa/uo7rXwbLduHVhXaF98W1x1OzTWMeELBhCqJmi7dyRxSkqguNDcDcubgLU3U4QV2T7j2Hv8PU2p/bNn9Qj88Cq/cyOXVIgo+0bw2TsLV3QXREc3y+cz7skUvpr/7ezMU7ai9EZMSoXpzRh3BmaTVQkpW63a86YGbcXDf0490k4nV0qXDCuxGkv6PXlgRuHbgCqSMrhqXjBABOBb3a/8diqOHkUCL00R0Ek7zhG35J+ZlrkhqYbM0VoAP43aGxb7UsC/mnIm4/RJaWsVaD2S5D2b0XDL/6n6LW044hvfDWXAR1Xqs3yuLVsnPpLK khT921Im i+XaiKMFCcyQaHXssDi/q4l9+9kfxmuWi8Ec1bzjTMbzEWfD6Nq+QjXwMS6SDW4mZkfFj84eOQ26/Bo4R5WPylUQaJT02kZBhptHvi/4XGhqxKtBd9u0p0g9emdUU1+k6CaJaUgSMaYucdXox79NGKkuawJSOeWT3xGEG0PaDljshqBKf25Mf7cPyt4TIQelPRWBjtYJpjVeAEbUKW8MYtsV7AsJVt1m0jkKREaXm9wlBNWAOga/82bTKhsKmEbyxOSOBftwKt6UOLFewBrKOGPCyA07w7bwZFAiUJYvz/uRlsJWLb9QD4F1idmb5inyKt0XBmhWThCbJ8tKvZFgmxYJN+6PWz4qBawqbwdWnYWy5nMCu5EEBtNoCouANe7v/KacjIvaB3aLlmqgy0mEpCFTtQZxow8z5vACQKzYbMYb2Pa5WJ0+DHRWUkjkmmqMIe0nxlF/uLv78iGs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The following functions are only used inside the mm subsystem, so it's better to move their declarations to the mm/internal.h file. 1. shrinker_debugfs_add() 2. shrinker_debugfs_detach() 3. shrinker_debugfs_remove() Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- include/linux/shrinker.h | 19 ------------------- mm/internal.h | 26 ++++++++++++++++++++++++++ mm/shrinker_debug.c | 2 ++ 3 files changed, 28 insertions(+), 19 deletions(-) diff --git a/include/linux/shrinker.h b/include/linux/shrinker.h index 224293b2dd06..8dc15aa37410 100644 --- a/include/linux/shrinker.h +++ b/include/linux/shrinker.h @@ -106,28 +106,9 @@ extern void free_prealloced_shrinker(struct shrinker *shrinker); extern void synchronize_shrinkers(void); #ifdef CONFIG_SHRINKER_DEBUG -extern int shrinker_debugfs_add(struct shrinker *shrinker); -extern struct dentry *shrinker_debugfs_detach(struct shrinker *shrinker, - int *debugfs_id); -extern void shrinker_debugfs_remove(struct dentry *debugfs_entry, - int debugfs_id); extern int __printf(2, 3) shrinker_debugfs_rename(struct shrinker *shrinker, const char *fmt, ...); #else /* CONFIG_SHRINKER_DEBUG */ -static inline int shrinker_debugfs_add(struct shrinker *shrinker) -{ - return 0; -} -static inline struct dentry *shrinker_debugfs_detach(struct shrinker *shrinker, - int *debugfs_id) -{ - *debugfs_id = -1; - return NULL; -} -static inline void shrinker_debugfs_remove(struct dentry *debugfs_entry, - int debugfs_id) -{ -} static inline __printf(2, 3) int shrinker_debugfs_rename(struct shrinker *shrinker, const char *fmt, ...) { diff --git a/mm/internal.h b/mm/internal.h index 7499b5ea1cf6..f30bb60e7790 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1155,4 +1155,30 @@ struct vma_prepare { struct vm_area_struct *remove; struct vm_area_struct *remove2; }; + +/* shrinker related functions */ + +#ifdef CONFIG_SHRINKER_DEBUG +extern int shrinker_debugfs_add(struct shrinker *shrinker); +extern struct dentry *shrinker_debugfs_detach(struct shrinker *shrinker, + int *debugfs_id); +extern void shrinker_debugfs_remove(struct dentry *debugfs_entry, + int debugfs_id); +#else /* CONFIG_SHRINKER_DEBUG */ +static inline int shrinker_debugfs_add(struct shrinker *shrinker) +{ + return 0; +} +static inline struct dentry *shrinker_debugfs_detach(struct shrinker *shrinker, + int *debugfs_id) +{ + *debugfs_id = -1; + return NULL; +} +static inline void shrinker_debugfs_remove(struct dentry *debugfs_entry, + int debugfs_id) +{ +} +#endif /* CONFIG_SHRINKER_DEBUG */ + #endif /* __MM_INTERNAL_H */ diff --git a/mm/shrinker_debug.c b/mm/shrinker_debug.c index 3ab53fad8876..ee0cddb4530f 100644 --- a/mm/shrinker_debug.c +++ b/mm/shrinker_debug.c @@ -6,6 +6,8 @@ #include #include +#include "internal.h" + /* defined in vmscan.c */ extern struct rw_semaphore shrinker_rwsem; extern struct list_head shrinker_list; From patchwork Thu Aug 24 03:35:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13363409 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 936B4C3DA6F for ; Thu, 24 Aug 2023 03:36:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 223A0280067; Wed, 23 Aug 2023 23:36:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1AD218E0011; Wed, 23 Aug 2023 23:36:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF381280067; Wed, 23 Aug 2023 23:36:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id DAB548E0011 for ; Wed, 23 Aug 2023 23:36:40 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 96B2F1C9071 for ; Thu, 24 Aug 2023 03:36:40 +0000 (UTC) X-FDA: 81157586160.27.EE7BFAE Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) by imf15.hostedemail.com (Postfix) with ESMTP id A3B31A000D for ; Thu, 24 Aug 2023 03:36:38 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=dpdiz4r7; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf15.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.182 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692848198; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=r3psnmGFCbLqWduUZROwVMCl8lwTTcvyNoVEFT074Yk=; b=xongSvnj9+Hmo7jYzvctiq5SAOGvt4n8NFJ963Be0rIwga48jLxp2w1okh8jkpcWH3aw46 o0aoIp7BgQWeke/vBmrKi8hU2+CdSlRtLQvIDKJrr2R+pYOk+RdmFtxsMRebzHp3PcmdkV 9M6DA8uqT+yFHpur0eIIcuh9GuEJAGU= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=dpdiz4r7; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf15.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.182 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692848198; a=rsa-sha256; cv=none; b=3N4RY7CUGnnQl4+nk6hdur02l7yUulhmiixHMv6hkSIgZqphS7cMK+JGAtXdrR0eaDEHNq KCrxkqpibudj/27ZP7WpY6NCRNBXA6EhXR9/YKpdmlzqGbeGV5YI4JuLDmx2cg/NkG1Px/ hHmyh7hgU9zoe74on/A58/PVGRg29kY= Received: by mail-pg1-f182.google.com with SMTP id 41be03b00d2f7-56f8334f15eso163006a12.1 for ; Wed, 23 Aug 2023 20:36:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1692848197; x=1693452997; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=r3psnmGFCbLqWduUZROwVMCl8lwTTcvyNoVEFT074Yk=; b=dpdiz4r7mm1pedbpooAfUpZx2b+HV7yPHuqAUPxNhr4M33+yrCqY8Figggq6aEUs4U Otrq28ouiXzWJml1oz3T5qNqZoEPP8cD01Ux87lCG6JRPJ3L73wKDofPMzBZeMVF7R+b 1ADxq2TDMxBkHnOG1m0Et2xg4gsoDt6tNyBMysJJN5wHas6gEAvKD3tfr5u56lBjtKOu Qr2pP8bC/zpufRps4U9UlxAoLbX5jn5CB3/uEFLve4E8r+6utnuqorrhfeoib63R1FM8 pjxwSscggZkz3y4tGA+Jgi3G8yY8xCWpYLtl8jE3n8x/QnzPE2PDlqasZWO+5kqOtnM1 8aLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692848197; x=1693452997; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=r3psnmGFCbLqWduUZROwVMCl8lwTTcvyNoVEFT074Yk=; b=I3XBttOUhP3scCRzYd5s6qsRuajQAo0M72bE+tDmBqp/RcE8KXZ6jA+/Nbg8ab3ofa V2iI2FKoRGZnVOsPPdI/NgWWMpmfHGtJBZywbiYP6ICcWW4pDIeCt9UqzST4MSo3jmv2 Pf+fih9ko8VblM+BoYD3Q5Mn4C4MiGXuGEgrBkiBRVFcbk36aGkT7P86wiBRGMA118tW U55eNF/psg7fK0CbJJFGXtYfidBhRGAJq6gw+v8rPHHctKa5obbZ6KIdRQjLZVaqTXlE Ib2V/dnEuDoVz3jiLXysy9Ab3+PpRH1yh7SWpsZ5M2n8lSCoF2zo4nuLQ7tqmF2Zdafg mcjg== X-Gm-Message-State: AOJu0Yzd/iAAbpikFIXf6+OUoA7nGZrhOk8mOfquKHoW1QyBNdmCpZIH Mk5VcByEJcTUfrDU1gz0+ukFHQ== X-Google-Smtp-Source: AGHT+IGxuU0Wej1QGWnYTl1eV5BlGNM3BG8vGM/tIQQelX/rcugYoOh3dIkC56Zqcea49J4sIM1kWA== X-Received: by 2002:a05:6a00:234b:b0:68a:6cbe:35a7 with SMTP id j11-20020a056a00234b00b0068a6cbe35a7mr6976163pfj.2.1692848196915; Wed, 23 Aug 2023 20:36:36 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.146]) by smtp.gmail.com with ESMTPSA id p16-20020a62ab10000000b0068b6137d144sm2996570pff.30.2023.08.23.20.36.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Aug 2023 20:36:36 -0700 (PDT) From: Qi Zheng To: akpm@linux-foundation.org, david@fromorbit.com, tkhai@ya.ru, vbabka@suse.cz, roman.gushchin@linux.dev, djwong@kernel.org, brauner@kernel.org, paulmck@kernel.org, tytso@mit.edu, steven.price@arm.com, cel@kernel.org, senozhatsky@chromium.org, yujie.liu@intel.com, gregkh@linuxfoundation.org, muchun.song@linux.dev, joel@joelfernandes.org, christian.koenig@amd.com, daniel@ffwll.ch Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, dri-devel@lists.freedesktop.org, linux-fsdevel@vger.kernel.org, Qi Zheng , Muchun Song Subject: [PATCH v3 2/4] mm: vmscan: move shrinker-related code into a separate file Date: Thu, 24 Aug 2023 11:35:37 +0800 Message-Id: <20230824033539.34570-3-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20230824033539.34570-1-zhengqi.arch@bytedance.com> References: <20230824033539.34570-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: A3B31A000D X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 83gwckgh1g5txxukmcuzm5kw7yu14hck X-HE-Tag: 1692848198-443733 X-HE-Meta: U2FsdGVkX19GBYKcdLTH+IqVPVrifna4jSwniHNjSuPCTnygcBtXo4BT3LAC9qKDBolyb+BFTYziYTAuK0eADZv/Pjwn4hDVP/CJg3nZSXJUllbC0DhjKMoiDOYDyiAmvbQTeelMuM0nZEz/9OP+2xHVFIR4c7WhmH/HusOmAv4JzgNu8TBlqedNTRLNwDyNwb7xnV9PPNCbpOOuIwLO2ouslsLWsvVzdZHwn5a/O6yunlE7G2EWfL0puHVevOFXsumnxNzTeDboDIKa4PkQz1zY+xBHb8y3aMKdIGQs1UVO1yv5PL4ITF8EinUY8UAS1Qd8qmAPtrDQrJz4tj46VgYWSvvvFUhxMNncvIT9GihAq9e31RUz4ktZq+Bb7wcmFB3gNlLiQb1cDvFHE8aW6F25GItl+Tns2txGu+O1xn+dph0IOh9lQY4/rXTHtvZsP4qL/IbROMruYeSWt0MFJkjLaeMMQGBPzgogHTUo7i4H0QvMZYvhQe/Ix26d/LjMl5By0R9WgRHxpVsL+TeiVnjnWaKM1qVMLDGNJ3HVHyi2wFlagb4rP6BWfpX86g46vE9qY/5RwWSWD1RUidZEVMmw0rJ4+mLc5OSCrmj2CU9LLEEjtY2jk/2xKTy1165MbmuES/vgYP2DuT6UDq5y2E0K0mE81vWtfqHj5c7SZBwoZou8SkZbPD5atminGdtoJhsaf6A0/GNzumwu+/EI9f5IxoW4SJRWolH7KZoRvsPTJo2oVI3pJigrwOiiofse0g4AO/yIaKB3GT064zN4LhKzMmLkaSZwEoRVtE7V7lP7EITPVjJ5rAdlmoQZvR0UK7CoBRTwsXEWJiSqvW2EX11ZqwPGNITyJ86oK5stXZEk345yclNgcWSyVQFuCxvWcnkauD/bO9/zfz/Tl+0c08dsSRfHfSFN1WzbWd2ebRF7mEm/cE5L0XBjTpb9nCXehbzonrsy65kz2Qir2vZ 27df+R8M mrunEGgqhmg79L7I6KsYdo9+CB2RSS4eynpwrJxBofqf1h/2CTYbDlch/idvwKT/sLB95O+DYenjcn2YwL9b0IDuSmo1C27pZwaojE4ozFedLyb2keydpnDWk5oPBPu2yDuT00cp5rUicIH4AYTeoeEYVMIcHdsZkFG7oN8Tk22xjb5Q6YkCswTQpuijqwgVY4sFa8EafU948ySlsqjCuC5op6zDU86tMVYnAEZaKNO01h0ZrswOJTVkmmnMQShDT6w7g2KgkMrAa7OwfOR37uXAjGzEO9TPkkP7c1L5GzaPzCBRT3qO0mO1Tn4AeuaK2LqliWCJHfD/Oi3knw5Xwyk8j7xaansypG20YzztD+F0JmMgdZnkuOrHgPUV7AwS9xhDsacrN0p5Mbx1bc7d1eCmFm8eCYc7G5Qw9Ja6ACNHGDhZcqGSSJitwqdRh4PQtVpzcArbQgmYZ/jZIqdj6EsHRYApNDNPZA3JqLGngMv/FSlu6s2LTFJGWbQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The mm/vmscan.c file is too large, so separate the shrinker-related code from it into a separate file. No functional changes. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/Makefile | 4 +- mm/internal.h | 2 + mm/shrinker.c | 709 ++++++++++++++++++++++++++++++++++++++++++++++++++ mm/vmscan.c | 701 ------------------------------------------------- 4 files changed, 713 insertions(+), 703 deletions(-) create mode 100644 mm/shrinker.c diff --git a/mm/Makefile b/mm/Makefile index ec65984e2ade..33873c8aedb3 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -48,8 +48,8 @@ endif obj-y := filemap.o mempool.o oom_kill.o fadvise.o \ maccess.o page-writeback.o folio-compat.o \ - readahead.o swap.o truncate.o vmscan.o shmem.o \ - util.o mmzone.o vmstat.o backing-dev.o \ + readahead.o swap.o truncate.o vmscan.o shrinker.o \ + shmem.o util.o mmzone.o vmstat.o backing-dev.o \ mm_init.o percpu.o slab_common.o \ compaction.o show_mem.o shmem_quota.o\ interval_tree.o list_lru.o workingset.o \ diff --git a/mm/internal.h b/mm/internal.h index f30bb60e7790..5d4697612073 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1157,6 +1157,8 @@ struct vma_prepare { }; /* shrinker related functions */ +unsigned long shrink_slab(gfp_t gfp_mask, int nid, struct mem_cgroup *memcg, + int priority); #ifdef CONFIG_SHRINKER_DEBUG extern int shrinker_debugfs_add(struct shrinker *shrinker); diff --git a/mm/shrinker.c b/mm/shrinker.c new file mode 100644 index 000000000000..043c87ccfab4 --- /dev/null +++ b/mm/shrinker.c @@ -0,0 +1,709 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include +#include +#include + +#include "internal.h" + +LIST_HEAD(shrinker_list); +DECLARE_RWSEM(shrinker_rwsem); + +#ifdef CONFIG_MEMCG +static int shrinker_nr_max; + +/* The shrinker_info is expanded in a batch of BITS_PER_LONG */ +static inline int shrinker_map_size(int nr_items) +{ + return (DIV_ROUND_UP(nr_items, BITS_PER_LONG) * sizeof(unsigned long)); +} + +static inline int shrinker_defer_size(int nr_items) +{ + return (round_up(nr_items, BITS_PER_LONG) * sizeof(atomic_long_t)); +} + +void free_shrinker_info(struct mem_cgroup *memcg) +{ + struct mem_cgroup_per_node *pn; + struct shrinker_info *info; + int nid; + + for_each_node(nid) { + pn = memcg->nodeinfo[nid]; + info = rcu_dereference_protected(pn->shrinker_info, true); + kvfree(info); + rcu_assign_pointer(pn->shrinker_info, NULL); + } +} + +int alloc_shrinker_info(struct mem_cgroup *memcg) +{ + struct shrinker_info *info; + int nid, size, ret = 0; + int map_size, defer_size = 0; + + down_write(&shrinker_rwsem); + map_size = shrinker_map_size(shrinker_nr_max); + defer_size = shrinker_defer_size(shrinker_nr_max); + size = map_size + defer_size; + for_each_node(nid) { + info = kvzalloc_node(sizeof(*info) + size, GFP_KERNEL, nid); + if (!info) { + free_shrinker_info(memcg); + ret = -ENOMEM; + break; + } + info->nr_deferred = (atomic_long_t *)(info + 1); + info->map = (void *)info->nr_deferred + defer_size; + info->map_nr_max = shrinker_nr_max; + rcu_assign_pointer(memcg->nodeinfo[nid]->shrinker_info, info); + } + up_write(&shrinker_rwsem); + + return ret; +} + +static struct shrinker_info *shrinker_info_protected(struct mem_cgroup *memcg, + int nid) +{ + return rcu_dereference_protected(memcg->nodeinfo[nid]->shrinker_info, + lockdep_is_held(&shrinker_rwsem)); +} + +static int expand_one_shrinker_info(struct mem_cgroup *memcg, + int map_size, int defer_size, + int old_map_size, int old_defer_size, + int new_nr_max) +{ + struct shrinker_info *new, *old; + struct mem_cgroup_per_node *pn; + int nid; + int size = map_size + defer_size; + + for_each_node(nid) { + pn = memcg->nodeinfo[nid]; + old = shrinker_info_protected(memcg, nid); + /* Not yet online memcg */ + if (!old) + return 0; + + /* Already expanded this shrinker_info */ + if (new_nr_max <= old->map_nr_max) + continue; + + new = kvmalloc_node(sizeof(*new) + size, GFP_KERNEL, nid); + if (!new) + return -ENOMEM; + + new->nr_deferred = (atomic_long_t *)(new + 1); + new->map = (void *)new->nr_deferred + defer_size; + new->map_nr_max = new_nr_max; + + /* map: set all old bits, clear all new bits */ + memset(new->map, (int)0xff, old_map_size); + memset((void *)new->map + old_map_size, 0, map_size - old_map_size); + /* nr_deferred: copy old values, clear all new values */ + memcpy(new->nr_deferred, old->nr_deferred, old_defer_size); + memset((void *)new->nr_deferred + old_defer_size, 0, + defer_size - old_defer_size); + + rcu_assign_pointer(pn->shrinker_info, new); + kvfree_rcu(old, rcu); + } + + return 0; +} + +static int expand_shrinker_info(int new_id) +{ + int ret = 0; + int new_nr_max = round_up(new_id + 1, BITS_PER_LONG); + int map_size, defer_size = 0; + int old_map_size, old_defer_size = 0; + struct mem_cgroup *memcg; + + if (!root_mem_cgroup) + goto out; + + lockdep_assert_held(&shrinker_rwsem); + + map_size = shrinker_map_size(new_nr_max); + defer_size = shrinker_defer_size(new_nr_max); + old_map_size = shrinker_map_size(shrinker_nr_max); + old_defer_size = shrinker_defer_size(shrinker_nr_max); + + memcg = mem_cgroup_iter(NULL, NULL, NULL); + do { + ret = expand_one_shrinker_info(memcg, map_size, defer_size, + old_map_size, old_defer_size, + new_nr_max); + if (ret) { + mem_cgroup_iter_break(NULL, memcg); + goto out; + } + } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL)) != NULL); +out: + if (!ret) + shrinker_nr_max = new_nr_max; + + return ret; +} + +void set_shrinker_bit(struct mem_cgroup *memcg, int nid, int shrinker_id) +{ + if (shrinker_id >= 0 && memcg && !mem_cgroup_is_root(memcg)) { + struct shrinker_info *info; + + rcu_read_lock(); + info = rcu_dereference(memcg->nodeinfo[nid]->shrinker_info); + if (!WARN_ON_ONCE(shrinker_id >= info->map_nr_max)) { + /* Pairs with smp mb in shrink_slab() */ + smp_mb__before_atomic(); + set_bit(shrinker_id, info->map); + } + rcu_read_unlock(); + } +} + +static DEFINE_IDR(shrinker_idr); + +static int prealloc_memcg_shrinker(struct shrinker *shrinker) +{ + int id, ret = -ENOMEM; + + if (mem_cgroup_disabled()) + return -ENOSYS; + + down_write(&shrinker_rwsem); + /* This may call shrinker, so it must use down_read_trylock() */ + id = idr_alloc(&shrinker_idr, shrinker, 0, 0, GFP_KERNEL); + if (id < 0) + goto unlock; + + if (id >= shrinker_nr_max) { + if (expand_shrinker_info(id)) { + idr_remove(&shrinker_idr, id); + goto unlock; + } + } + shrinker->id = id; + ret = 0; +unlock: + up_write(&shrinker_rwsem); + return ret; +} + +static void unregister_memcg_shrinker(struct shrinker *shrinker) +{ + int id = shrinker->id; + + BUG_ON(id < 0); + + lockdep_assert_held(&shrinker_rwsem); + + idr_remove(&shrinker_idr, id); +} + +static long xchg_nr_deferred_memcg(int nid, struct shrinker *shrinker, + struct mem_cgroup *memcg) +{ + struct shrinker_info *info; + + info = shrinker_info_protected(memcg, nid); + return atomic_long_xchg(&info->nr_deferred[shrinker->id], 0); +} + +static long add_nr_deferred_memcg(long nr, int nid, struct shrinker *shrinker, + struct mem_cgroup *memcg) +{ + struct shrinker_info *info; + + info = shrinker_info_protected(memcg, nid); + return atomic_long_add_return(nr, &info->nr_deferred[shrinker->id]); +} + +void reparent_shrinker_deferred(struct mem_cgroup *memcg) +{ + int i, nid; + long nr; + struct mem_cgroup *parent; + struct shrinker_info *child_info, *parent_info; + + parent = parent_mem_cgroup(memcg); + if (!parent) + parent = root_mem_cgroup; + + /* Prevent from concurrent shrinker_info expand */ + down_read(&shrinker_rwsem); + for_each_node(nid) { + child_info = shrinker_info_protected(memcg, nid); + parent_info = shrinker_info_protected(parent, nid); + for (i = 0; i < child_info->map_nr_max; i++) { + nr = atomic_long_read(&child_info->nr_deferred[i]); + atomic_long_add(nr, &parent_info->nr_deferred[i]); + } + } + up_read(&shrinker_rwsem); +} +#else +static int prealloc_memcg_shrinker(struct shrinker *shrinker) +{ + return -ENOSYS; +} + +static void unregister_memcg_shrinker(struct shrinker *shrinker) +{ +} + +static long xchg_nr_deferred_memcg(int nid, struct shrinker *shrinker, + struct mem_cgroup *memcg) +{ + return 0; +} + +static long add_nr_deferred_memcg(long nr, int nid, struct shrinker *shrinker, + struct mem_cgroup *memcg) +{ + return 0; +} +#endif /* CONFIG_MEMCG */ + +static long xchg_nr_deferred(struct shrinker *shrinker, + struct shrink_control *sc) +{ + int nid = sc->nid; + + if (!(shrinker->flags & SHRINKER_NUMA_AWARE)) + nid = 0; + + if (sc->memcg && + (shrinker->flags & SHRINKER_MEMCG_AWARE)) + return xchg_nr_deferred_memcg(nid, shrinker, + sc->memcg); + + return atomic_long_xchg(&shrinker->nr_deferred[nid], 0); +} + + +static long add_nr_deferred(long nr, struct shrinker *shrinker, + struct shrink_control *sc) +{ + int nid = sc->nid; + + if (!(shrinker->flags & SHRINKER_NUMA_AWARE)) + nid = 0; + + if (sc->memcg && + (shrinker->flags & SHRINKER_MEMCG_AWARE)) + return add_nr_deferred_memcg(nr, nid, shrinker, + sc->memcg); + + return atomic_long_add_return(nr, &shrinker->nr_deferred[nid]); +} + +#define SHRINK_BATCH 128 + +static unsigned long do_shrink_slab(struct shrink_control *shrinkctl, + struct shrinker *shrinker, int priority) +{ + unsigned long freed = 0; + unsigned long long delta; + long total_scan; + long freeable; + long nr; + long new_nr; + long batch_size = shrinker->batch ? shrinker->batch + : SHRINK_BATCH; + long scanned = 0, next_deferred; + + freeable = shrinker->count_objects(shrinker, shrinkctl); + if (freeable == 0 || freeable == SHRINK_EMPTY) + return freeable; + + /* + * copy the current shrinker scan count into a local variable + * and zero it so that other concurrent shrinker invocations + * don't also do this scanning work. + */ + nr = xchg_nr_deferred(shrinker, shrinkctl); + + if (shrinker->seeks) { + delta = freeable >> priority; + delta *= 4; + do_div(delta, shrinker->seeks); + } else { + /* + * These objects don't require any IO to create. Trim + * them aggressively under memory pressure to keep + * them from causing refetches in the IO caches. + */ + delta = freeable / 2; + } + + total_scan = nr >> priority; + total_scan += delta; + total_scan = min(total_scan, (2 * freeable)); + + trace_mm_shrink_slab_start(shrinker, shrinkctl, nr, + freeable, delta, total_scan, priority); + + /* + * Normally, we should not scan less than batch_size objects in one + * pass to avoid too frequent shrinker calls, but if the slab has less + * than batch_size objects in total and we are really tight on memory, + * we will try to reclaim all available objects, otherwise we can end + * up failing allocations although there are plenty of reclaimable + * objects spread over several slabs with usage less than the + * batch_size. + * + * We detect the "tight on memory" situations by looking at the total + * number of objects we want to scan (total_scan). If it is greater + * than the total number of objects on slab (freeable), we must be + * scanning at high prio and therefore should try to reclaim as much as + * possible. + */ + while (total_scan >= batch_size || + total_scan >= freeable) { + unsigned long ret; + unsigned long nr_to_scan = min(batch_size, total_scan); + + shrinkctl->nr_to_scan = nr_to_scan; + shrinkctl->nr_scanned = nr_to_scan; + ret = shrinker->scan_objects(shrinker, shrinkctl); + if (ret == SHRINK_STOP) + break; + freed += ret; + + count_vm_events(SLABS_SCANNED, shrinkctl->nr_scanned); + total_scan -= shrinkctl->nr_scanned; + scanned += shrinkctl->nr_scanned; + + cond_resched(); + } + + /* + * The deferred work is increased by any new work (delta) that wasn't + * done, decreased by old deferred work that was done now. + * + * And it is capped to two times of the freeable items. + */ + next_deferred = max_t(long, (nr + delta - scanned), 0); + next_deferred = min(next_deferred, (2 * freeable)); + + /* + * move the unused scan count back into the shrinker in a + * manner that handles concurrent updates. + */ + new_nr = add_nr_deferred(next_deferred, shrinker, shrinkctl); + + trace_mm_shrink_slab_end(shrinker, shrinkctl->nid, freed, nr, new_nr, total_scan); + return freed; +} + +#ifdef CONFIG_MEMCG +static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid, + struct mem_cgroup *memcg, int priority) +{ + struct shrinker_info *info; + unsigned long ret, freed = 0; + int i; + + if (!mem_cgroup_online(memcg)) + return 0; + + if (!down_read_trylock(&shrinker_rwsem)) + return 0; + + info = shrinker_info_protected(memcg, nid); + if (unlikely(!info)) + goto unlock; + + for_each_set_bit(i, info->map, info->map_nr_max) { + struct shrink_control sc = { + .gfp_mask = gfp_mask, + .nid = nid, + .memcg = memcg, + }; + struct shrinker *shrinker; + + shrinker = idr_find(&shrinker_idr, i); + if (unlikely(!shrinker || !(shrinker->flags & SHRINKER_REGISTERED))) { + if (!shrinker) + clear_bit(i, info->map); + continue; + } + + /* Call non-slab shrinkers even though kmem is disabled */ + if (!memcg_kmem_online() && + !(shrinker->flags & SHRINKER_NONSLAB)) + continue; + + ret = do_shrink_slab(&sc, shrinker, priority); + if (ret == SHRINK_EMPTY) { + clear_bit(i, info->map); + /* + * After the shrinker reported that it had no objects to + * free, but before we cleared the corresponding bit in + * the memcg shrinker map, a new object might have been + * added. To make sure, we have the bit set in this + * case, we invoke the shrinker one more time and reset + * the bit if it reports that it is not empty anymore. + * The memory barrier here pairs with the barrier in + * set_shrinker_bit(): + * + * list_lru_add() shrink_slab_memcg() + * list_add_tail() clear_bit() + * + * set_bit() do_shrink_slab() + */ + smp_mb__after_atomic(); + ret = do_shrink_slab(&sc, shrinker, priority); + if (ret == SHRINK_EMPTY) + ret = 0; + else + set_shrinker_bit(memcg, nid, i); + } + freed += ret; + + if (rwsem_is_contended(&shrinker_rwsem)) { + freed = freed ? : 1; + break; + } + } +unlock: + up_read(&shrinker_rwsem); + return freed; +} +#else /* !CONFIG_MEMCG */ +static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid, + struct mem_cgroup *memcg, int priority) +{ + return 0; +} +#endif /* CONFIG_MEMCG */ + +/** + * shrink_slab - shrink slab caches + * @gfp_mask: allocation context + * @nid: node whose slab caches to target + * @memcg: memory cgroup whose slab caches to target + * @priority: the reclaim priority + * + * Call the shrink functions to age shrinkable caches. + * + * @nid is passed along to shrinkers with SHRINKER_NUMA_AWARE set, + * unaware shrinkers will receive a node id of 0 instead. + * + * @memcg specifies the memory cgroup to target. Unaware shrinkers + * are called only if it is the root cgroup. + * + * @priority is sc->priority, we take the number of objects and >> by priority + * in order to get the scan target. + * + * Returns the number of reclaimed slab objects. + */ +unsigned long shrink_slab(gfp_t gfp_mask, int nid, struct mem_cgroup *memcg, + int priority) +{ + unsigned long ret, freed = 0; + struct shrinker *shrinker; + + /* + * The root memcg might be allocated even though memcg is disabled + * via "cgroup_disable=memory" boot parameter. This could make + * mem_cgroup_is_root() return false, then just run memcg slab + * shrink, but skip global shrink. This may result in premature + * oom. + */ + if (!mem_cgroup_disabled() && !mem_cgroup_is_root(memcg)) + return shrink_slab_memcg(gfp_mask, nid, memcg, priority); + + if (!down_read_trylock(&shrinker_rwsem)) + goto out; + + list_for_each_entry(shrinker, &shrinker_list, list) { + struct shrink_control sc = { + .gfp_mask = gfp_mask, + .nid = nid, + .memcg = memcg, + }; + + ret = do_shrink_slab(&sc, shrinker, priority); + if (ret == SHRINK_EMPTY) + ret = 0; + freed += ret; + /* + * Bail out if someone want to register a new shrinker to + * prevent the registration from being stalled for long periods + * by parallel ongoing shrinking. + */ + if (rwsem_is_contended(&shrinker_rwsem)) { + freed = freed ? : 1; + break; + } + } + + up_read(&shrinker_rwsem); +out: + cond_resched(); + return freed; +} + +/* + * Add a shrinker callback to be called from the vm. + */ +static int __prealloc_shrinker(struct shrinker *shrinker) +{ + unsigned int size; + int err; + + if (shrinker->flags & SHRINKER_MEMCG_AWARE) { + err = prealloc_memcg_shrinker(shrinker); + if (err != -ENOSYS) + return err; + + shrinker->flags &= ~SHRINKER_MEMCG_AWARE; + } + + size = sizeof(*shrinker->nr_deferred); + if (shrinker->flags & SHRINKER_NUMA_AWARE) + size *= nr_node_ids; + + shrinker->nr_deferred = kzalloc(size, GFP_KERNEL); + if (!shrinker->nr_deferred) + return -ENOMEM; + + return 0; +} + +#ifdef CONFIG_SHRINKER_DEBUG +int prealloc_shrinker(struct shrinker *shrinker, const char *fmt, ...) +{ + va_list ap; + int err; + + va_start(ap, fmt); + shrinker->name = kvasprintf_const(GFP_KERNEL, fmt, ap); + va_end(ap); + if (!shrinker->name) + return -ENOMEM; + + err = __prealloc_shrinker(shrinker); + if (err) { + kfree_const(shrinker->name); + shrinker->name = NULL; + } + + return err; +} +#else +int prealloc_shrinker(struct shrinker *shrinker, const char *fmt, ...) +{ + return __prealloc_shrinker(shrinker); +} +#endif + +void free_prealloced_shrinker(struct shrinker *shrinker) +{ +#ifdef CONFIG_SHRINKER_DEBUG + kfree_const(shrinker->name); + shrinker->name = NULL; +#endif + if (shrinker->flags & SHRINKER_MEMCG_AWARE) { + down_write(&shrinker_rwsem); + unregister_memcg_shrinker(shrinker); + up_write(&shrinker_rwsem); + return; + } + + kfree(shrinker->nr_deferred); + shrinker->nr_deferred = NULL; +} + +void register_shrinker_prepared(struct shrinker *shrinker) +{ + down_write(&shrinker_rwsem); + list_add_tail(&shrinker->list, &shrinker_list); + shrinker->flags |= SHRINKER_REGISTERED; + shrinker_debugfs_add(shrinker); + up_write(&shrinker_rwsem); +} + +static int __register_shrinker(struct shrinker *shrinker) +{ + int err = __prealloc_shrinker(shrinker); + + if (err) + return err; + register_shrinker_prepared(shrinker); + return 0; +} + +#ifdef CONFIG_SHRINKER_DEBUG +int register_shrinker(struct shrinker *shrinker, const char *fmt, ...) +{ + va_list ap; + int err; + + va_start(ap, fmt); + shrinker->name = kvasprintf_const(GFP_KERNEL, fmt, ap); + va_end(ap); + if (!shrinker->name) + return -ENOMEM; + + err = __register_shrinker(shrinker); + if (err) { + kfree_const(shrinker->name); + shrinker->name = NULL; + } + return err; +} +#else +int register_shrinker(struct shrinker *shrinker, const char *fmt, ...) +{ + return __register_shrinker(shrinker); +} +#endif +EXPORT_SYMBOL(register_shrinker); + +/* + * Remove one + */ +void unregister_shrinker(struct shrinker *shrinker) +{ + struct dentry *debugfs_entry; + int debugfs_id; + + if (!(shrinker->flags & SHRINKER_REGISTERED)) + return; + + down_write(&shrinker_rwsem); + list_del(&shrinker->list); + shrinker->flags &= ~SHRINKER_REGISTERED; + if (shrinker->flags & SHRINKER_MEMCG_AWARE) + unregister_memcg_shrinker(shrinker); + debugfs_entry = shrinker_debugfs_detach(shrinker, &debugfs_id); + up_write(&shrinker_rwsem); + + shrinker_debugfs_remove(debugfs_entry, debugfs_id); + + kfree(shrinker->nr_deferred); + shrinker->nr_deferred = NULL; +} +EXPORT_SYMBOL(unregister_shrinker); + +/** + * synchronize_shrinkers - Wait for all running shrinkers to complete. + * + * This is equivalent to calling unregister_shrink() and register_shrinker(), + * but atomically and with less overhead. This is useful to guarantee that all + * shrinker invocations have seen an update, before freeing memory, similar to + * rcu. + */ +void synchronize_shrinkers(void) +{ + down_write(&shrinker_rwsem); + up_write(&shrinker_rwsem); +} +EXPORT_SYMBOL(synchronize_shrinkers); diff --git a/mm/vmscan.c b/mm/vmscan.c index 6f13394b112e..62bbfea11835 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -35,7 +35,6 @@ #include #include #include -#include #include #include #include @@ -188,246 +187,7 @@ struct scan_control { */ int vm_swappiness = 60; -LIST_HEAD(shrinker_list); -DECLARE_RWSEM(shrinker_rwsem); - #ifdef CONFIG_MEMCG -static int shrinker_nr_max; - -/* The shrinker_info is expanded in a batch of BITS_PER_LONG */ -static inline int shrinker_map_size(int nr_items) -{ - return (DIV_ROUND_UP(nr_items, BITS_PER_LONG) * sizeof(unsigned long)); -} - -static inline int shrinker_defer_size(int nr_items) -{ - return (round_up(nr_items, BITS_PER_LONG) * sizeof(atomic_long_t)); -} - -static struct shrinker_info *shrinker_info_protected(struct mem_cgroup *memcg, - int nid) -{ - return rcu_dereference_protected(memcg->nodeinfo[nid]->shrinker_info, - lockdep_is_held(&shrinker_rwsem)); -} - -static int expand_one_shrinker_info(struct mem_cgroup *memcg, - int map_size, int defer_size, - int old_map_size, int old_defer_size, - int new_nr_max) -{ - struct shrinker_info *new, *old; - struct mem_cgroup_per_node *pn; - int nid; - int size = map_size + defer_size; - - for_each_node(nid) { - pn = memcg->nodeinfo[nid]; - old = shrinker_info_protected(memcg, nid); - /* Not yet online memcg */ - if (!old) - return 0; - - /* Already expanded this shrinker_info */ - if (new_nr_max <= old->map_nr_max) - continue; - - new = kvmalloc_node(sizeof(*new) + size, GFP_KERNEL, nid); - if (!new) - return -ENOMEM; - - new->nr_deferred = (atomic_long_t *)(new + 1); - new->map = (void *)new->nr_deferred + defer_size; - new->map_nr_max = new_nr_max; - - /* map: set all old bits, clear all new bits */ - memset(new->map, (int)0xff, old_map_size); - memset((void *)new->map + old_map_size, 0, map_size - old_map_size); - /* nr_deferred: copy old values, clear all new values */ - memcpy(new->nr_deferred, old->nr_deferred, old_defer_size); - memset((void *)new->nr_deferred + old_defer_size, 0, - defer_size - old_defer_size); - - rcu_assign_pointer(pn->shrinker_info, new); - kvfree_rcu(old, rcu); - } - - return 0; -} - -void free_shrinker_info(struct mem_cgroup *memcg) -{ - struct mem_cgroup_per_node *pn; - struct shrinker_info *info; - int nid; - - for_each_node(nid) { - pn = memcg->nodeinfo[nid]; - info = rcu_dereference_protected(pn->shrinker_info, true); - kvfree(info); - rcu_assign_pointer(pn->shrinker_info, NULL); - } -} - -int alloc_shrinker_info(struct mem_cgroup *memcg) -{ - struct shrinker_info *info; - int nid, size, ret = 0; - int map_size, defer_size = 0; - - down_write(&shrinker_rwsem); - map_size = shrinker_map_size(shrinker_nr_max); - defer_size = shrinker_defer_size(shrinker_nr_max); - size = map_size + defer_size; - for_each_node(nid) { - info = kvzalloc_node(sizeof(*info) + size, GFP_KERNEL, nid); - if (!info) { - free_shrinker_info(memcg); - ret = -ENOMEM; - break; - } - info->nr_deferred = (atomic_long_t *)(info + 1); - info->map = (void *)info->nr_deferred + defer_size; - info->map_nr_max = shrinker_nr_max; - rcu_assign_pointer(memcg->nodeinfo[nid]->shrinker_info, info); - } - up_write(&shrinker_rwsem); - - return ret; -} - -static int expand_shrinker_info(int new_id) -{ - int ret = 0; - int new_nr_max = round_up(new_id + 1, BITS_PER_LONG); - int map_size, defer_size = 0; - int old_map_size, old_defer_size = 0; - struct mem_cgroup *memcg; - - if (!root_mem_cgroup) - goto out; - - lockdep_assert_held(&shrinker_rwsem); - - map_size = shrinker_map_size(new_nr_max); - defer_size = shrinker_defer_size(new_nr_max); - old_map_size = shrinker_map_size(shrinker_nr_max); - old_defer_size = shrinker_defer_size(shrinker_nr_max); - - memcg = mem_cgroup_iter(NULL, NULL, NULL); - do { - ret = expand_one_shrinker_info(memcg, map_size, defer_size, - old_map_size, old_defer_size, - new_nr_max); - if (ret) { - mem_cgroup_iter_break(NULL, memcg); - goto out; - } - } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL)) != NULL); -out: - if (!ret) - shrinker_nr_max = new_nr_max; - - return ret; -} - -void set_shrinker_bit(struct mem_cgroup *memcg, int nid, int shrinker_id) -{ - if (shrinker_id >= 0 && memcg && !mem_cgroup_is_root(memcg)) { - struct shrinker_info *info; - - rcu_read_lock(); - info = rcu_dereference(memcg->nodeinfo[nid]->shrinker_info); - if (!WARN_ON_ONCE(shrinker_id >= info->map_nr_max)) { - /* Pairs with smp mb in shrink_slab() */ - smp_mb__before_atomic(); - set_bit(shrinker_id, info->map); - } - rcu_read_unlock(); - } -} - -static DEFINE_IDR(shrinker_idr); - -static int prealloc_memcg_shrinker(struct shrinker *shrinker) -{ - int id, ret = -ENOMEM; - - if (mem_cgroup_disabled()) - return -ENOSYS; - - down_write(&shrinker_rwsem); - /* This may call shrinker, so it must use down_read_trylock() */ - id = idr_alloc(&shrinker_idr, shrinker, 0, 0, GFP_KERNEL); - if (id < 0) - goto unlock; - - if (id >= shrinker_nr_max) { - if (expand_shrinker_info(id)) { - idr_remove(&shrinker_idr, id); - goto unlock; - } - } - shrinker->id = id; - ret = 0; -unlock: - up_write(&shrinker_rwsem); - return ret; -} - -static void unregister_memcg_shrinker(struct shrinker *shrinker) -{ - int id = shrinker->id; - - BUG_ON(id < 0); - - lockdep_assert_held(&shrinker_rwsem); - - idr_remove(&shrinker_idr, id); -} - -static long xchg_nr_deferred_memcg(int nid, struct shrinker *shrinker, - struct mem_cgroup *memcg) -{ - struct shrinker_info *info; - - info = shrinker_info_protected(memcg, nid); - return atomic_long_xchg(&info->nr_deferred[shrinker->id], 0); -} - -static long add_nr_deferred_memcg(long nr, int nid, struct shrinker *shrinker, - struct mem_cgroup *memcg) -{ - struct shrinker_info *info; - - info = shrinker_info_protected(memcg, nid); - return atomic_long_add_return(nr, &info->nr_deferred[shrinker->id]); -} - -void reparent_shrinker_deferred(struct mem_cgroup *memcg) -{ - int i, nid; - long nr; - struct mem_cgroup *parent; - struct shrinker_info *child_info, *parent_info; - - parent = parent_mem_cgroup(memcg); - if (!parent) - parent = root_mem_cgroup; - - /* Prevent from concurrent shrinker_info expand */ - down_read(&shrinker_rwsem); - for_each_node(nid) { - child_info = shrinker_info_protected(memcg, nid); - parent_info = shrinker_info_protected(parent, nid); - for (i = 0; i < child_info->map_nr_max; i++) { - nr = atomic_long_read(&child_info->nr_deferred[i]); - atomic_long_add(nr, &parent_info->nr_deferred[i]); - } - } - up_read(&shrinker_rwsem); -} /* Returns true for reclaim through cgroup limits or cgroup interfaces. */ static bool cgroup_reclaim(struct scan_control *sc) @@ -468,27 +228,6 @@ static bool writeback_throttling_sane(struct scan_control *sc) return false; } #else -static int prealloc_memcg_shrinker(struct shrinker *shrinker) -{ - return -ENOSYS; -} - -static void unregister_memcg_shrinker(struct shrinker *shrinker) -{ -} - -static long xchg_nr_deferred_memcg(int nid, struct shrinker *shrinker, - struct mem_cgroup *memcg) -{ - return 0; -} - -static long add_nr_deferred_memcg(long nr, int nid, struct shrinker *shrinker, - struct mem_cgroup *memcg) -{ - return 0; -} - static bool cgroup_reclaim(struct scan_control *sc) { return false; @@ -557,39 +296,6 @@ static void flush_reclaim_state(struct scan_control *sc) } } -static long xchg_nr_deferred(struct shrinker *shrinker, - struct shrink_control *sc) -{ - int nid = sc->nid; - - if (!(shrinker->flags & SHRINKER_NUMA_AWARE)) - nid = 0; - - if (sc->memcg && - (shrinker->flags & SHRINKER_MEMCG_AWARE)) - return xchg_nr_deferred_memcg(nid, shrinker, - sc->memcg); - - return atomic_long_xchg(&shrinker->nr_deferred[nid], 0); -} - - -static long add_nr_deferred(long nr, struct shrinker *shrinker, - struct shrink_control *sc) -{ - int nid = sc->nid; - - if (!(shrinker->flags & SHRINKER_NUMA_AWARE)) - nid = 0; - - if (sc->memcg && - (shrinker->flags & SHRINKER_MEMCG_AWARE)) - return add_nr_deferred_memcg(nr, nid, shrinker, - sc->memcg); - - return atomic_long_add_return(nr, &shrinker->nr_deferred[nid]); -} - static bool can_demote(int nid, struct scan_control *sc) { if (!numa_demotion_enabled) @@ -671,413 +377,6 @@ static unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru, return size; } -/* - * Add a shrinker callback to be called from the vm. - */ -static int __prealloc_shrinker(struct shrinker *shrinker) -{ - unsigned int size; - int err; - - if (shrinker->flags & SHRINKER_MEMCG_AWARE) { - err = prealloc_memcg_shrinker(shrinker); - if (err != -ENOSYS) - return err; - - shrinker->flags &= ~SHRINKER_MEMCG_AWARE; - } - - size = sizeof(*shrinker->nr_deferred); - if (shrinker->flags & SHRINKER_NUMA_AWARE) - size *= nr_node_ids; - - shrinker->nr_deferred = kzalloc(size, GFP_KERNEL); - if (!shrinker->nr_deferred) - return -ENOMEM; - - return 0; -} - -#ifdef CONFIG_SHRINKER_DEBUG -int prealloc_shrinker(struct shrinker *shrinker, const char *fmt, ...) -{ - va_list ap; - int err; - - va_start(ap, fmt); - shrinker->name = kvasprintf_const(GFP_KERNEL, fmt, ap); - va_end(ap); - if (!shrinker->name) - return -ENOMEM; - - err = __prealloc_shrinker(shrinker); - if (err) { - kfree_const(shrinker->name); - shrinker->name = NULL; - } - - return err; -} -#else -int prealloc_shrinker(struct shrinker *shrinker, const char *fmt, ...) -{ - return __prealloc_shrinker(shrinker); -} -#endif - -void free_prealloced_shrinker(struct shrinker *shrinker) -{ -#ifdef CONFIG_SHRINKER_DEBUG - kfree_const(shrinker->name); - shrinker->name = NULL; -#endif - if (shrinker->flags & SHRINKER_MEMCG_AWARE) { - down_write(&shrinker_rwsem); - unregister_memcg_shrinker(shrinker); - up_write(&shrinker_rwsem); - return; - } - - kfree(shrinker->nr_deferred); - shrinker->nr_deferred = NULL; -} - -void register_shrinker_prepared(struct shrinker *shrinker) -{ - down_write(&shrinker_rwsem); - list_add_tail(&shrinker->list, &shrinker_list); - shrinker->flags |= SHRINKER_REGISTERED; - shrinker_debugfs_add(shrinker); - up_write(&shrinker_rwsem); -} - -static int __register_shrinker(struct shrinker *shrinker) -{ - int err = __prealloc_shrinker(shrinker); - - if (err) - return err; - register_shrinker_prepared(shrinker); - return 0; -} - -#ifdef CONFIG_SHRINKER_DEBUG -int register_shrinker(struct shrinker *shrinker, const char *fmt, ...) -{ - va_list ap; - int err; - - va_start(ap, fmt); - shrinker->name = kvasprintf_const(GFP_KERNEL, fmt, ap); - va_end(ap); - if (!shrinker->name) - return -ENOMEM; - - err = __register_shrinker(shrinker); - if (err) { - kfree_const(shrinker->name); - shrinker->name = NULL; - } - return err; -} -#else -int register_shrinker(struct shrinker *shrinker, const char *fmt, ...) -{ - return __register_shrinker(shrinker); -} -#endif -EXPORT_SYMBOL(register_shrinker); - -/* - * Remove one - */ -void unregister_shrinker(struct shrinker *shrinker) -{ - struct dentry *debugfs_entry; - int debugfs_id; - - if (!(shrinker->flags & SHRINKER_REGISTERED)) - return; - - down_write(&shrinker_rwsem); - list_del(&shrinker->list); - shrinker->flags &= ~SHRINKER_REGISTERED; - if (shrinker->flags & SHRINKER_MEMCG_AWARE) - unregister_memcg_shrinker(shrinker); - debugfs_entry = shrinker_debugfs_detach(shrinker, &debugfs_id); - up_write(&shrinker_rwsem); - - shrinker_debugfs_remove(debugfs_entry, debugfs_id); - - kfree(shrinker->nr_deferred); - shrinker->nr_deferred = NULL; -} -EXPORT_SYMBOL(unregister_shrinker); - -/** - * synchronize_shrinkers - Wait for all running shrinkers to complete. - * - * This is equivalent to calling unregister_shrink() and register_shrinker(), - * but atomically and with less overhead. This is useful to guarantee that all - * shrinker invocations have seen an update, before freeing memory, similar to - * rcu. - */ -void synchronize_shrinkers(void) -{ - down_write(&shrinker_rwsem); - up_write(&shrinker_rwsem); -} -EXPORT_SYMBOL(synchronize_shrinkers); - -#define SHRINK_BATCH 128 - -static unsigned long do_shrink_slab(struct shrink_control *shrinkctl, - struct shrinker *shrinker, int priority) -{ - unsigned long freed = 0; - unsigned long long delta; - long total_scan; - long freeable; - long nr; - long new_nr; - long batch_size = shrinker->batch ? shrinker->batch - : SHRINK_BATCH; - long scanned = 0, next_deferred; - - freeable = shrinker->count_objects(shrinker, shrinkctl); - if (freeable == 0 || freeable == SHRINK_EMPTY) - return freeable; - - /* - * copy the current shrinker scan count into a local variable - * and zero it so that other concurrent shrinker invocations - * don't also do this scanning work. - */ - nr = xchg_nr_deferred(shrinker, shrinkctl); - - if (shrinker->seeks) { - delta = freeable >> priority; - delta *= 4; - do_div(delta, shrinker->seeks); - } else { - /* - * These objects don't require any IO to create. Trim - * them aggressively under memory pressure to keep - * them from causing refetches in the IO caches. - */ - delta = freeable / 2; - } - - total_scan = nr >> priority; - total_scan += delta; - total_scan = min(total_scan, (2 * freeable)); - - trace_mm_shrink_slab_start(shrinker, shrinkctl, nr, - freeable, delta, total_scan, priority); - - /* - * Normally, we should not scan less than batch_size objects in one - * pass to avoid too frequent shrinker calls, but if the slab has less - * than batch_size objects in total and we are really tight on memory, - * we will try to reclaim all available objects, otherwise we can end - * up failing allocations although there are plenty of reclaimable - * objects spread over several slabs with usage less than the - * batch_size. - * - * We detect the "tight on memory" situations by looking at the total - * number of objects we want to scan (total_scan). If it is greater - * than the total number of objects on slab (freeable), we must be - * scanning at high prio and therefore should try to reclaim as much as - * possible. - */ - while (total_scan >= batch_size || - total_scan >= freeable) { - unsigned long ret; - unsigned long nr_to_scan = min(batch_size, total_scan); - - shrinkctl->nr_to_scan = nr_to_scan; - shrinkctl->nr_scanned = nr_to_scan; - ret = shrinker->scan_objects(shrinker, shrinkctl); - if (ret == SHRINK_STOP) - break; - freed += ret; - - count_vm_events(SLABS_SCANNED, shrinkctl->nr_scanned); - total_scan -= shrinkctl->nr_scanned; - scanned += shrinkctl->nr_scanned; - - cond_resched(); - } - - /* - * The deferred work is increased by any new work (delta) that wasn't - * done, decreased by old deferred work that was done now. - * - * And it is capped to two times of the freeable items. - */ - next_deferred = max_t(long, (nr + delta - scanned), 0); - next_deferred = min(next_deferred, (2 * freeable)); - - /* - * move the unused scan count back into the shrinker in a - * manner that handles concurrent updates. - */ - new_nr = add_nr_deferred(next_deferred, shrinker, shrinkctl); - - trace_mm_shrink_slab_end(shrinker, shrinkctl->nid, freed, nr, new_nr, total_scan); - return freed; -} - -#ifdef CONFIG_MEMCG -static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid, - struct mem_cgroup *memcg, int priority) -{ - struct shrinker_info *info; - unsigned long ret, freed = 0; - int i; - - if (!mem_cgroup_online(memcg)) - return 0; - - if (!down_read_trylock(&shrinker_rwsem)) - return 0; - - info = shrinker_info_protected(memcg, nid); - if (unlikely(!info)) - goto unlock; - - for_each_set_bit(i, info->map, info->map_nr_max) { - struct shrink_control sc = { - .gfp_mask = gfp_mask, - .nid = nid, - .memcg = memcg, - }; - struct shrinker *shrinker; - - shrinker = idr_find(&shrinker_idr, i); - if (unlikely(!shrinker || !(shrinker->flags & SHRINKER_REGISTERED))) { - if (!shrinker) - clear_bit(i, info->map); - continue; - } - - /* Call non-slab shrinkers even though kmem is disabled */ - if (!memcg_kmem_online() && - !(shrinker->flags & SHRINKER_NONSLAB)) - continue; - - ret = do_shrink_slab(&sc, shrinker, priority); - if (ret == SHRINK_EMPTY) { - clear_bit(i, info->map); - /* - * After the shrinker reported that it had no objects to - * free, but before we cleared the corresponding bit in - * the memcg shrinker map, a new object might have been - * added. To make sure, we have the bit set in this - * case, we invoke the shrinker one more time and reset - * the bit if it reports that it is not empty anymore. - * The memory barrier here pairs with the barrier in - * set_shrinker_bit(): - * - * list_lru_add() shrink_slab_memcg() - * list_add_tail() clear_bit() - * - * set_bit() do_shrink_slab() - */ - smp_mb__after_atomic(); - ret = do_shrink_slab(&sc, shrinker, priority); - if (ret == SHRINK_EMPTY) - ret = 0; - else - set_shrinker_bit(memcg, nid, i); - } - freed += ret; - - if (rwsem_is_contended(&shrinker_rwsem)) { - freed = freed ? : 1; - break; - } - } -unlock: - up_read(&shrinker_rwsem); - return freed; -} -#else /* CONFIG_MEMCG */ -static unsigned long shrink_slab_memcg(gfp_t gfp_mask, int nid, - struct mem_cgroup *memcg, int priority) -{ - return 0; -} -#endif /* CONFIG_MEMCG */ - -/** - * shrink_slab - shrink slab caches - * @gfp_mask: allocation context - * @nid: node whose slab caches to target - * @memcg: memory cgroup whose slab caches to target - * @priority: the reclaim priority - * - * Call the shrink functions to age shrinkable caches. - * - * @nid is passed along to shrinkers with SHRINKER_NUMA_AWARE set, - * unaware shrinkers will receive a node id of 0 instead. - * - * @memcg specifies the memory cgroup to target. Unaware shrinkers - * are called only if it is the root cgroup. - * - * @priority is sc->priority, we take the number of objects and >> by priority - * in order to get the scan target. - * - * Returns the number of reclaimed slab objects. - */ -static unsigned long shrink_slab(gfp_t gfp_mask, int nid, - struct mem_cgroup *memcg, - int priority) -{ - unsigned long ret, freed = 0; - struct shrinker *shrinker; - - /* - * The root memcg might be allocated even though memcg is disabled - * via "cgroup_disable=memory" boot parameter. This could make - * mem_cgroup_is_root() return false, then just run memcg slab - * shrink, but skip global shrink. This may result in premature - * oom. - */ - if (!mem_cgroup_disabled() && !mem_cgroup_is_root(memcg)) - return shrink_slab_memcg(gfp_mask, nid, memcg, priority); - - if (!down_read_trylock(&shrinker_rwsem)) - goto out; - - list_for_each_entry(shrinker, &shrinker_list, list) { - struct shrink_control sc = { - .gfp_mask = gfp_mask, - .nid = nid, - .memcg = memcg, - }; - - ret = do_shrink_slab(&sc, shrinker, priority); - if (ret == SHRINK_EMPTY) - ret = 0; - freed += ret; - /* - * Bail out if someone want to register a new shrinker to - * prevent the registration from being stalled for long periods - * by parallel ongoing shrinking. - */ - if (rwsem_is_contended(&shrinker_rwsem)) { - freed = freed ? : 1; - break; - } - } - - up_read(&shrinker_rwsem); -out: - cond_resched(); - return freed; -} - static unsigned long drop_slab_node(int nid) { unsigned long freed = 0; From patchwork Thu Aug 24 03:35:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13363410 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1BA1BC71153 for ; Thu, 24 Aug 2023 03:36:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7C84F280069; Wed, 23 Aug 2023 23:36:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 751EC8E0011; Wed, 23 Aug 2023 23:36:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 62383280069; Wed, 23 Aug 2023 23:36:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 4F2288E0011 for ; Wed, 23 Aug 2023 23:36:49 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 20271B19D8 for ; Thu, 24 Aug 2023 03:36:49 +0000 (UTC) X-FDA: 81157586538.22.6EE84BC Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) by imf03.hostedemail.com (Postfix) with ESMTP id 5AB3F20005 for ; Thu, 24 Aug 2023 03:36:47 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=N1xChJFX; spf=pass (imf03.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692848207; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=q7bwmZAFhPRfqsSPFwB3IV8LJcvLn4gI9V7ELWjZ+Dw=; b=v+PHQbPy0PLMuV/lpm/R4AF5qBQsV7m2ygqzdWKBe1fmFCAXgfU3MvtEUo8kDv4GX4FQ1Y PGeaCndd8GdLr0QxSzcxyiaSEibCN+CtvxIx9ODKXZ5rLfYWDeBBZxN+BxfwJDAu82Hone 7WfRJufuz1MgwKFb62sEwgMBqbphQf0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692848207; a=rsa-sha256; cv=none; b=pY8u1f2O2U+Ybt0TfeiF5JaAJrrqqMpRRnvrGAWvESVVKcaEKTL9fF3xzZBMrOG7vLTRW7 AVnSWQ/DjYUNYQEYm0Gx3rUqP8bWd0Dk9iwiT+3B3tB4sMXV83k1+HfLGkyhqR5AzVfCFI nR/z6tlE98+uuYrxdnr/YmlXDaXhFZ4= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=N1xChJFX; spf=pass (imf03.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.210.176 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-68824a0e747so1236801b3a.1 for ; Wed, 23 Aug 2023 20:36:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1692848206; x=1693453006; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=q7bwmZAFhPRfqsSPFwB3IV8LJcvLn4gI9V7ELWjZ+Dw=; b=N1xChJFXyipb+5RQle6ZyegT3xc3OvIzC5mKhUrbULfJMef+H89T8PEOMnmL4nJdYQ N8zZwV0W6lbNmMmGxMNFgNlUOcPRpyP19+RyPJ8DlNbMVXU40NlXQPhjtgkkI+OTl/K7 8L5JzHE5Msr1tHKYrvJM6quTVdBT+mo9gcMuVKr0nx8NBTltN2C83er/zV0BtgV1PD4/ ACO9Ma9B1JjNMHcrUYOWt8NRs1kA8AusiSw9xGAg5XIzcL62aspNXe2GuAjNFyNoKWrc zIFZvqDP/A4zQqbJ8hoIzQk5YVJUVqKqQiRZYslXjGoF2Cw8Qu1g6Xr1Pa8bM2i/ytkS 4vNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692848206; x=1693453006; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=q7bwmZAFhPRfqsSPFwB3IV8LJcvLn4gI9V7ELWjZ+Dw=; b=IEjU8YFzF+ll0WOXqJYtfilCPkQoWJVIbhiNwAceIT6QkqXNkxHI36YiUiNLoGhNzN 2T4BaxBcC3o77DDVneaPfIouHLSTBv6Yyq+yJTdWbpS6mH2ey74eJyNTHT8gJ1+J53qx BtFdVQHX0n/ILXJaUkfYpYfFaKYB/QOPJnSssu4vivMnDkiWfKtboTC/jvaFwRY8Y05a SZiDFN/WcoPdg+ZsJcKaCWfbe53lDMWJcce9z9UsvtYMl9/8NQJZCZV/IuEoTrOQpG/t cu4uM6aP/xwqhrXpYKprtxirQh3glGbzfOlIOXv7t0EtFVupeZIolrDywaXeHfvV4sP1 DLVQ== X-Gm-Message-State: AOJu0YzpWQqmHPGw0O0gT1Eved3CnPMldHnJhQCGvJvM/q7gFRTGWzBE spVxE1Xn/KGYsb+Hkeig0F1uRQ== X-Google-Smtp-Source: AGHT+IF/No7OicKHtxqrUxwVNpcooeU5Pi6D1w0lXRqE33BiijaIAV5KxcraEu1IfHFFlHpUklkwxg== X-Received: by 2002:a05:6a00:1d85:b0:68a:6cec:e538 with SMTP id z5-20020a056a001d8500b0068a6cece538mr7234848pfw.3.1692848205878; Wed, 23 Aug 2023 20:36:45 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.146]) by smtp.gmail.com with ESMTPSA id p16-20020a62ab10000000b0068b6137d144sm2996570pff.30.2023.08.23.20.36.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Aug 2023 20:36:45 -0700 (PDT) From: Qi Zheng To: akpm@linux-foundation.org, david@fromorbit.com, tkhai@ya.ru, vbabka@suse.cz, roman.gushchin@linux.dev, djwong@kernel.org, brauner@kernel.org, paulmck@kernel.org, tytso@mit.edu, steven.price@arm.com, cel@kernel.org, senozhatsky@chromium.org, yujie.liu@intel.com, gregkh@linuxfoundation.org, muchun.song@linux.dev, joel@joelfernandes.org, christian.koenig@amd.com, daniel@ffwll.ch Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, dri-devel@lists.freedesktop.org, linux-fsdevel@vger.kernel.org, Qi Zheng , Muchun Song Subject: [PATCH v3 3/4] mm: shrinker: remove redundant shrinker_rwsem in debugfs operations Date: Thu, 24 Aug 2023 11:35:38 +0800 Message-Id: <20230824033539.34570-4-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20230824033539.34570-1-zhengqi.arch@bytedance.com> References: <20230824033539.34570-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5AB3F20005 X-Rspam-User: X-Stat-Signature: esjchsahcdzxb8qp6jmsfgocyni9urc8 X-Rspamd-Server: rspam03 X-HE-Tag: 1692848207-634368 X-HE-Meta: U2FsdGVkX1+lYFlSas/k24IPpYr8hK5TgehXYEmORVEQYK1+70yliUjfZnzDwxJ03UAWNg/FtyJcGaie2DvA+h1BkPpv4ekVGmMgp7rmV/Mp/hx5cO6F3deNBbZbf7wCy3yego0E1vF0ar/IBwoXhUFE4DT7DYper+NWuIUKwEGaA/oZwAMv0bY9brfTjeEYGauhmT8isO6nGYsjAz+f/zWB92gBbSwZpSJ90AOPewNL1UcrtwI1PLdk++8pfgkAeF40gaXPYnS8m1oDnx410c9pt38zceoaXUqhQVxr87/UWQcNCTO9un79bjoPoLT94nl//RxWtYTR2MZcsHT2P+naCJ1NkPgx4uMTsnFZH9sftGBPFD4xdthtxX/8kLXkh5cQNjf/KsrM8yPviOsdBuNddoGHzgQiCdtCxSUwsnT6eBb5S24g9t1tYCd36zGX05gDHv6f9bMDkrDOuyrm4vpPcnt+SRW349g5cV6HhPmVnzVDyVhbAa+7pgwIDLe48wNb58YjroRYCQeuZcXBXkz7EvK/4/m1jQNDB0urR5Lb/g89KizQJlq00DbeQ8yJAjp34Vt5iyUIGhMO8kB42rifZwp0Y+/o2uCeMkECkHDFnN/bKAM1He8YWhy4HDwS7af+5HibD+ziv5Ba3HguH06GHnRE1L9l1L+xQId14rZqs1IkNUoxT/+EFBCKdxi1D4aPCtX9m15xIKFgSTyomibS3Jq1kHS5vnSU3mTiX2YMxsNBX7g3q15gidOAQwSExAuWG997ocpH9yotHmAhL0LjDJUDEYY30eSWNsxr9RQQaD0hoKxptlwKSJxZFnMQmKU+e4Cj7mpiMBMBAa9Wpa0+ioSDFL5748D6IZM/HcdnXEzqoO3nRZ4cx/FlHCa/yVj33cSv1qCkjsoxVorkXLQb6ls58NCX9l9ZUibB7IGMgaNdsOF4QJ9lebChaP2aA3Thj5I/2C+CIbv6i2F ySyvbJKv 65tNnBxSZb0TKkwFqrdSXmU8rt7PGOcE/WKEEI3zJBWN2/JT9ebTcYBrWccdMXzxSqTZUcwwPOhkWbPPYuea7AUODOvLst7cj0ebdtmVnEE1KoPwRxhBLCPWi7sNlXlX7y65Asb9309c52txESa8kwk/DAxemItiDSlvSgGpBiLgW6ydskgZSzQoOMWfcxVGs+Fs8AVKP7B+SfTvX+VZoEsV8fTdfQFBIU9i5rcGi+55wjfIMQdvn6iLMZCGBJcbuj/Z0OU1cmuK78MJLnK/7336DqT8uLmdkJJ7mYx6iNUFCi9+XQQRKWNHpTNQZXfqC2IAkudffe7TauTDbxfzeLTvH2vJc66HW9pk30qzYnLeEKYW/msA+96PfjVVdFzlBRqwpehLQTSeTGyRMp+mH4IWZZnulIjRP6hIn6i49rCIo405AxRxInyAErEromNdUZoEZVrY9GXVwcgdp5UxlfoqQFw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The debugfs_remove_recursive() will wait for debugfs_file_put() to return, so the shrinker will not be freed when doing debugfs operations (such as shrinker_debugfs_count_show() and shrinker_debugfs_scan_write()), so there is no need to hold shrinker_rwsem during debugfs operations. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song --- mm/shrinker_debug.c | 16 +--------------- 1 file changed, 1 insertion(+), 15 deletions(-) diff --git a/mm/shrinker_debug.c b/mm/shrinker_debug.c index ee0cddb4530f..e4ce509f619e 100644 --- a/mm/shrinker_debug.c +++ b/mm/shrinker_debug.c @@ -51,17 +51,12 @@ static int shrinker_debugfs_count_show(struct seq_file *m, void *v) struct mem_cgroup *memcg; unsigned long total; bool memcg_aware; - int ret, nid; + int ret = 0, nid; count_per_node = kcalloc(nr_node_ids, sizeof(unsigned long), GFP_KERNEL); if (!count_per_node) return -ENOMEM; - ret = down_read_killable(&shrinker_rwsem); - if (ret) { - kfree(count_per_node); - return ret; - } rcu_read_lock(); memcg_aware = shrinker->flags & SHRINKER_MEMCG_AWARE; @@ -94,7 +89,6 @@ static int shrinker_debugfs_count_show(struct seq_file *m, void *v) } while ((memcg = mem_cgroup_iter(NULL, memcg, NULL)) != NULL); rcu_read_unlock(); - up_read(&shrinker_rwsem); kfree(count_per_node); return ret; @@ -119,7 +113,6 @@ static ssize_t shrinker_debugfs_scan_write(struct file *file, struct mem_cgroup *memcg = NULL; int nid; char kbuf[72]; - ssize_t ret; read_len = size < (sizeof(kbuf) - 1) ? size : (sizeof(kbuf) - 1); if (copy_from_user(kbuf, buf, read_len)) @@ -148,12 +141,6 @@ static ssize_t shrinker_debugfs_scan_write(struct file *file, return -EINVAL; } - ret = down_read_killable(&shrinker_rwsem); - if (ret) { - mem_cgroup_put(memcg); - return ret; - } - sc.nid = nid; sc.memcg = memcg; sc.nr_to_scan = nr_to_scan; @@ -161,7 +148,6 @@ static ssize_t shrinker_debugfs_scan_write(struct file *file, shrinker->scan_objects(shrinker, &sc); - up_read(&shrinker_rwsem); mem_cgroup_put(memcg); return size; From patchwork Thu Aug 24 03:35:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13363411 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42612C3DA6F for ; Thu, 24 Aug 2023 03:36:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A123B28006A; Wed, 23 Aug 2023 23:36:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9C3028E0011; Wed, 23 Aug 2023 23:36:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 83DCD28006A; Wed, 23 Aug 2023 23:36:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 733AC8E0011 for ; Wed, 23 Aug 2023 23:36:58 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 53127401B0 for ; Thu, 24 Aug 2023 03:36:58 +0000 (UTC) X-FDA: 81157586916.14.A289A86 Received: from mail-pg1-f173.google.com (mail-pg1-f173.google.com [209.85.215.173]) by imf24.hostedemail.com (Postfix) with ESMTP id 72E40180002 for ; Thu, 24 Aug 2023 03:36:56 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=JBGdR0nx; spf=pass (imf24.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.173 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692848216; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5OticL2Ym/Xp3/1qxI1A+TfW6cmSsSh/JXNH+7cjvxw=; b=PeSNXGIqhIUpt57UdS9BXf44gjRjvwg5bQz4MDGEMuoXHD4WCJG/8J52O0vpR9iPd9UJFx 33kxgvuwM0OmUmobxyFF5hbXg3rb9Idl+SxvLBUbppl4DJ9OkcELtNeXG4jYl6aJl6GtYn 3chFdFoPFJpIEWPSxf3hZpuhUzCiBVU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692848216; a=rsa-sha256; cv=none; b=qeYO+dAJiHDguxNnbMhj7DGk7H08N1cHmDxf+ywcKgj6TF4tlMPbcYhfJ0EuDmtjGrkmqz k3X8tqVR14kSuyRQzAAV3QSoiHpXsARElqMWJdSxAJJSg44yMQeUjSs7UaNmS8BPsQE30u glZfkbbjLOIfmzvMvbMZeC2Sma5oxPo= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=JBGdR0nx; spf=pass (imf24.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.215.173 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pg1-f173.google.com with SMTP id 41be03b00d2f7-565b053fe58so826509a12.1 for ; Wed, 23 Aug 2023 20:36:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1692848215; x=1693453015; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5OticL2Ym/Xp3/1qxI1A+TfW6cmSsSh/JXNH+7cjvxw=; b=JBGdR0nxHEMd+Kid2GCoKUm5r10wwOvWEU3QhglnYZkx5KhK5zs0/11xXXYBWi+g+j 8yvov2onxhP5YBjhFNrAG5tUOFbQi8pDKuWVKXBch4NhsrUV1d3HPrZyyBeBViChk23I 00rOf0JXjllllSco9fIFNaEQHyHzg4XHW2AYUPn87mU6kLmN9+01K70apj1RVlY0ICzb M5xvWFI2uJk8jt87e8aYoK3l7npYoU+JAnJZX0A/ZFW3a1/8piSWzkjImfUB2jeLy24G 2Mb6pMacsvTxbcPoU//WRqr8Wub3WLxRABjOjRTwybn241j0mUIDG4emEsw/GHzqg5cm oXDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692848215; x=1693453015; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5OticL2Ym/Xp3/1qxI1A+TfW6cmSsSh/JXNH+7cjvxw=; b=DjvjEt4mhj0cuFnEqhaMtp1tL4hu4lNWySW6eyeK06JSNcEKDeUbeyJetz8xy5+2C4 r9QVFxnRKYpYI+6+TZKcIaQ2c9UwDInAzIOSjsMpatAw8BfRHqXVJvdkNou5vJ6WHTK9 unMYIAC7XlRwCmNHdyk89Wxv0mHlbYLTj+rf3DbGUOCkM2b6L6Byd64kPeapOc5+FaqU duGtHcDIirdN9oJbUlQ7uHUEhwfjZJo7clHUD0SVxZKifLHPWi8KqlSC3v+F/3O8oYEB c8FnUEB8fsXsfPCHBATY5W7zh6sFWQRofiW/IHy3WKcdWhe/+q6t1ufLkJGeT6vyds2X JZ9w== X-Gm-Message-State: AOJu0YyKUA24U9t1Orc+zImVuYTji/dUvjg1uAN5g1iEOy5rQC/DuLde OSJycpQEDtHsspJ3c5AfgIDQ/w== X-Google-Smtp-Source: AGHT+IES43unKvAFTLYSHMe/zeQOxomYNWbuowDW3cLv1Miy6NSZqjDlL1eNhKGMV5wMuwnJItY5QQ== X-Received: by 2002:a05:6a00:1892:b0:68a:61fb:8025 with SMTP id x18-20020a056a00189200b0068a61fb8025mr8434319pfh.1.1692848215405; Wed, 23 Aug 2023 20:36:55 -0700 (PDT) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.146]) by smtp.gmail.com with ESMTPSA id p16-20020a62ab10000000b0068b6137d144sm2996570pff.30.2023.08.23.20.36.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Aug 2023 20:36:54 -0700 (PDT) From: Qi Zheng To: akpm@linux-foundation.org, david@fromorbit.com, tkhai@ya.ru, vbabka@suse.cz, roman.gushchin@linux.dev, djwong@kernel.org, brauner@kernel.org, paulmck@kernel.org, tytso@mit.edu, steven.price@arm.com, cel@kernel.org, senozhatsky@chromium.org, yujie.liu@intel.com, gregkh@linuxfoundation.org, muchun.song@linux.dev, joel@joelfernandes.org, christian.koenig@amd.com, daniel@ffwll.ch Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, dri-devel@lists.freedesktop.org, linux-fsdevel@vger.kernel.org, Qi Zheng , Muchun Song , Daniel Vetter Subject: [PATCH v3 4/4] drm/ttm: introduce pool_shrink_rwsem Date: Thu, 24 Aug 2023 11:35:39 +0800 Message-Id: <20230824033539.34570-5-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20230824033539.34570-1-zhengqi.arch@bytedance.com> References: <20230824033539.34570-1-zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 72E40180002 X-Rspam-User: X-Stat-Signature: e9kbnxb5ubmnbbi3ufqsypuizsnd16yq X-Rspamd-Server: rspam03 X-HE-Tag: 1692848216-430139 X-HE-Meta: U2FsdGVkX1/FA+xBg8EFuQC3vxEDKwxYmO6chmcytI0ar7Ujve3k9YEZ9FmiE5n+adLpkxg3sxpV2GYxoi+TkVLoshDBMonKi4TEjtLdh/rQS5GsG5lNh21MLR4uKOt0LDcWpB7Blyo2ZobSKPm6gTvUnnyDJ6FrgXYojb/5U2RMF5aNjh33X+47G/N7Zbok7DVQkQ5FfNFYY/0Xna+8JJBiMW5IFlCydZhD+QO67mBMzQMLbz5HFwc8eW3G+nk+llKRfcjt03r3Qzd+D6Sye54ZYBjeKC/xUaeUaAnGVUF23SZMsGlRJutCtLmJLyKnIG4nj+tWwA6I4A3hBmsj1GenbOeYE7Kz93einK0WtqLOVyqrfzq8RUedPPJnRUZXkiZx121LD/DhXU8r0Zf5ApxYL4inZ0qqS7Z/L3wsC8LCXDt44JsSbIibzbWGbckCA3oUY4chvuvyEWRWx0WQnZvZ2CLgdFnA/QxoonqZuqVn3jIDo6mqIxHn3jcOw/x0QkVjQlnWJ7nvqqBayRaFVgGSy0v0xDiFPlI89yAemzdM1czRHf7ssH+DJ30i6m7koNTz6F+F21EL4XpvUM5jD4Mr4CW8v3bTdyeqtLk2KntdDs41O0u8zWycELZKjqhW1gxSBtczj2TvZbugiqtbvJ7hPAIVGlD1fvOPnjtMbLJ/G/Zoy6i+MigPkbD/1sBR9XhXlOdzoK2F29ncRvepaBgX3KCojUFETI2EGfRRY5dO+1YN9BQtDjO1iH6W8R3IAH6RRfa9DdJxonQDaAP+mfwo4w+cqOKWuWTXFlXGeBJB5Rpf7RvIvjSjra+gz5yeRTvAh+KX+uUEgwkOl87nH04VggriGDlecF8HD/XNszICj5jFbH3kJHo58/brViYMBT9flT7mfw6gujxtmPq8rxEc+cMG/6NqxJwk+ACBB61CYpLHiFIJS+axVmclkT1pAzFZnBOtq209nVZqfIz k+V10KS4 9fetE1g/n5YVzTAM8oMxubVrDiqKWxK1dL1evmuqT8IwFEkg5RhUv4wN4acLEEL5D7lAFOUi+USddRPhtOzTB+TX/M31vzxgQWhfKoIBsUU6cGMMRGPAhBK1wQpdmNKaJLt+7sM0ChJaIemZrKhG4dqsS0xrqxaC6AIQ24V3LTC3nAwgdZQHJpA7AAm7b5P4AOpvxlpDirdsbhP+LCLCC9Pwb9j5JOcSDz7vWlma89qJYtM2icy0LbCwRfF6VabBF4LWSYT7mfDDxS3FPB6koUoGgD6BjT3iphKFpJZ8H9+GB3/M9vqyA+av8OHO3N4iSEDnbZXtJ5p8zB+Xzz4o86wPvHi9wDhJd+ooBz06TZWs0wZEF3tJVKHpwH41K1n30/2H+6IpHgHDMY4liKfG7v/tMjea0axyPvm7jYV93WmTUPnQqfNBN821Hnjp+gI21ne+4wG5gVXdxYxHpZXPCD6UAz/bxksruHBNPvyxS6ivyPd1y8Zb9/8CDHg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, the synchronize_shrinkers() is only used by TTM pool. It only requires that no shrinkers run in parallel. After we use RCU+refcount method to implement the lockless slab shrink, we can not use shrinker_rwsem or synchronize_rcu() to guarantee that all shrinker invocations have seen an update before freeing memory. So we introduce a new pool_shrink_rwsem to implement a private ttm_pool_synchronize_shrinkers(), so as to achieve the same purpose. Signed-off-by: Qi Zheng Reviewed-by: Muchun Song Reviewed-by: Christian König Acked-by: Daniel Vetter --- drivers/gpu/drm/ttm/ttm_pool.c | 17 ++++++++++++++++- include/linux/shrinker.h | 1 - mm/shrinker.c | 15 --------------- 3 files changed, 16 insertions(+), 17 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_pool.c b/drivers/gpu/drm/ttm/ttm_pool.c index cddb9151d20f..648ca70403a7 100644 --- a/drivers/gpu/drm/ttm/ttm_pool.c +++ b/drivers/gpu/drm/ttm/ttm_pool.c @@ -74,6 +74,7 @@ static struct ttm_pool_type global_dma32_uncached[MAX_ORDER + 1]; static spinlock_t shrinker_lock; static struct list_head shrinker_list; static struct shrinker mm_shrinker; +static DECLARE_RWSEM(pool_shrink_rwsem); /* Allocate pages of size 1 << order with the given gfp_flags */ static struct page *ttm_pool_alloc_page(struct ttm_pool *pool, gfp_t gfp_flags, @@ -317,6 +318,7 @@ static unsigned int ttm_pool_shrink(void) unsigned int num_pages; struct page *p; + down_read(&pool_shrink_rwsem); spin_lock(&shrinker_lock); pt = list_first_entry(&shrinker_list, typeof(*pt), shrinker_list); list_move_tail(&pt->shrinker_list, &shrinker_list); @@ -329,6 +331,7 @@ static unsigned int ttm_pool_shrink(void) } else { num_pages = 0; } + up_read(&pool_shrink_rwsem); return num_pages; } @@ -572,6 +575,18 @@ void ttm_pool_init(struct ttm_pool *pool, struct device *dev, } EXPORT_SYMBOL(ttm_pool_init); +/** + * ttm_pool_synchronize_shrinkers - Wait for all running shrinkers to complete. + * + * This is useful to guarantee that all shrinker invocations have seen an + * update, before freeing memory, similar to rcu. + */ +static void ttm_pool_synchronize_shrinkers(void) +{ + down_write(&pool_shrink_rwsem); + up_write(&pool_shrink_rwsem); +} + /** * ttm_pool_fini - Cleanup a pool * @@ -593,7 +608,7 @@ void ttm_pool_fini(struct ttm_pool *pool) /* We removed the pool types from the LRU, but we need to also make sure * that no shrinker is concurrently freeing pages from the pool. */ - synchronize_shrinkers(); + ttm_pool_synchronize_shrinkers(); } EXPORT_SYMBOL(ttm_pool_fini); diff --git a/include/linux/shrinker.h b/include/linux/shrinker.h index 8dc15aa37410..6b5843c3b827 100644 --- a/include/linux/shrinker.h +++ b/include/linux/shrinker.h @@ -103,7 +103,6 @@ extern int __printf(2, 3) register_shrinker(struct shrinker *shrinker, const char *fmt, ...); extern void unregister_shrinker(struct shrinker *shrinker); extern void free_prealloced_shrinker(struct shrinker *shrinker); -extern void synchronize_shrinkers(void); #ifdef CONFIG_SHRINKER_DEBUG extern int __printf(2, 3) shrinker_debugfs_rename(struct shrinker *shrinker, diff --git a/mm/shrinker.c b/mm/shrinker.c index 043c87ccfab4..a16cd448b924 100644 --- a/mm/shrinker.c +++ b/mm/shrinker.c @@ -692,18 +692,3 @@ void unregister_shrinker(struct shrinker *shrinker) shrinker->nr_deferred = NULL; } EXPORT_SYMBOL(unregister_shrinker); - -/** - * synchronize_shrinkers - Wait for all running shrinkers to complete. - * - * This is equivalent to calling unregister_shrink() and register_shrinker(), - * but atomically and with less overhead. This is useful to guarantee that all - * shrinker invocations have seen an update, before freeing memory, similar to - * rcu. - */ -void synchronize_shrinkers(void) -{ - down_write(&shrinker_rwsem); - up_write(&shrinker_rwsem); -} -EXPORT_SYMBOL(synchronize_shrinkers);