From patchwork Mon May 9 18:38:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roman Gushchin X-Patchwork-Id: 12843968 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 300CCC433F5 for ; Mon, 9 May 2022 18:38:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5BE4A6B0072; Mon, 9 May 2022 14:38:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 56E156B0073; Mon, 9 May 2022 14:38:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 435C46B0074; Mon, 9 May 2022 14:38:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 344616B0072 for ; Mon, 9 May 2022 14:38:31 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id EC5EF3137C for ; Mon, 9 May 2022 18:38:30 +0000 (UTC) X-FDA: 79447065180.14.EB7691F Received: from out0.migadu.com (out0.migadu.com [94.23.1.103]) by imf21.hostedemail.com (Postfix) with ESMTP id 016321C00A4 for ; Mon, 9 May 2022 18:38:22 +0000 (UTC) X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1652121507; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=wg7X+hDIQAnxZ2mBq8fPCOHY2m0N1zsR1d7bRCthk1o=; b=hhyxycifCkKzEFBEZ0ROTDSR94ZTAErkyneO1P5lkvPzD/OcFVlaht5ix+9lK53TKsSTDf TpIM/rq/Rc0SwbHQ6xGzba9e6sTgccp9TX0BP1+4UDMBBjtzmwdOWwgGNeucfkhLpOCENb wO1y9YFIJSPuLshp3Op8nj9AATOzo5Y= From: Roman Gushchin To: Andrew Morton , linux-mm@kvack.org Cc: Dave Chinner , linux-kernel@vger.kernel.org, Kent Overstreet , Hillf Danton , Christophe JAILLET , Roman Gushchin Subject: [PATCH v3 0/6] mm: introduce shrinker debugfs interface Date: Mon, 9 May 2022 11:38:14 -0700 Message-Id: <20220509183820.573666-1-roman.gushchin@linux.dev> MIME-Version: 1.0 X-Migadu-Flow: FLOW_OUT X-Migadu-Auth-User: linux.dev X-Rspamd-Queue-Id: 016321C00A4 X-Stat-Signature: b865txegxoqmw5a5bc4mrsebr9wkgr4h X-Rspam-User: Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=hhyxycif; spf=pass (imf21.hostedemail.com: domain of roman.gushchin@linux.dev designates 94.23.1.103 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Rspamd-Server: rspam09 X-HE-Tag: 1652121502-717097 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are 50+ different shrinkers in the kernel, many with their own bells and whistles. Under the memory pressure the kernel applies some pressure on each of them in the order of which they were created/registered in the system. Some of them can contain only few objects, some can be quite large. Some can be effective at reclaiming memory, some not. The only existing debugging mechanism is a couple of tracepoints in do_shrink_slab(): mm_shrink_slab_start and mm_shrink_slab_end. They aren't covering everything though: shrinkers which report 0 objects will never show up, there is no support for memcg-aware shrinkers. Shrinkers are identified by their scan function, which is not always enough (e.g. hard to guess which super block's shrinker it is having only "super_cache_scan"). To provide a better visibility and debug options for memory shrinkers this patchset introduces a /sys/kernel/debug/shrinker interface, to some extent similar to /sys/kernel/slab. For each shrinker registered in the system a directory is created. As now, the directory will contain only a "scan" file, which allows to get the number of managed objects for each memory cgroup (for memcg-aware shrinkers) and each numa node (for numa-aware shrinkers on a numa machine). Other interfaces might be added in the future. To make debugging more pleasant, the patchset also names all shrinkers, so that debugfs entries can have meaningful names. v3: 1) separated the "scan" part into a separate patch, by Dave 2) merged *_memcg, *_node and *_memcg_node interfaces, by Dave 3) shrinkers naming enhancements, by Christophe and Dave 4) added signal_pending() check, by Hillf 5) enabled by default, by Dave v2: 1) switched to debugfs, suggested by Mike, Andrew, Greg and others 2) switched to seq_file API for output, no PAGE_SIZE limit anymore, by Andrew 3) switched to down_read_killable(), suggested by Hillf 4) dropped stateful filtering and "freed" returning, by Kent 5) added docs, by Andrew 6) added memcg_shrinker.py tool rfc: https://lwn.net/Articles/891542/ Roman Gushchin (6): mm: memcontrol: introduce mem_cgroup_ino() and mem_cgroup_get_from_ino() mm: shrinkers: introduce debugfs interface for memory shrinkers mm: shrinkers: provide shrinkers with names mm: docs: document shrinker debugfs tools: add memcg_shrinker.py mm: shrinkers: add scan interface for shrinker debugfs Documentation/admin-guide/mm/index.rst | 1 + .../admin-guide/mm/shrinker_debugfs.rst | 131 ++++++++ arch/x86/kvm/mmu/mmu.c | 2 +- drivers/android/binder_alloc.c | 2 +- drivers/gpu/drm/i915/gem/i915_gem_shrinker.c | 3 +- drivers/gpu/drm/msm/msm_gem_shrinker.c | 2 +- .../gpu/drm/panfrost/panfrost_gem_shrinker.c | 2 +- drivers/gpu/drm/ttm/ttm_pool.c | 2 +- drivers/md/bcache/btree.c | 2 +- drivers/md/dm-bufio.c | 2 +- drivers/md/dm-zoned-metadata.c | 2 +- drivers/md/raid5.c | 2 +- drivers/misc/vmw_balloon.c | 2 +- drivers/virtio/virtio_balloon.c | 2 +- drivers/xen/xenbus/xenbus_probe_backend.c | 2 +- fs/btrfs/super.c | 2 + fs/erofs/utils.c | 2 +- fs/ext4/extents_status.c | 3 +- fs/f2fs/super.c | 2 +- fs/gfs2/glock.c | 2 +- fs/gfs2/main.c | 2 +- fs/jbd2/journal.c | 2 +- fs/mbcache.c | 2 +- fs/nfs/nfs42xattr.c | 7 +- fs/nfs/super.c | 2 +- fs/nfsd/filecache.c | 2 +- fs/nfsd/nfscache.c | 2 +- fs/quota/dquot.c | 2 +- fs/super.c | 6 +- fs/ubifs/super.c | 2 +- fs/xfs/xfs_buf.c | 2 +- fs/xfs/xfs_icache.c | 2 +- fs/xfs/xfs_qm.c | 2 +- include/linux/memcontrol.h | 21 ++ include/linux/shrinker.h | 31 +- kernel/rcu/tree.c | 2 +- lib/Kconfig.debug | 9 + mm/Makefile | 1 + mm/huge_memory.c | 4 +- mm/memcontrol.c | 23 ++ mm/shrinker_debug.c | 285 ++++++++++++++++++ mm/vmscan.c | 64 +++- mm/workingset.c | 2 +- mm/zsmalloc.c | 2 +- net/sunrpc/auth.c | 2 +- tools/cgroup/memcg_shrinker.py | 71 +++++ 46 files changed, 675 insertions(+), 47 deletions(-) create mode 100644 Documentation/admin-guide/mm/shrinker_debugfs.rst create mode 100644 mm/shrinker_debug.c create mode 100755 tools/cgroup/memcg_shrinker.py