From patchwork Mon Jul 15 20:29:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13733871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B89CBC3DA4B for ; Mon, 15 Jul 2024 20:29:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0DC986B00A5; Mon, 15 Jul 2024 16:29:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F24486B00AA; Mon, 15 Jul 2024 16:29:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B55286B00AB; Mon, 15 Jul 2024 16:29:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 762836B009B for ; Mon, 15 Jul 2024 16:29:43 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 0E859A01F2 for ; Mon, 15 Jul 2024 20:29:43 +0000 (UTC) X-FDA: 82343127846.21.4FFA45A Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf14.hostedemail.com (Postfix) with ESMTP id BFC16100011 for ; Mon, 15 Jul 2024 20:29:40 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=IchGYqVK; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=evMTmCrr; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=IchGYqVK; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=evMTmCrr; spf=pass (imf14.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721075342; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=63ptlEDSf2NF4mqHOwhlrxuC2a4rLEbYrta5gvgyEc0=; b=1YBsdWbzabCYpwBObTxiR617KIpsdd68+c36tvjVE7HXnU+BxzG2iFnJiOJlX+9VQeloKC KerIAA34CG08+qmBpJM8OPv6D+xkagMvLFQ7pj98RkABuuu5UNOVr/QFxZJIPBrV4hBBgo ok/SrgwePki92JclZuiLitf7UvNINVw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721075342; a=rsa-sha256; cv=none; b=dKMMzmkINKTz3mUsf2RseTiXFwkvErfoF6OyOprNKMuFEf+aPx5YC1DQFthQ5kgnu74Zel Je/atJQhoWkcPIFfjsw2ZbOFfmE/X51XrkZJudPgqaWyLLYxKfChe4TwyVerprCE+AmCx+ S33CwFLZd07PaW/qZCTxfSglSVTrCrE= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=IchGYqVK; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=evMTmCrr; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=IchGYqVK; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=evMTmCrr; spf=pass (imf14.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id BA01B210F6; Mon, 15 Jul 2024 20:29:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1721075378; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=63ptlEDSf2NF4mqHOwhlrxuC2a4rLEbYrta5gvgyEc0=; b=IchGYqVKTSTLgxc6iItiNxB8xYcgyho9SGEIHdG4AvGblPLUEGncmz+QAw9gCZeuYszYBo hDpm2S4wepd1ry/XsrtRiysjqoo2aldpOROpoUxAii8f9lIRXpgYcd3Oh+mmqeLWCu2J2D eXC1GfxqvuZMh8bTQhOMVpnEuPoDI3w= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1721075378; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=63ptlEDSf2NF4mqHOwhlrxuC2a4rLEbYrta5gvgyEc0=; b=evMTmCrrFOKvt+r76bJUkmHSGlLuXlmEdht9Fxt/VkvV8cIaoVObsQSvRCCjheyDPipECt Snp+IvBW3laNcQCg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1721075378; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=63ptlEDSf2NF4mqHOwhlrxuC2a4rLEbYrta5gvgyEc0=; b=IchGYqVKTSTLgxc6iItiNxB8xYcgyho9SGEIHdG4AvGblPLUEGncmz+QAw9gCZeuYszYBo hDpm2S4wepd1ry/XsrtRiysjqoo2aldpOROpoUxAii8f9lIRXpgYcd3Oh+mmqeLWCu2J2D eXC1GfxqvuZMh8bTQhOMVpnEuPoDI3w= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1721075378; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=63ptlEDSf2NF4mqHOwhlrxuC2a4rLEbYrta5gvgyEc0=; b=evMTmCrrFOKvt+r76bJUkmHSGlLuXlmEdht9Fxt/VkvV8cIaoVObsQSvRCCjheyDPipECt Snp+IvBW3laNcQCg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 979351395F; Mon, 15 Jul 2024 20:29:38 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 6Em9JLKGlWbvUwAAD6G6ig (envelope-from ); Mon, 15 Jul 2024 20:29:38 +0000 From: Vlastimil Babka Date: Mon, 15 Jul 2024 22:29:27 +0200 Subject: [PATCH RFC 1/6] mm, slab: make caches with refcount of 0 unmergeable MIME-Version: 1.0 Message-Id: <20240715-b4-slab-kfree_rcu-destroy-v1-1-46b2984c2205@suse.cz> References: <20240715-b4-slab-kfree_rcu-destroy-v1-0-46b2984c2205@suse.cz> In-Reply-To: <20240715-b4-slab-kfree_rcu-destroy-v1-0-46b2984c2205@suse.cz> To: "Paul E. McKenney" , Joel Fernandes , Josh Triplett , Boqun Feng , Christoph Lameter , David Rientjes Cc: Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Julia Lawall , Jakub Kicinski , "Jason A. Donenfeld" , "Uladzislau Rezki (Sony)" , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.0 X-Stat-Signature: iwz3q66tfytp9xod9uxr6tu3974uj8wb X-Rspamd-Queue-Id: BFC16100011 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1721075380-346428 X-HE-Meta: U2FsdGVkX18N1fF+JRmLSMS/giQ2flP9C/OG8ZcjNnMPL2U4v+l/LbYcJmgpZZk01UA/asApvBv4G6NJ2RcL1mGROAh3nACw/f7USWqSy5tT9DxfiUYtLtDJYcA8bNJnJrIypebndEdO6wbFuR0iGV7r55gsi2n57OS+/8AsESDynqT7YZntudkEq6o+XdJ1WTfAZoX8agln6gkW5+46fO4Ml3G5ly+7g+6W3Iz3Dlxdzwbae3ic2h1YDsAfqG6f6/jMe2GzU420hbQ1g4RrhzJ2CR47TgFMx2CADZe24CyfGLUYYTHJeDpeIwCoqs7XugT9Ez1KFbhbkj2v18IrYuwLAu/q2+Ozv+RS+2Y896lEfI/7QSG4dBI1yrjN35HhjYuWTZEfNGGr2fUHMpBlcR9if7c37mmWsOvEm8GkrDz66J9/lJeV2YKiL2GzdbKmS8/UTnnFybFjmtQb2oTnNGvktjTQiLHBZ1b/mq8qvuK9lHaOl6aNhinooR1KYPgXlodGjSLEbZur8Tg+7EYsuzfVB/eDEgp5gjXbp8L7Oamf166NXzLoqWObXDgDMyXvEmuV0mPSe/2Bx8jzOi/cQIzTKlp3roje5kNFKBD3I7R3jQrtVSt/Ag8ozBTMeEBo3U2PXh0sjTduPGLu+7CWhGGY/05IWkS7C+BjKcjADCPgukoninqHhssLmB5otOJ1fQGE8wiFckXFVkkTajJYk0Iqe3+KDXzZEWo8A7CNLq/cqfdCWh4x0s5HkZa27SpXNbPtvHWrVtHOXCl+iJyuwdvr7Kfpz9q3bjlEFjD7evXwB2rVGUnh8B6eyPkCEYH0eQChcgFL8ZK5tlv6gQvLpN4ynVWR5FwaIMBrGJrKekhnZ8rNvgXtDewsQKFzhMP7CLk+Raf9C/V0ed+MgYIZ7QneqOufrrJPXRy/+G8pevjTIXX/X65hLQ60ZqlDnP5mbgY/QvZcVcNlm7kMB2e yjz+/Adq iQ/hb/s3c87UDbFs0oeMCA19Jf1jREMWuec9GyKJPKR96C/GpB/6vhHrLUTLXTGggKPCQro9UIlibH6liSPly/+105uXRHXPQHVgoD52PIncdNaGRFCF+4O0lqCdcxRqI94/rofrxBE7bcBBKeL2d5oNPT2OimLZhDSXJ0nRhPPBxQg6d4Og9JY/A6SIgtOhK8zuIthGu8uHnrnFk/Uono+uTCxr9Tt1oKC4S7I4TlrWerAWZ8E48JgxD00lb47deGaEVkpYwxSp53y+qpaahWxEx5aEZdL0kvJgr8W4EP6uvsBA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Slab caches with refcount 0 are in the process of being destroyed so it's undesirable for new caches to attempt merging with them. A synchronous destruction happens under slab_mutex thus excluding concurrent cache creation and merging. Full destruction of SLAB_TYPESAFE_BY_RCU caches might be delayed, but the cache is still taken off the slab_caches list immediately, thus unreachable by cache creation. However a cache where __kmem_cache_shutdown() fails because it contains objects that were not freed (due to a bug in the cache user) will be left on the slab_caches list and might be considered for merging. Also the following patches will introduce a possibility of a cache with refcount 0 being temporarily reachable on the slab_list even in case of no bugs, due to kfree_rcu() in flight. For these reasons, prevent merging with caches that have zero refcount. Signed-off-by: Vlastimil Babka Acked-by: David Rientjes --- mm/slab_common.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 70943a4c1c4b..3ba205bda95d 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -150,9 +150,11 @@ int slab_unmergeable(struct kmem_cache *s) #endif /* - * We may have set a slab to be unmergeable during bootstrap. + * We may have set a cache to be unmergeable (-1) during bootstrap. + * 0 is for cache being destroyed asynchronously, or cache that failed + * to destroy due to outstanding objects. */ - if (s->refcount < 0) + if (s->refcount <= 0) return 1; return 0; From patchwork Mon Jul 15 20:29:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13733870 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A72BC3DA59 for ; Mon, 15 Jul 2024 20:29:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D11006B00A0; Mon, 15 Jul 2024 16:29:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C66E96B009E; Mon, 15 Jul 2024 16:29:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A1BFC6B00AA; Mon, 15 Jul 2024 16:29:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 76D716B009E for ; Mon, 15 Jul 2024 16:29:43 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 001E7140238 for ; Mon, 15 Jul 2024 20:29:42 +0000 (UTC) X-FDA: 82343127804.03.535ED42 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf04.hostedemail.com (Postfix) with ESMTP id C89804001D for ; Mon, 15 Jul 2024 20:29:40 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=ANbEASdW; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=4eHOsmkR; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=ANbEASdW; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=4eHOsmkR; spf=pass (imf04.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721075361; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hyldqlnDVTk50xyhWcs4CPBsygxILHPWHa8A+BcY6BU=; b=vfMvi9RzG/idelA3aLJGg87s7pO370iSY7BNf4fDyihcMeXV+ORC0aC2MJaZZOXFCyXoXK T4Wvzy4AyDSwY817amwKQbKhcQaOijNLc0cyHLXJNeVw+HIW14Fewkhqt35JLH5f/GtXlF z5GPNjNB82yUKJaJhhbsu/iFBaJh9gw= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=ANbEASdW; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=4eHOsmkR; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=ANbEASdW; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=4eHOsmkR; spf=pass (imf04.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721075361; a=rsa-sha256; cv=none; b=XHy5QPvXxOVMQMWZhx5M9rEogX4mRJKhBowA1IBxnlcEyuUsCHCUC+TR3rlL8v8vHPZ2E3 lVJ3s+M6yupQsC7tiCduu7PS6WG/OJnstoxIIecklmoypypvwV6oH+50zqXEp6cUJ2GkPC wAUt85WNpVwICF/5ixPkn9dZ18Gt7No= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id D58861F843; Mon, 15 Jul 2024 20:29:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1721075378; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hyldqlnDVTk50xyhWcs4CPBsygxILHPWHa8A+BcY6BU=; b=ANbEASdWGMdwl5qIXdPygLN5quDOUggtjyZj/rdVEODhlm+iIA64uMIgVwJdDUCncYcZI/ tNq53zYt3Ul/jLQpY+x02fL4MVwUY8mNk5iXoMQnwDUWRpXZfQyw2WHe5Tyltx22YZ/FyD 08Qt9lzqZt5wXVKR6LmkjZKOBINn3gw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1721075378; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hyldqlnDVTk50xyhWcs4CPBsygxILHPWHa8A+BcY6BU=; b=4eHOsmkRnhgqg1SSKo/heKcjThC68ZpN46jp+qFZPiqyFx4W5r7bcXY6t1L+lrj+mOkH2t quc3a/glQGts+RAQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1721075378; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hyldqlnDVTk50xyhWcs4CPBsygxILHPWHa8A+BcY6BU=; b=ANbEASdWGMdwl5qIXdPygLN5quDOUggtjyZj/rdVEODhlm+iIA64uMIgVwJdDUCncYcZI/ tNq53zYt3Ul/jLQpY+x02fL4MVwUY8mNk5iXoMQnwDUWRpXZfQyw2WHe5Tyltx22YZ/FyD 08Qt9lzqZt5wXVKR6LmkjZKOBINn3gw= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1721075378; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hyldqlnDVTk50xyhWcs4CPBsygxILHPWHa8A+BcY6BU=; b=4eHOsmkRnhgqg1SSKo/heKcjThC68ZpN46jp+qFZPiqyFx4W5r7bcXY6t1L+lrj+mOkH2t quc3a/glQGts+RAQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B38921396E; Mon, 15 Jul 2024 20:29:38 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id CBWUK7KGlWbvUwAAD6G6ig (envelope-from ); Mon, 15 Jul 2024 20:29:38 +0000 From: Vlastimil Babka Date: Mon, 15 Jul 2024 22:29:28 +0200 Subject: [PATCH RFC 2/6] mm, slab: always maintain per-node slab and object count MIME-Version: 1.0 Message-Id: <20240715-b4-slab-kfree_rcu-destroy-v1-2-46b2984c2205@suse.cz> References: <20240715-b4-slab-kfree_rcu-destroy-v1-0-46b2984c2205@suse.cz> In-Reply-To: <20240715-b4-slab-kfree_rcu-destroy-v1-0-46b2984c2205@suse.cz> To: "Paul E. McKenney" , Joel Fernandes , Josh Triplett , Boqun Feng , Christoph Lameter , David Rientjes Cc: Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Julia Lawall , Jakub Kicinski , "Jason A. Donenfeld" , "Uladzislau Rezki (Sony)" , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.0 X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: C89804001D X-Stat-Signature: yowb3bameiecxqhmxp34p6pgse9efr5u X-Rspam-User: X-HE-Tag: 1721075380-290124 X-HE-Meta: U2FsdGVkX19zPuphRY0b+yHHLGJGQBi8BfYtDOyEtMQWkt0xmJA8d1lk1sDONCRrNLOIms+535EP3j97gJjnCrG/OrIo62EQFpD1OlEiYeI5dVO/Xvy8KUEkDmZB1ewiMFXSYtKtZZjs2bTEgaAc+PAuPlVo9dhvNUaleL8b+8v5IO5JVhQ6mcNEvRaIueGsbIhDC7oba22o9PG9lXKGumaqfT3jiHJz290ZGD71UP5Mxe3jc0/B5X3DTsy+uixtdtw5Y68jUGoj5chx3VCUGusy8u4GRy0gdXR59w9mlIPpK0bOZBmnpGu4IVztgUsvOj2wM0A9TmNTgXby/3BC1jWsncHTG2xngraUUIGEQTvK6TRCfyQ7I0XiFALWlM7TWQsmpJsGMntOR9gHWAyD0yl7TSgSa1G6BSUvZJMQIEzs/n0Uo1mNlZPSZs9X0IBUn2oJZSw4rYuBRBEbPEeYyDjN75PlZtwhdMF5+KadHoZwVgJWfCtaF2xAYFbwVwtzBUJzBl/xxo+jNryLH2HiQrWuBYLSpPF0c5ZCjez+fQIa6h/QOk9XaLjJaDoPY9Zt9rQsJfYqzBAh7j/LWbFtAjhtMsQodkqBAwxN7iyizW+pR5CXhEekjXA6vbtIV/NEwVyuxI5e0A2jFOGLHOqydqSgWaklg+FTmbDAIaGzs6f/HBp6m/WCatrA8TP3/eJQA5bIiq+tv4r2coOtB39L5LWRJYn3J33CdQ2U6SQ6qfWl7K8SiouFzFLRw8GIi2SqwiL+Atc929CX8lQs7Znbjo/e+fzF7OdhcBALRK6L3aNMSOe0yaQ7nIAb0Q95kexdd1nKWVGkD9Ahr+OCC7/kdHD6OxNr89pHBLmcJHzig5qyiKWmx5UbQe1lnuR+uGPSNFxt8zky4kWBUliCfzSHv5Y9FLiRqGM+OqgkcRY8NeU8bGtRVbxQKmyOPXuRTKiay2Kzo0ocizscApjbkw4 Nrs6YPWk Uel9lVfJ1WJJpgX9hOux9H5tKER3QUqMEVGdmWIqDevDgRlekjQQ5mW9QH9SAFxr+2vtzNnDgD06MgX1iLm9XaJjsgsxhomQ0unCmPff/RO+b8Joi9bwfxxTuUGTgche1MvxATvAHYNCyxRndcfa16QNhPNiEfom59ZJkInY4dU5OAtZaS2PdTWuYyDOB6d9JlzpCirjmh4Tyi9ML/tCBHmldfNSCDmmhdxL5bLGZgnUKgZSEX8+hGH032rOpITOLHujxeBqjf4z7Ei8sTjDv6lSgoPusVIt4496XqqtyxsjqIW8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently SLUB counts per-node slabs and total objects only with CONFIG_SLUB_DEBUG, in order to minimize overhead. However, the detection in __kmem_cache_shutdown() whether there are no outstanding object relies on the per-node slab count (node_nr_slabs()) so it may be unreliable without CONFIG_SLUB_DEBUG. Thus we might be failing to warn about such situations, and instead destroy a cache while leaving its slab(s) around (due to a buggy slab user creating such a scenario, not in normal operation). We will also need node_nr_slabs() to be reliable in the following work to gracefully handle kmem_cache_destroy() with kfree_rcu() objects in flight. Thus make the counting of per-node slabs and objects unconditional. Note that CONFIG_SLUB_DEBUG is the default anyway, and the counting is done only when allocating or freeing a slab page, so even in !CONFIG_SLUB_DEBUG configs the overhead should be negligible. Signed-off-by: Vlastimil Babka Acked-by: David Rientjes --- mm/slub.c | 49 +++++++++++++++++++++---------------------------- 1 file changed, 21 insertions(+), 28 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 829a1f08e8a2..aa4d80109c49 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -426,9 +426,9 @@ struct kmem_cache_node { spinlock_t list_lock; unsigned long nr_partial; struct list_head partial; -#ifdef CONFIG_SLUB_DEBUG atomic_long_t nr_slabs; atomic_long_t total_objects; +#ifdef CONFIG_SLUB_DEBUG struct list_head full; #endif }; @@ -438,6 +438,26 @@ static inline struct kmem_cache_node *get_node(struct kmem_cache *s, int node) return s->node[node]; } +static inline unsigned long node_nr_slabs(struct kmem_cache_node *n) +{ + return atomic_long_read(&n->nr_slabs); +} + +static inline void inc_slabs_node(struct kmem_cache *s, int node, int objects) +{ + struct kmem_cache_node *n = get_node(s, node); + + atomic_long_inc(&n->nr_slabs); + atomic_long_add(objects, &n->total_objects); +} +static inline void dec_slabs_node(struct kmem_cache *s, int node, int objects) +{ + struct kmem_cache_node *n = get_node(s, node); + + atomic_long_dec(&n->nr_slabs); + atomic_long_sub(objects, &n->total_objects); +} + /* * Iterator over all nodes. The body will be executed for each node that has * a kmem_cache_node structure allocated (which is true for all online nodes) @@ -1511,26 +1531,6 @@ static void remove_full(struct kmem_cache *s, struct kmem_cache_node *n, struct list_del(&slab->slab_list); } -static inline unsigned long node_nr_slabs(struct kmem_cache_node *n) -{ - return atomic_long_read(&n->nr_slabs); -} - -static inline void inc_slabs_node(struct kmem_cache *s, int node, int objects) -{ - struct kmem_cache_node *n = get_node(s, node); - - atomic_long_inc(&n->nr_slabs); - atomic_long_add(objects, &n->total_objects); -} -static inline void dec_slabs_node(struct kmem_cache *s, int node, int objects) -{ - struct kmem_cache_node *n = get_node(s, node); - - atomic_long_dec(&n->nr_slabs); - atomic_long_sub(objects, &n->total_objects); -} - /* Object debug checks for alloc/free paths */ static void setup_object_debug(struct kmem_cache *s, void *object) { @@ -1871,13 +1871,6 @@ slab_flags_t kmem_cache_flags(slab_flags_t flags, const char *name) #define disable_higher_order_debug 0 -static inline unsigned long node_nr_slabs(struct kmem_cache_node *n) - { return 0; } -static inline void inc_slabs_node(struct kmem_cache *s, int node, - int objects) {} -static inline void dec_slabs_node(struct kmem_cache *s, int node, - int objects) {} - #ifndef CONFIG_SLUB_TINY static bool freelist_corrupted(struct kmem_cache *s, struct slab *slab, void **freelist, void *nextfree) From patchwork Mon Jul 15 20:29:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13733875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DCF50C3DA4B for ; Mon, 15 Jul 2024 20:29:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D8F8C6B009F; Mon, 15 Jul 2024 16:29:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A1BF36B00AE; Mon, 15 Jul 2024 16:29:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3BAB06B009F; Mon, 15 Jul 2024 16:29:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id AD0E16B00A8 for ; Mon, 15 Jul 2024 16:29:43 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 18F7C80242 for ; Mon, 15 Jul 2024 20:29:43 +0000 (UTC) X-FDA: 82343127846.15.07D6551 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf06.hostedemail.com (Postfix) with ESMTP id AE156180016 for ; Mon, 15 Jul 2024 20:29:40 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=hSGyH+m5; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=kAIi0fhY; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=hSGyH+m5; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=kAIi0fhY; spf=pass (imf06.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721075361; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tGLdjgoDRMwVFM9fTSoQ/YTustjDFDN8l8I6qmGg0KM=; b=bZceEPE4MIASaJkNCfYq/xTSkaj2xYAbMPirMY9HzUUrgRxs5axQAtQN5eqZo4W8DMkkG1 mIP6YtgKUhSIcfuM8eqL2DIYsWHW8y4XOqUzlc488UbeoBYJSqMcqDFqi72YLAc/Ux3Ohp Lv1Rudfms18adVXISTbug7yb0oQ98+g= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=hSGyH+m5; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=kAIi0fhY; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=hSGyH+m5; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=kAIi0fhY; spf=pass (imf06.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721075361; a=rsa-sha256; cv=none; b=zk48ScqcEkkhpTnNXuGzo/3Zj1d3TJr1aC9Vhkt+uss36dh0cFsfSiNGEEtAAjMWIWF2PB EZXGwaGjCI+Uk9pU7hofi19240Lmr6ERPAPtZ7UGHBHSPqNPzWl06YnEFt0y3kpl7yqM7p trwfDWYJqssEY8oZpD/V4ZyrxDLqZqw= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id EF0BC1F849; Mon, 15 Jul 2024 20:29:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tGLdjgoDRMwVFM9fTSoQ/YTustjDFDN8l8I6qmGg0KM=; b=hSGyH+m5FYyRHvKzXhLIpQGye0mEwLbM3RqIof47AS3U2ab1U27QYUbjQLDHoBm5COD+rq Ceso8rzC8jfjximaXlOSp7oRoim+/TN/R0Uq/5a8xFXCECpuSwariIoyRIXR2Ph34PPzs1 QVSlCwrPIzslLN7yLti+5OJPxWO/dfQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tGLdjgoDRMwVFM9fTSoQ/YTustjDFDN8l8I6qmGg0KM=; b=kAIi0fhYDpGbjURluCU3wBE0Tgnqg89fNVrOr5lDUl5eiKjVINACSfe8OycWeAGYyY3d+5 yNQIKHir85e7KmAg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tGLdjgoDRMwVFM9fTSoQ/YTustjDFDN8l8I6qmGg0KM=; b=hSGyH+m5FYyRHvKzXhLIpQGye0mEwLbM3RqIof47AS3U2ab1U27QYUbjQLDHoBm5COD+rq Ceso8rzC8jfjximaXlOSp7oRoim+/TN/R0Uq/5a8xFXCECpuSwariIoyRIXR2Ph34PPzs1 QVSlCwrPIzslLN7yLti+5OJPxWO/dfQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tGLdjgoDRMwVFM9fTSoQ/YTustjDFDN8l8I6qmGg0KM=; b=kAIi0fhYDpGbjURluCU3wBE0Tgnqg89fNVrOr5lDUl5eiKjVINACSfe8OycWeAGYyY3d+5 yNQIKHir85e7KmAg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id CEB9713A63; Mon, 15 Jul 2024 20:29:38 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id QGIrMrKGlWbvUwAAD6G6ig (envelope-from ); Mon, 15 Jul 2024 20:29:38 +0000 From: Vlastimil Babka Date: Mon, 15 Jul 2024 22:29:29 +0200 Subject: [PATCH RFC 3/6] mm, slab: unlink sysfs and debugfs immediately MIME-Version: 1.0 Message-Id: <20240715-b4-slab-kfree_rcu-destroy-v1-3-46b2984c2205@suse.cz> References: <20240715-b4-slab-kfree_rcu-destroy-v1-0-46b2984c2205@suse.cz> In-Reply-To: <20240715-b4-slab-kfree_rcu-destroy-v1-0-46b2984c2205@suse.cz> To: "Paul E. McKenney" , Joel Fernandes , Josh Triplett , Boqun Feng , Christoph Lameter , David Rientjes Cc: Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Julia Lawall , Jakub Kicinski , "Jason A. Donenfeld" , "Uladzislau Rezki (Sony)" , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.0 X-Rspamd-Action: no action X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: AE156180016 X-Stat-Signature: 4t95b1r8b8hy6fycaxkwtr9fyfhrtira X-Rspam-User: X-HE-Tag: 1721075380-931914 X-HE-Meta: U2FsdGVkX19m8fKh3WNUmovr029qQHnvyPzxB2eUfqAqyb3vYsvfkonBOkAdCBRbklHZ67X2Y6TAoTAj+aEslWQdDxTQW8reDAuVqHA7iJm5+SwLVFpJZVp+jb8K9z+PHgL3YgQNl1mahFRKpP9RT90xiDl0CAT0K3UhgKGeVeRqHyv87viViWNiOEDs/ZOLayksFfA28q1Aj1QKzU0aqg4bPNfSGrnqvOS5I26niDsLDuJJdEDn25zlZnM312Cy7Bv0TR+N+Qbel/ZP2A00zlhwdoxdNTLTWOkbNH/PlvWxcYTZA/nlgXBjHW4pwAiQ8AfXKRgxO/Wt4sEDe8Y6DqIc0nJH3PeIW7LIupy+p72ucIfoYU/1tPtQpShenzqq+neBaWm2wxOyIkF+mvsCGI+w6E12J/ty+zmkHDvkZ723Ej3LRyB+WsPTUrTtsQ0EPMHYFc5ijM1DvYAb42gCjDU7PBYqdTHplQIltlPBww8SYHTf9XKLt+5YUvHjEI0xLBzpMx3sMBw7/T97FI1STrqt6/fmwdNDBk1vf/6OoKsWMVbLNT1cmV1Y+qj+O8F5tLJ/Dozu+JEGmxMJoSSwD4YCMfvBuemZd97x1+7m5fzCvuyVESl63cz1aSoCCLycviapYmhjqookGDxIURmPxw7n0cNBviH8VSo4TmSZ3DeoXiZmPlHChbqtmIMYOW7PyKPg6GFCw6VHzhe3yVVn/9c6yd/0uBN3DOo35GBVuGS3UvNCUdQ5fzo9rxxPJGMMHsEnp9m5CAl7T4EwfNpQ+NBjyRyDPDqYbfoaRS4vLqvC6wlPZyCjJz4s1aN2BZjRyjWQJcXteWPX/wdm+rAjIzfoMXWjpmRTNdQ5FYtS9rVuW0VlKq3AlS5/gQoY1gBvuv52B6QISV3rS39ff1SU3oDcEf26SSl0bO53ET+B9Nb3DIT60Y2WTz2Ky75mzSS8d9qY+9iIaKHK/BO9HON JXTQiwmn INuBp149BAINyKqldjE3e9jaH0jYjtS0BquRJqC7Qc9+jkk3jXn+2NECJ9TPqh2myENM6DM0HkYKJu8YEWVrr7H871/+F7uTzM+JaDmmx+yw432orcyUyX2lawFLqJxfxZ5f8Qr6wVinzzIGL3ISRs+NdGachRSSR6EXWikcf+Ua6uv2KeTpGKfO1t76hARiKqklC0sC+rQkgYoNDL0ecxMI1j6ma9kJMgi0QTCS/N7DRl4a+sBZiPYGUJVn/HNVKv+wvHTLtipOmSVlfkGR2pSm69vCTlYoUY5ePfqXPXvYzf3U= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kmem_cache_destroy() includes removing the associated sysfs and debugfs directories. Currently this might not happen immediately when: - the cache is SLAB_TYPESAFE_BY_RCU and the cleanup is delayed, including the directores removal - __kmem_cache_shutdown() fails due to outstanding objects - the directories remain indefinitely When a cache is recreated with the same name, such as due to module unload followed by a load, the directories will fail to be recreated for the new instance of the cache due to the old directories being present. We also want to add another possibility of delayed cleanup due to kfree_rcu() in flight so let's fix this first and have the directories removed immediately in kmem_cache_destroy() and regardless of __kmem_cache_shutdown() success. This should not make debugging harder if __kmem_cache_shutdown() fails, because a detailed report of outstanding objects is printed into dmesg already due to the failure. Note the record in /proc/slabinfo will remain until the cleanup is finished (or indefinitely if __kmem_cache_shutdown() fails) but that does not prevent a new record to be added for a new cache instance. Signed-off-by: Vlastimil Babka --- mm/slab_common.c | 65 ++++++++++++++++++++++++++++---------------------------- 1 file changed, 32 insertions(+), 33 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 3ba205bda95d..2eef5ad37fa7 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -492,14 +492,10 @@ EXPORT_SYMBOL(kmem_buckets_create); * once or there will be a use-after-free problem. The actual deletion * and release of the kobject does not need slab_mutex or cpu_hotplug_lock * protection. So they are now done without holding those locks. - * - * Note that there will be a slight delay in the deletion of sysfs files - * if kmem_cache_release() is called indrectly from a work function. */ static void kmem_cache_release(struct kmem_cache *s) { if (slab_state >= FULL) { - sysfs_slab_unlink(s); sysfs_slab_release(s); } else { slab_kmem_cache_release(s); @@ -536,33 +532,11 @@ static void slab_caches_to_rcu_destroy_workfn(struct work_struct *work) rcu_barrier(); list_for_each_entry_safe(s, s2, &to_destroy, list) { - debugfs_slab_release(s); kfence_shutdown_cache(s); kmem_cache_release(s); } } -static int shutdown_cache(struct kmem_cache *s) -{ - /* free asan quarantined objects */ - kasan_cache_shutdown(s); - - if (__kmem_cache_shutdown(s) != 0) - return -EBUSY; - - list_del(&s->list); - - if (s->flags & SLAB_TYPESAFE_BY_RCU) { - list_add_tail(&s->list, &slab_caches_to_rcu_destroy); - schedule_work(&slab_caches_to_rcu_destroy_work); - } else { - kfence_shutdown_cache(s); - debugfs_slab_release(s); - } - - return 0; -} - void slab_kmem_cache_release(struct kmem_cache *s) { __kmem_cache_release(s); @@ -572,8 +546,8 @@ void slab_kmem_cache_release(struct kmem_cache *s) void kmem_cache_destroy(struct kmem_cache *s) { - int err = -EBUSY; bool rcu_set; + int err; if (unlikely(!s) || !kasan_check_byte(s)) return; @@ -581,20 +555,45 @@ void kmem_cache_destroy(struct kmem_cache *s) cpus_read_lock(); mutex_lock(&slab_mutex); + s->refcount--; + if (s->refcount) { + mutex_unlock(&slab_mutex); + cpus_read_unlock(); + return; + } + rcu_set = s->flags & SLAB_TYPESAFE_BY_RCU; - s->refcount--; - if (s->refcount) - goto out_unlock; + /* free asan quarantined objects */ + kasan_cache_shutdown(s); - err = shutdown_cache(s); + err = __kmem_cache_shutdown(s); WARN(err, "%s %s: Slab cache still has objects when called from %pS", __func__, s->name, (void *)_RET_IP_); -out_unlock: + + if (!err) + list_del(&s->list); + mutex_unlock(&slab_mutex); cpus_read_unlock(); - if (!err && !rcu_set) + + if (slab_state >= FULL) { + sysfs_slab_unlink(s); + } + debugfs_slab_release(s); + + if (err) + return; + + if (rcu_set) { + mutex_lock(&slab_mutex); + list_add_tail(&s->list, &slab_caches_to_rcu_destroy); + schedule_work(&slab_caches_to_rcu_destroy_work); + mutex_unlock(&slab_mutex); + } else { + kfence_shutdown_cache(s); kmem_cache_release(s); + } } EXPORT_SYMBOL(kmem_cache_destroy); From patchwork Mon Jul 15 20:29:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13733874 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71163C3DA4B for ; Mon, 15 Jul 2024 20:29:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BA53A6B009B; Mon, 15 Jul 2024 16:29:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7D50A6B00AA; Mon, 15 Jul 2024 16:29:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0CCC46B009E; Mon, 15 Jul 2024 16:29:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 9CEB96B00A5 for ; Mon, 15 Jul 2024 16:29:43 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 41BD1121385 for ; Mon, 15 Jul 2024 20:29:43 +0000 (UTC) X-FDA: 82343127846.23.880C2FF Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf03.hostedemail.com (Postfix) with ESMTP id CA70320029 for ; Mon, 15 Jul 2024 20:29:40 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=SwlIHH5d; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="o3BTn/mj"; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=SwlIHH5d; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="o3BTn/mj"; spf=pass (imf03.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721075361; a=rsa-sha256; cv=none; b=n8kyQ4kphARKuWg93XQu/scyPQrIsykJtLvHxpnKbPr7iEUjxmbkNQkI5gzKvdy12z6PPH ASSYskFbbDQmIthpWApuqZlpTGyqN4ewBKYJFVXoL/C8byeS4jj622hn+tpGYT7KRmD6aD 6t/uuBzAMutDEKSlcs6oZsfsjb3jDto= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=SwlIHH5d; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="o3BTn/mj"; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=SwlIHH5d; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="o3BTn/mj"; spf=pass (imf03.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721075361; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NU1ukOjEPV7zEMcpTecl7WdxnTW88aCY09Hfdb+DK10=; b=uqQkQJYRZPfG9c7OmzzvHwSV8ANsJNo/Zc8AcD9YjYFDBfs2QX2dryzbrssjGetUvrVvyV kWtaYWEfYtdV7o79JizPtD+T7jtdXQEUFRMFJ9XILvircSUmah7gHBJpOB07A/B8/dcqVE afhP6CVK2Xhu4FFffDRf322JXWDjGo4= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 161FB21BCE; Mon, 15 Jul 2024 20:29:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NU1ukOjEPV7zEMcpTecl7WdxnTW88aCY09Hfdb+DK10=; b=SwlIHH5d0ynfQ2BNBAJQLY3/sIgnYBrQFJjZOlOwPEVsX3LKgz9QDz/5pEvFogKr8ImAr3 uDSCuxn3SbLkgB0p6T5MXVnoZpkRoCxI0ZUjMLC+UWX1kYFRnwq85/vXupm0TkfoW6BEhh ucAONbfc+WVe/TH5KP+axsZGc9U6vpY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NU1ukOjEPV7zEMcpTecl7WdxnTW88aCY09Hfdb+DK10=; b=o3BTn/mjXjGJHsMvmm/YT9CypKvUmnDz59+qTmH6ye4Cy6nFtkAELn3eaYsl2cPgg2PfgP KtKQz92g/45hsCBw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NU1ukOjEPV7zEMcpTecl7WdxnTW88aCY09Hfdb+DK10=; b=SwlIHH5d0ynfQ2BNBAJQLY3/sIgnYBrQFJjZOlOwPEVsX3LKgz9QDz/5pEvFogKr8ImAr3 uDSCuxn3SbLkgB0p6T5MXVnoZpkRoCxI0ZUjMLC+UWX1kYFRnwq85/vXupm0TkfoW6BEhh ucAONbfc+WVe/TH5KP+axsZGc9U6vpY= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NU1ukOjEPV7zEMcpTecl7WdxnTW88aCY09Hfdb+DK10=; b=o3BTn/mjXjGJHsMvmm/YT9CypKvUmnDz59+qTmH6ye4Cy6nFtkAELn3eaYsl2cPgg2PfgP KtKQz92g/45hsCBw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id EACBD1395F; Mon, 15 Jul 2024 20:29:38 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id kEgRObKGlWbvUwAAD6G6ig (envelope-from ); Mon, 15 Jul 2024 20:29:38 +0000 From: Vlastimil Babka Date: Mon, 15 Jul 2024 22:29:30 +0200 Subject: [PATCH RFC 4/6] mm, slab: simplify kmem_cache_release() MIME-Version: 1.0 Message-Id: <20240715-b4-slab-kfree_rcu-destroy-v1-4-46b2984c2205@suse.cz> References: <20240715-b4-slab-kfree_rcu-destroy-v1-0-46b2984c2205@suse.cz> In-Reply-To: <20240715-b4-slab-kfree_rcu-destroy-v1-0-46b2984c2205@suse.cz> To: "Paul E. McKenney" , Joel Fernandes , Josh Triplett , Boqun Feng , Christoph Lameter , David Rientjes Cc: Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Julia Lawall , Jakub Kicinski , "Jason A. Donenfeld" , "Uladzislau Rezki (Sony)" , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.0 X-Rspamd-Action: no action X-Spamd-Bar: / X-Stat-Signature: njpsopsbxuqsgegebhny8t94zri3ukam X-Rspamd-Queue-Id: CA70320029 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1721075380-85854 X-HE-Meta: U2FsdGVkX1+qdUmWH+yqp2452bLn+LWWdfx6QoDoXbbPYO01XmzmK4q8mlT85YTq8hUgWheaex1cxaYUJQZl66RZ0elJhenykSl+nqZt5hxsEE6UQ+3bW5/dKSvVOZqgX0LeahZggh5bQM8MptDBmRp9lF91NYyDzPl81lkJ2ihBnVxwNjBlc3Ex9Lr7qgfOgrs9vEeKWZoTssPwZYFt/BvCPOzme+Q0BGhtPZeS9g4U5LQ2n7l8Svn3XfGWWyG+kmrNf4anWSrrvfwmr207NRtoNIS9IUxe48rlpXAiuBL1C9IC6W3zJyGTycimemNEBaIFEkH8S2l7zHlMPzWTGVEhUNVh+Ys3yhNQdG4zAVQbfiNQ0AXi4LFprPrJ6TIpl5B/zo7v+hW09AEeVHNhhnqYgG4IiKdhpjc2ZDQ2bPA2eJfOwsLJjLzVtmCXDd8dk5qzup2Xm3eGq0qMN6cVYwt48l9yNnd/l+C1j3KDyT30q9U8vPx1zVZ7MWbopvbFPlUKanx4U+sMhkWKz7p4N8XlsJLb6pQvlOJ8w+gMlPQGUqUFpABoYjKCcicycoc/0hes7Ga/bWJIW1REuKUFQvBMAOGed9MiNWk1gOcMR/OUoS7ue+xTDFoTw9z7RSps921Y3YPLRAFX9qYSaj9iUVhWoZPaF/GVDaGuKzgJRt0Q4Ec8dcqsWV7WHJbC6Tm05qGkx1K/fPIK2GCvYeiDODxrPEkHvhxsLjZlZZqpfmhX6o3BezSRV3af9/d+Bjtfu9Nvst2ZI5ROCkhCagrq+jub4QAgO1b7VxAelbZr41dysPGKqSUtxkC8rZdgb55yCJdzO1FGNNOrrM4bHj33PLKuQ2OXLdGNSpnD6h0scg/upQrBzwjHyrT2z9QVOclmSF5Pk8DNUYbwFDHEiLPL0XbJ/+ol1H+h6pZW3Vdf0fbsc6iiCQV5cNsZfDhkUL+YxGbLLycLboHSmGDCRUw efVnK3D9 nD5xuUcnpBmzkc39rfELQUGvX9EgLj4n3+7V0THyrOdZF7gpZpuctE5ARbWvlZVfEDPmH9/WB5gGKI3e29opab7vnEMridmuTz4+DOkUcepLRL7Wsi9iQ3q9jOQprGdDi1xn799fSVhD0VbrOX3PqG7S8aWqSEPhvPhVFNW4CfAstAxvo9F7CXKC3RhXM0bATVzdtfZa98i+UlV9Y65eyv2fooU0HO7eUkdfzwYbdfTR9d+oYM2ZXnk2nWmpaDAvUQUJ1z0b5/mXJxQ3PhnA+1mO13OCYyLCQwLyapoW/pOjuWIDQnrAEDVexvejwUZI1X0lC3M6iuGkTXuIMvsZgs0649A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kfence_shutdown_cache() is now called always just before kmem_cache_release() so move it there. Also replace two variants of the functions by using __is_defined(SLAB_SUPPORTS_SYSFS). Signed-off-by: Vlastimil Babka --- mm/slab_common.c | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 2eef5ad37fa7..57962e1a5a86 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -486,7 +486,6 @@ kmem_buckets *kmem_buckets_create(const char *name, slab_flags_t flags, } EXPORT_SYMBOL(kmem_buckets_create); -#ifdef SLAB_SUPPORTS_SYSFS /* * For a given kmem_cache, kmem_cache_destroy() should only be called * once or there will be a use-after-free problem. The actual deletion @@ -495,18 +494,12 @@ EXPORT_SYMBOL(kmem_buckets_create); */ static void kmem_cache_release(struct kmem_cache *s) { - if (slab_state >= FULL) { + kfence_shutdown_cache(s); + if (__is_defined(SLAB_SUPPORTS_SYSFS) && slab_state >= FULL) sysfs_slab_release(s); - } else { + else slab_kmem_cache_release(s); - } -} -#else -static void kmem_cache_release(struct kmem_cache *s) -{ - slab_kmem_cache_release(s); } -#endif static void slab_caches_to_rcu_destroy_workfn(struct work_struct *work) { @@ -531,10 +524,8 @@ static void slab_caches_to_rcu_destroy_workfn(struct work_struct *work) rcu_barrier(); - list_for_each_entry_safe(s, s2, &to_destroy, list) { - kfence_shutdown_cache(s); + list_for_each_entry_safe(s, s2, &to_destroy, list) kmem_cache_release(s); - } } void slab_kmem_cache_release(struct kmem_cache *s) @@ -591,7 +582,6 @@ void kmem_cache_destroy(struct kmem_cache *s) schedule_work(&slab_caches_to_rcu_destroy_work); mutex_unlock(&slab_mutex); } else { - kfence_shutdown_cache(s); kmem_cache_release(s); } } From patchwork Mon Jul 15 20:29:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13733873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D818BC3DA59 for ; Mon, 15 Jul 2024 20:29:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 89B216B009E; Mon, 15 Jul 2024 16:29:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 685DD6B00AB; Mon, 15 Jul 2024 16:29:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DECF06B00AC; Mon, 15 Jul 2024 16:29:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 7799E6B009F for ; Mon, 15 Jul 2024 16:29:43 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 091521A04C6 for ; Mon, 15 Jul 2024 20:29:43 +0000 (UTC) X-FDA: 82343127846.25.7B2930A Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf21.hostedemail.com (Postfix) with ESMTP id C0E471C0020 for ; Mon, 15 Jul 2024 20:29:40 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=pbG0llnr; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=97BjocNX; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=pbG0llnr; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=97BjocNX; spf=pass (imf21.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721075344; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=c5NdmWuv3Hvepg9STEhFTiZ7iZ+qzw7pFNvQWY9q/Os=; b=CwQweSOmH1dfn7lkYOixYiBUETE8iYfGeeHnjtmJgkCq4AkPsHMpKpw+43oodvK5DNlxwM WOdYTQitkOTeSvrgs3MFbxI+O2x9Fr84Snd+p9V9qlE/a+FPyB1yPvE08VC8apu/XCkXNZ AR4fjgDar4h4PeIpkDiJFh5LcEYx5zQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721075344; a=rsa-sha256; cv=none; b=llm+bqSRYkmB5jlxw+0z8myKjs6Mq6HwzlzCoC01zQ1b1R2ZG+qEGRXAHX0PFDNDRMDjXL ImaOLZoA+xvIydA7EspfkNONb6W6RNJyXwWeXIQcOi6oMmHefkYuN+cCEfTPqEXMk5AML1 JhL9pf5cXbXAPFxOgN6cCqFmvPGvygo= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=pbG0llnr; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=97BjocNX; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=pbG0llnr; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=97BjocNX; spf=pass (imf21.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 350E221BCF; Mon, 15 Jul 2024 20:29:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c5NdmWuv3Hvepg9STEhFTiZ7iZ+qzw7pFNvQWY9q/Os=; b=pbG0llnr/a+hFS/aFYyY7iQhUo9J3Wd8lOWvT1qWA9/XkF+qfpvwM//mHrWFKSTPxhXVEe zbwxz4zS8s/7SSuWKiw2wR23oMVa/QOGGzCmFoCeaXeLcrBVQrCw9TeP3gyEjSiP6Xax+a oBM6hahMKNsMzEDYtO0hstWCtbwQ084= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c5NdmWuv3Hvepg9STEhFTiZ7iZ+qzw7pFNvQWY9q/Os=; b=97BjocNXuZryya7AfWUdPflgXoUPJtywDuTsx6AhZ3hrUWAHocxKNhe1/FtxgkiAol+Mmj mG/gcI85urtW8DBg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c5NdmWuv3Hvepg9STEhFTiZ7iZ+qzw7pFNvQWY9q/Os=; b=pbG0llnr/a+hFS/aFYyY7iQhUo9J3Wd8lOWvT1qWA9/XkF+qfpvwM//mHrWFKSTPxhXVEe zbwxz4zS8s/7SSuWKiw2wR23oMVa/QOGGzCmFoCeaXeLcrBVQrCw9TeP3gyEjSiP6Xax+a oBM6hahMKNsMzEDYtO0hstWCtbwQ084= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c5NdmWuv3Hvepg9STEhFTiZ7iZ+qzw7pFNvQWY9q/Os=; b=97BjocNXuZryya7AfWUdPflgXoUPJtywDuTsx6AhZ3hrUWAHocxKNhe1/FtxgkiAol+Mmj mG/gcI85urtW8DBg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 13E72137EB; Mon, 15 Jul 2024 20:29:39 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id YBKJBLOGlWbvUwAAD6G6ig (envelope-from ); Mon, 15 Jul 2024 20:29:39 +0000 From: Vlastimil Babka Date: Mon, 15 Jul 2024 22:29:31 +0200 Subject: [PATCH RFC 5/6] mm, slab: asynchronously destroy caches with outstanding objects MIME-Version: 1.0 Message-Id: <20240715-b4-slab-kfree_rcu-destroy-v1-5-46b2984c2205@suse.cz> References: <20240715-b4-slab-kfree_rcu-destroy-v1-0-46b2984c2205@suse.cz> In-Reply-To: <20240715-b4-slab-kfree_rcu-destroy-v1-0-46b2984c2205@suse.cz> To: "Paul E. McKenney" , Joel Fernandes , Josh Triplett , Boqun Feng , Christoph Lameter , David Rientjes Cc: Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Julia Lawall , Jakub Kicinski , "Jason A. Donenfeld" , "Uladzislau Rezki (Sony)" , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.0 X-Rspamd-Queue-Id: C0E471C0020 X-Stat-Signature: dfnyrpycgoszez7s783et8wxsdatjhgi X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1721075380-587387 X-HE-Meta: U2FsdGVkX18JpEAodNreuajm8zmXq+ylxektzF9BFhP4ai95BrMS1qnmEdo47rFS8RVbsqS+AG7D4U68yWNghPfroMmzFZ8ujH3WNgd8IXMQ4PZLKN11R/CYOV7EaEYYm8GKaCzDNlimbJKIxxS5OYqBhAFdrN9Pfk3fp1Or5tKnolICIpWFlHesIYXpAS2lDM45gNFOgiLJoTDkcTbYHLJhqo98Ta3PYrc7yl8iozPIJF2Lz6LZAWgCfZUamGLGudaOE1hLbKzLS6KgqatdqycHiUQfa9rOJgF97xQaNH71xI7XXlpNSU7l3LzXFyOUzJF9d0N6Cru3BafquPPHKKoQ9UDQrUPzsaPbKSPmuVk2dMUQD9PzyT92cNvDhS3BWZr/2u5TnCwScxtdadqbf+Q2yG7N5G0V3vbgvS3KvNOcnR5EBVAV7FcuYeDlu4vf+zhKE6xVt4emN9idLO2XYCmnh0xlCXBv6ZbFjmvMlcyp8Fw3RBLHw4Qd/UP1SwTCphtSr3mM/40yvpb1XYJn8Qxxgp5PM6sST3UkF5PaIobZDoaAdx8KJMmNnN2iUFCM+c8ZbtdY88GKjUzmSIH8iRQFdaiNRxU6tvdPl1CUY5zuj3staH6XT7xazlox1B/gWiXh+GQSM5jjKQiIdDp/dHgUwp1dMiUsEKATWGpHsPmf987Q2huDhvMar8IuxIhS4WXXIsr8+9iC1I9S1zlX/tPRvr9kc5v2zDdsGDrYStJoS3caq2cgeE+r/AwPsBdi1NBdvyBsyo0Dm5YpB4+Y9N1diL9OSluexPr8x+alwEMu9boksJlPxPpmVnbdz/+KfHH/2v5jjfjE8SHcHv6lakXDcaWh+wgdjwOiq140oiesBs81Usn5PqTZccImg6igfNBtSr1w4V99CqJmW3eGMF21V0LHPAvIYhjRAIvX0TT6PuwBmszwxDfx5mznPMa6pWOU7mhJlN1IAlkH9fo fg3dlOMR nvghCActNRUkcWMuvfF7xcZaxFHPPCvUunq8ghxCRG4nVv420GkpIVQ++BJI9KrG8vwwGems89bjgpdNlQz6LNvvk51lX7yU/3H6fixirDsEwC5RHDVasPorMW82GPzSbXFZNzmi8T79klKOMGJAYTH0zMnr3aTeFzLhliIqZF8Yb8ZLGCfRZeJf3BhMGFFC4n9+Rz9WHjYHGKGmqBzC4P9oNtCAx5yaKRFAZO0p9yLmB9BlVqtbxAMDKm91Wc/ZpeHwUcEUgCCehIbzuyC81o19gQoc+DkHcsWIZBpB0cHoRcCI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We would like to replace call_rcu() users with kfree_rcu() where the existing callback is just a kmem_cache_free(). However this causes issues when the cache can be destroyed (such as due to module unload). Currently such modules should be issuing rcu_barrier() before kmem_cache_destroy() to have their call_rcu() callbacks processed first. This barrier is however not sufficient for kfree_rcu() in flight due to the batching introduced by a35d16905efc ("rcu: Add basic support for kfree_rcu() batching"). This is not a problem for kmalloc caches which are never destroyed, but since removing SLOB, kfree_rcu() is allowed also for any other cache, that might be destroyed. In order not to complicate the API, put the responsibility for handling outstanding kfree_rcu() in kmem_cache_destroy() itself. Use the result of __kmem_cache_shutdown() to determine if there are still allocated objects in the cache, and if there are, assume it's due to kfree_rcu(). In that case schedule a work item that will use the appropriate barrier and then attempt __kmem_cache_shutdown() again. Only if that fails as well, produce the usual warning about non-freed objects. Sysfs and debugs directories are removed immediately, so the cache can be recreated with the same name without issues, while the previous instance is still pending removal. Users of call_rcu() with arbitrary callbacks should still perform their own synchronous barrier before destroying the cache and unloading the module, as the callbacks may be invoking module code or perform other actions that are necessary for a successful unload. Note that another non-bug reason why there might be objects outstanding is the kasan quarantine. In that case the cleanup also becomes asynchronous, and flushing the quarantine by kasan_cache_shutdown(s) is only done in the workfn. Signed-off-by: Vlastimil Babka --- mm/slab.h | 4 +++- mm/slab_common.c | 52 ++++++++++++++++++++++++++++++++++++++++++++++++---- mm/slub.c | 9 +++++---- 3 files changed, 56 insertions(+), 9 deletions(-) diff --git a/mm/slab.h b/mm/slab.h index ece18ef5dd04..390a4e265f03 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -279,6 +279,8 @@ struct kmem_cache { unsigned int red_left_pad; /* Left redzone padding size */ const char *name; /* Name (only for display!) */ struct list_head list; /* List of slab caches */ + struct work_struct async_destroy_work; + #ifdef CONFIG_SYSFS struct kobject kobj; /* For sysfs */ #endif @@ -478,7 +480,7 @@ static inline bool is_kmalloc_cache(struct kmem_cache *s) SLAB_NO_USER_FLAGS) bool __kmem_cache_empty(struct kmem_cache *); -int __kmem_cache_shutdown(struct kmem_cache *); +int __kmem_cache_shutdown(struct kmem_cache *, bool); void __kmem_cache_release(struct kmem_cache *); int __kmem_cache_shrink(struct kmem_cache *); void slab_kmem_cache_release(struct kmem_cache *); diff --git a/mm/slab_common.c b/mm/slab_common.c index 57962e1a5a86..3e15525819b6 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -44,6 +44,8 @@ static LIST_HEAD(slab_caches_to_rcu_destroy); static void slab_caches_to_rcu_destroy_workfn(struct work_struct *work); static DECLARE_WORK(slab_caches_to_rcu_destroy_work, slab_caches_to_rcu_destroy_workfn); +static void kmem_cache_kfree_rcu_destroy_workfn(struct work_struct *work); + /* * Set of flags that will prevent slab merging @@ -235,6 +237,7 @@ static struct kmem_cache *create_cache(const char *name, s->refcount = 1; list_add(&s->list, &slab_caches); + INIT_WORK(&s->async_destroy_work, kmem_cache_kfree_rcu_destroy_workfn); return s; out_free_cache: @@ -535,6 +538,47 @@ void slab_kmem_cache_release(struct kmem_cache *s) kmem_cache_free(kmem_cache, s); } +static void kmem_cache_kfree_rcu_destroy_workfn(struct work_struct *work) +{ + struct kmem_cache *s; + bool rcu_set; + int err; + + s = container_of(work, struct kmem_cache, async_destroy_work); + + // XXX use the real kmem_cache_free_barrier() or similar thing here + rcu_barrier(); + + cpus_read_lock(); + mutex_lock(&slab_mutex); + + rcu_set = s->flags & SLAB_TYPESAFE_BY_RCU; + + /* free asan quarantined objects */ + kasan_cache_shutdown(s); + + err = __kmem_cache_shutdown(s, true); + WARN(err, "kmem_cache_destroy %s: Slab cache still has objects", + s->name); + + if (err) + goto out_unlock; + + list_del(&s->list); + + if (rcu_set) { + list_add_tail(&s->list, &slab_caches_to_rcu_destroy); + schedule_work(&slab_caches_to_rcu_destroy_work); + } + +out_unlock: + mutex_unlock(&slab_mutex); + cpus_read_unlock(); + + if (!err && !rcu_set) + kmem_cache_release(s); +} + void kmem_cache_destroy(struct kmem_cache *s) { bool rcu_set; @@ -558,9 +602,7 @@ void kmem_cache_destroy(struct kmem_cache *s) /* free asan quarantined objects */ kasan_cache_shutdown(s); - err = __kmem_cache_shutdown(s); - WARN(err, "%s %s: Slab cache still has objects when called from %pS", - __func__, s->name, (void *)_RET_IP_); + err = __kmem_cache_shutdown(s, false); if (!err) list_del(&s->list); @@ -573,8 +615,10 @@ void kmem_cache_destroy(struct kmem_cache *s) } debugfs_slab_release(s); - if (err) + if (err) { + schedule_work(&s->async_destroy_work); return; + } if (rcu_set) { mutex_lock(&slab_mutex); diff --git a/mm/slub.c b/mm/slub.c index aa4d80109c49..c1222467c346 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -5352,7 +5352,8 @@ static void list_slab_objects(struct kmem_cache *s, struct slab *slab, * This is called from __kmem_cache_shutdown(). We must take list_lock * because sysfs file might still access partial list after the shutdowning. */ -static void free_partial(struct kmem_cache *s, struct kmem_cache_node *n) +static void free_partial(struct kmem_cache *s, struct kmem_cache_node *n, + bool warn_inuse) { LIST_HEAD(discard); struct slab *slab, *h; @@ -5363,7 +5364,7 @@ static void free_partial(struct kmem_cache *s, struct kmem_cache_node *n) if (!slab->inuse) { remove_partial(n, slab); list_add(&slab->slab_list, &discard); - } else { + } else if (warn_inuse) { list_slab_objects(s, slab, "Objects remaining in %s on __kmem_cache_shutdown()"); } @@ -5388,7 +5389,7 @@ bool __kmem_cache_empty(struct kmem_cache *s) /* * Release all resources used by a slab cache. */ -int __kmem_cache_shutdown(struct kmem_cache *s) +int __kmem_cache_shutdown(struct kmem_cache *s, bool warn_inuse) { int node; struct kmem_cache_node *n; @@ -5396,7 +5397,7 @@ int __kmem_cache_shutdown(struct kmem_cache *s) flush_all_cpus_locked(s); /* Attempt to free all objects */ for_each_kmem_cache_node(s, node, n) { - free_partial(s, n); + free_partial(s, n, warn_inuse); if (n->nr_partial || node_nr_slabs(n)) return 1; } From patchwork Mon Jul 15 20:29:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13733872 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B7D7C3DA59 for ; Mon, 15 Jul 2024 20:29:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4A3276B00A8; Mon, 15 Jul 2024 16:29:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 401966B00AE; Mon, 15 Jul 2024 16:29:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D01C26B009B; Mon, 15 Jul 2024 16:29:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 7835C6B00A0 for ; Mon, 15 Jul 2024 16:29:43 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 2D9C6160234 for ; Mon, 15 Jul 2024 20:29:43 +0000 (UTC) X-FDA: 82343127846.30.1E2FD55 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf10.hostedemail.com (Postfix) with ESMTP id D84F9C0007 for ; Mon, 15 Jul 2024 20:29:40 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=prIKxLzt; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=aE2aSMpu; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=prIKxLzt; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=aE2aSMpu; spf=pass (imf10.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721075363; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yWewRUcOUAtVmN4JR7Gtyt2MolU1IAn0Ns4fcy6NIVg=; b=IH05l6sx6aIWPAEwjnnlSk5Ti4a7Y0U/BL/vxHxnrCubXSeoAbo73Yl/UG1Y0G0K/aYxHS BlA/eorOzrTWBf9Hs4PfzpNISvtT03DFNciNk6UO0MSQdF2n8fwOGkJshPwTwLYrFrltdY F3AVgk4lcG9V82G1iJ+hm3txWtlrHFs= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=prIKxLzt; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=aE2aSMpu; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=prIKxLzt; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=aE2aSMpu; spf=pass (imf10.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721075363; a=rsa-sha256; cv=none; b=fSdkTYDN30B4FeZeXBcoMTcSE+V9kb7Ayz0c9al0ukzZ6A7YlGkI/x1G5kx3I6Q/DMD+4Y 2chF49KhJcdCPSg0LIfbFrrC5l69nt7YZmeiH5CZdSVCnFOcJ9EI/F3c/vUxqEyL//1HWe 1TNbyKrzYx7llgH92jmZe1L3ErOwetw= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 5119A21BD0; Mon, 15 Jul 2024 20:29:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yWewRUcOUAtVmN4JR7Gtyt2MolU1IAn0Ns4fcy6NIVg=; b=prIKxLztGfLbdeAhqgcoCK1H8//juWuOKqdI91X4l4EJajX8iU9dBgUIGHI3oTtT90n1En fQg+/C1t4mT5ErXDMfBha2YRuPoew0wPFfBnkeYyzXsLYW1WxkU/p8BRc1tUfOWDsSWGWn HpwOqvdETyGUnPP9tPNEBEpZKg2A9ro= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yWewRUcOUAtVmN4JR7Gtyt2MolU1IAn0Ns4fcy6NIVg=; b=aE2aSMpuuGQWpz2WP9mQoUBD+DMuFoYlFRvPk5ajs9R0pwV7ByK+B24CBCTjYYCEhB3Ue/ /BDr04RNGcmpkdCA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yWewRUcOUAtVmN4JR7Gtyt2MolU1IAn0Ns4fcy6NIVg=; b=prIKxLztGfLbdeAhqgcoCK1H8//juWuOKqdI91X4l4EJajX8iU9dBgUIGHI3oTtT90n1En fQg+/C1t4mT5ErXDMfBha2YRuPoew0wPFfBnkeYyzXsLYW1WxkU/p8BRc1tUfOWDsSWGWn HpwOqvdETyGUnPP9tPNEBEpZKg2A9ro= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yWewRUcOUAtVmN4JR7Gtyt2MolU1IAn0Ns4fcy6NIVg=; b=aE2aSMpuuGQWpz2WP9mQoUBD+DMuFoYlFRvPk5ajs9R0pwV7ByK+B24CBCTjYYCEhB3Ue/ /BDr04RNGcmpkdCA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 31A361396E; Mon, 15 Jul 2024 20:29:39 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 0NDVC7OGlWbvUwAAD6G6ig (envelope-from ); Mon, 15 Jul 2024 20:29:39 +0000 From: Vlastimil Babka Date: Mon, 15 Jul 2024 22:29:32 +0200 Subject: [PATCH RFC 6/6] kunit, slub: add test_kfree_rcu() MIME-Version: 1.0 Message-Id: <20240715-b4-slab-kfree_rcu-destroy-v1-6-46b2984c2205@suse.cz> References: <20240715-b4-slab-kfree_rcu-destroy-v1-0-46b2984c2205@suse.cz> In-Reply-To: <20240715-b4-slab-kfree_rcu-destroy-v1-0-46b2984c2205@suse.cz> To: "Paul E. McKenney" , Joel Fernandes , Josh Triplett , Boqun Feng , Christoph Lameter , David Rientjes Cc: Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Julia Lawall , Jakub Kicinski , "Jason A. Donenfeld" , "Uladzislau Rezki (Sony)" , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.0 X-Rspamd-Action: no action X-Spamd-Bar: / X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: D84F9C0007 X-Stat-Signature: zcd7jfgbe1qi63izw1fdz7tif1w55n8n X-HE-Tag: 1721075380-425625 X-HE-Meta: U2FsdGVkX1/sVxklxlc2kWjie7kcmipoirfGxWJKggbzlrK21LZ9GzkunHnfJGYh+ADuANcqhr4MorThV45h8RJolYcr9CODmPUMOX2e7uTNk0HnX7vSgWiW0+WciiuaWhdis2MWnY88aweDcmWHQYivbSjfwwiogpyHSGPwy9QQWGFAcFcZ75ZFcD7m6SW+T3bbvw/BUP53QGVTjOo0jCTB8VBqc4LOcTSjK7UOuDhaB2Jd529D5lpKwlPSSGwnKHE9LRuGZ+gookMA1JL9sYawHRk0Y+ciiaWukO88oR9JPt5zFTT8RPXhpq7Jem45pTJNIWvyfsu6qHDbFMFHfOq6GOYtNuAvHAUYCt+s87n5mDc/e4HLgyrN6TvFuIgsFqKNiYP6BVoDCwzQPSRbdoik8qNDUPef2C47dBijiE7LASM4f0xAz4IyUis5ycZo0tEWUXrztSw6aC9J29d2Kw7G4phXPfLdqoZk9RM/+fjXvWg8PF7aHFeVU1Y2its5qyVMq1x5OI20mK4HC/sa/nn+6rJmK/wcJlh3ZszGXofo6MLktaB+R4MOhoTrrxeSxlRgfilbrPPqNoKpds3r5tT4Hg19CShcWK/Rg2GqsI+HovYMeiB9h9WHF/gkYvYD4XlzE0mG9Yuls1gcHDZFFvhrLC4YfNTrHxC/Gh9I5Q1q1bkgrL5pNqRgNC7C0N9rZFEdmdXo9npAiRo68AnIA2FahKjNOxMGGhn05+LRNuQQD7d5PHYshk6Qz9mLy4Ezz0OaLWSkRQSuhYoYzuocjC+lluOLWPgWnXYF3SrPqV2vmrLnISkLfvbD3j2gm393fD2fuCY6p2okzpgkcYrw8FIIREoMKsWRSoVWhAOOdAhhBVC27QvSOeTABzKwKsTMdkZ0wydhpZY7u7psPWGqmLs9tEeAtlBVJES6/qKDXm6gxdbasry56coOR2759WM8SJeameE+PDK6rn4Ag9a zfWRRbYX 6JUGX1Xs1EZCUYkzkal9C3FjFLVz0qgfZOCaHpFeMgg+PPpT7XctUyYBwRxj/N2hm15pC6TrgSNPWj4UJdn/RcS1+0U+WbxkSqXFLYi/or9cBisTEGGxLpW63qTXq09yEqmLBtG27By8fKsIcKMdkz8Z1k/Ujq2GdhrIPJvH5r8Gg1gxtGS0lEjBpERB4tEuZ+Wc1MfCRfovlv4PsXVjKKa5cgPzM7jU0qM0goc97wJw2V9R/1uTbWQO/F5gJg+tcUW2mu/my6sbYmTSzNrtWfhQ1mWPeyjSYGdjkbOdUGsEIgdKmhAD7E/l7nv0lxBeJ8kF//ZIbrm0b8xmE0JXYAPJyb3q5znXX2l4K X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add a test that will create cache, allocate one object, kfree_rcu() it and attempt to destroy it. If the asynchronous cache freeing works correctly, there should be no warnings in dmesg. Since the warnings in the failure case are produced by a work callback, I don't know if it's possible to capture it in the kunit test result properly. Signed-off-by: Vlastimil Babka --- lib/slub_kunit.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/lib/slub_kunit.c b/lib/slub_kunit.c index e6667a28c014..e3e4d0ca40b7 100644 --- a/lib/slub_kunit.c +++ b/lib/slub_kunit.c @@ -5,6 +5,7 @@ #include #include #include +#include #include "../mm/slab.h" static struct kunit_resource resource; @@ -157,6 +158,26 @@ static void test_kmalloc_redzone_access(struct kunit *test) kmem_cache_destroy(s); } +struct test_kfree_rcu_struct { + struct rcu_head rcu; +}; + +static void test_kfree_rcu(struct kunit *test) +{ + struct kmem_cache *s = test_kmem_cache_create("TestSlub_kfree_rcu", + sizeof(struct test_kfree_rcu_struct), + SLAB_NO_MERGE); + struct test_kfree_rcu_struct *p = kmem_cache_alloc(s, GFP_KERNEL); + + kasan_disable_current(); + + KUNIT_EXPECT_EQ(test, 0, slab_errors); + + kasan_enable_current(); + kfree_rcu(p, rcu); + kmem_cache_destroy(s); +} + static int test_init(struct kunit *test) { slab_errors = 0; @@ -177,6 +198,7 @@ static struct kunit_case test_cases[] = { KUNIT_CASE(test_clobber_redzone_free), KUNIT_CASE(test_kmalloc_redzone_access), + KUNIT_CASE(test_kfree_rcu), {} };