From patchwork Mon Jul 15 20:29:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13733875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DCF50C3DA4B for ; Mon, 15 Jul 2024 20:29:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D8F8C6B009F; Mon, 15 Jul 2024 16:29:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A1BF36B00AE; Mon, 15 Jul 2024 16:29:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3BAB06B009F; Mon, 15 Jul 2024 16:29:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id AD0E16B00A8 for ; Mon, 15 Jul 2024 16:29:43 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 18F7C80242 for ; Mon, 15 Jul 2024 20:29:43 +0000 (UTC) X-FDA: 82343127846.15.07D6551 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf06.hostedemail.com (Postfix) with ESMTP id AE156180016 for ; Mon, 15 Jul 2024 20:29:40 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=hSGyH+m5; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=kAIi0fhY; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=hSGyH+m5; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=kAIi0fhY; spf=pass (imf06.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721075361; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tGLdjgoDRMwVFM9fTSoQ/YTustjDFDN8l8I6qmGg0KM=; b=bZceEPE4MIASaJkNCfYq/xTSkaj2xYAbMPirMY9HzUUrgRxs5axQAtQN5eqZo4W8DMkkG1 mIP6YtgKUhSIcfuM8eqL2DIYsWHW8y4XOqUzlc488UbeoBYJSqMcqDFqi72YLAc/Ux3Ohp Lv1Rudfms18adVXISTbug7yb0oQ98+g= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=hSGyH+m5; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=kAIi0fhY; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=hSGyH+m5; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=kAIi0fhY; spf=pass (imf06.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721075361; a=rsa-sha256; cv=none; b=zk48ScqcEkkhpTnNXuGzo/3Zj1d3TJr1aC9Vhkt+uss36dh0cFsfSiNGEEtAAjMWIWF2PB EZXGwaGjCI+Uk9pU7hofi19240Lmr6ERPAPtZ7UGHBHSPqNPzWl06YnEFt0y3kpl7yqM7p trwfDWYJqssEY8oZpD/V4ZyrxDLqZqw= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id EF0BC1F849; Mon, 15 Jul 2024 20:29:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tGLdjgoDRMwVFM9fTSoQ/YTustjDFDN8l8I6qmGg0KM=; b=hSGyH+m5FYyRHvKzXhLIpQGye0mEwLbM3RqIof47AS3U2ab1U27QYUbjQLDHoBm5COD+rq Ceso8rzC8jfjximaXlOSp7oRoim+/TN/R0Uq/5a8xFXCECpuSwariIoyRIXR2Ph34PPzs1 QVSlCwrPIzslLN7yLti+5OJPxWO/dfQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tGLdjgoDRMwVFM9fTSoQ/YTustjDFDN8l8I6qmGg0KM=; b=kAIi0fhYDpGbjURluCU3wBE0Tgnqg89fNVrOr5lDUl5eiKjVINACSfe8OycWeAGYyY3d+5 yNQIKHir85e7KmAg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tGLdjgoDRMwVFM9fTSoQ/YTustjDFDN8l8I6qmGg0KM=; b=hSGyH+m5FYyRHvKzXhLIpQGye0mEwLbM3RqIof47AS3U2ab1U27QYUbjQLDHoBm5COD+rq Ceso8rzC8jfjximaXlOSp7oRoim+/TN/R0Uq/5a8xFXCECpuSwariIoyRIXR2Ph34PPzs1 QVSlCwrPIzslLN7yLti+5OJPxWO/dfQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1721075379; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tGLdjgoDRMwVFM9fTSoQ/YTustjDFDN8l8I6qmGg0KM=; b=kAIi0fhYDpGbjURluCU3wBE0Tgnqg89fNVrOr5lDUl5eiKjVINACSfe8OycWeAGYyY3d+5 yNQIKHir85e7KmAg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id CEB9713A63; Mon, 15 Jul 2024 20:29:38 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id QGIrMrKGlWbvUwAAD6G6ig (envelope-from ); Mon, 15 Jul 2024 20:29:38 +0000 From: Vlastimil Babka Date: Mon, 15 Jul 2024 22:29:29 +0200 Subject: [PATCH RFC 3/6] mm, slab: unlink sysfs and debugfs immediately MIME-Version: 1.0 Message-Id: <20240715-b4-slab-kfree_rcu-destroy-v1-3-46b2984c2205@suse.cz> References: <20240715-b4-slab-kfree_rcu-destroy-v1-0-46b2984c2205@suse.cz> In-Reply-To: <20240715-b4-slab-kfree_rcu-destroy-v1-0-46b2984c2205@suse.cz> To: "Paul E. McKenney" , Joel Fernandes , Josh Triplett , Boqun Feng , Christoph Lameter , David Rientjes Cc: Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Julia Lawall , Jakub Kicinski , "Jason A. Donenfeld" , "Uladzislau Rezki (Sony)" , Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.0 X-Rspamd-Action: no action X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: AE156180016 X-Stat-Signature: 4t95b1r8b8hy6fycaxkwtr9fyfhrtira X-Rspam-User: X-HE-Tag: 1721075380-931914 X-HE-Meta: U2FsdGVkX19m8fKh3WNUmovr029qQHnvyPzxB2eUfqAqyb3vYsvfkonBOkAdCBRbklHZ67X2Y6TAoTAj+aEslWQdDxTQW8reDAuVqHA7iJm5+SwLVFpJZVp+jb8K9z+PHgL3YgQNl1mahFRKpP9RT90xiDl0CAT0K3UhgKGeVeRqHyv87viViWNiOEDs/ZOLayksFfA28q1Aj1QKzU0aqg4bPNfSGrnqvOS5I26niDsLDuJJdEDn25zlZnM312Cy7Bv0TR+N+Qbel/ZP2A00zlhwdoxdNTLTWOkbNH/PlvWxcYTZA/nlgXBjHW4pwAiQ8AfXKRgxO/Wt4sEDe8Y6DqIc0nJH3PeIW7LIupy+p72ucIfoYU/1tPtQpShenzqq+neBaWm2wxOyIkF+mvsCGI+w6E12J/ty+zmkHDvkZ723Ej3LRyB+WsPTUrTtsQ0EPMHYFc5ijM1DvYAb42gCjDU7PBYqdTHplQIltlPBww8SYHTf9XKLt+5YUvHjEI0xLBzpMx3sMBw7/T97FI1STrqt6/fmwdNDBk1vf/6OoKsWMVbLNT1cmV1Y+qj+O8F5tLJ/Dozu+JEGmxMJoSSwD4YCMfvBuemZd97x1+7m5fzCvuyVESl63cz1aSoCCLycviapYmhjqookGDxIURmPxw7n0cNBviH8VSo4TmSZ3DeoXiZmPlHChbqtmIMYOW7PyKPg6GFCw6VHzhe3yVVn/9c6yd/0uBN3DOo35GBVuGS3UvNCUdQ5fzo9rxxPJGMMHsEnp9m5CAl7T4EwfNpQ+NBjyRyDPDqYbfoaRS4vLqvC6wlPZyCjJz4s1aN2BZjRyjWQJcXteWPX/wdm+rAjIzfoMXWjpmRTNdQ5FYtS9rVuW0VlKq3AlS5/gQoY1gBvuv52B6QISV3rS39ff1SU3oDcEf26SSl0bO53ET+B9Nb3DIT60Y2WTz2Ky75mzSS8d9qY+9iIaKHK/BO9HON JXTQiwmn INuBp149BAINyKqldjE3e9jaH0jYjtS0BquRJqC7Qc9+jkk3jXn+2NECJ9TPqh2myENM6DM0HkYKJu8YEWVrr7H871/+F7uTzM+JaDmmx+yw432orcyUyX2lawFLqJxfxZ5f8Qr6wVinzzIGL3ISRs+NdGachRSSR6EXWikcf+Ua6uv2KeTpGKfO1t76hARiKqklC0sC+rQkgYoNDL0ecxMI1j6ma9kJMgi0QTCS/N7DRl4a+sBZiPYGUJVn/HNVKv+wvHTLtipOmSVlfkGR2pSm69vCTlYoUY5ePfqXPXvYzf3U= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: kmem_cache_destroy() includes removing the associated sysfs and debugfs directories. Currently this might not happen immediately when: - the cache is SLAB_TYPESAFE_BY_RCU and the cleanup is delayed, including the directores removal - __kmem_cache_shutdown() fails due to outstanding objects - the directories remain indefinitely When a cache is recreated with the same name, such as due to module unload followed by a load, the directories will fail to be recreated for the new instance of the cache due to the old directories being present. We also want to add another possibility of delayed cleanup due to kfree_rcu() in flight so let's fix this first and have the directories removed immediately in kmem_cache_destroy() and regardless of __kmem_cache_shutdown() success. This should not make debugging harder if __kmem_cache_shutdown() fails, because a detailed report of outstanding objects is printed into dmesg already due to the failure. Note the record in /proc/slabinfo will remain until the cleanup is finished (or indefinitely if __kmem_cache_shutdown() fails) but that does not prevent a new record to be added for a new cache instance. Signed-off-by: Vlastimil Babka --- mm/slab_common.c | 65 ++++++++++++++++++++++++++++---------------------------- 1 file changed, 32 insertions(+), 33 deletions(-) diff --git a/mm/slab_common.c b/mm/slab_common.c index 3ba205bda95d..2eef5ad37fa7 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -492,14 +492,10 @@ EXPORT_SYMBOL(kmem_buckets_create); * once or there will be a use-after-free problem. The actual deletion * and release of the kobject does not need slab_mutex or cpu_hotplug_lock * protection. So they are now done without holding those locks. - * - * Note that there will be a slight delay in the deletion of sysfs files - * if kmem_cache_release() is called indrectly from a work function. */ static void kmem_cache_release(struct kmem_cache *s) { if (slab_state >= FULL) { - sysfs_slab_unlink(s); sysfs_slab_release(s); } else { slab_kmem_cache_release(s); @@ -536,33 +532,11 @@ static void slab_caches_to_rcu_destroy_workfn(struct work_struct *work) rcu_barrier(); list_for_each_entry_safe(s, s2, &to_destroy, list) { - debugfs_slab_release(s); kfence_shutdown_cache(s); kmem_cache_release(s); } } -static int shutdown_cache(struct kmem_cache *s) -{ - /* free asan quarantined objects */ - kasan_cache_shutdown(s); - - if (__kmem_cache_shutdown(s) != 0) - return -EBUSY; - - list_del(&s->list); - - if (s->flags & SLAB_TYPESAFE_BY_RCU) { - list_add_tail(&s->list, &slab_caches_to_rcu_destroy); - schedule_work(&slab_caches_to_rcu_destroy_work); - } else { - kfence_shutdown_cache(s); - debugfs_slab_release(s); - } - - return 0; -} - void slab_kmem_cache_release(struct kmem_cache *s) { __kmem_cache_release(s); @@ -572,8 +546,8 @@ void slab_kmem_cache_release(struct kmem_cache *s) void kmem_cache_destroy(struct kmem_cache *s) { - int err = -EBUSY; bool rcu_set; + int err; if (unlikely(!s) || !kasan_check_byte(s)) return; @@ -581,20 +555,45 @@ void kmem_cache_destroy(struct kmem_cache *s) cpus_read_lock(); mutex_lock(&slab_mutex); + s->refcount--; + if (s->refcount) { + mutex_unlock(&slab_mutex); + cpus_read_unlock(); + return; + } + rcu_set = s->flags & SLAB_TYPESAFE_BY_RCU; - s->refcount--; - if (s->refcount) - goto out_unlock; + /* free asan quarantined objects */ + kasan_cache_shutdown(s); - err = shutdown_cache(s); + err = __kmem_cache_shutdown(s); WARN(err, "%s %s: Slab cache still has objects when called from %pS", __func__, s->name, (void *)_RET_IP_); -out_unlock: + + if (!err) + list_del(&s->list); + mutex_unlock(&slab_mutex); cpus_read_unlock(); - if (!err && !rcu_set) + + if (slab_state >= FULL) { + sysfs_slab_unlink(s); + } + debugfs_slab_release(s); + + if (err) + return; + + if (rcu_set) { + mutex_lock(&slab_mutex); + list_add_tail(&s->list, &slab_caches_to_rcu_destroy); + schedule_work(&slab_caches_to_rcu_destroy_work); + mutex_unlock(&slab_mutex); + } else { + kfence_shutdown_cache(s); kmem_cache_release(s); + } } EXPORT_SYMBOL(kmem_cache_destroy);