From patchwork Mon Feb 3 09:28:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13957132 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DF904C02192 for ; Mon, 3 Feb 2025 09:30:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 79A7F28000F; Mon, 3 Feb 2025 04:30:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 72358280002; Mon, 3 Feb 2025 04:30:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 59E7428000F; Mon, 3 Feb 2025 04:30:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 32E05280002 for ; Mon, 3 Feb 2025 04:30:13 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 99CADB4F8D for ; Mon, 3 Feb 2025 09:29:32 +0000 (UTC) X-FDA: 83078110626.13.6FC4087 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf23.hostedemail.com (Postfix) with ESMTP id 74ADE140002 for ; Mon, 3 Feb 2025 09:29:30 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="sBMGJ/zh"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="X/7rN/En"; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="sBMGJ/zh"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="X/7rN/En"; spf=pass (imf23.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738574970; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=MfZfmqbXeAJR4AoyC4wgDr/6ru6EDZ/hvaXndR4NNN0=; b=7Z2xRsu4qgNq/c5NuoNc2IGADtyOSJLRe3ih0rX4DCY58GPJDbF/k0PwOW4mdusWJZ4UIZ A70wR9fA7JpFhDC034P79wER9tsb/R0ZUQ5cAcnd00xWIWksCUrxDLsvVgtoo9avWaWJ48 xkQ1O+x+3B6d7DRFktuD5JL+DTsSVAg= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="sBMGJ/zh"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="X/7rN/En"; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="sBMGJ/zh"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="X/7rN/En"; spf=pass (imf23.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738574970; a=rsa-sha256; cv=none; b=Xwuaa+7AIBHpvTmtFSxROQ/sXj7QYiCy/UDWRqo824uKjeeirQYUIqrpzzaRQdmMuvlwW0 urJyWCALzayQuW8biPRj/hA3qgOYBnv2qILoavRAn43AGm2OVYF49Qwb8n9ggUt/gq7IMh f/bTu81V+oFG/spbIC0+8X6rPVqq3V4= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 6C538210F5; Mon, 3 Feb 2025 09:29:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1738574968; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=MfZfmqbXeAJR4AoyC4wgDr/6ru6EDZ/hvaXndR4NNN0=; b=sBMGJ/zhBkW4MioftulY9t/iGHA6962bMzkH+Bx3NGT6WYz6oE5HOmiozexcNzAk63co6b LoGkyzwx3KFqrd3VypRfFF4Ab9E9w9lMG3pt5xg75ZV7JVulVutntjETbXbGvXE0eJ1sW0 mwVj6Av//HBRvUokVoxahFqkCeS/OnE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1738574968; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=MfZfmqbXeAJR4AoyC4wgDr/6ru6EDZ/hvaXndR4NNN0=; b=X/7rN/En/0RL/qYZO3ixWkVtJ6DusGgTpXwJNaMePOFHP+q0/dJgE/BDVw/+/XX/cE0g8g GTD6MbTOhXJ+ahAQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1738574968; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=MfZfmqbXeAJR4AoyC4wgDr/6ru6EDZ/hvaXndR4NNN0=; b=sBMGJ/zhBkW4MioftulY9t/iGHA6962bMzkH+Bx3NGT6WYz6oE5HOmiozexcNzAk63co6b LoGkyzwx3KFqrd3VypRfFF4Ab9E9w9lMG3pt5xg75ZV7JVulVutntjETbXbGvXE0eJ1sW0 mwVj6Av//HBRvUokVoxahFqkCeS/OnE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1738574968; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=MfZfmqbXeAJR4AoyC4wgDr/6ru6EDZ/hvaXndR4NNN0=; b=X/7rN/En/0RL/qYZO3ixWkVtJ6DusGgTpXwJNaMePOFHP+q0/dJgE/BDVw/+/XX/cE0g8g GTD6MbTOhXJ+ahAQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 41D9B13A78; Mon, 3 Feb 2025 09:29:28 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id N6u+D3iMoGfwYAAAD6G6ig (envelope-from ); Mon, 03 Feb 2025 09:29:28 +0000 From: Vlastimil Babka Subject: [PATCH 0/4] slab, rcu: move and consolidate TINY_RCU kvfree_rcu() to SLAB Date: Mon, 03 Feb 2025 10:28:46 +0100 Message-Id: <20250203-slub-tiny-kfree_rcu-v1-0-d4428bf9a8a1@suse.cz> MIME-Version: 1.0 X-B4-Tracking: v=1; b=H4sIAE6MoGcC/x3MwQpAQBCA4VfRnE0x4uBVJFlmmWhp1ork3W2O3 +H/H/Cswh7q5AHlU7xsLiJPExjm3k2MMkYDZVRmORXo12DwEHfjYpW50yGgGavSWCYiSxDLXdn K9V+b9n0/Y0EEkGUAAAA= X-Change-ID: 20250123-slub-tiny-kfree_rcu-bd65bfe222f2 To: Christoph Lameter , David Rientjes , "Paul E. McKenney" , Joel Fernandes , Josh Triplett , Boqun Feng , Uladzislau Rezki Cc: Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Steven Rostedt , Mathieu Desnoyers , Lai Jiangshan , Zqiang , rcu@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.2 X-Rspamd-Queue-Id: 74ADE140002 X-Stat-Signature: osq9gygpaws17fgdazer7osqiizt6gxo X-Rspamd-Server: rspam08 X-Rspam-User: X-HE-Tag: 1738574970-478883 X-HE-Meta: U2FsdGVkX1/uebJaDXKplyBSc4th7n8hEhp4CcFpj4TliIwZfEskBSNJPOXQ41M0kGgkHAk7vnAdXQHod/3wN0pF0FV0jILaPoNC1B927XjKyIbPhBAcqUsx/hdCasyRvnoWxqF7BA3TbP9aSgMFogaKtu+Q0FhTz7YjrX0wrvvGuvvpHWIAJBDJj+VJ0JODW2IynjVhCEo3j3vV+bE64yRVMTAuBhYygVJw+EO5i/U/u9s5tavZR8xuyW1M7sZx4xSz37L4p7eROQoaWq7F+MWsOrngcfqAQAbCn5Dtqm+2O2Yq4TCwVyslJIwn8G+iOOLMLwQU5FmFfhP/tuqjMeLsYcclKoctOkMv9gO1kYkBARYoKOwFWz9dQMGF5pMCCl1tBzqzpmqET16vjDgFoflTLkQtWzq+8eJDklop6Y1CXgpCqjmUug9F4AWifg1URwwriYtxzsEV5NBsnL9IYLnCiLV7rpzZ8nTTHS71Th0pH8ZLLviZ0QXPFXLIaCOW8PQRer++t36t9D7Q/RmJrgTWwrAgFk37NGkaQFeEevwEw+BsbyWrh/Whr58pRrS+tok6twG9jpuqE/R1Lw74bjWa3uK/B9qHOPdvFfePhycX08ClGqFJjeOvFbnjkGexTKhxrM6NBgYTKETsuoNVaaJuqlMcTrjvwt2FWwn8tU33xxSPbbYV9YLNCpMBElXEAQp4C+Squ8ShlyhamtaT8o49t6k0xr0aqEpvShe+0MiSWyLECTwNSzOpchxPMNsf4sFbs5aOv0q8GuZnFsbjWFmHsOlaUw2pNZ8aKRazehhtRw9Vzw5/BYYV4iUEMdrVlehSRJ2DPwt7ewXdborzx8m4gaVBEc4ruHyakCzazvM4BNn99kopgboCGeLpPMqbTAQAqWZnKm3lOIfS2kp1DCba3qCp5COWcaMUGsQeobIm+5RM7D5b5lNf95EdpJFqlU2HUtxcI71ggjJdJ50 5CQpmEb+ z2Va2iRjSdaZMt401QdLilgKY/i7n0dSzNVJM3XEp5TmXbSWByjG6NA94gK5BQb8o68MVva7keRdNgKBCuP4qMzRtwHtxiFd1Sl9Hoa6zhmmqLRd1M18a84Ac0uXiKPz6iFKA15xL03Q3gdKPG/BQSEvAe+LAodwyfe32Pvz8riCV/AZTM4IW3Pfj3VbJtsLRp6Nx9yi1rnjZ9yG51OYV1xi3FvRd3+m93nnhTovlvdYAxS+EsIOKGGjhsbXu9dDw+MmEgpxzc/7o7y/XE9VYsizEYdEDVflqnmYuuJe9GPIuZLZveLHflq1LQ/Tm7e8sLTcgY6iSgZr03LHRElZAZDGNhN+1Y5+SJEJqp84Q67F/2FXAhKg0Eq6WfyISV18SFDn/kOm7WQpz1B6i7AJiM+u3yrsjoqT7cFWEmbkSsV+dvbZ77VQCFl4tviApHM5QVbBGwjWcUX6KWECHiPrr+L7BgGtcSsJ1lN8ldGIGzeEMYgZL1+kq+vhXJyZOlcQSTpPKjKG6BcIiTsO+Cf0iX0pkMnyXZYnrc4edUbsDHZapkTTix6ePpyFOi+XU8JEC7cm7R75ndJfCSbP06IdC6a2RIve4hZFNx+RS/rixZ4AzKl58iC8LD1m2bCmgAsLffOYqukQZ9Ha0IVkaNMmJrrZBbxPAtiRxYPLDKPvteUPJco8HcsuCH6cfU8pNgSA2hbskrfWZpO8H6NEqdsPaMo7UBUqqVlgsKu7apg/FAOaMsQl9Dro29qxOwUt4hV9dEMv4kzd9MTt/g3VMkpGLLuuYJ3Vlg/f6mdgTtetHXAWZ7ybSAUTGaZUD7jeDK1XOkbR1sgBQqk6MYToXbDnk4cjIBQcpfbLOkum4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Following the move of the TREE_RCU batching kvfree_rcu() implementation to slab, we still have the simple non-batching implementation in tiny RCU, and RCU implementation specific ifdefs in slab code. Finish the move and integration into slab. Allow using the simple call_rcu() based implementation also with tree RCU when SLUB_TINY is enabled, as its goal is also to limit memory footprint with less concern for top performance. In order to avoid RCU having to recognize the fake callback function pointers (__is_kvfree_rcu_offset()) when handling call_rcu(), implement a callback that can calculate the object's address from the embedded rcu_head pointer without knowing the specific offset (previously SLOB would not have made it possible, but it's gone now). After this series, AFAIK only the following kvfree_rcu specific code remains in RCU: - a call to kfree_rcu_scheduler_running() from rcu_set_runtime_mode() - probably necessary and a generic registration interface would be unnecessary bloat? - declarations of kfree_rcu() API in include/linux/rcupdate.h - could be moved to slab.h after checking for/fixing up potential missing includes git tree: https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=b4/slub-tiny-kfree_rcu Changes since RFC [1]: - Rebased to v6.14-rc1 - Fixed build errors reported by bots. - Changed where kvfree_call_rcu() is moved to in patch 1 to avoid another move in patch 4. - R-b's by Ulad and Joel - Fix a memory leak in kvfree_rcu_list() thanks to Ulad - Various comments' improvements and fixes (Joel, Ulad) - Rename config to CONFIG_KFREE_RCU_BATCHED (Ulad) Due to changes, didn't keep Paul's Tested-by (but thanks!) Will include in slab/for-next if no objection. [1] https://lore.kernel.org/all/20250123-slub-tiny-kfree_rcu-v1-0-0e386ef1541a@suse.cz/ To: Christoph Lameter To: David Rientjes Cc: Andrew Morton Cc: Roman Gushchin Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com> Cc: linux-mm@kvack.org To: "Paul E. McKenney" To: Joel Fernandes To: Josh Triplett To: Boqun Feng To: Uladzislau Rezki Cc: Steven Rostedt Cc: Mathieu Desnoyers Cc: Lai Jiangshan Cc: Zqiang Cc: rcu@vger.kernel.org Signed-off-by: Vlastimil Babka --- Vlastimil Babka (4): slab, rcu: move TINY_RCU variant of kvfree_rcu() to SLAB rcu: remove trace_rcu_kvfree_callback rcu, slab: use a regular callback function for kvfree_rcu slab: don't batch kvfree_rcu() with SLUB_TINY include/linux/rcupdate.h | 33 ++++++++++++++++++--------------- include/linux/rcutiny.h | 36 ------------------------------------ include/linux/rcutree.h | 3 --- include/linux/slab.h | 14 ++++++++++++++ include/trace/events/rcu.h | 34 ---------------------------------- kernel/rcu/tiny.c | 25 ------------------------- kernel/rcu/tree.c | 9 ++------- mm/Kconfig | 4 ++++ mm/slab.h | 2 ++ mm/slab_common.c | 33 +++++++++++++++++++++++++++------ mm/slub.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 11 files changed, 113 insertions(+), 126 deletions(-) --- base-commit: 2014c95afecee3e76ca4a56956a936e23283f05b change-id: 20250123-slub-tiny-kfree_rcu-bd65bfe222f2 Best regards,