From patchwork Wed Jun 19 22:48:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vlastimil Babka X-Patchwork-Id: 13704692 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8CC6CC2BA1A for ; Wed, 19 Jun 2024 22:49:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ECEEE8D0084; Wed, 19 Jun 2024 18:49:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E7DF26B00BC; Wed, 19 Jun 2024 18:49:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD0768D0084; Wed, 19 Jun 2024 18:49:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id AD53B6B00BD for ; Wed, 19 Jun 2024 18:49:47 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 3861EA42BE for ; Wed, 19 Jun 2024 22:49:47 +0000 (UTC) X-FDA: 82249132014.08.FE11BD9 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf08.hostedemail.com (Postfix) with ESMTP id D6B7D16000A for ; Wed, 19 Jun 2024 22:49:44 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="smRbBT/c"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=g3E3XRzi; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="smRbBT/c"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=g3E3XRzi; spf=pass (imf08.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718837380; a=rsa-sha256; cv=none; b=4XFETcaG8/LRM3LJ2v1Zd8dwOuqR25qUIEQFgfmooExn7MzjvSMa4FTkgrBX7V9s9MxNTC 2uAH3XDRSu9hoh8IQLPXEaJ5MNN9XLvqGlO2frkRfta6DzbEgK/Dq6frvwQEaWbA3soYDz eR3oQNTpVgCxLJ+gB9wQ6fYjUzF/GFE= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="smRbBT/c"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=g3E3XRzi; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="smRbBT/c"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=g3E3XRzi; spf=pass (imf08.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.131 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718837380; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=r7eLs3qWVHkCYzNUctfASvMgvFekhPcbd8a5b9Uty8M=; b=AzhH21Oi3rS3rFLHj0Wd0s8kFZTvvd+OB2AQ1VYRmUnKDRV2I/iVvJGF8bI+qVGfxBNrX/ HoBf6XbdMdVG7PM+qX6Bj86oYtK7hl4q5tPqttgQ7AlGCkYkxb8cKvcu9+UuyudkHvXVIO UGqsG9EuqKTVXHN6hRMqDV2T9IgIDuI= Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 25D261F7F5; Wed, 19 Jun 2024 22:49:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1718837383; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r7eLs3qWVHkCYzNUctfASvMgvFekhPcbd8a5b9Uty8M=; b=smRbBT/csZSGJtlsJ4JN3oklSIC7vbquyuWyYFQK4X7uOph5tB+4a752AntXvJy1S6SyqH IS3Q3mHFdaV3Nuk7YxxoaL7OyDll3aJ5H4tMiqF2zEByxywgSPn6DUWnsDAtqd2l0oVEwI Ih3Y7GYugjJN390WQpBeTcoBGFUj6Z4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1718837383; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r7eLs3qWVHkCYzNUctfASvMgvFekhPcbd8a5b9Uty8M=; b=g3E3XRziF6HgkAAPS88wKQ5lNo4texbHFNeFeEwV0v4mYHhP+i771BZm1quZ/kUsnkbfs9 f1WTtAqzsMR7PdDw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1718837383; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r7eLs3qWVHkCYzNUctfASvMgvFekhPcbd8a5b9Uty8M=; b=smRbBT/csZSGJtlsJ4JN3oklSIC7vbquyuWyYFQK4X7uOph5tB+4a752AntXvJy1S6SyqH IS3Q3mHFdaV3Nuk7YxxoaL7OyDll3aJ5H4tMiqF2zEByxywgSPn6DUWnsDAtqd2l0oVEwI Ih3Y7GYugjJN390WQpBeTcoBGFUj6Z4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1718837383; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=r7eLs3qWVHkCYzNUctfASvMgvFekhPcbd8a5b9Uty8M=; b=g3E3XRziF6HgkAAPS88wKQ5lNo4texbHFNeFeEwV0v4mYHhP+i771BZm1quZ/kUsnkbfs9 f1WTtAqzsMR7PdDw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0152E13ABD; Wed, 19 Jun 2024 22:49:42 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 0PuTO4Zgc2aFIAAAD6G6ig (envelope-from ); Wed, 19 Jun 2024 22:49:42 +0000 From: Vlastimil Babka Date: Thu, 20 Jun 2024 00:48:55 +0200 Subject: [PATCH v2 1/7] fault-inject: add support for static keys around fault injection sites MIME-Version: 1.0 Message-Id: <20240620-fault-injection-statickeys-v2-1-e23947d3d84b@suse.cz> References: <20240620-fault-injection-statickeys-v2-0-e23947d3d84b@suse.cz> In-Reply-To: <20240620-fault-injection-statickeys-v2-0-e23947d3d84b@suse.cz> To: Akinobu Mita , Christoph Lameter , David Rientjes , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , "Naveen N. Rao" , Anil S Keshavamurthy , "David S. Miller" , Masami Hiramatsu , Steven Rostedt , Mark Rutland Cc: Jiri Olsa , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-kernel@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, linux-trace-kernel@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4698; i=vbabka@suse.cz; h=from:subject:message-id; bh=/cKM1m+IMVDzuJzFS+1fl48bPoWvR2tEgzVaVUXVT8A=; b=owEBbQGS/pANAwAIAbvgsHXSRYiaAcsmYgBmc2Byf1V3Bpp5LNGsY/c7Mzv3/GsPZ55FjBkLd LX7xvgBpSWJATMEAAEIAB0WIQR7u8hBFZkjSJZITfG74LB10kWImgUCZnNgcgAKCRC74LB10kWI mrooB/987gBj0pha0dvAhFXZiGUJdrVCDtQ6t2BV/SIlkveQM37wjVIcJKlT46POmu0EAVLbGts rfVWVX5e4aCI80nZ5KpkSs0COyqzR+4Eym09R0s0Mm29ctQ8BJYXISybtjvacF6lT2sDF1PvZG8 i3JzV58YERvX5BU+hxW2HjtgpEHgCzBBjFK/t/heKLFCeork00ZpNgQXER2fUDtpVCd4m62wTb9 aJr4p+K+XdUOvo164UnmBjTbqVN9BjUYA1laDaYyZ5+8942w7HBeyumYAqx3oJBarTi3ujoosjc 1Yyw2OhX6svXSoURISrllNMod+BTK/2jVgFrBLmcWyLPRwOW X-Developer-Key: i=vbabka@suse.cz; a=openpgp; fpr=A940D434992C2E8E99103D50224FA7E7CC82A664 X-Stat-Signature: k5ycomu18w6zjhefuuqgmqjh5wfjsh3m X-Rspamd-Queue-Id: D6B7D16000A X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1718837384-733428 X-HE-Meta: U2FsdGVkX18hz2Rgi/1uA+034FdGfgN4injVQLgWiI/jcdtiSzH0KNO632uSu4HtmDOvtpBeXyHLG3i5jNI5k1JeZj9OWtU1rjv4wiB1xUgFYbcGh4HYpf/d7cKlQvzq03SOQYPJCCq+yzPO7NT+omYN9iBWnxI0eR2NFlVgvBUO/t8sxDyQOGqHcMChzlA0oUZcIe3k0pN15aDJAJYX/s2ShbFGwznvpGbjoIpDQP0Ll/k12CUaII1ktT6JcXYBplWFLpzPNjo3yAVJt9VOU3rJhzhp2dmoVlTW0uuLVCYD+Pw38nR5ujjCSDk7Yxz8IgZktI4D00ZREwR4BAU4oHoR1KfHFcgkqSFolpaIDslXZVuszHcthUxC0s3LRnLEob5/PoID2kRvFVxHb496fPJSsDouyaFb1dzPU2Apvm60kzTEUrR4MphPzB2HVjpHE5y4AsRUsADq0KoQcQfWjXkJlvaWcpt3KYcgHnx36NXZ+l39mXXa/FHmnmji4+jCOdxRa7AH7eH7xd0sLk7zRFM1+iKBj6Q9G05GSstKN3dlSqPUN6IeU6PLRuG7pOYWcjetB6SHR8QkbcGoI5oufkvDYz/g4a38/cQBiQriiy8SrfdYuQWfTRKddr5PDOowM5gcUomKVrkaIJlYod/jBzXvrYxIU8u3/7v3SfZliAayK/RqxIwGghnGoxO1qhNh5vEWIREChV9nORB7+jL4/Vku0AcZNS/5Tl+mpoanHxXi9/zIsovGIWKeP9bxbjOaZo09dDYskdHoB2ge5zOBTQdIbGx4uQ3DqPtE8HYVnGgLD1RrPif+/vZCX4KJe85ucuUIyA7DqKJA6HPz1pXNghd8kFd30WD2ux1SpbS0SxTQ1iwPdGjKWv38Gb8gx/A0RQQjeHDpz7DBbr7FhmdqavA3cPmzDpyV2rzTvaUcIirpcx1477UCELWW6oflJ3pE9yTSFMlBWVyGF2K4JXV l3LYveSm sEzEVVEXJGHpIF5U3mD9Zk8hN4XHhNW64Le/crCZl+h+xAIi0zeO11UpNBifxBSOVFOuujLz848NTe4ml4rVKErfZFKaW8Yuq/o81PsPGuKWoa3uFVnWBzyzc6M6pEKubM1+yW1bJ68M5leBu7SzZd47jhI8KJ/ldifd5EDqd4sXZ20yr6hs0KegjXTSCMc3v7oZ5KA77+xFohvmo+9qpHP5dOEZfz1wM9lVCQBjlGb3kvtmquuSiWMWLT6lrQWh2yokmz8hrmk7EI9qkMUaDOY0Pw6AqfjBgulMgco2qTdEDF7Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Some fault injection sites are placed in hotpaths and incur overhead even if not enabled, due to one or more function calls leading up to should_fail_ex() that returns false due to attr->probability == 0. This overhead can be eliminated if the outermost call into the checks is guarded with a static key, so add support for that. The framework should be told that such static key exist for a fault_attr, by initializing fault_attr->active with the static key address. When it's not NULL, enable the static key from setup_fault_attr() when the fault probability is non-zero. Also wire up writing into debugfs "probability" file to enable or disable the static key when transitioning between zero and non-zero probability. For now, do not add configfs interface support as the immediate plan is to leverage this for should_failslab() and should_fail_alloc_page() after other necessary preparatory changes, and not for any of the configfs based fault injection users. Signed-off-by: Vlastimil Babka --- include/linux/fault-inject.h | 7 ++++++- lib/fault-inject.c | 43 ++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 48 insertions(+), 2 deletions(-) diff --git a/include/linux/fault-inject.h b/include/linux/fault-inject.h index 6d5edef09d45..cfe75cc1bac4 100644 --- a/include/linux/fault-inject.h +++ b/include/linux/fault-inject.h @@ -9,6 +9,7 @@ #include #include #include +#include /* * For explanation of the elements of this struct, see @@ -30,13 +31,14 @@ struct fault_attr { unsigned long count; struct ratelimit_state ratelimit_state; struct dentry *dname; + struct static_key *active; }; enum fault_flags { FAULT_NOWARN = 1 << 0, }; -#define FAULT_ATTR_INITIALIZER { \ +#define FAULT_ATTR_INITIALIZER_KEY(_key) { \ .interval = 1, \ .times = ATOMIC_INIT(1), \ .require_end = ULONG_MAX, \ @@ -44,8 +46,11 @@ enum fault_flags { .ratelimit_state = RATELIMIT_STATE_INIT_DISABLED, \ .verbose = 2, \ .dname = NULL, \ + .active = (_key), \ } +#define FAULT_ATTR_INITIALIZER FAULT_ATTR_INITIALIZER_KEY(NULL) + #define DECLARE_FAULT_ATTR(name) struct fault_attr name = FAULT_ATTR_INITIALIZER int setup_fault_attr(struct fault_attr *attr, char *str); bool should_fail_ex(struct fault_attr *attr, ssize_t size, int flags); diff --git a/lib/fault-inject.c b/lib/fault-inject.c index d608f9b48c10..de9552cb22d0 100644 --- a/lib/fault-inject.c +++ b/lib/fault-inject.c @@ -35,6 +35,9 @@ int setup_fault_attr(struct fault_attr *attr, char *str) atomic_set(&attr->times, times); atomic_set(&attr->space, space); + if (probability != 0 && attr->active) + static_key_slow_inc(attr->active); + return 1; } EXPORT_SYMBOL_GPL(setup_fault_attr); @@ -166,6 +169,12 @@ EXPORT_SYMBOL_GPL(should_fail); #ifdef CONFIG_FAULT_INJECTION_DEBUG_FS +/* + * Protect updating probability from debugfs as that may trigger static key + * changes when changing between zero and non-zero. + */ +static DEFINE_MUTEX(probability_mutex); + static int debugfs_ul_set(void *data, u64 val) { *(unsigned long *)data = val; @@ -186,6 +195,38 @@ static void debugfs_create_ul(const char *name, umode_t mode, debugfs_create_file(name, mode, parent, value, &fops_ul); } +static int debugfs_prob_set(void *data, u64 val) +{ + struct fault_attr *attr = data; + + mutex_lock(&probability_mutex); + + if (attr->active) { + if (attr->probability != 0 && val == 0) { + static_key_slow_dec(attr->active); + } else if (attr->probability == 0 && val != 0) { + static_key_slow_inc(attr->active); + } + } + + attr->probability = val; + + mutex_unlock(&probability_mutex); + + return 0; +} + +static int debugfs_prob_get(void *data, u64 *val) +{ + struct fault_attr *attr = data; + + *val = attr->probability; + + return 0; +} + +DEFINE_SIMPLE_ATTRIBUTE(fops_prob, debugfs_prob_get, debugfs_prob_set, "%llu\n"); + #ifdef CONFIG_FAULT_INJECTION_STACKTRACE_FILTER static int debugfs_stacktrace_depth_set(void *data, u64 val) @@ -218,7 +259,7 @@ struct dentry *fault_create_debugfs_attr(const char *name, if (IS_ERR(dir)) return dir; - debugfs_create_ul("probability", mode, dir, &attr->probability); + debugfs_create_file("probability", mode, dir, attr, &fops_prob); debugfs_create_ul("interval", mode, dir, &attr->interval); debugfs_create_atomic_t("times", mode, dir, &attr->times); debugfs_create_atomic_t("space", mode, dir, &attr->space);