From patchwork Mon Aug 26 06:31:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: NeilBrown X-Patchwork-Id: 13777132 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 02C3B136671; Mon, 26 Aug 2024 06:37:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724654269; cv=none; b=ZF6vgz5qM1qqJu0vBox/amIq4uieOc/n1U8TScQKra6YhgviWRGZYzM/TJRlc89wxMzRfaIsxEwqr/LCkS87bnXFTDYTC1eB9xm8XDQLhijVtxBvA6U3OcxW141J11g0Xzdo/QCIQXVbyAom1VefS1Bn0kfV/DJfW5r/CgFO0Dc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724654269; c=relaxed/simple; bh=f+YQ6nhv9D59ELA4Osdsn2vvm9PNb3OivnaHAdUAe30=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qPowdiPR3Thaj+JK5Z72U9RX0Z1o3pksIzgct7w0xCUn3LWFL3GyS14nx46FMzdJTQW0I5Uyqwow+iaWNpzvD9xDwyhWc043cVVrYZ0dB9piZ9T22CFqGwCSeH2uTBLiERWwR45NXsnHO9GLXuW8dOtyFt2ABibh08+6fbiYMdo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de; spf=pass smtp.mailfrom=suse.de; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=OosgwF+F; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=ACmOtA7f; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b=OosgwF+F; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b=ACmOtA7f; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=suse.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="OosgwF+F"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="ACmOtA7f"; dkim=pass (1024-bit key) header.d=suse.de header.i=@suse.de header.b="OosgwF+F"; dkim=permerror (0-bit key) header.d=suse.de header.i=@suse.de header.b="ACmOtA7f" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 439451F83B; Mon, 26 Aug 2024 06:37:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1724654266; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6dA2P7oI8aE0aoZymFQ2h8vW+8D9z3FjMzGxgLt+4O8=; b=OosgwF+FXEvP/Y+oYKD9SPnghUfcAKSyIUMrUyGZ1uqbZX8r0mnpwBdWXBSjbbW5pO+t2/ nbHq4HToIm64a3f7xmkngifXQAVANZWmA4m2oKOQHmnb8x2BKJ7pol5SOSmxcTHrxnJL0M MqFYQJT8jvCJVyKYzBXz92m1hsrbjb4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1724654266; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6dA2P7oI8aE0aoZymFQ2h8vW+8D9z3FjMzGxgLt+4O8=; b=ACmOtA7f85ztGOUqIzpQ4zaITbfZw8xCEj/DzuRteqFb9nZrX411IrH/BJjooexTYlEzzs JVXwL35Y12upMgBQ== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.de header.s=susede2_rsa header.b=OosgwF+F; dkim=pass header.d=suse.de header.s=susede2_ed25519 header.b=ACmOtA7f DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1724654266; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6dA2P7oI8aE0aoZymFQ2h8vW+8D9z3FjMzGxgLt+4O8=; b=OosgwF+FXEvP/Y+oYKD9SPnghUfcAKSyIUMrUyGZ1uqbZX8r0mnpwBdWXBSjbbW5pO+t2/ nbHq4HToIm64a3f7xmkngifXQAVANZWmA4m2oKOQHmnb8x2BKJ7pol5SOSmxcTHrxnJL0M MqFYQJT8jvCJVyKYzBXz92m1hsrbjb4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1724654266; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6dA2P7oI8aE0aoZymFQ2h8vW+8D9z3FjMzGxgLt+4O8=; b=ACmOtA7f85ztGOUqIzpQ4zaITbfZw8xCEj/DzuRteqFb9nZrX411IrH/BJjooexTYlEzzs JVXwL35Y12upMgBQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id D38FA13724; Mon, 26 Aug 2024 06:37:43 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id yDL0IbcizGZROAAAD6G6ig (envelope-from ); Mon, 26 Aug 2024 06:37:43 +0000 From: NeilBrown To: Ingo Molnar , Peter Zijlstra , Linus Torvalds , Jens Axboe Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org Subject: [PATCH 5/7] sched: Add test_and_clear_wake_up_bit() and atomic_dec_and_wake_up() Date: Mon, 26 Aug 2024 16:31:02 +1000 Message-ID: <20240826063659.15327-6-neilb@suse.de> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240826063659.15327-1-neilb@suse.de> References: <20240826063659.15327-1-neilb@suse.de> Precedence: bulk X-Mailing-List: linux-block@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Rspamd-Queue-Id: 439451F83B X-Spam-Level: X-Spamd-Result: default: False [-3.01 / 50.00]; BAYES_HAM(-3.00)[99.99%]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_MISSING_CHARSET(0.50)[]; R_DKIM_ALLOW(-0.20)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RBL_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:104:10:150:64:97:from]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FUZZY_BLOCKED(0.00)[rspamd.com]; DKIM_TRACE(0.00)[suse.de:+]; TO_DN_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_BLOCKED(0.00)[suse.de:dkim]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; RCPT_COUNT_SEVEN(0.00)[7]; R_RATELIMIT(0.00)[from(RLewrxuus8mos16izbn)]; RCVD_VIA_SMTP_AUTH(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.de:email,suse.de:dkim,suse.de:mid] X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -3.01 X-Spam-Flag: NO There are common patterns in the kernel of using test_and_clear_bit() before wake_up_bit(), and atomic_dec_and_test() before wake_up_var(). These combinations don't need extra barriers but sometimes include them unnecessarily. To help avoid the unnecessary barriers and to help discourage the general use of wake_up_bit/var (which is a fragile interface) introduce two combined functions which implement these patterns. Also add store_release_wake_up() which support the parts of simply setting a non-atomic variable as sending a wakeup. This pattern requires barriers which are often omitted. Signed-off-by: NeilBrown --- include/linux/wait_bit.h | 61 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/include/linux/wait_bit.h b/include/linux/wait_bit.h index ca5c6e70f908..c1675457c077 100644 --- a/include/linux/wait_bit.h +++ b/include/linux/wait_bit.h @@ -419,4 +419,65 @@ static inline void clear_and_wake_up_bit(int bit, unsigned long *word) wake_up_bit(word, bit); } +/** + * test_and_clear_wake_up_bit - clear a bit if it was set: wake up anyone waiting on that bit + * + * @bit: the bit of the word being waited on + * @word: the address of memory containing that bit + * + * If the bit is set and can be atomically cleared, any tasks waiting in + * wait_on_bit() or similar will be woken. This call has the same + * complete ordering semantics as test_and_clear_bit(). Any changes to + * memory made before this call are guaranteed to be visible after the + * corresponding wait_on_bit() completes. + * + * Returns %true if the bit was successfully set and the wake up was sent. + */ +static inline bool test_and_clear_wake_up_bit(int bit, unsigned long *word) +{ + if (!test_and_clear_bit(bit, word)) + return false; + /* no extra barrier required */ + wake_up_bit(word, bit); + return true; +} + +/** + * atomic_dec_and_wake_up - decrement an atomic_t and if zero, wake up waiters + * + * @var: the variable to dec and test + * + * Decrements the atomic variable and if it reaches zero, send a wake_up to any + * processes waiting on the variable. + * + * This function has the same complete ordering semantics as atomic_dec_and_test. + * + * Returns %true is the variable reaches zero and the wake up was sent. + */ + +static inline bool atomic_dec_and_wake_up(atomic_t *var) +{ + if (!atomic_dec_and_test(var)) + return false; + wake_up_var(var); + return true; +} + +/** + * store_release_wake_up - update a variable and send a wake_up + * @var: the address of the variable to be updated and woken + * @val: the value to store in the variable. + * + * Store the given value in the variable send a wake up to any tasks + * waiting on the variable. All necessary barriers are included to ensure + * the task calling wait_var_event() sees the new value and all values + * written to memory before this call. + */ +#define store_release_wake_up(var, val) \ +do { \ + smp_store_release(var, val); \ + smp_mb(); \ + wake_up_var(var); \ +} while (0) + #endif /* _LINUX_WAIT_BIT_H */