From patchwork Sun Jul 24 12:25:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 12927526 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5755DC433EF for ; Sun, 24 Jul 2022 12:25:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=kBxl61PUeRoGa+Bg4EUORc6DvaaViY4/QEWjIK+D3x8=; b=HkUQ4Rv0zMmq45 +39jMznPJTL+zDOxe7uUuFWTKtBO+25UrZmnxCiONgWZjKiQZ3qJ/HgpPeB/j7VZUOlkVYuzRo8Dw J7dLgzWLfgaLUvVzZCatQ/MDYUYP94AXtZr7tkO63PQX9V/1yyZrylSTf1tTkMXm5admSCxDPSQro XHDy9i999hxhRs86ZavlkMMv+Phf6YhQxAmHDuCjnjq7+ckCvWNYrCiH3FJff5PeClCvsdHhRAsVY a7/dc0fw7qfW0+FTTOtP126mxNDmaUlwu9dMT488Ox5vhwHgteJ/DpDr8PBqLNFUNbtrwMD6ljTgB R91aCkFkwQ5cNy9+7BPA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFag4-0060Z0-Re; Sun, 24 Jul 2022 12:25:44 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFag2-0060UU-Hi for linux-riscv@lists.infradead.org; Sun, 24 Jul 2022 12:25:43 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id BAB5561050; Sun, 24 Jul 2022 12:25:35 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C85A0C341C0; Sun, 24 Jul 2022 12:25:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1658665535; bh=x8ThgiIuKR7Cb/zZ7HilMYDMWlq4+j+CW+ILWn7Ti5k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kKZZEb6YKDRcOlsszyCzQ7M/uNpEnrWtnxpdvQHnPWNz6K51FSe01RcVzp+UyRzpo Y7gWUT7sY12kIkJR5bdYjqslJJh+yrQ8N0P1Skiv+7a2AThxxRTaRpSyzoJ5Ao7wwS irOpP/RkMvz7YrrfbUqsK/Gmt+hqmkZ2OlZd6LIwe8Nv5YxR/98QnkRfte95q2QhFg o+ZJTSbNZbOiaTbHrLMktgKfNHuiFCb5yAG7DMzP41IP9xpwjfN4fOnNOv2wzsFOov IpfSE7107PyFic9a3E9sYqpNyD84Ki2YVyFeZP3R2ugR4GwMP2A4Sx9tdkWPhPMeP/ Yp2xWgZPk1Lcw== From: guoren@kernel.org To: palmer@rivosinc.com, heiko@sntech.de, hch@infradead.org, arnd@arndb.de, peterz@infradead.org, will@kernel.org, boqun.feng@gmail.com, longman@redhat.com, mingo@redhat.com, philipp.tomsich@vrull.eu, cmuellner@linux.com, linux-kernel@vger.kernel.org, David.Laight@ACULAB.COM Cc: linux-riscv@lists.infradead.org, linux-csky@vger.kernel.org, Guo Ren , Guo Ren Subject: [PATCH V8 01/10] asm-generic: ticket-lock: Remove unnecessary atomic_read Date: Sun, 24 Jul 2022 08:25:08 -0400 Message-Id: <20220724122517.1019187-2-guoren@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220724122517.1019187-1-guoren@kernel.org> References: <20220724122517.1019187-1-guoren@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220724_052542_645337_A25CE799 X-CRM114-Status: UNSURE ( 9.55 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Guo Ren Remove unnecessary atomic_read in arch_spin_value_unlocked(lock), because the value has been in lock. This patch could prevent arch_spin_value_unlocked contend spin_lock data again. Signed-off-by: Guo Ren Signed-off-by: Guo Ren --- include/asm-generic/spinlock.h | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h index fdfebcb050f4..90803a826ba0 100644 --- a/include/asm-generic/spinlock.h +++ b/include/asm-generic/spinlock.h @@ -68,11 +68,18 @@ static __always_inline void arch_spin_unlock(arch_spinlock_t *lock) smp_store_release(ptr, (u16)val + 1); } +static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock) +{ + u32 val = lock.counter; + + return ((val >> 16) == (val & 0xffff)); +} + static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock) { - u32 val = atomic_read(lock); + arch_spinlock_t val = READ_ONCE(*lock); - return ((val >> 16) != (val & 0xffff)); + return !arch_spin_value_unlocked(val); } static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock) @@ -82,11 +89,6 @@ static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock) return (s16)((val >> 16) - (val & 0xffff)) > 1; } -static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock) -{ - return !arch_spin_is_locked(&lock); -} - #include #endif /* __ASM_GENERIC_SPINLOCK_H */ From patchwork Sun Jul 24 12:25:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 12927527 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6F992CCA48A for ; Sun, 24 Jul 2022 12:25:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=5jk3WsB8334eNTcYCXwRCTFiGyK7R5KVf4kGErfDj+g=; b=dmcGWFKcgXPIbZ hzI1u44WQ6Z9fFGZia3pi85tM4icIMmhIOsunoiFSmQZIeOYz5Xzm2Iwd+Gcvuyi6bBVxlQetXmYx CXWlO2H5wk0FxowbgfbjnFRgEFBX+2FVh5Uyvt1C+qEIzSUmttlhOw8ixFn7ujyc7J0oRQRJ2jgHX FVhgbzmOrAv/AOaff8z2oOy1LkmerySv1vuN/aIfeOpvHyvH/CV5iaBnJ9fP/IQDUcW2ztJhfIW04 nf3ZYpHop0ZMOiufNaL5AKgQM4MKvxHywh1/RVM7rpCd+PYC0owdlKO7Clpn/4Guk193AIry++dqN xfV9PngMP5Gteb5vq0NQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFag8-0060bi-LB; Sun, 24 Jul 2022 12:25:48 +0000 Received: from ams.source.kernel.org ([145.40.68.75]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFag5-0060YR-Aj for linux-riscv@lists.infradead.org; Sun, 24 Jul 2022 12:25:47 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id BAED6B80D6E; Sun, 24 Jul 2022 12:25:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D8681C3411E; Sun, 24 Jul 2022 12:25:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1658665541; bh=thtaHW8jAZpgDJtozuaQ3X5L4SWn53gdQRWmJtXyzQQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UnTJUpGnWFgQUfFUaP/J8Gr85hqNtITgyPJ7f1lf8YOMcFr04jUI8QGAG2EgxANxL rSNPl3PymOV0CzGRikYoGCPmxfc4oEmFPAZgpRRN/C7P82V0baZnBD7Jpd00B9coJ/ ZspOuxxtg9RnIUfGkW8bnM8V8nMj8T6Ks7hvrjOLGkd+0Z0RvGA5Zm0fRcOuwv3cEF dYAXbGNDX/f/FHqbrJ47l1ZAuxHR22CZmysINqyYZ40cefYoFpJrijBG7UaioUW0xB 8NAnGWNyYqgygeS4KcEwwMt/+pqV19pmDuvLdRZi5CMl0q8FoMVZANeK0PqCtNrp/6 TGaXv+x+jYWZQ== From: guoren@kernel.org To: palmer@rivosinc.com, heiko@sntech.de, hch@infradead.org, arnd@arndb.de, peterz@infradead.org, will@kernel.org, boqun.feng@gmail.com, longman@redhat.com, mingo@redhat.com, philipp.tomsich@vrull.eu, cmuellner@linux.com, linux-kernel@vger.kernel.org, David.Laight@ACULAB.COM Cc: linux-riscv@lists.infradead.org, linux-csky@vger.kernel.org, Guo Ren , Guo Ren Subject: [PATCH V8 02/10] asm-generic: ticket-lock: Use the same struct definitions with qspinlock Date: Sun, 24 Jul 2022 08:25:09 -0400 Message-Id: <20220724122517.1019187-3-guoren@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220724122517.1019187-1-guoren@kernel.org> References: <20220724122517.1019187-1-guoren@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220724_052545_708697_8AA95B45 X-CRM114-Status: GOOD ( 11.26 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Guo Ren Let ticket_lock use the same struct definitions with qspinlock, and then we could move to combo spinlock (combine ticket & queue). Signed-off-by: Guo Ren Signed-off-by: Guo Ren Reported-by: kernel test robot --- include/asm-generic/spinlock.h | 14 +++++++------- include/asm-generic/spinlock_types.h | 12 ++---------- 2 files changed, 9 insertions(+), 17 deletions(-) diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h index 90803a826ba0..4773334ee638 100644 --- a/include/asm-generic/spinlock.h +++ b/include/asm-generic/spinlock.h @@ -32,7 +32,7 @@ static __always_inline void arch_spin_lock(arch_spinlock_t *lock) { - u32 val = atomic_fetch_add(1<<16, lock); + u32 val = atomic_fetch_add(1<<16, &lock->val); u16 ticket = val >> 16; if (ticket == (u16)val) @@ -46,31 +46,31 @@ static __always_inline void arch_spin_lock(arch_spinlock_t *lock) * have no outstanding writes due to the atomic_fetch_add() the extra * orderings are free. */ - atomic_cond_read_acquire(lock, ticket == (u16)VAL); + atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL); smp_mb(); } static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock) { - u32 old = atomic_read(lock); + u32 old = atomic_read(&lock->val); if ((old >> 16) != (old & 0xffff)) return false; - return atomic_try_cmpxchg(lock, &old, old + (1<<16)); /* SC, for RCsc */ + return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */ } static __always_inline void arch_spin_unlock(arch_spinlock_t *lock) { u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN); - u32 val = atomic_read(lock); + u32 val = atomic_read(&lock->val); smp_store_release(ptr, (u16)val + 1); } static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock) { - u32 val = lock.counter; + u32 val = lock.val.counter; return ((val >> 16) == (val & 0xffff)); } @@ -84,7 +84,7 @@ static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock) static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock) { - u32 val = atomic_read(lock); + u32 val = atomic_read(&lock->val); return (s16)((val >> 16) - (val & 0xffff)) > 1; } diff --git a/include/asm-generic/spinlock_types.h b/include/asm-generic/spinlock_types.h index 8962bb730945..f534aa5de394 100644 --- a/include/asm-generic/spinlock_types.h +++ b/include/asm-generic/spinlock_types.h @@ -3,15 +3,7 @@ #ifndef __ASM_GENERIC_SPINLOCK_TYPES_H #define __ASM_GENERIC_SPINLOCK_TYPES_H -#include -typedef atomic_t arch_spinlock_t; - -/* - * qrwlock_types depends on arch_spinlock_t, so we must typedef that before the - * include. - */ -#include - -#define __ARCH_SPIN_LOCK_UNLOCKED ATOMIC_INIT(0) +#include +#include #endif /* __ASM_GENERIC_SPINLOCK_TYPES_H */ From patchwork Sun Jul 24 12:25:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 12927528 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D70D0CCA48A for ; Sun, 24 Jul 2022 12:26:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=qKKnr+BDJHJQjdIQ1kjo3iaYpPclAOEaRzYDJhbpUjs=; b=pTnNYc39hQlTHt JDebhpEJKSfbT5j5NzwZKckKB8rbPFUWVSiIGkiPhWBdVfh9N484BapY5EKc4enKZpXF6/U84w8k6 AMGqEtKB2A9S1X5WegWuJVIc/7Yw6OSJUE5OI8AuLpG4lRgfeBJB3f7WZWrDMrKesmMYl3saC+fAS ls7zmbhVgJw/nNuCWhpJPTpMBdID7amydaEwYsrNNKwDhhxvI7V+syniqWS2lezvdXYhIrXz6cK5W GTkFGxA7xDh3kIHy9Qo7cW8U+vK2sJJK4x7ngCYUZuI9m+BgihkWveAAFdFUdSdjuF9gQakrh4w83 I2a3c/gUruf355LzR+oA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFagE-0060fH-5F; Sun, 24 Jul 2022 12:25:54 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFagA-0060c1-CZ for linux-riscv@lists.infradead.org; Sun, 24 Jul 2022 12:25:52 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 1B369B80D66; Sun, 24 Jul 2022 12:25:49 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 45A7FC341C0; Sun, 24 Jul 2022 12:25:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1658665547; bh=1OxxvY2VVFPNhlKsoDbkdnbPpOuWZTQIuVCNxY6qQEI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ic84X6qjbX+xF8Qk/mlstm+l5D6UZSETZaKaD0Tp9ECRV8O0gaA7sncecFUzqaRKu SOAIBxonlhVrLVT+VWpSA5YExN44x7SwrHsK1a5O4sLkbh9D4gF3vQm5yXtj/JSjbr M2bZjwttuGTjAPFi84OXDKKazWrqoQ8Q1qVtHJ/SFiRgp0vsX41Fh14cBJIhno4NQk vghaUYIMF5JkUwPG4V2DDdTuMxiQ+Ymk7hc3kixUyjrIz6hng3sjTp61VKc0BIOtyr Bgj3PBiRGtcWJ5zexefTYzDlDcQEntYjzzFVIhzg/9jFePf9YN2noqDLFS7l2MWebs 4ocBUPFXwDBzw== From: guoren@kernel.org To: palmer@rivosinc.com, heiko@sntech.de, hch@infradead.org, arnd@arndb.de, peterz@infradead.org, will@kernel.org, boqun.feng@gmail.com, longman@redhat.com, mingo@redhat.com, philipp.tomsich@vrull.eu, cmuellner@linux.com, linux-kernel@vger.kernel.org, David.Laight@ACULAB.COM Cc: linux-riscv@lists.infradead.org, linux-csky@vger.kernel.org, Guo Ren , Guo Ren Subject: [PATCH V8 03/10] asm-generic: ticket-lock: Move into ticket_spinlock.h Date: Sun, 24 Jul 2022 08:25:10 -0400 Message-Id: <20220724122517.1019187-4-guoren@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220724122517.1019187-1-guoren@kernel.org> References: <20220724122517.1019187-1-guoren@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220724_052550_762999_AF982145 X-CRM114-Status: GOOD ( 20.87 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Guo Ren Move ticket-lock definition into an independent file. It's a preparation patch for the following combo spinlock. Signed-off-by: Guo Ren Signed-off-by: Guo Ren --- include/asm-generic/spinlock.h | 87 +--------------------- include/asm-generic/ticket_spinlock.h | 103 ++++++++++++++++++++++++++ 2 files changed, 104 insertions(+), 86 deletions(-) create mode 100644 include/asm-generic/ticket_spinlock.h diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h index 4773334ee638..970590baf61b 100644 --- a/include/asm-generic/spinlock.h +++ b/include/asm-generic/spinlock.h @@ -1,94 +1,9 @@ /* SPDX-License-Identifier: GPL-2.0 */ -/* - * 'Generic' ticket-lock implementation. - * - * It relies on atomic_fetch_add() having well defined forward progress - * guarantees under contention. If your architecture cannot provide this, stick - * to a test-and-set lock. - * - * It also relies on atomic_fetch_add() being safe vs smp_store_release() on a - * sub-word of the value. This is generally true for anything LL/SC although - * you'd be hard pressed to find anything useful in architecture specifications - * about this. If your architecture cannot do this you might be better off with - * a test-and-set. - * - * It further assumes atomic_*_release() + atomic_*_acquire() is RCpc and hence - * uses atomic_fetch_add() which is RCsc to create an RCsc hot path, along with - * a full fence after the spin to upgrade the otherwise-RCpc - * atomic_cond_read_acquire(). - * - * The implementation uses smp_cond_load_acquire() to spin, so if the - * architecture has WFE like instructions to sleep instead of poll for word - * modifications be sure to implement that (see ARM64 for example). - * - */ - #ifndef __ASM_GENERIC_SPINLOCK_H #define __ASM_GENERIC_SPINLOCK_H -#include -#include - -static __always_inline void arch_spin_lock(arch_spinlock_t *lock) -{ - u32 val = atomic_fetch_add(1<<16, &lock->val); - u16 ticket = val >> 16; - - if (ticket == (u16)val) - return; - - /* - * atomic_cond_read_acquire() is RCpc, but rather than defining a - * custom cond_read_rcsc() here we just emit a full fence. We only - * need the prior reads before subsequent writes ordering from - * smb_mb(), but as atomic_cond_read_acquire() just emits reads and we - * have no outstanding writes due to the atomic_fetch_add() the extra - * orderings are free. - */ - atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL); - smp_mb(); -} - -static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock) -{ - u32 old = atomic_read(&lock->val); - - if ((old >> 16) != (old & 0xffff)) - return false; - - return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */ -} - -static __always_inline void arch_spin_unlock(arch_spinlock_t *lock) -{ - u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN); - u32 val = atomic_read(&lock->val); - - smp_store_release(ptr, (u16)val + 1); -} - -static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock) -{ - u32 val = lock.val.counter; - - return ((val >> 16) == (val & 0xffff)); -} - -static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock) -{ - arch_spinlock_t val = READ_ONCE(*lock); - - return !arch_spin_value_unlocked(val); -} - -static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock) -{ - u32 val = atomic_read(&lock->val); - - return (s16)((val >> 16) - (val & 0xffff)) > 1; -} - +#include #include #endif /* __ASM_GENERIC_SPINLOCK_H */ diff --git a/include/asm-generic/ticket_spinlock.h b/include/asm-generic/ticket_spinlock.h new file mode 100644 index 000000000000..cfcff22b37b3 --- /dev/null +++ b/include/asm-generic/ticket_spinlock.h @@ -0,0 +1,103 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +/* + * 'Generic' ticket-lock implementation. + * + * It relies on atomic_fetch_add() having well defined forward progress + * guarantees under contention. If your architecture cannot provide this, stick + * to a test-and-set lock. + * + * It also relies on atomic_fetch_add() being safe vs smp_store_release() on a + * sub-word of the value. This is generally true for anything LL/SC although + * you'd be hard pressed to find anything useful in architecture specifications + * about this. If your architecture cannot do this you might be better off with + * a test-and-set. + * + * It further assumes atomic_*_release() + atomic_*_acquire() is RCpc and hence + * uses atomic_fetch_add() which is RCsc to create an RCsc hot path, along with + * a full fence after the spin to upgrade the otherwise-RCpc + * atomic_cond_read_acquire(). + * + * The implementation uses smp_cond_load_acquire() to spin, so if the + * architecture has WFE like instructions to sleep instead of poll for word + * modifications be sure to implement that (see ARM64 for example). + * + */ + +#ifndef __ASM_GENERIC_TICKET_SPINLOCK_H +#define __ASM_GENERIC_TICKET_SPINLOCK_H + +#include +#include + +static __always_inline void ticket_spin_lock(arch_spinlock_t *lock) +{ + u32 val = atomic_fetch_add(1<<16, &lock->val); + u16 ticket = val >> 16; + + if (ticket == (u16)val) + return; + + /* + * atomic_cond_read_acquire() is RCpc, but rather than defining a + * custom cond_read_rcsc() here we just emit a full fence. We only + * need the prior reads before subsequent writes ordering from + * smb_mb(), but as atomic_cond_read_acquire() just emits reads and we + * have no outstanding writes due to the atomic_fetch_add() the extra + * orderings are free. + */ + atomic_cond_read_acquire(&lock->val, ticket == (u16)VAL); + smp_mb(); +} + +static __always_inline bool ticket_spin_trylock(arch_spinlock_t *lock) +{ + u32 old = atomic_read(&lock->val); + + if ((old >> 16) != (old & 0xffff)) + return false; + + return atomic_try_cmpxchg(&lock->val, &old, old + (1<<16)); /* SC, for RCsc */ +} + +static __always_inline void ticket_spin_unlock(arch_spinlock_t *lock) +{ + u16 *ptr = (u16 *)lock + IS_ENABLED(CONFIG_CPU_BIG_ENDIAN); + u32 val = atomic_read(&lock->val); + + smp_store_release(ptr, (u16)val + 1); +} + +static __always_inline int ticket_spin_value_unlocked(arch_spinlock_t lock) +{ + u32 val = lock.val.counter; + + return ((val >> 16) == (val & 0xffff)); +} + +static __always_inline int ticket_spin_is_locked(arch_spinlock_t *lock) +{ + arch_spinlock_t val = READ_ONCE(*lock); + + return !ticket_spin_value_unlocked(val); +} + +static __always_inline int ticket_spin_is_contended(arch_spinlock_t *lock) +{ + u32 val = atomic_read(&lock->val); + + return (s16)((val >> 16) - (val & 0xffff)) > 1; +} + +/* + * Remapping spinlock architecture specific functions to the corresponding + * ticket spinlock functions. + */ +#define arch_spin_is_locked(l) ticket_spin_is_locked(l) +#define arch_spin_is_contended(l) ticket_spin_is_contended(l) +#define arch_spin_value_unlocked(l) ticket_spin_value_unlocked(l) +#define arch_spin_lock(l) ticket_spin_lock(l) +#define arch_spin_trylock(l) ticket_spin_trylock(l) +#define arch_spin_unlock(l) ticket_spin_unlock(l) + +#endif /* __ASM_GENERIC_TICKET_SPINLOCK_H */ From patchwork Sun Jul 24 12:25:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 12927529 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E432CC43334 for ; Sun, 24 Jul 2022 12:26:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=6OxDFELoSQ3KE0c7/k+u5X6PjY4a9uX/mz9wtrRi2Ns=; b=ID83X9zxtb0LwY PT+gbcJnEbAKp/2xGHGi6PNjHCfLgIHO/YduRKHeT+xyDf9XrS+vX5182vF/g6t24NUP7a0flE67R XXkLbvqWRDN0Jtqn9kBkIJdNzlTqqWIlWau78v/iek7BruTeXTGgFGPnAOuG2sgoUzOtGM0Ti/tuS +8VIrJ4n8EsaDcuEUylmHEd9trOd6tgd0xFAPf1RGZW64+OVmpiWn/A39qTR04tHRRElt03hoBBb1 JhyLSD3KWWm3FnLYNBalXwPJftSnQzQ17yiSi8KpI28Sj4+g27blCVk63CXdJEUd0q0jnW+brYkKb AVMHtyob9LOHRSx+5zXg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFagH-0060hy-Mo; Sun, 24 Jul 2022 12:25:57 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFagF-0060fj-1I for linux-riscv@lists.infradead.org; Sun, 24 Jul 2022 12:25:56 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 78DD861046; Sun, 24 Jul 2022 12:25:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8ED10C3411E; Sun, 24 Jul 2022 12:25:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1658665553; bh=CYylZgBgEWXe/AK9bUnLJ70ppNnmLn//vvrYUoL+UXI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JVZvlP9Ejw4lMPQy9eEKtpevIxYHCUuAJHNWVftrUiiPbLA6CEvWstyMqgThL2VIM jCzHwe/wyo3ChC2lq5GMdeHceQd+egrOOCvIiN3TKMsUI/9kxpJCm2QpCYg0Se499E xeacO7N5aY+CSiOuS8ppYoGGJQydpb9LFXBi9C6D03umN5R4042EqEOP3haWtDakKt Vqx6jvXQGrfbC34mSfu3db79meraV8ESab4BJ7IJvXW1LdwqLDLc+9Lj4NhiNSSj59 jFRq+5yoSzp/FTvGcMAarEZ0sav8r2nOKGM08aglcJuNdF6tjrbQfQneIQB09W8240 oMDxTQN27i6CQ== From: guoren@kernel.org To: palmer@rivosinc.com, heiko@sntech.de, hch@infradead.org, arnd@arndb.de, peterz@infradead.org, will@kernel.org, boqun.feng@gmail.com, longman@redhat.com, mingo@redhat.com, philipp.tomsich@vrull.eu, cmuellner@linux.com, linux-kernel@vger.kernel.org, David.Laight@ACULAB.COM Cc: linux-riscv@lists.infradead.org, linux-csky@vger.kernel.org, Guo Ren , Guo Ren Subject: [PATCH V8 04/10] asm-generic: spinlock: Add queued spinlock support in common header Date: Sun, 24 Jul 2022 08:25:11 -0400 Message-Id: <20220724122517.1019187-5-guoren@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220724122517.1019187-1-guoren@kernel.org> References: <20220724122517.1019187-1-guoren@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220724_052555_122372_7AB88BB2 X-CRM114-Status: UNSURE ( 8.75 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Guo Ren Select queued spinlock or ticket lock by CONFIG_QUEUED_SPINLOCKS in the common header file. Signed-off-by: Guo Ren Signed-off-by: Guo Ren --- include/asm-generic/spinlock.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/asm-generic/spinlock.h b/include/asm-generic/spinlock.h index 970590baf61b..f1c29db9c8b5 100644 --- a/include/asm-generic/spinlock.h +++ b/include/asm-generic/spinlock.h @@ -3,7 +3,11 @@ #ifndef __ASM_GENERIC_SPINLOCK_H #define __ASM_GENERIC_SPINLOCK_H +#ifdef CONFIG_QUEUED_SPINLOCKS +#include +#else #include +#endif #include #endif /* __ASM_GENERIC_SPINLOCK_H */ From patchwork Sun Jul 24 12:25:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 12927530 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DDFAAC43334 for ; Sun, 24 Jul 2022 12:26:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=zL0DXPfopZKVxyEDWG40eWv0gqIcNlI0hIBx7iRYixs=; b=thsFiKIvJc4W8T cfksbhxwWuIMwJ/27Jm1kSSy4qPDfDR+bk8cWEd70KVJTWAaLJM0sY8A6OBnuYt/UgCQ8qysTtQiP ffODHlC6dE16BA2ABBjhUSAlUCFbCI5NWRtNd7pMmSWrJg1ajS8KkMf6+ejKDxo0GQ01sdVfZUOLl FLWF7RTe7I2oUTb6DtJfT7mdtUgrd/ZrxMEcCc+HVbSK9vSBNgP2gwmQ6baZU+BcCDGwXSTHQmGCt KtPw3aEpGOblfI7uWdmHwpjyVe4PDxGXPCYJldJvYzQ+RgVIJoc6ftwf/Zjex8qGmHmNsGPyXT88w 9ORgBw89IIwwq9rbptoQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFagP-0060oW-Go; Sun, 24 Jul 2022 12:26:05 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFagM-0060lf-Kd for linux-riscv@lists.infradead.org; Sun, 24 Jul 2022 12:26:04 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 477E2B80D6B; Sun, 24 Jul 2022 12:26:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 97375C3411E; Sun, 24 Jul 2022 12:25:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1658665560; bh=vUseJCTrNzPDWvyzTJrjmT0HoGBBMBVECney2Q8pItc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dorbQK6XNrwRIg7t7ZoDjG6m5vvSYrrtpTw/55yU3+AxsgATYr6Ri7MgdUm0zQPEE MRwIbvU4A56OjXeYzJPBdxKtsQTkSSZxmG9qUfwZRS/MtxORCW1sAU2t2CxkGHyv25 Xw5w7w/gjSrNLssJt0P/TjtCHjgbnahPe1WF0ieK+nwI9sX+gUGf7DY98+YMnkzNwF 0jpvst9alrmopa+NUG/KBAfJosWF1wllbjxu61lbDHxg4lBJnG0YXU6lR0xC5jTjtT ILdjeEFaDH156aqDSQY1gdfKlmpPhgSMtLHdTjxmAbbWeW720Y3zfZqJZgPi3Yrr+Z kOP7cJxaWXnyA== From: guoren@kernel.org To: palmer@rivosinc.com, heiko@sntech.de, hch@infradead.org, arnd@arndb.de, peterz@infradead.org, will@kernel.org, boqun.feng@gmail.com, longman@redhat.com, mingo@redhat.com, philipp.tomsich@vrull.eu, cmuellner@linux.com, linux-kernel@vger.kernel.org, David.Laight@ACULAB.COM Cc: linux-riscv@lists.infradead.org, linux-csky@vger.kernel.org, Guo Ren , Guo Ren Subject: [PATCH V8 05/10] riscv: Enable ARCH_INLINE_READ*/WRITE*/SPIN* Date: Sun, 24 Jul 2022 08:25:12 -0400 Message-Id: <20220724122517.1019187-6-guoren@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220724122517.1019187-1-guoren@kernel.org> References: <20220724122517.1019187-1-guoren@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220724_052602_835912_DE35C2B6 X-CRM114-Status: UNSURE ( 8.29 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Guo Ren Enable ARCH_INLINE_READ*/WRITE*/SPIN* when !PREEMPTION, it is copied from arch/arm64. It could reduce procedure calls and improves performance. Signed-off-by: Guo Ren Signed-off-by: Guo Ren --- arch/riscv/Kconfig | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index fcbb81feb7ad..bff04916a6c5 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -32,6 +32,32 @@ config RISCV select ARCH_HAS_STRICT_MODULE_RWX if MMU && !XIP_KERNEL select ARCH_HAS_TICK_BROADCAST if GENERIC_CLOCKEVENTS_BROADCAST select ARCH_HAS_UBSAN_SANITIZE_ALL + select ARCH_INLINE_READ_LOCK if !PREEMPTION + select ARCH_INLINE_READ_LOCK_BH if !PREEMPTION + select ARCH_INLINE_READ_LOCK_IRQ if !PREEMPTION + select ARCH_INLINE_READ_LOCK_IRQSAVE if !PREEMPTION + select ARCH_INLINE_READ_UNLOCK if !PREEMPTION + select ARCH_INLINE_READ_UNLOCK_BH if !PREEMPTION + select ARCH_INLINE_READ_UNLOCK_IRQ if !PREEMPTION + select ARCH_INLINE_READ_UNLOCK_IRQRESTORE if !PREEMPTION + select ARCH_INLINE_WRITE_LOCK if !PREEMPTION + select ARCH_INLINE_WRITE_LOCK_BH if !PREEMPTION + select ARCH_INLINE_WRITE_LOCK_IRQ if !PREEMPTION + select ARCH_INLINE_WRITE_LOCK_IRQSAVE if !PREEMPTION + select ARCH_INLINE_WRITE_UNLOCK if !PREEMPTION + select ARCH_INLINE_WRITE_UNLOCK_BH if !PREEMPTION + select ARCH_INLINE_WRITE_UNLOCK_IRQ if !PREEMPTION + select ARCH_INLINE_WRITE_UNLOCK_IRQRESTORE if !PREEMPTION + select ARCH_INLINE_SPIN_TRYLOCK if !PREEMPTION + select ARCH_INLINE_SPIN_TRYLOCK_BH if !PREEMPTION + select ARCH_INLINE_SPIN_LOCK if !PREEMPTION + select ARCH_INLINE_SPIN_LOCK_BH if !PREEMPTION + select ARCH_INLINE_SPIN_LOCK_IRQ if !PREEMPTION + select ARCH_INLINE_SPIN_LOCK_IRQSAVE if !PREEMPTION + select ARCH_INLINE_SPIN_UNLOCK if !PREEMPTION + select ARCH_INLINE_SPIN_UNLOCK_BH if !PREEMPTION + select ARCH_INLINE_SPIN_UNLOCK_IRQ if !PREEMPTION + select ARCH_INLINE_SPIN_UNLOCK_IRQRESTORE if !PREEMPTION select ARCH_OPTIONAL_KERNEL_RWX if ARCH_HAS_STRICT_KERNEL_RWX select ARCH_OPTIONAL_KERNEL_RWX_DEFAULT select ARCH_STACKWALK From patchwork Sun Jul 24 12:25:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 12927531 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 61C49C43334 for ; Sun, 24 Jul 2022 12:26:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=CTX3v9oiZkUocukhPRt2bp6wQLJGJgMK1kZSHYLV0mI=; b=q4MSkssTc6KENt T97LjxJuNoHWOHeIQ7mC6YX1AERB+BJFo/j2bsYR3F21bLr0KE65rRgM1s0KubNDWUUO4fR4Idhbx 8AEf2IxiAgfsR3o9lobXkp8DKFlklxFU4y53llkccTeFqgWrsodnqygCPOtdTHTFY/0dBRYJ2TzN7 i31BQwGf1W0UshM1U30o1sXMH3fOuXA671QzfFurLYDcIrYHglfb6+X5WS95SeDkeYtWkkiJAf8qe rWMLH6T19ocN/i4yS4I3YGCDjGPKpno2jqYOAfbOQbrEkpqS6kXOD+C8LNjFcz725MjuU6Vx04IpL kvXWjqyWmkttPg/nES2g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFagV-0060tp-KV; Sun, 24 Jul 2022 12:26:11 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFagS-0060qA-I6 for linux-riscv@lists.infradead.org; Sun, 24 Jul 2022 12:26:10 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 3F46EB80D6E; Sun, 24 Jul 2022 12:26:07 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A32B4C341C0; Sun, 24 Jul 2022 12:26:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1658665566; bh=ui9v7vSQ2hZ9Vz4T0T/cwSBKPyXOYiw7vf10Zir0n/0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=u8tM3zQA2WovR9pT9WzJIjE98QNUnL4S8sdgOcnO2R/y5boR36AkueAY4wiA22wJb mC5v+aqu6nLFl0jo8qFZ7ZX/3zTAzQumB3Km6mtWd6/hn309e14rjRJYqandv8gHAT Oq9CS7IyVWM/Gcw9YRFLiw74pQeK9El8Y7TZpwWi7iU3ScSBFsgvhQ6t1RR/FLqmOX vbGI1NoY0xUjqX0kFG1YP32lyWN0ddJwrG6cmjCsLHhK2xZGNZPjlIetlVtU1FDmcJ Nl4fm4T142c6xW+vI1PuL3B4z05GBt0f7ojjkzYJHnTdI8JuZVWYW30KmWeUW5q+lB aCyDG6Oek9H6w== From: guoren@kernel.org To: palmer@rivosinc.com, heiko@sntech.de, hch@infradead.org, arnd@arndb.de, peterz@infradead.org, will@kernel.org, boqun.feng@gmail.com, longman@redhat.com, mingo@redhat.com, philipp.tomsich@vrull.eu, cmuellner@linux.com, linux-kernel@vger.kernel.org, David.Laight@ACULAB.COM Cc: linux-riscv@lists.infradead.org, linux-csky@vger.kernel.org, Guo Ren , Guo Ren Subject: [PATCH V8 06/10] riscv: atomic: Clean up unnecessary acquire and release definitions Date: Sun, 24 Jul 2022 08:25:13 -0400 Message-Id: <20220724122517.1019187-7-guoren@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220724122517.1019187-1-guoren@kernel.org> References: <20220724122517.1019187-1-guoren@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220724_052608_967107_0EA4BF98 X-CRM114-Status: UNSURE ( 9.16 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Guo Ren Clean up unnecessary xchg_acquire, xchg_release, and cmpxchg_release custom definitions, because the generic implementation is the same as the riscv custom implementation. Before the patch: 000000000000024e <.LBB238>: ops = xchg_acquire(pending_ipis, 0); 24e: 089937af amoswap.d a5,s1,(s2) 252: 0230000f fence r,rw 0000000000000256 <.LBB243>: ops = xchg_release(pending_ipis, 0); 256: 0310000f fence rw,w 25a: 089934af amoswap.d s1,s1,(s2) After the patch: 000000000000026e <.LBB245>: ops = xchg_acquire(pending_ipis, 0); 26e: 089937af amoswap.d a5,s1,(s2) 0000000000000272 <.LBE247>: 272: 0230000f fence r,rw 0000000000000276 <.LBB249>: ops = xchg_release(pending_ipis, 0); 276: 0310000f fence rw,w 000000000000027a <.LBB251>: 27a: 089934af amoswap.d s1,s1,(s2) Only cmpxchg_acquire is necessary (It prevents unnecessary acquire ordering when the value from lr is different from old). Signed-off-by: Guo Ren Signed-off-by: Guo Ren --- arch/riscv/include/asm/atomic.h | 19 ----- arch/riscv/include/asm/cmpxchg.h | 116 ------------------------------- 2 files changed, 135 deletions(-) diff --git a/arch/riscv/include/asm/atomic.h b/arch/riscv/include/asm/atomic.h index 0dfe9d857a76..83636320ba95 100644 --- a/arch/riscv/include/asm/atomic.h +++ b/arch/riscv/include/asm/atomic.h @@ -249,16 +249,6 @@ c_t arch_atomic##prefix##_xchg_relaxed(atomic##prefix##_t *v, c_t n) \ return __xchg_relaxed(&(v->counter), n, size); \ } \ static __always_inline \ -c_t arch_atomic##prefix##_xchg_acquire(atomic##prefix##_t *v, c_t n) \ -{ \ - return __xchg_acquire(&(v->counter), n, size); \ -} \ -static __always_inline \ -c_t arch_atomic##prefix##_xchg_release(atomic##prefix##_t *v, c_t n) \ -{ \ - return __xchg_release(&(v->counter), n, size); \ -} \ -static __always_inline \ c_t arch_atomic##prefix##_xchg(atomic##prefix##_t *v, c_t n) \ { \ return __xchg(&(v->counter), n, size); \ @@ -276,12 +266,6 @@ c_t arch_atomic##prefix##_cmpxchg_acquire(atomic##prefix##_t *v, \ return __cmpxchg_acquire(&(v->counter), o, n, size); \ } \ static __always_inline \ -c_t arch_atomic##prefix##_cmpxchg_release(atomic##prefix##_t *v, \ - c_t o, c_t n) \ -{ \ - return __cmpxchg_release(&(v->counter), o, n, size); \ -} \ -static __always_inline \ c_t arch_atomic##prefix##_cmpxchg(atomic##prefix##_t *v, c_t o, c_t n) \ { \ return __cmpxchg(&(v->counter), o, n, size); \ @@ -299,12 +283,9 @@ c_t arch_atomic##prefix##_cmpxchg(atomic##prefix##_t *v, c_t o, c_t n) \ ATOMIC_OPS() #define arch_atomic_xchg_relaxed arch_atomic_xchg_relaxed -#define arch_atomic_xchg_acquire arch_atomic_xchg_acquire -#define arch_atomic_xchg_release arch_atomic_xchg_release #define arch_atomic_xchg arch_atomic_xchg #define arch_atomic_cmpxchg_relaxed arch_atomic_cmpxchg_relaxed #define arch_atomic_cmpxchg_acquire arch_atomic_cmpxchg_acquire -#define arch_atomic_cmpxchg_release arch_atomic_cmpxchg_release #define arch_atomic_cmpxchg arch_atomic_cmpxchg #undef ATOMIC_OPS diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h index 12debce235e5..67ab6375b650 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -44,76 +44,6 @@ _x_, sizeof(*(ptr))); \ }) -#define __xchg_acquire(ptr, new, size) \ -({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(new) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) - -#define arch_xchg_acquire(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg_acquire((ptr), \ - _x_, sizeof(*(ptr))); \ -}) - -#define __xchg_release(ptr, new, size) \ -({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(new) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.w %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.d %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) - -#define arch_xchg_release(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg_release((ptr), \ - _x_, sizeof(*(ptr))); \ -}) - #define __xchg(ptr, new, size) \ ({ \ __typeof__(ptr) __ptr = (ptr); \ @@ -253,52 +183,6 @@ _o_, _n_, sizeof(*(ptr))); \ }) -#define __cmpxchg_release(ptr, old, new, size) \ -({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(*(ptr)) __old = (old); \ - __typeof__(*(ptr)) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ -}) - -#define arch_cmpxchg_release(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ = (o); \ - __typeof__(*(ptr)) _n_ = (n); \ - (__typeof__(*(ptr))) __cmpxchg_release((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) - #define __cmpxchg(ptr, old, new, size) \ ({ \ __typeof__(ptr) __ptr = (ptr); \ From patchwork Sun Jul 24 12:25:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 12927532 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 70595C433EF for ; Sun, 24 Jul 2022 12:26:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=9FKx/pCbObl51tg9H2wbtNLTaX32a0KSfzKIHeYaQoE=; b=H3AbnWmfpH5JiQ jGN899ils8Z3EQsp5IwfFsFSrlIMH8T7ZQEy3BfMy1WQ6UnoCwcnA9opN7ZkkR9uxahzi46gqZcuu KMA4ZWRV9Z02sdZ/WwCITRrmRAACWTyPsEDdJbSZPiKvuBO8dPmuOtfMJ7i0XPE5hqrBIowbMOVCh TzRogW3Gv3sXNgPO4peE2R4q3H0aZZrQWc9vV/Nd90OFRQk6OCMUNC4UhsACr46IcHBVugnYB1Q1Q tO+u8jrvG+Z31/alHJrwSfkF1PacT7UJjmCy9kqugfbT6NYai0QmLiVRvggOwMqLPhDW/wX3X9oiS DNtrR8vZcJN5HsB+J4kw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFaga-0060zI-FH; Sun, 24 Jul 2022 12:26:16 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFagX-0060vO-4f for linux-riscv@lists.infradead.org; Sun, 24 Jul 2022 12:26:14 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 99EC66103F; Sun, 24 Jul 2022 12:26:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ABF3DC3411E; Sun, 24 Jul 2022 12:26:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1658665572; bh=5kIVqhKqws1LvJ274twbut419msCbSFVG3i62G1GMz4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=n+GR3DtPlKwtsAoVzA2Q7OjeRqVeLSVehFABpXXHhcVotD2OBTjh6NhwVXdUqrIyU M2N4XM0LbMd/4nHTdg5JoPqImNW3BVGMu5k/U2GHxRuJy5wH3cK1PUshpMfZW3d7r4 oN8meBfVrqOPDRsysggmM6tyVqj1NwsNV7PzrpqVPT5rEPDGL1wmnoW+ZQlb7KkJds RZGbF9HPQ4O6Sjfk5qUDukB0a+YJX1QIQiMaTsG3zSXhzhYAtILlaZCHTiBaZjugWP sc7zjhwGpaiFOm8TxVZOo+VC5gGyBqiQq84UWQgorWyOKseM4LGiP+1LO1kM0BBmri mIWh7r2HIHXXg== From: guoren@kernel.org To: palmer@rivosinc.com, heiko@sntech.de, hch@infradead.org, arnd@arndb.de, peterz@infradead.org, will@kernel.org, boqun.feng@gmail.com, longman@redhat.com, mingo@redhat.com, philipp.tomsich@vrull.eu, cmuellner@linux.com, linux-kernel@vger.kernel.org, David.Laight@ACULAB.COM Cc: linux-riscv@lists.infradead.org, linux-csky@vger.kernel.org, Guo Ren , Guo Ren Subject: [PATCH V8 07/10] riscv: Add qspinlock support Date: Sun, 24 Jul 2022 08:25:14 -0400 Message-Id: <20220724122517.1019187-8-guoren@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220724122517.1019187-1-guoren@kernel.org> References: <20220724122517.1019187-1-guoren@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220724_052613_324301_22652A43 X-CRM114-Status: GOOD ( 12.01 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Guo Ren Enable qspinlock by the requirements mentioned in a8ad07e5240c9 ("asm-generic: qspinlock: Indicate the use of mixed-size atomics"). - RISC-V atomic_*_release()/atomic_*_acquire() are implemented with own relaxed version plus acquire/release_fence for RCsc synchronization. - RISC-V LR/SC pairs could provide a strong/weak forward guarantee that depends on micro-architecture. And RISC-V ISA spec has given out several limitations to let hardware support strict forward guarantee (RISC-V User ISA - 8.3 Eventual Success of Store-Conditional Instructions). Some riscv cores such as BOOMv3 & XiangShan could provide strict & strong forward guarantee (The cache line would be kept in an exclusive state for Backoff cycles, and only this core's interrupt could break the LR/SC pair). - RISC-V provides cheap atomic_fetch_or_acquire() with RCsc. - RISC-V only provides relaxed xchg16 to support qspinlock. Signed-off-by: Guo Ren Signed-off-by: Guo Ren Reported-by: kernel test robot --- arch/riscv/Kconfig | 16 ++++++++++++++++ arch/riscv/include/asm/Kbuild | 2 ++ arch/riscv/include/asm/cmpxchg.h | 17 +++++++++++++++++ 3 files changed, 35 insertions(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index bff04916a6c5..721f098228a8 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -359,6 +359,22 @@ config NODES_SHIFT Specify the maximum number of NUMA Nodes available on the target system. Increases memory reserved to accommodate various tables. +choice + prompt "RISC-V spinlock type" + default RISCV_TICKET_SPINLOCKS + +config RISCV_TICKET_SPINLOCKS + bool "Using ticket spinlock" + +config RISCV_QUEUED_SPINLOCKS + bool "Using queued spinlock" + depends on SMP && MMU + select ARCH_USE_QUEUED_SPINLOCKS + help + Make sure your micro arch LL/SC has a strong forward progress guarantee. + Otherwise, stay at ticket-lock/combo-lock. +endchoice + config RISCV_ALTERNATIVE bool depends on !XIP_KERNEL diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild index 504f8b7e72d4..2cce98c7b653 100644 --- a/arch/riscv/include/asm/Kbuild +++ b/arch/riscv/include/asm/Kbuild @@ -2,7 +2,9 @@ generic-y += early_ioremap.h generic-y += flat.h generic-y += kvm_para.h +generic-y += mcs_spinlock.h generic-y += parport.h +generic-y += qspinlock.h generic-y += spinlock.h generic-y += spinlock_types.h generic-y += qrwlock.h diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h index 67ab6375b650..6bf2726d4500 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -17,6 +17,23 @@ __typeof__(new) __new = (new); \ __typeof__(*(ptr)) __ret; \ switch (size) { \ + case 2: { \ + u32 tmp, ret; \ + u32 shif = ((ulong)__ptr & 2) ? 16 : 0; \ + u32 mask = 0xffff << shif; \ + __ptr = (__typeof__(ptr))((ulong)__ptr & ~2); \ + __asm__ __volatile__ ( \ + "0: lr.w %0, %2\n" \ + " and %1, %0, %z3\n" \ + " or %1, %1, %z4\n" \ + " sc.w %1, %1, %2\n" \ + " bnez %1, 0b\n" \ + : "=&r" (ret), "=&r" (tmp), "+A" (*__ptr) \ + : "rJ" (~mask), "rJ" ((u32)__new << shif) \ + : "memory"); \ + __ret = (__typeof__(*(ptr)))((ret & mask) >> shif); \ + break; \ + } \ case 4: \ __asm__ __volatile__ ( \ " amoswap.w %0, %2, %1\n" \ From patchwork Sun Jul 24 12:25:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 12927533 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 826BBC43334 for ; Sun, 24 Jul 2022 12:26:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=aRuGD4JgxGsA2WsqgBtwTLVOiEHjNhSTfD3JbeL330I=; b=zVE5960iRz9F3R Alzj6xwFuQUy4x2/52Qm3GH99fvbUBP+o384rQ+in0TxUQH+BNCFmvkH5rIkUbivBuvLoxfBu58PP 2qwn4MlasV8ObkL9KpQgMuh2H1A2D8XVq0PXlmo1cEnlDL20Q/093Qd7k6E6nQmi7baDmzV0eBweO GmXsvutHzV4Frs22hC9QrhnaJgeqSx0lWN5Aj0Ds6b1uY2hXhPX6wjGPImcnB++2YlF8FBrEMekDn TnPArM3qtev1X/l+6QdduMT2MebM7xizjzLIUSNU3ftFzGu95cDJdhp1BEEXn3iyQMIOdh9HhQAPM o9sOpLSR4K6fdRauQv4g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFagg-00615e-8B; Sun, 24 Jul 2022 12:26:22 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFagc-00611H-PZ for linux-riscv@lists.infradead.org; Sun, 24 Jul 2022 12:26:20 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 45E616106F; Sun, 24 Jul 2022 12:26:18 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B7EDAC341CF; Sun, 24 Jul 2022 12:26:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1658665578; bh=hwv8ZJ0arV6DKR5ahPEaSm+a2JYNeufj3cbLaNwOwFY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kUMqkB/MIL4posmwidps2X7SbqJHXWNu1WHwwi8YwdoqbaWQPwLOsNdrhpMPTSEAl efo1FwTvutclnE2x0xHHP1wu9QvAd6yW/1hOrBaoQrOVTOSgRbDOffHsY9nO3/ChgJ O6rFLJJ4aKrOUfnxpH9YFvPiifqjERto0628BWxkeGwrwGOhwsOcUq4dGr2UgPa6sM TfB3kQx29R5k2VhEmLXHiXUqMWQJrP3t6xZyhlUnm0/J7R3OPaxDMCSHiPorw3k6gE s1xVIYHO8KV4QBjPl+K9ZRl2iT8rjDavdAusDSvbeI3WRA/ZPrSEwFG+fyT1sv3TEF ZbVycELvr/Pcw== From: guoren@kernel.org To: palmer@rivosinc.com, heiko@sntech.de, hch@infradead.org, arnd@arndb.de, peterz@infradead.org, will@kernel.org, boqun.feng@gmail.com, longman@redhat.com, mingo@redhat.com, philipp.tomsich@vrull.eu, cmuellner@linux.com, linux-kernel@vger.kernel.org, David.Laight@ACULAB.COM Cc: linux-riscv@lists.infradead.org, linux-csky@vger.kernel.org, Guo Ren , Guo Ren Subject: [PATCH V8 08/10] riscv: Add combo spinlock support Date: Sun, 24 Jul 2022 08:25:15 -0400 Message-Id: <20220724122517.1019187-9-guoren@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220724122517.1019187-1-guoren@kernel.org> References: <20220724122517.1019187-1-guoren@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220724_052618_952051_5E41D654 X-CRM114-Status: GOOD ( 19.59 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Guo Ren Combo spinlock could support queued and ticket in one Linux Image and select them during boot time with command line option. Here is the func-size(Bytes) comparison table below: TYPE : COMBO | TICKET | QUEUED arch_spin_lock : 106 | 60 | 50 arch_spin_unlock : 54 | 36 | 26 arch_spin_trylock : 110 | 72 | 54 arch_spin_is_locked : 48 | 34 | 20 arch_spin_is_contended : 56 | 40 | 24 rch_spin_value_unlocked : 48 | 34 | 24 One example of disassemble combo arch_spin_unlock: 0xffffffff8000409c <+14>: nop # jump label slot 0xffffffff800040a0 <+18>: fence rw,w # queued spinlock start 0xffffffff800040a4 <+22>: sb zero,0(a4) # queued spinlock end 0xffffffff800040a8 <+26>: ld s0,8(sp) 0xffffffff800040aa <+28>: addi sp,sp,16 0xffffffff800040ac <+30>: ret 0xffffffff800040ae <+32>: lw a5,0(a4) # ticket spinlock start 0xffffffff800040b0 <+34>: sext.w a5,a5 0xffffffff800040b2 <+36>: fence rw,w 0xffffffff800040b6 <+40>: addiw a5,a5,1 0xffffffff800040b8 <+42>: slli a5,a5,0x30 0xffffffff800040ba <+44>: srli a5,a5,0x30 0xffffffff800040bc <+46>: sh a5,0(a4) # ticket spinlock end 0xffffffff800040c0 <+50>: ld s0,8(sp) 0xffffffff800040c2 <+52>: addi sp,sp,16 0xffffffff800040c4 <+54>: ret We could see queued spinlock is better than ticket-lock when there is no contend into queued_spin_lock_slowpath. And combo could provide a compatible Linux Image for a different kind of micro-arch design. Signed-off-by: Guo Ren Signed-off-by: Guo Ren --- arch/riscv/Kconfig | 9 +++- arch/riscv/include/asm/Kbuild | 1 - arch/riscv/include/asm/spinlock.h | 77 +++++++++++++++++++++++++++++++ arch/riscv/kernel/setup.c | 22 +++++++++ 4 files changed, 107 insertions(+), 2 deletions(-) create mode 100644 arch/riscv/include/asm/spinlock.h diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 721f098228a8..f1d68491a1cf 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -361,7 +361,7 @@ config NODES_SHIFT choice prompt "RISC-V spinlock type" - default RISCV_TICKET_SPINLOCKS + default RISCV_COMBO_SPINLOCKS config RISCV_TICKET_SPINLOCKS bool "Using ticket spinlock" @@ -373,6 +373,13 @@ config RISCV_QUEUED_SPINLOCKS help Make sure your micro arch LL/SC has a strong forward progress guarantee. Otherwise, stay at ticket-lock/combo-lock. + +config RISCV_COMBO_SPINLOCKS + bool "Using combo spinlock" + depends on SMP && MMU + select ARCH_USE_QUEUED_SPINLOCKS + help + Select queued spinlock or ticket-lock with jump_label. endchoice config RISCV_ALTERNATIVE diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild index 2cce98c7b653..59d5ea7390ea 100644 --- a/arch/riscv/include/asm/Kbuild +++ b/arch/riscv/include/asm/Kbuild @@ -5,7 +5,6 @@ generic-y += kvm_para.h generic-y += mcs_spinlock.h generic-y += parport.h generic-y += qspinlock.h -generic-y += spinlock.h generic-y += spinlock_types.h generic-y += qrwlock.h generic-y += qrwlock_types.h diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h new file mode 100644 index 000000000000..b079462d818b --- /dev/null +++ b/arch/riscv/include/asm/spinlock.h @@ -0,0 +1,77 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ASM_RISCV_SPINLOCK_H +#define __ASM_RISCV_SPINLOCK_H + +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS +#include + +#undef arch_spin_is_locked +#undef arch_spin_is_contended +#undef arch_spin_value_unlocked +#undef arch_spin_lock +#undef arch_spin_trylock +#undef arch_spin_unlock + +#include +#include + +#undef arch_spin_is_locked +#undef arch_spin_is_contended +#undef arch_spin_value_unlocked +#undef arch_spin_lock +#undef arch_spin_trylock +#undef arch_spin_unlock + +DECLARE_STATIC_KEY_TRUE(qspinlock_key); + +static __always_inline void arch_spin_lock(arch_spinlock_t *lock) +{ + if (static_branch_likely(&qspinlock_key)) + queued_spin_lock(lock); + else + ticket_spin_lock(lock); +} + +static __always_inline bool arch_spin_trylock(arch_spinlock_t *lock) +{ + if (static_branch_likely(&qspinlock_key)) + return queued_spin_trylock(lock); + return ticket_spin_trylock(lock); +} + +static __always_inline void arch_spin_unlock(arch_spinlock_t *lock) +{ + if (static_branch_likely(&qspinlock_key)) + queued_spin_unlock(lock); + else + ticket_spin_unlock(lock); +} + +static __always_inline int arch_spin_value_unlocked(arch_spinlock_t lock) +{ + if (static_branch_likely(&qspinlock_key)) + return queued_spin_value_unlocked(lock); + else + return ticket_spin_value_unlocked(lock); +} + +static __always_inline int arch_spin_is_locked(arch_spinlock_t *lock) +{ + if (static_branch_likely(&qspinlock_key)) + return queued_spin_is_locked(lock); + return ticket_spin_is_locked(lock); +} + +static __always_inline int arch_spin_is_contended(arch_spinlock_t *lock) +{ + if (static_branch_likely(&qspinlock_key)) + return queued_spin_is_contended(lock); + return ticket_spin_is_contended(lock); +} +#include +#else +#include +#endif /* CONFIG_RISCV_COMBO_SPINLOCKS */ + +#endif /* __ASM_RISCV_SPINLOCK_H */ diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c index f0f36a4a0e9b..b763039bf49b 100644 --- a/arch/riscv/kernel/setup.c +++ b/arch/riscv/kernel/setup.c @@ -261,6 +261,13 @@ static void __init parse_dtb(void) #endif } +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS +DEFINE_STATIC_KEY_TRUE_RO(qspinlock_key); +EXPORT_SYMBOL(qspinlock_key); + +static bool qspinlock_flag __initdata = false; +#endif + void __init setup_arch(char **cmdline_p) { parse_dtb(); @@ -295,10 +302,25 @@ void __init setup_arch(char **cmdline_p) setup_smp(); #endif +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS + if (!qspinlock_flag) + static_branch_disable(&qspinlock_key); +#endif + riscv_fill_hwcap(); apply_boot_alternatives(); } +#ifdef CONFIG_RISCV_COMBO_SPINLOCKS +static int __init enable_qspinlock(char *p) +{ + qspinlock_flag = true; + + return 0; +} +early_param("qspinlock", enable_qspinlock); +#endif + static int __init topology_init(void) { int i, ret; From patchwork Sun Jul 24 12:25:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 12927534 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 616D1C43334 for ; Sun, 24 Jul 2022 12:26:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=uJJNa1zgJ7AmmiS0a5dVciDDoTTN1r0hap9vjEM4r1U=; b=MfH+602P4IYWrf URgT90X9ezj+PqfxNm44sGi+yEHXEfZZpmEbERLfx9q+yKeA6eTYY7dqvI32lp0O16HKKoPGicN0+ 7mRvxR4/IBhZq5JIRl+80h23Ehd+wpUC8YNw5Iqzxgcp8z15ePYDmQKsEyb4lLI0nxDkvsANqMUqv cBXGx+QVIvJ9ftFwocPw6driz9HzDqpFT13Cn14tL2Ve6olmJdqwfRxeNoKy22T5mBdCJ3aFywq3U vprCYOqSCvBnn8oDCb6zHtjyQu+7cGthhK9sBhOuuYki04Tc3lGFTytZ84x6IRITdSskwupa/znBM E/mE+Hw5K4lCfmU6wIBw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFago-0061Fw-O8; Sun, 24 Jul 2022 12:26:30 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFagk-00619S-OW for linux-riscv@lists.infradead.org; Sun, 24 Jul 2022 12:26:28 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 60202B80D71; Sun, 24 Jul 2022 12:26:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C0295C385A2; Sun, 24 Jul 2022 12:26:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1658665584; bh=TntvPReuJPtm/qiF5zkEbZIbJqaHdY1xoBE45jhztPw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EPJhAETB/ZqTQnVFktFMepdVKYlphjry92IZblQTOLZlxU0UYThU9b2tUKt8pPiog GmtZpXduJBpd7drvjOtcETgnubOvsBOOhl0+wXj+iZncxwrG90Wy68y6saAdgVuKkQ GeSnzS+R7/0wu5I3pRqG2sgK/svzsONsYE2DShu4SccBavpViBvBIlNonqIq9XWE1K PW2UxIWAWPxsbxPxO7bABb89m0HCeMoAKBnZitSzUx6jasCKU+wyoy/vebAL4na6I0 3P6pnHIKjVySXbzldBjOCdD5/o+wnT5P2Ry/b0KCWEEXJ1A7u7+6d9GWrShyichSom KVIg8pv9nXfcQ== From: guoren@kernel.org To: palmer@rivosinc.com, heiko@sntech.de, hch@infradead.org, arnd@arndb.de, peterz@infradead.org, will@kernel.org, boqun.feng@gmail.com, longman@redhat.com, mingo@redhat.com, philipp.tomsich@vrull.eu, cmuellner@linux.com, linux-kernel@vger.kernel.org, David.Laight@ACULAB.COM Cc: linux-riscv@lists.infradead.org, linux-csky@vger.kernel.org, Guo Ren , Guo Ren Subject: [PATCH V8 09/10] csky: Enable ARCH_INLINE_READ*/WRITE*/SPIN* Date: Sun, 24 Jul 2022 08:25:16 -0400 Message-Id: <20220724122517.1019187-10-guoren@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220724122517.1019187-1-guoren@kernel.org> References: <20220724122517.1019187-1-guoren@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220724_052626_982336_12E7D551 X-CRM114-Status: UNSURE ( 8.18 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Guo Ren Enable ARCH_INLINE_READ*/WRITE*/SPIN* when !PREEMPTION, it is copied from arch/arm64. It could reduce procedure calls and improves performance. Signed-off-by: Guo Ren Signed-off-by: Guo Ren --- arch/csky/Kconfig | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/arch/csky/Kconfig b/arch/csky/Kconfig index 21d72b078eef..dfdb436b6078 100644 --- a/arch/csky/Kconfig +++ b/arch/csky/Kconfig @@ -8,6 +8,32 @@ config CSKY select ARCH_HAS_SYNC_DMA_FOR_DEVICE select ARCH_USE_BUILTIN_BSWAP select ARCH_USE_QUEUED_RWLOCKS + select ARCH_INLINE_READ_LOCK if !PREEMPTION + select ARCH_INLINE_READ_LOCK_BH if !PREEMPTION + select ARCH_INLINE_READ_LOCK_IRQ if !PREEMPTION + select ARCH_INLINE_READ_LOCK_IRQSAVE if !PREEMPTION + select ARCH_INLINE_READ_UNLOCK if !PREEMPTION + select ARCH_INLINE_READ_UNLOCK_BH if !PREEMPTION + select ARCH_INLINE_READ_UNLOCK_IRQ if !PREEMPTION + select ARCH_INLINE_READ_UNLOCK_IRQRESTORE if !PREEMPTION + select ARCH_INLINE_WRITE_LOCK if !PREEMPTION + select ARCH_INLINE_WRITE_LOCK_BH if !PREEMPTION + select ARCH_INLINE_WRITE_LOCK_IRQ if !PREEMPTION + select ARCH_INLINE_WRITE_LOCK_IRQSAVE if !PREEMPTION + select ARCH_INLINE_WRITE_UNLOCK if !PREEMPTION + select ARCH_INLINE_WRITE_UNLOCK_BH if !PREEMPTION + select ARCH_INLINE_WRITE_UNLOCK_IRQ if !PREEMPTION + select ARCH_INLINE_WRITE_UNLOCK_IRQRESTORE if !PREEMPTION + select ARCH_INLINE_SPIN_TRYLOCK if !PREEMPTION + select ARCH_INLINE_SPIN_TRYLOCK_BH if !PREEMPTION + select ARCH_INLINE_SPIN_LOCK if !PREEMPTION + select ARCH_INLINE_SPIN_LOCK_BH if !PREEMPTION + select ARCH_INLINE_SPIN_LOCK_IRQ if !PREEMPTION + select ARCH_INLINE_SPIN_LOCK_IRQSAVE if !PREEMPTION + select ARCH_INLINE_SPIN_UNLOCK if !PREEMPTION + select ARCH_INLINE_SPIN_UNLOCK_BH if !PREEMPTION + select ARCH_INLINE_SPIN_UNLOCK_IRQ if !PREEMPTION + select ARCH_INLINE_SPIN_UNLOCK_IRQRESTORE if !PREEMPTION select ARCH_WANT_FRAME_POINTERS if !CPU_CK610 && $(cc-option,-mbacktrace) select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT select COMMON_CLK From patchwork Sun Jul 24 12:25:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 12927535 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 47C2FCCA47C for ; Sun, 24 Jul 2022 12:26:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=DefRvucyATU6XHFzksfg6VFctXIMNgy1vXaK7KMOv+Y=; b=qQ0Q4OA+kzW+9e tj+53iydoDYn0D2QXO43shwUJWxpx6AtcvUkwFGl7aSZkdyZvAxKm0Zg79+Ru6MBma0OH57Em9tT2 ZgVUed1tMYtjJc5R61TuJ25SPQJrgFtamGaDYEuc3h1ZLBgLrAGirmq12GikgoWf3Ll4j9cTAp4dg 4ZQ2kJZ/RTXnaa6gXqka+9Fp6CDHKtE9fPPfRcnoJu+e3gaGNfYRDv/XEAPOEGLcvq31lb8OFT912 9JeS+xzgFBJn5q4JqFbo9PXu254oAqWR2QUnlPVEeFesJM6UytyWh04C98pqKp9dbO9S1Yl0ISa8e UIVjJ/TiV9DtY50L4oXg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFagt-0061K3-8J; Sun, 24 Jul 2022 12:26:35 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oFago-0061Fn-QG for linux-riscv@lists.infradead.org; Sun, 24 Jul 2022 12:26:32 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 56A1A6106F; Sun, 24 Jul 2022 12:26:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C77F7C385A5; Sun, 24 Jul 2022 12:26:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1658665590; bh=920mmnpEJPScVrA5dKwmyImPh9PCuH+GiFZjlE1o8zI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=KOdKpq+E4B/EJi49xbzc27v1TLG7ZtjFILg8aSF7N9+lSnRnEpR7LjSRUSbqeMr5L ZQAzxLJ3xxcSfBXGnpPhlbsaOsZUgB2f5Coi0rleUrDhjTQsHtAntM9kz0YHpGH2/x 8SDZPhVIT9NXdYQ+a5aF0lacSIH7mk/oI/Gs0GFgyQoz+d+WmU59fNUftUET9/CTFl 07mv1i8TnX854bxdJzBN7waa7YDuL9WjQpSlYSBtjAgMzjl5qvUt7VBbsgSivTjMTs AceE9qPAzul9KUiQIBQbEamCnpi8hpkTsb7O7rriYH9Rwui3zWwu81lDWuthAzyX86 qAB+XEfb3ayyA== From: guoren@kernel.org To: palmer@rivosinc.com, heiko@sntech.de, hch@infradead.org, arnd@arndb.de, peterz@infradead.org, will@kernel.org, boqun.feng@gmail.com, longman@redhat.com, mingo@redhat.com, philipp.tomsich@vrull.eu, cmuellner@linux.com, linux-kernel@vger.kernel.org, David.Laight@ACULAB.COM Cc: linux-riscv@lists.infradead.org, linux-csky@vger.kernel.org, Guo Ren , Guo Ren Subject: [PATCH V8 10/10] csky: Add qspinlock support Date: Sun, 24 Jul 2022 08:25:17 -0400 Message-Id: <20220724122517.1019187-11-guoren@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220724122517.1019187-1-guoren@kernel.org> References: <20220724122517.1019187-1-guoren@kernel.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220724_052631_033925_D4F3D7CA X-CRM114-Status: GOOD ( 11.22 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Guo Ren Enable qspinlock by the requirements mentioned in a8ad07e5240c9 ("asm-generic: qspinlock: Indicate the use of mixed-size atomics"). C-SKY only has "ldex/stex" for all atomic operations. So csky give a strong forward guarantee for "ldex/stex." That means when ldex grabbed the cache line into $L1, it would block other cores from snooping the address with several cycles. Signed-off-by: Guo Ren Signed-off-by: Guo Ren --- arch/csky/Kconfig | 16 ++++++++++++++++ arch/csky/include/asm/Kbuild | 2 ++ arch/csky/include/asm/cmpxchg.h | 20 ++++++++++++++++++++ 3 files changed, 38 insertions(+) diff --git a/arch/csky/Kconfig b/arch/csky/Kconfig index dfdb436b6078..09f7d1f06bca 100644 --- a/arch/csky/Kconfig +++ b/arch/csky/Kconfig @@ -354,6 +354,22 @@ config HAVE_EFFICIENT_UNALIGNED_STRING_OPS Say Y here to enable EFFICIENT_UNALIGNED_STRING_OPS. Some CPU models could deal with unaligned access by hardware. +choice + prompt "C-SKY spinlock type" + default CSKY_TICKET_SPINLOCKS + +config CSKY_TICKET_SPINLOCKS + bool "Using ticket spinlock" + +config CSKY_QUEUED_SPINLOCKS + bool "Using queued spinlock" + depends on SMP + select ARCH_USE_QUEUED_SPINLOCKS + help + Make sure your micro arch LL/SC has a strong forward progress guarantee. + Otherwise, stay at ticket-lock/combo-lock. +endchoice + endmenu source "arch/csky/Kconfig.platforms" diff --git a/arch/csky/include/asm/Kbuild b/arch/csky/include/asm/Kbuild index 103207a58f97..b70b14de904f 100644 --- a/arch/csky/include/asm/Kbuild +++ b/arch/csky/include/asm/Kbuild @@ -3,10 +3,12 @@ generic-y += asm-offsets.h generic-y += extable.h generic-y += gpio.h generic-y += kvm_para.h +generic-y += mcs_spinlock.h generic-y += spinlock.h generic-y += spinlock_types.h generic-y += qrwlock.h generic-y += qrwlock_types.h +generic-y += qspinlock.h generic-y += parport.h generic-y += user.h generic-y += vmlinux.lds.h diff --git a/arch/csky/include/asm/cmpxchg.h b/arch/csky/include/asm/cmpxchg.h index 5b8faccd65e4..5f693fadb56c 100644 --- a/arch/csky/include/asm/cmpxchg.h +++ b/arch/csky/include/asm/cmpxchg.h @@ -15,6 +15,26 @@ extern void __bad_xchg(void); __typeof__(*(ptr)) __ret; \ unsigned long tmp; \ switch (size) { \ + case 2: { \ + u32 ret; \ + u32 shif = ((ulong)__ptr & 2) ? 16 : 0; \ + u32 mask = 0xffff << shif; \ + __ptr = (__typeof__(ptr))((ulong)__ptr & ~2); \ + __asm__ __volatile__ ( \ + "1: ldex.w %0, (%4)\n" \ + " and %1, %0, %2\n" \ + " or %1, %1, %3\n" \ + " stex.w %1, (%4)\n" \ + " bez %1, 1b\n" \ + : "=&r" (ret), "=&r" (tmp) \ + : "r" (~mask), \ + "r" ((u32)__new << shif), \ + "r" (__ptr) \ + : "memory"); \ + __ret = (__typeof__(*(ptr))) \ + ((ret & mask) >> shif); \ + break; \ + } \ case 4: \ asm volatile ( \ "1: ldex.w %0, (%3) \n" \