From patchwork Sat Mar 27 18:06:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 12168597 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 995FBC433E0 for ; Sat, 27 Mar 2021 18:07:48 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3A17C61991 for ; Sat, 27 Mar 2021 18:07:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3A17C61991 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:MIME-Version:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:References:In-Reply-To:Message-Id:Date:Subject:Cc:To :From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=CZClBOtmakBvvcbtl15stU1bgx7StSx/WH55rHxovbs=; b=csh9l+tNkeUwalGjQzPrRNlE/p 4ER7Dl+5JHaCzjlqcqIkrNysW2DM6FKQz3ec1K5m40gQ4MhZ4a16mRpycsq53ieHL1mT4kg9fXq70 HX74oi0+mT2aPliO0C2rL679kgix1I1RrNtEkfa77Qn2esnYDaPTdzutRMX1OSXiJQPM2nSBTR/tD P3ZnxKU6OsdV9FcJpG5IBdkAfUcAvNjHuCxp5PgeAJGHph6pn8Hp1jlyzHnhpW1kvi4lyfW4pCnav s3a7fUobQOto7v24kfCWHdvWo2ZUe93qARcbeWa+pZcGfaJspd3CtjoT4aoDlw8Wv9Mcer3qYg1K4 mCjoTeHw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lQDLV-005OPD-TN; Sat, 27 Mar 2021 18:07:38 +0000 Received: from mail.kernel.org ([198.145.29.99]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lQDLP-005OO5-5z for linux-riscv@lists.infradead.org; Sat, 27 Mar 2021 18:07:33 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id C3CB3619BB; Sat, 27 Mar 2021 18:07:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1616868450; bh=DsrHsPtWDnnHmn30NT+r18yNYxdqI6C4be+2gqhoyC4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=riJh0/0iIDu7i9hn241DanZ0bIJyLp/J09SWZIldQpqVoJq6VRxGZZ/CyLDGsD7/9 miWSJHXEsYzRPFNJERuw1N0M32YNpBFbkw8hgNjkRlxzMXvV/oOcU4oz/CM/DiB8k4 sHEnXD0Y9dlznQv3jcBVKZkl27eaOxln6OnJNg4vbbrTsRbfMgI5o2ZXsYwZcNQokN BBqz2WnCEzzsGe0eRyGngpGATwdR9qCdj/B/O1akgK89sx7OvsJ7EZLxoC5luLz87o qh8+Iz151MquloFrNgC+/pEO9k4lqZktpB+jj03d1odC+WvspW4pRgwad4+9DGZwfI 5GnpMNSZGsqdQ== From: guoren@kernel.org To: guoren@kernel.org Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-arch@vger.kernel.org, Guo Ren , Peter Zijlstra , Michael Clark , Anup Patel , Arnd Bergmann , Palmer Dabbelt Subject: [PATCH v4 1/4] riscv: cmpxchg.h: Cleanup unused code Date: Sat, 27 Mar 2021 18:06:36 +0000 Message-Id: <1616868399-82848-2-git-send-email-guoren@kernel.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1616868399-82848-1-git-send-email-guoren@kernel.org> References: <1616868399-82848-1-git-send-email-guoren@kernel.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210327_180731_615461_DEAB8CD0 X-CRM114-Status: UNSURE ( 7.13 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Guo Ren Remove unnecessary marco, they are no use or handled by generic files (atomic-fallback.h, asm-generic/cmpxchg*). Signed-off-by: Guo Ren Link: https://lore.kernel.org/linux-riscv/CAJF2gTT1_mP-wiK2HsCpTeU61NqZVKZX1A5ye=TwqvGN4TPmrA@mail.gmail.com/ Cc: Peter Zijlstra Cc: Michael Clark Cc: Anup Patel Cc: Arnd Bergmann Cc: Palmer Dabbelt --- arch/riscv/include/asm/cmpxchg.h | 83 -------------------------------- 1 file changed, 83 deletions(-) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h index 262e5bbb2776..f1383c15e16b 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -72,13 +72,6 @@ __ret; \ }) -#define xchg_acquire(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg_acquire((ptr), \ - _x_, sizeof(*(ptr))); \ -}) - #define __xchg_release(ptr, new, size) \ ({ \ __typeof__(ptr) __ptr = (ptr); \ @@ -107,13 +100,6 @@ __ret; \ }) -#define xchg_release(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg_release((ptr), \ - _x_, sizeof(*(ptr))); \ -}) - #define __xchg(ptr, new, size) \ ({ \ __typeof__(ptr) __ptr = (ptr); \ @@ -140,24 +126,6 @@ __ret; \ }) -#define xchg(ptr, x) \ -({ \ - __typeof__(*(ptr)) _x_ = (x); \ - (__typeof__(*(ptr))) __xchg((ptr), _x_, sizeof(*(ptr))); \ -}) - -#define xchg32(ptr, x) \ -({ \ - BUILD_BUG_ON(sizeof(*(ptr)) != 4); \ - xchg((ptr), (x)); \ -}) - -#define xchg64(ptr, x) \ -({ \ - BUILD_BUG_ON(sizeof(*(ptr)) != 8); \ - xchg((ptr), (x)); \ -}) - /* * Atomic compare and exchange. Compare OLD with MEM, if identical, * store NEW in MEM. Return the initial value in MEM. Success is @@ -245,14 +213,6 @@ __ret; \ }) -#define cmpxchg_acquire(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ = (o); \ - __typeof__(*(ptr)) _n_ = (n); \ - (__typeof__(*(ptr))) __cmpxchg_acquire((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) - #define __cmpxchg_release(ptr, old, new, size) \ ({ \ __typeof__(ptr) __ptr = (ptr); \ @@ -291,14 +251,6 @@ __ret; \ }) -#define cmpxchg_release(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ = (o); \ - __typeof__(*(ptr)) _n_ = (n); \ - (__typeof__(*(ptr))) __cmpxchg_release((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) - #define __cmpxchg(ptr, old, new, size) \ ({ \ __typeof__(ptr) __ptr = (ptr); \ @@ -337,39 +289,4 @@ __ret; \ }) -#define cmpxchg(ptr, o, n) \ -({ \ - __typeof__(*(ptr)) _o_ = (o); \ - __typeof__(*(ptr)) _n_ = (n); \ - (__typeof__(*(ptr))) __cmpxchg((ptr), \ - _o_, _n_, sizeof(*(ptr))); \ -}) - -#define cmpxchg_local(ptr, o, n) \ - (__cmpxchg_relaxed((ptr), (o), (n), sizeof(*(ptr)))) - -#define cmpxchg32(ptr, o, n) \ -({ \ - BUILD_BUG_ON(sizeof(*(ptr)) != 4); \ - cmpxchg((ptr), (o), (n)); \ -}) - -#define cmpxchg32_local(ptr, o, n) \ -({ \ - BUILD_BUG_ON(sizeof(*(ptr)) != 4); \ - cmpxchg_relaxed((ptr), (o), (n)) \ -}) - -#define cmpxchg64(ptr, o, n) \ -({ \ - BUILD_BUG_ON(sizeof(*(ptr)) != 8); \ - cmpxchg((ptr), (o), (n)); \ -}) - -#define cmpxchg64_local(ptr, o, n) \ -({ \ - BUILD_BUG_ON(sizeof(*(ptr)) != 8); \ - cmpxchg_relaxed((ptr), (o), (n)); \ -}) - #endif /* _ASM_RISCV_CMPXCHG_H */ From patchwork Sat Mar 27 18:06:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 12168599 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8532DC433DB for ; Sat, 27 Mar 2021 18:07:50 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 27E8961991 for ; Sat, 27 Mar 2021 18:07:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 27E8961991 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:MIME-Version:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:References:In-Reply-To:Message-Id:Date:Subject:Cc:To :From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=usGa5Elc+XMqW+EAbY4pEWq5go/BbNL/5YFe4Xb3Xms=; b=XyedMVGchKWlXuQDouMXlE61tN /zWUyFUbVfqz139Lt2YFDPIjbaMF0hs5KOeMfVh5u89bVXcmPMmpKMIR+YcHytRzvv1F/jM/RdYY4 s8GMAyp709rhaFwNuIj3PnIgVLbJhgZN0v5vjvVwkyvGr2V1Ey6nzv+bGxpBwZbvDlBtM9U2Is8kC chbzAEn0i9ODGmLhZE1CMttwCSe6IuNAlWE/iDilYmA99ZFdOntfdv9Gr7ZSHg60+AZQw2GPwBRG4 M7MD9fzOBgg91YJ0qbbWRES98YGfWNA8iLo5w9qPgbVWiyzI9RkrWHsl/fNB5amc+JKG7LeINg+V4 /tsRDwmw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lQDLX-005OPj-Av; Sat, 27 Mar 2021 18:07:39 +0000 Received: from mail.kernel.org ([198.145.29.99]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lQDLR-005OOd-UX for linux-riscv@lists.infradead.org; Sat, 27 Mar 2021 18:07:36 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 6690361991; Sat, 27 Mar 2021 18:07:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1616868452; bh=Qa6OT7eFmBau8vPzDLJz0ISmmELm1dvO3bmq0s5juCw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LbI1c9dCyqddwz4XpqVGp+XIxzBmEdkrXa0WuxWogZgApEsrZref3oSR70ZG3PNKC 9nTWvZmlNYLxLAlIh2Z9oUPtYo1wteyvsIgHQp2gjmTyeIhg16wX7JxjYYkwnMyrTa 0WY5F1a6NsxxYDgcCOo89CSrMAWQX5J5yoZSsStRurunJMXf1W3MUatbAl5S5JK9c+ BcZKUkvuxUf5RUcuiU9usPQR1u1AUUw8XNdSmT1BraNS/reh16XqTw8Kqbi1rvVpVb bJAtLbDkYQnaK9gGd7bFMRtQ2ZVSzUwFYLEJkc9Mn9KZcn1R5L+e9zrnLevlst6V08 Jo4Iw8bGyxzxQ== From: guoren@kernel.org To: guoren@kernel.org Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-arch@vger.kernel.org, Guo Ren , Peter Zijlstra , Michael Clark , Anup Patel , Arnd Bergmann , Palmer Dabbelt Subject: [PATCH v4 2/4] riscv: cmpxchg.h: Merge macros Date: Sat, 27 Mar 2021 18:06:37 +0000 Message-Id: <1616868399-82848-3-git-send-email-guoren@kernel.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1616868399-82848-1-git-send-email-guoren@kernel.org> References: <1616868399-82848-1-git-send-email-guoren@kernel.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210327_180734_419709_25A550ED X-CRM114-Status: UNSURE ( 8.34 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Guo Ren To reduce assembly codes, let's merge duplicate codes into one (xchg_acquire, xchg_release, cmpxchg_release). Signed-off-by: Guo Ren Link: https://lore.kernel.org/linux-riscv/CAJF2gTT1_mP-wiK2HsCpTeU61NqZVKZX1A5ye=TwqvGN4TPmrA@mail.gmail.com/ Cc: Peter Zijlstra Cc: Michael Clark Cc: Anup Patel Cc: Arnd Bergmann Cc: Palmer Dabbelt --- arch/riscv/include/asm/cmpxchg.h | 92 +++++--------------------------- 1 file changed, 12 insertions(+), 80 deletions(-) diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h index f1383c15e16b..50513b95411d 100644 --- a/arch/riscv/include/asm/cmpxchg.h +++ b/arch/riscv/include/asm/cmpxchg.h @@ -11,6 +11,12 @@ #include #include +#define __local_acquire_fence() \ + __asm__ __volatile__(RISCV_ACQUIRE_BARRIER "" ::: "memory") + +#define __local_release_fence() \ + __asm__ __volatile__(RISCV_RELEASE_BARRIER "" ::: "memory") + #define __xchg_relaxed(ptr, new, size) \ ({ \ __typeof__(ptr) __ptr = (ptr); \ @@ -46,58 +52,16 @@ #define __xchg_acquire(ptr, new, size) \ ({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(new) __new = (new); \ __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - " amoswap.w %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - " amoswap.d %0, %2, %1\n" \ - RISCV_ACQUIRE_BARRIER \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ + __ret = __xchg_relaxed(ptr, new, size); \ + __local_acquire_fence(); \ __ret; \ }) #define __xchg_release(ptr, new, size) \ ({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(new) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.w %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - " amoswap.d %0, %2, %1\n" \ - : "=r" (__ret), "+A" (*__ptr) \ - : "r" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ + __local_release_fence(); \ + __xchg_relaxed(ptr, new, size); \ }) #define __xchg(ptr, new, size) \ @@ -215,40 +179,8 @@ #define __cmpxchg_release(ptr, old, new, size) \ ({ \ - __typeof__(ptr) __ptr = (ptr); \ - __typeof__(*(ptr)) __old = (old); \ - __typeof__(*(ptr)) __new = (new); \ - __typeof__(*(ptr)) __ret; \ - register unsigned int __rc; \ - switch (size) { \ - case 4: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.w %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.w %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" ((long)__old), "rJ" (__new) \ - : "memory"); \ - break; \ - case 8: \ - __asm__ __volatile__ ( \ - RISCV_RELEASE_BARRIER \ - "0: lr.d %0, %2\n" \ - " bne %0, %z3, 1f\n" \ - " sc.d %1, %z4, %2\n" \ - " bnez %1, 0b\n" \ - "1:\n" \ - : "=&r" (__ret), "=&r" (__rc), "+A" (*__ptr) \ - : "rJ" (__old), "rJ" (__new) \ - : "memory"); \ - break; \ - default: \ - BUILD_BUG(); \ - } \ - __ret; \ + __local_release_fence(); \ + __cmpxchg_relaxed(ptr, old, new, size); \ }) #define __cmpxchg(ptr, old, new, size) \ From patchwork Sat Mar 27 18:06:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 12168601 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12299C433DB for ; Sat, 27 Mar 2021 18:07:56 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9DEA861991 for ; Sat, 27 Mar 2021 18:07:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9DEA861991 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:MIME-Version:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:References:In-Reply-To:Message-Id:Date:Subject:Cc:To :From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=GUJj4nD/K8vHYaLc270WaaJzzUok56ga7OgXuGyUeNk=; b=kiPnbBI2e951K4RQCC6PAyG8zU iljQr6AijIbcmuuD6QtyMUz7x3LEFIwYYE27zONppiieKr006OrWAp75SeMlvxROjaSdj7BVvu4bA A4D1/tHXKEqos1Mabmec2gR7oAY8WJcdrVbHojq2nsX5jRYGra7AZLr1d7ZEWN9mskVlw+nFPeyQC EjAiu2MZGmug4W0wgm5UXaKTUuV1Q8U8ssnb7hfwp3odz/59lAfqCWP2R/RgGaLJQIFbeZNDctgwb 9RskoEMoJkoiZdwbqTSUfJ4GZ1lCkzbDosySURckI2F/tituhTbhLaQMQt0xwvc9e28vS75wMncGt I0MYoQkA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lQDLd-005OQl-DK; Sat, 27 Mar 2021 18:07:45 +0000 Received: from mail.kernel.org ([198.145.29.99]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lQDLU-005OP5-NT for linux-riscv@lists.infradead.org; Sat, 27 Mar 2021 18:07:39 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 6195161965; Sat, 27 Mar 2021 18:07:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1616868455; bh=LvRCrrCzj8udXkBA6WRXdyg71qL92AhvWoldyruR39k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=chiiL0yvB6GLw1pCkrzaWTJFvn76YRJU/n+BZBYztHPTGh+uoGLlmhRgoAQNlC54g OXCeIFjmYUhLqapkXYOi/+TB1ppmtdmV6JWFIoTlQf01SnYuPqe+BS30sWdry4VzOj iets2HU2I+h8s4RGn2zEFvrJPF6S3z7lPYsL+zTbW9dtL2wCMT0XrYdp+YEyYrXv3I zlBfOGPI6QPpa1G6XxskkEBivoktFSRd97qELc2fE4m63miZFBlygrMQ61hkcB91PA hbed4poQHYm7sNaEmRumdO5k4br16LR++Z3EUsqz7E6qeHcS8GnSNQpGabM1eDqiZO fMQgTZXR28bkA== From: guoren@kernel.org To: guoren@kernel.org Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-arch@vger.kernel.org, Guo Ren , Peter Zijlstra , Will Deacon , Ingo Molnar , Waiman Long , Arnd Bergmann , Anup Patel Subject: [PATCH v4 3/4] locking/qspinlock: Add ARCH_USE_QUEUED_SPINLOCKS_XCHG32 Date: Sat, 27 Mar 2021 18:06:38 +0000 Message-Id: <1616868399-82848-4-git-send-email-guoren@kernel.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1616868399-82848-1-git-send-email-guoren@kernel.org> References: <1616868399-82848-1-git-send-email-guoren@kernel.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210327_180737_765159_7B4CC0C4 X-CRM114-Status: GOOD ( 12.80 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Guo Ren Some architectures don't have sub-word swap atomic instruction, they only have the full word's one. The sub-word swap only improve the performance when: NR_CPUS < 16K * 0- 7: locked byte * 8: pending * 9-15: not used * 16-17: tail index * 18-31: tail cpu (+1) The 9-15 bits are wasted to use xchg16 in xchg_tail. Please let architecture select xchg16/xchg32 to implement xchg_tail. Signed-off-by: Guo Ren Cc: Peter Zijlstra Cc: Will Deacon Cc: Ingo Molnar Cc: Waiman Long Cc: Arnd Bergmann Cc: Anup Patel Acked-by: Stafford Horne --- kernel/Kconfig.locks | 3 +++ kernel/locking/qspinlock.c | 44 +++++++++++++++++++++----------------- 2 files changed, 27 insertions(+), 20 deletions(-) diff --git a/kernel/Kconfig.locks b/kernel/Kconfig.locks index 3de8fd11873b..d02f1261f73f 100644 --- a/kernel/Kconfig.locks +++ b/kernel/Kconfig.locks @@ -239,6 +239,9 @@ config LOCK_SPIN_ON_OWNER config ARCH_USE_QUEUED_SPINLOCKS bool +config ARCH_USE_QUEUED_SPINLOCKS_XCHG32 + bool + config QUEUED_SPINLOCKS def_bool y if ARCH_USE_QUEUED_SPINLOCKS depends on SMP diff --git a/kernel/locking/qspinlock.c b/kernel/locking/qspinlock.c index cbff6ba53d56..54de0632c6a8 100644 --- a/kernel/locking/qspinlock.c +++ b/kernel/locking/qspinlock.c @@ -163,26 +163,6 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock) WRITE_ONCE(lock->locked_pending, _Q_LOCKED_VAL); } -/* - * xchg_tail - Put in the new queue tail code word & retrieve previous one - * @lock : Pointer to queued spinlock structure - * @tail : The new queue tail code word - * Return: The previous queue tail code word - * - * xchg(lock, tail), which heads an address dependency - * - * p,*,* -> n,*,* ; prev = xchg(lock, node) - */ -static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail) -{ - /* - * We can use relaxed semantics since the caller ensures that the - * MCS node is properly initialized before updating the tail. - */ - return (u32)xchg_relaxed(&lock->tail, - tail >> _Q_TAIL_OFFSET) << _Q_TAIL_OFFSET; -} - #else /* _Q_PENDING_BITS == 8 */ /** @@ -206,6 +186,30 @@ static __always_inline void clear_pending_set_locked(struct qspinlock *lock) { atomic_add(-_Q_PENDING_VAL + _Q_LOCKED_VAL, &lock->val); } +#endif + +#if _Q_PENDING_BITS == 8 && !defined(CONFIG_ARCH_USE_QUEUED_SPINLOCKS_XCHG32) +/* + * xchg_tail - Put in the new queue tail code word & retrieve previous one + * @lock : Pointer to queued spinlock structure + * @tail : The new queue tail code word + * Return: The previous queue tail code word + * + * xchg(lock, tail), which heads an address dependency + * + * p,*,* -> n,*,* ; prev = xchg(lock, node) + */ +static __always_inline u32 xchg_tail(struct qspinlock *lock, u32 tail) +{ + /* + * We can use relaxed semantics since the caller ensures that the + * MCS node is properly initialized before updating the tail. + */ + return (u32)xchg_relaxed(&lock->tail, + tail >> _Q_TAIL_OFFSET) << _Q_TAIL_OFFSET; +} + +#else /** * xchg_tail - Put in the new queue tail code word & retrieve previous one From patchwork Sat Mar 27 18:06:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Guo Ren X-Patchwork-Id: 12168603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53238C433C1 for ; Sat, 27 Mar 2021 18:07:59 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F06F061971 for ; Sat, 27 Mar 2021 18:07:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F06F061971 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:MIME-Version:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:References:In-Reply-To:Message-Id:Date:Subject:Cc:To :From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=EJ/sVA5VVvL0XPmTrji9IaLYee6y987QirFlDiJgiEc=; b=QOCIbVLoo68wAE/gPSnnpMGkms JLYzRops5hG461CFYtt6FJauBQbjntbtLEPQyTnIcxl6PGaNU13D1n8ru9CX7rxVx/4x8oUieID3b fuNSY+mNMfLbEddCAr6fwpQ5OI4Q9f+aKXB1YLsYFAD8Sz0smB3DeKlipj62oe0H+8w/ZZbsJEEZD Lv7LThmp+6itX/c+ZDaH2fPhuRr5DeTVZWA1EcF2S9brr+rVkFDG97QZQ11YLHuLNeaCX0256kq2F qv7TjsVSMKa40tkjjeW0ynt1PuG9LsElgv7o/HaoegheSDCxewIDWvHudfaAE4xEM6z45QHPHtmfY hYZzbOOw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lQDLf-005ORD-5B; Sat, 27 Mar 2021 18:07:47 +0000 Received: from mail.kernel.org ([198.145.29.99]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lQDLX-005OPe-64 for linux-riscv@lists.infradead.org; Sat, 27 Mar 2021 18:07:42 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 078A6619C9; Sat, 27 Mar 2021 18:07:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1616868457; bh=6J9FKNfSghKRn1MDOq84VjUjsgWVuTuIQD6E9D3s7fA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qMnKVpC+8KBWbZTh5R7+nJDXD3FXhoONp+F+fNSMM7wAEy3fkfBBiHZ1XywaZNFKY /buaDYVJGJLf9u3DowXxt8TCubxCUli5p+uomIfnO8jQW9uBcIdDNf6B56bJRp1Oe2 bGWZDA6bIVi8oSrawqje9YCB3RDCEi0A6OwXEGjnibYYthPIUgXgoNU1mBR1unLAxv 31gzRbCwGVtjcEAF+F6MnrBGaunu9ysKQ39MeZZQ6sBUsu9XbL3gA4NjVmv4wteYE9 pA8t8n9CveXdQM6AkIYEAR796wE9bXywe1CsDT3/fJRheglOYpnfPey2bfQTHszp87 wQ/TPx8Auoqag== From: guoren@kernel.org To: guoren@kernel.org Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-arch@vger.kernel.org, Michael Clark , Peter Zijlstra , Anup Patel , Arnd Bergmann , Palmer Dabbelt Subject: [PATCH v4 4/4] riscv: Convert custom spinlock/rwlock to generic qspinlock/qrwlock Date: Sat, 27 Mar 2021 18:06:39 +0000 Message-Id: <1616868399-82848-5-git-send-email-guoren@kernel.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1616868399-82848-1-git-send-email-guoren@kernel.org> References: <1616868399-82848-1-git-send-email-guoren@kernel.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210327_180740_638180_40B61E1F X-CRM114-Status: GOOD ( 10.83 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Michael Clark Update the RISC-V port to use the generic qspinlock and qrwlock. This patch requires support for xchg for short which are added by a previous patch. Guo fixed up compile error which made by below include sequence: +#include +#include Signed-off-by: Michael Clark Co-developed-by: Guo Ren Tested-by: Guo Ren Link: https://lore.kernel.org/linux-riscv/20190211043829.30096-3-michaeljclark@mac.com/ Cc: Peter Zijlstra Cc: Anup Patel Cc: Arnd Bergmann Cc: Palmer Dabbelt --- arch/riscv/Kconfig | 3 + arch/riscv/include/asm/Kbuild | 3 + arch/riscv/include/asm/spinlock.h | 126 +----------------------- arch/riscv/include/asm/spinlock_types.h | 15 +-- 4 files changed, 11 insertions(+), 136 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 87d7b52f278f..67cc65ba1ea1 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -33,6 +33,9 @@ config RISCV select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU select ARCH_WANT_FRAME_POINTERS select ARCH_WANT_HUGE_PMD_SHARE if 64BIT + select ARCH_USE_QUEUED_RWLOCKS + select ARCH_USE_QUEUED_SPINLOCKS + select ARCH_USE_QUEUED_SPINLOCKS_XCHG32 select CLONE_BACKWARDS select CLINT_TIMER if !MMU select COMMON_CLK diff --git a/arch/riscv/include/asm/Kbuild b/arch/riscv/include/asm/Kbuild index 445ccc97305a..750c1056b90f 100644 --- a/arch/riscv/include/asm/Kbuild +++ b/arch/riscv/include/asm/Kbuild @@ -3,5 +3,8 @@ generic-y += early_ioremap.h generic-y += extable.h generic-y += flat.h generic-y += kvm_para.h +generic-y += mcs_spinlock.h +generic-y += qrwlock.h +generic-y += qspinlock.h generic-y += user.h generic-y += vmlinux.lds.h diff --git a/arch/riscv/include/asm/spinlock.h b/arch/riscv/include/asm/spinlock.h index f4f7fa1b7ca8..a557de67a425 100644 --- a/arch/riscv/include/asm/spinlock.h +++ b/arch/riscv/include/asm/spinlock.h @@ -7,129 +7,7 @@ #ifndef _ASM_RISCV_SPINLOCK_H #define _ASM_RISCV_SPINLOCK_H -#include -#include -#include - -/* - * Simple spin lock operations. These provide no fairness guarantees. - */ - -/* FIXME: Replace this with a ticket lock, like MIPS. */ - -#define arch_spin_is_locked(x) (READ_ONCE((x)->lock) != 0) - -static inline void arch_spin_unlock(arch_spinlock_t *lock) -{ - smp_store_release(&lock->lock, 0); -} - -static inline int arch_spin_trylock(arch_spinlock_t *lock) -{ - int tmp = 1, busy; - - __asm__ __volatile__ ( - " amoswap.w %0, %2, %1\n" - RISCV_ACQUIRE_BARRIER - : "=r" (busy), "+A" (lock->lock) - : "r" (tmp) - : "memory"); - - return !busy; -} - -static inline void arch_spin_lock(arch_spinlock_t *lock) -{ - while (1) { - if (arch_spin_is_locked(lock)) - continue; - - if (arch_spin_trylock(lock)) - break; - } -} - -/***********************************************************/ - -static inline void arch_read_lock(arch_rwlock_t *lock) -{ - int tmp; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bltz %1, 1b\n" - " addi %1, %1, 1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - : "+A" (lock->lock), "=&r" (tmp) - :: "memory"); -} - -static inline void arch_write_lock(arch_rwlock_t *lock) -{ - int tmp; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bnez %1, 1b\n" - " li %1, -1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - : "+A" (lock->lock), "=&r" (tmp) - :: "memory"); -} - -static inline int arch_read_trylock(arch_rwlock_t *lock) -{ - int busy; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bltz %1, 1f\n" - " addi %1, %1, 1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - "1:\n" - : "+A" (lock->lock), "=&r" (busy) - :: "memory"); - - return !busy; -} - -static inline int arch_write_trylock(arch_rwlock_t *lock) -{ - int busy; - - __asm__ __volatile__( - "1: lr.w %1, %0\n" - " bnez %1, 1f\n" - " li %1, -1\n" - " sc.w %1, %1, %0\n" - " bnez %1, 1b\n" - RISCV_ACQUIRE_BARRIER - "1:\n" - : "+A" (lock->lock), "=&r" (busy) - :: "memory"); - - return !busy; -} - -static inline void arch_read_unlock(arch_rwlock_t *lock) -{ - __asm__ __volatile__( - RISCV_RELEASE_BARRIER - " amoadd.w x0, %1, %0\n" - : "+A" (lock->lock) - : "r" (-1) - : "memory"); -} - -static inline void arch_write_unlock(arch_rwlock_t *lock) -{ - smp_store_release(&lock->lock, 0); -} +#include +#include #endif /* _ASM_RISCV_SPINLOCK_H */ diff --git a/arch/riscv/include/asm/spinlock_types.h b/arch/riscv/include/asm/spinlock_types.h index f398e7638dd6..d033a973f287 100644 --- a/arch/riscv/include/asm/spinlock_types.h +++ b/arch/riscv/include/asm/spinlock_types.h @@ -6,20 +6,11 @@ #ifndef _ASM_RISCV_SPINLOCK_TYPES_H #define _ASM_RISCV_SPINLOCK_TYPES_H -#ifndef __LINUX_SPINLOCK_TYPES_H +#if !defined(__LINUX_SPINLOCK_TYPES_H) && !defined(_ASM_RISCV_SPINLOCK_H) # error "please don't include this file directly" #endif -typedef struct { - volatile unsigned int lock; -} arch_spinlock_t; - -#define __ARCH_SPIN_LOCK_UNLOCKED { 0 } - -typedef struct { - volatile unsigned int lock; -} arch_rwlock_t; - -#define __ARCH_RW_LOCK_UNLOCKED { 0 } +#include +#include #endif /* _ASM_RISCV_SPINLOCK_TYPES_H */