From patchwork Sat May 5 13:27:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ingo Molnar X-Patchwork-Id: 10382157 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 8AAB160541 for ; Sat, 5 May 2018 13:28:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6EA8128E29 for ; Sat, 5 May 2018 13:28:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5241D28E74; Sat, 5 May 2018 13:28:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: * X-Spam-Status: No, score=1.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, FSL_HELO_FAKE, MAILING_LIST_MULTI, T_DKIM_INVALID autolearn=no version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 95ED728E29 for ; Sat, 5 May 2018 13:28:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=niS1lp+LVMq6cJMrgdikb4iDT56cNEEOwAsija3A1cY=; b=ktIooRKoVSs7xw jBxa4h5Jn/jAV9z2oYWpZwfQSIMmNj6vHXOF7q0caE6y4Qdgy8tRJDl4andW1EZiFVyOQzH52Kwdp cZqbDQeoKPVC0gdxYSwoKLdiAlCIbEQeYm9czEA3KfByM8CM0r/KuC/xHNVlOfSQkLsXzw/WhNQCW /mMn/jeAlV9MyOBQ6OjYhMkYPkakhTwQXRQ/D7XPOD9IUHQ3THOKKQJcTKFL+u9Vma6T5R5U0p7+j mpJyr8NyuciDmAPjUk3oLn1ZhZB5iGVPSiCjhlWWc15HkBmqQVBor5jhREn0n0NiBtFEv+PDRTrvO 6ArI/YAsgCUKtUCB1vYw==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fExEQ-0000T9-PN; Sat, 05 May 2018 13:28:10 +0000 Received: from mail-wm0-x242.google.com ([2a00:1450:400c:c09::242]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1fExEM-0000Rq-C9 for linux-arm-kernel@lists.infradead.org; Sat, 05 May 2018 13:28:08 +0000 Received: by mail-wm0-x242.google.com with SMTP id w194so7743929wmf.2 for ; Sat, 05 May 2018 06:27:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=YXZbFSLr0x48MMcGRgb8mbucgVJpAnz8UL5UDSABatM=; b=W/ryZJJ4JOWAcfv/Xrarh1Pay7NpW4xp1MX96//YYKJ6JEJDtTezpJ5gRqMF+Dyype CtLmCOopkxzkgsmgIg+1wvHRiVfGJ6HNwgeGxtKNDCfexJGikI2hNjRifc9WG2t5GF4t LWYdWwg14QMGA6Z07n5HGOYNlSgFGqbCXDUBVhBIeq+Un2qLcAZDx1a6ve7wJWiCExst vDd6mRRZxXJzObQRxXyaufPheZ1dpPa7eTaeu1ltGAvk3TJ+Wsf6+28YgCCwKNe+AJAk Ryc5YBTeVsHAAz4+QDeYpruPDSvoVC5hfBevoWJxNN9OAA9XNaKE9r80SW5KHixz+6sO C4+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=YXZbFSLr0x48MMcGRgb8mbucgVJpAnz8UL5UDSABatM=; b=bn7Qtxxv7vPCg5nqxmMbqyMxtujlQddNc3jpbUXI2UR64ItdePPXePLcXKg7f6lhFi 7Dc6f3uQ2KN2NV6VgUepzpVtYBXdlYPsTQrNXSpbYc93e7PFWeHyyY1PjFJe0Cu9GbnV HJQH8I0RV6tso+1V+19I9Dr2sP4CVdfS7cB+XLz2J6e5Z44ngI7Zjv92oigZvBAWfgO6 jKVZvj3iFYcxxxjrOQSnJSIsQ5py1/D0qp+V0O6XXUReW6PZG0UKlRyoOPPMnLsNWth9 rQjQVnWIPOCIcA6mbnFbXY4KAfmdZbM+Z/vUWfui3eQcQA4q4x3RszVAp01iZMFrpWEu Vb4w== X-Gm-Message-State: ALQs6tBmVZk2gmlhh9lASkYKXn70d8eXrJHtIaNVbsg1Og0iHLE6/MSg Dvx9SlsspE4gli/3+EoZH5o= X-Google-Smtp-Source: AB8JxZobQY4g8SY2lEJ/QpD0C+LlZMmrEa6yYEigQqEbnXIEnYRXodtGYfPvqbP0KJmItheW5KxhEg== X-Received: by 10.28.158.144 with SMTP id h138mr20380971wme.33.1525526874579; Sat, 05 May 2018 06:27:54 -0700 (PDT) Received: from gmail.com (2E8B0CD5.catv.pool.telekom.hu. [46.139.12.213]) by smtp.gmail.com with ESMTPSA id x7-v6sm9605670wrm.35.2018.05.05.06.27.53 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 05 May 2018 06:27:53 -0700 (PDT) Date: Sat, 5 May 2018 15:27:51 +0200 From: Ingo Molnar To: Boqun Feng Subject: [PATCH] locking/atomics/powerpc: Move cmpxchg helpers to asm/cmpxchg.h and define the full set of cmpxchg APIs Message-ID: <20180505132751.gwzu2vbzibr2risd@gmail.com> References: <20180504173937.25300-2-mark.rutland@arm.com> <20180504180105.GS12217@hirez.programming.kicks-ass.net> <20180504180909.dnhfflibjwywnm4l@lakrids.cambridge.arm.com> <20180505081100.nsyrqrpzq2vd27bk@gmail.com> <20180505084721.GA32344@noisy.programming.kicks-ass.net> <20180505090403.p2ywuen42rnlwizq@gmail.com> <20180505093829.xfylnedwd5nonhae@gmail.com> <20180505101609.5wb56j4mspjkokmw@tardis> <20180505103550.s7xsnto7tgppkmle@gmail.com> <20180505112817.ihrb726i37bwm4cj@tardis> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20180505112817.ihrb726i37bwm4cj@tardis> User-Agent: NeoMutt/20170609 (1.8.3) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180505_062806_412825_E99D7F4C X-CRM114-Status: GOOD ( 25.64 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Peter Zijlstra , catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, dvyukov@google.com, aryabinin@virtuozzo.com, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP * Boqun Feng wrote: > > May I suggest the patch below? No change in functionality, but it documents the > > lack of the cmpxchg_release() APIs and maps them explicitly to the full cmpxchg() > > version. (Which the generic code does now in a rather roundabout way.) > > > > Hmm.. cmpxchg_release() is actually lwsync() + cmpxchg_relaxed(), but > you just make it sync() + cmpxchg_relaxed() + sync() with the fallback, > and sync() is much heavier, so I don't think the fallback is correct. Indeed! The bit I missed previously is that PowerPC provides its own __atomic_op_release() method: #define __atomic_op_release(op, args...) \ ({ \ __asm__ __volatile__(PPC_RELEASE_BARRIER "" : : : "memory"); \ op##_relaxed(args); \ }) ... which maps to LWSYNC as you say, and my patch made that worse. > I think maybe you can move powerpc's __atomic_op_{acqurie,release}() > from atomic.h to cmpxchg.h (in arch/powerpc/include/asm), and > > #define cmpxchg_release __atomic_op_release(cmpxchg, __VA_ARGS__); > #define cmpxchg64_release __atomic_op_release(cmpxchg64, __VA_ARGS__); > > I put a diff below to say what I mean (untested). > > > Also, the change to arch/powerpc/include/asm/atomic.h has no functional effect > > right now either, but should anyone add a _relaxed() variant in the future, with > > this change atomic_cmpxchg_release() and atomic64_cmpxchg_release() will pick that > > up automatically. > > > > You mean with your other modification in include/linux/atomic.h, right? > Because with the unmodified include/linux/atomic.h, we already pick that > autmatically. If so, I think that's fine. > > Here is the diff for the modification for cmpxchg_release(), the idea is > we generate them in asm/cmpxchg.h other than linux/atomic.h for ppc, so > we keep the new linux/atomic.h working. Because if I understand > correctly, the next linux/atomic.h only accepts that > > 1) architecture only defines fully ordered primitives > > or > > 2) architecture only defines _relaxed primitives > > or > > 3) architecture defines all four (fully, _relaxed, _acquire, > _release) primitives > > So powerpc needs to define all four primitives in its only > asm/cmpxchg.h. Correct, although the new logic is still RFC, PeterZ didn't like the first version I proposed and might NAK them. Thanks for the patch - I have created the patch below from it and added your Signed-off-by. The only change I made beyond a trivial build fix is that I also added the release atomics variants explicitly: +#define atomic_cmpxchg_release(v, o, n) \ + cmpxchg_release(&((v)->counter), (o), (n)) +#define atomic64_cmpxchg_release(v, o, n) \ + cmpxchg_release(&((v)->counter), (o), (n)) It has passed a PowerPC cross-build test here, but no runtime tests. Does this patch look good to you? (Still subject to PeterZ's Ack/NAK.) Thanks, Ingo ======================> From: Boqun Feng Date: Sat, 5 May 2018 19:28:17 +0800 Subject: [PATCH] locking/atomics/powerpc: Move cmpxchg helpers to asm/cmpxchg.h and define the full set of cmpxchg APIs Move PowerPC's __op_{acqurie,release}() from atomic.h to cmpxchg.h (in arch/powerpc/include/asm), plus use them to define these two methods: #define cmpxchg_release __op_release(cmpxchg, __VA_ARGS__); #define cmpxchg64_release __op_release(cmpxchg64, __VA_ARGS__); ... the idea is to generate all these methods in cmpxchg.h and to define the full array of atomic primitives, including the cmpxchg_release() methods which were defined by the generic code before. Also define the atomic[64]_() variants explicitly. This ensures that all these low level cmpxchg APIs are defined in PowerPC headers, with no generic header fallbacks. No change in functionality or code generation. Signed-off-by: Boqun Feng Cc: Linus Torvalds Cc: Mark Rutland Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: aryabinin@virtuozzo.com Cc: catalin.marinas@arm.com Cc: dvyukov@google.com Cc: linux-arm-kernel@lists.infradead.org Cc: will.deacon@arm.com Link: http://lkml.kernel.org/r/20180505112817.ihrb726i37bwm4cj@tardis Signed-off-by: Ingo Molnar --- arch/powerpc/include/asm/atomic.h | 22 ++++------------------ arch/powerpc/include/asm/cmpxchg.h | 24 ++++++++++++++++++++++++ 2 files changed, 28 insertions(+), 18 deletions(-) diff --git a/arch/powerpc/include/asm/atomic.h b/arch/powerpc/include/asm/atomic.h index 682b3e6a1e21..4e06955ec10f 100644 --- a/arch/powerpc/include/asm/atomic.h +++ b/arch/powerpc/include/asm/atomic.h @@ -13,24 +13,6 @@ #define ATOMIC_INIT(i) { (i) } -/* - * Since *_return_relaxed and {cmp}xchg_relaxed are implemented with - * a "bne-" instruction at the end, so an isync is enough as a acquire barrier - * on the platform without lwsync. - */ -#define __atomic_op_acquire(op, args...) \ -({ \ - typeof(op##_relaxed(args)) __ret = op##_relaxed(args); \ - __asm__ __volatile__(PPC_ACQUIRE_BARRIER "" : : : "memory"); \ - __ret; \ -}) - -#define __atomic_op_release(op, args...) \ -({ \ - __asm__ __volatile__(PPC_RELEASE_BARRIER "" : : : "memory"); \ - op##_relaxed(args); \ -}) - static __inline__ int atomic_read(const atomic_t *v) { int t; @@ -213,6 +195,8 @@ static __inline__ int atomic_dec_return_relaxed(atomic_t *v) cmpxchg_relaxed(&((v)->counter), (o), (n)) #define atomic_cmpxchg_acquire(v, o, n) \ cmpxchg_acquire(&((v)->counter), (o), (n)) +#define atomic_cmpxchg_release(v, o, n) \ + cmpxchg_release(&((v)->counter), (o), (n)) #define atomic_xchg(v, new) (xchg(&((v)->counter), new)) #define atomic_xchg_relaxed(v, new) xchg_relaxed(&((v)->counter), (new)) @@ -519,6 +503,8 @@ static __inline__ long atomic64_dec_if_positive(atomic64_t *v) cmpxchg_relaxed(&((v)->counter), (o), (n)) #define atomic64_cmpxchg_acquire(v, o, n) \ cmpxchg_acquire(&((v)->counter), (o), (n)) +#define atomic64_cmpxchg_release(v, o, n) \ + cmpxchg_release(&((v)->counter), (o), (n)) #define atomic64_xchg(v, new) (xchg(&((v)->counter), new)) #define atomic64_xchg_relaxed(v, new) xchg_relaxed(&((v)->counter), (new)) diff --git a/arch/powerpc/include/asm/cmpxchg.h b/arch/powerpc/include/asm/cmpxchg.h index 9b001f1f6b32..e27a612b957f 100644 --- a/arch/powerpc/include/asm/cmpxchg.h +++ b/arch/powerpc/include/asm/cmpxchg.h @@ -8,6 +8,24 @@ #include #include +/* + * Since *_return_relaxed and {cmp}xchg_relaxed are implemented with + * a "bne-" instruction at the end, so an isync is enough as a acquire barrier + * on the platform without lwsync. + */ +#define __atomic_op_acquire(op, args...) \ +({ \ + typeof(op##_relaxed(args)) __ret = op##_relaxed(args); \ + __asm__ __volatile__(PPC_ACQUIRE_BARRIER "" : : : "memory"); \ + __ret; \ +}) + +#define __atomic_op_release(op, args...) \ +({ \ + __asm__ __volatile__(PPC_RELEASE_BARRIER "" : : : "memory"); \ + op##_relaxed(args); \ +}) + #ifdef __BIG_ENDIAN #define BITOFF_CAL(size, off) ((sizeof(u32) - size - off) * BITS_PER_BYTE) #else @@ -512,6 +530,9 @@ __cmpxchg_acquire(void *ptr, unsigned long old, unsigned long new, (unsigned long)_o_, (unsigned long)_n_, \ sizeof(*(ptr))); \ }) + +#define cmpxchg_release(...) __atomic_op_release(cmpxchg, __VA_ARGS__) + #ifdef CONFIG_PPC64 #define cmpxchg64(ptr, o, n) \ ({ \ @@ -533,6 +554,9 @@ __cmpxchg_acquire(void *ptr, unsigned long old, unsigned long new, BUILD_BUG_ON(sizeof(*(ptr)) != 8); \ cmpxchg_acquire((ptr), (o), (n)); \ }) + +#define cmpxchg64_release(...) __atomic_op_release(cmpxchg64, __VA_ARGS__) + #else #include #define cmpxchg64_local(ptr, o, n) __cmpxchg64_local_generic((ptr), (o), (n))