From patchwork Wed Jan 2 20:57:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 10746685 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A2E9513BF for ; Wed, 2 Jan 2019 20:58:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8EBB727FA3 for ; Wed, 2 Jan 2019 20:58:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7F34427F3E; Wed, 2 Jan 2019 20:58:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E7EE227F3E for ; Wed, 2 Jan 2019 20:58:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728381AbfABU54 (ORCPT ); Wed, 2 Jan 2019 15:57:56 -0500 Received: from mx1.redhat.com ([209.132.183.28]:40882 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728327AbfABU5z (ORCPT ); Wed, 2 Jan 2019 15:57:55 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 95CA7C0259; Wed, 2 Jan 2019 20:57:54 +0000 (UTC) Received: from redhat.com (ovpn-120-148.rdu2.redhat.com [10.10.120.148]) by smtp.corp.redhat.com (Postfix) with SMTP id D93C960C69; Wed, 2 Jan 2019 20:57:49 +0000 (UTC) Date: Wed, 2 Jan 2019 15:57:49 -0500 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Jason Wang , Alan Stern , Andrea Parri , Will Deacon , Peter Zijlstra , Boqun Feng , Nicholas Piggin , David Howells , Jade Alglave , Luc Maranget , "Paul E. McKenney" , Akira Yokosawa , Daniel Lustig , linux-arch@vger.kernel.org, netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, Eli Friedman , Joe Perches , Nick Desaulniers , Linus Torvalds , Luc Van Oostenryck , linux-sparse@vger.kernel.org Subject: [PATCH RFC 1/4] include/linux/compiler*.h: fix OPTIMIZER_HIDE_VAR Message-ID: <20190102205715.14054-2-mst@redhat.com> References: <20190102205715.14054-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190102205715.14054-1-mst@redhat.com> X-Mutt-Fcc: =sent X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Wed, 02 Jan 2019 20:57:55 +0000 (UTC) Sender: linux-sparse-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sparse@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Since commit 815f0ddb346c ("include/linux/compiler*.h: make compiler-*.h mutually exclusive") clang no longer reuses the OPTIMIZER_HIDE_VAR macro from compiler-gcc - instead it gets the version in include/linux/compiler.h. Unfortunately that version doesn't actually prevent compiler from optimizing out the variable. Fix up by moving the macro out from compiler-gcc.h to compiler.h. Compilers without incline asm support will keep working since it's protected by an ifdef. Also fix up comments to match reality since we are no longer overriding any macros. Build-tested with gcc and clang. Fixes: 815f0ddb346c ("include/linux/compiler*.h: make compiler-*.h mutually exclusive") Cc: Eli Friedman Cc: Joe Perches Cc: Nick Desaulniers Cc: Linus Torvalds Signed-off-by: Michael S. Tsirkin Reviewed-by: Nick Desaulniers --- include/linux/compiler-clang.h | 5 ++--- include/linux/compiler-gcc.h | 4 ---- include/linux/compiler-intel.h | 4 +--- include/linux/compiler.h | 4 +++- 4 files changed, 6 insertions(+), 11 deletions(-) diff --git a/include/linux/compiler-clang.h b/include/linux/compiler-clang.h index 3e7dafb3ea80..7ddaeb5182e3 100644 --- a/include/linux/compiler-clang.h +++ b/include/linux/compiler-clang.h @@ -3,9 +3,8 @@ #error "Please don't include directly, include instead." #endif -/* Some compiler specific definitions are overwritten here - * for Clang compiler - */ +/* Compiler specific definitions for Clang compiler */ + #define uninitialized_var(x) x = *(&(x)) /* same as gcc, this was present in clang-2.6 so we can assume it works diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h index 2010493e1040..72054d9f0eaa 100644 --- a/include/linux/compiler-gcc.h +++ b/include/linux/compiler-gcc.h @@ -58,10 +58,6 @@ (typeof(ptr)) (__ptr + (off)); \ }) -/* Make the optimizer believe the variable can be manipulated arbitrarily. */ -#define OPTIMIZER_HIDE_VAR(var) \ - __asm__ ("" : "=r" (var) : "0" (var)) - /* * A trick to suppress uninitialized variable warning without generating any * code diff --git a/include/linux/compiler-intel.h b/include/linux/compiler-intel.h index 517bd14e1222..b17f3cd18334 100644 --- a/include/linux/compiler-intel.h +++ b/include/linux/compiler-intel.h @@ -5,9 +5,7 @@ #ifdef __ECC -/* Some compiler specific definitions are overwritten here - * for Intel ECC compiler - */ +/* Compiler specific definitions for Intel ECC compiler */ #include diff --git a/include/linux/compiler.h b/include/linux/compiler.h index 06396c1cf127..1ad367b4cd8d 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -152,7 +152,9 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, #endif #ifndef OPTIMIZER_HIDE_VAR -#define OPTIMIZER_HIDE_VAR(var) barrier() +/* Make the optimizer believe the variable can be manipulated arbitrarily. */ +#define OPTIMIZER_HIDE_VAR(var) \ + __asm__ ("" : "=r" (var) : "0" (var)) #endif /* Not-quite-unique ID. */ From patchwork Wed Jan 2 20:57:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 10746681 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 564DC6C5 for ; Wed, 2 Jan 2019 20:58:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4659D27FA3 for ; Wed, 2 Jan 2019 20:58:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3A32D2818E; Wed, 2 Jan 2019 20:58:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E88522810E for ; Wed, 2 Jan 2019 20:58:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728335AbfABU57 (ORCPT ); Wed, 2 Jan 2019 15:57:59 -0500 Received: from mx1.redhat.com ([209.132.183.28]:33920 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728463AbfABU57 (ORCPT ); Wed, 2 Jan 2019 15:57:59 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D01202C977B; Wed, 2 Jan 2019 20:57:58 +0000 (UTC) Received: from redhat.com (ovpn-120-148.rdu2.redhat.com [10.10.120.148]) by smtp.corp.redhat.com (Postfix) with SMTP id 81F3160C47; Wed, 2 Jan 2019 20:57:55 +0000 (UTC) Date: Wed, 2 Jan 2019 15:57:54 -0500 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Jason Wang , Alan Stern , Andrea Parri , Will Deacon , Peter Zijlstra , Boqun Feng , Nicholas Piggin , David Howells , Jade Alglave , Luc Maranget , "Paul E. McKenney" , Akira Yokosawa , Daniel Lustig , linux-arch@vger.kernel.org, netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, Luc Van Oostenryck , linux-sparse@vger.kernel.org Subject: [PATCH RFC 2/4] include/linux/compiler.h: allow memory operands Message-ID: <20190102205715.14054-3-mst@redhat.com> References: <20190102205715.14054-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190102205715.14054-1-mst@redhat.com> X-Mutt-Fcc: =sent X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Wed, 02 Jan 2019 20:57:59 +0000 (UTC) Sender: linux-sparse-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sparse@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We don't really care whether the variable is in-register or in-memory. Relax the constraint accordingly. Signed-off-by: Michael S. Tsirkin --- include/linux/compiler.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/compiler.h b/include/linux/compiler.h index 1ad367b4cd8d..6601d39e8c48 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -154,7 +154,7 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, #ifndef OPTIMIZER_HIDE_VAR /* Make the optimizer believe the variable can be manipulated arbitrarily. */ #define OPTIMIZER_HIDE_VAR(var) \ - __asm__ ("" : "=r" (var) : "0" (var)) + __asm__ ("" : "=rm" (var) : "0" (var)) #endif /* Not-quite-unique ID. */ From patchwork Wed Jan 2 20:57:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Michael S. Tsirkin" X-Patchwork-Id: 10746675 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 672B86C5 for ; Wed, 2 Jan 2019 20:58:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5155E27FA3 for ; Wed, 2 Jan 2019 20:58:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 41DEB2818E; Wed, 2 Jan 2019 20:58:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BAE9D27FA3 for ; Wed, 2 Jan 2019 20:58:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729016AbfABU6K (ORCPT ); Wed, 2 Jan 2019 15:58:10 -0500 Received: from mx1.redhat.com ([209.132.183.28]:35662 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728953AbfABU6G (ORCPT ); Wed, 2 Jan 2019 15:58:06 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8ED1B806BC; Wed, 2 Jan 2019 20:58:03 +0000 (UTC) Received: from redhat.com (ovpn-120-148.rdu2.redhat.com [10.10.120.148]) by smtp.corp.redhat.com (Postfix) with SMTP id 679BD60C47; Wed, 2 Jan 2019 20:57:59 +0000 (UTC) Date: Wed, 2 Jan 2019 15:57:58 -0500 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Jason Wang , Alan Stern , Andrea Parri , Will Deacon , Peter Zijlstra , Boqun Feng , Nicholas Piggin , David Howells , Jade Alglave , Luc Maranget , "Paul E. McKenney" , Akira Yokosawa , Daniel Lustig , linux-arch@vger.kernel.org, netdev@vger.kernel.org, virtualization@lists.linux-foundation.org, Jonathan Corbet , Richard Henderson , Ivan Kokshaysky , Matt Turner , Arnd Bergmann , Luc Van Oostenryck , linux-doc@vger.kernel.org, linux-alpha@vger.kernel.org, linux-sparse@vger.kernel.org Subject: [PATCH RFC 3/4] barriers: convert a control to a data dependency Message-ID: <20190102205715.14054-4-mst@redhat.com> References: <20190102205715.14054-1-mst@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20190102205715.14054-1-mst@redhat.com> X-Mutt-Fcc: =sent X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Wed, 02 Jan 2019 20:58:05 +0000 (UTC) Sender: linux-sparse-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sparse@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP It's not uncommon to have two access two unrelated memory locations in a specific order. At the moment one has to use a memory barrier for this. However, if the first access was a read and the second used an address depending on the first one we would have a data dependency and no barrier would be necessary. This adds a new interface: dependent_ptr_mb which does exactly this: it returns a pointer with a data dependency on the supplied value. Signed-off-by: Michael S. Tsirkin --- Documentation/memory-barriers.txt | 20 ++++++++++++++++++++ arch/alpha/include/asm/barrier.h | 1 + include/asm-generic/barrier.h | 18 ++++++++++++++++++ include/linux/compiler.h | 4 ++++ 4 files changed, 43 insertions(+) diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt index c1d913944ad8..9dbaa2e1dbf6 100644 --- a/Documentation/memory-barriers.txt +++ b/Documentation/memory-barriers.txt @@ -691,6 +691,18 @@ case what's actually required is: p = READ_ONCE(b); } +Alternatively, a control dependency can be converted to a data dependency, +e.g.: + + q = READ_ONCE(a); + if (q) { + b = dependent_ptr_mb(b, q); + p = READ_ONCE(b); + } + +Note how the result of dependent_ptr_mb must be used with the following +accesses in order to have an effect. + However, stores are not speculated. This means that ordering -is- provided for load-store control dependencies, as in the following example: @@ -836,6 +848,12 @@ out-guess your code. More generally, although READ_ONCE() does force the compiler to actually emit code for a given load, it does not force the compiler to use the results. +Converting to a data dependency helps with this too: + + q = READ_ONCE(a); + b = dependent_ptr_mb(b, q); + WRITE_ONCE(b, 1); + In addition, control dependencies apply only to the then-clause and else-clause of the if-statement in question. In particular, it does not necessarily apply to code following the if-statement: @@ -875,6 +893,8 @@ to the CPU containing it. See the section on "Multicopy atomicity" for more information. + + In summary: (*) Control dependencies can order prior loads against later stores. diff --git a/arch/alpha/include/asm/barrier.h b/arch/alpha/include/asm/barrier.h index 92ec486a4f9e..b4934e8c551b 100644 --- a/arch/alpha/include/asm/barrier.h +++ b/arch/alpha/include/asm/barrier.h @@ -59,6 +59,7 @@ * as Alpha, "y" could be set to 3 and "x" to 0. Use rmb() * in cases like this where there are no data dependencies. */ +#define ARCH_NEEDS_READ_BARRIER_DEPENDS 1 #define read_barrier_depends() __asm__ __volatile__("mb": : :"memory") #ifdef CONFIG_SMP diff --git a/include/asm-generic/barrier.h b/include/asm-generic/barrier.h index 2cafdbb9ae4c..fa2e2ef72b68 100644 --- a/include/asm-generic/barrier.h +++ b/include/asm-generic/barrier.h @@ -70,6 +70,24 @@ #define __smp_read_barrier_depends() read_barrier_depends() #endif +#if defined(COMPILER_HAS_OPTIMIZER_HIDE_VAR) && \ + !defined(ARCH_NEEDS_READ_BARRIER_DEPENDS) + +#define dependent_ptr_mb(ptr, val) ({ \ + long dependent_ptr_mb_val = (long)(val); \ + long dependent_ptr_mb_ptr = (long)(ptr) - dependent_ptr_mb_val; \ + \ + BUILD_BUG_ON(sizeof(val) > sizeof(long)); \ + OPTIMIZER_HIDE_VAR(dependent_ptr_mb_val); \ + (typeof(ptr))(dependent_ptr_mb_ptr + dependent_ptr_mb_val); \ +}) + +#else + +#define dependent_ptr_mb(ptr, val) ({ mb(); (ptr); }) + +#endif + #ifdef CONFIG_SMP #ifndef smp_mb diff --git a/include/linux/compiler.h b/include/linux/compiler.h index 6601d39e8c48..f599c30f1b28 100644 --- a/include/linux/compiler.h +++ b/include/linux/compiler.h @@ -152,9 +152,13 @@ void ftrace_likely_update(struct ftrace_likely_data *f, int val, #endif #ifndef OPTIMIZER_HIDE_VAR + /* Make the optimizer believe the variable can be manipulated arbitrarily. */ #define OPTIMIZER_HIDE_VAR(var) \ __asm__ ("" : "=rm" (var) : "0" (var)) + +#define COMPILER_HAS_OPTIMIZER_HIDE_VAR 1 + #endif /* Not-quite-unique ID. */