From patchwork Tue Mar 13 22:46:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paolo Bonzini X-Patchwork-Id: 10281129 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 0C378601A0 for ; Tue, 13 Mar 2018 23:00:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EF9F028595 for ; Tue, 13 Mar 2018 23:00:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E4A79285A0; Tue, 13 Mar 2018 23:00:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8934C28595 for ; Tue, 13 Mar 2018 23:00:18 +0000 (UTC) Received: from localhost ([::1]:43348 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1evsu1-0008KJ-J8 for patchwork-qemu-devel@patchwork.kernel.org; Tue, 13 Mar 2018 19:00:17 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:58071) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1evsi1-0005kr-Up for qemu-devel@nongnu.org; Tue, 13 Mar 2018 18:47:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1evsi1-0003JV-2n for qemu-devel@nongnu.org; Tue, 13 Mar 2018 18:47:53 -0400 Received: from mail-wm0-x242.google.com ([2a00:1450:400c:c09::242]:33095) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1evsi0-0003I8-RY for qemu-devel@nongnu.org; Tue, 13 Mar 2018 18:47:52 -0400 Received: by mail-wm0-x242.google.com with SMTP id s206so16179041wme.0 for ; Tue, 13 Mar 2018 15:47:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:subject:date:message-id:in-reply-to:references; bh=ccO3p8t35wxJ+0nzNyklf4tZrUd4yZLT5/Kzmwd/wow=; b=sLV97+WlNqnvu/ac8gvyL/1Q37anXe8bdjsHmQdq0tl67tDoNXsk/DiQI8v5hqgqg2 0NN00fNu2dp/y2JFsZL5229MHx2fVS/l5U/Tlygszi2I9aWoVmQ8As5WyU3RB+I3Tbjw mFsypZ9WLZYh7kcw1JagR/jgInDa278cupGm+U1wX8TxFCykimfCyXcE3AtgBtLZ8fMU I8YR/FiTwcxZf25CfsLIO+y27P9pNnvLvpIjhLuJomA2I2EfeSKFla1/S0BupZO2coZl 3THS6Z6A349Qp7a79fIN6nldQYGjJFln3DRdtKDD2crK3uJy+XX9zTtTB2DDWesghmoh Czpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:subject:date:message-id :in-reply-to:references; bh=ccO3p8t35wxJ+0nzNyklf4tZrUd4yZLT5/Kzmwd/wow=; b=AV8XOfanpng0wrdSItFhnRnfM9kSPdUDY4gdNseyjd51OP1mVjWRoesprBE9J81d/S vEh5os15MedHgqxN+mdZAf9zIKHVZkMUbKH0/6QhW3W0GIEIHp/pR2FLpnht2p+rSC8y tWsNRlPfFA+/oOMhU6jbC09nkhbLtHTWsq0fIgBxujZSd8b1Yo/YEItKOt+Jufgl0NGz nKDxZWFBzNf8lJJd5lST1ndpPFKKWyFNRHNmJ5Hucjh5GdF2g9+7fKgmyBnpFmO4NZGW qaW1bba9NHDkra+lLgVve7eL2CATDwCFkvMlhRz9SW18Nnh1pgPKBh+9ts64pB321yh+ T3kw== X-Gm-Message-State: AElRT7FjbW7DtMAD4k7kr+vxAFn6hVMaQys5Z/2P4hMOZfE8VDSdlX47 L1bbV3iKX1qVluw+DGrqGspfiS/c X-Google-Smtp-Source: AG47ELsA+5IFKeSdlU94folMaef9j7LMf1QMQAaDJUxINb9jJGsJh5AWGfqYSv8fECP9uSlcxduBIQ== X-Received: by 10.28.180.132 with SMTP id d126mr2114056wmf.93.1520981271366; Tue, 13 Mar 2018 15:47:51 -0700 (PDT) Received: from donizetti.lan (94-36-191-219.adsl-ull.clienti.tiscali.it. [94.36.191.219]) by smtp.gmail.com with ESMTPSA id x107sm1557951wrb.97.2018.03.13.15.47.49 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 13 Mar 2018 15:47:50 -0700 (PDT) From: Paolo Bonzini To: qemu-devel@nongnu.org Date: Tue, 13 Mar 2018 23:46:34 +0100 Message-Id: <20180313224719.4954-25-pbonzini@redhat.com> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180313224719.4954-1-pbonzini@redhat.com> References: <20180313224719.4954-1-pbonzini@redhat.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::242 Subject: [Qemu-devel] [PULL 24/69] rcu: make memory barriers more explicit X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP Prepare for introducing smp_mb_placeholder() and smp_mb_global(). The new smp_mb() in synchronize_rcu() is not strictly necessary, since the first atomic_mb_set for rcu_gp_ctr provides the required ordering. However, synchronize_rcu is not performance critical, and it *will* be necessary to introduce a smp_mb_global before calling wait_for_readers(). Signed-off-by: Paolo Bonzini --- include/qemu/rcu.h | 15 +++++++++++++-- util/rcu.c | 12 +++++++++--- 2 files changed, 22 insertions(+), 5 deletions(-) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index f19413d649..625f09ac09 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h @@ -79,7 +79,10 @@ static inline void rcu_read_lock(void) } ctr = atomic_read(&rcu_gp_ctr); - atomic_xchg(&p_rcu_reader->ctr, ctr); + atomic_set(&p_rcu_reader->ctr, ctr); + + /* Write p_rcu_reader->ctr before reading RCU-protected pointers. */ + smp_mb(); } static inline void rcu_read_unlock(void) @@ -91,7 +94,15 @@ static inline void rcu_read_unlock(void) return; } - atomic_xchg(&p_rcu_reader->ctr, 0); + /* Ensure that the critical section is seen to precede the + * store to p_rcu_reader->ctr. Together with the following + * smp_mb(), this ensures writes to p_rcu_reader->ctr + * are sequentially consistent. + */ + atomic_store_release(&p_rcu_reader->ctr, 0); + + /* Write p_rcu_reader->ctr before reading p_rcu_reader->waiting. */ + smp_mb(); if (unlikely(atomic_read(&p_rcu_reader->waiting))) { atomic_set(&p_rcu_reader->waiting, false); qemu_event_set(&rcu_gp_event); diff --git a/util/rcu.c b/util/rcu.c index f4d09c8304..7366dc50dd 100644 --- a/util/rcu.c +++ b/util/rcu.c @@ -92,8 +92,9 @@ static void wait_for_readers(void) atomic_set(&index->waiting, true); } - /* Here, order the stores to index->waiting before the - * loads of index->ctr. + /* Here, order the stores to index->waiting before the loads of + * index->ctr. Pairs with smp_mb() in rcu_read_unlock(), + * ensuring that the loads of index->ctr are sequentially consistent. */ smp_mb(); @@ -142,8 +143,13 @@ static void wait_for_readers(void) void synchronize_rcu(void) { qemu_mutex_lock(&rcu_sync_lock); - qemu_mutex_lock(&rcu_registry_lock); + /* Write RCU-protected pointers before reading p_rcu_reader->ctr. + * Pairs with smp_mb() in rcu_read_lock(). + */ + smp_mb(); + + qemu_mutex_lock(&rcu_registry_lock); if (!QLIST_EMPTY(®istry)) { /* In either case, the atomic_mb_set below blocks stores that free * old RCU-protected pointers.