From patchwork Thu Dec 13 23:58:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Benjamin Herrenschmidt X-Patchwork-Id: 10730119 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 056811751 for ; Thu, 13 Dec 2018 23:59:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EC90D2CD9B for ; Thu, 13 Dec 2018 23:59:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E12402CDA2; Thu, 13 Dec 2018 23:59:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 787EA2CD9F for ; Thu, 13 Dec 2018 23:59:44 +0000 (UTC) Received: from localhost ([::1]:57470 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXatL-0000Wk-Pn for patchwork-qemu-devel@patchwork.kernel.org; Thu, 13 Dec 2018 18:59:43 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33380) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gXasT-0000UQ-TT for qemu-devel@nongnu.org; Thu, 13 Dec 2018 18:58:50 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gXasP-00077J-Sj for qemu-devel@nongnu.org; Thu, 13 Dec 2018 18:58:49 -0500 Received: from gate.crashing.org ([63.228.1.57]:37791) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gXasP-000743-IG for qemu-devel@nongnu.org; Thu, 13 Dec 2018 18:58:45 -0500 Received: from pasglop.ozlabs.ibm.com (localhost.localdomain [127.0.0.1]) by gate.crashing.org (8.14.1/8.14.1) with ESMTP id wBDNwDG8012013; Thu, 13 Dec 2018 17:58:15 -0600 From: Benjamin Herrenschmidt To: qemu-devel@nongnu.org Date: Fri, 14 Dec 2018 10:58:02 +1100 Message-Id: <20181213235804.14956-1-benh@kernel.crashing.org> X-Mailer: git-send-email 2.19.2 MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x [fuzzy] X-Received-From: 63.228.1.57 Subject: [Qemu-devel] [PATCH 1/3] memory_ldst: Add atomic ops for PTE updates X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" X-Virus-Scanned: ClamAV using ClamSMTP On some architectures, PTE updates for dirty and changed bits need to be performed atomically. This adds a couple of address_space_cmpxchg* helpers for that purpose. Signed-off-by: Benjamin Herrenschmidt --- include/exec/memory_ldst.inc.h | 6 +++ memory_ldst.inc.c | 78 ++++++++++++++++++++++++++++++++++ 2 files changed, 84 insertions(+) diff --git a/include/exec/memory_ldst.inc.h b/include/exec/memory_ldst.inc.h index 272c20f02e..f3cfa7e9a6 100644 --- a/include/exec/memory_ldst.inc.h +++ b/include/exec/memory_ldst.inc.h @@ -28,6 +28,12 @@ extern uint64_t glue(address_space_ldq, SUFFIX)(ARG1_DECL, hwaddr addr, MemTxAttrs attrs, MemTxResult *result); extern void glue(address_space_stl_notdirty, SUFFIX)(ARG1_DECL, hwaddr addr, uint32_t val, MemTxAttrs attrs, MemTxResult *result); +extern uint32_t glue(address_space_cmpxchgl_notdirty, SUFFIX)(ARG1_DECL, + hwaddr addr, uint32_t old, uint32_t new, MemTxAttrs attrs, + MemTxResult *result); +extern uint32_t glue(address_space_cmpxchgq_notdirty, SUFFIX)(ARG1_DECL, + hwaddr addr, uint64_t old, uint64_t new, MemTxAttrs attrs, + MemTxResult *result); extern void glue(address_space_stw, SUFFIX)(ARG1_DECL, hwaddr addr, uint32_t val, MemTxAttrs attrs, MemTxResult *result); extern void glue(address_space_stl, SUFFIX)(ARG1_DECL, diff --git a/memory_ldst.inc.c b/memory_ldst.inc.c index acf865b900..7ab6de37ba 100644 --- a/memory_ldst.inc.c +++ b/memory_ldst.inc.c @@ -320,6 +320,84 @@ void glue(address_space_stl_notdirty, SUFFIX)(ARG1_DECL, RCU_READ_UNLOCK(); } +/* This is meant to be used for atomic PTE updates under MT-TCG */ +uint32_t glue(address_space_cmpxchgl_notdirty, SUFFIX)(ARG1_DECL, + hwaddr addr, uint32_t old, uint32_t new, MemTxAttrs attrs, MemTxResult *result) +{ + uint8_t *ptr; + MemoryRegion *mr; + hwaddr l = 4; + hwaddr addr1; + MemTxResult r; + uint8_t dirty_log_mask; + + /* Must test result */ + assert(result); + + RCU_READ_LOCK(); + mr = TRANSLATE(addr, &addr1, &l, true, attrs); + if (l < 4 || !memory_access_is_direct(mr, true)) { + r = MEMTX_ERROR; + } else { + uint32_t orig = old; + + ptr = qemu_map_ram_ptr(mr->ram_block, addr1); + old = atomic_cmpxchg(ptr, orig, new); + + if (old == orig) { + dirty_log_mask = memory_region_get_dirty_log_mask(mr); + dirty_log_mask &= ~(1 << DIRTY_MEMORY_CODE); + cpu_physical_memory_set_dirty_range(memory_region_get_ram_addr(mr) + addr, + 4, dirty_log_mask); + } + r = MEMTX_OK; + } + *result = r; + RCU_READ_UNLOCK(); + + return old; +} + +#ifdef CONFIG_ATOMIC64 +/* This is meant to be used for atomic PTE updates under MT-TCG */ +uint32_t glue(address_space_cmpxchgq_notdirty, SUFFIX)(ARG1_DECL, + hwaddr addr, uint64_t old, uint64_t new, MemTxAttrs attrs, MemTxResult *result) +{ + uint8_t *ptr; + MemoryRegion *mr; + hwaddr l = 8; + hwaddr addr1; + MemTxResult r; + uint8_t dirty_log_mask; + + /* Must test result */ + assert(result); + + RCU_READ_LOCK(); + mr = TRANSLATE(addr, &addr1, &l, true, attrs); + if (l < 8 || !memory_access_is_direct(mr, true)) { + r = MEMTX_ERROR; + } else { + uint32_t orig = old; + + ptr = qemu_map_ram_ptr(mr->ram_block, addr1); + old = atomic_cmpxchg(ptr, orig, new); + + if (old == orig) { + dirty_log_mask = memory_region_get_dirty_log_mask(mr); + dirty_log_mask &= ~(1 << DIRTY_MEMORY_CODE); + cpu_physical_memory_set_dirty_range(memory_region_get_ram_addr(mr) + addr, + 8, dirty_log_mask); + } + r = MEMTX_OK; + } + *result = r; + RCU_READ_UNLOCK(); + + return old; +} +#endif /* CONFIG_ATOMIC64 */ + /* warning: addr must be aligned */ static inline void glue(address_space_stl_internal, SUFFIX)(ARG1_DECL, hwaddr addr, uint32_t val, MemTxAttrs attrs,