From patchwork Fri Mar 17 08:18:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13178673 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4F59EC7618A for ; Fri, 17 Mar 2023 08:20:49 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pd5JE-0006yk-M2; Fri, 17 Mar 2023 04:19:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pd5JC-0006xx-Pe for qemu-devel@nongnu.org; Fri, 17 Mar 2023 04:19:30 -0400 Received: from mail-pj1-x1036.google.com ([2607:f8b0:4864:20::1036]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pd5J9-0006Qt-Fg for qemu-devel@nongnu.org; Fri, 17 Mar 2023 04:19:30 -0400 Received: by mail-pj1-x1036.google.com with SMTP id y2so4342217pjg.3 for ; Fri, 17 Mar 2023 01:19:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1679041161; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1ynCqpQWIExEB2abHWcbxc3wkhfGjOmuAhmDexPEdiM=; b=dYruDzLAqNxBdnUWrEU1Yr3l5C3+JXH5xdfHE6Tc7MqjSVHAdTq92YnDhOa0DIK8OO VL5inJ7BNczSILVFidFwpart/4I3pBUbP1+trUYkPMvZMa0IFCTuBRtK3OfWg0uFrIFd 61ny5GRIjMqOzRGK3d4RhujTlPxRIufyYrZTb9EMrjgfVEGIBwU75ykD9AdpIlWjgxSU R64XXs1+0sVVtZqgDVTLxSmLNFbfcDfxFCdFnHuD9sYvdmZ1P/KnFiwJ67Tatv37QVfy GNDpl0TpCwNO61hw7Ci2jAUTgZjdSmW+hX8C2iXQ1mf8CGvHyL3Uiz3w9AoR1v/98uTE ATZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679041161; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1ynCqpQWIExEB2abHWcbxc3wkhfGjOmuAhmDexPEdiM=; b=QJzxmn0ftbMpOGTAgAPs9cHz8Pk61vnIVb468A4cTkq44f+EWk86KGU7L76L4m5wRK H30SBRNv2g5/QhbZ48uogwGQ3Xfph2J+dpL5hNuGewLALCy2OUetB74Y4mIt3c1lToxr 8vPrhtaykqALAasv2f+8vgrkm2bgOLcI2LSuQpv1TIPzYvDyoWDYHDOA44V8zPuJYqBE ZNsHT0d+KmYJdjBerS+fpem0xdLM5q+WKt69CFjv3DHMlo/GewKPmOdMO6VD1xrqOEXq fpfvuexRSlx3dYBtRJiaj9WgvFszuS15YBmbt6ncZUzqMIdG5+E+3gvuNmtrCYMxp6KC iDtA== X-Gm-Message-State: AO0yUKXDmGNQ6ORsvrddx+uVdf0tixsiKUJeBJSF2QGY+60RzmLROufI 50AOrIrjlaBngmJ/swuJwP0dBsaYGW4wRTRFtB4= X-Google-Smtp-Source: AK7set+spPOmNTnMioN6TBmzmrpBYuxAgsCz0g2vjiF+39z5QfTMx2YyZFb02mcnjWbvU9iWYEhlgQ== X-Received: by 2002:a05:6a20:6da3:b0:d7:19b3:a9db with SMTP id gl35-20020a056a206da300b000d719b3a9dbmr3937046pzb.1.1679041161319; Fri, 17 Mar 2023 01:19:21 -0700 (PDT) Received: from MacBook-Pro.local.bytedance.net ([139.177.225.249]) by smtp.gmail.com with ESMTPSA id p36-20020a631e64000000b0050be8e0b94csm873304pgm.90.2023.03.17.01.19.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Mar 2023 01:19:21 -0700 (PDT) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com Subject: [PATCH v8 1/6] memory: Reference as->current_map directly in memory commit Date: Fri, 17 Mar 2023 16:18:59 +0800 Message-Id: <20230317081904.24389-2-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20230317081904.24389-1-xuchuangxclwt@bytedance.com> References: <20230317081904.24389-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1036; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pj1-x1036.google.com X-Spam_score_int: 4 X-Spam_score: 0.4 X-Spam_bar: / X-Spam_report: (0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Xu Calling RCU variants of address_space_get|to_flatview() during memory commit (flatview updates, triggering memory listeners, or updating ioeventfds, etc.) is not 100% accurate, because commit() requires BQL rather than RCU read lock, so the context exclusively owns current_map and can be directly referenced. Neither does it need a refcount to current_map because it cannot be freed from under the caller. Add address_space_get_flatview_raw() for the case where the context holds BQL rather than RCU read lock and use it across the core memory updates, Drop the extra refcounts on FlatView*. Signed-off-by: Peter Xu --- softmmu/memory.c | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/softmmu/memory.c b/softmmu/memory.c index 4699ba55ec..a992a365d9 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -61,6 +61,13 @@ struct AddrRange { Int128 size; }; +/* Called with BQL held */ +static inline FlatView *address_space_to_flatview_raw(AddressSpace *as) +{ + assert(qemu_mutex_iothread_locked()); + return as->current_map; +} + static AddrRange addrrange_make(Int128 start, Int128 size) { return (AddrRange) { start, size }; @@ -155,7 +162,7 @@ enum ListenerDirection { Forward, Reverse }; #define MEMORY_LISTENER_UPDATE_REGION(fr, as, dir, callback, _args...) \ do { \ MemoryRegionSection mrs = section_from_flat_range(fr, \ - address_space_to_flatview(as)); \ + address_space_to_flatview_raw(as)); \ MEMORY_LISTENER_CALL(as, callback, dir, &mrs, ##_args); \ } while(0) @@ -753,6 +760,7 @@ static FlatView *generate_memory_topology(MemoryRegion *mr) } static void address_space_add_del_ioeventfds(AddressSpace *as, + FlatView *view, MemoryRegionIoeventfd *fds_new, unsigned fds_new_nb, MemoryRegionIoeventfd *fds_old, @@ -774,7 +782,7 @@ static void address_space_add_del_ioeventfds(AddressSpace *as, &fds_new[inew]))) { fd = &fds_old[iold]; section = (MemoryRegionSection) { - .fv = address_space_to_flatview(as), + .fv = view, .offset_within_address_space = int128_get64(fd->addr.start), .size = fd->addr.size, }; @@ -787,7 +795,7 @@ static void address_space_add_del_ioeventfds(AddressSpace *as, &fds_old[iold]))) { fd = &fds_new[inew]; section = (MemoryRegionSection) { - .fv = address_space_to_flatview(as), + .fv = view, .offset_within_address_space = int128_get64(fd->addr.start), .size = fd->addr.size, }; @@ -833,7 +841,7 @@ static void address_space_update_ioeventfds(AddressSpace *as) ioeventfd_max = QEMU_ALIGN_UP(as->ioeventfd_nb, 4); ioeventfds = g_new(MemoryRegionIoeventfd, ioeventfd_max); - view = address_space_get_flatview(as); + view = address_space_to_flatview_raw(as); FOR_EACH_FLAT_RANGE(fr, view) { for (i = 0; i < fr->mr->ioeventfd_nb; ++i) { tmp = addrrange_shift(fr->mr->ioeventfds[i].addr, @@ -852,13 +860,12 @@ static void address_space_update_ioeventfds(AddressSpace *as) } } - address_space_add_del_ioeventfds(as, ioeventfds, ioeventfd_nb, + address_space_add_del_ioeventfds(as, view, ioeventfds, ioeventfd_nb, as->ioeventfds, as->ioeventfd_nb); g_free(as->ioeventfds); as->ioeventfds = ioeventfds; as->ioeventfd_nb = ioeventfd_nb; - flatview_unref(view); } /* @@ -1026,7 +1033,7 @@ static void flatviews_reset(void) static void address_space_set_flatview(AddressSpace *as) { - FlatView *old_view = address_space_to_flatview(as); + FlatView *old_view = address_space_to_flatview_raw(as); MemoryRegion *physmr = memory_region_get_flatview_root(as->root); FlatView *new_view = g_hash_table_lookup(flat_views, physmr); @@ -2979,8 +2986,7 @@ static void listener_add_address_space(MemoryListener *listener, listener->log_global_start(listener); } } - - view = address_space_get_flatview(as); + view = address_space_to_flatview_raw(as); FOR_EACH_FLAT_RANGE(fr, view) { MemoryRegionSection section = section_from_flat_range(fr, view); @@ -2994,7 +3000,6 @@ static void listener_add_address_space(MemoryListener *listener, if (listener->commit) { listener->commit(listener); } - flatview_unref(view); } static void listener_del_address_space(MemoryListener *listener, @@ -3006,7 +3011,7 @@ static void listener_del_address_space(MemoryListener *listener, if (listener->begin) { listener->begin(listener); } - view = address_space_get_flatview(as); + view = address_space_to_flatview_raw(as); FOR_EACH_FLAT_RANGE(fr, view) { MemoryRegionSection section = section_from_flat_range(fr, view); @@ -3020,7 +3025,6 @@ static void listener_del_address_space(MemoryListener *listener, if (listener->commit) { listener->commit(listener); } - flatview_unref(view); } void memory_listener_register(MemoryListener *listener, AddressSpace *as) From patchwork Fri Mar 17 08:19:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13178676 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2A091C74A5B for ; Fri, 17 Mar 2023 08:21:25 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pd5JG-0006zR-0Z; Fri, 17 Mar 2023 04:19:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pd5JD-0006yd-UY for qemu-devel@nongnu.org; Fri, 17 Mar 2023 04:19:31 -0400 Received: from mail-pj1-x1034.google.com ([2607:f8b0:4864:20::1034]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pd5J9-0006RA-4I for qemu-devel@nongnu.org; Fri, 17 Mar 2023 04:19:31 -0400 Received: by mail-pj1-x1034.google.com with SMTP id om3-20020a17090b3a8300b0023efab0e3bfso8178924pjb.3 for ; Fri, 17 Mar 2023 01:19:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1679041165; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=me/gqRlFyHbSVeRcW2F2ZB6r0FGs3WMizMuVDN3fKSU=; b=jNH4dxEvj9341HvVljpM9YDzaGuv3id0GVsq3Zkz/6ngojk1jE4EZk+0PaJIfO57/E lURZg6K50KdPndIMTBLYar5R5pB2cILvgDkr1cjeN432YY923TuSWIDmO73gwH5z5vEf iN6mHE67qp1hvNbQ9XvxFMQVqAS3H+vPrNw11E7Eu703kKG/bpF1OOAF+w7KvyIHeXmE ltQjRtSoi9ImEiAvqNB1cXeV9QrGrhw5Y+Ef4bCovMCuJ5cu+dL4VitYrgkWuJAqpYEi UpyIg6e+M3WqIzvfoqvgcnIpjY2yVNWa4qom4D/7hcpqRezw9jbfw9BhJPdEyZm1u0tt +bAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679041165; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=me/gqRlFyHbSVeRcW2F2ZB6r0FGs3WMizMuVDN3fKSU=; b=29uuF37C9sLr4EVCfxPKuN8HZvsi30UkQ6m5eg8JLlqYjXNOYt38gZtZ1tLY1tjm33 RqR2ei3RLaqVOd+HDebvEvcz3NmDTUNv8SCL1FrymrXns/jt8JC5qacTYt32KoKk0HAI AQkHlVmHiY7Hqpm+FDhPtbwWx+PVJ9ZLn3y3cxtf2yFVrmDm92h0MM5425LieKsQlqoN wCnFwYteLFbhRRqog+7TEnLhGXFNrWUGPoYfrCD/ACCRutT4l+tidakvwU1JCKOYEepV 53B8syabX0I3w2eK3zB4wmKyLGx4rsjqoW9+HMKpxhx0hc9NVyZ+wBPcFjiAo0d7BECQ LMjg== X-Gm-Message-State: AO0yUKUmDUR0xMVSOVAStP8o6KfZ/OKfX2CNxmCW0a8rXni/gjCvDrTO zsmyDQb1gWZ4z5IeSZicbjrEG8f+7Y6KtLQJBNQ= X-Google-Smtp-Source: AK7set/Ri4Rx8VcCxKM+dpgY+DWAsX6vfawxWI06NDzs3aJOpKIQfwE3AwCHHBYguNqzxrvkRv5Eng== X-Received: by 2002:a17:90b:4f90:b0:23a:5f51:6ee5 with SMTP id qe16-20020a17090b4f9000b0023a5f516ee5mr7327867pjb.12.1679041165119; Fri, 17 Mar 2023 01:19:25 -0700 (PDT) Received: from MacBook-Pro.local.bytedance.net ([139.177.225.249]) by smtp.gmail.com with ESMTPSA id p36-20020a631e64000000b0050be8e0b94csm873304pgm.90.2023.03.17.01.19.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Mar 2023 01:19:24 -0700 (PDT) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [PATCH v8 2/6] rcu: Introduce rcu_read_is_locked() Date: Fri, 17 Mar 2023 16:19:00 +0800 Message-Id: <20230317081904.24389-3-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20230317081904.24389-1-xuchuangxclwt@bytedance.com> References: <20230317081904.24389-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1034; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pj1-x1034.google.com X-Spam_score_int: 4 X-Spam_score: 0.4 X-Spam_bar: / X-Spam_report: (0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add rcu_read_is_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu --- include/qemu/rcu.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index 313fc414bc..7bf45602e1 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h @@ -115,6 +115,13 @@ static inline void rcu_read_unlock(void) } } +static inline bool rcu_read_is_locked(void) +{ + struct rcu_reader_data *p_rcu_reader = get_ptr_rcu_reader(); + + return p_rcu_reader->depth > 0; +} + extern void synchronize_rcu(void); /* From patchwork Fri Mar 17 08:19:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13178677 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A5FC4C76195 for ; Fri, 17 Mar 2023 08:21:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pd5JG-0006zw-QA; Fri, 17 Mar 2023 04:19:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pd5JE-0006yo-ON for qemu-devel@nongnu.org; Fri, 17 Mar 2023 04:19:33 -0400 Received: from mail-pl1-x636.google.com ([2607:f8b0:4864:20::636]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pd5JC-0006RN-Uv for qemu-devel@nongnu.org; Fri, 17 Mar 2023 04:19:32 -0400 Received: by mail-pl1-x636.google.com with SMTP id u5so4555427plq.7 for ; Fri, 17 Mar 2023 01:19:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1679041169; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GG2aqHgGCiNg5mZ5qJ5G1bt8sGQLaCInJg5o3vlkQmc=; b=NdBW6N6cleaBklMeIPcenkiLkKNGdSUVoOxrC8eQU8gwukL1XNGjnAEh3NqNmGTNrZ x2t97sH31KbfZapDPtcLK186vmjYKQQP+VG8uLGDbr5wqirMwqz+4BVn704eYWUtzriB IUADIsVspY5mGKQ3Fx0oeOyALKt09FYG/OibmehPHptscgXY9klvLp5q0kE3kc3fF7xK 7XwTG0KJvuBS2Ov8GRtCYQ2pWAio27PP2rvvNhPo9QKKHBF3PfMeq0ptwFim+J774Fze EPP17JXgxjSvOnnflEZJxMx9R/Wqi8bFx5zUmbZRdNDNiWQJiNYV7wSPlAAicH+hlaay 1Qow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679041169; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GG2aqHgGCiNg5mZ5qJ5G1bt8sGQLaCInJg5o3vlkQmc=; b=piZphegFL2NMpvlYSf9KSJQWdYVQVsvjiJ3pMgu499suvDVUmak5jj1pC27vLiesrf fUPZ7cRRmDyemVYSIGN9F6pqQ2b3KQSNx1VZad2DsDEvWRIbug2rDc01k9S4LKm6g6Lo kebSU0TCVAKJyZzdLdqjixNaFJ+MDFHlxS6nn1j3LniftKPq1GG3Xg3uutUjwY9X9wb8 dtk2MKoFFLvt/4xcbDGvqRxIpi6N+T7UemXELApuBthHRM9W+vzScjitndgko3H7+jJJ oU9ne1DgePVGLSjqQnAPihOMOzFrAe1I7G51D4OpyqnyQk9grWvSCqSP6oCXB4T6ZtE2 MvwA== X-Gm-Message-State: AO0yUKVqbqQTk3s2HeR8jg+72OXQG0tsapJ8dFg/mwcMyEbHJULNxNw9 3JUEQ9pFUa30hCL0AirY0HrLcUzovVztWwKRZTo= X-Google-Smtp-Source: AK7set9wJ4R9anzfyXdbxYPRegGTQ3LtMZ/GMexmI8rTosrN6/SUeoa+1OvkZ0BlsLuZ0FO1aBJYMQ== X-Received: by 2002:a05:6a20:8f25:b0:d6:9939:b0a9 with SMTP id b37-20020a056a208f2500b000d69939b0a9mr2356238pzk.25.1679041168814; Fri, 17 Mar 2023 01:19:28 -0700 (PDT) Received: from MacBook-Pro.local.bytedance.net ([139.177.225.249]) by smtp.gmail.com with ESMTPSA id p36-20020a631e64000000b0050be8e0b94csm873304pgm.90.2023.03.17.01.19.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Mar 2023 01:19:28 -0700 (PDT) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [PATCH v8 3/6] memory: Introduce memory_region_transaction_do_commit() Date: Fri, 17 Mar 2023 16:19:01 +0800 Message-Id: <20230317081904.24389-4-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20230317081904.24389-1-xuchuangxclwt@bytedance.com> References: <20230317081904.24389-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::636; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pl1-x636.google.com X-Spam_score_int: 4 X-Spam_score: 0.4 X-Spam_bar: / X-Spam_report: (0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Split memory_region_transaction_do_commit() from memory_region_transaction_commit(). We'll call do_commit() in address_space_to_flatview() in the later patch. Signed-off-by: Chuang Xu --- softmmu/memory.c | 47 +++++++++++++++++++++++++++-------------------- 1 file changed, 27 insertions(+), 20 deletions(-) diff --git a/softmmu/memory.c b/softmmu/memory.c index a992a365d9..33ecc62ee9 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -1093,34 +1093,41 @@ void memory_region_transaction_begin(void) ++memory_region_transaction_depth; } -void memory_region_transaction_commit(void) +void memory_region_transaction_do_commit(void) { AddressSpace *as; - assert(memory_region_transaction_depth); assert(qemu_mutex_iothread_locked()); - --memory_region_transaction_depth; - if (!memory_region_transaction_depth) { - if (memory_region_update_pending) { - flatviews_reset(); + if (memory_region_update_pending) { + flatviews_reset(); - MEMORY_LISTENER_CALL_GLOBAL(begin, Forward); + MEMORY_LISTENER_CALL_GLOBAL(begin, Forward); - QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) { - address_space_set_flatview(as); - address_space_update_ioeventfds(as); - } - memory_region_update_pending = false; - ioeventfd_update_pending = false; - MEMORY_LISTENER_CALL_GLOBAL(commit, Forward); - } else if (ioeventfd_update_pending) { - QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) { - address_space_update_ioeventfds(as); - } - ioeventfd_update_pending = false; + QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) { + address_space_set_flatview(as); + address_space_update_ioeventfds(as); + } + memory_region_update_pending = false; + ioeventfd_update_pending = false; + MEMORY_LISTENER_CALL_GLOBAL(commit, Forward); + } else if (ioeventfd_update_pending) { + QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) { + address_space_update_ioeventfds(as); } - } + ioeventfd_update_pending = false; + } +} + +void memory_region_transaction_commit(void) +{ + assert(memory_region_transaction_depth); + assert(qemu_mutex_iothread_locked()); + + --memory_region_transaction_depth; + if (!memory_region_transaction_depth) { + memory_region_transaction_do_commit(); + } } static void memory_region_destructor_none(MemoryRegion *mr) From patchwork Fri Mar 17 08:19:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13178674 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B4581C74A5B for ; Fri, 17 Mar 2023 08:21:05 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pd5JJ-00070J-GI; Fri, 17 Mar 2023 04:19:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pd5JH-000707-LL for qemu-devel@nongnu.org; Fri, 17 Mar 2023 04:19:35 -0400 Received: from mail-pl1-x62a.google.com ([2607:f8b0:4864:20::62a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pd5JG-0006SN-06 for qemu-devel@nongnu.org; Fri, 17 Mar 2023 04:19:35 -0400 Received: by mail-pl1-x62a.google.com with SMTP id p20so4538663plw.13 for ; Fri, 17 Mar 2023 01:19:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1679041172; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g9aFN8Ma91UuxeJMWpFWAGfhx+Xhxyy3JILvaccFafU=; b=QwminStduRfZUjPoT7+Lo7Y/tM/saw/kCDKmFYfWdnmIN7dkJXMyFBOIG4Nc+X88BV ddo32y2CQ4Yf3RJoIWVIYbG+/lHXSuPim7G3YiD2iUwD3DuSX0RJuY+qR4odDGaXcWjn h/+TkeC/i33INuhq0gs9bXE93er+wJpSivNsXm3LuiwsBYdAp5f04QlL4+GLk1gohrFU hZT5674oIIl8gUZPK0u0wy6ea8pWHvriut+W+YehuG67VrNbXmLRLofxwS2dX8K+y6Ab Vsx+8XRorK1KpErUBQJcaf+MlDgoRRdGXpT0pgSbfx+MvdNi9Cpga4lMczmvYZYkvT6A eCTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679041172; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=g9aFN8Ma91UuxeJMWpFWAGfhx+Xhxyy3JILvaccFafU=; b=T7bcYj9Wo1ziznWmM3JvrVyx4euxQglhSlmRawV6dudwqxak2UsLOFgJ5jGSR7BUFd MZsmmic0B13RbyKRixBcrdtUEj2EAyBltxD/opk+BtxjuwQwFs17xlTC2FoZ0anZo2dJ MXVPulXXXf1ov2kaDbl2Z4EFLBltXCB3tKrmVugo+F308ZxfAfZWqC/GoODshKgzTxsF bXNukwYcLS3JsTkBBht9LGoR0AKRtpEa5kJbcAJCo4Eyi5YtqN5aGaYWApFi/OSAWMuE hNR8fQwpV5IilXxBw9OqEewUJxl7/JmDiiZKY1U2K8kysdLYN6PrcRMFzF0T8dRsr79V S+KA== X-Gm-Message-State: AO0yUKXyp6n8h8bP+elk2hcnMs1BvQVZ5SOF3cb1KjQrzVpLZCxXIWqK HXaV9HjAKfVAZM/oIiUS9OWGCjmzeLhXMXQG170= X-Google-Smtp-Source: AK7set+yxP3QMX7Xxk+8LV3KBjD4zvMl90+zIuqRPxUk4e0T/n/o0gizYHPZDgB6N5zDb9kFgCNvQA== X-Received: by 2002:a05:6a20:a111:b0:d3:76e0:89fd with SMTP id q17-20020a056a20a11100b000d376e089fdmr8204758pzk.50.1679041172550; Fri, 17 Mar 2023 01:19:32 -0700 (PDT) Received: from MacBook-Pro.local.bytedance.net ([139.177.225.249]) by smtp.gmail.com with ESMTPSA id p36-20020a631e64000000b0050be8e0b94csm873304pgm.90.2023.03.17.01.19.29 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Mar 2023 01:19:32 -0700 (PDT) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [PATCH v8 4/6] memory: Add do_commit() and sanity check in address_space_to_flatview Date: Fri, 17 Mar 2023 16:19:02 +0800 Message-Id: <20230317081904.24389-5-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20230317081904.24389-1-xuchuangxclwt@bytedance.com> References: <20230317081904.24389-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62a; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pl1-x62a.google.com X-Spam_score_int: 4 X-Spam_score: 0.4 X-Spam_bar: / X-Spam_report: (0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org In next patch, we wrap the vm_load into a whole mr transaction to speed up vm_load. This brings a problem, old flatviews may be referenced during the vm_load. vm_load contains far more memory updates than referencing to a specific flatview, hence we introduce do_commit to make sure address_space_to_flatview will return the newest flatview and it should logically only be triggered in a few spots during vm_load. Other than that, sanity check whether BQL or rcu is held before using any flatview. Signed-off-by: Chuang Xu --- include/exec/memory.h | 23 +++++++++++++++++++++++ softmmu/memory.c | 5 +++++ 2 files changed, 28 insertions(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index 6fa0b071f0..d6fd89db64 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -27,6 +27,7 @@ #include "qemu/notify.h" #include "qom/object.h" #include "qemu/rcu.h" +#include "qemu/main-loop.h" #define RAM_ADDR_INVALID (~(ram_addr_t)0) @@ -1095,8 +1096,30 @@ struct FlatView { MemoryRegion *root; }; +bool memory_region_transaction_in_progress(void); + +void memory_region_transaction_do_commit(void); + static inline FlatView *address_space_to_flatview(AddressSpace *as) { + if (qemu_mutex_iothread_locked()) { + /* We exclusively own the flatview now.. */ + if (memory_region_transaction_in_progress()) { + /* + * Fetch the flatview within a transaction in-progress, it + * means current_map may not be the latest, we need to update + * immediately to make sure the caller won't see obsolete + * mapping. + */ + memory_region_transaction_do_commit(); + } + + /* No further protection needed to access current_map */ + return as->current_map; + } + + /* Otherwise we must have had the RCU lock or something went wrong */ + assert(rcu_read_is_locked()); return qatomic_rcu_read(&as->current_map); } diff --git a/softmmu/memory.c b/softmmu/memory.c index 33ecc62ee9..6a8e8b4e71 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -1130,6 +1130,11 @@ void memory_region_transaction_commit(void) } } +bool memory_region_transaction_in_progress(void) +{ + return memory_region_transaction_depth != 0; +} + static void memory_region_destructor_none(MemoryRegion *mr) { } From patchwork Fri Mar 17 08:19:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13178672 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BD13BC76195 for ; Fri, 17 Mar 2023 08:20:46 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pd5JO-00071P-TD; Fri, 17 Mar 2023 04:19:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pd5JL-00070a-Jk for qemu-devel@nongnu.org; Fri, 17 Mar 2023 04:19:39 -0400 Received: from mail-pj1-x1033.google.com ([2607:f8b0:4864:20::1033]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pd5JJ-0006Sv-VN for qemu-devel@nongnu.org; Fri, 17 Mar 2023 04:19:39 -0400 Received: by mail-pj1-x1033.google.com with SMTP id h12-20020a17090aea8c00b0023d1311fab3so4431019pjz.1 for ; Fri, 17 Mar 2023 01:19:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1679041176; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AP9kMZ+IWsggIfpBp8YbbnkPCWJhhQYUg0buOuv8PNk=; b=gRAF9DZyS0JAjPLvmwqkHXrj3tHTnGGmdo7IthY2iFKoucar8b2TT5Nf2OD8/jW0m1 aRkhWf3rBUWNgEBkotWeYgLDKT4b4SkTkg6nUr2m6gQNRUdIxLPIPA9je/ZEO1iGUNVx xIRkwNuhz7dq/Jrzc8PDYi80byHFMQfBwdGzpf1xKlVTw18OUwTVJ+guFyBpjycXvMdA 94uxsizsh3xHafjH2DJhqXWb2J6cAh32AfuRqEIU1WJUZqNrulNCn15t5bNzbaFG+/uX QNmJv+Vs1IzqUEVSMzmMP2/OhXgERTS5GZAAWa7AIqC3PYsjo4hH/2zX/ZjVo0Lf3o7N trOg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679041176; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AP9kMZ+IWsggIfpBp8YbbnkPCWJhhQYUg0buOuv8PNk=; b=J8lynyInKCTh87YANmYZ7P/oeddLk3boVkU280uu3JBvCLLYdiWfMrjaL8I6FDGdSy iPYhOyu5omp15rET6m3k5/6tGw494d0GqzMuCaUbdSjBS0oeRTAOdvcmpIe1+QyURU6D 7Bjt0sMcJvp7q1MK32WMiw0og4CTmtddBK32WKHUAo2Eqzew8fLFFkvYBvgYd98m/RCh suRRmY7UCLsuEh9U1D6AkhT+WXXsU7hu+lI7IuhmevbVW853cBEasEI6euWF2SJz3mmW sv3uJJSlyLfIQReYYVTXMYw6x3Ibwnm2CClIAztGvLhhEGnbgdjJsViIXO4NSqItbV1c fSpw== X-Gm-Message-State: AO0yUKUiWISiZ7d3/LGwux1h3N+uKulQho08XFizQYQvbDnm9n6Hjbtq TTdX+o4XJt78mY9wn7aZluMnFwMz8lRdigL3lXI= X-Google-Smtp-Source: AK7set9M7O2VuxnX/bIofmPuDuUcXRApeW7JjYibZetUIFAA9mTSZ58FB5k2KjW1QNawcBMh+huHQw== X-Received: by 2002:a17:903:2306:b0:19a:b588:6fe2 with SMTP id d6-20020a170903230600b0019ab5886fe2mr2299083plh.13.1679041176339; Fri, 17 Mar 2023 01:19:36 -0700 (PDT) Received: from MacBook-Pro.local.bytedance.net ([139.177.225.249]) by smtp.gmail.com with ESMTPSA id p36-20020a631e64000000b0050be8e0b94csm873304pgm.90.2023.03.17.01.19.32 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Mar 2023 01:19:35 -0700 (PDT) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [PATCH v8 5/6] migration: Reduce time of loading non-iterable vmstate Date: Fri, 17 Mar 2023 16:19:03 +0800 Message-Id: <20230317081904.24389-6-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20230317081904.24389-1-xuchuangxclwt@bytedance.com> References: <20230317081904.24389-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1033; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pj1-x1033.google.com X-Spam_score_int: 4 X-Spam_score: 0.4 X-Spam_bar: / X-Spam_report: (0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop and ending with the timestamp of target qemu start). Most of the time is spent committing memory region changes repeatedly. This patch packs all the changes to memory region during the period of loading non-iterable vmstate in a single memory transaction. With the increase of devices, this patch will greatly improve the performance. Note that the following test results are based on the application of the next patch. Without the next patch, the improvement will be reduced. Here are the test1 results: test info: - Host - Intel(R) Xeon(R) Platinum 8362 CPU - Mellanox Technologies MT28841 - VM - 32 CPUs 128GB RAM VM - 8 16-queue vhost-net device - 16 4-queue vhost-user-blk device. time of loading non-iterable vmstate downtime before about 112 ms 285 ms after about 20 ms 194 ms In test2, we keep the number of the device the same as test1, reduce the number of queues per device: Here are the test2 results: test info: - Host - Intel(R) Xeon(R) Platinum 8362 CPU - Mellanox Technologies MT28841 - VM - 32 CPUs 128GB RAM VM - 8 1-queue vhost-net device - 16 1-queue vhost-user-blk device. time of loading non-iterable vmstate downtime before about 65 ms about 151 ms after about 19 ms about 100 ms In test3, we keep the number of queues per device the same as test1, reduce the number of devices: Here are the test3 results: test info: - Host - Intel(R) Xeon(R) Platinum 8362 CPU - Mellanox Technologies MT28841 - VM - 32 CPUs 128GB RAM VM - 1 16-queue vhost-net device - 1 4-queue vhost-user-blk device. time of loading non-iterable vmstate downtime before about 24 ms about 51 ms after about 9 ms about 36 ms As we can see from the test results above, both the number of queues and the number of devices have a great impact on the time of loading non-iterable vmstate. The growth of the number of devices and queues will lead to more mr commits, and the time consumption caused by the flatview reconstruction will also increase. Signed-off-by: Chuang Xu --- migration/savevm.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index aa54a67fda..ecf7b27000 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2787,7 +2787,25 @@ int qemu_loadvm_state(QEMUFile *f) cpu_synchronize_all_pre_loadvm(); + /* + * Call memory_region_transaction_begin() before loading vmstate. + * This call is paired with memory_region_transaction_commit() at + * the end of qemu_loadvm_state_main(), in order to pack all the + * changes to memory region during the period of loading + * non-iterable vmstate in a single memory transaction. + * This operation will reduce time of loading non-iterable vmstate + */ + memory_region_transaction_begin(); + ret = qemu_loadvm_state_main(f, mis); + + /* + * Call memory_region_transaction_commit() after loading vmstate. + * At this point, qemu actually completes all the previous memory + * region transactions. + */ + memory_region_transaction_commit(); + qemu_event_set(&mis->main_thread_load_event); trace_qemu_loadvm_state_post_main(ret); From patchwork Fri Mar 17 08:19:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13178671 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 55129C74A5B for ; Fri, 17 Mar 2023 08:20:46 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pd5JS-00072V-EQ; Fri, 17 Mar 2023 04:19:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pd5JP-00071y-Lx for qemu-devel@nongnu.org; Fri, 17 Mar 2023 04:19:43 -0400 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pd5JN-0006TT-UL for qemu-devel@nongnu.org; Fri, 17 Mar 2023 04:19:43 -0400 Received: by mail-pl1-x62e.google.com with SMTP id ix20so4572256plb.3 for ; Fri, 17 Mar 2023 01:19:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1679041180; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3N7ETY31dIiZEWsYXp+kYxAK9LnKSJagn3s6+DE/F+4=; b=IFPpNjMHV1ck7PnTn3vHG8VVtYPyG8+J9UuJwsXfkVC7ooCR5LHC1uJBePAhBXYWa2 s1jvIL5MPghh3xn5ZwGNU9YZjmDith5CTGvkxzGH73idelxNDauzN+Bsxr3TlX2Wn86c 27vjK2iQVjmcNdhjp4jfoYQr+xSTg2ztL9dfhzz9bq5kDLyhyAeLIR2HTSqmUoTyC1n8 Bb5KxQ0jqYWdggqPyEKEYLhZEOtobriePAP7SEOWLFqrQRfhV+D9XiQlhWdUv9l4k693 MWVn0AIRU0NQcw9zxAq4A71ZqTFhNtXaT4XaKoKhvA04+RkYJl0dV2E//nfab828z7Io VaeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679041180; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3N7ETY31dIiZEWsYXp+kYxAK9LnKSJagn3s6+DE/F+4=; b=Z2U9wRgd5tKt0aZnyT31mM1NyyUGAhiF7S0l8SeJCA1WfaRT9A1HscEEUp9ArjyZQE GGAiZrSGiAJbTBwhGZu3UfL1Xo/EgucDC3HG1I3THU1w3u1je/kOpEED2Hwf8ikspSgs AA7p3qMCtdFpJLlhaRq8AIRKbi2sSXmLs6gocusVN3wtd7TtaW8KfN5lmjgS1TvaiaxQ /0hhMchJm64GdOG/LpC9+us6kB+uq5wt5TpJOAjkZ4p3tEifdEksJelp5hY1ri2/asj2 6eaCeRM+aP5OdCUAhFO40woGcUm5DHMRveWkUqZm0DVjAX2zWcKwB4ZEHQOXAuaKg7+5 maMA== X-Gm-Message-State: AO0yUKUuva14mT/S7jxl1NYWiCQjP0rqyhywcQe8xCuv+3DP7o3Gpreq a2Wu/iEq0MBYarsl17+IQ3nrgFCeCtrX6pE9/2o= X-Google-Smtp-Source: AK7set/aDNHq18k+E9fQx6HGB0SlmUm11TixjXBx68hPi0WI+PS7o/9Nx4d3GU5Aj8DDkJ6AqZ2Drg== X-Received: by 2002:a05:6a20:3d02:b0:d5:d300:adea with SMTP id y2-20020a056a203d0200b000d5d300adeamr2490439pzi.8.1679041180104; Fri, 17 Mar 2023 01:19:40 -0700 (PDT) Received: from MacBook-Pro.local.bytedance.net ([139.177.225.249]) by smtp.gmail.com with ESMTPSA id p36-20020a631e64000000b0050be8e0b94csm873304pgm.90.2023.03.17.01.19.36 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 17 Mar 2023 01:19:39 -0700 (PDT) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [PATCH v8 6/6] memory: Introduce address_space_to_flatview_rcu() Date: Fri, 17 Mar 2023 16:19:04 +0800 Message-Id: <20230317081904.24389-7-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <20230317081904.24389-1-xuchuangxclwt@bytedance.com> References: <20230317081904.24389-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62e; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pl1-x62e.google.com X-Spam_score_int: 4 X-Spam_score: 0.4 X-Spam_bar: / X-Spam_report: (0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org In last patch, we wrap vm_load with begin/commit, here we introduce address_space_to_flatview_rcu() to avoid unnecessary enforce commit during vm_load. Signed-off-by: Chuang Xu --- include/exec/memory.h | 17 +++++++++++++++++ softmmu/memory.c | 2 +- 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/include/exec/memory.h b/include/exec/memory.h index d6fd89db64..2bf702dc94 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -1123,6 +1123,23 @@ static inline FlatView *address_space_to_flatview(AddressSpace *as) return qatomic_rcu_read(&as->current_map); } +/* + * We recommend using address_space_to_flatview() rather than this one. + * Note that if we use this during a memory region transaction, we may + * see obsolete flatviews. We must bear with an obsolete map until commit. + * And outside a memory region transaction, this is basically the same as + * address_space_to_flatview(). + */ +static inline FlatView *address_space_to_flatview_rcu(AddressSpace *as) +{ + /* + * Before using any flatview, sanity check BQL or RCU is held. + */ + assert(qemu_mutex_iothread_locked() || rcu_read_is_locked()); + + return qatomic_rcu_read(&as->current_map); +} + /** * typedef flatview_cb: callback for flatview_for_each_range() * diff --git a/softmmu/memory.c b/softmmu/memory.c index 6a8e8b4e71..33d14e967d 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -815,7 +815,7 @@ FlatView *address_space_get_flatview(AddressSpace *as) RCU_READ_LOCK_GUARD(); do { - view = address_space_to_flatview(as); + view = address_space_to_flatview_rcu(as); /* If somebody has replaced as->current_map concurrently, * flatview_ref returns false. */