From patchwork Fri Mar 10 02:24:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13168634 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9EF45C61DA4 for ; Fri, 10 Mar 2023 02:25:22 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1paSR0-0005c8-3H; Thu, 09 Mar 2023 21:24:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1paSQx-0005bb-GY for qemu-devel@nongnu.org; Thu, 09 Mar 2023 21:24:40 -0500 Received: from mail-pj1-x102f.google.com ([2607:f8b0:4864:20::102f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1paSQv-0001vN-Ag for qemu-devel@nongnu.org; Thu, 09 Mar 2023 21:24:39 -0500 Received: by mail-pj1-x102f.google.com with SMTP id m8-20020a17090a4d8800b002377bced051so8375355pjh.0 for ; Thu, 09 Mar 2023 18:24:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1678415075; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zjDnw/VppzDNDqJf2mtBwD3QCqXkpc97TNAYmWYJprM=; b=PYxcpVumTUwo5jx7mrXelLqmvp8iKhjDnDFwxtOL27GVrgUMbQXpPMfIkInDNR/XdK o/u0hVQ7tVyXh6WgIX86QOgX3aZhbCOD0rRftdgzfAsgPgDInrSZtvw7GVyPhSO5PTkW CcrPX2sgRwYG5vyORXFNSTbq8BbvpAHCQNngNn/buN/ojQ0XEknT8Wdu0hVyqPQzuYyy 8dpjwd1ZuucS1wyBu1RZZ9yyYScTKf/8UH3BC/NBESEnezXdQG0d3lE9ItP59r0/AJ8A V/MytoY2W/dXh7O5HcD33z0xLMvcDRC4AD7vUzWq7qEF0LvdXCKAIGNkGvNrVUvpkT3g XmCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678415075; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zjDnw/VppzDNDqJf2mtBwD3QCqXkpc97TNAYmWYJprM=; b=ywFMIBYRFmeQAONADTmvDMZ6Ax1G5qs0c4XNv+e50mseLyXN5+sw4xdUcmaPBys4vn jeXPUR6PA5gw2DHrvaNB3WLktlQzWGh0rxi1SwvoxCZZeMsdZlP3CelYk+nKWwXP5DFD a45pgWMOrw53NRkFEnpq7k/3dtwbNHdwzRk3en+kgs4Y2kkCf0/82aiZcOIFMcw3L6Ac 1Az0mDRI5dUFGqoRiKZzfSDRKX74EjmYZpQxKxPUpyC2eYs2jxlY8n48dPz4jAOjmgYv HMrnKHLUb8qz/dh0DYvFoAM1BQVZg2KsMjPDHEcyHgzqw8cGkmu1AAQDTFOsfVkf6Igr LPGA== X-Gm-Message-State: AO0yUKV3FupKgE10GHn8kKCU/RDfu6sq4yjnUhsFCV6ERhTL/KVgCHNy /YI6JpG/nbjOIYoQfRHPAzXZQZNyt4vmPrDZzSw= X-Google-Smtp-Source: AK7set9UbXTS68jHl1h9iNtxOt9CkfnyyD9Zm0FvVfPkn/SBLkAug8qwC+ckRXw8vHUDnV9AHDzGMw== X-Received: by 2002:a05:6a20:8403:b0:cc:76ec:70dc with SMTP id c3-20020a056a20840300b000cc76ec70dcmr30397749pzd.26.1678415075361; Thu, 09 Mar 2023 18:24:35 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.28]) by smtp.gmail.com with ESMTPSA id g7-20020aa78187000000b00571f66721aesm246081pfi.42.2023.03.09.18.24.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Mar 2023 18:24:34 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com Subject: [PATCH v7 1/6] memory: Reference as->current_map directly in memory commit Date: Fri, 10 Mar 2023 10:24:20 +0800 Message-Id: <20230310022425.2992472-2-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230310022425.2992472-1-xuchuangxclwt@bytedance.com> References: <20230310022425.2992472-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102f; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pj1-x102f.google.com X-Spam_score_int: 4 X-Spam_score: 0.4 X-Spam_bar: / X-Spam_report: (0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Xu Calling RCU variance of address_space_get|to_flatview() during memory commit (flatview updates, triggering memory listeners, or updating ioeventfds, etc.) is not 100% accurate, because commit() requires BQL rather than RCU read lock, so the context exclusively owns current_map and can be directly referenced. Neither does it need a refcount to current_map because it cannot be freed from under the caller. Add address_space_get_flatview_raw() for the case where the context holds BQL rather than RCU read lock and use it across the core memory updates, Drop the extra refcounts on FlatView*. Signed-off-by: Peter Xu --- softmmu/memory.c | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/softmmu/memory.c b/softmmu/memory.c index 4699ba55ec..a992a365d9 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -61,6 +61,13 @@ struct AddrRange { Int128 size; }; +/* Called with BQL held */ +static inline FlatView *address_space_to_flatview_raw(AddressSpace *as) +{ + assert(qemu_mutex_iothread_locked()); + return as->current_map; +} + static AddrRange addrrange_make(Int128 start, Int128 size) { return (AddrRange) { start, size }; @@ -155,7 +162,7 @@ enum ListenerDirection { Forward, Reverse }; #define MEMORY_LISTENER_UPDATE_REGION(fr, as, dir, callback, _args...) \ do { \ MemoryRegionSection mrs = section_from_flat_range(fr, \ - address_space_to_flatview(as)); \ + address_space_to_flatview_raw(as)); \ MEMORY_LISTENER_CALL(as, callback, dir, &mrs, ##_args); \ } while(0) @@ -753,6 +760,7 @@ static FlatView *generate_memory_topology(MemoryRegion *mr) } static void address_space_add_del_ioeventfds(AddressSpace *as, + FlatView *view, MemoryRegionIoeventfd *fds_new, unsigned fds_new_nb, MemoryRegionIoeventfd *fds_old, @@ -774,7 +782,7 @@ static void address_space_add_del_ioeventfds(AddressSpace *as, &fds_new[inew]))) { fd = &fds_old[iold]; section = (MemoryRegionSection) { - .fv = address_space_to_flatview(as), + .fv = view, .offset_within_address_space = int128_get64(fd->addr.start), .size = fd->addr.size, }; @@ -787,7 +795,7 @@ static void address_space_add_del_ioeventfds(AddressSpace *as, &fds_old[iold]))) { fd = &fds_new[inew]; section = (MemoryRegionSection) { - .fv = address_space_to_flatview(as), + .fv = view, .offset_within_address_space = int128_get64(fd->addr.start), .size = fd->addr.size, }; @@ -833,7 +841,7 @@ static void address_space_update_ioeventfds(AddressSpace *as) ioeventfd_max = QEMU_ALIGN_UP(as->ioeventfd_nb, 4); ioeventfds = g_new(MemoryRegionIoeventfd, ioeventfd_max); - view = address_space_get_flatview(as); + view = address_space_to_flatview_raw(as); FOR_EACH_FLAT_RANGE(fr, view) { for (i = 0; i < fr->mr->ioeventfd_nb; ++i) { tmp = addrrange_shift(fr->mr->ioeventfds[i].addr, @@ -852,13 +860,12 @@ static void address_space_update_ioeventfds(AddressSpace *as) } } - address_space_add_del_ioeventfds(as, ioeventfds, ioeventfd_nb, + address_space_add_del_ioeventfds(as, view, ioeventfds, ioeventfd_nb, as->ioeventfds, as->ioeventfd_nb); g_free(as->ioeventfds); as->ioeventfds = ioeventfds; as->ioeventfd_nb = ioeventfd_nb; - flatview_unref(view); } /* @@ -1026,7 +1033,7 @@ static void flatviews_reset(void) static void address_space_set_flatview(AddressSpace *as) { - FlatView *old_view = address_space_to_flatview(as); + FlatView *old_view = address_space_to_flatview_raw(as); MemoryRegion *physmr = memory_region_get_flatview_root(as->root); FlatView *new_view = g_hash_table_lookup(flat_views, physmr); @@ -2979,8 +2986,7 @@ static void listener_add_address_space(MemoryListener *listener, listener->log_global_start(listener); } } - - view = address_space_get_flatview(as); + view = address_space_to_flatview_raw(as); FOR_EACH_FLAT_RANGE(fr, view) { MemoryRegionSection section = section_from_flat_range(fr, view); @@ -2994,7 +3000,6 @@ static void listener_add_address_space(MemoryListener *listener, if (listener->commit) { listener->commit(listener); } - flatview_unref(view); } static void listener_del_address_space(MemoryListener *listener, @@ -3006,7 +3011,7 @@ static void listener_del_address_space(MemoryListener *listener, if (listener->begin) { listener->begin(listener); } - view = address_space_get_flatview(as); + view = address_space_to_flatview_raw(as); FOR_EACH_FLAT_RANGE(fr, view) { MemoryRegionSection section = section_from_flat_range(fr, view); @@ -3020,7 +3025,6 @@ static void listener_del_address_space(MemoryListener *listener, if (listener->commit) { listener->commit(listener); } - flatview_unref(view); } void memory_listener_register(MemoryListener *listener, AddressSpace *as) From patchwork Fri Mar 10 02:24:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13168635 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 58036C64EC4 for ; Fri, 10 Mar 2023 02:25:22 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1paSR1-0005cZ-SS; Thu, 09 Mar 2023 21:24:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1paSQy-0005bz-RT for qemu-devel@nongnu.org; Thu, 09 Mar 2023 21:24:40 -0500 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1paSQx-0001vc-DA for qemu-devel@nongnu.org; Thu, 09 Mar 2023 21:24:40 -0500 Received: by mail-pf1-x434.google.com with SMTP id cp12so2773372pfb.5 for ; Thu, 09 Mar 2023 18:24:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1678415078; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=me/gqRlFyHbSVeRcW2F2ZB6r0FGs3WMizMuVDN3fKSU=; b=FXtdwo4OWKftpuNQlnSmtUIdmUmDRZRUKXJGn9bUxdjGZ7bFdh/uniNaIQByOcgl0A 7hgUDBVIwmVKSN1hi8sAP8lWSlaEr3nJgzu7RDE9r0N7f3RgFlxVJ3JzhfGgzW8ME16g ZQ80BukSC0GZfmI6pMweoltO2rcI9ATHjpZXO3vj6Vcg1KasofOKImkDnscgafy3vr1U b1UffddVpXQqx9Pn01HwAvKWSqZv+h27q0P8mX2Vla4CCSrY6M1k0nrKGhG0L405GYfD e9UOe1gwdvjNx7n+cr/gbZtjbLa8nndME8g8V9C2zzYWp7nwUqBT0ajUqZsOa36z4cEs s6eA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678415078; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=me/gqRlFyHbSVeRcW2F2ZB6r0FGs3WMizMuVDN3fKSU=; b=zP2UIaU5uwKN65pefS3F9s/l/JAO2oxvGnNiyEKq90q+Re9bYamdRFzQq18/DNaHZ2 bDDt+WhlHtBT2dyVBkSUs6f+RsGcqvOhUphSSwq+CnyBXNYv4la7n9RLGpg4pks1CvKG uCXqza/liVZe+VK1TBwTVNSHrMPCaBYUG8yfbWLHNqhDTYSRXZExtZ/3mPmYtkcBhSRq bER7MfmV741jX0ZaSGC4vyJRN+6eiIJAiqOuYnJUhbTB83zm9T1O33/hWpgF3FdaOge7 pTHG/atNM2bSsu/LImiAlM/MZvESenY2PylKiEBlXZR2G/LBPvhWTI9l8YZemXfLF0V8 Is0g== X-Gm-Message-State: AO0yUKVF4EJlU/npUZZ23ne3Q2Imemte+bY47pymy2lI41+dJX2J1JHQ uVqSo/PWT2YwYccvKWeyr2/hhV0ImUYTpZ0W9Yg= X-Google-Smtp-Source: AK7set9AYRRdBV0S9Ifvl1GRW57HgbMK2KBQDCQdd5Swugo7ftzZw9gvHaCSgf1YLCme8R3BuAzUCQ== X-Received: by 2002:a62:1bd2:0:b0:5a9:b6f4:778a with SMTP id b201-20020a621bd2000000b005a9b6f4778amr21547284pfb.24.1678415077969; Thu, 09 Mar 2023 18:24:37 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.28]) by smtp.gmail.com with ESMTPSA id g7-20020aa78187000000b00571f66721aesm246081pfi.42.2023.03.09.18.24.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Mar 2023 18:24:37 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [PATCH v7 2/6] rcu: Introduce rcu_read_is_locked() Date: Fri, 10 Mar 2023 10:24:21 +0800 Message-Id: <20230310022425.2992472-3-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230310022425.2992472-1-xuchuangxclwt@bytedance.com> References: <20230310022425.2992472-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::434; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pf1-x434.google.com X-Spam_score_int: 4 X-Spam_score: 0.4 X-Spam_bar: / X-Spam_report: (0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add rcu_read_is_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu Reviewed-by: Peter Xu --- include/qemu/rcu.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index 313fc414bc..7bf45602e1 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h @@ -115,6 +115,13 @@ static inline void rcu_read_unlock(void) } } +static inline bool rcu_read_is_locked(void) +{ + struct rcu_reader_data *p_rcu_reader = get_ptr_rcu_reader(); + + return p_rcu_reader->depth > 0; +} + extern void synchronize_rcu(void); /* From patchwork Fri Mar 10 02:24:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13168638 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 93436C64EC4 for ; Fri, 10 Mar 2023 02:26:08 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1paSR2-0005ch-FS; Thu, 09 Mar 2023 21:24:44 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1paSR1-0005cV-Jc for qemu-devel@nongnu.org; Thu, 09 Mar 2023 21:24:43 -0500 Received: from mail-pg1-x52f.google.com ([2607:f8b0:4864:20::52f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1paSR0-0001wF-0W for qemu-devel@nongnu.org; Thu, 09 Mar 2023 21:24:43 -0500 Received: by mail-pg1-x52f.google.com with SMTP id s17so2266687pgv.4 for ; Thu, 09 Mar 2023 18:24:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1678415080; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GG2aqHgGCiNg5mZ5qJ5G1bt8sGQLaCInJg5o3vlkQmc=; b=GYkSFtI3dAfTikw1U/qOvXdvNQnPugbe3wMoUTZvBt40L0U3VkmOrmqm6BF41mHBjJ g8bn4PdgQ4rvxiJd6O3K4HV14jASSoTPxLCv2dwbcXZMxD5ek9wqh4mHf51NHJAcVuHy RZeNWvRNBc7nlUo0zfTTaRrqWzi6QZ1Gle9Z5dgp6ZPNurTnVNq0tuFiKKUPwGF24tI9 lxp5k9jVzyKFDgtQgc7WNA9ksbhO+VfzPq1RxaeblJP7JoromCfSKoIC4FWeSt6rd3KL t1/CxUvnF/DA/X6w97VgnFOYSlQ+SLcBVifZthEeWI/w3hzfa2Ttq8YoBn/AnepvRCN5 DDrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678415080; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GG2aqHgGCiNg5mZ5qJ5G1bt8sGQLaCInJg5o3vlkQmc=; b=hXJkT4ZcGaX3tS+YvNsf8pN5WFbVWnu2LpuHfKSvmK1x+LQL4ElcAqqaq89dsdLTDy gBT2gfggqZLYRFpT2LA5VGKjNu67YAOL53uSAr9KywCDt5bRFROS2H6i53A9HyCme1N8 eEDsZNjnhaFd2E6kghzGv+nBL5O/ELGagd5ke0CR6Y9g3XNxIAdB5QaFSWlV+S2g630o bhRiuDfqzpS73N2aDa0/khn54XvXBWGU3ohK8gzGCAQt2JjRiECPhePUIri6CwLTelXT 2UVDhaqbTFvhbql+r3PtrocL6FFHsQjIEPdOdK6/v2OCnB04ToiizJuxyNAC6XjrX82L SB7Q== X-Gm-Message-State: AO0yUKW/4gQs+C01Z/DBt0waO6LCEWMBiADGiefWMNHpxS/vHfq/MtNw ilmLysAOEGxUwD2MnKJwzI3FrCghZ/zpfBzi8aE= X-Google-Smtp-Source: AK7set8LqLdX79l/eu5fEi8C9WP65pcU5KzykTMQKlNPWgDsgvRaWkun7YUMYsXXrWpTMg1SiJI52Q== X-Received: by 2002:a62:1c48:0:b0:60e:d286:56d with SMTP id c69-20020a621c48000000b0060ed286056dmr19365644pfc.1.1678415080471; Thu, 09 Mar 2023 18:24:40 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.28]) by smtp.gmail.com with ESMTPSA id g7-20020aa78187000000b00571f66721aesm246081pfi.42.2023.03.09.18.24.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Mar 2023 18:24:40 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [PATCH v7 3/6] memory: Introduce memory_region_transaction_do_commit() Date: Fri, 10 Mar 2023 10:24:22 +0800 Message-Id: <20230310022425.2992472-4-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230310022425.2992472-1-xuchuangxclwt@bytedance.com> References: <20230310022425.2992472-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52f; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pg1-x52f.google.com X-Spam_score_int: 4 X-Spam_score: 0.4 X-Spam_bar: / X-Spam_report: (0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Split memory_region_transaction_do_commit() from memory_region_transaction_commit(). We'll call do_commit() in address_space_to_flatview() in the later patch. Signed-off-by: Chuang Xu Reviewed-by: Peter Xu --- softmmu/memory.c | 47 +++++++++++++++++++++++++++-------------------- 1 file changed, 27 insertions(+), 20 deletions(-) diff --git a/softmmu/memory.c b/softmmu/memory.c index a992a365d9..33ecc62ee9 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -1093,34 +1093,41 @@ void memory_region_transaction_begin(void) ++memory_region_transaction_depth; } -void memory_region_transaction_commit(void) +void memory_region_transaction_do_commit(void) { AddressSpace *as; - assert(memory_region_transaction_depth); assert(qemu_mutex_iothread_locked()); - --memory_region_transaction_depth; - if (!memory_region_transaction_depth) { - if (memory_region_update_pending) { - flatviews_reset(); + if (memory_region_update_pending) { + flatviews_reset(); - MEMORY_LISTENER_CALL_GLOBAL(begin, Forward); + MEMORY_LISTENER_CALL_GLOBAL(begin, Forward); - QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) { - address_space_set_flatview(as); - address_space_update_ioeventfds(as); - } - memory_region_update_pending = false; - ioeventfd_update_pending = false; - MEMORY_LISTENER_CALL_GLOBAL(commit, Forward); - } else if (ioeventfd_update_pending) { - QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) { - address_space_update_ioeventfds(as); - } - ioeventfd_update_pending = false; + QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) { + address_space_set_flatview(as); + address_space_update_ioeventfds(as); + } + memory_region_update_pending = false; + ioeventfd_update_pending = false; + MEMORY_LISTENER_CALL_GLOBAL(commit, Forward); + } else if (ioeventfd_update_pending) { + QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) { + address_space_update_ioeventfds(as); } - } + ioeventfd_update_pending = false; + } +} + +void memory_region_transaction_commit(void) +{ + assert(memory_region_transaction_depth); + assert(qemu_mutex_iothread_locked()); + + --memory_region_transaction_depth; + if (!memory_region_transaction_depth) { + memory_region_transaction_do_commit(); + } } static void memory_region_destructor_none(MemoryRegion *mr) From patchwork Fri Mar 10 02:24:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13168637 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E63A3C61DA4 for ; Fri, 10 Mar 2023 02:25:58 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1paSR6-0005dn-Jv; Thu, 09 Mar 2023 21:24:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1paSR5-0005dc-DF for qemu-devel@nongnu.org; Thu, 09 Mar 2023 21:24:47 -0500 Received: from mail-pj1-x1036.google.com ([2607:f8b0:4864:20::1036]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1paSR3-0001ws-R2 for qemu-devel@nongnu.org; Thu, 09 Mar 2023 21:24:47 -0500 Received: by mail-pj1-x1036.google.com with SMTP id 6-20020a17090a190600b00237c5b6ecd7so8320655pjg.4 for ; Thu, 09 Mar 2023 18:24:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1678415083; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KGcBg4/xLOZLgwxdzdMMAnYKMA5aafW6ecNJRQ6F0x4=; b=Pl0W6z+NOBy2rjKTRAZm6FMOHkYLGTTm7EWHWEind5IgzHNnDQrPOmdlYoG4jPYtg/ yikQmxiLVuPSZgh3+3m/UfUEcpL42MblDDziZeM6FhT54KjfDZyOssaABNsYkAaQYNJL gdYw1+uCC4JM2sbvTLL8oIydrgbu7s38LusPYXzO8LlL+Eqc3QsI+AAzEHlpfo4KhZN9 szbZqXZHgUK60KsmF4Bi0a5rgkd8799oM7HmAHfRE7fS5nvdMLmAqx2oCjiUscG4cbc1 VJAr4uHpsEanAtf+FRlOXntl4mmuEUBbQPHEqpbJy0cGqBMHqk/RX6pL3ISwJPNapu+N 19fA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678415083; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KGcBg4/xLOZLgwxdzdMMAnYKMA5aafW6ecNJRQ6F0x4=; b=mWrPpxesGd+rXRSsAK7wbo0j9/lCC7WNkuKzTO/LdeEIMXcJAhs/hf/Lm+Pi14qsn9 nFGSVyrM15eNA3g5bcbga3oJ5PuIJZ8XnarLhcCwnmi6JXPyQJJ6lvtgIH5C4QU2/Hhp JXbsGHJW6FlUr9u68yH1HY4uvL/8Mmdq21aKiS7OvamUTPROgc2OJM/1sPBcoezqITQV GMaZg9cEqL3T5PvzpEcW1BbmRhXnLBCnqLbkWJ/JbnMOAchTSorW6xecbugYf0YD5roQ AY72ZEQBHPQqVkmA8EuuxvWCvkPNrZssYTtzC1cxnxspxhIO39fccV7/7q+84ZZ2oNtm xvhQ== X-Gm-Message-State: AO0yUKWCOTsEwTJXABALlxsA2xazl6H3x+74QX7VUIWAYSx9coQefvnf bsEbCsuQBJZPRx8ZZ97zGkFVxZfcZJioO8GXdVQ= X-Google-Smtp-Source: AK7set8JWDB6SErjJoYzl2QNY8uGEb8TO9UUMnHbJ/M4+wV22+W7KpskRSd/wwyeSuPdfkIz0V7p+A== X-Received: by 2002:a05:6a20:6982:b0:cd:ed5c:513 with SMTP id t2-20020a056a20698200b000cded5c0513mr1032392pzk.31.1678415083025; Thu, 09 Mar 2023 18:24:43 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.28]) by smtp.gmail.com with ESMTPSA id g7-20020aa78187000000b00571f66721aesm246081pfi.42.2023.03.09.18.24.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Mar 2023 18:24:42 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [PATCH v7 4/6] memory: Add sanity check in address_space_to_flatview Date: Fri, 10 Mar 2023 10:24:23 +0800 Message-Id: <20230310022425.2992472-5-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230310022425.2992472-1-xuchuangxclwt@bytedance.com> References: <20230310022425.2992472-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1036; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pj1-x1036.google.com X-Spam_score_int: 4 X-Spam_score: 0.4 X-Spam_bar: / X-Spam_report: (0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Before using any flatview, sanity check whether BQL or rcu is held. And if we're during a memory region transaction, try to immediately update mappings, or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 23 +++++++++++++++++++++++ softmmu/memory.c | 5 +++++ 2 files changed, 28 insertions(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index 6fa0b071f0..d6fd89db64 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -27,6 +27,7 @@ #include "qemu/notify.h" #include "qom/object.h" #include "qemu/rcu.h" +#include "qemu/main-loop.h" #define RAM_ADDR_INVALID (~(ram_addr_t)0) @@ -1095,8 +1096,30 @@ struct FlatView { MemoryRegion *root; }; +bool memory_region_transaction_in_progress(void); + +void memory_region_transaction_do_commit(void); + static inline FlatView *address_space_to_flatview(AddressSpace *as) { + if (qemu_mutex_iothread_locked()) { + /* We exclusively own the flatview now.. */ + if (memory_region_transaction_in_progress()) { + /* + * Fetch the flatview within a transaction in-progress, it + * means current_map may not be the latest, we need to update + * immediately to make sure the caller won't see obsolete + * mapping. + */ + memory_region_transaction_do_commit(); + } + + /* No further protection needed to access current_map */ + return as->current_map; + } + + /* Otherwise we must have had the RCU lock or something went wrong */ + assert(rcu_read_is_locked()); return qatomic_rcu_read(&as->current_map); } diff --git a/softmmu/memory.c b/softmmu/memory.c index 33ecc62ee9..6a8e8b4e71 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -1130,6 +1130,11 @@ void memory_region_transaction_commit(void) } } +bool memory_region_transaction_in_progress(void) +{ + return memory_region_transaction_depth != 0; +} + static void memory_region_destructor_none(MemoryRegion *mr) { } From patchwork Fri Mar 10 02:24:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13168640 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9A47EC61DA4 for ; Fri, 10 Mar 2023 02:26:15 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1paSRN-0005eM-UT; Thu, 09 Mar 2023 21:25:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1paSR6-0005dp-V2 for qemu-devel@nongnu.org; Thu, 09 Mar 2023 21:24:48 -0500 Received: from mail-pj1-x1034.google.com ([2607:f8b0:4864:20::1034]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1paSR5-0001xN-9v for qemu-devel@nongnu.org; Thu, 09 Mar 2023 21:24:48 -0500 Received: by mail-pj1-x1034.google.com with SMTP id ce8-20020a17090aff0800b0023a61cff2c6so7210061pjb.0 for ; Thu, 09 Mar 2023 18:24:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1678415086; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9wK7Kao4Ay8cM21jN0qeB4dy3IR6Pt8A2UW1ABAkymk=; b=RSSqqrdkAhMj8GRW/4evaDHHWulxsjHj9OkLePcvcjjCOhe+EOoQ5yilgRqeLh5rhF Ym4fq5znyJajPNGnMTDPcryaBIW0Bdb3ykQGK+dkbhRiSVgO6R+qnaeSS9ytASY/pLRc 8HVCvA1jBo8biWYL1WCOIgNe4z0nc/EOEDqSo0Jhu0mE74GZBqPKIMhTyQY2heDk00MI u4wuEfLUzHkEtFm+X9afPUH5ep/926Rz1PpNQ6GN0fR7tgieJhG95RCf2xt57d+F3fy7 nrq3wy2+UYr8OsF7JbThKe69IoPPvwQ9qU9iQgAfaGIxrgACV2V1589zDRsjXjpD1Grg ZLDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678415086; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9wK7Kao4Ay8cM21jN0qeB4dy3IR6Pt8A2UW1ABAkymk=; b=KvLK0Up67xNx7WJiZvk0CcThGb1rGrodobdhPF/sKGh+QxgRlaOdTcbqasNnjBg0d2 AwrhBEcRcOCcngqzwvyphQILublXjw+MNhxqiE+mRg+NshWvnmw2ntHoCmvtn6tgsyBv Q8POQPiCgE/4qajYz+WiKdlifjY1EZhrIsiuDJTW5xO8aVJJR9D36gYrrZGAogtyLt96 S2A8VrAo0UGqYDNJYjweavyKeQR+l3+lNFHw+Vaq0fqQn23h46QGhbHs38L1eBMb3B7z u6syTXJdkZVSPlbAqqv3rXVgRFBN2MaoDMfzYCCncNV+LWfR/sdbwqnZd0dPVDr6nowa qU0w== X-Gm-Message-State: AO0yUKVDAjdfUhEl2JFVZTOXPu+DybrBWdGD+sHz8xXE0sT35UXUyc96 bRl1pRpme0BKeoJA7w7bUIEyhkN0/HMB5xqywt8= X-Google-Smtp-Source: AK7set94HUnUqNaVTMoWhtiZyScsxILPYf4FtqTnUCOp+HmMQBVC/NhjlxqRNqiXC23C+Mkd+P83dw== X-Received: by 2002:a05:6a20:5489:b0:cc:120c:b259 with SMTP id i9-20020a056a20548900b000cc120cb259mr30209129pzk.39.1678415085774; Thu, 09 Mar 2023 18:24:45 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.28]) by smtp.gmail.com with ESMTPSA id g7-20020aa78187000000b00571f66721aesm246081pfi.42.2023.03.09.18.24.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Mar 2023 18:24:45 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [PATCH v7 5/6] migration: Reduce time of loading non-iterable vmstate Date: Fri, 10 Mar 2023 10:24:24 +0800 Message-Id: <20230310022425.2992472-6-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230310022425.2992472-1-xuchuangxclwt@bytedance.com> References: <20230310022425.2992472-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1034; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pj1-x1034.google.com X-Spam_score_int: 4 X-Spam_score: 0.4 X-Spam_bar: / X-Spam_report: (0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop and ending with the timestamp of target qemu start). Most of the time is spent committing memory region changes repeatedly. This patch packs all the changes to memory region during the period of loading non-iterable vmstate in a single memory transaction. With the increase of devices, this patch will greatly improve the performance. Note that the following test results are based on the application of the next patch. Without the next patch, the improvement will be reduced. Here are the test1 results: test info: - Host - Intel(R) Xeon(R) Platinum 8362 CPU - Mellanox Technologies MT28841 - VM - 32 CPUs 128GB RAM VM - 8 16-queue vhost-net device - 16 4-queue vhost-user-blk device. time of loading non-iterable vmstate downtime before about 112 ms 285 ms after about 20 ms 194 ms In test2, we keep the number of the device the same as test1, reduce the number of queues per device: Here are the test2 results: test info: - Host - Intel(R) Xeon(R) Platinum 8362 CPU - Mellanox Technologies MT28841 - VM - 32 CPUs 128GB RAM VM - 8 1-queue vhost-net device - 16 1-queue vhost-user-blk device. time of loading non-iterable vmstate downtime before about 65 ms about 151 ms after about 19 ms about 100 ms In test3, we keep the number of queues per device the same as test1, reduce the number of devices: Here are the test3 results: test info: - Host - Intel(R) Xeon(R) Platinum 8362 CPU - Mellanox Technologies MT28841 - VM - 32 CPUs 128GB RAM VM - 1 16-queue vhost-net device - 1 4-queue vhost-user-blk device. time of loading non-iterable vmstate downtime before about 24 ms about 51 ms after about 9 ms about 36 ms As we can see from the test results above, both the number of queues and the number of devices have a great impact on the time of loading non-iterable vmstate. The growth of the number of devices and queues will lead to more mr commits, and the time consumption caused by the flatview reconstruction will also increase. Signed-off-by: Chuang Xu Reviewed-by: Peter Xu Reviewed-by: Juan Quintela --- migration/savevm.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index aa54a67fda..9a7d3e40d6 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2762,6 +2762,7 @@ out: goto retry; } } + return ret; } @@ -2787,7 +2788,25 @@ int qemu_loadvm_state(QEMUFile *f) cpu_synchronize_all_pre_loadvm(); + /* + * Call memory_region_transaction_begin() before loading vmstate. + * This call is paired with memory_region_transaction_commit() at + * the end of qemu_loadvm_state_main(), in order to pack all the + * changes to memory region during the period of loading + * non-iterable vmstate in a single memory transaction. + * This operation will reduce time of loading non-iterable vmstate + */ + memory_region_transaction_begin(); + ret = qemu_loadvm_state_main(f, mis); + + /* + * Call memory_region_transaction_commit() after loading vmstate. + * At this point, qemu actually completes all the previous memory + * region transactions. + */ + memory_region_transaction_commit(); + qemu_event_set(&mis->main_thread_load_event); trace_qemu_loadvm_state_post_main(ret); From patchwork Fri Mar 10 02:24:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13168639 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A12D8C6FD1F for ; Fri, 10 Mar 2023 02:26:08 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1paSRU-0005fO-Jt; Thu, 09 Mar 2023 21:25:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1paSRC-0005eN-4b for qemu-devel@nongnu.org; Thu, 09 Mar 2023 21:24:59 -0500 Received: from mail-pj1-x102c.google.com ([2607:f8b0:4864:20::102c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1paSR7-0001y0-SK for qemu-devel@nongnu.org; Thu, 09 Mar 2023 21:24:51 -0500 Received: by mail-pj1-x102c.google.com with SMTP id l1so3997798pjt.2 for ; Thu, 09 Mar 2023 18:24:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1678415088; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LnbAFlawzzcdAvvBRPDdzj3j81OcWEFdiP4smmXMd+s=; b=DieprWgRgRew+dXyufCsVgj0d50LlijffMFjhqZKc+vBW5HH4BMYPACjkuvAEJmuYZ f4/99bgSq3Vxn//Ay2n87IxglJffmGBRO3ISv44Z96ObO1Y/zgSPdSPuR/LkGsIBzjqr G/9MWbNF7llg9COoWpPnOs2rFkCd7UV1GxusnDDfocl2TEib6b2CqiYEYnBpqbWduFeY 92R4rTYKYyAljtDXLvZGh4creWQPm0TB0QHWvANtEJmGtqdmKRTA7gT9iJM62kym0lx1 Vuzka+uFxlj0YEk0ukY5uprZqDzLLv4pipqPkMOd53Xw71oF5juYJBfb1/fdok07unke 3qew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678415088; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LnbAFlawzzcdAvvBRPDdzj3j81OcWEFdiP4smmXMd+s=; b=j/7AeAj7cx+RXXQD8+8A75hASxXMIJ2RtTc7Zc0TsXh3Yy7br6b8leT9rT9yqdYrqZ 0r2XrpKEJ2UqOiBTbSfP39p5zx3FMvp42fKjkhdMoBDB58Lxc+fgm8r0f65RIiyUk0i7 YP8IVCc4kVGRK8TG1kmsqI4r7T5HFcEXaPpWErCJlniAocW1K/x3OSxxz4vry6Cnrir9 r4Guxwv9hKdq0v2gpsdBwiA9olwrhK6mae0KuEiNMPVq59QQG0wIQSbm7qAZpfIrFx8R 9Earw6hV92AADD3NWpeqRQxiQYpt6codKve7f1bbArILMoG3ohmkgYNYJz9GXmhKvGWG Evcg== X-Gm-Message-State: AO0yUKXzEME6VR+gAHHIYg0rwoUimH6faYvfTL7CWzitRmuvoVIRwVQJ HyTwnWUMjysVhFAm7dXL2aVRsTdVWWQU9XmmwGs= X-Google-Smtp-Source: AK7set9HMN7cMugKW2H0r49hoKs1J7aGA/7PPyDgIX2Jcx0vqC8gcJPFSXgUXV3FM6DWiWIeMfXt4w== X-Received: by 2002:a05:6a20:8b23:b0:cc:5be9:fe63 with SMTP id l35-20020a056a208b2300b000cc5be9fe63mr19877271pzh.8.1678415088476; Thu, 09 Mar 2023 18:24:48 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.28]) by smtp.gmail.com with ESMTPSA id g7-20020aa78187000000b00571f66721aesm246081pfi.42.2023.03.09.18.24.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Mar 2023 18:24:48 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [PATCH v7 6/6] memory: Introduce address_space_to_flatview_rcu() Date: Fri, 10 Mar 2023 10:24:25 +0800 Message-Id: <20230310022425.2992472-7-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230310022425.2992472-1-xuchuangxclwt@bytedance.com> References: <20230310022425.2992472-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102c; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pj1-x102c.google.com X-Spam_score_int: 4 X-Spam_score: 0.4 X-Spam_bar: / X-Spam_report: (0.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.999, HK_RANDOM_FROM=1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org In last patch, we wrap vm_load with begin/commit, here we introduce address_space_to_flatview_rcu() to avoid unnecessary enforce commit during vm_load. Signed-off-by: Chuang Xu --- include/exec/memory-internal.h | 2 +- include/exec/memory.h | 20 ++++++++++++++++++++ softmmu/memory.c | 2 +- 3 files changed, 22 insertions(+), 2 deletions(-) diff --git a/include/exec/memory-internal.h b/include/exec/memory-internal.h index 100c1237ac..1432240449 100644 --- a/include/exec/memory-internal.h +++ b/include/exec/memory-internal.h @@ -30,7 +30,7 @@ static inline AddressSpaceDispatch *flatview_to_dispatch(FlatView *fv) static inline AddressSpaceDispatch *address_space_to_dispatch(AddressSpace *as) { - return flatview_to_dispatch(address_space_to_flatview(as)); + return flatview_to_dispatch(address_space_to_flatview_rcu(as)); } FlatView *address_space_get_flatview(AddressSpace *as); diff --git a/include/exec/memory.h b/include/exec/memory.h index d6fd89db64..235e3017bc 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -1100,6 +1100,9 @@ bool memory_region_transaction_in_progress(void); void memory_region_transaction_do_commit(void); +/* + * We recommend using this by default. + */ static inline FlatView *address_space_to_flatview(AddressSpace *as) { if (qemu_mutex_iothread_locked()) { @@ -1123,6 +1126,23 @@ static inline FlatView *address_space_to_flatview(AddressSpace *as) return qatomic_rcu_read(&as->current_map); } +/* + * We recommend using address_space_to_flatview() rather than this one. + * Note that if we use this during a memory region transaction, we may + * see obsolete flatviews. We must bear with an obsolete map until commit. + * And outside a memory region transaction, this is basically the same as + * address_space_to_flatview(). + */ +static inline FlatView *address_space_to_flatview_rcu(AddressSpace *as) +{ + /* + * Before using any flatview, sanity check BQL or RCU is held. + */ + assert(qemu_mutex_iothread_locked() || rcu_read_is_locked()); + + return qatomic_rcu_read(&as->current_map); +} + /** * typedef flatview_cb: callback for flatview_for_each_range() * diff --git a/softmmu/memory.c b/softmmu/memory.c index 6a8e8b4e71..33d14e967d 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -815,7 +815,7 @@ FlatView *address_space_get_flatview(AddressSpace *as) RCU_READ_LOCK_GUARD(); do { - view = address_space_to_flatview(as); + view = address_space_to_flatview_rcu(as); /* If somebody has replaced as->current_map concurrently, * flatview_ref returns false. */