From patchwork Fri Mar 3 10:56:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13158653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 49882C7EE33 for ; Fri, 3 Mar 2023 10:57:56 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pY36F-0006rF-Se; Fri, 03 Mar 2023 05:57:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY36B-0006qo-Tt for qemu-devel@nongnu.org; Fri, 03 Mar 2023 05:57:15 -0500 Received: from mail-pj1-x1035.google.com ([2607:f8b0:4864:20::1035]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pY366-00041n-BS for qemu-devel@nongnu.org; Fri, 03 Mar 2023 05:57:12 -0500 Received: by mail-pj1-x1035.google.com with SMTP id me6-20020a17090b17c600b0023816b0c7ceso5754467pjb.2 for ; Fri, 03 Mar 2023 02:57:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1677841027; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=z7tZa1mBhRjtuXfAzw01UoTlbuOJ1Zi3UEwNndiwjX8=; b=gV0t3Oqd6Tq72faktp+MKJUV86o9VcOAlKO7Rx5fGYAw3XIcXweZSwfDSLrWwuZ7Er 0bptEP8mwwplbAqRicXyzihKIio6SpwvKM8EwlVOmt06dMCNVIEA0ssdCLGeu9oWxbE9 Z5aVWmgL47zjsA8qLv1NtDnI2wfXmYwu/Ep5y1g/aifsjZUGdYFEI+ETlfCel4Mey7BM VT82clXU8LWc5EnMPCw3OflDIiEtnYE1P4ulv0RdWN0JZfqfLSKxtxN09tkVegXaoCQG IXfAoOGyyYIvE5WjehY3g7lE1HzkN+pH2wEeInNaNEYQZnQS4bTLeToGsdUr3rdIrPIG 78Wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677841027; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=z7tZa1mBhRjtuXfAzw01UoTlbuOJ1Zi3UEwNndiwjX8=; b=gvcQsV1o7A+vRGc1B1xV/tF7k2J/EaRmObVTyg35luxGj2JxqjMVCYpmS3FveQ5IBu XfXRChdPVm1Xr3QbUJ1DsIPMm8QcbsioLIIOCz36EIJGVzMQXzTSJi4uayFC0pYtZj1M PdlEhDmwetiC21RW5ghvXWZoxcV8vQ/KhWwcYM+eIeyC9rIcoCp/xC9yVZYKD9Q9RErb 62WQPR8uUNfApUvU5bNbofiaZ0bOHgxm0BW0EYgilyTp9idjT5H9kH0KzWefDMrTwOG4 34tjfexZPjVs8iMY1QlphoFt9TZ4NxWLxcpgJPde2PfsVSeGNwMnB8IxBVzb/QuvJaCa 5xBA== X-Gm-Message-State: AO0yUKWX1I1Yl4haaY0GAHV5fIHOelLCGrXa++IVfDW4haawdY92tiE2 +qRFovenptCDR9fpMxcwOYbvlq2Mwic73FXS X-Google-Smtp-Source: AK7set8rfM5s4npK6W7fZDA6UoipD5QmJUtPrUe7XIhTeRG1Lw0pybJKIc/sPrBovP8001cOe/GUjA== X-Received: by 2002:a05:6a20:7b30:b0:cc:109b:6c85 with SMTP id s48-20020a056a207b3000b000cc109b6c85mr1630785pzh.7.1677841027319; Fri, 03 Mar 2023 02:57:07 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.12]) by smtp.gmail.com with ESMTPSA id r15-20020a63a54f000000b004fadb547d0csm1248000pgu.61.2023.03.03.02.57.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Mar 2023 02:57:06 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com Subject: [PATCH RESEND v6 1/5] memory: Reference as->current_map directly in memory commit Date: Fri, 3 Mar 2023 18:56:51 +0800 Message-Id: <20230303105655.396446-2-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230303105655.396446-1-xuchuangxclwt@bytedance.com> References: <20230303105655.396446-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1035; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pj1-x1035.google.com X-Spam_score_int: -5 X-Spam_score: -0.6 X-Spam_bar: / X-Spam_report: (-0.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.001, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Peter Xu Calling RCU variance of address_space_get|to_flatview() during memory commit (flatview updates, triggering memory listeners, or updating ioeventfds, etc.) is not 100% accurate, because commit() requires BQL rather than RCU read lock, so the context exclusively owns current_map and can be directly referenced. Neither does it need a refcount to current_map because it cannot be freed from under the caller. Add address_space_get_flatview_raw() for the case where the context holds BQL rather than RCU read lock and use it across the core memory updates, Drop the extra refcounts on FlatView*. Signed-off-by: Peter Xu --- softmmu/memory.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/softmmu/memory.c b/softmmu/memory.c index 9d64efca26..213496802b 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -61,6 +61,13 @@ struct AddrRange { Int128 size; }; +/* Called with BQL held */ +static inline FlatView *address_space_to_flatview_raw(AddressSpace *as) +{ + assert(qemu_mutex_iothread_locked()); + return as->current_map; +} + static AddrRange addrrange_make(Int128 start, Int128 size) { return (AddrRange) { start, size }; @@ -155,7 +162,7 @@ enum ListenerDirection { Forward, Reverse }; #define MEMORY_LISTENER_UPDATE_REGION(fr, as, dir, callback, _args...) \ do { \ MemoryRegionSection mrs = section_from_flat_range(fr, \ - address_space_to_flatview(as)); \ + address_space_to_flatview_raw(as)); \ MEMORY_LISTENER_CALL(as, callback, dir, &mrs, ##_args); \ } while(0) @@ -753,6 +760,7 @@ static FlatView *generate_memory_topology(MemoryRegion *mr) } static void address_space_add_del_ioeventfds(AddressSpace *as, + FlatView *view, MemoryRegionIoeventfd *fds_new, unsigned fds_new_nb, MemoryRegionIoeventfd *fds_old, @@ -774,7 +782,7 @@ static void address_space_add_del_ioeventfds(AddressSpace *as, &fds_new[inew]))) { fd = &fds_old[iold]; section = (MemoryRegionSection) { - .fv = address_space_to_flatview(as), + .fv = view, .offset_within_address_space = int128_get64(fd->addr.start), .size = fd->addr.size, }; @@ -787,7 +795,7 @@ static void address_space_add_del_ioeventfds(AddressSpace *as, &fds_old[iold]))) { fd = &fds_new[inew]; section = (MemoryRegionSection) { - .fv = address_space_to_flatview(as), + .fv = view, .offset_within_address_space = int128_get64(fd->addr.start), .size = fd->addr.size, }; @@ -833,7 +841,7 @@ static void address_space_update_ioeventfds(AddressSpace *as) ioeventfd_max = QEMU_ALIGN_UP(as->ioeventfd_nb, 4); ioeventfds = g_new(MemoryRegionIoeventfd, ioeventfd_max); - view = address_space_get_flatview(as); + view = address_space_to_flatview_raw(as); FOR_EACH_FLAT_RANGE(fr, view) { for (i = 0; i < fr->mr->ioeventfd_nb; ++i) { tmp = addrrange_shift(fr->mr->ioeventfds[i].addr, @@ -852,13 +860,12 @@ static void address_space_update_ioeventfds(AddressSpace *as) } } - address_space_add_del_ioeventfds(as, ioeventfds, ioeventfd_nb, + address_space_add_del_ioeventfds(as, view, ioeventfds, ioeventfd_nb, as->ioeventfds, as->ioeventfd_nb); g_free(as->ioeventfds); as->ioeventfds = ioeventfds; as->ioeventfd_nb = ioeventfd_nb; - flatview_unref(view); } /* @@ -1026,7 +1033,7 @@ static void flatviews_reset(void) static void address_space_set_flatview(AddressSpace *as) { - FlatView *old_view = address_space_to_flatview(as); + FlatView *old_view = address_space_to_flatview_raw(as); MemoryRegion *physmr = memory_region_get_flatview_root(as->root); FlatView *new_view = g_hash_table_lookup(flat_views, physmr); From patchwork Fri Mar 3 10:56:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13158649 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 35F0CC7EE30 for ; Fri, 3 Mar 2023 10:57:56 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pY36F-0006rM-Sx; Fri, 03 Mar 2023 05:57:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY36B-0006qp-Tp for qemu-devel@nongnu.org; Fri, 03 Mar 2023 05:57:15 -0500 Received: from mail-pj1-x1031.google.com ([2607:f8b0:4864:20::1031]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pY367-00042a-GK for qemu-devel@nongnu.org; Fri, 03 Mar 2023 05:57:12 -0500 Received: by mail-pj1-x1031.google.com with SMTP id bo22so2080346pjb.4 for ; Fri, 03 Mar 2023 02:57:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1677841030; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oybiXPgSNhTbwAYyTDdPbCgWTqNqgxckr+HdWHzShIw=; b=TuiDmPECvvzsufgR2b12xOg5iblTfTLs7/7HfTzqxVsdOWeKbcrk3usCdqXbJCkedB N7sl3TS8KbfjcG7bXXo5fPf3gt4B7yki5IYc3dGWwdeqcNoP0N1qxHuDS4dx061JQMhT EtTgBLJRK3tZi+y1p0BKszcceVwRGOoVPcyIgsplFaBfwUCP6qiO9gadhHzHCD/W7SBz D3QnkoDLqJNHlkvRKHVDptpXIUPhSvj17jSFhihOeLYszrelcjyk4j3sYt3nDCMy4aEb YttkQxfP2lWjPXCeg+LKfGUWlwXoybjyf8DUhXtzTrr6R4ZOs8gJo/0l7ds0+urDwm7k fcUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677841030; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oybiXPgSNhTbwAYyTDdPbCgWTqNqgxckr+HdWHzShIw=; b=wGNUiTRTpUTyNuDM4yhvSJeqo0gma/TnMAukhyqA2iMr9kUJ1LCTnooEOIgYXnK/M/ H420uLfDDSsH5v8zywZZ2V8eWOq6LuILQYXz3O+mDr7uiF8+RAXFmUOw17eQKrQgYAr6 FlXApgjnMZdzpXI5ctBdhofct1nH78wmPWdRU0/UqOi8CXr4GEUmMx4bPXZ6YtBf8Ycb 5YO2dG1z6sM6d/dUeO5UEBWrbngKUEZdqcdiq4pKEkELlANZFMWKrb7aJVGvdZgwdJPk Blt716CXf0zpQQk6TRgH9xm5zQoQaj8zEHhp7sYSbgFC2aQsgbuELdUEE8Nj5ejCPYiZ 61Hg== X-Gm-Message-State: AO0yUKXQ6GWADNbznWr4+GZRRF1bATeewqvTRyfjMB9/Psh4chM6RoAr n4GeG+4SyQtWaQXxAm/uMlr9PjYgCtNpdAW0 X-Google-Smtp-Source: AK7set+vIcjtaGVwkrZotsKW9QFWKa14+/Q22pQG3UIku7JdCN/vSb40vaMzp7CbqJXdcDSCBz4ehQ== X-Received: by 2002:a05:6a20:6d2a:b0:cc:395f:e1ed with SMTP id fv42-20020a056a206d2a00b000cc395fe1edmr1665385pzb.22.1677841030070; Fri, 03 Mar 2023 02:57:10 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.12]) by smtp.gmail.com with ESMTPSA id r15-20020a63a54f000000b004fadb547d0csm1248000pgu.61.2023.03.03.02.57.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Mar 2023 02:57:09 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [PATCH RESEND v6 2/5] rcu: Introduce rcu_read_is_locked() Date: Fri, 3 Mar 2023 18:56:52 +0800 Message-Id: <20230303105655.396446-3-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230303105655.396446-1-xuchuangxclwt@bytedance.com> References: <20230303105655.396446-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1031; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pj1-x1031.google.com X-Spam_score_int: -5 X-Spam_score: -0.6 X-Spam_bar: / X-Spam_report: (-0.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.001, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Add rcu_read_is_locked() to detect holding of rcu lock. Signed-off-by: Chuang Xu --- include/qemu/rcu.h | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h index b063c6fde8..719916d9d3 100644 --- a/include/qemu/rcu.h +++ b/include/qemu/rcu.h @@ -119,6 +119,13 @@ static inline void rcu_read_unlock(void) } } +static inline bool rcu_read_is_locked(void) +{ + struct rcu_reader_data *p_rcu_reader = get_ptr_rcu_reader(); + + return p_rcu_reader->depth > 0; +} + extern void synchronize_rcu(void); /* From patchwork Fri Mar 3 10:56:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13158654 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 24582C64EC4 for ; Fri, 3 Mar 2023 10:58:04 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pY36U-0006sl-7n; Fri, 03 Mar 2023 05:57:34 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY36D-0006r7-9u for qemu-devel@nongnu.org; Fri, 03 Mar 2023 05:57:18 -0500 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pY36B-00042l-MV for qemu-devel@nongnu.org; Fri, 03 Mar 2023 05:57:17 -0500 Received: by mail-pl1-x634.google.com with SMTP id i3so2278269plg.6 for ; Fri, 03 Mar 2023 02:57:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1677841033; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Hz+uaABV0KE9Rq3NUQUeNyj98Vrl/3eewrNbYhH9fiA=; b=kyjHV6by01FycEwGIm3N0qZr2uDUQaaA25gU40cEYA2MzQvBPv55MpPtqGKz3h8+bt bE6PCpyRZlPYT9dlrStjPMiWPXXZWg605WQrtw/KsxAJY8pi48Qv69iKgAMOOw96c0X0 UyGnfGMAqFSHYQkwKKmPFNByBhPR/j8jRzYjo7GQG8MHQ+0XSKQmUXDenKxAWlcXebK+ QtNBV5Z5cPFKVK0+SUjUV/D003v15W2BgmZbpytbHgqAps9reDjE8JqAUIHjjzTrCHsp PZLQhzeZtuf99zdjvUJzeVIqomv74LuNd8G34eRdYWdNAHD1fELCupkG9uPD3RFRCsLj Ahlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677841033; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Hz+uaABV0KE9Rq3NUQUeNyj98Vrl/3eewrNbYhH9fiA=; b=J3q4L1p1lV+hp9NKqXNcbLGIPn2QDjdJhfZzsDMY/1suyCjkAxKKcuwkvM0YHEIB2l WTsxFd390f6rS8N0KTEWOKGQw+4Y7PoDFMps0vFnRAGCITEKd2MFEzimyXymnOnuHkgn YNur/F8YGqlk9qjItGsRfe+teqeDddCsryd5DsgnxwjmrZrFgb7u6fjeH+O9NdK0OxSa pUfclkhPBKu12BA71/tnqDUSqTbIcAAWQ6CsgJm+GtKF6934w59b2zY2EWf0FIUXRHLB Nk3MeG7/mWr0/xl3c16eM+6vvbYDGENyuYkH7J9kst35TZpGsepfRFjEIQHjI0Kl8AGU vnUQ== X-Gm-Message-State: AO0yUKU2aeJjhtt2H9B+rpGxWynlPZ6jYF1AIVHnE/KtTo9njqGflw53 kguIZgX2VbUWq8qKt2zkZqoWlSNtySV8AgUU X-Google-Smtp-Source: AK7set/XkAONYYAw0B65Fpn0KGZWkyIbWmDx7Y9W9oc3dYTy9IU4mXtt29lsAGkLui41Le2FWm3Fkw== X-Received: by 2002:a05:6a20:8353:b0:cc:f057:d3d7 with SMTP id z19-20020a056a20835300b000ccf057d3d7mr1616969pzc.11.1677841032869; Fri, 03 Mar 2023 02:57:12 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.12]) by smtp.gmail.com with ESMTPSA id r15-20020a63a54f000000b004fadb547d0csm1248000pgu.61.2023.03.03.02.57.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Mar 2023 02:57:12 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [PATCH RESEND v6 3/5] memory: Introduce memory_region_transaction_do_commit() Date: Fri, 3 Mar 2023 18:56:53 +0800 Message-Id: <20230303105655.396446-4-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230303105655.396446-1-xuchuangxclwt@bytedance.com> References: <20230303105655.396446-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pl1-x634.google.com X-Spam_score_int: -5 X-Spam_score: -0.6 X-Spam_bar: / X-Spam_report: (-0.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.001, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Split memory_region_transaction_do_commit() from memory_region_transaction_commit(). We'll call do_commit() in address_space_to_flatview() in the later patch. Signed-off-by: Chuang Xu --- softmmu/memory.c | 47 +++++++++++++++++++++++++++-------------------- 1 file changed, 27 insertions(+), 20 deletions(-) diff --git a/softmmu/memory.c b/softmmu/memory.c index 213496802b..b89abf400e 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -1093,34 +1093,41 @@ void memory_region_transaction_begin(void) ++memory_region_transaction_depth; } -void memory_region_transaction_commit(void) +void memory_region_transaction_do_commit(void) { AddressSpace *as; - assert(memory_region_transaction_depth); assert(qemu_mutex_iothread_locked()); - --memory_region_transaction_depth; - if (!memory_region_transaction_depth) { - if (memory_region_update_pending) { - flatviews_reset(); + if (memory_region_update_pending) { + flatviews_reset(); - MEMORY_LISTENER_CALL_GLOBAL(begin, Forward); + MEMORY_LISTENER_CALL_GLOBAL(begin, Forward); - QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) { - address_space_set_flatview(as); - address_space_update_ioeventfds(as); - } - memory_region_update_pending = false; - ioeventfd_update_pending = false; - MEMORY_LISTENER_CALL_GLOBAL(commit, Forward); - } else if (ioeventfd_update_pending) { - QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) { - address_space_update_ioeventfds(as); - } - ioeventfd_update_pending = false; + QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) { + address_space_set_flatview(as); + address_space_update_ioeventfds(as); + } + memory_region_update_pending = false; + ioeventfd_update_pending = false; + MEMORY_LISTENER_CALL_GLOBAL(commit, Forward); + } else if (ioeventfd_update_pending) { + QTAILQ_FOREACH(as, &address_spaces, address_spaces_link) { + address_space_update_ioeventfds(as); } - } + ioeventfd_update_pending = false; + } +} + +void memory_region_transaction_commit(void) +{ + assert(memory_region_transaction_depth); + assert(qemu_mutex_iothread_locked()); + + --memory_region_transaction_depth; + if (!memory_region_transaction_depth) { + memory_region_transaction_do_commit(); + } } static void memory_region_destructor_none(MemoryRegion *mr) From patchwork Fri Mar 3 10:56:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13158650 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 48BC4C7EE39 for ; Fri, 3 Mar 2023 10:57:56 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pY36U-0006tC-Rz; Fri, 03 Mar 2023 05:57:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY36E-0006rL-QM for qemu-devel@nongnu.org; Fri, 03 Mar 2023 05:57:19 -0500 Received: from mail-pj1-x1029.google.com ([2607:f8b0:4864:20::1029]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pY36D-00043D-8s for qemu-devel@nongnu.org; Fri, 03 Mar 2023 05:57:18 -0500 Received: by mail-pj1-x1029.google.com with SMTP id m8-20020a17090a4d8800b002377bced051so5798504pjh.0 for ; Fri, 03 Mar 2023 02:57:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1677841035; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OioqYVW2wWPI0FbDvxDEk2G5WH+pqlZIjQGBZIZ5kBI=; b=YJeOnPgfSF4AwJwFJrcfz0vEn+Ns6CqdMdjO9FcYspmXhgJpDMjgcgwItfmPuEfziN 5nLzNZuxmaeS6ltHkezwsnLvHt0kaaPmIbteZofN+QhqwDRK1H1p1smC9hzML3LY4B6W jO5Exb3EWzJo7NHx8HdTJNu3Au+hW0Wmws5gJrNcskOd3T2m7oIRaI+NE/u1toP52E4f /DxDhYqwlN+Pr0IaesB6vOFHX2AL4J1ZL0QjGv9Wvze7KjNDlvrhl6q6wJ6tBEmy/ADm aTOveAXRvsHtNS2IBNPzhz38iAyNLMERFEhhUFJuvQsOg2xLWphkBz/P1G0T/9atUJWf qGaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677841035; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OioqYVW2wWPI0FbDvxDEk2G5WH+pqlZIjQGBZIZ5kBI=; b=zD6ZUCdiyKR4UDzKsqQ0iClUqu+Lxp/7W41N6djnvJxAhLskvto5cTf5loxNiqlXqC UbeQx3CL7OI4hjRkIMZuE+a2f+CXX3Bcsy8EbbkMynXAotkGRGV9lSJOha9YigLlKBeD Df5AtfPmWF5J69C8SDRmJmJDXU4UZ2GyOpSIEmDDFSzobJUlDI1f5btRmu+hHMFsMhD2 yQUYaa4j00TbrHftMY0TqWGKe6B7gG8wSUyzQPKpR8XEniIHGBTSZbF/RBYpbHnivAkA mOeAxO2nkMrt2BXNG8y05BR0+hZQ8KGpcOwSXB9vY+7wBxRVT2tEIA/mT78ITjuOn5U9 N4Pg== X-Gm-Message-State: AO0yUKVKc7PkrWVKaDFDz4YAehE1JFD/OhwovrBO1GsSO6s6827iZduU 651KhZDjopeOZJA8gqRUhhEYP3N8C4KEnENO X-Google-Smtp-Source: AK7set9D2OpvDDSr+GLNHXs/C7w7wBvafYGQC2fQWVbz9EArsKG5FZAcat4y22/spEvbn2qMSa1j/w== X-Received: by 2002:a05:6a20:6a9e:b0:cd:71de:e757 with SMTP id bi30-20020a056a206a9e00b000cd71dee757mr1525697pzb.32.1677841035669; Fri, 03 Mar 2023 02:57:15 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.12]) by smtp.gmail.com with ESMTPSA id r15-20020a63a54f000000b004fadb547d0csm1248000pgu.61.2023.03.03.02.57.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Mar 2023 02:57:15 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [PATCH RESEND v6 4/5] memory: Add sanity check in address_space_to_flatview Date: Fri, 3 Mar 2023 18:56:54 +0800 Message-Id: <20230303105655.396446-5-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230303105655.396446-1-xuchuangxclwt@bytedance.com> References: <20230303105655.396446-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1029; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pj1-x1029.google.com X-Spam_score_int: -5 X-Spam_score: -0.6 X-Spam_bar: / X-Spam_report: (-0.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.001, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Before using any flatview, sanity check whether BQL or rcu is held. And if we're during a memory region transaction, try to immediately update mappings, or the map can be invalid. Signed-off-by: Chuang Xu --- include/exec/memory.h | 23 +++++++++++++++++++++++ softmmu/memory.c | 5 +++++ 2 files changed, 28 insertions(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index 2e602a2fad..84b531c6ff 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -27,6 +27,7 @@ #include "qemu/notify.h" #include "qom/object.h" #include "qemu/rcu.h" +#include "qemu/main-loop.h" #define RAM_ADDR_INVALID (~(ram_addr_t)0) @@ -1095,8 +1096,30 @@ struct FlatView { MemoryRegion *root; }; +bool memory_region_transaction_in_progress(void); + +void memory_region_transaction_do_commit(void); + static inline FlatView *address_space_to_flatview(AddressSpace *as) { + if (qemu_mutex_iothread_locked()) { + /* We exclusively own the flatview now.. */ + if (memory_region_transaction_in_progress()) { + /* + * Fetch the flatview within a transaction in-progress, it + * means current_map may not be the latest, we need to update + * immediately to make sure the caller won't see obsolete + * mapping. + */ + memory_region_transaction_do_commit(); + } + + /* No further protection needed to access current_map */ + return as->current_map; + } + + /* Otherwise we must have had the RCU lock or something went wrong */ + assert(rcu_read_is_locked()); return qatomic_rcu_read(&as->current_map); } diff --git a/softmmu/memory.c b/softmmu/memory.c index b89abf400e..1834e14cc8 100644 --- a/softmmu/memory.c +++ b/softmmu/memory.c @@ -1130,6 +1130,11 @@ void memory_region_transaction_commit(void) } } +bool memory_region_transaction_in_progress(void) +{ + return memory_region_transaction_depth != 0; +} + static void memory_region_destructor_none(MemoryRegion *mr) { } From patchwork Fri Mar 3 10:56:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuang Xu X-Patchwork-Id: 13158651 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3FA74C7EE36 for ; Fri, 3 Mar 2023 10:57:56 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pY36f-000728-Nv; Fri, 03 Mar 2023 05:57:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pY36H-0006rr-EX for qemu-devel@nongnu.org; Fri, 03 Mar 2023 05:57:24 -0500 Received: from mail-pg1-x531.google.com ([2607:f8b0:4864:20::531]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pY36F-00043a-Qc for qemu-devel@nongnu.org; Fri, 03 Mar 2023 05:57:21 -0500 Received: by mail-pg1-x531.google.com with SMTP id q23so1201763pgt.7 for ; Fri, 03 Mar 2023 02:57:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1677841038; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=s1mu46Tlnl5UKUBS8hg1pN4oSgMp+Y6WnRUDWtPACnY=; b=TBod0AQt1EbahiJeDlDu6v2ypiNGP7HA+v3tVBP1fXicE5RF0e2FOtEUfEk7Eeun+3 kUony4TLh8ytC/SQGVCRbLxZsX+vnTuYRyxWTJUpCU2Dy4IewqVTjnrMGt7S1qz0qgoN 6lckw1TzHdYXvymzCiKL9+Jy8gXN1pIhoq4CdMj/Ix/kr7/wK+G0bupjGzDp60ejVTlr Tj3sVippOjFLXEx0ElQWrGTMW/fborlI8ZvZjwoyVYz3VBQRfMlyrfAkjSa8wdcK+YG3 GB596jfCPRuHMV4k4I896zl7qjK4mFNbCdkfbv9O1KIm6YM/I5GdUxNtjdDGJ+FqOi6i /MYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677841038; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=s1mu46Tlnl5UKUBS8hg1pN4oSgMp+Y6WnRUDWtPACnY=; b=PcyCJRI+juzMCEn0+VaB3qWELLLt7PQh6vVqbMsGHXmHIKfB1XZ7idUZykZQfQcL8A AU0AdXuVuLJ5uaLYCa+t3mtbBBtg1wmyoQVWwkzluj+beZ9Dk78xI0OKXO4QyiPQFDUV jg96thayCa/oF4ABIhioIJvYdMVBlbruT8M2FWP7Yo6OxIeo91LFn5258ODZbGegzMle bbWcQOaKqBNrznhaQFBOB2UfpqWhayKwmw6dM8c02uIkSjmL1Q+1bKKY+CW6IqbjGm8v D1xwnFuJRwnjBEs1Tdch1VpxjhvRAED0IRIrqokQnBhn6vufrP1bM645j4VhSVYJOV63 3izw== X-Gm-Message-State: AO0yUKXDDTipTF77YQxyJ7+rg1TFQolrHOWvZ2iUxCudiUbm9nmr+Pkr 3jaRbim3iUWgcmTfPgAxjNrtOpwn6dY2PIzU X-Google-Smtp-Source: AK7set+5fI0xmzV3N7clBXvwtmNnVuQ6sRUw3GpqOzk7F76/iu/RqbvWrZcrahQYkZNU+Z8pM/gM7Q== X-Received: by 2002:a62:65c1:0:b0:60b:e13:a10b with SMTP id z184-20020a6265c1000000b0060b0e13a10bmr1472783pfb.3.1677841038303; Fri, 03 Mar 2023 02:57:18 -0800 (PST) Received: from n250-032-048.byted.org ([221.194.189.12]) by smtp.gmail.com with ESMTPSA id r15-20020a63a54f000000b004fadb547d0csm1248000pgu.61.2023.03.03.02.57.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Mar 2023 02:57:18 -0800 (PST) From: Chuang Xu To: qemu-devel@nongnu.org Cc: dgilbert@redhat.com, quintela@redhat.com, pbonzini@redhat.com, peterx@redhat.com, david@redhat.com, philmd@linaro.org, zhouyibo@bytedance.com, Chuang Xu Subject: [PATCH RESEND v6 5/5] migration: Reduce time of loading non-iterable vmstate Date: Fri, 3 Mar 2023 18:56:55 +0800 Message-Id: <20230303105655.396446-6-xuchuangxclwt@bytedance.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230303105655.396446-1-xuchuangxclwt@bytedance.com> References: <20230303105655.396446-1-xuchuangxclwt@bytedance.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::531; envelope-from=xuchuangxclwt@bytedance.com; helo=mail-pg1-x531.google.com X-Spam_score_int: -5 X-Spam_score: -0.6 X-Spam_bar: / X-Spam_report: (-0.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_LOCAL_NOVOWEL=0.5, HK_RANDOM_ENVFROM=0.001, HK_RANDOM_FROM=0.999, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The duration of loading non-iterable vmstate accounts for a significant portion of downtime (starting with the timestamp of source qemu stop and ending with the timestamp of target qemu start). Most of the time is spent committing memory region changes repeatedly. This patch packs all the changes to memory region during the period of loading non-iterable vmstate in a single memory transaction. With the increase of devices, this patch will greatly improve the performance. Here are the test1 results: test info: - Host - Intel(R) Xeon(R) Platinum 8362 CPU - Mellanox Technologies MT28841 - VM - 32 CPUs 128GB RAM VM - 8 16-queue vhost-net device - 16 4-queue vhost-user-blk device. time of loading non-iterable vmstate downtime before about 112 ms 285 ms after about 44 ms 208 ms In test2, we keep the number of the device the same as test1, reduce the number of queues per device: Here are the test2 results: test info: - Host - Intel(R) Xeon(R) Platinum 8362 CPU - Mellanox Technologies MT28841 - VM - 32 CPUs 128GB RAM VM - 8 1-queue vhost-net device - 16 1-queue vhost-user-blk device. time of loading non-iterable vmstate downtime before about 65 ms about 151 ms after about 30 ms about 110 ms In test3, we keep the number of queues per device the same as test1, reduce the number of devices: Here are the test3 results: test info: - Host - Intel(R) Xeon(R) Platinum 8362 CPU - Mellanox Technologies MT28841 - VM - 32 CPUs 128GB RAM VM - 1 16-queue vhost-net device - 1 4-queue vhost-user-blk device. time of loading non-iterable vmstate downtime before about 24 ms about 51 ms after about 12 ms about 38 ms As we can see from the test results above, both the number of queues and the number of devices have a great impact on the time of loading non-iterable vmstate. The growth of the number of devices and queues will lead to more mr commits, and the time consumption caused by the flatview reconstruction will also increase. Signed-off-by: Chuang Xu --- migration/savevm.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/migration/savevm.c b/migration/savevm.c index b5e6962bb6..3dd9daabd8 100644 --- a/migration/savevm.c +++ b/migration/savevm.c @@ -2770,6 +2770,7 @@ out: goto retry; } } + return ret; } @@ -2795,7 +2796,25 @@ int qemu_loadvm_state(QEMUFile *f) cpu_synchronize_all_pre_loadvm(); + /* + * Call memory_region_transaction_begin() before loading vmstate. + * This call is paired with memory_region_transaction_commit() at + * the end of qemu_loadvm_state_main(), in order to pack all the + * changes to memory region during the period of loading + * non-iterable vmstate in a single memory transaction. + * This operation will reduce time of loading non-iterable vmstate + */ + memory_region_transaction_begin(); + ret = qemu_loadvm_state_main(f, mis); + + /* + * Call memory_region_transaction_commit() after loading vmstate. + * At this point, qemu actually completes all the previous memory + * region transactions. + */ + memory_region_transaction_commit(); + qemu_event_set(&mis->main_thread_load_event); trace_qemu_loadvm_state_post_main(ret);