From patchwork Wed Sep 4 22:35:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13791476 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B6439CD4F46 for ; Wed, 4 Sep 2024 22:36:29 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slyav-0003nz-S5; Wed, 04 Sep 2024 18:35:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slyat-0003m4-Ma for qemu-devel@nongnu.org; Wed, 04 Sep 2024 18:35:19 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slyar-00019B-NA for qemu-devel@nongnu.org; Wed, 04 Sep 2024 18:35:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725489316; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PSegrKh4aO8PowvNwpz62I125Ra5M8gvAX+e6Q3xOzM=; b=GGnjVjIHP0xM5L7vslCLl5qu2LGVIKBMEzBqLSUn30rsvLv/IwK2EGE+pjcHXyPCGA/DJa GqBzB6bQQpjODGROyzCV52arKAB0m1fckgtoT/M31uq/Fej0H4eU4A6hjZNxBxd/iS4GNA J/BC8UKqgDFiZ/qBgc2/K+HnVz6jNXI= Received: from mail-oi1-f198.google.com (mail-oi1-f198.google.com [209.85.167.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-220-K0fzEa1ON0SeCUvbAVISrA-1; Wed, 04 Sep 2024 18:35:15 -0400 X-MC-Unique: K0fzEa1ON0SeCUvbAVISrA-1 Received: by mail-oi1-f198.google.com with SMTP id 5614622812f47-3df1cd35a90so77463b6e.0 for ; Wed, 04 Sep 2024 15:35:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725489314; x=1726094114; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PSegrKh4aO8PowvNwpz62I125Ra5M8gvAX+e6Q3xOzM=; b=qw0R89IjFjf/ukOsGvthlJNcJAJAdfB5rgVqTDc2nihBxVnnhnFUc+HdqQUPiraSlP XzWVWarAIremUPi3HK1wsZgni0t6ijp7LiGxMbGtStKsiKO+zd3toqVlA5MA3LuPwWxD QwLfV3nyjV0P5tp5GQ2LyR09+IajZVYP0bmtMURfIQauMmAtZxyVvIUpDHYp/wQqbtvl S4U9tYOxdtjbvVB/72BnsmXrpbeNlkeQ4ibz2ntvgO3ZQepF1KaKovsjsg924ggUF71W mKODe10mu2erPNpNrNNerEA10GllPVzBsqz8VbpruJXV2Xl2uf9AKe1DhQQ2pXqDT5U3 C5QQ== X-Gm-Message-State: AOJu0Yzoh9sRABS3b6AqHlkGjzkkWJ0kmCAONTHYzZtdgm4uoxk51jff ix1q6Ia10SUYcvmPsh8qAR261MKs06NZeQtBDUDC8BWdYlyOXQxtVU6CHTqSWTtVcTCRvqzhs7J zLx8sBLOCzFM+my8jG73CkwvUW36ISXZK8AZzAgbfMUSFJF0RKQW1aM1+yaB1i4vmb+0BTRo/g2 rjfWFcJLfwZ0dMFM2l9rqx8lr/8B68rQH5xs+t X-Received: by 2002:a05:6808:14d1:b0:3db:2cf3:36e with SMTP id 5614622812f47-3e00cb979ccmr8954324b6e.28.1725489314450; Wed, 04 Sep 2024 15:35:14 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG+vrRJODVklsEV5v6ZUr1AkYDbsGeJpOnkoNwoXnitU7WelxMD1fK/AI+mXPcmJXtlOxRlOg== X-Received: by 2002:a05:6808:14d1:b0:3db:2cf3:36e with SMTP id 5614622812f47-3e00cb979ccmr8954288b6e.28.1725489314041; Wed, 04 Sep 2024 15:35:14 -0700 (PDT) Received: from x1n.redhat.com (pool-99-254-121-117.cpe.net.cable.rogers.com. [99.254.121.117]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45801b35df7sm2428881cf.33.2024.09.04.15.35.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 15:35:13 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Vitaly Kuznetsov , Prasad Pandit , Julia Suvorova , peterx@redhat.com, Fabiano Rosas , Paolo Bonzini , Juraj Marcin , David Hildenbrand , qemu-stable , Zhiyi Guo Subject: [PATCH v2 1/4] KVM: Dynamic sized kvm memslots array Date: Wed, 4 Sep 2024 18:35:07 -0400 Message-ID: <20240904223510.3519358-2-peterx@redhat.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240904223510.3519358-1-peterx@redhat.com> References: <20240904223510.3519358-1-peterx@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.142, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Zhiyi reported an infinite loop issue in VFIO use case. The cause of that was a separate discussion, however during that I found a regression of dirty sync slowness when profiling. Each KVMMemoryListerner maintains an array of kvm memslots. Currently it's statically allocated to be the max supported by the kernel. However after Linux commit 4fc096a99e ("KVM: Raise the maximum number of user memslots"), the max supported memslots reported now grows to some number large enough so that it may not be wise to always statically allocate with the max reported. What's worse, QEMU kvm code still walks all the allocated memslots entries to do any form of lookups. It can drastically slow down all memslot operations because each of such loop can run over 32K times on the new kernels. Fix this issue by making the memslots to be allocated dynamically. Here the initial size was set to 16 because it should cover the basic VM usages, so that the hope is the majority VM use case may not even need to grow at all (e.g. if one starts a VM with ./qemu-system-x86_64 by default it'll consume 9 memslots), however not too large to waste memory. There can also be even better way to address this, but so far this is the simplest and should be already better even than before we grow the max supported memslots. For example, in the case of above issue when VFIO was attached on a 32GB system, there are only ~10 memslots used. So it could be good enough as of now. In the above VFIO context, measurement shows that the precopy dirty sync shrinked from ~86ms to ~3ms after this patch applied. It should also apply to any KVM enabled VM even without VFIO. NOTE: we don't have a FIXES tag for this patch because there's no real commit that regressed this in QEMU. Such behavior existed for a long time, but only start to be a problem when the kernel reports very large nr_slots_max value. However that's pretty common now (the kernel change was merged in 2021) so we attached cc:stable because we'll want this change to be backported to stable branches. Cc: qemu-stable Reported-by: Zhiyi Guo Tested-by: Zhiyi Guo Acked-by: David Hildenbrand Signed-off-by: Peter Xu --- include/sysemu/kvm_int.h | 1 + accel/kvm/kvm-all.c | 93 +++++++++++++++++++++++++++++++++------- accel/kvm/trace-events | 1 + 3 files changed, 80 insertions(+), 15 deletions(-) diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h index 1d8fb1473b..48e496b3d4 100644 --- a/include/sysemu/kvm_int.h +++ b/include/sysemu/kvm_int.h @@ -46,6 +46,7 @@ typedef struct KVMMemoryListener { MemoryListener listener; KVMSlot *slots; unsigned int nr_used_slots; + unsigned int nr_slots_allocated; int as_id; QSIMPLEQ_HEAD(, KVMMemoryUpdate) transaction_add; QSIMPLEQ_HEAD(, KVMMemoryUpdate) transaction_del; diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index 75d11a07b2..f9368494a8 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -69,6 +69,9 @@ #define KVM_GUESTDBG_BLOCKIRQ 0 #endif +/* Default num of memslots to be allocated when VM starts */ +#define KVM_MEMSLOTS_NR_ALLOC_DEFAULT 16 + struct KVMParkedVcpu { unsigned long vcpu_id; int kvm_fd; @@ -165,6 +168,57 @@ void kvm_resample_fd_notify(int gsi) } } +/** + * kvm_slots_grow(): Grow the slots[] array in the KVMMemoryListener + * + * @kml: The KVMMemoryListener* to grow the slots[] array + * @nr_slots_new: The new size of slots[] array + * + * Returns: True if the array grows larger, false otherwise. + */ +static bool kvm_slots_grow(KVMMemoryListener *kml, unsigned int nr_slots_new) +{ + unsigned int i, cur = kml->nr_slots_allocated; + KVMSlot *slots; + + if (nr_slots_new > kvm_state->nr_slots) { + nr_slots_new = kvm_state->nr_slots; + } + + if (cur >= nr_slots_new) { + /* Big enough, no need to grow, or we reached max */ + return false; + } + + if (cur == 0) { + slots = g_new0(KVMSlot, nr_slots_new); + } else { + assert(kml->slots); + slots = g_renew(KVMSlot, kml->slots, nr_slots_new); + /* + * g_renew() doesn't initialize extended buffers, however kvm + * memslots require fields to be zero-initialized. E.g. pointers, + * memory_size field, etc. + */ + memset(&slots[cur], 0x0, sizeof(slots[0]) * (nr_slots_new - cur)); + } + + for (i = cur; i < nr_slots_new; i++) { + slots[i].slot = i; + } + + kml->slots = slots; + kml->nr_slots_allocated = nr_slots_new; + trace_kvm_slots_grow(cur, nr_slots_new); + + return true; +} + +static bool kvm_slots_double(KVMMemoryListener *kml) +{ + return kvm_slots_grow(kml, kml->nr_slots_allocated * 2); +} + unsigned int kvm_get_max_memslots(void) { KVMState *s = KVM_STATE(current_accel()); @@ -193,15 +247,20 @@ unsigned int kvm_get_free_memslots(void) /* Called with KVMMemoryListener.slots_lock held */ static KVMSlot *kvm_get_free_slot(KVMMemoryListener *kml) { - KVMState *s = kvm_state; int i; - for (i = 0; i < s->nr_slots; i++) { +retry: + for (i = 0; i < kml->nr_slots_allocated; i++) { if (kml->slots[i].memory_size == 0) { return &kml->slots[i]; } } + /* If no free slots, try to grow first by doubling */ + if (kvm_slots_double(kml)) { + goto retry; + } + return NULL; } @@ -222,10 +281,9 @@ static KVMSlot *kvm_lookup_matching_slot(KVMMemoryListener *kml, hwaddr start_addr, hwaddr size) { - KVMState *s = kvm_state; int i; - for (i = 0; i < s->nr_slots; i++) { + for (i = 0; i < kml->nr_slots_allocated; i++) { KVMSlot *mem = &kml->slots[i]; if (start_addr == mem->start_addr && size == mem->memory_size) { @@ -267,7 +325,7 @@ int kvm_physical_memory_addr_from_host(KVMState *s, void *ram, int i, ret = 0; kvm_slots_lock(); - for (i = 0; i < s->nr_slots; i++) { + for (i = 0; i < kml->nr_slots_allocated; i++) { KVMSlot *mem = &kml->slots[i]; if (ram >= mem->ram && ram < mem->ram + mem->memory_size) { @@ -1071,7 +1129,7 @@ static int kvm_physical_log_clear(KVMMemoryListener *kml, kvm_slots_lock(); - for (i = 0; i < s->nr_slots; i++) { + for (i = 0; i < kml->nr_slots_allocated; i++) { mem = &kml->slots[i]; /* Discard slots that are empty or do not overlap the section */ if (!mem->memory_size || @@ -1719,12 +1777,8 @@ static void kvm_log_sync_global(MemoryListener *l, bool last_stage) /* Flush all kernel dirty addresses into KVMSlot dirty bitmap */ kvm_dirty_ring_flush(); - /* - * TODO: make this faster when nr_slots is big while there are - * only a few used slots (small VMs). - */ kvm_slots_lock(); - for (i = 0; i < s->nr_slots; i++) { + for (i = 0; i < kml->nr_slots_allocated; i++) { mem = &kml->slots[i]; if (mem->memory_size && mem->flags & KVM_MEM_LOG_DIRTY_PAGES) { kvm_slot_sync_dirty_pages(mem); @@ -1839,12 +1893,9 @@ void kvm_memory_listener_register(KVMState *s, KVMMemoryListener *kml, { int i; - kml->slots = g_new0(KVMSlot, s->nr_slots); kml->as_id = as_id; - for (i = 0; i < s->nr_slots; i++) { - kml->slots[i].slot = i; - } + kvm_slots_grow(kml, KVM_MEMSLOTS_NR_ALLOC_DEFAULT); QSIMPLEQ_INIT(&kml->transaction_add); QSIMPLEQ_INIT(&kml->transaction_del); @@ -2461,6 +2512,18 @@ static int kvm_init(MachineState *ms) s->nr_slots = 32; } + /* + * A VM will at least require a few memslots to work, or it can even + * fail to boot. Make sure the supported value is always at least + * larger than what we will initially allocate. + */ + if (s->nr_slots < KVM_MEMSLOTS_NR_ALLOC_DEFAULT) { + ret = -EINVAL; + fprintf(stderr, "KVM max supported number of slots (%d) too small\n", + s->nr_slots); + goto err; + } + s->nr_as = kvm_check_extension(s, KVM_CAP_MULTI_ADDRESS_SPACE); if (s->nr_as <= 1) { s->nr_as = 1; diff --git a/accel/kvm/trace-events b/accel/kvm/trace-events index 37626c1ac5..ad2ae6fca5 100644 --- a/accel/kvm/trace-events +++ b/accel/kvm/trace-events @@ -36,3 +36,4 @@ kvm_io_window_exit(void) "" kvm_run_exit_system_event(int cpu_index, uint32_t event_type) "cpu_index %d, system_even_type %"PRIu32 kvm_convert_memory(uint64_t start, uint64_t size, const char *msg) "start 0x%" PRIx64 " size 0x%" PRIx64 " %s" kvm_memory_fault(uint64_t start, uint64_t size, uint64_t flags) "start 0x%" PRIx64 " size 0x%" PRIx64 " flags 0x%" PRIx64 +kvm_slots_grow(unsigned int old, unsigned int new) "%u -> %u" From patchwork Wed Sep 4 22:35:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13791477 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2AAD8CD4F3C for ; Wed, 4 Sep 2024 22:36:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slyav-0003o9-WE; Wed, 04 Sep 2024 18:35:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slyau-0003mY-IZ for qemu-devel@nongnu.org; Wed, 04 Sep 2024 18:35:20 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slyas-00019O-Hn for qemu-devel@nongnu.org; Wed, 04 Sep 2024 18:35:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725489318; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=n96AsVK5wxnnnoMRLrLFJzon8QcyNjwp4J+Dv4Znv6c=; b=XL7K1PkEXPoYxDurqlfzB+QPn4fhhyp7PXHVDvUJx5qDR1dU4RFSmmb9kj/tAyx1L1n7LF 8ZYZjeJo+wC6BE1SzlmSeyPgrNqLppz1dbhLNeV7t5DvKivXsN62I/7ykhKGuMbRX1oKop lqkUaj4Gg/nsfrjFel6Fgm0P1q6J/eU= Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-468-RvUdrT8ZNQSFrlTe42xOYw-1; Wed, 04 Sep 2024 18:35:17 -0400 X-MC-Unique: RvUdrT8ZNQSFrlTe42xOYw-1 Received: by mail-qt1-f197.google.com with SMTP id d75a77b69052e-457d35e797bso1436091cf.2 for ; Wed, 04 Sep 2024 15:35:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725489316; x=1726094116; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=n96AsVK5wxnnnoMRLrLFJzon8QcyNjwp4J+Dv4Znv6c=; b=sVhTTu0hDD0Sz4v7aDUzG/UnS7PNJsVgW6uZ22POX7GIfipf3MU00BIKcGxqF6yQLg hjP6tOgjMd2wmitB1nhvinvLuZMaLgz/2DwpcFsRYpJLimLSRU8AKttQfFHUvOre8QqE JPwn+uDOJnasDBxWye1Q9ikw5iKsqOt7BmWHj4EFk+lRtiynoCFQ16ma0pflvl31xDMC ycX4mLoR2d6keW9Fgpty3MW4DschOi47TUzloXDf7dK6ZveeJA/3NkZvUXX1GlQsqFyj H5C+y7Ry80tbven1Q1a9HqhkhyQkaak9lo6Z6HzzD9VzrGSZg/t8nj0O/3iz393a3rmB JA1Q== X-Gm-Message-State: AOJu0YxXdgb98JT7BcyFxo0M8ah5fYM2OvXJ5iJM9uNNcH7lLPFrQm0D VIrrj3/AnMe60FK7jAflxPY8mzuwQh6Jc0PUfQix+feiB00odDFVGN473GN6P+fmuCDxV4nT/hV 97Q9KVcFolz2x1LDTGqAMCkz52Xhg34G4s/5M25hAJTYkZAYzc/uCySpWKqmBPUgU+Q4Vb4ByU+ rnQ6eLudg/0oYfzDRdiFLhcrHXV1QNMIriHOx+ X-Received: by 2002:a05:622a:2505:b0:455:1f:c876 with SMTP id d75a77b69052e-457e2d28ef2mr77535721cf.13.1725489316006; Wed, 04 Sep 2024 15:35:16 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEej6DTJw87HRtWA9LFEYoflZXflYbD22qJixPEd6eqCMhYMl/22a2S+6MTDmdU5xt5S3LKpg== X-Received: by 2002:a05:622a:2505:b0:455:1f:c876 with SMTP id d75a77b69052e-457e2d28ef2mr77535471cf.13.1725489315655; Wed, 04 Sep 2024 15:35:15 -0700 (PDT) Received: from x1n.redhat.com (pool-99-254-121-117.cpe.net.cable.rogers.com. [99.254.121.117]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45801b35df7sm2428881cf.33.2024.09.04.15.35.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 15:35:14 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Vitaly Kuznetsov , Prasad Pandit , Julia Suvorova , peterx@redhat.com, Fabiano Rosas , Paolo Bonzini , Juraj Marcin , David Hildenbrand Subject: [PATCH v2 2/4] KVM: Define KVM_MEMSLOTS_NUM_MAX_DEFAULT Date: Wed, 4 Sep 2024 18:35:08 -0400 Message-ID: <20240904223510.3519358-3-peterx@redhat.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240904223510.3519358-1-peterx@redhat.com> References: <20240904223510.3519358-1-peterx@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.133.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.142, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Make the default max nr_slots a macro, it's only used when KVM reports nothing. Reviewed-by: David Hildenbrand Signed-off-by: Peter Xu --- accel/kvm/kvm-all.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index f9368494a8..bc1b039190 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -71,6 +71,8 @@ /* Default num of memslots to be allocated when VM starts */ #define KVM_MEMSLOTS_NR_ALLOC_DEFAULT 16 +/* Default max allowed memslots if kernel reported nothing */ +#define KVM_MEMSLOTS_NR_MAX_DEFAULT 32 struct KVMParkedVcpu { unsigned long vcpu_id; @@ -2509,7 +2511,7 @@ static int kvm_init(MachineState *ms) /* If unspecified, use the default value */ if (!s->nr_slots) { - s->nr_slots = 32; + s->nr_slots_max = KVM_MEMSLOTS_NR_MAX_DEFAULT; } /* From patchwork Wed Sep 4 22:35:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13791475 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3C3ABCD4F45 for ; Wed, 4 Sep 2024 22:36:29 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slyay-0003x2-37; Wed, 04 Sep 2024 18:35:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slyaw-0003qY-Ij for qemu-devel@nongnu.org; Wed, 04 Sep 2024 18:35:22 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slyau-00019k-Tm for qemu-devel@nongnu.org; Wed, 04 Sep 2024 18:35:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725489320; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bDftqUi+TODvxn/v8AdpNLGIJkWpOz4ULtJCbY+H3Mc=; b=iJGNVHV1PJY9pK/MZS/xZOqcmi4rM+bSqU7ALItDu1pX0EX3kKyWdjPkdjn1/IBwlzImQc Yc+AhorbYtPwFP2annGqemZuvhDpc1RnzyrxbzcMlqeAAbebnEIoIVkiGQi4DJhnhF1kzS MotHRG+KCbzmUG7o1o0rTXCB7AWbzf4= Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-311-kpNfUwJYOr2EpIrgic_Dng-1; Wed, 04 Sep 2024 18:35:19 -0400 X-MC-Unique: kpNfUwJYOr2EpIrgic_Dng-1 Received: by mail-qt1-f199.google.com with SMTP id d75a77b69052e-457c70a2da0so1271001cf.2 for ; Wed, 04 Sep 2024 15:35:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725489318; x=1726094118; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bDftqUi+TODvxn/v8AdpNLGIJkWpOz4ULtJCbY+H3Mc=; b=wjd1T/U1siRq67B8BgwfzHnc4rM4ZdB5TQIcVINKdns44nfzDXOtHSeP9jPpJ2IEeB Ow9A61wXVpMDZHh9ohhrdG8YUSF3lADyc+NhOZ+cenJ8Lim8vo+RKK/knvnTJmTxKuvy FQVT4TKqdg6w5PontxBvXEqUCW3e9ca98BNVm1DNCInqyM5IJ0zLR1NQbEGZY2IpoScc J4+mfMB5uEm7jVbeXmdU3jLvDSfJaljflDcmvcHf2/fyxS9GILvuptzN1OfCJOArvpUq wZ/klyqQ5dMx1YOhA2h2C272P5lBIwU5BPrlEZNg+BhB39UQxogigmBx/vNdGecBoCvt nJnQ== X-Gm-Message-State: AOJu0YyGbIaseH6IFPT6OTe6o43l2HKcTCcUGo5sFcoZumjybTn8aMlW LCgLxrmc1rZTMZ5LTabAB0meWs+TpXk2WziPscwyIU66C3y2koLz9H55WXTR5C+CRgbWMrM5LIq sdkZOJ/8LD7axMnRue2Qy8o6quUwWRVzYfXUteCLAkT/gGpQuj4xabaxMk7E1WpE3clf0Dd57m3 4wFtYmhx7kMF+5OtqDwq4cgorJPc7lCBcjXiBm X-Received: by 2002:a05:622a:993:b0:456:8ee8:f5f3 with SMTP id d75a77b69052e-456fd7d8632mr186080831cf.33.1725489317808; Wed, 04 Sep 2024 15:35:17 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFnanzZ3lnpXZrxYOqOHeANrVZMrQvjX/09uT0b3vzCXL42rtL5rdMjlBhd2PyGuZIOcyBJlA== X-Received: by 2002:a05:622a:993:b0:456:8ee8:f5f3 with SMTP id d75a77b69052e-456fd7d8632mr186080391cf.33.1725489317230; Wed, 04 Sep 2024 15:35:17 -0700 (PDT) Received: from x1n.redhat.com (pool-99-254-121-117.cpe.net.cable.rogers.com. [99.254.121.117]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45801b35df7sm2428881cf.33.2024.09.04.15.35.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 15:35:16 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Vitaly Kuznetsov , Prasad Pandit , Julia Suvorova , peterx@redhat.com, Fabiano Rosas , Paolo Bonzini , Juraj Marcin , David Hildenbrand Subject: [PATCH v2 3/4] KVM: Rename KVMMemoryListener.nr_used_slots to nr_slots_used Date: Wed, 4 Sep 2024 18:35:09 -0400 Message-ID: <20240904223510.3519358-4-peterx@redhat.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240904223510.3519358-1-peterx@redhat.com> References: <20240904223510.3519358-1-peterx@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.142, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This will make all nr_slots counters to be named in the same manner. Reviewed-by: David Hildenbrand Signed-off-by: Peter Xu --- include/sysemu/kvm_int.h | 2 +- accel/kvm/kvm-all.c | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h index 48e496b3d4..b705dfc9b4 100644 --- a/include/sysemu/kvm_int.h +++ b/include/sysemu/kvm_int.h @@ -45,7 +45,7 @@ typedef struct KVMMemoryUpdate { typedef struct KVMMemoryListener { MemoryListener listener; KVMSlot *slots; - unsigned int nr_used_slots; + unsigned int nr_slots_used; unsigned int nr_slots_allocated; int as_id; QSIMPLEQ_HEAD(, KVMMemoryUpdate) transaction_add; diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index bc1b039190..b7fb73ae18 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -239,7 +239,7 @@ unsigned int kvm_get_free_memslots(void) if (!s->as[i].ml) { continue; } - used_slots = MAX(used_slots, s->as[i].ml->nr_used_slots); + used_slots = MAX(used_slots, s->as[i].ml->nr_slots_used); } kvm_slots_unlock(); @@ -1510,7 +1510,7 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, } start_addr += slot_size; size -= slot_size; - kml->nr_used_slots--; + kml->nr_slots_used--; } while (size); return; } @@ -1549,7 +1549,7 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml, ram_start_offset += slot_size; ram += slot_size; size -= slot_size; - kml->nr_used_slots++; + kml->nr_slots_used++; } while (size); } From patchwork Wed Sep 4 22:35:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peter Xu X-Patchwork-Id: 13791473 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A5CCDCD4F3C for ; Wed, 4 Sep 2024 22:36:28 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1slyb0-00045h-47; Wed, 04 Sep 2024 18:35:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slyax-0003va-Ob for qemu-devel@nongnu.org; Wed, 04 Sep 2024 18:35:23 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1slyaw-0001A8-7t for qemu-devel@nongnu.org; Wed, 04 Sep 2024 18:35:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1725489321; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EkbxL0DLwlIg9hHpDaAUsSOs36b2eDu6NFzXSaDhuLM=; b=go2YM2DYC6uDF90o41SbVZdvjv4OSbE07UHquDkVm1N2WOhNFBLyu6RcMUXHq5f1zxj3ad tjJ1MI3FCiY5JkQxKO1AXqc31+MUJWETSARvKmQLqA1hD3bcNfefP1BiI7389GUdXmiPvk qHNgt6x6fO+8MqbUCfJC1SR+bhrmZZI= Received: from mail-oi1-f200.google.com (mail-oi1-f200.google.com [209.85.167.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-447-PpirzaWNORa7rlFsz63AAg-1; Wed, 04 Sep 2024 18:35:20 -0400 X-MC-Unique: PpirzaWNORa7rlFsz63AAg-1 Received: by mail-oi1-f200.google.com with SMTP id 5614622812f47-3df033c1e08so75842b6e.2 for ; Wed, 04 Sep 2024 15:35:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725489319; x=1726094119; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EkbxL0DLwlIg9hHpDaAUsSOs36b2eDu6NFzXSaDhuLM=; b=MvgschJMkTsn+gSsjsSkRnOzFurVD2RrU33/X0fVkezJdUXfHPZ4Ow1mgYotq0rD/a dlSjwiAlZL3FBCwPI2U/aDsyWkRh66sCQDsDFY1F+h+cGhX/G+uupidFzmFvxFytx6lR 3LRsP3867/7FZYeZG3DvNbFNJDHA7EeCww+p5UEJwOI75Yv58RMt8AeIwU3rJn8UWnhA yRTQQcjPcoOrDWIaa+bOjy7jFzKlQct8mh4gHeFloRGr7SELM0GNUx4bh/wOtfasx9bF LBavyNaX0atRhAksUir+z7RqLOIJTi01H7lRA8w0pytwbpuBrVbPzELhs0al37iurxd5 5ooA== X-Gm-Message-State: AOJu0Yz572WFMj3Eo/7dZJbN94z+CoHN54wn1tIN9djg78SBfCYdBXdc 3j8nEDeqJXnauCUDfD8r28zRCpeMB8UW9/TqL3Ldo9qjlbUuGJ2Ba+u0xxFs8Itxi9h9BrTnL85 ztn7IezjTuzOmGbD2uVQQ+2DhQkKsx3EBP1RVIxXDe/9r43A+WrUdHMcdKOOsfJ+suLjsyF1+wK B4uyGJLb0VBInGxvw3A3VthS3l7p7vn4O1Lz5U X-Received: by 2002:a05:6808:1a0e:b0:3df:1625:d993 with SMTP id 5614622812f47-3df1ca27bf5mr18219464b6e.34.1725489319433; Wed, 04 Sep 2024 15:35:19 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGsIqhbfHNOPbcqocHfuPo8FFcU7ygheSRmWWbrNer0Jfo/Fy4cUuihzIBzE/fIYQlY7MK25g== X-Received: by 2002:a05:6808:1a0e:b0:3df:1625:d993 with SMTP id 5614622812f47-3df1ca27bf5mr18219442b6e.34.1725489319087; Wed, 04 Sep 2024 15:35:19 -0700 (PDT) Received: from x1n.redhat.com (pool-99-254-121-117.cpe.net.cable.rogers.com. [99.254.121.117]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45801b35df7sm2428881cf.33.2024.09.04.15.35.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Sep 2024 15:35:18 -0700 (PDT) From: Peter Xu To: qemu-devel@nongnu.org Cc: Vitaly Kuznetsov , Prasad Pandit , Julia Suvorova , peterx@redhat.com, Fabiano Rosas , Paolo Bonzini , Juraj Marcin , David Hildenbrand Subject: [PATCH v2 4/4] KVM: Rename KVMState->nr_slots to nr_slots_max Date: Wed, 4 Sep 2024 18:35:10 -0400 Message-ID: <20240904223510.3519358-5-peterx@redhat.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240904223510.3519358-1-peterx@redhat.com> References: <20240904223510.3519358-1-peterx@redhat.com> MIME-Version: 1.0 Received-SPF: pass client-ip=170.10.129.124; envelope-from=peterx@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.142, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This value used to reflect the maximum supported memslots from KVM kernel. Rename it to be clearer. Reviewed-by: David Hildenbrand Signed-off-by: Peter Xu --- include/sysemu/kvm_int.h | 4 ++-- accel/kvm/kvm-all.c | 16 ++++++++-------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h index b705dfc9b4..2c57194b6b 100644 --- a/include/sysemu/kvm_int.h +++ b/include/sysemu/kvm_int.h @@ -103,8 +103,8 @@ struct KVMDirtyRingReaper { struct KVMState { AccelState parent_obj; - - int nr_slots; + /* Max number of KVM slots supported */ + int nr_slots_max; int fd; int vmfd; int coalesced_mmio; diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index b7fb73ae18..020fd16ab8 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -183,8 +183,8 @@ static bool kvm_slots_grow(KVMMemoryListener *kml, unsigned int nr_slots_new) unsigned int i, cur = kml->nr_slots_allocated; KVMSlot *slots; - if (nr_slots_new > kvm_state->nr_slots) { - nr_slots_new = kvm_state->nr_slots; + if (nr_slots_new > kvm_state->nr_slots_max) { + nr_slots_new = kvm_state->nr_slots_max; } if (cur >= nr_slots_new) { @@ -225,7 +225,7 @@ unsigned int kvm_get_max_memslots(void) { KVMState *s = KVM_STATE(current_accel()); - return s->nr_slots; + return s->nr_slots_max; } unsigned int kvm_get_free_memslots(void) @@ -243,7 +243,7 @@ unsigned int kvm_get_free_memslots(void) } kvm_slots_unlock(); - return s->nr_slots - used_slots; + return s->nr_slots_max - used_slots; } /* Called with KVMMemoryListener.slots_lock held */ @@ -2507,10 +2507,10 @@ static int kvm_init(MachineState *ms) (kvm_supported_memory_attributes & KVM_MEMORY_ATTRIBUTE_PRIVATE); kvm_immediate_exit = kvm_check_extension(s, KVM_CAP_IMMEDIATE_EXIT); - s->nr_slots = kvm_check_extension(s, KVM_CAP_NR_MEMSLOTS); + s->nr_slots_max = kvm_check_extension(s, KVM_CAP_NR_MEMSLOTS); /* If unspecified, use the default value */ - if (!s->nr_slots) { + if (!s->nr_slots_max) { s->nr_slots_max = KVM_MEMSLOTS_NR_MAX_DEFAULT; } @@ -2519,10 +2519,10 @@ static int kvm_init(MachineState *ms) * fail to boot. Make sure the supported value is always at least * larger than what we will initially allocate. */ - if (s->nr_slots < KVM_MEMSLOTS_NR_ALLOC_DEFAULT) { + if (s->nr_slots_max < KVM_MEMSLOTS_NR_ALLOC_DEFAULT) { ret = -EINVAL; fprintf(stderr, "KVM max supported number of slots (%d) too small\n", - s->nr_slots); + s->nr_slots_max); goto err; }