From patchwork Wed Feb 12 13:42:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11378585 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 67DB6924 for ; Wed, 12 Feb 2020 13:46:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3FA7B206B6 for ; Wed, 12 Feb 2020 13:46:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="L1oSSqvG" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3FA7B206B6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38082 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sLb-0002vp-CC for patchwork-qemu-devel@patchwork.kernel.org; Wed, 12 Feb 2020 08:46:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39207) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sIT-0006M7-JY for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1sIQ-0003fw-O6 for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:21 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:51184 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j1sIQ-0003fE-6w for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:18 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581514997; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SDigtZPzr12awgdVXgpZVqFUZEj/xk07krCG/vw+Z6U=; b=L1oSSqvGLZjBCpLol/qkP9QHoYBjlOjdXf9aKQRfBWAHKxGiS6PwYi+WLt1mUjlVafug8w RWquEsWrMXUPvtBCOFOYnYiYC6KArLbDKIIs/EBIwNlhQoVHiFKDVvQsVwCzf8uMEvPICh voPC7mnfUv/eBvszLSeJDXIrZBJXoG8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-105-LFRU4pKLNLOYxAb42U7F-A-1; Wed, 12 Feb 2020 08:43:14 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2962E100550E; Wed, 12 Feb 2020 13:43:13 +0000 (UTC) Received: from t480s.redhat.com (ovpn-117-92.ams2.redhat.com [10.36.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id D2E821001B07; Wed, 12 Feb 2020 13:43:10 +0000 (UTC) From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v2 fixed 01/16] util: vfio-helpers: Factor out and fix processing of existing ram blocks Date: Wed, 12 Feb 2020 14:42:39 +0100 Message-Id: <20200212134254.11073-2-david@redhat.com> In-Reply-To: <20200212134254.11073-1-david@redhat.com> References: <20200212134254.11073-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: LFRU4pKLNLOYxAb42U7F-A-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S . Tsirkin" , David Hildenbrand , Stefan Hajnoczi , "Dr . David Alan Gilbert" , Alex Williamson , Igor Mammedov , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Factor it out into common code when a new notifier is registered, just as done with the memory region notifier. This allows us to have the logic about how to process existing ram blocks at a central place (which will be extended soon). Just like when adding a new ram block, we have to register the max_length for now. We don't have a way to get notified about resizes yet, and some memory would not be mapped when growing the ram block. Note: Currently, ram blocks are only "fake resized". All memory (max_length) is accessible. We can get rid of a bunch of functions in stubs/ram-block.c . Print the warning from inside qemu_vfio_ram_block_added(). Cc: Richard Henderson Cc: Paolo Bonzini Cc: Eduardo Habkost Cc: Marcel Apfelbaum Cc: Alex Williamson Cc: Stefan Hajnoczi Signed-off-by: David Hildenbrand --- exec.c | 5 +++++ hw/core/numa.c | 14 ++++++++++++++ include/exec/cpu-common.h | 1 + stubs/ram-block.c | 20 -------------------- util/vfio-helpers.c | 28 +++++++--------------------- 5 files changed, 27 insertions(+), 41 deletions(-) diff --git a/exec.c b/exec.c index 67e520d18e..05cfe868ab 100644 --- a/exec.c +++ b/exec.c @@ -2017,6 +2017,11 @@ ram_addr_t qemu_ram_get_used_length(RAMBlock *rb) return rb->used_length; } +ram_addr_t qemu_ram_get_max_length(RAMBlock *rb) +{ + return rb->max_length; +} + bool qemu_ram_is_shared(RAMBlock *rb) { return rb->flags & RAM_SHARED; diff --git a/hw/core/numa.c b/hw/core/numa.c index 0d1b4be76a..6599c69e05 100644 --- a/hw/core/numa.c +++ b/hw/core/numa.c @@ -899,9 +899,23 @@ void query_numa_node_mem(NumaNodeMem node_mem[], MachineState *ms) } } +static int ram_block_notify_add_single(RAMBlock *rb, void *opaque) +{ + const ram_addr_t max_size = qemu_ram_get_max_length(rb); + void *host = qemu_ram_get_host_addr(rb); + RAMBlockNotifier *notifier = opaque; + + if (host) { + notifier->ram_block_added(notifier, host, max_size); + } + return 0; +} + void ram_block_notifier_add(RAMBlockNotifier *n) { QLIST_INSERT_HEAD(&ram_list.ramblock_notifiers, n, next); + /* Notify about all existing ram blocks. */ + qemu_ram_foreach_block(ram_block_notify_add_single, n); } void ram_block_notifier_remove(RAMBlockNotifier *n) diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h index 81753bbb34..9760ac9068 100644 --- a/include/exec/cpu-common.h +++ b/include/exec/cpu-common.h @@ -59,6 +59,7 @@ const char *qemu_ram_get_idstr(RAMBlock *rb); void *qemu_ram_get_host_addr(RAMBlock *rb); ram_addr_t qemu_ram_get_offset(RAMBlock *rb); ram_addr_t qemu_ram_get_used_length(RAMBlock *rb); +ram_addr_t qemu_ram_get_max_length(RAMBlock *rb); bool qemu_ram_is_shared(RAMBlock *rb); bool qemu_ram_is_uf_zeroable(RAMBlock *rb); void qemu_ram_set_uf_zeroable(RAMBlock *rb); diff --git a/stubs/ram-block.c b/stubs/ram-block.c index 73c0a3ee08..10855b52dd 100644 --- a/stubs/ram-block.c +++ b/stubs/ram-block.c @@ -2,21 +2,6 @@ #include "exec/ramlist.h" #include "exec/cpu-common.h" -void *qemu_ram_get_host_addr(RAMBlock *rb) -{ - return 0; -} - -ram_addr_t qemu_ram_get_offset(RAMBlock *rb) -{ - return 0; -} - -ram_addr_t qemu_ram_get_used_length(RAMBlock *rb) -{ - return 0; -} - void ram_block_notifier_add(RAMBlockNotifier *n) { } @@ -24,8 +9,3 @@ void ram_block_notifier_add(RAMBlockNotifier *n) void ram_block_notifier_remove(RAMBlockNotifier *n) { } - -int qemu_ram_foreach_block(RAMBlockIterFunc func, void *opaque) -{ - return 0; -} diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c index 813f7ec564..71e02e7f35 100644 --- a/util/vfio-helpers.c +++ b/util/vfio-helpers.c @@ -376,8 +376,13 @@ static void qemu_vfio_ram_block_added(RAMBlockNotifier *n, void *host, size_t size) { QEMUVFIOState *s = container_of(n, QEMUVFIOState, ram_notifier); + int ret; + trace_qemu_vfio_ram_block_added(s, host, size); - qemu_vfio_dma_map(s, host, size, false, NULL); + ret = qemu_vfio_dma_map(s, host, size, false, NULL); + if (ret) { + error_report("qemu_vfio_dma_map(%p, %zu) failed: %d", host, size, ret); + } } static void qemu_vfio_ram_block_removed(RAMBlockNotifier *n, @@ -390,33 +395,14 @@ static void qemu_vfio_ram_block_removed(RAMBlockNotifier *n, } } -static int qemu_vfio_init_ramblock(RAMBlock *rb, void *opaque) -{ - void *host_addr = qemu_ram_get_host_addr(rb); - ram_addr_t length = qemu_ram_get_used_length(rb); - int ret; - QEMUVFIOState *s = opaque; - - if (!host_addr) { - return 0; - } - ret = qemu_vfio_dma_map(s, host_addr, length, false, NULL); - if (ret) { - fprintf(stderr, "qemu_vfio_init_ramblock: failed %p %" PRId64 "\n", - host_addr, (uint64_t)length); - } - return 0; -} - static void qemu_vfio_open_common(QEMUVFIOState *s) { qemu_mutex_init(&s->lock); s->ram_notifier.ram_block_added = qemu_vfio_ram_block_added; s->ram_notifier.ram_block_removed = qemu_vfio_ram_block_removed; - ram_block_notifier_add(&s->ram_notifier); s->low_water_mark = QEMU_VFIO_IOVA_MIN; s->high_water_mark = QEMU_VFIO_IOVA_MAX; - qemu_ram_foreach_block(qemu_vfio_init_ramblock, s); + ram_block_notifier_add(&s->ram_notifier); } /** From patchwork Wed Feb 12 13:42:40 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11378591 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4ED0A924 for ; Wed, 12 Feb 2020 13:49:30 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 26451206B6 for ; Wed, 12 Feb 2020 13:49:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BSZhPomi" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 26451206B6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38122 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sOP-00081U-CF for patchwork-qemu-devel@patchwork.kernel.org; Wed, 12 Feb 2020 08:49:29 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39221) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sIW-0006SO-26 for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:25 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1sIR-0003gR-6W for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:23 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:47158 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j1sIQ-0003ff-TH for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581514998; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yNuQlSgldSAkpIuDIJbk/zpSbtxwoK8DXhH4/lGJbpk=; b=BSZhPomigrkX+nu0cr+P3CyJMgYtPp/c9iAHxHKYUZ7zGrQmRvP4nMRvIxHonHbHojDFj9 6hNbWvd+a/UqcGLx+5bawFFCHFn7G/6D46R1+Ix8KtLiZZmEPqiDBYZYI7DP6NpFArQg98 2DJB/00AbVElwKpouEh/B1VqymT7MLg= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-328-DP7iRCqnOeOoOXvupSnS5Q-1; Wed, 12 Feb 2020 08:43:16 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C0F0A801E67; Wed, 12 Feb 2020 13:43:15 +0000 (UTC) Received: from t480s.redhat.com (ovpn-117-92.ams2.redhat.com [10.36.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id 75F601001B07; Wed, 12 Feb 2020 13:43:13 +0000 (UTC) From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v2 fixed 02/16] util: vfio-helpers: Fix qemu_vfio_close() Date: Wed, 12 Feb 2020 14:42:40 +0100 Message-Id: <20200212134254.11073-3-david@redhat.com> In-Reply-To: <20200212134254.11073-1-david@redhat.com> References: <20200212134254.11073-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: DP7iRCqnOeOoOXvupSnS5Q-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S . Tsirkin" , David Hildenbrand , Stefan Hajnoczi , "Dr . David Alan Gilbert" , Alex Williamson , Igor Mammedov , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" qemu_vfio_undo_mapping() will decrement the number of mappings and reshuffle the array elements to fit into the reduced size. Iterating over all elements like this does not work as expected, let's make sure to remove all mappings properly. Cc: Richard Henderson Cc: Paolo Bonzini Cc: Eduardo Habkost Cc: Marcel Apfelbaum Cc: Alex Williamson Cc: Stefan Hajnoczi Signed-off-by: David Hildenbrand Reviewed-by: Peter Xu --- util/vfio-helpers.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c index 71e02e7f35..d6332522c1 100644 --- a/util/vfio-helpers.c +++ b/util/vfio-helpers.c @@ -694,13 +694,11 @@ static void qemu_vfio_reset(QEMUVFIOState *s) /* Close and free the VFIO resources. */ void qemu_vfio_close(QEMUVFIOState *s) { - int i; - if (!s) { return; } - for (i = 0; i < s->nr_mappings; ++i) { - qemu_vfio_undo_mapping(s, &s->mappings[i], NULL); + while (s->nr_mappings) { + qemu_vfio_undo_mapping(s, &s->mappings[s->nr_mappings - 1], NULL); } ram_block_notifier_remove(&s->ram_notifier); qemu_vfio_reset(s); From patchwork Wed Feb 12 13:42:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11378589 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5DB8F924 for ; Wed, 12 Feb 2020 13:48:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 35742206B6 for ; Wed, 12 Feb 2020 13:48:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="U8PGEkN8" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 35742206B6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38106 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sNX-00062u-9L for patchwork-qemu-devel@patchwork.kernel.org; Wed, 12 Feb 2020 08:48:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39252) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sIe-0006o6-If for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:33 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1sId-0003xL-FI for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:32 -0500 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:23538 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j1sId-0003x3-BA for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581515011; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=A/W0Rnt5R1bACGOIM7oXhJcO9iZmEXLI8cWkfuHUQwA=; b=U8PGEkN8XX/ti8SN1jK9uvVmehtDuC2hOgrfXqf255ZdErOpSelHh7TyvrzRyYI7zKi9ov gIkv2zxpSokb3SoMkvvj556qqs6kOjcA8ivLcGRtKH29+64ckbZyvQvdLscpTyyd5fLMWH Wcc62YLEw9Y1TI/br6OWBSnHYl0w5EU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-330-bxJFlNPqM5K9syowjz41hQ-1; Wed, 12 Feb 2020 08:43:29 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 443CC10054E3; Wed, 12 Feb 2020 13:43:28 +0000 (UTC) Received: from t480s.redhat.com (ovpn-117-92.ams2.redhat.com [10.36.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1DE451001281; Wed, 12 Feb 2020 13:43:15 +0000 (UTC) From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v2 fixed 03/16] util: vfio-helpers: Remove Error parameter from qemu_vfio_undo_mapping() Date: Wed, 12 Feb 2020 14:42:41 +0100 Message-Id: <20200212134254.11073-4-david@redhat.com> In-Reply-To: <20200212134254.11073-1-david@redhat.com> References: <20200212134254.11073-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: bxJFlNPqM5K9syowjz41hQ-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S . Tsirkin" , David Hildenbrand , Stefan Hajnoczi , "Dr . David Alan Gilbert" , Alex Williamson , Igor Mammedov , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Everybody discards the error. Let's error_report() instead so this error doesn't get lost. Cc: Richard Henderson Cc: Paolo Bonzini Cc: Eduardo Habkost Cc: Marcel Apfelbaum Cc: Alex Williamson Cc: Stefan Hajnoczi Signed-off-by: David Hildenbrand --- util/vfio-helpers.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c index d6332522c1..13dd962d95 100644 --- a/util/vfio-helpers.c +++ b/util/vfio-helpers.c @@ -540,8 +540,7 @@ static int qemu_vfio_do_mapping(QEMUVFIOState *s, void *host, size_t size, /** * Undo the DMA mapping from @s with VFIO, and remove from mapping list. */ -static void qemu_vfio_undo_mapping(QEMUVFIOState *s, IOVAMapping *mapping, - Error **errp) +static void qemu_vfio_undo_mapping(QEMUVFIOState *s, IOVAMapping *mapping) { int index; struct vfio_iommu_type1_dma_unmap unmap = { @@ -556,7 +555,7 @@ static void qemu_vfio_undo_mapping(QEMUVFIOState *s, IOVAMapping *mapping, assert(QEMU_IS_ALIGNED(mapping->size, qemu_real_host_page_size)); assert(index >= 0 && index < s->nr_mappings); if (ioctl(s->container, VFIO_IOMMU_UNMAP_DMA, &unmap)) { - error_setg(errp, "VFIO_UNMAP_DMA failed: %d", -errno); + error_report("VFIO_UNMAP_DMA failed: %d", -errno); } memmove(mapping, &s->mappings[index + 1], sizeof(s->mappings[0]) * (s->nr_mappings - index - 1)); @@ -621,7 +620,7 @@ int qemu_vfio_dma_map(QEMUVFIOState *s, void *host, size_t size, assert(qemu_vfio_verify_mappings(s)); ret = qemu_vfio_do_mapping(s, host, size, iova0); if (ret) { - qemu_vfio_undo_mapping(s, mapping, NULL); + qemu_vfio_undo_mapping(s, mapping); goto out; } s->low_water_mark += size; @@ -681,7 +680,7 @@ void qemu_vfio_dma_unmap(QEMUVFIOState *s, void *host) if (!m) { goto out; } - qemu_vfio_undo_mapping(s, m, NULL); + qemu_vfio_undo_mapping(s, m); out: qemu_mutex_unlock(&s->lock); } @@ -698,7 +697,7 @@ void qemu_vfio_close(QEMUVFIOState *s) return; } while (s->nr_mappings) { - qemu_vfio_undo_mapping(s, &s->mappings[s->nr_mappings - 1], NULL); + qemu_vfio_undo_mapping(s, &s->mappings[s->nr_mappings - 1]); } ram_block_notifier_remove(&s->ram_notifier); qemu_vfio_reset(s); From patchwork Wed Feb 12 13:42:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11378595 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B00BD924 for ; Wed, 12 Feb 2020 13:50:50 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8701A206B6 for ; Wed, 12 Feb 2020 13:50:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="OiD1rfHC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8701A206B6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38160 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sPh-0002IW-P4 for patchwork-qemu-devel@patchwork.kernel.org; Wed, 12 Feb 2020 08:50:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39274) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sIh-0006vd-Kl for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:36 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1sIg-00040Y-AC for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:35 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:49261 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j1sIg-000408-5y for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581515013; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qepC1WP47Sat3hY176Yi0qhBDAWOu4/qmPiAqR1VGoI=; b=OiD1rfHCqmrbWUFBgXqN1iANkXv1+AZmiJG7MT146fAs5CX89B7JOnPSsP7lCELsyhfu3j qhRJ7lI012GukJ0+DvmVPudbzo59vuhXL5MaoejHWx4tlTJh9oDzwWPxuYKmarzhVVPzbg B9p9uLbU/u2n2otxh89g08BfG+eXX5A= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-434-9MvaIf2aOqmiAlzZkmZ4HQ-1; Wed, 12 Feb 2020 08:43:31 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D3AB9107ACC9; Wed, 12 Feb 2020 13:43:30 +0000 (UTC) Received: from t480s.redhat.com (ovpn-117-92.ams2.redhat.com [10.36.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9096E1001DDE; Wed, 12 Feb 2020 13:43:28 +0000 (UTC) From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v2 fixed 04/16] util: vfio-helpers: Factor out removal from qemu_vfio_undo_mapping() Date: Wed, 12 Feb 2020 14:42:42 +0100 Message-Id: <20200212134254.11073-5-david@redhat.com> In-Reply-To: <20200212134254.11073-1-david@redhat.com> References: <20200212134254.11073-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: 9MvaIf2aOqmiAlzZkmZ4HQ-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S . Tsirkin" , David Hildenbrand , Stefan Hajnoczi , "Dr . David Alan Gilbert" , Alex Williamson , Igor Mammedov , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Factor it out and properly use it where applicable. Make qemu_vfio_undo_mapping() look like qemu_vfio_do_mapping(), passing the size and iova, not the mapping. Cc: Richard Henderson Cc: Paolo Bonzini Cc: Eduardo Habkost Cc: Marcel Apfelbaum Cc: Alex Williamson Cc: Stefan Hajnoczi Signed-off-by: David Hildenbrand Reviewed-by: Peter Xu --- util/vfio-helpers.c | 43 +++++++++++++++++++++++++++---------------- 1 file changed, 27 insertions(+), 16 deletions(-) diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c index 13dd962d95..b3adc328db 100644 --- a/util/vfio-helpers.c +++ b/util/vfio-helpers.c @@ -517,6 +517,20 @@ static IOVAMapping *qemu_vfio_add_mapping(QEMUVFIOState *s, return insert; } +/** + * Remove the mapping from @s and free it. + */ +static void qemu_vfio_remove_mapping(QEMUVFIOState *s, IOVAMapping *mapping) +{ + const int index = mapping - s->mappings; + + assert(index >= 0 && index < s->nr_mappings); + memmove(mapping, &s->mappings[index + 1], + sizeof(s->mappings[0]) * (s->nr_mappings - index - 1)); + s->nr_mappings--; + s->mappings = g_renew(IOVAMapping, s->mappings, s->nr_mappings); +} + /* Do the DMA mapping with VFIO. */ static int qemu_vfio_do_mapping(QEMUVFIOState *s, void *host, size_t size, uint64_t iova) @@ -538,29 +552,22 @@ static int qemu_vfio_do_mapping(QEMUVFIOState *s, void *host, size_t size, } /** - * Undo the DMA mapping from @s with VFIO, and remove from mapping list. + * Undo the DMA mapping from @s with VFIO. */ -static void qemu_vfio_undo_mapping(QEMUVFIOState *s, IOVAMapping *mapping) +static void qemu_vfio_undo_mapping(QEMUVFIOState *s, size_t size, uint64_t iova) { - int index; struct vfio_iommu_type1_dma_unmap unmap = { .argsz = sizeof(unmap), .flags = 0, - .iova = mapping->iova, - .size = mapping->size, + .iova = iova, + .size = size, }; - index = mapping - s->mappings; - assert(mapping->size > 0); - assert(QEMU_IS_ALIGNED(mapping->size, qemu_real_host_page_size)); - assert(index >= 0 && index < s->nr_mappings); + assert(size > 0); + assert(QEMU_IS_ALIGNED(size, qemu_real_host_page_size)); if (ioctl(s->container, VFIO_IOMMU_UNMAP_DMA, &unmap)) { error_report("VFIO_UNMAP_DMA failed: %d", -errno); } - memmove(mapping, &s->mappings[index + 1], - sizeof(s->mappings[0]) * (s->nr_mappings - index - 1)); - s->nr_mappings--; - s->mappings = g_renew(IOVAMapping, s->mappings, s->nr_mappings); } /* Check if the mapping list is (ascending) ordered. */ @@ -620,7 +627,7 @@ int qemu_vfio_dma_map(QEMUVFIOState *s, void *host, size_t size, assert(qemu_vfio_verify_mappings(s)); ret = qemu_vfio_do_mapping(s, host, size, iova0); if (ret) { - qemu_vfio_undo_mapping(s, mapping); + qemu_vfio_remove_mapping(s, mapping); goto out; } s->low_water_mark += size; @@ -680,7 +687,8 @@ void qemu_vfio_dma_unmap(QEMUVFIOState *s, void *host) if (!m) { goto out; } - qemu_vfio_undo_mapping(s, m); + qemu_vfio_undo_mapping(s, m->size, m->iova); + qemu_vfio_remove_mapping(s, m); out: qemu_mutex_unlock(&s->lock); } @@ -697,7 +705,10 @@ void qemu_vfio_close(QEMUVFIOState *s) return; } while (s->nr_mappings) { - qemu_vfio_undo_mapping(s, &s->mappings[s->nr_mappings - 1]); + IOVAMapping *m = &s->mappings[s->nr_mappings - 1]; + + qemu_vfio_undo_mapping(s, m->size, m->iova); + qemu_vfio_remove_mapping(s, m); } ram_block_notifier_remove(&s->ram_notifier); qemu_vfio_reset(s); From patchwork Wed Feb 12 13:42:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11378593 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1F98E13A4 for ; Wed, 12 Feb 2020 13:49:50 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EAF3A206B6 for ; Wed, 12 Feb 2020 13:49:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="XMz2kupM" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EAF3A206B6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38124 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sOj-0000I7-4W for patchwork-qemu-devel@patchwork.kernel.org; Wed, 12 Feb 2020 08:49:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39304) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sIl-000753-Hd for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:40 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1sIk-00044X-Dj for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:39 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:58922 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j1sIk-000447-9u for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:38 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581515017; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zxlDY+iam1eGvrSP9JAuoFgfMKsrZ8lZwfOq1IgFDoE=; b=XMz2kupMbqLyupEc6zIXNc+UBJEXSjpkdUp/ukzeLmdrcXENIBJ+zMoCcNTiUTBJjnqvIV uiGqoSoLb4EjPbdTN+6L2z8bgp5vU3T3JsHBrLehdipEXVWanEc7MoMrZTSM1Fjq3xA2RW o3ySSkNMGX5+8JiLbkpnrQ9jdqctaF4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-56-5VpgLspdNxWtiMZ0y68tQQ-1; Wed, 12 Feb 2020 08:43:34 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 20B65800D4E; Wed, 12 Feb 2020 13:43:33 +0000 (UTC) Received: from t480s.redhat.com (ovpn-117-92.ams2.redhat.com [10.36.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3084C1001B08; Wed, 12 Feb 2020 13:43:31 +0000 (UTC) From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v2 fixed 05/16] exec: Factor out setting ram settings (madvise ...) into qemu_ram_apply_settings() Date: Wed, 12 Feb 2020 14:42:43 +0100 Message-Id: <20200212134254.11073-6-david@redhat.com> In-Reply-To: <20200212134254.11073-1-david@redhat.com> References: <20200212134254.11073-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: 5VpgLspdNxWtiMZ0y68tQQ-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S . Tsirkin" , David Hildenbrand , Richard Henderson , "Dr . David Alan Gilbert" , Igor Mammedov , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Factor all settings out into qemu_ram_apply_settings(). For memory_try_enable_merging(), the important bit is that it won't be called with XEN - which is now still the case as new_block->host will remain NULL. Reviewed-by: Richard Henderson Cc: Richard Henderson Cc: Paolo Bonzini Cc: Igor Mammedov Signed-off-by: David Hildenbrand Reviewed-by: Peter Xu --- exec.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/exec.c b/exec.c index 05cfe868ab..31a462a7d3 100644 --- a/exec.c +++ b/exec.c @@ -2121,6 +2121,15 @@ static int memory_try_enable_merging(void *addr, size_t len) return qemu_madvise(addr, len, QEMU_MADV_MERGEABLE); } +static void qemu_ram_apply_settings(void *host, size_t length) +{ + memory_try_enable_merging(host, length); + qemu_ram_setup_dump(host, length); + qemu_madvise(host, length, QEMU_MADV_HUGEPAGE); + /* MADV_DONTFORK is also needed by KVM in absence of synchronous MMU */ + qemu_madvise(host, length, QEMU_MADV_DONTFORK); +} + /* Only legal before guest might have detected the memory size: e.g. on * incoming migration, or right after reset. * @@ -2271,7 +2280,6 @@ static void ram_block_add(RAMBlock *new_block, Error **errp, bool shared) qemu_mutex_unlock_ramlist(); return; } - memory_try_enable_merging(new_block->host, new_block->max_length); } } @@ -2309,10 +2317,7 @@ static void ram_block_add(RAMBlock *new_block, Error **errp, bool shared) DIRTY_CLIENTS_ALL); if (new_block->host) { - qemu_ram_setup_dump(new_block->host, new_block->max_length); - qemu_madvise(new_block->host, new_block->max_length, QEMU_MADV_HUGEPAGE); - /* MADV_DONTFORK is also needed by KVM in absence of synchronous MMU */ - qemu_madvise(new_block->host, new_block->max_length, QEMU_MADV_DONTFORK); + qemu_ram_apply_settings(new_block->host, new_block->max_length); ram_block_notify_add(new_block->host, new_block->max_length); } } From patchwork Wed Feb 12 13:42:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11378597 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D17C5924 for ; Wed, 12 Feb 2020 13:51:09 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A6E37206B6 for ; Wed, 12 Feb 2020 13:51:09 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fthkQl54" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A6E37206B6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38162 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sQ0-0002vu-Rn for patchwork-qemu-devel@patchwork.kernel.org; Wed, 12 Feb 2020 08:51:08 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39306) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sIl-00075R-MZ for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:40 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1sIk-00044l-Km for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:39 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:57230 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j1sIk-00044J-H3 for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:38 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581515018; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=X6WUlVonFsZtKBb2gz2kBtwE/N1ki16+gfem00/U4IQ=; b=fthkQl54GpsGQWP5CmuBQ7vZaAxX0PFEUnAfbX8kIYG6wrLMceIlYo5g9ylahlTiu92qiJ 9ZsyA6ov+hBI8oSiCbDPhdRCPZZ0sqHxu2WK1STtolt+CxVxocFL5HXd6uzUlz7IfkCiGI JDxYQZl75vw8YCzBaPlDCH+vldyVNvU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-29-cj85Ev7bMKW_HkOU6GKc-A-1; Wed, 12 Feb 2020 08:43:36 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 577F6DBB8; Wed, 12 Feb 2020 13:43:35 +0000 (UTC) Received: from t480s.redhat.com (ovpn-117-92.ams2.redhat.com [10.36.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6E6031001B08; Wed, 12 Feb 2020 13:43:33 +0000 (UTC) From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v2 fixed 06/16] exec: Reuse qemu_ram_apply_settings() in qemu_ram_remap() Date: Wed, 12 Feb 2020 14:42:44 +0100 Message-Id: <20200212134254.11073-7-david@redhat.com> In-Reply-To: <20200212134254.11073-1-david@redhat.com> References: <20200212134254.11073-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: cj85Ev7bMKW_HkOU6GKc-A-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S . Tsirkin" , David Hildenbrand , Richard Henderson , "Dr . David Alan Gilbert" , Igor Mammedov , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" I don't see why we shouldn't apply all settings to make it look like the surrounding RAM (and enable proper VMA merging). Note: memory backend settings might have overridden these settings. We would need a callback to let the memory backend fix that up. Reviewed-by: Richard Henderson Cc: Richard Henderson Cc: Paolo Bonzini Cc: Igor Mammedov Signed-off-by: David Hildenbrand Reviewed-by: Peter Xu --- exec.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/exec.c b/exec.c index 31a462a7d3..f7525867ec 100644 --- a/exec.c +++ b/exec.c @@ -2552,8 +2552,7 @@ void qemu_ram_remap(ram_addr_t addr, ram_addr_t length) length, addr); exit(1); } - memory_try_enable_merging(vaddr, length); - qemu_ram_setup_dump(vaddr, length); + qemu_ram_apply_settings(vaddr, length); } } } From patchwork Wed Feb 12 13:42:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11378609 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1442F13A4 for ; Wed, 12 Feb 2020 13:54:22 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AF49B2082F for ; Wed, 12 Feb 2020 13:54:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="i+AtvvCw" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AF49B2082F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38230 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sT6-00015C-Nq for patchwork-qemu-devel@patchwork.kernel.org; Wed, 12 Feb 2020 08:54:20 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39357) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sIt-0007OG-9f for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:48 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1sIs-0004DM-5T for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:47 -0500 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:25859 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j1sIs-0004D3-1R for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581515025; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H3rvnuT51cs0LPxsa94uLPi5fmCSxlFI0oHScm/fHqA=; b=i+AtvvCwxCDk4t5aYpSHp1TyxWPl0ZB7P0Bztq70xRkzWiCftzxY/+aI7MNgDELQ8ZZTFT XuQk60FWx7PH3/nDnVOOL7AGpMm7uRhlTkFe92HRdArPVu+Sq1nst8zoRStobJ0fDZpxa0 B9FxE3GgPFAi1bKeFlIwkpdtO1YbN7g= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-382-aSDbROOIMWGF7INZVsGtsw-1; Wed, 12 Feb 2020 08:43:44 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EF272801E78; Wed, 12 Feb 2020 13:43:42 +0000 (UTC) Received: from t480s.redhat.com (ovpn-117-92.ams2.redhat.com [10.36.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id A71AF1001B08; Wed, 12 Feb 2020 13:43:35 +0000 (UTC) From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v2 fixed 07/16] exec: Drop "shared" parameter from ram_block_add() Date: Wed, 12 Feb 2020 14:42:45 +0100 Message-Id: <20200212134254.11073-8-david@redhat.com> In-Reply-To: <20200212134254.11073-1-david@redhat.com> References: <20200212134254.11073-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: aSDbROOIMWGF7INZVsGtsw-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Igor Kotrasinski , Eduardo Habkost , "Michael S . Tsirkin" , David Hildenbrand , Richard Henderson , "Dr . David Alan Gilbert" , Igor Mammedov , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Properly store it in the flags of the ram block instead (and the flag even already exists and is used). E.g., qemu_ram_is_shared() now properly succeeds on all ram blocks that are actually shared. Reviewed-by: Igor Kotrasinski Reviewed-by: Richard Henderson Cc: Richard Henderson Cc: Paolo Bonzini Cc: Igor Mammedov Signed-off-by: David Hildenbrand Reviewed-by: Peter Xu --- exec.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/exec.c b/exec.c index f7525867ec..fc65c4f7ca 100644 --- a/exec.c +++ b/exec.c @@ -2249,7 +2249,7 @@ static void dirty_memory_extend(ram_addr_t old_ram_size, } } -static void ram_block_add(RAMBlock *new_block, Error **errp, bool shared) +static void ram_block_add(RAMBlock *new_block, Error **errp) { RAMBlock *block; RAMBlock *last_block = NULL; @@ -2272,7 +2272,8 @@ static void ram_block_add(RAMBlock *new_block, Error **errp, bool shared) } } else { new_block->host = phys_mem_alloc(new_block->max_length, - &new_block->mr->align, shared); + &new_block->mr->align, + qemu_ram_is_shared(new_block)); if (!new_block->host) { error_setg_errno(errp, errno, "cannot set up guest memory '%s'", @@ -2376,7 +2377,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr, return NULL; } - ram_block_add(new_block, &local_err, ram_flags & RAM_SHARED); + ram_block_add(new_block, &local_err); if (local_err) { g_free(new_block); error_propagate(errp, local_err); @@ -2438,10 +2439,13 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size, if (host) { new_block->flags |= RAM_PREALLOC; } + if (share) { + new_block->flags |= RAM_SHARED; + } if (resizeable) { new_block->flags |= RAM_RESIZEABLE; } - ram_block_add(new_block, &local_err, share); + ram_block_add(new_block, &local_err); if (local_err) { g_free(new_block); error_propagate(errp, local_err); From patchwork Wed Feb 12 13:42:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11378601 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E65D213A4 for ; Wed, 12 Feb 2020 13:52:22 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BDD852082F for ; Wed, 12 Feb 2020 13:52:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="WTUprPGK" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BDD852082F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38186 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sRB-0005R5-VJ for patchwork-qemu-devel@patchwork.kernel.org; Wed, 12 Feb 2020 08:52:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39381) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sIw-0007Wk-NS for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1sIv-0004HW-Ht for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:50 -0500 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:31265 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j1sIv-0004HA-EA for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:49 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581515029; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=PeNV+R9SkRzKBUQroDIA7kCYiKwN6U06vRHB5GJs2/4=; b=WTUprPGKFGVnUM6ZbQwk3xW5zb5quqr2KmtcucZ+skYzUvNq6z3Dcz2wr9N47C5J74h+qK 84bBSJGh8riDfhutMg8gPjIQRu7nS3rzal0J8bleQp0T5gRILEZCiJz9c+cxO8xO6RorzU gZmDv1oCAMw2xRRe/JRKdjIriogkCjI= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-285-T93ADSFIMOOdDdh7nezg0w-1; Wed, 12 Feb 2020 08:43:47 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B530BDBA3; Wed, 12 Feb 2020 13:43:45 +0000 (UTC) Received: from t480s.redhat.com (ovpn-117-92.ams2.redhat.com [10.36.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4B41D1001B08; Wed, 12 Feb 2020 13:43:43 +0000 (UTC) From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v2 fixed 08/16] util/mmap-alloc: Factor out calculation of pagesize to mmap_pagesize() Date: Wed, 12 Feb 2020 14:42:46 +0100 Message-Id: <20200212134254.11073-9-david@redhat.com> In-Reply-To: <20200212134254.11073-1-david@redhat.com> References: <20200212134254.11073-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: T93ADSFIMOOdDdh7nezg0w-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Igor Kotrasinski , Eduardo Habkost , "Michael S . Tsirkin" , David Hildenbrand , Richard Henderson , "Dr . David Alan Gilbert" , Greg Kurz , Murilo Opsfelder Araujo , Igor Mammedov , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Factor it out and add a comment. Reviewed-by: Igor Kotrasinski Acked-by: Murilo Opsfelder Araujo Reviewed-by: Richard Henderson Cc: "Michael S. Tsirkin" Cc: Murilo Opsfelder Araujo Cc: Greg Kurz Cc: Eduardo Habkost Cc: "Dr. David Alan Gilbert" Cc: Igor Mammedov Signed-off-by: David Hildenbrand --- util/mmap-alloc.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/util/mmap-alloc.c b/util/mmap-alloc.c index 27dcccd8ec..82f02a2cec 100644 --- a/util/mmap-alloc.c +++ b/util/mmap-alloc.c @@ -82,17 +82,27 @@ size_t qemu_mempath_getpagesize(const char *mem_path) return qemu_real_host_page_size; } +static inline size_t mmap_pagesize(int fd) +{ +#if defined(__powerpc64__) && defined(__linux__) + /* Mappings in the same segment must share the same page size */ + return qemu_fd_getpagesize(fd); +#else + return qemu_real_host_page_size; +#endif +} + void *qemu_ram_mmap(int fd, size_t size, size_t align, bool shared, bool is_pmem) { + const size_t pagesize = mmap_pagesize(fd); int flags; int map_sync_flags = 0; int guardfd; size_t offset; - size_t pagesize; size_t total; void *guardptr; void *ptr; @@ -113,7 +123,6 @@ void *qemu_ram_mmap(int fd, * anonymous memory is OK. */ flags = MAP_PRIVATE; - pagesize = qemu_fd_getpagesize(fd); if (fd == -1 || pagesize == qemu_real_host_page_size) { guardfd = -1; flags |= MAP_ANONYMOUS; @@ -123,7 +132,6 @@ void *qemu_ram_mmap(int fd, } #else guardfd = -1; - pagesize = qemu_real_host_page_size; flags = MAP_PRIVATE | MAP_ANONYMOUS; #endif @@ -198,15 +206,10 @@ void *qemu_ram_mmap(int fd, void qemu_ram_munmap(int fd, void *ptr, size_t size) { - size_t pagesize; + const size_t pagesize = mmap_pagesize(fd); if (ptr) { /* Unmap both the RAM block and the guard page */ -#if defined(__powerpc64__) && defined(__linux__) - pagesize = qemu_fd_getpagesize(fd); -#else - pagesize = qemu_real_host_page_size; -#endif munmap(ptr, size + pagesize); } } From patchwork Wed Feb 12 13:42:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11378607 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 363C513A4 for ; Wed, 12 Feb 2020 13:53:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0D9582082F for ; Wed, 12 Feb 2020 13:53:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="JOwam6XZ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0D9582082F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38208 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sSN-00086o-5c for patchwork-qemu-devel@patchwork.kernel.org; Wed, 12 Feb 2020 08:53:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39406) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sJ3-0007dy-CB for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:59 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1sJ0-0004MI-Oq for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:57 -0500 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:44153 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j1sJ0-0004LD-L2 for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581515033; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GZqr2smF5vISL5p0H9inXxoTmM/MfLDgolYLBIyYsHg=; b=JOwam6XZ9tL4F+fRTtk17quh2e6ZKBKky/60zLrJ7hco/gbyA8wEe5IZ9jkQc5R/wIOWre Tm8eWhEcp41RWQdY+0m/YBr/LWF+7P1P0m8S9lKe3rk4tbNL5i6pBjghl3TkgcmLkO2t3F b+QeUKpSujIaWh/97TSGwojSJz/+QAs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-62-euDxcVwzPWqbraSwLypKLQ-1; Wed, 12 Feb 2020 08:43:49 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 78C2BDBA3; Wed, 12 Feb 2020 13:43:48 +0000 (UTC) Received: from t480s.redhat.com (ovpn-117-92.ams2.redhat.com [10.36.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0EE8D1001B08; Wed, 12 Feb 2020 13:43:45 +0000 (UTC) From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v2 fixed 09/16] util/mmap-alloc: Factor out reserving of a memory region to mmap_reserve() Date: Wed, 12 Feb 2020 14:42:47 +0100 Message-Id: <20200212134254.11073-10-david@redhat.com> In-Reply-To: <20200212134254.11073-1-david@redhat.com> References: <20200212134254.11073-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: euDxcVwzPWqbraSwLypKLQ-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Igor Kotrasinski , Eduardo Habkost , "Michael S . Tsirkin" , David Hildenbrand , Richard Henderson , "Dr . David Alan Gilbert" , Greg Kurz , Murilo Opsfelder Araujo , Igor Mammedov , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We want to reserve a memory region without actually populating memory. Let's factor that out. Reviewed-by: Igor Kotrasinski Acked-by: Murilo Opsfelder Araujo Reviewed-by: Richard Henderson Cc: "Michael S. Tsirkin" Cc: Greg Kurz Cc: Murilo Opsfelder Araujo Cc: Eduardo Habkost Cc: "Dr. David Alan Gilbert" Cc: Igor Mammedov Signed-off-by: David Hildenbrand Reviewed-by: Peter Xu --- util/mmap-alloc.c | 58 +++++++++++++++++++++++++++-------------------- 1 file changed, 33 insertions(+), 25 deletions(-) diff --git a/util/mmap-alloc.c b/util/mmap-alloc.c index 82f02a2cec..43a26f38a8 100644 --- a/util/mmap-alloc.c +++ b/util/mmap-alloc.c @@ -82,6 +82,38 @@ size_t qemu_mempath_getpagesize(const char *mem_path) return qemu_real_host_page_size; } +/* + * Reserve a new memory region of the requested size to be used for mapping + * from the given fd (if any). + */ +static void *mmap_reserve(size_t size, int fd) +{ + int flags = MAP_PRIVATE; + +#if defined(__powerpc64__) && defined(__linux__) + /* + * On ppc64 mappings in the same segment (aka slice) must share the same + * page size. Since we will be re-allocating part of this segment + * from the supplied fd, we should make sure to use the same page size, to + * this end we mmap the supplied fd. In this case, set MAP_NORESERVE to + * avoid allocating backing store memory. + * We do this unless we are using the system page size, in which case + * anonymous memory is OK. + */ + if (fd == -1 || qemu_fd_getpagesize(fd) == qemu_real_host_page_size) { + fd = -1; + flags |= MAP_ANONYMOUS; + } else { + flags |= MAP_NORESERVE; + } +#else + fd = -1; + flags |= MAP_ANONYMOUS; +#endif + + return mmap(0, size, PROT_NONE, flags, fd, 0); +} + static inline size_t mmap_pagesize(int fd) { #if defined(__powerpc64__) && defined(__linux__) @@ -101,7 +133,6 @@ void *qemu_ram_mmap(int fd, const size_t pagesize = mmap_pagesize(fd); int flags; int map_sync_flags = 0; - int guardfd; size_t offset; size_t total; void *guardptr; @@ -113,30 +144,7 @@ void *qemu_ram_mmap(int fd, */ total = size + align; -#if defined(__powerpc64__) && defined(__linux__) - /* On ppc64 mappings in the same segment (aka slice) must share the same - * page size. Since we will be re-allocating part of this segment - * from the supplied fd, we should make sure to use the same page size, to - * this end we mmap the supplied fd. In this case, set MAP_NORESERVE to - * avoid allocating backing store memory. - * We do this unless we are using the system page size, in which case - * anonymous memory is OK. - */ - flags = MAP_PRIVATE; - if (fd == -1 || pagesize == qemu_real_host_page_size) { - guardfd = -1; - flags |= MAP_ANONYMOUS; - } else { - guardfd = fd; - flags |= MAP_NORESERVE; - } -#else - guardfd = -1; - flags = MAP_PRIVATE | MAP_ANONYMOUS; -#endif - - guardptr = mmap(0, total, PROT_NONE, flags, guardfd, 0); - + guardptr = mmap_reserve(total, fd); if (guardptr == MAP_FAILED) { return MAP_FAILED; } From patchwork Wed Feb 12 13:42:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11378599 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3F3B5924 for ; Wed, 12 Feb 2020 13:52:04 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 16A092082F for ; Wed, 12 Feb 2020 13:52:04 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="eTdOIlUh" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 16A092082F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38184 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sQt-0004na-5M for patchwork-qemu-devel@patchwork.kernel.org; Wed, 12 Feb 2020 08:52:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39418) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sJ5-0007ez-4Y for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:00 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1sJ3-0004Ok-B0 for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:58 -0500 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:52431 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j1sJ2-0004NP-CE for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:57 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581515036; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Nkt3nPeMNuejDJ2QcKQSthQlrGGILC3VWmryZ4CTcDY=; b=eTdOIlUhsRVcnpPf+mh7llO1fKB1WRDDGvcUsk11oPTBseGuOHTW8GbvPdwV8IFI3+g4xT jWWfubb8w381CGpIVVu9DRBU7UmczBsxKjybCXxbwu1nWqZ5bYASugWMAV9sT8JB+M2hZI aVC2+sJ099vkkWDCR1eSsmEEXEtRSFQ= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-133-BajUEzHeORaazmsIg0SVzg-1; Wed, 12 Feb 2020 08:43:52 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3F9C41005F6C; Wed, 12 Feb 2020 13:43:51 +0000 (UTC) Received: from t480s.redhat.com (ovpn-117-92.ams2.redhat.com [10.36.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id C8B141001281; Wed, 12 Feb 2020 13:43:48 +0000 (UTC) From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v2 fixed 10/16] util/mmap-alloc: Factor out populating of memory to mmap_populate() Date: Wed, 12 Feb 2020 14:42:48 +0100 Message-Id: <20200212134254.11073-11-david@redhat.com> In-Reply-To: <20200212134254.11073-1-david@redhat.com> References: <20200212134254.11073-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: BajUEzHeORaazmsIg0SVzg-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Igor Kotrasinski , Eduardo Habkost , "Michael S . Tsirkin" , David Hildenbrand , Richard Henderson , "Dr . David Alan Gilbert" , Greg Kurz , Murilo Opsfelder Araujo , Igor Mammedov , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We want to populate memory within a reserved memory region. Let's factor that out. Reviewed-by: Richard Henderson Acked-by: Murilo Opsfelder Araujo Cc: Igor Kotrasinski Cc: "Michael S. Tsirkin" Cc: Greg Kurz Cc: Murilo Opsfelder Araujo Cc: Eduardo Habkost Cc: "Dr. David Alan Gilbert" Cc: Igor Mammedov Signed-off-by: David Hildenbrand Reviewed-by: Peter Xu --- util/mmap-alloc.c | 89 +++++++++++++++++++++++++---------------------- 1 file changed, 47 insertions(+), 42 deletions(-) diff --git a/util/mmap-alloc.c b/util/mmap-alloc.c index 43a26f38a8..2f366dae72 100644 --- a/util/mmap-alloc.c +++ b/util/mmap-alloc.c @@ -114,6 +114,50 @@ static void *mmap_reserve(size_t size, int fd) return mmap(0, size, PROT_NONE, flags, fd, 0); } +/* + * Populate memory in a reserved region from the given fd (if any). + */ +static void *mmap_populate(void *ptr, size_t size, int fd, bool shared, + bool is_pmem) +{ + int map_sync_flags = 0; + int flags = MAP_FIXED; + void *populated_ptr; + + flags |= fd == -1 ? MAP_ANONYMOUS : 0; + flags |= shared ? MAP_SHARED : MAP_PRIVATE; + if (shared && is_pmem) { + map_sync_flags = MAP_SYNC | MAP_SHARED_VALIDATE; + } + + populated_ptr = mmap(ptr, size, PROT_READ | PROT_WRITE, + flags | map_sync_flags, fd, 0); + if (populated_ptr == MAP_FAILED && map_sync_flags) { + if (errno == ENOTSUP) { + char *proc_link = g_strdup_printf("/proc/self/fd/%d", fd); + char *file_name = g_malloc0(PATH_MAX); + int len = readlink(proc_link, file_name, PATH_MAX - 1); + + if (len < 0) { + len = 0; + } + file_name[len] = '\0'; + fprintf(stderr, "Warning: requesting persistence across crashes " + "for backend file %s failed. Proceeding without " + "persistence, data might become corrupted in case of host " + "crash.\n", file_name); + g_free(proc_link); + g_free(file_name); + } + /* + * If mmap failed with MAP_SHARED_VALIDATE | MAP_SYNC, we will try + * again without these flags to handle backwards compatibility. + */ + populated_ptr = mmap(ptr, size, PROT_READ | PROT_WRITE, flags, fd, 0); + } + return populated_ptr; +} + static inline size_t mmap_pagesize(int fd) { #if defined(__powerpc64__) && defined(__linux__) @@ -131,12 +175,8 @@ void *qemu_ram_mmap(int fd, bool is_pmem) { const size_t pagesize = mmap_pagesize(fd); - int flags; - int map_sync_flags = 0; - size_t offset; - size_t total; - void *guardptr; - void *ptr; + size_t offset, total; + void *ptr, *guardptr; /* * Note: this always allocates at least one extra page of virtual address @@ -153,44 +193,9 @@ void *qemu_ram_mmap(int fd, /* Always align to host page size */ assert(align >= pagesize); - flags = MAP_FIXED; - flags |= fd == -1 ? MAP_ANONYMOUS : 0; - flags |= shared ? MAP_SHARED : MAP_PRIVATE; - if (shared && is_pmem) { - map_sync_flags = MAP_SYNC | MAP_SHARED_VALIDATE; - } - offset = QEMU_ALIGN_UP((uintptr_t)guardptr, align) - (uintptr_t)guardptr; - ptr = mmap(guardptr + offset, size, PROT_READ | PROT_WRITE, - flags | map_sync_flags, fd, 0); - - if (ptr == MAP_FAILED && map_sync_flags) { - if (errno == ENOTSUP) { - char *proc_link, *file_name; - int len; - proc_link = g_strdup_printf("/proc/self/fd/%d", fd); - file_name = g_malloc0(PATH_MAX); - len = readlink(proc_link, file_name, PATH_MAX - 1); - if (len < 0) { - len = 0; - } - file_name[len] = '\0'; - fprintf(stderr, "Warning: requesting persistence across crashes " - "for backend file %s failed. Proceeding without " - "persistence, data might become corrupted in case of host " - "crash.\n", file_name); - g_free(proc_link); - g_free(file_name); - } - /* - * if map failed with MAP_SHARED_VALIDATE | MAP_SYNC, - * we will remove these flags to handle compatibility. - */ - ptr = mmap(guardptr + offset, size, PROT_READ | PROT_WRITE, - flags, fd, 0); - } - + ptr = mmap_populate(guardptr + offset, size, fd, shared, is_pmem); if (ptr == MAP_FAILED) { munmap(guardptr, total); return MAP_FAILED; From patchwork Wed Feb 12 13:42:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11378605 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E95E6924 for ; Wed, 12 Feb 2020 13:53:13 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C16FC2082F for ; Wed, 12 Feb 2020 13:53:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="RcN6dMjC" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C16FC2082F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38206 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sS1-0007KT-0Q for patchwork-qemu-devel@patchwork.kernel.org; Wed, 12 Feb 2020 08:53:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39437) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sJ8-0007hs-Jo for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:03 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1sJ6-0004Rm-FG for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:02 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:34853 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j1sJ5-0004QC-7T for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:43:59 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581515038; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tha9YJ740MpzJF2OZkPvucAf9Y+sGM5RvzSXdztglkY=; b=RcN6dMjCfBQ9RAFeMUmNuCTrEU8nfdsyLjVfzwvjwAOoXBihLUudltnl3tbhAJF7plWuxy t1dYoRlHb+79dmN7/3HkayaLeXSaXtHBErkrlLUWyUO0HPgqP7YNgio8fQ0SA+KDEYzoVL yOAr03qxeGtVpsH5s92hVhEoZc1UWs4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-346-kVqOpsjBMCiSkq2yW5Vgew-1; Wed, 12 Feb 2020 08:43:55 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 007311005510; Wed, 12 Feb 2020 13:43:54 +0000 (UTC) Received: from t480s.redhat.com (ovpn-117-92.ams2.redhat.com [10.36.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8C0D31001B08; Wed, 12 Feb 2020 13:43:51 +0000 (UTC) From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v2 fixed 11/16] util/mmap-alloc: Prepare for resizable mmaps Date: Wed, 12 Feb 2020 14:42:49 +0100 Message-Id: <20200212134254.11073-12-david@redhat.com> In-Reply-To: <20200212134254.11073-1-david@redhat.com> References: <20200212134254.11073-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: kVqOpsjBMCiSkq2yW5Vgew-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Igor Kotrasinski , Eduardo Habkost , "Michael S . Tsirkin" , David Hildenbrand , Richard Henderson , "Dr . David Alan Gilbert" , Greg Kurz , Murilo Opsfelder Araujo , Igor Mammedov , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" When shrinking a mmap we want to re-reserve the already populated area. When growing a memory region, we want to populate starting with a given fd_offset. Prepare by allowing to pass these parameters. Also, let's make sure we always process full pages, to avoid unmapping/remapping pages that are already in use when growing/shrinking. Add some asserts. Reviewed-by: Richard Henderson Cc: Igor Kotrasinski Cc: Murilo Opsfelder Araujo Cc: "Michael S. Tsirkin" Cc: Greg Kurz Cc: Murilo Opsfelder Araujo Cc: Eduardo Habkost Cc: "Dr. David Alan Gilbert" Cc: Igor Mammedov Signed-off-by: David Hildenbrand Reviewed-by: Peter Xu --- util/mmap-alloc.c | 31 ++++++++++++++++++++----------- 1 file changed, 20 insertions(+), 11 deletions(-) diff --git a/util/mmap-alloc.c b/util/mmap-alloc.c index 2f366dae72..fb7ef588fe 100644 --- a/util/mmap-alloc.c +++ b/util/mmap-alloc.c @@ -83,12 +83,12 @@ size_t qemu_mempath_getpagesize(const char *mem_path) } /* - * Reserve a new memory region of the requested size to be used for mapping - * from the given fd (if any). + * Reserve a new memory region of the requested size or re-reserve parts + * of an existing region to be used for mapping from the given fd (if any). */ -static void *mmap_reserve(size_t size, int fd) +static void *mmap_reserve(void *ptr, size_t size, int fd) { - int flags = MAP_PRIVATE; + int flags = MAP_PRIVATE | (ptr ? MAP_FIXED : 0); #if defined(__powerpc64__) && defined(__linux__) /* @@ -111,19 +111,23 @@ static void *mmap_reserve(size_t size, int fd) flags |= MAP_ANONYMOUS; #endif - return mmap(0, size, PROT_NONE, flags, fd, 0); + return mmap(ptr, size, PROT_NONE, flags, fd, 0); } /* * Populate memory in a reserved region from the given fd (if any). */ -static void *mmap_populate(void *ptr, size_t size, int fd, bool shared, - bool is_pmem) +static void *mmap_populate(void *ptr, size_t size, int fd, size_t fd_offset, + bool shared, bool is_pmem) { int map_sync_flags = 0; int flags = MAP_FIXED; void *populated_ptr; + if (fd == -1) { + fd_offset = 0; + } + flags |= fd == -1 ? MAP_ANONYMOUS : 0; flags |= shared ? MAP_SHARED : MAP_PRIVATE; if (shared && is_pmem) { @@ -131,7 +135,7 @@ static void *mmap_populate(void *ptr, size_t size, int fd, bool shared, } populated_ptr = mmap(ptr, size, PROT_READ | PROT_WRITE, - flags | map_sync_flags, fd, 0); + flags | map_sync_flags, fd, fd_offset); if (populated_ptr == MAP_FAILED && map_sync_flags) { if (errno == ENOTSUP) { char *proc_link = g_strdup_printf("/proc/self/fd/%d", fd); @@ -153,7 +157,8 @@ static void *mmap_populate(void *ptr, size_t size, int fd, bool shared, * If mmap failed with MAP_SHARED_VALIDATE | MAP_SYNC, we will try * again without these flags to handle backwards compatibility. */ - populated_ptr = mmap(ptr, size, PROT_READ | PROT_WRITE, flags, fd, 0); + populated_ptr = mmap(ptr, size, PROT_READ | PROT_WRITE, flags, fd, + fd_offset); } return populated_ptr; } @@ -178,13 +183,15 @@ void *qemu_ram_mmap(int fd, size_t offset, total; void *ptr, *guardptr; + g_assert(QEMU_IS_ALIGNED(size, pagesize)); + /* * Note: this always allocates at least one extra page of virtual address * space, even if size is already aligned. */ total = size + align; - guardptr = mmap_reserve(total, fd); + guardptr = mmap_reserve(0, total, fd); if (guardptr == MAP_FAILED) { return MAP_FAILED; } @@ -195,7 +202,7 @@ void *qemu_ram_mmap(int fd, offset = QEMU_ALIGN_UP((uintptr_t)guardptr, align) - (uintptr_t)guardptr; - ptr = mmap_populate(guardptr + offset, size, fd, shared, is_pmem); + ptr = mmap_populate(guardptr + offset, size, fd, 0, shared, is_pmem); if (ptr == MAP_FAILED) { munmap(guardptr, total); return MAP_FAILED; @@ -221,6 +228,8 @@ void qemu_ram_munmap(int fd, void *ptr, size_t size) { const size_t pagesize = mmap_pagesize(fd); + g_assert(QEMU_IS_ALIGNED(size, pagesize)); + if (ptr) { /* Unmap both the RAM block and the guard page */ munmap(ptr, size + pagesize); From patchwork Wed Feb 12 13:42:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11378615 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C5C61138D for ; Wed, 12 Feb 2020 13:55:49 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9C72B2082F for ; Wed, 12 Feb 2020 13:55:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Dm3c9poY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9C72B2082F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38266 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sUW-0003Fu-RL for patchwork-qemu-devel@patchwork.kernel.org; Wed, 12 Feb 2020 08:55:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39480) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sJH-0007tV-5N for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:12 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1sJF-0004bl-Pu for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:11 -0500 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:33943 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j1sJF-0004bT-M8 for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:09 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581515049; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LnJzdyGqoAQKx+1rnNUMeCg39x2uBiI+oIICFGxMjgU=; b=Dm3c9poYRwF6xEmz2Z4uuV5UnsEfGKuRiuqZcrAks4NMNDE06VC+yAlf8WbFnaoJgDoCF5 0mdrksPGrOoMUf6U4OesGW1v4t3RhyfDQMkdDxtfgEgA1tAtFLlCMsNRQGmdFZTfEJkhhx mglIDrpxQePDIm/pEgLln01rGtvyNH0= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-151-yX8BHRaIOYup158mDf75Yg-1; Wed, 12 Feb 2020 08:44:06 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 50651190D340; Wed, 12 Feb 2020 13:44:04 +0000 (UTC) Received: from t480s.redhat.com (ovpn-117-92.ams2.redhat.com [10.36.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7552F1001281; Wed, 12 Feb 2020 13:43:54 +0000 (UTC) From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v2 fixed 12/16] util/mmap-alloc: Implement resizable mmaps Date: Wed, 12 Feb 2020 14:42:50 +0100 Message-Id: <20200212134254.11073-13-david@redhat.com> In-Reply-To: <20200212134254.11073-1-david@redhat.com> References: <20200212134254.11073-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: yX8BHRaIOYup158mDf75Yg-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Igor Kotrasinski , Eduardo Habkost , "Michael S . Tsirkin" , David Hildenbrand , Richard Henderson , "Dr . David Alan Gilbert" , Greg Kurz , Murilo Opsfelder Araujo , Igor Mammedov , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Implement resizable mmaps. For now, the actual resizing is not wired up. Introduce qemu_ram_mmap_resizable() and qemu_ram_mmap_resize(). Make qemu_ram_mmap() a wrapper of qemu_ram_mmap_resizable(). Cc: Richard Henderson Cc: Igor Kotrasinski Cc: Murilo Opsfelder Araujo Cc: "Michael S. Tsirkin" Cc: Greg Kurz Cc: Eduardo Habkost Cc: "Dr. David Alan Gilbert" Cc: Igor Mammedov Signed-off-by: David Hildenbrand --- include/qemu/mmap-alloc.h | 21 ++++++++++++-------- util/mmap-alloc.c | 42 ++++++++++++++++++++++++++++----------- 2 files changed, 43 insertions(+), 20 deletions(-) diff --git a/include/qemu/mmap-alloc.h b/include/qemu/mmap-alloc.h index e786266b92..3a219721e3 100644 --- a/include/qemu/mmap-alloc.h +++ b/include/qemu/mmap-alloc.h @@ -7,11 +7,13 @@ size_t qemu_fd_getpagesize(int fd); size_t qemu_mempath_getpagesize(const char *mem_path); /** - * qemu_ram_mmap: mmap the specified file or device. + * qemu_ram_mmap_resizable: reserve a memory region of @max_size to mmap the + * specified file or device and mmap @size of it. * * Parameters: * @fd: the file or the device to mmap * @size: the number of bytes to be mmaped + * @max_size: the number of bytes to be reserved * @align: if not zero, specify the alignment of the starting mapping address; * otherwise, the alignment in use will be determined by QEMU. * @shared: map has RAM_SHARED flag. @@ -21,12 +23,15 @@ size_t qemu_mempath_getpagesize(const char *mem_path); * On success, return a pointer to the mapped area. * On failure, return MAP_FAILED. */ -void *qemu_ram_mmap(int fd, - size_t size, - size_t align, - bool shared, - bool is_pmem); - -void qemu_ram_munmap(int fd, void *ptr, size_t size); +void *qemu_ram_mmap_resizable(int fd, size_t size, size_t max_size, + size_t align, bool shared, bool is_pmem); +bool qemu_ram_mmap_resize(void *ptr, int fd, size_t old_size, size_t new_size, + bool shared, bool is_pmem); +static inline void *qemu_ram_mmap(int fd, size_t size, size_t align, + bool shared, bool is_pmem) +{ + return qemu_ram_mmap_resizable(fd, size, size, align, shared, is_pmem); +} +void qemu_ram_munmap(int fd, void *ptr, size_t max_size); #endif diff --git a/util/mmap-alloc.c b/util/mmap-alloc.c index fb7ef588fe..164b88a088 100644 --- a/util/mmap-alloc.c +++ b/util/mmap-alloc.c @@ -173,23 +173,22 @@ static inline size_t mmap_pagesize(int fd) #endif } -void *qemu_ram_mmap(int fd, - size_t size, - size_t align, - bool shared, - bool is_pmem) +void *qemu_ram_mmap_resizable(int fd, size_t size, size_t max_size, + size_t align, bool shared, bool is_pmem) { const size_t pagesize = mmap_pagesize(fd); size_t offset, total; void *ptr, *guardptr; g_assert(QEMU_IS_ALIGNED(size, pagesize)); + g_assert(QEMU_IS_ALIGNED(max_size, pagesize)); /* * Note: this always allocates at least one extra page of virtual address - * space, even if size is already aligned. + * space, even if the size is already aligned. We will reserve an area of + * at least max_size, but only populate the requested part of it. */ - total = size + align; + total = max_size + align; guardptr = mmap_reserve(0, total, fd); if (guardptr == MAP_FAILED) { @@ -217,21 +216,40 @@ void *qemu_ram_mmap(int fd, * a guard page guarding against potential buffer overflows. */ total -= offset; - if (total > size + pagesize) { - munmap(ptr + size + pagesize, total - size - pagesize); + if (total > max_size + pagesize) { + munmap(ptr + max_size + pagesize, total - max_size - pagesize); } return ptr; } -void qemu_ram_munmap(int fd, void *ptr, size_t size) +bool qemu_ram_mmap_resize(void *ptr, int fd, size_t old_size, size_t new_size, + bool shared, bool is_pmem) { const size_t pagesize = mmap_pagesize(fd); - g_assert(QEMU_IS_ALIGNED(size, pagesize)); + g_assert(QEMU_IS_ALIGNED(old_size, pagesize)); + g_assert(QEMU_IS_ALIGNED(new_size, pagesize)); + + if (old_size < new_size) { + /* populate the missing piece into the reserved area */ + ptr = mmap_populate(ptr + old_size, new_size - old_size, fd, old_size, + shared, is_pmem); + } else if (old_size > new_size) { + /* discard this piece, marking it reserved */ + ptr = mmap_reserve(ptr + new_size, old_size - new_size, fd); + } + return ptr != MAP_FAILED; +} + +void qemu_ram_munmap(int fd, void *ptr, size_t max_size) +{ + const size_t pagesize = mmap_pagesize(fd); + + g_assert(QEMU_IS_ALIGNED(max_size, pagesize)); if (ptr) { /* Unmap both the RAM block and the guard page */ - munmap(ptr, size + pagesize); + munmap(ptr, max_size + pagesize); } } From patchwork Wed Feb 12 13:42:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11378619 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4EBFE1395 for ; Wed, 12 Feb 2020 13:56:55 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1495220724 for ; Wed, 12 Feb 2020 13:56:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="A/ITxfTn" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1495220724 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38290 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sVa-000500-7f for patchwork-qemu-devel@patchwork.kernel.org; Wed, 12 Feb 2020 08:56:54 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39519) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sJL-00083T-Tj for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:17 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1sJJ-0004ey-P0 for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:15 -0500 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:33770 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j1sJJ-0004eZ-KV for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:13 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581515053; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZyIsCGbp0q7rUNVxi3tQZc/wfPXKE8Dd+matyQg6+Kw=; b=A/ITxfTnEctdIV+rpWv0h6CVznAJDByfi8Zqum0UvUoZ/O0V7NknFkPgrOoufudiCKfjc/ s7oXq6rnpOedPmzRb/8VIOXJxMbqePY9jIeyVrTyQAroRXRqrXIyyWQhiv/tbJgcSNOSMP seCy5ubcLL5kOrzPq3Fua9XHeYVbb7o= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-431-JcoS9hQuPluo-HF9K9lF9Q-1; Wed, 12 Feb 2020 08:44:11 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B44478017CC; Wed, 12 Feb 2020 13:44:09 +0000 (UTC) Received: from t480s.redhat.com (ovpn-117-92.ams2.redhat.com [10.36.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id A21F71001281; Wed, 12 Feb 2020 13:44:04 +0000 (UTC) From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v2 fixed 13/16] numa: Teach ram block notifiers about resizable ram blocks Date: Wed, 12 Feb 2020 14:42:51 +0100 Message-Id: <20200212134254.11073-14-david@redhat.com> In-Reply-To: <20200212134254.11073-1-david@redhat.com> References: <20200212134254.11073-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: JcoS9hQuPluo-HF9K9lF9Q-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 207.211.31.81 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Eduardo Habkost , "Michael S . Tsirkin" , David Hildenbrand , "Dr . David Alan Gilbert" , Paul Durrant , xen-devel@lists.xenproject.org, Igor Mammedov , Anthony Perard , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We want to actually resize ram blocks (make everything between used_length and max_length inaccessible) - however, not all ram block notifiers will support that. Let's teach the notifier that ram blocks are indeed resizable, but keep using max_size in the existing notifiers. Supply the max_size when adding and removing ram blocks. Also, notify on resizes. Introduce a way to detect if any registered notifier does not support resizes - ram_block_notifiers_support_resize() - which we can later use to fallback to legacy handling if a registered notifier (esp., SEV and HAX) does not support actual resizes. Cc: Richard Henderson Cc: Paolo Bonzini Cc: "Dr. David Alan Gilbert" Cc: Eduardo Habkost Cc: Marcel Apfelbaum Cc: Stefano Stabellini Cc: Anthony Perard Cc: Paul Durrant Cc: "Michael S. Tsirkin" Cc: xen-devel@lists.xenproject.org Cc: Igor Mammedov Signed-off-by: David Hildenbrand Acked-by: Paul Durrant --- exec.c | 13 +++++++++++-- hw/core/numa.c | 34 +++++++++++++++++++++++++++++----- hw/i386/xen/xen-mapcache.c | 7 ++++--- include/exec/ramlist.h | 14 ++++++++++---- target/i386/hax-mem.c | 5 +++-- target/i386/sev.c | 18 ++++++++++-------- util/vfio-helpers.c | 17 +++++++++-------- 7 files changed, 76 insertions(+), 32 deletions(-) diff --git a/exec.c b/exec.c index fc65c4f7ca..f2d30479b8 100644 --- a/exec.c +++ b/exec.c @@ -2139,6 +2139,8 @@ static void qemu_ram_apply_settings(void *host, size_t length) */ int qemu_ram_resize(RAMBlock *block, ram_addr_t newsize, Error **errp) { + const ram_addr_t oldsize = block->used_length; + assert(block); newsize = HOST_PAGE_ALIGN(newsize); @@ -2167,6 +2169,11 @@ int qemu_ram_resize(RAMBlock *block, ram_addr_t newsize, Error **errp) block->used_length = newsize; cpu_physical_memory_set_dirty_range(block->offset, block->used_length, DIRTY_CLIENTS_ALL); + + if (block->host) { + ram_block_notify_resized(block->host, oldsize, newsize); + } + memory_region_set_size(block->mr, newsize); if (block->resized) { block->resized(block->idstr, newsize, block->host); @@ -2319,7 +2326,8 @@ static void ram_block_add(RAMBlock *new_block, Error **errp) if (new_block->host) { qemu_ram_apply_settings(new_block->host, new_block->max_length); - ram_block_notify_add(new_block->host, new_block->max_length); + ram_block_notify_add(new_block->host, new_block->used_length, + new_block->max_length); } } @@ -2502,7 +2510,8 @@ void qemu_ram_free(RAMBlock *block) } if (block->host) { - ram_block_notify_remove(block->host, block->max_length); + ram_block_notify_remove(block->host, block->used_length, + block->max_length); } qemu_mutex_lock_ramlist(); diff --git a/hw/core/numa.c b/hw/core/numa.c index 6599c69e05..5b20dc726d 100644 --- a/hw/core/numa.c +++ b/hw/core/numa.c @@ -902,11 +902,12 @@ void query_numa_node_mem(NumaNodeMem node_mem[], MachineState *ms) static int ram_block_notify_add_single(RAMBlock *rb, void *opaque) { const ram_addr_t max_size = qemu_ram_get_max_length(rb); + const ram_addr_t size = qemu_ram_get_used_length(rb); void *host = qemu_ram_get_host_addr(rb); RAMBlockNotifier *notifier = opaque; if (host) { - notifier->ram_block_added(notifier, host, max_size); + notifier->ram_block_added(notifier, host, size, max_size); } return 0; } @@ -923,20 +924,43 @@ void ram_block_notifier_remove(RAMBlockNotifier *n) QLIST_REMOVE(n, next); } -void ram_block_notify_add(void *host, size_t size) +void ram_block_notify_add(void *host, size_t size, size_t max_size) { RAMBlockNotifier *notifier; QLIST_FOREACH(notifier, &ram_list.ramblock_notifiers, next) { - notifier->ram_block_added(notifier, host, size); + notifier->ram_block_added(notifier, host, size, max_size); } } -void ram_block_notify_remove(void *host, size_t size) +void ram_block_notify_remove(void *host, size_t size, size_t max_size) { RAMBlockNotifier *notifier; QLIST_FOREACH(notifier, &ram_list.ramblock_notifiers, next) { - notifier->ram_block_removed(notifier, host, size); + notifier->ram_block_removed(notifier, host, size, max_size); } } + +void ram_block_notify_resized(void *host, size_t old_size, size_t new_size) +{ + RAMBlockNotifier *notifier; + + QLIST_FOREACH(notifier, &ram_list.ramblock_notifiers, next) { + if (notifier->ram_block_resized) { + notifier->ram_block_resized(notifier, host, old_size, new_size); + } + } +} + +bool ram_block_notifiers_support_resize(void) +{ + RAMBlockNotifier *notifier; + + QLIST_FOREACH(notifier, &ram_list.ramblock_notifiers, next) { + if (!notifier->ram_block_resized) { + return false; + } + } + return true; +} diff --git a/hw/i386/xen/xen-mapcache.c b/hw/i386/xen/xen-mapcache.c index 5b120ed44b..d6dcea65d1 100644 --- a/hw/i386/xen/xen-mapcache.c +++ b/hw/i386/xen/xen-mapcache.c @@ -169,7 +169,8 @@ static void xen_remap_bucket(MapCacheEntry *entry, if (entry->vaddr_base != NULL) { if (!(entry->flags & XEN_MAPCACHE_ENTRY_DUMMY)) { - ram_block_notify_remove(entry->vaddr_base, entry->size); + ram_block_notify_remove(entry->vaddr_base, entry->size, + entry->size); } if (munmap(entry->vaddr_base, entry->size) != 0) { perror("unmap fails"); @@ -211,7 +212,7 @@ static void xen_remap_bucket(MapCacheEntry *entry, } if (!(entry->flags & XEN_MAPCACHE_ENTRY_DUMMY)) { - ram_block_notify_add(vaddr_base, size); + ram_block_notify_add(vaddr_base, size, size); } entry->vaddr_base = vaddr_base; @@ -452,7 +453,7 @@ static void xen_invalidate_map_cache_entry_unlocked(uint8_t *buffer) } pentry->next = entry->next; - ram_block_notify_remove(entry->vaddr_base, entry->size); + ram_block_notify_remove(entry->vaddr_base, entry->size, entry->size); if (munmap(entry->vaddr_base, entry->size) != 0) { perror("unmap fails"); exit(-1); diff --git a/include/exec/ramlist.h b/include/exec/ramlist.h index bc4faa1b00..92e548461e 100644 --- a/include/exec/ramlist.h +++ b/include/exec/ramlist.h @@ -65,15 +65,21 @@ void qemu_mutex_lock_ramlist(void); void qemu_mutex_unlock_ramlist(void); struct RAMBlockNotifier { - void (*ram_block_added)(RAMBlockNotifier *n, void *host, size_t size); - void (*ram_block_removed)(RAMBlockNotifier *n, void *host, size_t size); + void (*ram_block_added)(RAMBlockNotifier *n, void *host, size_t size, + size_t max_size); + void (*ram_block_removed)(RAMBlockNotifier *n, void *host, size_t size, + size_t max_size); + void (*ram_block_resized)(RAMBlockNotifier *n, void *host, size_t old_size, + size_t new_size); QLIST_ENTRY(RAMBlockNotifier) next; }; void ram_block_notifier_add(RAMBlockNotifier *n); void ram_block_notifier_remove(RAMBlockNotifier *n); -void ram_block_notify_add(void *host, size_t size); -void ram_block_notify_remove(void *host, size_t size); +void ram_block_notify_add(void *host, size_t size, size_t max_size); +void ram_block_notify_remove(void *host, size_t size, size_t max_size); +void ram_block_notify_resized(void *host, size_t old_size, size_t new_size); +bool ram_block_notifiers_support_resize(void); void ram_block_dump(Monitor *mon); diff --git a/target/i386/hax-mem.c b/target/i386/hax-mem.c index 6bb5a24917..454d7fb212 100644 --- a/target/i386/hax-mem.c +++ b/target/i386/hax-mem.c @@ -293,7 +293,8 @@ static MemoryListener hax_memory_listener = { .priority = 10, }; -static void hax_ram_block_added(RAMBlockNotifier *n, void *host, size_t size) +static void hax_ram_block_added(RAMBlockNotifier *n, void *host, size_t size, + size_t max_size) { /* * We must register each RAM block with the HAXM kernel module, or @@ -304,7 +305,7 @@ static void hax_ram_block_added(RAMBlockNotifier *n, void *host, size_t size) * host physical pages for the RAM block as part of this registration * process, hence the name hax_populate_ram(). */ - if (hax_populate_ram((uint64_t)(uintptr_t)host, size) < 0) { + if (hax_populate_ram((uint64_t)(uintptr_t)host, max_size) < 0) { fprintf(stderr, "HAX failed to populate RAM\n"); abort(); } diff --git a/target/i386/sev.c b/target/i386/sev.c index 024bb24e51..6b4cee24a2 100644 --- a/target/i386/sev.c +++ b/target/i386/sev.c @@ -129,7 +129,8 @@ sev_set_guest_state(SevState new_state) } static void -sev_ram_block_added(RAMBlockNotifier *n, void *host, size_t size) +sev_ram_block_added(RAMBlockNotifier *n, void *host, size_t size, + size_t max_size) { int r; struct kvm_enc_region range; @@ -146,19 +147,20 @@ sev_ram_block_added(RAMBlockNotifier *n, void *host, size_t size) } range.addr = (__u64)(unsigned long)host; - range.size = size; + range.size = max_size; - trace_kvm_memcrypt_register_region(host, size); + trace_kvm_memcrypt_register_region(host, max_size); r = kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_REG_REGION, &range); if (r) { error_report("%s: failed to register region (%p+%#zx) error '%s'", - __func__, host, size, strerror(errno)); + __func__, host, max_size, strerror(errno)); exit(1); } } static void -sev_ram_block_removed(RAMBlockNotifier *n, void *host, size_t size) +sev_ram_block_removed(RAMBlockNotifier *n, void *host, size_t size, + size_t max_size) { int r; struct kvm_enc_region range; @@ -175,13 +177,13 @@ sev_ram_block_removed(RAMBlockNotifier *n, void *host, size_t size) } range.addr = (__u64)(unsigned long)host; - range.size = size; + range.size = max_size; - trace_kvm_memcrypt_unregister_region(host, size); + trace_kvm_memcrypt_unregister_region(host, max_size); r = kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_UNREG_REGION, &range); if (r) { error_report("%s: failed to unregister region (%p+%#zx)", - __func__, host, size); + __func__, host, max_size); } } diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c index b3adc328db..3db6aa49f4 100644 --- a/util/vfio-helpers.c +++ b/util/vfio-helpers.c @@ -372,25 +372,26 @@ fail_container: return ret; } -static void qemu_vfio_ram_block_added(RAMBlockNotifier *n, - void *host, size_t size) +static void qemu_vfio_ram_block_added(RAMBlockNotifier *n, void *host, + size_t size, size_t max_size) { QEMUVFIOState *s = container_of(n, QEMUVFIOState, ram_notifier); int ret; - trace_qemu_vfio_ram_block_added(s, host, size); - ret = qemu_vfio_dma_map(s, host, size, false, NULL); + trace_qemu_vfio_ram_block_added(s, host, max_size); + ret = qemu_vfio_dma_map(s, host, max_size, false, NULL); if (ret) { - error_report("qemu_vfio_dma_map(%p, %zu) failed: %d", host, size, ret); + error_report("qemu_vfio_dma_map(%p, %zu) failed: %d", host, + max_size, ret); } } -static void qemu_vfio_ram_block_removed(RAMBlockNotifier *n, - void *host, size_t size) +static void qemu_vfio_ram_block_removed(RAMBlockNotifier *n, void *host, + size_t size, size_t max_size) { QEMUVFIOState *s = container_of(n, QEMUVFIOState, ram_notifier); if (host) { - trace_qemu_vfio_ram_block_removed(s, host, size); + trace_qemu_vfio_ram_block_removed(s, host, max_size); qemu_vfio_dma_unmap(s, host); } } From patchwork Wed Feb 12 13:42:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11378611 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8757413A4 for ; Wed, 12 Feb 2020 13:54:31 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5DAB72082F for ; Wed, 12 Feb 2020 13:54:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LgvM4Tml" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5DAB72082F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38232 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sTG-0001Oy-JO for patchwork-qemu-devel@patchwork.kernel.org; Wed, 12 Feb 2020 08:54:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39575) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sJd-0000M5-Uu for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:35 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1sJc-0004xJ-Bh for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:33 -0500 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:21992 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j1sJc-0004wu-6S for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581515071; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mplzcwPG73COXHnghBhE4N6/huJASTrDghs1zBoH9YM=; b=LgvM4Tml7TJC9B4Q8ltHs0+cUAc6zy4VH8O8JvBcHt5dWVyTUI/B2Oi+JP23EQ4WBn5jwx oep2yuPwP7mK9wZYecL/PWCFFVtfA/1w5b+o6mivhKvmUHx8xdpzIEiShUnxJ73ucifzs5 UXn1BJVwTfD4otNDoGZC+RB7LhbBphk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-312-XCFNyH4ONouLpBJHwJnMag-1; Wed, 12 Feb 2020 08:44:28 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 47CF510054E3; Wed, 12 Feb 2020 13:44:27 +0000 (UTC) Received: from t480s.redhat.com (ovpn-117-92.ams2.redhat.com [10.36.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0F02F10027A8; Wed, 12 Feb 2020 13:44:09 +0000 (UTC) From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v2 fixed 14/16] util: vfio-helpers: Implement ram_block_resized() Date: Wed, 12 Feb 2020 14:42:52 +0100 Message-Id: <20200212134254.11073-15-david@redhat.com> In-Reply-To: <20200212134254.11073-1-david@redhat.com> References: <20200212134254.11073-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: XCFNyH4ONouLpBJHwJnMag-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S . Tsirkin" , David Hildenbrand , Stefan Hajnoczi , "Dr . David Alan Gilbert" , Alex Williamson , Igor Mammedov , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Let's implement ram_block_resized(), allowing resizable mappings. For resizable mappings, we reserve $max_size IOVA address space, but only map $size of it. When resizing, unmap the old part and remap the new part. We'll need a new ioctl to do this atomically (e.g., to resize while the guest is running - not allowed for now). Cc: Richard Henderson Cc: Paolo Bonzini Cc: Eduardo Habkost Cc: Marcel Apfelbaum Cc: Alex Williamson Cc: Stefan Hajnoczi Cc: "Dr. David Alan Gilbert" Signed-off-by: David Hildenbrand --- util/trace-events | 5 ++-- util/vfio-helpers.c | 70 ++++++++++++++++++++++++++++++++++++++++----- 2 files changed, 66 insertions(+), 9 deletions(-) diff --git a/util/trace-events b/util/trace-events index 83b6639018..88b7dbf4a5 100644 --- a/util/trace-events +++ b/util/trace-events @@ -74,8 +74,9 @@ qemu_mutex_unlock(void *mutex, const char *file, const int line) "released mutex # vfio-helpers.c qemu_vfio_dma_reset_temporary(void *s) "s %p" -qemu_vfio_ram_block_added(void *s, void *p, size_t size) "s %p host %p size 0x%zx" -qemu_vfio_ram_block_removed(void *s, void *p, size_t size) "s %p host %p size 0x%zx" +qemu_vfio_ram_block_added(void *s, void *p, size_t size, size_t max_size) "s %p host %p size 0x%zx max_size 0x%zx" +qemu_vfio_ram_block_removed(void *s, void *p, size_t size, size_t max_size) "s %p host %p size 0x%zx max_size 0x%zx" +qemu_vfio_ram_block_resized(void *s, void *p, size_t old_size, size_t new_sizze) "s %p host %p old_size 0x%zx new_size 0x%zx" qemu_vfio_find_mapping(void *s, void *p) "s %p host %p" qemu_vfio_new_mapping(void *s, void *host, size_t size, int index, uint64_t iova) "s %p host %p size %zu index %d iova 0x%"PRIx64 qemu_vfio_do_mapping(void *s, void *host, size_t size, uint64_t iova) "s %p host %p size %zu iova 0x%"PRIx64 diff --git a/util/vfio-helpers.c b/util/vfio-helpers.c index 3db6aa49f4..70877b9ebd 100644 --- a/util/vfio-helpers.c +++ b/util/vfio-helpers.c @@ -372,14 +372,20 @@ fail_container: return ret; } +static int qemu_vfio_dma_map_resizable(QEMUVFIOState *s, void *host, + size_t size, size_t max_size, + bool temporary, uint64_t *iova); +static void qemu_vfio_dma_map_resize(QEMUVFIOState *s, void *host, + size_t old_size, size_t new_size); + static void qemu_vfio_ram_block_added(RAMBlockNotifier *n, void *host, size_t size, size_t max_size) { QEMUVFIOState *s = container_of(n, QEMUVFIOState, ram_notifier); int ret; - trace_qemu_vfio_ram_block_added(s, host, max_size); - ret = qemu_vfio_dma_map(s, host, max_size, false, NULL); + trace_qemu_vfio_ram_block_added(s, host, size, max_size); + ret = qemu_vfio_dma_map_resizable(s, host, size, max_size, false, NULL); if (ret) { error_report("qemu_vfio_dma_map(%p, %zu) failed: %d", host, max_size, ret); @@ -391,16 +397,28 @@ static void qemu_vfio_ram_block_removed(RAMBlockNotifier *n, void *host, { QEMUVFIOState *s = container_of(n, QEMUVFIOState, ram_notifier); if (host) { - trace_qemu_vfio_ram_block_removed(s, host, max_size); + trace_qemu_vfio_ram_block_removed(s, host, size, max_size); qemu_vfio_dma_unmap(s, host); } } +static void qemu_vfio_ram_block_resized(RAMBlockNotifier *n, void *host, + size_t old_size, size_t new_size) +{ + QEMUVFIOState *s = container_of(n, QEMUVFIOState, ram_notifier); + + if (host) { + trace_qemu_vfio_ram_block_resized(s, host, old_size, new_size); + qemu_vfio_dma_map_resize(s, host, old_size, new_size); + } +} + static void qemu_vfio_open_common(QEMUVFIOState *s) { qemu_mutex_init(&s->lock); s->ram_notifier.ram_block_added = qemu_vfio_ram_block_added; s->ram_notifier.ram_block_removed = qemu_vfio_ram_block_removed; + s->ram_notifier.ram_block_resized = qemu_vfio_ram_block_resized; s->low_water_mark = QEMU_VFIO_IOVA_MIN; s->high_water_mark = QEMU_VFIO_IOVA_MAX; ram_block_notifier_add(&s->ram_notifier); @@ -597,9 +615,14 @@ static bool qemu_vfio_verify_mappings(QEMUVFIOState *s) * the result in @iova if not NULL. The caller need to make sure the area is * aligned to page size, and mustn't overlap with existing mapping areas (split * mapping status within this area is not allowed). + * + * If size < max_size, a region of max_size in IOVA address is reserved, such + * that the mapping can later be resized. Resizable mappings are only allowed + * for !temporary mappings. */ -int qemu_vfio_dma_map(QEMUVFIOState *s, void *host, size_t size, - bool temporary, uint64_t *iova) +static int qemu_vfio_dma_map_resizable(QEMUVFIOState *s, void *host, + size_t size, size_t max_size, + bool temporary, uint64_t *iova) { int ret = 0; int index; @@ -608,13 +631,17 @@ int qemu_vfio_dma_map(QEMUVFIOState *s, void *host, size_t size, assert(QEMU_PTR_IS_ALIGNED(host, qemu_real_host_page_size)); assert(QEMU_IS_ALIGNED(size, qemu_real_host_page_size)); + assert(QEMU_IS_ALIGNED(max_size, qemu_real_host_page_size)); + assert(size == max_size || !temporary); + assert(size <= max_size); + trace_qemu_vfio_dma_map(s, host, size, temporary, iova); qemu_mutex_lock(&s->lock); mapping = qemu_vfio_find_mapping(s, host, &index); if (mapping) { iova0 = mapping->iova + ((uint8_t *)host - (uint8_t *)mapping->host); } else { - if (s->high_water_mark - s->low_water_mark + 1 < size) { + if (s->high_water_mark - s->low_water_mark + 1 < max_size) { ret = -ENOMEM; goto out; } @@ -631,7 +658,7 @@ int qemu_vfio_dma_map(QEMUVFIOState *s, void *host, size_t size, qemu_vfio_remove_mapping(s, mapping); goto out; } - s->low_water_mark += size; + s->low_water_mark += max_size; qemu_vfio_dump_mappings(s); } else { iova0 = s->high_water_mark - size; @@ -650,6 +677,12 @@ out: return ret; } +int qemu_vfio_dma_map(QEMUVFIOState *s, void *host, size_t size, + bool temporary, uint64_t *iova) +{ + return qemu_vfio_dma_map_resizable(s, host, size, size, temporary, iova); +} + /* Reset the high watermark and free all "temporary" mappings. */ int qemu_vfio_dma_reset_temporary(QEMUVFIOState *s) { @@ -694,6 +727,29 @@ out: qemu_mutex_unlock(&s->lock); } +static void qemu_vfio_dma_map_resize(QEMUVFIOState *s, void *host, + size_t old_size, size_t new_size) +{ + IOVAMapping *m; + int index = 0; + + qemu_mutex_lock(&s->lock); + m = qemu_vfio_find_mapping(s, host, &index); + if (!m) { + return; + } + assert(m->size == old_size); + + /* Note: Not atomic - we need a new ioctl for that. */ + qemu_vfio_undo_mapping(s, m->iova, m->size); + qemu_vfio_do_mapping(s, host, m->iova, new_size); + + m->size = new_size; + assert(qemu_vfio_verify_mappings(s)); + + qemu_mutex_unlock(&s->lock); +} + static void qemu_vfio_reset(QEMUVFIOState *s) { ioctl(s->device, VFIO_DEVICE_RESET); From patchwork Wed Feb 12 13:42:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11378583 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 37B5813A4 for ; Wed, 12 Feb 2020 13:46:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0D6A8206B6 for ; Wed, 12 Feb 2020 13:46:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="W50PBK27" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0D6A8206B6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38080 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sLb-0002vO-2f for patchwork-qemu-devel@patchwork.kernel.org; Wed, 12 Feb 2020 08:46:35 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39614) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sJs-0000ue-9i for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:49 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1sJq-00059V-V5 for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:48 -0500 Received: from us-smtp-delivery-1.mimecast.com ([205.139.110.120]:31646 helo=us-smtp-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j1sJq-000596-RN for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581515086; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dvyH6dpXKalWf9UO+cg9MYwefzVgUpzVUOYn5jFtHs0=; b=W50PBK27SdZz4RMK4jI1Jw/JfFRejph1aJnkUoafVAGqkCEXcP+dMYp2a/nA9DTzGDn6gy f7CR9j8tITHJV1F0HdsfmuBBBJid/gn670ij844qRu4mHA5nVdTLziVlhmF6f97KW/irOi a72nwdUrR611jXfDGef0sJOu5xVioww= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-364-2vk34nHQP2-CsQY7Iphvxw-1; Wed, 12 Feb 2020 08:44:45 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E7671800D55; Wed, 12 Feb 2020 13:44:43 +0000 (UTC) Received: from t480s.redhat.com (ovpn-117-92.ams2.redhat.com [10.36.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id 96D771001B09; Wed, 12 Feb 2020 13:44:27 +0000 (UTC) From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v2 fixed 15/16] util: oslib: Resizable anonymous allocations under POSIX Date: Wed, 12 Feb 2020 14:42:53 +0100 Message-Id: <20200212134254.11073-16-david@redhat.com> In-Reply-To: <20200212134254.11073-1-david@redhat.com> References: <20200212134254.11073-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: 2vk34nHQP2-CsQY7Iphvxw-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.120 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S . Tsirkin" , Stefan Weil , David Hildenbrand , "Dr . David Alan Gilbert" , Igor Mammedov , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" Introduce qemu_anon_ram_alloc_resizable() and qemu_anon_ram_resize(). Implement them under POSIX and make them return NULL under WIN32. Under POSIX, we make use of resizable mmaps. An implementation under WIN32 is theoretically possible AFAIK and can be added later. In qemu_anon_ram_free(), rename the size parameter to max_size, to make it clearer that we have to use the maximum size when freeing resizable anonymous allocations. Cc: Richard Henderson Cc: Paolo Bonzini Cc: "Dr. David Alan Gilbert" Cc: Eduardo Habkost Cc: Marcel Apfelbaum Cc: Stefan Weil Cc: Igor Mammedov Signed-off-by: David Hildenbrand --- include/qemu/osdep.h | 6 +++++- util/oslib-posix.c | 37 ++++++++++++++++++++++++++++++++++--- util/oslib-win32.c | 14 ++++++++++++++ util/trace-events | 4 +++- 4 files changed, 56 insertions(+), 5 deletions(-) diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h index 9bd3dcfd13..84c54c1647 100644 --- a/include/qemu/osdep.h +++ b/include/qemu/osdep.h @@ -311,8 +311,12 @@ int qemu_daemon(int nochdir, int noclose); void *qemu_try_memalign(size_t alignment, size_t size); void *qemu_memalign(size_t alignment, size_t size); void *qemu_anon_ram_alloc(size_t size, uint64_t *align, bool shared); +void *qemu_anon_ram_alloc_resizable(size_t size, size_t max_size, + uint64_t *align, bool shared); +bool qemu_anon_ram_resize(void *ptr, size_t old_size, size_t new_size, + bool shared); void qemu_vfree(void *ptr); -void qemu_anon_ram_free(void *ptr, size_t size); +void qemu_anon_ram_free(void *ptr, size_t max_size); #define QEMU_MADV_INVALID -1 diff --git a/util/oslib-posix.c b/util/oslib-posix.c index 5a291cc982..147246d543 100644 --- a/util/oslib-posix.c +++ b/util/oslib-posix.c @@ -219,16 +219,47 @@ void *qemu_anon_ram_alloc(size_t size, uint64_t *alignment, bool shared) return ptr; } +void *qemu_anon_ram_alloc_resizable(size_t size, size_t max_size, + uint64_t *alignment, bool shared) +{ + size_t align = QEMU_VMALLOC_ALIGN; + void *ptr = qemu_ram_mmap_resizable(-1, size, max_size, align, shared, + false); + + if (ptr == MAP_FAILED) { + return NULL; + } + + if (alignment) { + *alignment = align; + } + + trace_qemu_anon_ram_alloc_resizable(size, max_size, ptr); + return ptr; +} + +bool qemu_anon_ram_resize(void *ptr, size_t old_size, size_t new_size, + bool shared) +{ + bool resized = qemu_ram_mmap_resize(ptr, -1, old_size, new_size, shared, + false); + + if (resized) { + trace_qemu_anon_ram_resize(old_size, new_size, ptr); + } + return resized; +} + void qemu_vfree(void *ptr) { trace_qemu_vfree(ptr); free(ptr); } -void qemu_anon_ram_free(void *ptr, size_t size) +void qemu_anon_ram_free(void *ptr, size_t max_size) { - trace_qemu_anon_ram_free(ptr, size); - qemu_ram_munmap(-1, ptr, size); + trace_qemu_anon_ram_free(ptr, max_size); + qemu_ram_munmap(-1, ptr, max_size); } void qemu_set_block(int fd) diff --git a/util/oslib-win32.c b/util/oslib-win32.c index e9b14ab178..5ba872bd3b 100644 --- a/util/oslib-win32.c +++ b/util/oslib-win32.c @@ -90,6 +90,20 @@ void *qemu_anon_ram_alloc(size_t size, uint64_t *align, bool shared) return ptr; } +void *qemu_anon_ram_alloc_resizable(size_t size, size_t max_size, + uint64_t *align, bool shared) +{ + /* resizable ram not implemented yet */ + return NULL; +} + +bool qemu_anon_ram_resize(void *ptr, size_t old_size, size_t new_size, + bool shared) +{ + /* resizable ram not implemented yet */ + return false; +} + void qemu_vfree(void *ptr) { trace_qemu_vfree(ptr); diff --git a/util/trace-events b/util/trace-events index 88b7dbf4a5..8f44dcc1a0 100644 --- a/util/trace-events +++ b/util/trace-events @@ -46,8 +46,10 @@ qemu_co_mutex_unlock_return(void *mutex, void *self) "mutex %p self %p" # oslib-posix.c qemu_memalign(size_t alignment, size_t size, void *ptr) "alignment %zu size %zu ptr %p" qemu_anon_ram_alloc(size_t size, void *ptr) "size %zu ptr %p" +qemu_anon_ram_alloc_resizable(size_t size, size_t max_size, void *ptr) "size %zu max_size %zu ptr %p" +qemu_anon_ram_resize(size_t old_size, size_t new_size, void *ptr) "old_size %zu new_size %zu ptr %p" qemu_vfree(void *ptr) "ptr %p" -qemu_anon_ram_free(void *ptr, size_t size) "ptr %p size %zu" +qemu_anon_ram_free(void *ptr, size_t max_size) "ptr %p max_size %zu" # hbitmap.c hbitmap_iter_skip_words(const void *hb, void *hbi, uint64_t pos, unsigned long cur) "hb %p hbi %p pos %"PRId64" cur 0x%lx" From patchwork Wed Feb 12 13:42:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 11378617 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7AD3B138D for ; Wed, 12 Feb 2020 13:56:00 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5256C2082F for ; Wed, 12 Feb 2020 13:56:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Ci+wOv5J" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5256C2082F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:38268 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sUh-0003Xp-IC for patchwork-qemu-devel@patchwork.kernel.org; Wed, 12 Feb 2020 08:55:59 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:39639) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j1sJv-00013B-GO for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:53 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j1sJt-0005CK-OY for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:51 -0500 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:25766 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j1sJt-0005Br-KC for qemu-devel@nongnu.org; Wed, 12 Feb 2020 08:44:49 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1581515089; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=paiODFbGD5fd47o3hrnj9jzAWs9R0kAqAVdxlWzm1Ig=; b=Ci+wOv5JiUuWZzudZDyxGRsmHQfZE+DcrcidU0OEfIH3z9xZ/uCRb17ACS7Qp8wo5zMwbM rF4CT4MTxVm4erteL0JuqDIZ2oMp0htWKbGO8VCFUHyd/24QsvVSsoHKx55fU+zT/XJH39 FM/kPUW5JsDtXAfHkqfO3bft6ew+iPw= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-268-HNNGAaqFMSKuyypWtD84UA-1; Wed, 12 Feb 2020 08:44:47 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8226EDBCC; Wed, 12 Feb 2020 13:44:46 +0000 (UTC) Received: from t480s.redhat.com (ovpn-117-92.ams2.redhat.com [10.36.117.92]) by smtp.corp.redhat.com (Postfix) with ESMTP id 419961001281; Wed, 12 Feb 2020 13:44:44 +0000 (UTC) From: David Hildenbrand To: qemu-devel@nongnu.org Subject: [PATCH v2 fixed 16/16] exec: Ram blocks with resizable anonymous allocations under POSIX Date: Wed, 12 Feb 2020 14:42:54 +0100 Message-Id: <20200212134254.11073-17-david@redhat.com> In-Reply-To: <20200212134254.11073-1-david@redhat.com> References: <20200212134254.11073-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: HNNGAaqFMSKuyypWtD84UA-1 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 205.139.110.61 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "Michael S . Tsirkin" , Stefan Weil , David Hildenbrand , "Dr . David Alan Gilbert" , Shameerali Kolothum Thodi , Igor Mammedov , Paolo Bonzini , Richard Henderson Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We can now make use of resizable anonymous allocations to implement actually resizable ram blocks. Resizable anonymous allocations are not implemented under WIN32 yet and are not available when using alternative allocators. Fall back to the existing handling. We also have to fallback to the existing handling in case any ram block notifier does not support resizing (esp., AMD SEV, HAX) yet. Remember in RAM_RESIZEABLE_ALLOC if we are using resizable anonymous allocations. As the mmap()-hackery will invalidate some madvise settings, we have to re-apply them after resizing. After resizing, notify the ram block notifiers. Try to grow early, as that can easily fail if out of memory. Shrink late and ignore errors (nothing will actually break). Warn only. The benefit of actually resizable ram blocks is that e.g., under Linux, only the actual size will be reserved (even if "/proc/sys/vm/overcommit_memory" is set to "never"). Additional memory will be reserved when trying to resize, which allows to have ram blocks that start small but can theoretically grow very large. Note1: We are not able to create resizable ram blocks with pre-allocated memory yet, so prealloc is not affected. Note2: mlock should work as it used to as os_mlock() does a mlockall(MCL_CURRENT | MCL_FUTURE), which includes future mappings. Note3: Nobody should access memory beyond used_length. Memory notifiers already properly take care of this, only ram block notifiers violate this constraint and, therefore, have to be special-cased. Especially, any ram block notifier that might dynamically register at runtime (e.g., vfio), has to support resizes. Add an assert for that. Both, HAX and SEV register early, so they are fine. Cc: Richard Henderson Cc: Paolo Bonzini Cc: "Dr. David Alan Gilbert" Cc: Eduardo Habkost Cc: Marcel Apfelbaum Cc: Stefan Weil Cc: Igor Mammedov Cc: Shameerali Kolothum Thodi Signed-off-by: David Hildenbrand --- exec.c | 60 ++++++++++++++++++++++++++++++++++++--- hw/core/numa.c | 7 +++++ include/exec/cpu-common.h | 2 ++ include/exec/memory.h | 8 ++++++ 4 files changed, 73 insertions(+), 4 deletions(-) diff --git a/exec.c b/exec.c index f2d30479b8..71e32dcc11 100644 --- a/exec.c +++ b/exec.c @@ -2053,6 +2053,16 @@ void qemu_ram_unset_migratable(RAMBlock *rb) rb->flags &= ~RAM_MIGRATABLE; } +bool qemu_ram_is_resizable(RAMBlock *rb) +{ + return rb->flags & RAM_RESIZEABLE; +} + +bool qemu_ram_is_resizable_alloc(RAMBlock *rb) +{ + return rb->flags & RAM_RESIZEABLE_ALLOC; +} + /* Called with iothread lock held. */ void qemu_ram_set_idstr(RAMBlock *new_block, const char *name, DeviceState *dev) { @@ -2139,6 +2149,7 @@ static void qemu_ram_apply_settings(void *host, size_t length) */ int qemu_ram_resize(RAMBlock *block, ram_addr_t newsize, Error **errp) { + const bool shared = block->flags & RAM_SHARED; const ram_addr_t oldsize = block->used_length; assert(block); @@ -2149,7 +2160,7 @@ int qemu_ram_resize(RAMBlock *block, ram_addr_t newsize, Error **errp) return 0; } - if (!(block->flags & RAM_RESIZEABLE)) { + if (!qemu_ram_is_resizable(block)) { error_setg_errno(errp, EINVAL, "Length mismatch: %s: 0x" RAM_ADDR_FMT " in != 0x" RAM_ADDR_FMT, block->idstr, @@ -2165,6 +2176,12 @@ int qemu_ram_resize(RAMBlock *block, ram_addr_t newsize, Error **errp) return -EINVAL; } + if (oldsize < newsize && qemu_ram_is_resizable_alloc(block) && + !qemu_anon_ram_resize(block->host, oldsize, newsize, shared)) { + error_setg_errno(errp, -ENOMEM, "Cannot allocate enough memory."); + return -ENOMEM; + } + cpu_physical_memory_clear_dirty_range(block->offset, block->used_length); block->used_length = newsize; cpu_physical_memory_set_dirty_range(block->offset, block->used_length, @@ -2178,6 +2195,21 @@ int qemu_ram_resize(RAMBlock *block, ram_addr_t newsize, Error **errp) if (block->resized) { block->resized(block->idstr, newsize, block->host); } + + /* + * Shrinking will only fail in rare scenarios (e.g., maximum number of + * mappings reached), and can be ignored. Warn only. + */ + if (newsize < oldsize && qemu_ram_is_resizable_alloc(block) && + !qemu_anon_ram_resize(block->host, oldsize, newsize, shared)) { + warn_report("Shrinking memory allocation failed."); + } + + if (block->host && qemu_ram_is_resizable_alloc(block)) { + /* re-apply settings that might have been overriden by the resize */ + qemu_ram_apply_settings(block->host, block->max_length); + } + return 0; } @@ -2256,6 +2288,28 @@ static void dirty_memory_extend(ram_addr_t old_ram_size, } } +static void ram_block_alloc_ram(RAMBlock *rb) +{ + const bool shared = qemu_ram_is_shared(rb); + + /* + * If we can, try to allocate actually resizable ram. Will also fail + * if qemu_anon_ram_alloc_resizable() is not implemented. + */ + if (phys_mem_alloc == qemu_anon_ram_alloc && + qemu_ram_is_resizable(rb) && + ram_block_notifiers_support_resize()) { + rb->host = qemu_anon_ram_alloc_resizable(rb->used_length, + rb->max_length, &rb->mr->align, + shared); + if (rb->host) { + rb->flags |= RAM_RESIZEABLE_ALLOC; + return; + } + } + rb->host = phys_mem_alloc(rb->max_length, &rb->mr->align, shared); +} + static void ram_block_add(RAMBlock *new_block, Error **errp) { RAMBlock *block; @@ -2278,9 +2332,7 @@ static void ram_block_add(RAMBlock *new_block, Error **errp) return; } } else { - new_block->host = phys_mem_alloc(new_block->max_length, - &new_block->mr->align, - qemu_ram_is_shared(new_block)); + ram_block_alloc_ram(new_block); if (!new_block->host) { error_setg_errno(errp, errno, "cannot set up guest memory '%s'", diff --git a/hw/core/numa.c b/hw/core/numa.c index 5b20dc726d..601cf9f603 100644 --- a/hw/core/numa.c +++ b/hw/core/numa.c @@ -907,6 +907,13 @@ static int ram_block_notify_add_single(RAMBlock *rb, void *opaque) RAMBlockNotifier *notifier = opaque; if (host) { + /* + * Dynamically adding notifiers that don't support resizes is forbidden + * when dealing with resizable ram blocks that have actually resizable + * allocations. + */ + g_assert(!qemu_ram_is_resizable_alloc(rb) || + notifier->ram_block_resized); notifier->ram_block_added(notifier, host, size, max_size); } return 0; diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h index 9760ac9068..a9c76bd5ef 100644 --- a/include/exec/cpu-common.h +++ b/include/exec/cpu-common.h @@ -66,6 +66,8 @@ void qemu_ram_set_uf_zeroable(RAMBlock *rb); bool qemu_ram_is_migratable(RAMBlock *rb); void qemu_ram_set_migratable(RAMBlock *rb); void qemu_ram_unset_migratable(RAMBlock *rb); +bool qemu_ram_is_resizable(RAMBlock *rb); +bool qemu_ram_is_resizable_alloc(RAMBlock *rb); size_t qemu_ram_pagesize(RAMBlock *block); size_t qemu_ram_pagesize_largest(void); diff --git a/include/exec/memory.h b/include/exec/memory.h index e85b7de99a..19417943a2 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -129,6 +129,14 @@ typedef struct IOMMUNotifier IOMMUNotifier; /* RAM is a persistent kind memory */ #define RAM_PMEM (1 << 5) +/* + * Implies RAM_RESIZEABLE. Memory beyond the used_length is inaccessible + * (esp. initially and after resizing). For such memory blocks, only the + * used_length is reserved in the OS - resizing might fail. Will only be + * used with host OS support and if all ram block notifiers support resizing. + */ +#define RAM_RESIZEABLE_ALLOC (1 << 6) + static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn, IOMMUNotifierFlag flags, hwaddr start, hwaddr end,