From patchwork Wed Jan 22 15:17:28 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?C=C3=A9dric_Le_Goater?= X-Patchwork-Id: 13947425 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7FADFC0218D for ; Wed, 22 Jan 2025 15:19:37 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tacUG-0004bW-N6; Wed, 22 Jan 2025 10:17:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tacUE-0004b5-Iz for qemu-devel@nongnu.org; Wed, 22 Jan 2025 10:17:46 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tacUC-0004Y7-Cl for qemu-devel@nongnu.org; Wed, 22 Jan 2025 10:17:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1737559062; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bPCmnBBmQ21SuuJg+NeSSYHdwCKST5iZyoGTs4BTCfs=; b=CTRAzlLZlU4OJYjDQJsPbOqAYWH8sKNIS/4svl+To99+WmW5BowdofG/UphzDtQeT3mN9d DWcPlAiZcO3nVri7LSjuueXDaOi8RR9z0JqWDlKaIeKghLVaIj+Lu9s3p5ukyn8PYBptdU SYpy+/2mOk5LL/oXxKLyUA+qMiaM6Pw= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-287-PpEqxVkoOaikbenfHIt6TA-1; Wed, 22 Jan 2025 10:17:40 -0500 X-MC-Unique: PpEqxVkoOaikbenfHIt6TA-1 X-Mimecast-MFC-AGG-ID: PpEqxVkoOaikbenfHIt6TA Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3AEC01956058 for ; Wed, 22 Jan 2025 15:17:39 +0000 (UTC) Received: from corto.redhat.com (unknown [10.39.192.154]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 21C7419560AA; Wed, 22 Jan 2025 15:17:37 +0000 (UTC) From: =?utf-8?q?C=C3=A9dric_Le_Goater?= To: qemu-devel@nongnu.org Cc: Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_G?= =?utf-8?q?oater?= Subject: [PATCH 1/5] vfio/pci: Replace "iommu_device" by "vIOMMU" Date: Wed, 22 Jan 2025 16:17:28 +0100 Message-ID: <20250122151732.1351821-2-clg@redhat.com> In-Reply-To: <20250122151732.1351821-1-clg@redhat.com> References: <20250122151732.1351821-1-clg@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.996, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This is to be consistent with other reported errors related to vIOMMU devices. Signed-off-by: Cédric Le Goater --- hw/vfio/pci.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index cf14987e42bd9188d5040b51a2f84cfa959f632d..ad326839db49cf3a50524d5443ceedac66e1df3d 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3131,7 +3131,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) if (!vbasedev->mdev && !pci_device_set_iommu_device(pdev, vbasedev->hiod, errp)) { - error_prepend(errp, "Failed to set iommu_device: "); + error_prepend(errp, "Failed to set vIOMMU: "); goto out_teardown; } From patchwork Wed Jan 22 15:17:29 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?C=C3=A9dric_Le_Goater?= X-Patchwork-Id: 13947424 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6DDCCC0218C for ; Wed, 22 Jan 2025 15:19:37 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tacUG-0004bc-Sr; Wed, 22 Jan 2025 10:17:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tacUF-0004bH-Pd for qemu-devel@nongnu.org; Wed, 22 Jan 2025 10:17:47 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tacUE-0004YT-Ag for qemu-devel@nongnu.org; Wed, 22 Jan 2025 10:17:47 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1737559065; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uCH6PrnlcUl+bRd4c5qQ28sEk/fKhW973tgX711Eulk=; b=G0CbGUu/bPCjUqmFgvJsOkvI0Tgk2PlvAlbGaeB3gRxKe7K37KFkjofQtPNuT0ukbAagNb NQYMMS36an9Va+EbDqMRrhfJ+XqRThMyt7/MmmJ2Ru2AsUL46EQgivjepeHNCKtPW+eFum clpGRFFqmfbRnxuiGNbBQusCmZCgYf0= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-157-uNqx4XTTOJeTRPK0ulYp2g-1; Wed, 22 Jan 2025 10:17:42 -0500 X-MC-Unique: uNqx4XTTOJeTRPK0ulYp2g-1 X-Mimecast-MFC-AGG-ID: uNqx4XTTOJeTRPK0ulYp2g Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B473E19560B3 for ; Wed, 22 Jan 2025 15:17:40 +0000 (UTC) Received: from corto.redhat.com (unknown [10.39.192.154]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 9A22119560AA; Wed, 22 Jan 2025 15:17:39 +0000 (UTC) From: =?utf-8?q?C=C3=A9dric_Le_Goater?= To: qemu-devel@nongnu.org Cc: Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_G?= =?utf-8?q?oater?= Subject: [PATCH 2/5] vfio: Modify vfio_viommu_preset() parameter Date: Wed, 22 Jan 2025 16:17:29 +0100 Message-ID: <20250122151732.1351821-3-clg@redhat.com> In-Reply-To: <20250122151732.1351821-1-clg@redhat.com> References: <20250122151732.1351821-1-clg@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.996, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We plan to use vfio_viommu_preset() in MemoryListener handlers which operate at the container level. Change the parameter to VFIOContainerBase to ease future changes. Signed-off-by: Cédric Le Goater --- include/hw/vfio/vfio-common.h | 2 +- hw/vfio/common.c | 4 ++-- hw/vfio/migration.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 48018dc751e51066769b23bc6e4675a7167b099e..fcfe0b4b8cbe907877e366117e7bb7f74311d4f6 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -274,7 +274,7 @@ extern int vfio_kvm_device_fd; bool vfio_mig_active(void); int vfio_block_multiple_devices_migration(VFIODevice *vbasedev, Error **errp); void vfio_unblock_multiple_devices_migration(void); -bool vfio_viommu_preset(VFIODevice *vbasedev); +bool vfio_viommu_preset(VFIOContainerBase *bcontainer); int64_t vfio_mig_bytes_transferred(void); void vfio_reset_bytes_transferred(void); bool vfio_device_state_is_running(VFIODevice *vbasedev); diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 2660a42f9edc9346f2e62652efb0c78a8b48b52b..3ca5dbf883ed2262e36952fcc47e717ff4154f12 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -142,9 +142,9 @@ void vfio_unblock_multiple_devices_migration(void) migrate_del_blocker(&multiple_devices_migration_blocker); } -bool vfio_viommu_preset(VFIODevice *vbasedev) +bool vfio_viommu_preset(VFIOContainerBase *bcontainer) { - return vbasedev->bcontainer->space->as != &address_space_memory; + return bcontainer->space->as != &address_space_memory; } static void vfio_set_migration_error(int ret) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index adfa752db5272e37d73fc0a435a0834e74e3f2fe..347390adb27bc3cd123f3eec1de6dc6986d8d952 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -1069,7 +1069,7 @@ bool vfio_migration_realize(VFIODevice *vbasedev, Error **errp) goto out_deinit; } - if (vfio_viommu_preset(vbasedev)) { + if (vfio_viommu_preset(vbasedev->bcontainer)) { error_setg(&err, "%s: Migration is currently not supported " "with vIOMMU enabled", vbasedev->name); goto add_blocker; From patchwork Wed Jan 22 15:17:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?C=C3=A9dric_Le_Goater?= X-Patchwork-Id: 13947426 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2BFC1C02181 for ; Wed, 22 Jan 2025 15:19:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tacUK-0004cV-AN; Wed, 22 Jan 2025 10:17:52 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tacUH-0004c3-A7 for qemu-devel@nongnu.org; Wed, 22 Jan 2025 10:17:49 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tacUE-0004Ye-Vo for qemu-devel@nongnu.org; Wed, 22 Jan 2025 10:17:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1737559066; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=++9SJZ2wWg4Pbvd90BZMzoeNDlPsajzo7tBFGHJvvZU=; b=HCCfCk0sVKKmMi0RQpWMQa9GaDIIPcmWXPSS8BCXGEnoTKX1iogp14EDsxLjkAcyNyo75U HMUat6HV4boya4qgBzUVmCDjHqumzJz9M50wOeFQMS/y1eq+mOCkYIesKpP956A0fkaVWS hosTiCZKwQHFVSPOKLgJfk1UTao2ZYo= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-515-Z2lNY_ThMi6fCFKiCqt9Vg-1; Wed, 22 Jan 2025 10:17:44 -0500 X-MC-Unique: Z2lNY_ThMi6fCFKiCqt9Vg-1 X-Mimecast-MFC-AGG-ID: Z2lNY_ThMi6fCFKiCqt9Vg Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D8F211955D53 for ; Wed, 22 Jan 2025 15:17:43 +0000 (UTC) Received: from corto.redhat.com (unknown [10.39.192.154]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2196D19560AA; Wed, 22 Jan 2025 15:17:40 +0000 (UTC) From: =?utf-8?q?C=C3=A9dric_Le_Goater?= To: qemu-devel@nongnu.org Cc: Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_G?= =?utf-8?q?oater?= Subject: [PATCH 3/5] vfio: Improve error reporting when MMIO region mapping fails Date: Wed, 22 Jan 2025 16:17:30 +0100 Message-ID: <20250122151732.1351821-4-clg@redhat.com> In-Reply-To: <20250122151732.1351821-1-clg@redhat.com> References: <20250122151732.1351821-1-clg@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.996, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org When the IOMMU address space width is smaller than the physical address width, a MMIO region of a device can fail to map because the region is outside the supported IOVA ranges of the VM. In this case, PCI peer-to-peer transactions on BARs are not supported. This can occur with the 39-bit IOMMU address space width, as can be the case on consumer processors or when using a vIOMMU device with default settings. The current error message is unclear. Change the error report to a warning because it is a non fatal condition for the VM, clarify the induced limitations for the user and provide advice on how to possibly resolve this issue: setting the CPU address space width of the IOMMU address space width accordingly. Signed-off-by: Cédric Le Goater --- hw/vfio/common.c | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 3ca5dbf883ed2262e36952fcc47e717ff4154f12..561be7f57cf903e6e2bcbbb708be9e4d4ee8941c 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -569,6 +569,20 @@ static bool vfio_get_section_iova_range(VFIOContainerBase *bcontainer, return true; } +static void vfio_container_p2p_error_append(VFIOContainerBase *bcontainer, + Error **errp) +{ + error_append_hint(errp, "PCI peer-to-peer transactions on BARs " + "are not supported. "); + if (vfio_viommu_preset(bcontainer)) { + error_append_hint(errp, "Try setting the vIOMMU \"aw-bits\" " + "property to match CPU address space width\n"); + } else { + error_append_hint(errp, "Try setting the CPU \"phys-bits\" " + "property to match IOMMU address space width\n"); + } +} + static void vfio_listener_region_add(MemoryListener *listener, MemoryRegionSection *section) { @@ -685,9 +699,13 @@ static void vfio_listener_region_add(MemoryListener *listener, "0x%"HWADDR_PRIx", %p) = %d (%s)", bcontainer, iova, int128_get64(llsize), vaddr, ret, strerror(-ret)); + /* + * MMIO region mapping failures are not fatal but in this case + * PCI peer-to-peer transactions are broken. + */ if (memory_region_is_ram_device(section->mr)) { - /* Allow unexpected mappings not to be fatal for RAM devices */ - error_report_err(err); + vfio_container_p2p_error_append(bcontainer, &err); + warn_report_err(err); return; } goto fail; From patchwork Wed Jan 22 15:17:31 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?C=C3=A9dric_Le_Goater?= X-Patchwork-Id: 13947422 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0B1E2C02182 for ; Wed, 22 Jan 2025 15:19:24 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tacUN-0004dA-0m; Wed, 22 Jan 2025 10:17:55 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tacUL-0004cp-Ll for qemu-devel@nongnu.org; Wed, 22 Jan 2025 10:17:53 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tacUJ-0004Zl-Rg for qemu-devel@nongnu.org; Wed, 22 Jan 2025 10:17:53 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1737559071; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3/JMgGWmATOznOhuHhwYa5PU35n44LVPlDniBVxJxpQ=; b=T5wdxC5NxlrDj8INL1BgTfXN+8qFQyi4Xdrj/dtpEeCTMZKN4g5LoWdML9qgPK4tA+klx8 fYkyB5G9t8vwsGChw0EPr6/ZHnh9UFxsCFwt1Qmy5uy5wINv4mAe20xAlQaJtcZgOhIG3p 9jkUZu82hnmRTGzMjkypySGQZtMXvo8= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-587-tWjpqeJJNPewrviODl0NUQ-1; Wed, 22 Jan 2025 10:17:48 -0500 X-MC-Unique: tWjpqeJJNPewrviODl0NUQ-1 X-Mimecast-MFC-AGG-ID: tWjpqeJJNPewrviODl0NUQ Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 64AB01956051; Wed, 22 Jan 2025 15:17:47 +0000 (UTC) Received: from corto.redhat.com (unknown [10.39.192.154]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 46EB519560AA; Wed, 22 Jan 2025 15:17:44 +0000 (UTC) From: =?utf-8?q?C=C3=A9dric_Le_Goater?= To: qemu-devel@nongnu.org Cc: Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_G?= =?utf-8?q?oater?= , Richard Henderson , Paolo Bonzini , Eduardo Habkost , Marcel Apfelbaum , =?utf-8?q?Philippe_Mathieu-D?= =?utf-8?q?aud=C3=A9?= , Yanan Wang , Zhao Liu Subject: [PATCH 4/5] cpu: Introduce cpu_get_phys_bits() Date: Wed, 22 Jan 2025 16:17:31 +0100 Message-ID: <20250122151732.1351821-5-clg@redhat.com> In-Reply-To: <20250122151732.1351821-1-clg@redhat.com> References: <20250122151732.1351821-1-clg@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.996, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org The Intel CPU has a complex history regarding setting of the physical address space width on KVM. A 'phys_bits' field and a "phys-bits" property were added by commit af45907a1328 ("target-i386: Allow physical address bits to be set") to tune this value. In certain circumstances, it is interesting to know this value to check that all the conditions are met for optimal operation. For instance, when the system has a 39-bit IOMMU address space width and a larger CPU physical address space, we expect issues when mapping the MMIO regions of passthrough devices and it would good to report to the user. These hybrid HW configs can be found on some consumer grade processors or when using a vIOMMU device with default settings. For this purpose, add an helper routine and a CPUClass callback to return the physical address space width of a CPU. Cc: Richard Henderson Cc: Paolo Bonzini Cc: Eduardo Habkost Cc: Marcel Apfelbaum Cc: "Philippe Mathieu-Daudé" Cc: Yanan Wang Cc: Zhao Liu Signed-off-by: Cédric Le Goater --- include/hw/core/cpu.h | 9 +++++++++ include/hw/core/sysemu-cpu-ops.h | 6 ++++++ cpu-target.c | 5 +++++ hw/core/cpu-system.c | 11 +++++++++++ target/i386/cpu.c | 6 ++++++ 5 files changed, 37 insertions(+) diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h index fb397cdfc53d12d40d3e4e7f86251fc31c48b9f6..1b3eead102ce62fcee55ab0ed5e0dff327fa2fc5 100644 --- a/include/hw/core/cpu.h +++ b/include/hw/core/cpu.h @@ -748,6 +748,14 @@ int cpu_asidx_from_attrs(CPUState *cpu, MemTxAttrs attrs); */ bool cpu_virtio_is_big_endian(CPUState *cpu); +/** + * cpu_get_phys_bits: + * @cpu: CPU + * + * Return the physical address space width of the CPU @cpu. + */ +uint32_t cpu_get_phys_bits(const CPUState *cpu); + #endif /* CONFIG_USER_ONLY */ /** @@ -1168,6 +1176,7 @@ void cpu_exec_unrealizefn(CPUState *cpu); void cpu_exec_reset_hold(CPUState *cpu); const char *target_name(void); +uint32_t target_phys_bits(void); #ifdef COMPILING_PER_TARGET diff --git a/include/hw/core/sysemu-cpu-ops.h b/include/hw/core/sysemu-cpu-ops.h index 0df5b058f50073e47d2a6b8286be5204776520d2..210b3ed57985525795b81559e41e0085969210d5 100644 --- a/include/hw/core/sysemu-cpu-ops.h +++ b/include/hw/core/sysemu-cpu-ops.h @@ -81,6 +81,12 @@ typedef struct SysemuCPUOps { */ bool (*virtio_is_big_endian)(CPUState *cpu); + /** + * @get_phys_bits: Callback to return the physical address space + * width of a CPU. + */ + uint32_t (*get_phys_bits)(const CPUState *cpu); + /** * @legacy_vmsd: Legacy state for migration. * Do not use in new targets, use #DeviceClass::vmsd instead. diff --git a/cpu-target.c b/cpu-target.c index 667688332c929aa53782c94343def34571272d5f..88158272c06cc42424d435b9701e33735f080239 100644 --- a/cpu-target.c +++ b/cpu-target.c @@ -472,3 +472,8 @@ const char *target_name(void) { return TARGET_NAME; } + +uint32_t target_phys_bits(void) +{ + return TARGET_PHYS_ADDR_SPACE_BITS; +} diff --git a/hw/core/cpu-system.c b/hw/core/cpu-system.c index 6aae28a349a7a377d010ff9dcab5ebc29e1126ca..05067d84f4258facf4252216f17729e390d38eae 100644 --- a/hw/core/cpu-system.c +++ b/hw/core/cpu-system.c @@ -60,6 +60,17 @@ hwaddr cpu_get_phys_page_attrs_debug(CPUState *cpu, vaddr addr, return cc->sysemu_ops->get_phys_page_debug(cpu, addr); } +uint32_t cpu_get_phys_bits(const CPUState *cpu) +{ + CPUClass *cc = CPU_GET_CLASS(cpu); + + if (cc->sysemu_ops->get_phys_bits) { + return cc->sysemu_ops->get_phys_bits(cpu); + } + + return target_phys_bits(); +} + hwaddr cpu_get_phys_page_debug(CPUState *cpu, vaddr addr) { MemTxAttrs attrs = {}; diff --git a/target/i386/cpu.c b/target/i386/cpu.c index 1b9c11022c48e3103627d370f7fbdb2ae94a9f81..8f9f75de7cafaca72b4eb32e8229a7a7668f5c1c 100644 --- a/target/i386/cpu.c +++ b/target/i386/cpu.c @@ -8239,6 +8239,11 @@ static bool x86_cpu_get_paging_enabled(const CPUState *cs) return cpu->env.cr[0] & CR0_PG_MASK; } + +static uint32_t x86_cpu_get_phys_bits(const CPUState *cs) +{ + return X86_CPU(cs)->phys_bits; +} #endif /* !CONFIG_USER_ONLY */ static void x86_cpu_set_pc(CPUState *cs, vaddr value) @@ -8547,6 +8552,7 @@ static const struct SysemuCPUOps i386_sysemu_ops = { .get_memory_mapping = x86_cpu_get_memory_mapping, .get_paging_enabled = x86_cpu_get_paging_enabled, .get_phys_page_attrs_debug = x86_cpu_get_phys_page_attrs_debug, + .get_phys_bits = x86_cpu_get_phys_bits, .asidx_from_attrs = x86_asidx_from_attrs, .get_crash_info = x86_cpu_get_crash_info, .write_elf32_note = x86_cpu_write_elf32_note, From patchwork Wed Jan 22 15:17:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?C=C3=A9dric_Le_Goater?= X-Patchwork-Id: 13947423 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B9600C02181 for ; Wed, 22 Jan 2025 15:19:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tacUT-0004e3-DN; Wed, 22 Jan 2025 10:18:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tacUO-0004dn-N1 for qemu-devel@nongnu.org; Wed, 22 Jan 2025 10:17:56 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tacUM-0004a9-UB for qemu-devel@nongnu.org; Wed, 22 Jan 2025 10:17:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1737559072; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=3BoilxGy/dKTFh734KqqGJ1Ffq33SYL1+VhPSneWoXI=; b=B7JLFcMEw5BR23mK6RNKXc3DIJCwHNZloYWeaXSpTgzzP8kFK9iAYjSU0h598uK0fq/aty Tkv9/PVWltZtYsZXv7XspbU3rWinQq0jggOvG0wgcz66AOuIb38mTFHwC5zyHsVpLxeBDX dVD6iOQrjMGv66HS0OaD4V1eXx6RS/g= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-325-7Zww7tDyOxawawF2WtrZww-1; Wed, 22 Jan 2025 10:17:49 -0500 X-MC-Unique: 7Zww7tDyOxawawF2WtrZww-1 X-Mimecast-MFC-AGG-ID: 7Zww7tDyOxawawF2WtrZww Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E96681956055 for ; Wed, 22 Jan 2025 15:17:48 +0000 (UTC) Received: from corto.redhat.com (unknown [10.39.192.154]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id DC4BD19560AA; Wed, 22 Jan 2025 15:17:47 +0000 (UTC) From: =?utf-8?q?C=C3=A9dric_Le_Goater?= To: qemu-devel@nongnu.org Cc: Alex Williamson , =?utf-8?q?C=C3=A9dric_Le_G?= =?utf-8?q?oater?= Subject: [PATCH 5/5] vfio: Check compatibility of CPU and IOMMU address space width Date: Wed, 22 Jan 2025 16:17:32 +0100 Message-ID: <20250122151732.1351821-6-clg@redhat.com> In-Reply-To: <20250122151732.1351821-1-clg@redhat.com> References: <20250122151732.1351821-1-clg@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 Received-SPF: pass client-ip=170.10.129.124; envelope-from=clg@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.996, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Print a warning if IOMMU address space width is smaller than the physical address width. In this case, PCI peer-to-peer transactions on BARs are not supported and failures of device MMIO regions are to be expected. This can occur with the 39-bit IOMMU address space width as found on consumer grade processors or when using a vIOMMU device with default settings. Signed-off-by: Cédric Le Goater --- hw/vfio/common.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 561be7f57cf903e6e2bcbbb708be9e4d4ee8941c..d0104976e9d99c3f64ec716accd3adb4a70d9afe 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -44,6 +44,8 @@ #include "migration/qemu-file.h" #include "system/tpm.h" +#include "hw/core/cpu.h" + VFIODeviceList vfio_device_list = QLIST_HEAD_INITIALIZER(vfio_device_list); static QLIST_HEAD(, VFIOAddressSpace) vfio_address_spaces = @@ -1561,12 +1563,28 @@ retry: return info; } +static bool vfio_device_check_address_space(VFIODevice *vbasedev, Error **errp) +{ + uint32_t cpu_aw_bits = cpu_get_phys_bits(first_cpu); + uint32_t iommu_aw_bits = vfio_device_get_aw_bits(vbasedev); + + if (cpu_aw_bits && cpu_aw_bits > iommu_aw_bits) { + error_setg(errp, "Host physical address space (%u) is larger than " + "the host IOMMU address space (%u).", cpu_aw_bits, + iommu_aw_bits); + vfio_container_p2p_error_append(vbasedev->bcontainer, errp); + return false; + } + return true; +} + bool vfio_attach_device(char *name, VFIODevice *vbasedev, AddressSpace *as, Error **errp) { const VFIOIOMMUClass *ops = VFIO_IOMMU_CLASS(object_class_by_name(TYPE_VFIO_IOMMU_LEGACY)); HostIOMMUDevice *hiod = NULL; + Error *local_err = NULL; if (vbasedev->iommufd) { ops = VFIO_IOMMU_CLASS(object_class_by_name(TYPE_VFIO_IOMMU_IOMMUFD)); @@ -1586,6 +1604,9 @@ bool vfio_attach_device(char *name, VFIODevice *vbasedev, return false; } + if (!vfio_device_check_address_space(vbasedev, &local_err)) { + warn_report_err(local_err); + } return true; }