From patchwork Thu Feb 1 07:28:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540724 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9ECD6C47DDF for ; Thu, 1 Feb 2024 07:31:25 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRWq-0005Dj-GN; Thu, 01 Feb 2024 02:30:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRWo-0005Cw-NB for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:30:31 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRWh-0001Up-Fh for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:30:29 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772623; x=1738308623; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Lhqz2iRToADp66tcHjAaagZoKGFICWwlIhQ9qtn5y64=; b=gt6gb1x4LBgYXrwMXz22NDrugrqP7U9FBNj6Cj4cbiq3PYxaCaPn3GCh 15RkxdDskKmd/CykYAkOj4dL6O6+emRsYOdM+WwqBta5KvWdNScz+KB31 EbvJbUC6FeOnbjFwB1FeON3pvIOlHHSo3jNc1VzpI4fVB3si6I1dJHR2O kgAg9986uJ6q3Ih9v2GrDOgptyT3zOYWBgcqGsn5PUO55zRHR9b/66ANO ouB4AULwWyuIxcfWjD6QwpiGwHorldD2DHR4Cc6zr4bxwxEaAtE1PjTZA HfTnRoiqMqTUMgdB9VDhWJgv3C0hdTmXcK+dIsiGpFDRgpdNvDqnBgRTu A==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402676610" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402676610" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:30:22 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443140" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:30:18 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Zhenzhong Duan Subject: [PATCH rfcv2 01/18] Introduce a common abstract struct HostIOMMUDevice Date: Thu, 1 Feb 2024 15:28:01 +0800 Message-Id: <20240201072818.327930-2-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org HostIOMMUDevice will be inherited by two sub classes, legacy and iommufd currently. Introduce a helper function host_iommu_base_device_init to initialize it. Suggested-by: Eric Auger Signed-off-by: Zhenzhong Duan --- include/sysemu/host_iommu_device.h | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 include/sysemu/host_iommu_device.h diff --git a/include/sysemu/host_iommu_device.h b/include/sysemu/host_iommu_device.h new file mode 100644 index 0000000000..fe80ab25fb --- /dev/null +++ b/include/sysemu/host_iommu_device.h @@ -0,0 +1,22 @@ +#ifndef HOST_IOMMU_DEVICE_H +#define HOST_IOMMU_DEVICE_H + +typedef enum HostIOMMUDevice_Type { + HID_LEGACY, + HID_IOMMUFD, + HID_MAX, +} HostIOMMUDevice_Type; + +typedef struct HostIOMMUDevice { + HostIOMMUDevice_Type type; + size_t size; +} HostIOMMUDevice; + +static inline void host_iommu_base_device_init(HostIOMMUDevice *dev, + HostIOMMUDevice_Type type, + size_t size) +{ + dev->type = type; + dev->size = size; +} +#endif From patchwork Thu Feb 1 07:28:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DB8E3C47DDF for ; Thu, 1 Feb 2024 07:31:28 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRWv-0005Em-Qh; Thu, 01 Feb 2024 02:30:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRWp-0005Cx-BB for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:30:31 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRWl-0001R8-L8 for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:30:31 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772627; x=1738308627; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=sAIPXTm87HB/uGpJ/9YNiXxwn8eta5zQASgPvOQiYZs=; b=RuV8fY7LGwZtXStYP9Cg2KK8VUo5C9WUs/pamSJP/pc2+2ntMyOzfxrI kjWk+lkWkOSBDbs5gr5corHnztTb88blaudnkFWRW/Yo5tGJMhn3CfHD9 dc9hnIE5Bh4ez+EpzKBk/GlzWosrBzspkPdVX3nR1Ae5pr4RQYFZfGuk7 Hxos8dilViyB2Rgth5T7+AkLhfmd5p/yN7nEG2iLgwquigwx+soHpn5If gX7tmwWTOYWdcrnrsfZFdMFsz3kKNkl4BnQpPa7TcpVROrcL0dm1+QhNy Vu0KqQAiOJZtQuRTqB3ysOUX9weHfcAPs+aKvBQteQiwymAfvV9o2/9qy A==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402676638" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402676638" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:30:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443168" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:30:23 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Yi Sun Subject: [PATCH rfcv2 02/18] backends/iommufd: Introduce IOMMUFDDevice Date: Thu, 1 Feb 2024 15:28:02 +0800 Message-Id: <20240201072818.327930-3-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org IOMMUFDDevice represents a device in iommufd and can be used as a communication interface between devices (i.e., VFIO, VDPA) and vIOMMU. Currently it includes only public iommufd handle and device id which could be used by vIOMMU to get hw IOMMU information. There will also be some elements in private field in future, i.e., capability bits for dirty tracking; when nested translation is supported in future, vIOMMU is going to have more iommufd related operations like allocate hwpt for a device, attach/detach hwpt, etc. So IOMMUFDDevice will be further extended with those needs. IOMMUFDDevice is willingly not a QOM object because we don't want it to be visible from the user interface. Introduce a helper iommufd_device_init to initialize IOMMUFDDevice. Originally-by: Yi Liu Signed-off-by: Yi Sun Signed-off-by: Zhenzhong Duan --- include/sysemu/iommufd.h | 14 ++++++++++++++ backends/iommufd.c | 6 ++++++ 2 files changed, 20 insertions(+) diff --git a/include/sysemu/iommufd.h b/include/sysemu/iommufd.h index 9af27ebd6c..c3f3469760 100644 --- a/include/sysemu/iommufd.h +++ b/include/sysemu/iommufd.h @@ -4,6 +4,7 @@ #include "qom/object.h" #include "exec/hwaddr.h" #include "exec/cpu-common.h" +#include "sysemu/host_iommu_device.h" #define TYPE_IOMMUFD_BACKEND "iommufd" OBJECT_DECLARE_TYPE(IOMMUFDBackend, IOMMUFDBackendClass, IOMMUFD_BACKEND) @@ -33,4 +34,17 @@ int iommufd_backend_map_dma(IOMMUFDBackend *be, uint32_t ioas_id, hwaddr iova, ram_addr_t size, void *vaddr, bool readonly); int iommufd_backend_unmap_dma(IOMMUFDBackend *be, uint32_t ioas_id, hwaddr iova, ram_addr_t size); + + +/* Abstraction of host IOMMUFD device */ +typedef struct IOMMUFDDevice { + HostIOMMUDevice base; + /* private: */ + + /* public: */ + IOMMUFDBackend *iommufd; + uint32_t devid; +} IOMMUFDDevice; + +void iommufd_device_init(IOMMUFDDevice *idev); #endif diff --git a/backends/iommufd.c b/backends/iommufd.c index 1ef683c7b0..d92791bba9 100644 --- a/backends/iommufd.c +++ b/backends/iommufd.c @@ -231,3 +231,9 @@ static void register_types(void) } type_init(register_types); + +void iommufd_device_init(IOMMUFDDevice *idev) +{ + host_iommu_base_device_init(&idev->base, HID_IOMMUFD, + sizeof(IOMMUFDDevice)); +} From patchwork Thu Feb 1 07:28:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540723 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E01E8C47DDF for ; Thu, 1 Feb 2024 07:31:20 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRX5-0005KK-VR; Thu, 01 Feb 2024 02:30:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRWy-0005IZ-KU for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:30:42 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRWs-0001bF-Pu for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:30:39 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772634; x=1738308634; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=N3xsv2GtRVX6aSgVHRkuCisTKekGHMSdCU/SFK0nNzk=; b=MsFKUGqbGoQUacOgpGOHqsucBigpWLKA+yhjm/Y5rEpMaQfe/91RN4b6 tNNeqdE5CtYNlFe3D2HkWPGzEmgTpuzjBs2lzKw0MaEwaaMLq6A+dOC6B GsNAV5Duu+jUo0+lcQkAqVK+rLdrB3j88RPI5nWsmuv4zwiPnPS67NMQM fVZEGRqkcEaqi6Pijmt5dAilQJbrfcTNSDvbsvWUnYEuWKrqCmLmqpajp p9UWaKVyvR6B6VY1raSL3Wkb57sUBcGGelp5A3r3HwSnkGRJ6y2/W0F3m QnfDP5tLfJO3gfbh9nZW4fJBKkT1lMiVHu33S6jdo9mCIrtoxWbv9Hcdo w==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402676650" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402676650" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:30:31 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443189" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:30:27 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Zhenzhong Duan Subject: [PATCH rfcv2 03/18] vfio: Introduce IOMMULegacyDevice Date: Thu, 1 Feb 2024 15:28:03 +0800 Message-Id: <20240201072818.327930-4-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Similar as IOMMUFDDevice, IOMMULegacyDevice represents a device in legacy mode and can be used as a communication interface between devices (i.e., VFIO, VDPA) and vIOMMU. Currently it includes nothing legacy specific, but could be extended with any wanted info of legacy mode when necessary. IOMMULegacyDevice is willingly not a QOM object because we don't want it to be visible from the user interface. Suggested-by: Eric Auger Signed-off-by: Zhenzhong Duan --- include/hw/vfio/vfio-common.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 9b7ef7d02b..8bfb9cbe94 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -31,6 +31,7 @@ #endif #include "sysemu/sysemu.h" #include "hw/vfio/vfio-container-base.h" +#include "sysemu/host_iommu_device.h" #define VFIO_MSG_PREFIX "vfio %s: " @@ -97,6 +98,11 @@ typedef struct VFIOIOMMUFDContainer { uint32_t ioas_id; } VFIOIOMMUFDContainer; +/* Abstraction of host IOMMU legacy device */ +typedef struct IOMMULegacyDevice { + HostIOMMUDevice base; +} IOMMULegacyDevice; + typedef struct VFIODeviceOps VFIODeviceOps; typedef struct VFIODevice { From patchwork Thu Feb 1 07:28:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540726 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 27E26C47DDF for ; Thu, 1 Feb 2024 07:31:34 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRX5-0005KJ-Ui; Thu, 01 Feb 2024 02:30:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRWy-0005IY-KD for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:30:42 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRWw-0001bh-8D for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:30:39 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772638; x=1738308638; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=CTDEsm16kYgmZEZrIhRTFu5mtzMEIhIQsKe4RoTDPvE=; b=npxF3gjRfWw/d0UvzbrjEsMvPWsysYW3xkdZrQKeANjFyfwt3Q+7fwpq ++n+nL2F1j5E3OSz1p2bSfqhJFlGgQbKYWdJ+wy4j/4rqIAexWq3N2AUq 3KtZn0dgQX1/kw6/jRbulSjwvevk+OBN6EYuBmhk6/UzzM5eBrPszfnEy f5PXQGgCVPU75BaeU/CpAxtwOsGPcWVVlwsy5fEdbZ0+fu6XJi8Jlq1+w PXo/HaAfsFVPHtnow8ECs+SRM0OoryR36qo5GB5vn3J/9fliy29lJga6d RYuzxv38vXKgvekS0icRZwG7h7sjoYlr6oPmxdXKdn0lBvOtyyHEUXIyN g==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402676666" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402676666" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:30:36 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443212" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:30:32 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Zhenzhong Duan Subject: [PATCH rfcv2 04/18] vfio: Add host iommu device instance into VFIODevice Date: Thu, 1 Feb 2024 15:28:04 +0800 Message-Id: <20240201072818.327930-5-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Either IOMMULegacyDevice or IOMMUFDDevice into VFIODevice, neither both. Signed-off-by: Zhenzhong Duan --- include/hw/vfio/vfio-common.h | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 8bfb9cbe94..1bbad003ee 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -32,6 +32,7 @@ #include "sysemu/sysemu.h" #include "hw/vfio/vfio-container-base.h" #include "sysemu/host_iommu_device.h" +#include "sysemu/iommufd.h" #define VFIO_MSG_PREFIX "vfio %s: " @@ -132,8 +133,18 @@ typedef struct VFIODevice { bool dirty_tracking; int devid; IOMMUFDBackend *iommufd; + union { + HostIOMMUDevice base_hdev; + IOMMULegacyDevice legacy_dev; + IOMMUFDDevice iommufd_dev; + }; } VFIODevice; +QEMU_BUILD_BUG_ON(offsetof(VFIODevice, legacy_dev.base) != + offsetof(VFIODevice, base_hdev)); +QEMU_BUILD_BUG_ON(offsetof(VFIODevice, iommufd_dev.base) != + offsetof(VFIODevice, base_hdev)); + struct VFIODeviceOps { void (*vfio_compute_needs_reset)(VFIODevice *vdev); int (*vfio_hot_reset_multi)(VFIODevice *vdev); From patchwork Thu Feb 1 07:28:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540722 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 30292C47DDB for ; Thu, 1 Feb 2024 07:31:09 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRX9-0005VQ-TX; Thu, 01 Feb 2024 02:30:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRX8-0005UK-Fo; Thu, 01 Feb 2024 02:30:50 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRX4-0001ck-1Q; Thu, 01 Feb 2024 02:30:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772646; x=1738308646; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=TgSkZfw+it2zJiFcuri/JTQyY0VaeQ1D5N/sRZjophw=; b=JX6ODzplnBgH+po5S9cAPaejaiR/If0WC7RtpNl8YcSDQa5iMRo529xT OX7iKnPsSuWu+18btu+0tFf4a3Cq0EiUMaWjBXfRnEE8Q0p0vDQGrXG4F dJuhghg7xNXYAsWKmOReteCZyovjvIcGa1AT5G+vKTMRS5QSmLvVL27fl e+VQ0ylxmyn38WdlXK+6Jo9vsGtNV+lECzxfm2phwnhC8SM07uiIRIhVE zhrKtjSyvROskVY2i1sWRrkwE588OaC/3RTdxjLtJsU85sZ3HqTkRJ+58 YKDModsGZmBYHt76m3SpVdwSt5yCVshXNvY/1Z7eFKbRMU9G1k/G1Eilw w==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402676690" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402676690" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:30:42 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443226" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:30:37 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Thomas Huth , Tony Krowiak , Halil Pasic , Jason Herne , Eric Farman , Matthew Rosato , qemu-s390x@nongnu.org (open list:S390 general arch...) Subject: [PATCH rfcv2 05/18] vfio: Remove redundant iommufd and devid elements in VFIODevice Date: Thu, 1 Feb 2024 15:28:05 +0800 Message-Id: <20240201072818.327930-6-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org iommufd and devid in VFIODevice are redundant with the ones in IOMMUFDDevice, so remove them. Suggested-by: Eric Auger Signed-off-by: Zhenzhong Duan --- include/hw/vfio/vfio-common.h | 2 -- hw/vfio/ap.c | 2 +- hw/vfio/ccw.c | 2 +- hw/vfio/common.c | 2 +- hw/vfio/helpers.c | 2 +- hw/vfio/iommufd.c | 26 ++++++++++++++------------ hw/vfio/pci.c | 2 +- hw/vfio/platform.c | 3 ++- 8 files changed, 21 insertions(+), 20 deletions(-) diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 1bbad003ee..24e3eaaf3d 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -131,8 +131,6 @@ typedef struct VFIODevice { OnOffAuto pre_copy_dirty_page_tracking; bool dirty_pages_supported; bool dirty_tracking; - int devid; - IOMMUFDBackend *iommufd; union { HostIOMMUDevice base_hdev; IOMMULegacyDevice legacy_dev; diff --git a/hw/vfio/ap.c b/hw/vfio/ap.c index e157aa1ff7..11526d93d4 100644 --- a/hw/vfio/ap.c +++ b/hw/vfio/ap.c @@ -198,7 +198,7 @@ static void vfio_ap_unrealize(DeviceState *dev) static Property vfio_ap_properties[] = { DEFINE_PROP_STRING("sysfsdev", VFIOAPDevice, vdev.sysfsdev), #ifdef CONFIG_IOMMUFD - DEFINE_PROP_LINK("iommufd", VFIOAPDevice, vdev.iommufd, + DEFINE_PROP_LINK("iommufd", VFIOAPDevice, vdev.iommufd_dev.iommufd, TYPE_IOMMUFD_BACKEND, IOMMUFDBackend *), #endif DEFINE_PROP_END_OF_LIST(), diff --git a/hw/vfio/ccw.c b/hw/vfio/ccw.c index 90e4a53437..b1b75ffa2a 100644 --- a/hw/vfio/ccw.c +++ b/hw/vfio/ccw.c @@ -667,7 +667,7 @@ static Property vfio_ccw_properties[] = { DEFINE_PROP_STRING("sysfsdev", VFIOCCWDevice, vdev.sysfsdev), DEFINE_PROP_BOOL("force-orb-pfch", VFIOCCWDevice, force_orb_pfch, false), #ifdef CONFIG_IOMMUFD - DEFINE_PROP_LINK("iommufd", VFIOCCWDevice, vdev.iommufd, + DEFINE_PROP_LINK("iommufd", VFIOCCWDevice, vdev.iommufd_dev.iommufd, TYPE_IOMMUFD_BACKEND, IOMMUFDBackend *), #endif DEFINE_PROP_END_OF_LIST(), diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 059bfdc07a..8b3b575c9d 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1505,7 +1505,7 @@ int vfio_attach_device(char *name, VFIODevice *vbasedev, const VFIOIOMMUClass *ops = VFIO_IOMMU_CLASS(object_class_by_name(TYPE_VFIO_IOMMU_LEGACY)); - if (vbasedev->iommufd) { + if (vbasedev->iommufd_dev.iommufd) { ops = VFIO_IOMMU_CLASS(object_class_by_name(TYPE_VFIO_IOMMU_IOMMUFD)); } diff --git a/hw/vfio/helpers.c b/hw/vfio/helpers.c index 6789870802..e5457ca326 100644 --- a/hw/vfio/helpers.c +++ b/hw/vfio/helpers.c @@ -626,7 +626,7 @@ int vfio_device_get_name(VFIODevice *vbasedev, Error **errp) vbasedev->name = g_path_get_basename(vbasedev->sysfsdev); } } else { - if (!vbasedev->iommufd) { + if (!vbasedev->iommufd_dev.iommufd) { error_setg(errp, "Use FD passing only with iommufd backend"); return -EINVAL; } diff --git a/hw/vfio/iommufd.c b/hw/vfio/iommufd.c index 9bfddc1360..5d50549713 100644 --- a/hw/vfio/iommufd.c +++ b/hw/vfio/iommufd.c @@ -65,7 +65,7 @@ static void iommufd_cdev_kvm_device_del(VFIODevice *vbasedev) static int iommufd_cdev_connect_and_bind(VFIODevice *vbasedev, Error **errp) { - IOMMUFDBackend *iommufd = vbasedev->iommufd; + IOMMUFDBackend *iommufd = vbasedev->iommufd_dev.iommufd; struct vfio_device_bind_iommufd bind = { .argsz = sizeof(bind), .flags = 0, @@ -96,9 +96,10 @@ static int iommufd_cdev_connect_and_bind(VFIODevice *vbasedev, Error **errp) goto err_bind; } - vbasedev->devid = bind.out_devid; + vbasedev->iommufd_dev.devid = bind.out_devid; trace_iommufd_cdev_connect_and_bind(bind.iommufd, vbasedev->name, - vbasedev->fd, vbasedev->devid); + vbasedev->fd, + vbasedev->iommufd_dev.devid); return ret; err_bind: iommufd_cdev_kvm_device_del(vbasedev); @@ -111,7 +112,7 @@ static void iommufd_cdev_unbind_and_disconnect(VFIODevice *vbasedev) { /* Unbind is automatically conducted when device fd is closed */ iommufd_cdev_kvm_device_del(vbasedev); - iommufd_backend_disconnect(vbasedev->iommufd); + iommufd_backend_disconnect(vbasedev->iommufd_dev.iommufd); } static int iommufd_cdev_getfd(const char *sysfs_path, Error **errp) @@ -181,7 +182,7 @@ out_free_path: static int iommufd_cdev_attach_ioas_hwpt(VFIODevice *vbasedev, uint32_t id, Error **errp) { - int ret, iommufd = vbasedev->iommufd->fd; + int ret, iommufd = vbasedev->iommufd_dev.iommufd->fd; struct vfio_device_attach_iommufd_pt attach_data = { .argsz = sizeof(attach_data), .flags = 0, @@ -203,7 +204,7 @@ static int iommufd_cdev_attach_ioas_hwpt(VFIODevice *vbasedev, uint32_t id, static int iommufd_cdev_detach_ioas_hwpt(VFIODevice *vbasedev, Error **errp) { - int ret, iommufd = vbasedev->iommufd->fd; + int ret, iommufd = vbasedev->iommufd_dev.iommufd->fd; struct vfio_device_detach_iommufd_pt detach_data = { .argsz = sizeof(detach_data), .flags = 0, @@ -337,7 +338,7 @@ static int iommufd_cdev_attach(const char *name, VFIODevice *vbasedev, QLIST_FOREACH(bcontainer, &space->containers, next) { container = container_of(bcontainer, VFIOIOMMUFDContainer, bcontainer); if (bcontainer->ops != iommufd_vioc || - vbasedev->iommufd != container->be) { + vbasedev->iommufd_dev.iommufd != container->be) { continue; } if (iommufd_cdev_attach_container(vbasedev, container, &err)) { @@ -358,15 +359,16 @@ static int iommufd_cdev_attach(const char *name, VFIODevice *vbasedev, } /* Need to allocate a new dedicated container */ - ret = iommufd_backend_alloc_ioas(vbasedev->iommufd, &ioas_id, errp); + ret = iommufd_backend_alloc_ioas(vbasedev->iommufd_dev.iommufd, + &ioas_id, errp); if (ret < 0) { goto err_alloc_ioas; } - trace_iommufd_cdev_alloc_ioas(vbasedev->iommufd->fd, ioas_id); + trace_iommufd_cdev_alloc_ioas(vbasedev->iommufd_dev.iommufd->fd, ioas_id); container = g_malloc0(sizeof(*container)); - container->be = vbasedev->iommufd; + container->be = vbasedev->iommufd_dev.iommufd; container->ioas_id = ioas_id; bcontainer = &container->bcontainer; @@ -479,7 +481,7 @@ static VFIODevice *iommufd_cdev_pci_find_by_devid(__u32 devid) if (vbasedev_iter->bcontainer->ops != iommufd_vioc) { continue; } - if (devid == vbasedev_iter->devid) { + if (devid == vbasedev_iter->iommufd_dev.devid) { return vbasedev_iter; } } @@ -492,7 +494,7 @@ iommufd_cdev_dep_get_realized_vpdev(struct vfio_pci_dependent_device *dep_dev, { VFIODevice *vbasedev_tmp; - if (dep_dev->devid == reset_dev->devid || + if (dep_dev->devid == reset_dev->iommufd_dev.devid || dep_dev->devid == VFIO_PCI_DEVID_OWNED) { return NULL; } diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index 4fa387f043..d1e1b8cb89 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3385,7 +3385,7 @@ static Property vfio_pci_dev_properties[] = { DEFINE_PROP_OFF_AUTO_PCIBAR("x-msix-relocation", VFIOPCIDevice, msix_relo, OFF_AUTOPCIBAR_OFF), #ifdef CONFIG_IOMMUFD - DEFINE_PROP_LINK("iommufd", VFIOPCIDevice, vbasedev.iommufd, + DEFINE_PROP_LINK("iommufd", VFIOPCIDevice, vbasedev.iommufd_dev.iommufd, TYPE_IOMMUFD_BACKEND, IOMMUFDBackend *), #endif DEFINE_PROP_END_OF_LIST(), diff --git a/hw/vfio/platform.c b/hw/vfio/platform.c index a8d9b7da63..823a17b9c5 100644 --- a/hw/vfio/platform.c +++ b/hw/vfio/platform.c @@ -646,7 +646,8 @@ static Property vfio_platform_dev_properties[] = { mmap_timeout, 1100), DEFINE_PROP_BOOL("x-irqfd", VFIOPlatformDevice, irqfd_allowed, true), #ifdef CONFIG_IOMMUFD - DEFINE_PROP_LINK("iommufd", VFIOPlatformDevice, vbasedev.iommufd, + DEFINE_PROP_LINK("iommufd", VFIOPlatformDevice, + vbasedev.iommufd_dev.iommufd, TYPE_IOMMUFD_BACKEND, IOMMUFDBackend *), #endif DEFINE_PROP_END_OF_LIST(), From patchwork Thu Feb 1 07:28:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6043EC47DDF for ; Thu, 1 Feb 2024 07:31:38 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRXB-0005Wi-Nd; Thu, 01 Feb 2024 02:30:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRX9-0005VA-MK for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:30:51 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRX7-0001dL-U6 for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:30:51 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772649; x=1738308649; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=L1Ps1z7JR0EkT48WfCZvtyE+oHYsvw5z4u4xqJeBqVQ=; b=hyLcv/bF3JGtS/L1BoYxR7le4NuzLY5oezXOFlEO+Zge7xilS4VnhaNi PMEwXR4/E/eNslAZFFxKp2vTvEh5Nq4GZWwg/OxTc1rvQeRI924oQXd6f loEE0iy7JiZ0aPTLCsu1h4uux25+4qhjgh2EQKUHTiTD6lk06YQES+D+6 XF/Fnj7oWNUhiKYdRsYTaPAfKvEFwHjdiivM5bfdM0jTGRHjrp4NVRJ11 ngdMOW3E80BrsdRJc8ROIfe21aOTVWpJG4qGdPw2xUioaAoqgwbPy5dw+ lxIsNsXAYEK+sUg9lm7hHGadkuIUMlB+cm9VkFeiARmvUMbFOD4v3Uk62 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402676700" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402676700" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:30:47 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443242" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:30:43 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Zhenzhong Duan Subject: [PATCH rfcv2 06/18] vfio: Introduce host_iommu_device_init callback Date: Thu, 1 Feb 2024 15:28:06 +0800 Message-Id: <20240201072818.327930-7-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Introduce host_iommu_device_init callback and a wrapper for it. Signed-off-by: Zhenzhong Duan --- include/hw/vfio/vfio-common.h | 1 + include/hw/vfio/vfio-container-base.h | 1 + hw/vfio/common.c | 8 ++++++++ 3 files changed, 10 insertions(+) diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h index 24e3eaaf3d..9c4b60c906 100644 --- a/include/hw/vfio/vfio-common.h +++ b/include/hw/vfio/vfio-common.h @@ -216,6 +216,7 @@ struct vfio_device_info *vfio_get_device_info(int fd); int vfio_attach_device(char *name, VFIODevice *vbasedev, AddressSpace *as, Error **errp); void vfio_detach_device(VFIODevice *vbasedev); +void host_iommu_device_init(VFIODevice *vbasedev); int vfio_kvm_device_add_fd(int fd, Error **errp); int vfio_kvm_device_del_fd(int fd, Error **errp); diff --git a/include/hw/vfio/vfio-container-base.h b/include/hw/vfio/vfio-container-base.h index b2813b0c11..c71f4abb2d 100644 --- a/include/hw/vfio/vfio-container-base.h +++ b/include/hw/vfio/vfio-container-base.h @@ -120,6 +120,7 @@ struct VFIOIOMMUClass { int (*attach_device)(const char *name, VFIODevice *vbasedev, AddressSpace *as, Error **errp); void (*detach_device)(VFIODevice *vbasedev); + void (*host_iommu_device_init)(VFIODevice *vbasedev); /* migration feature */ int (*set_dirty_page_tracking)(const VFIOContainerBase *bcontainer, bool start); diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 8b3b575c9d..f7f85160be 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -1521,3 +1521,11 @@ void vfio_detach_device(VFIODevice *vbasedev) } vbasedev->bcontainer->ops->detach_device(vbasedev); } + +void host_iommu_device_init(VFIODevice *vbasedev) +{ + const VFIOIOMMUClass *ops = vbasedev->bcontainer->ops; + + assert(ops->host_iommu_device_init); + ops->host_iommu_device_init(vbasedev); +} From patchwork Thu Feb 1 07:28:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540730 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 67ED7C47DDB for ; Thu, 1 Feb 2024 07:32:07 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRXG-0005ag-BI; Thu, 01 Feb 2024 02:30:58 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRXE-0005XG-TX for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:30:56 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRXD-0001fn-3H for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:30:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772655; x=1738308655; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=D4cTHuJ5P52PDk7zGzN3Kqt39x1mp1P+J3KDpszCppQ=; b=OL6vd2qSfvTRzigpy0IV2B3xNCkoVDo5d87uW1rIn+bIMZCJlhAfDU8L JKXKRA9Cj9L2GcWmd14XzUyb2+BsU+cugx/YS+DZO4o0qW0zcoBa4+8yo e/b/PRv2+IAA3nsUDjRAUKMfJcVYsjSWnieZjO1ZcuMx2YJ6ytXvBZvpQ s1Tk7x35MUPvORYRdcoh/V95YvgzRf4GU9n3/w5yckYZnWd1XHDgynY9A +4X3OBOj1u0UjMCqo3R4Be059442kv4vg+Tw1sQVrmEhftWzI2OBUnrMG X9eYeJb06gNsitQgQ2eRBPgPV2NK3mkk1AeyduObxtGBUevsmtlHn0YY2 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402676716" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402676716" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:30:53 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443256" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:30:48 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Zhenzhong Duan Subject: [PATCH rfcv2 07/18] vfio/container: Implement host_iommu_device_init callback in legacy mode Date: Thu, 1 Feb 2024 15:28:07 +0800 Message-Id: <20240201072818.327930-8-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This callback will be used to initialize base and public elements in IOMMULegacyDevice. Signed-off-by: Zhenzhong Duan --- hw/vfio/container.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/hw/vfio/container.c b/hw/vfio/container.c index bd25b9fbad..8fafd4b4e5 100644 --- a/hw/vfio/container.c +++ b/hw/vfio/container.c @@ -1120,6 +1120,12 @@ out_single: return ret; } +static void vfio_legacy_host_iommu_device_init(VFIODevice *vbasedev) +{ + host_iommu_base_device_init(&vbasedev->base_hdev, HID_LEGACY, + sizeof(IOMMULegacyDevice)); +} + static void vfio_iommu_legacy_class_init(ObjectClass *klass, void *data) { VFIOIOMMUClass *vioc = VFIO_IOMMU_CLASS(klass); @@ -1132,6 +1138,7 @@ static void vfio_iommu_legacy_class_init(ObjectClass *klass, void *data) vioc->set_dirty_page_tracking = vfio_legacy_set_dirty_page_tracking; vioc->query_dirty_bitmap = vfio_legacy_query_dirty_bitmap; vioc->pci_hot_reset = vfio_legacy_pci_hot_reset; + vioc->host_iommu_device_init = vfio_legacy_host_iommu_device_init; }; static const TypeInfo types[] = { From patchwork Thu Feb 1 07:28:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540729 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 02903C47DDF for ; Thu, 1 Feb 2024 07:31:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRXe-0005xD-3T; Thu, 01 Feb 2024 02:31:22 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRXb-0005so-B9 for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:19 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRXS-0001h2-02 for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:19 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772670; x=1738308670; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8xJbm0mkCQz03mGu/N6iVObLbl1o4y3Umlds/PQNw6g=; b=KDGaiE2mX9Gcj6HKMbR6ggWvD4LTNUPESVT2UzmaEwQJAfgrlIBchjjh GHFR/k352/eUPYaL7v8r5g8lNPYXhuTy4ZCTdqOrVLbtQ22wkq0woG3CS 3iD2g/SoAVL9upsfp7pbZqx+kvWaDdEl7DZK51BMvPY/IQ4pYBikzzQCx U+b4n1LppIV6f/qIRwCBSYclNlJS8E2dOQ7U8zUs7amSspU9dg3MRnOhL 4yne22ga6DFGSGhCpGeY0FGODjzzuBWYLME39qc8pafRavVxzdq0aWTZx a28m76ki4bT5AudIWngHjqTBb2HHf0n3VzNx3S2srAZyo5man7Lu5Pvn4 w==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402676732" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402676732" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:30:57 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443267" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:30:53 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Zhenzhong Duan Subject: [PATCH rfcv2 08/18] vfio/iommufd: Implement host_iommu_device_init callback in iommufd mode Date: Thu, 1 Feb 2024 15:28:08 +0800 Message-Id: <20240201072818.327930-9-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, T_SCC_BODY_TEXT_LINE=-0.01, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This callback will be used to initialize base and public elements in IOMMUFDDevice, with the exception of iommufd and devid which are initialized early in attachment. Signed-off-by: Zhenzhong Duan --- hw/vfio/iommufd.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/hw/vfio/iommufd.c b/hw/vfio/iommufd.c index 5d50549713..7d39d7a5fa 100644 --- a/hw/vfio/iommufd.c +++ b/hw/vfio/iommufd.c @@ -621,6 +621,11 @@ out_single: return ret; } +static void vfio_cdev_host_iommu_device_init(VFIODevice *vbasedev) +{ + iommufd_device_init(&vbasedev->iommufd_dev); +} + static void vfio_iommu_iommufd_class_init(ObjectClass *klass, void *data) { VFIOIOMMUClass *vioc = VFIO_IOMMU_CLASS(klass); @@ -630,6 +635,7 @@ static void vfio_iommu_iommufd_class_init(ObjectClass *klass, void *data) vioc->attach_device = iommufd_cdev_attach; vioc->detach_device = iommufd_cdev_detach; vioc->pci_hot_reset = iommufd_cdev_pci_hot_reset; + vioc->host_iommu_device_init = vfio_cdev_host_iommu_device_init; }; static const TypeInfo types[] = { From patchwork Thu Feb 1 07:28:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540736 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 47644C47DDF for ; Thu, 1 Feb 2024 07:33:35 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRXf-0005xi-2Z; Thu, 01 Feb 2024 02:31:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRXc-0005up-GZ for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:21 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRXT-0001hD-8C for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:20 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772671; x=1738308671; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=sw26yu89PSA3YDAaJKd7N9dsi9pYZXg/gUW5kOZpYbQ=; b=QdrX89KJM1sTXC29x4Ee7LGZw5CJSAtzYe7Q3tDGPGjppduGAiWfcrUm 8GzZ/eLoCMmjJ7IbCR3j2xmehcNEdYkTA2jUMVAy/QCO0NVRjxGhGfQvc wveWrMAeZ0wcO1GJJS1cf/wk0zzsTvyZ1OQA2CB+8VHkJVxXPKgmd23t5 NQQ3OuF8P9XNWsT1CdbVCEXcp9tiC3imB9Oe7RrbEmoe6Avnxb/XOlMcZ r6lG+3Pi+7gpuegxeo1+zHnv0iLH2fchA2/LjvDyYtxSapc3XsWmHtjk7 pxfHM0+2h3uHrNsCZ8obaqSsi17Rhbf2+GjOGg47ji9+JQr50WtH3g9w3 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402676746" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402676746" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:01 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443278" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:30:57 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Zhenzhong Duan Subject: [PATCH rfcv2 09/18] vfio/pci: Initialize host iommu device instance after attachment Date: Thu, 1 Feb 2024 15:28:09 +0800 Message-Id: <20240201072818.327930-10-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Zhenzhong Duan --- hw/vfio/pci.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index d1e1b8cb89..dedb64fc08 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3006,6 +3006,9 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) goto error; } + /* Initialize host iommu device after attachment succeed */ + host_iommu_device_init(vbasedev); + vfio_populate_device(vdev, &err); if (err) { error_propagate(errp, err); From patchwork Thu Feb 1 07:28:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540734 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 22DB0C47DDF for ; Thu, 1 Feb 2024 07:33:17 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRXf-0005xy-Nm; Thu, 01 Feb 2024 02:31:23 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRXd-0005x3-S9 for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:21 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRXV-0001hL-7u for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:21 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772673; x=1738308673; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=id+Z1Tr+dw3DYND8HRmXBEtVfPDghhlU8wnr/4KPBxo=; b=Y0plqqLpXo9yJF7h5T5W2kCwCu1FWzia/w7oSjEhzhM8RuJY3g1RieEz ES048kHXaxks2DZ4A4KqIVAzb2vJgYc2l01shpqjxfghRwD3KzdGw2JMK qNS3tvhNtoLh36JwTy+fpt2ZIavCpxj1/HD3uGgiaeMfjoiENOGv5XpBK 7BW5wsBU2wnjhPy3aIBmIGYqGwakBkTGtIXp000A7Dux/LtVfxgks3hWE PNwJP54LarMLwoLzxsWW8KunwAV4g4maXlVpaQkiqe0vjH3oH/ZZSFkKu i+qBAxL8B+fG80MJ4bzyLOfKLgfmrAsgXemdptCE4Ad46lNoMHqE9vf2F A==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402676763" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402676763" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443297" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:02 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Yi Sun , Zhenzhong Duan , Marcel Apfelbaum Subject: [PATCH rfcv2 10/18] hw/pci: Introduce pci_device_set/unset_iommu_device() Date: Thu, 1 Feb 2024 15:28:10 +0800 Message-Id: <20240201072818.327930-11-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Yi Liu This adds pci_device_set/unset_iommu_device() to set/unset HostIOMMUDevice for a given PCIe device. Caller of set should fail if set operation fails. Extract out pci_device_get_iommu_bus_devfn() to facilitate implementation of pci_device_set/unset_iommu_device(). Signed-off-by: Yi Liu Signed-off-by: Yi Sun Signed-off-by: Nicolin Chen Signed-off-by: Zhenzhong Duan --- include/hw/pci/pci.h | 38 ++++++++++++++++++++++++++- hw/pci/pci.c | 62 +++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 96 insertions(+), 4 deletions(-) diff --git a/include/hw/pci/pci.h b/include/hw/pci/pci.h index fa6313aabc..5b471fd380 100644 --- a/include/hw/pci/pci.h +++ b/include/hw/pci/pci.h @@ -3,6 +3,7 @@ #include "exec/memory.h" #include "sysemu/dma.h" +#include "sysemu/host_iommu_device.h" /* PCI includes legacy ISA access. */ #include "hw/isa/isa.h" @@ -384,10 +385,45 @@ typedef struct PCIIOMMUOps { * * @devfn: device and function number */ - AddressSpace * (*get_address_space)(PCIBus *bus, void *opaque, int devfn); + AddressSpace * (*get_address_space)(PCIBus *bus, void *opaque, int devfn); + /** + * @set_iommu_device: set iommufd device for a PCI device to vIOMMU + * + * Optional callback, if not implemented in vIOMMU, then vIOMMU can't + * utilize iommufd specific features. + * + * Return true if iommufd device is accepted, or else return false with + * errp set. + * + * @bus: the #PCIBus of the PCI device. + * + * @opaque: the data passed to pci_setup_iommu(). + * + * @devfn: device and function number of the PCI device. + * + * @dev: the data structure representing host assigned device. + * + */ + int (*set_iommu_device)(PCIBus *bus, void *opaque, int devfn, + HostIOMMUDevice *dev, Error **errp); + /** + * @unset_iommu_device: unset iommufd device for a PCI device from vIOMMU + * + * Optional callback. + * + * @bus: the #PCIBus of the PCI device. + * + * @opaque: the data passed to pci_setup_iommu(). + * + * @devfn: device and function number of the PCI device. + */ + void (*unset_iommu_device)(PCIBus *bus, void *opaque, int devfn); } PCIIOMMUOps; AddressSpace *pci_device_iommu_address_space(PCIDevice *dev); +int pci_device_set_iommu_device(PCIDevice *dev, HostIOMMUDevice *base_dev, + Error **errp); +void pci_device_unset_iommu_device(PCIDevice *dev); /** * pci_setup_iommu: Initialize specific IOMMU handlers for a PCIBus diff --git a/hw/pci/pci.c b/hw/pci/pci.c index 76080af580..8078307963 100644 --- a/hw/pci/pci.c +++ b/hw/pci/pci.c @@ -2672,11 +2672,14 @@ static void pci_device_class_base_init(ObjectClass *klass, void *data) } } -AddressSpace *pci_device_iommu_address_space(PCIDevice *dev) +static void pci_device_get_iommu_bus_devfn(PCIDevice *dev, + PCIBus **aliased_bus, + PCIBus **piommu_bus, + int *aliased_devfn) { PCIBus *bus = pci_get_bus(dev); PCIBus *iommu_bus = bus; - uint8_t devfn = dev->devfn; + int devfn = dev->devfn; while (iommu_bus && !iommu_bus->iommu_ops && iommu_bus->parent_dev) { PCIBus *parent_bus = pci_get_bus(iommu_bus->parent_dev); @@ -2717,13 +2720,66 @@ AddressSpace *pci_device_iommu_address_space(PCIDevice *dev) iommu_bus = parent_bus; } - if (!pci_bus_bypass_iommu(bus) && iommu_bus->iommu_ops) { + + assert(0 <= devfn && devfn < PCI_DEVFN_MAX); + assert(iommu_bus); + + if (pci_bus_bypass_iommu(bus) || !iommu_bus->iommu_ops) { + iommu_bus = NULL; + } + + *piommu_bus = iommu_bus; + + if (aliased_bus) { + *aliased_bus = bus; + } + + if (aliased_devfn) { + *aliased_devfn = devfn; + } +} + +AddressSpace *pci_device_iommu_address_space(PCIDevice *dev) +{ + PCIBus *bus; + PCIBus *iommu_bus; + int devfn; + + pci_device_get_iommu_bus_devfn(dev, &bus, &iommu_bus, &devfn); + if (iommu_bus) { return iommu_bus->iommu_ops->get_address_space(bus, iommu_bus->iommu_opaque, devfn); } return &address_space_memory; } +int pci_device_set_iommu_device(PCIDevice *dev, HostIOMMUDevice *base_dev, + Error **errp) +{ + PCIBus *iommu_bus; + + pci_device_get_iommu_bus_devfn(dev, NULL, &iommu_bus, NULL); + if (iommu_bus && iommu_bus->iommu_ops->set_iommu_device) { + return iommu_bus->iommu_ops->set_iommu_device(pci_get_bus(dev), + iommu_bus->iommu_opaque, + dev->devfn, base_dev, + errp); + } + return 0; +} + +void pci_device_unset_iommu_device(PCIDevice *dev) +{ + PCIBus *iommu_bus; + + pci_device_get_iommu_bus_devfn(dev, NULL, &iommu_bus, NULL); + if (iommu_bus && iommu_bus->iommu_ops->unset_iommu_device) { + return iommu_bus->iommu_ops->unset_iommu_device(pci_get_bus(dev), + iommu_bus->iommu_opaque, + dev->devfn); + } +} + void pci_setup_iommu(PCIBus *bus, const PCIIOMMUOps *ops, void *opaque) { /* From patchwork Thu Feb 1 07:28:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540738 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CAC2DC48286 for ; Thu, 1 Feb 2024 07:33:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRXh-00062f-VV; Thu, 01 Feb 2024 02:31:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRXf-0005xS-IX for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:23 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRXb-0001h2-LZ for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772679; x=1738308679; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pb+pcDN+DMxwXrY6c0YeGaqOsNGV5EP3UG4ckxSOIyU=; b=RzAUl9z/GRZHN+rJ6j/MlSZnrbEIV5y0m17TkV+XuJcv7/+zZIzpBTNz e/WBm8smNG70Jp16hWe1ODtNZebh1DgJXjTzX4at5xOTWw4GBFF1zLJT2 3pTUhZf1cjynjlnsz6rfcEeDOcismy1e11O8H7bVV163fzbngog/4H91r 3MNb7mSRCwZ7RF5WxoEe7ZjmPWJUYOCQC+biEGgBWX2OUTFBsptxXPCKc A266xWmbG9p+ARHkU1ZZx8fHIkay9zNQHIZ8wFJLaTcPXxKE+yYRTIJOy TgYzyE6Vrv+L80XGbSstOmL3GaFPOd857VMHKGPmQNQvIUO9W/fDU/pqy A==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402676811" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402676811" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443313" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:07 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Yi Sun , Zhenzhong Duan , Paolo Bonzini , Richard Henderson , Eduardo Habkost , Marcel Apfelbaum Subject: [PATCH rfcv2 11/18] intel_iommu: Add set/unset_iommu_device callback Date: Thu, 1 Feb 2024 15:28:11 +0800 Message-Id: <20240201072818.327930-12-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Yi Liu This adds set/unset_iommu_device() implementation in Intel vIOMMU. In set call, a pointer to host IOMMU device info is stored in hash table indexed by PCI BDF. Signed-off-by: Yi Liu Signed-off-by: Yi Sun Signed-off-by: Zhenzhong Duan --- hw/i386/intel_iommu_internal.h | 14 +++++++ include/hw/i386/intel_iommu.h | 2 + hw/i386/intel_iommu.c | 74 ++++++++++++++++++++++++++++++++++ 3 files changed, 90 insertions(+) diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h index f8cf99bddf..3301f54b35 100644 --- a/hw/i386/intel_iommu_internal.h +++ b/hw/i386/intel_iommu_internal.h @@ -28,6 +28,8 @@ #ifndef HW_I386_INTEL_IOMMU_INTERNAL_H #define HW_I386_INTEL_IOMMU_INTERNAL_H #include "hw/i386/intel_iommu.h" +#include "sysemu/host_iommu_device.h" +#include "hw/vfio/vfio-common.h" /* * Intel IOMMU register specification @@ -537,4 +539,16 @@ typedef struct VTDRootEntry VTDRootEntry; #define VTD_SL_IGN_COM 0xbff0000000000000ULL #define VTD_SL_TM (1ULL << 62) + +typedef struct VTDHostIOMMUDevice { + IntelIOMMUState *iommu_state; + PCIBus *bus; + uint8_t devfn; + union { + HostIOMMUDevice *dev; + IOMMULegacyDevice *ldev; + IOMMUFDDevice *idev; + }; + QLIST_ENTRY(VTDHostIOMMUDevice) next; +} VTDHostIOMMUDevice; #endif diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h index 7fa0a695c8..bbc7b96add 100644 --- a/include/hw/i386/intel_iommu.h +++ b/include/hw/i386/intel_iommu.h @@ -292,6 +292,8 @@ struct IntelIOMMUState { /* list of registered notifiers */ QLIST_HEAD(, VTDAddressSpace) vtd_as_with_notifiers; + GHashTable *vtd_host_iommu_dev; /* VTDHostIOMMUDevice */ + /* interrupt remapping */ bool intr_enabled; /* Whether guest enabled IR */ dma_addr_t intr_root; /* Interrupt remapping table pointer */ diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 1a07faddb4..9b62441439 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -237,6 +237,13 @@ static gboolean vtd_as_equal(gconstpointer v1, gconstpointer v2) (key1->pasid == key2->pasid); } +static gboolean vtd_as_idev_equal(gconstpointer v1, gconstpointer v2) +{ + const struct vtd_as_key *key1 = v1; + const struct vtd_as_key *key2 = v2; + + return (key1->bus == key2->bus) && (key1->devfn == key2->devfn); +} /* * Note that we use pointer to PCIBus as the key, so hashing/shifting * based on the pointer value is intended. Note that we deal with @@ -3812,6 +3819,68 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, return vtd_dev_as; } +static int vtd_dev_set_iommu_device(PCIBus *bus, void *opaque, int devfn, + HostIOMMUDevice *base_dev, Error **errp) +{ + IntelIOMMUState *s = opaque; + VTDHostIOMMUDevice *vtd_hdev; + struct vtd_as_key key = { + .bus = bus, + .devfn = devfn, + }; + struct vtd_as_key *new_key; + + assert(base_dev); + + vtd_iommu_lock(s); + + vtd_hdev = g_hash_table_lookup(s->vtd_host_iommu_dev, &key); + + if (vtd_hdev) { + error_setg(errp, "IOMMUFD device already exist"); + vtd_iommu_unlock(s); + return -EEXIST; + } + + vtd_hdev = g_malloc0(sizeof(VTDHostIOMMUDevice)); + vtd_hdev->bus = bus; + vtd_hdev->devfn = (uint8_t)devfn; + vtd_hdev->iommu_state = s; + vtd_hdev->dev = base_dev; + + new_key = g_malloc(sizeof(*new_key)); + new_key->bus = bus; + new_key->devfn = devfn; + + g_hash_table_insert(s->vtd_host_iommu_dev, new_key, vtd_hdev); + + vtd_iommu_unlock(s); + + return 0; +} + +static void vtd_dev_unset_iommu_device(PCIBus *bus, void *opaque, int devfn) +{ + IntelIOMMUState *s = opaque; + VTDHostIOMMUDevice *vtd_hdev; + struct vtd_as_key key = { + .bus = bus, + .devfn = devfn, + }; + + vtd_iommu_lock(s); + + vtd_hdev = g_hash_table_lookup(s->vtd_host_iommu_dev, &key); + if (!vtd_hdev) { + vtd_iommu_unlock(s); + return; + } + + g_hash_table_remove(s->vtd_host_iommu_dev, &key); + + vtd_iommu_unlock(s); +} + /* Unmap the whole range in the notifier's scope. */ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n) { @@ -4107,6 +4176,8 @@ static AddressSpace *vtd_host_dma_iommu(PCIBus *bus, void *opaque, int devfn) static PCIIOMMUOps vtd_iommu_ops = { .get_address_space = vtd_host_dma_iommu, + .set_iommu_device = vtd_dev_set_iommu_device, + .unset_iommu_device = vtd_dev_unset_iommu_device, }; static bool vtd_decide_config(IntelIOMMUState *s, Error **errp) @@ -4230,6 +4301,9 @@ static void vtd_realize(DeviceState *dev, Error **errp) g_free, g_free); s->vtd_address_spaces = g_hash_table_new_full(vtd_as_hash, vtd_as_equal, g_free, g_free); + s->vtd_host_iommu_dev = g_hash_table_new_full(vtd_as_hash, + vtd_as_idev_equal, + g_free, g_free); vtd_init(s); pci_setup_iommu(bus, &vtd_iommu_ops, dev); /* Pseudo address space under root PCI bus. */ From patchwork Thu Feb 1 07:28:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540737 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4EBA5C47DDB for ; Thu, 1 Feb 2024 07:33:36 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRXi-00064V-RD; Thu, 01 Feb 2024 02:31:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRXf-0005y2-M2 for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:23 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRXc-0001hD-Rp for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:22 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772680; x=1738308680; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KwVL3I9PgS4s8Z7KV0SvAHDS/KkgvNyyFOLt/4IucP0=; b=ZfbG9opJRQkXgCBNyLh6Puze+DFDpHZ28yQXa/GGV4zPeRlbrd1QOjhd DBf68QTkxqLuyGf3Oe2ZTkgssX5lWlA+iNlIq4k2K9/gJbrg2KIuYR5Wv IYslmgNRCBoEq8uQjs0xHKUeAZUrOS7iakatpge/N680WbQInO6zpFgUm wMzOmifr0IsepRgb7hN1iXU8yRld85/sgvHdrnFQQicTJj2NXtttnkw54 4rdqezoxxUZFK5XHMYlICyMBLXARfoq6esUaaZGI1j1KEO8SJ83aVjkIQ n1DGXz0EDhH+NaNnhdw7u4lH1/JGTtBHd4+YfzFyYiLdRX5ZkvqIZiH8R A==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402676829" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402676829" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:17 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443327" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:13 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Yi Sun Subject: [PATCH rfcv2 12/18] vfio: Initialize host IOMMU device and pass to vIOMMU Date: Thu, 1 Feb 2024 15:28:12 +0800 Message-Id: <20240201072818.327930-13-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Initialize host IOMMU device in vfio and pass to vIOMMU, so that vIOMMU could get hw IOMMU information. Support both iommufd and legacy backend. Originally-by: Yi Liu Signed-off-by: Nicolin Chen Signed-off-by: Yi Sun Signed-off-by: Zhenzhong Duan --- hw/vfio/pci.c | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c index dedb64fc08..b23c5ea790 100644 --- a/hw/vfio/pci.c +++ b/hw/vfio/pci.c @@ -3112,11 +3112,17 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) vfio_bars_register(vdev); - ret = vfio_add_capabilities(vdev, errp); + ret = pci_device_set_iommu_device(pdev, &vbasedev->base_hdev, errp); if (ret) { + error_prepend(errp, "Failed to set iommu_device: "); goto out_teardown; } + ret = vfio_add_capabilities(vdev, errp); + if (ret) { + goto out_unset_idev; + } + if (vdev->vga) { vfio_vga_quirk_setup(vdev); } @@ -3133,7 +3139,7 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) error_setg(errp, "cannot support IGD OpRegion feature on hotplugged " "device"); - goto out_teardown; + goto out_unset_idev; } ret = vfio_get_dev_region_info(vbasedev, @@ -3142,13 +3148,13 @@ static void vfio_realize(PCIDevice *pdev, Error **errp) if (ret) { error_setg_errno(errp, -ret, "does not support requested IGD OpRegion feature"); - goto out_teardown; + goto out_unset_idev; } ret = vfio_pci_igd_opregion_init(vdev, opregion, errp); g_free(opregion); if (ret) { - goto out_teardown; + goto out_unset_idev; } } @@ -3234,6 +3240,8 @@ out_deregister: if (vdev->intx.mmap_timer) { timer_free(vdev->intx.mmap_timer); } +out_unset_idev: + pci_device_unset_iommu_device(pdev); out_teardown: vfio_teardown_msi(vdev); vfio_bars_exit(vdev); @@ -3262,6 +3270,7 @@ static void vfio_instance_finalize(Object *obj) static void vfio_exitfn(PCIDevice *pdev) { VFIOPCIDevice *vdev = VFIO_PCI(pdev); + VFIODevice *vbasedev = &vdev->vbasedev; vfio_unregister_req_notifier(vdev); vfio_unregister_err_notifier(vdev); @@ -3276,7 +3285,8 @@ static void vfio_exitfn(PCIDevice *pdev) vfio_teardown_msi(vdev); vfio_pci_disable_rp_atomics(vdev); vfio_bars_exit(vdev); - vfio_migration_exit(&vdev->vbasedev); + vfio_migration_exit(vbasedev); + pci_device_unset_iommu_device(pdev); } static void vfio_pci_reset(DeviceState *dev) From patchwork Thu Feb 1 07:28:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540756 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4C0A9C47DDF for ; Thu, 1 Feb 2024 07:33:57 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRXn-00069L-9u; Thu, 01 Feb 2024 02:31:31 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRXi-00063e-Eb for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:26 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRXg-0001hD-Ji for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:26 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772684; x=1738308684; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=648gzLYMico2GjbX5DwPYwkXtPATDWt90fsopq2YixU=; b=Tb4nRNxRu3Ss823aZgaDi1s28ANp+VPaug2w7MclDPFlMgptT7RYcWty 13Q3FTF1X9X0jPQwvgxMfPP3vTn5f3MVerIVLKfSkffZDIl73d2HS62cR OPWoxalJTWaL2SpF+/jQf0tW6Ymmxt0aw/E4lx9rJpu1mrcCbafof9M3m 5O4QwljBVLknsLf6LH1+K4BFVN14lVM+NPzgf7HmL+DlBv389vRrVPL3H 3jK2wsECyhhAijqkp7ItBlD+ZSBtj/rCrPpo4XkvYPr5rH0HJBLSQXZ8p v8pnRYmByxnUsnmw4px2mIPl9B3uwrrIDbS1x72HPZ7vDqT6ivlAkrqy0 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402676856" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402676856" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:23 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443346" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:18 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Marcel Apfelbaum , Paolo Bonzini , Richard Henderson , Eduardo Habkost Subject: [PATCH rfcv2 13/18] intel_iommu: Extract out vtd_cap_init to initialize cap/ecap Date: Thu, 1 Feb 2024 15:28:13 +0800 Message-Id: <20240201072818.327930-14-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org This is a prerequisite for host cap/ecap sync. No functional change intended. Reviewed-by: Eric Auger Signed-off-by: Zhenzhong Duan --- hw/i386/intel_iommu.c | 93 ++++++++++++++++++++++++------------------- 1 file changed, 51 insertions(+), 42 deletions(-) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 9b62441439..ffa1ad6429 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -4003,30 +4003,10 @@ static void vtd_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n) return; } -/* Do the initialization. It will also be called when reset, so pay - * attention when adding new initialization stuff. - */ -static void vtd_init(IntelIOMMUState *s) +static void vtd_cap_init(IntelIOMMUState *s) { X86IOMMUState *x86_iommu = X86_IOMMU_DEVICE(s); - memset(s->csr, 0, DMAR_REG_SIZE); - memset(s->wmask, 0, DMAR_REG_SIZE); - memset(s->w1cmask, 0, DMAR_REG_SIZE); - memset(s->womask, 0, DMAR_REG_SIZE); - - s->root = 0; - s->root_scalable = false; - s->dmar_enabled = false; - s->intr_enabled = false; - s->iq_head = 0; - s->iq_tail = 0; - s->iq = 0; - s->iq_size = 0; - s->qi_enabled = false; - s->iq_last_desc_type = VTD_INV_DESC_NONE; - s->iq_dw = false; - s->next_frcd_reg = 0; s->cap = VTD_CAP_FRO | VTD_CAP_NFR | VTD_CAP_ND | VTD_CAP_MAMV | VTD_CAP_PSI | VTD_CAP_SLLPS | VTD_CAP_MGAW(s->aw_bits); @@ -4043,27 +4023,6 @@ static void vtd_init(IntelIOMMUState *s) } s->ecap = VTD_ECAP_QI | VTD_ECAP_IRO; - /* - * Rsvd field masks for spte - */ - vtd_spte_rsvd[0] = ~0ULL; - vtd_spte_rsvd[1] = VTD_SPTE_PAGE_L1_RSVD_MASK(s->aw_bits, - x86_iommu->dt_supported); - vtd_spte_rsvd[2] = VTD_SPTE_PAGE_L2_RSVD_MASK(s->aw_bits); - vtd_spte_rsvd[3] = VTD_SPTE_PAGE_L3_RSVD_MASK(s->aw_bits); - vtd_spte_rsvd[4] = VTD_SPTE_PAGE_L4_RSVD_MASK(s->aw_bits); - - vtd_spte_rsvd_large[2] = VTD_SPTE_LPAGE_L2_RSVD_MASK(s->aw_bits, - x86_iommu->dt_supported); - vtd_spte_rsvd_large[3] = VTD_SPTE_LPAGE_L3_RSVD_MASK(s->aw_bits, - x86_iommu->dt_supported); - - if (s->scalable_mode || s->snoop_control) { - vtd_spte_rsvd[1] &= ~VTD_SPTE_SNP; - vtd_spte_rsvd_large[2] &= ~VTD_SPTE_SNP; - vtd_spte_rsvd_large[3] &= ~VTD_SPTE_SNP; - } - if (x86_iommu_ir_supported(x86_iommu)) { s->ecap |= VTD_ECAP_IR | VTD_ECAP_MHMV; if (s->intr_eim == ON_OFF_AUTO_ON) { @@ -4096,6 +4055,56 @@ static void vtd_init(IntelIOMMUState *s) if (s->pasid) { s->ecap |= VTD_ECAP_PASID; } +} + +/* + * Do the initialization. It will also be called when reset, so pay + * attention when adding new initialization stuff. + */ +static void vtd_init(IntelIOMMUState *s) +{ + X86IOMMUState *x86_iommu = X86_IOMMU_DEVICE(s); + + memset(s->csr, 0, DMAR_REG_SIZE); + memset(s->wmask, 0, DMAR_REG_SIZE); + memset(s->w1cmask, 0, DMAR_REG_SIZE); + memset(s->womask, 0, DMAR_REG_SIZE); + + s->root = 0; + s->root_scalable = false; + s->dmar_enabled = false; + s->intr_enabled = false; + s->iq_head = 0; + s->iq_tail = 0; + s->iq = 0; + s->iq_size = 0; + s->qi_enabled = false; + s->iq_last_desc_type = VTD_INV_DESC_NONE; + s->iq_dw = false; + s->next_frcd_reg = 0; + + vtd_cap_init(s); + + /* + * Rsvd field masks for spte + */ + vtd_spte_rsvd[0] = ~0ULL; + vtd_spte_rsvd[1] = VTD_SPTE_PAGE_L1_RSVD_MASK(s->aw_bits, + x86_iommu->dt_supported); + vtd_spte_rsvd[2] = VTD_SPTE_PAGE_L2_RSVD_MASK(s->aw_bits); + vtd_spte_rsvd[3] = VTD_SPTE_PAGE_L3_RSVD_MASK(s->aw_bits); + vtd_spte_rsvd[4] = VTD_SPTE_PAGE_L4_RSVD_MASK(s->aw_bits); + + vtd_spte_rsvd_large[2] = VTD_SPTE_LPAGE_L2_RSVD_MASK(s->aw_bits, + x86_iommu->dt_supported); + vtd_spte_rsvd_large[3] = VTD_SPTE_LPAGE_L3_RSVD_MASK(s->aw_bits, + x86_iommu->dt_supported); + + if (s->scalable_mode || s->snoop_control) { + vtd_spte_rsvd[1] &= ~VTD_SPTE_SNP; + vtd_spte_rsvd_large[2] &= ~VTD_SPTE_SNP; + vtd_spte_rsvd_large[3] &= ~VTD_SPTE_SNP; + } vtd_reset_caches(s); From patchwork Thu Feb 1 07:28:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540739 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1F90CC47DDB for ; Thu, 1 Feb 2024 07:33:48 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRXs-0006ST-QI; Thu, 01 Feb 2024 02:31:36 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRXp-0006HV-B5 for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:33 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRXl-0001hD-Mo for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772689; x=1738308689; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Idmve9hI3M5yptuDUtlLrnV5peWHH6TGUubN6UCkLTk=; b=BBYYiSRWEizSLO01xrEc3EKKfUxAenLp46LpDg941Q0bPfMukQYkaFZj 84XDzm1B6SBLr892ZMZQLLzLHvDeYtnH4hAxg3D5csfV+Q1NF1UeDtnFq /Lsut2yX+ttc6dSSQ4ktYiRo/C+z/+mdCASYq6dZBczheIQVnNwcxJklg FVKO6yVdXrKGvAY02qz36ZG+apqknoyXoXLTxdxmK6Uc4iRIlHZQ15HWv YOwq/8Hj2ScBj5O1EqorZfSQlvtcNxZmh7X0cM0COujuJ++tGwVKV+hzP tDpXmN9SC/gx3Q3V0QPQdkFj8LdHVGru4Pzlwjy2K61ttzYIy9Ko+lAVF w==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402676889" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402676889" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:28 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443363" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:23 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Yi Sun , Zhenzhong Duan , Paolo Bonzini , Richard Henderson , Eduardo Habkost , Marcel Apfelbaum Subject: [PATCH rfcv2 14/18] intel_iommu: Add a framework to check and sync host IOMMU cap/ecap Date: Thu, 1 Feb 2024 15:28:14 +0800 Message-Id: <20240201072818.327930-15-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Yi Liu Add a framework to check and synchronize host IOMMU cap/ecap with vIOMMU cap/ecap. The sequence will be: vtd_cap_init() initializes iommu->cap/ecap. vtd_check_hdev() update iommu->cap/ecap based on host cap/ecap. iommu->cap_frozen set when machine create done, iommu->cap/ecap become readonly. Implementation details for different backends will be in following patches. Signed-off-by: Yi Liu Signed-off-by: Yi Sun Signed-off-by: Zhenzhong Duan --- include/hw/i386/intel_iommu.h | 1 + hw/i386/intel_iommu.c | 41 ++++++++++++++++++++++++++++++++++- 2 files changed, 41 insertions(+), 1 deletion(-) diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h index bbc7b96add..c71a133820 100644 --- a/include/hw/i386/intel_iommu.h +++ b/include/hw/i386/intel_iommu.h @@ -283,6 +283,7 @@ struct IntelIOMMUState { uint64_t cap; /* The value of capability reg */ uint64_t ecap; /* The value of extended capability reg */ + bool cap_frozen; /* cap/ecap become read-only after frozen */ uint32_t context_cache_gen; /* Should be in [1,MAX] */ GHashTable *iotlb; /* IOTLB */ diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index ffa1ad6429..7ed2b79669 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -3819,6 +3819,31 @@ VTDAddressSpace *vtd_find_add_as(IntelIOMMUState *s, PCIBus *bus, return vtd_dev_as; } +static int vtd_check_legacy_hdev(IntelIOMMUState *s, + IOMMULegacyDevice *ldev, + Error **errp) +{ + return 0; +} + +static int vtd_check_iommufd_hdev(IntelIOMMUState *s, + IOMMUFDDevice *idev, + Error **errp) +{ + return 0; +} + +static int vtd_check_hdev(IntelIOMMUState *s, VTDHostIOMMUDevice *vtd_hdev, + Error **errp) +{ + HostIOMMUDevice *base_dev = vtd_hdev->dev; + + if (base_dev->type == HID_LEGACY) { + return vtd_check_legacy_hdev(s, vtd_hdev->ldev, errp); + } + return vtd_check_iommufd_hdev(s, vtd_hdev->idev, errp); +} + static int vtd_dev_set_iommu_device(PCIBus *bus, void *opaque, int devfn, HostIOMMUDevice *base_dev, Error **errp) { @@ -3829,6 +3854,7 @@ static int vtd_dev_set_iommu_device(PCIBus *bus, void *opaque, int devfn, .devfn = devfn, }; struct vtd_as_key *new_key; + int ret; assert(base_dev); @@ -3848,6 +3874,13 @@ static int vtd_dev_set_iommu_device(PCIBus *bus, void *opaque, int devfn, vtd_hdev->iommu_state = s; vtd_hdev->dev = base_dev; + ret = vtd_check_hdev(s, vtd_hdev, errp); + if (ret) { + g_free(vtd_hdev); + vtd_iommu_unlock(s); + return ret; + } + new_key = g_malloc(sizeof(*new_key)); new_key->bus = bus; new_key->devfn = devfn; @@ -4083,7 +4116,9 @@ static void vtd_init(IntelIOMMUState *s) s->iq_dw = false; s->next_frcd_reg = 0; - vtd_cap_init(s); + if (!s->cap_frozen) { + vtd_cap_init(s); + } /* * Rsvd field masks for spte @@ -4254,6 +4289,10 @@ static int vtd_machine_done_notify_one(Object *child, void *unused) static void vtd_machine_done_hook(Notifier *notifier, void *unused) { + IntelIOMMUState *iommu = INTEL_IOMMU_DEVICE(x86_iommu_get_default()); + + iommu->cap_frozen = true; + object_child_foreach_recursive(object_get_root(), vtd_machine_done_notify_one, NULL); } From patchwork Thu Feb 1 07:28:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540732 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7670FC47DDB for ; Thu, 1 Feb 2024 07:32:15 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRYE-00075p-Ro; Thu, 01 Feb 2024 02:31:58 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRY4-0006kL-Qq for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:50 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRXr-0001hD-Ob for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:37 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772695; x=1738308695; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=4UuNU52+wzlDsbtsKgRQzd/clhv4zIKmdvD3x0j7SeU=; b=kB4Z1eRUBjkQFRWxhO0LUzRLJ5dWICyd+d325pT6Q20nAIGLKOgYvdSG G/NXXOeoiSB7fx5VI84jchp69LtKahcY8n9XzhJb7hSDHcCKKwQ6yRpvw WczHERK7Lma11lP2xNps6aSDmgVDMe67PcvEJ5f0eiFRHBkjCAxQLSlqc mEdq+B2ebH4XRP6bu9dDzJKcMLwLzjQYFdOXeB8ym5CfSJ3Tv4Zvo5cYO kXWarlxntbM6sAVVeYaWqZZah58Jf0OBBii30W8Fa0BEwW20coZgA9HWV LxSuKz0xmeNavcX6PyHgBV2t6XgUxlbdtfsu6O63Vw/lO8lEdItqS+pTP Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402676925" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402676925" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443378" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:29 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Yi Sun Subject: [PATCH rfcv2 15/18] backends/iommufd: Introduce helper function iommufd_device_get_info() Date: Thu, 1 Feb 2024 15:28:15 +0800 Message-Id: <20240201072818.327930-16-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Introduce a helper function iommufd_device_get_info() to get host IOMMU related information through iommufd uAPI. Signed-off-by: Yi Liu Signed-off-by: Yi Sun Signed-off-by: Zhenzhong Duan --- include/sysemu/iommufd.h | 4 ++++ backends/iommufd.c | 25 ++++++++++++++++++++++++- 2 files changed, 28 insertions(+), 1 deletion(-) diff --git a/include/sysemu/iommufd.h b/include/sysemu/iommufd.h index c3f3469760..ec8b80d8d9 100644 --- a/include/sysemu/iommufd.h +++ b/include/sysemu/iommufd.h @@ -4,6 +4,7 @@ #include "qom/object.h" #include "exec/hwaddr.h" #include "exec/cpu-common.h" +#include #include "sysemu/host_iommu_device.h" #define TYPE_IOMMUFD_BACKEND "iommufd" @@ -47,4 +48,7 @@ typedef struct IOMMUFDDevice { } IOMMUFDDevice; void iommufd_device_init(IOMMUFDDevice *idev); +int iommufd_device_get_info(IOMMUFDDevice *idev, + enum iommu_hw_info_type *type, + uint32_t len, void *data, Error **errp); #endif diff --git a/backends/iommufd.c b/backends/iommufd.c index d92791bba9..1b0b991747 100644 --- a/backends/iommufd.c +++ b/backends/iommufd.c @@ -20,7 +20,6 @@ #include "monitor/monitor.h" #include "trace.h" #include -#include static void iommufd_backend_init(Object *obj) { @@ -237,3 +236,27 @@ void iommufd_device_init(IOMMUFDDevice *idev) host_iommu_base_device_init(&idev->base, HID_IOMMUFD, sizeof(IOMMUFDDevice)); } + +int iommufd_device_get_info(IOMMUFDDevice *idev, + enum iommu_hw_info_type *type, + uint32_t len, void *data, Error **errp) +{ + struct iommu_hw_info info = { + .size = sizeof(info), + .flags = 0, + .dev_id = idev->devid, + .data_len = len, + .__reserved = 0, + .data_uptr = (uintptr_t)data, + }; + int ret; + + ret = ioctl(idev->iommufd->fd, IOMMU_GET_HW_INFO, &info); + if (ret) { + error_setg_errno(errp, errno, "Failed to get hardware info"); + } else { + *type = info.out_data_type; + } + + return ret; +} From patchwork Thu Feb 1 07:28:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540733 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C7DCAC47DDF for ; Thu, 1 Feb 2024 07:32:35 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRYJ-0007ke-Gh; Thu, 01 Feb 2024 02:32:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRYA-0006qH-Jl for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:58 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRY4-0001kb-Lv for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772708; x=1738308708; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Wif64BZtnQFnyitRvIJHnqYL2A3FK/+JPBbzKxxW/VY=; b=Tmy11YyoTBAel9Vv2+XctvkcXI7jCPpsRvE1CYKyU5jHlZI7y3UcKvqu /kD1D0nUBitZZlAK3z9OcX/TIAQ4lUWgXwo9JVATMjbmIczsPLuqxdEZT cdUXEKmtaQvOhjLjHHgO0QMPE8VAHldxFqqJycT0QSkkzpAb2vbyCftli JlL0mkFsWJ73uc3YmITIJaVeSuECDfGpmxoEMjg+R93Ih3huE91EYb0+Z vOIHwql97k7K9MSSS4N4k+j13ujaOaTXNLxWvnm93QZ+3TJQtFmmY5yqR siqU1JxFtAhwUDQRaetCoSSr75nPVV2X+C3bb+FS5sET3XnrghEZXfZpw A==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402676948" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402676948" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:39 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443395" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:34 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Yi Sun , Marcel Apfelbaum , Paolo Bonzini , Richard Henderson , Eduardo Habkost Subject: [PATCH rfcv2 16/18] intel_iommu: Implement check and sync mechanism in iommufd mode Date: Thu, 1 Feb 2024 15:28:16 +0800 Message-Id: <20240201072818.327930-17-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org We use cap_frozen to mark cap/ecap read/writable or read-only, At init stage, we allow to update cap/ecap based on host IOMMU cap/ecap, but when machine create done, cap_frozen is set and we only allow checking cap/ecap for compatibility. Currently only stage-2 translation is supported which is backed by shadow page table on host side. So we don't need exact matching of each bit of cap/ecap between vIOMMU and host. However, we can still ensure compatibility of host and vIOMMU's address width at least, i.e., vIOMMU's mgaw <= host IOMMU mgaw, which is missed before. When stage-1 translation is supported in future, a.k.a. scalable modern mode, this mechanism will be further extended to check more bits. Signed-off-by: Yi Liu Signed-off-by: Yi Sun Signed-off-by: Zhenzhong Duan --- hw/i386/intel_iommu_internal.h | 1 + include/hw/i386/intel_iommu.h | 1 + hw/i386/intel_iommu.c | 29 +++++++++++++++++++++++++++++ 3 files changed, 31 insertions(+) diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h index 3301f54b35..33d2298dce 100644 --- a/hw/i386/intel_iommu_internal.h +++ b/hw/i386/intel_iommu_internal.h @@ -206,6 +206,7 @@ #define VTD_DOMAIN_ID_MASK ((1UL << VTD_DOMAIN_ID_SHIFT) - 1) #define VTD_CAP_ND (((VTD_DOMAIN_ID_SHIFT - 4) / 2) & 7ULL) #define VTD_ADDRESS_SIZE(aw) (1ULL << (aw)) +#define VTD_CAP_MGAW_MASK (0x3fULL << 16) #define VTD_CAP_MGAW(aw) ((((aw) - 1) & 0x3fULL) << 16) #define VTD_MAMV 18ULL #define VTD_CAP_MAMV (VTD_MAMV << 48) diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h index c71a133820..a0b530ebc6 100644 --- a/include/hw/i386/intel_iommu.h +++ b/include/hw/i386/intel_iommu.h @@ -47,6 +47,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(IntelIOMMUState, INTEL_IOMMU_DEVICE) #define VTD_HOST_AW_48BIT 48 #define VTD_HOST_ADDRESS_WIDTH VTD_HOST_AW_39BIT #define VTD_HAW_MASK(aw) ((1ULL << (aw)) - 1) +#define VTD_MGAW_FROM_CAP(cap) (((cap >> 16) & 0x3fULL) + 1) #define DMAR_REPORT_F_INTR (1) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 7ed2b79669..409f8a59c3 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -35,6 +35,7 @@ #include "sysemu/kvm.h" #include "sysemu/dma.h" #include "sysemu/sysemu.h" +#include "sysemu/iommufd.h" #include "hw/i386/apic_internal.h" #include "kvm/kvm_i386.h" #include "migration/vmstate.h" @@ -3830,6 +3831,34 @@ static int vtd_check_iommufd_hdev(IntelIOMMUState *s, IOMMUFDDevice *idev, Error **errp) { + struct iommu_hw_info_vtd vtd; + enum iommu_hw_info_type type = IOMMU_HW_INFO_TYPE_INTEL_VTD; + long host_mgaw, viommu_mgaw = VTD_MGAW_FROM_CAP(s->cap); + uint64_t tmp_cap = s->cap; + int ret; + + ret = iommufd_device_get_info(idev, &type, sizeof(vtd), &vtd, errp); + if (ret) { + return ret; + } + + if (type != IOMMU_HW_INFO_TYPE_INTEL_VTD) { + error_setg(errp, "IOMMU hardware is not compatible"); + return -EINVAL; + } + + host_mgaw = VTD_MGAW_FROM_CAP(vtd.cap_reg); + if (viommu_mgaw > host_mgaw) { + if (s->cap_frozen) { + error_setg(errp, "mgaw %" PRId64 " > host mgaw %" PRId64, + viommu_mgaw, host_mgaw); + return -EINVAL; + } + tmp_cap &= ~VTD_CAP_MGAW_MASK; + tmp_cap |= VTD_CAP_MGAW(host_mgaw + 1); + } + + s->cap = tmp_cap; return 0; } From patchwork Thu Feb 1 07:28:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540735 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BA6ABC47DDB for ; Thu, 1 Feb 2024 07:33:23 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRYH-0007UM-H7; Thu, 01 Feb 2024 02:32:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRYA-0006qI-KO for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:58 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRY4-0001lf-Lv for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:50 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772708; x=1738308708; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=BrGr1sksyixCvlwpq+tJwUA82IYeGZtvh6mz2FrbZg4=; b=bHAODC7OXgSG5t88frW7ahNNldIaqWbWuyfQD9zWDhFTGEXEoItZ0xBB PDLJy+iszW0l6426UufwygJNFq0Wk210q3XW7lI0TrpuLYM8qH7hDWQLT JjNqGCjFrY6HOoIeSavxinjo/2HhDqwT5ngQSLm+2oZ52gICEiJ7Rt+0s dcApCZImgw8m0LKsou8E74DsWB2LHgw3SkFCdLdwiejm8ceGNTWHvzy1Y J0mHSEBVOd0CU5O+o78NjpJTuNH6OXC/v8vnYHTGZxMc+cpprbADOnWel RGnUD6XIXb2Dl1FCUymJb0n5z4AGFnr+5u+wvI/nHbWsO/t4I1Gm7JVEW Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402676996" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402676996" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443418" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:40 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Igor Mammedov , Ani Sinha , Paolo Bonzini , Richard Henderson , Eduardo Habkost , Marcel Apfelbaum Subject: [PATCH rfcv2 17/18] intel_iommu: Use mgaw instead of s->aw_bits Date: Thu, 1 Feb 2024 15:28:17 +0800 Message-Id: <20240201072818.327930-18-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Because vIOMMU mgaw can be updated based on host IOMMU mgaw, s->aw_bits does't necessarily represent the final mgaw now but the mgaw field in s->cap does. Replace reference to s->aw_bits with a MACRO S_AW_BITS to fetch mgaw from s->cap. There are two exceptions on this, aw_bits value sanity check and s->cap initialization. ACPI DMAR table is also updated with right mgaw value. Signed-off-by: Zhenzhong Duan --- hw/i386/acpi-build.c | 3 ++- hw/i386/intel_iommu.c | 44 ++++++++++++++++++++++--------------------- 2 files changed, 25 insertions(+), 22 deletions(-) diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c index edc979379c..6467157686 100644 --- a/hw/i386/acpi-build.c +++ b/hw/i386/acpi-build.c @@ -2159,7 +2159,8 @@ build_dmar_q35(GArray *table_data, BIOSLinker *linker, const char *oem_id, acpi_table_begin(&table, table_data); /* Host Address Width */ - build_append_int_noprefix(table_data, intel_iommu->aw_bits - 1, 1); + build_append_int_noprefix(table_data, + VTD_MGAW_FROM_CAP(intel_iommu->cap), 1); build_append_int_noprefix(table_data, dmar_flags, 1); /* Flags */ g_array_append_vals(table_data, rsvd10, sizeof(rsvd10)); /* Reserved */ diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 409f8a59c3..72cc8b2c71 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -41,6 +41,8 @@ #include "migration/vmstate.h" #include "trace.h" +#define S_AW_BITS (VTD_MGAW_FROM_CAP(s->cap) + 1) + /* context entry operations */ #define VTD_CE_GET_RID2PASID(ce) \ ((ce)->val[1] & VTD_SM_CONTEXT_ENTRY_RID2PASID_MASK) @@ -1409,13 +1411,13 @@ static int vtd_root_entry_rsvd_bits_check(IntelIOMMUState *s, { /* Legacy Mode reserved bits check */ if (!s->root_scalable && - (re->hi || (re->lo & VTD_ROOT_ENTRY_RSVD(s->aw_bits)))) + (re->hi || (re->lo & VTD_ROOT_ENTRY_RSVD(S_AW_BITS)))) goto rsvd_err; /* Scalable Mode reserved bits check */ if (s->root_scalable && - ((re->lo & VTD_ROOT_ENTRY_RSVD(s->aw_bits)) || - (re->hi & VTD_ROOT_ENTRY_RSVD(s->aw_bits)))) + ((re->lo & VTD_ROOT_ENTRY_RSVD(S_AW_BITS)) || + (re->hi & VTD_ROOT_ENTRY_RSVD(S_AW_BITS)))) goto rsvd_err; return 0; @@ -1432,7 +1434,7 @@ static inline int vtd_context_entry_rsvd_bits_check(IntelIOMMUState *s, { if (!s->root_scalable && (ce->hi & VTD_CONTEXT_ENTRY_RSVD_HI || - ce->lo & VTD_CONTEXT_ENTRY_RSVD_LO(s->aw_bits))) { + ce->lo & VTD_CONTEXT_ENTRY_RSVD_LO(S_AW_BITS))) { error_report_once("%s: invalid context entry: hi=%"PRIx64 ", lo=%"PRIx64" (reserved nonzero)", __func__, ce->hi, ce->lo); @@ -1440,7 +1442,7 @@ static inline int vtd_context_entry_rsvd_bits_check(IntelIOMMUState *s, } if (s->root_scalable && - (ce->val[0] & VTD_SM_CONTEXT_ENTRY_RSVD_VAL0(s->aw_bits) || + (ce->val[0] & VTD_SM_CONTEXT_ENTRY_RSVD_VAL0(S_AW_BITS) || ce->val[1] & VTD_SM_CONTEXT_ENTRY_RSVD_VAL1 || ce->val[2] || ce->val[3])) { @@ -1571,7 +1573,7 @@ static int vtd_sync_shadow_page_table_range(VTDAddressSpace *vtd_as, .hook_fn = vtd_sync_shadow_page_hook, .private = (void *)&vtd_as->iommu, .notify_unmap = true, - .aw = s->aw_bits, + .aw = S_AW_BITS, .as = vtd_as, .domain_id = vtd_get_domain_id(s, ce, vtd_as->pasid), }; @@ -1990,7 +1992,7 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus, } ret_fr = vtd_iova_to_slpte(s, &ce, addr, is_write, &slpte, &level, - &reads, &writes, s->aw_bits, pasid); + &reads, &writes, S_AW_BITS, pasid); if (ret_fr) { vtd_report_fault(s, -ret_fr, is_fpd_set, source_id, addr, is_write, pasid != PCI_NO_PASID, pasid); @@ -2004,7 +2006,7 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus, out: vtd_iommu_unlock(s); entry->iova = addr & page_mask; - entry->translated_addr = vtd_get_slpte_addr(slpte, s->aw_bits) & page_mask; + entry->translated_addr = vtd_get_slpte_addr(slpte, S_AW_BITS) & page_mask; entry->addr_mask = ~page_mask; entry->perm = access_flags; return true; @@ -2021,7 +2023,7 @@ error: static void vtd_root_table_setup(IntelIOMMUState *s) { s->root = vtd_get_quad_raw(s, DMAR_RTADDR_REG); - s->root &= VTD_RTADDR_ADDR_MASK(s->aw_bits); + s->root &= VTD_RTADDR_ADDR_MASK(S_AW_BITS); vtd_update_scalable_state(s); @@ -2039,7 +2041,7 @@ static void vtd_interrupt_remap_table_setup(IntelIOMMUState *s) uint64_t value = 0; value = vtd_get_quad_raw(s, DMAR_IRTA_REG); s->intr_size = 1UL << ((value & VTD_IRTA_SIZE_MASK) + 1); - s->intr_root = value & VTD_IRTA_ADDR_MASK(s->aw_bits); + s->intr_root = value & VTD_IRTA_ADDR_MASK(S_AW_BITS); s->intr_eime = value & VTD_IRTA_EIME; /* Notify global invalidation */ @@ -2322,7 +2324,7 @@ static void vtd_handle_gcmd_qie(IntelIOMMUState *s, bool en) trace_vtd_inv_qi_enable(en); if (en) { - s->iq = iqa_val & VTD_IQA_IQA_MASK(s->aw_bits); + s->iq = iqa_val & VTD_IQA_IQA_MASK(S_AW_BITS); /* 2^(x+8) entries */ s->iq_size = 1UL << ((iqa_val & VTD_IQA_QS) + 8 - (s->iq_dw ? 1 : 0)); s->qi_enabled = true; @@ -3958,12 +3960,12 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n) * VT-d spec), otherwise we need to consider overflow of 64 bits. */ - if (end > VTD_ADDRESS_SIZE(s->aw_bits) - 1) { + if (end > VTD_ADDRESS_SIZE(S_AW_BITS) - 1) { /* * Don't need to unmap regions that is bigger than the whole * VT-d supported address space size */ - end = VTD_ADDRESS_SIZE(s->aw_bits) - 1; + end = VTD_ADDRESS_SIZE(S_AW_BITS) - 1; } assert(start <= end); @@ -3971,7 +3973,7 @@ static void vtd_address_space_unmap(VTDAddressSpace *as, IOMMUNotifier *n) while (remain >= VTD_PAGE_SIZE) { IOMMUTLBEvent event; - uint64_t mask = dma_aligned_pow2_mask(start, end, s->aw_bits); + uint64_t mask = dma_aligned_pow2_mask(start, end, S_AW_BITS); uint64_t size = mask + 1; assert(size); @@ -4050,7 +4052,7 @@ static void vtd_iommu_replay(IOMMUMemoryRegion *iommu_mr, IOMMUNotifier *n) .hook_fn = vtd_replay_hook, .private = (void *)n, .notify_unmap = false, - .aw = s->aw_bits, + .aw = S_AW_BITS, .as = vtd_as, .domain_id = vtd_get_domain_id(s, &ce, vtd_as->pasid), }; @@ -4153,15 +4155,15 @@ static void vtd_init(IntelIOMMUState *s) * Rsvd field masks for spte */ vtd_spte_rsvd[0] = ~0ULL; - vtd_spte_rsvd[1] = VTD_SPTE_PAGE_L1_RSVD_MASK(s->aw_bits, + vtd_spte_rsvd[1] = VTD_SPTE_PAGE_L1_RSVD_MASK(S_AW_BITS, x86_iommu->dt_supported); - vtd_spte_rsvd[2] = VTD_SPTE_PAGE_L2_RSVD_MASK(s->aw_bits); - vtd_spte_rsvd[3] = VTD_SPTE_PAGE_L3_RSVD_MASK(s->aw_bits); - vtd_spte_rsvd[4] = VTD_SPTE_PAGE_L4_RSVD_MASK(s->aw_bits); + vtd_spte_rsvd[2] = VTD_SPTE_PAGE_L2_RSVD_MASK(S_AW_BITS); + vtd_spte_rsvd[3] = VTD_SPTE_PAGE_L3_RSVD_MASK(S_AW_BITS); + vtd_spte_rsvd[4] = VTD_SPTE_PAGE_L4_RSVD_MASK(S_AW_BITS); - vtd_spte_rsvd_large[2] = VTD_SPTE_LPAGE_L2_RSVD_MASK(s->aw_bits, + vtd_spte_rsvd_large[2] = VTD_SPTE_LPAGE_L2_RSVD_MASK(S_AW_BITS, x86_iommu->dt_supported); - vtd_spte_rsvd_large[3] = VTD_SPTE_LPAGE_L3_RSVD_MASK(s->aw_bits, + vtd_spte_rsvd_large[3] = VTD_SPTE_LPAGE_L3_RSVD_MASK(S_AW_BITS, x86_iommu->dt_supported); if (s->scalable_mode || s->snoop_control) { From patchwork Thu Feb 1 07:28:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Duan, Zhenzhong" X-Patchwork-Id: 13540731 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 62089C47DDF for ; Thu, 1 Feb 2024 07:32:14 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rVRYG-0007Hc-6I; Thu, 01 Feb 2024 02:32:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRYC-0006ub-VC for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:58 -0500 Received: from mgamail.intel.com ([192.55.52.120]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rVRYA-0001hD-C8 for qemu-devel@nongnu.org; Thu, 01 Feb 2024 02:31:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706772714; x=1738308714; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SxIm2x19TwG5HPffy2XnHOf3K+46VOZDLtqOFlpVjxc=; b=Ss97+wJ3i5FPUzbgH8T9QxxGHpRu/SOxDEe4s/XBDSdjHPttAat5BH1U xiIkH/inH/mZlC7nM82Y+iYOY8T3M0EKzd4hHL/1H0pcl5uyzSedMjtOI /id1AQAFUf4JWMXau+8/1NspKZuwtMDdCm4MW7xVRIxfrB0fjifbjD3j4 z4UJF3eGF64j9TK9vfxKLd+5BSe4Qi8BDmOAxZBby0Jp04qAE+iZU3F72 D4Ls8FEGTh8f5pXunhFlqQaOdNcy43bSSqHiKZcUkHxtmmqI7k7zVbVfO U+0exeUSvVLpsqrTalPy5HgaXE94c1fSInxHN5RMrL1kp6fnsobLGv9cE A==; X-IronPort-AV: E=McAfee;i="6600,9927,10969"; a="402677024" X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="402677024" Received: from orviesa004.jf.intel.com ([10.64.159.144]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:51 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,234,1701158400"; d="scan'208";a="4443435" Received: from spr-s2600bt.bj.intel.com ([10.240.192.124]) by orviesa004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Jan 2024 23:31:46 -0800 From: Zhenzhong Duan To: qemu-devel@nongnu.org Cc: alex.williamson@redhat.com, clg@redhat.com, eric.auger@redhat.com, peterx@redhat.com, jasowang@redhat.com, mst@redhat.com, jgg@nvidia.com, nicolinc@nvidia.com, joao.m.martins@oracle.com, kevin.tian@intel.com, yi.l.liu@intel.com, yi.y.sun@intel.com, chao.p.peng@intel.com, Zhenzhong Duan , Paolo Bonzini , Richard Henderson , Eduardo Habkost , Marcel Apfelbaum Subject: [PATCH rfcv2 18/18] intel_iommu: Block migration if cap is updated Date: Thu, 1 Feb 2024 15:28:18 +0800 Message-Id: <20240201072818.327930-19-zhenzhong.duan@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240201072818.327930-1-zhenzhong.duan@intel.com> References: <20240201072818.327930-1-zhenzhong.duan@intel.com> MIME-Version: 1.0 Received-SPF: pass client-ip=192.55.52.120; envelope-from=zhenzhong.duan@intel.com; helo=mgamail.intel.com X-Spam_score_int: -56 X-Spam_score: -5.7 X-Spam_bar: ----- X-Spam_report: (-5.7 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.292, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org When there is VFIO device and vIOMMU cap/ecap is updated based on host IOMMU cap/ecap, migration should be blocked. Signed-off-by: Zhenzhong Duan --- hw/i386/intel_iommu.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c index 72cc8b2c71..7f9ff653b2 100644 --- a/hw/i386/intel_iommu.c +++ b/hw/i386/intel_iommu.c @@ -39,6 +39,7 @@ #include "hw/i386/apic_internal.h" #include "kvm/kvm_i386.h" #include "migration/vmstate.h" +#include "migration/blocker.h" #include "trace.h" #define S_AW_BITS (VTD_MGAW_FROM_CAP(s->cap) + 1) @@ -3829,6 +3830,8 @@ static int vtd_check_legacy_hdev(IntelIOMMUState *s, return 0; } +static Error *vtd_mig_blocker; + static int vtd_check_iommufd_hdev(IntelIOMMUState *s, IOMMUFDDevice *idev, Error **errp) @@ -3860,8 +3863,17 @@ static int vtd_check_iommufd_hdev(IntelIOMMUState *s, tmp_cap |= VTD_CAP_MGAW(host_mgaw + 1); } - s->cap = tmp_cap; - return 0; + if (s->cap != tmp_cap) { + if (vtd_mig_blocker == NULL) { + error_setg(&vtd_mig_blocker, + "cap/ecap update from host IOMMU block migration"); + ret = migrate_add_blocker(&vtd_mig_blocker, errp); + } + if (!ret) { + s->cap = tmp_cap; + } + } + return ret; } static int vtd_check_hdev(IntelIOMMUState *s, VTDHostIOMMUDevice *vtd_hdev,