From patchwork Tue Jan 8 10:26:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751771 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0C9601399 for ; Tue, 8 Jan 2019 10:27:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ED8D428BC4 for ; Tue, 8 Jan 2019 10:27:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E0F4328BDE; Tue, 8 Jan 2019 10:27:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C3B1228BC4 for ; Tue, 8 Jan 2019 10:26:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727689AbfAHK0y (ORCPT ); Tue, 8 Jan 2019 05:26:54 -0500 Received: from mx1.redhat.com ([209.132.183.28]:37818 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727001AbfAHK0y (ORCPT ); Tue, 8 Jan 2019 05:26:54 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id A335B8AE46; Tue, 8 Jan 2019 10:26:53 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9B29E601A4; Tue, 8 Jan 2019 10:26:49 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 01/21] iommu: Introduce set_pasid_table API Date: Tue, 8 Jan 2019 11:26:13 +0100 Message-Id: <20190108102633.17482-2-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 08 Jan 2019 10:26:53 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jacob Pan In virtualization use case, when a guest is assigned a PCI host device, protected by a virtual IOMMU on a guest, the physical IOMMU must be programmed to be consistent with the guest mappings. If the physical IOMMU supports two translation stages it makes sense to program guest mappings onto the first stage/level (ARM/VTD terminology) while to host owns the stage/level 2. In that case, it is mandated to trap on guest configuration settings and pass those to the physical iommu driver. This patch adds a new API to the iommu subsystem that allows to set the pasid table information. A generic iommu_pasid_table_config struct is introduced in a new iommu.h uapi header. This is going to be used by the VFIO user API. We foresee at least two specializations of this struct, for PASID table passing and ARM SMMUv3. Signed-off-by: Jean-Philippe Brucker Signed-off-by: Liu, Yi L Signed-off-by: Ashok Raj Signed-off-by: Jacob Pan Signed-off-by: Eric Auger --- This patch generalizes the API introduced by Jacob & co-authors in https://lwn.net/Articles/754331/ v2 -> v3: - replace unbind/bind by set_pasid_table - move table pointer and pasid bits in the generic part of the struct v1 -> v2: - restore the original pasid table name - remove the struct device * parameter in the API - reworked iommu_pasid_smmuv3 --- drivers/iommu/iommu.c | 10 ++++++++ include/linux/iommu.h | 14 +++++++++++ include/uapi/linux/iommu.h | 50 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 74 insertions(+) create mode 100644 include/uapi/linux/iommu.h diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 3ed4db334341..0f2b7f1fc7c8 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -1393,6 +1393,16 @@ int iommu_attach_device(struct iommu_domain *domain, struct device *dev) } EXPORT_SYMBOL_GPL(iommu_attach_device); +int iommu_set_pasid_table(struct iommu_domain *domain, + struct iommu_pasid_table_config *cfg) +{ + if (unlikely(!domain->ops->set_pasid_table)) + return -ENODEV; + + return domain->ops->set_pasid_table(domain, cfg); +} +EXPORT_SYMBOL_GPL(iommu_set_pasid_table); + static void __iommu_detach_device(struct iommu_domain *domain, struct device *dev) { diff --git a/include/linux/iommu.h b/include/linux/iommu.h index e90da6b6f3d1..1da2a2357ea4 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -25,6 +25,7 @@ #include #include #include +#include #define IOMMU_READ (1 << 0) #define IOMMU_WRITE (1 << 1) @@ -184,6 +185,7 @@ struct iommu_resv_region { * @domain_window_disable: Disable a particular window for a domain * @of_xlate: add OF master IDs to iommu grouping * @pgsize_bitmap: bitmap of all possible supported page sizes + * @set_pasid_table: set pasid table */ struct iommu_ops { bool (*capable)(enum iommu_cap); @@ -226,6 +228,9 @@ struct iommu_ops { int (*of_xlate)(struct device *dev, struct of_phandle_args *args); bool (*is_attach_deferred)(struct iommu_domain *domain, struct device *dev); + int (*set_pasid_table)(struct iommu_domain *domain, + struct iommu_pasid_table_config *cfg); + unsigned long pgsize_bitmap; }; @@ -287,6 +292,8 @@ extern int iommu_attach_device(struct iommu_domain *domain, struct device *dev); extern void iommu_detach_device(struct iommu_domain *domain, struct device *dev); +extern int iommu_set_pasid_table(struct iommu_domain *domain, + struct iommu_pasid_table_config *cfg); extern struct iommu_domain *iommu_get_domain_for_dev(struct device *dev); extern struct iommu_domain *iommu_get_dma_domain(struct device *dev); extern int iommu_map(struct iommu_domain *domain, unsigned long iova, @@ -696,6 +703,13 @@ const struct iommu_ops *iommu_ops_from_fwnode(struct fwnode_handle *fwnode) return NULL; } +static inline +int iommu_set_pasid_table(struct iommu_domain *domain, + struct iommu_pasid_table_config *cfg) +{ + return -ENODEV; +} + #endif /* CONFIG_IOMMU_API */ #ifdef CONFIG_IOMMU_DEBUGFS diff --git a/include/uapi/linux/iommu.h b/include/uapi/linux/iommu.h new file mode 100644 index 000000000000..7a7cf7a3de7c --- /dev/null +++ b/include/uapi/linux/iommu.h @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * IOMMU user API definitions + * + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + */ + +#ifndef _UAPI_IOMMU_H +#define _UAPI_IOMMU_H + +#include + +/** + * SMMUv3 Stream Table Entry stage 1 related information + * @abort: shall the STE lead to abort + * @s1fmt: STE s1fmt field as set by the guest + * @s1dss: STE s1dss as set by the guest + * All field names match the smmu 3.0/3.1 spec (ARM IHI 0070A) + */ +struct iommu_pasid_smmuv3 { + __u8 abort; + __u8 s1fmt; + __u8 s1dss; +}; + +/** + * PASID table data used to bind guest PASID table to the host IOMMU + * Note PASID table corresponds to the Context Table on ARM SMMUv3. + * + * @version: API version to prepare for future extensions + * @format: format of the PASID table + * + */ +struct iommu_pasid_table_config { +#define PASID_TABLE_CFG_VERSION_1 1 + __u32 version; +#define IOMMU_PASID_FORMAT_SMMUV3 (1 << 0) + __u32 format; + __u64 base_ptr; + __u8 pasid_bits; + __u8 bypass; + union { + struct iommu_pasid_smmuv3 smmuv3; + }; +}; + +#endif /* _UAPI_IOMMU_H */ From patchwork Tue Jan 8 10:26:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751773 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2993A1399 for ; Tue, 8 Jan 2019 10:27:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 17E9828BDC for ; Tue, 8 Jan 2019 10:27:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0C12228BDF; Tue, 8 Jan 2019 10:27:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5D34628BDC for ; Tue, 8 Jan 2019 10:27:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728519AbfAHK07 (ORCPT ); Tue, 8 Jan 2019 05:26:59 -0500 Received: from mx1.redhat.com ([209.132.183.28]:58954 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727001AbfAHK07 (ORCPT ); Tue, 8 Jan 2019 05:26:59 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1DFE5C023D; Tue, 8 Jan 2019 10:26:58 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id 018866013A; Tue, 8 Jan 2019 10:26:53 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 02/21] iommu: Introduce cache_invalidate API Date: Tue, 8 Jan 2019 11:26:14 +0100 Message-Id: <20190108102633.17482-3-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Tue, 08 Jan 2019 10:26:58 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: "Liu, Yi L" In any virtualization use case, when the first translation stage is "owned" by the guest OS, the host IOMMU driver has no knowledge of caching structure updates unless the guest invalidation activities are trapped by the virtualizer and passed down to the host. Since the invalidation data are obtained from user space and will be written into physical IOMMU, we must allow security check at various layers. Therefore, generic invalidation data format are proposed here, model specific IOMMU drivers need to convert them into their own format. Signed-off-by: Liu, Yi L Signed-off-by: Jean-Philippe Brucker Signed-off-by: Jacob Pan Signed-off-by: Ashok Raj Signed-off-by: Eric Auger --- v1 -> v2: - add arch_id field - renamed tlb_invalidate into cache_invalidate as this API allows to invalidate context caches on top of IOTLBs v1: renamed sva_invalidate into tlb_invalidate and add iommu_ prefix in header. Commit message reworded. --- drivers/iommu/iommu.c | 14 ++++++ include/linux/iommu.h | 14 ++++++ include/uapi/linux/iommu.h | 95 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 123 insertions(+) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 0f2b7f1fc7c8..b2e248770508 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -1403,6 +1403,20 @@ int iommu_set_pasid_table(struct iommu_domain *domain, } EXPORT_SYMBOL_GPL(iommu_set_pasid_table); +int iommu_cache_invalidate(struct iommu_domain *domain, struct device *dev, + struct iommu_cache_invalidate_info *inv_info) +{ + int ret = 0; + + if (unlikely(!domain->ops->cache_invalidate)) + return -ENODEV; + + ret = domain->ops->cache_invalidate(domain, dev, inv_info); + + return ret; +} +EXPORT_SYMBOL_GPL(iommu_cache_invalidate); + static void __iommu_detach_device(struct iommu_domain *domain, struct device *dev) { diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 1da2a2357ea4..96d59886f230 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -186,6 +186,7 @@ struct iommu_resv_region { * @of_xlate: add OF master IDs to iommu grouping * @pgsize_bitmap: bitmap of all possible supported page sizes * @set_pasid_table: set pasid table + * @cache_invalidate: invalidate translation caches */ struct iommu_ops { bool (*capable)(enum iommu_cap); @@ -231,6 +232,9 @@ struct iommu_ops { int (*set_pasid_table)(struct iommu_domain *domain, struct iommu_pasid_table_config *cfg); + int (*cache_invalidate)(struct iommu_domain *domain, struct device *dev, + struct iommu_cache_invalidate_info *inv_info); + unsigned long pgsize_bitmap; }; @@ -294,6 +298,9 @@ extern void iommu_detach_device(struct iommu_domain *domain, struct device *dev); extern int iommu_set_pasid_table(struct iommu_domain *domain, struct iommu_pasid_table_config *cfg); +extern int iommu_cache_invalidate(struct iommu_domain *domain, + struct device *dev, + struct iommu_cache_invalidate_info *inv_info); extern struct iommu_domain *iommu_get_domain_for_dev(struct device *dev); extern struct iommu_domain *iommu_get_dma_domain(struct device *dev); extern int iommu_map(struct iommu_domain *domain, unsigned long iova, @@ -709,6 +716,13 @@ int iommu_set_pasid_table(struct iommu_domain *domain, { return -ENODEV; } +static inline int +iommu_cache_invalidate(struct iommu_domain *domain, + struct device *dev, + struct iommu_cache_invalidate_info *inv_info) +{ + return -ENODEV; +} #endif /* CONFIG_IOMMU_API */ diff --git a/include/uapi/linux/iommu.h b/include/uapi/linux/iommu.h index 7a7cf7a3de7c..4605f5cfac84 100644 --- a/include/uapi/linux/iommu.h +++ b/include/uapi/linux/iommu.h @@ -47,4 +47,99 @@ struct iommu_pasid_table_config { }; }; +/** + * enum iommu_inv_granularity - Generic invalidation granularity + * @IOMMU_INV_GRANU_DOMAIN_ALL_PASID: TLB entries or PASID caches of all + * PASIDs associated with a domain ID + * @IOMMU_INV_GRANU_PASID_SEL: TLB entries or PASID cache associated + * with a PASID and a domain + * @IOMMU_INV_GRANU_PAGE_PASID: TLB entries of selected page range + * within a PASID + * + * When an invalidation request is passed down to IOMMU to flush translation + * caches, it may carry different granularity levels, which can be specific + * to certain types of translation caches. + * This enum is a collection of granularities for all types of translation + * caches. The idea is to make it easy for IOMMU model specific driver to + * convert from generic to model specific value. Each IOMMU driver + * can enforce check based on its own conversion table. The conversion is + * based on 2D look-up with inputs as follows: + * - translation cache types + * - granularity + * + * type | DTLB | TLB | PASID | + * granule | | | cache | + * -----------------+-----------+-----------+-----------+ + * DN_ALL_PASID | Y | Y | Y | + * PASID_SEL | Y | Y | Y | + * PAGE_PASID | Y | Y | N/A | + * + */ +enum iommu_inv_granularity { + IOMMU_INV_GRANU_DOMAIN_ALL_PASID, + IOMMU_INV_GRANU_PASID_SEL, + IOMMU_INV_GRANU_PAGE_PASID, + IOMMU_INV_NR_GRANU, +}; + +/** + * enum iommu_inv_type - Generic translation cache types for invalidation + * + * @IOMMU_INV_TYPE_DTLB: device IOTLB + * @IOMMU_INV_TYPE_TLB: IOMMU paging structure cache + * @IOMMU_INV_TYPE_PASID: PASID cache + * Invalidation requests sent to IOMMU for a given device need to indicate + * which type of translation cache to be operated on. Combined with enum + * iommu_inv_granularity, model specific driver can do a simple lookup to + * convert from generic to model specific value. + */ +enum iommu_inv_type { + IOMMU_INV_TYPE_DTLB, + IOMMU_INV_TYPE_TLB, + IOMMU_INV_TYPE_PASID, + IOMMU_INV_NR_TYPE +}; + +/** + * Translation cache invalidation header that contains mandatory meta data. + * @version: info format version, expecting future extesions + * @type: type of translation cache to be invalidated + */ +struct iommu_cache_invalidate_hdr { + __u32 version; +#define TLB_INV_HDR_VERSION_1 1 + enum iommu_inv_type type; +}; + +/** + * Translation cache invalidation information, contains generic IOMMU + * data which can be parsed based on model ID by model specific drivers. + * Since the invalidation of second level page tables are included in the + * unmap operation, this info is only applicable to the first level + * translation caches, i.e. DMA request with PASID. + * + * @granularity: requested invalidation granularity, type dependent + * @size: 2^size of 4K pages, 0 for 4k, 9 for 2MB, etc. + * @nr_pages: number of pages to invalidate + * @pasid: processor address space ID value per PCI spec. + * @arch_id: architecture dependent id characterizing a context + * and tagging the caches, ie. domain Identfier on VTD, + * asid on ARM SMMU + * @addr: page address to be invalidated + * @flags IOMMU_INVALIDATE_ADDR_LEAF: leaf paging entries + * IOMMU_INVALIDATE_GLOBAL_PAGE: global pages + * + */ +struct iommu_cache_invalidate_info { + struct iommu_cache_invalidate_hdr hdr; + enum iommu_inv_granularity granularity; + __u32 flags; +#define IOMMU_INVALIDATE_ADDR_LEAF (1 << 0) +#define IOMMU_INVALIDATE_GLOBAL_PAGE (1 << 1) + __u8 size; + __u64 nr_pages; + __u32 pasid; + __u64 arch_id; + __u64 addr; +}; #endif /* _UAPI_IOMMU_H */ From patchwork Tue Jan 8 10:26:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751777 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3C93013B4 for ; Tue, 8 Jan 2019 10:27:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2A3EA28BDE for ; Tue, 8 Jan 2019 10:27:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1E88928BE1; Tue, 8 Jan 2019 10:27:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AF31128BDE for ; Tue, 8 Jan 2019 10:27:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728501AbfAHK1L (ORCPT ); Tue, 8 Jan 2019 05:27:11 -0500 Received: from mx1.redhat.com ([209.132.183.28]:59066 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727867AbfAHK1K (ORCPT ); Tue, 8 Jan 2019 05:27:10 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EB96381F13; Tue, 8 Jan 2019 10:27:09 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5510E605AE; Tue, 8 Jan 2019 10:26:58 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 03/21] iommu: Introduce bind_guest_msi Date: Tue, 8 Jan 2019 11:26:15 +0100 Message-Id: <20190108102633.17482-4-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Tue, 08 Jan 2019 10:27:10 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On ARM, MSI are translated by the SMMU. An IOVA is allocated for each MSI doorbell. If both the host and the guest are exposed with SMMUs, we end up with 2 different IOVAs allocated by each. guest allocates an IOVA (gIOVA) to map onto the guest MSI doorbell (gDB). The Host allocates another IOVA (hIOVA) to map onto the physical doorbell (hDB). So we end up with 2 untied mappings: S1 S2 gIOVA -> gDB hIOVA -> gDB Currently the PCI device is programmed by the host with hIOVA as MSI doorbell. So this does not work. This patch introduces an API to pass gIOVA/gDB to the host so that gIOVA can be reused by the host instead of re-allocating a new IOVA. So the goal is to create the following nested mapping: S1 S2 gIOVA -> gDB -> hDB and program the PCI device with gIOVA MSI doorbell. Signed-off-by: Eric Auger --- v2 -> v3: - add a struct device handle --- drivers/iommu/iommu.c | 10 ++++++++++ include/linux/iommu.h | 13 +++++++++++++ include/uapi/linux/iommu.h | 6 ++++++ 3 files changed, 29 insertions(+) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index b2e248770508..ea11442e7054 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -1431,6 +1431,16 @@ static void __iommu_detach_device(struct iommu_domain *domain, trace_detach_device_from_domain(dev); } +int iommu_bind_guest_msi(struct iommu_domain *domain, struct device *dev, + struct iommu_guest_msi_binding *binding) +{ + if (unlikely(!domain->ops->bind_guest_msi)) + return -ENODEV; + + return domain->ops->bind_guest_msi(domain, dev, binding); +} +EXPORT_SYMBOL_GPL(iommu_bind_guest_msi); + void iommu_detach_device(struct iommu_domain *domain, struct device *dev) { struct iommu_group *group; diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 96d59886f230..244c1a3d5989 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -235,6 +235,9 @@ struct iommu_ops { int (*cache_invalidate)(struct iommu_domain *domain, struct device *dev, struct iommu_cache_invalidate_info *inv_info); + int (*bind_guest_msi)(struct iommu_domain *domain, struct device *dev, + struct iommu_guest_msi_binding *binding); + unsigned long pgsize_bitmap; }; @@ -301,6 +304,9 @@ extern int iommu_set_pasid_table(struct iommu_domain *domain, extern int iommu_cache_invalidate(struct iommu_domain *domain, struct device *dev, struct iommu_cache_invalidate_info *inv_info); +extern int iommu_bind_guest_msi(struct iommu_domain *domain, struct device *dev, + struct iommu_guest_msi_binding *binding); + extern struct iommu_domain *iommu_get_domain_for_dev(struct device *dev); extern struct iommu_domain *iommu_get_dma_domain(struct device *dev); extern int iommu_map(struct iommu_domain *domain, unsigned long iova, @@ -724,6 +730,13 @@ iommu_cache_invalidate(struct iommu_domain *domain, return -ENODEV; } +static inline +int iommu_bind_guest_msi(struct iommu_domain *domain, struct device *dev, + struct iommu_guest_msi_binding *binding) +{ + return -ENODEV; +} + #endif /* CONFIG_IOMMU_API */ #ifdef CONFIG_IOMMU_DEBUGFS diff --git a/include/uapi/linux/iommu.h b/include/uapi/linux/iommu.h index 4605f5cfac84..f28cd9a1aa96 100644 --- a/include/uapi/linux/iommu.h +++ b/include/uapi/linux/iommu.h @@ -142,4 +142,10 @@ struct iommu_cache_invalidate_info { __u64 arch_id; __u64 addr; }; + +struct iommu_guest_msi_binding { + __u64 iova; + __u64 gpa; + __u32 granule; +}; #endif /* _UAPI_IOMMU_H */ From patchwork Tue Jan 8 10:26:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751781 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3979813B4 for ; Tue, 8 Jan 2019 10:27:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2647628BC4 for ; Tue, 8 Jan 2019 10:27:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1A24A28BDE; Tue, 8 Jan 2019 10:27:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B0F3328BC4 for ; Tue, 8 Jan 2019 10:27:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728622AbfAHK1Q (ORCPT ); Tue, 8 Jan 2019 05:27:16 -0500 Received: from mx1.redhat.com ([209.132.183.28]:55432 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727867AbfAHK1P (ORCPT ); Tue, 8 Jan 2019 05:27:15 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 95A4490C9C; Tue, 8 Jan 2019 10:27:14 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4A9926013A; Tue, 8 Jan 2019 10:27:10 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 04/21] vfio: VFIO_IOMMU_SET_PASID_TABLE Date: Tue, 8 Jan 2019 11:26:16 +0100 Message-Id: <20190108102633.17482-5-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Tue, 08 Jan 2019 10:27:14 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: "Liu, Yi L" This patch adds VFIO_IOMMU_SET_PASID_TABLE ioctl which aims at passing the virtual iommu guest configuration to the VFIO driver downto to the iommu subsystem. Signed-off-by: Jacob Pan Signed-off-by: Liu, Yi L Signed-off-by: Eric Auger --- v2 -> v3: - s/BIND_PASID_TABLE/SET_PASID_TABLE v1 -> v2: - s/BIND_GUEST_STAGE/BIND_PASID_TABLE - remove the struct device arg --- drivers/vfio/vfio_iommu_type1.c | 31 +++++++++++++++++++++++++++++++ include/uapi/linux/vfio.h | 8 ++++++++ 2 files changed, 39 insertions(+) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 7651cfb14836..d9dd23f64f00 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -1644,6 +1644,24 @@ static int vfio_domains_have_iommu_cache(struct vfio_iommu *iommu) return ret; } +static int +vfio_set_pasid_table(struct vfio_iommu *iommu, + struct vfio_iommu_type1_set_pasid_table *ustruct) +{ + struct vfio_domain *d; + int ret = 0; + + mutex_lock(&iommu->lock); + + list_for_each_entry(d, &iommu->domain_list, next) { + ret = iommu_set_pasid_table(d->domain, &ustruct->config); + if (ret) + break; + } + mutex_unlock(&iommu->lock); + return ret; +} + static long vfio_iommu_type1_ioctl(void *iommu_data, unsigned int cmd, unsigned long arg) { @@ -1714,6 +1732,19 @@ static long vfio_iommu_type1_ioctl(void *iommu_data, return copy_to_user((void __user *)arg, &unmap, minsz) ? -EFAULT : 0; + } else if (cmd == VFIO_IOMMU_SET_PASID_TABLE) { + struct vfio_iommu_type1_set_pasid_table ustruct; + + minsz = offsetofend(struct vfio_iommu_type1_set_pasid_table, + config); + + if (copy_from_user(&ustruct, (void __user *)arg, minsz)) + return -EFAULT; + + if (ustruct.argsz < minsz || ustruct.flags) + return -EINVAL; + + return vfio_set_pasid_table(iommu, &ustruct); } return -ENOTTY; diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 02bb7ad6e986..0d9f4090c95d 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -14,6 +14,7 @@ #include #include +#include #define VFIO_API_VERSION 0 @@ -759,6 +760,13 @@ struct vfio_iommu_type1_dma_unmap { #define VFIO_IOMMU_ENABLE _IO(VFIO_TYPE, VFIO_BASE + 15) #define VFIO_IOMMU_DISABLE _IO(VFIO_TYPE, VFIO_BASE + 16) +struct vfio_iommu_type1_set_pasid_table { + __u32 argsz; + __u32 flags; + struct iommu_pasid_table_config config; +}; +#define VFIO_IOMMU_SET_PASID_TABLE _IO(VFIO_TYPE, VFIO_BASE + 22) + /* -------- Additional API for SPAPR TCE (Server POWERPC) IOMMU -------- */ /* From patchwork Tue Jan 8 10:26:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751779 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 55F401399 for ; Tue, 8 Jan 2019 10:27:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 43A8928BC4 for ; Tue, 8 Jan 2019 10:27:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3748828BDE; Tue, 8 Jan 2019 10:27:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C78A328BC4 for ; Tue, 8 Jan 2019 10:27:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728689AbfAHK1U (ORCPT ); Tue, 8 Jan 2019 05:27:20 -0500 Received: from mx1.redhat.com ([209.132.183.28]:59266 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727980AbfAHK1T (ORCPT ); Tue, 8 Jan 2019 05:27:19 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 338879D501; Tue, 8 Jan 2019 10:27:19 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id EAD53605AE; Tue, 8 Jan 2019 10:27:14 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 05/21] vfio: VFIO_IOMMU_CACHE_INVALIDATE Date: Tue, 8 Jan 2019 11:26:17 +0100 Message-Id: <20190108102633.17482-6-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Tue, 08 Jan 2019 10:27:19 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: "Liu, Yi L" When the guest "owns" the stage 1 translation structures, the host IOMMU driver has no knowledge of caching structure updates unless the guest invalidation requests are trapped and passed down to the host. This patch adds the VFIO_IOMMU_CACHE_INVALIDATE ioctl with aims at propagating guest stage1 IOMMU cache invalidations to the host. Signed-off-by: Liu, Yi L Signed-off-by: Eric Auger --- v2 -> v3: - introduce vfio_iommu_for_each_dev back in this patch v1 -> v2: - s/TLB/CACHE - remove vfio_iommu_task usage - commit message rewording --- drivers/vfio/vfio_iommu_type1.c | 47 +++++++++++++++++++++++++++++++++ include/uapi/linux/vfio.h | 7 +++++ 2 files changed, 54 insertions(+) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index d9dd23f64f00..c3ba3f249438 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -113,6 +113,26 @@ struct vfio_regions { #define IS_IOMMU_CAP_DOMAIN_IN_CONTAINER(iommu) \ (!list_empty(&iommu->domain_list)) +/* iommu->lock must be held */ +static int +vfio_iommu_for_each_dev(struct vfio_iommu *iommu, void *data, + int (*fn)(struct device *, void *)) +{ + struct vfio_domain *d; + struct vfio_group *g; + int ret = 0; + + list_for_each_entry(d, &iommu->domain_list, next) { + list_for_each_entry(g, &d->group_list, next) { + ret = iommu_group_for_each_dev(g->iommu_group, + data, fn); + if (ret) + break; + } + } + return ret; +} + static int put_pfn(unsigned long pfn, int prot); /* @@ -1644,6 +1664,15 @@ static int vfio_domains_have_iommu_cache(struct vfio_iommu *iommu) return ret; } +static int vfio_cache_inv_fn(struct device *dev, void *data) +{ + struct vfio_iommu_type1_cache_invalidate *ustruct = + (struct vfio_iommu_type1_cache_invalidate *)data; + struct iommu_domain *d = iommu_get_domain_for_dev(dev); + + return iommu_cache_invalidate(d, dev, &ustruct->info); +} + static int vfio_set_pasid_table(struct vfio_iommu *iommu, struct vfio_iommu_type1_set_pasid_table *ustruct) @@ -1745,6 +1774,24 @@ static long vfio_iommu_type1_ioctl(void *iommu_data, return -EINVAL; return vfio_set_pasid_table(iommu, &ustruct); + } else if (cmd == VFIO_IOMMU_CACHE_INVALIDATE) { + struct vfio_iommu_type1_cache_invalidate ustruct; + int ret; + + minsz = offsetofend(struct vfio_iommu_type1_cache_invalidate, + info); + + if (copy_from_user(&ustruct, (void __user *)arg, minsz)) + return -EFAULT; + + if (ustruct.argsz < minsz || ustruct.flags) + return -EINVAL; + + mutex_lock(&iommu->lock); + ret = vfio_iommu_for_each_dev(iommu, &ustruct, + vfio_cache_inv_fn); + mutex_unlock(&iommu->lock); + return ret; } return -ENOTTY; diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 0d9f4090c95d..11a07165e7e1 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -767,6 +767,13 @@ struct vfio_iommu_type1_set_pasid_table { }; #define VFIO_IOMMU_SET_PASID_TABLE _IO(VFIO_TYPE, VFIO_BASE + 22) +struct vfio_iommu_type1_cache_invalidate { + __u32 argsz; + __u32 flags; + struct iommu_cache_invalidate_info info; +}; +#define VFIO_IOMMU_CACHE_INVALIDATE _IO(VFIO_TYPE, VFIO_BASE + 23) + /* -------- Additional API for SPAPR TCE (Server POWERPC) IOMMU -------- */ /* From patchwork Tue Jan 8 10:26:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751787 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8AF721399 for ; Tue, 8 Jan 2019 10:27:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 78DAC28BC4 for ; Tue, 8 Jan 2019 10:27:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6C7DE28BDE; Tue, 8 Jan 2019 10:27:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5C1F628BC4 for ; Tue, 8 Jan 2019 10:27:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728675AbfAHK1b (ORCPT ); Tue, 8 Jan 2019 05:27:31 -0500 Received: from mx1.redhat.com ([209.132.183.28]:55670 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727381AbfAHK1a (ORCPT ); Tue, 8 Jan 2019 05:27:30 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id F3CB890C9C; Tue, 8 Jan 2019 10:27:29 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id 818EB6013A; Tue, 8 Jan 2019 10:27:19 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 06/21] vfio: VFIO_IOMMU_BIND_MSI Date: Tue, 8 Jan 2019 11:26:18 +0100 Message-Id: <20190108102633.17482-7-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Tue, 08 Jan 2019 10:27:30 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch adds the VFIO_IOMMU_BIND_MSI ioctl which aims at passing the guest MSI binding to the host. Signed-off-by: Eric Auger --- v2 -> v3: - adapt to new proto of bind_guest_msi - directly use vfio_iommu_for_each_dev v1 -> v2: - s/vfio_iommu_type1_guest_msi_binding/vfio_iommu_type1_bind_guest_msi --- drivers/vfio/vfio_iommu_type1.c | 27 +++++++++++++++++++++++++++ include/uapi/linux/vfio.h | 7 +++++++ 2 files changed, 34 insertions(+) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index c3ba3f249438..59229f6e2d84 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -1673,6 +1673,15 @@ static int vfio_cache_inv_fn(struct device *dev, void *data) return iommu_cache_invalidate(d, dev, &ustruct->info); } +static int vfio_bind_guest_msi_fn(struct device *dev, void *data) +{ + struct vfio_iommu_type1_bind_guest_msi *ustruct = + (struct vfio_iommu_type1_bind_guest_msi *)data; + struct iommu_domain *d = iommu_get_domain_for_dev(dev); + + return iommu_bind_guest_msi(d, dev, &ustruct->binding); +} + static int vfio_set_pasid_table(struct vfio_iommu *iommu, struct vfio_iommu_type1_set_pasid_table *ustruct) @@ -1792,6 +1801,24 @@ static long vfio_iommu_type1_ioctl(void *iommu_data, vfio_cache_inv_fn); mutex_unlock(&iommu->lock); return ret; + } else if (cmd == VFIO_IOMMU_BIND_MSI) { + struct vfio_iommu_type1_bind_guest_msi ustruct; + int ret; + + minsz = offsetofend(struct vfio_iommu_type1_bind_guest_msi, + binding); + + if (copy_from_user(&ustruct, (void __user *)arg, minsz)) + return -EFAULT; + + if (ustruct.argsz < minsz || ustruct.flags) + return -EINVAL; + + mutex_lock(&iommu->lock); + ret = vfio_iommu_for_each_dev(iommu, &ustruct, + vfio_bind_guest_msi_fn); + mutex_unlock(&iommu->lock); + return ret; } return -ENOTTY; diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 11a07165e7e1..352e795a93c8 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -774,6 +774,13 @@ struct vfio_iommu_type1_cache_invalidate { }; #define VFIO_IOMMU_CACHE_INVALIDATE _IO(VFIO_TYPE, VFIO_BASE + 23) +struct vfio_iommu_type1_bind_guest_msi { + __u32 argsz; + __u32 flags; + struct iommu_guest_msi_binding binding; +}; +#define VFIO_IOMMU_BIND_MSI _IO(VFIO_TYPE, VFIO_BASE + 24) + /* -------- Additional API for SPAPR TCE (Server POWERPC) IOMMU -------- */ /* From patchwork Tue Jan 8 10:26:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751783 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C941413B4 for ; Tue, 8 Jan 2019 10:27:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B652B28BC4 for ; Tue, 8 Jan 2019 10:27:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A9CDA28BDE; Tue, 8 Jan 2019 10:27:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 47D9228BC4 for ; Tue, 8 Jan 2019 10:27:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728731AbfAHK1f (ORCPT ); Tue, 8 Jan 2019 05:27:35 -0500 Received: from mx1.redhat.com ([209.132.183.28]:38402 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728728AbfAHK1e (ORCPT ); Tue, 8 Jan 2019 05:27:34 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4FAF888E7E; Tue, 8 Jan 2019 10:27:34 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id 52F4D601A4; Tue, 8 Jan 2019 10:27:30 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 07/21] iommu/arm-smmu-v3: Link domains and devices Date: Tue, 8 Jan 2019 11:26:19 +0100 Message-Id: <20190108102633.17482-8-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 08 Jan 2019 10:27:34 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jean-Philippe Brucker When removing a mapping from a domain, we need to send an invalidation to all devices that might have stored it in their Address Translation Cache (ATC). In addition with SVM, we'll need to invalidate context descriptors of all devices attached to a live domain. Maintain a list of devices in each domain, protected by a spinlock. It is updated every time we attach or detach devices to and from domains. It needs to be a spinlock because we'll invalidate ATC entries from within hardirq-safe contexts, but it may be possible to relax the read side with RCU later. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 0d284029dc73..ce222705f52b 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -595,6 +595,11 @@ struct arm_smmu_device { struct arm_smmu_master_data { struct arm_smmu_device *smmu; struct arm_smmu_strtab_ent ste; + + struct arm_smmu_domain *domain; + struct list_head list; /* domain->devices */ + + struct device *dev; }; /* SMMU private data for an IOMMU domain */ @@ -619,6 +624,9 @@ struct arm_smmu_domain { }; struct iommu_domain domain; + + struct list_head devices; + spinlock_t devices_lock; }; struct arm_smmu_option_prop { @@ -1494,6 +1502,9 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) } mutex_init(&smmu_domain->init_mutex); + INIT_LIST_HEAD(&smmu_domain->devices); + spin_lock_init(&smmu_domain->devices_lock); + return &smmu_domain->domain; } @@ -1714,6 +1725,16 @@ static void arm_smmu_detach_dev(struct device *dev) { struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct arm_smmu_master_data *master = fwspec->iommu_priv; + unsigned long flags; + struct arm_smmu_domain *smmu_domain = master->domain; + + if (smmu_domain) { + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_del(&master->list); + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + + master->domain = NULL; + } master->ste.assigned = false; arm_smmu_install_ste_for_dev(fwspec); @@ -1723,6 +1744,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret = 0; struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + unsigned long flags; struct arm_smmu_device *smmu; struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); struct arm_smmu_master_data *master; @@ -1758,6 +1780,11 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) } ste->assigned = true; + master->domain = smmu_domain; + + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_add(&master->list, &smmu_domain->devices); + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); if (smmu_domain->stage == ARM_SMMU_DOMAIN_BYPASS) { ste->s1_cfg = NULL; @@ -1884,6 +1911,7 @@ static int arm_smmu_add_device(struct device *dev) return -ENOMEM; master->smmu = smmu; + master->dev = dev; fwspec->iommu_priv = master; } From patchwork Tue Jan 8 10:26:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751785 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9EB721399 for ; Tue, 8 Jan 2019 10:27:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8BE3228BC4 for ; Tue, 8 Jan 2019 10:27:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7F86F28BDE; Tue, 8 Jan 2019 10:27:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F326228BC4 for ; Tue, 8 Jan 2019 10:27:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728727AbfAHK1l (ORCPT ); Tue, 8 Jan 2019 05:27:41 -0500 Received: from mx1.redhat.com ([209.132.183.28]:50750 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728158AbfAHK1l (ORCPT ); Tue, 8 Jan 2019 05:27:41 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 7E3A8C05CDEB; Tue, 8 Jan 2019 10:27:40 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id B4675601A3; Tue, 8 Jan 2019 10:27:34 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 08/21] iommu/arm-smmu-v3: Maintain a SID->device structure Date: Tue, 8 Jan 2019 11:26:20 +0100 Message-Id: <20190108102633.17482-9-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 08 Jan 2019 10:27:40 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jean-Philippe Brucker When handling faults from the event or PRI queue, we need to find the struct device associated to a SID. Add a rb_tree to keep track of SIDs. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 136 ++++++++++++++++++++++++++++++++++-- 1 file changed, 132 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index ce222705f52b..9af68266bbb1 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -589,6 +589,16 @@ struct arm_smmu_device { /* IOMMU core code handle */ struct iommu_device iommu; + + struct rb_root streams; + struct mutex streams_mutex; + +}; + +struct arm_smmu_stream { + u32 id; + struct arm_smmu_master_data *master; + struct rb_node node; }; /* SMMU private data for each master */ @@ -598,6 +608,7 @@ struct arm_smmu_master_data { struct arm_smmu_domain *domain; struct list_head list; /* domain->devices */ + struct arm_smmu_stream *streams; struct device *dev; }; @@ -1244,6 +1255,32 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid) return 0; } +__maybe_unused +static struct arm_smmu_master_data * +arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid) +{ + struct rb_node *node; + struct arm_smmu_stream *stream; + struct arm_smmu_master_data *master = NULL; + + mutex_lock(&smmu->streams_mutex); + node = smmu->streams.rb_node; + while (node) { + stream = rb_entry(node, struct arm_smmu_stream, node); + if (stream->id < sid) { + node = node->rb_right; + } else if (stream->id > sid) { + node = node->rb_left; + } else { + master = stream->master; + break; + } + } + mutex_unlock(&smmu->streams_mutex); + + return master; +} + /* IRQ and event handlers */ static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev) { @@ -1882,6 +1919,71 @@ static bool arm_smmu_sid_in_range(struct arm_smmu_device *smmu, u32 sid) return sid < limit; } +static int arm_smmu_insert_master(struct arm_smmu_device *smmu, + struct arm_smmu_master_data *master) +{ + int i; + int ret = 0; + struct arm_smmu_stream *new_stream, *cur_stream; + struct rb_node **new_node, *parent_node = NULL; + struct iommu_fwspec *fwspec = master->dev->iommu_fwspec; + + master->streams = kcalloc(fwspec->num_ids, + sizeof(struct arm_smmu_stream), GFP_KERNEL); + if (!master->streams) + return -ENOMEM; + + mutex_lock(&smmu->streams_mutex); + for (i = 0; i < fwspec->num_ids && !ret; i++) { + new_stream = &master->streams[i]; + new_stream->id = fwspec->ids[i]; + new_stream->master = master; + + new_node = &(smmu->streams.rb_node); + while (*new_node) { + cur_stream = rb_entry(*new_node, struct arm_smmu_stream, + node); + parent_node = *new_node; + if (cur_stream->id > new_stream->id) { + new_node = &((*new_node)->rb_left); + } else if (cur_stream->id < new_stream->id) { + new_node = &((*new_node)->rb_right); + } else { + dev_warn(master->dev, + "stream %u already in tree\n", + cur_stream->id); + ret = -EINVAL; + break; + } + } + + if (!ret) { + rb_link_node(&new_stream->node, parent_node, new_node); + rb_insert_color(&new_stream->node, &smmu->streams); + } + } + mutex_unlock(&smmu->streams_mutex); + + return ret; +} + +static void arm_smmu_remove_master(struct arm_smmu_device *smmu, + struct arm_smmu_master_data *master) +{ + int i; + struct iommu_fwspec *fwspec = master->dev->iommu_fwspec; + + if (!master->streams) + return; + + mutex_lock(&smmu->streams_mutex); + for (i = 0; i < fwspec->num_ids; i++) + rb_erase(&master->streams[i].node, &smmu->streams); + mutex_unlock(&smmu->streams_mutex); + + kfree(master->streams); +} + static struct iommu_ops arm_smmu_ops; static int arm_smmu_add_device(struct device *dev) @@ -1930,13 +2032,35 @@ static int arm_smmu_add_device(struct device *dev) } } + ret = iommu_device_link(&smmu->iommu, dev); + if (ret) + goto err_free_master; + + ret = arm_smmu_insert_master(smmu, master); + if (ret) + goto err_unlink; + group = iommu_group_get_for_dev(dev); - if (!IS_ERR(group)) { - iommu_group_put(group); - iommu_device_link(&smmu->iommu, dev); + if (IS_ERR(group)) { + ret = PTR_ERR(group); + goto err_remove_master; } - return PTR_ERR_OR_ZERO(group); + iommu_group_put(group); + + return 0; + +err_remove_master: + arm_smmu_remove_master(smmu, master); + +err_unlink: + iommu_device_unlink(&smmu->iommu, dev); + +err_free_master: + kfree(master); + fwspec->iommu_priv = NULL; + + return ret; } static void arm_smmu_remove_device(struct device *dev) @@ -1953,6 +2077,7 @@ static void arm_smmu_remove_device(struct device *dev) if (master && master->ste.assigned) arm_smmu_detach_dev(dev); iommu_group_remove_device(dev); + arm_smmu_remove_master(smmu, master); iommu_device_unlink(&smmu->iommu, dev); kfree(master); iommu_fwspec_free(dev); @@ -2266,6 +2391,9 @@ static int arm_smmu_init_structures(struct arm_smmu_device *smmu) { int ret; + mutex_init(&smmu->streams_mutex); + smmu->streams = RB_ROOT; + ret = arm_smmu_init_queues(smmu); if (ret) return ret; From patchwork Tue Jan 8 10:26:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751791 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5C5FA13B4 for ; Tue, 8 Jan 2019 10:28:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4A3C028BC4 for ; Tue, 8 Jan 2019 10:28:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3E2DD28BDE; Tue, 8 Jan 2019 10:28:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BBC5428BC4 for ; Tue, 8 Jan 2019 10:28:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728775AbfAHK1v (ORCPT ); Tue, 8 Jan 2019 05:27:51 -0500 Received: from mx1.redhat.com ([209.132.183.28]:55826 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728158AbfAHK1v (ORCPT ); Tue, 8 Jan 2019 05:27:51 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 94767461CF; Tue, 8 Jan 2019 10:27:50 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id D0248601A3; Tue, 8 Jan 2019 10:27:40 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 09/21] iommu/smmuv3: Get prepared for nested stage support Date: Tue, 8 Jan 2019 11:26:21 +0100 Message-Id: <20190108102633.17482-10-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Tue, 08 Jan 2019 10:27:51 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP To allow nested stage support, we need to store both stage 1 and stage 2 configurations (and remove the former union). arm_smmu_write_strtab_ent() is modified to write both stage fields in the STE. We add a nested_bypass field to the S1 configuration as the first stage can be bypassed. Also the guest may force the STE to abort: this information gets stored into the nested_abort field. Only S2 stage is "finalized" as the host does not configure S1 CD, guest does. Signed-off-by: Eric Auger --- v1 -> v2: - invalidate the STE before moving from a live STE config to another - add the nested_abort and nested_bypass fields --- drivers/iommu/arm-smmu-v3.c | 43 ++++++++++++++++++++++++++++--------- 1 file changed, 33 insertions(+), 10 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 9af68266bbb1..9716a301d9ae 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -212,6 +212,7 @@ #define STRTAB_STE_0_CFG_BYPASS 4 #define STRTAB_STE_0_CFG_S1_TRANS 5 #define STRTAB_STE_0_CFG_S2_TRANS 6 +#define STRTAB_STE_0_CFG_NESTED 7 #define STRTAB_STE_0_S1FMT GENMASK_ULL(5, 4) #define STRTAB_STE_0_S1FMT_LINEAR 0 @@ -491,6 +492,10 @@ struct arm_smmu_strtab_l1_desc { struct arm_smmu_s1_cfg { __le64 *cdptr; dma_addr_t cdptr_dma; + /* in nested mode, tells s1 must be bypassed */ + bool nested_bypass; + /* in nested mode, abort is forced by guest */ + bool nested_abort; struct arm_smmu_ctx_desc { u16 asid; @@ -515,6 +520,7 @@ struct arm_smmu_strtab_ent { * configured according to the domain type. */ bool assigned; + bool nested; struct arm_smmu_s1_cfg *s1_cfg; struct arm_smmu_s2_cfg *s2_cfg; }; @@ -629,10 +635,8 @@ struct arm_smmu_domain { bool non_strict; enum arm_smmu_domain_stage stage; - union { - struct arm_smmu_s1_cfg s1_cfg; - struct arm_smmu_s2_cfg s2_cfg; - }; + struct arm_smmu_s1_cfg s1_cfg; + struct arm_smmu_s2_cfg s2_cfg; struct iommu_domain domain; @@ -1139,10 +1143,11 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, break; case STRTAB_STE_0_CFG_S1_TRANS: case STRTAB_STE_0_CFG_S2_TRANS: + case STRTAB_STE_0_CFG_NESTED: ste_live = true; break; case STRTAB_STE_0_CFG_ABORT: - if (disable_bypass) + if (disable_bypass || ste->nested) break; default: BUG(); /* STE corruption */ @@ -1154,7 +1159,8 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, /* Bypass/fault */ if (!ste->assigned || !(ste->s1_cfg || ste->s2_cfg)) { - if (!ste->assigned && disable_bypass) + if ((!ste->assigned && disable_bypass) || + (ste->s1_cfg && ste->s1_cfg->nested_abort)) val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT); else val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS); @@ -1172,8 +1178,17 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, return; } + if (ste->nested && ste_live) { + /* + * When enabling nested, the STE may be transitionning from + * s2 to nested and back. Invalidate the STE before changing it. + */ + dst[0] = cpu_to_le64(0); + arm_smmu_sync_ste_for_sid(smmu, sid); + val = STRTAB_STE_0_V; + } + if (ste->s1_cfg) { - BUG_ON(ste_live); dst[1] = cpu_to_le64( FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) | FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) | @@ -1187,12 +1202,12 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_device *smmu, u32 sid, !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE)) dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD); - val |= (ste->s1_cfg->cdptr_dma & STRTAB_STE_0_S1CTXPTR_MASK) | - FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS); + if (!ste->s1_cfg->nested_bypass) + val |= (ste->s1_cfg->cdptr_dma & STRTAB_STE_0_S1CTXPTR_MASK) | + FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS); } if (ste->s2_cfg) { - BUG_ON(ste_live); dst[2] = cpu_to_le64( FIELD_PREP(STRTAB_STE_2_S2VMID, ste->s2_cfg->vmid) | FIELD_PREP(STRTAB_STE_2_VTCR, ste->s2_cfg->vtcr) | @@ -1454,6 +1469,10 @@ static void arm_smmu_tlb_inv_context(void *cookie) cmd.opcode = CMDQ_OP_TLBI_NH_ASID; cmd.tlbi.asid = smmu_domain->s1_cfg.cd.asid; cmd.tlbi.vmid = 0; + } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED) { + cmd.opcode = CMDQ_OP_TLBI_NH_ASID; + cmd.tlbi.asid = smmu_domain->s1_cfg.cd.asid; + cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; } else { cmd.opcode = CMDQ_OP_TLBI_S12_VMALL; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; @@ -1484,6 +1503,10 @@ static void arm_smmu_tlb_inv_range_nosync(unsigned long iova, size_t size, if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { cmd.opcode = CMDQ_OP_TLBI_NH_VA; cmd.tlbi.asid = smmu_domain->s1_cfg.cd.asid; + } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED) { + cmd.opcode = CMDQ_OP_TLBI_NH_VA; + cmd.tlbi.asid = smmu_domain->s1_cfg.cd.asid; + cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; } else { cmd.opcode = CMDQ_OP_TLBI_S2_IPA; cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid; From patchwork Tue Jan 8 10:26:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751789 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A6ACD13B4 for ; Tue, 8 Jan 2019 10:28:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 93C2428BC4 for ; Tue, 8 Jan 2019 10:28:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 836D428BDE; Tue, 8 Jan 2019 10:28:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1D33328BC4 for ; Tue, 8 Jan 2019 10:28:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728656AbfAHK1z (ORCPT ); Tue, 8 Jan 2019 05:27:55 -0500 Received: from mx1.redhat.com ([209.132.183.28]:49830 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727597AbfAHK1z (ORCPT ); Tue, 8 Jan 2019 05:27:55 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D632D89AE8; Tue, 8 Jan 2019 10:27:54 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id E918A601A4; Tue, 8 Jan 2019 10:27:50 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 10/21] iommu/smmuv3: Implement set_pasid_table Date: Tue, 8 Jan 2019 11:26:22 +0100 Message-Id: <20190108102633.17482-11-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Tue, 08 Jan 2019 10:27:55 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On set_pasid_table() we program STE S1 related info set by the guest into the actual physical STEs. At minimum we need to program the context descriptor GPA and compute whether the guest wanted to bypass the stage 1 or induce aborts for this STE. Signed-off-by: Eric Auger --- v2 -> v3: - callback now is named set_pasid_table and struct fields are laid out differently. v1 -> v2: - invalidate the STE before changing them - hold init_mutex - handle new fields --- drivers/iommu/arm-smmu-v3.c | 68 +++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 9716a301d9ae..0e006babc8a6 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -2226,6 +2226,73 @@ static void arm_smmu_put_resv_regions(struct device *dev, kfree(entry); } +static int arm_smmu_set_pasid_table(struct iommu_domain *domain, + struct iommu_pasid_table_config *cfg) +{ + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_master_data *entry; + struct arm_smmu_s1_cfg *s1_cfg; + struct arm_smmu_device *smmu; + unsigned long flags; + int ret = -EINVAL; + + if (cfg->format != IOMMU_PASID_FORMAT_SMMUV3) + return -EINVAL; + + mutex_lock(&smmu_domain->init_mutex); + + smmu = smmu_domain->smmu; + + if (!smmu) + goto out; + + if (!((smmu->features & ARM_SMMU_FEAT_TRANS_S1) && + (smmu->features & ARM_SMMU_FEAT_TRANS_S2))) { + dev_info(smmu_domain->smmu->dev, + "does not implement two stages\n"); + goto out; + } + + if (smmu_domain->stage != ARM_SMMU_DOMAIN_NESTED) + goto out; + + if (cfg->bypass) { + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(entry, &smmu_domain->devices, list) { + entry->ste.s1_cfg = NULL; + entry->ste.nested = false; + arm_smmu_install_ste_for_dev(entry->dev->iommu_fwspec); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + + s1_cfg->nested_abort = false; + s1_cfg->nested_bypass = false; + ret = 0; + goto out; + } + + /* we currently support a single CD. S1DSS and S1FMT are ignored */ + if (cfg->pasid_bits) + goto out; + + s1_cfg = &smmu_domain->s1_cfg; + s1_cfg->nested_bypass = cfg->bypass; + s1_cfg->nested_abort = cfg->smmuv3.abort; + s1_cfg->cdptr_dma = cfg->base_ptr; + + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(entry, &smmu_domain->devices, list) { + entry->ste.s1_cfg = &smmu_domain->s1_cfg; + entry->ste.nested = true; + arm_smmu_install_ste_for_dev(entry->dev->iommu_fwspec); + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + ret = 0; +out: + mutex_unlock(&smmu_domain->init_mutex); + return ret; +} + static struct iommu_ops arm_smmu_ops = { .capable = arm_smmu_capable, .domain_alloc = arm_smmu_domain_alloc, @@ -2244,6 +2311,7 @@ static struct iommu_ops arm_smmu_ops = { .of_xlate = arm_smmu_of_xlate, .get_resv_regions = arm_smmu_get_resv_regions, .put_resv_regions = arm_smmu_put_resv_regions, + .set_pasid_table = arm_smmu_set_pasid_table, .pgsize_bitmap = -1UL, /* Restricted during device attach */ }; From patchwork Tue Jan 8 10:26:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751793 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D95671399 for ; Tue, 8 Jan 2019 10:28:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C6F7D28BC4 for ; Tue, 8 Jan 2019 10:28:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BACEE28BDE; Tue, 8 Jan 2019 10:28:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 638AB28BC4 for ; Tue, 8 Jan 2019 10:28:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728305AbfAHK2F (ORCPT ); Tue, 8 Jan 2019 05:28:05 -0500 Received: from mx1.redhat.com ([209.132.183.28]:51054 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727368AbfAHK2F (ORCPT ); Tue, 8 Jan 2019 05:28:05 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 64214C0528D7; Tue, 8 Jan 2019 10:27:59 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id 35BC7601A4; Tue, 8 Jan 2019 10:27:55 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 11/21] iommu/smmuv3: Implement cache_invalidate Date: Tue, 8 Jan 2019 11:26:23 +0100 Message-Id: <20190108102633.17482-12-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 08 Jan 2019 10:28:04 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Implement IOMMU_INV_TYPE_TLB invalidations. When nr_pages is null we interpret this as a context invalidation. Signed-off-by: Eric Auger --- The user API needs to be refined to discriminate context invalidations from NH_VA invalidations. Also the leaf attribute is not yet properly handled. v2 -> v3: - replace __arm_smmu_tlb_sync by arm_smmu_cmdq_issue_sync v1 -> v2: - properly pass the asid --- drivers/iommu/arm-smmu-v3.c | 40 +++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 0e006babc8a6..ca72e0ce92f6 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -2293,6 +2293,45 @@ static int arm_smmu_set_pasid_table(struct iommu_domain *domain, return ret; } +static int +arm_smmu_cache_invalidate(struct iommu_domain *domain, struct device *dev, + struct iommu_cache_invalidate_info *inv_info) +{ + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_device *smmu = smmu_domain->smmu; + + if (smmu_domain->stage != ARM_SMMU_DOMAIN_NESTED) + return -EINVAL; + + if (!smmu) + return -EINVAL; + + switch (inv_info->hdr.type) { + case IOMMU_INV_TYPE_TLB: + /* + * TODO: On context invalidation, the userspace sets nr_pages + * to 0. Refine the API to add a dedicated flags and also + * properly handle the leaf parameter. + */ + if (!inv_info->nr_pages) { + smmu_domain->s1_cfg.cd.asid = inv_info->arch_id; + arm_smmu_tlb_inv_context(smmu_domain); + } else { + size_t granule = 1 << (inv_info->size + 12); + size_t size = inv_info->nr_pages * granule; + + smmu_domain->s1_cfg.cd.asid = inv_info->arch_id; + arm_smmu_tlb_inv_range_nosync(inv_info->addr, size, + granule, false, + smmu_domain); + arm_smmu_cmdq_issue_sync(smmu); + } + return 0; + default: + return -EINVAL; + } +} + static struct iommu_ops arm_smmu_ops = { .capable = arm_smmu_capable, .domain_alloc = arm_smmu_domain_alloc, @@ -2312,6 +2351,7 @@ static struct iommu_ops arm_smmu_ops = { .get_resv_regions = arm_smmu_get_resv_regions, .put_resv_regions = arm_smmu_put_resv_regions, .set_pasid_table = arm_smmu_set_pasid_table, + .cache_invalidate = arm_smmu_cache_invalidate, .pgsize_bitmap = -1UL, /* Restricted during device attach */ }; From patchwork Tue Jan 8 10:26:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751799 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0380713B4 for ; Tue, 8 Jan 2019 10:28:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E2FD528BC4 for ; Tue, 8 Jan 2019 10:28:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D181C28BDE; Tue, 8 Jan 2019 10:28:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2A7F028BC4 for ; Tue, 8 Jan 2019 10:28:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728648AbfAHK2M (ORCPT ); Tue, 8 Jan 2019 05:28:12 -0500 Received: from mx1.redhat.com ([209.132.183.28]:46098 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728062AbfAHK2M (ORCPT ); Tue, 8 Jan 2019 05:28:12 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E68838E596; Tue, 8 Jan 2019 10:28:11 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id B8C27601A3; Tue, 8 Jan 2019 10:27:59 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 12/21] dma-iommu: Implement NESTED_MSI cookie Date: Tue, 8 Jan 2019 11:26:24 +0100 Message-Id: <20190108102633.17482-13-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 08 Jan 2019 10:28:12 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Up to now, when the type was UNMANAGED, we used to allocate IOVA pages within a range provided by the user. This does not work in nested mode. If both the host and the guest are exposed with SMMUs, each would allocate an IOVA. The guest allocates an IOVA (gIOVA) to map onto the guest MSI doorbell (gDB). The Host allocates another IOVA (hIOVA) to map onto the physical doorbell (hDB). So we end up with 2 unrelated mappings, at S1 and S2: S1 S2 gIOVA -> gDB hIOVA -> hDB The PCI device would be programmed with hIOVA. iommu_dma_bind_doorbell allows to pass gIOVA/gDB to the host so that gIOVA can be used by the host instead of re-allocating a new IOVA. The device handle also is passed to garantee devices belonging to different stage1 domains record distinguishable stage1 mappings. That way the host can create the following nested mapping: S1 S2 gIOVA -> gDB -> hDB this time, the PCI device will be programmed with the gIOVA MSI doorbell which is correctly mapped through the 2 stages. Signed-off-by: Eric Auger --- v2 -> v3: - also store the device handle on S1 mapping registration. This garantees we associate the associated S2 mapping binds to the correct physical MSI controller. v1 -> v2: - unmap stage2 on put() --- drivers/iommu/dma-iommu.c | 112 ++++++++++++++++++++++++++++++++++++-- include/linux/dma-iommu.h | 11 ++++ 2 files changed, 119 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index d19f3d6b43c1..19af8107e959 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -35,12 +35,15 @@ struct iommu_dma_msi_page { struct list_head list; dma_addr_t iova; + dma_addr_t ipa; phys_addr_t phys; + struct device *dev; }; enum iommu_dma_cookie_type { IOMMU_DMA_IOVA_COOKIE, IOMMU_DMA_MSI_COOKIE, + IOMMU_DMA_NESTED_MSI_COOKIE, }; struct iommu_dma_cookie { @@ -110,14 +113,17 @@ EXPORT_SYMBOL(iommu_get_dma_cookie); * * Users who manage their own IOVA allocation and do not want DMA API support, * but would still like to take advantage of automatic MSI remapping, can use - * this to initialise their own domain appropriately. Users should reserve a + * this to initialise their own domain appropriately. Users may reserve a * contiguous IOVA region, starting at @base, large enough to accommodate the * number of PAGE_SIZE mappings necessary to cover every MSI doorbell address - * used by the devices attached to @domain. + * used by the devices attached to @domain. The other way round is to provide + * usable iova pages through the iommu_dma_bind_doorbell API (nested stages + * use case) */ int iommu_get_msi_cookie(struct iommu_domain *domain, dma_addr_t base) { struct iommu_dma_cookie *cookie; + int nesting, ret; if (domain->type != IOMMU_DOMAIN_UNMANAGED) return -EINVAL; @@ -125,7 +131,12 @@ int iommu_get_msi_cookie(struct iommu_domain *domain, dma_addr_t base) if (domain->iova_cookie) return -EEXIST; - cookie = cookie_alloc(IOMMU_DMA_MSI_COOKIE); + ret = iommu_domain_get_attr(domain, DOMAIN_ATTR_NESTING, &nesting); + if (!ret && nesting) + cookie = cookie_alloc(IOMMU_DMA_NESTED_MSI_COOKIE); + else + cookie = cookie_alloc(IOMMU_DMA_MSI_COOKIE); + if (!cookie) return -ENOMEM; @@ -146,6 +157,7 @@ void iommu_put_dma_cookie(struct iommu_domain *domain) { struct iommu_dma_cookie *cookie = domain->iova_cookie; struct iommu_dma_msi_page *msi, *tmp; + bool s2_unmap = false; if (!cookie) return; @@ -153,7 +165,15 @@ void iommu_put_dma_cookie(struct iommu_domain *domain) if (cookie->type == IOMMU_DMA_IOVA_COOKIE && cookie->iovad.granule) put_iova_domain(&cookie->iovad); + if (cookie->type == IOMMU_DMA_NESTED_MSI_COOKIE) + s2_unmap = true; + list_for_each_entry_safe(msi, tmp, &cookie->msi_page_list, list) { + if (s2_unmap && msi->phys) { + size_t size = cookie_msi_granule(cookie); + + WARN_ON(iommu_unmap(domain, msi->ipa, size) != size); + } list_del(&msi->list); kfree(msi); } @@ -162,6 +182,52 @@ void iommu_put_dma_cookie(struct iommu_domain *domain) } EXPORT_SYMBOL(iommu_put_dma_cookie); +/** + * iommu_dma_bind_doorbell - Allows to provide a usable IOVA page + * @domain: domain handle + * @dev: device handle + * @binding: IOVA/IPA binding + * + * In nested stage use case, the user can provide IOVA/IPA bindings + * corresponding to a guest MSI stage 1 mapping. When the host needs + * to map its own MSI doorbells, it can use the IPA as stage 2 input + * and map it onto the physical MSI doorbell. + */ +int iommu_dma_bind_doorbell(struct iommu_domain *domain, struct device *dev, + struct iommu_guest_msi_binding *binding) +{ + struct iommu_dma_cookie *cookie = domain->iova_cookie; + struct iommu_dma_msi_page *msi; + dma_addr_t ipa, iova; + size_t size; + + if (!cookie) + return -EINVAL; + + if (cookie->type != IOMMU_DMA_NESTED_MSI_COOKIE) + return -EINVAL; + + size = 1 << binding->granule; + iova = binding->iova & ~(phys_addr_t)(size - 1); + ipa = binding->gpa & ~(phys_addr_t)(size - 1); + + list_for_each_entry(msi, &cookie->msi_page_list, list) { + if (msi->iova == iova && msi->dev == dev) + return 0; /* this page is already registered */ + } + + msi = kzalloc(sizeof(*msi), GFP_KERNEL); + if (!msi) + return -ENOMEM; + + msi->iova = iova; + msi->ipa = ipa; + msi->dev = dev; + list_add(&msi->list, &cookie->msi_page_list); + return 0; +} +EXPORT_SYMBOL(iommu_dma_bind_doorbell); + /** * iommu_dma_get_resv_regions - Reserved region driver helper * @dev: Device from iommu_get_resv_regions() @@ -856,6 +922,16 @@ void iommu_dma_unmap_resource(struct device *dev, dma_addr_t handle, __iommu_dma_unmap(iommu_get_dma_domain(dev), handle, size); } +static bool msi_page_match(struct iommu_dma_msi_page *msi_page, + struct device *dev, phys_addr_t msi_addr) +{ + bool match = msi_page->phys == msi_addr; + + if (msi_page->dev) + match &= (msi_page->dev == dev); + return match; +} + static struct iommu_dma_msi_page *iommu_dma_get_msi_page(struct device *dev, phys_addr_t msi_addr, struct iommu_domain *domain) { @@ -867,9 +943,37 @@ static struct iommu_dma_msi_page *iommu_dma_get_msi_page(struct device *dev, msi_addr &= ~(phys_addr_t)(size - 1); list_for_each_entry(msi_page, &cookie->msi_page_list, list) - if (msi_page->phys == msi_addr) + if (msi_page_match(msi_page, dev, msi_addr)) return msi_page; + /* + * In nested stage mode, we do not allocate an MSI page in + * a range provided by the user. Instead, IOVA/IPA bindings are + * individually provided. We reuse thise IOVAs to build the + * IOVA -> IPA -> MSI PA nested stage mapping. + */ + if (cookie->type == IOMMU_DMA_NESTED_MSI_COOKIE) { + list_for_each_entry(msi_page, &cookie->msi_page_list, list) + if (!msi_page->phys && msi_page->dev == dev) { + dma_addr_t ipa = msi_page->ipa; + int ret; + + msi_page->phys = msi_addr; + + /* do the stage 2 mapping */ + ret = iommu_map(domain, ipa, msi_addr, size, + IOMMU_MMIO | IOMMU_WRITE); + if (ret) { + pr_warn("MSI S2 mapping failed (%d)\n", + ret); + return NULL; + } + return msi_page; + } + pr_warn("%s no MSI binding found\n", __func__); + return NULL; + } + msi_page = kzalloc(sizeof(*msi_page), GFP_ATOMIC); if (!msi_page) return NULL; diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h index e760dc5d1fa8..778243719462 100644 --- a/include/linux/dma-iommu.h +++ b/include/linux/dma-iommu.h @@ -24,6 +24,7 @@ #include #include #include +#include int iommu_dma_init(void); @@ -73,12 +74,15 @@ void iommu_dma_unmap_resource(struct device *dev, dma_addr_t handle, /* The DMA API isn't _quite_ the whole story, though... */ void iommu_dma_map_msi_msg(int irq, struct msi_msg *msg); void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list); +int iommu_dma_bind_doorbell(struct iommu_domain *domain, struct device *dev, + struct iommu_guest_msi_binding *binding); #else struct iommu_domain; struct msi_msg; struct device; +struct iommu_guest_msi_binding; static inline int iommu_dma_init(void) { @@ -103,6 +107,13 @@ static inline void iommu_dma_map_msi_msg(int irq, struct msi_msg *msg) { } +static inline int +iommu_dma_bind_doorbell(struct iommu_domain *domain, + struct iommu_guest_msi_binding *binding) +{ + return -ENODEV; +} + static inline void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list) { } From patchwork Tue Jan 8 10:26:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751797 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ACFF51399 for ; Tue, 8 Jan 2019 10:28:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9997F28BC4 for ; Tue, 8 Jan 2019 10:28:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8C45C28BDE; Tue, 8 Jan 2019 10:28:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 34DA028BC4 for ; Tue, 8 Jan 2019 10:28:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728844AbfAHK2S (ORCPT ); Tue, 8 Jan 2019 05:28:18 -0500 Received: from mx1.redhat.com ([209.132.183.28]:55664 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728833AbfAHK2R (ORCPT ); Tue, 8 Jan 2019 05:28:17 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 76FBA3E2CC; Tue, 8 Jan 2019 10:28:16 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4C490601A3; Tue, 8 Jan 2019 10:28:12 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 13/21] iommu/smmuv3: Implement bind_guest_msi Date: Tue, 8 Jan 2019 11:26:25 +0100 Message-Id: <20190108102633.17482-14-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 08 Jan 2019 10:28:16 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The bind_guest_msi() callback checks the domain is NESTED and redirect to the dma-iommu implementation. Signed-off-by: Eric Auger --- drivers/iommu/arm-smmu-v3.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index ca72e0ce92f6..999ee470a2ae 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -2226,6 +2226,28 @@ static void arm_smmu_put_resv_regions(struct device *dev, kfree(entry); } +static int arm_smmu_bind_guest_msi(struct iommu_domain *domain, + struct device *dev, + struct iommu_guest_msi_binding *binding) +{ + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_device *smmu; + int ret = -EINVAL; + + mutex_lock(&smmu_domain->init_mutex); + smmu = smmu_domain->smmu; + if (!smmu) + goto out; + + if (smmu_domain->stage != ARM_SMMU_DOMAIN_NESTED) + goto out; + + ret = iommu_dma_bind_doorbell(domain, dev, binding); +out: + mutex_unlock(&smmu_domain->init_mutex); + return ret; +} + static int arm_smmu_set_pasid_table(struct iommu_domain *domain, struct iommu_pasid_table_config *cfg) { @@ -2352,6 +2374,7 @@ static struct iommu_ops arm_smmu_ops = { .put_resv_regions = arm_smmu_put_resv_regions, .set_pasid_table = arm_smmu_set_pasid_table, .cache_invalidate = arm_smmu_cache_invalidate, + .bind_guest_msi = arm_smmu_bind_guest_msi, .pgsize_bitmap = -1UL, /* Restricted during device attach */ }; From patchwork Tue Jan 8 10:26:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751795 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B895F13B4 for ; Tue, 8 Jan 2019 10:28:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A546028BC4 for ; Tue, 8 Jan 2019 10:28:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 992F628BDE; Tue, 8 Jan 2019 10:28:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5DBAA28BC4 for ; Tue, 8 Jan 2019 10:28:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728684AbfAHK2W (ORCPT ); Tue, 8 Jan 2019 05:28:22 -0500 Received: from mx1.redhat.com ([209.132.183.28]:59772 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728398AbfAHK2V (ORCPT ); Tue, 8 Jan 2019 05:28:21 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id E8679804F2; Tue, 8 Jan 2019 10:28:20 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id CCDAD601A3; Tue, 8 Jan 2019 10:28:16 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 14/21] iommu: introduce device fault data Date: Tue, 8 Jan 2019 11:26:26 +0100 Message-Id: <20190108102633.17482-15-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Tue, 08 Jan 2019 10:28:21 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jacob Pan Device faults detected by IOMMU can be reported outside IOMMU subsystem for further processing. This patch intends to provide a generic device fault data such that device drivers can be communicated with IOMMU faults without model specific knowledge. The proposed format is the result of discussion at: https://lkml.org/lkml/2017/11/10/291 Part of the code is based on Jean-Philippe Brucker's patchset (https://patchwork.kernel.org/patch/9989315/). The assumption is that model specific IOMMU driver can filter and handle most of the internal faults if the cause is within IOMMU driver control. Therefore, the fault reasons can be reported are grouped and generalized based common specifications such as PCI ATS. Signed-off-by: Jacob Pan Signed-off-by: Jean-Philippe Brucker Signed-off-by: Liu, Yi L Signed-off-by: Ashok Raj Signed-off-by: Eric Auger [moved part of the iommu_fault_event struct in the uapi, enriched the fault reasons to be able to map unrecoverable SMMUv3 errors] --- include/linux/iommu.h | 55 ++++++++++++++++++++++++- include/uapi/linux/iommu.h | 83 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 136 insertions(+), 2 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 244c1a3d5989..1dedc2d247c2 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -49,13 +49,17 @@ struct bus_type; struct device; struct iommu_domain; struct notifier_block; +struct iommu_fault_event; /* iommu fault flags */ -#define IOMMU_FAULT_READ 0x0 -#define IOMMU_FAULT_WRITE 0x1 +#define IOMMU_FAULT_READ (1 << 0) +#define IOMMU_FAULT_WRITE (1 << 1) +#define IOMMU_FAULT_EXEC (1 << 2) +#define IOMMU_FAULT_PRIV (1 << 3) typedef int (*iommu_fault_handler_t)(struct iommu_domain *, struct device *, unsigned long, int, void *); +typedef int (*iommu_dev_fault_handler_t)(struct iommu_fault_event *, void *); struct iommu_domain_geometry { dma_addr_t aperture_start; /* First address that can be mapped */ @@ -255,6 +259,52 @@ struct iommu_device { struct device *dev; }; +/** + * struct iommu_fault_event - Generic per device fault data + * + * - PCI and non-PCI devices + * - Recoverable faults (e.g. page request), information based on PCI ATS + * and PASID spec. + * - Un-recoverable faults of device interest + * - DMA remapping and IRQ remapping faults + * + * @fault: fault descriptor + * @device_private: if present, uniquely identify device-specific + * private data for an individual page request. + * @iommu_private: used by the IOMMU driver for storing fault-specific + * data. Users should not modify this field before + * sending the fault response. + */ +struct iommu_fault_event { + struct iommu_fault fault; + u64 device_private; + u64 iommu_private; +}; + +/** + * struct iommu_fault_param - per-device IOMMU fault data + * @dev_fault_handler: Callback function to handle IOMMU faults at device level + * @data: handler private data + * + */ +struct iommu_fault_param { + iommu_dev_fault_handler_t handler; + void *data; +}; + +/** + * struct iommu_param - collection of per-device IOMMU data + * + * @fault_param: IOMMU detected device fault reporting data + * + * TODO: migrate other per device data pointers under iommu_dev_data, e.g. + * struct iommu_group *iommu_group; + * struct iommu_fwspec *iommu_fwspec; + */ +struct iommu_param { + struct iommu_fault_param *fault_param; +}; + int iommu_device_register(struct iommu_device *iommu); void iommu_device_unregister(struct iommu_device *iommu); int iommu_device_sysfs_add(struct iommu_device *iommu, @@ -438,6 +488,7 @@ struct iommu_ops {}; struct iommu_group {}; struct iommu_fwspec {}; struct iommu_device {}; +struct iommu_fault_param {}; static inline bool iommu_present(struct bus_type *bus) { diff --git a/include/uapi/linux/iommu.h b/include/uapi/linux/iommu.h index f28cd9a1aa96..e9b5330a13c8 100644 --- a/include/uapi/linux/iommu.h +++ b/include/uapi/linux/iommu.h @@ -148,4 +148,87 @@ struct iommu_guest_msi_binding { __u64 gpa; __u32 granule; }; + +/* Generic fault types, can be expanded IRQ remapping fault */ +enum iommu_fault_type { + IOMMU_FAULT_DMA_UNRECOV = 1, /* unrecoverable fault */ + IOMMU_FAULT_PAGE_REQ, /* page request fault */ +}; + +enum iommu_fault_reason { + IOMMU_FAULT_REASON_UNKNOWN = 0, + + /* IOMMU internal error, no specific reason to report out */ + IOMMU_FAULT_REASON_INTERNAL, + + /* Could not access the PASID table (fetch caused external abort) */ + IOMMU_FAULT_REASON_PASID_FETCH, + + /* could not access the device context (fetch caused external abort) */ + IOMMU_FAULT_REASON_DEVICE_CONTEXT_FETCH, + + /* pasid entry is invalid or has configuration errors */ + IOMMU_FAULT_REASON_BAD_PASID_ENTRY, + + /* device context entry is invalid or has configuration errors */ + IOMMU_FAULT_REASON_BAD_DEVICE_CONTEXT_ENTRY, + /* + * PASID is out of range (e.g. exceeds the maximum PASID + * supported by the IOMMU) or disabled. + */ + IOMMU_FAULT_REASON_PASID_INVALID, + + /* source id is out of range */ + IOMMU_FAULT_REASON_SOURCEID_INVALID, + + /* + * An external abort occurred fetching (or updating) a translation + * table descriptor + */ + IOMMU_FAULT_REASON_WALK_EABT, + + /* + * Could not access the page table entry (Bad address), + * actual translation fault + */ + IOMMU_FAULT_REASON_PTE_FETCH, + + /* Protection flag check failed */ + IOMMU_FAULT_REASON_PERMISSION, + + /* access flag check failed */ + IOMMU_FAULT_REASON_ACCESS, + + /* Output address of a translation stage caused Address Size fault */ + IOMMU_FAULT_REASON_OOR_ADDRESS +}; + +/** + * struct iommu_fault - Generic fault data + * + * @type contains fault type + * @reason fault reasons if relevant outside IOMMU driver. + * IOMMU driver internal faults are not reported. + * @addr: tells the offending page address + * @fetch_addr: tells the address that caused an abort, if any + * @pasid: contains process address space ID, used in shared virtual memory + * @page_req_group_id: page request group index + * @last_req: last request in a page request group + * @pasid_valid: indicates if the PRQ has a valid PASID + * @prot: page access protection flag: + * IOMMU_FAULT_READ, IOMMU_FAULT_WRITE + */ + +struct iommu_fault { + __u32 type; /* enum iommu_fault_type */ + __u32 reason; /* enum iommu_fault_reason */ + __u64 addr; + __u64 fetch_addr; + __u32 pasid; + __u32 page_req_group_id; + __u32 last_req; + __u32 pasid_valid; + __u32 prot; + __u32 access; +}; #endif /* _UAPI_IOMMU_H */ From patchwork Tue Jan 8 10:26:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751805 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 746081399 for ; Tue, 8 Jan 2019 10:28:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6394B1FFE6 for ; Tue, 8 Jan 2019 10:28:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 56C6727E01; Tue, 8 Jan 2019 10:28:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F16C01FFE6 for ; Tue, 8 Jan 2019 10:28:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728870AbfAHK2c (ORCPT ); Tue, 8 Jan 2019 05:28:32 -0500 Received: from mx1.redhat.com ([209.132.183.28]:46282 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728062AbfAHK2b (ORCPT ); Tue, 8 Jan 2019 05:28:31 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id DFBC088314; Tue, 8 Jan 2019 10:28:30 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id 474916013A; Tue, 8 Jan 2019 10:28:21 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 15/21] driver core: add per device iommu param Date: Tue, 8 Jan 2019 11:26:27 +0100 Message-Id: <20190108102633.17482-16-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 08 Jan 2019 10:28:31 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jacob Pan DMA faults can be detected by IOMMU at device level. Adding a pointer to struct device allows IOMMU subsystem to report relevant faults back to the device driver for further handling. For direct assigned device (or user space drivers), guest OS holds responsibility to handle and respond per device IOMMU fault. Therefore we need fault reporting mechanism to propagate faults beyond IOMMU subsystem. There are two other IOMMU data pointers under struct device today, here we introduce iommu_param as a parent pointer such that all device IOMMU data can be consolidated here. The idea was suggested here by Greg KH and Joerg. The name iommu_param is chosen here since iommu_data has been used. Suggested-by: Greg Kroah-Hartman Reviewed-by: Greg Kroah-Hartman Signed-off-by: Jacob Pan Link: https://lkml.org/lkml/2017/10/6/81 --- include/linux/device.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/device.h b/include/linux/device.h index 6cb4640b6160..fd7f9fae404e 100644 --- a/include/linux/device.h +++ b/include/linux/device.h @@ -42,6 +42,7 @@ struct iommu_ops; struct iommu_group; struct iommu_fwspec; struct dev_pin_info; +struct iommu_param; struct bus_attribute { struct attribute attr; @@ -950,6 +951,7 @@ struct dev_links_info { * device (i.e. the bus driver that discovered the device). * @iommu_group: IOMMU group the device belongs to. * @iommu_fwspec: IOMMU-specific properties supplied by firmware. + * @iommu_param: Per device generic IOMMU runtime data * * @offline_disabled: If set, the device is permanently online. * @offline: Set after successful invocation of bus type's .offline(). @@ -1042,6 +1044,7 @@ struct device { void (*release)(struct device *dev); struct iommu_group *iommu_group; struct iommu_fwspec *iommu_fwspec; + struct iommu_param *iommu_param; bool offline_disabled:1; bool offline:1; From patchwork Tue Jan 8 10:26:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751803 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0F5781399 for ; Tue, 8 Jan 2019 10:28:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EEB152580E for ; Tue, 8 Jan 2019 10:28:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DD8421FFE6; Tue, 8 Jan 2019 10:28:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F10911FFE6 for ; Tue, 8 Jan 2019 10:28:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728897AbfAHK2h (ORCPT ); Tue, 8 Jan 2019 05:28:37 -0500 Received: from mx1.redhat.com ([209.132.183.28]:51592 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728873AbfAHK2g (ORCPT ); Tue, 8 Jan 2019 05:28:36 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 59F53C049580; Tue, 8 Jan 2019 10:28:35 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3E9B9601A3; Tue, 8 Jan 2019 10:28:31 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 16/21] iommu: introduce device fault report API Date: Tue, 8 Jan 2019 11:26:28 +0100 Message-Id: <20190108102633.17482-17-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Tue, 08 Jan 2019 10:28:35 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Jacob Pan Traditionally, device specific faults are detected and handled within their own device drivers. When IOMMU is enabled, faults such as DMA related transactions are detected by IOMMU. There is no generic reporting mechanism to report faults back to the in-kernel device driver or the guest OS in case of assigned devices. Faults detected by IOMMU is based on the transaction's source ID which can be reported at per device basis, regardless of the device type is a PCI device or not. The fault types include recoverable (e.g. page request) and unrecoverable faults(e.g. access error). In most cases, faults can be handled by IOMMU drivers internally. The primary use cases are as follows: 1. page request fault originated from an SVM capable device that is assigned to guest via vIOMMU. In this case, the first level page tables are owned by the guest. Page request must be propagated to the guest to let guest OS fault in the pages then send page response. In this mechanism, the direct receiver of IOMMU fault notification is VFIO, which can relay notification events to QEMU or other user space software. 2. faults need more subtle handling by device drivers. Other than simply invoke reset function, there are needs to let device driver handle the fault with a smaller impact. This patchset is intended to create a generic fault report API such that it can scale as follows: - all IOMMU types - PCI and non-PCI devices - recoverable and unrecoverable faults - VFIO and other other in kernel users - DMA & IRQ remapping (TBD) The original idea was brought up by David Woodhouse and discussions summarized at https://lwn.net/Articles/608914/. Signed-off-by: Jacob Pan Signed-off-by: Ashok Raj Signed-off-by: Jean-Philippe Brucker Signed-off-by: Eric Auger [adapt to new iommu_fault fault field, test fault_param on iommu_unregister_device_fault_handler] --- drivers/iommu/iommu.c | 153 +++++++++++++++++++++++++++++++++++++++++- include/linux/iommu.h | 33 ++++++++- 2 files changed, 184 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index ea11442e7054..fb13d83914a6 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -648,6 +648,13 @@ int iommu_group_add_device(struct iommu_group *group, struct device *dev) goto err_free_name; } + dev->iommu_param = kzalloc(sizeof(*dev->iommu_param), GFP_KERNEL); + if (!dev->iommu_param) { + ret = -ENOMEM; + goto err_free_name; + } + mutex_init(&dev->iommu_param->lock); + kobject_get(group->devices_kobj); dev->iommu_group = group; @@ -678,6 +685,7 @@ int iommu_group_add_device(struct iommu_group *group, struct device *dev) mutex_unlock(&group->mutex); dev->iommu_group = NULL; kobject_put(group->devices_kobj); + kfree(dev->iommu_param); err_free_name: kfree(device->name); err_remove_link: @@ -724,7 +732,7 @@ void iommu_group_remove_device(struct device *dev) sysfs_remove_link(&dev->kobj, "iommu_group"); trace_remove_device_from_group(group->id, dev); - + kfree(dev->iommu_param); kfree(device->name); kfree(device); dev->iommu_group = NULL; @@ -858,6 +866,149 @@ int iommu_group_unregister_notifier(struct iommu_group *group, } EXPORT_SYMBOL_GPL(iommu_group_unregister_notifier); +/** + * iommu_register_device_fault_handler() - Register a device fault handler + * @dev: the device + * @handler: the fault handler + * @data: private data passed as argument to the handler + * + * When an IOMMU fault event is received, call this handler with the fault event + * and data as argument. The handler should return 0 on success. If the fault is + * recoverable (IOMMU_FAULT_PAGE_REQ), the handler can also complete + * the fault by calling iommu_page_response() with one of the following + * response code: + * - IOMMU_PAGE_RESP_SUCCESS: retry the translation + * - IOMMU_PAGE_RESP_INVALID: terminate the fault + * - IOMMU_PAGE_RESP_FAILURE: terminate the fault and stop reporting + * page faults if possible. + * + * Return 0 if the fault handler was installed successfully, or an error. + */ +int iommu_register_device_fault_handler(struct device *dev, + iommu_dev_fault_handler_t handler, + void *data) +{ + struct iommu_param *param = dev->iommu_param; + int ret = 0; + + /* + * Device iommu_param should have been allocated when device is + * added to its iommu_group. + */ + if (!param) + return -EINVAL; + + mutex_lock(¶m->lock); + /* Only allow one fault handler registered for each device */ + if (param->fault_param) { + ret = -EBUSY; + goto done_unlock; + } + + get_device(dev); + param->fault_param = + kzalloc(sizeof(struct iommu_fault_param), GFP_KERNEL); + if (!param->fault_param) { + put_device(dev); + ret = -ENOMEM; + goto done_unlock; + } + mutex_init(¶m->fault_param->lock); + param->fault_param->handler = handler; + param->fault_param->data = data; + INIT_LIST_HEAD(¶m->fault_param->faults); + +done_unlock: + mutex_unlock(¶m->lock); + + return ret; +} +EXPORT_SYMBOL_GPL(iommu_register_device_fault_handler); + +/** + * iommu_unregister_device_fault_handler() - Unregister the device fault handler + * @dev: the device + * + * Remove the device fault handler installed with + * iommu_register_device_fault_handler(). + * + * Return 0 on success, or an error. + */ +int iommu_unregister_device_fault_handler(struct device *dev) +{ + struct iommu_param *param = dev->iommu_param; + int ret = 0; + + if (!param) + return -EINVAL; + + mutex_lock(¶m->lock); + + if (!param->fault_param) + goto unlock; + + /* we cannot unregister handler if there are pending faults */ + if (!list_empty(¶m->fault_param->faults)) { + ret = -EBUSY; + goto unlock; + } + + kfree(param->fault_param); + param->fault_param = NULL; + put_device(dev); +unlock: + mutex_unlock(¶m->lock); + + return ret; +} +EXPORT_SYMBOL_GPL(iommu_unregister_device_fault_handler); + + +/** + * iommu_report_device_fault() - Report fault event to device + * @dev: the device + * @evt: fault event data + * + * Called by IOMMU model specific drivers when fault is detected, typically + * in a threaded IRQ handler. + * + * Return 0 on success, or an error. + */ +int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt) +{ + int ret = 0; + struct iommu_fault_event *evt_pending; + struct iommu_fault_param *fparam; + + /* iommu_param is allocated when device is added to group */ + if (!dev->iommu_param | !evt) + return -EINVAL; + /* we only report device fault if there is a handler registered */ + mutex_lock(&dev->iommu_param->lock); + if (!dev->iommu_param->fault_param || + !dev->iommu_param->fault_param->handler) { + ret = -EINVAL; + goto done_unlock; + } + fparam = dev->iommu_param->fault_param; + if (evt->fault.type == IOMMU_FAULT_PAGE_REQ && evt->fault.last_req) { + evt_pending = kmemdup(evt, sizeof(struct iommu_fault_event), + GFP_KERNEL); + if (!evt_pending) { + ret = -ENOMEM; + goto done_unlock; + } + mutex_lock(&fparam->lock); + list_add_tail(&evt_pending->list, &fparam->faults); + mutex_unlock(&fparam->lock); + } + ret = fparam->handler(evt, fparam->data); +done_unlock: + mutex_unlock(&dev->iommu_param->lock); + return ret; +} +EXPORT_SYMBOL_GPL(iommu_report_device_fault); + /** * iommu_group_id - Return ID for a group * @group: the group to ID diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 1dedc2d247c2..a39bf9e040d4 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -269,6 +269,7 @@ struct iommu_device { * - DMA remapping and IRQ remapping faults * * @fault: fault descriptor + * @list pending fault event list, used for tracking responses * @device_private: if present, uniquely identify device-specific * private data for an individual page request. * @iommu_private: used by the IOMMU driver for storing fault-specific @@ -276,6 +277,7 @@ struct iommu_device { * sending the fault response. */ struct iommu_fault_event { + struct list_head list; struct iommu_fault fault; u64 device_private; u64 iommu_private; @@ -285,10 +287,13 @@ struct iommu_fault_event { * struct iommu_fault_param - per-device IOMMU fault data * @dev_fault_handler: Callback function to handle IOMMU faults at device level * @data: handler private data - * + * @faults: holds the pending faults which needs response, e.g. page response. + * @lock: protect pending PRQ event list */ struct iommu_fault_param { iommu_dev_fault_handler_t handler; + struct list_head faults; + struct mutex lock; void *data; }; @@ -302,6 +307,7 @@ struct iommu_fault_param { * struct iommu_fwspec *iommu_fwspec; */ struct iommu_param { + struct mutex lock; struct iommu_fault_param *fault_param; }; @@ -402,6 +408,14 @@ extern int iommu_group_register_notifier(struct iommu_group *group, struct notifier_block *nb); extern int iommu_group_unregister_notifier(struct iommu_group *group, struct notifier_block *nb); +extern int iommu_register_device_fault_handler(struct device *dev, + iommu_dev_fault_handler_t handler, + void *data); + +extern int iommu_unregister_device_fault_handler(struct device *dev); + +extern int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt); + extern int iommu_group_id(struct iommu_group *group); extern struct iommu_group *iommu_group_get_for_dev(struct device *dev); extern struct iommu_domain *iommu_group_default_domain(struct iommu_group *); @@ -682,6 +696,23 @@ static inline int iommu_group_unregister_notifier(struct iommu_group *group, return 0; } +static inline int iommu_register_device_fault_handler(struct device *dev, + iommu_dev_fault_handler_t handler, + void *data) +{ + return -ENODEV; +} + +static inline int iommu_unregister_device_fault_handler(struct device *dev) +{ + return 0; +} + +static inline int iommu_report_device_fault(struct device *dev, struct iommu_fault_event *evt) +{ + return -ENODEV; +} + static inline int iommu_group_id(struct iommu_group *group) { return -ENODEV; From patchwork Tue Jan 8 10:26:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751801 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6EE8713B4 for ; Tue, 8 Jan 2019 10:28:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5D7EC1FFE6 for ; Tue, 8 Jan 2019 10:28:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4FFB926E51; Tue, 8 Jan 2019 10:28:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BBF691FFE6 for ; Tue, 8 Jan 2019 10:28:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728922AbfAHK2l (ORCPT ); Tue, 8 Jan 2019 05:28:41 -0500 Received: from mx1.redhat.com ([209.132.183.28]:40986 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728875AbfAHK2k (ORCPT ); Tue, 8 Jan 2019 05:28:40 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 92DA3A6DFA; Tue, 8 Jan 2019 10:28:39 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id ADAF5601A3; Tue, 8 Jan 2019 10:28:35 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 17/21] iommu/smmuv3: Report non recoverable faults Date: Tue, 8 Jan 2019 11:26:29 +0100 Message-Id: <20190108102633.17482-18-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Tue, 08 Jan 2019 10:28:39 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When a stage 1 related fault event is read from the event queue, let's propagate it to potential external fault listeners, ie. users who registered a fault handler. Signed-off-by: Eric Auger --- drivers/iommu/arm-smmu-v3.c | 124 ++++++++++++++++++++++++++++++++---- 1 file changed, 113 insertions(+), 11 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 999ee470a2ae..6a711cbbb228 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -168,6 +168,26 @@ #define ARM_SMMU_PRIQ_IRQ_CFG1 0xd8 #define ARM_SMMU_PRIQ_IRQ_CFG2 0xdc +/* Events */ +#define ARM_SMMU_EVT_F_UUT 0x01 +#define ARM_SMMU_EVT_C_BAD_STREAMID 0x02 +#define ARM_SMMU_EVT_F_STE_FETCH 0x03 +#define ARM_SMMU_EVT_C_BAD_STE 0x04 +#define ARM_SMMU_EVT_F_BAD_ATS_TREQ 0x05 +#define ARM_SMMU_EVT_F_STREAM_DISABLED 0x06 +#define ARM_SMMU_EVT_F_TRANSL_FORBIDDEN 0x07 +#define ARM_SMMU_EVT_C_BAD_SUBSTREAMID 0x08 +#define ARM_SMMU_EVT_F_CD_FETCH 0x09 +#define ARM_SMMU_EVT_C_BAD_CD 0x0a +#define ARM_SMMU_EVT_F_WALK_EABT 0x0b +#define ARM_SMMU_EVT_F_TRANSLATION 0x10 +#define ARM_SMMU_EVT_F_ADDR_SIZE 0x11 +#define ARM_SMMU_EVT_F_ACCESS 0x12 +#define ARM_SMMU_EVT_F_PERMISSION 0x13 +#define ARM_SMMU_EVT_F_TLB_CONFLICT 0x20 +#define ARM_SMMU_EVT_F_CFG_CONFLICT 0x21 +#define ARM_SMMU_EVT_E_PAGE_REQUEST 0x24 + /* Common MSI config fields */ #define MSI_CFG0_ADDR_MASK GENMASK_ULL(51, 2) #define MSI_CFG2_SH GENMASK(5, 4) @@ -333,6 +353,11 @@ #define EVTQ_MAX_SZ_SHIFT 7 #define EVTQ_0_ID GENMASK_ULL(7, 0) +#define EVTQ_0_SUBSTREAMID GENMASK_ULL(31, 12) +#define EVTQ_0_STREAMID GENMASK_ULL(63, 32) +#define EVTQ_1_S2 GENMASK_ULL(39, 39) +#define EVTQ_1_CLASS GENMASK_ULL(40, 41) +#define EVTQ_3_FETCH_ADDR GENMASK_ULL(51, 3) /* PRI queue */ #define PRIQ_ENT_DWORDS 2 @@ -1270,7 +1295,6 @@ static int arm_smmu_init_l2_strtab(struct arm_smmu_device *smmu, u32 sid) return 0; } -__maybe_unused static struct arm_smmu_master_data * arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid) { @@ -1296,24 +1320,102 @@ arm_smmu_find_master(struct arm_smmu_device *smmu, u32 sid) return master; } +static void arm_smmu_report_event(struct arm_smmu_device *smmu, u64 *evt) +{ + u64 fetch_addr = FIELD_GET(EVTQ_3_FETCH_ADDR, evt[3]); + u32 sid = FIELD_GET(EVTQ_0_STREAMID, evt[0]); + bool s1 = !FIELD_GET(EVTQ_1_S2, evt[1]); + u8 type = FIELD_GET(EVTQ_0_ID, evt[0]); + struct arm_smmu_master_data *master; + struct iommu_fault_event event; + bool propagate = true; + u64 addr = evt[2]; + int i; + + master = arm_smmu_find_master(smmu, sid); + if (WARN_ON(!master)) + return; + + event.fault.type = IOMMU_FAULT_DMA_UNRECOV; + + switch (type) { + case ARM_SMMU_EVT_C_BAD_STREAMID: + event.fault.reason = IOMMU_FAULT_REASON_SOURCEID_INVALID; + break; + case ARM_SMMU_EVT_F_STREAM_DISABLED: + case ARM_SMMU_EVT_C_BAD_SUBSTREAMID: + event.fault.reason = IOMMU_FAULT_REASON_PASID_INVALID; + break; + case ARM_SMMU_EVT_F_CD_FETCH: + event.fault.reason = IOMMU_FAULT_REASON_PASID_FETCH; + break; + case ARM_SMMU_EVT_F_WALK_EABT: + event.fault.reason = IOMMU_FAULT_REASON_WALK_EABT; + event.fault.addr = addr; + event.fault.fetch_addr = fetch_addr; + propagate = s1; + break; + case ARM_SMMU_EVT_F_TRANSLATION: + event.fault.reason = IOMMU_FAULT_REASON_PTE_FETCH; + event.fault.addr = addr; + event.fault.fetch_addr = fetch_addr; + propagate = s1; + break; + case ARM_SMMU_EVT_F_PERMISSION: + event.fault.reason = IOMMU_FAULT_REASON_PERMISSION; + event.fault.addr = addr; + propagate = s1; + break; + case ARM_SMMU_EVT_F_ACCESS: + event.fault.reason = IOMMU_FAULT_REASON_ACCESS; + event.fault.addr = addr; + propagate = s1; + break; + case ARM_SMMU_EVT_C_BAD_STE: + event.fault.reason = IOMMU_FAULT_REASON_BAD_DEVICE_CONTEXT_ENTRY; + break; + case ARM_SMMU_EVT_C_BAD_CD: + event.fault.reason = IOMMU_FAULT_REASON_BAD_PASID_ENTRY; + break; + case ARM_SMMU_EVT_F_ADDR_SIZE: + event.fault.reason = IOMMU_FAULT_REASON_OOR_ADDRESS; + propagate = s1; + break; + case ARM_SMMU_EVT_F_STE_FETCH: + event.fault.reason = IOMMU_FAULT_REASON_DEVICE_CONTEXT_FETCH; + event.fault.fetch_addr = fetch_addr; + break; + /* End of addition */ + case ARM_SMMU_EVT_E_PAGE_REQUEST: + case ARM_SMMU_EVT_F_TLB_CONFLICT: + case ARM_SMMU_EVT_F_CFG_CONFLICT: + case ARM_SMMU_EVT_F_BAD_ATS_TREQ: + case ARM_SMMU_EVT_F_TRANSL_FORBIDDEN: + case ARM_SMMU_EVT_F_UUT: + default: + event.fault.reason = IOMMU_FAULT_REASON_UNKNOWN; + } + /* only propagate the error if it relates to stage 1 */ + if (s1) + iommu_report_device_fault(master->dev, &event); + + dev_info(smmu->dev, "event 0x%02x received:\n", type); + for (i = 0; i < EVTQ_ENT_DWORDS; ++i) { + dev_info(smmu->dev, "\t0x%016llx\n", + (unsigned long long)evt[i]); + } +} + /* IRQ and event handlers */ static irqreturn_t arm_smmu_evtq_thread(int irq, void *dev) { - int i; struct arm_smmu_device *smmu = dev; struct arm_smmu_queue *q = &smmu->evtq.q; u64 evt[EVTQ_ENT_DWORDS]; do { - while (!queue_remove_raw(q, evt)) { - u8 id = FIELD_GET(EVTQ_0_ID, evt[0]); - - dev_info(smmu->dev, "event 0x%02x received:\n", id); - for (i = 0; i < ARRAY_SIZE(evt); ++i) - dev_info(smmu->dev, "\t0x%016llx\n", - (unsigned long long)evt[i]); - - } + while (!queue_remove_raw(q, evt)) + arm_smmu_report_event(smmu, evt); /* * Not much we can do on overflow, so scream and pretend we're From patchwork Tue Jan 8 10:26:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751811 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8351D1399 for ; Tue, 8 Jan 2019 10:29:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 722EB1FFE6 for ; Tue, 8 Jan 2019 10:29:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 657A827E01; Tue, 8 Jan 2019 10:29:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D195C1FFE6 for ; Tue, 8 Jan 2019 10:29:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728875AbfAHK2y (ORCPT ); Tue, 8 Jan 2019 05:28:54 -0500 Received: from mx1.redhat.com ([209.132.183.28]:39150 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728004AbfAHK2x (ORCPT ); Tue, 8 Jan 2019 05:28:53 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id C5AF87F6CB; Tue, 8 Jan 2019 10:28:52 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id E8C88601A5; Tue, 8 Jan 2019 10:28:39 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 18/21] vfio-pci: Add a new VFIO_REGION_TYPE_NESTED region type Date: Tue, 8 Jan 2019 11:26:30 +0100 Message-Id: <20190108102633.17482-19-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Tue, 08 Jan 2019 10:28:53 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch adds a new 64kB region aiming to report nested mode translation faults. The region contains a header with the size of the queue, the producer and consumer indices and then the actual fault queue data. The producer is updated by the kernel while the consumer is updated by the userspace. Signed-off-by: Eric Auger --- --- drivers/vfio/pci/vfio_pci.c | 102 +++++++++++++++++++++++++++- drivers/vfio/pci/vfio_pci_private.h | 2 + include/uapi/linux/vfio.h | 15 ++++ 3 files changed, 118 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index ff60bd1ea587..2ba181ab2edd 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -56,6 +56,11 @@ module_param(disable_idle_d3, bool, S_IRUGO | S_IWUSR); MODULE_PARM_DESC(disable_idle_d3, "Disable using the PCI D3 low power state for idle, unused devices"); +#define VFIO_FAULT_REGION_SIZE 0x10000 +#define VFIO_FAULT_QUEUE_SIZE \ + ((VFIO_FAULT_REGION_SIZE - sizeof(struct vfio_fault_region_header)) / \ + sizeof(struct iommu_fault)) + static inline bool vfio_vga_disabled(void) { #ifdef CONFIG_VFIO_PCI_VGA @@ -1226,6 +1231,100 @@ static const struct vfio_device_ops vfio_pci_ops = { static int vfio_pci_reflck_attach(struct vfio_pci_device *vdev); static void vfio_pci_reflck_put(struct vfio_pci_reflck *reflck); +static size_t +vfio_pci_dma_fault_rw(struct vfio_pci_device *vdev, char __user *buf, + size_t count, loff_t *ppos, bool iswrite) +{ + unsigned int i = VFIO_PCI_OFFSET_TO_INDEX(*ppos) - VFIO_PCI_NUM_REGIONS; + void *base = vdev->region[i].data; + loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK; + + if (pos >= vdev->region[i].size) + return -EINVAL; + + count = min(count, (size_t)(vdev->region[i].size - pos)); + + if (copy_to_user(buf, base + pos, count)) + return -EFAULT; + + *ppos += count; + + return count; +} + +static int vfio_pci_dma_fault_mmap(struct vfio_pci_device *vdev, + struct vfio_pci_region *region, + struct vm_area_struct *vma) +{ + u64 phys_len, req_len, pgoff, req_start; + unsigned long long addr; + unsigned int index; + + index = vma->vm_pgoff >> (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT); + + if (vma->vm_end < vma->vm_start) + return -EINVAL; + if ((vma->vm_flags & VM_SHARED) == 0) + return -EINVAL; + + phys_len = VFIO_FAULT_REGION_SIZE; + + req_len = vma->vm_end - vma->vm_start; + pgoff = vma->vm_pgoff & + ((1U << (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT)) - 1); + req_start = pgoff << PAGE_SHIFT; + + if (req_start + req_len > phys_len) + return -EINVAL; + + addr = virt_to_phys(vdev->fault_region); + vma->vm_private_data = vdev; + vma->vm_pgoff = (addr >> PAGE_SHIFT) + pgoff; + + return remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff, + req_len, vma->vm_page_prot); +} + +void vfio_pci_dma_fault_release(struct vfio_pci_device *vdev, + struct vfio_pci_region *region) +{ +} + +static const struct vfio_pci_regops vfio_pci_dma_fault_regops = { + .rw = vfio_pci_dma_fault_rw, + .mmap = vfio_pci_dma_fault_mmap, + .release = vfio_pci_dma_fault_release, +}; + +static int vfio_pci_init_dma_fault_region(struct vfio_pci_device *vdev) +{ + u32 flags = VFIO_REGION_INFO_FLAG_READ | VFIO_REGION_INFO_FLAG_WRITE | + VFIO_REGION_INFO_FLAG_MMAP; + int ret; + + spin_lock_init(&vdev->fault_queue_lock); + + vdev->fault_region = kmalloc(VFIO_FAULT_REGION_SIZE, GFP_KERNEL); + if (!vdev->fault_region) + return -ENOMEM; + + ret = vfio_pci_register_dev_region(vdev, + VFIO_REGION_TYPE_NESTED, + VFIO_REGION_SUBTYPE_NESTED_FAULT_REGION, + &vfio_pci_dma_fault_regops, VFIO_FAULT_REGION_SIZE, + flags, vdev->fault_region); + if (ret) { + kfree(vdev->fault_region); + return ret; + } + + vdev->fault_region->header.prod = 0; + vdev->fault_region->header.cons = 0; + vdev->fault_region->header.reserved = 0; + vdev->fault_region->header.size = VFIO_FAULT_QUEUE_SIZE; + return 0; +} + static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) { struct vfio_pci_device *vdev; @@ -1300,7 +1399,7 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) pci_set_power_state(pdev, PCI_D3hot); } - return ret; + return vfio_pci_init_dma_fault_region(vdev); } static void vfio_pci_remove(struct pci_dev *pdev) @@ -1315,6 +1414,7 @@ static void vfio_pci_remove(struct pci_dev *pdev) vfio_iommu_group_put(pdev->dev.iommu_group, &pdev->dev); kfree(vdev->region); + kfree(vdev->fault_region); mutex_destroy(&vdev->ioeventfds_lock); kfree(vdev); diff --git a/drivers/vfio/pci/vfio_pci_private.h b/drivers/vfio/pci/vfio_pci_private.h index 8c0009f00818..38b5d1764a26 100644 --- a/drivers/vfio/pci/vfio_pci_private.h +++ b/drivers/vfio/pci/vfio_pci_private.h @@ -120,6 +120,8 @@ struct vfio_pci_device { int ioeventfds_nr; struct eventfd_ctx *err_trigger; struct eventfd_ctx *req_trigger; + spinlock_t fault_queue_lock; + struct vfio_fault_region *fault_region; struct list_head dummy_resources_list; struct mutex ioeventfds_lock; struct list_head ioeventfds_list; diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 352e795a93c8..b78c2c62af6d 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -307,6 +307,9 @@ struct vfio_region_info_cap_type { #define VFIO_REGION_TYPE_GFX (1) #define VFIO_REGION_SUBTYPE_GFX_EDID (1) +#define VFIO_REGION_TYPE_NESTED (2) +#define VFIO_REGION_SUBTYPE_NESTED_FAULT_REGION (1) + /** * struct vfio_region_gfx_edid - EDID region layout. * @@ -697,6 +700,18 @@ struct vfio_device_ioeventfd { #define VFIO_DEVICE_IOEVENTFD _IO(VFIO_TYPE, VFIO_BASE + 16) +struct vfio_fault_region_header { + __u32 size; /* Read-Only */ + __u32 prod; /* Read-Only */ + __u32 cons; + __u32 reserved; /* must be 0 */ +}; + +struct vfio_fault_region { + struct vfio_fault_region_header header; + struct iommu_fault queue[0]; +}; + /* -------- API for Type1 VFIO IOMMU -------- */ /** From patchwork Tue Jan 8 10:26:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751807 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 53D0213B4 for ; Tue, 8 Jan 2019 10:29:05 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 40EC91FFE6 for ; Tue, 8 Jan 2019 10:29:05 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 33DDE27E01; Tue, 8 Jan 2019 10:29:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C752B1FFE6 for ; Tue, 8 Jan 2019 10:29:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728570AbfAHK27 (ORCPT ); Tue, 8 Jan 2019 05:28:59 -0500 Received: from mx1.redhat.com ([209.132.183.28]:46602 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728920AbfAHK26 (ORCPT ); Tue, 8 Jan 2019 05:28:58 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 96B86637EF; Tue, 8 Jan 2019 10:28:57 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3670B601A4; Tue, 8 Jan 2019 10:28:52 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 19/21] vfio-pci: Register an iommu fault handler Date: Tue, 8 Jan 2019 11:26:31 +0100 Message-Id: <20190108102633.17482-20-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Tue, 08 Jan 2019 10:28:57 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch registers a fault handler which records faults in a circular buffer and then signals an eventfd. This buffer is exposed within the fault region. Signed-off-by: Eric Auger --- drivers/vfio/pci/vfio_pci.c | 44 ++++++++++++++++++++++++++++- drivers/vfio/pci/vfio_pci_private.h | 1 + 2 files changed, 44 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 2ba181ab2edd..f9e2c8292e60 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -29,6 +29,7 @@ #include #include #include +#include #include "vfio_pci_private.h" @@ -1296,6 +1297,44 @@ static const struct vfio_pci_regops vfio_pci_dma_fault_regops = { .release = vfio_pci_dma_fault_release, }; +int vfio_pci_iommu_dev_fault_handler(struct iommu_fault_event *evt, void *data) +{ + struct vfio_pci_device *vdev = (struct vfio_pci_device *) data; + int prod, cons, size; + unsigned long flags; + + spin_lock_irqsave(&vdev->fault_queue_lock, flags); + prod = vdev->fault_region->header.prod; + cons = vdev->fault_region->header.cons; + size = vdev->fault_region->header.size; + + if (cons > VFIO_FAULT_QUEUE_SIZE - 1) + goto unlock; + if (prod > VFIO_FAULT_QUEUE_SIZE - 1) + goto unlock; + if (size != VFIO_FAULT_QUEUE_SIZE) + goto unlock; + if (vdev->fault_region->header.reserved) + goto unlock; + if (CIRC_SPACE(prod, cons, size) < 1) + goto unlock; + + vdev->fault_region->queue[prod] = evt->fault; + prod = (prod + 1) % size; + vdev->fault_region->header.prod = prod; + spin_unlock_irqrestore(&vdev->fault_queue_lock, flags); + + mutex_lock(&vdev->igate); + if (vdev->dma_fault_trigger) + eventfd_signal(vdev->dma_fault_trigger, 1); + mutex_unlock(&vdev->igate); + return 0; + +unlock: + spin_unlock_irqrestore(&vdev->fault_queue_lock, flags); + return -EINVAL; +} + static int vfio_pci_init_dma_fault_region(struct vfio_pci_device *vdev) { u32 flags = VFIO_REGION_INFO_FLAG_READ | VFIO_REGION_INFO_FLAG_WRITE | @@ -1322,7 +1361,9 @@ static int vfio_pci_init_dma_fault_region(struct vfio_pci_device *vdev) vdev->fault_region->header.cons = 0; vdev->fault_region->header.reserved = 0; vdev->fault_region->header.size = VFIO_FAULT_QUEUE_SIZE; - return 0; + return iommu_register_device_fault_handler(&vdev->pdev->dev, + vfio_pci_iommu_dev_fault_handler, + vdev); } static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) @@ -1414,6 +1455,7 @@ static void vfio_pci_remove(struct pci_dev *pdev) vfio_iommu_group_put(pdev->dev.iommu_group, &pdev->dev); kfree(vdev->region); + iommu_unregister_device_fault_handler(&pdev->dev); kfree(vdev->fault_region); mutex_destroy(&vdev->ioeventfds_lock); kfree(vdev); diff --git a/drivers/vfio/pci/vfio_pci_private.h b/drivers/vfio/pci/vfio_pci_private.h index 38b5d1764a26..5936802cbbd0 100644 --- a/drivers/vfio/pci/vfio_pci_private.h +++ b/drivers/vfio/pci/vfio_pci_private.h @@ -120,6 +120,7 @@ struct vfio_pci_device { int ioeventfds_nr; struct eventfd_ctx *err_trigger; struct eventfd_ctx *req_trigger; + struct eventfd_ctx *dma_fault_trigger; spinlock_t fault_queue_lock; struct vfio_fault_region *fault_region; struct list_head dummy_resources_list; From patchwork Tue Jan 8 10:26:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751809 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2A10613B4 for ; Tue, 8 Jan 2019 10:29:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 19E0A1FFE6 for ; Tue, 8 Jan 2019 10:29:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0D5DE27E01; Tue, 8 Jan 2019 10:29:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A99341FFE6 for ; Tue, 8 Jan 2019 10:29:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728971AbfAHK3E (ORCPT ); Tue, 8 Jan 2019 05:29:04 -0500 Received: from mx1.redhat.com ([209.132.183.28]:56716 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728530AbfAHK3D (ORCPT ); Tue, 8 Jan 2019 05:29:03 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 781031555A; Tue, 8 Jan 2019 10:29:02 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id EB210605AE; Tue, 8 Jan 2019 10:28:57 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 20/21] vfio-pci: Add VFIO_PCI_DMA_FAULT_IRQ_INDEX Date: Tue, 8 Jan 2019 11:26:32 +0100 Message-Id: <20190108102633.17482-21-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Tue, 08 Jan 2019 10:29:02 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add a new VFIO_PCI_DMA_FAULT_IRQ_INDEX index. This allows to set/unset an eventfd that will be triggered when DMA translation faults are detected at physical level when the nested mode is used. Signed-off-by: Eric Auger --- drivers/vfio/pci/vfio_pci.c | 3 +++ drivers/vfio/pci/vfio_pci_intrs.c | 19 +++++++++++++++++++ include/uapi/linux/vfio.h | 1 + 3 files changed, 23 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index f9e2c8292e60..66d44736e71d 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -495,6 +495,8 @@ static int vfio_pci_get_irq_count(struct vfio_pci_device *vdev, int irq_type) return 1; } else if (irq_type == VFIO_PCI_REQ_IRQ_INDEX) { return 1; + } else if (irq_type == VFIO_PCI_DMA_FAULT_IRQ_INDEX) { + return 1; } return 0; @@ -822,6 +824,7 @@ static long vfio_pci_ioctl(void *device_data, switch (info.index) { case VFIO_PCI_INTX_IRQ_INDEX ... VFIO_PCI_MSIX_IRQ_INDEX: case VFIO_PCI_REQ_IRQ_INDEX: + case VFIO_PCI_DMA_FAULT_IRQ_INDEX: break; case VFIO_PCI_ERR_IRQ_INDEX: if (pci_is_pcie(vdev->pdev)) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 1c46045b0e7f..28a96117daf3 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -622,6 +622,18 @@ static int vfio_pci_set_req_trigger(struct vfio_pci_device *vdev, count, flags, data); } +static int vfio_pci_set_dma_fault_trigger(struct vfio_pci_device *vdev, + unsigned index, unsigned start, + unsigned count, uint32_t flags, + void *data) +{ + if (index != VFIO_PCI_DMA_FAULT_IRQ_INDEX || start != 0 || count > 1) + return -EINVAL; + + return vfio_pci_set_ctx_trigger_single(&vdev->dma_fault_trigger, + count, flags, data); +} + int vfio_pci_set_irqs_ioctl(struct vfio_pci_device *vdev, uint32_t flags, unsigned index, unsigned start, unsigned count, void *data) @@ -671,6 +683,13 @@ int vfio_pci_set_irqs_ioctl(struct vfio_pci_device *vdev, uint32_t flags, break; } break; + case VFIO_PCI_DMA_FAULT_IRQ_INDEX: + switch (flags & VFIO_IRQ_SET_ACTION_TYPE_MASK) { + case VFIO_IRQ_SET_ACTION_TRIGGER: + func = vfio_pci_set_dma_fault_trigger; + break; + } + break; } if (!func) diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index b78c2c62af6d..47b65ef9d448 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -554,6 +554,7 @@ enum { VFIO_PCI_MSIX_IRQ_INDEX, VFIO_PCI_ERR_IRQ_INDEX, VFIO_PCI_REQ_IRQ_INDEX, + VFIO_PCI_DMA_FAULT_IRQ_INDEX, VFIO_PCI_NUM_IRQS }; From patchwork Tue Jan 8 10:26:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Auger X-Patchwork-Id: 10751813 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2F47613B4 for ; Tue, 8 Jan 2019 10:29:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1CC9328B90 for ; Tue, 8 Jan 2019 10:29:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 10F7528BDB; Tue, 8 Jan 2019 10:29:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9BA3328B90 for ; Tue, 8 Jan 2019 10:29:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728412AbfAHK3Q (ORCPT ); Tue, 8 Jan 2019 05:29:16 -0500 Received: from mx1.redhat.com ([209.132.183.28]:56226 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727614AbfAHK3Q (ORCPT ); Tue, 8 Jan 2019 05:29:16 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 537933F731; Tue, 8 Jan 2019 10:29:15 +0000 (UTC) Received: from laptop.redhat.com (ovpn-116-249.ams2.redhat.com [10.36.116.249]) by smtp.corp.redhat.com (Postfix) with ESMTP id DF6336013A; Tue, 8 Jan 2019 10:29:02 +0000 (UTC) From: Eric Auger To: eric.auger.pro@gmail.com, eric.auger@redhat.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, joro@8bytes.org, alex.williamson@redhat.com, jacob.jun.pan@linux.intel.com, yi.l.liu@linux.intel.com, jean-philippe.brucker@arm.com, will.deacon@arm.com, robin.murphy@arm.com Cc: kevin.tian@intel.com, ashok.raj@intel.com, marc.zyngier@arm.com, christoffer.dall@arm.com, peter.maydell@linaro.org Subject: [RFC v3 21/21] vfio: Document nested stage control Date: Tue, 8 Jan 2019 11:26:33 +0100 Message-Id: <20190108102633.17482-22-eric.auger@redhat.com> In-Reply-To: <20190108102633.17482-1-eric.auger@redhat.com> References: <20190108102633.17482-1-eric.auger@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Tue, 08 Jan 2019 10:29:15 +0000 (UTC) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP New iotcls were introduced to pass information about guest stage1 to the host through VFIO. Let's document the nested stage control. Signed-off-by: Eric Auger --- v2 -> v3: - document the new fault API v1 -> v2: - use the new ioctl names - add doc related to fault handling --- Documentation/vfio.txt | 62 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) diff --git a/Documentation/vfio.txt b/Documentation/vfio.txt index f1a4d3c3ba0b..620e38ed0c4a 100644 --- a/Documentation/vfio.txt +++ b/Documentation/vfio.txt @@ -239,6 +239,68 @@ group and can access them as follows:: /* Gratuitous device reset and go... */ ioctl(device, VFIO_DEVICE_RESET); +IOMMU Dual Stage Control +------------------------ + +Some IOMMUs support 2 stages/levels of translation. "Stage" corresponds to +the ARM terminology while "level" corresponds to Intel's VTD terminology. In +the following text we use either without distinction. + +This is useful when the guest is exposed with a virtual IOMMU and some +devices are assigned to the guest through VFIO. Then the guest OS can use +stage 1 (IOVA -> GPA), while the hypervisor uses stage 2 for VM isolation +(GPA -> HPA). + +The guest gets ownership of the stage 1 page tables and also owns stage 1 +configuration structures. The hypervisor owns the root configuration structure +(for security reason), including stage 2 configuration. This works as long +configuration structures and page table format are compatible between the +virtual IOMMU and the physical IOMMU. + +Assuming the HW supports it, this nested mode is selected by choosing the +VFIO_TYPE1_NESTING_IOMMU type through: + +ioctl(container, VFIO_SET_IOMMU, VFIO_TYPE1_NESTING_IOMMU); + +This forces the hypervisor to use the stage 2, leaving stage 1 available for +guest usage. + +Once groups are attached to the container, the guest stage 1 translation +configuration data can be passed to VFIO by using + +ioctl(container, VFIO_IOMMU_BIND_PASID_TABLE, &pasid_table_info); + +This allows to combine guest stage 1 configuration structure along with +hypervisor stage 2 configuration structure. stage 1 configuration structures +are dependent on the IOMMU type. + +As the stage 1 translation is fully delegated to the HW, physical events that +may occur (especially translation faults), need to be propagated up to +the virtualizer and re-injected into the guest. + +By using VFIO_DEVICE_SET_IRQS along with the VFIO_PCI_DMA_FAULT_IRQ_INDEX +index, the virtualizer can register an eventfd signalled whenever a +fault is observed at physical level. The actual faults can be retrieved +from the device fault region whose type/subtype is: +VFIO_REGION_TYPE_NESTED/VFIO_REGION_SUBTYPE_NESTED_FAULT_REGION. + +This region can be mmapped. When a fault is consumed, the user must increment +the consumer index. + +When the guest invalidates stage 1 related caches, invalidations must be +forwarded to the host through +ioctl(container, VFIO_IOMMU_CACHE_INVALIDATE, &inv_data); +Those invalidations can happen at various granularity levels, page, context, ... + +The ARM SMMU specification introduces another challenge: MSIs are translated by +both the virtual SMMU and the physical SMMU. To build a nested mapping for the +IOVA programmed into the assigned device, the guest needs to pass its IOVA/MSI +doorbell GPA binding to the host. Then the hypervisor can build a nested stage 2 +binding eventually translating into the physical MSI doorbell. + +This is achieved by +ioctl(container, VFIO_IOMMU_BIND_MSI, &guest_binding); + VFIO User API -------------------------------------------------------------------------------