From patchwork Thu May 18 20:46:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247356 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6E20C77B73 for ; Thu, 18 May 2023 20:47:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229977AbjERUrs (ORCPT ); Thu, 18 May 2023 16:47:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55728 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229574AbjERUrq (ORCPT ); Thu, 18 May 2023 16:47:46 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 47F78F7 for ; Thu, 18 May 2023 13:47:45 -0700 (PDT) Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIx6iJ025284; Thu, 18 May 2023 20:47:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=lmBX3d6EEqid/HWfvIQqyMVUgRO0Wer5JoCL94qDets=; b=qA27Ed6TF7M4KQu1JB/enySS1cHu83hGHvO469u8Em4h0xp5F5DwbMaN4ETx0B+hWF0d uwWobPyMGVd4GNEM3DdLwhb/TKnrx6t8Y9Ft8yfOUJgHCBLDPxaQkYXv7oEtBQP29g34 LouvDc2fZcBCz8SoEH+NJNZx9XxdvIOIVo0TCfkwZ2sbXBSTipnprj6ozTdK2NjZHHuo ZDtoqDiOkzibnQRd1qoKl9915Dit3fo3Ey10dOMeRdRY2XTV3nsACmzE48sUcdRWW3XY RQ208itkEi6pJ/lAwBtqRaiKpTrGFJtboQnJtrmyCpdwWwpE6fZLALPVFtZlyXzpXMZ+ Zw== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qj1fc97c4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:47:20 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJ3Pn5032141; Thu, 18 May 2023 20:47:19 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10dae7g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:47:19 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE2n033533; Thu, 18 May 2023 20:47:18 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-2; Thu, 18 May 2023 20:47:18 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 01/24] iommu: Add RCU-protected page free support Date: Thu, 18 May 2023 21:46:27 +0100 Message-Id: <20230518204650.14541-2-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180170 X-Proofpoint-ORIG-GUID: dDDr2URMhFRURT5eRyrAX1r_Ov3r1IFZ X-Proofpoint-GUID: dDDr2URMhFRURT5eRyrAX1r_Ov3r1IFZ Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Lu Baolu The IOMMU page tables are updated using iommu_map/unmap() interfaces. Currently, there is no mandatory requirement for drivers to use locks to ensure concurrent updates to page tables, because it's assumed that overlapping IOVA ranges do not have concurrent updates. Therefore the IOMMU drivers only need to take care of concurrent updates to level page table entries. But enabling new features challenges this assumption. For example, the hardware assisted dirty page tracking feature requires scanning page tables in interfaces other than mapping and unmapping. This might result in a use-after-free scenario in which a level page table has been freed by the unmap() interface, while another thread is scanning the next level page table. This adds RCU-protected page free support so that the pages are really freed and reused after a RCU grace period. Hence, the page tables are safe for scanning within a rcu_read_lock critical region. Considering that scanning the page table is a rare case, this also adds a domain flag and the RCU-protected page free is only used when this flat is set. Signed-off-by: Lu Baolu --- drivers/iommu/iommu.c | 23 +++++++++++++++++++++++ include/linux/iommu.h | 10 ++++++++++ 2 files changed, 33 insertions(+) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 91573efd9488..2088caae5074 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -3432,3 +3432,26 @@ struct iommu_domain *iommu_sva_domain_alloc(struct device *dev, return domain; } + +static void pgtble_page_free_rcu(struct rcu_head *rcu) +{ + struct page *page = container_of(rcu, struct page, rcu_head); + + __free_pages(page, 0); +} + +void iommu_free_pgtbl_pages(struct iommu_domain *domain, + struct list_head *pages) +{ + struct page *page, *next; + + if (!domain->concurrent_traversal) { + put_pages_list(pages); + return; + } + + list_for_each_entry_safe(page, next, pages, lru) { + list_del(&page->lru); + call_rcu(&page->rcu_head, pgtble_page_free_rcu); + } +} diff --git a/include/linux/iommu.h b/include/linux/iommu.h index e8c9a7da1060..39d25645a5ab 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -110,6 +110,7 @@ struct iommu_domain { int users; }; }; + unsigned long concurrent_traversal:1; }; static inline bool iommu_is_dma_domain(struct iommu_domain *domain) @@ -697,6 +698,12 @@ static inline void dev_iommu_priv_set(struct device *dev, void *priv) dev->iommu->priv = priv; } +static inline void domain_set_concurrent_traversal(struct iommu_domain *domain, + bool value) +{ + domain->concurrent_traversal = value; +} + int iommu_probe_device(struct device *dev); int iommu_dev_enable_feature(struct device *dev, enum iommu_dev_features f); @@ -721,6 +728,9 @@ void iommu_detach_device_pasid(struct iommu_domain *domain, struct iommu_domain * iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid, unsigned int type); + +void iommu_free_pgtbl_pages(struct iommu_domain *domain, + struct list_head *pages); #else /* CONFIG_IOMMU_API */ struct iommu_ops {}; From patchwork Thu May 18 20:46:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247357 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66D4AC77B7A for ; Thu, 18 May 2023 20:47:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230091AbjERUrt (ORCPT ); Thu, 18 May 2023 16:47:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55730 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229799AbjERUrq (ORCPT ); Thu, 18 May 2023 16:47:46 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F13ACEE for ; Thu, 18 May 2023 13:47:45 -0700 (PDT) Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIxAv7015429; Thu, 18 May 2023 20:47:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=Hs/FL0pNDu6fa17nEH3p23V6JtyjGWxlEHZV3Uz4db4=; b=uNyRKD9PnPQ6HpOvxUMSVqaTpiYTmjcXl8jqWJq3Yec+p3n3wuGZU49BmW/cW421PYaR OY1o4KumwMiReyjG43d5okYhTP/Qn6TzId8//It5i61KgDTXC4BitlyF8PhyHysV7NCG OeZ/0dtxnytYXTeNB6vcCwK7g4QGtnWLxrGyiB3yqV2KRjRJOxNhvz+KFu79yLx7pgp5 oHA4BYytW7bvvBAdZifB6vKZ2M5193CmUzI+LP89QjUXGHqrZCXbm18HBmun2zNfQMGY 2TXY5MHFzZJVlQedSxmoWTu3KMvOYkvq9lPfTK0EGj2pEelpQFxCBzSfPLtOSSiYHMo9 ug== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qmx8j3n3b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:47:25 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJISCN032224; Thu, 18 May 2023 20:47:24 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10dae96-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:47:24 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE2p033533; Thu, 18 May 2023 20:47:23 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-3; Thu, 18 May 2023 20:47:22 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 02/24] iommu: Replace put_pages_list() with iommu_free_pgtbl_pages() Date: Thu, 18 May 2023 21:46:28 +0100 Message-Id: <20230518204650.14541-3-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180170 X-Proofpoint-GUID: _DS4keRxAQIrfPtqBNEIZsJ7yZroPJ7f X-Proofpoint-ORIG-GUID: _DS4keRxAQIrfPtqBNEIZsJ7yZroPJ7f Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Lu Baolu Therefore, RCU protected page free will take effect if necessary. Signed-off-by: Lu Baolu --- drivers/iommu/amd/io_pgtable.c | 5 ++--- drivers/iommu/dma-iommu.c | 6 ++++-- drivers/iommu/intel/iommu.c | 4 ++-- 3 files changed, 8 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/amd/io_pgtable.c b/drivers/iommu/amd/io_pgtable.c index 1b67116882be..666b643106f8 100644 --- a/drivers/iommu/amd/io_pgtable.c +++ b/drivers/iommu/amd/io_pgtable.c @@ -430,7 +430,7 @@ static int iommu_v1_map_pages(struct io_pgtable_ops *ops, unsigned long iova, } /* Everything flushed out, free pages now */ - put_pages_list(&freelist); + iommu_free_pgtbl_pages(&dom->domain, &freelist); return ret; } @@ -511,8 +511,7 @@ static void v1_free_pgtable(struct io_pgtable *iop) /* Make changes visible to IOMMUs */ amd_iommu_domain_update(dom); - - put_pages_list(&freelist); + iommu_free_pgtbl_pages(&dom->domain, &freelist); } static struct io_pgtable *v1_alloc_pgtable(struct io_pgtable_cfg *cfg, void *cookie) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 7a9f0b0bddbd..33925b9249b3 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -136,7 +136,8 @@ static void fq_ring_free(struct iommu_dma_cookie *cookie, struct iova_fq *fq) if (fq->entries[idx].counter >= counter) break; - put_pages_list(&fq->entries[idx].freelist); + iommu_free_pgtbl_pages(cookie->fq_domain, + &fq->entries[idx].freelist); free_iova_fast(&cookie->iovad, fq->entries[idx].iova_pfn, fq->entries[idx].pages); @@ -232,7 +233,8 @@ static void iommu_dma_free_fq(struct iommu_dma_cookie *cookie) struct iova_fq *fq = per_cpu_ptr(cookie->fq, cpu); fq_ring_for_each(idx, fq) - put_pages_list(&fq->entries[idx].freelist); + iommu_free_pgtbl_pages(cookie->fq_domain, + &fq->entries[idx].freelist); } free_percpu(cookie->fq); diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index b871a6afd803..4662292d60ba 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -1827,7 +1827,7 @@ static void domain_exit(struct dmar_domain *domain) LIST_HEAD(freelist); domain_unmap(domain, 0, DOMAIN_MAX_PFN(domain->gaw), &freelist); - put_pages_list(&freelist); + iommu_free_pgtbl_pages(&domain->domain, &freelist); } if (WARN_ON(!list_empty(&domain->devices))) @@ -4286,7 +4286,7 @@ static void intel_iommu_tlb_sync(struct iommu_domain *domain, start_pfn, nrpages, list_empty(&gather->freelist), 0); - put_pages_list(&gather->freelist); + iommu_free_pgtbl_pages(domain, &gather->freelist); } static phys_addr_t intel_iommu_iova_to_phys(struct iommu_domain *domain, From patchwork Thu May 18 20:46:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247360 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1ECCFC77B7A for ; Thu, 18 May 2023 20:48:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230305AbjERUsF (ORCPT ); Thu, 18 May 2023 16:48:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229799AbjERUsD (ORCPT ); Thu, 18 May 2023 16:48:03 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B075D10CF for ; Thu, 18 May 2023 13:48:00 -0700 (PDT) Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIx5Gp025230; Thu, 18 May 2023 20:47:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=hnzbDzcuADQdXdX6l91Ma65MoXvbmxZbCuNBaDlEMLw=; b=dXr3iIexO3u2FVTW26XMXOk6lf0oxGWABRN0AJFiM/YpIDQ5FK7jBnhXUJje0ENN5OhC eQF0UqQ+tM0OySoP/hY7gASvwT9g1chBXUcKet6O22WPvquXnW4HWtxD3XSoHE3AdhqN ggCl+jvR15mIZtozYz6LktpO9BzOyx12/GqLC4e/CYPzRcRaUKhqWU5zgKnxo3I+Z7jg 5/MtMwoiP42jtgsXyMwUsoMaWv2LkjoKeceBsvPHD/v7NMrcN+QX558qcVdLWc6yMKp+ gKeS6ItFSxT8U21xQzB5ymZE62Q01Oqw2O/c1C6Bvz3NRtjGgRKkc2QbqTeQDMA/P4Ug /Q== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qj1fc97ch-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:47:29 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJFRxA032132; Thu, 18 May 2023 20:47:28 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10daeah-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:47:28 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE2r033533; Thu, 18 May 2023 20:47:27 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-4; Thu, 18 May 2023 20:47:27 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 03/24] vfio: Move iova_bitmap into iommu core Date: Thu, 18 May 2023 21:46:29 +0100 Message-Id: <20230518204650.14541-4-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=715 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180170 X-Proofpoint-ORIG-GUID: ebGehFK4xMnLwN1Mf4khudJ4ibTWxQUP X-Proofpoint-GUID: ebGehFK4xMnLwN1Mf4khudJ4ibTWxQUP Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Both VFIO and IOMMUFD will need iova bitmap for storing dirties and walking the user bitmaps, so move to the common dependency into IOMMU core. IOMMUFD can't exactly host it given that VFIO dirty tracking can be used without IOMMUFD. Signed-off-by: Joao Martins --- drivers/iommu/Makefile | 1 + drivers/{vfio => iommu}/iova_bitmap.c | 0 drivers/vfio/Makefile | 3 +-- 3 files changed, 2 insertions(+), 2 deletions(-) rename drivers/{vfio => iommu}/iova_bitmap.c (100%) diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile index 769e43d780ce..9d9dfbd2dfc2 100644 --- a/drivers/iommu/Makefile +++ b/drivers/iommu/Makefile @@ -10,6 +10,7 @@ obj-$(CONFIG_IOMMU_IO_PGTABLE_ARMV7S) += io-pgtable-arm-v7s.o obj-$(CONFIG_IOMMU_IO_PGTABLE_LPAE) += io-pgtable-arm.o obj-$(CONFIG_IOMMU_IO_PGTABLE_DART) += io-pgtable-dart.o obj-$(CONFIG_IOMMU_IOVA) += iova.o +obj-$(CONFIG_IOMMU_IOVA) += iova_bitmap.o obj-$(CONFIG_OF_IOMMU) += of_iommu.o obj-$(CONFIG_MSM_IOMMU) += msm_iommu.o obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o diff --git a/drivers/vfio/iova_bitmap.c b/drivers/iommu/iova_bitmap.c similarity index 100% rename from drivers/vfio/iova_bitmap.c rename to drivers/iommu/iova_bitmap.c diff --git a/drivers/vfio/Makefile b/drivers/vfio/Makefile index 57c3515af606..f9cc32a9810c 100644 --- a/drivers/vfio/Makefile +++ b/drivers/vfio/Makefile @@ -1,8 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_VFIO) += vfio.o -vfio-y += vfio_main.o \ - iova_bitmap.o +vfio-y += vfio_main.o vfio-$(CONFIG_VFIO_DEVICE_CDEV) += device_cdev.o vfio-$(CONFIG_VFIO_GROUP) += group.o vfio-$(CONFIG_IOMMUFD) += iommufd.o From patchwork Thu May 18 20:46:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247359 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B3ABC77B7A for ; Thu, 18 May 2023 20:48:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230169AbjERUr7 (ORCPT ); Thu, 18 May 2023 16:47:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55920 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230122AbjERUr6 (ORCPT ); Thu, 18 May 2023 16:47:58 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8FBBA121 for ; Thu, 18 May 2023 13:47:56 -0700 (PDT) Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIx7f8032103; Thu, 18 May 2023 20:47:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=glDG6s0ji8MaAa65dGHIt3BUdaE70T22AYrJcvt0ijo=; b=jfyk2w7a2vs2oYpU3NJKXmNBtljwu4APN55l/7V2zd4EMQZKpUp1YawVFdd7QGMP7T5l CAVhfVw+QYQT01HirXBJsD65P3dvjUpj5jK8G+MyIn4AMxf9A9TS2F0oe5/dv96oEQdq Gsy6j4IKy7mX/xy+CrAHKMYB/AG9jWkX9KjXRrn6p4/xSGbBu4u9uXOfEl+bNFcW6Cfr cxG0oSqnJMFtc2ivTIWiCg9nE4kmgSZYSr7WIHr/F7mkYlSyaxkY+HQwNMJv5pmcG2aO UBPxtNHs1GbA8RW25uIfQeesPlui/cLm++WPULpMSHNuftg40Pnyb+a49EDGbgbuGTxH DQ== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qj33v0v9h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:47:33 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IKXqw9032130; Thu, 18 May 2023 20:47:32 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10daec7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:47:32 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE2t033533; Thu, 18 May 2023 20:47:31 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-5; Thu, 18 May 2023 20:47:31 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 04/24] iommu: Add iommu_domain ops for dirty tracking Date: Thu, 18 May 2023 21:46:30 +0100 Message-Id: <20230518204650.14541-5-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180170 X-Proofpoint-GUID: vxAUhsfdm4GWue53sJs6RVElhtq324AN X-Proofpoint-ORIG-GUID: vxAUhsfdm4GWue53sJs6RVElhtq324AN Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add to iommu domain operations a set of callbacks to perform dirty tracking, particulary to start and stop tracking and finally to read and clear the dirty data. Drivers are generally expected to dynamically change its translation structures to toggle the tracking and flush some form of control state structure that stands in the IOVA translation path. Though it's not mandatory, as drivers will be enable dirty tracking at boot, and just flush the IO pagetables when setting dirty tracking. For each of the newly added IOMMU core APIs: .supported_flags[IOMMU_DOMAIN_F_ENFORCE_DIRTY]: Introduce a set of flags that enforce certain restrictions in the iommu_domain object. For dirty tracking this means that when IOMMU_DOMAIN_F_ENFORCE_DIRTY is set via its helper iommu_domain_set_flags(...) devices attached via attach_dev will fail on devices that do *not* have dirty tracking supported. IOMMU drivers that support dirty tracking should advertise this flag, while enforcing that dirty tracking is supported by the device in its .attach_dev iommu op. iommu_cap::IOMMU_CAP_DIRTY: new device iommu_capable value when probing for capabilities of the device. .set_dirty_tracking(): an iommu driver is expected to change its translation structures and enable dirty tracking for the devices in the iommu_domain. For drivers making dirty tracking always-enabled, it should just return 0. .read_and_clear_dirty(): an iommu driver is expected to walk the iova range passed in and use iommu_dirty_bitmap_record() to record dirty info per IOVA. When detecting a given IOVA is dirty it should also clear its dirty state from the PTE, *unless* the flag IOMMU_DIRTY_NO_CLEAR is passed in -- flushing is steered from the caller of the domain_op via iotlb_gather. The iommu core APIs use the same data structure in use for dirty tracking for VFIO device dirty (struct iova_bitmap) abstracted by iommu_dirty_bitmap_record() helper function. Signed-off-by: Joao Martins --- drivers/iommu/iommu.c | 11 +++++++ include/linux/io-pgtable.h | 4 +++ include/linux/iommu.h | 67 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 82 insertions(+) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 2088caae5074..95acc543e8fb 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -2013,6 +2013,17 @@ struct iommu_domain *iommu_domain_alloc(const struct bus_type *bus) } EXPORT_SYMBOL_GPL(iommu_domain_alloc); +int iommu_domain_set_flags(struct iommu_domain *domain, + const struct bus_type *bus, unsigned long val) +{ + if (!(val & bus->iommu_ops->supported_flags)) + return -EINVAL; + + domain->flags |= val; + return 0; +} +EXPORT_SYMBOL_GPL(iommu_domain_set_flags); + void iommu_domain_free(struct iommu_domain *domain) { if (domain->type == IOMMU_DOMAIN_SVA) diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h index 1b7a44b35616..25142a0e2fc2 100644 --- a/include/linux/io-pgtable.h +++ b/include/linux/io-pgtable.h @@ -166,6 +166,10 @@ struct io_pgtable_ops { struct iommu_iotlb_gather *gather); phys_addr_t (*iova_to_phys)(struct io_pgtable_ops *ops, unsigned long iova); + int (*read_and_clear_dirty)(struct io_pgtable_ops *ops, + unsigned long iova, size_t size, + unsigned long flags, + struct iommu_dirty_bitmap *dirty); }; /** diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 39d25645a5ab..992ea87f2f8e 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -13,6 +13,7 @@ #include #include #include +#include #include #define IOMMU_READ (1 << 0) @@ -65,6 +66,11 @@ struct iommu_domain_geometry { #define __IOMMU_DOMAIN_SVA (1U << 4) /* Shared process address space */ +/* Domain feature flags that do not define domain types */ +#define IOMMU_DOMAIN_F_ENFORCE_DIRTY (1U << 6) /* Enforce attachment of + dirty tracking supported + devices */ + /* * This are the possible domain-types * @@ -93,6 +99,7 @@ struct iommu_domain_geometry { struct iommu_domain { unsigned type; + unsigned flags; const struct iommu_domain_ops *ops; unsigned long pgsize_bitmap; /* Bitmap of page sizes in use */ struct iommu_domain_geometry geometry; @@ -128,6 +135,7 @@ enum iommu_cap { * this device. */ IOMMU_CAP_ENFORCE_CACHE_COHERENCY, + IOMMU_CAP_DIRTY, /* IOMMU supports dirty tracking */ }; /* These are the possible reserved region types */ @@ -220,6 +228,17 @@ struct iommu_iotlb_gather { bool queued; }; +/** + * struct iommu_dirty_bitmap - Dirty IOVA bitmap state + * + * @bitmap: IOVA bitmap + * @gather: Range information for a pending IOTLB flush + */ +struct iommu_dirty_bitmap { + struct iova_bitmap *bitmap; + struct iommu_iotlb_gather *gather; +}; + /** * struct iommu_ops - iommu ops and capabilities * @capable: check capability @@ -248,6 +267,7 @@ struct iommu_iotlb_gather { * pasid, so that any DMA transactions with this pasid * will be blocked by the hardware. * @pgsize_bitmap: bitmap of all possible supported page sizes + * @flags: All non domain type supported features * @owner: Driver module providing these ops */ struct iommu_ops { @@ -281,6 +301,7 @@ struct iommu_ops { const struct iommu_domain_ops *default_domain_ops; unsigned long pgsize_bitmap; + unsigned long supported_flags; struct module *owner; }; @@ -316,6 +337,11 @@ struct iommu_ops { * @enable_nesting: Enable nesting * @set_pgtable_quirks: Set io page table quirks (IO_PGTABLE_QUIRK_*) * @free: Release the domain after use. + * @set_dirty_tracking: Enable or Disable dirty tracking on the iommu domain + * @read_and_clear_dirty: Walk IOMMU page tables for dirtied PTEs marshalled + * into a bitmap, with a bit represented as a page. + * Reads the dirty PTE bits and clears it from IO + * pagetables. */ struct iommu_domain_ops { int (*attach_dev)(struct iommu_domain *domain, struct device *dev); @@ -348,6 +374,12 @@ struct iommu_domain_ops { unsigned long quirks); void (*free)(struct iommu_domain *domain); + + int (*set_dirty_tracking)(struct iommu_domain *domain, bool enabled); + int (*read_and_clear_dirty)(struct iommu_domain *domain, + unsigned long iova, size_t size, + unsigned long flags, + struct iommu_dirty_bitmap *dirty); }; /** @@ -461,6 +493,9 @@ extern bool iommu_present(const struct bus_type *bus); extern bool device_iommu_capable(struct device *dev, enum iommu_cap cap); extern bool iommu_group_has_isolated_msi(struct iommu_group *group); extern struct iommu_domain *iommu_domain_alloc(const struct bus_type *bus); +extern int iommu_domain_set_flags(struct iommu_domain *domain, + const struct bus_type *bus, + unsigned long flags); extern void iommu_domain_free(struct iommu_domain *domain); extern int iommu_attach_device(struct iommu_domain *domain, struct device *dev); @@ -627,6 +662,28 @@ static inline bool iommu_iotlb_gather_queued(struct iommu_iotlb_gather *gather) return gather && gather->queued; } +static inline void iommu_dirty_bitmap_init(struct iommu_dirty_bitmap *dirty, + struct iova_bitmap *bitmap, + struct iommu_iotlb_gather *gather) +{ + if (gather) + iommu_iotlb_gather_init(gather); + + dirty->bitmap = bitmap; + dirty->gather = gather; +} + +static inline void +iommu_dirty_bitmap_record(struct iommu_dirty_bitmap *dirty, unsigned long iova, + unsigned long length) +{ + if (dirty->bitmap) + iova_bitmap_set(dirty->bitmap, iova, length); + + if (dirty->gather) + iommu_iotlb_gather_add_range(dirty->gather, iova, length); +} + /* PCI device grouping function */ extern struct iommu_group *pci_device_group(struct device *dev); /* Generic device grouping function */ @@ -657,6 +714,9 @@ struct iommu_fwspec { /* ATS is supported */ #define IOMMU_FWSPEC_PCI_RC_ATS (1 << 0) +/* Read but do not clear any dirty bits */ +#define IOMMU_DIRTY_NO_CLEAR (1 << 0) + /** * struct iommu_sva - handle to a device-mm bond */ @@ -755,6 +815,13 @@ static inline struct iommu_domain *iommu_domain_alloc(const struct bus_type *bus return NULL; } +static inline int iommu_domain_set_flags(struct iommu_domain *domain, + const struct bus_type *bus, + unsigned long flags) +{ + return -ENODEV; +} + static inline void iommu_domain_free(struct iommu_domain *domain) { } From patchwork Thu May 18 20:46:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247361 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18FF7C77B73 for ; Thu, 18 May 2023 20:48:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229635AbjERUsI (ORCPT ); Thu, 18 May 2023 16:48:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56118 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230144AbjERUsG (ORCPT ); Thu, 18 May 2023 16:48:06 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 831AB189 for ; Thu, 18 May 2023 13:48:01 -0700 (PDT) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIxeiR012540; Thu, 18 May 2023 20:47:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=QcIsCoHtQ7tdFGqnxoNBNnKAFVBbCL6Lhv33lo+g6fU=; b=nIzaPt4rqGDzI6yOUWmN+fTE2/eiW2EiHpXUuJ3aQuOOCMCNCpdWBcmrrDcop2IkcPAO GHewE9Yagz8nf8i1Qk1BEopWRC/61evLxV5kaDidSJl7BJ+l85zwdmhWKF6UDeI/EMDo KgAtOGWpUMEv3scOn9dW6I4DrRaoO7fitUGvxNanvW41h/t/oLTN1csVGsZezwf7+cqY pv0gAoCESyfzWPQAQdtNaD0a9QQmbTDEjPrAr7HgKmYgu5LDwBQNgv5IYXAc9J/2MJJf ukFsoI0Y6Blckedlg/ZinJygeRaj+/vfNlc2W/LvoptEMbaY9ZL/1/ewWKTnFn3h2rVo dg== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qmxfc3jx4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:47:37 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJ2W5a032112; Thu, 18 May 2023 20:47:36 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10daedy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:47:36 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE2v033533; Thu, 18 May 2023 20:47:35 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-6; Thu, 18 May 2023 20:47:35 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 05/24] iommufd: Add a flag to enforce dirty tracking on attach Date: Thu, 18 May 2023 21:46:31 +0100 Message-Id: <20230518204650.14541-6-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180170 X-Proofpoint-GUID: -JeeAC5ISuAFikfuevNFnqcc00JsaYFW X-Proofpoint-ORIG-GUID: -JeeAC5ISuAFikfuevNFnqcc00JsaYFW Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Throughout IOMMU domain lifetime that wants to use dirty tracking, some guarantees are needed such that any device attached to the iommu_domain supports dirty tracking. The idea is to handle a case where IOMMUs are assymetric feature-wise and thus the capability may not be advertised for all devices. This is done by adding a flag into HWPT_ALLOC namely: IOMMUFD_HWPT_ALLOC_ENFORCE_DIRTY .. Passed in HWPT_ALLOC ioctl flags. The enforcement is done by creating a iommu_domain and setting the associated flags (via iommu_domain_set_flags) cross-checking with IOMMU driver advertised flags (and failing if it's not advertised). Advertising the new IOMMU domain feature flag requires that the individual iommu driver capability is supported when we attach a new device to the iommu_domain or otherwise fail the attachment if the capability is not set in the device. Userspace will have also the option of checking which that dirty tracking is supported in the IOMMU behind the device. Link: https://lore.kernel.org/kvm/20220721142421.GB4609@nvidia.com/ Signed-off-by: Joao Martins --- drivers/iommu/iommufd/device.c | 2 +- drivers/iommu/iommufd/hw_pagetable.c | 29 ++++++++++++++++++++++--- drivers/iommu/iommufd/iommufd_private.h | 4 +++- include/uapi/linux/iommufd.h | 9 ++++++++ 4 files changed, 39 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/iommufd/device.c b/drivers/iommu/iommufd/device.c index 989bd485f92f..48d1300f0350 100644 --- a/drivers/iommu/iommufd/device.c +++ b/drivers/iommu/iommufd/device.c @@ -511,7 +511,7 @@ iommufd_device_auto_get_domain(struct iommufd_device *idev, } hwpt = iommufd_hw_pagetable_alloc(idev->ictx, ioas, idev, - immediate_attach); + immediate_attach, false); if (IS_ERR(hwpt)) { destroy_hwpt = ERR_CAST(hwpt); goto out_unlock; diff --git a/drivers/iommu/iommufd/hw_pagetable.c b/drivers/iommu/iommufd/hw_pagetable.c index cf2c1504e20d..4f0b72737ae2 100644 --- a/drivers/iommu/iommufd/hw_pagetable.c +++ b/drivers/iommu/iommufd/hw_pagetable.c @@ -55,12 +55,26 @@ int iommufd_hw_pagetable_enforce_cc(struct iommufd_hw_pagetable *hwpt) return 0; } +int iommufd_hw_pagetable_enforce_dirty(struct iommufd_hw_pagetable *hwpt, + struct iommufd_device *idev) +{ + hwpt->enforce_dirty = + !iommu_domain_set_flags(hwpt->domain, idev->dev->bus, + IOMMU_DOMAIN_F_ENFORCE_DIRTY); + if (!hwpt->enforce_dirty) + return -EINVAL; + + return 0; +} + /** * iommufd_hw_pagetable_alloc() - Get an iommu_domain for a device * @ictx: iommufd context * @ioas: IOAS to associate the domain with * @idev: Device to get an iommu_domain for * @immediate_attach: True if idev should be attached to the hwpt + * @enforce_dirty: True if dirty tracking support should be enforce + * on device attach * * Allocate a new iommu_domain and return it as a hw_pagetable. The HWPT * will be linked to the given ioas and upon return the underlying iommu_domain @@ -72,7 +86,8 @@ int iommufd_hw_pagetable_enforce_cc(struct iommufd_hw_pagetable *hwpt) */ struct iommufd_hw_pagetable * iommufd_hw_pagetable_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas, - struct iommufd_device *idev, bool immediate_attach) + struct iommufd_device *idev, bool immediate_attach, + bool enforce_dirty) { struct iommufd_hw_pagetable *hwpt; int rc; @@ -107,6 +122,12 @@ iommufd_hw_pagetable_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas, goto out_abort; } + if (enforce_dirty) { + rc = iommufd_hw_pagetable_enforce_dirty(hwpt, idev); + if (rc) + goto out_abort; + } + /* * immediate_attach exists only to accommodate iommu drivers that cannot * directly allocate a domain. These drivers do not finish creating the @@ -141,7 +162,8 @@ int iommufd_hwpt_alloc(struct iommufd_ucmd *ucmd) struct iommufd_ioas *ioas; int rc; - if (cmd->flags || cmd->__reserved) + if ((cmd->flags & ~(IOMMU_HWPT_ALLOC_ENFORCE_DIRTY)) || + cmd->__reserved) return -EOPNOTSUPP; idev = iommufd_get_device(ucmd, cmd->dev_id); @@ -155,7 +177,8 @@ int iommufd_hwpt_alloc(struct iommufd_ucmd *ucmd) } mutex_lock(&ioas->mutex); - hwpt = iommufd_hw_pagetable_alloc(ucmd->ictx, ioas, idev, false); + hwpt = iommufd_hw_pagetable_alloc(ucmd->ictx, ioas, idev, false, + cmd->flags & IOMMU_HWPT_ALLOC_ENFORCE_DIRTY); if (IS_ERR(hwpt)) { rc = PTR_ERR(hwpt); goto out_unlock; diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h index dba730129b8c..2552eb44d83a 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -247,6 +247,7 @@ struct iommufd_hw_pagetable { struct iommu_domain *domain; bool auto_domain : 1; bool enforce_cache_coherency : 1; + bool enforce_dirty : 1; bool msi_cookie : 1; /* Head at iommufd_ioas::hwpt_list */ struct list_head hwpt_item; @@ -254,7 +255,8 @@ struct iommufd_hw_pagetable { struct iommufd_hw_pagetable * iommufd_hw_pagetable_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas, - struct iommufd_device *idev, bool immediate_attach); + struct iommufd_device *idev, bool immediate_attach, + bool enforce_dirty); int iommufd_hw_pagetable_enforce_cc(struct iommufd_hw_pagetable *hwpt); int iommufd_hw_pagetable_attach(struct iommufd_hw_pagetable *hwpt, struct iommufd_device *idev); diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 8245c01adca6..1cd9c54d0f64 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -346,6 +346,15 @@ struct iommu_vfio_ioas { }; #define IOMMU_VFIO_IOAS _IO(IOMMUFD_TYPE, IOMMUFD_CMD_VFIO_IOAS) +/** + * enum iommufd_hwpt_alloc_flags - Flags for alloc hwpt + * @IOMMU_HWPT_ALL_ENFORCE_DIRTY: Dirty tracking support for device IOMMU is + * enforced on device attachment + */ +enum iommufd_hwpt_alloc_flags { + IOMMU_HWPT_ALLOC_ENFORCE_DIRTY = 1 << 0, +}; + /** * struct iommu_hwpt_alloc - ioctl(IOMMU_HWPT_ALLOC) * @size: sizeof(struct iommu_hwpt_alloc) From patchwork Thu May 18 20:46:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247362 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FD9DC7EE2A for ; Thu, 18 May 2023 20:48:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230144AbjERUsK (ORCPT ); Thu, 18 May 2023 16:48:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56190 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229555AbjERUsJ (ORCPT ); Thu, 18 May 2023 16:48:09 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C0D8E7F for ; Thu, 18 May 2023 13:48:03 -0700 (PDT) Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIx8m9032118; Thu, 18 May 2023 20:47:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=iMktio9VTgbJ1yfqMEUUnV1ANkjO6zvNSUX5TxAJMrU=; b=ptwdTBuf1JcJzHelS2lyO/TdiOfO+V4FJmr4k2i45kDclFjYYylPnwSFn+KkGwVjzdd0 M7mluSL+e4tpC5whfaaHsEqHiJwGpDgdLKYAIN8IoAaKuozMYwzQzvGoSyxdfKvwgKKo WCYf7XrRCCAYQjQyAU0b9UjDYLfaETf36PL3KBNSxWTF+L+UwUPzhmsaDKrvuheAVvNh 5PSZf0bWA8V8Ao4tErxH3IIwobuRuFYg0IyaEc2mZc/yg9heTeGYrCkLvDV3Z26TGPY6 s0YSZXsoA9Y752HqssDs53x2LP7zBxDKB3x8go7O7bJA+j2F6wyUZRGZkUKJ4Yaueh2q 5Q== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qj33v0v9w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:47:41 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJkARw032121; Thu, 18 May 2023 20:47:40 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10daeff-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:47:40 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE2x033533; Thu, 18 May 2023 20:47:39 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-7; Thu, 18 May 2023 20:47:39 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 06/24] iommufd/selftest: Add a flags to _test_cmd_{hwpt_alloc,mock_domain} Date: Thu, 18 May 2023 21:46:32 +0100 Message-Id: <20230518204650.14541-7-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=865 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180170 X-Proofpoint-GUID: 5j-peOTwN9q8QBBoW7Zrh2AkwgJ4B1n7 X-Proofpoint-ORIG-GUID: 5j-peOTwN9q8QBBoW7Zrh2AkwgJ4B1n7 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org In preparation to test passing flags to HWPT_ALLOC (particularly IOMMU_HWPT_ALLOC_ENFORCE_DIRTY), add a flags argument into the test functions. Signed-off-by: Joao Martins --- tools/testing/selftests/iommu/iommufd.c | 3 ++- .../selftests/iommu/iommufd_fail_nth.c | 24 +++++++++++-------- tools/testing/selftests/iommu/iommufd_utils.h | 22 ++++++++++------- 3 files changed, 30 insertions(+), 19 deletions(-) diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selftests/iommu/iommufd.c index dc09c1de319f..771e4a40200f 100644 --- a/tools/testing/selftests/iommu/iommufd.c +++ b/tools/testing/selftests/iommu/iommufd.c @@ -1346,7 +1346,8 @@ TEST_F(iommufd_mock_domain, alloc_hwpt) uint32_t stddev_id; uint32_t hwpt_id; - test_cmd_hwpt_alloc(self->idev_ids[0], self->ioas_id, &hwpt_id); + test_cmd_hwpt_alloc(self->idev_ids[0], self->ioas_id, + 0, &hwpt_id); test_cmd_mock_domain(hwpt_id, &stddev_id, NULL, NULL); test_ioctl_destroy(stddev_id); test_ioctl_destroy(hwpt_id); diff --git a/tools/testing/selftests/iommu/iommufd_fail_nth.c b/tools/testing/selftests/iommu/iommufd_fail_nth.c index d4c552e56948..0e003069bb2a 100644 --- a/tools/testing/selftests/iommu/iommufd_fail_nth.c +++ b/tools/testing/selftests/iommu/iommufd_fail_nth.c @@ -315,7 +315,8 @@ TEST_FAIL_NTH(basic_fail_nth, map_domain) fail_nth_enable(); - if (_test_cmd_mock_domain(self->fd, ioas_id, &stdev_id, &hwpt_id, NULL)) + if (_test_cmd_mock_domain(self->fd, ioas_id, 0, &stdev_id, + &hwpt_id, NULL)) return -1; if (_test_ioctl_ioas_map(self->fd, ioas_id, buffer, 262144, &iova, @@ -326,7 +327,8 @@ TEST_FAIL_NTH(basic_fail_nth, map_domain) if (_test_ioctl_destroy(self->fd, stdev_id)) return -1; - if (_test_cmd_mock_domain(self->fd, ioas_id, &stdev_id, &hwpt_id, NULL)) + if (_test_cmd_mock_domain(self->fd, ioas_id, 0, + &stdev_id, &hwpt_id, NULL)) return -1; return 0; } @@ -350,13 +352,14 @@ TEST_FAIL_NTH(basic_fail_nth, map_two_domains) if (_test_ioctl_set_temp_memory_limit(self->fd, 32)) return -1; - if (_test_cmd_mock_domain(self->fd, ioas_id, &stdev_id, &hwpt_id, NULL)) + if (_test_cmd_mock_domain(self->fd, ioas_id, 0, + &stdev_id, &hwpt_id, NULL)) return -1; fail_nth_enable(); - if (_test_cmd_mock_domain(self->fd, ioas_id, &stdev_id2, &hwpt_id2, - NULL)) + if (_test_cmd_mock_domain(self->fd, ioas_id, 0, + &stdev_id2, &hwpt_id2, NULL)) return -1; if (_test_ioctl_ioas_map(self->fd, ioas_id, buffer, 262144, &iova, @@ -370,9 +373,9 @@ TEST_FAIL_NTH(basic_fail_nth, map_two_domains) if (_test_ioctl_destroy(self->fd, stdev_id2)) return -1; - if (_test_cmd_mock_domain(self->fd, ioas_id, &stdev_id, &hwpt_id, NULL)) + if (_test_cmd_mock_domain(self->fd, ioas_id, 0, &stdev_id, &hwpt_id, NULL)) return -1; - if (_test_cmd_mock_domain(self->fd, ioas_id, &stdev_id2, &hwpt_id2, + if (_test_cmd_mock_domain(self->fd, ioas_id, 0, &stdev_id2, &hwpt_id2, NULL)) return -1; return 0; @@ -530,7 +533,8 @@ TEST_FAIL_NTH(basic_fail_nth, access_pin_domain) if (_test_ioctl_set_temp_memory_limit(self->fd, 32)) return -1; - if (_test_cmd_mock_domain(self->fd, ioas_id, &stdev_id, &hwpt_id, NULL)) + if (_test_cmd_mock_domain(self->fd, ioas_id, 0, + &stdev_id, &hwpt_id, NULL)) return -1; if (_test_ioctl_ioas_map(self->fd, ioas_id, buffer, BUFFER_SIZE, &iova, @@ -607,11 +611,11 @@ TEST_FAIL_NTH(basic_fail_nth, device) fail_nth_enable(); - if (_test_cmd_mock_domain(self->fd, ioas_id, &stdev_id, NULL, + if (_test_cmd_mock_domain(self->fd, ioas_id, 0, &stdev_id, NULL, &idev_id)) return -1; - if (_test_cmd_hwpt_alloc(self->fd, idev_id, ioas_id, &hwpt_id)) + if (_test_cmd_hwpt_alloc(self->fd, idev_id, ioas_id, 0, &hwpt_id)) return -1; if (_test_cmd_mock_domain_replace(self->fd, stdev_id, ioas_id2, NULL)) diff --git a/tools/testing/selftests/iommu/iommufd_utils.h b/tools/testing/selftests/iommu/iommufd_utils.h index 53b4d3f2d9fc..04871bcfd34b 100644 --- a/tools/testing/selftests/iommu/iommufd_utils.h +++ b/tools/testing/selftests/iommu/iommufd_utils.h @@ -40,7 +40,8 @@ static unsigned long PAGE_SIZE; &test_cmd)); \ }) -static int _test_cmd_mock_domain(int fd, unsigned int ioas_id, __u32 *stdev_id, +static int _test_cmd_mock_domain(int fd, unsigned int ioas_id, + __u32 stdev_flags, __u32 *stdev_id, __u32 *hwpt_id, __u32 *idev_id) { struct iommu_test_cmd cmd = { @@ -64,10 +65,13 @@ static int _test_cmd_mock_domain(int fd, unsigned int ioas_id, __u32 *stdev_id, return 0; } #define test_cmd_mock_domain(ioas_id, stdev_id, hwpt_id, idev_id) \ - ASSERT_EQ(0, _test_cmd_mock_domain(self->fd, ioas_id, stdev_id, \ - hwpt_id, idev_id)) -#define test_err_mock_domain(_errno, ioas_id, stdev_id, hwpt_id) \ - EXPECT_ERRNO(_errno, _test_cmd_mock_domain(self->fd, ioas_id, \ + ASSERT_EQ(0, _test_cmd_mock_domain(self->fd, ioas_id, 0, \ + stdev_id, hwpt_id, idev_id)) +#define test_err_mock_domain(_errno, ioas_id, stdev_id, hwpt_id) \ + EXPECT_ERRNO(_errno, _test_cmd_mock_domain(self->fd, ioas_id, 0, \ + stdev_id, hwpt_id, NULL)) +#define test_err_mock_domain_flags(_errno, ioas_id, flags, stdev_id, hwpt_id) \ + EXPECT_ERRNO(_errno, _test_cmd_mock_domain(self->fd, ioas_id, flags, \ stdev_id, hwpt_id, NULL)) static int _test_cmd_mock_domain_replace(int fd, __u32 stdev_id, __u32 pt_id, @@ -99,10 +103,11 @@ static int _test_cmd_mock_domain_replace(int fd, __u32 stdev_id, __u32 pt_id, pt_id, NULL)) static int _test_cmd_hwpt_alloc(int fd, __u32 device_id, __u32 pt_id, - __u32 *hwpt_id) + __u32 flags, __u32 *hwpt_id) { struct iommu_hwpt_alloc cmd = { .size = sizeof(cmd), + .flags = flags, .dev_id = device_id, .pt_id = pt_id, }; @@ -116,8 +121,9 @@ static int _test_cmd_hwpt_alloc(int fd, __u32 device_id, __u32 pt_id, return 0; } -#define test_cmd_hwpt_alloc(device_id, pt_id, hwpt_id) \ - ASSERT_EQ(0, _test_cmd_hwpt_alloc(self->fd, device_id, pt_id, hwpt_id)) +#define test_cmd_hwpt_alloc(device_id, pt_id, flags, hwpt_id) \ + ASSERT_EQ(0, _test_cmd_hwpt_alloc(self->fd, device_id, pt_id, flags, \ + hwpt_id)) static int _test_cmd_create_access(int fd, unsigned int ioas_id, __u32 *access_id, unsigned int flags) From patchwork Thu May 18 20:46:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247363 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D1A8C77B73 for ; Thu, 18 May 2023 20:48:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230328AbjERUsd (ORCPT ); Thu, 18 May 2023 16:48:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56470 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230045AbjERUsX (ORCPT ); Thu, 18 May 2023 16:48:23 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1C40A170B for ; Thu, 18 May 2023 13:48:10 -0700 (PDT) Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIwuEr031068; Thu, 18 May 2023 20:47:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=L0Q7NWjsGheg+wnN5fTWgVsgopnQoZX5y/zPqPeSaRs=; b=U0U8y5pETs+0vjqBAjTiOz60zLgnWLHLzJhLXwX1XP43a3rvJo+8iEgkcjrSc5kIaqP5 Hr6V7lKf5Z9SleArtuqDrr5VcRWcekykoIISWYdeA2N/+pHurFpJ5yHzl3qiJcb7vQ73 TBqKIG2nFtdBI6iJChS9MaD4PYED6lSVzqDWx8/+5N/cBdKAxkc25ArrNcEpNWN2ME/i 5ig2NDSCFCvr3BJ3WGlmDYEUvPVeCyKK23QFbJ8WuM+GCl7IdTKQwnOF9rDKqWJLfEvl I+3qOOlN1rfAoPscffmVyAn5LQ4n0Zk2DucyIbkNhQSEs2GJjg9Rowk51cJYPK5nz9rd 3g== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qj0ye8t4v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:47:45 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJ7NOv032193; Thu, 18 May 2023 20:47:44 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10daehk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:47:44 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE31033533; Thu, 18 May 2023 20:47:43 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-8; Thu, 18 May 2023 20:47:43 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 07/24] iommufd/selftest: Test IOMMU_HWPT_ALLOC_ENFORCE_DIRTY Date: Thu, 18 May 2023 21:46:33 +0100 Message-Id: <20230518204650.14541-8-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180170 X-Proofpoint-ORIG-GUID: mtExfqFsCotpv87Lvg5oij-QJiEvRvVL X-Proofpoint-GUID: mtExfqFsCotpv87Lvg5oij-QJiEvRvVL Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org In order to selftest the iommu domain dirty enforcing we implement the mock_domain necessary support and add a new dev_flags to test that the attach_device fails as expected. Expand the existing mock_domain fixture with a enforce_dirty test that exercises the hwpt_alloc and device attachment. Signed-off-by: Joao Martins --- drivers/iommu/iommufd/iommufd_test.h | 5 +++ drivers/iommu/iommufd/selftest.c | 16 +++++++- tools/testing/selftests/iommu/Makefile | 3 ++ tools/testing/selftests/iommu/iommufd.c | 39 +++++++++++++++++++ tools/testing/selftests/iommu/iommufd_utils.h | 2 +- 5 files changed, 62 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/iommufd/iommufd_test.h b/drivers/iommu/iommufd/iommufd_test.h index dd9168a20ddf..9abcc3231137 100644 --- a/drivers/iommu/iommufd/iommufd_test.h +++ b/drivers/iommu/iommufd/iommufd_test.h @@ -39,6 +39,10 @@ enum { MOCK_FLAGS_ACCESS_CREATE_NEEDS_PIN_PAGES = 1 << 0, }; +enum { + MOCK_FLAGS_DEVICE_NO_DIRTY = 1 << 0, +}; + struct iommu_test_cmd { __u32 size; __u32 op; @@ -50,6 +54,7 @@ struct iommu_test_cmd { __aligned_u64 length; } add_reserved; struct { + __u32 dev_flags; __u32 out_stdev_id; __u32 out_hwpt_id; /* out_idev_id is the standard iommufd_bind object */ diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c index 9d43334e4faf..65daceb6e0dc 100644 --- a/drivers/iommu/iommufd/selftest.c +++ b/drivers/iommu/iommufd/selftest.c @@ -93,6 +93,7 @@ enum selftest_obj_type { struct mock_dev { struct device dev; + unsigned long flags; }; struct selftest_obj { @@ -115,6 +116,12 @@ static void mock_domain_blocking_free(struct iommu_domain *domain) static int mock_domain_nop_attach(struct iommu_domain *domain, struct device *dev) { + struct mock_dev *mdev = container_of(dev, struct mock_dev, dev); + + if ((domain->flags & IOMMU_DOMAIN_F_ENFORCE_DIRTY) && + (mdev->flags & MOCK_FLAGS_DEVICE_NO_DIRTY)) + return -EINVAL; + return 0; } @@ -278,6 +285,7 @@ static void mock_domain_set_plaform_dma_ops(struct device *dev) static const struct iommu_ops mock_ops = { .owner = THIS_MODULE, + .supported_flags = IOMMU_DOMAIN_F_ENFORCE_DIRTY, .pgsize_bitmap = MOCK_IO_PAGE_SIZE, .domain_alloc = mock_domain_alloc, .capable = mock_domain_capable, @@ -328,18 +336,22 @@ static void mock_dev_release(struct device *dev) kfree(mdev); } -static struct mock_dev *mock_dev_create(void) +static struct mock_dev *mock_dev_create(unsigned long dev_flags) { struct iommu_group *iommu_group; struct dev_iommu *dev_iommu; struct mock_dev *mdev; int rc; + if (dev_flags & ~(MOCK_FLAGS_DEVICE_NO_DIRTY)) + return ERR_PTR(-EINVAL); + mdev = kzalloc(sizeof(*mdev), GFP_KERNEL); if (!mdev) return ERR_PTR(-ENOMEM); device_initialize(&mdev->dev); + mdev->flags = dev_flags; mdev->dev.release = mock_dev_release; mdev->dev.bus = &iommufd_mock_bus_type; @@ -422,7 +434,7 @@ static int iommufd_test_mock_domain(struct iommufd_ucmd *ucmd, sobj->idev.ictx = ucmd->ictx; sobj->type = TYPE_IDEV; - sobj->idev.mock_dev = mock_dev_create(); + sobj->idev.mock_dev = mock_dev_create(cmd->mock_domain.dev_flags); if (IS_ERR(sobj->idev.mock_dev)) { rc = PTR_ERR(sobj->idev.mock_dev); goto out_sobj; diff --git a/tools/testing/selftests/iommu/Makefile b/tools/testing/selftests/iommu/Makefile index 32c5fdfd0eef..f1aee4e5ec2e 100644 --- a/tools/testing/selftests/iommu/Makefile +++ b/tools/testing/selftests/iommu/Makefile @@ -1,5 +1,8 @@ # SPDX-License-Identifier: GPL-2.0-only CFLAGS += -Wall -O2 -Wno-unused-function +CFLAGS += -I../../../../tools/include/ +CFLAGS += -I../../../../include/uapi/ +CFLAGS += -I../../../../include/ CFLAGS += $(KHDR_INCLUDES) CFLAGS += -D_GNU_SOURCE diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selftests/iommu/iommufd.c index 771e4a40200f..da7d1dad1816 100644 --- a/tools/testing/selftests/iommu/iommufd.c +++ b/tools/testing/selftests/iommu/iommufd.c @@ -1354,6 +1354,45 @@ TEST_F(iommufd_mock_domain, alloc_hwpt) } } +FIXTURE(iommufd_dirty_tracking) +{ + int fd; + uint32_t ioas_id; + uint32_t hwpt_id; + uint32_t stdev_id; + uint32_t idev_id; +}; + +FIXTURE_SETUP(iommufd_dirty_tracking) +{ + self->fd = open("/dev/iommu", O_RDWR); + ASSERT_NE(-1, self->fd); + + test_ioctl_ioas_alloc(&self->ioas_id); + test_cmd_mock_domain(self->ioas_id, &self->stdev_id, + &self->hwpt_id, &self->idev_id); +} + +FIXTURE_TEARDOWN(iommufd_dirty_tracking) +{ + teardown_iommufd(self->fd, _metadata); +} + +TEST_F(iommufd_dirty_tracking, enforce_dirty) +{ + uint32_t dev_flags = MOCK_FLAGS_DEVICE_NO_DIRTY; + uint32_t stddev_id; + uint32_t hwpt_id; + + test_cmd_hwpt_alloc(self->idev_id, self->ioas_id, + IOMMU_HWPT_ALLOC_ENFORCE_DIRTY, &hwpt_id); + test_cmd_mock_domain(hwpt_id, &stddev_id, NULL, NULL); + test_err_mock_domain_flags(EINVAL, hwpt_id, dev_flags, + &stddev_id, NULL); + test_ioctl_destroy(stddev_id); + test_ioctl_destroy(hwpt_id); +} + /* VFIO compatibility IOCTLs */ TEST_F(iommufd, simple_ioctls) diff --git a/tools/testing/selftests/iommu/iommufd_utils.h b/tools/testing/selftests/iommu/iommufd_utils.h index 04871bcfd34b..f8c926f96f23 100644 --- a/tools/testing/selftests/iommu/iommufd_utils.h +++ b/tools/testing/selftests/iommu/iommufd_utils.h @@ -48,7 +48,7 @@ static int _test_cmd_mock_domain(int fd, unsigned int ioas_id, .size = sizeof(cmd), .op = IOMMU_TEST_OP_MOCK_DOMAIN, .id = ioas_id, - .mock_domain = {}, + .mock_domain = { .dev_flags = stdev_flags }, }; int ret; From patchwork Thu May 18 20:46:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247364 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22521C77B73 for ; Thu, 18 May 2023 20:48:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229737AbjERUs6 (ORCPT ); Thu, 18 May 2023 16:48:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56834 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229948AbjERUs4 (ORCPT ); Thu, 18 May 2023 16:48:56 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 302C810E5 for ; Thu, 18 May 2023 13:48:37 -0700 (PDT) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIx6tA032295; Thu, 18 May 2023 20:48:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=u/U+v7/5wsTNvcp+fP9iSCgVpBo4DhTF0KBqbtAIEYw=; b=b/PTmdv9ds5Ta5bI71gznMbo2WN9/pa1fm8aEkzTN1tZUqrTw+2mmiMHLnsyXd+bU2Y6 ALwhuTI4X5Z2sWeKcdNTBZRRM4uEznmABau4p83HFHx5hhNphD8btK9RZ2O0c97VEmNt TcLFKrWVXkcu2UBXHjMKFNruvPJl2c/uK4UdHx9eqQMBTddk+rIhq+GO6VeRwyRDluUy LGcut1ik2F0NcblkEV3a3+SMJEXM/8485gIZ7AYHnkJSIrnJjk4PnXqGMCVRdgle14Cr 1RGaEKHFD8t3V/P1/pJRwngm4K8zScT3xymzZAhB7cqz5eoF93OmvOGa2fplfs7b/hG/ fQ== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qnkux92kq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:17 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJPApp032227; Thu, 18 May 2023 20:48:16 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10daex1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:16 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE33033533; Thu, 18 May 2023 20:48:15 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-9; Thu, 18 May 2023 20:48:15 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 08/24] iommufd: Dirty tracking data support Date: Thu, 18 May 2023 21:46:34 +0100 Message-Id: <20230518204650.14541-9-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=814 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180171 X-Proofpoint-ORIG-GUID: CPJOkXA26O1RcAo3BX73B8FXQx7xEGrE X-Proofpoint-GUID: CPJOkXA26O1RcAo3BX73B8FXQx7xEGrE Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add an IO pagetable API iopt_read_and_clear_dirty_data() that performs the reading of dirty IOPTEs for a given IOVA range and then copying back to userspace bitmap. Underneath it uses the IOMMU domain kernel API which will read the dirty bits, as well as atomically clearing the IOPTE dirty bit and flushing the IOTLB at the end. The IOVA bitmaps usage takes care of the iteration of the bitmaps user pages efficiently and without copies. Signed-off-by: Joao Martins --- drivers/iommu/iommufd/io_pagetable.c | 70 +++++++++++++++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 14 +++++ 2 files changed, 84 insertions(+) diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c index 21052f64f956..187626e5f2bc 100644 --- a/drivers/iommu/iommufd/io_pagetable.c +++ b/drivers/iommu/iommufd/io_pagetable.c @@ -15,6 +15,7 @@ #include #include #include +#include #include "io_pagetable.h" #include "double_span.h" @@ -412,6 +413,75 @@ int iopt_map_user_pages(struct iommufd_ctx *ictx, struct io_pagetable *iopt, return 0; } +struct iova_bitmap_fn_arg { + struct iommu_domain *domain; + struct iommu_dirty_bitmap *dirty; +}; + +static int __iommu_read_and_clear_dirty(struct iova_bitmap *bitmap, + unsigned long iova, size_t length, + void *opaque) +{ + struct iova_bitmap_fn_arg *arg = opaque; + struct iommu_domain *domain = arg->domain; + const struct iommu_domain_ops *ops = domain->ops; + struct iommu_dirty_bitmap *dirty = arg->dirty; + + return ops->read_and_clear_dirty(domain, iova, length, 0, dirty); +} + +static int iommu_read_and_clear_dirty(struct iommu_domain *domain, + unsigned long flags, + struct iommufd_dirty_data *bitmap) +{ + const struct iommu_domain_ops *ops = domain->ops; + struct iommu_iotlb_gather gather; + struct iommu_dirty_bitmap dirty; + struct iova_bitmap_fn_arg arg; + struct iova_bitmap *iter; + int ret = 0; + + if (!ops || !ops->read_and_clear_dirty) + return -EOPNOTSUPP; + + iter = iova_bitmap_alloc(bitmap->iova, bitmap->length, + bitmap->page_size, bitmap->data); + if (IS_ERR(iter)) + return -ENOMEM; + + iommu_dirty_bitmap_init(&dirty, iter, &gather); + + arg.domain = domain; + arg.dirty = &dirty; + iova_bitmap_for_each(iter, &arg, __iommu_read_and_clear_dirty); + + iommu_iotlb_sync(domain, &gather); + iova_bitmap_free(iter); + + return ret; +} + +int iopt_read_and_clear_dirty_data(struct io_pagetable *iopt, + struct iommu_domain *domain, + unsigned long flags, + struct iommufd_dirty_data *bitmap) +{ + unsigned long last_iova, iova = bitmap->iova; + unsigned long length = bitmap->length; + int ret = -EOPNOTSUPP; + + if ((iova & (iopt->iova_alignment - 1))) + return -EINVAL; + + if (check_add_overflow(iova, length - 1, &last_iova)) + return -EOVERFLOW; + + down_read(&iopt->iova_rwsem); + ret = iommu_read_and_clear_dirty(domain, flags, bitmap); + up_read(&iopt->iova_rwsem); + return ret; +} + int iopt_get_pages(struct io_pagetable *iopt, unsigned long iova, unsigned long length, struct list_head *pages_list) { diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h index 2552eb44d83a..2259b15340e4 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -8,6 +8,8 @@ #include #include #include +#include +#include struct iommu_domain; struct iommu_group; @@ -70,6 +72,18 @@ int iopt_unmap_iova(struct io_pagetable *iopt, unsigned long iova, unsigned long length, unsigned long *unmapped); int iopt_unmap_all(struct io_pagetable *iopt, unsigned long *unmapped); +struct iommufd_dirty_data { + unsigned long iova; + unsigned long length; + unsigned long page_size; + unsigned long long *data; +}; + +int iopt_read_and_clear_dirty_data(struct io_pagetable *iopt, + struct iommu_domain *domain, + unsigned long flags, + struct iommufd_dirty_data *bitmap); + void iommufd_access_notify_unmap(struct io_pagetable *iopt, unsigned long iova, unsigned long length); int iopt_table_add_domain(struct io_pagetable *iopt, From patchwork Thu May 18 20:46:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1802CC77B7A for ; Thu, 18 May 2023 20:49:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229936AbjERUtJ (ORCPT ); Thu, 18 May 2023 16:49:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230045AbjERUtE (ORCPT ); Thu, 18 May 2023 16:49:04 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 81C2110E9 for ; Thu, 18 May 2023 13:48:43 -0700 (PDT) Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIx5mj025244; Thu, 18 May 2023 20:48:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=Zo1jLF/mimg0CRENi9hdWwSatdNwv2JQ/hbvjLQOaQk=; b=R5y+MfQeheDzytVuFdVjovz0EYXqco4w5hDnxvb4iTfIFE3lKs2ioOKkiN/hvLtFexVi 0jeaKbVmGUAoL2RhupvDBkExDYZXFmSbJCAlvsmQ3J2J/cbXS01SLRhXlJVtVlLnWWqF Lzo/i2RqodXXHM6gVjzunLka06/EFdRNdTqw+6UkJ8La960wWmTQKgSrSD+oqFs6PBuS qwdJNttJvgrBP8wqsOiXk/Xp+RQRHGfvQk5crE+CyBTHdzK7kI6xI3IwH5/7E6NccJ8R OwaGF6tOl67ck+5fl3gIxB8fGrisE2d8wYrOayELyPLhTdkPwgnNvj9WHPgfql+HjCu3 RQ== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qj1fc97dj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:22 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IKXqwQ032130; Thu, 18 May 2023 20:48:20 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10daeyu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:20 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE35033533; Thu, 18 May 2023 20:48:20 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-10; Thu, 18 May 2023 20:48:19 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 09/24] iommufd: Add IOMMU_HWPT_SET_DIRTY Date: Thu, 18 May 2023 21:46:35 +0100 Message-Id: <20230518204650.14541-10-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=881 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180171 X-Proofpoint-ORIG-GUID: wDOjp4O33BkPZTujXR17B-FiRDBN-kVe X-Proofpoint-GUID: wDOjp4O33BkPZTujXR17B-FiRDBN-kVe Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Every IOMMU driver should be able to implement the needed iommu domain ops to control dirty tracking. Connect a hw_pagetable to the IOMMU core dirty tracking ops, specifically the ability to enable/disable dirty tracking on an IOMMU domain (hw_pagetable id). To that end add an io_pagetable kernel API to toggle dirty tracking: * iopt_set_dirty_tracking(iopt, [domain], state) The intended caller of this is via the hw_pagetable object that is created. Internally we will ensure the leftover dirty state is cleared /right before/ we start dirty tracking. This is also useful for iommu drivers which may decide that dirty tracking is always-enabled without being able to disable it via iommu domain op. Signed-off-by: Joao Martins --- drivers/iommu/iommufd/hw_pagetable.c | 24 +++++++++++++++++ drivers/iommu/iommufd/io_pagetable.c | 36 +++++++++++++++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 11 ++++++++ drivers/iommu/iommufd/main.c | 3 +++ include/uapi/linux/iommufd.h | 27 +++++++++++++++++++ 5 files changed, 101 insertions(+) diff --git a/drivers/iommu/iommufd/hw_pagetable.c b/drivers/iommu/iommufd/hw_pagetable.c index 4f0b72737ae2..7acbd88d05b7 100644 --- a/drivers/iommu/iommufd/hw_pagetable.c +++ b/drivers/iommu/iommufd/hw_pagetable.c @@ -200,3 +200,27 @@ int iommufd_hwpt_alloc(struct iommufd_ucmd *ucmd) iommufd_put_object(&idev->obj); return rc; } + +int iommufd_hwpt_set_dirty(struct iommufd_ucmd *ucmd) +{ + struct iommu_hwpt_set_dirty *cmd = ucmd->cmd; + struct iommufd_hw_pagetable *hwpt; + struct iommufd_ioas *ioas; + int rc = -EOPNOTSUPP; + bool enable; + + hwpt = iommufd_get_hwpt(ucmd, cmd->hwpt_id); + if (IS_ERR(hwpt)) + return PTR_ERR(hwpt); + + if (!hwpt->enforce_dirty) + return -EOPNOTSUPP; + + ioas = hwpt->ioas; + enable = cmd->flags & IOMMU_DIRTY_TRACKING_ENABLED; + + rc = iopt_set_dirty_tracking(&ioas->iopt, hwpt->domain, enable); + + iommufd_put_object(&hwpt->obj); + return rc; +} diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c index 187626e5f2bc..01adb0f7e4d0 100644 --- a/drivers/iommu/iommufd/io_pagetable.c +++ b/drivers/iommu/iommufd/io_pagetable.c @@ -479,6 +479,42 @@ int iopt_read_and_clear_dirty_data(struct io_pagetable *iopt, down_read(&iopt->iova_rwsem); ret = iommu_read_and_clear_dirty(domain, flags, bitmap); up_read(&iopt->iova_rwsem); + + return ret; +} + +int iopt_set_dirty_tracking(struct io_pagetable *iopt, + struct iommu_domain *domain, bool enable) +{ + const struct iommu_domain_ops *ops = domain->ops; + struct iommu_dirty_bitmap dirty; + struct iommu_iotlb_gather gather; + struct iopt_area *area; + int ret = 0; + + if (!ops->set_dirty_tracking) + return -EOPNOTSUPP; + + iommu_dirty_bitmap_init(&dirty, NULL, &gather); + + down_write(&iopt->iova_rwsem); + for (area = iopt_area_iter_first(iopt, 0, ULONG_MAX); + area && enable; + area = iopt_area_iter_next(area, 0, ULONG_MAX)) { + ret = ops->read_and_clear_dirty(domain, + iopt_area_iova(area), + iopt_area_length(area), 0, + &dirty); + if (ret) + goto out_unlock; + } + + iommu_iotlb_sync(domain, &gather); + + ret = ops->set_dirty_tracking(domain, enable); + +out_unlock: + up_write(&iopt->iova_rwsem); return ret; } diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h index 2259b15340e4..e902197a6a42 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -10,6 +10,7 @@ #include #include #include +#include struct iommu_domain; struct iommu_group; @@ -83,6 +84,8 @@ int iopt_read_and_clear_dirty_data(struct io_pagetable *iopt, struct iommu_domain *domain, unsigned long flags, struct iommufd_dirty_data *bitmap); +int iopt_set_dirty_tracking(struct io_pagetable *iopt, + struct iommu_domain *domain, bool enable); void iommufd_access_notify_unmap(struct io_pagetable *iopt, unsigned long iova, unsigned long length); @@ -267,6 +270,14 @@ struct iommufd_hw_pagetable { struct list_head hwpt_item; }; +static inline struct iommufd_hw_pagetable *iommufd_get_hwpt( + struct iommufd_ucmd *ucmd, u32 id) +{ + return container_of(iommufd_get_object(ucmd->ictx, id, + IOMMUFD_OBJ_HW_PAGETABLE), + struct iommufd_hw_pagetable, obj); +} +int iommufd_hwpt_set_dirty(struct iommufd_ucmd *ucmd); struct iommufd_hw_pagetable * iommufd_hw_pagetable_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas, struct iommufd_device *idev, bool immediate_attach, diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index 3932fe26522b..8c4640df0547 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -277,6 +277,7 @@ union ucmd_buffer { struct iommu_ioas_unmap unmap; struct iommu_option option; struct iommu_vfio_ioas vfio_ioas; + struct iommu_hwpt_set_dirty set_dirty; #ifdef CONFIG_IOMMUFD_TEST struct iommu_test_cmd test; #endif @@ -318,6 +319,8 @@ static const struct iommufd_ioctl_op iommufd_ioctl_ops[] = { val64), IOCTL_OP(IOMMU_VFIO_IOAS, iommufd_vfio_ioas, struct iommu_vfio_ioas, __reserved), + IOCTL_OP(IOMMU_HWPT_SET_DIRTY, iommufd_hwpt_set_dirty, + struct iommu_hwpt_set_dirty, __reserved), #ifdef CONFIG_IOMMUFD_TEST IOCTL_OP(IOMMU_TEST_CMD, iommufd_test, struct iommu_test_cmd, last), #endif diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 1cd9c54d0f64..85498f14b3ae 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -46,6 +46,7 @@ enum { IOMMUFD_CMD_OPTION, IOMMUFD_CMD_VFIO_IOAS, IOMMUFD_CMD_HWPT_ALLOC, + IOMMUFD_CMD_HWPT_SET_DIRTY, }; /** @@ -379,4 +380,30 @@ struct iommu_hwpt_alloc { __u32 __reserved; }; #define IOMMU_HWPT_ALLOC _IO(IOMMUFD_TYPE, IOMMUFD_CMD_HWPT_ALLOC) + +/** + * enum iommufd_set_dirty_flags - Flags for steering dirty tracking + * @IOMMU_DIRTY_TRACKING_DISABLED: Disables dirty tracking + * @IOMMU_DIRTY_TRACKING_ENABLED: Enables dirty tracking + */ +enum iommufd_hwpt_set_dirty_flags { + IOMMU_DIRTY_TRACKING_DISABLED = 0, + IOMMU_DIRTY_TRACKING_ENABLED = 1, +}; + +/** + * struct iommu_hwpt_set_dirty - ioctl(IOMMU_HWPT_SET_DIRTY) + * @size: sizeof(struct iommu_hwpt_set_dirty) + * @flags: Flags to control dirty tracking status. + * @hwpt_id: HW pagetable ID that represents the IOMMU domain. + * + * Toggle dirty tracking on an HW pagetable. + */ +struct iommu_hwpt_set_dirty { + __u32 size; + __u32 flags; + __u32 hwpt_id; + __u32 __reserved; +}; +#define IOMMU_HWPT_SET_DIRTY _IO(IOMMUFD_TYPE, IOMMUFD_CMD_HWPT_SET_DIRTY) #endif From patchwork Thu May 18 20:46:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247366 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A5B6C77B73 for ; Thu, 18 May 2023 20:49:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229933AbjERUtK (ORCPT ); Thu, 18 May 2023 16:49:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230331AbjERUtG (ORCPT ); Thu, 18 May 2023 16:49:06 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E6511701 for ; Thu, 18 May 2023 13:48:46 -0700 (PDT) Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIxAvA015429; Thu, 18 May 2023 20:48:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=0M9fuTvW7n8YESX/WjBfXuzWqz7zmpS+sUo0kbpONCk=; b=3XwVsmKJCZQqrYO6iIJRmix4pOb2pYE/gLmTbhBG5lOe/QPYds51qkk07U1+OyFrhwCi z6/QiSr6pi/tjlidm9wJO4/ROqAlgvdouTyIFtOzvQFj+lgSxRcPN7itif6pyeZZBWeN 6NL1mpUlnqZ728vEUCy9kR74DP3uGof5uM2vyYUsbSSSI6sxNaXKmhL/qc+EzUNSUkUD QijndwG25AJYNZC6Vy4vM2qvY18SRcRFzPfqje26pIpE6c1WTg9hUfunDOwresaE9U5m Z+iKn4IzJ3VGnzNEpkFVmDMh5vJCiKUF5D2b9c4XryT0VBeCo3QGmuGKf6+UZYFXOLdy OQ== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qmx8j3n4u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:26 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJPApt032227; Thu, 18 May 2023 20:48:24 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10daf1w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:24 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE37033533; Thu, 18 May 2023 20:48:24 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-11; Thu, 18 May 2023 20:48:23 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 10/24] iommufd/selftest: Test IOMMU_HWPT_SET_DIRTY Date: Thu, 18 May 2023 21:46:36 +0100 Message-Id: <20230518204650.14541-11-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180171 X-Proofpoint-GUID: WbVSU2uJvZrucqjVqzpeEhgXoMTAPY7i X-Proofpoint-ORIG-GUID: WbVSU2uJvZrucqjVqzpeEhgXoMTAPY7i Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Change mock_domain to supporting dirty tracking and add tests to exercise the new SET_DIRTY API in the mock_domain selftest fixture. Signed-off-by: Joao Martins --- drivers/iommu/iommufd/selftest.c | 21 +++++++++++++++++++ tools/testing/selftests/iommu/iommufd.c | 15 +++++++++++++ tools/testing/selftests/iommu/iommufd_utils.h | 18 ++++++++++++++++ 3 files changed, 54 insertions(+) diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c index 65daceb6e0dc..ee7523c8d46a 100644 --- a/drivers/iommu/iommufd/selftest.c +++ b/drivers/iommu/iommufd/selftest.c @@ -21,6 +21,7 @@ static struct dentry *dbgfs_root; size_t iommufd_test_memory_limit = 65536; enum { + MOCK_DIRTY_TRACK = 1, MOCK_IO_PAGE_SIZE = PAGE_SIZE / 2, /* @@ -83,6 +84,7 @@ void iommufd_test_syz_conv_iova_id(struct iommufd_ucmd *ucmd, } struct mock_iommu_domain { + unsigned long flags; struct iommu_domain domain; struct xarray pfns; }; @@ -283,6 +285,24 @@ static void mock_domain_set_plaform_dma_ops(struct device *dev) */ } +static int mock_domain_set_dirty_tracking(struct iommu_domain *domain, + bool enable) +{ + struct mock_iommu_domain *mock = + container_of(domain, struct mock_iommu_domain, domain); + unsigned long flags = mock->flags; + + /* No change? */ + if (!(enable ^ !!(flags & MOCK_DIRTY_TRACK))) + return -EINVAL; + + flags = (enable ? + flags | MOCK_DIRTY_TRACK : flags & ~MOCK_DIRTY_TRACK); + + mock->flags = flags; + return 0; +} + static const struct iommu_ops mock_ops = { .owner = THIS_MODULE, .supported_flags = IOMMU_DOMAIN_F_ENFORCE_DIRTY, @@ -297,6 +317,7 @@ static const struct iommu_ops mock_ops = { .map_pages = mock_domain_map_pages, .unmap_pages = mock_domain_unmap_pages, .iova_to_phys = mock_domain_iova_to_phys, + .set_dirty_tracking = mock_domain_set_dirty_tracking, }, }; diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selftests/iommu/iommufd.c index da7d1dad1816..8adccdde5ecc 100644 --- a/tools/testing/selftests/iommu/iommufd.c +++ b/tools/testing/selftests/iommu/iommufd.c @@ -1393,6 +1393,21 @@ TEST_F(iommufd_dirty_tracking, enforce_dirty) test_ioctl_destroy(hwpt_id); } +TEST_F(iommufd_dirty_tracking, set_dirty) +{ + uint32_t stddev_id; + uint32_t hwpt_id; + + test_cmd_hwpt_alloc(self->idev_id, self->ioas_id, + IOMMU_HWPT_ALLOC_ENFORCE_DIRTY, &hwpt_id); + test_cmd_mock_domain(hwpt_id, &stddev_id, NULL, NULL); + test_cmd_set_dirty(hwpt_id, true); + test_cmd_set_dirty(hwpt_id, false); + + test_ioctl_destroy(stddev_id); + test_ioctl_destroy(hwpt_id); +} + /* VFIO compatibility IOCTLs */ TEST_F(iommufd, simple_ioctls) diff --git a/tools/testing/selftests/iommu/iommufd_utils.h b/tools/testing/selftests/iommu/iommufd_utils.h index f8c926f96f23..3629c531ec9f 100644 --- a/tools/testing/selftests/iommu/iommufd_utils.h +++ b/tools/testing/selftests/iommu/iommufd_utils.h @@ -125,6 +125,24 @@ static int _test_cmd_hwpt_alloc(int fd, __u32 device_id, __u32 pt_id, ASSERT_EQ(0, _test_cmd_hwpt_alloc(self->fd, device_id, pt_id, flags, \ hwpt_id)) +static int _test_cmd_set_dirty(int fd, __u32 hwpt_id, bool enabled) +{ + struct iommu_hwpt_set_dirty cmd = { + .size = sizeof(cmd), + .flags = enabled ? IOMMU_DIRTY_TRACKING_ENABLED : + IOMMU_DIRTY_TRACKING_DISABLED, + .hwpt_id = hwpt_id, + }; + int ret; + + ret = ioctl(fd, IOMMU_HWPT_SET_DIRTY, &cmd); + if (ret) + return ret; + return 0; +} + +#define test_cmd_set_dirty(hwpt_id, enabled) \ + ASSERT_EQ(0, _test_cmd_set_dirty(self->fd, hwpt_id, enabled)) static int _test_cmd_create_access(int fd, unsigned int ioas_id, __u32 *access_id, unsigned int flags) { From patchwork Thu May 18 20:46:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247367 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C012C77B7A for ; Thu, 18 May 2023 20:49:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230141AbjERUtL (ORCPT ); Thu, 18 May 2023 16:49:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57070 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229951AbjERUtG (ORCPT ); Thu, 18 May 2023 16:49:06 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3BF2D1703 for ; Thu, 18 May 2023 13:48:49 -0700 (PDT) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIxZTY012446; Thu, 18 May 2023 20:48:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=gDSYNGIHrGuf8J0xePV4A5sx2JRGL/2I7xS8EwFae6A=; b=Cud2iGssGRNLELNjJRgxSog3qPaCgsLFfA9DCjUMQCTUVD8XYCQJvIOlHrALGU8ze8r2 0psPmi7imUkAi1FXXmuy8rtgWE1pzJRsr+t2K7o0glVU93E9hRSqMAmdt3NuKoWNE0Fj Pnb1R/nDjFtBshamWXhDzFOJ2a8fb75LP0fuyQ+IsRPD6nXm5Jg8AOjdVS6HdDjlbT6K 9DQs1sBtIm/GGAlHweQqkur012RliNnfEjpUC/OXnlmSgbdWl8/QfMU6thMvD+WIHf7t LjPJcMRkoGNU2Vy0Of7McoWdgtloPMUXUE28JvUomwRsAnOzHoecC2mIFLeIQHzLbRKP hA== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qmxfc3jyg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:30 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJBjk6032143; Thu, 18 May 2023 20:48:29 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10daf4e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:29 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE39033533; Thu, 18 May 2023 20:48:28 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-12; Thu, 18 May 2023 20:48:28 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 11/24] iommufd: Add IOMMU_HWPT_GET_DIRTY_IOVA Date: Thu, 18 May 2023 21:46:37 +0100 Message-Id: <20230518204650.14541-12-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180171 X-Proofpoint-GUID: w4t0YsHyKiD5nXNvsY-5Yd862JJYu-jY X-Proofpoint-ORIG-GUID: w4t0YsHyKiD5nXNvsY-5Yd862JJYu-jY Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Connect a hw_pagetable to the IOMMU core dirty tracking read_and_clear_dirty iommu domain op. It exposes all of the functionality for the UAPI that read the dirtied IOVAs while clearing the Dirty bits from the PTEs In doing so the previously internal iommufd_dirty_data structure is moved over as the UAPI intermediate structure for representing iommufd dirty bitmaps. Contrary to past incantation of a similar interface in VFIO the IOVA range to be scanned is tied in to the bitmap size, thus the application needs to pass a appropriately sized bitmap address taking into account the iova range being passed *and* page size ... as opposed to allowing bitmap-iova != iova. Signed-off-by: Joao Martins --- drivers/iommu/iommufd/hw_pagetable.c | 58 +++++++++++++++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 11 ++--- drivers/iommu/iommufd/main.c | 3 ++ include/uapi/linux/iommufd.h | 36 +++++++++++++++ 4 files changed, 101 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/iommufd/hw_pagetable.c b/drivers/iommu/iommufd/hw_pagetable.c index 7acbd88d05b7..25860aa0a1f8 100644 --- a/drivers/iommu/iommufd/hw_pagetable.c +++ b/drivers/iommu/iommufd/hw_pagetable.c @@ -224,3 +224,61 @@ int iommufd_hwpt_set_dirty(struct iommufd_ucmd *ucmd) iommufd_put_object(&hwpt->obj); return rc; } + +int iommufd_check_iova_range(struct iommufd_ioas *ioas, + struct iommufd_dirty_data *bitmap) +{ + unsigned long pgshift, npages; + size_t iommu_pgsize; + int rc = -EINVAL; + + pgshift = __ffs(bitmap->page_size); + npages = bitmap->length >> pgshift; + + if (!npages || (npages > ULONG_MAX)) + return rc; + + iommu_pgsize = 1 << __ffs(ioas->iopt.iova_alignment); + + /* allow only smallest supported pgsize */ + if (bitmap->page_size != iommu_pgsize) + return rc; + + if (bitmap->iova & (iommu_pgsize - 1)) + return rc; + + if (!bitmap->length || bitmap->length & (iommu_pgsize - 1)) + return rc; + + return 0; +} + +int iommufd_hwpt_get_dirty_iova(struct iommufd_ucmd *ucmd) +{ + struct iommu_hwpt_get_dirty_iova *cmd = ucmd->cmd; + struct iommufd_hw_pagetable *hwpt; + struct iommufd_ioas *ioas; + int rc = -EOPNOTSUPP; + + if ((cmd->flags || cmd->__reserved)) + return -EOPNOTSUPP; + + hwpt = iommufd_get_hwpt(ucmd, cmd->hwpt_id); + if (IS_ERR(hwpt)) + return PTR_ERR(hwpt); + + if (!hwpt->enforce_dirty) + return -EOPNOTSUPP; + + ioas = hwpt->ioas; + rc = iommufd_check_iova_range(ioas, &cmd->bitmap); + if (rc) + goto out_put; + + rc = iopt_read_and_clear_dirty_data(&ioas->iopt, hwpt->domain, + cmd->flags, &cmd->bitmap); + +out_put: + iommufd_put_object(&hwpt->obj); + return rc; +} diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h index e902197a6a42..3de8046fee07 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -73,13 +73,6 @@ int iopt_unmap_iova(struct io_pagetable *iopt, unsigned long iova, unsigned long length, unsigned long *unmapped); int iopt_unmap_all(struct io_pagetable *iopt, unsigned long *unmapped); -struct iommufd_dirty_data { - unsigned long iova; - unsigned long length; - unsigned long page_size; - unsigned long long *data; -}; - int iopt_read_and_clear_dirty_data(struct io_pagetable *iopt, struct iommu_domain *domain, unsigned long flags, @@ -251,6 +244,8 @@ int iommufd_option_rlimit_mode(struct iommu_option *cmd, struct iommufd_ctx *ictx); int iommufd_vfio_ioas(struct iommufd_ucmd *ucmd); +int iommufd_check_iova_range(struct iommufd_ioas *ioas, + struct iommufd_dirty_data *bitmap); /* * A HW pagetable is called an iommu_domain inside the kernel. This user object @@ -278,6 +273,8 @@ static inline struct iommufd_hw_pagetable *iommufd_get_hwpt( struct iommufd_hw_pagetable, obj); } int iommufd_hwpt_set_dirty(struct iommufd_ucmd *ucmd); +int iommufd_hwpt_get_dirty_iova(struct iommufd_ucmd *ucmd); + struct iommufd_hw_pagetable * iommufd_hw_pagetable_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas, struct iommufd_device *idev, bool immediate_attach, diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index 8c4640df0547..f34b309a1baf 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -278,6 +278,7 @@ union ucmd_buffer { struct iommu_option option; struct iommu_vfio_ioas vfio_ioas; struct iommu_hwpt_set_dirty set_dirty; + struct iommu_hwpt_get_dirty_iova get_dirty_iova; #ifdef CONFIG_IOMMUFD_TEST struct iommu_test_cmd test; #endif @@ -321,6 +322,8 @@ static const struct iommufd_ioctl_op iommufd_ioctl_ops[] = { __reserved), IOCTL_OP(IOMMU_HWPT_SET_DIRTY, iommufd_hwpt_set_dirty, struct iommu_hwpt_set_dirty, __reserved), + IOCTL_OP(IOMMU_HWPT_GET_DIRTY_IOVA, iommufd_hwpt_get_dirty_iova, + struct iommu_hwpt_get_dirty_iova, bitmap.data), #ifdef CONFIG_IOMMUFD_TEST IOCTL_OP(IOMMU_TEST_CMD, iommufd_test, struct iommu_test_cmd, last), #endif diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 85498f14b3ae..44f9ddcfda58 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -47,6 +47,7 @@ enum { IOMMUFD_CMD_VFIO_IOAS, IOMMUFD_CMD_HWPT_ALLOC, IOMMUFD_CMD_HWPT_SET_DIRTY, + IOMMUFD_CMD_HWPT_GET_DIRTY_IOVA, }; /** @@ -406,4 +407,39 @@ struct iommu_hwpt_set_dirty { __u32 __reserved; }; #define IOMMU_HWPT_SET_DIRTY _IO(IOMMUFD_TYPE, IOMMUFD_CMD_HWPT_SET_DIRTY) + +/** + * struct iommufd_dirty_bitmap - Dirty IOVA tracking bitmap + * @iova: base IOVA of the bitmap + * @length: IOVA size + * @page_size: page size granularity of each bit in the bitmap + * @data: bitmap where to set the dirty bits. The bitmap bits each + * represent a page_size which you deviate from an arbitrary iova. + * Checking a given IOVA is dirty: + * + * data[(iova / page_size) / 64] & (1ULL << (iova % 64)) + */ +struct iommufd_dirty_data { + __aligned_u64 iova; + __aligned_u64 length; + __aligned_u64 page_size; + __aligned_u64 *data; +}; + +/** + * struct iommu_hwpt_get_dirty_iova - ioctl(IOMMU_HWPT_GET_DIRTY_IOVA) + * @size: sizeof(struct iommu_hwpt_get_dirty_iova) + * @hwpt_id: HW pagetable ID that represents the IOMMU domain. + * @flags: Flags to control dirty tracking status. + * @bitmap: Bitmap of the range of IOVA to read out + */ +struct iommu_hwpt_get_dirty_iova { + __u32 size; + __u32 hwpt_id; + __u32 flags; + __u32 __reserved; + struct iommufd_dirty_data bitmap; +}; +#define IOMMU_HWPT_GET_DIRTY_IOVA _IO(IOMMUFD_TYPE, IOMMUFD_CMD_HWPT_GET_DIRTY_IOVA) + #endif From patchwork Thu May 18 20:46:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247369 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BD2CC7EE2A for ; Thu, 18 May 2023 20:49:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229456AbjERUtP (ORCPT ); Thu, 18 May 2023 16:49:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230102AbjERUtL (ORCPT ); Thu, 18 May 2023 16:49:11 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D4B16137 for ; Thu, 18 May 2023 13:48:56 -0700 (PDT) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIx6tE032295; Thu, 18 May 2023 20:48:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=EFvSg79Rv2Ate9GlGq+dGjSzoLuLv3TJBFevauYPCxU=; b=t5kpYyBlTHtZhDwU5QQ/jMpfbCRHhF5tRHzzMS7iGqLa+aUNHozlghBlONbbwhj18CG3 5U/Oge/UdBNODe8DVx0FfMQgzJuSas5OOIZq3X0biDwKnHo4WqEns4mIuxJotGGJmAEK iNa/RpjyRTl9hQrlEDbdvKmwO0kpRqE/0ihmQPlrsnROuQIDuDBjQOZVgyC9SO4FR9ts 6ORWEI17DyHGK3gRU8IHOcDtaFm00YYuObPg0VgPiCBoW0ZzZS5D9gBlHMhTzYL5o3yW r9XalwNSwYFBAxQgckhno/v+jifa4pQtMGDAFebNEKakDG1POuCf3yCzI/1ATwoe9YQl cg== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qnkux92ms-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:34 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJJYQf032277; Thu, 18 May 2023 20:48:33 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10daf6e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:33 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE3B033533; Thu, 18 May 2023 20:48:32 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-13; Thu, 18 May 2023 20:48:32 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 12/24] iommufd/selftest: Test IOMMU_HWPT_GET_DIRTY_IOVA Date: Thu, 18 May 2023 21:46:38 +0100 Message-Id: <20230518204650.14541-13-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180171 X-Proofpoint-ORIG-GUID: JGWrncO-jNQMK83klwDTkHS58c5bwlxT X-Proofpoint-GUID: JGWrncO-jNQMK83klwDTkHS58c5bwlxT Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add a new test ioctl for simulating the dirty IOVAs in the mock domain, and implement the mock iommu domain ops that get the dirty tracking supported. The selftest exercises the usual main workflow of: 1) Setting dirty tracking from the iommu domain 2) Read and clear dirty IOPTEs Different fixtures will test different IOVA range sizes, that exercise corner cases of the bitmaps. Signed-off-by: Joao Martins --- drivers/iommu/iommufd/iommufd_test.h | 9 ++ drivers/iommu/iommufd/selftest.c | 97 ++++++++++++++- tools/testing/selftests/iommu/iommufd.c | 99 ++++++++++++++++ tools/testing/selftests/iommu/iommufd_utils.h | 112 ++++++++++++++++++ 4 files changed, 314 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/iommufd/iommufd_test.h b/drivers/iommu/iommufd/iommufd_test.h index 9abcc3231137..8c9f0d95ed94 100644 --- a/drivers/iommu/iommufd/iommufd_test.h +++ b/drivers/iommu/iommufd/iommufd_test.h @@ -18,6 +18,7 @@ enum { IOMMU_TEST_OP_ACCESS_RW, IOMMU_TEST_OP_SET_TEMP_MEMORY_LIMIT, IOMMU_TEST_OP_MOCK_DOMAIN_REPLACE, + IOMMU_TEST_OP_DIRTY, }; enum { @@ -96,6 +97,14 @@ struct iommu_test_cmd { struct { __u32 limit; } memory_limit; + struct { + __u32 flags; + __aligned_u64 iova; + __aligned_u64 length; + __aligned_u64 page_size; + __aligned_u64 uptr; + __aligned_u64 out_nr_dirty; + } dirty; }; __u32 last; }; diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c index ee7523c8d46a..3ec0eb4dfe97 100644 --- a/drivers/iommu/iommufd/selftest.c +++ b/drivers/iommu/iommufd/selftest.c @@ -34,6 +34,7 @@ enum { _MOCK_PFN_START = MOCK_PFN_MASK + 1, MOCK_PFN_START_IOVA = _MOCK_PFN_START, MOCK_PFN_LAST_IOVA = _MOCK_PFN_START, + MOCK_PFN_DIRTY_IOVA = _MOCK_PFN_START << 1, }; /* @@ -234,7 +235,7 @@ static size_t mock_domain_unmap_pages(struct iommu_domain *domain, for (cur = 0; cur != pgsize; cur += MOCK_IO_PAGE_SIZE) { ent = xa_erase(&mock->pfns, iova / MOCK_IO_PAGE_SIZE); - WARN_ON(!ent); + /* * iommufd generates unmaps that must be a strict * superset of the map's performend So every starting @@ -244,12 +245,12 @@ static size_t mock_domain_unmap_pages(struct iommu_domain *domain, * passed to map_pages */ if (first) { - WARN_ON(!(xa_to_value(ent) & + WARN_ON(ent && !(xa_to_value(ent) & MOCK_PFN_START_IOVA)); first = false; } if (pgcount == 1 && cur + MOCK_IO_PAGE_SIZE == pgsize) - WARN_ON(!(xa_to_value(ent) & + WARN_ON(ent && !(xa_to_value(ent) & MOCK_PFN_LAST_IOVA)); iova += MOCK_IO_PAGE_SIZE; @@ -303,6 +304,39 @@ static int mock_domain_set_dirty_tracking(struct iommu_domain *domain, return 0; } +static int mock_domain_read_and_clear_dirty(struct iommu_domain *domain, + unsigned long iova, size_t size, + unsigned long flags, + struct iommu_dirty_bitmap *dirty) +{ + struct mock_iommu_domain *mock = + container_of(domain, struct mock_iommu_domain, domain); + unsigned long i, max = size / MOCK_IO_PAGE_SIZE; + void *ent, *old; + + if (!(mock->flags & MOCK_DIRTY_TRACK) && dirty->bitmap) + return -EINVAL; + + for (i = 0; i < max; i++) { + unsigned long cur = iova + i * MOCK_IO_PAGE_SIZE; + + ent = xa_load(&mock->pfns, cur / MOCK_IO_PAGE_SIZE); + if (ent && + (xa_to_value(ent) & MOCK_PFN_DIRTY_IOVA)) { + unsigned long val; + + /* Clear dirty */ + val = xa_to_value(ent) & ~MOCK_PFN_DIRTY_IOVA; + old = xa_store(&mock->pfns, cur / MOCK_IO_PAGE_SIZE, + xa_mk_value(val), GFP_KERNEL); + WARN_ON_ONCE(ent != old); + iommu_dirty_bitmap_record(dirty, cur, MOCK_IO_PAGE_SIZE); + } + } + + return 0; +} + static const struct iommu_ops mock_ops = { .owner = THIS_MODULE, .supported_flags = IOMMU_DOMAIN_F_ENFORCE_DIRTY, @@ -318,6 +352,7 @@ static const struct iommu_ops mock_ops = { .unmap_pages = mock_domain_unmap_pages, .iova_to_phys = mock_domain_iova_to_phys, .set_dirty_tracking = mock_domain_set_dirty_tracking, + .read_and_clear_dirty = mock_domain_read_and_clear_dirty, }, }; @@ -994,6 +1029,56 @@ static_assert((unsigned int)MOCK_ACCESS_RW_WRITE == IOMMUFD_ACCESS_RW_WRITE); static_assert((unsigned int)MOCK_ACCESS_RW_SLOW_PATH == __IOMMUFD_ACCESS_RW_SLOW_PATH); +static int iommufd_test_dirty(struct iommufd_ucmd *ucmd, + unsigned int mockpt_id, unsigned long iova, + size_t length, unsigned long page_size, + void __user *uptr, u32 flags) +{ + unsigned long i, max = length / page_size; + struct iommu_test_cmd *cmd = ucmd->cmd; + struct iommufd_hw_pagetable *hwpt; + struct mock_iommu_domain *mock; + int rc, count = 0; + + if (iova % page_size || length % page_size || + (uintptr_t)uptr % page_size) + return -EINVAL; + + hwpt = get_md_pagetable(ucmd, mockpt_id, &mock); + if (IS_ERR(hwpt)) + return PTR_ERR(hwpt); + + if (!(mock->flags & MOCK_DIRTY_TRACK)) { + rc = -EINVAL; + goto out_put; + } + + for (i = 0; i < max; i++) { + unsigned long cur = iova + i * page_size; + void *ent, *old; + + if (!test_bit(i, (unsigned long *) uptr)) + continue; + + ent = xa_load(&mock->pfns, cur / page_size); + if (ent) { + unsigned long val; + + val = xa_to_value(ent) | MOCK_PFN_DIRTY_IOVA; + old = xa_store(&mock->pfns, cur / page_size, + xa_mk_value(val), GFP_KERNEL); + WARN_ON_ONCE(ent != old); + count++; + } + } + + cmd->dirty.out_nr_dirty = count; + rc = iommufd_ucmd_respond(ucmd, sizeof(*cmd)); +out_put: + iommufd_put_object(&hwpt->obj); + return rc; +} + void iommufd_selftest_destroy(struct iommufd_object *obj) { struct selftest_obj *sobj = container_of(obj, struct selftest_obj, obj); @@ -1055,6 +1140,12 @@ int iommufd_test(struct iommufd_ucmd *ucmd) return -EINVAL; iommufd_test_memory_limit = cmd->memory_limit.limit; return 0; + case IOMMU_TEST_OP_DIRTY: + return iommufd_test_dirty( + ucmd, cmd->id, cmd->dirty.iova, + cmd->dirty.length, cmd->dirty.page_size, + u64_to_user_ptr(cmd->dirty.uptr), + cmd->dirty.flags); default: return -EOPNOTSUPP; } diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selftests/iommu/iommufd.c index 8adccdde5ecc..818e78cd889a 100644 --- a/tools/testing/selftests/iommu/iommufd.c +++ b/tools/testing/selftests/iommu/iommufd.c @@ -12,6 +12,7 @@ static unsigned long HUGEPAGE_SIZE; #define MOCK_PAGE_SIZE (PAGE_SIZE / 2) +#define BITS_PER_BYTE 8 static unsigned long get_huge_page_size(void) { @@ -1361,13 +1362,47 @@ FIXTURE(iommufd_dirty_tracking) uint32_t hwpt_id; uint32_t stdev_id; uint32_t idev_id; + unsigned long page_size; + unsigned long bitmap_size; + void *bitmap; + void *buffer; +}; + +FIXTURE_VARIANT(iommufd_dirty_tracking) +{ + unsigned long buffer_size; }; FIXTURE_SETUP(iommufd_dirty_tracking) { + void *vrc; + int rc; + self->fd = open("/dev/iommu", O_RDWR); ASSERT_NE(-1, self->fd); + rc = posix_memalign(&self->buffer, HUGEPAGE_SIZE, variant->buffer_size); + if (rc || !self->buffer) { + SKIP(return, "Skipping buffer_size=%lu due to errno=%d", + variant->buffer_size, rc); + } + + assert((uintptr_t)self->buffer % HUGEPAGE_SIZE == 0); + vrc = mmap(self->buffer, variant->buffer_size, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_ANONYMOUS | MAP_FIXED, -1, 0); + assert(vrc == self->buffer); + + self->page_size = MOCK_PAGE_SIZE; + self->bitmap_size = variant->buffer_size / + self->page_size / BITS_PER_BYTE; + + /* Provision with an extra (MOCK_PAGE_SIZE) for the unaligned case */ + rc = posix_memalign(&self->bitmap, PAGE_SIZE, + self->bitmap_size + MOCK_PAGE_SIZE); + assert(!rc); + assert(self->bitmap); + assert((uintptr_t)self->bitmap % PAGE_SIZE == 0); + test_ioctl_ioas_alloc(&self->ioas_id); test_cmd_mock_domain(self->ioas_id, &self->stdev_id, &self->hwpt_id, &self->idev_id); @@ -1375,9 +1410,41 @@ FIXTURE_SETUP(iommufd_dirty_tracking) FIXTURE_TEARDOWN(iommufd_dirty_tracking) { + munmap(self->buffer, variant->buffer_size); + munmap(self->bitmap, self->bitmap_size); teardown_iommufd(self->fd, _metadata); } +FIXTURE_VARIANT_ADD(iommufd_dirty_tracking, domain_dirty128k) +{ + /* one u32 index bitmap */ + .buffer_size = 128UL * 1024UL, +}; + +FIXTURE_VARIANT_ADD(iommufd_dirty_tracking, domain_dirty256k) +{ + /* one u64 index bitmap */ + .buffer_size = 256UL * 1024UL, +}; + +FIXTURE_VARIANT_ADD(iommufd_dirty_tracking, domain_dirty640k) +{ + /* two u64 index and trailing end bitmap */ + .buffer_size = 640UL * 1024UL, +}; + +FIXTURE_VARIANT_ADD(iommufd_dirty_tracking, domain_dirty128M) +{ + /* 4K bitmap (128M IOVA range) */ + .buffer_size = 128UL * 1024UL * 1024UL, +}; + +FIXTURE_VARIANT_ADD(iommufd_dirty_tracking, domain_dirty256M) +{ + /* 8K bitmap (256M IOVA range) */ + .buffer_size = 256UL * 1024UL * 1024UL, +}; + TEST_F(iommufd_dirty_tracking, enforce_dirty) { uint32_t dev_flags = MOCK_FLAGS_DEVICE_NO_DIRTY; @@ -1408,6 +1475,38 @@ TEST_F(iommufd_dirty_tracking, set_dirty) test_ioctl_destroy(hwpt_id); } +TEST_F(iommufd_dirty_tracking, get_dirty_iova) +{ + uint32_t stddev_id; + uint32_t hwpt_id; + uint32_t ioas_id; + + test_ioctl_ioas_alloc(&ioas_id); + test_ioctl_ioas_map_fixed_id(ioas_id, self->buffer, + variant->buffer_size, + MOCK_APERTURE_START); + + test_cmd_hwpt_alloc(self->idev_id, ioas_id, + IOMMU_HWPT_ALLOC_ENFORCE_DIRTY, &hwpt_id); + test_cmd_mock_domain(hwpt_id, &stddev_id, NULL, NULL); + + test_cmd_set_dirty(hwpt_id, true); + + test_mock_dirty_bitmaps(hwpt_id, variant->buffer_size, + MOCK_APERTURE_START, + self->page_size, self->bitmap, + self->bitmap_size, _metadata); + + /* PAGE_SIZE unaligned bitmap */ + test_mock_dirty_bitmaps(hwpt_id, variant->buffer_size, + MOCK_APERTURE_START, + self->page_size, self->bitmap + MOCK_PAGE_SIZE, + self->bitmap_size, _metadata); + + test_ioctl_destroy(stddev_id); + test_ioctl_destroy(hwpt_id); +} + /* VFIO compatibility IOCTLs */ TEST_F(iommufd, simple_ioctls) diff --git a/tools/testing/selftests/iommu/iommufd_utils.h b/tools/testing/selftests/iommu/iommufd_utils.h index 3629c531ec9f..4d428fbb12e2 100644 --- a/tools/testing/selftests/iommu/iommufd_utils.h +++ b/tools/testing/selftests/iommu/iommufd_utils.h @@ -9,6 +9,8 @@ #include #include #include +#include +#include #include "../kselftest_harness.h" #include "../../../../drivers/iommu/iommufd/iommufd_test.h" @@ -143,6 +145,105 @@ static int _test_cmd_set_dirty(int fd, __u32 hwpt_id, bool enabled) #define test_cmd_set_dirty(hwpt_id, enabled) \ ASSERT_EQ(0, _test_cmd_set_dirty(self->fd, hwpt_id, enabled)) + +static int _test_cmd_get_dirty_iova(int fd, __u32 hwpt_id, size_t length, + __u64 iova, size_t page_size, __u64 *bitmap) +{ + struct iommu_hwpt_get_dirty_iova cmd = { + .size = sizeof(cmd), + .hwpt_id = hwpt_id, + .bitmap = { + .iova = iova, + .length = length, + .page_size = page_size, + .data = bitmap, + } + }; + int ret; + + ret = ioctl(fd, IOMMU_HWPT_GET_DIRTY_IOVA, &cmd); + if (ret) + return ret; + return 0; +} + +#define test_cmd_get_dirty_iova(fd, hwpt_id, length, iova, page_size, bitmap) \ + ASSERT_EQ(0, _test_cmd_get_dirty_iova(fd, hwpt_id, length, \ + iova, page_size, bitmap)) + +static int _test_cmd_mock_domain_set_dirty(int fd, __u32 hwpt_id, size_t length, + __u64 iova, size_t page_size, + __u64 *bitmap, __u64 *dirty) +{ + struct iommu_test_cmd cmd = { + .size = sizeof(cmd), + .op = IOMMU_TEST_OP_DIRTY, + .id = hwpt_id, + .dirty = { + .iova = iova, + .length = length, + .page_size = page_size, + .uptr = (uintptr_t) bitmap, + } + }; + int ret; + + ret = ioctl(fd, _IOMMU_TEST_CMD(IOMMU_TEST_OP_DIRTY), &cmd); + if (ret) + return -ret; + if (dirty) + *dirty = cmd.dirty.out_nr_dirty; + return 0; +} + +#define test_cmd_mock_domain_set_dirty(fd, hwpt_id, length, iova, page_size, bitmap, nr) \ + ASSERT_EQ(0, _test_cmd_mock_domain_set_dirty(fd, hwpt_id, \ + length, iova, \ + page_size, bitmap, \ + nr)) + +static int _test_mock_dirty_bitmaps(int fd, __u32 hwpt_id, size_t length, + __u64 iova, size_t page_size, + __u64 *bitmap, __u64 bitmap_size, + struct __test_metadata *_metadata) +{ + unsigned long i, count, nbits = bitmap_size * BITS_PER_BYTE; + unsigned long nr = nbits / 2; + __u64 out_dirty = 0; + + /* Mark all even bits as dirty in the mock domain */ + for (count = 0, i = 0; i < nbits; count += !(i%2), i++) + if (!(i % 2)) + __set_bit(i, (unsigned long *) bitmap); + ASSERT_EQ(nr, count); + + test_cmd_mock_domain_set_dirty(fd, hwpt_id, length, iova, page_size, + bitmap, &out_dirty); + ASSERT_EQ(nr, out_dirty); + + /* Expect all even bits as dirty in the user bitmap */ + memset(bitmap, 0, bitmap_size); + test_cmd_get_dirty_iova(fd, hwpt_id, length, iova, page_size, bitmap); + for (count = 0, i = 0; i < nbits; count += !(i%2), i++) + ASSERT_EQ(!(i % 2), test_bit(i, (unsigned long *) bitmap)); + ASSERT_EQ(count, out_dirty); + + memset(bitmap, 0, bitmap_size); + test_cmd_get_dirty_iova(fd, hwpt_id, length, iova, page_size, bitmap); + + /* It as read already -- expect all zeroes */ + for (i = 0; i < nbits; i++) + ASSERT_EQ(0, test_bit(i, (unsigned long *) bitmap)); + + return 0; +} +#define test_mock_dirty_bitmaps(hwpt_id, length, iova, page_size, bitmap, \ + bitmap_size, _metadata) \ + ASSERT_EQ(0, _test_mock_dirty_bitmaps(self->fd, hwpt_id, \ + length, iova, \ + page_size, bitmap, \ + bitmap_size, _metadata)) + static int _test_cmd_create_access(int fd, unsigned int ioas_id, __u32 *access_id, unsigned int flags) { @@ -267,6 +368,17 @@ static int _test_ioctl_ioas_map(int fd, unsigned int ioas_id, void *buffer, IOMMU_IOAS_MAP_READABLE)); \ }) +#define test_ioctl_ioas_map_fixed_id(ioas_id, buffer, length, iova) \ + ({ \ + __u64 __iova = iova; \ + ASSERT_EQ(0, _test_ioctl_ioas_map( \ + self->fd, ioas_id, buffer, length, \ + &__iova, \ + IOMMU_IOAS_MAP_FIXED_IOVA | \ + IOMMU_IOAS_MAP_WRITEABLE | \ + IOMMU_IOAS_MAP_READABLE)); \ + }) + #define test_err_ioctl_ioas_map_fixed(_errno, buffer, length, iova) \ ({ \ __u64 __iova = iova; \ From patchwork Thu May 18 20:46:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247368 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97A3DC77B73 for ; Thu, 18 May 2023 20:49:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229951AbjERUtN (ORCPT ); Thu, 18 May 2023 16:49:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230038AbjERUtK (ORCPT ); Thu, 18 May 2023 16:49:10 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 107B4EE for ; Thu, 18 May 2023 13:48:56 -0700 (PDT) Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIxAnN015445; Thu, 18 May 2023 20:48:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=CY7DPvw82imjKSegForCze2+T0wx2g71jXQTjdS3kYg=; b=yvwl24vgUG6Vclh39wgaMI7zyrY0fRZn6yhn5oPU1oDOi/knfoswgXMFSlpVhGth4JRc Jcj6QQDo/y31EthnCVwuq+6yxunWQLAvqHlLo0z3s7YMj0DSnvJpPR/cJDksSd1S0xNt i2LLtrB4Qe27aSPhqFGDkAIxwboc/YWYU0n2ayP6RG50W7JsshNr0Dj8OUzfOqUsWe8O m5XRsWqZ18xfUOPVPMlafVaRSHcCZydFDII06aVP7K8o9zPw+mfVHXfORJ7+/IYkWhGt i6B+vZReH79Mc0nbO29Q+/OPR8GSkKedCifcR7nyrjRgVWalRK6amqC0SI6c+aXjBvbs UA== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qmx8j3n57-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:38 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJPApx032227; Thu, 18 May 2023 20:48:37 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10daf8m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:37 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE3D033533; Thu, 18 May 2023 20:48:36 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-14; Thu, 18 May 2023 20:48:36 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 13/24] iommufd: Add IOMMU_DEVICE_GET_CAPS Date: Thu, 18 May 2023 21:46:39 +0100 Message-Id: <20230518204650.14541-14-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180171 X-Proofpoint-GUID: flZONY3ZXnpcPSNEL-7abX4_ZBbdmyhy X-Proofpoint-ORIG-GUID: flZONY3ZXnpcPSNEL-7abX4_ZBbdmyhy Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Add IOMMU_DEVICE_GET_CAPS op for querying iommu capabilities for a given device. Capabilities are IOMMU agnostic and use device_iommu_capable() API passing one of the IOMMU_CAP_*. Enumerate IOMMU_CAP_DIRTY for now in the out_caps field returned back to userspace. Signed-off-by: Joao Martins --- drivers/iommu/iommufd/device.c | 26 +++++++++++++++++++++++++ drivers/iommu/iommufd/iommufd_private.h | 1 + drivers/iommu/iommufd/main.c | 3 +++ include/uapi/linux/iommufd.h | 23 ++++++++++++++++++++++ 4 files changed, 53 insertions(+) diff --git a/drivers/iommu/iommufd/device.c b/drivers/iommu/iommufd/device.c index 48d1300f0350..63e2ffe21653 100644 --- a/drivers/iommu/iommufd/device.c +++ b/drivers/iommu/iommufd/device.c @@ -263,6 +263,32 @@ u32 iommufd_device_to_id(struct iommufd_device *idev) } EXPORT_SYMBOL_NS_GPL(iommufd_device_to_id, IOMMUFD); +int iommufd_device_get_caps(struct iommufd_ucmd *ucmd) +{ + struct iommu_device_get_caps *cmd = ucmd->cmd; + struct iommufd_object *obj; + struct iommufd_device *idev; + int rc; + + obj = iommufd_get_object(ucmd->ictx, cmd->dev_id, IOMMUFD_OBJ_DEVICE); + if (IS_ERR(obj)) + return PTR_ERR(obj); + + idev = container_of(obj, struct iommufd_device, obj); + + cmd->out_caps = 0; + if (device_iommu_capable(idev->dev, IOMMU_CAP_DIRTY)) + cmd->out_caps |= IOMMUFD_CAP_DIRTY_TRACKING; + + rc = iommufd_ucmd_respond(ucmd, sizeof(*cmd)); + if (rc) + goto out_put; + +out_put: + iommufd_put_object(obj); + return rc; +} + static int iommufd_group_setup_msi(struct iommufd_group *igroup, struct iommufd_hw_pagetable *hwpt) { diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h index 3de8046fee07..e5782459e4aa 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -246,6 +246,7 @@ int iommufd_option_rlimit_mode(struct iommu_option *cmd, int iommufd_vfio_ioas(struct iommufd_ucmd *ucmd); int iommufd_check_iova_range(struct iommufd_ioas *ioas, struct iommufd_dirty_data *bitmap); +int iommufd_device_get_caps(struct iommufd_ucmd *ucmd); /* * A HW pagetable is called an iommu_domain inside the kernel. This user object diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index f34b309a1baf..c4c6f900ef0a 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -279,6 +279,7 @@ union ucmd_buffer { struct iommu_vfio_ioas vfio_ioas; struct iommu_hwpt_set_dirty set_dirty; struct iommu_hwpt_get_dirty_iova get_dirty_iova; + struct iommu_device_get_caps get_caps; #ifdef CONFIG_IOMMUFD_TEST struct iommu_test_cmd test; #endif @@ -324,6 +325,8 @@ static const struct iommufd_ioctl_op iommufd_ioctl_ops[] = { struct iommu_hwpt_set_dirty, __reserved), IOCTL_OP(IOMMU_HWPT_GET_DIRTY_IOVA, iommufd_hwpt_get_dirty_iova, struct iommu_hwpt_get_dirty_iova, bitmap.data), + IOCTL_OP(IOMMU_DEVICE_GET_CAPS, iommufd_device_get_caps, + struct iommu_device_get_caps, out_caps), #ifdef CONFIG_IOMMUFD_TEST IOCTL_OP(IOMMU_TEST_CMD, iommufd_test, struct iommu_test_cmd, last), #endif diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 44f9ddcfda58..c256f7354867 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -48,6 +48,7 @@ enum { IOMMUFD_CMD_HWPT_ALLOC, IOMMUFD_CMD_HWPT_SET_DIRTY, IOMMUFD_CMD_HWPT_GET_DIRTY_IOVA, + IOMMUFD_CMD_DEVICE_GET_CAPS, }; /** @@ -442,4 +443,26 @@ struct iommu_hwpt_get_dirty_iova { }; #define IOMMU_HWPT_GET_DIRTY_IOVA _IO(IOMMUFD_TYPE, IOMMUFD_CMD_HWPT_GET_DIRTY_IOVA) + +/** + * enum iommufd_device_caps + * @IOMMU_CAP_DIRTY_TRACKING: IOMMU device support for dirty tracking + */ +enum iommufd_device_caps { + IOMMUFD_CAP_DIRTY_TRACKING = 1 << 0, +}; + +/* + * struct iommu_device_caps - ioctl(IOMMU_DEVICE_GET_CAPS) + * @size: sizeof(struct iommu_device_caps) + * @dev_id: the device to query + * @caps: IOMMU capabilities of the device + */ +struct iommu_device_get_caps { + __u32 size; + __u32 dev_id; + __aligned_u64 out_caps; +}; +#define IOMMU_DEVICE_GET_CAPS _IO(IOMMUFD_TYPE, IOMMUFD_CMD_DEVICE_GET_CAPS) + #endif From patchwork Thu May 18 20:46:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247374 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5C71C7EE2A for ; Thu, 18 May 2023 20:49:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229622AbjERUtn (ORCPT ); Thu, 18 May 2023 16:49:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57700 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229973AbjERUtk (ORCPT ); Thu, 18 May 2023 16:49:40 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C2631735 for ; Thu, 18 May 2023 13:49:14 -0700 (PDT) Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIxA4f015466; Thu, 18 May 2023 20:48:58 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=ik0nQef5BcCOZZtfW1MdDQzVXmMsarHrUpNkmcjWPVw=; b=0v7RRdoUi18G5JxKodk1mB6yGwEvGjsP+hdXpgg5YQMRR3mK+0EiEecqSdPrLK3Xg4iv EyYvvuDaktLYjgi7zvpDp0TX9iyeopyndH2fYzMZd38qKMWNi7cTtCUsmrSvzUuq8/ss pr18BlF4G+atA/vOckFt6Yj5++t3aVpwnAzoUoWZ10pg7XjXtITQdNjkJAJxhced9vlO FQTmATAnNCKZj5unhiVo/AttUUY0hCEdwh+hCeR22y0pmGP+sIJxK10IZNHoJ7+6DQQ8 FeJ+zJzYNt4Le9BxLzkTTgdIoOn9egxJY5FxnCRBS9VVkkZE2JmA6NheBIkHruj8jfXK qQ== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qmx8j3n5b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:57 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJFRxh032132; Thu, 18 May 2023 20:48:41 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10dafa9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:41 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE3F033533; Thu, 18 May 2023 20:48:40 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-15; Thu, 18 May 2023 20:48:40 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 14/24] iommufd/selftest: Test IOMMU_DEVICE_GET_CAPS Date: Thu, 18 May 2023 21:46:40 +0100 Message-Id: <20230518204650.14541-15-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180171 X-Proofpoint-GUID: bSS_9b94F57sYjNVj6g-8CDmDIJAYvYR X-Proofpoint-ORIG-GUID: bSS_9b94F57sYjNVj6g-8CDmDIJAYvYR Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Enumerate the capabilities from the mock device and test whether it advertises as expected. Include it as part of the iommufd_dirty_tracking suite. Signed-off-by: Joao Martins --- drivers/iommu/iommufd/selftest.c | 10 +++++++++- tools/testing/selftests/iommu/iommufd.c | 14 ++++++++++++++ tools/testing/selftests/iommu/iommufd_utils.h | 19 +++++++++++++++++++ 3 files changed, 42 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c index 3ec0eb4dfe97..d81a977bf3af 100644 --- a/drivers/iommu/iommufd/selftest.c +++ b/drivers/iommu/iommufd/selftest.c @@ -275,7 +275,15 @@ static phys_addr_t mock_domain_iova_to_phys(struct iommu_domain *domain, static bool mock_domain_capable(struct device *dev, enum iommu_cap cap) { - return cap == IOMMU_CAP_CACHE_COHERENCY; + switch (cap) { + case IOMMU_CAP_CACHE_COHERENCY: + case IOMMU_CAP_DIRTY: + return true; + default: + break; + } + + return false; } static void mock_domain_set_plaform_dma_ops(struct device *dev) diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selftests/iommu/iommufd.c index 818e78cd889a..dad1eca3aa09 100644 --- a/tools/testing/selftests/iommu/iommufd.c +++ b/tools/testing/selftests/iommu/iommufd.c @@ -1475,6 +1475,19 @@ TEST_F(iommufd_dirty_tracking, set_dirty) test_ioctl_destroy(hwpt_id); } +TEST_F(iommufd_dirty_tracking, device_dirty_capability) +{ + uint32_t stddev_id; + uint32_t hwpt_id; + + test_cmd_hwpt_alloc(self->idev_id, self->ioas_id, 0, &hwpt_id); + test_cmd_mock_domain(hwpt_id, &stddev_id, NULL, NULL); + test_cmd_get_device_caps(self->idev_id, IOMMUFD_CAP_DIRTY_TRACKING); + + test_ioctl_destroy(stddev_id); + test_ioctl_destroy(hwpt_id); +} + TEST_F(iommufd_dirty_tracking, get_dirty_iova) { uint32_t stddev_id; @@ -1507,6 +1520,7 @@ TEST_F(iommufd_dirty_tracking, get_dirty_iova) test_ioctl_destroy(hwpt_id); } + /* VFIO compatibility IOCTLs */ TEST_F(iommufd, simple_ioctls) diff --git a/tools/testing/selftests/iommu/iommufd_utils.h b/tools/testing/selftests/iommu/iommufd_utils.h index 4d428fbb12e2..e942bc781f34 100644 --- a/tools/testing/selftests/iommu/iommufd_utils.h +++ b/tools/testing/selftests/iommu/iommufd_utils.h @@ -146,6 +146,25 @@ static int _test_cmd_set_dirty(int fd, __u32 hwpt_id, bool enabled) #define test_cmd_set_dirty(hwpt_id, enabled) \ ASSERT_EQ(0, _test_cmd_set_dirty(self->fd, hwpt_id, enabled)) +static int _test_cmd_get_device_caps(int fd, __u32 dev_id, __u64 capability) +{ + struct iommu_device_get_caps cmd = { + .size = sizeof(cmd), + .dev_id = dev_id, + }; + int ret; + + ret = ioctl(fd, IOMMU_DEVICE_GET_CAPS, &cmd); + if (ret) + return ret; + + return cmd.out_caps & capability; +} + +#define test_cmd_get_device_caps(dev_id, expected) \ + ASSERT_EQ(expected, _test_cmd_get_device_caps(self->fd, dev_id, \ + expected)) + static int _test_cmd_get_dirty_iova(int fd, __u32 hwpt_id, size_t length, __u64 iova, size_t page_size, __u64 *bitmap) { From patchwork Thu May 18 20:46:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247370 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE555C77B73 for ; Thu, 18 May 2023 20:49:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230168AbjERUtb (ORCPT ); Thu, 18 May 2023 16:49:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57578 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229622AbjERUt3 (ORCPT ); Thu, 18 May 2023 16:49:29 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A9817170E for ; Thu, 18 May 2023 13:49:09 -0700 (PDT) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIx5IS032279; Thu, 18 May 2023 20:48:51 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=dgIJgLe0f4AFWEuVxcCq+v9L7jR0hnU2/PxyXCkvh9o=; b=MqkUnr7A91I4gDFzm0o7pqlkwC0CNqViM2MQMgbjmpUlnwiSRKwM6XuhCGe5fgzKA2Ys sj10UJdOZXmH5omQHXXutTfs0xr0nYZbpwqPI76/yC40RKYRPE1OTZZCqLAq/1jEhtLW LHaKDY9QD0ORBC2DG+FEfZSlwH7GZBZDkeROzSaPkgIzQhBs5Ax/rOACwPSxuJvaDL1s OP1UqC2Viuc127L3C+tt2wQgBQJOJKd9ITiOInXA8mWYr/M4Gw+iezLUcdW2NdfwCACt KecgKnfpjlAs36d0NHfXeo8u4eKUdbO4vTXOsm65lHo5uvXxrw6aUqvZKyOTTgCZJq11 gg== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qnkux92n8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:50 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJV8d6032116; Thu, 18 May 2023 20:48:45 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10dafc3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:45 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE3H033533; Thu, 18 May 2023 20:48:44 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-16; Thu, 18 May 2023 20:48:44 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 15/24] iommufd: Add a flag to skip clearing of IOPTE dirty Date: Thu, 18 May 2023 21:46:41 +0100 Message-Id: <20230518204650.14541-16-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180171 X-Proofpoint-ORIG-GUID: fQO7HOcsGn1E-efwFHvDeN3M0s6hON-Q X-Proofpoint-GUID: fQO7HOcsGn1E-efwFHvDeN3M0s6hON-Q Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org VFIO has an operation where you an unmap an IOVA while returning a bitmap with the dirty data. In reality the operation doesn't quite query the IO pagetables that the the PTE was dirty or not, but it marks as dirty in the bitmap anything that was mapped all in one operation. In IOMMUFD this equivalent can be done in two operations by querying with GET_DIRTY_IOVA followed by UNMAP_IOVA. However, this would incur two TLB flushes given that after clearing dirty bits IOMMU implementations require invalidating their IOTLB, plus another invalidation needed for the UNMAP. To allow dirty bits to be queried faster, we add a flag (IOMMU_GET_DIRTY_IOVA_NO_CLEAR) that requests to not clear the dirty bits from the PTE (but just reading them), under the expectation that the next operation is the unmap. An alternative is to unmap and just perpectually mark as dirty as that's the same behaviour as today. So here equivalent functionally can be provided, and if real dirty info is required we amortize the cost while querying. There's still a race against DMA where in theory the unmap of the IOVA (when the guest invalidates the IOTLB via emulated iommu) would race against the VF performing DMA on the same IOVA being invalidated which would be marking the PTE as dirty but losing the update in the unmap-related IOTLB flush. The way to actually prevent the race would be to write-protect the IOPTE, then query dirty bits and flush the IOTLB in the unmap after. However, this remains an issue that is so far theoretically possible, but lacks an use case or whether the race is relevant in the first place that justifies such complexity. Link: https://lore.kernel.org/linux-iommu/20220502185239.GR8364@nvidia.com/ Signed-off-by: Joao Martins --- drivers/iommu/iommufd/hw_pagetable.c | 3 ++- drivers/iommu/iommufd/io_pagetable.c | 9 +++++++-- include/uapi/linux/iommufd.h | 12 ++++++++++++ 3 files changed, 21 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/iommufd/hw_pagetable.c b/drivers/iommu/iommufd/hw_pagetable.c index 25860aa0a1f8..bcb81eefe16c 100644 --- a/drivers/iommu/iommufd/hw_pagetable.c +++ b/drivers/iommu/iommufd/hw_pagetable.c @@ -260,7 +260,8 @@ int iommufd_hwpt_get_dirty_iova(struct iommufd_ucmd *ucmd) struct iommufd_ioas *ioas; int rc = -EOPNOTSUPP; - if ((cmd->flags || cmd->__reserved)) + if ((cmd->flags & ~(IOMMU_GET_DIRTY_IOVA_NO_CLEAR)) || + cmd->__reserved) return -EOPNOTSUPP; hwpt = iommufd_get_hwpt(ucmd, cmd->hwpt_id); diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c index 01adb0f7e4d0..ef58910ec747 100644 --- a/drivers/iommu/iommufd/io_pagetable.c +++ b/drivers/iommu/iommufd/io_pagetable.c @@ -414,6 +414,7 @@ int iopt_map_user_pages(struct iommufd_ctx *ictx, struct io_pagetable *iopt, } struct iova_bitmap_fn_arg { + unsigned long flags; struct iommu_domain *domain; struct iommu_dirty_bitmap *dirty; }; @@ -426,8 +427,9 @@ static int __iommu_read_and_clear_dirty(struct iova_bitmap *bitmap, struct iommu_domain *domain = arg->domain; const struct iommu_domain_ops *ops = domain->ops; struct iommu_dirty_bitmap *dirty = arg->dirty; + unsigned long flags = arg->flags; - return ops->read_and_clear_dirty(domain, iova, length, 0, dirty); + return ops->read_and_clear_dirty(domain, iova, length, flags, dirty); } static int iommu_read_and_clear_dirty(struct iommu_domain *domain, @@ -451,11 +453,14 @@ static int iommu_read_and_clear_dirty(struct iommu_domain *domain, iommu_dirty_bitmap_init(&dirty, iter, &gather); + arg.flags = flags; arg.domain = domain; arg.dirty = &dirty; iova_bitmap_for_each(iter, &arg, __iommu_read_and_clear_dirty); - iommu_iotlb_sync(domain, &gather); + if (!(flags & IOMMU_DIRTY_NO_CLEAR)) + iommu_iotlb_sync(domain, &gather); + iova_bitmap_free(iter); return ret; diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index c256f7354867..a91bf845e679 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -427,6 +427,18 @@ struct iommufd_dirty_data { __aligned_u64 *data; }; +/** + * enum iommufd_get_dirty_iova_flags - Flags for getting dirty bits + * @IOMMU_GET_DIRTY_IOVA_NO_CLEAR: Just read the PTEs without clearing any dirty + * bits metadata. This flag can be passed in the + * expectation where the next operation is + * an unmap of the same IOVA range. + * + */ +enum iommufd_hwpt_get_dirty_iova_flags { + IOMMU_GET_DIRTY_IOVA_NO_CLEAR = 1, +}; + /** * struct iommu_hwpt_get_dirty_iova - ioctl(IOMMU_HWPT_GET_DIRTY_IOVA) * @size: sizeof(struct iommu_hwpt_get_dirty_iova) From patchwork Thu May 18 20:46:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247373 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E26B8C77B73 for ; Thu, 18 May 2023 20:49:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229869AbjERUtl (ORCPT ); Thu, 18 May 2023 16:49:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57724 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229622AbjERUth (ORCPT ); Thu, 18 May 2023 16:49:37 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0493F1727 for ; Thu, 18 May 2023 13:49:12 -0700 (PDT) Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIx57J025260; Thu, 18 May 2023 20:48:54 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=2D7UxbOumN8S0VQCXY9A6pctFVNdyDKDTmUQXX+6IOs=; b=aT8NjB7M4OPvcKWeYlK5EhZySaCUitE5tkeVu47DQXbB4M571j8XWwxVY7zE7UNKCwc3 jPRHHVKUmnvuZQpQa7gFQq4ZdJ2CCgJzN+aT+Uxl6J9EKr/FPSNPspIb+GTCffuz4mHd WkykXXuGjJEZKLennSyKWw4Jy6H21lBPzRB29kpAzyWe3TpadxxGTmcf1DCux4Z6Cf9W eYUSPJNta47HFZQQZnU9DDYGl5Wmt0xIgJ4jpHF1RJ1wFL4KlYNC8eiiycwu4v52g4Vo 7HGYsvBbE9QhFXukju6KmyjtD6LAysJAF4MF/83eB1LAUIrN7qTnSz5NXRPsEl8mNF4v jQ== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qj1fc97ef-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:53 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJQch1032127; Thu, 18 May 2023 20:48:50 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10dafe3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:49 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE3J033533; Thu, 18 May 2023 20:48:49 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-17; Thu, 18 May 2023 20:48:48 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 16/24] iommufd/selftest: Test IOMMU_GET_DIRTY_IOVA_NO_CLEAR flag Date: Thu, 18 May 2023 21:46:42 +0100 Message-Id: <20230518204650.14541-17-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=899 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180171 X-Proofpoint-ORIG-GUID: Q7m0MiiD7sC4JYr4zEqgxCysLj3rQHmU X-Proofpoint-GUID: Q7m0MiiD7sC4JYr4zEqgxCysLj3rQHmU Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Change test_mock_dirty_bitmaps() to pass a flag where we specify the flag under test. The test does the same thing as the GET_DIRTY_IOVA regular test. Except that we test whether the bits we dirtied are fetched all the same a second time as opposed to observing them cleared. Signed-off-by: Joao Martins --- drivers/iommu/iommufd/selftest.c | 15 ++++--- tools/testing/selftests/iommu/iommufd.c | 40 ++++++++++++++++++- tools/testing/selftests/iommu/iommufd_utils.h | 26 +++++++----- 3 files changed, 64 insertions(+), 17 deletions(-) diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c index d81a977bf3af..ae8e94259b21 100644 --- a/drivers/iommu/iommufd/selftest.c +++ b/drivers/iommu/iommufd/selftest.c @@ -331,13 +331,16 @@ static int mock_domain_read_and_clear_dirty(struct iommu_domain *domain, ent = xa_load(&mock->pfns, cur / MOCK_IO_PAGE_SIZE); if (ent && (xa_to_value(ent) & MOCK_PFN_DIRTY_IOVA)) { - unsigned long val; - /* Clear dirty */ - val = xa_to_value(ent) & ~MOCK_PFN_DIRTY_IOVA; - old = xa_store(&mock->pfns, cur / MOCK_IO_PAGE_SIZE, - xa_mk_value(val), GFP_KERNEL); - WARN_ON_ONCE(ent != old); + if (!(flags & IOMMU_GET_DIRTY_IOVA_NO_CLEAR)) { + unsigned long val; + + val = xa_to_value(ent) & ~MOCK_PFN_DIRTY_IOVA; + old = xa_store(&mock->pfns, + cur / MOCK_IO_PAGE_SIZE, + xa_mk_value(val), GFP_KERNEL); + WARN_ON_ONCE(ent != old); + } iommu_dirty_bitmap_record(dirty, cur, MOCK_IO_PAGE_SIZE); } } diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selftests/iommu/iommufd.c index dad1eca3aa09..7ee788ce80c8 100644 --- a/tools/testing/selftests/iommu/iommufd.c +++ b/tools/testing/selftests/iommu/iommufd.c @@ -1508,13 +1508,49 @@ TEST_F(iommufd_dirty_tracking, get_dirty_iova) test_mock_dirty_bitmaps(hwpt_id, variant->buffer_size, MOCK_APERTURE_START, self->page_size, self->bitmap, - self->bitmap_size, _metadata); + self->bitmap_size, 0, _metadata); /* PAGE_SIZE unaligned bitmap */ test_mock_dirty_bitmaps(hwpt_id, variant->buffer_size, MOCK_APERTURE_START, self->page_size, self->bitmap + MOCK_PAGE_SIZE, - self->bitmap_size, _metadata); + self->bitmap_size, 0, _metadata); + + test_ioctl_destroy(stddev_id); + test_ioctl_destroy(hwpt_id); +} + +TEST_F(iommufd_dirty_tracking, get_dirty_iova_no_clear) +{ + uint32_t stddev_id; + uint32_t hwpt_id; + uint32_t ioas_id; + + test_ioctl_ioas_alloc(&ioas_id); + test_ioctl_ioas_map_fixed_id(ioas_id, self->buffer, + variant->buffer_size, + MOCK_APERTURE_START); + + test_cmd_hwpt_alloc(self->idev_id, ioas_id, + IOMMU_HWPT_ALLOC_ENFORCE_DIRTY, &hwpt_id); + test_cmd_mock_domain(hwpt_id, &stddev_id, NULL, NULL); + + test_cmd_set_dirty(hwpt_id, true); + + test_mock_dirty_bitmaps(hwpt_id, variant->buffer_size, + MOCK_APERTURE_START, + self->page_size, self->bitmap, + self->bitmap_size, + IOMMU_GET_DIRTY_IOVA_NO_CLEAR, + _metadata); + + /* Unaligned bitmap */ + test_mock_dirty_bitmaps(hwpt_id, variant->buffer_size, + MOCK_APERTURE_START, + self->page_size, self->bitmap + MOCK_PAGE_SIZE, + self->bitmap_size, + IOMMU_GET_DIRTY_IOVA_NO_CLEAR, + _metadata); test_ioctl_destroy(stddev_id); test_ioctl_destroy(hwpt_id); diff --git a/tools/testing/selftests/iommu/iommufd_utils.h b/tools/testing/selftests/iommu/iommufd_utils.h index e942bc781f34..1c0b942bcb4a 100644 --- a/tools/testing/selftests/iommu/iommufd_utils.h +++ b/tools/testing/selftests/iommu/iommufd_utils.h @@ -166,11 +166,13 @@ static int _test_cmd_get_device_caps(int fd, __u32 dev_id, __u64 capability) expected)) static int _test_cmd_get_dirty_iova(int fd, __u32 hwpt_id, size_t length, - __u64 iova, size_t page_size, __u64 *bitmap) + __u64 iova, size_t page_size, __u64 *bitmap, + __u32 flags) { struct iommu_hwpt_get_dirty_iova cmd = { .size = sizeof(cmd), .hwpt_id = hwpt_id, + .flags = flags, .bitmap = { .iova = iova, .length = length, @@ -186,9 +188,10 @@ static int _test_cmd_get_dirty_iova(int fd, __u32 hwpt_id, size_t length, return 0; } -#define test_cmd_get_dirty_iova(fd, hwpt_id, length, iova, page_size, bitmap) \ +#define test_cmd_get_dirty_iova(fd, hwpt_id, length, iova, page_size, bitmap, \ + flags) \ ASSERT_EQ(0, _test_cmd_get_dirty_iova(fd, hwpt_id, length, \ - iova, page_size, bitmap)) + iova, page_size, bitmap, flags)) static int _test_cmd_mock_domain_set_dirty(int fd, __u32 hwpt_id, size_t length, __u64 iova, size_t page_size, @@ -224,6 +227,7 @@ static int _test_cmd_mock_domain_set_dirty(int fd, __u32 hwpt_id, size_t length, static int _test_mock_dirty_bitmaps(int fd, __u32 hwpt_id, size_t length, __u64 iova, size_t page_size, __u64 *bitmap, __u64 bitmap_size, + __u32 flags, struct __test_metadata *_metadata) { unsigned long i, count, nbits = bitmap_size * BITS_PER_BYTE; @@ -242,26 +246,30 @@ static int _test_mock_dirty_bitmaps(int fd, __u32 hwpt_id, size_t length, /* Expect all even bits as dirty in the user bitmap */ memset(bitmap, 0, bitmap_size); - test_cmd_get_dirty_iova(fd, hwpt_id, length, iova, page_size, bitmap); + test_cmd_get_dirty_iova(fd, hwpt_id, length, iova, + page_size, bitmap, flags); for (count = 0, i = 0; i < nbits; count += !(i%2), i++) ASSERT_EQ(!(i % 2), test_bit(i, (unsigned long *) bitmap)); ASSERT_EQ(count, out_dirty); memset(bitmap, 0, bitmap_size); - test_cmd_get_dirty_iova(fd, hwpt_id, length, iova, page_size, bitmap); + test_cmd_get_dirty_iova(fd, hwpt_id, length, iova, + page_size, bitmap, flags); /* It as read already -- expect all zeroes */ - for (i = 0; i < nbits; i++) - ASSERT_EQ(0, test_bit(i, (unsigned long *) bitmap)); + for (i = 0; i < nbits; i++) { + ASSERT_EQ(!(i % 2) && (flags & IOMMU_GET_DIRTY_IOVA_NO_CLEAR), + test_bit(i, (unsigned long *) bitmap)); + } return 0; } #define test_mock_dirty_bitmaps(hwpt_id, length, iova, page_size, bitmap, \ - bitmap_size, _metadata) \ + bitmap_size, flags, _metadata) \ ASSERT_EQ(0, _test_mock_dirty_bitmaps(self->fd, hwpt_id, \ length, iova, \ page_size, bitmap, \ - bitmap_size, _metadata)) + bitmap_size, flags, _metadata)) static int _test_cmd_create_access(int fd, unsigned int ioas_id, __u32 *access_id, unsigned int flags) From patchwork Thu May 18 20:46:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247379 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F012C77B73 for ; Thu, 18 May 2023 20:50:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230377AbjERUu3 (ORCPT ); Thu, 18 May 2023 16:50:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58408 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230147AbjERUuY (ORCPT ); Thu, 18 May 2023 16:50:24 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 04857189 for ; Thu, 18 May 2023 13:50:00 -0700 (PDT) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIxYl6012418; Thu, 18 May 2023 20:49:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=rarZQdWfoONRlaZJKhm0wS9rPiQ9FWSS3CiTbDMh+yU=; b=YVl0/JuyzGQPa3yhr/6wz2rnEVpyNvFWr1qXsIdvuyHB6eNJnQhhQtq1HJrvVX16R4AB zTva1YZxeRFqI2QpE2CAO1L+42y1vRDY6JGEWFa7erAhNLPmrPO9HRkvNxKjNgq1yVZJ tPJ7JfQ43xF73kjJoU/TQEEESUkpzcvJcoBF/Gtx04EQadqyvtPjejA05NuPexIGqCnn nmzmXBey0qeGvF5Jq1Z/Ey54X4mINOcLQQL3ue1LVjSQ5yY34hdWs6+/rLj8DeoPBwEy 8VwTe6SBRcuW3SieAjDcVpe+G3W28lAGwCE4nCJlaac0pLhM9qVaeAOXS2E7Yo/QsF7u 8g== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qmxfc3k04-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:55 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJ1gqV032153; Thu, 18 May 2023 20:48:53 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10daffx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:53 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE3L033533; Thu, 18 May 2023 20:48:53 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-18; Thu, 18 May 2023 20:48:52 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 17/24] iommu/amd: Access/Dirty bit support in IOPTEs Date: Thu, 18 May 2023 21:46:43 +0100 Message-Id: <20230518204650.14541-18-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180171 X-Proofpoint-GUID: RaRMpkRs2DAbAA8PItrBG-mVO30-SbQQ X-Proofpoint-ORIG-GUID: RaRMpkRs2DAbAA8PItrBG-mVO30-SbQQ Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org IOMMU advertises Access/Dirty bits if the extended feature register reports it. Relevant AMD IOMMU SDM ref[0] "1.3.8 Enhanced Support for Access and Dirty Bits" To enable it set the DTE flag in bits 7 and 8 to enable access, or access+dirty. With that, the IOMMU starts marking the D and A flags on every Memory Request or ATS translation request. It is on the VMM side to steer whether to enable dirty tracking or not, rather than wrongly doing in IOMMU. Relevant AMD IOMMU SDM ref [0], "Table 7. Device Table Entry (DTE) Field Definitions" particularly the entry "HAD". To actually toggle on and off it's relatively simple as it's setting 2 bits on DTE and flush the device DTE cache. To get what's dirtied use existing AMD io-pgtable support, by walking the pagetables over each IOVA, with fetch_pte(). The IOTLB flushing is left to the caller (much like unmap), and iommu_dirty_bitmap_record() is the one adding page-ranges to invalidate. This allows caller to batch the flush over a big span of IOVA space, without the iommu wondering about when to flush. Worthwhile sections from AMD IOMMU SDM: "2.2.3.1 Host Access Support" "2.2.3.2 Host Dirty Support" For details on how IOMMU hardware updates the dirty bit see, and expects from its consequent clearing by CPU: "2.2.7.4 Updating Accessed and Dirty Bits in the Guest Address Tables" "2.2.7.5 Clearing Accessed and Dirty Bits" Quoting the SDM: "The setting of accessed and dirty status bits in the page tables is visible to both the CPU and the peripheral when sharing guest page tables. The IOMMU interlocked operations to update A and D bits must be 64-bit operations and naturally aligned on a 64-bit boundary" .. and for the IOMMU update sequence to Dirty bit, essentially is states: 1. Decodes the read and write intent from the memory access. 2. If P=0 in the page descriptor, fail the access. 3. Compare the A & D bits in the descriptor with the read and write intent in the request. 4. If the A or D bits need to be updated in the descriptor: * Start atomic operation. * Read the descriptor as a 64-bit access. * If the descriptor no longer appears to require an update, release the atomic lock with no further action and continue to step 5. * Calculate the new A & D bits. * Write the descriptor as a 64-bit access. * End atomic operation. 5. Continue to the next stage of translation or to the memory access. Access/Dirty bits readout also need to consider the non-default page-sizes (aka replicated PTEs as mentined by manual), as AMD supports all powers of two (except 512G) page sizes. Signed-off-by: Joao Martins --- drivers/iommu/amd/amd_iommu.h | 1 + drivers/iommu/amd/amd_iommu_types.h | 12 +++++ drivers/iommu/amd/init.c | 5 ++ drivers/iommu/amd/io_pgtable.c | 84 +++++++++++++++++++++++++++++ drivers/iommu/amd/iommu.c | 81 ++++++++++++++++++++++++++++ 5 files changed, 183 insertions(+) diff --git a/drivers/iommu/amd/amd_iommu.h b/drivers/iommu/amd/amd_iommu.h index e98f20a9bdd8..62567f275878 100644 --- a/drivers/iommu/amd/amd_iommu.h +++ b/drivers/iommu/amd/amd_iommu.h @@ -35,6 +35,7 @@ extern int amd_iommu_enable_faulting(void); extern int amd_iommu_guest_ir; extern enum io_pgtable_fmt amd_iommu_pgtable; extern int amd_iommu_gpt_level; +extern bool amd_iommu_had_support; /* IOMMUv2 specific functions */ struct iommu_domain; diff --git a/drivers/iommu/amd/amd_iommu_types.h b/drivers/iommu/amd/amd_iommu_types.h index 2ddbda3a4374..3138c257338d 100644 --- a/drivers/iommu/amd/amd_iommu_types.h +++ b/drivers/iommu/amd/amd_iommu_types.h @@ -97,7 +97,9 @@ #define FEATURE_GATS_MASK (3ULL) #define FEATURE_GAM_VAPIC (1ULL<<21) #define FEATURE_GIOSUP (1ULL<<48) +#define FEATURE_HASUP (1ULL<<49) #define FEATURE_EPHSUP (1ULL<<50) +#define FEATURE_HDSUP (1ULL<<52) #define FEATURE_SNP (1ULL<<63) #define FEATURE_PASID_SHIFT 32 @@ -208,6 +210,7 @@ /* macros and definitions for device table entries */ #define DEV_ENTRY_VALID 0x00 #define DEV_ENTRY_TRANSLATION 0x01 +#define DEV_ENTRY_HAD 0x07 #define DEV_ENTRY_PPR 0x34 #define DEV_ENTRY_IR 0x3d #define DEV_ENTRY_IW 0x3e @@ -366,10 +369,16 @@ #define PTE_LEVEL_PAGE_SIZE(level) \ (1ULL << (12 + (9 * (level)))) +/* + * The IOPTE dirty bit + */ +#define IOMMU_PTE_HD_BIT (6) + /* * Bit value definition for I/O PTE fields */ #define IOMMU_PTE_PR (1ULL << 0) +#define IOMMU_PTE_HD (1ULL << IOMMU_PTE_HD_BIT) #define IOMMU_PTE_U (1ULL << 59) #define IOMMU_PTE_FC (1ULL << 60) #define IOMMU_PTE_IR (1ULL << 61) @@ -380,6 +389,7 @@ */ #define DTE_FLAG_V (1ULL << 0) #define DTE_FLAG_TV (1ULL << 1) +#define DTE_FLAG_HAD (3ULL << 7) #define DTE_FLAG_IR (1ULL << 61) #define DTE_FLAG_IW (1ULL << 62) @@ -409,6 +419,7 @@ #define IOMMU_PAGE_MASK (((1ULL << 52) - 1) & ~0xfffULL) #define IOMMU_PTE_PRESENT(pte) ((pte) & IOMMU_PTE_PR) +#define IOMMU_PTE_DIRTY(pte) ((pte) & IOMMU_PTE_HD) #define IOMMU_PTE_PAGE(pte) (iommu_phys_to_virt((pte) & IOMMU_PAGE_MASK)) #define IOMMU_PTE_MODE(pte) (((pte) >> 9) & 0x07) @@ -559,6 +570,7 @@ struct protection_domain { int nid; /* Node ID */ u64 *gcr3_tbl; /* Guest CR3 table */ unsigned long flags; /* flags to find out type of domain */ + bool dirty_tracking; /* dirty tracking is enabled in the domain */ unsigned dev_cnt; /* devices assigned to this domain */ unsigned dev_iommu[MAX_IOMMUS]; /* per-IOMMU reference count */ }; diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c index 329a406cc37d..082f47e22c6e 100644 --- a/drivers/iommu/amd/init.c +++ b/drivers/iommu/amd/init.c @@ -151,6 +151,7 @@ struct ivmd_header { bool amd_iommu_dump; bool amd_iommu_irq_remap __read_mostly; +bool amd_iommu_had_support __read_mostly; enum io_pgtable_fmt amd_iommu_pgtable = AMD_IOMMU_V1; /* Guest page table level */ @@ -2201,6 +2202,10 @@ static int __init amd_iommu_init_pci(void) for_each_iommu(iommu) iommu_flush_all_caches(iommu); + if (check_feature_on_all_iommus(FEATURE_HASUP) && + check_feature_on_all_iommus(FEATURE_HDSUP)) + amd_iommu_had_support = true; + print_iommu_info(); out: diff --git a/drivers/iommu/amd/io_pgtable.c b/drivers/iommu/amd/io_pgtable.c index 666b643106f8..63b4b3ae7c71 100644 --- a/drivers/iommu/amd/io_pgtable.c +++ b/drivers/iommu/amd/io_pgtable.c @@ -486,6 +486,89 @@ static phys_addr_t iommu_v1_iova_to_phys(struct io_pgtable_ops *ops, unsigned lo return (__pte & ~offset_mask) | (iova & offset_mask); } +static bool pte_test_dirty(u64 *ptep, unsigned long size) +{ + bool dirty = false; + int i, count; + + /* + * 2.2.3.2 Host Dirty Support + * When a non-default page size is used , software must OR the + * Dirty bits in all of the replicated host PTEs used to map + * the page. The IOMMU does not guarantee the Dirty bits are + * set in all of the replicated PTEs. Any portion of the page + * may have been written even if the Dirty bit is set in only + * one of the replicated PTEs. + */ + count = PAGE_SIZE_PTE_COUNT(size); + for (i = 0; i < count; i++) { + if (test_bit(IOMMU_PTE_HD_BIT, (unsigned long *) &ptep[i])) { + dirty = true; + break; + } + } + + return dirty; +} + +static bool pte_test_and_clear_dirty(u64 *ptep, unsigned long size) +{ + bool dirty = false; + int i, count; + + /* + * 2.2.3.2 Host Dirty Support + * When a non-default page size is used , software must OR the + * Dirty bits in all of the replicated host PTEs used to map + * the page. The IOMMU does not guarantee the Dirty bits are + * set in all of the replicated PTEs. Any portion of the page + * may have been written even if the Dirty bit is set in only + * one of the replicated PTEs. + */ + count = PAGE_SIZE_PTE_COUNT(size); + for (i = 0; i < count; i++) + if (test_and_clear_bit(IOMMU_PTE_HD_BIT, + (unsigned long *) &ptep[i])) + dirty = true; + + return dirty; +} + +static int iommu_v1_read_and_clear_dirty(struct io_pgtable_ops *ops, + unsigned long iova, size_t size, + unsigned long flags, + struct iommu_dirty_bitmap *dirty) +{ + struct amd_io_pgtable *pgtable = io_pgtable_ops_to_data(ops); + unsigned long end = iova + size - 1; + + do { + unsigned long pgsize = 0; + u64 *ptep, pte; + + ptep = fetch_pte(pgtable, iova, &pgsize); + if (ptep) + pte = READ_ONCE(*ptep); + if (!ptep || !IOMMU_PTE_PRESENT(pte)) { + pgsize = pgsize ?: PTE_LEVEL_PAGE_SIZE(0); + iova += pgsize; + continue; + } + + /* + * Mark the whole IOVA range as dirty even if only one of + * the replicated PTEs were marked dirty. + */ + if (((flags & IOMMU_DIRTY_NO_CLEAR) && + pte_test_dirty(ptep, pgsize)) || + pte_test_and_clear_dirty(ptep, pgsize)) + iommu_dirty_bitmap_record(dirty, iova, pgsize); + iova += pgsize; + } while (iova < end); + + return 0; +} + /* * ---------------------------------------------------- */ @@ -526,6 +609,7 @@ static struct io_pgtable *v1_alloc_pgtable(struct io_pgtable_cfg *cfg, void *coo pgtable->iop.ops.map_pages = iommu_v1_map_pages; pgtable->iop.ops.unmap_pages = iommu_v1_unmap_pages; pgtable->iop.ops.iova_to_phys = iommu_v1_iova_to_phys; + pgtable->iop.ops.read_and_clear_dirty = iommu_v1_read_and_clear_dirty; return &pgtable->iop; } diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index 4a314647d1f7..ddb92005f018 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -1586,6 +1586,9 @@ static void set_dte_entry(struct amd_iommu *iommu, u16 devid, pte_root |= 1ULL << DEV_ENTRY_PPR; } + if (domain->dirty_tracking) + pte_root |= DTE_FLAG_HAD; + if (domain->flags & PD_IOMMUV2_MASK) { u64 gcr3 = iommu_virt_to_phys(domain->gcr3_tbl); u64 glx = domain->glx; @@ -2177,6 +2180,10 @@ static int amd_iommu_attach_device(struct iommu_domain *dom, dev_data->defer_attach = false; + if (dom->flags & IOMMU_DOMAIN_F_ENFORCE_DIRTY && + (iommu && !(iommu->features & FEATURE_HDSUP))) + return -EINVAL; + if (dev_data->domain) detach_device(dev); @@ -2293,6 +2300,8 @@ static bool amd_iommu_capable(struct device *dev, enum iommu_cap cap) return amdr_ivrs_remap_support; case IOMMU_CAP_ENFORCE_CACHE_COHERENCY: return true; + case IOMMU_CAP_DIRTY: + return amd_iommu_had_support; default: break; } @@ -2300,6 +2309,75 @@ static bool amd_iommu_capable(struct device *dev, enum iommu_cap cap) return false; } +static int amd_iommu_set_dirty_tracking(struct iommu_domain *domain, + bool enable) +{ + struct protection_domain *pdomain = to_pdomain(domain); + struct dev_table_entry *dev_table; + struct iommu_dev_data *dev_data; + struct amd_iommu *iommu; + unsigned long flags; + u64 pte_root; + + spin_lock_irqsave(&pdomain->lock, flags); + if (!(pdomain->dirty_tracking ^ enable)) { + spin_unlock_irqrestore(&pdomain->lock, flags); + return 0; + } + + list_for_each_entry(dev_data, &pdomain->dev_list, list) { + iommu = rlookup_amd_iommu(dev_data->dev); + if (!iommu) + continue; + + dev_table = get_dev_table(iommu); + pte_root = dev_table[dev_data->devid].data[0]; + + pte_root = (enable ? + pte_root | DTE_FLAG_HAD : pte_root & ~DTE_FLAG_HAD); + + /* Flush device DTE */ + dev_table[dev_data->devid].data[0] = pte_root; + device_flush_dte(dev_data); + } + + /* Flush IOTLB to mark IOPTE dirty on the next translation(s) */ + amd_iommu_domain_flush_tlb_pde(pdomain); + amd_iommu_domain_flush_complete(pdomain); + pdomain->dirty_tracking = enable; + spin_unlock_irqrestore(&pdomain->lock, flags); + + return 0; +} + +static int amd_iommu_read_and_clear_dirty(struct iommu_domain *domain, + unsigned long iova, size_t size, + unsigned long flags, + struct iommu_dirty_bitmap *dirty) +{ + struct protection_domain *pdomain = to_pdomain(domain); + struct io_pgtable_ops *ops = &pdomain->iop.iop.ops; + unsigned long lflags; + int ret; + + if (!ops || !ops->read_and_clear_dirty) + return -EOPNOTSUPP; + + spin_lock_irqsave(&pdomain->lock, lflags); + if (!pdomain->dirty_tracking && dirty->bitmap) { + spin_unlock_irqrestore(&pdomain->lock, lflags); + return -EINVAL; + } + spin_unlock_irqrestore(&pdomain->lock, lflags); + + rcu_read_lock(); + ret = ops->read_and_clear_dirty(ops, iova, size, flags, dirty); + rcu_read_unlock(); + + return ret; +} + + static void amd_iommu_get_resv_regions(struct device *dev, struct list_head *head) { @@ -2432,6 +2510,7 @@ const struct iommu_ops amd_iommu_ops = { .get_resv_regions = amd_iommu_get_resv_regions, .is_attach_deferred = amd_iommu_is_attach_deferred, .pgsize_bitmap = AMD_IOMMU_PGSIZES, + .supported_flags = IOMMU_DOMAIN_F_ENFORCE_DIRTY, .def_domain_type = amd_iommu_def_domain_type, .default_domain_ops = &(const struct iommu_domain_ops) { .attach_dev = amd_iommu_attach_device, @@ -2443,6 +2522,8 @@ const struct iommu_ops amd_iommu_ops = { .iotlb_sync = amd_iommu_iotlb_sync, .free = amd_iommu_domain_free, .enforce_cache_coherency = amd_iommu_enforce_cache_coherency, + .set_dirty_tracking = amd_iommu_set_dirty_tracking, + .read_and_clear_dirty = amd_iommu_read_and_clear_dirty, } }; From patchwork Thu May 18 20:46:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247375 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39097C77B7A for ; Thu, 18 May 2023 20:49:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230049AbjERUtp (ORCPT ); Thu, 18 May 2023 16:49:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57802 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229717AbjERUto (ORCPT ); Thu, 18 May 2023 16:49:44 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ABF62E7D for ; Thu, 18 May 2023 13:49:16 -0700 (PDT) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIx4mR032260; Thu, 18 May 2023 20:48:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=nSr+1XoQLx2Xe3M6TjCM5Ex2L+ELRf1SGpP6Wj+ieq8=; b=HB2l1AoR/Zvx2tyllH5b900dBv8C7QXOUuHcsHU8WKiuGuftVqBQNqseNQoeZabSmxBx GuuO+eZD/yPS3SYyw7xGLLxLFsWDviRsIbBElUFE83Cuuo6PsltDgJh4I16+YJq5FElp vdG/h2yak8i8iWIcQmvY0/cjPSzW1ZY6wxcpMbjORlOxoo5VsE543zUqDNzWAz1sYnoh Gz3L+WBUEmBuwOzoBb+TebPdKc3gkaBIND/fXmH0umvwfjlAe3ynEg/FunbJHgNBKlrQ w1RBqWrnn2e7imQE3nWxfllWTO6yTzeAU0twBJ1Zbv9hzl7YVWGYw1ikT2z0vre6OpIh gw== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qnkux92nm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:58 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IKfQLF032131; Thu, 18 May 2023 20:48:57 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10dafjh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:48:57 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE3N033533; Thu, 18 May 2023 20:48:57 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-19; Thu, 18 May 2023 20:48:56 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 18/24] iommu/amd: Print access/dirty bits if supported Date: Thu, 18 May 2023 21:46:44 +0100 Message-Id: <20230518204650.14541-19-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180171 X-Proofpoint-ORIG-GUID: zNCrPmLGMZHZrwx0YhQB_vYuEgxrssT1 X-Proofpoint-GUID: zNCrPmLGMZHZrwx0YhQB_vYuEgxrssT1 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Print the feature, much like other kernel-supported features. One can still probe its actual hw support via sysfs, regardless of what the kernel does. Signed-off-by: Joao Martins --- drivers/iommu/amd/init.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c index 082f47e22c6e..102440316c4a 100644 --- a/drivers/iommu/amd/init.c +++ b/drivers/iommu/amd/init.c @@ -2151,6 +2151,10 @@ static void print_iommu_info(void) if (iommu->features & FEATURE_GAM_VAPIC) pr_cont(" GA_vAPIC"); + if (iommu->features & FEATURE_HASUP) + pr_cont(" HASup"); + if (iommu->features & FEATURE_HDSUP) + pr_cont(" HDSup"); if (iommu->features & FEATURE_SNP) pr_cont(" SNP"); From patchwork Thu May 18 20:46:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247376 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E626C77B7A for ; Thu, 18 May 2023 20:49:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230318AbjERUtz (ORCPT ); Thu, 18 May 2023 16:49:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57962 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230062AbjERUty (ORCPT ); Thu, 18 May 2023 16:49:54 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 16A5A173F for ; Thu, 18 May 2023 13:49:22 -0700 (PDT) Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIx9kD015384; Thu, 18 May 2023 20:49:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=8dUdIZkk4mF1VFVAvcQyDH3mky5o3NyzbdKt58jwQPE=; b=G1AJwzMntb50af6hyzrfXVmuB/xeNfYb1rvfAKMSf+Kx5nquQDVXfKCga0ZLycKsN3PD /pC+ABaT7vfqmi/3Oleb4CInVueMiFsEbSC2/8W6sute7ZTJNvS/vBubIc/w+aaPaA9C YNQNAgjdvDIZZV8fya/dFPLO2WpX8i0bZzUZaA3lRJPuwBKcS7dbpCv55x3lxx+en9Eg fIniGY+Szu1wVJ9Z6SAlw1pR8rR1WfO+JRVIidisi29KmrXmnsOcRnAEQ1pf+iaAODkP 0iPYdMZLtFHqHvvdreO1Kt7Ah2Q7byXNHGf+59WxI+IU7W2MZmoQh9rUa1l+JEUAEFJ1 3A== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qmx8j3n69-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:49:03 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJBdG1032190; Thu, 18 May 2023 20:49:02 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10dafme-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:49:01 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE3P033533; Thu, 18 May 2023 20:49:01 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-20; Thu, 18 May 2023 20:49:01 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 19/24] iommu/intel: Access/Dirty bit support for SL domains Date: Thu, 18 May 2023 21:46:45 +0100 Message-Id: <20230518204650.14541-20-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180171 X-Proofpoint-GUID: D0VI0rHUKneO4gbYsi5H0D4GtHTHxkbJ X-Proofpoint-ORIG-GUID: D0VI0rHUKneO4gbYsi5H0D4GtHTHxkbJ Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org IOMMU advertises Access/Dirty bits for second-stage page table if the extended capability DMAR register reports it (ECAP, mnemonic ECAP.SSADS). The first stage table is compatible with CPU page table thus A/D bits are implicitly supported. Relevant Intel IOMMU SDM ref for first stage table "3.6.2 Accessed, Extended Accessed, and Dirty Flags" and second stage table "3.7.2 Accessed and Dirty Flags". First stage page table is enabled by default so it's allowed to set dirty tracking and no control bits needed, we just return 0. To use SSADS, we set bit 9 (SSADE) in the scalable-mode PASID table entry and flush the IOTLB via pasid_flush_caches(). Relevant SDM refs: "3.7.2 Accessed and Dirty Flags" "6.5.3.3 Guidance to Software for Invalidations, Table 23. Guidance to Software for Invalidations" PTE dirty bit is located in bit 9 and it's cached in the IOTLB so we also need to flush IOTLB to make sure IOMMU attempts to set the dirty bit again. Note that iommu_dirty_bitmap_record() will add the IOVA to iotlb_gather and thus the caller of the iommu op will flush the IOTLB. Relevant manuals over the hardware translation is chapter 6 with some special mention to: "6.2.3.1 Scalable-Mode PASID-Table Entry Programming Considerations" "6.2.4 IOTLB" Signed-off-by: Joao Martins --- The IOPTE walker is still inneficient, but v3 will change that. Mainly purpose here would be to make sure the UAPI/IOMMUFD is solid and agreed upon. --- drivers/iommu/intel/iommu.c | 88 ++++++++++++++++++++++++++++++ drivers/iommu/intel/iommu.h | 15 ++++++ drivers/iommu/intel/pasid.c | 103 ++++++++++++++++++++++++++++++++++++ drivers/iommu/intel/pasid.h | 4 ++ 4 files changed, 210 insertions(+) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c index 4662292d60ba..6cf9cbe4c299 100644 --- a/drivers/iommu/intel/iommu.c +++ b/drivers/iommu/intel/iommu.c @@ -4112,6 +4112,10 @@ static int prepare_domain_attach_device(struct iommu_domain *domain, if (!iommu) return -ENODEV; + if (domain->flags & IOMMU_DOMAIN_F_ENFORCE_DIRTY && + !ecap_slads(iommu->ecap)) + return -EINVAL; + if (dmar_domain->force_snooping && !ecap_sc_support(iommu->ecap)) return -EINVAL; @@ -4374,6 +4378,9 @@ static bool intel_iommu_capable(struct device *dev, enum iommu_cap cap) return dmar_platform_optin(); case IOMMU_CAP_ENFORCE_CACHE_COHERENCY: return ecap_sc_support(info->iommu->ecap); + case IOMMU_CAP_DIRTY: + return sm_supported(info->iommu) && + ecap_slads(info->iommu->ecap); default: return false; } @@ -4739,6 +4746,84 @@ static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) intel_pasid_tear_down_entry(iommu, dev, pasid, false); } +static int intel_iommu_set_dirty_tracking(struct iommu_domain *domain, + bool enable) +{ + struct dmar_domain *dmar_domain = to_dmar_domain(domain); + struct device_domain_info *info; + int ret = -EINVAL; + + spin_lock(&dmar_domain->lock); + if (!(dmar_domain->dirty_tracking ^ enable) || + list_empty(&dmar_domain->devices)) { + spin_unlock(&dmar_domain->lock); + return 0; + } + + list_for_each_entry(info, &dmar_domain->devices, link) { + /* First-level page table always enables dirty bit*/ + if (dmar_domain->use_first_level) { + ret = 0; + break; + } + + ret = intel_pasid_setup_dirty_tracking(info->iommu, info->domain, + info->dev, PASID_RID2PASID, + enable); + if (ret) + break; + + } + + if (!ret) + dmar_domain->dirty_tracking = enable; + spin_unlock(&dmar_domain->lock); + + return ret; +} + +static int intel_iommu_read_and_clear_dirty(struct iommu_domain *domain, + unsigned long iova, size_t size, + unsigned long flags, + struct iommu_dirty_bitmap *dirty) +{ + struct dmar_domain *dmar_domain = to_dmar_domain(domain); + unsigned long end = iova + size - 1; + unsigned long pgsize; + bool ad_enabled; + + spin_lock(&dmar_domain->lock); + ad_enabled = dmar_domain->dirty_tracking; + spin_unlock(&dmar_domain->lock); + + if (!ad_enabled && dirty->bitmap) + return -EINVAL; + + rcu_read_lock(); + do { + struct dma_pte *pte; + int lvl = 0; + + pte = pfn_to_dma_pte(dmar_domain, iova >> VTD_PAGE_SHIFT, &lvl, + GFP_ATOMIC); + pgsize = level_size(lvl) << VTD_PAGE_SHIFT; + if (!pte || !dma_pte_present(pte)) { + iova += pgsize; + continue; + } + + /* It is writable, set the bitmap */ + if (((flags & IOMMU_DIRTY_NO_CLEAR) && + dma_sl_pte_dirty(pte)) || + dma_sl_pte_test_and_clear_dirty(pte)) + iommu_dirty_bitmap_record(dirty, iova, pgsize); + iova += pgsize; + } while (iova < end); + rcu_read_unlock(); + + return 0; +} + const struct iommu_ops intel_iommu_ops = { .capable = intel_iommu_capable, .domain_alloc = intel_iommu_domain_alloc, @@ -4753,6 +4838,7 @@ const struct iommu_ops intel_iommu_ops = { .def_domain_type = device_def_domain_type, .remove_dev_pasid = intel_iommu_remove_dev_pasid, .pgsize_bitmap = SZ_4K, + .supported_flags = IOMMU_DOMAIN_F_ENFORCE_DIRTY, #ifdef CONFIG_INTEL_IOMMU_SVM .page_response = intel_svm_page_response, #endif @@ -4766,6 +4852,8 @@ const struct iommu_ops intel_iommu_ops = { .iova_to_phys = intel_iommu_iova_to_phys, .free = intel_iommu_domain_free, .enforce_cache_coherency = intel_iommu_enforce_cache_coherency, + .set_dirty_tracking = intel_iommu_set_dirty_tracking, + .read_and_clear_dirty = intel_iommu_read_and_clear_dirty, } }; diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h index 1c5e1d88862b..56ee6ce2e09d 100644 --- a/drivers/iommu/intel/iommu.h +++ b/drivers/iommu/intel/iommu.h @@ -48,6 +48,9 @@ #define DMA_FL_PTE_DIRTY BIT_ULL(6) #define DMA_FL_PTE_XD BIT_ULL(63) +#define DMA_SL_PTE_DIRTY_BIT 9 +#define DMA_SL_PTE_DIRTY BIT_ULL(DMA_SL_PTE_DIRTY_BIT) + #define ADDR_WIDTH_5LEVEL (57) #define ADDR_WIDTH_4LEVEL (48) @@ -592,6 +595,7 @@ struct dmar_domain { * otherwise, goes through the second * level. */ + u8 dirty_tracking:1; /* Dirty tracking is enabled */ spinlock_t lock; /* Protect device tracking lists */ struct list_head devices; /* all devices' list */ @@ -774,6 +778,17 @@ static inline bool dma_pte_present(struct dma_pte *pte) return (pte->val & 3) != 0; } +static inline bool dma_sl_pte_dirty(struct dma_pte *pte) +{ + return (pte->val & DMA_SL_PTE_DIRTY) != 0; +} + +static inline bool dma_sl_pte_test_and_clear_dirty(struct dma_pte *pte) +{ + return test_and_clear_bit(DMA_SL_PTE_DIRTY_BIT, + (unsigned long *)&pte->val); +} + static inline bool dma_pte_superpage(struct dma_pte *pte) { return (pte->val & DMA_PTE_LARGE_PAGE); diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c index c5d479770e12..c7cfa0387277 100644 --- a/drivers/iommu/intel/pasid.c +++ b/drivers/iommu/intel/pasid.c @@ -277,6 +277,11 @@ static inline void pasid_set_bits(u64 *ptr, u64 mask, u64 bits) WRITE_ONCE(*ptr, (old & ~mask) | bits); } +static inline u64 pasid_get_bits(u64 *ptr) +{ + return READ_ONCE(*ptr); +} + /* * Setup the DID(Domain Identifier) field (Bit 64~79) of scalable mode * PASID entry. @@ -335,6 +340,45 @@ static inline void pasid_set_fault_enable(struct pasid_entry *pe) pasid_set_bits(&pe->val[0], 1 << 1, 0); } +/* + * Enable second level A/D bits by setting the SLADE (Second Level + * Access Dirty Enable) field (Bit 9) of a scalable mode PASID + * entry. + */ +static inline void pasid_set_ssade(struct pasid_entry *pe) +{ + pasid_set_bits(&pe->val[0], 1 << 9, 1 << 9); +} + +/* + * Enable second level A/D bits by setting the SLADE (Second Level + * Access Dirty Enable) field (Bit 9) of a scalable mode PASID + * entry. + */ +static inline void pasid_clear_ssade(struct pasid_entry *pe) +{ + pasid_set_bits(&pe->val[0], 1 << 9, 0); +} + +/* + * Checks if second level A/D bits by setting the SLADE (Second Level + * Access Dirty Enable) field (Bit 9) of a scalable mode PASID + * entry is enabled. + */ +static inline bool pasid_get_ssade(struct pasid_entry *pe) +{ + return pasid_get_bits(&pe->val[0]) & (1 << 9); +} + +/* + * Setup the SRE(Supervisor Request Enable) field (Bit 128) of a + * scalable mode PASID entry. + */ +static inline void pasid_set_sre(struct pasid_entry *pe) +{ + pasid_set_bits(&pe->val[2], 1 << 0, 1); +} + /* * Setup the WPE(Write Protect Enable) field (Bit 132) of a * scalable mode PASID entry. @@ -627,6 +671,8 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu, pasid_set_translation_type(pte, PASID_ENTRY_PGTT_SL_ONLY); pasid_set_fault_enable(pte); pasid_set_page_snoop(pte, !!ecap_smpwc(iommu->ecap)); + if (domain->dirty_tracking) + pasid_set_ssade(pte); pasid_set_present(pte); spin_unlock(&iommu->lock); @@ -636,6 +682,63 @@ int intel_pasid_setup_second_level(struct intel_iommu *iommu, return 0; } +/* + * Set up dirty tracking on a second only translation type. + */ +int intel_pasid_setup_dirty_tracking(struct intel_iommu *iommu, + struct dmar_domain *domain, + struct device *dev, u32 pasid, + bool enabled) +{ + struct pasid_entry *pte; + u16 did, pgtt; + + spin_lock(&iommu->lock); + + did = domain_id_iommu(domain, iommu); + pte = intel_pasid_get_entry(dev, pasid); + if (!pte) { + spin_unlock(&iommu->lock); + dev_err(dev, "Failed to get pasid entry of PASID %d\n", pasid); + return -ENODEV; + } + + pgtt = pasid_pte_get_pgtt(pte); + + if (enabled) + pasid_set_ssade(pte); + else + pasid_clear_ssade(pte); + spin_unlock(&iommu->lock); + + /* + * From VT-d spec table 25 "Guidance to Software for Invalidations": + * + * - PASID-selective-within-Domain PASID-cache invalidation + * If (PGTT=SS or Nested) + * - Domain-selective IOTLB invalidation + * Else + * - PASID-selective PASID-based IOTLB invalidation + * - If (pasid is RID_PASID) + * - Global Device-TLB invalidation to affected functions + * Else + * - PASID-based Device-TLB invalidation (with S=1 and + * Addr[63:12]=0x7FFFFFFF_FFFFF) to affected functions + */ + pasid_cache_invalidation_with_pasid(iommu, did, pasid); + + if (pgtt == PASID_ENTRY_PGTT_SL_ONLY || pgtt == PASID_ENTRY_PGTT_NESTED) + iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH); + else + qi_flush_piotlb(iommu, did, pasid, 0, -1, 0); + + /* Device IOTLB doesn't need to be flushed in caching mode. */ + if (!cap_caching_mode(iommu->cap)) + devtlb_invalidation_with_pasid(iommu, dev, pasid); + + return 0; +} + /* * Set up the scalable mode pasid entry for passthrough translation type. */ diff --git a/drivers/iommu/intel/pasid.h b/drivers/iommu/intel/pasid.h index d6b7d21244b1..3fc5aba02971 100644 --- a/drivers/iommu/intel/pasid.h +++ b/drivers/iommu/intel/pasid.h @@ -108,6 +108,10 @@ int intel_pasid_setup_first_level(struct intel_iommu *iommu, int intel_pasid_setup_second_level(struct intel_iommu *iommu, struct dmar_domain *domain, struct device *dev, u32 pasid); +int intel_pasid_setup_dirty_tracking(struct intel_iommu *iommu, + struct dmar_domain *domain, + struct device *dev, u32 pasid, + bool enabled); int intel_pasid_setup_pass_through(struct intel_iommu *iommu, struct dmar_domain *domain, struct device *dev, u32 pasid); From patchwork Thu May 18 20:46:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247378 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EFF4CC77B7A for ; Thu, 18 May 2023 20:50:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230363AbjERUuT (ORCPT ); Thu, 18 May 2023 16:50:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230259AbjERUuR (ORCPT ); Thu, 18 May 2023 16:50:17 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B3EE0171F for ; Thu, 18 May 2023 13:49:44 -0700 (PDT) Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIx0ho029765; Thu, 18 May 2023 20:49:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=NuZcDcoI+IWzUpgCs/AeMI0KnHsXKTSMrRq5A3YGHq0=; b=bU33w9W1eiPTCsdTtgg25nT5o5mnrUDF8h8lIlAH2XUVENcm6jqgXR87/IRUXwctIzqm djIlpeM7oKJPHQHyp+ayr0o6jUpDM0dUzPVdI0SRbBBNxsSjkD1unYBnnFN5OzxCOSxQ oNzk1eR/ZdMu0UTBtM2o2JXCbJkCSM27v77x6HkBPEikuqi52ofsg+x4B64iWh5uL2sz 4EpmGaCVotVBAFCXWPBpd8zomltoc8pXFgRPNsUzzVqG4YNzCnTaQbuPvJpaBn0TXfGA CupCVHy5DphIWGFfdcaltL42TfaOlNyGRSd23XkNBPRZiteqMmhKYnUaOJj2y7HJdXAr Yg== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qmxwpkg8s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:49:07 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJkASX032121; Thu, 18 May 2023 20:49:06 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10dafpu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:49:06 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE3R033533; Thu, 18 May 2023 20:49:06 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-21; Thu, 18 May 2023 20:49:05 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 20/24] iommu/arm-smmu-v3: Add feature detection for HTTU Date: Thu, 18 May 2023 21:46:46 +0100 Message-Id: <20230518204650.14541-21-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180171 X-Proofpoint-GUID: UFIsGWGleXVn72z1SX64bXTcxJ_ILb4I X-Proofpoint-ORIG-GUID: UFIsGWGleXVn72z1SX64bXTcxJ_ILb4I Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Jean-Philippe Brucker If the SMMU supports it and the kernel was built with HTTU support, Probe support for Hardware Translation Table Update (HTTU) which is essentially to enable hardware update of access and dirty flags. Probe and set the smmu::features for Hardware Dirty and Hardware Access bits. This is in preparation, to enable it on the context descriptors of stage 1 format. Link: https://lore.kernel.org/lkml/20210413085457.25400-5-zhukeqian1@huawei.com/ Signed-off-by: Jean-Philippe Brucker [joaomart: Change commit message to reflect the underlying changes, the Link points to the original version this was based] Signed-off-by: Joao Martins --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 32 +++++++++++++++++++++ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 5 ++++ 2 files changed, 37 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 3fd83fb75722..e110ff4710bf 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -3429,6 +3429,28 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu, bool bypass) return 0; } +static void arm_smmu_get_httu(struct arm_smmu_device *smmu, u32 reg) +{ + u32 fw_features = smmu->features & (ARM_SMMU_FEAT_HA | ARM_SMMU_FEAT_HD); + u32 features = 0; + + switch (FIELD_GET(IDR0_HTTU, reg)) { + case IDR0_HTTU_ACCESS_DIRTY: + features |= ARM_SMMU_FEAT_HD; + fallthrough; + case IDR0_HTTU_ACCESS: + features |= ARM_SMMU_FEAT_HA; + } + + if (smmu->dev->of_node) + smmu->features |= features; + else if (features != fw_features) + /* ACPI IORT sets the HTTU bits */ + dev_warn(smmu->dev, + "IDR0.HTTU overridden by FW configuration (0x%x)\n", + fw_features); +} + static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu) { u32 reg; @@ -3489,6 +3511,8 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu) smmu->features |= ARM_SMMU_FEAT_E2H; } + arm_smmu_get_httu(smmu, reg); + /* * The coherency feature as set by FW is used in preference to the ID * register, but warn on mismatch. @@ -3675,6 +3699,14 @@ static int arm_smmu_device_acpi_probe(struct platform_device *pdev, if (iort_smmu->flags & ACPI_IORT_SMMU_V3_COHACC_OVERRIDE) smmu->features |= ARM_SMMU_FEAT_COHERENCY; + switch (FIELD_GET(ACPI_IORT_SMMU_V3_HTTU_OVERRIDE, iort_smmu->flags)) { + case IDR0_HTTU_ACCESS_DIRTY: + smmu->features |= ARM_SMMU_FEAT_HD; + fallthrough; + case IDR0_HTTU_ACCESS: + smmu->features |= ARM_SMMU_FEAT_HA; + } + return 0; } #else diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index b574c58a3487..d82dd125446c 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -33,6 +33,9 @@ #define IDR0_ASID16 (1 << 12) #define IDR0_ATS (1 << 10) #define IDR0_HYP (1 << 9) +#define IDR0_HTTU GENMASK(7, 6) +#define IDR0_HTTU_ACCESS 1 +#define IDR0_HTTU_ACCESS_DIRTY 2 #define IDR0_COHACC (1 << 4) #define IDR0_TTF GENMASK(3, 2) #define IDR0_TTF_AARCH64 2 @@ -639,6 +642,8 @@ struct arm_smmu_device { #define ARM_SMMU_FEAT_BTM (1 << 16) #define ARM_SMMU_FEAT_SVA (1 << 17) #define ARM_SMMU_FEAT_E2H (1 << 18) +#define ARM_SMMU_FEAT_HA (1 << 19) +#define ARM_SMMU_FEAT_HD (1 << 20) u32 features; #define ARM_SMMU_OPT_SKIP_PREFETCH (1 << 0) From patchwork Thu May 18 20:46:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247377 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DB4CC77B73 for ; Thu, 18 May 2023 20:50:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230321AbjERUuR (ORCPT ); Thu, 18 May 2023 16:50:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58272 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230062AbjERUuQ (ORCPT ); Thu, 18 May 2023 16:50:16 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 80DB610E5 for ; Thu, 18 May 2023 13:49:43 -0700 (PDT) Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIxAXG015520; Thu, 18 May 2023 20:49:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=LMvi/hYF8ATr0QTdOqJooiRhnqgeYdhXDKmWJ5nZxc0=; b=Q4Fc4H1ibmadDmr5uAj7AnGtb9hm3aV+7wcds45UyuagmFocFsnhtIRo3TtjCGbPg8fq mmYRjX80np8zgvky4j6xo7eVVtsOxlXI5UPhSvVUSUBDjzI71VlgnlbEJzw8actggYMU zrFcLWA5yWyOdmbrCuI43Q46P1vZcGEPl8Weeyyirl3aM4RAWOgqfyDPW3q3Ms66FxVG poXyd7D76NlY/SP7a18ZZoNrWlLPFqbmmvgjImYNNkBkQSYjqFeMgoDK2lCKToenXDlS WNR0FZqR3TLKcHc5N6uSHO8qoBP5R5YkP2S4dGC4UEklhb0x9sN5QQHxwNellPdwlQWN 5Q== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qmx8j3n6y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:49:12 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJK7xd032108; Thu, 18 May 2023 20:49:11 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10dafrr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:49:11 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE3T033533; Thu, 18 May 2023 20:49:10 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-22; Thu, 18 May 2023 20:49:09 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 21/24] iommu/arm-smmu-v3: Enable HTTU for stage1 with io-pgtable mapping Date: Thu, 18 May 2023 21:46:47 +0100 Message-Id: <20230518204650.14541-22-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180171 X-Proofpoint-GUID: nucMYkaRm7ZmTX6k_82ZCTKIKWBw4VsW X-Proofpoint-ORIG-GUID: nucMYkaRm7ZmTX6k_82ZCTKIKWBw4VsW Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Kunkun Jiang As nested mode is not upstreamed now, we just aim to support dirty log tracking for stage1 with io-pgtable mapping (means not support SVA mapping). If HTTU is supported, we enable HA/HD bits in the SMMU CD and transfer ARM_HD quirk to io-pgtable. We additionally filter out HD|HA if not supportted. The CD.HD bit is not particularly useful unless we toggle the DBM bit in the PTE entries. Link: https://lore.kernel.org/lkml/20210413085457.25400-6-zhukeqian1@huawei.com/ Co-developed-by: Keqian Zhu Signed-off-by: Keqian Zhu Signed-off-by: Kunkun Jiang [joaomart:Convey HD|HA bits over to the context descriptor and update commit message; original in Link, where this is based on] Signed-off-by: Joao Martins --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 10 ++++++++++ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 3 +++ drivers/iommu/io-pgtable-arm.c | 11 +++++++++-- include/linux/io-pgtable.h | 4 ++++ 4 files changed, 26 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index e110ff4710bf..e2b98a6a6b74 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -1998,6 +1998,11 @@ static const struct iommu_flush_ops arm_smmu_flush_ops = { .tlb_add_page = arm_smmu_tlb_inv_page_nosync, }; +static bool arm_smmu_dbm_capable(struct arm_smmu_device *smmu) +{ + return smmu->features & (ARM_SMMU_FEAT_HD | ARM_SMMU_FEAT_COHERENCY); +} + /* IOMMU API */ static bool arm_smmu_capable(struct device *dev, enum iommu_cap cap) { @@ -2124,6 +2129,8 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, FIELD_PREP(CTXDESC_CD_0_TCR_SH0, tcr->sh) | FIELD_PREP(CTXDESC_CD_0_TCR_IPS, tcr->ips) | CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64; + if (pgtbl_cfg->quirks & IO_PGTABLE_QUIRK_ARM_HD) + cfg->cd.tcr |= CTXDESC_CD_0_TCR_HA | CTXDESC_CD_0_TCR_HD; cfg->cd.mair = pgtbl_cfg->arm_lpae_s1_cfg.mair; /* @@ -2226,6 +2233,9 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain, .iommu_dev = smmu->dev, }; + if (smmu->features & arm_smmu_dbm_capable(smmu)) + pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_ARM_HD; + pgtbl_ops = alloc_io_pgtable_ops(fmt, &pgtbl_cfg, smmu_domain); if (!pgtbl_ops) return -ENOMEM; diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h index d82dd125446c..83d6f3a2554f 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h @@ -288,6 +288,9 @@ #define CTXDESC_CD_0_TCR_IPS GENMASK_ULL(34, 32) #define CTXDESC_CD_0_TCR_TBI0 (1ULL << 38) +#define CTXDESC_CD_0_TCR_HA (1UL << 43) +#define CTXDESC_CD_0_TCR_HD (1UL << 42) + #define CTXDESC_CD_0_AA64 (1UL << 41) #define CTXDESC_CD_0_S (1UL << 44) #define CTXDESC_CD_0_R (1UL << 45) diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c index 72dcdd468cf3..b2f470529459 100644 --- a/drivers/iommu/io-pgtable-arm.c +++ b/drivers/iommu/io-pgtable-arm.c @@ -75,6 +75,7 @@ #define ARM_LPAE_PTE_NSTABLE (((arm_lpae_iopte)1) << 63) #define ARM_LPAE_PTE_XN (((arm_lpae_iopte)3) << 53) +#define ARM_LPAE_PTE_DBM (((arm_lpae_iopte)1) << 51) #define ARM_LPAE_PTE_AF (((arm_lpae_iopte)1) << 10) #define ARM_LPAE_PTE_SH_NS (((arm_lpae_iopte)0) << 8) #define ARM_LPAE_PTE_SH_OS (((arm_lpae_iopte)2) << 8) @@ -84,7 +85,7 @@ #define ARM_LPAE_PTE_ATTR_LO_MASK (((arm_lpae_iopte)0x3ff) << 2) /* Ignore the contiguous bit for block splitting */ -#define ARM_LPAE_PTE_ATTR_HI_MASK (((arm_lpae_iopte)6) << 52) +#define ARM_LPAE_PTE_ATTR_HI_MASK (((arm_lpae_iopte)13) << 51) #define ARM_LPAE_PTE_ATTR_MASK (ARM_LPAE_PTE_ATTR_LO_MASK | \ ARM_LPAE_PTE_ATTR_HI_MASK) /* Software bit for solving coherency races */ @@ -93,6 +94,9 @@ /* Stage-1 PTE */ #define ARM_LPAE_PTE_AP_UNPRIV (((arm_lpae_iopte)1) << 6) #define ARM_LPAE_PTE_AP_RDONLY (((arm_lpae_iopte)2) << 6) +#define ARM_LPAE_PTE_AP_RDONLY_BIT 7 +#define ARM_LPAE_PTE_AP_WRITABLE (ARM_LPAE_PTE_AP_RDONLY | \ + ARM_LPAE_PTE_DBM) #define ARM_LPAE_PTE_ATTRINDX_SHIFT 2 #define ARM_LPAE_PTE_nG (((arm_lpae_iopte)1) << 11) @@ -407,6 +411,8 @@ static arm_lpae_iopte arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data, pte = ARM_LPAE_PTE_nG; if (!(prot & IOMMU_WRITE) && (prot & IOMMU_READ)) pte |= ARM_LPAE_PTE_AP_RDONLY; + else if (data->iop.cfg.quirks & IO_PGTABLE_QUIRK_ARM_HD) + pte |= ARM_LPAE_PTE_AP_WRITABLE; if (!(prot & IOMMU_PRIV)) pte |= ARM_LPAE_PTE_AP_UNPRIV; } else { @@ -804,7 +810,8 @@ arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie) if (cfg->quirks & ~(IO_PGTABLE_QUIRK_ARM_NS | IO_PGTABLE_QUIRK_ARM_TTBR1 | - IO_PGTABLE_QUIRK_ARM_OUTER_WBWA)) + IO_PGTABLE_QUIRK_ARM_OUTER_WBWA | + IO_PGTABLE_QUIRK_ARM_HD)) return NULL; data = arm_lpae_alloc_pgtable(cfg); diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h index 25142a0e2fc2..9a996ba7856d 100644 --- a/include/linux/io-pgtable.h +++ b/include/linux/io-pgtable.h @@ -85,6 +85,8 @@ struct io_pgtable_cfg { * * IO_PGTABLE_QUIRK_ARM_OUTER_WBWA: Override the outer-cacheability * attributes set in the TCR for a non-coherent page-table walker. + * + * IO_PGTABLE_QUIRK_ARM_HD: Enables dirty tracking. */ #define IO_PGTABLE_QUIRK_ARM_NS BIT(0) #define IO_PGTABLE_QUIRK_NO_PERMS BIT(1) @@ -92,6 +94,8 @@ struct io_pgtable_cfg { #define IO_PGTABLE_QUIRK_ARM_MTK_TTBR_EXT BIT(4) #define IO_PGTABLE_QUIRK_ARM_TTBR1 BIT(5) #define IO_PGTABLE_QUIRK_ARM_OUTER_WBWA BIT(6) + #define IO_PGTABLE_QUIRK_ARM_HD BIT(7) + unsigned long quirks; unsigned long pgsize_bitmap; unsigned int ias; From patchwork Thu May 18 20:46:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247380 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72E37C77B7A for ; Thu, 18 May 2023 20:50:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230355AbjERUue (ORCPT ); Thu, 18 May 2023 16:50:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58538 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230368AbjERUuc (ORCPT ); Thu, 18 May 2023 16:50:32 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DD0901729 for ; Thu, 18 May 2023 13:50:13 -0700 (PDT) Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIxZTg012446; Thu, 18 May 2023 20:49:17 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=GuoSj46gusCPx2n8p781D/gEMzPiYIlbDruZ4mnxf18=; b=P7nMhyREZdOL9tPzY0ptU6Xo9IcPA9WI3yZumnJs8H7cLRYO5+AlU9uB7Jw2C1Z14UcE DmiYVhP7nD09sE9bkIOafDqmg3mfJL13rIXHHVhDAwb2yinwk48vHGhQfsm8XBfoo/W7 6CFrpXXqpAqij9lPUWgdsET8a/WpDIVSK57d79lralMIR9iOpSIrKIe6aGOYMvy1rGO1 8WwUV5rtpM1ibc4CKuZwLKbHEJG4tkLg7sdMGZfNfOO8UQgXccMrdUQCfEYh1pkzRhrC EiDRzRdrNexURMXM/UkXcdtV6WLv1J/2sD6+rY5Wo7216SUSxsf15QYpQi3wNfeoVFca lg== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qmxfc3k16-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:49:17 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJ3Pnp032141; Thu, 18 May 2023 20:49:16 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10daftw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:49:15 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE3V033533; Thu, 18 May 2023 20:49:15 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-23; Thu, 18 May 2023 20:49:14 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 22/24] iommu/arm-smmu-v3: Add read_and_clear_dirty() support Date: Thu, 18 May 2023 21:46:48 +0100 Message-Id: <20230518204650.14541-23-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=709 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180171 X-Proofpoint-GUID: pVWoFGVw22b47mNe1uOMa0FvadM2rC3W X-Proofpoint-ORIG-GUID: pVWoFGVw22b47mNe1uOMa0FvadM2rC3W Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org From: Keqian Zhu .read_and_clear_dirty() IOMMU domain op takes care of reading the dirty bits (i.e. PTE has both DBM and AP[2] set) and marshalling into a bitmap of a given page size. While reading the dirty bits we also clear the PTE AP[2] bit to mark it as writable-clean depending on read_and_clear_dirty() flags. Structure it in a way that the IOPTE walker is generic, and so we pass a function pointer over what to do on a per-PTE basis. [Link below points to the original version that was based on] Link: https://lore.kernel.org/lkml/20210413085457.25400-11-zhukeqian1@huawei.com/ Co-developed-by: Keqian Zhu Co-developed-by: Kunkun Jiang Signed-off-by: Kunkun Jiang [joaomart: Massage commit message] Co-developed-by: Joao Martins Signed-off-by: Joao Martins --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 23 +++++ drivers/iommu/io-pgtable-arm.c | 104 ++++++++++++++++++++ 2 files changed, 127 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index e2b98a6a6b74..2cde14003469 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2765,6 +2765,28 @@ static int arm_smmu_enable_nesting(struct iommu_domain *domain) return ret; } +static int arm_smmu_read_and_clear_dirty(struct iommu_domain *domain, + unsigned long iova, size_t size, + unsigned long flags, + struct iommu_dirty_bitmap *dirty) +{ + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops; + int ret; + + if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1) + return -EINVAL; + + if (!ops || !ops->read_and_clear_dirty) { + pr_err_once("io-pgtable don't support dirty tracking\n"); + return -ENODEV; + } + + ret = ops->read_and_clear_dirty(ops, iova, size, flags, dirty); + + return ret; +} + static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args) { return iommu_fwspec_add_ids(dev, args->args, 1); @@ -2893,6 +2915,7 @@ static struct iommu_ops arm_smmu_ops = { .iova_to_phys = arm_smmu_iova_to_phys, .enable_nesting = arm_smmu_enable_nesting, .free = arm_smmu_domain_free, + .read_and_clear_dirty = arm_smmu_read_and_clear_dirty, } }; diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c index b2f470529459..de9e61f8452d 100644 --- a/drivers/iommu/io-pgtable-arm.c +++ b/drivers/iommu/io-pgtable-arm.c @@ -717,6 +717,109 @@ static phys_addr_t arm_lpae_iova_to_phys(struct io_pgtable_ops *ops, return iopte_to_paddr(pte, data) | iova; } +struct arm_lpae_iopte_read_dirty { + unsigned long flags; + struct iommu_dirty_bitmap *dirty; +}; + +static int __arm_lpae_read_and_clear_dirty(unsigned long iova, size_t size, + arm_lpae_iopte *ptep, void *opaque) +{ + struct arm_lpae_iopte_read_dirty *arg = opaque; + struct iommu_dirty_bitmap *dirty = arg->dirty; + arm_lpae_iopte pte; + + pte = READ_ONCE(*ptep); + if (WARN_ON(!pte)) + return -EINVAL; + + if ((pte & ARM_LPAE_PTE_AP_WRITABLE) == ARM_LPAE_PTE_AP_WRITABLE) + return 0; + + iommu_dirty_bitmap_record(dirty, iova, size); + if (!(arg->flags & IOMMU_DIRTY_NO_CLEAR)) + set_bit(ARM_LPAE_PTE_AP_RDONLY_BIT, (unsigned long *)ptep); + return 0; +} + +static int __arm_lpae_iopte_walk(struct arm_lpae_io_pgtable *data, + unsigned long iova, size_t size, + int lvl, arm_lpae_iopte *ptep, + int (*fn)(unsigned long iova, size_t size, + arm_lpae_iopte *pte, void *opaque), + void *opaque) +{ + arm_lpae_iopte pte; + struct io_pgtable *iop = &data->iop; + size_t base, next_size; + int ret; + + if (WARN_ON_ONCE(!fn)) + return -EINVAL; + + if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS)) + return -EINVAL; + + ptep += ARM_LPAE_LVL_IDX(iova, lvl, data); + pte = READ_ONCE(*ptep); + if (WARN_ON(!pte)) + return -EINVAL; + + if (size == ARM_LPAE_BLOCK_SIZE(lvl, data)) { + if (iopte_leaf(pte, lvl, iop->fmt)) + return fn(iova, size, ptep, opaque); + + /* Current level is table, traverse next level */ + next_size = ARM_LPAE_BLOCK_SIZE(lvl + 1, data); + ptep = iopte_deref(pte, data); + for (base = 0; base < size; base += next_size) { + ret = __arm_lpae_iopte_walk(data, iova + base, + next_size, lvl + 1, ptep, + fn, opaque); + if (ret) + return ret; + } + return 0; + } else if (iopte_leaf(pte, lvl, iop->fmt)) { + return fn(iova, size, ptep, opaque); + } + + /* Keep on walkin */ + ptep = iopte_deref(pte, data); + return __arm_lpae_iopte_walk(data, iova, size, lvl + 1, ptep, + fn, opaque); +} + +static int arm_lpae_read_and_clear_dirty(struct io_pgtable_ops *ops, + unsigned long iova, size_t size, + unsigned long flags, + struct iommu_dirty_bitmap *dirty) +{ + struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops); + struct io_pgtable_cfg *cfg = &data->iop.cfg; + struct arm_lpae_iopte_read_dirty arg = { + .flags = flags, .dirty = dirty, + }; + arm_lpae_iopte *ptep = data->pgd; + int lvl = data->start_level; + long iaext = (s64)iova >> cfg->ias; + + if (WARN_ON(!size || (size & cfg->pgsize_bitmap) != size)) + return -EINVAL; + + if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_TTBR1) + iaext = ~iaext; + if (WARN_ON(iaext)) + return -EINVAL; + + if (data->iop.fmt != ARM_64_LPAE_S1 && + data->iop.fmt != ARM_32_LPAE_S1) + return -EINVAL; + + return __arm_lpae_iopte_walk(data, iova, size, lvl, ptep, + __arm_lpae_read_and_clear_dirty, &arg); +} + static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg) { unsigned long granule, page_sizes; @@ -795,6 +898,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg) .map_pages = arm_lpae_map_pages, .unmap_pages = arm_lpae_unmap_pages, .iova_to_phys = arm_lpae_iova_to_phys, + .read_and_clear_dirty = arm_lpae_read_and_clear_dirty, }; return data; From patchwork Thu May 18 20:46:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247381 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F3C3EC77B73 for ; Thu, 18 May 2023 20:50:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230391AbjERUuh (ORCPT ); Thu, 18 May 2023 16:50:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58322 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230349AbjERUue (ORCPT ); Thu, 18 May 2023 16:50:34 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 015901712 for ; Thu, 18 May 2023 13:50:18 -0700 (PDT) Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIwpfg018288; Thu, 18 May 2023 20:49:20 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=w0o6G0yQRdrkGIcIUpsgd4Lt8hKCCSm8IByMw8BYHjA=; b=NCXglvP4UsVZqjzl6W1YewcTzFcrM/60QBvdjh6NJG5kfGVlAh98ICeRsIneCj4K9Azi O3jZCZ9WfsE9lwVt1MrBxm5iWSHkf6qhe7eTXMKSkVKPWxgEJIbGVeOyT4UlqT3INqVw oB+ueQxJze6tuG7TJpC5Q3AMg/7AXnw/TiIbbJmcBzats3VyLII6CANiP1Fv1AhGWYnv 8YAYmapw6K3fRU03lAG//3jLDSyqW96FqjANLXCs5MGrYC3dArEmXgtNiOM43MPHDV+N 7RE+uwZVnBpRDfb3ZG/po6yoqXfwTGcN1PQOeH4DswqEDH1skXYfuvDckdWlErWh29+w nQ== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qj2kds9bu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:49:20 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IKXqwo032130; Thu, 18 May 2023 20:49:20 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10dafvn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:49:20 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE3X033533; Thu, 18 May 2023 20:49:19 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-24; Thu, 18 May 2023 20:49:19 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 23/24] iommu/arm-smmu-v3: Add set_dirty_tracking() support Date: Thu, 18 May 2023 21:46:49 +0100 Message-Id: <20230518204650.14541-24-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=878 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180171 X-Proofpoint-GUID: f0-uzCHn62GNlAOgiAgJSOIt59HUOCq- X-Proofpoint-ORIG-GUID: f0-uzCHn62GNlAOgiAgJSOIt59HUOCq- Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Dirty tracking will always be enabled with DBM=1 modifier enabled by default. Signed-off-by: Joao Martins --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 22 +++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 2cde14003469..bf0aac333725 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2787,6 +2787,27 @@ static int arm_smmu_read_and_clear_dirty(struct iommu_domain *domain, return ret; } +static int arm_smmu_set_dirty_tracking(struct iommu_domain *domain, + bool enabled) +{ + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops; + + if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1) + return -EINVAL; + + if (!ops) { + pr_err_once("io-pgtable don't support dirty tracking\n"); + return -ENODEV; + } + + /* + * Always enabled and the dirty bitmap is cleared prior to + * set_dirty_tracking(). + */ + return 0; +} + static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args) { return iommu_fwspec_add_ids(dev, args->args, 1); @@ -2916,6 +2937,7 @@ static struct iommu_ops arm_smmu_ops = { .enable_nesting = arm_smmu_enable_nesting, .free = arm_smmu_domain_free, .read_and_clear_dirty = arm_smmu_read_and_clear_dirty, + .set_dirty_tracking = arm_smmu_set_dirty_tracking, } }; From patchwork Thu May 18 20:46:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joao Martins X-Patchwork-Id: 13247382 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD4CCC77B7A for ; Thu, 18 May 2023 20:50:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230361AbjERUu6 (ORCPT ); Thu, 18 May 2023 16:50:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230171AbjERUut (ORCPT ); Thu, 18 May 2023 16:50:49 -0400 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 403981986 for ; Thu, 18 May 2023 13:50:31 -0700 (PDT) Received: from pps.filterd (m0333520.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 34IIx3RR018492; Thu, 18 May 2023 20:49:25 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2023-03-30; bh=LQKbjMN345OghwKCC/x/MrOorlZAr2BYoI+k4BaQA+Y=; b=sGHL2F+8xs6TCPX9puedXFBKd19Dsatyq5wQ5myWqPwZ1n1Q3rdcGTfKfRUjUIe4PMN2 Fwi0247qu4rlEeBp0sGPYuy4MV4PeHOO5bzG5etWLa8fMTcgYiB8/Z7wwNE6z9XWItqt XRur3415EPCU5Iw8PimfdpFK5DXVxT8yMQoeaerG7sOP+w8l6O4MzqleUPoeDPYPSILd 8qVp7dXaTrJfHMvdkHMqm4xMv04W2qmQXJMIdaD/ORAI/J5qncdrKdcz5Yj/EU6IYIA/ deNwI4PJOyvzmfkL8sOSuySepPUltwKdUA4iUmMMQCxa21QuyYPzJsg4zA+2TGsSdBrI Dg== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3qj2kds9ca-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:49:25 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 34IJV8dK032116; Thu, 18 May 2023 20:49:24 GMT Received: from pps.reinject (localhost [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3qj10dafy2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 18 May 2023 20:49:24 +0000 Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 34IKlE3Z033533; Thu, 18 May 2023 20:49:23 GMT Received: from joaomart-mac.uk.oracle.com (dhcp-10-175-172-172.vpn.oracle.com [10.175.172.172]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTP id 3qj10dae46-25; Thu, 18 May 2023 20:49:23 +0000 From: Joao Martins To: iommu@lists.linux.dev Cc: Jason Gunthorpe , Kevin Tian , Shameerali Kolothum Thodi , Lu Baolu , Yi Liu , Yi Y Sun , Eric Auger , Nicolin Chen , Joerg Roedel , Jean-Philippe Brucker , Suravee Suthikulpanit , Will Deacon , Robin Murphy , Alex Williamson , kvm@vger.kernel.org, Joao Martins Subject: [PATCH RFCv2 24/24] iommu/arm-smmu-v3: Advertise IOMMU_DOMAIN_F_ENFORCE_DIRTY Date: Thu, 18 May 2023 21:46:50 +0100 Message-Id: <20230518204650.14541-25-joao.m.martins@oracle.com> In-Reply-To: <20230518204650.14541-1-joao.m.martins@oracle.com> References: <20230518204650.14541-1-joao.m.martins@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-05-18_15,2023-05-17_02,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=902 adultscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2304280000 definitions=main-2305180171 X-Proofpoint-GUID: YKl1ERjnwCeW7oQrrjbN608uKWTdMhdh X-Proofpoint-ORIG-GUID: YKl1ERjnwCeW7oQrrjbN608uKWTdMhdh Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Now that we probe, and handle the DBM bit modifier, unblock the kAPI usage by exposing the IOMMU_DOMAIN_F_ENFORCE_DIRTY and implement it's requirement of revoking device attachment in the iommu_capable. Finally expose the IOMMU_CAP_DIRTY to users (IOMMUFD_DEVICE_GET_CAPS). Signed-off-by: Joao Martins --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index bf0aac333725..71dd95a687fd 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2014,6 +2014,8 @@ static bool arm_smmu_capable(struct device *dev, enum iommu_cap cap) return master->smmu->features & ARM_SMMU_FEAT_COHERENCY; case IOMMU_CAP_NOEXEC: return true; + case IOMMU_CAP_DIRTY: + return arm_smmu_dbm_capable(master->smmu); default: return false; } @@ -2430,6 +2432,11 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) master = dev_iommu_priv_get(dev); smmu = master->smmu; + if (domain->flags & IOMMU_DOMAIN_F_ENFORCE_DIRTY && + !arm_smmu_dbm_capable(smmu)) + return -EINVAL; + + /* * Checking that SVA is disabled ensures that this device isn't bound to * any mm, and can be safely detached from its old domain. Bonds cannot @@ -2913,6 +2920,7 @@ static void arm_smmu_remove_dev_pasid(struct device *dev, ioasid_t pasid) } static struct iommu_ops arm_smmu_ops = { + .supported_flags = IOMMU_DOMAIN_F_ENFORCE_DIRTY, .capable = arm_smmu_capable, .domain_alloc = arm_smmu_domain_alloc, .probe_device = arm_smmu_probe_device,