From patchwork Thu Apr 14 18:46:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12813880 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6F3EC433F5 for ; Thu, 14 Apr 2022 18:46:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245117AbiDNSsl (ORCPT ); Thu, 14 Apr 2022 14:48:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52718 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237531AbiDNSsi (ORCPT ); Thu, 14 Apr 2022 14:48:38 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2087.outbound.protection.outlook.com [40.107.220.87]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 95BE4DB4A3 for ; Thu, 14 Apr 2022 11:46:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=B6e86QUUtekzQ3KCTB+NM1UXJdqiZLmBDJWjmvbCYIvD/6f8QQ5Klkb41XfjPgvpxoGiTKFFaYz05ojqW/aMZGrD7DqVs7ArdOBQcrDRtg99Ifxprl2Oe2TqfQ3E2SEMmEPaSRlGkFPFJhonmNU7akjNXhvlGyfABY454pzp5qKBIQgYcJsc0RUy0MpGtdeZVsI7neYYl8WPVMIsHDHIBEVoIJfMM3fWVkRn6d4wk6/wqJHmECcDIh6uvxUz6RGkCa4szErvycjEtiQGMPZD4YcKpHQFoaOlggDAJxYSaNp9gY0QBlNSUpM5KadJ/0HYHd7eSUEiyrR6pKSTsatoTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=O/SNkKGKalrgBn8yEcmVHVUP0zmqxUg29UXwAH85OOM=; b=UQfmKmNKAuF52VhmRA/g8mkiIDl0Iz+LmwAIA7Se+oFO/qmbX2/KeJK9JajM8xezBL0MrbMGof8PgkJDUiVDdGvaZV+nHQj64Ogkxd5VF97fG9CS4ufiklSmeEYlsLAEV2kXGuOe6u9TqVuV6lUhmToMc8OaF8Wx6bsilcM8sxq2CXkUGrFDVFyjAfxNrxLYwFpFYXxnJxTRf438mkWjLVVUUYYoNZK/a26e/CE6Uo8RX145epO83KHbPjyhjwh53hSfpB20Hk8ao3j6uIWg01v6pGfvvRRM5jV2xiYkSo19RCNW7fm9/9/oRAydJ6ndCkIB+JDnVPLOLlZBJsE2kw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=O/SNkKGKalrgBn8yEcmVHVUP0zmqxUg29UXwAH85OOM=; b=jjvjhPMsrITrwI6OD9yTo3k5Y/CCk4xeMMJK7RFBb0a8p+VtbJiQ6rXcYFetgmuwL+w3GEXcX4NTSiKR5xw/6KQgdLoVk0ejdGRudB+7V3LbE9eIZbLiryNSHj6L2qEQZRrGN2bQYPO0dfDDp2l0Tu12PLItmRJlaiZQjMA60pDqq9AceXnFzpkIyugGAFtW0PsoiDsIGaO4Ez5E6cpbiUQZKLOf2T+/n5G6pQGywgAQ5i3Y4lJtfkcmQIjmQJSR1CYK5oH/6Hjn8sH7Oceoq5L+zNis/nkHYEKd5Upu3c4j3xb4hjSobGS7Q30SAgAlSlCCkMZOBTAm8mau+HNgGg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BL1PR12MB5350.namprd12.prod.outlook.com (2603:10b6:208:31d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 18:46:10 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%5]) with mapi id 15.20.5164.020; Thu, 14 Apr 2022 18:46:10 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Yi Liu Subject: [PATCH 01/10] kvm/vfio: Move KVM_DEV_VFIO_GROUP_* ioctls into functions Date: Thu, 14 Apr 2022 15:46:00 -0300 Message-Id: <1-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR02CA0058.namprd02.prod.outlook.com (2603:10b6:207:3d::35) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c4f854b9-8dbf-44a8-6284-08da1e470afa X-MS-TrafficTypeDiagnostic: BL1PR12MB5350:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HRUaRpQDuR9uttGn2TrJ/XbPIWtNg2uCoh0X41vnHyHxBaLsqL3K/iNeb/hqG7ZGZXw2JgNc+DzWAzbzoj6dwLe+t6p1ZcygIJ3AZ5d1I5DQ2YVXegOBE1vuDujB0xmazZlhGxEZPQ7YUHQS1qjhKdo6GuWsg+Por9HxKiQAhvGun3caonjOldpd/4UTLcSrxOSYcY9yZexChQy0QfPuCK1QSAfDW5jpKczGSKLa7vfO4twFPU5DzoO9jYBibJCPKWHsIzqnriguQcSFU/SLOxnApizWpTdVfjqORR9O8N21hwxWEhXimwHVSFywHb8eIDwn0khYyli9BvTkJjj9m+sRPn5fy5ds/L/Zzi8wUKrO/EIRI+hzu1JNzj7g2ATmRRg0JZVwv9bunz8QWCEzQalFjXWNM0J/KypfmhzgaDLRQRN0EEKt/SM4LO8uAOIhpF177/jFTdoO10QIAm6WKnWkfcjtU6LOCYHosm0+Oel6qKKeXWDvqx9VawIl1vSaXRLdDepxcG97Pq5b/YB95zGzhB0sWo8UxzVH51YGeHNCdp7Sjfad5e6cxo9VDjL0QzPnlJiQH4SdFd7lQ2qm2wIUWayh7/zhJLHdln4JU/fs1nuG9x0BAWLAjr2iXNMoJZPJbQ94SDulhamwMXDysdh/S3r+eBio8uc82W4oxx5Ut4bTszIus+rr6npd/vp3 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(2616005)(86362001)(186003)(38100700002)(83380400001)(110136005)(66946007)(66476007)(66556008)(8936002)(4326008)(8676002)(316002)(5660300002)(54906003)(26005)(6506007)(6512007)(6666004)(36756003)(2906002)(508600001)(6486002)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: xZaJ7/l6PEIWKadt90JAqcfmNspUSEPrtmPhcgNSrsn+aGIl2Oj4piS9+uZaeZWA77d90EIqm01qBXXwznPH8tSqwnofifudbJyPT493OKzvucb43y+IcmG3koB7FaeXka0MCag7uuBcAa6h2dq1Cf34SWOCIzE2JR3WiDFnhfk1uCzTNq7CUjUnvopTp+P7ny+E/G1J8f2+glZ00OPQWApzZIRPJ3NJJ2UhMtgvnct09OVGTcEck41Z7+ILh/GhSFoyH+ai4TwaLeI5de29ZgksAqv8wvneiAnDzjJw9x/M+nJVhFx4NI4bu+IQv5KbC2NEsPB5kxshstnDoZXP3koYcx72u8HCKq40w8xLgV0eVH3TE2BQ9rhBB1Bgl21SIDmC4HNcMFQSPTI6jQUJsXQCiRL/jTCNaNyrlmh1Frso+pfsK+TBq+Z4FNL39/kgM86Ah6iaQ2f7X4tXfQICJYASXPvg7u1Ykg/BoT9uQSdPO9Gvy3kDVj9j8eFw6Z4ETn+omFUPDIRN2cMMBBCBMCluv/knkcuveBdJUbpQVRX0aLBWn0vI8mERiSDuUobBV3ukcA/MovrHITigqcziPctZrBbnGP0UmS7jvOCIXtKBkiNKkTy7UPv+uqe1ofdjJFtP65rhTA3fFqrwla8IgQZPbZQRybbcnyS0AVnFsFNZumn1h2dyPuOkWzr0fGdANKgkSL+EfIKlDaxSwMfsedHynVV/Yh9Z9QUMAEB43Q5glNZ26nb0Y2sIZS6+c0ac5vbrlJBf4aLC2jREccaJKqbg+qV4SmGuHdSknuyK4XqqHwrzjJUCgYn0B6pDSY1KSUYM77FQc4WK/it1lx9v0ZP8yLIKESVVi2d8rwkXzGC+m2a2U0d56iZyHWl57G5BN60DY0sLLG2mPVv3XirCl35545M9X6z2VYJRL64Hc+Is6tcKHQ7Y1TkK/A0xhXYBbUnMk5djo/Y13QJ8gp7nFY8ZQOUERe5n3bICukxW1LUjyjpMIOt/bY5xGq0gvuL8IpzV5CORPOlcLhjClBg6pi0Sep4ldpjt2qZsYsXkstynYJ8j4M4y0dY4mAiQCdioaX27VRslmKZPsgw7kprpKwmqableLBtMpVk8Wk7sswvUa4gXJ9xoKCoSbN0X2FW7SAaAncSx9oUdlzcyiOurSvj0py8xk89A2SubsNOCz83s1fuGF49cMTwvBDcFMVlqM2dyvtVORIsR8pvdFWJiBXn9wDIEUnWPmUD3zdUPH652+WvWREdF7nLNUAz+u0EOwe1FaoNDMGbrf79xhzN7k3Nt3k0RPBiTRuKKZb1QRWDDTCj+TClHtZrbCAaQklqgZ1b6p23H4/J0wle8qFyUUtkwzFagXrUDBfLKTXsUVsECVNF8JAepX+38Y/rV342i2prpJM07o3movzqfPuscOn56Wgx5lhA1uNNAuLcD9WoUvdAk3KfzscTTUmBdMBXVoXVsIxN44yAwh9DPDjLrUB6TwH2Fc0HO0U/Yww3vbkYEHTcZBUdOnmSzWID1NkEOFsXmk24VnzvkC+M2vuLWGUiiunrIy19uq5INY+VK3m8UbHrULamkxLVE2WFLP9cgiFoCV7CJcsdoRVMTSfqfCOCtM1KG4vcvJaq15ZbqVZPZ7NeJ53dw1Ubw7yuhlYmcBacbOtGOHKfGiELexT7BsSdiH7m8IzqZXzvDZavknYgOIOdosMV/bhaHYKvlUO5FBa4QKCkxjSR7LuqERNiJ/w== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: c4f854b9-8dbf-44a8-6284-08da1e470afa X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 18:46:10.3757 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: qQV5tAcboHhY0063LVQDZ4MTWpVVasMxi1vHbvjx3vmGVzf/BwEyNQsTLyYG4RWg X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5350 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org To make it easier to read and change in following patches. Signed-off-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Christoph Hellwig --- virt/kvm/vfio.c | 271 ++++++++++++++++++++++++++---------------------- 1 file changed, 146 insertions(+), 125 deletions(-) This is best viewed using 'git diff -b' to ignore the whitespace change. diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index 8fcbc50221c2d2..a1167ab7a2246f 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -181,149 +181,170 @@ static void kvm_vfio_update_coherency(struct kvm_device *dev) mutex_unlock(&kv->lock); } -static int kvm_vfio_set_group(struct kvm_device *dev, long attr, u64 arg) +static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) { struct kvm_vfio *kv = dev->private; struct vfio_group *vfio_group; struct kvm_vfio_group *kvg; - int32_t __user *argp = (int32_t __user *)(unsigned long)arg; struct fd f; - int32_t fd; int ret; + f = fdget(fd); + if (!f.file) + return -EBADF; + + vfio_group = kvm_vfio_group_get_external_user(f.file); + fdput(f); + + if (IS_ERR(vfio_group)) + return PTR_ERR(vfio_group); + + mutex_lock(&kv->lock); + + list_for_each_entry(kvg, &kv->group_list, node) { + if (kvg->vfio_group == vfio_group) { + ret = -EEXIST; + goto err_unlock; + } + } + + kvg = kzalloc(sizeof(*kvg), GFP_KERNEL_ACCOUNT); + if (!kvg) { + ret = -ENOMEM; + goto err_unlock; + } + + list_add_tail(&kvg->node, &kv->group_list); + kvg->vfio_group = vfio_group; + + kvm_arch_start_assignment(dev->kvm); + + mutex_unlock(&kv->lock); + + kvm_vfio_group_set_kvm(vfio_group, dev->kvm); + kvm_vfio_update_coherency(dev); + + return 0; +err_unlock: + mutex_unlock(&kv->lock); + kvm_vfio_group_put_external_user(vfio_group); + return ret; +} + +static int kvm_vfio_group_del(struct kvm_device *dev, unsigned int fd) +{ + struct kvm_vfio *kv = dev->private; + struct kvm_vfio_group *kvg; + struct fd f; + int ret; + + f = fdget(fd); + if (!f.file) + return -EBADF; + + ret = -ENOENT; + + mutex_lock(&kv->lock); + + list_for_each_entry(kvg, &kv->group_list, node) { + if (!kvm_vfio_external_group_match_file(kvg->vfio_group, + f.file)) + continue; + + list_del(&kvg->node); + kvm_arch_end_assignment(dev->kvm); +#ifdef CONFIG_SPAPR_TCE_IOMMU + kvm_spapr_tce_release_vfio_group(dev->kvm, kvg->vfio_group); +#endif + kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); + kvm_vfio_group_put_external_user(kvg->vfio_group); + kfree(kvg); + ret = 0; + break; + } + + mutex_unlock(&kv->lock); + + fdput(f); + + kvm_vfio_update_coherency(dev); + + return ret; +} + +#ifdef CONFIG_SPAPR_TCE_IOMMU +static int kvm_vfio_group_set_spapr_tce(struct kvm_device *dev, + void __user *arg) +{ + struct kvm_vfio_spapr_tce param; + struct kvm_vfio *kv = dev->private; + struct vfio_group *vfio_group; + struct kvm_vfio_group *kvg; + struct fd f; + struct iommu_group *grp; + int ret; + + if (copy_from_user(¶m, arg, sizeof(struct kvm_vfio_spapr_tce))) + return -EFAULT; + + f = fdget(param.groupfd); + if (!f.file) + return -EBADF; + + vfio_group = kvm_vfio_group_get_external_user(f.file); + fdput(f); + + if (IS_ERR(vfio_group)) + return PTR_ERR(vfio_group); + + grp = kvm_vfio_group_get_iommu_group(vfio_group); + if (WARN_ON_ONCE(!grp)) { + ret = -EIO; + goto err_put_external; + } + + ret = -ENOENT; + + mutex_lock(&kv->lock); + + list_for_each_entry(kvg, &kv->group_list, node) { + if (kvg->vfio_group != vfio_group) + continue; + + ret = kvm_spapr_tce_attach_iommu_group(dev->kvm, param.tablefd, + grp); + break; + } + + mutex_unlock(&kv->lock); + + iommu_group_put(grp); +err_put_external: + kvm_vfio_group_put_external_user(vfio_group); + return ret; +} +#endif + +static int kvm_vfio_set_group(struct kvm_device *dev, long attr, u64 arg) +{ + int32_t __user *argp = (int32_t __user *)(unsigned long)arg; + int32_t fd; + switch (attr) { case KVM_DEV_VFIO_GROUP_ADD: if (get_user(fd, argp)) return -EFAULT; - - f = fdget(fd); - if (!f.file) - return -EBADF; - - vfio_group = kvm_vfio_group_get_external_user(f.file); - fdput(f); - - if (IS_ERR(vfio_group)) - return PTR_ERR(vfio_group); - - mutex_lock(&kv->lock); - - list_for_each_entry(kvg, &kv->group_list, node) { - if (kvg->vfio_group == vfio_group) { - mutex_unlock(&kv->lock); - kvm_vfio_group_put_external_user(vfio_group); - return -EEXIST; - } - } - - kvg = kzalloc(sizeof(*kvg), GFP_KERNEL_ACCOUNT); - if (!kvg) { - mutex_unlock(&kv->lock); - kvm_vfio_group_put_external_user(vfio_group); - return -ENOMEM; - } - - list_add_tail(&kvg->node, &kv->group_list); - kvg->vfio_group = vfio_group; - - kvm_arch_start_assignment(dev->kvm); - - mutex_unlock(&kv->lock); - - kvm_vfio_group_set_kvm(vfio_group, dev->kvm); - - kvm_vfio_update_coherency(dev); - - return 0; + return kvm_vfio_group_add(dev, fd); case KVM_DEV_VFIO_GROUP_DEL: if (get_user(fd, argp)) return -EFAULT; + return kvm_vfio_group_del(dev, fd); - f = fdget(fd); - if (!f.file) - return -EBADF; - - ret = -ENOENT; - - mutex_lock(&kv->lock); - - list_for_each_entry(kvg, &kv->group_list, node) { - if (!kvm_vfio_external_group_match_file(kvg->vfio_group, - f.file)) - continue; - - list_del(&kvg->node); - kvm_arch_end_assignment(dev->kvm); #ifdef CONFIG_SPAPR_TCE_IOMMU - kvm_spapr_tce_release_vfio_group(dev->kvm, - kvg->vfio_group); + case KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE: + return kvm_vfio_group_set_spapr_tce(dev, (void __user *)arg); #endif - kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); - kvm_vfio_group_put_external_user(kvg->vfio_group); - kfree(kvg); - ret = 0; - break; - } - - mutex_unlock(&kv->lock); - - fdput(f); - - kvm_vfio_update_coherency(dev); - - return ret; - -#ifdef CONFIG_SPAPR_TCE_IOMMU - case KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE: { - struct kvm_vfio_spapr_tce param; - struct kvm_vfio *kv = dev->private; - struct vfio_group *vfio_group; - struct kvm_vfio_group *kvg; - struct fd f; - struct iommu_group *grp; - - if (copy_from_user(¶m, (void __user *)arg, - sizeof(struct kvm_vfio_spapr_tce))) - return -EFAULT; - - f = fdget(param.groupfd); - if (!f.file) - return -EBADF; - - vfio_group = kvm_vfio_group_get_external_user(f.file); - fdput(f); - - if (IS_ERR(vfio_group)) - return PTR_ERR(vfio_group); - - grp = kvm_vfio_group_get_iommu_group(vfio_group); - if (WARN_ON_ONCE(!grp)) { - kvm_vfio_group_put_external_user(vfio_group); - return -EIO; - } - - ret = -ENOENT; - - mutex_lock(&kv->lock); - - list_for_each_entry(kvg, &kv->group_list, node) { - if (kvg->vfio_group != vfio_group) - continue; - - ret = kvm_spapr_tce_attach_iommu_group(dev->kvm, - param.tablefd, grp); - break; - } - - mutex_unlock(&kv->lock); - - iommu_group_put(grp); - kvm_vfio_group_put_external_user(vfio_group); - - return ret; - } -#endif /* CONFIG_SPAPR_TCE_IOMMU */ } return -ENXIO; From patchwork Thu Apr 14 18:46:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12813885 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0746CC433F5 for ; Thu, 14 Apr 2022 18:46:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344774AbiDNSst (ORCPT ); Thu, 14 Apr 2022 14:48:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52762 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243078AbiDNSsk (ORCPT ); Thu, 14 Apr 2022 14:48:40 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2086.outbound.protection.outlook.com [40.107.220.86]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3B08FDBD0E for ; Thu, 14 Apr 2022 11:46:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mJ4VUFqjNpF0qcWl0H5R1wUAiXO4mtQM2Wy/q1AEMqqRIDpNf7OBR5/Ok+pdrdWv2JWXdl/F2voh//cUNDVw7cZYAqFu3Re4GGMcDRJPxDBB2ttNx4mzkv0mAw8AqLQxeXVFXf8STM0lN/s7s98ihsM1TtY92Cy5BLVkWp3/SwHcj2zkzZl3ZXYkJyF4GHSZb4r+gA1njmtacAvygKwGIm+C9rufs5chNyHM/7SaIeqzi0vk3kmTUcn/eT7rM07ip+N3P4FCZQYRNunebZK7s+orKmH0VM+/hoSuTzMz01JvWnTCrzd9w+Jn7A2lSvlH0F9vobv7WqN3j7LV9MRs+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NEXEuNzQXJGQo/64zsNXYPhQ/3p/634HLl7StvnVBQ4=; b=Mr0hqPMcPTOStiqiHoxWLy8XzNy0qSYA3tj0jWc/Kv0HmQ2KpdTD5dj9f92uRnNbq/vqj51cTeFEiH4XFC5MzOcOvSjFbFgko+6rPZrEoDbKy7ZyxVO3LToROnF33uQKGOKTFed7J77lJ1VJN9jofrWK3NuEW8RFKOiQvTLbmK54wD5iB18RmQKgei5zKpWbgyelJA9mJGEhrbLHtY5zEBCQjij0KayhkOfcvZmV6m+kBPMD4GE0YWW8i+JctwrDEkGGLtCA9Qbw26sjoBSvrU96j2U66I7GGCLYtEqbEbXlkDWvz/BKrjmaXNH9z0DN50CeHJ+2NWvqSAS9BrzxKg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NEXEuNzQXJGQo/64zsNXYPhQ/3p/634HLl7StvnVBQ4=; b=eqWDI+ZkscRLmeXwctde8ljh8yPIqu+eG6ym1tcFk7RBGCHRfAvJ6K9Rd+SnMIVZXw2g1jjYih8CAAS+FF36iUEM/CvOKBVS+ec6lnlGhfwiiozS/3NR/+U5kRt7akkt5EVpAI2qJ5kX5DZnDto/sVw7emR38DuJ2Ymd+QszvjLhoOxqWxyb6fuxkHj3OFDAOxiUdagV3gI+AbCYWl7hS11k8F/NrPAPXL+HWPvgKPSkaxLDWcSnc7Mru+Zc+IIzu3ZuhmWhGhbo5Jt3Z8wzQIDDJcldAae2GEx0sGAx3W7FyKXfSlMnby9GmUg43kTsADPhWHXSugwIr2JyVwNr/w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BL1PR12MB5350.namprd12.prod.outlook.com (2603:10b6:208:31d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 18:46:12 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%5]) with mapi id 15.20.5164.020; Thu, 14 Apr 2022 18:46:12 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Yi Liu Subject: [PATCH 02/10] kvm/vfio: Reduce the scope of PPC #ifdefs Date: Thu, 14 Apr 2022 15:46:01 -0300 Message-Id: <2-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR11CA0012.namprd11.prod.outlook.com (2603:10b6:208:23b::17) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 08a4d47b-809b-4be1-433d-08da1e470ba1 X-MS-TrafficTypeDiagnostic: BL1PR12MB5350:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0G2Ni0ZwipN6bDmz5MMvwGW0+RCRizeYShJ5Kw8u79qrAuOcN1RvtvFYR1Y5n09yNLpT9JQ/IIHpQvoZFTg6HsjpIi6G8N6E0vcbEHPHUH13YQ8uWtQO8Ot0okkuI/LWXFKDryfPECW7QBjgLLsCF4WPBpVcvBnCQXIiL7wLlf9HKX1VhUkB98FhSSQ+t/LbJK01G1Kb9NtBBX0xaPRrE6E2Aym9oWiZoLnDSf2gp1g07p+qXaVtS1e79WhKDDEfbyR+QpzYkww3KM4Km/tdqeQKPR6ZNkOdTN/gSRbIfXxuGevIGRRpcG+kSstHqwjq/0eFi3a/Il2G5tWctVnKahsGplkFa5jqe1Fik6Yrl11kyl6LvycFYU1Jyzdn0bsvFffE98cs/fCL0YAYffnI1MiuJxSRMq88Yuodso1sKI4NtrvjsfDooZVfQDBD8378fGWnm9wJsSe0ryungQsTpoWAfOPkjVSE0tH2fjl6HQ3jcMLXURpssM+2Z1k4hNxJYCUoY34nD7QqfERre4X/v39q/Na3rF3MXXU7gPRrRrmZ+ZNMM4X4fZItAHExTxghtILcPSiZvs47UX+uypXB/011RzOOhtoAMwx0Dkq6Iigq3gGl7aAzKsN78KG/fCDlNa9NqVX1SJdeBqkiv8HULurrSImo6Ro+dZWbT4ECy2uoz23CNwg7yqNU7u3Cr1NSF6l+uQKG+45lqwwv4XwdPA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(2616005)(86362001)(186003)(38100700002)(83380400001)(110136005)(66946007)(66476007)(66556008)(8936002)(4326008)(8676002)(316002)(5660300002)(54906003)(26005)(6506007)(6512007)(6666004)(36756003)(2906002)(508600001)(6486002)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 0t8FL0+rwf//OXx80FSLEPImgO3/TdJ8MZ6M6t0PL2KuNGb/00HXQvdHcC8GUOiwBpS3FCPcJWvRbGMpKROGlzAu5Fd4gHO5xk4Fnb9nCUrzMYg6JK46f6j27R02+6FhTduF8+dSr0w0sgYa7mtRV5XJC98pHdE3y1dBWd3c17oWjVRx5atBjX27bvKa7vUfCnbltRCx2MSyl8iRii4KBcvZHeH2oxJVk3vLtZpqD6rymFZQPnDL0DSuMvCWtE1gAd9mU4HEglC2WtABu05q4bWEIX/ffO0C9Wja18/XVfN213gKWiJW3qbz3mMNMSGtq8e3CCwlWVXzo67kn4Ln5GQgkm0qc7daJVH5PSzpuj/NFkZxsfJ/3NE8DAsS1PCNTayOozI1H0tm2xuaBarV1/oM0CbKyPMKj9MLCGX2NKbpgTplK3C+aM7k9eZCPt0hHQeG6NnwOcsSvOMJHrEZlIE3uOkbjMASRGmOsgzlmE0MyEJqE4Ud8Iwn7tLyOa2geovzdhOP4w6DBEAk/+28dz+WLfPiGYtimta3cRylRZl7U6mFRymPKG6ZrRXNR/RCDiRiDTulguqDinjQbUHcFVs+c1qMJ1JIlaug889Zmxtr5N8ZGPx6SosYkdBLND9ujTdBU91xQPPbjuaE+am0ab/EORlRmu6IKi2wvoXVA5FY6hXzSzvsU0w76hQNTjBg/GZ8fzmEzTaMayCriS7yZGwkLIRXpWo6UUcBK9wWXrhrm1cr4gYfs3Q7pYo/m13RSc0FsqKhUFxXQrozHWXtB8PDse5M4CM8J1UCrnfJgJJ6e+33nzvYXol2Gcq61ACO72WCCWBo7H2DdETcUKKnkg8AnWXCmNQcoeue7TfcRmnoG121fjeZ/VF/aoqjcbdyW3pk6NFQkgp4lfBcTvp7376n+4jI9poRtU7uI/Px039xT/1QKEXwJwCEqAkosinBU3shbGYZCne0Ar0w9dB9h35vbYJBaLeeE97G63KIgz7Q24NVQ0NDagIxXWru1I48M3fcCmlr4A6oZO6zOvs0cLAOIVsn8CyPELNdKHixoRrmDTfQ7d63BDrN1lGIL1JeVqUQBuppnmw+loFzXC5i+v0UIYkVaqcZ2Lfh/A90/muPWkLt7YHUmqrVztcOfMqJ3IWRqd8GAMCMGvptHp6mZEkQaA5ECyYlKCaaQZTGj6pH3ljOhB1ts3sIxo5QT/tCTivZ9djGZYdZFv2iaJaC6h/oyJYTvh6BJeTlLhYXbuW8La159CvcwQ/pLPHixQMHz/iWxy9HQWqyu47dF3WMRYxS1VjUuURsQPSNqVr4RdoKri4/gT2rX73SBhW8Mm8mcKbUYNysiBp+OulXTk3F08Y3qiJPHQbx4iG+9lcmnD/kPmyAPVj+8keIKfdQO8qjUOZBfvmwUz3tTk8qoL6UQiR+66+V8JAnbmhrWIFs+Y8SqZG3sHGJK9FXaCuPhopyXIR5R8kpIujkSt7LDwu1F5CglbtwVOPpkQhozSeU6GKzzkX7FkgH4adPVgYolCtDv5Zyya034Se2PnACsCsXaNEORREQO9VCrUumQL+ypEwHAo2oTLFkKxU5rFetOV24+INMRafc+9Ag++EhD5Xwr1jWNpS2mHphHHaelBuRbl+vRPEHZWCh5WnY6mwhIKeg9HAVIo3ZRU8FiaZ5Tw+7u8UJQZZHhWyrrq/xjSCdB4PFNAqcds9kVlW/9DhC+uxyqm4PhYuzeOKfEAYH6qn1sQ== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 08a4d47b-809b-4be1-433d-08da1e470ba1 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 18:46:11.4694 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: WpsMU+igl2XKh0MMeiCAkILH5CuJdCzZK1sBvJZFImY+7EA90VsTzVQ9cql3SYQL X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5350 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Use IS_ENABLED and static inlines instead of just ifdef'ing away all the PPC code. This allows it to be compile tested on all platforms and makes it easier to maintain. Signed-off-by: Jason Gunthorpe --- virt/kvm/vfio.c | 34 +++++++++++++++++++++------------- 1 file changed, 21 insertions(+), 13 deletions(-) I'm not a big fan if #ifdefs around #include, but lack a better idea - these weird spapr things should not be part of the generic kvm arch API at least, IMHO. diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index a1167ab7a2246f..9b942f447df79d 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -19,6 +19,16 @@ #ifdef CONFIG_SPAPR_TCE_IOMMU #include +#else +static long kvm_spapr_tce_attach_iommu_group(struct kvm *kvm, int tablefd, + struct iommu_group *grp) +{ + return -EINVAL; +} +static void kvm_spapr_tce_release_iommu_group(struct kvm *kvm, + struct iommu_group *grp) +{ +} #endif struct kvm_vfio_group { @@ -106,7 +116,6 @@ static bool kvm_vfio_group_is_coherent(struct vfio_group *vfio_group) return ret > 0; } -#ifdef CONFIG_SPAPR_TCE_IOMMU static int kvm_vfio_external_user_iommu_id(struct vfio_group *vfio_group) { int (*fn)(struct vfio_group *); @@ -137,15 +146,18 @@ static struct iommu_group *kvm_vfio_group_get_iommu_group( static void kvm_spapr_tce_release_vfio_group(struct kvm *kvm, struct vfio_group *vfio_group) { - struct iommu_group *grp = kvm_vfio_group_get_iommu_group(vfio_group); + struct iommu_group *grp; + if (!IS_ENABLED(CONFIG_SPAPR_TCE_IOMMU)) + return; + + grp = kvm_vfio_group_get_iommu_group(vfio_group); if (WARN_ON_ONCE(!grp)) return; kvm_spapr_tce_release_iommu_group(kvm, grp); iommu_group_put(grp); } -#endif /* * Groups can use the same or different IOMMU domains. If the same then @@ -253,9 +265,7 @@ static int kvm_vfio_group_del(struct kvm_device *dev, unsigned int fd) list_del(&kvg->node); kvm_arch_end_assignment(dev->kvm); -#ifdef CONFIG_SPAPR_TCE_IOMMU kvm_spapr_tce_release_vfio_group(dev->kvm, kvg->vfio_group); -#endif kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); kvm_vfio_group_put_external_user(kvg->vfio_group); kfree(kvg); @@ -272,7 +282,6 @@ static int kvm_vfio_group_del(struct kvm_device *dev, unsigned int fd) return ret; } -#ifdef CONFIG_SPAPR_TCE_IOMMU static int kvm_vfio_group_set_spapr_tce(struct kvm_device *dev, void __user *arg) { @@ -284,6 +293,9 @@ static int kvm_vfio_group_set_spapr_tce(struct kvm_device *dev, struct iommu_group *grp; int ret; + if (!IS_ENABLED(CONFIG_SPAPR_TCE_IOMMU)) + return -ENXIO; + if (copy_from_user(¶m, arg, sizeof(struct kvm_vfio_spapr_tce))) return -EFAULT; @@ -323,7 +335,6 @@ static int kvm_vfio_group_set_spapr_tce(struct kvm_device *dev, kvm_vfio_group_put_external_user(vfio_group); return ret; } -#endif static int kvm_vfio_set_group(struct kvm_device *dev, long attr, u64 arg) { @@ -341,10 +352,8 @@ static int kvm_vfio_set_group(struct kvm_device *dev, long attr, u64 arg) return -EFAULT; return kvm_vfio_group_del(dev, fd); -#ifdef CONFIG_SPAPR_TCE_IOMMU case KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE: return kvm_vfio_group_set_spapr_tce(dev, (void __user *)arg); -#endif } return -ENXIO; @@ -369,9 +378,10 @@ static int kvm_vfio_has_attr(struct kvm_device *dev, switch (attr->attr) { case KVM_DEV_VFIO_GROUP_ADD: case KVM_DEV_VFIO_GROUP_DEL: -#ifdef CONFIG_SPAPR_TCE_IOMMU + return 0; case KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE: -#endif + if (!IS_ENABLED(CONFIG_SPAPR_TCE_IOMMU)) + break; return 0; } @@ -387,9 +397,7 @@ static void kvm_vfio_destroy(struct kvm_device *dev) struct kvm_vfio_group *kvg, *tmp; list_for_each_entry_safe(kvg, tmp, &kv->group_list, node) { -#ifdef CONFIG_SPAPR_TCE_IOMMU kvm_spapr_tce_release_vfio_group(dev->kvm, kvg->vfio_group); -#endif kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); kvm_vfio_group_put_external_user(kvg->vfio_group); list_del(&kvg->node); From patchwork Thu Apr 14 18:46:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12813882 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B767C433FE for ; Thu, 14 Apr 2022 18:46:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S245484AbiDNSsp (ORCPT ); Thu, 14 Apr 2022 14:48:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52750 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241689AbiDNSsj (ORCPT ); Thu, 14 Apr 2022 14:48:39 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2086.outbound.protection.outlook.com [40.107.220.86]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6D0CDDB4A3 for ; Thu, 14 Apr 2022 11:46:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FcbMO4hIVmQYwrK/4YyD0ZgD6kOyzDivWDWzQJEJUGIo9sE9GKFfTlC1EayzoudhG6m96Gb9r6EpRLVdrmHzx5hs3mOESCuKHURZu+hA+u1VRVsclqO1L1lwcIgrmA7w7EDiq5JPxEeOnhTCU6d4dYEzq4YtcTXUw5wAHhlqLjVHoNeHZscfnf06YMov/4P4j9FhDLB5TZGca4rg3X8Ccs6Vlia2GRlz51B7jBgHoTsqDh+g2xu0oBzgqLN826uEiXrg4NIXGtg5cf25d4UQ7ZXuWdcTXxyX0QwYD728NckNkPUeeKIQbkahSraJSdb7UsA2+74Tkrn+9xBcHc6p1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vGtSlZNRZeoDsrjCSeQUZLNLc/2rz2tha0i7ukvfd9I=; b=FUWRsOSjJKtnOW1TB38wH6sUJiALgmDRDgMtHAhHFpIiTpjmqrDbUaCjulLCXDHqqK6elz+jlm5DY4Ux27abWW4hK8jb7Nw7uQ4Qz5KFseSFn7r14zOUC4pZQWKQHxaLqaSdh4MwtOkBPyg5G10IJtf5zUHHpMsnk4c7oHZbf0xKc50HYspjtT7aK4aW7lyG4UYMZkK0UzmSDJdCitPsm0aqI/ZMZCFbumgTneJht/AsiyO80Lk7PjkkEqZhapmsOFsxJXsm3qrhkkpkI/XBfPtDWDkfdytTJ6Y0dVIroPMi0MeNrBbOvlg2AfXomfSFvGMW0cOIlPGSaKIMSAUrog== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vGtSlZNRZeoDsrjCSeQUZLNLc/2rz2tha0i7ukvfd9I=; b=GFv8Kh4TEHvU4/SFojX3FbzIuA7kqdqtIU8dEGHviWQIkMUWOCLj8G/4a91zokiBoxTYliZTPELabMJy4W3mIAotZmCjDoDLNZS3V4wauGgCjgYu6WXNAMPVFXC4enL34199QasbVYguUbV34YGxPlUPcZw4GKQYQxFN1PbyWOXUIgO4LGd/Zp/mdMqPdPyOF+TMpogP+FxsFv/aUD6IBLvjF1A3KicR5OWZeU94/LIyLU0Tw/BGuY8PAU3kRg3Fgc73wjMxozGael0uAhzKGbT+Zia7DxehFYDYVu9fLZipbn8i3MIXIjg6AcRzPqtKrVpgkQpA5J8GRKyQPE6x7Q== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BL1PR12MB5350.namprd12.prod.outlook.com (2603:10b6:208:31d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 18:46:11 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%5]) with mapi id 15.20.5164.020; Thu, 14 Apr 2022 18:46:11 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Yi Liu Subject: [PATCH 03/10] kvm/vfio: Store the struct file in the kvm_vfio_group Date: Thu, 14 Apr 2022 15:46:02 -0300 Message-Id: <3-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL1P221CA0009.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:2c5::24) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a45cd342-6b5b-4686-afe9-08da1e470b08 X-MS-TrafficTypeDiagnostic: BL1PR12MB5350:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: EwqP/qWP8EoGsY3jFFzasK0YZ1teLl1ajX9Vf1KGExnz1rr2PglzMiNX+nCjeSzluj8DlBQxLjfyOV5QEbom3Xx7iPqe8Pqsru435fdqA+bn1hIO2pOxC4xFeadx0k870kmHdLszReB/j6Oxb0qinuPz8LVcBskCYZeSj+ER3dsGPXZ3pIYusYKqKjrhBfWmyB1F4hQu/yF/qoU1D269SlAvm6lQdVLFw9hp+L0mBBaxINBXOpNkACLLxk7alnzDxRZ+qpOpV8PONCvNUPIJg4xU0KN904+jsvfHq+2iPS3FfN+O7E4qrIKm4LLYdszUE9QPgx9QnZlMYUSWxr6zRRG+MkemO2QGY63Owu4k7t4iq+xiHco30p+tuP7kTeHICb5ZQdDdQdBA17HoTL7GommTWlHDafyw+oQEikycpudVfdX90mshr4pf2Oy6vl/JGyj/q2Z3W6+9wkExnLSV9cfjV4pDv79KPQ1c3e4nTkPDoQJuxn6zLtF4GJzzoLmTzRLrkdVLjWK69l/MVUA/uiyGYOcy7iIy0iGKY65THH6VrRq1Mp18fwChjS58mDE1LMelDfD02r5hXSl8+PtDVhCaa7TGX7swmxEZ9i/AfEXK93fnMC5E7HjY8JNknwL2fu7LR1ytdHe47VGGj1+zqbNyEQU5tukpaDUrNa8lintNshdNgmWbR7TDuR7DmsuL X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(2616005)(86362001)(186003)(38100700002)(83380400001)(110136005)(66946007)(66476007)(66556008)(8936002)(4326008)(8676002)(316002)(5660300002)(54906003)(26005)(6506007)(6512007)(6666004)(36756003)(2906002)(508600001)(6486002)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: mTcFp2a1qhEhrqMudTarAw109ZLiM/RDIBq57DdxZI2NVmpzGMsAUu2iNXmKQpKrurr+lhQi3HayKDR0CWp0fq1RN0S7/7qQxD5T/kATMn7yD3Oc8/6zVgiDLu8NSKmwdCcJ71h0LZyNNb85gX0pPEVkVnEIfJdQinDTW/j7Pt1OO6JIiRHiPCWtvFcJwtO8Ja5MifKKKe/GTWwi5l3G3EcSWaSkEzTRqOrWbhVyGeiqGhfQVd3zLit5UDf3aC4y+2hKis8bnJDw+2mDCpkw43XdJ/jyGjA1aZhwx/ywxdk9lzLxrJwPkez/i7v1N7mANLbfiARCSjocrV3FYZo12yU9jRVrcg4f5oF/AUEb7/mMslaNNvucSIaaGdzh0VRxq+gNO9ZCncEQb0OIq175VJVe6Zaua2SmfPcMv4FXwD+piJeWvbAnYt15Bq3zGWLD0LWwwRa4DL0a6RpAezZv+kFwww1D6m5svTs2Gp/Ci4BtdjirnwHZE1iNGtGJsQnjuBY2Qlcypbquiic3p4sv9ELkwhrG8k+mFbvIP+smOoss7SuKaSyooaJyObU3cyr0A0C2xWx+csy2at7H+w2djZEG3+37wjky38OVYi6ah0sNUICK2vrGYGWDKkb5exPTl8tMTZkLVkfTJ2VnLOGgoQbj8U1e8ABFkHdHIOLP/q1m3VK7n59TBtSbEpFTsaSaezU80wp0U6sdmvVRiuMXgbNuSBtDbLOp/1VTxecy4rCyI0HGoDo/3HTzohVoT1ixY3PSuwY9Gl9C7PMWP8KSROPZ9gsvUfd1Vyzj9ynoQxhJ1ii6GdE/uBMGoUpCWJK9pUJmtNqri3jXKSfgonpAFOse06WBxX46uinbX6vARWoQ7VeoBfBRjA8wNQcHj+1oRqGx9Ke2yMpi4/HpHhpTbNPGhX2TZ+qjdzj0ZZJPiWeDIyJpsCAAILH9HFoG9pc2waDcQ007jF8WgoKLzbZPRLmeeg538mUg+7knPH8OCWww8vNpVLKHtUV4w/k292rmFAE8/N3vqptECxWgAkrw/Ha12IXtjmXn+KvPrVQ+qyCoJxi4eqXM0lBqLemDezJ8oqQP1vmrN4UPDyju3bbk42g4k/3L2YPPmuCdrnX+p8pBBO7UwYR+y0cyo3G+QAu2y8K9yt544d3i/mc/5gc37Enydux/0zUPuUV8MH/G6GpbwsIOjblHfsgbiSidnBbXN4KEvPTJwe5r2DgC9ciItTf8OJVHxA6UlN73fla3uRTtfH4xG6E3eQpkEJSJb3MG10o9WsQ1xamtI3eV3Zz+mKKHhgKkju1IV2/DtZT/4iwv/rd4kY4BDCNnzWz3VVg0lZW/ulDM9921s/cS4JkVRZPQ3Ruorcz9PPBJr26xXUaCDb8tlXk4cc/5WrisPwnLyfnvMgsje9fLcYreulzqzdyjFseyKuJrnfPOYmzNY48DMWtZwWQKSA1EuUui6MRFClzAeITblfzfg3YEaTRVir0EQLGylO6hFSptpOingkQvyq+zAwkjWqmQc/S8HmTXnthlbM5543bF3QJ9sfThGqDJBxvSINVGHJxGIbCs+VCWUgxd3iW0UaLZsPs/QduL9yyHbVhkSWsfdRAsRM5ceVNlvmtQgZth8Q1m5lL1wZYoH9JEOET9jp/SSQR9Bbk0H6MbJ8gZXcVOBYzzr474SRAV0Yd/+Bj7+M78TrymRBRgn7/7erlucGvi1fi2aVlNiW1W4vfF0pO/HG1D3daDfA== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: a45cd342-6b5b-4686-afe9-08da1e470b08 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 18:46:10.5163 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 4pVVpu2yaElKZF4msOOhLZPDiwWT4WOH3cmCQ2mVjFEMaSveitsShtrNFLb9hqe7 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5350 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Following patches will chage the APIs to use the struct file as the handle instead of the vfio_group, so hang on to a reference to it with the same duration of as the vfio_group. Signed-off-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Christoph Hellwig --- virt/kvm/vfio.c | 40 ++++++++++++++++++++++++---------------- 1 file changed, 24 insertions(+), 16 deletions(-) diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index 9b942f447df79d..7e1793a1f5f1fd 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -33,6 +33,7 @@ static void kvm_spapr_tce_release_iommu_group(struct kvm *kvm, struct kvm_vfio_group { struct list_head node; + struct file *filp; struct vfio_group *vfio_group; }; @@ -198,23 +199,17 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) struct kvm_vfio *kv = dev->private; struct vfio_group *vfio_group; struct kvm_vfio_group *kvg; - struct fd f; + struct file *filp; int ret; - f = fdget(fd); - if (!f.file) + filp = fget(fd); + if (!filp) return -EBADF; - vfio_group = kvm_vfio_group_get_external_user(f.file); - fdput(f); - - if (IS_ERR(vfio_group)) - return PTR_ERR(vfio_group); - mutex_lock(&kv->lock); list_for_each_entry(kvg, &kv->group_list, node) { - if (kvg->vfio_group == vfio_group) { + if (kvg->filp == filp) { ret = -EEXIST; goto err_unlock; } @@ -226,6 +221,13 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) goto err_unlock; } + vfio_group = kvm_vfio_group_get_external_user(filp); + if (IS_ERR(vfio_group)) { + ret = PTR_ERR(vfio_group); + goto err_free; + } + + kvg->filp = filp; list_add_tail(&kvg->node, &kv->group_list); kvg->vfio_group = vfio_group; @@ -237,9 +239,11 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) kvm_vfio_update_coherency(dev); return 0; +err_free: + kfree(kvg); err_unlock: mutex_unlock(&kv->lock); - kvm_vfio_group_put_external_user(vfio_group); + fput(filp); return ret; } @@ -268,6 +272,7 @@ static int kvm_vfio_group_del(struct kvm_device *dev, unsigned int fd) kvm_spapr_tce_release_vfio_group(dev->kvm, kvg->vfio_group); kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); kvm_vfio_group_put_external_user(kvg->vfio_group); + fput(kvg->filp); kfree(kvg); ret = 0; break; @@ -304,10 +309,10 @@ static int kvm_vfio_group_set_spapr_tce(struct kvm_device *dev, return -EBADF; vfio_group = kvm_vfio_group_get_external_user(f.file); - fdput(f); - - if (IS_ERR(vfio_group)) - return PTR_ERR(vfio_group); + if (IS_ERR(vfio_group)) { + ret = PTR_ERR(vfio_group); + goto err_fdput; + } grp = kvm_vfio_group_get_iommu_group(vfio_group); if (WARN_ON_ONCE(!grp)) { @@ -320,7 +325,7 @@ static int kvm_vfio_group_set_spapr_tce(struct kvm_device *dev, mutex_lock(&kv->lock); list_for_each_entry(kvg, &kv->group_list, node) { - if (kvg->vfio_group != vfio_group) + if (kvg->filp != f.file) continue; ret = kvm_spapr_tce_attach_iommu_group(dev->kvm, param.tablefd, @@ -333,6 +338,8 @@ static int kvm_vfio_group_set_spapr_tce(struct kvm_device *dev, iommu_group_put(grp); err_put_external: kvm_vfio_group_put_external_user(vfio_group); +err_fdput: + fdput(f); return ret; } @@ -400,6 +407,7 @@ static void kvm_vfio_destroy(struct kvm_device *dev) kvm_spapr_tce_release_vfio_group(dev->kvm, kvg->vfio_group); kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); kvm_vfio_group_put_external_user(kvg->vfio_group); + fput(kvg->filp); list_del(&kvg->node); kfree(kvg); kvm_arch_end_assignment(dev->kvm); From patchwork Thu Apr 14 18:46:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12813887 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67285C433FE for ; Thu, 14 Apr 2022 18:46:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344765AbiDNSsx (ORCPT ); Thu, 14 Apr 2022 14:48:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236852AbiDNSsl (ORCPT ); Thu, 14 Apr 2022 14:48:41 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2087.outbound.protection.outlook.com [40.107.220.87]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BD28CDB498 for ; Thu, 14 Apr 2022 11:46:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=h4zoa6/th40iCi7eYivr7a8dEdSNTp6b5NlIlmL1dyMeh94hS3KOanys1oNvTYX7Fa8oMFxo01aqSMhrs9dd51FgbWDku1gStrdtHy3tZdAsGF3mhLgDd+Oml+0DttAijWS5Nnh8iyKazDbTZ2ciUayUF3SUzPOtd9WPv2kM3BCPB4M3UlLev69jXRnEZNZ1dVzE+AK+1EFzNaWfYriDrrti3sY6iLXjXsrIC6IhIgy7xv+x7WyLE4tvixdwTp2D1A6gPqaUOFgD97FnIfjWwrg7Fu6Iq4qAcJjmlNHKw99ywHDzBMkz0C19G3TgoFlR8/Kx+kTErY/mCbuKyilkNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7CLxETMMldOr4VkojCs1afE53FK2CKd42RuE+PWxnUU=; b=hGVLlfvSAVCa4eubmmOSqMZf49qR2xhzQu0hk+gv7Wud4C5t1w7RXZC/uF5Ar0qKdkY4JxsopyemEELNjI+w9CEfy1ezk6TV5QoNY/DOw4eDVzsm03lOtZk0zeWdYJOukb13J0O7fDpsAb0gfKP41O26K1hUFA7DqtBq1u+w/xKxi/NY9NrENmN2QhmBTq+YXQiwrmUln2b4xf2EnltCitEkABQ5ERgJS15NtlrxHV82PB1VS+Aqq4VEixxDgHld0YvAmvr/CkAvZsIhMoeVPHTR7ubpiStN4qICT1rufkllizbf9O1oQCAw9/ccTBr9mAo7IHg3VwDtljxuA7ksMg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7CLxETMMldOr4VkojCs1afE53FK2CKd42RuE+PWxnUU=; b=tPXtrxf1pVp4c69uZc2JWbbgu1sUoLJk/KToIUqF47jTcwel2PSLaroIpDN9mv2l2NclT0dboC9v+l1Q+06iu+IESRUWgSPnRTxbubrepItG3hqjsr9aG9fLgJXuqZpj5y2dCcVv0G8MBgkQxmKw8kDNiWCHg4Mvlmq3TUc2ES/m2rHcFnoBd+92LhlBzTVi4nnasPO4DXyKX3aPzrJSIiSD4aLIbZ4zHsb/hGjCU5fEphpa1wyVjvSOXXlUbfssZ4Nm9cGV+3yravYi58CZ0Jhi6x5J3m4un5wgfUUEIUSuhSOvaTaXkJSETuQs7JnCtQpbsf/MMsqhw088PecwpA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BL1PR12MB5350.namprd12.prod.outlook.com (2603:10b6:208:31d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 18:46:12 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%5]) with mapi id 15.20.5164.020; Thu, 14 Apr 2022 18:46:12 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Yi Liu Subject: [PATCH 04/10] vfio: Use a struct of function pointers instead of a many symbol_get()'s Date: Thu, 14 Apr 2022 15:46:03 -0300 Message-Id: <4-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL1P221CA0020.NAMP221.PROD.OUTLOOK.COM (2603:10b6:208:2c5::7) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 30394e9a-690b-47b1-ed23-08da1e470bef X-MS-TrafficTypeDiagnostic: BL1PR12MB5350:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6dHusWEm7uSe7FcVkbzXY6v1/Mt//eQtS6+aqkaPxL5Rax7wKHrArO9Gx7zf8NaTokw6EGddtwXyGaGH0Kiihd+QybLs3XtocyVmZNVnnbFywufvGkfdoRp20wMC2Mwkz9rf2aMbjlwYwvlswey1V52epYrI5DzO/Ot8WXYQpnmaTuskHciHARTlpEPDZBgGWolUYgnf/MEWkP3romd0dRLNZvWlQyMWcGqj1sHjQXboVoXs+6/KZFASpae/wxlIP8VYRXkw6CFB7b+FD/ngmxfWfY0dpYWUS5RPgQGdodLcAK/4wvJDLq3LtARj5DU6V5Gq2f4btZu1WkutFLxCJnhnukln8cDHdt9U429Uw6kZQcvpvBVA65/8wX9asfjNZSkF36NUZ3csZt9kS3EB3axBTAY68GAIpTsqKOHbYiDCiVkb53sKvWb70UCw9y60deqz+uI2Z5uxcjEdrKny5Kg0BqH58u1hlcUMt0/3h1cOZF1z9gufwoaRwrXfdynkA7RYd4b0qP9myDBozAAqKn1wBqSOBbuMrUyUr2iWC2kzJC/NXaS3e6XRaps9XEIIZrI3LwKn1MAdSm9xPzCQDDzicgfgGyMZmC9dDV0qaXY7ksMuPiKth3XoBcO5DkibRhKK2f8rPIArs0TzWWQnOQ0TJIMCIrt1LAAUBY+LbPVywzUlr9CHW1yTYIKvTlad X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(2616005)(86362001)(186003)(38100700002)(83380400001)(110136005)(66946007)(66476007)(66556008)(8936002)(4326008)(8676002)(316002)(5660300002)(54906003)(26005)(6506007)(6512007)(6666004)(36756003)(2906002)(508600001)(6486002)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: TGaZdQU6s2p+n2jvCqPMWjEtYZu4BpbMKH5D9Uy7+rWF4K2fKvbj577W4RtdM3oYQCg4JpbDTq3TFa+lnmZfqgl2/2cOrNP0FmN4RcThJJMxO0qE20JAuv0Eb6Z47Tsjxgt3XcfNeJo888qWyCjEEZxYZqId1GgRL+3TQ6R8E2W1Nsj7WUi85d+CS9lhK39T2CdeEOAhzWx8GFGVMd+oPjJuqQtxebArqvWDLquEXmu0BM/wGkFs4u67TYjiK3NkcTYagvXWFIVxhsNb8zrpJ6Uy61eIdPtTS/cA/PY51Bjrzs+yuOiL2O8QDl/MQdQnPLna+v1Y7KGCBIg7hyKNNiiQHig4O8+6x1withDG9w8dMLVifNIbx3nGQVoftUUiIHJztKfhYVvbhqiMts0msoFUWyR2LqVOgq7xuhqRy3SgrZj5EEE8prwj79/ZlnPAExpW6xDG6xJNqrhI7xHQJJxMN3CZpuIRubOMzqF4iFBBS7wa0uxuJhabKutv9MpHbdX4z75ha7RwScxF5N35jNDgUnlXQqA+OXIRg8WcbvzYsNI9FSHR25/4V04Dvzu9Ag015eWrXs+vDmLaGL8wzERDdDhR4Ybz0efhdGagbDyPwEoy1ojzds1rfm9SQm6mVmE5u4uU4g27vqXyO8xMY9iA5Gn/I3Z/7Dx/a5qY0IdGa+FwkpMtxfsKiwU/o/pPGCwNk1R20kNbi95Z6B6ahYwjCpnPhYjuwOpaG0vjRDQvBDm8y/L6EtLdb213/kSxjzs4GsjQE9b6w7EhVc26B/KzsA6srjKsb1VeDTlcOeKA4aNyCkMad1ZwWoHhSiOAj7peNNIqrysdAD4+6Ln2U5ervOtS5yGpEQgWJUdla75fXVBKCLWLgLqr+QBHiOX1SzYpEqF9F6YWb+bMO5rvRJSjPiPEDjemJH6AbQTPsnf6ldPRApQINVJ3hS1A7RKbgBNnHxiYAdPAkZWmpXKWHefE1R3QgA2Z4qIrpczmWNtw+AvDXMC5w1Vptl8t9AhKDl+uwWg1PPGUlYGY1aFqPYPkrPL0NaaSQdfdvvvC9yNkrcn4cczLge6uvntODPD6fTuwnUEdOcrhkg0p1/Svl5ENLBWTUnX2xfGRmFe4QwJSwupv95adiBO7PAyts+bL4GVHA9YARtipQxC/w5hnYspb80/4OBZT/VKKlSOLlMtl9vNHS0+k04z/oqYdhd2MdcYh9JD7kgEBuyEOgYoNrXzS3haQRIk5O5poREKhTFusuXIu2E0P1Yk2JwqzCyFjina1cz88jKjR4zqZNdIk6YEJ/pQY+xUumRuf4qoGMY/Qr5A/OKgvEIh1NiIqPSzHZFfR0N4F8sQ8iWt+m1Rk3LTlb19llttTsfqETd5jW36/1FsMkRBLEa3+yCUyGK2VQOFUAn80KxwPabuwCR5F5m7kEHTq26yj1KfvOvOG08Rv/Vw4XWYsr5vgkkAHfGJciEw1Q+7iRJpO1vIP2N2phLv/1AlWMqnQ0ZOugklKl+9Eku+thLGK95CJryKhT8Rnq/G0648NhOKiP0GP5ClMqARZ12E/INW3bTt0zj3XABmc/90GbMrQHKBlzDhjs/VYpnn5CdU6BVHr+rRKvNLTmPQEhlArpb5I0Ism2Idgl73ugY6NNXob5UXqMq80RSTaQErgcT7DDA0Kwb9LgbXYQmLDyxL42ydSxIAWB/hNMu15hPnzcHIMY9ZRiQjiyQnGNplczWzZwHkHuouNcV+Z1g== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 30394e9a-690b-47b1-ed23-08da1e470bef X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 18:46:12.0318 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: K5mtGNm3iXhHnMdQzvis1mJZBmoPCPDIo0yBvffDOOsOkCwUK7s9nZNFYfFh2H1y X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5350 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org kvm and VFIO need to be coupled together however neither is willing to tolerate a direct module dependency. Instead when kvm is given a VFIO FD it uses many symbol_get()'s to access VFIO. Provide a single VFIO function vfio_file_get_ops() which validates the given struct file * is a VFIO file and then returns a struct of ops. Following patches will redo each of the symbol_get() calls into an indirection through this ops struct. Signed-off-by: Jason Gunthorpe Reviewed-by: Kevin Tian --- drivers/vfio/vfio.c | 19 +++++++++++++++++++ include/linux/vfio.h | 3 +++ virt/kvm/vfio.c | 14 ++++++++++++++ 3 files changed, 36 insertions(+) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index a4555014bd1e72..93508f6a8beda5 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -2013,6 +2013,25 @@ long vfio_external_check_extension(struct vfio_group *group, unsigned long arg) } EXPORT_SYMBOL_GPL(vfio_external_check_extension); +static const struct vfio_file_ops vfio_file_group_ops = { +}; + +/** + * vfio_file_get_ops - Return a struct of function pointers to use with the file + * @filep: VFIO file to return pointers for + * + * This API exists to allow KVM to call into VFIO without tightly coupling the + * VFIO and KVM modules together. KVM will call this using symbol_get() and from + * then on will call VFIO through the returned function pointers. + */ +const struct vfio_file_ops *vfio_file_get_ops(struct file *filep) +{ + if (filep->f_op != &vfio_group_fops) + return ERR_PTR(-EINVAL); + return &vfio_file_group_ops; +} +EXPORT_SYMBOL_GPL(vfio_file_get_ops); + /* * Sub-module support */ diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 66dda06ec42d1b..409bbf817206cc 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -138,6 +138,8 @@ int vfio_mig_get_next_state(struct vfio_device *device, /* * External user API */ +struct vfio_file_ops { +}; extern struct vfio_group *vfio_group_get_external_user(struct file *filep); extern void vfio_group_put_external_user(struct vfio_group *group); extern struct vfio_group *vfio_group_get_external_user_from_dev(struct device @@ -147,6 +149,7 @@ extern bool vfio_external_group_match_file(struct vfio_group *group, extern int vfio_external_user_iommu_id(struct vfio_group *group); extern long vfio_external_check_extension(struct vfio_group *group, unsigned long arg); +const struct vfio_file_ops *vfio_file_get_ops(struct file *filep); #define VFIO_PIN_PAGES_MAX_ENTRIES (PAGE_SIZE/sizeof(unsigned long)) diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index 7e1793a1f5f1fd..254d8c18378163 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -34,6 +34,7 @@ static void kvm_spapr_tce_release_iommu_group(struct kvm *kvm, struct kvm_vfio_group { struct list_head node; struct file *filp; + const struct vfio_file_ops *ops; struct vfio_group *vfio_group; }; @@ -196,6 +197,7 @@ static void kvm_vfio_update_coherency(struct kvm_device *dev) static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) { + const struct vfio_file_ops *(*fn)(struct file *filep); struct kvm_vfio *kv = dev->private; struct vfio_group *vfio_group; struct kvm_vfio_group *kvg; @@ -221,6 +223,18 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) goto err_unlock; } + fn = symbol_get(vfio_file_get_ops); + if (!fn) { + ret = -EINVAL; + goto err_free; + } + kvg->ops = fn(filp); + symbol_put(vfio_file_get_ops); + if (IS_ERR(kvg->ops)) { + ret = PTR_ERR(kvg->ops); + goto err_free; + } + vfio_group = kvm_vfio_group_get_external_user(filp); if (IS_ERR(vfio_group)) { ret = PTR_ERR(vfio_group); From patchwork Thu Apr 14 18:46:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12813883 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7983FC433EF for ; Thu, 14 Apr 2022 18:46:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344758AbiDNSsq (ORCPT ); Thu, 14 Apr 2022 14:48:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243792AbiDNSsl (ORCPT ); Thu, 14 Apr 2022 14:48:41 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2086.outbound.protection.outlook.com [40.107.220.86]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F037DDBD19 for ; Thu, 14 Apr 2022 11:46:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hL8vN1IVVx2I/iMwr5Mjfk83OuRQsQRHFCJM2wUVa5kjnvgxazEx637dbJt2npAgGyuUduaSYpYt+YT0lORIl1Fb3V8yiRkxgc8Dab14UhF15w8oeiFIdprGGRFwmRi0h6FJCmIAm+JHWdBCG+DWf2X0D0HAtNbqfaZPUCUYi92h+SoHWYqac0mmbBUONry4B7aCffNkS0Saijm7JghbFJTghivn0dmdLeSrrA4BPn45igBFaGt316lnDcgMEOVlwTZP2m4ZjYNbnFlYjGxEiVfRrmY27dGGNMFaqOgYd0IJFGdfPZB+O7mjvnxKH2Jz65o2zfbr0aR77m/qWhFgPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wD3PR8nXG8xboi9+HtgVwnkCapHVD1Z+eBp6VEY7N8Y=; b=ZeH/b1VJm7JGwVfMUvIhoSJ2PXMT/o5kmHjIGYTBxYmoaJiXJzj+Xi9cAME3h6EHnVgZeIYqIsR0HpVE/ZtgUyeFCiwtytV6xH1UrtqByyK9qLVSPF6dpOWKmYlf9GonPxuvA9GxjSR4DsqpCeE7upO916uOBWZeBL30LxSURNjo1O55gp/P2R5HqcQEd1fbdpDre1GONNZCmMkt4pqxPuUvP0DDkmFImcAOQ9vOrJ1m/+PuVULqsg/sEs2L4JsfktCgJxyOczMu3LrhO4W44TDyd+gaQH0tD+MrvuQcRw9+Dj/A/y/FTmUa9lpQSOLceU1KG5tej8LHorBb0qGvNQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wD3PR8nXG8xboi9+HtgVwnkCapHVD1Z+eBp6VEY7N8Y=; b=iLL0XdofCncukcO4OimxIhDmVtfz5fP6KIdgp/NH4zxvLl5zjJFlWMPdpoxyghRbmAP69zCpAor4o/drzCeFh0OhKlGT1EbC3qZeMaR+hFY5rQGqNu5MFx0yDN5ykRJUBpXhQuFxcUDyjVlMqGQGvwOBUnOaSjTr9dmPQ8TI1Vr4E9lpn8zm4f4pkWyhwM1Wyqk7IaaFWDDQCjnBgRBnIPP6VRAiSFHw3v38wWOYbeelYCoibdT+KVeOYeBogCE4x0TE6/44Yk1rHoy+55hLRrz2jItoYb93HsrIPOnFFeiYn/JNpUU9IyqFrtBbUCj4Albb36yane21oq/lK6Tq4A== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BL1PR12MB5350.namprd12.prod.outlook.com (2603:10b6:208:31d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 18:46:12 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%5]) with mapi id 15.20.5164.020; Thu, 14 Apr 2022 18:46:12 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Yi Liu Subject: [PATCH 05/10] vfio: Move vfio_external_user_iommu_id() to vfio_file_ops Date: Thu, 14 Apr 2022 15:46:04 -0300 Message-Id: <5-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR02CA0055.namprd02.prod.outlook.com (2603:10b6:207:3d::32) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e9018770-4872-45d5-433d-08da1e470ba1 X-MS-TrafficTypeDiagnostic: BL1PR12MB5350:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cAx5X8qz8VzGo6B6LA62+0bpAqsBBe425auHYdoZqYqIUykm8KPNoaoyLfVYzlrnWJRSFAWiy1YYTHb5UUY+2W4G6vCUiWIbkhs+QwZr2n5+aupfDiI99fZDn9iXe+kI9Q68ugtWst/thf+hoQ0RipfPm7TxNr1Pw2aJ47Vzwq+LuS3d+4eFGuNFYujFtGoaZFZq5NewkoRNFEO28+fpj3vZ7p1fvse+s21xixVK5g1sHH0c1ACdr8qLU+flsGhpvzNZTJrmK0TPX/gC0pKK31nwFIlkbpaeXeT+raPB7w6q2qfztiz6bFo9wt3Z2mH9MNhhc1QjCXseQGtbFS9ccNCFF5uzbzHLoD85Jw/aOTmYI1/xKhtSzRjRGoVaTIbIk+rbRRQX5or2+LOpl/b/4y+tc+gX9wMQewu0UJOFM1Sr96pkan8mZ3rgcK83YVFUx3EGrTKEmlUPjXBVwkqmwoLtsGC6dpmv3c9DrRgKVHbwKTAygBOV0vlFmU8wWHqZoZ77uJkhfUSln5xB7UmlFKiMe7p3Os2P8eiV5ePhh86DJIc/P9QtPsmyXNMOfafxAW8coJ6faL/DpKmgzIeSCABk+UteUtDdafi3md1BzmUt8VjIPubnyy7GhDPRxkvYLUJvlXIflXIduFTwYmdUBLsPaAXXkisCytaQNjP0le2sztbYOzrFYhZkViRBe8Sk X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(2616005)(86362001)(186003)(38100700002)(83380400001)(110136005)(66946007)(66476007)(66556008)(8936002)(4326008)(8676002)(316002)(5660300002)(54906003)(26005)(6506007)(6512007)(6666004)(36756003)(2906002)(508600001)(6486002)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: bByDAfBRB/wQOLVLHtwSqzyuT+A/ltSIDENZFyi2Fj8mB/8dJYOXQ+CFDwHq8AHt+NXPg46A3/Hoqq5OWcm45vZ7TkdUSVKzneOhCugjn063c6NxCXQGdcfYV/7I1Fb3lrce//kxzPtHNwZMdSU2tsMHRBMixLX5fUqAfgt43oia2JyFEb5R/+DvjbkW5xUla1D+i82XpEJhksoK+5V9ZFSp/HOOYTpZMRX4SgTfNjVgmx6TBcjqEo3pz6zwTo17HyKQajSfepbZQxo9ghVA6BZJHF4Wsz8d/bU1mr8M0T/kYJ2XdDPhpgcFXmQPvBvZKnWOR+0eV3pukJCDHxtZpqHmuBw5DiyFB8p/IckZp57jtU3dy9VjKHcI4bRv1gsNDl1vAQ1gp5KVNtKo/UXWHm1d6xeyUgOHiTBT6/fuXHlgKCCOSbT3Pah7Y0g50kciA+YEvFJ1xfX6gG3uNsvAiKb3YWVGgk/poLM5owk1HBEq8CLEMRwun2CrMExT1bQsC1u6z3T8mbQdfB982OTn+BLZZ1MSf3/UWguCzjQtr6zE4wwqZqCcChFvoGuebGAjmIvml14Bks/nphCA66rYa/LSvUsC923N53QkrGApsySoh7an+JTrqc87+vwdRp5u2a+tLBHp/kY7hEm07tI9LnjZH34WURf7zf9iWhmGG80sN7oYMdWpVTd1T+L2C0nmCVso+AN00l7fxZggQTyGiJ2NfpghgmVqHGLmb0gkM8F69byqEY2TwBtW4FsRsAmxPHopZZx3+KBli3rHNmU9wSLg6U+B3THD864Dd16ym2GlJLmBLt3HW7u5o8e6iDU8gutC3Cnw9JyA5En45Pb+YQZ5COusHy0UMBe7zBFsgJ8CAna+QFN6+EhY1U7V6/sJadfBw4FXAiis+FpeOGJa83apkVzzZrM/DfngZPOUc3JhUxltRWnLd+nEsT6xOXIwiZ+HVD3Hy7kmXk2VTNooin9QGXmAkqSiRT5T39eXpC9ifyhYHWMUQzZt8SRjd2gYFRLbZUuGMQeAOmyuxGYZ1UhAo8WOJGdGe6ZdGem2zSStoHwAZr1uyTQ4R5ikF3k7zIZ3aacltTJGWzAwleoDOO9ftbpIGQq4MjFTnPnxAyWtRJqzB6GWfCAD8ulTk9Ez0mXwX4Gt6u+6xclFHaOcg4+oNrJL9gqcpvcogYJYmL2nl6TUTOzhAt7Ldv6xf4BYtDHCaublarVoWTN1wGrKhN0Q+Zui4mp6LHDsZclSandpiqG2MIkJTOA4mEG9F+HvM2HLp/mkqc5xU2IqXm/NqZ+Pcr6nFm8R0+V4jcjtLwjCBkQpNqY8EnyFsxX1IN0w6MElZ5VEbjSgg6eN4E0vZlBbzwip5S07wEMK1VNLpJ25GfrZ71HS2h6YLVDv90aTwpEYr5WCjVKpJCsLdMBiXHdG4zFoAIJJZc9NyM91DyIbIhsr83aW7f4xSaIqdfKm3mVSFbKTo7h2SNWRazDfFDQQE6iDiyeJxnNsPBrgOh4KOMHQFY0Fi12nGVggf2QVhgrMjU94/yiFJJmF9dz7tXmmRE5H/L36rUWlyfZH++YgLpsw9fRSJLDPDzzi1CW170c3yffY801liel6Fc0sxcQwuiEupHuOO0WGIKoYRryYrKCNge7c1wPfGclPgY6wVLkzc0ynHtxYqSoNP/cCgk7pd2p0G9ptCvqvmU+O9EghOu/jY/G0EiXURMRsD6T3qcpULPNhIPK5mxuL4h5NXg== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e9018770-4872-45d5-433d-08da1e470ba1 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 18:46:11.5787 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: qSjkzDpmMzWBehOV6Ok/n3JtuVi5luAHvpgLFJndTCovdpGa8HFaRXF5Z/2TLHW0 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5350 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The only user wants to get a pointer to the struct iommu_group associated with the VFIO file being used. Instead of returning the group ID then searching sysfs for that string just directly return the iommu_group pointer already held by the vfio_group struct. It already has a safe lifetime due to the struct file kref. Signed-off-by: Jason Gunthorpe Reviewed-by: Kevin Tian --- drivers/vfio/vfio.c | 20 ++++++++----- include/linux/vfio.h | 2 +- virt/kvm/vfio.c | 68 ++++++-------------------------------------- 3 files changed, 22 insertions(+), 68 deletions(-) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 93508f6a8beda5..4d62de69705573 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -1919,10 +1919,7 @@ static const struct file_operations vfio_device_fops = { * increments the container user counter to prevent * the VFIO group from disposal before KVM exits. * - * 3. The external user calls vfio_external_user_iommu_id() - * to know an IOMMU ID. - * - * 4. When the external KVM finishes, it calls + * 3. When the external KVM finishes, it calls * vfio_group_put_external_user() to release the VFIO group. * This call decrements the container user counter. */ @@ -2001,11 +1998,19 @@ bool vfio_external_group_match_file(struct vfio_group *test_group, } EXPORT_SYMBOL_GPL(vfio_external_group_match_file); -int vfio_external_user_iommu_id(struct vfio_group *group) +/** + * vfio_file_iommu_group - Return the struct iommu_group for the vfio file + * @filep: VFIO file + * + * The returned iommu_group is valid as long as a ref is held on the filep. + * VFIO files always have an iommu_group, so this cannot fail. + */ +static struct iommu_group *vfio_file_iommu_group(struct file *filep) { - return iommu_group_id(group->iommu_group); + struct vfio_group *group = filep->private_data; + + return group->iommu_group; } -EXPORT_SYMBOL_GPL(vfio_external_user_iommu_id); long vfio_external_check_extension(struct vfio_group *group, unsigned long arg) { @@ -2014,6 +2019,7 @@ long vfio_external_check_extension(struct vfio_group *group, unsigned long arg) EXPORT_SYMBOL_GPL(vfio_external_check_extension); static const struct vfio_file_ops vfio_file_group_ops = { + .get_iommu_group = vfio_file_iommu_group, }; /** diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 409bbf817206cc..e5ca7d5a0f1584 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -139,6 +139,7 @@ int vfio_mig_get_next_state(struct vfio_device *device, * External user API */ struct vfio_file_ops { + struct iommu_group *(*get_iommu_group)(struct file *filep); }; extern struct vfio_group *vfio_group_get_external_user(struct file *filep); extern void vfio_group_put_external_user(struct vfio_group *group); @@ -146,7 +147,6 @@ extern struct vfio_group *vfio_group_get_external_user_from_dev(struct device *dev); extern bool vfio_external_group_match_file(struct vfio_group *group, struct file *filep); -extern int vfio_external_user_iommu_id(struct vfio_group *group); extern long vfio_external_check_extension(struct vfio_group *group, unsigned long arg); const struct vfio_file_ops *vfio_file_get_ops(struct file *filep); diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index 254d8c18378163..743e4870fa1825 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -118,47 +118,14 @@ static bool kvm_vfio_group_is_coherent(struct vfio_group *vfio_group) return ret > 0; } -static int kvm_vfio_external_user_iommu_id(struct vfio_group *vfio_group) -{ - int (*fn)(struct vfio_group *); - int ret = -EINVAL; - - fn = symbol_get(vfio_external_user_iommu_id); - if (!fn) - return ret; - - ret = fn(vfio_group); - - symbol_put(vfio_external_user_iommu_id); - - return ret; -} - -static struct iommu_group *kvm_vfio_group_get_iommu_group( - struct vfio_group *group) -{ - int group_id = kvm_vfio_external_user_iommu_id(group); - - if (group_id < 0) - return NULL; - - return iommu_group_get_by_id(group_id); -} - static void kvm_spapr_tce_release_vfio_group(struct kvm *kvm, - struct vfio_group *vfio_group) + struct kvm_vfio_group *kvg) { - struct iommu_group *grp; - if (!IS_ENABLED(CONFIG_SPAPR_TCE_IOMMU)) return; - grp = kvm_vfio_group_get_iommu_group(vfio_group); - if (WARN_ON_ONCE(!grp)) - return; - - kvm_spapr_tce_release_iommu_group(kvm, grp); - iommu_group_put(grp); + kvm_spapr_tce_release_iommu_group(kvm, + kvg->ops->get_iommu_group(kvg->filp)); } /* @@ -283,7 +250,7 @@ static int kvm_vfio_group_del(struct kvm_device *dev, unsigned int fd) list_del(&kvg->node); kvm_arch_end_assignment(dev->kvm); - kvm_spapr_tce_release_vfio_group(dev->kvm, kvg->vfio_group); + kvm_spapr_tce_release_vfio_group(dev->kvm, kvg); kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); kvm_vfio_group_put_external_user(kvg->vfio_group); fput(kvg->filp); @@ -306,10 +273,8 @@ static int kvm_vfio_group_set_spapr_tce(struct kvm_device *dev, { struct kvm_vfio_spapr_tce param; struct kvm_vfio *kv = dev->private; - struct vfio_group *vfio_group; struct kvm_vfio_group *kvg; struct fd f; - struct iommu_group *grp; int ret; if (!IS_ENABLED(CONFIG_SPAPR_TCE_IOMMU)) @@ -322,18 +287,6 @@ static int kvm_vfio_group_set_spapr_tce(struct kvm_device *dev, if (!f.file) return -EBADF; - vfio_group = kvm_vfio_group_get_external_user(f.file); - if (IS_ERR(vfio_group)) { - ret = PTR_ERR(vfio_group); - goto err_fdput; - } - - grp = kvm_vfio_group_get_iommu_group(vfio_group); - if (WARN_ON_ONCE(!grp)) { - ret = -EIO; - goto err_put_external; - } - ret = -ENOENT; mutex_lock(&kv->lock); @@ -341,18 +294,13 @@ static int kvm_vfio_group_set_spapr_tce(struct kvm_device *dev, list_for_each_entry(kvg, &kv->group_list, node) { if (kvg->filp != f.file) continue; - - ret = kvm_spapr_tce_attach_iommu_group(dev->kvm, param.tablefd, - grp); + ret = kvm_spapr_tce_attach_iommu_group( + dev->kvm, param.tablefd, + kvg->ops->get_iommu_group(kvg->filp)); break; } mutex_unlock(&kv->lock); - - iommu_group_put(grp); -err_put_external: - kvm_vfio_group_put_external_user(vfio_group); -err_fdput: fdput(f); return ret; } @@ -418,7 +366,7 @@ static void kvm_vfio_destroy(struct kvm_device *dev) struct kvm_vfio_group *kvg, *tmp; list_for_each_entry_safe(kvg, tmp, &kv->group_list, node) { - kvm_spapr_tce_release_vfio_group(dev->kvm, kvg->vfio_group); + kvm_spapr_tce_release_vfio_group(dev->kvm, kvg); kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); kvm_vfio_group_put_external_user(kvg->vfio_group); fput(kvg->filp); From patchwork Thu Apr 14 18:46:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12813879 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2DCFC433F5 for ; Thu, 14 Apr 2022 18:46:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240163AbiDNSsi (ORCPT ); Thu, 14 Apr 2022 14:48:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52706 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233450AbiDNSsh (ORCPT ); Thu, 14 Apr 2022 14:48:37 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2087.outbound.protection.outlook.com [40.107.220.87]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8C829DB498 for ; Thu, 14 Apr 2022 11:46:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=M/BtMIkxxh3TfIg+XWgNNjoGsQmRN93J0D173VYCDfSqE+Scnb1Wl/5RRNEVJgq80WCYTLeGOogZMP6R7acyCDpBmkB5oay2khFbMUHSHS+b0jp/Fo3UNekuJJTyvMTIKxjxnYn2kPpTq+Nof//oOevw78BTavQhhhvKqufkwTnRqSybSvM2KyFYRvTinqrUBE5Jz6CIPfvAQn6mXnykfGaW8hsuBuyOiIpHCBhoJEbkRCttjS6a/2oK4c72npzemL13TfyYWKz+5EIuGVR1AlQa1KbFLAV7EXuDCe6M0d6vj7YpCElgwQlE8yE91CFQkZuXMpMKkt+8Z5c3RiEY0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YKavb8q6HM8BxRHUCIzLukvxZ2U+kIoAL1WUbjJXBTc=; b=SuDtRaFYp9qjOGb+Oy0KfozjmcpeeB6Rp5wMz/jH/7nATJYR3eUq5eMcha+6EXybDzxlBa1ZDSDujh9q3qLl46lfNTKLvjlkFC1GJwA7bWGezvTCd6vz4acMDTDrvVclgc174a09RRP/hs5Gs6R1XfJcIZqKHX8wGfLHNCkDtqb49IeiyD57gzOqTjlPmf1SmgK/aPPKT1CGTExPKaFAUcnek1t0loRef4DtRs4r9X/KOABOfoH3zm/vnWyk/tXOyX0w+pUYk2dMPbhQ26sGDgSQ+7wIt28ikmd/Nlct/Kd1H2UfK3e991PGqFVTo8bVgPksiifgL3444Te2ykN6sQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YKavb8q6HM8BxRHUCIzLukvxZ2U+kIoAL1WUbjJXBTc=; b=udl7t3FoOXYZcPuuxzHjOhq3ZLu6j8xviuude5sGkq5KIA6zNOYQDx7M6jMxgnL8kWsqwqFjjOdTxE1CIj9UdN3GKuAJ/IVMdUS2riWuT9MOLFONso5nbcBKkpH01UEkMPBTnQbBacOWbhIR+WSHtG/GaRQ/q0CDhiRftCePynJbf9yCw6BYDi6rZFBAupOMddAHw1h0ynlQO84mdvqTyuclxwossOHkhvK4mamwyKSJcu3Zv15P3UHjGdl2nsELKfEsrhRyTmHF9IsvuzE/5uylZKs+KkoGyH8AOMJoWlU7cXwBFhWm4jcy71f9hdm+bFve2ybuyh6UVmklSAh7/g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BL1PR12MB5350.namprd12.prod.outlook.com (2603:10b6:208:31d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 18:46:10 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%5]) with mapi id 15.20.5164.020; Thu, 14 Apr 2022 18:46:10 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Yi Liu Subject: [PATCH 06/10] vfio: Remove vfio_external_group_match_file() Date: Thu, 14 Apr 2022 15:46:05 -0300 Message-Id: <6-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR05CA0009.namprd05.prod.outlook.com (2603:10b6:208:91::19) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 14d388df-5d38-4867-26f6-08da1e470aff X-MS-TrafficTypeDiagnostic: BL1PR12MB5350:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: niZyzWJRHm/JyQCPPWl/Xh93eHqhi4iDnUPrQTnxdFoeqvPs6ZF/vAFVF6q4GmbHzTPKcGJbvxYdf4tHzzGxjvBKpqaBxcrGt1is/Q6yy+fuasXR/IGSfBRpdlpmuXnpzD8N4peWVrYocbOVghNmi7vQeCh5ofz4f4S8WXPrVaqgmozioV93W3j4Jo13nLsS3vzAWiFNERCqeCu1PRQPk8PwGnuWoFg3rNeMDiG+c0Fg+3r4Q9vyCgxxvl0SFuudVwGclX9OhQ1WnTE+AKKj24rJialT1fq50GIQU7ohyRhddlZoAVxp6j2J3IpAhtCD2KS1nDMQewSAQuSeRJFmGZ0eAkXopk+XhPcFO2XdwQC3McYb8giGtmaAe5KDqp9bZKcEcqs1BjR+778eMRj9WlD8BnuyxqPqLNQfyQuO26nhznucU3j5i21vGC86TAmOoz29YxfR2niZDS1KGL4MVzF1nPhwxZ0nc4W6ntif237+k4A+XN+YTJk8IjwrLnaNmzrH4EmHZ8/nhoERmNhVhWr1tfaWD94jXUpux4wst5H5NkYQQK6XytNFd3pFtuQDj3fVG+o7dNoOBZXZBdwBVXahXU7noB+ZHBotahXLM97cQfbzOdMZl30yJjOtrcxBQtj1OMz1y+rvRcV3+bRPhzSKpTt1KHbxjD5Xnd4nDRCyH92d077UjZNogh8vVYJ2 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(2616005)(86362001)(186003)(38100700002)(83380400001)(110136005)(66946007)(66476007)(66556008)(8936002)(4326008)(8676002)(316002)(5660300002)(54906003)(26005)(6506007)(6512007)(6666004)(36756003)(2906002)(508600001)(6486002)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: hvToUDGhdWVFjBtwPay2W45VmX+1+ycv4Y1MPaFpKso48TNByjQ0lTRaJ/Zyoj/UuQbjVLSy5nEgHZTHqHVX01+kxrvUx5LCgUrOFv1jiigg+rAhsvuHW/YaW4FmgUjM4T8tLFZEttDwxaUv7moEd8ib8XgFThguZ+bE7aHiIZzjbYRPK4qgSKwmbq/CNl0KzWUOC6JA9qh6kPFdL3BQiguT4bXLh5QEF/xOIwH9J0uxXiyJ546vDIsnVJEWy6aMj3FkCFyBRT1PB06EcGzN6uh2ewbUhBXp3u4Yfzq57Jtutx9cVYdhwsytCVHpIAIknY08e7d/5AQlddv3asZ+scHjhSqTtIqV8UO977AiHKII8GiOLMcEUBpw1ILNKIlHohvUYXhjLHr0/67U69zSvmyaY9cSpy6aOX+NtxmX5j4zxXGY+/O24eg35DVj1TOhST3JzxoOHNr35r91Tw2cjnlNmAmEWHlVXAt6mBvVq1D2fuRgy712qVGQkRtp5cPVshmZhpR4puGqrAyd3fYanfsJ6W2qBHO1PhcdiPHNs/SZ9ySL3sB1Y2FEd2AxWQ7deIEsfFW2TJM7p7jhQ/Y5PaNceKd8omlpAGzVIlIDT9J/AT1D2CEZ32aNDvvwMniYYBmlW2HV0Qt5l/kaSn5PbHFEjsSmV3dgDPUkGVXxioNWFoYVNkPTtrlJBr1bd6Uyg3Dr/vlwzSv/WDuDMVwY1vip/AlnJkdCucZ6D6A7Wp40GVldwlRtyBxVohnFpSyITxSGeXmrBmlS8s+A8d/5GIsGPqIDR1Xm07rs4/cb8K8AVpeQO+xbPiWmEAJpOGvVD7vId93JT0+Hnp8H3hZsFBYxNrgSj54oXlDyv7FaK3RIgub7cfcHUzR2Qz2oTcWf40MTePGzdTgMeyR28L4J8xPR4fkj+zAkXBnXxpAh4SYJrO3rd5cjccjuL5bHJ/Lml39LocBqZEj0eOVG9z0Tq1rxW+pKF+3IL7Nhcjyu2thHHmi8BomJVwTnt3lPvbcvdC+77g8YhlRkZbc+Yc6qLN5cSpWBSsZmtqN9oisUmTdEryRMpRq3EvB7k2NRfbZSZosYoXYqVPtT2jE4ZIBr8WAqnDTzOLJ4KuJzPoR7sCoC/XQwHSYdt8wfBAhQFhI9wWXkvuqknRlM1RasrRbXnyAL7U0UnUkLgdRBM8MWlZPI8UXj36xkGFykOKCsMApMEYuc6RiYamDvTvj4tvYyRM7xQP3TgXie610imEPzdufWA0ogLgMh0vg/EHoQd2IgTb8bs7m7Ptc9C89LBSOIUoy8SCmzmQXJp7yJ2AAN7jdS5qsViYdMn91ZjHjioNekKTHVLOalGTmpz9uUHDlGqybRw+5ImFs2ydpO9Q7FEbtgxztzDWZI+jQq208X+E2QLgMtQz4x3aePCZmgtvKfMzws5o0npeko2GuvIgcxynhnxrI3fe1UGYNEV82918bSuJ5xSafWSVYFPt0XTTZF9HSq7JFWT2uX4h03OdZO0EGJiPiZ4Q4GV9+odYsxdAwOKMBzVHoKh7YS82o7dQ318Wlu5+OBGxlJLIrSVXFu762Xa6tbARC9B6d0E/nmBhggXfZHlJ2/s0FLX+vwprnhNv/nPoDmJTZCTv09IRdryyNX2LQJRFzWIkd97at4i46WgcPOBs5lNtggA7h3UIyeomoJAFqjmnCsNZ6OLcIUetaSCPd6HurGfhGaygnVnzV3VmyaTcptxstJT4vqfrtKRA== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 14d388df-5d38-4867-26f6-08da1e470aff X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 18:46:10.3601 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: mk3sgu4Uhi5wd9TE50eed+r2J72LTnILFatHfflWqSYTw367VaJTEnEqpci9ulns X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5350 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org vfio_group_fops_open() ensures there is only ever one struct file open for any struct vfio_group at any time: /* Do we need multiple instances of the group open? Seems not. */ opened = atomic_cmpxchg(&group->opened, 0, 1); if (opened) { vfio_group_put(group); return -EBUSY; Therefor the struct file * can be used directly to search the list of VFIO groups that KVM keeps instead of using the vfio_external_group_match_file() callback to try to figure out if the passed in FD matches the list or not. Delete vfio_external_group_match_file(). Signed-off-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Christoph Hellwig --- drivers/vfio/vfio.c | 9 --------- include/linux/vfio.h | 2 -- virt/kvm/vfio.c | 19 +------------------ 3 files changed, 1 insertion(+), 29 deletions(-) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 4d62de69705573..eb65b4c80ece64 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -1989,15 +1989,6 @@ void vfio_group_put_external_user(struct vfio_group *group) } EXPORT_SYMBOL_GPL(vfio_group_put_external_user); -bool vfio_external_group_match_file(struct vfio_group *test_group, - struct file *filep) -{ - struct vfio_group *group = filep->private_data; - - return (filep->f_op == &vfio_group_fops) && (group == test_group); -} -EXPORT_SYMBOL_GPL(vfio_external_group_match_file); - /** * vfio_file_iommu_group - Return the struct iommu_group for the vfio file * @filep: VFIO file diff --git a/include/linux/vfio.h b/include/linux/vfio.h index e5ca7d5a0f1584..d09a1856d4e5ea 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -145,8 +145,6 @@ extern struct vfio_group *vfio_group_get_external_user(struct file *filep); extern void vfio_group_put_external_user(struct vfio_group *group); extern struct vfio_group *vfio_group_get_external_user_from_dev(struct device *dev); -extern bool vfio_external_group_match_file(struct vfio_group *group, - struct file *filep); extern long vfio_external_check_extension(struct vfio_group *group, unsigned long arg); const struct vfio_file_ops *vfio_file_get_ops(struct file *filep); diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index 743e4870fa1825..955cabc0683b29 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -60,22 +60,6 @@ static struct vfio_group *kvm_vfio_group_get_external_user(struct file *filep) return vfio_group; } -static bool kvm_vfio_external_group_match_file(struct vfio_group *group, - struct file *filep) -{ - bool ret, (*fn)(struct vfio_group *, struct file *); - - fn = symbol_get(vfio_external_group_match_file); - if (!fn) - return false; - - ret = fn(group, filep); - - symbol_put(vfio_external_group_match_file); - - return ret; -} - static void kvm_vfio_group_put_external_user(struct vfio_group *vfio_group) { void (*fn)(struct vfio_group *); @@ -244,8 +228,7 @@ static int kvm_vfio_group_del(struct kvm_device *dev, unsigned int fd) mutex_lock(&kv->lock); list_for_each_entry(kvg, &kv->group_list, node) { - if (!kvm_vfio_external_group_match_file(kvg->vfio_group, - f.file)) + if (kvg->filp != f.file) continue; list_del(&kvg->node); From patchwork Thu Apr 14 18:46:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12813889 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35083C433EF for ; Thu, 14 Apr 2022 18:46:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236852AbiDNSs4 (ORCPT ); Thu, 14 Apr 2022 14:48:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52796 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1343506AbiDNSsm (ORCPT ); Thu, 14 Apr 2022 14:48:42 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2087.outbound.protection.outlook.com [40.107.220.87]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8A19EDB4AF for ; Thu, 14 Apr 2022 11:46:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bzb28QTmgBe3JHjHuqhIr9J4iHJmiUwrOuaYR7fzTRxRXJO28b2QksUnMCt4j9F1DByZnxfsNxZQSmipeB15VhcnjdDP8af4mvk69YiwOd0K+poKkYeiSZyb5sE9Q2YjVpbTNG8g50qfNy++jFHRaV8oY8H4lG9enLKQxNcU6kEA0cKvJ3eP6hDl+u62z0Mh1UBoeiP6ThUH+wdcAhVvC2PGZbO1usJtPyV60tDqz6sy4cU1AU18QN82DnQyBcan34ZyGpFASkBhXuLBVBUUN+hg0MQN8+ttx8IASA/FYbMXYnQJV4o+7JXzakjQdLyRv04bIuwQzV3hteL2WCm+7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=b7HwqU4CVVpa1hZL/FSuZyYkP6DLD+qB3i4kl7jhiic=; b=DZmBE34vcWdk3YR5Z6qpeXOuduc2657PaXTAmWy/TxybVCl+53GkJE34WfzTxBjNO92ysq1xkQa63olU7D4jt4aLg4/5kBuFAgsfZVrFJrUsg2av6ZSOuw35U8iBpm4AXhB82w8JG1iULVXY9f/CAXiEic/xoLnDSZAIT16WhKyuWWTXzqAofUmHl2wAuUJdbiXk2IxIyhvOzN4Ss5FeDWqDYGBQx1WZWwLyFNv/PFGhmmRPlu+p0TNluq+mnMfhJrlGk46oNl8N2UryXL3zuBg8e05+OL7ijmZ4viKm105Nj9W0feSMeDyvp4Ibwh+SyERuOn+6MWGiK5PLSm9LhA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=b7HwqU4CVVpa1hZL/FSuZyYkP6DLD+qB3i4kl7jhiic=; b=YPN45g/VHfMcrPudc/lpyq2K8Q0OYWjunwq4UqTjDjSQMIFRhEQI9d5pydehCJlmk5aFiyt1E2HdhA8KpKmdP7BnW+g40R/PNLwCv5sS9fUdZ9nglUDy+fKT2B7L5eqHD+qHvieFsQuxXmEQiC3eotWbaRPwu2ndYdmC1y7ZOhfbfnbBNzNmRX2Cwr7WVm0kIStEGoFo4x8/sV1A0RKeQ1ErL6HKbOnYJiNqEtle5aIl5IcRjbpdtQtWa+1T0F/OWwK18197t/lu4KUuOB/p6Ksc75BZZFioRSkX/ekkkM7rZ7sVQAOezf3sBGYravmJWpcG9ouwso7AwQEvGsQl7w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BL1PR12MB5350.namprd12.prod.outlook.com (2603:10b6:208:31d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 18:46:13 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%5]) with mapi id 15.20.5164.020; Thu, 14 Apr 2022 18:46:13 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Yi Liu Subject: [PATCH 07/10] vfio: Move vfio_external_check_extension() to vfio_file_ops Date: Thu, 14 Apr 2022 15:46:06 -0300 Message-Id: <7-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR07CA0013.namprd07.prod.outlook.com (2603:10b6:208:1a0::23) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4cfafcfe-607d-4d1b-e49d-08da1e470c4c X-MS-TrafficTypeDiagnostic: BL1PR12MB5350:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0LlZm2sRbYS44pnBYfupwqTNmCIjorWcJq+Q6CuWDG7udfO0CmDxSyTdPCcHMnnGU7HKa/gFgwPwoGgl++OAQNTt5pSgmCscxYFrsY6Os+kwhmSuZqTbSzWxOZV4XmCzr89iM/6Bt3pk8gUe4n8h9M/fPhUc9EjMqYxoUS0Ub978uIEwXUtgTTlQqHj9gHG2+ZwSl4xgUBHZZ/1DBNXjm2UhYknMapf2GV4o6SjhsM6mdzpYMIKRFDDzA+qol8fDjlNQ9wqW/sXeA+0BnkwoGGtwpE+nLRMCwbbFicJJC94lAoul55rHIONtSwWoL+uumloN7uNP8qcEOd85N4iyMK/wJFKEh4hUvJWJJWRTMjkPxLe24AkOm5mWeV/AY8aMeOUuVrn/uzTVahMS7wnYnTBnqHyAItO4dCL28AJGn5nuREZH6E4pBbVyEJd9ZoWxN9xpgE3nfG9u9TiqEs4dnHnZrkgx7wPK5FCkNIaz3Lwtb04M4k/cEkBDbpoLog97xMHnPoo8juulSJCVKFIaDC1AA+8KI+cqKjXWXORJdwdKsnWOeIkRD3Owq6GdPUp5SanwaImW6ubIzLibTN8nXlKfICkeWkXxoM+7VGoMKMuPDDzNpVb1nG6VJRze42K6WrcZyft+crtxNGj5vO//SpkgD5L3WCMTD4PP4lhC3/E089+m+0aIO0QE8HMu0g6O X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(2616005)(86362001)(186003)(38100700002)(83380400001)(110136005)(66946007)(66476007)(66556008)(8936002)(4326008)(8676002)(316002)(5660300002)(54906003)(26005)(6506007)(6512007)(6666004)(36756003)(2906002)(508600001)(6486002)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 3QwU6r4NDdBLl0WaBoLuTwtMtrjXoQ69kUftn7Ndr8eGLGSV6X+2B0CPItDm/tLJt6ybmHJGRMcm6Gh73af4Nu+Hl8UeXHlXZdFGw03LBQyH+6c52UYf8cw+ePXfDjjXqmgMprKaTORnLqoy6HulQ2ZoFJxqKrk+ZInH7g010HNL4XH+J4tT/WdtDz1wOo+kK52yCnvbCVY+hSgkWj7FbWbQoaBOXuZvSGx8O88zOGSICBqWgI4QFYFcSsAnSVXyfRehwLhOZIPY+UWt0F3di9MwFn5F9IQgvLO93sopUfpw2MxRC+FYzL6JjlWJTcNqsdu8fjcJXMS+XYX4/EGA4376rNnNdy1vMbPCE5j3PO4D2Les2mt3iJ5H8q2Ek04cM61bs/DcXrKfyZLy6NwxHXsRpG7EkDVkHyRpomRLTOXt3733YPs154NZxb5yL1OB9/2CO8gxRkU7gxLuhKIVMu4d4ZthOmizw57gcyj4k6QP/Rlo0gTY+2UjxLbZWAR0Eh52hJ5UVS0mTnQFN+1iCAuWAE7jAGE6ljZflSjhWbxUzZvnXgg0sOSO0wRTQ5jK1WjwIi3DTvpDFepV1hBKB7EAHBKi1i5i8Vz6GDIol+l7QS0tHDCeSWBXIFRwAc3O2e6up3yIJm9PIAAX41z/XYcjGOZQCluX3cBXxjHpR0eeewRbrQIlnmmPZoA/P8+2IWns2QLbZwX8XkXAN/13gM2Rc99DWMYAJHot9ATxTx5PAznbtpYETnAzxIYuDBr0t0Y8lsce4Ll2po97SvLGta46qzXYi78ZBa0dCBIEi3951lt9Nm96anxao0S1ZEwdbfiurOKYyEcpU+Aq6qhl47T9ylmNaS8B1gHjRR+6szqP1q4DW5EDh9Q4fZR0mXBNf1GeeP2IQsa+3qpzjWguEaj3ckFizpMfl5arFgS2z2oBHoRR9Fpr6EmGAVhdKAK4NSHb0AbQnAaIuxT7OR2aWN5XodwY0SdzSwXtIAj8/+L0sMS15As1l4AjeH6CeZngpNKfWiqBpISnsZrIfj6T0tEsrXD5LXDfKKYPGxPptzpWuujLNShMQU+c9vjr2lbgoB9co7xEtGSCVgBJvx+iPPWEUwsUCcsOATz0A6c11KDFxFwxpNNCWKp2g5sDrRA5zLplVO/1OGRIWzekPV1Hg82J7m6S0RGJvDxLwOnJJvA9q54GQrH6z3vI+KpekqCRchs+kdlF9RdGw6aOFVyA78C92DRV+c+mLZFXfCDI0xCRUl884iD0CV0UKbgqlPP8KvIS80/DcXzaAvFb0rJ8elsZueT2bGDoopqYyRdrk9B3GtSXCrRrDKDrOB7CbC4XR7I9PaqFobx2+CcgkDPyGiPj9nbzjrzoAi2tOWqLCt8c++i3rXLjVfh1T8IADj1rvjeQ8j3TIwHuzKiNFMwqT3ERzZrkKDXoGQYIIlYi1TeYGbu3YYN2xLOSCnP7V5iq76+dcBzZYj6Ub23eW9kjEkQLMtV898CFEOc+mfx/oEs+4Sd96x0wIixhQ85ebPUDVH2vcUJgyP2vW2ieTXwZzGygZdWQxHkhp7g60slWbXWqlSMy7fQ5TPqqjgbs+4DkAX6jmqe8vVmjIDC0RG1o7E/dhLbcRjJwSgvMlINvRHbHqM32WoVE9uT5Q51KTV+WLk5HQmM2mFmJkP7xFWvaAMpgQQG1dWUEmYLQu5KP+r/asZn/jOX4tcjC2YVd0hP+3Q/c31sgdsZpHAyWiAT+Rg== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4cfafcfe-607d-4d1b-e49d-08da1e470c4c X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 18:46:12.5785 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lMtgOLAwCZ7Zzotrz2ohW4s8wagSQxMLF37bB+2IEo9STKXvhagYS1SCI29E6F20 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5350 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Focus the new op into is_enforced_coherent() which only checks the enforced DMA coherency property of the file. Make the new op self contained by properly refcounting the container before touching it. Signed-off-by: Jason Gunthorpe --- drivers/vfio/vfio.c | 27 ++++++++++++++++++++++++--- include/linux/vfio.h | 3 +-- virt/kvm/vfio.c | 18 +----------------- 3 files changed, 26 insertions(+), 22 deletions(-) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index eb65b4c80ece64..c08093fb6d28d5 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -2003,14 +2003,35 @@ static struct iommu_group *vfio_file_iommu_group(struct file *filep) return group->iommu_group; } -long vfio_external_check_extension(struct vfio_group *group, unsigned long arg) +/** + * vfio_file_enforced_coherent - True if the DMA associated with the VFIO file + * is always CPU cache coherent + * @filep: VFIO file + * + * Enforced coherent means that the IOMMU ignores things like the PCIe no-snoop + * bit in DMA transactions. A return of false indicates that the user has + * rights to access additional instructions such as wbinvd on x86. + */ +static bool vfio_file_enforced_coherent(struct file *filep) { - return vfio_ioctl_check_extension(group->container, arg); + struct vfio_group *group = filep->private_data; + bool ret; + + /* + * Since the coherency state is determined only once a container is + * attached the user must do so before they can prove they have + * permission. + */ + if (vfio_group_add_container_user(group)) + return true; + ret = vfio_ioctl_check_extension(group->container, VFIO_DMA_CC_IOMMU); + vfio_group_try_dissolve_container(group); + return ret; } -EXPORT_SYMBOL_GPL(vfio_external_check_extension); static const struct vfio_file_ops vfio_file_group_ops = { .get_iommu_group = vfio_file_iommu_group, + .is_enforced_coherent = vfio_file_enforced_coherent, }; /** diff --git a/include/linux/vfio.h b/include/linux/vfio.h index d09a1856d4e5ea..b1583eb80f12e6 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -140,13 +140,12 @@ int vfio_mig_get_next_state(struct vfio_device *device, */ struct vfio_file_ops { struct iommu_group *(*get_iommu_group)(struct file *filep); + bool (*is_enforced_coherent)(struct file *filep); }; extern struct vfio_group *vfio_group_get_external_user(struct file *filep); extern void vfio_group_put_external_user(struct vfio_group *group); extern struct vfio_group *vfio_group_get_external_user_from_dev(struct device *dev); -extern long vfio_external_check_extension(struct vfio_group *group, - unsigned long arg); const struct vfio_file_ops *vfio_file_get_ops(struct file *filep); #define VFIO_PIN_PAGES_MAX_ENTRIES (PAGE_SIZE/sizeof(unsigned long)) diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index 955cabc0683b29..f5ef78192a97ab 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -86,22 +86,6 @@ static void kvm_vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm) symbol_put(vfio_group_set_kvm); } -static bool kvm_vfio_group_is_coherent(struct vfio_group *vfio_group) -{ - long (*fn)(struct vfio_group *, unsigned long); - long ret; - - fn = symbol_get(vfio_external_check_extension); - if (!fn) - return false; - - ret = fn(vfio_group, VFIO_DMA_CC_IOMMU); - - symbol_put(vfio_external_check_extension); - - return ret > 0; -} - static void kvm_spapr_tce_release_vfio_group(struct kvm *kvm, struct kvm_vfio_group *kvg) { @@ -128,7 +112,7 @@ static void kvm_vfio_update_coherency(struct kvm_device *dev) mutex_lock(&kv->lock); list_for_each_entry(kvg, &kv->group_list, node) { - if (!kvm_vfio_group_is_coherent(kvg->vfio_group)) { + if (!kvg->ops->is_enforced_coherent(kvg->filp)) { noncoherent = true; break; } From patchwork Thu Apr 14 18:46:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12813886 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0775C433EF for ; Thu, 14 Apr 2022 18:46:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344776AbiDNSsu (ORCPT ); Thu, 14 Apr 2022 14:48:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52778 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243184AbiDNSsl (ORCPT ); Thu, 14 Apr 2022 14:48:41 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2087.outbound.protection.outlook.com [40.107.220.87]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B0C8BDBD13 for ; Thu, 14 Apr 2022 11:46:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ODzD7opW+b6+es/RzB3R5z0lBonBHRwkusN4t/5+Sd6Ndj8dEt66PnxUve4RCmyzXHfdHib0uqsi5q6D1ARfoDBwDz/ycYekUw4KMAdsrRAPeVMVCxI6tHrGVk2FtuguREEgfUbcjCzRnbcVFNfeA5bGl/k36Ak37IUz0ySHeC5MLKN2Hx2Va/UNUgPUne3GkK/EwWw1m2f8SNAtlcDPyUiAvOWsRm+JW6CXj5MM2ucMu1vXJwkBN481z48Mp55/gpbImDCPX3U0uPWXPw1wJiBVHm5/o+4bLMKbQWDoZsG9ho6XQVV6JjVj1yTik7W0xAsLJ63BRvenq5LMMRgIbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dPYNkGtKhIC3GQC4bpyRromr8EXvQn3unFBLkP+MEF0=; b=myXDoDVYKVtRn+Elgqs2f9TVP3Qf2+vwpDSEiEYNUPKYNmRwpOoFx9qjAP+tsGuva25RBESxB+PvgG+jV4KjqdEX5y0rDDHLvtUS7RpUNZ7fCChX4w7fWy4bozxRbEKnkSMSi/cBlOaNoeHEw2yrmkDBVrRwntpvj7v9PdLZ+xjY8Mh/N+RkkY/BZwKOn7xw6G2KeTHFd1+V1zgeiLNml8ezBPN3rtK3ucPMFSRy9vhmjB+4C3d+3Sf4BX/6F6rAibIJjCWVmo8Pc8eTUkROffnBxEADQDBulN2IMVY0uRngQgoomp6TeKWXRx2MsF20MyAHvucVo4aLbiazdH02Tw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dPYNkGtKhIC3GQC4bpyRromr8EXvQn3unFBLkP+MEF0=; b=hD28ThIdCg3LQe9lKwAWm9DjbOKftQvrZzfW3Q3jYXOB+/7mGppDOJCInueHXa1eQks1XhDwhetcDVuu4fof/FPC7Kk1syhgZwsJ/A/adKlaY7iVEyewvp78WpNekTqWk8YYkn0X/Rrx6Mcw8YUTFxcLiHNTDafIGCWT69qc/99huEsXMc2meENHVnQoH06tz25rnxs91OwSGdAafdVp9Tjzq7D418g3AhtDP41MpbMLSQ+R26EdxZVMJi8Xb5B6F/Bt8xtGz/7SwcPSNkwe6rfptRnmbgtCnvDMzkArG7QwIP6APnf16mFNLUNQDILCX98KoE9hGfErizwyaOmDMA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BL1PR12MB5350.namprd12.prod.outlook.com (2603:10b6:208:31d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 18:46:12 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%5]) with mapi id 15.20.5164.020; Thu, 14 Apr 2022 18:46:12 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Yi Liu Subject: [PATCH 08/10] vfio: Move vfio_group_set_kvm() into vfio_file_ops Date: Thu, 14 Apr 2022 15:46:07 -0300 Message-Id: <8-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BLAPR03CA0061.namprd03.prod.outlook.com (2603:10b6:208:329::6) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f77520be-4761-4205-433d-08da1e470ba1 X-MS-TrafficTypeDiagnostic: BL1PR12MB5350:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VlIKjzHkgZWPKjcF/5QZ1MM6ilDaxAZK5k7bKSL/2foJ/6SlNawmU1ni81QjqP05QDPeeN3vBwA07MEWjb7ZGuC915j6rgAhGCkIpc6x+yf/ACQ/cXrTp7JZdSSO3xEB6m+zLdAHIaCN8eDLwU6n4nZjP6c3bmpQaFsd2HzuAzbeRt78QFwGX8GK0SU5hkYPo4FnEABPqEoayE+qMUPw1GAMK0cAJmC/P23YIvcVJ3qMINr0/F6IV4oXCUXeqy+E9PRA3xg0KStMYgIZFQpMnFF7mPE4UeckgV9gV0n/eor7Lh4Iskqhf48l2GdQgPadHAPAQn6oWwj7kqqTTnKdVc/zE6CHH5TFLrw+J5Onu9+EzxJzT9EBjdagKRXJHeMSuUoWWvA0V6FAEV84PqOvS4bIbEc5UMKGBSgKCMl3M4pjPXUZ8kIYd62toibR1TLOnsIZmUz3siCqxnhxO4CzRLviFmuqW2Ax+O6C3gOsyR4luEPQccUYgZmwolxlc0UMKWz08P/Hg/249nZlx1jz9hdr+cMZXoyYk27nqd/tmWlX/L074LlcvCAINNq43FOZBGz3JFkGEJVVvBq6Mq9iNlgPakSRFOOLpjNCyT/Ott28jbushaLlL0HTqqe6C1nbn+y6YgkAKuCPNyClDjgl4RMLscwtzNhSoL3S7O03qnnpKoWAiaIe5K8omnFrcLsmUrr72UZaHeGttXPfNjFfGQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(2616005)(86362001)(186003)(38100700002)(83380400001)(110136005)(66946007)(66476007)(66556008)(8936002)(4326008)(8676002)(316002)(5660300002)(54906003)(26005)(6506007)(6512007)(6666004)(36756003)(2906002)(508600001)(6486002)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: BTOK42c3eRd8ngsDU1IUmHfaW0H2ga2Uh/2ChPhizgcyltGvAUrxMUaLLNYXjDdMMhhl/X2Yh0tG4HqOV/2t7eWw/SNIlYy30Y495SHyYIAdXcejt1YSCfDYSLqFG6+461I1vlUJCJS7J8hchuDnDgFAh4baE7p2JO1mUM1WQ3j7vD+xrt7Vl0hC0SK6hNqD4tuQ9STQEwFtcyymYOB7X6K1ChpNoUsPulP/D1HsbumJXQef0cE7QJXnmfFzpZ/G0CQ+hOMt7ns78wWjGRRiY9Wq4ovipOpLREO+7iWN7w+pi47g4MUpF6ERhRX+6SmVxldm66g9wro3fNimRFBnoLYMtHeEMxQSv8NFQOPo9UMZp9q0znfwzsWgem9tsLZkczDUM+1TZ+m7GsCqUrYJ1HnjQsOxbb0rEUtaHnsc7JMFcrZPGQjuvNNhb2Jcf1DL4CFPxX4HaouZCd5SymvelrCfGX6ogqNME6GsOKQ6pzpO+4PWFPsHWw/GNt9ej7CKgn3SYvjir/SR2pikJLF+W8lG0Gsl8hLvm7qkCtqWvbfXYfjBx8pO50JbhKTNz/uLit4uOFjSKIffC8I3ZQsSWjQdaVTIYzIXKZCTq1n/aHVeOc1PpJQnoUYey4O7Thp2uInJrdp5eK3PzfULrJYafj9UZfxDVXBpmODySES3jaOvLSDg4hpkBni1g1CgFv+cp2fwaZYeUg5LEOsGiXLY9MhU9q0M8vKpgVLTlDM9XXaRWDDGOj2F97wFbFfWfiR4SLywCIMM6j+/1jfuup1u2rdA2ajWk8PsA13Hx5LablyNlUl9iLR/NqfHlxDmUJLkuU5QlBGht6WSgtlrwc/xuSr+10FIa5jWmABN8HIfoaxyliLj/1peJ5uoAv58Yu7Ti17iRYlzFgkmSfTjVxh8Lu6vGL9KB9Fxgd0Di3yTPXklu/yIIR8oNZ+W/QDRE6VQIdH4Vx1kQHhgI9jweEkTcFonL22cOprVTubXaoKIf6LyU2rDv5meZ/FEz0DWDfW6QZvVsgkD0trlomsj1pfyZ/u/nGoqRJyZvLs6RRadxnfkiXlv+d2OfuBk6YWlr5SGBVGuprHBTxso2kXduMbf42w/n7jy6QrdgDEa1EUQe4b4+h8XNiABUvNm9rMfZC9pkUFQhbRrPmGgwsGlz3TiQPUQpsGrQgqCBqLanngF8oYFvRblzvlnhQPIbvoO0H4HmUuTNz1MaI+r9YEPMH3nDmtbAOwJNonXtItMzxD64Hhpli5CwHhDT1Sn/x/KIpetcNBJ6rqVAxIwQxffzdkWKlxD8KfKcSi4PnSq8ImzcrC5SVNecuhdXIZnFp8pVc9lrpaP77Ve+HvDD0rFrQr/XLpr4wCUTDVZQrMjHWXWbOEbDZMa1WbrFFUgl6cSS3tuFTfXAuruySYOIMWg+1v5/dDtSqj9JLe3dGOHm0n0Kuqv7V2R8d/aCi7zbi2E9ev+8MeGwxlI52K2vih7aktVi0mNoBcDQw+K5d6eoZ+2iN4M2PMuvH8lM339Tb+Mab4GHJ2fYnTWX+QsArPA1PjINFS5DJN/rdqB8GFnnRZMhK1nhraEg32qMxHeBAmC3gwmYgsiU5PvMjhIAoOOPkv0fbvJjTYhl0ZLTSvvEVuFKAM5AUpQG+uIKg3J8wcK7FUGehB+Jnm4+BncLzmem0wLp22Iq5ytHWPsLo6swYCSNSVgbhl36uoLa7w/yHS5tbdU+T+mcu02sB/sn2gn6cPzkQ== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: f77520be-4761-4205-433d-08da1e470ba1 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 18:46:11.5475 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UeqbMCuHMxhlnIIrissNd7o+cYHojSNyH03+gLI0XoNqh6u1jmbPf1WxwUZD+N55 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5350 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Just move the exported function into the op and have it accept the struct file instead of vfio_group. Signed-off-by: Jason Gunthorpe Reviewed-by: Kevin Tian --- drivers/vfio/vfio.c | 26 ++++++++++++++++++-------- include/linux/vfio.h | 5 +++-- virt/kvm/vfio.c | 19 +++---------------- 3 files changed, 24 insertions(+), 26 deletions(-) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index c08093fb6d28d5..2eb63d9dded8fb 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -2029,9 +2029,27 @@ static bool vfio_file_enforced_coherent(struct file *filep) return ret; } +/** + * vfio_file_set_kvm - Link a kvm with VFIO drivers + * @filep: VFIO file + * @kvm: KVM to link + * + * The kvm pointer will be forwarded to all the vfio_device's attached to the + * VFIO file via the VFIO_GROUP_NOTIFY_SET_KVM notifier. + */ +static void vfio_file_set_kvm(struct file *filep, struct kvm *kvm) +{ + struct vfio_group *group = filep->private_data; + + group->kvm = kvm; + blocking_notifier_call_chain(&group->notifier, + VFIO_GROUP_NOTIFY_SET_KVM, kvm); +} + static const struct vfio_file_ops vfio_file_group_ops = { .get_iommu_group = vfio_file_iommu_group, .is_enforced_coherent = vfio_file_enforced_coherent, + .set_kvm = vfio_file_set_kvm, }; /** @@ -2461,14 +2479,6 @@ static int vfio_unregister_iommu_notifier(struct vfio_group *group, return ret; } -void vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm) -{ - group->kvm = kvm; - blocking_notifier_call_chain(&group->notifier, - VFIO_GROUP_NOTIFY_SET_KVM, kvm); -} -EXPORT_SYMBOL_GPL(vfio_group_set_kvm); - static int vfio_register_group_notifier(struct vfio_group *group, unsigned long *events, struct notifier_block *nb) diff --git a/include/linux/vfio.h b/include/linux/vfio.h index b1583eb80f12e6..bc6e47f3f26560 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -15,6 +15,8 @@ #include #include +struct kvm; + /* * VFIO devices can be placed in a set, this allows all devices to share this * structure and the VFIO core will provide a lock that is held around @@ -141,6 +143,7 @@ int vfio_mig_get_next_state(struct vfio_device *device, struct vfio_file_ops { struct iommu_group *(*get_iommu_group)(struct file *filep); bool (*is_enforced_coherent)(struct file *filep); + void (*set_kvm)(struct file *filep, struct kvm *kvm); }; extern struct vfio_group *vfio_group_get_external_user(struct file *filep); extern void vfio_group_put_external_user(struct vfio_group *group); @@ -186,8 +189,6 @@ extern int vfio_unregister_notifier(struct device *dev, enum vfio_notify_type type, struct notifier_block *nb); -struct kvm; -extern void vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm); /* * Sub-module helpers diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index f5ef78192a97ab..9baf04c5b0cc3d 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -73,19 +73,6 @@ static void kvm_vfio_group_put_external_user(struct vfio_group *vfio_group) symbol_put(vfio_group_put_external_user); } -static void kvm_vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm) -{ - void (*fn)(struct vfio_group *, struct kvm *); - - fn = symbol_get(vfio_group_set_kvm); - if (!fn) - return; - - fn(group, kvm); - - symbol_put(vfio_group_set_kvm); -} - static void kvm_spapr_tce_release_vfio_group(struct kvm *kvm, struct kvm_vfio_group *kvg) { @@ -184,7 +171,7 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) mutex_unlock(&kv->lock); - kvm_vfio_group_set_kvm(vfio_group, dev->kvm); + kvg->ops->set_kvm(kvg->filp, dev->kvm); kvm_vfio_update_coherency(dev); return 0; @@ -218,7 +205,7 @@ static int kvm_vfio_group_del(struct kvm_device *dev, unsigned int fd) list_del(&kvg->node); kvm_arch_end_assignment(dev->kvm); kvm_spapr_tce_release_vfio_group(dev->kvm, kvg); - kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); + kvg->ops->set_kvm(kvg->filp, NULL); kvm_vfio_group_put_external_user(kvg->vfio_group); fput(kvg->filp); kfree(kvg); @@ -334,7 +321,7 @@ static void kvm_vfio_destroy(struct kvm_device *dev) list_for_each_entry_safe(kvg, tmp, &kv->group_list, node) { kvm_spapr_tce_release_vfio_group(dev->kvm, kvg); - kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); + kvg->ops->set_kvm(kvg->filp, NULL); kvm_vfio_group_put_external_user(kvg->vfio_group); fput(kvg->filp); list_del(&kvg->node); From patchwork Thu Apr 14 18:46:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12813884 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B95BC4332F for ; Thu, 14 Apr 2022 18:46:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344218AbiDNSsr (ORCPT ); Thu, 14 Apr 2022 14:48:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52758 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243073AbiDNSsk (ORCPT ); Thu, 14 Apr 2022 14:48:40 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2087.outbound.protection.outlook.com [40.107.220.87]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B684CDB4A9 for ; Thu, 14 Apr 2022 11:46:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dGh5hqDmYA1NsPq0s09MHDtWY5Cncu9058UFA38vjECviWSwwSsPcYMWe7zD4j0VUN4LoYDNZM6FxVb1wAJhlpnemC07OUr7D2gizfImyTyDsiNYhcKyDXmrHzD+urDFKY+k7wJh19fWazPTFl5luEABgwl5EhtJOS+LlDaF+LvyuvtBrXtI1nGfWlT6KTznzL57QL7J6dG0UgMHfvFmXTVSEMHXgl5lLsqmYwtp9XgwJp+qlLkjqq8uUSQZKhxQDuLGyLfimTaqBsJV1Ec6dX/2n0k1njI5buhGpbtwFA4zEoTrJw9eEnbeoX82wSI1UtvbPA9kwJrC6wetfJaKDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=OeB8zsfNocOWeq+KHtYMknWpSMIwC5gV9pKb5oElmAE=; b=lc7D1MO7har8Enp456300ZLS6Pt/SzjIY0bxqPWP8dujoHkCrJOCq+/jLYa+65lo8Ab547eZtnq4AdyGZj6k0e9giEAx6CeE3Dy4M8KxdhoXeX6M0b+Q1ibvxnzYpIarioXkcZnCmMdjLH7geJJHYCdN6RqhPVBDmRBtpxCLRY6XlNH4mvOsEOTScA5Nu5Nn3ht4f4ZM0piUIFWijgtQBC7fRhY18sf7oolzso4IBW4gbpnhqezHuUycaxzR83xsE/vaWjvSgNOs96ypht706UcwjKvlavyLRwhw7K1vUj1ZOFT5sZp/fGPStzjp13RAUVQzEudMHdV7Gk2rbg0IcA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OeB8zsfNocOWeq+KHtYMknWpSMIwC5gV9pKb5oElmAE=; b=j8+zeeT5UPui7kuEQsjEYRtrsgixvbSgYIHQNdATSmjiPQdQV/+bMko4QcXxR5QWti/4yTg6QGguLYzoKoOAV7pzg4Ck2jpJ9DpPDSuEgNt3K4hkOMEBkLkyTn1BIMQyX79wgbZrTxQh37EQHfK/h7pReJM+nrjE7NkVZYLGPRfGk2qjJhRNkpUywMrJRgp+lNnNivMkij5Gj5oj0ApPD12io+W8Nea6eBlHkTsIKmKcCXdaHS4bqudzmEGZyrMwlPMNCRKhwWcPppgSVQFxU5P7Vzpj7WXpBNNmgiW1Kfs53x4CZUUMxpp7AIdE/OtJgJbB1pRQkw4QFtcn3e6Hxw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BL1PR12MB5350.namprd12.prod.outlook.com (2603:10b6:208:31d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 18:46:11 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%5]) with mapi id 15.20.5164.020; Thu, 14 Apr 2022 18:46:11 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Yi Liu Subject: [PATCH 09/10] kvm/vfio: Remove vfio_group from kvm Date: Thu, 14 Apr 2022 15:46:08 -0300 Message-Id: <9-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR07CA0016.namprd07.prod.outlook.com (2603:10b6:208:1a0::26) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e8d9dbc7-1c21-4577-3337-08da1e470b50 X-MS-TrafficTypeDiagnostic: BL1PR12MB5350:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: fLIhzwZ4KvaXkGd+Wc2XeKH8G+nr7SMoioCjW5MNj/NJt6/GjpWdloM7aNcZVZfZx+pUiUV6Q0HD/3NauwK4Vfschc+u/jJlF73UwEhHWp7qCw92L/i5rZAyA0ei2VIe7eiNsT2x1c9koymXwVOHJWuzzTMHz81RhaA/DxqUR36X7jRrkUph1AHmXobZCuso5ptyZksuRQ34+FMz8hARyv2ZOY2I+xw8s8COxx/NeqGmsSr0jXidnBUU33tU4i7UljAFoSVfCWcxvc45g6WurUPYuXWRZkgtaDtzV+TWfiCP//dE2kj0mNl2H/pjJL5pWDj5Stjp+g8ZC/PYFHTkkwnTwyDkUw7cul5z4B0bKswkdT0qxx0BhcM70IODQlRW/j80hJXLffoO5QHrMKNIaMtZTsrXqH7PypY2gjVPXmDDWtdLy/Zw0RnJ/pcJi7sNfF59pqzclVORRlJAGbrCWz1ur4gRIoa7453/+K5rbJOlEkwbESka1TEptz+a5u9XYMn2BXRbLTBIOagtfzvLjtNI3yXPW7/7IziYfoy2gud4TKlGSc/u2GrT0Nph2TbP8kDz4A2DekfhB44Wr0fDwsyyCsfbpnDt7Espc/pJ46mdsC0f5P20FEIsJeRTe9ZCV5VmC5EJRpfFDMOXCn8NqA2x8SeI8+e6ctZmpNneyJfQz3fHxKe9Op8WFijhq2SkR25ljFppwMM/hj/DkfJx5w== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(2616005)(86362001)(186003)(38100700002)(83380400001)(110136005)(66946007)(66476007)(66556008)(8936002)(4326008)(8676002)(316002)(5660300002)(54906003)(26005)(6506007)(6512007)(6666004)(36756003)(2906002)(508600001)(6486002)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: n6NucSvzO+JdJOyQV9OJOAurb57VHD5qhknIQc9fumTFBpUUz9qz6ZlLpbYPg4n37np+1ifBu3JPRA+7Hv0mzI7nJwDS2uVmKm1IJeglG4JtEPubRmYOnMYlNTyO+vQ03JlE1tLhEvdIro5ITajKpUdtN2Y6XODXrvp4ka1CCO62MD45Ks5tG/EGGRB4KIxejrZ9aZeE78pt65ClS+jjJ4K5o7Ad81X3F2tW0qiS2tWKgeA0sJXoS7SDGc3ppgU1T0BuoP/gBEpzqSuBYKlFOCOfszG0GQgqQYJfBkjf2GFHHz7WrbTeHdpWNbVzhcrTiSEz+11H/dlasezdGSfI3dt0kndvAP/cF60CnJcnoaJa777uECz/Gt+8yd5/WNats5d8qRRh19rD5acJW5JJZ9w1ZQ8BEhQgZBZqCzMV77Z6ioeHxLhKVbKJcT5mTxWf/HBLG8SbJ6s1t4vUOGvHpeX5sQFW3p/bTolOXVrPJa3C5g8ywx2Quva2qfDy9MCLMON8QmE49bZr9rKD4OUAKEEikQB63KFacDMAvGh1zrTHHDDTUjWNBZuPdNg1sTIN+DrEHlNjbTcAXJu8fgaYQix2jRiTkLx/7Mf5N0M8idXgK/g6jhn2UR1wtMQ6a0Pq2BkBt63Hg4WtgWlcb64najkIMkEWXWA86wKoRrGcyLM36uJhelV94ysfwlkQZxE+yWzQYagB27xmKcYJ+Z3rqZuUUUNGZ7uQi+s6HZG5/sfNAqHFwvgYCbKZD+LfoJHiuoI4YuwnBnzBPkGNN/VpXiyZnpa2hxXIBF0qyaba50Hih2poAF+zNTyxDNIOBmEeYKbO5VW5jm5UqqLHjDx4wyCJPUiozCpSYYqu6/3jNYqWBtRTGulhPtyqy8esPxNIzOW80YzlsQmlRzvBMFQNqSk8qwEWt2x0DC80UaYXAcJYq8K///iiw/eyIWyl6/h1p0uj6ldrVp3jrTsIj39IRw0P+eXfTcYkuEuwD6/sePj61hVP3/TklPwdLPXiT4hueKsfbmqDa4AzPjeUDmxLAclkJXoxDf0tjYBNcBgwVRlw06VJxmJwNUyYETntPH6lUNQmoAleqDYWUnms4Tf0ykxwlx36p3bnTBLQXJ5A5H95PMbJU8WhOOPe3ve0FHekTs1LQeVwIs/oHgd8+6+AThbfQDE+DucBuQzVAggvZPd4HSp6V70WEcZXzlxH1XMgc4xfp9fWeWafZENBTS/eOsvgk6nIwOhg+f5giSu8//16ij1+soZCzHKhMaxHWRfb2INMmMwu9UtACclKvZQNcRbd6XUmq23ACCQwQ3fxuQAFCLMzddbkVf+y2WktBQUNC9jkHJ9N7bKzkBY5o/EcCW7/xkzR9A3vjXJE35xQz8bgB4PNj9WcN5dvn0V3pDqMsCF1pZYjvV3XerB2tANk4wZ7DXTUALz48ICvEgq5t6KhFXyE6KKon5ak1fVEJHpot9MQw7h3PuSgesHHEO1EpD2WgKcQI16W93F5qEgXBqgtHVD9v0wGp1/5LiXOlewQ+peSpdeUFe/PHGlOyV7kRvFbaa+iGUrcuZjSux/2WbXwKhqdZ+CjHXQdCxrE3a3aLL9CMdHrlIMAqi7kFEEQwObR0GjIaIDLOJ18LHAHRUCvqA7HlQgZDmdOtEfclj6oI1cr39wgnhHc1Yd6urtbFeCRo2GxDUGbzfrVDB1a29HKu9OpyrjTHXQEGwQqZw9V9TH5Tp9OKdXWg5Rqv3AkgA== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e8d9dbc7-1c21-4577-3337-08da1e470b50 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 18:46:10.8912 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: FVKsOe0RJa3IHMd4oCdlrn1WBeJlWekoRsPkqYTCTB6k2y/AClTUNQ544KaF9Xko X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5350 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org None of the VFIO APIs take in the vfio_group anymore, so we can remove it completely. This has a subtle side effect on the enforced coherency tracking. The vfio_group_get_external_user() was holding on to the container_users which would prevent the iommu_domain and thus the enforced coherency value from changing while the group is registered with kvm. It changes the security proof slightly into 'user must hold a group FD that has a device that cannot enforce DMA coherence'. As opening the group FD, not attaching the container, is the privileged operation this doesn't change the security properties much. On the flip side it paves the way to changing the iommu_domain/container attached to a group at runtime which is something that will be required to support nested translation. Signed-off-by: Jason Gunthorpe --- virt/kvm/vfio.c | 40 ---------------------------------------- 1 file changed, 40 deletions(-) diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index 9baf04c5b0cc3d..39834a0653d83a 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -35,7 +35,6 @@ struct kvm_vfio_group { struct list_head node; struct file *filp; const struct vfio_file_ops *ops; - struct vfio_group *vfio_group; }; struct kvm_vfio { @@ -44,35 +43,6 @@ struct kvm_vfio { bool noncoherent; }; -static struct vfio_group *kvm_vfio_group_get_external_user(struct file *filep) -{ - struct vfio_group *vfio_group; - struct vfio_group *(*fn)(struct file *); - - fn = symbol_get(vfio_group_get_external_user); - if (!fn) - return ERR_PTR(-EINVAL); - - vfio_group = fn(filep); - - symbol_put(vfio_group_get_external_user); - - return vfio_group; -} - -static void kvm_vfio_group_put_external_user(struct vfio_group *vfio_group) -{ - void (*fn)(struct vfio_group *); - - fn = symbol_get(vfio_group_put_external_user); - if (!fn) - return; - - fn(vfio_group); - - symbol_put(vfio_group_put_external_user); -} - static void kvm_spapr_tce_release_vfio_group(struct kvm *kvm, struct kvm_vfio_group *kvg) { @@ -121,7 +91,6 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) { const struct vfio_file_ops *(*fn)(struct file *filep); struct kvm_vfio *kv = dev->private; - struct vfio_group *vfio_group; struct kvm_vfio_group *kvg; struct file *filp; int ret; @@ -157,15 +126,8 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) goto err_free; } - vfio_group = kvm_vfio_group_get_external_user(filp); - if (IS_ERR(vfio_group)) { - ret = PTR_ERR(vfio_group); - goto err_free; - } - kvg->filp = filp; list_add_tail(&kvg->node, &kv->group_list); - kvg->vfio_group = vfio_group; kvm_arch_start_assignment(dev->kvm); @@ -206,7 +168,6 @@ static int kvm_vfio_group_del(struct kvm_device *dev, unsigned int fd) kvm_arch_end_assignment(dev->kvm); kvm_spapr_tce_release_vfio_group(dev->kvm, kvg); kvg->ops->set_kvm(kvg->filp, NULL); - kvm_vfio_group_put_external_user(kvg->vfio_group); fput(kvg->filp); kfree(kvg); ret = 0; @@ -322,7 +283,6 @@ static void kvm_vfio_destroy(struct kvm_device *dev) list_for_each_entry_safe(kvg, tmp, &kv->group_list, node) { kvm_spapr_tce_release_vfio_group(dev->kvm, kvg); kvg->ops->set_kvm(kvg->filp, NULL); - kvm_vfio_group_put_external_user(kvg->vfio_group); fput(kvg->filp); list_del(&kvg->node); kfree(kvg); From patchwork Thu Apr 14 18:46:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12813881 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D35CFC433EF for ; Thu, 14 Apr 2022 18:46:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237531AbiDNSsn (ORCPT ); Thu, 14 Apr 2022 14:48:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52736 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240994AbiDNSsj (ORCPT ); Thu, 14 Apr 2022 14:48:39 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2087.outbound.protection.outlook.com [40.107.220.87]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B446EDB498 for ; Thu, 14 Apr 2022 11:46:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ghbqc4WS7XYsjIhNtO+5MdoSNdeL2u76LhKt6LQLPBxeEpqoitJQsd9zf+op8QRwnJAg3LCN8RIZH7d1nY6qVbgjay5i8UV4vFZB/XNpPXhOJRhic5rpdaE8e1RsKtdpXa2SEbH/7MQkrD6ql8HKKBGQt0CBKCzUUswA/xBRMC356asZEnSt7SB4k0zUWvM2cJnQ4ilCa4M7osewUDK3IDJB7aA58TzudmxXBf/cT3N15Yg0VBljuVQZT40P3WepvCXs0v/fYzUkcdS7qxs5tr8pFTjdDmCL3X1nuo44XuQJvDjBgSkSZD7wYDhOkY/s3OqwuIhrIg4rznlZ0NwUnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=THEAxfMrOGSEQREsGwAGlsIQ4YJwVl9+3C4Ycni3wg4=; b=FWklssiV5fnIv/hhA+wWoPqBT4jci6uIPico6A7xSQYaCfgEDs5IZYB+lVp/6RM4LE58QmD0k4AaSJ2HH8VhBjHR9O0AZxDm0GbqbJwrMpcKZ9ceuJ9XgDbqmOzWe3tEZ0+wKQTpekUKusVVzvQvoSvqoKSonalIxhM/r0q6cWd+m4fScvtkuuZOED1bzAejbB6c9gHEYBHi/bMbgHZeI70DpMStgHOuRqBxYElxGAiny9J0h74D9oc+G92XLJ1I5dT13T6ytC49yIQVCVUNfTPsNWan5jh7lIHkXRtOTncFKFkfVXJw2syaHFZs3S6DynohXm75OFw1Bgu0vk3aHA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=THEAxfMrOGSEQREsGwAGlsIQ4YJwVl9+3C4Ycni3wg4=; b=U8G+sYe3PhnzmDIkZatgrMW/2laYczsNxfhTPgyy680oDv9xuloA6GoTgEokm6Ede8zvqE0sEW23bGgsgOJVSGCOnnOnBWtwj8Qy09Z2DIl5xbJ0hy6YUdMiVBx3A2nWBs/nicKLegjMaSHvgYCgzFhqttkt/5OZnlAOh9015G8M3i19r1oAUXzdwYwaW+6zDl0n7eltlWyCoeIXbQP9MJJtUlrhCzixMqcK8EfG6ftavyl4DSUIsaOfE/yE4GGpDG2MINK9aNAt/TVuzjDXlL3RDEGFqVAiBBL30EhOehG8hFIS5rosBxBl/N3ftW8kyphuVI1M5WK2sq43iJHO9Q== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BL1PR12MB5350.namprd12.prod.outlook.com (2603:10b6:208:31d::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Thu, 14 Apr 2022 18:46:11 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%5]) with mapi id 15.20.5164.020; Thu, 14 Apr 2022 18:46:11 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Yi Liu Subject: [PATCH 10/10] vfio/pci: Use the struct file as the handle not the vfio_group Date: Thu, 14 Apr 2022 15:46:09 -0300 Message-Id: <10-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v1-33906a626da1+16b0-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL1PR13CA0408.namprd13.prod.outlook.com (2603:10b6:208:2c2::23) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e7891a50-4679-493f-5d28-08da1e470b38 X-MS-TrafficTypeDiagnostic: BL1PR12MB5350:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: xHS/PqMNYPqWqgmTFbaxJqKGRLLJLEbd7uSHSkuq8RFQbVlTHiSpESNSfftg3fP0l1J9MjtbP9D6Mo8fIDJIv/+hNU8c5bPB6NPocrJ9s47JX+EongZ29sLWiziintovxDwRuyt70ot/2Cgrc8G/i2o54ggSMjH4iNVpnCAqqHm1PXpXCP1vyQ7cOznb59jO7JK7O0sp1KUMaM6Lhk1eubUjr1Hk07EaZ9dtWARIFbgstrA5We4sff+4oEMsqu/h2A5rdJHrDkkvFUh6zZGkO2XezqaZj+AqhabCXyWp3Tn7bQbKE//KrXUtpJZoHf34gyKCNRAFeVTLy85ZBzIcBCl+jvUMewYrafQpUt2p94efn1facUlpu5sLc442MqavUFi80+zBYPis70U2uswJ33MUDfMHYZfVyACO1lmyQdY054ZDdA+hyHCqzDDoC798VsoSMdrefuVg4uPdfGla05SPtUMaa96mcfG4fcuhOzxN/dzUGGtmcUOk1EBYdGnJgsvOi6+A9FYNpL7+1PSMgtDwH1Z+h4aRFFMUaFNtu47irdwq+M/lwHUSGIoTSh/uy5HyuNHOkwMNLeQ+7jWz7JWhnlXW/qE7Ldw+WZ6SRlmtJuBw/xnWvt1D98a3WeG/wyYx/Eayu/cB7T5sHUOULA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(2616005)(86362001)(186003)(38100700002)(83380400001)(110136005)(66946007)(66476007)(66556008)(8936002)(4326008)(8676002)(316002)(5660300002)(54906003)(26005)(6506007)(6512007)(36756003)(2906002)(508600001)(6486002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: rIgatBKH8IHx32rHsaufjKOc0DSJrijnVz1yWpounbxDSdlybrQyAuZ8zdw4hXWrrPavfaZIsRT2sBNEeLSLMpAsCm4VutELnIkxmC9g4B72wsItywwjpQxeq9xZverH7v0vrjXWHr5VOB8RX7b0FZFyFQZN5FJ88tgYROrMOeeehqsjKdTqXHM0Ci3SyVpp4NiUlNxa1um+zdlFYjB5zoQRC3wjH2knNMN0EDSP2yfJ3ZxXQwC9/5vzVD+h9h9yvw0CT0ZPbIgSpAdVur5Pt5ncpwE42y9yrti7V3ZJ/wuFdIhv2x4101LCTGyp+hkLGCdlsK8WwjOwfUVnycsav26lsn4QAHI/G0XVH9cKQ5EIRZvIJzEir4Sif7uba8R02vgv3l2U+c1wSDSPfQpbFvOVVO3yaMbzSV6lJK9JT3kvBWyg/w3U1NP+/1VwAzzy+TdH8ee29DsgTYArFaBcQgt6VzggfngjbQ/4zsT5gLaPjdIcqaCdG4p8ayMTWAhrkLROvfs7xQ04Y+mEB8NkGgItilBNrfX4tKw2IE7XgAah7FMk6BtH+MvqY0si1nVNt57/z8sIE99O0yioCwfCjpZJgkZvJ22UPiA/ysfyr/oM9pUzlmwwX2+mzCf+41xD9GCEAPUPB1u8HQTsS4nh5asv0dkiGZVo9iR49KmrF5URtTw8lHUtL3pV6ivt2U5jwQ47RnHJVFtsYD6AFwhmQ6kM+Nj20eaDNkkWOAXsHhbdmjUW5oIzOb1h0ak61jHS/r+61ynvafrCRsbIA5WTX+JSk5u6Iq3uwWEt8Jc4uvUHmJ7P033s/prfl0xByj2zwbt82YOJsWtVPOCDyVgcQmdxEELbBmI/6U9kjq77O9a1E8U0BV3IuI6TFNLE/UW3rn3RdDuBhtkp0eOzElQ+1AfJDVlPQFNRXGfJ4YRtI1gxJeqB2eZdRSst3AeVhzGtKU9QVW+qwacXDwjSX0GN14nrso3w+SeV5/JckF8uWKdsJMpNNgLQT763jaXmmTVqRBTnwsMMTegI6rQfr9Ea1d2lSclVw5XZBkN3mM3SthXL1Nk5btSqunYMTFDZntPED1w+NX5lvwHnK1SYRGE4dhwr87HpaV4kSbp6Hx4RRaTKuugqkRDu1sc0iM1jo8C8Sa7PqM1/11vxhmsXVlMqGQUG4BRjgOcQzvGjQtuC9K2v+mB5458P71MHtT7mWxIDxV2zK0YRGrTDJgNIlZZ2XYZvGvSjBi8uMr0Sh5IB6GepmTbqxkvXG8tJS59vOIyqgt3KaedU4QsvDGOH6yrPHKEoB7xaxMUxR2Dvly6RkmO7f5QQ7pr+NxgvGXkwsWPYvQ3KTW2fwqngTSee8/dck+a0ug2tFtb/CtVNbfTWyNSd34vbYq0akWVZ87i4ukGVMTQ6fhnJTFwaIAck4zYEruRYaNbxa3oABrh2kwDLMyLu1X29tXP8lel3lBDA1XwyQlWRmp3SMnajyXhP7bw/4FP5vTatm5dnvqlqncZDmjCsFBU6YmY+jbAxg4kmTMMLlcUearPYSbqh5LnQ3y1zjOssFqZoN98XE2SCCAw5DMDOWWlLb73khF6KU7bxKhTtF2O5nBOTCymIb8h9EIArg+9hqBYcyy7bpGQkUmHPueQqqmbvkVOv8yEX372+uZGrhIfsE5LZ1to+8/IirFH+MUX6SBZMzSybsjgw7NUOx4WJY4quOm9mIrIw53q6//+/YKnAq4uzAj94gGcIN99G6A== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e7891a50-4679-493f-5d28-08da1e470b38 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2022 18:46:10.7819 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: EdY9/QOqTNCWTV+mLKwKAmHfe6STHbeICQEo3OtLOcyTh8zZlIZmtYALtzNGbimY X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5350 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org VFIO PCI does a security check as part of hot reset to prove that the user has permission to manipulate all the devices that will be impacted by the reset. Use a new API vfio_file_has_dev() to perform this security check against the struct file directly and remove the vfio_group from VFIO PCI. Since VFIO PCI was the last user of vfio_group_get_external_user() remove it as well. Signed-off-by: Jason Gunthorpe --- drivers/vfio/pci/vfio_pci_core.c | 43 +++++++++++----------- drivers/vfio/vfio.c | 63 +++++++++----------------------- include/linux/vfio.h | 2 +- 3 files changed, 41 insertions(+), 67 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index b7bb16f92ac628..ea584934683848 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -577,7 +577,7 @@ static int vfio_pci_fill_devs(struct pci_dev *pdev, void *data) struct vfio_pci_group_info { int count; - struct vfio_group **groups; + struct file **files; }; static bool vfio_pci_dev_below_slot(struct pci_dev *pdev, struct pci_slot *slot) @@ -1039,10 +1039,10 @@ long vfio_pci_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd, } else if (cmd == VFIO_DEVICE_PCI_HOT_RESET) { struct vfio_pci_hot_reset hdr; int32_t *group_fds; - struct vfio_group **groups; + struct file **files; struct vfio_pci_group_info info; bool slot = false; - int group_idx, count = 0, ret = 0; + int file_idx, count = 0, ret = 0; minsz = offsetofend(struct vfio_pci_hot_reset, count); @@ -1075,17 +1075,17 @@ long vfio_pci_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd, return -EINVAL; group_fds = kcalloc(hdr.count, sizeof(*group_fds), GFP_KERNEL); - groups = kcalloc(hdr.count, sizeof(*groups), GFP_KERNEL); - if (!group_fds || !groups) { + files = kcalloc(hdr.count, sizeof(*files), GFP_KERNEL); + if (!group_fds || !files) { kfree(group_fds); - kfree(groups); + kfree(files); return -ENOMEM; } if (copy_from_user(group_fds, (void __user *)(arg + minsz), hdr.count * sizeof(*group_fds))) { kfree(group_fds); - kfree(groups); + kfree(files); return -EFAULT; } @@ -1094,22 +1094,23 @@ long vfio_pci_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd, * user interface and store the group and iommu ID. This * ensures the group is held across the reset. */ - for (group_idx = 0; group_idx < hdr.count; group_idx++) { - struct vfio_group *group; - struct fd f = fdget(group_fds[group_idx]); - if (!f.file) { + for (file_idx = 0; file_idx < hdr.count; file_idx++) { + struct file *file = fget(group_fds[file_idx]); + const struct vfio_file_ops *ops; + + if (!file) { ret = -EBADF; break; } - group = vfio_group_get_external_user(f.file); - fdput(f); - if (IS_ERR(group)) { - ret = PTR_ERR(group); + ops = vfio_file_get_ops(file); + if (IS_ERR(ops)) { + fput(file); + ret = PTR_ERR(ops); break; } - groups[group_idx] = group; + files[file_idx] = file; } kfree(group_fds); @@ -1119,15 +1120,15 @@ long vfio_pci_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd, goto hot_reset_release; info.count = hdr.count; - info.groups = groups; + info.files = files; ret = vfio_pci_dev_set_hot_reset(vdev->vdev.dev_set, &info); hot_reset_release: - for (group_idx--; group_idx >= 0; group_idx--) - vfio_group_put_external_user(groups[group_idx]); + for (file_idx--; file_idx >= 0; file_idx--) + fput(files[file_idx]); - kfree(groups); + kfree(files); return ret; } else if (cmd == VFIO_DEVICE_IOEVENTFD) { struct vfio_device_ioeventfd ioeventfd; @@ -1964,7 +1965,7 @@ static bool vfio_dev_in_groups(struct vfio_pci_core_device *vdev, unsigned int i; for (i = 0; i < groups->count; i++) - if (groups->groups[i] == vdev->vdev.group) + if (vfio_file_has_dev(groups->files[i], &vdev->vdev)) return true; return false; } diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 2eb63d9dded8fb..69772105ec43e8 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -1899,51 +1899,6 @@ static const struct file_operations vfio_device_fops = { .mmap = vfio_device_fops_mmap, }; -/* - * External user API, exported by symbols to be linked dynamically. - * - * The protocol includes: - * 1. do normal VFIO init operation: - * - opening a new container; - * - attaching group(s) to it; - * - setting an IOMMU driver for a container. - * When IOMMU is set for a container, all groups in it are - * considered ready to use by an external user. - * - * 2. User space passes a group fd to an external user. - * The external user calls vfio_group_get_external_user() - * to verify that: - * - the group is initialized; - * - IOMMU is set for it. - * If both checks passed, vfio_group_get_external_user() - * increments the container user counter to prevent - * the VFIO group from disposal before KVM exits. - * - * 3. When the external KVM finishes, it calls - * vfio_group_put_external_user() to release the VFIO group. - * This call decrements the container user counter. - */ -struct vfio_group *vfio_group_get_external_user(struct file *filep) -{ - struct vfio_group *group = filep->private_data; - int ret; - - if (filep->f_op != &vfio_group_fops) - return ERR_PTR(-EINVAL); - - ret = vfio_group_add_container_user(group); - if (ret) - return ERR_PTR(ret); - - /* - * Since the caller holds the fget on the file group->users must be >= 1 - */ - vfio_group_get(group); - - return group; -} -EXPORT_SYMBOL_GPL(vfio_group_get_external_user); - /* * External user API, exported by symbols to be linked dynamically. * The external user passes in a device pointer @@ -2068,6 +2023,24 @@ const struct vfio_file_ops *vfio_file_get_ops(struct file *filep) } EXPORT_SYMBOL_GPL(vfio_file_get_ops); +/** + * vfio_file_has_dev - True if the VFIO file is a handle for device + * @filep: VFIO file to check + * @device: Device that must be part of the file + * + * Returns true if given file has permission to manipulate the given device. + */ +bool vfio_file_has_dev(struct file *filep, struct vfio_device *device) +{ + struct vfio_group *group = filep->private_data; + + if (filep->f_op != &vfio_group_fops) + return false; + + return group == device->group; +} +EXPORT_SYMBOL_GPL(vfio_file_has_dev); + /* * Sub-module support */ diff --git a/include/linux/vfio.h b/include/linux/vfio.h index bc6e47f3f26560..ea254283fa0522 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -145,11 +145,11 @@ struct vfio_file_ops { bool (*is_enforced_coherent)(struct file *filep); void (*set_kvm)(struct file *filep, struct kvm *kvm); }; -extern struct vfio_group *vfio_group_get_external_user(struct file *filep); extern void vfio_group_put_external_user(struct vfio_group *group); extern struct vfio_group *vfio_group_get_external_user_from_dev(struct device *dev); const struct vfio_file_ops *vfio_file_get_ops(struct file *filep); +bool vfio_file_has_dev(struct file *filep, struct vfio_device *device); #define VFIO_PIN_PAGES_MAX_ENTRIES (PAGE_SIZE/sizeof(unsigned long))