From patchwork Wed Apr 20 19:23:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12820727 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35120C433EF for ; Wed, 20 Apr 2022 19:23:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1381769AbiDTT0I (ORCPT ); Wed, 20 Apr 2022 15:26:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46332 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1381764AbiDTT0H (ORCPT ); Wed, 20 Apr 2022 15:26:07 -0400 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam08on2070.outbound.protection.outlook.com [40.107.102.70]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 52463424BF for ; Wed, 20 Apr 2022 12:23:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=g6biPazQvtfOCyBfTDVk2o7uEcrX2YCU/SPd1Y7OvfUCK+kMw2uzE02pqJbBY7S8UxtisDPf9yYTe4cFJajKvO2VFfY2Wwzoewctbj2qZ0foGemqeh4yRXWOrmyPz5yNR7tDoOFPEHzehYLxA0pET3rxwNQMc11iB5edH48EUtxo65f5zEnkiukAmdTRYVUYP8W5EoRCne76OVjOjdemf8RBLeVq9FmT1P6JBYXOknAyy4hvURasUTQ0pWUXYCICHUjgL4gFlNHqlSLaQe6mykNtC6MlnIKGlU84iaOthZr20I3M4Q5UmR3H8Z5+WAzXNL1AQJ/pp7cSOOxxPfdZ3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=039dlDF4xOBIrNO6BimXWmwlz4m6tv4rP/nmsVn1s9c=; b=USU0hr+7UucSfaRbR3BYiuXXlEu1rhW3RCoUEURu/SfzjRyMvatlDh/iXsyahhnp1CIAyEXorHDyAbfipSqYI8Ly18Eo1YK9a+/9Pux62jg6n6KuMGDnWZGzDbAT3yW4aa/M0m07FTi1p0RP2LWWtbxP4NADrKfdaQvWNaok0RNhobc9xKyGCdSM/ZklLP32C1rEf2THiljXgAoZlMFmtuZfOzxdC2f6hlO+pYLHIYnf3P6D73DKB9hxomQa6vvBAjhNeWxUB9NOTgXqg3jgsD4C+uG/TBd8cj1S1rcAdWngekzkIYRaBZUvb7YwGdFlob77K71fZQH18in7Nwq2QQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=039dlDF4xOBIrNO6BimXWmwlz4m6tv4rP/nmsVn1s9c=; b=lZvwP5Mr0IrsVnXY95qZWy3+igJ16TpNuNXphp3Ax9Qzyq/Er0WjlOH6TCCEbs89KpXyD4/13eE3sg6S78JRTbMvqqgWbKSjh6CzVHX2X4irymvfLBRCwifhIBXXX7sK97QeB0bs562F2EhzBb3+BoeFtuyhgzBCFGtCgh4nbyDMWF5N7HxRCeAiTiHzcG+uod+1u7MyRuGUMyXAVq84Mmq07SYIJYur1se6vwlZXzK9bGw9KwzQg4CIMuCM1/I0QgonhV93RacKYZFuT0Ysvbf8z2kxXuBD40sbN8k4xkFcAHgaW/cndawNafjW9zGkjCH2RPE1no+v4k5wcw/4Qg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by DM5PR12MB4679.namprd12.prod.outlook.com (2603:10b6:4:a2::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.14; Wed, 20 Apr 2022 19:23:18 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5186.014; Wed, 20 Apr 2022 19:23:18 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Kevin Tian , Yi Liu Subject: [PATCH v2 1/8] kvm/vfio: Move KVM_DEV_VFIO_GROUP_* ioctls into functions Date: Wed, 20 Apr 2022 16:23:10 -0300 Message-Id: <1-v2-6a528653a750+1578a-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v2-6a528653a750+1578a-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR05CA0037.namprd05.prod.outlook.com (2603:10b6:208:236::6) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 93511acf-9106-4dda-aaf9-08da2303398f X-MS-TrafficTypeDiagnostic: DM5PR12MB4679:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oF89w1TLolCXw1Wzcz7N3VvtUIR/viS16wSDa4yxsf5gBevOlnlTtCbs3fcCcqTyueEODWXWxN2qUn9QG5XcubsHWOwAxX0bAAnUBdZ88isUZZLqlV0/kmd8IcOx05aUieyn+7/kg4iRVtUmgoIeBX4z3SCzmI54i8HrsO0+3jdjzXzjQOylO0VYniSjfPw/n6w2y4A7SKgMQ/ahiiC4OfFUSmEoTyUJIx4AFqHKaC68XJmVypXd83nmwrUWIpvWuqhGdN0/BzlaJFZJAmhVDWS5JyK/zO5+UvOiQRtRTjoI7tkpeLZmbBRinJ6bRMtjKlOQugGw0eNAgCVCYXXriK2vpUnEBcZS8yKDHC5c2LpYGgynuntgDvzDHu8akV5i2ms3Y7a5E9TCxmf3I49FD8MR5EjWsBT3ZbavYiG3o0LhH7LOCPcAEAJCzXswQ3BhREguSGF27CghpyGhdTO/gJf+c7TScQkJcyxPEeeB6Mdq0lOH9aw2TAS8DpKRz3kRGVlxk4W/J+uUCt/lazG33yzc0Ju1XLUhUh0boJw4vicoNWGCPVqP2sS6CF4mTu9AFISOH7+rkn/Ml8S1vfL5hXSkp2IXYC+RsnnFgHm94RXxFOOPyPc1h0IoKE+pC46s9AqpHyxkVLdw8WY98wMnvW3cVtqqWjBjH0wLqDEGGeEw18Dh/aB2UPj3Ju9d3BLg X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(508600001)(86362001)(5660300002)(83380400001)(6486002)(36756003)(4326008)(2616005)(186003)(8676002)(8936002)(2906002)(66556008)(6506007)(66476007)(6512007)(6666004)(26005)(38100700002)(110136005)(54906003)(66946007)(316002)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: zvuibC7lJfaaBH5hnFpBSGHJL20wD3szly1rNoa9JcEFckDH6DMyoUeCR59kQI3BcchSzNEzlWRnblaEfGv7kx3cN7re4SH9N5O6RvxwikqrBI0CINqmVaWSQxbj0ogH6+Hq9uL/xtsL8Ry6miwTljjBnzfTfbQ5AI3+6CFWIN+D7fC5nVSmMh+0TDpFl5ndj5UWWxf0T9fWhkuRSXojvm8KinihpWOoVsZwjVVf0biIn3Tu3PO4kXr4c/hyDGf+AAzO3p4H4Vl/5roKpTvQzWQMuUO8Bi6ZfH5nOkb9wrfEI+68aAgiOcgFgDwEdL85eM5elnYxCxKHfLcjDngSoIABgGJmSiIQi4OfOv7eHa4yPy0QEpdDtoZ0mHCiuDpy0rwZAmIyBJzdjkSQ0OCijQw7o4iPgrzXrIRqZ+Cdi7Rx+ZirUZKTH5/s4hGbdBB7Sn7KaAFvze92/De8DtZHf0/FELu3n5SHR/VpzQBvzLGZ0zGWlcpa2L8sFXP3qJgI9vaeMuQmJOQEzLDLsrlEWhm92WDMqH0v59AOkwXIyV2KWb8jma+zoDr6TVSyWQZLTE4kJedomL4/UkENVQCql6SvmY0JUgV3Bt51IkF7Mb6zUrQn6a8VFGjXud3DnbdSmCQLSC1wQVZTnUaFZuBoJEqbCq/jXqcNgKdGRK9Ru5z5xeSOalvA/rlmnDnfxfpR1mvPxTMuapIoc377YccrSjVAvJ3sMiD6MBDRJXkJZLyeFxRI75KvJbFQy1LxtepSh6OKxa2fZmMdd1hVEqONaPVMIU2awgcCkzzk/vHggyc6u0yLEb3tJjKGSgk9uiasU5rICz6TfXvLAY1zvFK3c9hWW7kmpE9HB1RZOXLNXgor58/CN4HypAoQS2tMWHZayR9xMS8KyL8lFX7hJSHyukCaA/GauAKRq74FTvScsGn3tAGgxuYcQM3r/Vti0bjcNbXt6cC5iUrl2v97Q11C5X9n7K4kfY0pGsMAR/+iwdVikTruTPIl2yRsXtPUO7lyQNoyRsN4yafKcLcFTq8aqPDZv3fkcivt+O7qZATUQ7+/BBscqRwwxrGxgD9h+3QRMweDVjBurA0yWGXiGcqIzuUsXUaNlp27ZmEpaIIQbF3iw9AXJRVJXJu2vk6ScjK7W3UqtpIwX5CsCtGWQMeNDYJdoctBYA1vDOnUMirF7u7LkhYfq5mJ8Td5Lz9uCrO+EemexgcyHpgUikKJvItOTjRnd5oMHwGV2LkRxn2y0gV2KNJv29yURiWcPxk37WTApf5vsArYpXerhmywxR0Zlw2Tj/UUaYCrgdHsvtiqVZ6REMAUMA5NTapDHeajnr7rTpHj7Qcg1cigca8yF+RNpQoa1vBqoVkUvirWlcS6Hokf+d9xll5HOAY/pxn0rR+1g2uG9eCHE91BTdqwPjbJ4niUH5+f9GFOCQy8sgnufB16l9SGJOHavfbXmJd2pmswDeFICKk96YdbzFhA7rzHj9mmeBJE0mMBaAVaq+inzgKp/75bkpcNjqFaEFvtFPE4892hDO48NXUT2ccAsJawgcCajRACyCXmsjxerJDHfY/R9k89GOFGOF1Nqq3o2XtJaWCF4CUfOA1htHrwYKlUbm2cZ/ksFpX/OvZTABshQPqI+I3FkumIIbnUAtKub3svXo/JTjRDVsGCRFZj2uskZ2X1/DAdybqn9i7idppBQ1g75/7cwK9K+i0mh7PF1EhFzkOBSSTNnQOMF9/WW6M0Pg== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 93511acf-9106-4dda-aaf9-08da2303398f X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Apr 2022 19:23:18.5968 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Yzl62jM6uDdEZQXAzwTMMOCR/ARbRgkNk7neCuSE4j4yUVVhqPueqktpfBwdIVlb X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB4679 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org To make it easier to read and change in following patches. Reviewed-by: Kevin Tian Reviewed-by: Christoph Hellwig Signed-off-by: Jason Gunthorpe Reviewed-by: Cornelia Huck Reviewed-by: Yi Liu --- virt/kvm/vfio.c | 271 ++++++++++++++++++++++++++---------------------- 1 file changed, 146 insertions(+), 125 deletions(-) This is best viewed with 'git show -b' diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index 8fcbc50221c2d2..a1167ab7a2246f 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -181,149 +181,170 @@ static void kvm_vfio_update_coherency(struct kvm_device *dev) mutex_unlock(&kv->lock); } -static int kvm_vfio_set_group(struct kvm_device *dev, long attr, u64 arg) +static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) { struct kvm_vfio *kv = dev->private; struct vfio_group *vfio_group; struct kvm_vfio_group *kvg; - int32_t __user *argp = (int32_t __user *)(unsigned long)arg; struct fd f; - int32_t fd; int ret; + f = fdget(fd); + if (!f.file) + return -EBADF; + + vfio_group = kvm_vfio_group_get_external_user(f.file); + fdput(f); + + if (IS_ERR(vfio_group)) + return PTR_ERR(vfio_group); + + mutex_lock(&kv->lock); + + list_for_each_entry(kvg, &kv->group_list, node) { + if (kvg->vfio_group == vfio_group) { + ret = -EEXIST; + goto err_unlock; + } + } + + kvg = kzalloc(sizeof(*kvg), GFP_KERNEL_ACCOUNT); + if (!kvg) { + ret = -ENOMEM; + goto err_unlock; + } + + list_add_tail(&kvg->node, &kv->group_list); + kvg->vfio_group = vfio_group; + + kvm_arch_start_assignment(dev->kvm); + + mutex_unlock(&kv->lock); + + kvm_vfio_group_set_kvm(vfio_group, dev->kvm); + kvm_vfio_update_coherency(dev); + + return 0; +err_unlock: + mutex_unlock(&kv->lock); + kvm_vfio_group_put_external_user(vfio_group); + return ret; +} + +static int kvm_vfio_group_del(struct kvm_device *dev, unsigned int fd) +{ + struct kvm_vfio *kv = dev->private; + struct kvm_vfio_group *kvg; + struct fd f; + int ret; + + f = fdget(fd); + if (!f.file) + return -EBADF; + + ret = -ENOENT; + + mutex_lock(&kv->lock); + + list_for_each_entry(kvg, &kv->group_list, node) { + if (!kvm_vfio_external_group_match_file(kvg->vfio_group, + f.file)) + continue; + + list_del(&kvg->node); + kvm_arch_end_assignment(dev->kvm); +#ifdef CONFIG_SPAPR_TCE_IOMMU + kvm_spapr_tce_release_vfio_group(dev->kvm, kvg->vfio_group); +#endif + kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); + kvm_vfio_group_put_external_user(kvg->vfio_group); + kfree(kvg); + ret = 0; + break; + } + + mutex_unlock(&kv->lock); + + fdput(f); + + kvm_vfio_update_coherency(dev); + + return ret; +} + +#ifdef CONFIG_SPAPR_TCE_IOMMU +static int kvm_vfio_group_set_spapr_tce(struct kvm_device *dev, + void __user *arg) +{ + struct kvm_vfio_spapr_tce param; + struct kvm_vfio *kv = dev->private; + struct vfio_group *vfio_group; + struct kvm_vfio_group *kvg; + struct fd f; + struct iommu_group *grp; + int ret; + + if (copy_from_user(¶m, arg, sizeof(struct kvm_vfio_spapr_tce))) + return -EFAULT; + + f = fdget(param.groupfd); + if (!f.file) + return -EBADF; + + vfio_group = kvm_vfio_group_get_external_user(f.file); + fdput(f); + + if (IS_ERR(vfio_group)) + return PTR_ERR(vfio_group); + + grp = kvm_vfio_group_get_iommu_group(vfio_group); + if (WARN_ON_ONCE(!grp)) { + ret = -EIO; + goto err_put_external; + } + + ret = -ENOENT; + + mutex_lock(&kv->lock); + + list_for_each_entry(kvg, &kv->group_list, node) { + if (kvg->vfio_group != vfio_group) + continue; + + ret = kvm_spapr_tce_attach_iommu_group(dev->kvm, param.tablefd, + grp); + break; + } + + mutex_unlock(&kv->lock); + + iommu_group_put(grp); +err_put_external: + kvm_vfio_group_put_external_user(vfio_group); + return ret; +} +#endif + +static int kvm_vfio_set_group(struct kvm_device *dev, long attr, u64 arg) +{ + int32_t __user *argp = (int32_t __user *)(unsigned long)arg; + int32_t fd; + switch (attr) { case KVM_DEV_VFIO_GROUP_ADD: if (get_user(fd, argp)) return -EFAULT; - - f = fdget(fd); - if (!f.file) - return -EBADF; - - vfio_group = kvm_vfio_group_get_external_user(f.file); - fdput(f); - - if (IS_ERR(vfio_group)) - return PTR_ERR(vfio_group); - - mutex_lock(&kv->lock); - - list_for_each_entry(kvg, &kv->group_list, node) { - if (kvg->vfio_group == vfio_group) { - mutex_unlock(&kv->lock); - kvm_vfio_group_put_external_user(vfio_group); - return -EEXIST; - } - } - - kvg = kzalloc(sizeof(*kvg), GFP_KERNEL_ACCOUNT); - if (!kvg) { - mutex_unlock(&kv->lock); - kvm_vfio_group_put_external_user(vfio_group); - return -ENOMEM; - } - - list_add_tail(&kvg->node, &kv->group_list); - kvg->vfio_group = vfio_group; - - kvm_arch_start_assignment(dev->kvm); - - mutex_unlock(&kv->lock); - - kvm_vfio_group_set_kvm(vfio_group, dev->kvm); - - kvm_vfio_update_coherency(dev); - - return 0; + return kvm_vfio_group_add(dev, fd); case KVM_DEV_VFIO_GROUP_DEL: if (get_user(fd, argp)) return -EFAULT; + return kvm_vfio_group_del(dev, fd); - f = fdget(fd); - if (!f.file) - return -EBADF; - - ret = -ENOENT; - - mutex_lock(&kv->lock); - - list_for_each_entry(kvg, &kv->group_list, node) { - if (!kvm_vfio_external_group_match_file(kvg->vfio_group, - f.file)) - continue; - - list_del(&kvg->node); - kvm_arch_end_assignment(dev->kvm); #ifdef CONFIG_SPAPR_TCE_IOMMU - kvm_spapr_tce_release_vfio_group(dev->kvm, - kvg->vfio_group); + case KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE: + return kvm_vfio_group_set_spapr_tce(dev, (void __user *)arg); #endif - kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); - kvm_vfio_group_put_external_user(kvg->vfio_group); - kfree(kvg); - ret = 0; - break; - } - - mutex_unlock(&kv->lock); - - fdput(f); - - kvm_vfio_update_coherency(dev); - - return ret; - -#ifdef CONFIG_SPAPR_TCE_IOMMU - case KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE: { - struct kvm_vfio_spapr_tce param; - struct kvm_vfio *kv = dev->private; - struct vfio_group *vfio_group; - struct kvm_vfio_group *kvg; - struct fd f; - struct iommu_group *grp; - - if (copy_from_user(¶m, (void __user *)arg, - sizeof(struct kvm_vfio_spapr_tce))) - return -EFAULT; - - f = fdget(param.groupfd); - if (!f.file) - return -EBADF; - - vfio_group = kvm_vfio_group_get_external_user(f.file); - fdput(f); - - if (IS_ERR(vfio_group)) - return PTR_ERR(vfio_group); - - grp = kvm_vfio_group_get_iommu_group(vfio_group); - if (WARN_ON_ONCE(!grp)) { - kvm_vfio_group_put_external_user(vfio_group); - return -EIO; - } - - ret = -ENOENT; - - mutex_lock(&kv->lock); - - list_for_each_entry(kvg, &kv->group_list, node) { - if (kvg->vfio_group != vfio_group) - continue; - - ret = kvm_spapr_tce_attach_iommu_group(dev->kvm, - param.tablefd, grp); - break; - } - - mutex_unlock(&kv->lock); - - iommu_group_put(grp); - kvm_vfio_group_put_external_user(vfio_group); - - return ret; - } -#endif /* CONFIG_SPAPR_TCE_IOMMU */ } return -ENXIO; From patchwork Wed Apr 20 19:23:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12820728 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8CDECC433F5 for ; Wed, 20 Apr 2022 19:23:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1381776AbiDTT0J (ORCPT ); Wed, 20 Apr 2022 15:26:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46370 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1381767AbiDTT0I (ORCPT ); Wed, 20 Apr 2022 15:26:08 -0400 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam08on2070.outbound.protection.outlook.com [40.107.102.70]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 87FB846145 for ; Wed, 20 Apr 2022 12:23:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HsLaz5Jd+4nbahhaz37Uyi9K/9tOiAHN57x58S4FwQrQhW82eDmI52SXTxf45ZdJcSstfqemPSLZIOCnBa3RrVgPkZNA+kZnHJSfoAFhkOowj2BkQ4Qo20REwvq0aAAY3HiiW7ghOiSsjwZMdMJgbG3CiHQKzi83eeM7DqBcXw2tU7IwuuMbvQi1Q05F/ByswAdK41RbqRzXftCrVbsrcUROdqY5knewdXcCPgbkvVHznMLAhYxXI5WugJkQyw9BgDnwqVCgk0XYsjdGlRV9tRcG1PU7vS49S7m4nHhYj0i+4QTQvoFm4Dy2rnH3dJl/8NqAiad2XIioFDDTBgfaAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zn1b/fLCtFnyGYgnpenaucag+dFyfPTCA0mVoJxvcn8=; b=EiGyzHGPk44P8Ae9TVzhrPsfQgds7+I78MdNf1lSP+MCmzGKuGKY0x4PxXWnEWU6VryBMrI2eokQdw+rMVpUHeXQl4p/qI2vZJxu6idhz6KQ2+H3Q2z5T4lepf0I/s9RVSji6CdcPXUSreQMI3sr9ckBasax6DWV6yz8EFuw08uWvLQVPoDP5VVmYlM9O3nsVps1SIOsH4fWYCPiE8Usf+OeJBsM/42JO7UcVdItiRsl1US7RHMjPptKO8ACRBNW5EMmerBPGPtsHCv0jQ1SVNTZYDvhxtfmiCoX7rEkxTHYzUIAxs9gP3VB6h3IfM+WQ6HGSbNiJN2cVBJv2PrZcQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zn1b/fLCtFnyGYgnpenaucag+dFyfPTCA0mVoJxvcn8=; b=DZlOFGDjVGXNN3fD2e2+aD+UC1oJevw69d9bERvFX+uvUeGD2x3aBAy1/7NNPqiEDLHAlyzWjsJ2VuesdzuZOSIgoOapCBrytltbIcIxRlWL+ZjZ0+aS2GT8DyQLv7AE5UfTEpigvUunBw/fjXiOtniVipBUeX+ISPvvclK8FqVXK/8P4hLN8Fn+VV137VlvR2Nxa/uD8ekt7Hd2S9eYyx7acZ9bd6GBZNmWCGsvjBQ1h1ZhzUUfWy+CFlyH7hqlHZ/gE6Bh+JXd/BFBpjfclGTGiTx6k6MV+Ovdagehmx9FNwuYBE6yoMudpnKsr+T43tUOu0GweZWNW8ATINIE5g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by DM5PR12MB4679.namprd12.prod.outlook.com (2603:10b6:4:a2::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.14; Wed, 20 Apr 2022 19:23:19 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5186.014; Wed, 20 Apr 2022 19:23:19 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Kevin Tian , Yi Liu Subject: [PATCH v2 2/8] kvm/vfio: Store the struct file in the kvm_vfio_group Date: Wed, 20 Apr 2022 16:23:11 -0300 Message-Id: <2-v2-6a528653a750+1578a-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v2-6a528653a750+1578a-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL1PR13CA0126.namprd13.prod.outlook.com (2603:10b6:208:2bb::11) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 40f24c1d-3103-4707-8128-08da230339a7 X-MS-TrafficTypeDiagnostic: DM5PR12MB4679:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: vKUlQSt3U+3qfVwcLDrW/5ACZwesbxaAGmJLWMrGCzbKZZAxRb3OMxbO+on+sOrOOk2m1bmDQtjPOXHqJUcxZQyJCd9wn5oPJYZfrQjYwty/YM/9bSSLXbSRVnyHpv+BotuaulPAaHbRSaCjupkWhSAtfQyOGEeOBmczN+1YzQ3hCPCMG1LhvlKz0FMSC0AySPTV3PBIISMKtakl0Cgw5TwzFLlTRlSLN+LjF6jFseO4TQAJURjZZamV2bdke9G1y0A3X51JGm/9ug7zCsKDTYX0E5MuIhR60MX6t+AsGyC5EYMwI8OAkm1CzViY+bi/a06Pf8pFXwt9Ne7K8CTqMWb0Jo1I0buSMRNXRfYXHUtCpdSznbtcEcspifbZYtQJDHn0WZlxthQ+8erN+hopblKnqe5hYTgIXAzDZwjsHT/dIo9wV8EG6WfztD4++FHg6gSHxxSMpkfC2WqvtIIvyAQw7w6HzPguWP4ENHRP4TvdRZ2iLs7aq681adw7IkLF6iwjG1P32HhN36a1dcwHfDUqQHi+9EsdswlvWQFVu4l6gksdI/oO6V0xUNGIDY9fKiTyRERZBIxaM799SBvvTRQ3Uk34eFk1ga0ZMcMviyv1V325YLMPBUacCPOFvR2oVLJzj6mzgN1YarxKTEAQovRM01wBqfI0XJoT05IzWeCkPiIWB4DZyFlDy00b/CBc X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(508600001)(86362001)(5660300002)(83380400001)(6486002)(36756003)(4326008)(2616005)(186003)(8676002)(8936002)(2906002)(66556008)(6506007)(66476007)(6512007)(6666004)(26005)(38100700002)(110136005)(54906003)(66946007)(316002)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: mN8tJmay41JaOXvEikzfioRfS9v+UgtPmU+xW15+QQBZwWw6vnqy+t7OcEGoqi+5zt+ixChQNESChRck6bfRaow8wgbl35RnxZRKQJFZ0B2Nw0PAmx9jqFWjJDii3MVdK2VuJStS2a4pwF7eDUb4jCSxOblb42zhjNkdDgfHCuy0c8lm6jPxJCUM6pxZn7ez+BbjD5ZPkeyp7lw6jtUGh4bGv6nomeYewFmRbSwCED/A4YKEaJqPN55b5ApiNQTiNsT6hEMG5UQFCDFWp0TmK0pz6oG1lZPLvQfajEWbpEFGh68syetz3deSumFBZlCJsRzRsg1ylmk0EdrNrk7eWe1ETlhYg2LEnZsuoAlpDRTOF+iy5ZuizE6Vk5hPURoqPt9armq3ieCbrbLdrHQjNgdUrxg5+OMxVPYfsm/RcJ4L5fBiuSDfw1u6aQ8H7woJFyNk7ZtUQBgsYF+3JXZxkqj2q6Md0yGe/2ZbV+oLx6FEwCNnyoQJX3A0KfzpLC7qPLXqWntMaJQWjFZ7PVAk81NUpZPB0xxE+Wq+jhyoQ+FXtGSG7kbZFJg86dHNcXzOOj+NoW85L0adcXnXel6xcz0qdySZ4PxwV8z49TQS7zboQKk17iWGmgSpUnFF+ZIUMqCTikKnMSTF+JZp2addA9JLBXnjQ1bMJr52bGQVdsw9Jot//+nhh0wQpCz8/5Kol9r1OtHj2xUtoU4PvoyUtuFoYx0E1J+lptwbCrI1omXrLsONmNLZJMQ+HzwpDcfsdRupxP7VWFhvsNGMkcfNG/t4qcNshULdeTZn7hV5p3tDccY66DUsf7z/NUtTzvGp3OfsYP2r09m0+QNVwAPoN55Pwle+7YEjrtOQyt1oS7yobR0s4B02gn9BxYGd02vhMe+zN537dJ7lMHzlr7Q8hrRqWxYWmySpnHjEYajImHuQDFdJwJEmz784HjOVa/W5Z0nb7UDGbHd/tNNkHV6LQRvJH3dp38bfBpJ0c0+JpyI1TxmOiBFG8vI9/g2bKdrFsu0aJdHTJNAwqkF5fDRbtM9V6erHGDD31b4BYsBAllGromsqTRWVkG5kBjnIz8PFhBr8KVemjWBPAPQGIMSubvmLua2otMuX1dLHrgWerufdDVKnqC709V187C58SZbSzRiq93ZXCtRbK8ov16bJcKf+6QBMNzA++HVxHtwTI1rntxYkCoptW8ns+9hGFlcFT5ZanVh9LQVPsXoCjQBuUzCMubRLMVbD+HM6Xx36UEIg1lxay1ygH6kIpLAXFtzKGVREJ87jjKfhqwNdHTfcKcMJqD+nmTAFB2DqAdqd1AoxUNl+7i61eLXzjEQscjzb1d1RTKf0rR54AZne4zujMBOZ7clgkvgku6UuHunWwKtJA1Dj/wjNRb3sk/XBhsNdVk5TmMtLdSMurSAIGagB7Wt6X+m3piknPoaPwNU3FPEinL3KgFJvpBFhjqTDarlnTNrD9iVnpz8tNZh0GUj1RDy3rGLQWv9lkGaGquK5dkZFly+fsIhzYxvRnnhLGD/xzr+eRWP12ffgrv/Zjc2QyBcLyEQ2i9/RJ7A5itirM4nFNJ2nXhmOR9zlF0g4gjCqkUx6QQTyVx/TZljd4CUPRGEKHtgXaLWQjL6vfclkqgEYSrvYCc+QR05C+eMVEU2M8teQ08Pd6x1LwQjFbhj2yoLOUXcMYCPISgguiqJf+iEyLBBsvtntTA9bpfsrCYemel7Fg8SulUQxNM5Onl74lQ== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 40f24c1d-3103-4707-8128-08da230339a7 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Apr 2022 19:23:18.6905 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: KPGB95eVjrj+yyrxqYvJLT9lNzWTN1P8L/HVKg1LZ1rYj33Gt/hL02TF68jrGUpK X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB4679 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Following patches will change the APIs to use the struct file as the handle instead of the vfio_group, so hang on to a reference to it with the same duration of as the vfio_group. Reviewed-by: Kevin Tian Reviewed-by: Christoph Hellwig Signed-off-by: Jason Gunthorpe Reviewed-by: Yi Liu --- virt/kvm/vfio.c | 59 ++++++++++++++++++++++++------------------------- 1 file changed, 29 insertions(+), 30 deletions(-) diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index a1167ab7a2246f..07ee54a62b560d 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -23,6 +23,7 @@ struct kvm_vfio_group { struct list_head node; + struct file *file; struct vfio_group *vfio_group; }; @@ -186,23 +187,17 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) struct kvm_vfio *kv = dev->private; struct vfio_group *vfio_group; struct kvm_vfio_group *kvg; - struct fd f; + struct file *filp; int ret; - f = fdget(fd); - if (!f.file) + filp = fget(fd); + if (!filp) return -EBADF; - vfio_group = kvm_vfio_group_get_external_user(f.file); - fdput(f); - - if (IS_ERR(vfio_group)) - return PTR_ERR(vfio_group); - mutex_lock(&kv->lock); list_for_each_entry(kvg, &kv->group_list, node) { - if (kvg->vfio_group == vfio_group) { + if (kvg->file == filp) { ret = -EEXIST; goto err_unlock; } @@ -214,6 +209,13 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) goto err_unlock; } + vfio_group = kvm_vfio_group_get_external_user(filp); + if (IS_ERR(vfio_group)) { + ret = PTR_ERR(vfio_group); + goto err_free; + } + + kvg->file = filp; list_add_tail(&kvg->node, &kv->group_list); kvg->vfio_group = vfio_group; @@ -225,9 +227,11 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) kvm_vfio_update_coherency(dev); return 0; +err_free: + kfree(kvg); err_unlock: mutex_unlock(&kv->lock); - kvm_vfio_group_put_external_user(vfio_group); + fput(filp); return ret; } @@ -258,6 +262,7 @@ static int kvm_vfio_group_del(struct kvm_device *dev, unsigned int fd) #endif kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); kvm_vfio_group_put_external_user(kvg->vfio_group); + fput(kvg->file); kfree(kvg); ret = 0; break; @@ -278,10 +283,8 @@ static int kvm_vfio_group_set_spapr_tce(struct kvm_device *dev, { struct kvm_vfio_spapr_tce param; struct kvm_vfio *kv = dev->private; - struct vfio_group *vfio_group; struct kvm_vfio_group *kvg; struct fd f; - struct iommu_group *grp; int ret; if (copy_from_user(¶m, arg, sizeof(struct kvm_vfio_spapr_tce))) @@ -291,36 +294,31 @@ static int kvm_vfio_group_set_spapr_tce(struct kvm_device *dev, if (!f.file) return -EBADF; - vfio_group = kvm_vfio_group_get_external_user(f.file); - fdput(f); - - if (IS_ERR(vfio_group)) - return PTR_ERR(vfio_group); - - grp = kvm_vfio_group_get_iommu_group(vfio_group); - if (WARN_ON_ONCE(!grp)) { - ret = -EIO; - goto err_put_external; - } - ret = -ENOENT; mutex_lock(&kv->lock); list_for_each_entry(kvg, &kv->group_list, node) { - if (kvg->vfio_group != vfio_group) + struct iommu_group *grp; + + if (kvg->file != f.file) continue; + grp = kvm_vfio_group_get_iommu_group(kvg->vfio_group); + if (WARN_ON_ONCE(!grp)) { + ret = -EIO; + goto err_fdput; + } + ret = kvm_spapr_tce_attach_iommu_group(dev->kvm, param.tablefd, grp); + iommu_group_put(grp); break; } mutex_unlock(&kv->lock); - - iommu_group_put(grp); -err_put_external: - kvm_vfio_group_put_external_user(vfio_group); +err_fdput: + fdput(f); return ret; } #endif @@ -392,6 +390,7 @@ static void kvm_vfio_destroy(struct kvm_device *dev) #endif kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); kvm_vfio_group_put_external_user(kvg->vfio_group); + fput(kvg->file); list_del(&kvg->node); kfree(kvg); kvm_arch_end_assignment(dev->kvm); From patchwork Wed Apr 20 19:23:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12820733 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1A60C433EF for ; Wed, 20 Apr 2022 19:23:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1381788AbiDTT0T (ORCPT ); Wed, 20 Apr 2022 15:26:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46522 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1381760AbiDTT0N (ORCPT ); Wed, 20 Apr 2022 15:26:13 -0400 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2069.outbound.protection.outlook.com [40.107.223.69]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 23CB045AED for ; Wed, 20 Apr 2022 12:23:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OSKimlfPXeCWCTtYmx/mFTTda16kkMTSjh4gsd8TXokF1ZlSQj+GPlp4Uc7ABif64dd74wIKqAauzBrUIPxuj5nmBhW8QzAxVyjOrYBxXbRnBNr8re72S4tJGERoY9pP0b4jSPenYIn5IVAZkUfC7GLtIE5WzNDIN9IlQBn5crh6ccJINZ6l9rQCrt6zJ/1itokK3gl5qfSaFXKsoj5jO3v5dSl9BxHTH2RdYxl9SHRqB3deL3z6tJuKGGR60GkPqvuFKoq/rWI71yLcDQGrW/+WmsE5YrGPJ/8Ak6DBV+KqPyT3qz4bCicYj0ainuqdu3m6aNY9PqMQBX20t4QKow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Sw0tlOPVjKyCw7QL852cDybyT7Whvv1AnaS7edjuJng=; b=UbF0ZGQ27Bv48snuXqA8Fxq1CSjBA3hNhWCZTFW42yIpN0bWobnvdifiqeSIWmWGbBhRAKxWslatycMiLMAjJ9z5Ih9sLSpbTwMEoPoEzPS21sIdpnJWgF1mDGASJmlLCp/Lsig89uYTsDGKSdTsN/s2pf8axBLRbJ//q5h9cJ+HDJwKvpobpdAhUKpnYNF+SPHfc5b/xjh0pUFKGfRKAEJrMV+HCjm8ElJi+cuYCm/6pXw0R5ev0o8vamBq7sSPyN4TS6XGZdyZllKMUYqGaoLARGh+TY8lyMYRIxEV3ymw8775Oz1jeG3TucmEA7FQQkAsAI1xMBSHASY0RrVVug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Sw0tlOPVjKyCw7QL852cDybyT7Whvv1AnaS7edjuJng=; b=lnu99vC31d9sLOMzDTTsSzeLdZqiQlDQGcdtJUVPbyEtse0IOe4SfrKNv0JURXuZIuEljTqvpfa0Pc4njdgIys9lDQ0R+aOz4fKFXPqUp9OMagkLPrTD/qtKGQs9g3mb23yMacYgVEtPjqtAzR/c75FLcjXFb80M1hsWUHWLLr+SplgTAB7eill6DKEyZM3kzbmc39JG6+BnCxd76QNDXMTkYxdFttfPrrsnFFAp5W2sUV2EKK68zMFUciuBbTmaYcUd8h5dU9wkwJqTwHNwSeY4kyTt4XQL1JvUozw6RmcmsOVEzy+8MExhbI6ZXF10hm8FFuHBtUEM/yHPPgiCQA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BY5PR12MB4131.namprd12.prod.outlook.com (2603:10b6:a03:212::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.13; Wed, 20 Apr 2022 19:23:22 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5186.014; Wed, 20 Apr 2022 19:23:22 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Kevin Tian , Yi Liu Subject: [PATCH v2 3/8] vfio: Change vfio_external_user_iommu_id() to vfio_file_iommu_group() Date: Wed, 20 Apr 2022 16:23:12 -0300 Message-Id: <3-v2-6a528653a750+1578a-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v2-6a528653a750+1578a-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL1PR13CA0147.namprd13.prod.outlook.com (2603:10b6:208:2bb::32) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: aede5d1a-60ac-44f7-a96c-08da23033a25 X-MS-TrafficTypeDiagnostic: BY5PR12MB4131:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wUhn8lrwQOAdLCOEztoXH1uh2dQxNv/rMaKC5DJI/TJuybz9bghmIEAyI7uZ6qDEQu90HgaiWF4t5BNm2yf/oYk7rgTn+nZ7ciKxEfjI+mQvyoPlakWpMy/96BeNddtkDfedHUjdcWNsSYbUM0ao0gVD8rtqDSh/YoRGXWBAQF3gYhsYJmx5NNsGWfu403S90Rvq/Rs8/Db4H7gbobqvQX8fNiLf8UmN6KWScsyStcb16xKUkisbTPU07YzXSx9Edf6qxIwQN1woZr2baX2fTd+6+iGVW4KpQf9ZD/7A3xZBZJwP9GJCzqCus/VU6VP/Zp4j4GFd1IUSEOMzmVZ9uESN8JxybbKPdq3pkdw2zc7E9wBAe7XsyTjxn+0Janh6clc7UVPsGW74T16IrZdtuX5q7JrjgjcCFJQ4mEBI/HOqHxbkj5cZhYj+KrCHqfwniEOZz77gPVLKU/jsBztLYEIclZm1+1a/+0hug1O4dBBxu/7EEmrFFjA4FfEiLJLNmHMT4y+a3wAIo3xudl3+F/LcELE6an9oEAfqAMyMRFHkV41584w01hkNHhn3PAd98qsya+ONsxTkHf8bU/dSfMGf2BP9Z9uXhaOEuor3UMgbnPqFC8P9XTr18TAQtjDBCX1HZj5GSTLk06PpVLecDmjZpn/LhcDZfgK9qKRuWNSFt+NUZzrL+v2FDXG2mRKc X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(26005)(38100700002)(6512007)(36756003)(6506007)(66946007)(2616005)(186003)(6666004)(66476007)(8676002)(66556008)(8936002)(316002)(4326008)(5660300002)(86362001)(83380400001)(110136005)(54906003)(508600001)(6486002)(2906002)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: +Gmf46IlB99YR6WyPJgP+1XqPRsrWqYNREouhEleaZ8wUZsJFEKKbPoEgOkueRJ7TT+VP0iWkuHqr00LbNJAGfzEN6j/Aw/uW1IHNKxQ8nAo6/ByaZTlMPe67ymG4Ig/H4IhRHUgi9UH74Lm8bbKNCFahnVRj9n25X72A3lDKx+O/d1dF0OS63728DDXLWSUiCKcblBGEIgOPZ2fjgbHKCJsmc1UhvRHZBePcS0ku+Xbn13tQf3lFG41/QO1o8AjMuZWzvZnGCkRl1CXY+a/mm7/cG/VR2q4OXvHk0GEtNQOJ0luwGZaJ0L8NDACt6GmhJEKmohBI6lIH3BSwzysOXIuFwPz0KZg/nHba8J9gPwZTN999L4Fy5QPc07F2hu8g+WMbqOw0uU9Y4RvyuoY/LFsrrmU54kuDFYwA5bJ2AEfLoYvhO3AaHLkdeVL//0sxCFCH1EwX4DCXOqfVBrD0RNc3a6XJOo84I7y/aGHRenqhepytR+aYPFz83OGuHg0deIQbBOiCEulJx06NIElgHp0vPGzJ0o5r8S0HitlvGAHpgXvABxxlDXxK5umRk1q6apRMBr4Igbf5f/h1cVvYKZ/p7ILs5BG55h//UJ7/VoLhqJ0hC8jGCNp2kYeYGlT0S4Ah0pBqMoAIj1MESRkSvgN7xJvG+l7QOQJov8DaLAQGFycrUYxO/DmJNEPGodmlhoZbjq/hpSoDnPUXCkBy0fyD4qTgd/PnUSTdTsKAf6eQxRI2bY9/Fc28MSkKVvF3qeVcNDvTUGlk2xMMovJYlaKVa4WCs8ZOzULzMjAruf8WgYb+2tqtx9bONX2ld3zfmf5I6auTplZ3kkZpIlRBOn0ldnkB4HxekSBqtwzk8gr9/Lf60rBcB9KqdDpYsrfHgbpr9sKe+o45KlPv8Qfuh7SQ3PGtS6aGeK02+QwweCcXp4vvU/bsRaW9kZdCk2gIetMPL0yCmsb/DLCRq1hNIon9IPsLDrfzHNuioQwqOAkggzsKbsUlEGr3lVANgRf8Ayr2JDCzVT7vnOpJZ7r2tEQ+qrwzSjgYKwFTSajr+bsATqcOnaLRN86ZcSub65woE6eKXcH/fIKQR1A66w3uKWl9T5OJDmm60Qs6BDxIfctdf33NIU9+if20SIfQnbr+J6oL/ieLYVtqRYNeiI2Jr2+30QuKMVH/ntjKjA+SP1D5PKmmeJGGnXoyZ9FkhTxgL1Cfhu93785h/QrjIx993dVgTkoGK4n5EDzR/NNZOGrx8RjaPsdI5eayB6hULlll+Ut8JPsnVfNnduQ8ODu3oX0XYPArxJJSqDxD2Ss1m399RB70/ElKCTDY+nIfuK/IkXBI6c6+x884kfxze1zAoc7ooCNWnQl5HwDZfOEN3foYL1zaTluykhr2EyahV63zbnsgikQK6Q+RcCEzv0gf8ht6mco5DlghNxO5HSuTMnnPO10kOyG0XfR4P2WJ6EAeEGvzOVqVy5u3BmRl/WMY7er6oClJ4CdGsDWfK2LhqjTVIm1hSV23Kdv3VbIB5eBpHkDG/PKkYnjYcTwMCNNYnoJBVaC/PZfhrwXXuyqEDAA0boKnCusG40mK2XNjuau/PXKquumhQMM3uHnpkCqEdMCG+gxbV54nujPuoSuxp5e+CeRvGLtbfJBwbAl3xd3oj7BuuQfPF/NK/g5/EP5po2Srl2sFFzklHlqukDIfhE6jRg0+4O8LTmUeJh+yUS746qZOX9S2toUl7gXrpcF1A== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: aede5d1a-60ac-44f7-a96c-08da23033a25 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Apr 2022 19:23:19.5173 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: /hQcjjBgL/rPB+GVjb2p3rmvUTwM+eaSZRqwLf6K/bFGNwfglGTpzZvsM/JoY9Qt X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4131 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The only user wants to get a pointer to the struct iommu_group associated with the VFIO group file being used. Instead of returning the group ID then searching sysfs for that string just directly return the iommu_group pointer already held by the vfio_group struct. It already has a safe lifetime due to the struct file kref, the vfio_group and thus the iommu_group cannot be destroyed while the group file is open. Signed-off-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Christoph Hellwig Reviewed-by: Yi Liu --- drivers/vfio/vfio.c | 21 ++++++++++++++------- include/linux/vfio.h | 2 +- virt/kvm/vfio.c | 37 ++++++++++++------------------------- 3 files changed, 27 insertions(+), 33 deletions(-) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index a4555014bd1e72..3444d36714e933 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -1919,10 +1919,7 @@ static const struct file_operations vfio_device_fops = { * increments the container user counter to prevent * the VFIO group from disposal before KVM exits. * - * 3. The external user calls vfio_external_user_iommu_id() - * to know an IOMMU ID. - * - * 4. When the external KVM finishes, it calls + * 3. When the external KVM finishes, it calls * vfio_group_put_external_user() to release the VFIO group. * This call decrements the container user counter. */ @@ -2001,11 +1998,21 @@ bool vfio_external_group_match_file(struct vfio_group *test_group, } EXPORT_SYMBOL_GPL(vfio_external_group_match_file); -int vfio_external_user_iommu_id(struct vfio_group *group) +/** + * vfio_file_iommu_group - Return the struct iommu_group for the vfio group file + * @file: VFIO group file + * + * The returned iommu_group is valid as long as a ref is held on the file. + */ +struct iommu_group *vfio_file_iommu_group(struct file *file) { - return iommu_group_id(group->iommu_group); + struct vfio_group *group = file->private_data; + + if (file->f_op != &vfio_group_fops) + return NULL; + return group->iommu_group; } -EXPORT_SYMBOL_GPL(vfio_external_user_iommu_id); +EXPORT_SYMBOL_GPL(vfio_file_iommu_group); long vfio_external_check_extension(struct vfio_group *group, unsigned long arg) { diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 66dda06ec42d1b..8b53fd9920d24a 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -144,7 +144,7 @@ extern struct vfio_group *vfio_group_get_external_user_from_dev(struct device *dev); extern bool vfio_external_group_match_file(struct vfio_group *group, struct file *filep); -extern int vfio_external_user_iommu_id(struct vfio_group *group); +extern struct iommu_group *vfio_file_iommu_group(struct file *file); extern long vfio_external_check_extension(struct vfio_group *group, unsigned long arg); diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index 07ee54a62b560d..1655d3aebd16b4 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -108,43 +108,31 @@ static bool kvm_vfio_group_is_coherent(struct vfio_group *vfio_group) } #ifdef CONFIG_SPAPR_TCE_IOMMU -static int kvm_vfio_external_user_iommu_id(struct vfio_group *vfio_group) +static struct iommu_group *kvm_vfio_file_iommu_group(struct file *file) { - int (*fn)(struct vfio_group *); - int ret = -EINVAL; + struct iommu_group *(*fn)(struct file *file); + struct iommu_group *ret; - fn = symbol_get(vfio_external_user_iommu_id); + fn = symbol_get(vfio_file_iommu_group); if (!fn) - return ret; + return NULL; - ret = fn(vfio_group); + ret = fn(file); - symbol_put(vfio_external_user_iommu_id); + symbol_put(vfio_file_iommu_group); return ret; } -static struct iommu_group *kvm_vfio_group_get_iommu_group( - struct vfio_group *group) -{ - int group_id = kvm_vfio_external_user_iommu_id(group); - - if (group_id < 0) - return NULL; - - return iommu_group_get_by_id(group_id); -} - static void kvm_spapr_tce_release_vfio_group(struct kvm *kvm, - struct vfio_group *vfio_group) + struct kvm_vfio_group *kvg) { - struct iommu_group *grp = kvm_vfio_group_get_iommu_group(vfio_group); + struct iommu_group *grp = kvm_vfio_file_iommu_group(kvg->file); if (WARN_ON_ONCE(!grp)) return; kvm_spapr_tce_release_iommu_group(kvm, grp); - iommu_group_put(grp); } #endif @@ -258,7 +246,7 @@ static int kvm_vfio_group_del(struct kvm_device *dev, unsigned int fd) list_del(&kvg->node); kvm_arch_end_assignment(dev->kvm); #ifdef CONFIG_SPAPR_TCE_IOMMU - kvm_spapr_tce_release_vfio_group(dev->kvm, kvg->vfio_group); + kvm_spapr_tce_release_vfio_group(dev->kvm, kvg); #endif kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); kvm_vfio_group_put_external_user(kvg->vfio_group); @@ -304,7 +292,7 @@ static int kvm_vfio_group_set_spapr_tce(struct kvm_device *dev, if (kvg->file != f.file) continue; - grp = kvm_vfio_group_get_iommu_group(kvg->vfio_group); + grp = kvm_vfio_file_iommu_group(kvg->file); if (WARN_ON_ONCE(!grp)) { ret = -EIO; goto err_fdput; @@ -312,7 +300,6 @@ static int kvm_vfio_group_set_spapr_tce(struct kvm_device *dev, ret = kvm_spapr_tce_attach_iommu_group(dev->kvm, param.tablefd, grp); - iommu_group_put(grp); break; } @@ -386,7 +373,7 @@ static void kvm_vfio_destroy(struct kvm_device *dev) list_for_each_entry_safe(kvg, tmp, &kv->group_list, node) { #ifdef CONFIG_SPAPR_TCE_IOMMU - kvm_spapr_tce_release_vfio_group(dev->kvm, kvg->vfio_group); + kvm_spapr_tce_release_vfio_group(dev->kvm, kvg); #endif kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); kvm_vfio_group_put_external_user(kvg->vfio_group); From patchwork Wed Apr 20 19:23:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12820735 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07D37C433EF for ; Wed, 20 Apr 2022 19:23:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1381794AbiDTT0W (ORCPT ); Wed, 20 Apr 2022 15:26:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46730 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1381783AbiDTT0R (ORCPT ); Wed, 20 Apr 2022 15:26:17 -0400 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2069.outbound.protection.outlook.com [40.107.223.69]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F419A4664C for ; Wed, 20 Apr 2022 12:23:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MePK3S6Be0gs3LsQeS8zlH1X/xwlpfwoUImifiRqCc5JDnuOTcZ26j/59LKN6NWciv1YCSgsbN7S+7qTKE4XXecPpV1UjgWMXUDcncnCoVK+EST6DbtPQsc6naFp29aeODWbdqfVmRvEPJ59/y3xti8/11SoRvaskvKGlAg3CLKl7Cszd+m8iisKPwcFtDi801dx2bunGGztmcW2zaZvBtjIH9lTrYf6ka514+4mFZxIVWdEyR0MFjCPH+8Ynwbx4T12o21GokS02CTSvjPoEieM0n7ES5xhmtNjd9kPLUgn1erLtL+7jUNlItzhezEUC2nKxLl6tnqs2dj6+KYBvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ah5ovzd4PiV/QUaYlz0fMzMtAV6K5NiUJprsfm/iGpY=; b=cYimPLbRjRpctNpd1V1EQRaJAme5q5aAbamvqipUKItVXg6xSxglXNDcjrePx+J/nAeP5/1ItPHu7k+o9BWQbxXF96WLIoIpn+0dP9ZSp6dLr2vlVxga6192x/dh1zall+sdpTfvRY1BKUdvp3YaxLk1GU0gG3avGhIxymms4eCMqOP1IoDhr2FDLHI5Nhu9nJF0IyeefRBLsXj/9JocfHkkeqwhUoSrvDUgxLt7Vn1ViHj8b8sVd/6rVuoS1PAHnOalVaYRpFmkjvRsmzEg5gD+cOePwu4Lm1Ih2TN9tBNVnXEOdRGHS32VlhPCBCzN4c3jiXYFAEuQl+RhnZMErg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ah5ovzd4PiV/QUaYlz0fMzMtAV6K5NiUJprsfm/iGpY=; b=uB0jTSztOdjLdMkxQ7M/+VmxJvg9u8Ok9S7M0nhY/CsNIf5yJBDZ/O1TH8wui705tNESNaQGWs45zONo8Bg/oufGBqVN0a8tVoWmRmHpYS+qeqgaZaHPIMddRNUTjYiMPOEV5JlqDh0/BuBPyss7eQfjRVkd/rHWvA82uRlyFk8b5QwD7jtwmy2+musrQwkLmH4+gUxqIoVWMXiJ+u5FyjuJ/lyU7i1l/C6eu+GwEA/obGh+JXGyEYA/M7wDGY2XryHIeFdzgqkIHYu9/KpKVnxHbH/VrwBvO08ItYhACjzw8Tvm4VGmbD4vd96ZjLkVWqBANc0JAp77LjJQ3B8/qA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BY5PR12MB4131.namprd12.prod.outlook.com (2603:10b6:a03:212::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.13; Wed, 20 Apr 2022 19:23:24 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5186.014; Wed, 20 Apr 2022 19:23:24 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Kevin Tian , Yi Liu Subject: [PATCH v2 4/8] vfio: Remove vfio_external_group_match_file() Date: Wed, 20 Apr 2022 16:23:13 -0300 Message-Id: <4-v2-6a528653a750+1578a-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v2-6a528653a750+1578a-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BLAPR03CA0098.namprd03.prod.outlook.com (2603:10b6:208:32a::13) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1d529cea-fbfa-48f9-d023-08da23033a2c X-MS-TrafficTypeDiagnostic: BY5PR12MB4131:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Jv4QrZGwtRxsPABM5fWdG5rtdtoxAkdVL8M09IvO35WWb7BNIU9pX8XDd3iDnOGQ7xXch4WPCCbf2g9Yl3dXUGVVWnNNRXV5Ek6SITyPBjXHYD1JuzXDHazFqmi5gjgP9FS+HmcfAiQJ+Uc1uocny2iQjuspaDk+kah2Zdg2Ifad4PaejRKCiF4scid0QrGzbGh0u8GPzDR7g5xhJbHscrg4cJAYDruwakZDhZQuTqCsdoMlyrvhQTChHf36485rCErsh2fQa+RZFQg7YIXJD/ZlZAREPASZ99MPoZq+1W3+Yd5Jt4kfL8hnenyQaZdIJWNxycQ5lMEsXe3OPJ53dSoHbfolHNWMKkOyr37nIU5Ow2BBoOxIBOVn1eHXUxwn42wfNmilwKc31bHMmPFQialkarHyWKXRcNwTZrolfeuYaFXDscpL0UTTrtPk1hZpmUoKBsrrxuWOLWJv0yEIQ+gIJmDFY+kS1BYbgHeFkXkSdJli9yeqLVO9MU4F8MMjvNoO5OsFjCTxMwtlWpId8wlNdjyQTMBnBn8IYnGHrvlORRwc0cxeXYWWrp9+I01wMFhm3qKe6DP8sl0jseSrq6gAeLB4Zf3XAIp4mTLDQLnThniPzA5HaKrkxQJjwYReAWEMJbEchnyEpqTUFWYnkVYmGG4AvZo0hQZyHxHUpj8PwRrjqehiuQkRVGXN/VVd X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(26005)(38100700002)(6512007)(36756003)(6506007)(66946007)(2616005)(186003)(6666004)(66476007)(8676002)(66556008)(8936002)(316002)(4326008)(5660300002)(86362001)(83380400001)(110136005)(54906003)(508600001)(6486002)(2906002)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: HVDW3kJ9YvfEf5Tsq6M8/BtGhJ7s1ADDCnpr1q1jR4Oopi19KK50LZyFSbN6gLIcdJh0Hglb/Kfcb7YSWZL7FZOeck1ZCwo9USX5hTPGlRSXjRdMLABj8LOwbZLqSEkT92NZEQsr1EaEXaCWSsURjscpNi3l2e2osSd0Y98nuomjgoBFjxUykR/DvZEpg/u2ufAh7tszv+/+CKHPa5LOnwxPN5r2ExTZRFuiWqm9TvQNVpOjZvlTvpPOhLKVa63QmP/Bca052dizioKXIyXQOlExiO4QDM4sIrJnBhtzlc2XyNqem4vcutohegHbRMArZ05ztK23dCEFKKGebe8EDw96qRKuW1fMVASeq5ONlcwrt8a3G34QdTJ7yGYvg89TnthxdnBXU/h2jKm28xgmQCluyLFs53wfIMhWZzR5jOsyC2Otiqf1QKQtejSCwrF3Yx2bQdef96d+eM3xHrZdP9RX9v9auBHuHUagitqgtTdNiJ+o4Rk5AdjGJZ1/s3EO6WrJHwlUWFee7Ny7B8fLHQtQeSzuaKj8pXeKrQbPfC5kQVt9EsJjIWqx9gJ2V/P0kGWgs7qCEd/CEXVdVkLNNxQakRuzgZ+VTdEx9EyQUGT7+JIhBjklefMSDr2F5/GW21ezijuOdRAOmuHwIfWQfFDzeJtDjqcO7e7VCwAKmprPI4roJ70y13fusdKKJ8JyOhvVv+OVcyAJA9ftFSkvtmmKYubtL1oBXvGr9I46Tjo1jd58G2/d/8LPtVPRlNTGR12spGKLMScKbSlRePX7hD9yoe8POZwUDrYG/90j47naN/LbAmliF0r3vMP/aia6V/1sy8XlbUf3a+jbQ2Zr2cvtl2Eb2CNx4+rTkdMiQGlPoWpIeIzGz/cyPxyf3wsTFD4fQnVuhMJ8fVeqAI2rJySQq6UxXTCU5R77SH37VpU12UScfBoLzWQHrD7b7SiZXAv4C3eV96QKPZTe3Gg8xosgM3Qxq2SpLq7xqxRYDg+kokV6dNFBE4HS7qFhEQ+mP79MZ6jTPqxRxoGV0lzgugcXOos/J00eimwWYMMfsYbICqY5MHYPJgV+83T1IfGb1so+V4YwHdHECOAAbhKheS9/f4n/LsPRTbyNKMuAcTgWFnJghbnYlY4lO0anoaUfqE4JAywdzxvTVviYETyA+SGG05MeShkfKjDj0a3rmDmPeJfSWlV97nSrSQiyYiTpkgI3NpTtPPISA8WeyLfl3c4f+EOrquB6sT2a9WbWI0t6YyJQVZRQpOfA/fyjaDEy8OUI1QCxi0BP2eRe1qoQ74OkcPl+bBJZmGZlAodTLf4Inkv0QgkaQOiOOHJGdZaQVi7u5x2pNUNzK8UAL6D0e18CY+376AwTRz0INKAaDX4CzcF5xxoriy79j7wDa5sBpXHGvorFMqjmG+980W8l+g0hLW7MybwbsVvjMVLqY6sKtddtiYOg+0ry8n/E0r8n7JHEUNOI/BnVcRECSgGBrJeKpHcRsPDryz56qo7F5Xmav1/9n4XGycu5U9MMZ6RisYcMN9Gi7DEoMRVDnn6km/T2mmRZKxA1zMfRBYnXyytr51Bh7s5sVNSajBqIz/Ai4YwFMZUn4Z6Ex86TsQzWaZHzijqbi16WBJSDvgC8lh7v4e+IJvAwQ1ERtKrFQcsknVIyk/Wfs/7m4fsm7QkZEdjOQFkfKUANu98+g8bHef+Z+qBLI3rK9DDk6BvHdPtv+riNUV55NOOIDXFKCv6gbw== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1d529cea-fbfa-48f9-d023-08da23033a2c X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Apr 2022 19:23:19.6266 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: S3lYffw9rZmoEBq9uUm1vXuQ17wsWoJX3wyC6fjP07eD9M0ymuuZycr+L2SwL0xN X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4131 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org vfio_group_fops_open() ensures there is only ever one struct file open for any struct vfio_group at any time: /* Do we need multiple instances of the group open? Seems not. */ opened = atomic_cmpxchg(&group->opened, 0, 1); if (opened) { vfio_group_put(group); return -EBUSY; Therefor the struct file * can be used directly to search the list of VFIO groups that KVM keeps instead of using the vfio_external_group_match_file() callback to try to figure out if the passed in FD matches the list or not. Delete vfio_external_group_match_file(). Reviewed-by: Kevin Tian Reviewed-by: Christoph Hellwig Signed-off-by: Jason Gunthorpe Reviewed-by: Christoph Hellwig Reviewed-by: Yi Liu --- drivers/vfio/vfio.c | 9 --------- include/linux/vfio.h | 2 -- virt/kvm/vfio.c | 19 +------------------ 3 files changed, 1 insertion(+), 29 deletions(-) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 3444d36714e933..c9122c84583aa1 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -1989,15 +1989,6 @@ void vfio_group_put_external_user(struct vfio_group *group) } EXPORT_SYMBOL_GPL(vfio_group_put_external_user); -bool vfio_external_group_match_file(struct vfio_group *test_group, - struct file *filep) -{ - struct vfio_group *group = filep->private_data; - - return (filep->f_op == &vfio_group_fops) && (group == test_group); -} -EXPORT_SYMBOL_GPL(vfio_external_group_match_file); - /** * vfio_file_iommu_group - Return the struct iommu_group for the vfio group file * @file: VFIO group file diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 8b53fd9920d24a..132cf3e7cda8db 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -142,8 +142,6 @@ extern struct vfio_group *vfio_group_get_external_user(struct file *filep); extern void vfio_group_put_external_user(struct vfio_group *group); extern struct vfio_group *vfio_group_get_external_user_from_dev(struct device *dev); -extern bool vfio_external_group_match_file(struct vfio_group *group, - struct file *filep); extern struct iommu_group *vfio_file_iommu_group(struct file *file); extern long vfio_external_check_extension(struct vfio_group *group, unsigned long arg); diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index 1655d3aebd16b4..50193ae270faca 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -49,22 +49,6 @@ static struct vfio_group *kvm_vfio_group_get_external_user(struct file *filep) return vfio_group; } -static bool kvm_vfio_external_group_match_file(struct vfio_group *group, - struct file *filep) -{ - bool ret, (*fn)(struct vfio_group *, struct file *); - - fn = symbol_get(vfio_external_group_match_file); - if (!fn) - return false; - - ret = fn(group, filep); - - symbol_put(vfio_external_group_match_file); - - return ret; -} - static void kvm_vfio_group_put_external_user(struct vfio_group *vfio_group) { void (*fn)(struct vfio_group *); @@ -239,8 +223,7 @@ static int kvm_vfio_group_del(struct kvm_device *dev, unsigned int fd) mutex_lock(&kv->lock); list_for_each_entry(kvg, &kv->group_list, node) { - if (!kvm_vfio_external_group_match_file(kvg->vfio_group, - f.file)) + if (kvg->file != f.file) continue; list_del(&kvg->node); From patchwork Wed Apr 20 19:23:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12820729 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9819EC433F5 for ; Wed, 20 Apr 2022 19:23:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1381775AbiDTT0L (ORCPT ); Wed, 20 Apr 2022 15:26:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1381771AbiDTT0J (ORCPT ); Wed, 20 Apr 2022 15:26:09 -0400 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2041.outbound.protection.outlook.com [40.107.243.41]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BFDFA13FBB for ; Wed, 20 Apr 2022 12:23:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dq32ijUjAubBAjgugAWm6XsvrQ5hRzy3XuQz2IKyh1a3X9pUwf348wtv2fgrd43MNo9olPJCapmEXEYtedynIt2IQQkqf7bqgWeLZ5GaDO6DeRx/xiIX9oqMby4olSrik2Q5mX22rr9vzVM5+WhFR13Zs39dWD8u+/qd7TqWmJfSXb2l4gCaKP+hTsNR3tuNaXAzY4X8q1QKbCUmOQ7cBynZpxOWo8mtMQl0Rj7MGdHrzT0bKxZOIPhUv8u825v8jqJao2ufO32XQ8UrLORkBENv2QXHIZWMQJNx4tEfCdbKVBRnXGJfQ26+XnO8xZKnU8ke+FTdH4i9Xyof3XaagQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pJkuj6aBfZTAR2SqokRd7NKbrY4cT6dx0hVCCImfhqw=; b=iUd682Xz2Ha3AWyA0WhYkrP04nARW7trZowVXsUphFLafbhLR6cTaY/O81G7A6Gf+EZkHwPXaOcodwFFlYQsUvlkQ6yVwETn7V0uNNZ6mSxPxOvlKgIlrfam8nj5NQirLkYQwRLcFTseVemaxsOyO5PRh/rUZOPfd9bpTIin+6/B51BVCovO1qKquUxhPXBTA8JHGBM4ey0bsX8QId7PVm1Jf9MUAXUq4ZqZ71Nz0GgjjPAJo0fHxRomFu1az10CQ2TZ9iziVKEYWzDqReiN6Oiw7DuqRWPEUtLamNxcIsJhV9k5xSuQcKBzZxdt5O31s+ZCPsEDC/THZVY32D1khw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=pJkuj6aBfZTAR2SqokRd7NKbrY4cT6dx0hVCCImfhqw=; b=X3AusgRgmeDCJNjvW/y9sGDa6uSMuMuzo/szgXjm6aH2ed/eyU835QkxyDqnAT92AlL4H1yks6iVqAyPYffgGUJzPmHVR8hLTRIVfC1ESeXMG8fwxhTejeg7BCt0jbduHt8MTaDZw4jhDpirSvKiF5ner6eDfI0gBLfwbgXxaQb4Yq/IcHHv2GlXlWGg1jE+KA3X4JqK8e9x4fB41FIYPId0iYR5mnTIyjfCAKWBWlALAX1K0hhaJtMQOnxMINYqRdpyR9ypkq3aHw+wEgu9b1nldn0UAFI09DjJhELpGqG/jVXPZ6zCcjd7z3g+XVrcKwgCuqGAcNbSDznnjeXjDQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by CH2PR12MB4325.namprd12.prod.outlook.com (2603:10b6:610:a9::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5164.20; Wed, 20 Apr 2022 19:23:20 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5186.014; Wed, 20 Apr 2022 19:23:20 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Kevin Tian , Yi Liu Subject: [PATCH v2 5/8] vfio: Change vfio_external_check_extension() to vfio_file_enforced_coherent() Date: Wed, 20 Apr 2022 16:23:14 -0300 Message-Id: <5-v2-6a528653a750+1578a-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v2-6a528653a750+1578a-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BLAPR03CA0093.namprd03.prod.outlook.com (2603:10b6:208:32a::8) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0cf1a8c5-9ad8-442f-2ddf-08da230339d7 X-MS-TrafficTypeDiagnostic: CH2PR12MB4325:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: w9yUBdf5o9NlE68b+zWTxVtMbfmfqlnnoiksAyrUe9S4uf2dyiMzUAYaVIPVjOsz+xyHkWN7VBGAblqRHbzkjeVBx2YGIjbuaXuLROUr9DvMNQJoDvEXkjgQTcierrW3Tgf0VbHm414YR77zN+myhtkpU/E++DFxmnHb3fFeOik3K71cbvlUNLag18TOQH9jJMYvutJ6m5A1+7+ZWqNKHLTOr8IvoOejlcNOAa3la+FmWIoV8FSpqSNV7reydSCfaeiPS7ieSH7esMh9jhRpglQ/ZjbXkY+apo38W/GMO0HzXuRp8gY517Y6HccCyIbfLD0er1zeVRC/rydelDkFFMUJZUGmxn3G40IQmoQY7QrW0a5EoiTlBaJNGgbRq+M28YnYzgCesn8gJMuIVfUAZEW4dYTpftoIcjyvtV+LHDNwjnJMuO6b3e5S+xkQe55jDHfDSCye//w4CddYEIQaD5d3sZ2abNeXBxxDYMPOujXrHIXTmcnom8PekmpzarzFPX88zi8OE6VV4sh91rKjOypid56wErlMLIlw5j/BndLOkU6mAhQnAyi751UaFPYGw6j0HaW47fvguszcQywHRJAjPFmdYXDBrnGlnHfkKxA18TJaqM1yj5r5MAxfSXTDLtWfPXKeld8qE+pRTU8LwUnlNbYBylvJnZyDN0Q4F/xRs5SepvhBoHArezAE7AjK X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(316002)(110136005)(6666004)(6512007)(26005)(6506007)(2616005)(66476007)(66556008)(8676002)(4326008)(54906003)(83380400001)(2906002)(508600001)(5660300002)(86362001)(6486002)(36756003)(38100700002)(186003)(8936002)(66946007)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: bryKtkfguhkhi7su8rPDgq0rkX1V5+EieloM4/S1rrHgEahaOv9uc5pfzZHinws2PeFmV3FfS28P4F7sJHpTRRfeih8W2Nnvke50eDN6uCM87YbzWtfTfVYGlrshtQuzik2jZKfHQ7wvdwCXfcYnhkyCm3l5P3h3t0LE88CSKTd9UCB7OgRxuLE/XaTkGVRau6dm4i97XVGKY/B1F3VIZkqjkbXn+60f1f4kd5qnf/oyr39C5IgJKFqH5HXg3HCK+o3lpW8ZFiTH1kzE/dyWkCtJgxl+XUX1xC5vB9I+XHH4a0kFWDbwQTM8y6e5mfPRbvzda4vQt7pIdZPhzs/kgxuzkFBavvDnQQnMCNxhkUi3EzMdcj0+Oq8IiAWQ9AQHLXNTW573pRK90Ze+6o5sDf9WoythTNb/7tlRpb5o4E0Q/NxHS9FsN5Begy5oTq77ckVrsElhf+hsJi0jZ9dlOwMLzvjqo3Y4oSUa/4LAYgRMIJqIlcAfivc3hJTd3xHbPWACNVrPT0S42dQpSbPHd3QhZqXy6FIz0c+26IJyG99E9avU2GG5L2c1NdRKl4Uw2SOEYAdTNsE+1wEV0hQmhIoG1Rld5rgxyks2B98LsrGWYjdR8rCuH44oiOd4m3uMA6uTkeK/ubIRzlKAAvup9UUAV6vTkpN7E420fJXC6+lzqFJC/srIq1cd0Dv/EVwmXe92WjfqqzaP1Q5DDWFhZcV2StWTFR8yI7N3ELtiiyp3L0D1tcp54nmNV9yklraPop/VDpRInblr7+cc/PblIXkbi2asGlDws7JUTOhtdo3sZ4w2Uo4PQo6t9A7ZUmvGxyNdn2PrGr/yk8ET79MhIsj8SZJRAUUPI4iXVIPWaukUIhBh2uWunr05GuME0jWU42dTlyLpQ6dVtjPtZOBV/79VNWe2X1qKtS5Uq8B5Qstra+X/2vsAaA8jbfV3DpMQgB+hP5ZrwxHCu6gWDRj89NaVv0ajlDKPi9HPGFIYq174GalyZD65SDgl9EBwjWPb182fir2pGJSjvfG/UC+xUrbNv8gOij9zda3FowiPXhe3QIAjuBJqvrxPRsqVruc4FLbpjeVwsibSlF38Vu0IFfQyl58p1y8Aw0nKk5m0yxpqgLWavW+gRGJ1VxQ8skJXlzoY1WDRjDbthnAsmgPMb8n4ea3T/OiRtid2neYXppnf1TZNRsDCWmBfIRnDfkNqILzvz5ZDE95+mDkfubmEHtpMmijXgjqU/oNffitzyeEgXb9OtyBeyJuVJsLWNtSo2JC/xfhSnDYezW2gWW6syOhEWX4cjebL42+Hb43aiI7Yq1q99xOOWzzgMkJqHv+d2KD12tbZyNZ9+tm7Z0Tm00OYtqqG11XAxvoWFR5JjoBvLSZvCe9/zxdo+kk/HRkHDt5GkQu4P3pCpRRYzedfZiZsVYth0rd6NqMoyhze6QqoLiHCqoYlj4hWAc3Fxxwh4MaboPMOmsCV1pojwmyt2r+IHzyNhj+4UGHOpS8bouGf5eiVoTW9egiz/c3RdSUXYkCSm2g+5ZJ+cEcafn8SdZ4AuQ2EK3lEFbCqWYdcRcIiF8W1cCAiOW6nhS+6cogghQYkerxFtBXhHlYvWSH6yGGPRf3/gTqCwRbKQb9NcP6CGO5QTN+/i4F/ZKbLsLu2KdFnLakPJihGj3AHaalz4xRseFY//EBgaO4w0/KMaSj2FQ0A02xtrJKqZ0NsrcgGq/SYHNFq3oISDAwpNrdaGQ== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0cf1a8c5-9ad8-442f-2ddf-08da230339d7 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Apr 2022 19:23:18.9717 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9MqmIK5yedIWya1qvWQdMKzpLir/xfxNAeER0EkJ9LEURQ4hMjDH1NSwjN2L6BM1 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4325 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Instead of a general extension check change the function into a limited test if the iommu_domain has enforced coherency, which is the only thing kvm needs to query. Make the new op self contained by properly refcounting the container before touching it. Signed-off-by: Jason Gunthorpe Reviewed-by: Christoph Hellwig --- drivers/vfio/vfio.c | 30 +++++++++++++++++++++++++++--- include/linux/vfio.h | 3 +-- virt/kvm/vfio.c | 16 ++++++++-------- 3 files changed, 36 insertions(+), 13 deletions(-) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index c9122c84583aa1..ae3e802991edf2 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -2005,11 +2005,35 @@ struct iommu_group *vfio_file_iommu_group(struct file *file) } EXPORT_SYMBOL_GPL(vfio_file_iommu_group); -long vfio_external_check_extension(struct vfio_group *group, unsigned long arg) +/** + * vfio_file_enforced_coherent - True if the DMA associated with the VFIO file + * is always CPU cache coherent + * @file: VFIO group file + * + * Enforced coherency means that the IOMMU ignores things like the PCIe no-snoop + * bit in DMA transactions. A return of false indicates that the user has + * rights to access additional instructions such as wbinvd on x86. + */ +bool vfio_file_enforced_coherent(struct file *file) { - return vfio_ioctl_check_extension(group->container, arg); + struct vfio_group *group = file->private_data; + bool ret; + + if (file->f_op != &vfio_group_fops) + return true; + + /* + * Since the coherency state is determined only once a container is + * attached the user must do so before they can prove they have + * permission. + */ + if (vfio_group_add_container_user(group)) + return true; + ret = vfio_ioctl_check_extension(group->container, VFIO_DMA_CC_IOMMU); + vfio_group_try_dissolve_container(group); + return ret; } -EXPORT_SYMBOL_GPL(vfio_external_check_extension); +EXPORT_SYMBOL_GPL(vfio_file_enforced_coherent); /* * Sub-module support diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 132cf3e7cda8db..7f022ae126a392 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -143,8 +143,7 @@ extern void vfio_group_put_external_user(struct vfio_group *group); extern struct vfio_group *vfio_group_get_external_user_from_dev(struct device *dev); extern struct iommu_group *vfio_file_iommu_group(struct file *file); -extern long vfio_external_check_extension(struct vfio_group *group, - unsigned long arg); +extern bool vfio_file_enforced_coherent(struct file *file); #define VFIO_PIN_PAGES_MAX_ENTRIES (PAGE_SIZE/sizeof(unsigned long)) diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index 50193ae270faca..2330b0c272e671 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -75,20 +75,20 @@ static void kvm_vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm) symbol_put(vfio_group_set_kvm); } -static bool kvm_vfio_group_is_coherent(struct vfio_group *vfio_group) +static bool kvm_vfio_file_enforced_coherent(struct file *file) { - long (*fn)(struct vfio_group *, unsigned long); - long ret; + bool (*fn)(struct file *file); + bool ret; - fn = symbol_get(vfio_external_check_extension); + fn = symbol_get(vfio_file_enforced_coherent); if (!fn) return false; - ret = fn(vfio_group, VFIO_DMA_CC_IOMMU); + ret = fn(file); - symbol_put(vfio_external_check_extension); + symbol_put(vfio_file_enforced_coherent); - return ret > 0; + return ret; } #ifdef CONFIG_SPAPR_TCE_IOMMU @@ -136,7 +136,7 @@ static void kvm_vfio_update_coherency(struct kvm_device *dev) mutex_lock(&kv->lock); list_for_each_entry(kvg, &kv->group_list, node) { - if (!kvm_vfio_group_is_coherent(kvg->vfio_group)) { + if (!kvm_vfio_file_enforced_coherent(kvg->file)) { noncoherent = true; break; } From patchwork Wed Apr 20 19:23:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12820731 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40FD7C433FE for ; Wed, 20 Apr 2022 19:23:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1381781AbiDTT0O (ORCPT ); Wed, 20 Apr 2022 15:26:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46434 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1381767AbiDTT0K (ORCPT ); Wed, 20 Apr 2022 15:26:10 -0400 Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam08on2070.outbound.protection.outlook.com [40.107.102.70]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8F6A1427FE for ; Wed, 20 Apr 2022 12:23:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hcM1cye7VtTS/PiA1L4ZZ8g4SWsJWNRircIDvNwxZBchnwwdvYgT73eg2scBklPlYInTFJdTf6JgDfk45159+D3BbHD8YioOnIViZJxU+B3U0K5t4qGQ0DSV8f3bpOICE1gegB1sTV51r0TDy5jMAvMXnjtRdxGwswjg9T4NQiBXOrO+Cl6I4lumGZoIYzQwsVhkA2rhQ/wecEdu6cSodETsKdlZ1hCkrjmJOFW58VF2VRegpMFYn7FJmjmMpFI9CwQBRIuW1Z/j0Xg/rfXQpxbAyF0kYqecMqbmwM2mLV8Ha1uHdiHr662n2X1Or3EfuQkN97hoLj1U4T/WDh6Vfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ugI/oGBMEue8Bi3Gq2aim10r2xqeY2bEycUnr59bEis=; b=B6ppXePPlRaSAGGV3bdBbgYcEfdufhJJBjUaRKRNWt9LHDyGuH4Vbi+qxR+GBH8a0CwLtnvf2iZ6DXIiaT+YQ2EmcBIbldjItxR8w6HqiXjN1ja2wy1xBs+0Izck3mowpO6lohULREtSlTQq1LgVMhQEOr89c8uQCl6iOTS2CxuIbBcYz8qKtiOqve6dBkQSS77lp8BJF35sievY79L43jqk4Z78XXhmD9fcKXgFl0xJscZ+FOGKIrzvsoBntLYv0aM4prmD5tHSDo13rOrrYz5aSA3aePUfIFgo9h6KdkEIbql/Vnworg1S3R4EqQyZDKHD23LrfXWRpR3bvJV7dQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ugI/oGBMEue8Bi3Gq2aim10r2xqeY2bEycUnr59bEis=; b=pjj6sPg7KkCeIsF9uqzI1XCGRKNBtjnYnoT/JPW05cG3fgYURXkaIDOMnfopV2B8ZEyMqAvtKhPtXoVkHHQ/vfdhyNK2xzg7Mlmz1e/ZXgxtGcySbKiUgyJwpQMOzlHG0MYX9lU66oliiMlswzEPcTChJhusrm+bRhdQnH5FM8vYenxgMUEZp9MT95d3nRunzNN1XnWaf6t4HcqCRLBiMwxbB7MURXebi7PKYZOb3qeRrJk+AE5w9vNM2bWYcSf46iBcpdgD5jGiRZ06Od7hCkxXVpuFoe/RPuSbMVbTLbhhXlI/GQD244nqwyI9Nv/RnS0MSK7KdUdHJpxLi/dbfg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by DM5PR12MB4679.namprd12.prod.outlook.com (2603:10b6:4:a2::37) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.14; Wed, 20 Apr 2022 19:23:19 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5186.014; Wed, 20 Apr 2022 19:23:19 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Kevin Tian , Yi Liu Subject: [PATCH v2 6/8] vfio: Change vfio_group_set_kvm() to vfio_file_set_kvm() Date: Wed, 20 Apr 2022 16:23:15 -0300 Message-Id: <6-v2-6a528653a750+1578a-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v2-6a528653a750+1578a-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL1PR13CA0128.namprd13.prod.outlook.com (2603:10b6:208:2bb::13) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1a5350d1-4393-4ddf-0a1c-08da230339b1 X-MS-TrafficTypeDiagnostic: DM5PR12MB4679:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yIvYgaUN58bcA8b/HIMYSNIOcVEr2/YVdp6SZfGkq06+3oJrigr9FW768oAADkzzOVT/ZhXfeZmVl6c2A6XAs7L2lSojjEmirHPvbuOVCTyM3lq/OWRbQ1XFMc8alQgMvsPT1ZwClE0XiI8jvNg6zg3eLXJvOT+O3uFgHGltg8sB8xVoGdL6srT61cWcsAByNnWjg4l83h4inG0t2ryvyvvwZULVkSFkiiEcbjuOkMsmBPDVcRZb+jvKrInsofmSk6OhO7Qv9UwbLYU4ySTArraG+W5Ddv7C5k/6qj5TyJ7YjsJXefp1N/vwjrZu684IEoZsRZXyRbJ5R9gONC/l11LmD2qF7Tph5b1IPMEb3u8fBKOjmSgzFl8YiUZifcdkBHbc9JkY//S3fAM5p+P3QDhB9kU4sZl5C6CILUWQZWgV5ptO8Dd/I5oDU8NeyoK0eGZJCqQx6dfK4baYzn23jopi6D4wFcqXk3HwILCH4+6fHcHj4sZ5TzCDYTLlmPqB4Hn89sYVlhXUNnvR/CL1Oh7Yp/Ag0VUd2fUX9iI1CzwPxifNhin1/NH9eb5cupuePTRkGnPDqGHRw96PKYE9iWVJUq6ry995GKGSqHTzJJ28xzI2yyQhrChRFsFUqgct+BfKXfINxH/EaXf38pvWIyvheJdKtIdZFyHQAFTzfboivF2evTWgk85VxztyAXsA X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(508600001)(86362001)(5660300002)(83380400001)(6486002)(36756003)(4326008)(2616005)(186003)(8676002)(8936002)(2906002)(66556008)(6506007)(66476007)(6512007)(6666004)(26005)(38100700002)(110136005)(54906003)(66946007)(316002)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: XRLUlqL1hjwSoxngPtXZdiIk5OB7e0yJwq0qRpmY17S3Ykko3rwnpWBTP2PTs3Ywryfv81MxzGp19UW1OojYbYgZiu8YNQQ0ui23gYZTo+iqCWb3G83yHAKilwc1pIl3oCl1hFu4U7f645bh3CBnFqO7iEzmtxBZL1cPcdAdUJSS217pr0/gpWBsU/EgFEnSXMDCoYdUN7c7EVFmcRPJHnDxwr+RTc8yzCPY7jnNa4X9i4EtjLof3TNDZN5nkpE7QIMPRA6JDeIwuIZrp7NPj2cF+QrrV22NsHFzAtHyxF4BmcYKn47Y/A1OOTsW7xYBan4r70KVJDOMNxrK24PvnZp1bzjHyCcL37lZE1WOla3zyB5kkOchF+snkgmumK8+KTNQ/6IBxvvYAZ+KyvvvlcDwD2ySyEwlx7xLQLshigzNbuwAC92VEhiL+0C8dAINa5BasR7QZZi8r4n+tTXJ+SUPmIRWy0uINLPNu9t1LaaOiMbEbfASPcQD2k3ZC2mL4p8FaJqXmITjxo5yma+IjZuBpuIwjhtJlsDrlhw07gNxcOwNRu1kvEmFrk0NcwUc9swI/hZXI+wfPq4YeCBzqC2QjySYWjc3mdsCwEHbQZTMy7jIRKcvHE7izsHj4bv3ox9vMHzt9MfWpMdAL9+wib4GZAqNc9CqRisQrN5EYjTq5f8ixozPtXj3Kq0CjoKND+Gg/AycktGqLK8CaBwxdKyhLv4vNipDxQwEyu793ZyglopaK0Nk+AgQ9JSe56NTVE1OqV37bV3XEEz3eUb+wCgOTfEa9g/foeXB8fdYKF5u+5IaGci98RE5JMo2OuUcL4vQjTnIsMt1lxTNIiPHRLZsPuZ8HACPnkUvNfw+as1FbBhz06b50/G8/mSEN1CrQKSQVYF9Il8cOeX3WLqgaLffWj9Nf4rC+fGf6PDvKRpoBTsok//ajdNqiNbKsxBm9j+Ut2/ynKaVD5YKWoYwEuNeGlEUEQLT7wlO4CDNp0p2k8deff4QZ6G36nXRo0u/PqjNj9CdXCtFVZSMtUH4olCM+w2RJP1qtgE4u+Rfb67bOIfwiQgm2+Wi2EFOoBMj58fEo1icshxdS2v0RvCeAliUobEdwLTH/P/SxWrNWPCZXq/p0HLvM5SYf+GDwSYGkosJfJNy9wDyTS6BKDVtwdVtrOtGMw9X8CrpXrXLuc6fxdktLQt5oldQVncPRj1BsixUhvcqDO7y3z1VRBBcHsfsX0PpSgjJjrVs8iADH4MHL754d21aXenLOAYHJElaC5V67sgQ6MTjNJq3KHvfF1MdOCQxwPkzy48VKP2jHwEZHBfQOto9SedkoVUCH//eCPgYbIIUtsGQGSKN+QKHpoN3xMF9SCCuPGZYC2fcImISyX9Rg1AaOwhKoE15A36lAbvaxQD2xKIItFYiOSc9GCBENMqvsEWpYzNY6TqetLY4IxpXhVrk/1dfjqeJf6F94im9dCTNqWc7EjOnqbRGmYT24eOS8LDHHncC0CCdC27B3eFVbUxxv8LjwqpdvnYsplo6zfT289SKEz08l6kQPErQ0cMf9a/v8Ulp1c+g13QmZ6EPfoCgHi85xDvuqFA8K2n+clhmwjwd629WshVMpbMVgt219hGjShdmhq+mzhi4aPjc/wT8NGqHCkHczv8SUi+5gbEH6RN+HrauQkjtZvisNIcwT2XsIJPu9o16atuV4MIZNaq/0gohAUnPG0JIYeEavdb19whjJ7qlE0EABw== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1a5350d1-4393-4ddf-0a1c-08da230339b1 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Apr 2022 19:23:18.7217 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lDFsH3MNuM3BuiYkaF2CDcAIfT5vW0P+xrYqRvyL9BYo6rfPtpHFV5MJM8k6/KXZ X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB4679 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Just change the argument from struct vfio_group to struct file *. Signed-off-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Christoph Hellwig --- drivers/vfio/vfio.c | 29 +++++++++++++++++++++-------- include/linux/vfio.h | 5 +++-- virt/kvm/vfio.c | 16 ++++++++-------- 3 files changed, 32 insertions(+), 18 deletions(-) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index ae3e802991edf2..7d0fad02936f69 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -2035,6 +2035,27 @@ bool vfio_file_enforced_coherent(struct file *file) } EXPORT_SYMBOL_GPL(vfio_file_enforced_coherent); +/** + * vfio_file_set_kvm - Link a kvm with VFIO drivers + * @file: VFIO group file + * @kvm: KVM to link + * + * The kvm pointer will be forwarded to all the vfio_device's attached to the + * VFIO file via the VFIO_GROUP_NOTIFY_SET_KVM notifier. + */ +void vfio_file_set_kvm(struct file *file, struct kvm *kvm) +{ + struct vfio_group *group = file->private_data; + + if (file->f_op != &vfio_group_fops) + return; + + group->kvm = kvm; + blocking_notifier_call_chain(&group->notifier, + VFIO_GROUP_NOTIFY_SET_KVM, kvm); +} +EXPORT_SYMBOL_GPL(vfio_file_set_kvm); + /* * Sub-module support */ @@ -2446,14 +2467,6 @@ static int vfio_unregister_iommu_notifier(struct vfio_group *group, return ret; } -void vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm) -{ - group->kvm = kvm; - blocking_notifier_call_chain(&group->notifier, - VFIO_GROUP_NOTIFY_SET_KVM, kvm); -} -EXPORT_SYMBOL_GPL(vfio_group_set_kvm); - static int vfio_register_group_notifier(struct vfio_group *group, unsigned long *events, struct notifier_block *nb) diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 7f022ae126a392..cbd9103b5c1223 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -15,6 +15,8 @@ #include #include +struct kvm; + /* * VFIO devices can be placed in a set, this allows all devices to share this * structure and the VFIO core will provide a lock that is held around @@ -144,6 +146,7 @@ extern struct vfio_group *vfio_group_get_external_user_from_dev(struct device *dev); extern struct iommu_group *vfio_file_iommu_group(struct file *file); extern bool vfio_file_enforced_coherent(struct file *file); +extern void vfio_file_set_kvm(struct file *file, struct kvm *kvm); #define VFIO_PIN_PAGES_MAX_ENTRIES (PAGE_SIZE/sizeof(unsigned long)) @@ -183,8 +186,6 @@ extern int vfio_unregister_notifier(struct device *dev, enum vfio_notify_type type, struct notifier_block *nb); -struct kvm; -extern void vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm); /* * Sub-module helpers diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index 2330b0c272e671..2aeb53247001cc 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -62,17 +62,17 @@ static void kvm_vfio_group_put_external_user(struct vfio_group *vfio_group) symbol_put(vfio_group_put_external_user); } -static void kvm_vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm) +static void kvm_vfio_file_set_kvm(struct file *file, struct kvm *kvm) { - void (*fn)(struct vfio_group *, struct kvm *); + void (*fn)(struct file *file, struct kvm *kvm); - fn = symbol_get(vfio_group_set_kvm); + fn = symbol_get(vfio_file_set_kvm); if (!fn) return; - fn(group, kvm); + fn(file, kvm); - symbol_put(vfio_group_set_kvm); + symbol_put(vfio_file_set_kvm); } static bool kvm_vfio_file_enforced_coherent(struct file *file) @@ -195,7 +195,7 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) mutex_unlock(&kv->lock); - kvm_vfio_group_set_kvm(vfio_group, dev->kvm); + kvm_vfio_file_set_kvm(kvg->file, dev->kvm); kvm_vfio_update_coherency(dev); return 0; @@ -231,7 +231,7 @@ static int kvm_vfio_group_del(struct kvm_device *dev, unsigned int fd) #ifdef CONFIG_SPAPR_TCE_IOMMU kvm_spapr_tce_release_vfio_group(dev->kvm, kvg); #endif - kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); + kvm_vfio_file_set_kvm(kvg->file, NULL); kvm_vfio_group_put_external_user(kvg->vfio_group); fput(kvg->file); kfree(kvg); @@ -358,7 +358,7 @@ static void kvm_vfio_destroy(struct kvm_device *dev) #ifdef CONFIG_SPAPR_TCE_IOMMU kvm_spapr_tce_release_vfio_group(dev->kvm, kvg); #endif - kvm_vfio_group_set_kvm(kvg->vfio_group, NULL); + kvm_vfio_file_set_kvm(kvg->file, NULL); kvm_vfio_group_put_external_user(kvg->vfio_group); fput(kvg->file); list_del(&kvg->node); From patchwork Wed Apr 20 19:23:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12820730 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5E67C433EF for ; Wed, 20 Apr 2022 19:23:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1381784AbiDTT0N (ORCPT ); Wed, 20 Apr 2022 15:26:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1381763AbiDTT0K (ORCPT ); Wed, 20 Apr 2022 15:26:10 -0400 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2069.outbound.protection.outlook.com [40.107.223.69]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 163DC4551A for ; Wed, 20 Apr 2022 12:23:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=d07xQzTrEAE4aAaW9phUtxLww/DuvB+ajbEUuPcoyQSvhjGUg8Nx9neNUi5GejpVG92Wi5hk1Py3rgDdZGAGm3gCDiUWv5MYxC2jLch6nCl2o/1y+xc7v30M96a1f7uplGUYZOPiywCCTxePncVQhB+3r/HE9oijNVJLrbmNzExejpkNxXQH7zQ5JXNLs88DbcXnuQcmiRIpE8sKrOZUjFa5tIyZGr/sC8t+QtcHqFp2iHdXpu5YOKfNjK8NCHM78eUI3dGj7Y4YrBa5pe+7+Zo77x6WQCA251aBEtKcGE2P+FurxmQP+EfhQ8mKwzyyYzbEy4WAtGbDGgfp8qAksg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SOR40/At1boJtRmvFgNosCrdc34vkjdQ7fwVQW6HJJo=; b=Uz2HhY0GXuOi+AwOi7tQjKobHVhMOURpKnBouYDvuck2JRr47bOn3B4e+tbZAx8qrU9xVTt4rldLYcYNQZrXeW+oPp2yO8BEqQ3ku8jBddZanWj9hBSpkTuozvlJExRa+1Zo828Loht8C1S2g+xqXqWcdgGZw6v9UA0B0owW2+3QzrQKKNnbrY6s46tMvaoi54EePNh9jnl+UcdDLXcKYO16yXQ63XU1XU/IPU6W6bBd0dQVYRLzZj0BxTD8s4OvTnfkj4K1LHYM6ozIM6ZEdn/s+xfv/IcAnw56qsTWLAXyeghCyZsV7YOZjVk18w/nsefEfLqkLX6ahUvapcEhFQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SOR40/At1boJtRmvFgNosCrdc34vkjdQ7fwVQW6HJJo=; b=g8IjjDODz1ZoBU/RS3i2f+DKU6n4YEPOYGd9MIFTtyjsOvLBuQL5+BMVpdoZ/Elo0oRo4Le6Rw5sXGAjwsHLhlrRdH4YgRSVStFGlWQYFITv3CDVUIlnJBjl+RpKQefCEeQKTxgY4An1Rz3jlVAezwkDz5h9PjUvPTGQJT4y2L9gkXgKV3oWkaGwJ0mwaWCsYL5tmPtL289nBlWYFQ6vx7RnPBpi8JLsRDxaxruiZIgP1m7wgiELxfE/Vp3Tpy6WHJGeIm6Y9bxrscNtNxZgUAaG2+l+tC8uLx7fh5HfxM9vFK7KOo+to9+5Nd3TrkPnbzkCRZPajhVPUwdhsb4K1Q== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BY5PR12MB4131.namprd12.prod.outlook.com (2603:10b6:a03:212::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.13; Wed, 20 Apr 2022 19:23:21 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5186.014; Wed, 20 Apr 2022 19:23:20 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Kevin Tian , Yi Liu Subject: [PATCH v2 7/8] kvm/vfio: Remove vfio_group from kvm Date: Wed, 20 Apr 2022 16:23:16 -0300 Message-Id: <7-v2-6a528653a750+1578a-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v2-6a528653a750+1578a-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BLAPR03CA0120.namprd03.prod.outlook.com (2603:10b6:208:32a::35) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 89f7c04e-aea1-476c-54a1-08da230339de X-MS-TrafficTypeDiagnostic: BY5PR12MB4131:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: tnBIAEabVV0ZUuweRI4Q1bZI45wpwSCSpQcumx12Fs9skCwAJNkQ5P/UQhkBABSUwUoBpCRC2R9O7uedr089cUv5LPMIiNzCaZYyXUJfKeJAh6CrSQBhgAHJtlS8Mh0+gofat9XpNaWn/KleWeBx7SeOLjCOMPT/I/rm8uYV3jJtf4YdEDkbHwurzdKc/agqT8SVKGhp5NhbTHNME6JZ1H2vIy1YnnL0PYUK6vB0IaXEQ97l/HjJ89iCYe7FHFV8uOAyX73X5cggbXDCWL507Bz2CN/0vW5B5DkoTaZE5TY9BZv6GQ78UxgRvmBUqKPFGhJMbSiplmAE0/XpEk+5DlWBPI2mVC75AY3GNjb9KvthMDDTwsV/bI2trd9SQVD/paDbnwZAjes2WBAINIp4CULhtd9PqK+/l5uVkWuqBHEK2qlfkOLWdZTEoKHr7KWa2iGfzR4s8ucsBu9LpW42z4eOG/eXvaT8HoAxb5uyCPjafmF+YV3txoEyqs2ziBtFIJU5m0akWVQ4/ZAAkCaVi6qUGYaop6OUK/xIkvZkOaZFkkAovHDcrGEJwmOOD9HwFsb13RdJoBTNa8KaJa9mnQvA0rE0ulKH1I3+ACUfWurNiksZM6AS6iHzd8Sy3FwstIfF8wFFwmn+3E710TEbszE9inyP/Ax9X0Oq4ToauI9OXKLRj72owlRh6U/V/DM8 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(26005)(38100700002)(6512007)(36756003)(6506007)(66946007)(2616005)(186003)(6666004)(66476007)(8676002)(66556008)(8936002)(316002)(4326008)(5660300002)(86362001)(83380400001)(110136005)(54906003)(508600001)(6486002)(2906002)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: zHZm8KhE2UX87/qtyccIM1EXcZSb3lhy22qd7YJ7UGUguT2VFHmpXZefLIqJUsQ4Fba8tDmjh0ou4TpDJNv4uQF72rF0aQS79KXE9onuqb7ud7H1pfYtEOhV0tUdMPmkwwi7AyHf9c3Ezl/Y/qQmjIEMEWVg4FD+I5Ue0AViVO5crJM4qcokvg48yRLhLfOzPeVHXb9pz8llp7a0Jo824fKyTSB0H8GYOAPL7wr8H2tzjRmdZ4bO464lRDLSGT/q1/iEw87Q0I67kwUOkjpFL0Dfv6iw7P35+1YWs2zGpsAunwafGOFn+5LwsIL2KoZjlTuqHeZDOxTuBMC+DKZb2SJNpwYDWU0gesh3WYhjbJNMBnDyQBtyiB48wvva7bJKF6WerG1HMxJVLjg1ereIDt0uSTSXIGHEjC0OpVcSlKd4hxVH7+IZnAzLYnHJ/U1LJP7kRGYzu8bMocjpvdcWeBuzgTGHP98oTjYDYnebHQHTfPaFiW+EUxe3nPjS4RpFkALtjuIsnvtQeAET05u5nt9pIUcB3PnH65xYo2LX5k5/kathAfpkCzr/VB6pKIvONlU/wTdU8JiAzWJvsmMFPXN4s+tAVn9ca5AUsdmzr+Zn/lSNJP3bL/5NEx3AK0uEWFFDZhgjIyNkjJZLkl/nA7s7hsvvt/9XWHBvH5IGczoU9opK3N4SF1RG+JX0lMGq+Nvrjlw816bxeqZkR08ibYJ8RFexHA1MZBFEQAF7U5VzUm0tx+vzPVyHFuDUhz+m+A1hCaBwTVkbLRD9LbyekW9CV3Uq/z9GXdZrGEIA6DELDRXasx5LtjoFI5CNHJyjPOKR8Zrd5CKFdBT0gR97FsPmTM82Th//YaVLjPswiW/9ucMH5boyJJwaeSKkuy65ZeSoxxEb63RIy9l2Y8+Hm5lZHnKmc0ZKn0y2XkLainOlv+djr4kvuXNcaOd2haw/+hyFYcnykZawUYLvEJlAJ+7nLbhczdTgk78yipXIe+Ljm5KeTmRUdZE1xL9vJa37DvswlM86EXc2s6HfvQzRxajA3BrIpCPkr/coJWbD6XeuzX9vVRVPSIsTD/XT978Ze9cQgo/aC0BCCqqJnvvQeMIlpjv9Gzs5XRG/iUk7RiBrtgYPbjVM3fE/+/rjygdk1bQiye0oParK/BEy0uVlVFSoB76V8BAyGTCu6EYO4bZu1udvKmkjwhYzml2Zv076+8Cohk2kvwbgNzNtRNJ6E9USzTFR5etr8axowPo5JHKbMy/cjxCaWn+XsgQ+R5ku3CRrdzbtHCxczD+Pz3k/o3pYaIHVXBGKjPmJd+N9n3zt4N9oMJY4yINmfqL4+ApT7ygaRuD2ZTcTIariqgN7j7Vb/qI5pB/kncAbPd5pJyj/UwrqP283PIgQ61CPAE3JyqOBitZ8ubA4gF/Z5riyUf5BWaMdXKuUp5ZOX1jrYvSRJnMioIgwtF/kyCzDbhJbzedDnHGK2XGgjlpDpA63PC4SKDs8u/8A6zfOmgoxTEHbZ8lmnjUW21MdDsF5yelH95xWD14BY5xQxWaZ7B/uOetGPLbzMGWVpXJYTHRJUCC742xaMT3VopZy/hdXFls7YSLrDRYZJQF8dnr3YTXI0826b/SX693FiG0DThOj8/vQ/x9yhg1rkSI8knBLQk1uCdA+gmBaQNG6ndXnsS6RQ79bJZbnfN6AGRYQfssIjOik502tJS/o0ChEvDNYa5tP7bKQ0nu7A7CtqAc6Hr0oIQ== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 89f7c04e-aea1-476c-54a1-08da230339de X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Apr 2022 19:23:19.0342 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7ks8XHlf/7QfAaiGRDzw/usruiYnkhENz0v8StZniGRSB3jyMKiLteMNB7xr88Hk X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4131 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org None of the VFIO APIs take in the vfio_group anymore, so we can remove it completely. This has a subtle side effect on the enforced coherency tracking. The vfio_group_get_external_user() was holding on to the container_users which would prevent the iommu_domain and thus the enforced coherency value from changing while the group is registered with kvm. It changes the security proof slightly into 'user must hold a group FD that has a device that cannot enforce DMA coherence'. As opening the group FD, not attaching the container, is the privileged operation this doesn't change the security properties much. On the flip side it paves the way to changing the iommu_domain/container attached to a group at runtime which is something that will be required to support nested translation. Signed-off-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Christoph Hellwig --- virt/kvm/vfio.c | 51 ++++++++----------------------------------------- 1 file changed, 8 insertions(+), 43 deletions(-) diff --git a/virt/kvm/vfio.c b/virt/kvm/vfio.c index 2aeb53247001cc..f78c2fe3659c1a 100644 --- a/virt/kvm/vfio.c +++ b/virt/kvm/vfio.c @@ -24,7 +24,6 @@ struct kvm_vfio_group { struct list_head node; struct file *file; - struct vfio_group *vfio_group; }; struct kvm_vfio { @@ -33,35 +32,6 @@ struct kvm_vfio { bool noncoherent; }; -static struct vfio_group *kvm_vfio_group_get_external_user(struct file *filep) -{ - struct vfio_group *vfio_group; - struct vfio_group *(*fn)(struct file *); - - fn = symbol_get(vfio_group_get_external_user); - if (!fn) - return ERR_PTR(-EINVAL); - - vfio_group = fn(filep); - - symbol_put(vfio_group_get_external_user); - - return vfio_group; -} - -static void kvm_vfio_group_put_external_user(struct vfio_group *vfio_group) -{ - void (*fn)(struct vfio_group *); - - fn = symbol_get(vfio_group_put_external_user); - if (!fn) - return; - - fn(vfio_group); - - symbol_put(vfio_group_put_external_user); -} - static void kvm_vfio_file_set_kvm(struct file *file, struct kvm *kvm) { void (*fn)(struct file *file, struct kvm *kvm); @@ -91,7 +61,6 @@ static bool kvm_vfio_file_enforced_coherent(struct file *file) return ret; } -#ifdef CONFIG_SPAPR_TCE_IOMMU static struct iommu_group *kvm_vfio_file_iommu_group(struct file *file) { struct iommu_group *(*fn)(struct file *file); @@ -108,6 +77,7 @@ static struct iommu_group *kvm_vfio_file_iommu_group(struct file *file) return ret; } +#ifdef CONFIG_SPAPR_TCE_IOMMU static void kvm_spapr_tce_release_vfio_group(struct kvm *kvm, struct kvm_vfio_group *kvg) { @@ -157,7 +127,6 @@ static void kvm_vfio_update_coherency(struct kvm_device *dev) static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) { struct kvm_vfio *kv = dev->private; - struct vfio_group *vfio_group; struct kvm_vfio_group *kvg; struct file *filp; int ret; @@ -166,6 +135,12 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) if (!filp) return -EBADF; + /* Ensure the FD is a vfio group FD.*/ + if (!kvm_vfio_file_iommu_group(filp)) { + ret = -EINVAL; + goto err_fput; + } + mutex_lock(&kv->lock); list_for_each_entry(kvg, &kv->group_list, node) { @@ -181,15 +156,8 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) goto err_unlock; } - vfio_group = kvm_vfio_group_get_external_user(filp); - if (IS_ERR(vfio_group)) { - ret = PTR_ERR(vfio_group); - goto err_free; - } - kvg->file = filp; list_add_tail(&kvg->node, &kv->group_list); - kvg->vfio_group = vfio_group; kvm_arch_start_assignment(dev->kvm); @@ -199,10 +167,9 @@ static int kvm_vfio_group_add(struct kvm_device *dev, unsigned int fd) kvm_vfio_update_coherency(dev); return 0; -err_free: - kfree(kvg); err_unlock: mutex_unlock(&kv->lock); +err_fput: fput(filp); return ret; } @@ -232,7 +199,6 @@ static int kvm_vfio_group_del(struct kvm_device *dev, unsigned int fd) kvm_spapr_tce_release_vfio_group(dev->kvm, kvg); #endif kvm_vfio_file_set_kvm(kvg->file, NULL); - kvm_vfio_group_put_external_user(kvg->vfio_group); fput(kvg->file); kfree(kvg); ret = 0; @@ -359,7 +325,6 @@ static void kvm_vfio_destroy(struct kvm_device *dev) kvm_spapr_tce_release_vfio_group(dev->kvm, kvg); #endif kvm_vfio_file_set_kvm(kvg->file, NULL); - kvm_vfio_group_put_external_user(kvg->vfio_group); fput(kvg->file); list_del(&kvg->node); kfree(kvg); From patchwork Wed Apr 20 19:23:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12820732 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5DA4C433EF for ; Wed, 20 Apr 2022 19:23:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1381790AbiDTT0R (ORCPT ); Wed, 20 Apr 2022 15:26:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46490 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1381779AbiDTT0M (ORCPT ); Wed, 20 Apr 2022 15:26:12 -0400 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2069.outbound.protection.outlook.com [40.107.223.69]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 742F745AE8 for ; Wed, 20 Apr 2022 12:23:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QOrMqnCUYWSEYyZsd2NLLQg8GqZsqOATrJehxeJK8MLdxHILz9gzY6+nSdMr8zomoqY19q/KDr/6PSkf/d/D8A1Et/Sj/L8fkZhjxHxsL9qZOBCUIQTHQ8xTJA5ljX8UKfSRj/N3s8xDLZOKukeeyCGrX7o/GfX1zhaiw53wPhiMUYLrToKlGxWZk3/GUsvh9gWhLX1/fW21axeTnzL+QzBp6RVn80/t2B+WaJd6Ab6Pn+4OEvfAMPQXJS3YWe+nqBo/cIcGMBWA0p4il8BocsTyinHzix4rucBKz5xR+fwDl/kehyBtHLt+/kKr+uH/qARyQCemGKY7zs5tD3cPyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=peG8yk59Lq+LS+BJ9+PXG8JsMnm5EijEobHJD/bmmP0=; b=Zj7C4SrVWm8NxHhmrshWoXU3KOXDflWK2rS3XPuaa7yJpVFxyY6jVMQdGZSK4vthBK6ITvqu9ZkDtrSnaEutrTB1fazQpe6DVUxEcXg0nQpD4iayz278e/wXFGSKQ7T/BTcWa/UjWGsBqpRYxCpKX2XAQrpGr0LW6MiW5cmb/A4wDdVFgX5Yu/sMDp1zKZZPpOijnHJrmDq9inovnpNVwmpLerRZ6xyh4bklYqKhKutUHAp8drHV/ZOPa6SK0HMU8YB4md8ZHddTXCzFpCaNSD+luXmTuGsqp88uVOEVN2/SiJliyq4w3yDLAD3UQPsyxKHYCoejPslenmUKyMFQDw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=peG8yk59Lq+LS+BJ9+PXG8JsMnm5EijEobHJD/bmmP0=; b=sJhU6Q7CrYRlbA0/dQ5ZnZFix/wrRTLpbv+K1KbHQwwRVqJUTSnEtir4o3viiZok35Zbai/sIFB42c/mFNzCCSJlp7iI3CJ/dEP/by/NARxjeO0jNq1ts0TF/Q/qJUmkVGHZyXjW58Ln55Rv8QZK72tQOhC3Mm/oQQrgB7ORfxXxkB6FkzrKWrUrQEg3IP2EL2OlL/mt0SGDQ/KH5OD5TrnQCXyK79INROVVnz+jWo9k+Zj94v2OwY4r62zMgHnd19MmGxusMVmupMalJvWCaOogIrjNwCrVYrj6zTlL+ese3cVaDMSAbITYUC++zR3tgj5wC4QCn2bQqEEtVqKRtQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BY5PR12MB4131.namprd12.prod.outlook.com (2603:10b6:a03:212::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.13; Wed, 20 Apr 2022 19:23:22 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5186.014; Wed, 20 Apr 2022 19:23:22 +0000 From: Jason Gunthorpe To: Alex Williamson , Cornelia Huck , kvm@vger.kernel.org, Paolo Bonzini Cc: Eric Auger , Christoph Hellwig , Kevin Tian , Yi Liu Subject: [PATCH v2 8/8] vfio/pci: Use the struct file as the handle not the vfio_group Date: Wed, 20 Apr 2022 16:23:17 -0300 Message-Id: <8-v2-6a528653a750+1578a-vfio_kvm_no_group_jgg@nvidia.com> In-Reply-To: <0-v2-6a528653a750+1578a-vfio_kvm_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR05CA0051.namprd05.prod.outlook.com (2603:10b6:208:236::20) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 439581c0-8970-4e0b-51b3-08da230339fd X-MS-TrafficTypeDiagnostic: BY5PR12MB4131:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: nOVdiXhxzk56p5/JBa5LBetfLAbywf0oEZMzCxWOhTMZm+Q3UCuyH9eaTOBsAuOtdqpB3CKn5LF30Fj8FRQFXCp42wjYwBSQPR4Pdygxe4lfSs2eV+CG7OuhG2s4yp+1l8EjJp7N3DTkiAKy7wjzuoGo2S6dftv69ncNyb2BDnIGOTR4nLdyS8/bDdWyAz1Zfn3qPo9dtridhlU6x9R5kTjkPYVccLAbTpcAX89PSXtGBnzgOICHez+tcE3vAFMiMak9nborD3fB0cFz3d3ehBYOeDZxwAeHkvO2KfKd/kCU3dZOZRXeat5fPOn63a/aN4COOpGgD3XNWGKeusQP+/TxEMuKWODI4hZB0GUdKVMz46bY3z0PLnUpc1pMMMG5wbItijnl2vQHDsPqjFBVEGCAG262O50wGTVSc4e2c1fNhyu83XDL1rVkz2qkAEfvi3toyOaQj8jwQxvE3/6ijlnqVpiCoHu4MTzQd07HUvWbwWMBBMv5mF/6VSYQzQFbv3JRA2AfvkGXbWlEn+WKZRvFE8VyelKMoGxkHjKeUyL0SOicCYJCvvHG5gJtR+QxXNh5hSXW+pLHVf9yBNr2EwkHezbsyv2Vw1BSuFn3genuarxVLuyyZLFT5ufXTE5A8LRQ7YWb+DKflQQt3NAxiz2uIh+SD/Dj6mcKJCG3oydFgcF9MtPk7LmnBVxtiWj2 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(26005)(38100700002)(6512007)(36756003)(6506007)(66946007)(2616005)(186003)(6666004)(66476007)(8676002)(66556008)(8936002)(316002)(4326008)(5660300002)(86362001)(83380400001)(110136005)(54906003)(508600001)(6486002)(2906002)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: AWPeHTDLI1/IdCQk4z5ETjNaqR3xm600NtoUQlxvIsX0NYuvHb5FxDQ9vt2Nqy15h3MvBqWOYsXdQMoLxySVWZH1DJ6xw74Wdt11zQZIuqNkZb8jO9hJ9Np4xN+EGeZwMCf/NC2OFC0aDvFbzQdYfNyLyKlEaSDzZRnSQl2DGgIJ8Z1S3Pd+ePOhHWfGy+Xf5+hNSkSsl4Aq7U7eY7JM7qJWwVgSr0ReiLAm85JiUqohCd/JI2fuyHTi2e0s7+6HeBXa+1TuMlkLYRNL3c0SjSUHaLBjOzew5GYQDzBp3Vn/WnyWe1o9XJzhvOFKq0XI02Gmf1dsxBGm7kwNSBQ7BaRLO7X/X03QP2KOspTD7Tb02v+ud7VPyHErAcnm5HMGnx7DTHaJ5JoiEd0SRBgYOxpfeWEFrXih2Qvo7H0xyF5+uyhQTHaqMt7yLmNZ9lKRfnkbL3Ojz3KD15lCqpX3ale4g+Zw8LveYs8AFHbKL8foCmj3QTvZkhQtDW9SOc2dZ1FaqL7cDe1lGd5JjHGITv36HKHamZ7bqAKZQWsBHwd97ebVR+cSN0SoaTTbBbZ5MR09bi5ouvV552oyhjGyk7Ophcn7vfOFkRTn8HQ7oqQNBFL9+U9OGBMLFEQc7RU25acXHnxrH1dZZZL2E8t9ootevnRuYyjCsCNsviFSTQhlpI2WCg+WVPKfHbrQplNR9oPYdd174TDNfKZgh/HEnVw4l6otYL/4qmGhXkCNqA5B3o/gPAIkkEPkUleDrWiYRYEwx7l1Hdr1HxG60nyD0vYIBM5dKNR+jyoCXZtxURn57ijVrmemS4sjk7ZFGIwpRP5+FZAUjTk9aBoEcgYouG6YEQZ9ipLDpCzej7zj9HIISNc2A5U/5fLY6UV5Zft15vF8vC62v3+mN2BQdDwfMMJx8bYLPT1FzBV0MQUk8SAgbaAvbQRMsZ96u3YLqhwJXVfC3Wb2/1uYTMwW1rig4Jygfa5pB+GDI5IG9Q2PDKSvukbw2r96gqqFjzs+2nb8/tCrK/LLG2ZTBwtj4MORypdEGtPj28Wmx7hWgdinVsHcCAu752bSRch2zorEf0e9oDxWaPnrPX3H5qHysTp7IpNvcaGEnl00N3sjRlgvv3OJKqKosZF2ziJbI+HV0CJlX4ZQw4iU7e1yVdQljRDon5EUjlpSqBLEYxf/nyNnE97pPUtOwU80k1TUkbb4EJI6eeG7AvfRFr99IzswtobHXpNfTdFukDBW2KLHalG2AC/plEPt6TqYL+B5RZJjuNYZnQsndc+i/T25Th7VwO1T5f6kcrq2rkSyJnRMqD7W+Zl2XIWxdl+XySIeNSiRvCDbsYhm6qjWwMx69i6YVWhX+re5WQY59gJwrrDVRtgMbve1x5RPIcs/QvKebiZE31ZhWmJauMYxit/d2RDFtMtR08+rbtlFCoykbeVEQmfGUhDeG+jv9hoUbtlMeaZbHhq4NPiPuQNcAbyO1NFyGCVFXSswHPy3CowfivjmKbgwmJOiwyy28wCDWZX+AA0tNM+4zJg87CyO0/QNpB4CmV+G6c0K17+StO7Q7hwyhA772AqxrGcfOT1MGrt2EbYwTFUB5frH/Y/K5YiqkoqXmSB2/YYz/QaEX1kEw6Z8YfjMFeCo6cYpxqZ6eaXTA+9I4mQcMuka3AVqMkq2CV9cb3FO+q4JATKgJ/FxCz3o3cRuNRztt6bIRh2s3jaaFNpSTzl7ky7SqjJGPg4ohWe++fpK1g== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 439581c0-8970-4e0b-51b3-08da230339fd X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Apr 2022 19:23:19.2674 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 96vzdnlgJdQ2EkPI5+GHw4uFMo/9B/FciVD7nEuBKKc9PVsMx3JQiNXDFZBnYtPj X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4131 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org VFIO PCI does a security check as part of hot reset to prove that the user has permission to manipulate all the devices that will be impacted by the reset. Use a new API vfio_file_has_dev() to perform this security check against the struct file directly and remove the vfio_group from VFIO PCI. Since VFIO PCI was the last user of vfio_group_get_external_user() remove it as well. Signed-off-by: Jason Gunthorpe Reviewed-by: Kevin Tian Reviewed-by: Christoph Hellwig --- drivers/vfio/pci/vfio_pci_core.c | 42 ++++++++++----------- drivers/vfio/vfio.c | 63 +++++++++----------------------- include/linux/vfio.h | 2 +- 3 files changed, 40 insertions(+), 67 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index b7bb16f92ac628..465c42f53fd2fc 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -577,7 +577,7 @@ static int vfio_pci_fill_devs(struct pci_dev *pdev, void *data) struct vfio_pci_group_info { int count; - struct vfio_group **groups; + struct file **files; }; static bool vfio_pci_dev_below_slot(struct pci_dev *pdev, struct pci_slot *slot) @@ -1039,10 +1039,10 @@ long vfio_pci_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd, } else if (cmd == VFIO_DEVICE_PCI_HOT_RESET) { struct vfio_pci_hot_reset hdr; int32_t *group_fds; - struct vfio_group **groups; + struct file **files; struct vfio_pci_group_info info; bool slot = false; - int group_idx, count = 0, ret = 0; + int file_idx, count = 0, ret = 0; minsz = offsetofend(struct vfio_pci_hot_reset, count); @@ -1075,17 +1075,17 @@ long vfio_pci_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd, return -EINVAL; group_fds = kcalloc(hdr.count, sizeof(*group_fds), GFP_KERNEL); - groups = kcalloc(hdr.count, sizeof(*groups), GFP_KERNEL); - if (!group_fds || !groups) { + files = kcalloc(hdr.count, sizeof(*files), GFP_KERNEL); + if (!group_fds || !files) { kfree(group_fds); - kfree(groups); + kfree(files); return -ENOMEM; } if (copy_from_user(group_fds, (void __user *)(arg + minsz), hdr.count * sizeof(*group_fds))) { kfree(group_fds); - kfree(groups); + kfree(files); return -EFAULT; } @@ -1094,22 +1094,22 @@ long vfio_pci_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd, * user interface and store the group and iommu ID. This * ensures the group is held across the reset. */ - for (group_idx = 0; group_idx < hdr.count; group_idx++) { - struct vfio_group *group; - struct fd f = fdget(group_fds[group_idx]); - if (!f.file) { + for (file_idx = 0; file_idx < hdr.count; file_idx++) { + struct file *file = fget(group_fds[file_idx]); + + if (!file) { ret = -EBADF; break; } - group = vfio_group_get_external_user(f.file); - fdput(f); - if (IS_ERR(group)) { - ret = PTR_ERR(group); + /* Ensure the FD is a vfio group FD.*/ + if (!vfio_file_iommu_group(file)) { + fput(file); + ret = -EINVAL; break; } - groups[group_idx] = group; + files[file_idx] = file; } kfree(group_fds); @@ -1119,15 +1119,15 @@ long vfio_pci_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd, goto hot_reset_release; info.count = hdr.count; - info.groups = groups; + info.files = files; ret = vfio_pci_dev_set_hot_reset(vdev->vdev.dev_set, &info); hot_reset_release: - for (group_idx--; group_idx >= 0; group_idx--) - vfio_group_put_external_user(groups[group_idx]); + for (file_idx--; file_idx >= 0; file_idx--) + fput(files[file_idx]); - kfree(groups); + kfree(files); return ret; } else if (cmd == VFIO_DEVICE_IOEVENTFD) { struct vfio_device_ioeventfd ioeventfd; @@ -1964,7 +1964,7 @@ static bool vfio_dev_in_groups(struct vfio_pci_core_device *vdev, unsigned int i; for (i = 0; i < groups->count; i++) - if (groups->groups[i] == vdev->vdev.group) + if (vfio_file_has_dev(groups->files[i], &vdev->vdev)) return true; return false; } diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 7d0fad02936f69..ff5f6e0f285faa 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -1899,51 +1899,6 @@ static const struct file_operations vfio_device_fops = { .mmap = vfio_device_fops_mmap, }; -/* - * External user API, exported by symbols to be linked dynamically. - * - * The protocol includes: - * 1. do normal VFIO init operation: - * - opening a new container; - * - attaching group(s) to it; - * - setting an IOMMU driver for a container. - * When IOMMU is set for a container, all groups in it are - * considered ready to use by an external user. - * - * 2. User space passes a group fd to an external user. - * The external user calls vfio_group_get_external_user() - * to verify that: - * - the group is initialized; - * - IOMMU is set for it. - * If both checks passed, vfio_group_get_external_user() - * increments the container user counter to prevent - * the VFIO group from disposal before KVM exits. - * - * 3. When the external KVM finishes, it calls - * vfio_group_put_external_user() to release the VFIO group. - * This call decrements the container user counter. - */ -struct vfio_group *vfio_group_get_external_user(struct file *filep) -{ - struct vfio_group *group = filep->private_data; - int ret; - - if (filep->f_op != &vfio_group_fops) - return ERR_PTR(-EINVAL); - - ret = vfio_group_add_container_user(group); - if (ret) - return ERR_PTR(ret); - - /* - * Since the caller holds the fget on the file group->users must be >= 1 - */ - vfio_group_get(group); - - return group; -} -EXPORT_SYMBOL_GPL(vfio_group_get_external_user); - /* * External user API, exported by symbols to be linked dynamically. * The external user passes in a device pointer @@ -2056,6 +2011,24 @@ void vfio_file_set_kvm(struct file *file, struct kvm *kvm) } EXPORT_SYMBOL_GPL(vfio_file_set_kvm); +/** + * vfio_file_has_dev - True if the VFIO file is a handle for device + * @file: VFIO file to check + * @device: Device that must be part of the file + * + * Returns true if given file has permission to manipulate the given device. + */ +bool vfio_file_has_dev(struct file *file, struct vfio_device *device) +{ + struct vfio_group *group = file->private_data; + + if (file->f_op != &vfio_group_fops) + return false; + + return group == device->group; +} +EXPORT_SYMBOL_GPL(vfio_file_has_dev); + /* * Sub-module support */ diff --git a/include/linux/vfio.h b/include/linux/vfio.h index cbd9103b5c1223..e8be8ec40f2b50 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -140,13 +140,13 @@ int vfio_mig_get_next_state(struct vfio_device *device, /* * External user API */ -extern struct vfio_group *vfio_group_get_external_user(struct file *filep); extern void vfio_group_put_external_user(struct vfio_group *group); extern struct vfio_group *vfio_group_get_external_user_from_dev(struct device *dev); extern struct iommu_group *vfio_file_iommu_group(struct file *file); extern bool vfio_file_enforced_coherent(struct file *file); extern void vfio_file_set_kvm(struct file *file, struct kvm *kvm); +extern bool vfio_file_has_dev(struct file *file, struct vfio_device *device); #define VFIO_PIN_PAGES_MAX_ENTRIES (PAGE_SIZE/sizeof(unsigned long))