From patchwork Fri May 6 00:08:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12840349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 624E1C433FE for ; Fri, 6 May 2022 00:08:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1387369AbiEFAMg (ORCPT ); Thu, 5 May 2022 20:12:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47476 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1387341AbiEFAMd (ORCPT ); Thu, 5 May 2022 20:12:33 -0400 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2048.outbound.protection.outlook.com [40.107.236.48]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B98C54FC51; Thu, 5 May 2022 17:08:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Qsb5poXawrXlOPYibLl5vJvawZWMpifqXTCoTkUW9FiUEmRv4UjFU0q0FblAJkBb2heyufS943e6jT3DAbzhQAPkUmgDb3C4qn0sj1AYAVxlvx+EbM63rpQQdptL59X1upIwG3tSgjCt5pqQWnsaALGawRZ78bR+sGaxbiR/R/w0IN0Qb01La074BrJQLmFCmMTfcss0W3P4M7iXv9j/TIP548Fl3Mj8mNhNv40OTp3kbXzBQbVmQY/SwhTzUJbLf1JaRLWYWTIAWnckuv0zFHkGAAuH81zZnsvSRA6WOrOPFW6PbPRgTuwVC4k0VPI2fJZQLnAnh7LibemKlRKYOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rMPVTVXzR088Hrf3D1a0rC9YtKaLbMYSxuJPfXntvlo=; b=eqwlGKXPlw2FaYgDVWs5TGNot/hfhNoV00YdJMDcQcU8CSxtKMab+mwX2r5MZemHJrbAhrXssGliu1gPqcTNwZvRbmKYnmP78nMB96lr2l3eJaC6fB0moO54yVPitGG7KSca+kIob0Fzqfk0OEchBNQbGkqn6Y1j0FlY6ya1kX8dnTJmqcN2gq3cFetg9xZYyjKjGOjliAJn5cTzAjyT0XDTfzRkJZZsCr+wust7Espbim8LzJJdK5h7b9TgslHbpL4ED0oNOCCziaoYbx4d9NoLc9f6dMqbS5pSEgkitJHCaoq0/0pik8igBmBkl7KoNm6Jdc/9WnJ+TwLmTvnMCA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rMPVTVXzR088Hrf3D1a0rC9YtKaLbMYSxuJPfXntvlo=; b=NxgrLzwkDm5XkQ0p8KjI0yhXXhtqW91X93K5QAqNcmHOYksD+GRe2vO065wq34hnqxh0twzObjAMux8J7NMyId45Qbo7Kxl76RBfQpRwzoarxliS2HUsgKV7+BjZHp8/0hcQNMKVflJtVEsUhsbknaolhM3TO39n5ci9tcb3HixXpg1v9Tk8ZFVzv+thE/Kh2Ctq7WWJzWGbrffqiITBCiHpoQAZEYIWtuxS+1DVNp9tf2taw0hhVMyGPW/wBzWWRT7mGF/IQeccxBNgiECEPYp+2NfoS1KIELHKsLBNKSQNEw9JGPOCxJp7niumHyYB5S6maxM9DGhR02VuFIzRdw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BYAPR12MB2934.namprd12.prod.outlook.com (2603:10b6:a03:13b::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.24; Fri, 6 May 2022 00:08:48 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5206.027; Fri, 6 May 2022 00:08:48 +0000 From: Jason Gunthorpe To: Alexander Gordeev , David Airlie , Alex Williamson , Christian Borntraeger , Cornelia Huck , Jonathan Corbet , Daniel Vetter , dri-devel@lists.freedesktop.org, Harald Freudenberger , Vasily Gorbik , Heiko Carstens , intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org, Jani Nikula , Jason Herne , Joonas Lahtinen , kvm@vger.kernel.org, Kirti Wankhede , linux-doc@vger.kernel.org, linux-s390@vger.kernel.org, Matthew Rosato , Peter Oberparleiter , Halil Pasic , Rodrigo Vivi , Sven Schnelle , Tvrtko Ursulin , Vineeth Vijayan , Zhenyu Wang , Zhi Wang Cc: Tony Krowiak , Eric Farman , Christoph Hellwig , Kevin Tian , "Liu, Yi L" Subject: [PATCH v4 1/7] vfio: Make vfio_(un)register_notifier accept a vfio_device Date: Thu, 5 May 2022 21:08:39 -0300 Message-Id: <1-v4-8045e76bf00b+13d-vfio_mdev_no_group_jgg@nvidia.com> In-Reply-To: <0-v4-8045e76bf00b+13d-vfio_mdev_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR05CA0029.namprd05.prod.outlook.com (2603:10b6:208:91::39) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b28a964d-5178-496e-9491-08da2ef496f2 X-MS-TrafficTypeDiagnostic: BYAPR12MB2934:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 8aRt0nHF2e8NgMtoKuWIkxI2KMfS9QXpV75ofJzPxVd+OoQp3tWxb9f7ANvn2sR9DE94cFS5YcXTllcc/y3V3eLh2M6kjXunSmkmcFsnfGVRNnOMsL4ji0HFmDMc0xgXI+vjfaSWhfORIlGma6cSucwAMvpmgv5HgMulyQe1zsnqUBU5AJn8/Yk3pwIi8LHZwX7nUefDx+gGURbLM1hTrN6tGbL3f111vSOnRnw8yN3Xqmg1ebFrsopjUL9CIIm75W01ahnqb/8U1GK7CNuoPejDKCVar3ZNx6f9Miyn++71dAo3H/n3O/QQRIuZN3wl/UNaguvkdgB5i36ahGwrMph8Cjjt+Cz2x6qPHZI2hCJPGJrLHTxzsdZ+uBG7e2504SdG3l68dTt4CeIkaq3od15+ozjb4KNgtxe08INnMMxxbAa3Kg+WlliU+JFypDm+9z1xVVGRMXq4aV7Kc8ZPqBK96gCj1yvWNngmZqpmYQHU6aUO/524pDH82lE5K6vFz5Y773wG4Cp+IaiSqnQlKsCn5cio5hpjfGgiXMrabf7ax+qEnb+2ZH+kscpWOm51aQkBYjz+a7dNvjWA+LbBpZ5gXJXHtwrxyu0zV1qQGbUXBu0hOkSOu1qn3fWx7nkXxS7bFfHUAgHAz63Ur/0sWPMBOJV3efwixpQ2ncEbmZ+Qk/xgSXMsEZgVRWDTWxzo8+OHKBR6S5HhY1672Yn6fw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(5660300002)(2906002)(66946007)(66556008)(54906003)(8676002)(86362001)(110136005)(4326008)(6486002)(38100700002)(508600001)(316002)(66476007)(6666004)(6512007)(6506007)(186003)(2616005)(921005)(26005)(7416002)(8936002)(83380400001)(7406005)(36756003)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: JFY2fURJhqqQRNrNC3amN8aOh9iD5v0SKLb1fMAHwwZqeDjQwheyJ8oTC/H9hVe4q2C71iQo60oXWdmcA40xoiE/8yG1N2EwgB8eTV+QlZcxIQvnMNDsS8WBDf+99wEmXyDEQaIkRjwJYjK5HWP/N3TfmDYv1043P/W/0/n3b2otzOfFG0SGEKJV+auVQOiXf72HO230HByBQZpvw6+FYVuhGBAUyt+yue08fCdqI2WIic3BFWGxRMGJXbvhlBJHNsnlmSpvpsrgNk5pSYnrDUsR8E9xeQe+uxJOGH5f/8gLQZAzz34OVrr5+Gx6PIVcIjuBDkQVyWpia9i/8oh0Q3yL8ES9nVdbbuXg9PSblc7KznDIWiix9+uzc6gOoSZfI9D7Gco1Mwbm1D1fhlhO730v493yOH6pWYVrgmQWm8+RNWHKthww9qxTMhfouYANmX4hEOFwUeaybZEYgSXwRh3RCkuGkbrJfICUgPTpFpF8jYq9fwmYXZ3waVbQOootAXcuVkanipj3Hr9wineLPd4zZSRIGmJXNwNn1xWYZZsP44OBq+VvZahTH2gHlZbtQOx2S9dn10BE/Gat7ftA/gIPLR66VBBaRf/q3hy+35SmnIWOwaAWdajeBoz8zbJ6Ld5n96XAyLD9Cel7ht4jz9kFfqHw+d/136Hi84rqI88Zck/CwawJTVEhZayavzeTyCD0vSMRyqC+Bt+diRCOPK9eO9tyV902LCjiUizEX6+aGOyVMjXI4Szj3UXLBWDhYfEvjsUMOSsSv2AxEo9gKGORTwXn34PSCG2/5dBy/fcvtLbikfq89jOFHaMWqszCnoBgOkOmLfx7bcfk2z8r04SBJfd+On+CzBBRUd1Ps9E3jL4H9skR5fx9ZQFYhW+AEWhtTRZRNtAP5pNm6trs5y18Eqvwb32lDBRmZcdeuUhFR1NUI+aqWZWWW+PqIEllTOxTUArBYasaEqHxmu3To3wEN+SN4NTy8Q9E8ONjvz/RnfS2i4fTC8pMohGxsXnhLPsjamuP/Hq2Jd2oG2ZhWykUFXLP+mCk9OiCCrgASKn+3Mt6Lps6cIf++7d3VhjrqKZP41ZrkyaVBgZsNa9qvvG10qv1q3LjbdDo94NX9YSylLms7hUqovwL/7Ypxzy5kaiXAqqIUD8Y+BdgL+hOR59gJJ8srb8Vmxp0egOx+Tg4BdvKKeZIF5Zfd1hiYevyAjIvuuLw7Leu/cAgS/V5tHXKVSlivYuh0+n7seoLEZvwG25w9Ic7yOaj2u+Y6sV98UCxODuvRGgCDVuKuGcnWmH/iWVZe030cQ3B7UlcV/r1gXyXQsbiIee1JrY3IxuVrB8RjE6JhUjHp8esrZkkmdgYTh2xNmIdzodEmDcH6JGOI6peSne3E9Lb76fadLoxZO8XD2iNOSQeh8ehvE3cr7pD3OsCHGxSOoyJx5dYzOBAt6aKLwAk4S/DHCkwvKbtb4GH/S9sb+s+YZ+oxG/2bpVcBw/D8HngWwFHLyB6YonSQvJaxTZu23pUpEBEJP5ggcM3vX1hEZEKmUYn75/sOha7/cEvDtD+qdOMcwt2lhdDZWc68LBgTa0lcSuzgUubi/iJZIvg82gKgknm33uNoeBD6QR5Z77sbbaHx1vFtPyWaLVgzgPM6e4vYpmsL6/euSWO8qlSjOI1Zh/s/MnB2TFDevIUOmkkqy7cCc3B1VR6Ye4EnnXYUGVivcwJD62L7EyjEpw7ii8lyGR9El9I6Q== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: b28a964d-5178-496e-9491-08da2ef496f2 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2022 00:08:46.9186 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Snug5YQW140BLekvpue18kfz14cL8tdi7uvZPDFkQdOvolh56J7h6HRmhL0LUXN/ X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB2934 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org All callers have a struct vfio_device trivially available, pass it in directly and avoid calling the expensive vfio_group_get_from_dev(). Acked-by: Eric Farman Reviewed-by: Jason J. Herne Reviewed-by: Tony Krowiak Reviewed-by: Kevin Tian Reviewed-by: Christoph Hellwig Signed-off-by: Jason Gunthorpe --- drivers/gpu/drm/i915/gvt/kvmgt.c | 24 ++++++++++++------------ drivers/s390/cio/vfio_ccw_ops.c | 7 +++---- drivers/s390/crypto/vfio_ap_ops.c | 14 +++++++------- drivers/vfio/vfio.c | 28 +++++++++------------------- include/linux/vfio.h | 4 ++-- 5 files changed, 33 insertions(+), 44 deletions(-) diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c index 0787ba5c301f5e..1cec4f1fdfaced 100644 --- a/drivers/gpu/drm/i915/gvt/kvmgt.c +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c @@ -810,8 +810,8 @@ static int intel_vgpu_open_device(struct vfio_device *vfio_dev) vgpu->group_notifier.notifier_call = intel_vgpu_group_notifier; events = VFIO_IOMMU_NOTIFY_DMA_UNMAP; - ret = vfio_register_notifier(vfio_dev->dev, VFIO_IOMMU_NOTIFY, &events, - &vgpu->iommu_notifier); + ret = vfio_register_notifier(vfio_dev, VFIO_IOMMU_NOTIFY, &events, + &vgpu->iommu_notifier); if (ret != 0) { gvt_vgpu_err("vfio_register_notifier for iommu failed: %d\n", ret); @@ -819,8 +819,8 @@ static int intel_vgpu_open_device(struct vfio_device *vfio_dev) } events = VFIO_GROUP_NOTIFY_SET_KVM; - ret = vfio_register_notifier(vfio_dev->dev, VFIO_GROUP_NOTIFY, &events, - &vgpu->group_notifier); + ret = vfio_register_notifier(vfio_dev, VFIO_GROUP_NOTIFY, &events, + &vgpu->group_notifier); if (ret != 0) { gvt_vgpu_err("vfio_register_notifier for group failed: %d\n", ret); @@ -873,12 +873,12 @@ static int intel_vgpu_open_device(struct vfio_device *vfio_dev) vgpu->vfio_group = NULL; undo_register: - vfio_unregister_notifier(vfio_dev->dev, VFIO_GROUP_NOTIFY, - &vgpu->group_notifier); + vfio_unregister_notifier(vfio_dev, VFIO_GROUP_NOTIFY, + &vgpu->group_notifier); undo_iommu: - vfio_unregister_notifier(vfio_dev->dev, VFIO_IOMMU_NOTIFY, - &vgpu->iommu_notifier); + vfio_unregister_notifier(vfio_dev, VFIO_IOMMU_NOTIFY, + &vgpu->iommu_notifier); out: return ret; } @@ -907,13 +907,13 @@ static void __intel_vgpu_release(struct intel_vgpu *vgpu) intel_gvt_release_vgpu(vgpu); - ret = vfio_unregister_notifier(vgpu->vfio_device.dev, VFIO_IOMMU_NOTIFY, - &vgpu->iommu_notifier); + ret = vfio_unregister_notifier(&vgpu->vfio_device, VFIO_IOMMU_NOTIFY, + &vgpu->iommu_notifier); drm_WARN(&i915->drm, ret, "vfio_unregister_notifier for iommu failed: %d\n", ret); - ret = vfio_unregister_notifier(vgpu->vfio_device.dev, VFIO_GROUP_NOTIFY, - &vgpu->group_notifier); + ret = vfio_unregister_notifier(&vgpu->vfio_device, VFIO_GROUP_NOTIFY, + &vgpu->group_notifier); drm_WARN(&i915->drm, ret, "vfio_unregister_notifier for group failed: %d\n", ret); diff --git a/drivers/s390/cio/vfio_ccw_ops.c b/drivers/s390/cio/vfio_ccw_ops.c index c4d60cdbf247bf..b49e2e9db2dc6f 100644 --- a/drivers/s390/cio/vfio_ccw_ops.c +++ b/drivers/s390/cio/vfio_ccw_ops.c @@ -183,7 +183,7 @@ static int vfio_ccw_mdev_open_device(struct vfio_device *vdev) private->nb.notifier_call = vfio_ccw_mdev_notifier; - ret = vfio_register_notifier(vdev->dev, VFIO_IOMMU_NOTIFY, + ret = vfio_register_notifier(vdev, VFIO_IOMMU_NOTIFY, &events, &private->nb); if (ret) return ret; @@ -204,8 +204,7 @@ static int vfio_ccw_mdev_open_device(struct vfio_device *vdev) out_unregister: vfio_ccw_unregister_dev_regions(private); - vfio_unregister_notifier(vdev->dev, VFIO_IOMMU_NOTIFY, - &private->nb); + vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &private->nb); return ret; } @@ -223,7 +222,7 @@ static void vfio_ccw_mdev_close_device(struct vfio_device *vdev) cp_free(&private->cp); vfio_ccw_unregister_dev_regions(private); - vfio_unregister_notifier(vdev->dev, VFIO_IOMMU_NOTIFY, &private->nb); + vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &private->nb); } static ssize_t vfio_ccw_mdev_read_io_region(struct vfio_ccw_private *private, diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c index ee0a3bf8f476ca..bfa7ee6ef532d9 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -1406,21 +1406,21 @@ static int vfio_ap_mdev_open_device(struct vfio_device *vdev) matrix_mdev->group_notifier.notifier_call = vfio_ap_mdev_group_notifier; events = VFIO_GROUP_NOTIFY_SET_KVM; - ret = vfio_register_notifier(vdev->dev, VFIO_GROUP_NOTIFY, - &events, &matrix_mdev->group_notifier); + ret = vfio_register_notifier(vdev, VFIO_GROUP_NOTIFY, &events, + &matrix_mdev->group_notifier); if (ret) return ret; matrix_mdev->iommu_notifier.notifier_call = vfio_ap_mdev_iommu_notifier; events = VFIO_IOMMU_NOTIFY_DMA_UNMAP; - ret = vfio_register_notifier(vdev->dev, VFIO_IOMMU_NOTIFY, - &events, &matrix_mdev->iommu_notifier); + ret = vfio_register_notifier(vdev, VFIO_IOMMU_NOTIFY, &events, + &matrix_mdev->iommu_notifier); if (ret) goto out_unregister_group; return 0; out_unregister_group: - vfio_unregister_notifier(vdev->dev, VFIO_GROUP_NOTIFY, + vfio_unregister_notifier(vdev, VFIO_GROUP_NOTIFY, &matrix_mdev->group_notifier); return ret; } @@ -1430,9 +1430,9 @@ static void vfio_ap_mdev_close_device(struct vfio_device *vdev) struct ap_matrix_mdev *matrix_mdev = container_of(vdev, struct ap_matrix_mdev, vdev); - vfio_unregister_notifier(vdev->dev, VFIO_IOMMU_NOTIFY, + vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &matrix_mdev->iommu_notifier); - vfio_unregister_notifier(vdev->dev, VFIO_GROUP_NOTIFY, + vfio_unregister_notifier(vdev, VFIO_GROUP_NOTIFY, &matrix_mdev->group_notifier); vfio_ap_mdev_unset_kvm(matrix_mdev); } diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 0c766384cee0f8..d70890ab9cea92 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -2269,19 +2269,16 @@ static int vfio_unregister_group_notifier(struct vfio_group *group, return ret; } -int vfio_register_notifier(struct device *dev, enum vfio_notify_type type, - unsigned long *events, struct notifier_block *nb) +int vfio_register_notifier(struct vfio_device *device, + enum vfio_notify_type type, unsigned long *events, + struct notifier_block *nb) { - struct vfio_group *group; + struct vfio_group *group = device->group; int ret; - if (!dev || !nb || !events || (*events == 0)) + if (!nb || !events || (*events == 0)) return -EINVAL; - group = vfio_group_get_from_dev(dev); - if (!group) - return -ENODEV; - switch (type) { case VFIO_IOMMU_NOTIFY: ret = vfio_register_iommu_notifier(group, events, nb); @@ -2292,25 +2289,20 @@ int vfio_register_notifier(struct device *dev, enum vfio_notify_type type, default: ret = -EINVAL; } - - vfio_group_put(group); return ret; } EXPORT_SYMBOL(vfio_register_notifier); -int vfio_unregister_notifier(struct device *dev, enum vfio_notify_type type, +int vfio_unregister_notifier(struct vfio_device *device, + enum vfio_notify_type type, struct notifier_block *nb) { - struct vfio_group *group; + struct vfio_group *group = device->group; int ret; - if (!dev || !nb) + if (!nb) return -EINVAL; - group = vfio_group_get_from_dev(dev); - if (!group) - return -ENODEV; - switch (type) { case VFIO_IOMMU_NOTIFY: ret = vfio_unregister_iommu_notifier(group, nb); @@ -2321,8 +2313,6 @@ int vfio_unregister_notifier(struct device *dev, enum vfio_notify_type type, default: ret = -EINVAL; } - - vfio_group_put(group); return ret; } EXPORT_SYMBOL(vfio_unregister_notifier); diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 66dda06ec42d1b..a00fd722f044b0 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -178,11 +178,11 @@ enum vfio_notify_type { /* events for VFIO_GROUP_NOTIFY */ #define VFIO_GROUP_NOTIFY_SET_KVM BIT(0) -extern int vfio_register_notifier(struct device *dev, +extern int vfio_register_notifier(struct vfio_device *device, enum vfio_notify_type type, unsigned long *required_events, struct notifier_block *nb); -extern int vfio_unregister_notifier(struct device *dev, +extern int vfio_unregister_notifier(struct vfio_device *device, enum vfio_notify_type type, struct notifier_block *nb); From patchwork Fri May 6 00:08:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12840353 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96788C4167D for ; Fri, 6 May 2022 00:09:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1387408AbiEFAMn (ORCPT ); Thu, 5 May 2022 20:12:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47526 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1387379AbiEFAMh (ORCPT ); Thu, 5 May 2022 20:12:37 -0400 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2048.outbound.protection.outlook.com [40.107.236.48]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E706350B2F; Thu, 5 May 2022 17:08:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MoZnEq4ZBnsaCKdkRX0K3djof5jfGvjal+lhAKcfBEsxIAVpoKMxNekmkVnRs/aMzCkApW//xxFCDEUkp8613JNyHXnY41rgUOOoqJCO4Cp0ru/7yCArQUe/RDo66LGhfKID6X7Rq2VdbewnauQJdyhG+FAwJ7TB1HEFcg8JJIMuxrhwNtVfjF/No12vnCABzo31FmMGLeUA4kOWU4I32OuGYgfyh8nHzKIhKX9bRJuOFeyVP5x6hbLqTsbhoMuIeBPtfhDC0/A/f4ShFnW45UD/8my32Cp15FzwpeqWaNM8PKPxXOu8nAKoOLOURq1mfuoATnA0LDIcSQh8SkPoBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8LrDPKMVIsusxztPgCpb23x63uX0ihT59Q0QLuOEqwE=; b=KgMASJvzVw8UNGT2/7nPgNuh/xzTxe4CEcvwgrSSmgf81ET3C2BUiUgdLv1cO1W4eM3q9FnTRyrXxgXmDrpOF0tnYKIk1Pg1DTrxgbMy1GtMTXOgwa7Cz5uixjddBS1kvC8WeS+PPKFMJU3alTFb0ChrW4EZrdYjaKh1f7K+0/xAnNjQA3StxdS8wOFuNPVPHT16XHxwvzEQUQNUvom6l3Fo5eP60JOGLdOxPfmaQe4xDXivCmxLUFsWmCp/TRvG0meLTVrc9b3N79kIvHMiLZ2fVx8xmNjsc2XnCLIAuw4U97g9UrDE103UDBiZXRTMfremetY3uquormpySpnowg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8LrDPKMVIsusxztPgCpb23x63uX0ihT59Q0QLuOEqwE=; b=dgnqklOS7ffbpTGI8SXtd0gyoJ+cUh6w+kYufmiHkqzRjLj5Quc0+eUbE3wab7D++QO4cGumcpkgbmsNFe4uhBiRmdJhQzIMRP3v9a/kRekeF3f63E0HANeAUXcl2QGItEm6E4XTXrlew9l+wjWNV54PtSkuKEjAZwsrgU9WN64PHCgBNH7JV2QDejwM6wdluw+yaA/glvuZeUSSHf8a74dASol9MrlXkK2GrN5F6WF307ehfUMR+/YTl/EDuETVTa1cJt76rexgjhYS3ExlizWeHNRm1XFB3Y+W8J1TM8gikTYc0CRSt6/hPUFvVPjs8ZX5gSU63x5T9HEQBEclVw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BYAPR12MB2934.namprd12.prod.outlook.com (2603:10b6:a03:13b::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.24; Fri, 6 May 2022 00:08:51 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5206.027; Fri, 6 May 2022 00:08:51 +0000 From: Jason Gunthorpe To: Alexander Gordeev , David Airlie , Alex Williamson , Christian Borntraeger , Cornelia Huck , Jonathan Corbet , Daniel Vetter , dri-devel@lists.freedesktop.org, Harald Freudenberger , Vasily Gorbik , Heiko Carstens , intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org, Jani Nikula , Jason Herne , Joonas Lahtinen , kvm@vger.kernel.org, Kirti Wankhede , linux-doc@vger.kernel.org, linux-s390@vger.kernel.org, Matthew Rosato , Peter Oberparleiter , Halil Pasic , Rodrigo Vivi , Sven Schnelle , Tvrtko Ursulin , Vineeth Vijayan , Zhenyu Wang , Zhi Wang Cc: Tony Krowiak , Eric Farman , Christoph Hellwig , Kevin Tian , "Liu, Yi L" Subject: [PATCH v4 2/7] vfio/ccw: Remove mdev from struct channel_program Date: Thu, 5 May 2022 21:08:40 -0300 Message-Id: <2-v4-8045e76bf00b+13d-vfio_mdev_no_group_jgg@nvidia.com> In-Reply-To: <0-v4-8045e76bf00b+13d-vfio_mdev_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR05CA0004.namprd05.prod.outlook.com (2603:10b6:208:91::14) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 087555bd-4d89-4c16-7d3c-08da2ef497c9 X-MS-TrafficTypeDiagnostic: BYAPR12MB2934:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: M39zZsiYk3FNh+CWIayoFH3zcqKPxru8Yjtijpd/1kDfkZtuzpvH1u7wZVng5DYys8wOzksMN6h9b0C+xs+HCcuV7uqNytfxoFV9YzSa2cVqfXC+0zSH3jfexHFpFAvWtII4oifJI2b1KRbLTsQvkVh8TMyVMKdKONENNR5J5Y6k74MhYQrFRhtVtfdX3nIMfU2rhp8YLgcy4HKJVcPtPdeQss5V7lp9G7LXaGWY29aaBkyv8C6LA1ULfmLLvRi4l5XdqnEUnQ5sTAu0P42MTR+la2k49e3lVFoPkyCEKcs92xrsETBlta2TZ3ag2MomFDC1BcNdTOP/Q5+F4Kq5K6SjR3OzprB1HAE0lsJsE12xdcahWB0SR+yfMqqIYE6hgermUe/iC1HZYc9t/fGf39zemMKDTHh/5/72Y6nAY0IbdELQQYp9LrgYT4fmX/fybCDyBgcyeKn4FNvwYDZcJAQrb0laxTvBMW3EtuYNPL28JKRG8B5OhRN8YLGR46W1OoatkA9TOMujuzAEdUqAdsgZsMA4MszjijHc7EDkLA8pUvo6GQF+6qC3Fv9hkKBQeKNKC1TxqNwGgxF49AypdgxUCdbxV1hQSPmbMYuKDhC1VxvpjtbGbOqIsDKhf2qTrl8I/Gv11LAMC/JsNW5byq+eoBD+UbHSQtFHESid0ZrjXrdYYNjqat/LKkT4y/j4H1WBHKG9YXS+SBjVwPN97A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(5660300002)(2906002)(66946007)(66556008)(54906003)(8676002)(86362001)(110136005)(4326008)(6486002)(38100700002)(508600001)(316002)(66476007)(6666004)(6512007)(6506007)(186003)(2616005)(921005)(26005)(7416002)(8936002)(83380400001)(7406005)(36756003)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: eKkbs0DKkj4GddW3uTpVItuEMOkntqLIbFygBrsDMHzeSL6ud32AdWQodV9faLfcDugeDGpjfnpHt9zGTunS80lCH9fgxgWyCJGAYuL8f6agiKIIGOQbf0Zh82oOWG83HuBiv4UP0ospSUDsHmiodui15PaXWaKBNuN9ZteTMKUaVO32PCZL8IPAurgKvxF9NqvWq5Au5ZzL+c3+hcxTDx6Mo6R4V6db7iLLbX3OIQpawFUVhR7vqrWlHghGHYooUm3ryKUQtOFkefjZIoaUWPBHHFP5IX2XFdcsOr4c84y8HPow7aPvxHJvnzyofSEATBx6zljbHCnHQZGLmlBe18hs1o/7tbwqHn11wcmfAC0VqbqH0s0b8BfppwwJNr6DDlDqdsKj34TCEIU2Ytvq8GiB6ipEshF/9DRt+N0DPv80EXV0lvehabv3Hif00lGYKXMz6NFAcyeNlT8QcsOSDINC57SSbNmeTH1pxoMnym+ynyrArAvWoaEduQDdwhLlPVrzIGI3ah6ytf2efjUAlhRtkUqhrcG62PhsrbRMp5V9va9/gZKtsTeV7Qc8WaF2xRjJJbpSp2+H5v5lnCPfI7bV32UtfLZ82rHQzj0x1l+ZQTIrcRqYkxgZjwpIrum/cmhhXj8vrvFpRfTCLMRm6QNeWFDavYPmRFeiohlSpTKJqPBERWUcWKZBO3KGUjST8uZKfncG+efsyGqV6orrgqJw79mcVqBjdtTCaEEaD6QxX8Zdd26Gvv1yXfHgVFwgGqeRdmDh5RAC8XDW6V1SCXUwG9pfK+D6oFHac6mD/rYAHs7uSWPRqiFv1vi0Mz4dqZRMIIKmyqKjHMEaCPcz/SGpTAGs7aABl9Pn3aMLI1CzfCWcVE15fVqyfcGRq4wLWqQZ65I5Ww4IkRZTjn9qSuboeZc5UPjHUqDTZYzVQ0WD0eV9lIzUtmzCEJtolY/jyvVzKYSBaTql34G9wIwIWwl2DuuX/uox0RCcm+SUcNmBXE+nWPayd5kV8vPgWdiKhict+UBDi9So9ca5mB+ErYrCGQoCF/5X2dxdaoOcX/FNUtqxAn8VCOnPeY0dj/o57NAqqyPgekCf8CvnFeL0LbB2IT+XqeIIsdN+2puNAV+kPWMduIZNSzhBNi2/zvsVHg8Ds9IjvwwwGY9tvz5LR2xK5+2NEGEpx/9Cz14eTQVOHgrb/FbZP3YhB6kn4gkwTeJhXDvoZXaf5H2dDZBBW3e0sLzZVIU/rK00oEjZL90PEucdLGTTuAkT/o2DJFwvFUcCxMvNOUmzupyRBEUYmYujOuc28vbJOaGMB7aNNCUWdMrxsHaynM4SEbB7XnTqlOY4c5H0MbzfwbysQvzsNWEWjBcNzuazEnhi6+LQNkHEilrNef7feVVsiJibs0B8ZT9ob7IjXzjfxZcqK/bc5XFjA6mbYfS/7QDN6QBg5sH8UwjfrDk6D2bGPmnbRgqkrakN1Xl5JBXLgdEHAQJrDiEYfASEB3wSmQy8Xbc6dm8TV8wEwWHqQzye7MkaSPL3L1oP0wmpcpLUkHFtCe+BO46kbYHvtubsGD7atzBBKnNOGF0j0RwuW5wEyIIpmepZfrdcovVPl9YYSWY6wU1ntXNpotEf+i6xKEUmR8NBIcpndK2HE8QCTDhYWHHGHIKUDQ9ssWyDQDQU6PSANrN1TVFzf9HsSs0R1mGDjpt4ofqOT0LDX+UVM3Soiig/7nMi/+vjBshkMtKlKyK5xutWLw== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 087555bd-4d89-4c16-7d3c-08da2ef497c9 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2022 00:08:48.1683 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: m9S6D0nZc6MM8/EkrjyKwONzc3wnKf2FhUW8d3CLyizAcM/CKKWKnrWFm+Rm62Y3 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB2934 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The next patch wants the vfio_device instead. There is no reason to store a pointer here since we can container_of back to the vfio_device. Reviewed-by: Eric Farman Signed-off-by: Jason Gunthorpe --- drivers/s390/cio/vfio_ccw_cp.c | 47 ++++++++++++++++++++------------- drivers/s390/cio/vfio_ccw_cp.h | 4 +-- drivers/s390/cio/vfio_ccw_fsm.c | 3 +-- 3 files changed, 30 insertions(+), 24 deletions(-) diff --git a/drivers/s390/cio/vfio_ccw_cp.c b/drivers/s390/cio/vfio_ccw_cp.c index 8d1b2771c1aa02..7a1cf3091cd647 100644 --- a/drivers/s390/cio/vfio_ccw_cp.c +++ b/drivers/s390/cio/vfio_ccw_cp.c @@ -16,6 +16,7 @@ #include #include "vfio_ccw_cp.h" +#include "vfio_ccw_private.h" struct pfn_array { /* Starting guest physical I/O address. */ @@ -98,17 +99,17 @@ static int pfn_array_alloc(struct pfn_array *pa, u64 iova, unsigned int len) * If the pin request partially succeeds, or fails completely, * all pages are left unpinned and a negative error value is returned. */ -static int pfn_array_pin(struct pfn_array *pa, struct device *mdev) +static int pfn_array_pin(struct pfn_array *pa, struct vfio_device *vdev) { int ret = 0; - ret = vfio_pin_pages(mdev, pa->pa_iova_pfn, pa->pa_nr, + ret = vfio_pin_pages(vdev->dev, pa->pa_iova_pfn, pa->pa_nr, IOMMU_READ | IOMMU_WRITE, pa->pa_pfn); if (ret < 0) { goto err_out; } else if (ret > 0 && ret != pa->pa_nr) { - vfio_unpin_pages(mdev, pa->pa_iova_pfn, ret); + vfio_unpin_pages(vdev->dev, pa->pa_iova_pfn, ret); ret = -EINVAL; goto err_out; } @@ -122,11 +123,11 @@ static int pfn_array_pin(struct pfn_array *pa, struct device *mdev) } /* Unpin the pages before releasing the memory. */ -static void pfn_array_unpin_free(struct pfn_array *pa, struct device *mdev) +static void pfn_array_unpin_free(struct pfn_array *pa, struct vfio_device *vdev) { /* Only unpin if any pages were pinned to begin with */ if (pa->pa_nr) - vfio_unpin_pages(mdev, pa->pa_iova_pfn, pa->pa_nr); + vfio_unpin_pages(vdev->dev, pa->pa_iova_pfn, pa->pa_nr); pa->pa_nr = 0; kfree(pa->pa_iova_pfn); } @@ -190,8 +191,7 @@ static void convert_ccw0_to_ccw1(struct ccw1 *source, unsigned long len) * Within the domain (@mdev), copy @n bytes from a guest physical * address (@iova) to a host physical address (@to). */ -static long copy_from_iova(struct device *mdev, - void *to, u64 iova, +static long copy_from_iova(struct vfio_device *vdev, void *to, u64 iova, unsigned long n) { struct pfn_array pa = {0}; @@ -203,9 +203,9 @@ static long copy_from_iova(struct device *mdev, if (ret < 0) return ret; - ret = pfn_array_pin(&pa, mdev); + ret = pfn_array_pin(&pa, vdev); if (ret < 0) { - pfn_array_unpin_free(&pa, mdev); + pfn_array_unpin_free(&pa, vdev); return ret; } @@ -226,7 +226,7 @@ static long copy_from_iova(struct device *mdev, break; } - pfn_array_unpin_free(&pa, mdev); + pfn_array_unpin_free(&pa, vdev); return l; } @@ -423,11 +423,13 @@ static int ccwchain_loop_tic(struct ccwchain *chain, static int ccwchain_handle_ccw(u32 cda, struct channel_program *cp) { + struct vfio_device *vdev = + &container_of(cp, struct vfio_ccw_private, cp)->vdev; struct ccwchain *chain; int len, ret; /* Copy 2K (the most we support today) of possible CCWs */ - len = copy_from_iova(cp->mdev, cp->guest_cp, cda, + len = copy_from_iova(vdev, cp->guest_cp, cda, CCWCHAIN_LEN_MAX * sizeof(struct ccw1)); if (len) return len; @@ -508,6 +510,8 @@ static int ccwchain_fetch_direct(struct ccwchain *chain, int idx, struct channel_program *cp) { + struct vfio_device *vdev = + &container_of(cp, struct vfio_ccw_private, cp)->vdev; struct ccw1 *ccw; struct pfn_array *pa; u64 iova; @@ -526,7 +530,7 @@ static int ccwchain_fetch_direct(struct ccwchain *chain, if (ccw_is_idal(ccw)) { /* Read first IDAW to see if it's 4K-aligned or not. */ /* All subsequent IDAws will be 4K-aligned. */ - ret = copy_from_iova(cp->mdev, &iova, ccw->cda, sizeof(iova)); + ret = copy_from_iova(vdev, &iova, ccw->cda, sizeof(iova)); if (ret) return ret; } else { @@ -555,7 +559,7 @@ static int ccwchain_fetch_direct(struct ccwchain *chain, if (ccw_is_idal(ccw)) { /* Copy guest IDAL into host IDAL */ - ret = copy_from_iova(cp->mdev, idaws, ccw->cda, idal_len); + ret = copy_from_iova(vdev, idaws, ccw->cda, idal_len); if (ret) goto out_unpin; @@ -574,7 +578,7 @@ static int ccwchain_fetch_direct(struct ccwchain *chain, } if (ccw_does_data_transfer(ccw)) { - ret = pfn_array_pin(pa, cp->mdev); + ret = pfn_array_pin(pa, vdev); if (ret < 0) goto out_unpin; } else { @@ -590,7 +594,7 @@ static int ccwchain_fetch_direct(struct ccwchain *chain, return 0; out_unpin: - pfn_array_unpin_free(pa, cp->mdev); + pfn_array_unpin_free(pa, vdev); out_free_idaws: kfree(idaws); out_init: @@ -632,8 +636,10 @@ static int ccwchain_fetch_one(struct ccwchain *chain, * Returns: * %0 on success and a negative error value on failure. */ -int cp_init(struct channel_program *cp, struct device *mdev, union orb *orb) +int cp_init(struct channel_program *cp, union orb *orb) { + struct vfio_device *vdev = + &container_of(cp, struct vfio_ccw_private, cp)->vdev; /* custom ratelimit used to avoid flood during guest IPL */ static DEFINE_RATELIMIT_STATE(ratelimit_state, 5 * HZ, 1); int ret; @@ -650,11 +656,12 @@ int cp_init(struct channel_program *cp, struct device *mdev, union orb *orb) * the problem if something does break. */ if (!orb->cmd.pfch && __ratelimit(&ratelimit_state)) - dev_warn(mdev, "Prefetching channel program even though prefetch not specified in ORB"); + dev_warn( + vdev->dev, + "Prefetching channel program even though prefetch not specified in ORB"); INIT_LIST_HEAD(&cp->ccwchain_list); memcpy(&cp->orb, orb, sizeof(*orb)); - cp->mdev = mdev; /* Build a ccwchain for the first CCW segment */ ret = ccwchain_handle_ccw(orb->cmd.cpa, cp); @@ -682,6 +689,8 @@ int cp_init(struct channel_program *cp, struct device *mdev, union orb *orb) */ void cp_free(struct channel_program *cp) { + struct vfio_device *vdev = + &container_of(cp, struct vfio_ccw_private, cp)->vdev; struct ccwchain *chain, *temp; int i; @@ -691,7 +700,7 @@ void cp_free(struct channel_program *cp) cp->initialized = false; list_for_each_entry_safe(chain, temp, &cp->ccwchain_list, next) { for (i = 0; i < chain->ch_len; i++) { - pfn_array_unpin_free(chain->ch_pa + i, cp->mdev); + pfn_array_unpin_free(chain->ch_pa + i, vdev); ccwchain_cda_free(chain, i); } ccwchain_free(chain); diff --git a/drivers/s390/cio/vfio_ccw_cp.h b/drivers/s390/cio/vfio_ccw_cp.h index ba31240ce96594..e4c436199b4cda 100644 --- a/drivers/s390/cio/vfio_ccw_cp.h +++ b/drivers/s390/cio/vfio_ccw_cp.h @@ -37,13 +37,11 @@ struct channel_program { struct list_head ccwchain_list; union orb orb; - struct device *mdev; bool initialized; struct ccw1 *guest_cp; }; -extern int cp_init(struct channel_program *cp, struct device *mdev, - union orb *orb); +extern int cp_init(struct channel_program *cp, union orb *orb); extern void cp_free(struct channel_program *cp); extern int cp_prefetch(struct channel_program *cp); extern union orb *cp_get_orb(struct channel_program *cp, u32 intparm, u8 lpm); diff --git a/drivers/s390/cio/vfio_ccw_fsm.c b/drivers/s390/cio/vfio_ccw_fsm.c index e435a9cd92dacf..8483a266051c21 100644 --- a/drivers/s390/cio/vfio_ccw_fsm.c +++ b/drivers/s390/cio/vfio_ccw_fsm.c @@ -262,8 +262,7 @@ static void fsm_io_request(struct vfio_ccw_private *private, errstr = "transport mode"; goto err_out; } - io_region->ret_code = cp_init(&private->cp, mdev_dev(mdev), - orb); + io_region->ret_code = cp_init(&private->cp, orb); if (io_region->ret_code) { VFIO_CCW_MSG_EVENT(2, "%pUl (%x.%x.%04x): cp_init=%d\n", From patchwork Fri May 6 00:08:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12840348 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F14FEC4332F for ; Fri, 6 May 2022 00:08:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1387343AbiEFAMd (ORCPT ); Thu, 5 May 2022 20:12:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47470 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241593AbiEFAMc (ORCPT ); Thu, 5 May 2022 20:12:32 -0400 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2048.outbound.protection.outlook.com [40.107.236.48]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9314F4EDFB; Thu, 5 May 2022 17:08:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=S1UoZeBlXwslVK1vQ1bbf3hrCzkAAzitfGiQWEAMNABkqlrMs0zKrOnu9crMYVzisMLm+EUqO/yYEM+ac1DlACY6JK5dQjJiQF2mBUtPBMWKiArUK39J0cAXv6IeahRLaBEh2qjagutY1QwCHziUpi4atDjWwanXJNWCLGuUvkPY5LB709LBY7pZi1Xngzt9x6/tGhcduxk4MTw3fcsv8dqN7DoswfHQHQbCnS1OKhwooDeM3H6DTF+m+F3BArKPJ42VIMCvyIVRZfmHeR8jE08qYBBM0Xjr5GiNOHAE5NnLBlZ8C5xG3f9wHSM1PS0ls22MaH9Wq4QfQAeBnnF0Iw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=007v1D6OZ6VdXRrP2OlGA8BaIYAWfzduE7Z598Jqj6s=; b=iM2I2S2yMbFySK5qFmSYgh9y2Kx8n+yewT9n6mgW/Q8hbZOEU3I4UQRLpb5T+65IuBuyO+y1fwuPVra8RLH/EZAK4Rx27v9Xj0hf6MP0FpGTxE2mk6XAIfobMsUv/Grqyj5K8v1ku3hapabK+2UPByYsGGICBEBvEwadrEMy/TjPlFVZqL7zNc/XyxttwbqBLbiA3fyFy+WhVeoeNnhjxUn2r0w/Qta6ok0HHSU9XctPai2ohSVgP6TMiq9GFlWuOJJc8xM3J81d66frk5uB9JnRT9Ma7sxyBQ5ZGRmAfxN0DcLLUZT6Lr5KFOnU9exvkH5xgy0aMyS9HxL7Y5nkqQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=007v1D6OZ6VdXRrP2OlGA8BaIYAWfzduE7Z598Jqj6s=; b=cm04q43ORbqfEkXErSOMwFhoGnSouEF/EaxRjV7Ey+Ex/jElfBwCIUm1pP8nh75w4L9P6pheTjXGjgJ2nTzrvC5E7HtZnFRX02G324sfiqsjEDarRwZqKBUmvrDu+rspO7hWfwqzFTHqI2tIFTJK6cPu1afvLDpzNXww7OR6jTJwl6QvBg5dMozXtmGjMp1S5K72T+lkMMXJz6eiRlgAvWMps6Bw8ERynUZOGxzrVqffgZs/PbneNtavBDfPNiBNkcM7iZJJ2YIc+M8h4S/+ef7X12ysek8kjMRuN3DJKuzXHI9nifRxqw5HQ2y86uN1xe+kTArX4EmklM2WzfQ7aw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BYAPR12MB2934.namprd12.prod.outlook.com (2603:10b6:a03:13b::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.24; Fri, 6 May 2022 00:08:48 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5206.027; Fri, 6 May 2022 00:08:46 +0000 From: Jason Gunthorpe To: Alexander Gordeev , David Airlie , Alex Williamson , Christian Borntraeger , Cornelia Huck , Jonathan Corbet , Daniel Vetter , dri-devel@lists.freedesktop.org, Harald Freudenberger , Vasily Gorbik , Heiko Carstens , intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org, Jani Nikula , Jason Herne , Joonas Lahtinen , kvm@vger.kernel.org, Kirti Wankhede , linux-doc@vger.kernel.org, linux-s390@vger.kernel.org, Matthew Rosato , Peter Oberparleiter , Halil Pasic , Rodrigo Vivi , Sven Schnelle , Tvrtko Ursulin , Vineeth Vijayan , Zhenyu Wang , Zhi Wang Cc: Tony Krowiak , Eric Farman , Christoph Hellwig , Kevin Tian , "Liu, Yi L" Subject: [PATCH v4 3/7] vfio/mdev: Pass in a struct vfio_device * to vfio_pin/unpin_pages() Date: Thu, 5 May 2022 21:08:41 -0300 Message-Id: <3-v4-8045e76bf00b+13d-vfio_mdev_no_group_jgg@nvidia.com> In-Reply-To: <0-v4-8045e76bf00b+13d-vfio_mdev_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR10CA0004.namprd10.prod.outlook.com (2603:10b6:208:120::17) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2d8aa6d9-bb34-4f78-5909-08da2ef496f7 X-MS-TrafficTypeDiagnostic: BYAPR12MB2934:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Y3RS2vDMjBKCcSvwuVbV+46AhVhujKOdjhfzh47rPeCCFwt4NBdDYDMCL5OpARpAvzA0haDGwZD6fnjtSDuHG4ZOqR1q7pwC+KCp19lSJWO8hTINB1XrYG30AJze43v3bPFRPiTUIuJyb1LDo708FVzGzHkMFnCb9hDdBnXsrpm34BX9RjanH4+Byu0QkJrfW5t+OQJzzHf36JwOwUoKeiLdaFd7AgIoMb4NvlCYZ11Dc+NCargcC9jfsyvTnBCbPjV+c5cBwxMndGfw3GkK5G9syLYvNci7Br5fJ6tLc1y4vcaGubD+eeNh2lLTx295cgBsu7+Xp3B9CST1gHy3k6pAg3M6W81icL1yN1xLwo/S0BvLfwzlwhsFh0HipVZtn8hawXr2mxUQ8wSgAN0BMVRhiHxf2OjSOfigf4S0LE/Ewg8FgpsvQ3gjKRIKS9dsOAIslMdSqBpjUX1y1+3fHeGzAP0+DTlfWFg3gBCOEqoPqobI7umQ3YfkEVKVy57HN5MBdHTVCu5QeA6q6BD7La3F2UvcSkAOp9xYPKoVjR2iZ5uuGB1TFv5udRAUSzRhSpGmNEQJvnmzu6Vw0FPRECO1vXHyeXei6fHtWwVi3TikCQ9Y5QNHKYVU+bCA+o+RX8iMpW0o5PfCo88scfSNbRsHoSA/a9SVCjnQi41imRfvb8xVpR+0hbVsaUYUIIR2DjA3dym9HS7CZcVQC1yDfQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(5660300002)(2906002)(66946007)(66556008)(54906003)(8676002)(86362001)(110136005)(4326008)(6486002)(38100700002)(508600001)(316002)(66476007)(6666004)(6512007)(6506007)(186003)(2616005)(921005)(26005)(7416002)(8936002)(83380400001)(7406005)(36756003)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: o3Bpl9JqnGFxs5pxG0hCmV4ZJoLg9PUtCGK4cn88aF6wvERCa8T9I/xOn1AT2JPAw74vlmD5GBusho6FF1sdzXRALKrm2+BkxisWSqLoq2UK3DpQInpWeQc82cW+3qSUmQAImSuwsXBF4M7t+UNk0g4p5t4NukACInUE94OhLvdqGr7uLGbLjqYJ+3R+PpgNJxJstv+Vm9uV1wUnOoOeaa/mJYvuSOIs0+yVB2LI1GRPTq/0SEuRPiA5vxBKmKxAaq2sB81oAh0BplsBObu9ew2H3cQIpmvrI8BmHPPw5uup9eL5WLxaH7nI15YOqwJf8wa9D8NL9ynFHy2nhFnluw51ZBlWyf3HxG0vQeINBrjbNBbjyCOMYpnyB6tB+VGB8JLlmc27NShAmakMl9k/hZ3WdA1v4ROLl9/jPDHUqq0pMI9sIM9af9K+rL+2Z4WkJGmpnOdxoaxOYuL1V/L/1FexFx4O6RpYCjb4iOELDLQgD0li5wVkZjL4WUP0fZtdE9/Xy8IzuSW1wz2tIiUacV5p4mA0L6toCDSGf4I1Cu89lX+zi9sevrRtONmic9/R7McHLrtK46Cwt6kDBSoAjky6zsyc0ejvSMGrytxTg9dIxkHy2lnMc05+V/v+mdAfknm24OgOYFBrn7vYJ2ZIHFoO2qvFxPE4/VgSwwlzcVt7UbSMzaqQAf1XLmgrUQnqge9e/0f5Fj/x9jNHVl8Qt1ZxS6vWmFTKwiPnP7pW2hFjKiCGynBnCDFAsKoYTZG/Z8c+bmCPF3t9VU0S1ZRHkkoHnbGgjc5Xqv53y03Df4qxCLMV7zWm1jbhQXAnNQcub5UURdRzgrG5JcIv3M+5G9+oT5rThzeUZFlWFJ785AglQqADqmLXksX9gySmTsM+h7pYxhBBcADsvwttbGR9OYBkBfXgMJzpgX6opVM3tXqCSmrJjuAH2wbxY99+f0H3gOnGOqMpoD+lDz+lrVn17uSTAmYVdVUT6W1WMF6U5wH6veD/HWlZT/woBr91ZqecoALjUNA1ysDQQjy29HpJ1oHxwnqik1tIDGBuomTH6aOzi9ikrDex6ibPneU6JF5Yg8kmXE/aFNp8LiqKL5jac87NYmWzdj8WVVruVqs4h3EdIlAPH6LaB+h7jt4zRCIl/tNPCu1k7sZsBuH4VckrHm/yAjfYOLQ22aR3MU4bXXgBCOLgU62yrgZcj6T7uuwCrA1P35ElEOoOI38YfYLjQ/NIjH6X9s+mQLXd3S2JEe9Hvz+bQ7EKYtoIpYkBqdzbtzo6svKiNdeGm+iwaiFQ6BbPV5auNfGjdchKacRiNDn3jO2cnGMOM8+6Uev0LyO4PTLIA8+eIooJOrSWoCPhXFnEwYFOIjCtP+CspY9GXE9vROu/K/9IY8hoB7OowcmWRi5glxPSYUpbu08gxE9liFK+3ekblTUHyTZBAvn280pSzYb1KHKfLMob72NQweprbZJv1rKZML4TUaQ0iRL6Nbl6BmELMMw46iqqb8L4kdjDETEQDge5FjfKAUuCZfMkEtL9TKgm8kmaTQdjcHsurmFYx09iab5rwvX8zaONzjLNv6aAG99v7UUN4Qh8y3fK2I24vVp5i/3oHiA/F9dnqsVgwtVDIOmpIl1JKW9DQ+hNoX53ZBCG5m33W1iiBNDimHDBXpgApou8sgn7TO+GF0UWal7llqKhVQyV+Wiwr07h18tpq/D9mRBpPF9DyW2jPLi6hUf0qy8OcAtaxRsPFw== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2d8aa6d9-bb34-4f78-5909-08da2ef496f7 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2022 00:08:46.9029 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: MREoh2h6HWmtvX7NJqLPsXDjMBUzKb8nmglfq40KTA/p3sTpXAPv/V4D7Ezksv67 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB2934 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Every caller has a readily available vfio_device pointer, use that instead of passing in a generic struct device. The struct vfio_device already contains the group we need so this avoids complexity, extra refcountings, and a confusing lifecycle model. Reviewed-by: Christoph Hellwig Acked-by: Eric Farman Reviewed-by: Jason J. Herne Reviewed-by: Tony Krowiak Reviewed-by: Kevin Tian Signed-off-by: Jason Gunthorpe --- .../driver-api/vfio-mediated-device.rst | 4 +- drivers/s390/cio/vfio_ccw_cp.c | 6 +-- drivers/s390/crypto/vfio_ap_ops.c | 9 ++-- drivers/vfio/vfio.c | 46 +++++++------------ include/linux/vfio.h | 4 +- 5 files changed, 27 insertions(+), 42 deletions(-) diff --git a/Documentation/driver-api/vfio-mediated-device.rst b/Documentation/driver-api/vfio-mediated-device.rst index 784bbeb22adcf5..eded8719180fba 100644 --- a/Documentation/driver-api/vfio-mediated-device.rst +++ b/Documentation/driver-api/vfio-mediated-device.rst @@ -262,10 +262,10 @@ Translation APIs for Mediated Devices The following APIs are provided for translating user pfn to host pfn in a VFIO driver:: - extern int vfio_pin_pages(struct device *dev, unsigned long *user_pfn, + int vfio_pin_pages(struct vfio_device *device, unsigned long *user_pfn, int npage, int prot, unsigned long *phys_pfn); - extern int vfio_unpin_pages(struct device *dev, unsigned long *user_pfn, + int vfio_unpin_pages(struct vfio_device *device, unsigned long *user_pfn, int npage); These functions call back into the back-end IOMMU module by using the pin_pages diff --git a/drivers/s390/cio/vfio_ccw_cp.c b/drivers/s390/cio/vfio_ccw_cp.c index 7a1cf3091cd647..0c2be9421ab78f 100644 --- a/drivers/s390/cio/vfio_ccw_cp.c +++ b/drivers/s390/cio/vfio_ccw_cp.c @@ -103,13 +103,13 @@ static int pfn_array_pin(struct pfn_array *pa, struct vfio_device *vdev) { int ret = 0; - ret = vfio_pin_pages(vdev->dev, pa->pa_iova_pfn, pa->pa_nr, + ret = vfio_pin_pages(vdev, pa->pa_iova_pfn, pa->pa_nr, IOMMU_READ | IOMMU_WRITE, pa->pa_pfn); if (ret < 0) { goto err_out; } else if (ret > 0 && ret != pa->pa_nr) { - vfio_unpin_pages(vdev->dev, pa->pa_iova_pfn, ret); + vfio_unpin_pages(vdev, pa->pa_iova_pfn, ret); ret = -EINVAL; goto err_out; } @@ -127,7 +127,7 @@ static void pfn_array_unpin_free(struct pfn_array *pa, struct vfio_device *vdev) { /* Only unpin if any pages were pinned to begin with */ if (pa->pa_nr) - vfio_unpin_pages(vdev->dev, pa->pa_iova_pfn, pa->pa_nr); + vfio_unpin_pages(vdev, pa->pa_iova_pfn, pa->pa_nr); pa->pa_nr = 0; kfree(pa->pa_iova_pfn); } diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c index bfa7ee6ef532d9..e8914024f5b1af 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -124,8 +124,7 @@ static void vfio_ap_free_aqic_resources(struct vfio_ap_queue *q) q->saved_isc = VFIO_AP_ISC_INVALID; } if (q->saved_pfn && !WARN_ON(!q->matrix_mdev)) { - vfio_unpin_pages(mdev_dev(q->matrix_mdev->mdev), - &q->saved_pfn, 1); + vfio_unpin_pages(&q->matrix_mdev->vdev, &q->saved_pfn, 1); q->saved_pfn = 0; } } @@ -258,7 +257,7 @@ static struct ap_queue_status vfio_ap_irq_enable(struct vfio_ap_queue *q, return status; } - ret = vfio_pin_pages(mdev_dev(q->matrix_mdev->mdev), &g_pfn, 1, + ret = vfio_pin_pages(&q->matrix_mdev->vdev, &g_pfn, 1, IOMMU_READ | IOMMU_WRITE, &h_pfn); switch (ret) { case 1: @@ -301,7 +300,7 @@ static struct ap_queue_status vfio_ap_irq_enable(struct vfio_ap_queue *q, break; case AP_RESPONSE_OTHERWISE_CHANGED: /* We could not modify IRQ setings: clear new configuration */ - vfio_unpin_pages(mdev_dev(q->matrix_mdev->mdev), &g_pfn, 1); + vfio_unpin_pages(&q->matrix_mdev->vdev, &g_pfn, 1); kvm_s390_gisc_unregister(kvm, isc); break; default: @@ -1250,7 +1249,7 @@ static int vfio_ap_mdev_iommu_notifier(struct notifier_block *nb, struct vfio_iommu_type1_dma_unmap *unmap = data; unsigned long g_pfn = unmap->iova >> PAGE_SHIFT; - vfio_unpin_pages(mdev_dev(matrix_mdev->mdev), &g_pfn, 1); + vfio_unpin_pages(&matrix_mdev->vdev, &g_pfn, 1); return NOTIFY_OK; } diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index d70890ab9cea92..93caab1f29dbd7 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -1919,7 +1919,7 @@ EXPORT_SYMBOL(vfio_set_irqs_validate_and_prepare); /* * Pin a set of guest PFNs and return their associated host PFNs for local * domain only. - * @dev [in] : device + * @device [in] : device * @user_pfn [in]: array of user/guest PFNs to be pinned. * @npage [in] : count of elements in user_pfn array. This count should not * be greater VFIO_PIN_PAGES_MAX_ENTRIES. @@ -1927,32 +1927,26 @@ EXPORT_SYMBOL(vfio_set_irqs_validate_and_prepare); * @phys_pfn[out]: array of host PFNs * Return error or number of pages pinned. */ -int vfio_pin_pages(struct device *dev, unsigned long *user_pfn, int npage, - int prot, unsigned long *phys_pfn) +int vfio_pin_pages(struct vfio_device *device, unsigned long *user_pfn, + int npage, int prot, unsigned long *phys_pfn) { struct vfio_container *container; - struct vfio_group *group; + struct vfio_group *group = device->group; struct vfio_iommu_driver *driver; int ret; - if (!dev || !user_pfn || !phys_pfn || !npage) + if (!user_pfn || !phys_pfn || !npage) return -EINVAL; if (npage > VFIO_PIN_PAGES_MAX_ENTRIES) return -E2BIG; - group = vfio_group_get_from_dev(dev); - if (!group) - return -ENODEV; - - if (group->dev_counter > 1) { - ret = -EINVAL; - goto err_pin_pages; - } + if (group->dev_counter > 1) + return -EINVAL; ret = vfio_group_add_container_user(group); if (ret) - goto err_pin_pages; + return ret; container = group->container; driver = container->iommu_driver; @@ -1965,43 +1959,37 @@ int vfio_pin_pages(struct device *dev, unsigned long *user_pfn, int npage, vfio_group_try_dissolve_container(group); -err_pin_pages: - vfio_group_put(group); return ret; } EXPORT_SYMBOL(vfio_pin_pages); /* * Unpin set of host PFNs for local domain only. - * @dev [in] : device + * @device [in] : device * @user_pfn [in]: array of user/guest PFNs to be unpinned. Number of user/guest * PFNs should not be greater than VFIO_PIN_PAGES_MAX_ENTRIES. * @npage [in] : count of elements in user_pfn array. This count should not * be greater than VFIO_PIN_PAGES_MAX_ENTRIES. * Return error or number of pages unpinned. */ -int vfio_unpin_pages(struct device *dev, unsigned long *user_pfn, int npage) +int vfio_unpin_pages(struct vfio_device *device, unsigned long *user_pfn, + int npage) { struct vfio_container *container; - struct vfio_group *group; struct vfio_iommu_driver *driver; int ret; - if (!dev || !user_pfn || !npage) + if (!user_pfn || !npage) return -EINVAL; if (npage > VFIO_PIN_PAGES_MAX_ENTRIES) return -E2BIG; - group = vfio_group_get_from_dev(dev); - if (!group) - return -ENODEV; - - ret = vfio_group_add_container_user(group); + ret = vfio_group_add_container_user(device->group); if (ret) - goto err_unpin_pages; + return ret; - container = group->container; + container = device->group->container; driver = container->iommu_driver; if (likely(driver && driver->ops->unpin_pages)) ret = driver->ops->unpin_pages(container->iommu_data, user_pfn, @@ -2009,10 +1997,8 @@ int vfio_unpin_pages(struct device *dev, unsigned long *user_pfn, int npage) else ret = -ENOTTY; - vfio_group_try_dissolve_container(group); + vfio_group_try_dissolve_container(device->group); -err_unpin_pages: - vfio_group_put(group); return ret; } EXPORT_SYMBOL(vfio_unpin_pages); diff --git a/include/linux/vfio.h b/include/linux/vfio.h index a00fd722f044b0..bddc70f88899c3 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -150,9 +150,9 @@ extern long vfio_external_check_extension(struct vfio_group *group, #define VFIO_PIN_PAGES_MAX_ENTRIES (PAGE_SIZE/sizeof(unsigned long)) -extern int vfio_pin_pages(struct device *dev, unsigned long *user_pfn, +extern int vfio_pin_pages(struct vfio_device *device, unsigned long *user_pfn, int npage, int prot, unsigned long *phys_pfn); -extern int vfio_unpin_pages(struct device *dev, unsigned long *user_pfn, +extern int vfio_unpin_pages(struct vfio_device *device, unsigned long *user_pfn, int npage); extern int vfio_group_pin_pages(struct vfio_group *group, From patchwork Fri May 6 00:08:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12840352 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0886CC43217 for ; Fri, 6 May 2022 00:09:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1387401AbiEFAMm (ORCPT ); Thu, 5 May 2022 20:12:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47522 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1387374AbiEFAMg (ORCPT ); Thu, 5 May 2022 20:12:36 -0400 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2053.outbound.protection.outlook.com [40.107.236.53]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5042F506E2; Thu, 5 May 2022 17:08:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ps3JYd1cyiiTCtSdl25iVwPwdc4M/cCqGP7Mq6+P2K8bF2MzVx3Mw6vypgNOE6qHaQJmYvlw7EvJe6Dy11oNkQcp3Fptj51ibUvbtRjibgKZdREeUogwqGMw5q8yCxV+BB/oghemv+HEPYVCnUopNJ68GA8bm1D9ozlWMsC7xxu9luqIfwFw4v0YS6xtljJWqgJMS6bQkbmdjRrXVsDHNuJimBPHIZoHYYpMCNDun0JHBusX1IzVF36msVzzJj88cRu15SfW4ZyYRNN/qepc0qREFEPgNP/lbLX/ZgS2BAQ/Vo/pSYMrQgWF/VKswueDnGDWNUJdOwCOuqoYAyEBaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JKgMH0C+sdZdX+Xl3LsrWj+gZJ/nBXyZObIa3eQ9eZM=; b=T8NdWD/iPa6bu5WablAAqNT3J3IkO6GIhiUs+UIzC2VwCEY2287DL+bbRppvPfhimJyT6RXk/rPsKyhvpEF5WS0BlExEP9wot7qSn27ZloukRLdOTkNLFE38KpnpOyyGktpeKwkU3kNQl3vHrG5XvkHnhXdroJYjQtMJqKjqCcl4/rlIL03pciSlmSAnXBeI7ztQsvWl/dn7uNxdDCqo80W6231D9TEsmpYVOu9L13ylMapLO9r8mxzUw1j0BX3GMCrVlcArDojUxHVvst8qMUjXg2G+nHXYBJo+iwK1lQecDC1R3IcgVUc47MQkeZoZ7EH5JU0ajqYEaMQkDic6pw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JKgMH0C+sdZdX+Xl3LsrWj+gZJ/nBXyZObIa3eQ9eZM=; b=mBQE/pAWn1osKhlZAHJ7lULDG9ZLthCQNF6V8RqxIoJvhRr9gmO1M0l1l0Qb01x5YGP/IOB2QLBaeRzsLb6Q+aaqbvgGeOMiAIocDQ/TuHqz7X8oUIDpHMXyyD3KRuIIfqGGR9Ne3RK1C56cMj72MUy4TTsWHhjcVBOJ/Rdvf8P3wMD1mHLngbxG0OneJW8VBsAsixwVWQzN6iw+TNhcX2jkYAh8PowdfyaK+nYWynyn947ZnkX62+o/+fzspP38uBckxwFQ9WmJwFTbHHNnt9s7C576cYcySUn/P/DjtYz9P31tSu4fC7+59fGWoVzKbiRhFrMkCqkajHvklV9JVw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BYAPR12MB2934.namprd12.prod.outlook.com (2603:10b6:a03:13b::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.24; Fri, 6 May 2022 00:08:49 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5206.027; Fri, 6 May 2022 00:08:49 +0000 From: Jason Gunthorpe To: Alexander Gordeev , David Airlie , Alex Williamson , Christian Borntraeger , Cornelia Huck , Jonathan Corbet , Daniel Vetter , dri-devel@lists.freedesktop.org, Harald Freudenberger , Vasily Gorbik , Heiko Carstens , intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org, Jani Nikula , Jason Herne , Joonas Lahtinen , kvm@vger.kernel.org, Kirti Wankhede , linux-doc@vger.kernel.org, linux-s390@vger.kernel.org, Matthew Rosato , Peter Oberparleiter , Halil Pasic , Rodrigo Vivi , Sven Schnelle , Tvrtko Ursulin , Vineeth Vijayan , Zhenyu Wang , Zhi Wang Cc: Tony Krowiak , Eric Farman , Christoph Hellwig , Kevin Tian , "Liu, Yi L" Subject: [PATCH v4 4/7] vfio/mdev: Pass in a struct vfio_device * to vfio_dma_rw() Date: Thu, 5 May 2022 21:08:42 -0300 Message-Id: <4-v4-8045e76bf00b+13d-vfio_mdev_no_group_jgg@nvidia.com> In-Reply-To: <0-v4-8045e76bf00b+13d-vfio_mdev_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR1501CA0002.namprd15.prod.outlook.com (2603:10b6:207:17::15) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0affd359-3fca-4001-75aa-08da2ef49759 X-MS-TrafficTypeDiagnostic: BYAPR12MB2934:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zXS34a/bDPPYkuTo1Q0WywF5xF1I0H3H2+sXWUXtSszgsHfMv3p2F6Of+Pnan+9yzDzp9/OReQ6t4D6ZncvM0T2y+RDHaJ9LHb/Zh9HFfRF7PI1h6ZmAOVF4XFxXax0WIih8qtUvB2Pe+3hJTibyTR4ynZ4vMzbp7S37HOe420/0Qext3iDIzazfYYy1vhTZgfhvjEfYvSRkG71fLl1wNK2a3qjCtZRgSWboF4GwvVenguI+I1qD5W/4uUUsiIcbU3M9MqFaGuXT4656wHqYoQJ1HcXE+WQBYP+h15+bzSRGAfNcSBjJFHmm9fex3b9cbqfb7cKhbKayCg1tc4bJeD3XCmE5PI86l6UQ3nt/xMWAxjFDQYhYjl0Ekbkw3SHpePY16MXWy8+suG4nXU0tNHOMhMW+owP7HzZXODFh0MCVh0yC2jRkGTRvVTmSDkCfWywmiljb0XNvzDFxxRES9oy7UltdrjKfiMsO8gmenF4W3P6xH6XLLOHektz5VhjcvGh1qQ0ooG85+8/k9lVZxNg7y06m5IQGhiA+W5e3xxJN/oSzZhtlAhz1Y14RRK4EBBUqeihWt3eJXoeqDtEuqnB/npgFlaG9s/SNSQP76/aYD2t4XkWYUmdeD+7K+Wi53oRMZxJDrpgIkh9I3PDPcAthkwazc3NTK+iiQD3hS0sYrnqRtxoeXwElWSQz/m+LatjiJpXhJ9K3AOq0bnETGg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(5660300002)(2906002)(66946007)(66556008)(54906003)(8676002)(86362001)(110136005)(4326008)(6486002)(38100700002)(508600001)(316002)(66476007)(6666004)(6512007)(6506007)(186003)(2616005)(921005)(26005)(7416002)(8936002)(83380400001)(7406005)(36756003)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: p88Q0GQQ4kiyeChsMliXvExb1zgjmfr5Cw8qL5L+AfP0ENP9QRwnzj1pPf1TQB19ThEU8OGsM3P8cE2ecMTvFCQAev/F3AqWwRx7RF9x+JiHVxqpY5/n77nkU/EblQ5YiswsfBzW6TNW/q8Yy+tx2VZ5f7jm1FrxbbBPFpTtbH+7AczlBgYkS5dAcggZSZe1/y66nSYgKMGh5RJ8P97CT2KEjtQ66sU8HqKcxqBrpyhlbGUj3P/Z4hA6Rn8CPJFBRR/ZoOJzI093Kkr6fAi5jn1hAHGv98AiW44uHXV5K2bWZqdtoIq31c7NYHbrQEGCziPcrPQaNYYv9wXvBb3AcJqah9eEYISIU4llCE/YoUr9FsFeVPqbEuNzrUSo43z7/3HxElsoW1Rc3718PM9QjsWks4Y/qgNZZ3xkygSXRJUTWmFAuhtphubn3K09FPeihKW/5ip1g7pnGR0h/uvhDx5r7ClB3Ix1TRM6Y4SQusjIvRp7pYmMUdHh0bDMWTMHYYtFY7FP8BR6EmMjO3UlgiymbG8Dmz8nrvfwpSHeTuYTBJNH0iFkVIBAAxj7aK/cssre+XIABHC72k58ZvpvCFm1ZOehFgXyrcn/d4LfegyJ7eUU+98wAR9VLitgtz54e5Yl+JP88nbo4AVXALaPeytwgRWv0GxDt5u8U2d+o0CZFLihccD6lXYHZXNU8UsdIiGgpBVqeqXLviiXXsX0wYXTIkcIiu0h9CQMw0jfeRTJNgKoIwf5R4pfH0CQakXmWoC59Dh4ZhPAeYVBC9CY6vVJvMyW3oLrmF8fQ3scT2CJm1DWR4jShjIhgrg/l0QbMDr7lliGfijEjoZlTpArs/rZ++fuKygdNKxdi/EO6jFUcTv3dbu77YHX9rjdDKuLUUM+shVCoEtkaO4nTYGhNiYMgx78ztq6FH32Poixc+V6v8BhnQXOja7T/aL7fUUTEOVMmgotSkJOU3y16V99ptb3zHoc9tBQ+ZFckW57gpZE1fJxvcJeivpWN74CIBk2YRIKDvGhSZhA2/28ZKpgxcepZVHvH7wjQxGiC5WTY7FhsaYRgkd32jHNOcGLFRc5RXCKU/NlJ4gFOLPfauw6RE3l65yVAz3oFLyCgoFFMTTF32zvYbrVeU/80E+nlKQb0f2DVisRQSyHdQbEzTEIBBmq4Oa3NgEAuzlSQ/2U8nhu57BVBqrsvLyPE95Ksqb/FjjRU7R6/s4HjXPuV00UKZARkF8QS7z/bNoxpbFve6ObrGjvL5WYNEIOJGHPtkZnblCI+I04Ibbt4EckULcKHn0ad1lbU99jkV7rbTItCmU9atGBnHO2YwOTfOL2iDLUD5szmtYY451I8iy6LmGYIGTgr3Isv5/xCkCnR1JMqFCp6jNFJu2kjGFYRxq+q2SNyLG2j8u5BCRidab5jfa0nzXAMBGgFszERzxVsmLK78Tmisd1osNavwYAaZ9hUgE4iKXh8Sk3qbyNpovB/Ugw2GfYEty8J86mC7RoSm3nokR6/juvKiZQWMSuriHZlLtk4TUzmEU8U8RUWT1gdtGguj0+i4/4q7Q8eRC2Kt4BlF/07JDPa7uYVAGfpDhSgOzdrPOIVDQvpbMdafl7lvOK9SOlwUdmmWdV9MkDGdEuOo1mranknqo87iZk7wR6s1T/qbGzBzzZAgZlo7IRaHC8+nzuKTKTpepm1qFO0EMQsQkmn2FI+o9kyf1d/IsO9UtpDOqcWcfSBsacYXIrzwdo/Q== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0affd359-3fca-4001-75aa-08da2ef49759 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2022 00:08:47.4028 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: L4raNAuH7sGGQzFLa9VsF5OhfvyTnqnHCK4XlB9iHsY0Ie5dhchtONGMo5Ms3kuO X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB2934 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Every caller has a readily available vfio_device pointer, use that instead of passing in a generic struct device. Change vfio_dma_rw() to take in the struct vfio_device and move the container users that would have been held by vfio_group_get_external_user_from_dev() to vfio_dma_rw() directly, like vfio_pin/unpin_pages(). Reviewed-by: Christoph Hellwig Reviewed-by: Kevin Tian Signed-off-by: Jason Gunthorpe --- drivers/gpu/drm/i915/gvt/gvt.h | 4 ++-- drivers/vfio/vfio.c | 24 +++++++++++------------- include/linux/vfio.h | 2 +- 3 files changed, 14 insertions(+), 16 deletions(-) diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h index 03ecffc2ba56a9..5a28ee965b7f3e 100644 --- a/drivers/gpu/drm/i915/gvt/gvt.h +++ b/drivers/gpu/drm/i915/gvt/gvt.h @@ -732,7 +732,7 @@ static inline int intel_gvt_read_gpa(struct intel_vgpu *vgpu, unsigned long gpa, { if (!vgpu->attached) return -ESRCH; - return vfio_dma_rw(vgpu->vfio_group, gpa, buf, len, false); + return vfio_dma_rw(&vgpu->vfio_device, gpa, buf, len, false); } /** @@ -750,7 +750,7 @@ static inline int intel_gvt_write_gpa(struct intel_vgpu *vgpu, { if (!vgpu->attached) return -ESRCH; - return vfio_dma_rw(vgpu->vfio_group, gpa, buf, len, true); + return vfio_dma_rw(&vgpu->vfio_device, gpa, buf, len, true); } void intel_gvt_debugfs_remove_vgpu(struct intel_vgpu *vgpu); diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 93caab1f29dbd7..85e1304099b8a5 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -2108,32 +2108,28 @@ EXPORT_SYMBOL(vfio_group_unpin_pages); * As the read/write of user space memory is conducted via the CPUs and is * not a real device DMA, it is not necessary to pin the user space memory. * - * The caller needs to call vfio_group_get_external_user() or - * vfio_group_get_external_user_from_dev() prior to calling this interface, - * so as to prevent the VFIO group from disposal in the middle of the call. - * But it can keep the reference to the VFIO group for several calls into - * this interface. - * After finishing using of the VFIO group, the caller needs to release the - * VFIO group by calling vfio_group_put_external_user(). - * - * @group [in] : VFIO group + * @device [in] : VFIO device * @user_iova [in] : base IOVA of a user space buffer * @data [in] : pointer to kernel buffer * @len [in] : kernel buffer length * @write : indicate read or write * Return error code on failure or 0 on success. */ -int vfio_dma_rw(struct vfio_group *group, dma_addr_t user_iova, - void *data, size_t len, bool write) +int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova, void *data, + size_t len, bool write) { struct vfio_container *container; struct vfio_iommu_driver *driver; int ret = 0; - if (!group || !data || len <= 0) + if (!data || len <= 0) return -EINVAL; - container = group->container; + ret = vfio_group_add_container_user(device->group); + if (ret) + return ret; + + container = device->group->container; driver = container->iommu_driver; if (likely(driver && driver->ops->dma_rw)) @@ -2142,6 +2138,8 @@ int vfio_dma_rw(struct vfio_group *group, dma_addr_t user_iova, else ret = -ENOTTY; + vfio_group_try_dissolve_container(device->group); + return ret; } EXPORT_SYMBOL(vfio_dma_rw); diff --git a/include/linux/vfio.h b/include/linux/vfio.h index bddc70f88899c3..8a151025871776 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -161,7 +161,7 @@ extern int vfio_group_pin_pages(struct vfio_group *group, extern int vfio_group_unpin_pages(struct vfio_group *group, unsigned long *user_iova_pfn, int npage); -extern int vfio_dma_rw(struct vfio_group *group, dma_addr_t user_iova, +extern int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova, void *data, size_t len, bool write); extern struct iommu_domain *vfio_group_iommu_domain(struct vfio_group *group); From patchwork Fri May 6 00:08:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12840350 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C937DC43217 for ; Fri, 6 May 2022 00:08:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1387387AbiEFAMi (ORCPT ); Thu, 5 May 2022 20:12:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47488 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241593AbiEFAMe (ORCPT ); Thu, 5 May 2022 20:12:34 -0400 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2048.outbound.protection.outlook.com [40.107.236.48]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA5264EDFB; Thu, 5 May 2022 17:08:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F83c/8/j1qe6fJlz+Pu0f9r/T1EOYel/3ee6zwTUeTWt0Zp+tjIAOpf6IVd7BbdGDA2t7wrg09ZyoPB6UirxoSE1+XQmpu9rTixOx34ZDTmN+Wz1L8viS2ebHhK2cUsa2yE55Rgsye6uhFGSUFenjUFv25Bv1QNKnhucA0WVlJcvvO8N2QcIu27y6vPFLxW2ikxCBp+Xd0QwjL9zyzpQpotGatYsIaDxZGoLM47pLAZORM+JKCOIVFSR7Gu/5A1Q68hsuc9soIDHrpQKca1pz24JC1LHtpksyhhxec5py69/XXcWrsqrgMMrQnwit5EXdHVBgcC3kp6/h6CljokALw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rX3j7rOWSWO7c4hbUYMNBvD/GJno+vvgfbHyrqgd16w=; b=U7IZPT6N5intDWNYuhbEA10x+uJ7UdT7TDJEsmZAQGSdUW2ptcoAz0/8ofj6RUFhhj4SDffnMlDnuqkw6ld5MUuiUH7rN8tz5fDlsFdJIvZcOIChAmzQBfRFiT7TKZ5fnU33vRCxm/BaJYgGhMdVF2TSIPET1XyhGnfbHoDfbxqdnV3QeQV2SET5riAvt0NS73zYLtl+N3CDbapTWjOWPghzZLyfQnchI45d/r0ka4H4cmXZ8kpKtB9gHnh3rTiG5fQufgTbkNECYJ5ujQDBW8mhfSkUXejNUAiQMIyUYtQ2DNK8cy70LFzeZRkABfGq43IbMtJ9h0puW1RsZ5bbFA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rX3j7rOWSWO7c4hbUYMNBvD/GJno+vvgfbHyrqgd16w=; b=Nib7vTME4/3+G4RPZERFEOjwgLLpDpmbUIYUaiacJbcu4ft3L3dU3G26tO7b9GFuWDMG19G/Vsqs7t+bkpHxm/vAaLyFB/i7Jfxx1MeW8NoSS/iIUIzLJgkplrCfh1JeRzLugHKc3sQNBXynUSpYZnhPPIBsvQYOU4wyFhcXGuNt3B1j1H+KfJu/yKt0etSgC7CSkXWfPL5ax0u8JKk1tapq2EubQeZOLE4vMr/Kpm4D8Qe7f8DdjXhWd84OObLVXp4XgeNJLnIDqUZWpidRI5znINcow/si7AkAOF7iiMIhCFvfrI377FY+tKsk66kXrDQbnQjSVu5dy2ov4Bs+4A== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BYAPR12MB2934.namprd12.prod.outlook.com (2603:10b6:a03:13b::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.24; Fri, 6 May 2022 00:08:49 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5206.027; Fri, 6 May 2022 00:08:49 +0000 From: Jason Gunthorpe To: Alexander Gordeev , David Airlie , Alex Williamson , Christian Borntraeger , Cornelia Huck , Jonathan Corbet , Daniel Vetter , dri-devel@lists.freedesktop.org, Harald Freudenberger , Vasily Gorbik , Heiko Carstens , intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org, Jani Nikula , Jason Herne , Joonas Lahtinen , kvm@vger.kernel.org, Kirti Wankhede , linux-doc@vger.kernel.org, linux-s390@vger.kernel.org, Matthew Rosato , Peter Oberparleiter , Halil Pasic , Rodrigo Vivi , Sven Schnelle , Tvrtko Ursulin , Vineeth Vijayan , Zhenyu Wang , Zhi Wang Cc: Tony Krowiak , Eric Farman , Christoph Hellwig , Kevin Tian , "Liu, Yi L" Subject: [PATCH v4 5/7] drm/i915/gvt: Change from vfio_group_(un)pin_pages to vfio_(un)pin_pages Date: Thu, 5 May 2022 21:08:43 -0300 Message-Id: <5-v4-8045e76bf00b+13d-vfio_mdev_no_group_jgg@nvidia.com> In-Reply-To: <0-v4-8045e76bf00b+13d-vfio_mdev_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR10CA0022.namprd10.prod.outlook.com (2603:10b6:208:120::35) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 01f245fb-4775-4e7b-c9ef-08da2ef4970a X-MS-TrafficTypeDiagnostic: BYAPR12MB2934:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Waapk8o2Jy5/7jqWCMX41amOUJHsPXJk/pCkvUhHyMMukiexL9kFUnR+6zrVO4YF8ewA07u6zoJcDL47VI54DiA8EM5/EgdSdtDV014+Ro5EiP8h7bcTwuoFqYXU8MgUpoMG+Tg1AYVfYa54zNgIosm+NsMMjy1bWemSKT7/iF1+dhf8RxVY5IaZL9EZlSF5o6W3Rij5HVoCi9XOZQxkhfENV/NOo/Pe1ASM6nx2zCEqNwm0slcxe+IIkCyEdfhFPONUIeDuM7fK9RdOE3faFR3oRo4Ry3B8u2k0o1qBMu2MPFUMMx7TMQ2tGcNncPwVGKUp6JdrBX0W02fnhFdPPU/fzFhf9vH2wlz9AHAsLVbf9A88IVPtygHhua4HUKoumtIN7ldienz4gPht7SbZmf0Ff4ZU2rmSfyx1L16sG6qHgBlWICBC5yGr4Vu4qDGG5MtpaMb5GfKj9oNcJntD6fJ9687r/fwwaI4Pt/BdpdVcB/SizZupMTOlj0AqOWnTsZwtWgfTBJxQ/RdFY1QduF6DELpJBAlwQkOUxGKOS5tLWlmf323mu0pxgNYI8kVj1L0NIqpF1ZsRTTEaWkYhiumWjTd05Bh/7cSmSJ3KNiW8oABP5P9bp9Xb700R7LYcB8saKkYrCKxkXVJDNZB/iJYUREMN6nwDIHX8fiSBxN8zphLGFha1N9RmkXl0fAc2c9n4yOy3IZUUbk84HMPAsA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(5660300002)(2906002)(66946007)(66556008)(54906003)(8676002)(86362001)(110136005)(4326008)(6486002)(38100700002)(508600001)(316002)(66476007)(6666004)(6512007)(6506007)(186003)(2616005)(921005)(26005)(7416002)(8936002)(83380400001)(7406005)(36756003)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: djQjSXX8PUiIltogHVtQFwR5No0DMZH7ORS47rwoYB0tHAke83wNRA35fGHeVel5rIznDu+PPx+LBWWaaas/UcnsdAtDmgSfvRfD8nGKlti/kPxCBTPJpBkmBo11MECyN85vjwoeitXXbj5PDxlxQkGOEokyOaHffzJS3A/xA3bM7g0z6BNH/FOgpYK60hC0Ncssjx2f8XRDX62cm+PFQijpv72MTWRQ0iXioHi/sN0QsDHgeKGHGSX1hGHEEHxdJoxNnS03QyZQ1MqVjaG//2eM+3e+yXUcn4ixh2/e78qFO3pjk8K9sy5NZ0zUnIjFR5Zy5aWJYecpJ6HWHCJGwK4dpukVpD309sRSXS80zNRWnlY2IjFEySyJKrLS28+aK6VpyA4JI3LVqsDWifEUB+zZL6louKfgExm+JbjXRIDb8SWyuD/7OFJ8UbRYWVUtwT7IYqsHru5p/pdc6OmN6fSDT+p0+YgGz0kReKpsAjsSwwlZnkrZ9i9fGFi468pHK8+mIItIbfLFBwYMFwJElf1VwUDl60dqq7eHuTFbYiy90f2Rof/k4aawi2SrOVEs03Sfl0Og2sgNUIBZVu9MQsz/Ygiaae8stYax5yVdKAnAIOwQ468lFNfBN4xKXOWLLQnpbHhzTnZ38GI34jMy4UgxF+zuIhS1rkIa3f9S29vr4rYNCHFHY4VS7XmoxrwTJnXKi7TlhTJHhhssZZrNdVTCF+COPS4W/mAxQDKqdgjYyEmq9EasroGfbEumzdllq7kXe0kKEznHDPTDHfdFWX7Xkt+Tj1WCPjleQ+nizhDCKAmQjbXbsOVwJl5JodwOgFhxF7+Ibtw1N5wHQEzlKhVOKG8+cdHkvFL0oYiflfnsanI8z6QeWDkGFWyZvLrCmZMiz4GByezm50l7k5AADImCge57LA37mIJpEkBXDHvfvPxjBciLhwsnxCw/I8FpUuCFUxDfQ45Tt3K3CbhLs0hZ3cTCMb9NuwEzw0I2a/tAOUeAPu69vHJE8I2QxdRBeW0Pjt1as8rz3Iks8T3SSQ8+t+mlsmCsww9o6XtAVqvvzwJrzSJ9MphdD5bJlPSPyUtNvS7lo2mJKEfAMVlj5iki0Dze1kKSbgFTUmHzWvWphRFzYL7hKu1uvyG84z7kYUodeBhH6cgsSh7ii6depvpJVrNbwDKB4+CvXfSU861p9NnYH+/6h+Hu5855wIqLXJuDz5OqrycHTgSO2uNzGgvhKUIzj/SJyC/voa3fH4E3t/Hpvvy4USh/RD1GCdw12xr+sw9ahH8BiZngPUuHANjs4uCDadsUdsJbGYvY+VawsuNfep+K1y/rImD03SMkuJM4+rQEDLqfo2hNB9HNG4J39+zEHDfh1oUKuBHqrPzwsa7sNvr2uUNpE5z8qzY260nRgzN6rWQMWZihQ23dUjnDLfho6ihoT4DUSMtedODvugIoG9MTbgFaLQHvTVZaK4L+4/9fN9YObaeFB1WjWWvdhsrf+dkNCCgDcYsr4tORW2ESW9XvJQ6XNb5Ac6ESYMbFt7TuYE1T/qmSDnkGK3ONLs1kGdTk1mqH4aQuVYzeHLa4tc8BvmkIaZrAZoFitw9Z9CazdKCdmfUNGaeamodUK9SN9yDSrmtirVc9nrO3jbld+xS3VBCqyJtXHg6+ZGfQTqgntSjuY9Qqt1dMHWwPv2mUYJHBRXgzRrFwEooohASB+tUAdtwgV2bxuo47ThtUtYEnlYoj3vb4/yZBsw== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 01f245fb-4775-4e7b-c9ef-08da2ef4970a X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2022 00:08:46.9966 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: R/OhNmtSWVMZoA/ldNYZWB/F8An/LW4+IvlZiIxStpGJ64IY7H9UozXFG/HMfVRp X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB2934 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Use the existing vfio_device versions of vfio_(un)pin_pages(). There is no reason to use a group interface here, kvmgt has easy access to a vfio_device. Delete kvmgt_vdev::vfio_group since these calls were the last users. Reviewed-by: Kevin Tian Reviewed-by: Christoph Hellwig Signed-off-by: Jason Gunthorpe Acked-by: Zhi Wang --- drivers/gpu/drm/i915/gvt/gvt.h | 1 - drivers/gpu/drm/i915/gvt/kvmgt.c | 27 ++++++--------------------- 2 files changed, 6 insertions(+), 22 deletions(-) diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h index 5a28ee965b7f3e..2af4c83e733c6c 100644 --- a/drivers/gpu/drm/i915/gvt/gvt.h +++ b/drivers/gpu/drm/i915/gvt/gvt.h @@ -231,7 +231,6 @@ struct intel_vgpu { struct kvm *kvm; struct work_struct release_work; atomic_t released; - struct vfio_group *vfio_group; struct kvm_page_track_notifier_node track_node; #define NR_BKT (1 << 18) diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c index 1cec4f1fdfaced..7655ffa97d5116 100644 --- a/drivers/gpu/drm/i915/gvt/kvmgt.c +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c @@ -243,7 +243,7 @@ static void gvt_unpin_guest_page(struct intel_vgpu *vgpu, unsigned long gfn, for (npage = 0; npage < total_pages; npage++) { unsigned long cur_gfn = gfn + npage; - ret = vfio_group_unpin_pages(vgpu->vfio_group, &cur_gfn, 1); + ret = vfio_unpin_pages(&vgpu->vfio_device, &cur_gfn, 1); drm_WARN_ON(&i915->drm, ret != 1); } } @@ -266,8 +266,8 @@ static int gvt_pin_guest_page(struct intel_vgpu *vgpu, unsigned long gfn, unsigned long cur_gfn = gfn + npage; unsigned long pfn; - ret = vfio_group_pin_pages(vgpu->vfio_group, &cur_gfn, 1, - IOMMU_READ | IOMMU_WRITE, &pfn); + ret = vfio_pin_pages(&vgpu->vfio_device, &cur_gfn, 1, + IOMMU_READ | IOMMU_WRITE, &pfn); if (ret != 1) { gvt_vgpu_err("vfio_pin_pages failed for gfn 0x%lx, ret %d\n", cur_gfn, ret); @@ -804,7 +804,6 @@ static int intel_vgpu_open_device(struct vfio_device *vfio_dev) struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev); unsigned long events; int ret; - struct vfio_group *vfio_group; vgpu->iommu_notifier.notifier_call = intel_vgpu_iommu_notifier; vgpu->group_notifier.notifier_call = intel_vgpu_group_notifier; @@ -827,28 +826,19 @@ static int intel_vgpu_open_device(struct vfio_device *vfio_dev) goto undo_iommu; } - vfio_group = - vfio_group_get_external_user_from_dev(vgpu->vfio_device.dev); - if (IS_ERR_OR_NULL(vfio_group)) { - ret = !vfio_group ? -EFAULT : PTR_ERR(vfio_group); - gvt_vgpu_err("vfio_group_get_external_user_from_dev failed\n"); - goto undo_register; - } - vgpu->vfio_group = vfio_group; - ret = -EEXIST; if (vgpu->attached) - goto undo_group; + goto undo_register; ret = -ESRCH; if (!vgpu->kvm || vgpu->kvm->mm != current->mm) { gvt_vgpu_err("KVM is required to use Intel vGPU\n"); - goto undo_group; + goto undo_register; } ret = -EEXIST; if (__kvmgt_vgpu_exist(vgpu)) - goto undo_group; + goto undo_register; vgpu->attached = true; kvm_get_kvm(vgpu->kvm); @@ -868,10 +858,6 @@ static int intel_vgpu_open_device(struct vfio_device *vfio_dev) atomic_set(&vgpu->released, 0); return 0; -undo_group: - vfio_group_put_external_user(vgpu->vfio_group); - vgpu->vfio_group = NULL; - undo_register: vfio_unregister_notifier(vfio_dev, VFIO_GROUP_NOTIFY, &vgpu->group_notifier); @@ -925,7 +911,6 @@ static void __intel_vgpu_release(struct intel_vgpu *vgpu) gvt_cache_destroy(vgpu); intel_vgpu_release_msi_eventfd_ctx(vgpu); - vfio_group_put_external_user(vgpu->vfio_group); vgpu->kvm = NULL; vgpu->attached = false; From patchwork Fri May 6 00:08:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12840354 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C8B7C433EF for ; Fri, 6 May 2022 00:09:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1387409AbiEFAMp (ORCPT ); Thu, 5 May 2022 20:12:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47546 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1387386AbiEFAMi (ORCPT ); Thu, 5 May 2022 20:12:38 -0400 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2053.outbound.protection.outlook.com [40.107.236.53]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 56B6150B39; Thu, 5 May 2022 17:08:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AIJGH4P/HQ+5FgZT4GvfZapE13J18DlGcdAP9Ala3q6z4DdTWpQKDkOD7aLDLTwxArSOdt+cZst8TTSEOYolPw4HaxDKzMgYTNIWtQiS8Xz9TD0/Ss2oOH3CPZaDlZJiul7Q/rZPpBgzirjkIZGaxu1y6KzqgQ4MW3m9IMB0fVoTgBhiB1IafSOS4GeYnTRXzdSm8F/rGCkTaYB2FmO79fnQcLMY6PFPoUta8CnFgC1r6nBo9cu+9yi0J0oybpWbyvXZecNrd5xGLF/qmKrWPG2HX6MaYoyYEdUHTFSt7Hy8S8H6loq1r2weBYCjdetyPbY5aqnMS4bM6lgHmHBkwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1wtrIJOQpRDgxcZmQdrCfkkxhbwhBQUNhRJIwP0ntSg=; b=bbPtS3l8Ck1RondQ3YTa6GZyFc40KTXCv9FIMKl378BocCTD0t0wx0ZPxGvHlSBHmiq5UqkMMGOQ+cKJIZu5kXPhHjQh8cweN6gniGDp7lLj1QbxqEPv5G3cgaYsCAKkd53EC+P/vY+ewY6qHAHBxQcj8/R2OHUUElbeYyxuEHUqTzL95j+ASGzizeZ7UkVGV7n+HdGEcEtXk4SPFrhXg5HmpXJCHsR81TS2RHnWEwS7VqRIBq1AF1upfQRgCpWuCEy7E5nuxhL4kiCX03Ez5p2hXmBjGH0mtQLUkS/uV+OnlMlIKsivcaryIwpc+HvLEVZ0RzlSrgOl9zL9i7+ITA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1wtrIJOQpRDgxcZmQdrCfkkxhbwhBQUNhRJIwP0ntSg=; b=BRLPxmfHEnKHcLnsRMgCaU/RDHbICPlHFAjLd0mKOotL3dQAXEK3DxvUvscRSJ6KIODCpanFgceHHBpWtCZQzXqKDGEpmlbWNndw+amzLqV3ZzCTZ1nK8NW2PGl9s1nA58ZyzJIf+I3xLFkIkh6l7Vc8QpHoXGqslE0dpAI21E73tSyDEPeBVZP6Lh9eaALy6eAJIL6qmxURSlA0aKjAVaTPZkiEzEiLQX5HvF7KrZ8VkJQc0j3JYYxUccERNxZF1Mij8nfP1QGuzP68ny5NFx4LvEiEJLmkzt4J/P1RKXRhEtd8u0q6pQlBcCtbT+/TZ55ivlhYym/kSy4QrYZ+Ng== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BYAPR12MB2934.namprd12.prod.outlook.com (2603:10b6:a03:13b::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.24; Fri, 6 May 2022 00:08:51 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5206.027; Fri, 6 May 2022 00:08:51 +0000 From: Jason Gunthorpe To: Alexander Gordeev , David Airlie , Alex Williamson , Christian Borntraeger , Cornelia Huck , Jonathan Corbet , Daniel Vetter , dri-devel@lists.freedesktop.org, Harald Freudenberger , Vasily Gorbik , Heiko Carstens , intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org, Jani Nikula , Jason Herne , Joonas Lahtinen , kvm@vger.kernel.org, Kirti Wankhede , linux-doc@vger.kernel.org, linux-s390@vger.kernel.org, Matthew Rosato , Peter Oberparleiter , Halil Pasic , Rodrigo Vivi , Sven Schnelle , Tvrtko Ursulin , Vineeth Vijayan , Zhenyu Wang , Zhi Wang Cc: Tony Krowiak , Eric Farman , Christoph Hellwig , Kevin Tian , "Liu, Yi L" Subject: [PATCH v4 6/7] vfio: Remove dead code Date: Thu, 5 May 2022 21:08:44 -0300 Message-Id: <6-v4-8045e76bf00b+13d-vfio_mdev_no_group_jgg@nvidia.com> In-Reply-To: <0-v4-8045e76bf00b+13d-vfio_mdev_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR1501CA0036.namprd15.prod.outlook.com (2603:10b6:207:17::49) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: de61833c-58dd-4461-b5c9-08da2ef497e3 X-MS-TrafficTypeDiagnostic: BYAPR12MB2934:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YQe9NBAPbSC9BTiexdHW6dzTJfK6WKvYratYWGCPOiEEq2Y+JDTCZUu1J6at0EnO/p1bZ909jg7ahTvM+g6SgnWYrDn0LH5Zt2j94IWX4jkkWpWQWgdxjkrAenkTLJJoWUlXimx3LEgNQcjLVcOp+ATdQ52wsXdf/zfXCpdbHJDI/AH0+uK3cL6szHidllV1NPuuYigLRpldIN3V7N6LVv+bhKaiBdDPPo5a3w92frOfInKBUca+P6Rue5xPMA/P/XTdEzhAihJ23p8Kmt/eCg9YvQLu4ZoNKsZxt1uXZJl3yJ4Y+0U7yTrBQ719qpS+ujypmcfJCUts9Wz1/10BkMW20L0BwWLXcpnBgNUD+F9BlKfu8VBKGcelTP1s8AcZQbyV1BwP4NN2gdCJDPUiynGwLl1dD8C0WhG1u0nNyyvq7ElwNBNgoiVcB/+997/kt0sKm9llzloyr8p+CFo1qTJuxfkyZ3iJrl/JbM21UQNDqVYxftfh0MvKPmeYSNJHY3LthvT1YXIHACvMoMma2zQFrItJDo5rXJQ3BRqbSqBlxuI4yaAAU6gUYsgVCVUX7+dNGjjI8zSYY9IYZXaetneTQ5m+wEPOC1MmYjsQtlQDf1z5YpYpJKbXK7vsXVQcUd7PD4JX+zyiqQ4lEfEgn8E5BJOA8TIwz/hfxVES4aeNJSW1Pe5weE/Qn6tAnJ3QgtsFQTh6Ns/Ohp4MdEj/4w== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(5660300002)(2906002)(66946007)(66556008)(54906003)(8676002)(86362001)(110136005)(4326008)(6486002)(38100700002)(508600001)(316002)(66476007)(6666004)(6512007)(6506007)(186003)(2616005)(921005)(26005)(7416002)(8936002)(83380400001)(7406005)(36756003)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: TN9n1k7Y5YMAgedRWhQbf8Vha2spycWVVuvg/Dp1WcqRpiArJPFpECnXHSYPtZHlMPzg+kNRuYuOaWOMi6M0t4HH9KpsxDu2NcZOVMnZhk66cqyNue+zF0rxcWD10Wcn+k7NOCg9D5WTrSuJHDajoLhhsx59w/kqcrBlfvnuAAiUCFAnjny3pgy0bL7GncB7TmOikhvPRIam3ek/NUkbuRQ2DzQNhFrMOnwy73P54ayiV/oZmmdjDalZnJxbkUm8TwPGrHmhdci2BvkBUF9lKaQUGrIQlLqHwnhNhdDcBXwx4Fg5kAiuKJu+s7iSZElApIH/Fd/Q6Ga8G5rEohz6h725JwcMa1DpRd4M3YP7Et3pktwuO98nROmCTY+p4hWrjleECm7MFd4iw7wTtoEAfebtg9NsbfVAN1zyQrpBcaXI1FscjgFQEzB+QBgubtHog8K575SVEq7ux/cMAL2lPUNYc7nhnOA4O5pkCgvKNaZzpmSKTf2pgLrMhBrZQzF6S6Sxl1wfL4ANLsNSiOMIjcNMCFQrHNIWjMh+wLfTuGXN1I9TUycu+lvuNnTSuwUPm0je5nFEXQFiF43CjvfcyGu6zVnzGjlCTXBmj4vYCSBHogJ97V+QPFdYV1UDR1BWTO+rNrJJRwFyMJu2r/RDfglI32T6f2u/TS1UeMaSFbvgmG1XJv7ln8BSxMe5mESLE+TuUJ9A2NMQqb46H9m0erdTtA6dsTiDRzdTOAvXv3k3bjd7t5K7PpNQST+HClH3qiN4OI2IIcbHmN1CAIGUPio195jSykzVg97G+fDE/Q75ansi9p1TFWC2GwjNPNQSuZnSmu3QGd5XceGDg/H1ehW3pwaKKoTmnCpPnfy4iXMHvy36WIWi/GQJxwZuD56JpIuJ7i3WFa3oBN+uqGG7wdoejwrgqL1wegj4tFsO5kC+wDtC0SP9kD604o2jqeoUECjdZ2DoaMd5GG4GJQ/qpVGb5fsP8NukjDs3MoeOvBIql9pU9qLWWgTNG6vfLda9ew6t6+5e1pNbYIHnoZY/q3E3W/HXj0ieIjbT1DD49jlsgnbay4U2GPjwb9KuYKTFTWoJihfg30vU0wLrgNLmmdUUjtZxhVrUbZigUM9bJsKWpCh/c6yJxCuPeMNcsX2RL9be7Kovi7XlU5a4tT4Z1e9gcC8mEKwB7rrVPLdV1ldPKmeuL/UX6tZhvFql9bDQqohj3wOslbAZomZM+cJHTE3gl1Xc/6guyfRSaMNlxfe46EqCL4QyVPydE8XqeYVbhXtOkjykMvQeAcdFlEC+/03UTqvXifrhIO60Dsv2rT+e+YcAESEW4YTBdIbqTpSSft3Z/lfmXAko3TTxkxMf4kKc0sSbaJ4V8dc3qiDhsje+/8kLxA0SyrZ6b6XWKSg1gJF89ft+/0XRPY3GIsrKUj30i5zJ4VjI66y1x+hMMhPeSJjK5+yV8+Olga/8sGg4ya08qqpFRwPnafaXCdgaf4lbRptqNBm3+dks4XMtvmw7FsC2szIoqp+SRuqTrDF8NuIrHtEfcDOfCS2mBTAfpQGw/q6H5qygXASuQkd63k2wDq4U01E+NgUXdomxvkHtgocJe99Hge6nZzw8vSKEJajN0Oy9HLB5oepPDQM5yGMOnd51GMxBr/kK2wfzOqIHoUa/bzy0bD88sJUf0378ppeqqIv6S1ggfPF3snx3oYsHpk6yCKB/fmm3YQyi7vrPFU2k7uY0c2MhiZyRBR7NqA== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: de61833c-58dd-4461-b5c9-08da2ef497e3 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2022 00:08:48.3714 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 4FgDQHxGUqPI/ygt2CgZC8bXthyuGBm7BtnxFjkJEz16uTBhwkHTBss/tfWeY+3u X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB2934 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Now that callers have been updated to use the vfio_device APIs the driver facing group interface is no longer used, delete it: - vfio_group_get_external_user_from_dev() - vfio_group_pin_pages() - vfio_group_unpin_pages() - vfio_group_iommu_domain() Reviewed-by: Christoph Hellwig Reviewed-by: Kevin Tian Signed-off-by: Jason Gunthorpe --- FIXME: vfio_group_put_external_user() is removable too when combined with the KVM series --- drivers/vfio/vfio.c | 151 ------------------------------------------- include/linux/vfio.h | 11 ---- 2 files changed, 162 deletions(-) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 85e1304099b8a5..c651c4805acd59 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -1732,44 +1732,6 @@ struct vfio_group *vfio_group_get_external_user(struct file *filep) } EXPORT_SYMBOL_GPL(vfio_group_get_external_user); -/* - * External user API, exported by symbols to be linked dynamically. - * The external user passes in a device pointer - * to verify that: - * - A VFIO group is assiciated with the device; - * - IOMMU is set for the group. - * If both checks passed, vfio_group_get_external_user_from_dev() - * increments the container user counter to prevent the VFIO group - * from disposal before external user exits and returns the pointer - * to the VFIO group. - * - * When the external user finishes using the VFIO group, it calls - * vfio_group_put_external_user() to release the VFIO group and - * decrement the container user counter. - * - * @dev [in] : device - * Return error PTR or pointer to VFIO group. - */ - -struct vfio_group *vfio_group_get_external_user_from_dev(struct device *dev) -{ - struct vfio_group *group; - int ret; - - group = vfio_group_get_from_dev(dev); - if (!group) - return ERR_PTR(-ENODEV); - - ret = vfio_group_add_container_user(group); - if (ret) { - vfio_group_put(group); - return ERR_PTR(ret); - } - - return group; -} -EXPORT_SYMBOL_GPL(vfio_group_get_external_user_from_dev); - void vfio_group_put_external_user(struct vfio_group *group) { vfio_group_try_dissolve_container(group); @@ -2003,101 +1965,6 @@ int vfio_unpin_pages(struct vfio_device *device, unsigned long *user_pfn, } EXPORT_SYMBOL(vfio_unpin_pages); -/* - * Pin a set of guest IOVA PFNs and return their associated host PFNs for a - * VFIO group. - * - * The caller needs to call vfio_group_get_external_user() or - * vfio_group_get_external_user_from_dev() prior to calling this interface, - * so as to prevent the VFIO group from disposal in the middle of the call. - * But it can keep the reference to the VFIO group for several calls into - * this interface. - * After finishing using of the VFIO group, the caller needs to release the - * VFIO group by calling vfio_group_put_external_user(). - * - * @group [in] : VFIO group - * @user_iova_pfn [in] : array of user/guest IOVA PFNs to be pinned. - * @npage [in] : count of elements in user_iova_pfn array. - * This count should not be greater - * VFIO_PIN_PAGES_MAX_ENTRIES. - * @prot [in] : protection flags - * @phys_pfn [out] : array of host PFNs - * Return error or number of pages pinned. - */ -int vfio_group_pin_pages(struct vfio_group *group, - unsigned long *user_iova_pfn, int npage, - int prot, unsigned long *phys_pfn) -{ - struct vfio_container *container; - struct vfio_iommu_driver *driver; - int ret; - - if (!group || !user_iova_pfn || !phys_pfn || !npage) - return -EINVAL; - - if (group->dev_counter > 1) - return -EINVAL; - - if (npage > VFIO_PIN_PAGES_MAX_ENTRIES) - return -E2BIG; - - container = group->container; - driver = container->iommu_driver; - if (likely(driver && driver->ops->pin_pages)) - ret = driver->ops->pin_pages(container->iommu_data, - group->iommu_group, user_iova_pfn, - npage, prot, phys_pfn); - else - ret = -ENOTTY; - - return ret; -} -EXPORT_SYMBOL(vfio_group_pin_pages); - -/* - * Unpin a set of guest IOVA PFNs for a VFIO group. - * - * The caller needs to call vfio_group_get_external_user() or - * vfio_group_get_external_user_from_dev() prior to calling this interface, - * so as to prevent the VFIO group from disposal in the middle of the call. - * But it can keep the reference to the VFIO group for several calls into - * this interface. - * After finishing using of the VFIO group, the caller needs to release the - * VFIO group by calling vfio_group_put_external_user(). - * - * @group [in] : vfio group - * @user_iova_pfn [in] : array of user/guest IOVA PFNs to be unpinned. - * @npage [in] : count of elements in user_iova_pfn array. - * This count should not be greater than - * VFIO_PIN_PAGES_MAX_ENTRIES. - * Return error or number of pages unpinned. - */ -int vfio_group_unpin_pages(struct vfio_group *group, - unsigned long *user_iova_pfn, int npage) -{ - struct vfio_container *container; - struct vfio_iommu_driver *driver; - int ret; - - if (!group || !user_iova_pfn || !npage) - return -EINVAL; - - if (npage > VFIO_PIN_PAGES_MAX_ENTRIES) - return -E2BIG; - - container = group->container; - driver = container->iommu_driver; - if (likely(driver && driver->ops->unpin_pages)) - ret = driver->ops->unpin_pages(container->iommu_data, - user_iova_pfn, npage); - else - ret = -ENOTTY; - - return ret; -} -EXPORT_SYMBOL(vfio_group_unpin_pages); - - /* * This interface allows the CPUs to perform some sort of virtual DMA on * behalf of the device. @@ -2301,24 +2168,6 @@ int vfio_unregister_notifier(struct vfio_device *device, } EXPORT_SYMBOL(vfio_unregister_notifier); -struct iommu_domain *vfio_group_iommu_domain(struct vfio_group *group) -{ - struct vfio_container *container; - struct vfio_iommu_driver *driver; - - if (!group) - return ERR_PTR(-EINVAL); - - container = group->container; - driver = container->iommu_driver; - if (likely(driver && driver->ops->group_iommu_domain)) - return driver->ops->group_iommu_domain(container->iommu_data, - group->iommu_group); - - return ERR_PTR(-ENOTTY); -} -EXPORT_SYMBOL_GPL(vfio_group_iommu_domain); - /* * Module/class support */ diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 8a151025871776..6195edd2edcd7b 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -140,8 +140,6 @@ int vfio_mig_get_next_state(struct vfio_device *device, */ extern struct vfio_group *vfio_group_get_external_user(struct file *filep); extern void vfio_group_put_external_user(struct vfio_group *group); -extern struct vfio_group *vfio_group_get_external_user_from_dev(struct device - *dev); extern bool vfio_external_group_match_file(struct vfio_group *group, struct file *filep); extern int vfio_external_user_iommu_id(struct vfio_group *group); @@ -154,18 +152,9 @@ extern int vfio_pin_pages(struct vfio_device *device, unsigned long *user_pfn, int npage, int prot, unsigned long *phys_pfn); extern int vfio_unpin_pages(struct vfio_device *device, unsigned long *user_pfn, int npage); - -extern int vfio_group_pin_pages(struct vfio_group *group, - unsigned long *user_iova_pfn, int npage, - int prot, unsigned long *phys_pfn); -extern int vfio_group_unpin_pages(struct vfio_group *group, - unsigned long *user_iova_pfn, int npage); - extern int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova, void *data, size_t len, bool write); -extern struct iommu_domain *vfio_group_iommu_domain(struct vfio_group *group); - /* each type has independent events */ enum vfio_notify_type { VFIO_IOMMU_NOTIFY = 0, From patchwork Fri May 6 00:08:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12840355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C33DC433EF for ; Fri, 6 May 2022 00:09:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1387423AbiEFAMs (ORCPT ); Thu, 5 May 2022 20:12:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47546 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1387389AbiEFAMi (ORCPT ); Thu, 5 May 2022 20:12:38 -0400 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2048.outbound.protection.outlook.com [40.107.236.48]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7D842506E2; Thu, 5 May 2022 17:08:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ypida9ZNEFagFH3myqSIFQi6ldDtls3+P0jxruTxhbX9c8eE7NTC8+W/xYtkRt/dD3BpOwsXa4lpgIhFnNIIIVTgsDxWg8hB/+N3+j/ofaoOyoGSSl67gGpW3PGMUGwwpBgSjAUoOdk6vEDpK8aX+30ZcNf1MM6pT0UgOZv2UljRzW/O+BD/ajQ/seXSBF9v6cdaBjohHhYX/TXqoOf/IK/o2RJBiqacaEoB4AkN+WpGAD0itjjhpKY5NZpdfQRhJxT/DdiQhkHW4MoU51efVtLsNqIiDNv1gMax57dc2LC5UyQsqhFJjhsBEzNLcubmnJBqvKxJES+TMSNgUYPEDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BGsDroh7318M7IFYvIVTolUnNNiYq8HH8XrljJJ+kew=; b=JhkdRZ1d+22I/p0d9yhHvFRzvWrhS13kRK4eMnzhDrgS66yLE1t0dq1PPfToXy0i2gtfYqfcv7avWXMivycYZqKaV/AFf4pcZEIWICeoJN0H4hvmomEvxpJeTu73TB3f5PDdT2oDwJjZK8D8D/ttczshH10nCpeXO6CyIoDFqmmMcl9FGm/vrtAh1i+AS+Wvj3PuuULHNliN2L1r0TvJSXp79X//lEWousPahB1fA4cTjCc3hDwgf2veY9yZTwoQXUc5n4aC5VfpGGPTvr01JmPx4/Nx0aezo6UzUjslJx79FDiV2oXTZcnEr/Kta/73lZh8dhKt94QDKCEawQRWZA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BGsDroh7318M7IFYvIVTolUnNNiYq8HH8XrljJJ+kew=; b=icsCAIUQ1I0rdU9TIgStTYSz56LZ4szZaDbM8XGMBSpeFBKDQlxyJ9V1pgeoxBNsStNRTP2sJU5viszKb37c5vJdoarSBv1h0BvcgzkirVMSEQjf7p8XcM5u2TxzkmZ6HQrJ+optOOFwNhi6U5bg5h+jtNfxmUT7E6bdlo/QBW8336dTsy4IqT/xI1PsTgr5vbIA5Ov398g3Qfm5l1EDv+UTMKwfw8gaDH3POsIscaYs4VJbMBlD/CkJzR7vrVeeuSHrVtMOPGfzjyYjqi/bq3h7jaj5TmRvJtBXXqqaKxDV5WPf7ctSKOUUIA1YE3oooL7rNSn8CMe6D3jmjMErrw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BYAPR12MB2934.namprd12.prod.outlook.com (2603:10b6:a03:13b::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.24; Fri, 6 May 2022 00:08:52 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5206.027; Fri, 6 May 2022 00:08:52 +0000 From: Jason Gunthorpe To: Alexander Gordeev , David Airlie , Alex Williamson , Christian Borntraeger , Cornelia Huck , Jonathan Corbet , Daniel Vetter , dri-devel@lists.freedesktop.org, Harald Freudenberger , Vasily Gorbik , Heiko Carstens , intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org, Jani Nikula , Jason Herne , Joonas Lahtinen , kvm@vger.kernel.org, Kirti Wankhede , linux-doc@vger.kernel.org, linux-s390@vger.kernel.org, Matthew Rosato , Peter Oberparleiter , Halil Pasic , Rodrigo Vivi , Sven Schnelle , Tvrtko Ursulin , Vineeth Vijayan , Zhenyu Wang , Zhi Wang Cc: Tony Krowiak , Eric Farman , Christoph Hellwig , Kevin Tian , "Liu, Yi L" Subject: [PATCH v4 7/7] vfio: Remove calls to vfio_group_add_container_user() Date: Thu, 5 May 2022 21:08:45 -0300 Message-Id: <7-v4-8045e76bf00b+13d-vfio_mdev_no_group_jgg@nvidia.com> In-Reply-To: <0-v4-8045e76bf00b+13d-vfio_mdev_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR02CA0014.namprd02.prod.outlook.com (2603:10b6:208:fc::27) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e75e0ca0-a46e-419d-033c-08da2ef497f2 X-MS-TrafficTypeDiagnostic: BYAPR12MB2934:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: b8LNGpS/fzCumYu9pIiCr4gF2ktWu4z9bm9fBTzTP1Md7mRLlXtnTjPvlzDg6LxhSiot+uubJzMee8OWIwmT2PhLOO+QFRFKtvFrZe7texfz0sS9JNDLqSgPKu6155J2si3XhB2wivvb5NWVR5SggDxOefVtumP38YoYKJs1Ing6aEcwta6xEH6MnDQikzuf3W1JWRQbVzL0MipxiKsc5+XqA9Y1xYE+QFYca+RtVrSzyy/0bmAtI5wI0IowNsupTkSRAms7/2v7I4KCDOI1V8i/h+dbC2TNETqsuFbZ2o5EYtA3hK+p2Pt7o+OiuU8KRvpu7m01npj+XytpEKUTL6us35P/R8TlPIlfcEUe+bMC/Wcrx5tCvT2UedTclOazMq0uxTUiH3ELD5BFD2MFkFYYOol6CQiqdlIuOlYGjO2iBsrZqPjQBk4kHzNn88MzJCMoSUrM/4MCciFbKdmKfWrgxfDWWFdwUFBoiy1+ye1HttrNGc9lLD3luRu1GK+HuiVBQncapWLQqzeOXq/dFpmooZrU3hn8BzoYf7LWAwMeKnIpLbRtEl3Mz2JjX7Goq0CalfOa1Uu0md8sOGOt4E2ZSFFg0GQLHF7mdh/dd6VgffKq9TpbGxI97CrQHtzZREFtQ9BxSXHGBmQdOu+pm3kbqq+PZ+e0SkSCLVPUIsUzNOa2LkKbDcDf14opkrQHNWSku4Sp5lIcgKDeIddrtw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(5660300002)(2906002)(66946007)(66556008)(54906003)(8676002)(86362001)(110136005)(4326008)(6486002)(38100700002)(508600001)(316002)(66476007)(6666004)(6512007)(6506007)(186003)(2616005)(921005)(26005)(7416002)(8936002)(83380400001)(7406005)(36756003)(4216001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: AZzzVS6y7ADOqpqDnmazmBdP6pQnC/uVqg+fB+6D6ml3ec0zTI9vSls041ezoGOhp+djy7z/mIPVXegNHRpq/GLLUzwspAky7XTkyDxL352gmyCabbw1SRQTPn/eVVEcUyYIqHH43LbQSi+ZTZV0IscnF1Q+HD9jODe57IZum29EePMSKd2g1L/vSRx/Rdbk9p9XYHPADJ/1ZmSS6mLzjG3hbB5OQVkp567RxFJL3498FH08ELjnRFRDRBHgyQvs1nIKelDQgfP19EKH3ZNKAPYoN044p4Eul7UnCehcuJrET2QEufI/jCaPMjoPgeaQ8sgqW1jlbrJrx1QP7rS7iGRWUBXL3WP8biZWIG0qa0/L7zlLf2NgBjp8AXZa7cuHv+5dNSGkFErmjaFV5d1JC6F7u8q65pQ9ziUf9fgnCQR4ydsXyTUVkShkJRp2ik+fZWT/7dvloxvqMdbVilz+qNUJte2KbbAzv+0h2iofVGrubZ9iO0WC/Pc67/bfi3j5dZ92p/hkafOkNcEoXo/o96IWF6qhQUH3G/9rh5FpiqSZLMomN6AcFnCCLvonCPLbMtbp1LQNYf4Qn4Fc5RCcW0pgQsSNQ6MwVFku9Sit0Kh/3TQy7e819vhlQLwWAyykkV3zjO+94LkxaChyCLc92R/N7wPLFk6mhknzahP+CWMGJXhhF/0p/K9nco/fxD60wW3ri8X/fFMUKfr2m5OOA+HciCgzPu2x7gTULM+b7j7FiHvAJfQ+w0FQyV+UtPHmbhwf12QqOWlh0JL76f9EGTnWCTRhNDmVBO95+x2M7ABMfe0LkE6TKgHYE1s1J7ngFiDoJMtfHChfd+bZ9A/l/PnCgRSlN9fgPmPivHDriExDA+XaZHL1XSJfCOwkhRONxLXj9tjOva1mIrcjeI9ssvZnIiNeSEMHciXqvEstBASbuUtJeH4UksRNTeK1kSdns4ER/JlmsxdFB0eRzzgqCXH0v7amgFfq2/azfTe9GQaY5lDcrGbDlx7KJy9YGCNUZjsgXxN8w6KcyA3CWqjw0nN7RBeBI7qOclgxI66BAUr2zt79DlbNRoNZ2zorVnuLpcbDAR0RFj3oSjMoPIZs4cadU/nB7gwTCY5H+4UPWNCcdPQ+gE3ur8xwHlpWlApUySe3t4Nfx5iO/DJYO0hBBWjhGsVOvwTMx9JNYgfjmhkAVrFRj9sNHCmmZmAOij/O31qf88vIUreDtwodp1TUuUMrdAYNFGk2Zhe9RYzNwQ8y3h4S7wtZN4bJoQGLsnBDWgO3HFlS5KXyZUVpNm/X8wgfSa0vkPb3m9N4sOEgv/7D4rnAYiKDXhrRHHaikIv5t33NWl/3nu3dDKA/xW5SBbJeGuwtNw2E3dzMGIzN2uWvmWDQT/l/I9qpNXx8M5iHGiPErwEjMILfDeGZmyVZh2x+XZHY2BfvupR971hrwm4AJAxptDbJZ4KGQJm9YhvImzkfiwXg0GiSYif3QyLDzGDtjqpkjagRIZiudMTEePOMI0gPTGIj11dwumVjkM25ya7/QBWniVm3CA+Uf0DfJC9xg8UaTslzQ/iBq2sWzlQEZjuqerPJDRZo6ruEj6BgRkrDDfUDt7GxvwR6RmQi6TnASCEg1iz519T+aIEYrnstpoROhirtwyqZZzesuv5GQ8JSho/hfiONUne0JGOcCW0DGEcj+CZ5OK4tgdUi9y1u/NeVWQR73TAp50DpQI+UXsDETC4Wml0x2BzSOeJwsA== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e75e0ca0-a46e-419d-033c-08da2ef497f2 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 May 2022 00:08:48.4495 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1V/dWNUDTf5dRUdWrNNSzLfiuxiQfQvMtkTA4BPDhzL33lTzLZFvYtO1gIyRkD9u X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB2934 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When the open_device() op is called the container_users is incremented and held incremented until close_device(). Thus, so long as drivers call functions within their open_device()/close_device() region they do not need to worry about the container_users. These functions can all only be called between open_device() and close_device(): vfio_pin_pages() vfio_unpin_pages() vfio_dma_rw() vfio_register_notifier() vfio_unregister_notifier() Eliminate the calls to vfio_group_add_container_user() and add vfio_assert_device_open() to detect driver mis-use. This causes the close_device() op to check device->open_count so always leave it elevated while calling the op. Reviewed-by: Christoph Hellwig Reviewed-by: Kevin Tian Signed-off-by: Jason Gunthorpe --- drivers/vfio/vfio.c | 80 ++++++++++----------------------------------- 1 file changed, 17 insertions(+), 63 deletions(-) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index c651c4805acd59..8bb38941c1dfd8 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -1115,6 +1115,12 @@ static int vfio_group_add_container_user(struct vfio_group *group) static const struct file_operations vfio_device_fops; +/* true if the vfio_device has open_device() called but not close_device() */ +static bool vfio_assert_device_open(struct vfio_device *device) +{ + return !WARN_ON_ONCE(!READ_ONCE(device->open_count)); +} + static int vfio_group_get_device_fd(struct vfio_group *group, char *buf) { struct vfio_device *device; @@ -1329,8 +1335,10 @@ static int vfio_device_fops_release(struct inode *inode, struct file *filep) struct vfio_device *device = filep->private_data; mutex_lock(&device->dev_set->lock); - if (!--device->open_count && device->ops->close_device) + vfio_assert_device_open(device); + if (device->open_count == 1 && device->ops->close_device) device->ops->close_device(device); + device->open_count--; mutex_unlock(&device->dev_set->lock); module_put(device->dev->driver->owner); @@ -1897,7 +1905,8 @@ int vfio_pin_pages(struct vfio_device *device, unsigned long *user_pfn, struct vfio_iommu_driver *driver; int ret; - if (!user_pfn || !phys_pfn || !npage) + if (!user_pfn || !phys_pfn || !npage || + !vfio_assert_device_open(device)) return -EINVAL; if (npage > VFIO_PIN_PAGES_MAX_ENTRIES) @@ -1906,10 +1915,6 @@ int vfio_pin_pages(struct vfio_device *device, unsigned long *user_pfn, if (group->dev_counter > 1) return -EINVAL; - ret = vfio_group_add_container_user(group); - if (ret) - return ret; - container = group->container; driver = container->iommu_driver; if (likely(driver && driver->ops->pin_pages)) @@ -1919,8 +1924,6 @@ int vfio_pin_pages(struct vfio_device *device, unsigned long *user_pfn, else ret = -ENOTTY; - vfio_group_try_dissolve_container(group); - return ret; } EXPORT_SYMBOL(vfio_pin_pages); @@ -1941,16 +1944,12 @@ int vfio_unpin_pages(struct vfio_device *device, unsigned long *user_pfn, struct vfio_iommu_driver *driver; int ret; - if (!user_pfn || !npage) + if (!user_pfn || !npage || !vfio_assert_device_open(device)) return -EINVAL; if (npage > VFIO_PIN_PAGES_MAX_ENTRIES) return -E2BIG; - ret = vfio_group_add_container_user(device->group); - if (ret) - return ret; - container = device->group->container; driver = container->iommu_driver; if (likely(driver && driver->ops->unpin_pages)) @@ -1959,8 +1958,6 @@ int vfio_unpin_pages(struct vfio_device *device, unsigned long *user_pfn, else ret = -ENOTTY; - vfio_group_try_dissolve_container(device->group); - return ret; } EXPORT_SYMBOL(vfio_unpin_pages); @@ -1989,13 +1986,9 @@ int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova, void *data, struct vfio_iommu_driver *driver; int ret = 0; - if (!data || len <= 0) + if (!data || len <= 0 || !vfio_assert_device_open(device)) return -EINVAL; - ret = vfio_group_add_container_user(device->group); - if (ret) - return ret; - container = device->group->container; driver = container->iommu_driver; @@ -2004,9 +1997,6 @@ int vfio_dma_rw(struct vfio_device *device, dma_addr_t user_iova, void *data, user_iova, data, len, write); else ret = -ENOTTY; - - vfio_group_try_dissolve_container(device->group); - return ret; } EXPORT_SYMBOL(vfio_dma_rw); @@ -2019,10 +2009,6 @@ static int vfio_register_iommu_notifier(struct vfio_group *group, struct vfio_iommu_driver *driver; int ret; - ret = vfio_group_add_container_user(group); - if (ret) - return -EINVAL; - container = group->container; driver = container->iommu_driver; if (likely(driver && driver->ops->register_notifier)) @@ -2030,9 +2016,6 @@ static int vfio_register_iommu_notifier(struct vfio_group *group, events, nb); else ret = -ENOTTY; - - vfio_group_try_dissolve_container(group); - return ret; } @@ -2043,10 +2026,6 @@ static int vfio_unregister_iommu_notifier(struct vfio_group *group, struct vfio_iommu_driver *driver; int ret; - ret = vfio_group_add_container_user(group); - if (ret) - return -EINVAL; - container = group->container; driver = container->iommu_driver; if (likely(driver && driver->ops->unregister_notifier)) @@ -2054,9 +2033,6 @@ static int vfio_unregister_iommu_notifier(struct vfio_group *group, nb); else ret = -ENOTTY; - - vfio_group_try_dissolve_container(group); - return ret; } @@ -2085,10 +2061,6 @@ static int vfio_register_group_notifier(struct vfio_group *group, if (*events) return -EINVAL; - ret = vfio_group_add_container_user(group); - if (ret) - return -EINVAL; - ret = blocking_notifier_chain_register(&group->notifier, nb); /* @@ -2098,25 +2070,6 @@ static int vfio_register_group_notifier(struct vfio_group *group, if (!ret && set_kvm && group->kvm) blocking_notifier_call_chain(&group->notifier, VFIO_GROUP_NOTIFY_SET_KVM, group->kvm); - - vfio_group_try_dissolve_container(group); - - return ret; -} - -static int vfio_unregister_group_notifier(struct vfio_group *group, - struct notifier_block *nb) -{ - int ret; - - ret = vfio_group_add_container_user(group); - if (ret) - return -EINVAL; - - ret = blocking_notifier_chain_unregister(&group->notifier, nb); - - vfio_group_try_dissolve_container(group); - return ret; } @@ -2127,7 +2080,8 @@ int vfio_register_notifier(struct vfio_device *device, struct vfio_group *group = device->group; int ret; - if (!nb || !events || (*events == 0)) + if (!nb || !events || (*events == 0) || + !vfio_assert_device_open(device)) return -EINVAL; switch (type) { @@ -2151,7 +2105,7 @@ int vfio_unregister_notifier(struct vfio_device *device, struct vfio_group *group = device->group; int ret; - if (!nb) + if (!nb || !vfio_assert_device_open(device)) return -EINVAL; switch (type) { @@ -2159,7 +2113,7 @@ int vfio_unregister_notifier(struct vfio_device *device, ret = vfio_unregister_iommu_notifier(group, nb); break; case VFIO_GROUP_NOTIFY: - ret = vfio_unregister_group_notifier(group, nb); + ret = blocking_notifier_chain_unregister(&group->notifier, nb); break; default: ret = -EINVAL;