From patchwork Mon May 2 17:31:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12834559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1E6DC433EF for ; Mon, 2 May 2022 17:31:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386668AbiEBRfO (ORCPT ); Mon, 2 May 2022 13:35:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40272 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348252AbiEBRfK (ORCPT ); Mon, 2 May 2022 13:35:10 -0400 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2063.outbound.protection.outlook.com [40.107.93.63]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D07516268 for ; Mon, 2 May 2022 10:31:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Rs/g1Y4rIF+Z31aVmUQT5BfBHWH2HUUDUT13iR4fXMnQseV0O9LnhgzAyvoJ2tcmmWCOgmbu8ztzksbnYwl24FL1I7YG0U5C1HHK2u/c1n4+oJEJqq4ipCbbhWujqAgydWPxaWm8ADSLp+BhZHamqrUMhICnUPlt7OWvdaes40pykSewGNFL+yj9eMeT8/xtOM0ykQRjGlQ3Sr1se4PPr7kwk4FOmeaTDCg5mbMzKwrqLVkwt259IGqS+KlokzNU/7zBD7pOxAZzv9gg4nwHRbKLa0U3YBkDx5wYhcJBaaFDKpyfdLcIFk4k+87ccSx/V90AJgYOJWkxrtOS5zoHfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rhKOFvPBOr57GR3wpx0tt61S6YsPM0lONuG7hz9QHl0=; b=oDkZM+qE3uD/G41gIvhcfRofSs2FyRsXzulDSmWZeYzRBzivfgmjRWV7fK/Vr2jt4aWF0pdVMAJFycJBDD9NjdF+OCuAnqb0wsuCHzMNysTDr81xyNIcMSvc2tIOjl/qvs69n4CgQoQucM76XfQ/BhCHe9uLo2/kyQqMfJWa/zeHhrg6adkiCs/WqUKjQ+vw4y12BN2nEi5v9BeQsOEt4hTyD+f8Oo7JyFfYTVRShkzIkwC2lTAGKIdxX0n9sFNa2dWunRF8LwiRS0uOXlW3bhwjWKqbRnPb6PpbFcTlwiffkwPG7yw39+oL0QWUcwBAncko3pKKnlfpFrqklpQ0Kw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rhKOFvPBOr57GR3wpx0tt61S6YsPM0lONuG7hz9QHl0=; b=o48yLoW487gHsGjEUfyxnSD+8/mDy1GTGsdAbFe5NNTibx7fSwAOogprBQFZeR+r93Bnradd7YLFs5p2IzIEDQFfnQeDCg+4IhIv54vusupL/4nE6A+5fK5m+IyrY7OAVBdk1Zg1M2mFzLs28Ty+WLB7ZxPuYcb14I+BQ0ju4H+OdqD6wZYEeGN/O5ejaPPeMLrlHjigIGvgt94clAG97yyMGA2w7v/sfqXzT/h1yY9LQDqHC2rbcKd0YJF4964Xp9bf4RsvVnEGCFGhKOAyR3qlpg3ICpIlUF6FC4Ebpg0iOl6cpGdQO//O1xM/BW6IV/z1P56owhK1hc5O0TqB1g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by CO6PR12MB5458.namprd12.prod.outlook.com (2603:10b6:5:35b::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.13; Mon, 2 May 2022 17:31:39 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5206.024; Mon, 2 May 2022 17:31:39 +0000 From: Jason Gunthorpe Cc: Tony Krowiak , Eric Farman , Christoph Hellwig , intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org, "Jason J. Herne" , "Tian, Kevin" , kvm@vger.kernel.org, "Liu, Yi L" Subject: [PATCH v3 1/7] vfio: Make vfio_(un)register_notifier accept a vfio_device Date: Mon, 2 May 2022 14:31:31 -0300 Message-Id: <1-v3-e131a9b6b467+14b6-vfio_mdev_no_group_jgg@nvidia.com> In-Reply-To: <0-v3-e131a9b6b467+14b6-vfio_mdev_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR02CA0055.namprd02.prod.outlook.com (2603:10b6:207:3d::32) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0bf891b4-730a-49b0-511a-08da2c619cbe X-MS-TrafficTypeDiagnostic: CO6PR12MB5458:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yW3aUvfG/Bb6zCK7htv95z0tYxmKTz4B6sNUL4rd0IcdK3ns2CLt+ppop/0iIpqnOV2ykoTBZTCoHQ7EmXMLyeEWoICm8oprJxtpRYG4wuNG6MLK1aMXT3y3Ej2IDvuKLF/bsTxC+MCS8g51Nh2gPmbts1wxcL4o/o93NaQGMDDDDUUVaXDmV55TF6WpX5iAQBMJXkufnbfZgrXxcDv2O/POSo+DLDtGTsIUcdOluUQLjiVPq1Wa/IIxDsAWH6LNJpQmYZ1Nl5I2c5zFB79+6mibIL8XBdp5DD16IIwd7xvivgmV9T68Uf57kThfW/IGDWB/3f6uZ5ngm8L9qdpNnR5alEvSUIjoM9zQAGNv0wNYbW3aU3kzK0HquVxjdr+ygimp9EvQFRLhNQrmDbNbi1GKxIib0EmVWvRL6AjO/WU4pAAZSBsXm/VT3TlpMcDPDqudQCsQnVPmkEIEUFM7sQt6LflaVCghQbvcVdxHZnuc7uy59T56RfB/6cGl4U28CUlSSz2uXjyNDiSwAcYUe6PgpSt5ktdIPZHqzO/iiBhOvLuvXv8gGP7W34t7cmEd6txJc5PzEygYyrUsfYP3MyZCf3kfg6kxREWMeEshJtIff9JntJHkeXWmD10/9igsS3CrgpnXs6CmKErt5ZrwKudbsmzwhRQMRN/bho8jMkCOqY8LGswLH6xnGC1UI4wdoh7jS8mc3pfOwQrinoFZAIwrxY8lNPX64TxnGrEsPMk= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(109986005)(6486002)(508600001)(2616005)(86362001)(6512007)(26005)(6506007)(38100700002)(6666004)(316002)(66476007)(2906002)(66946007)(66556008)(186003)(36756003)(5660300002)(8936002)(54906003)(4326008)(83380400001)(8676002)(4216001)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: Jncrx2KeKtNh0a2E+BJx9IZB7vwxuAkagEW/118DzGgTCrZUS2vZKy1xu1S2mw6I+bmkoxoErAXvN9I4L2PRPz70EJYeDxzKhrYe3ALePcwwdBrtPg4fMyGiqyfAHFQsnRKdVyykTRA8jWFmq+sqTiHNTcU4c+j3RgkK40ZmI3PD8aZ+DBBKHt1i44IZ+h/2xNA0ZtTww5gYoM6Pnuc6MjLsJ+Q4XGJ/0SmnkD9DANqDQIZysIDZdSFSTRPkuMVhAy1en/kCd8Ty4JHdyi0b8bHcgQz+hOA9s04o8SZS1252W9+rCtbPDPkKBaDrKe+iS5IXn/K4C3iBnzpLRh1MWb37y32A40QsGuvQiQ326vsMP89Zxb+RExDHN+dDGBY+0LETRgXSlDnG3uE85Lgu4xsBp+KjXWtrlxLYvfQE1KcgKvEddl4BXRSLV8QvJB7NTITtaiWCFEBY9/gaSS5PhdUKvpMeQBeDVgJOAafUCfrX7p4halIw/3a6iGXHEgKgMYIt5AuxwiilaHwRLmB4jP8buYTjr9pnEPT3BgtjRkxSt6x6E8PCuqUiRQX9gSE89jTxDPd+VdUcx9GgNSeIyhtlCBrTyjY7xtgG2UkrCxPivs/FDGJ6jwuiu63Uit19w5axsfKt1B7Ppr0jqeSOn8nfZ5OCnMrEDKxSsrjlUSrqrCqesFq67b6etDwSnKWl4ZuFIUJerKZkw0FjAn15Xd8HS6ftJAX4bzi6jFq41GxPIzPML7ciq1Ahz2kk1V0Nz0jBsWh/+AkHfUXy/gc0I83b8amHCwYwAEhdlIwLleHGcXX7HtD1nkQ/kuuEVN+dTYuJ7mKLzYXF/zzNwp/NqBIB6jPIJaheEINaJPlbQalh/d+ylXSJrhEwpXTLENZsr+c/ijk08nZmHyRN/WafbT83ejzC0bn3wZjHrXfdamXC/VZX3Q5hNMezhnRyF+FQ6Hzlo0DOjYDkKwqQtB3Mq1z9mNOJO9tSXX4+bVS7LPOs3klWuYJkaX6wqSPHyH8/IeN1CFVV5awj56/uDh44eYX/WdBeC4wpnpuLZkyIfP4mymO2lcph9iAJwNRhjBcZD7nfhnd83M91CagQTkrQCN7hNfTkWo2Ys9FbWyj/hvVgNW/F2oTkDIyY3NOR5e7G7wQcJf1rxxZSVKtbQTDcgMnKtzBwZlEauJmpFl/g0AdlMl8hXaXJUTHze8jVtafJqvc3POm8p2FD4ZBht9RedZtd5Z62tDeIBD9flTcUHn57rr9+NDUhdwP3xOPMyV1OaezeaKqHymfTpvcNkpVnGFnV9xgxlitBFVp7vBeKEGdh/bEzYA6QHTOGjSn+H9MSzFw9l1YBVo+9Y/eWKJLeYDjeF0ZFK2/GDEYDqkeEu8/EVysae9a/eS2AaibXoZsc8VCCvNQh/XybGG6kc45iGjcG3XZY2gBt5xkbp8PZSgOv+i3iaTdBl5BoMAvhDfvdUWUMgG2U+kjZZJs2B7cvInra31owidNFvaOY4M1Fy5o0aDuufRxpM9hX6fRQfdTN5FTG6JwwViyX5BFZEZotwz8nKCbFHSocM6tXmDKeJCrDxDs8+Il/lIZBEMdFvvWvbeh8rXu7VPpejIgymwh0yBi1v5K+4hqBu3STC06uWvpOhRrWnAl+MuJaQTy7ZLOssS5To65c2qEwGZKsjCqwTo6lqDZoWR7H91v7hUMYk2KvYHqlAqPqua/DoLj8NymurUYzsgldJK+CKgCvafTfzw== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0bf891b4-730a-49b0-511a-08da2c619cbe X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2022 17:31:38.1389 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fLEUGzveikM3Slp/20IGtkMQW3okJzHyIZvkm/htZ2IjdNptGplF9UtHwLIK3bjg X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR12MB5458 To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org All callers have a struct vfio_device trivially available, pass it in directly and avoid calling the expensive vfio_group_get_from_dev(). Acked-by: Eric Farman Reviewed-by: Jason J. Herne Reviewed-by: Tony Krowiak Reviewed-by: Kevin Tian Reviewed-by: Christoph Hellwig Signed-off-by: Jason Gunthorpe --- drivers/gpu/drm/i915/gvt/kvmgt.c | 24 ++++++++++++------------ drivers/s390/cio/vfio_ccw_ops.c | 7 +++---- drivers/s390/crypto/vfio_ap_ops.c | 14 +++++++------- drivers/vfio/vfio.c | 25 +++++++------------------ include/linux/vfio.h | 4 ++-- 5 files changed, 31 insertions(+), 43 deletions(-) diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c index 0787ba5c301f5e..1cec4f1fdfaced 100644 --- a/drivers/gpu/drm/i915/gvt/kvmgt.c +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c @@ -810,8 +810,8 @@ static int intel_vgpu_open_device(struct vfio_device *vfio_dev) vgpu->group_notifier.notifier_call = intel_vgpu_group_notifier; events = VFIO_IOMMU_NOTIFY_DMA_UNMAP; - ret = vfio_register_notifier(vfio_dev->dev, VFIO_IOMMU_NOTIFY, &events, - &vgpu->iommu_notifier); + ret = vfio_register_notifier(vfio_dev, VFIO_IOMMU_NOTIFY, &events, + &vgpu->iommu_notifier); if (ret != 0) { gvt_vgpu_err("vfio_register_notifier for iommu failed: %d\n", ret); @@ -819,8 +819,8 @@ static int intel_vgpu_open_device(struct vfio_device *vfio_dev) } events = VFIO_GROUP_NOTIFY_SET_KVM; - ret = vfio_register_notifier(vfio_dev->dev, VFIO_GROUP_NOTIFY, &events, - &vgpu->group_notifier); + ret = vfio_register_notifier(vfio_dev, VFIO_GROUP_NOTIFY, &events, + &vgpu->group_notifier); if (ret != 0) { gvt_vgpu_err("vfio_register_notifier for group failed: %d\n", ret); @@ -873,12 +873,12 @@ static int intel_vgpu_open_device(struct vfio_device *vfio_dev) vgpu->vfio_group = NULL; undo_register: - vfio_unregister_notifier(vfio_dev->dev, VFIO_GROUP_NOTIFY, - &vgpu->group_notifier); + vfio_unregister_notifier(vfio_dev, VFIO_GROUP_NOTIFY, + &vgpu->group_notifier); undo_iommu: - vfio_unregister_notifier(vfio_dev->dev, VFIO_IOMMU_NOTIFY, - &vgpu->iommu_notifier); + vfio_unregister_notifier(vfio_dev, VFIO_IOMMU_NOTIFY, + &vgpu->iommu_notifier); out: return ret; } @@ -907,13 +907,13 @@ static void __intel_vgpu_release(struct intel_vgpu *vgpu) intel_gvt_release_vgpu(vgpu); - ret = vfio_unregister_notifier(vgpu->vfio_device.dev, VFIO_IOMMU_NOTIFY, - &vgpu->iommu_notifier); + ret = vfio_unregister_notifier(&vgpu->vfio_device, VFIO_IOMMU_NOTIFY, + &vgpu->iommu_notifier); drm_WARN(&i915->drm, ret, "vfio_unregister_notifier for iommu failed: %d\n", ret); - ret = vfio_unregister_notifier(vgpu->vfio_device.dev, VFIO_GROUP_NOTIFY, - &vgpu->group_notifier); + ret = vfio_unregister_notifier(&vgpu->vfio_device, VFIO_GROUP_NOTIFY, + &vgpu->group_notifier); drm_WARN(&i915->drm, ret, "vfio_unregister_notifier for group failed: %d\n", ret); diff --git a/drivers/s390/cio/vfio_ccw_ops.c b/drivers/s390/cio/vfio_ccw_ops.c index c4d60cdbf247bf..b49e2e9db2dc6f 100644 --- a/drivers/s390/cio/vfio_ccw_ops.c +++ b/drivers/s390/cio/vfio_ccw_ops.c @@ -183,7 +183,7 @@ static int vfio_ccw_mdev_open_device(struct vfio_device *vdev) private->nb.notifier_call = vfio_ccw_mdev_notifier; - ret = vfio_register_notifier(vdev->dev, VFIO_IOMMU_NOTIFY, + ret = vfio_register_notifier(vdev, VFIO_IOMMU_NOTIFY, &events, &private->nb); if (ret) return ret; @@ -204,8 +204,7 @@ static int vfio_ccw_mdev_open_device(struct vfio_device *vdev) out_unregister: vfio_ccw_unregister_dev_regions(private); - vfio_unregister_notifier(vdev->dev, VFIO_IOMMU_NOTIFY, - &private->nb); + vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &private->nb); return ret; } @@ -223,7 +222,7 @@ static void vfio_ccw_mdev_close_device(struct vfio_device *vdev) cp_free(&private->cp); vfio_ccw_unregister_dev_regions(private); - vfio_unregister_notifier(vdev->dev, VFIO_IOMMU_NOTIFY, &private->nb); + vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &private->nb); } static ssize_t vfio_ccw_mdev_read_io_region(struct vfio_ccw_private *private, diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c index ee0a3bf8f476ca..bfa7ee6ef532d9 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -1406,21 +1406,21 @@ static int vfio_ap_mdev_open_device(struct vfio_device *vdev) matrix_mdev->group_notifier.notifier_call = vfio_ap_mdev_group_notifier; events = VFIO_GROUP_NOTIFY_SET_KVM; - ret = vfio_register_notifier(vdev->dev, VFIO_GROUP_NOTIFY, - &events, &matrix_mdev->group_notifier); + ret = vfio_register_notifier(vdev, VFIO_GROUP_NOTIFY, &events, + &matrix_mdev->group_notifier); if (ret) return ret; matrix_mdev->iommu_notifier.notifier_call = vfio_ap_mdev_iommu_notifier; events = VFIO_IOMMU_NOTIFY_DMA_UNMAP; - ret = vfio_register_notifier(vdev->dev, VFIO_IOMMU_NOTIFY, - &events, &matrix_mdev->iommu_notifier); + ret = vfio_register_notifier(vdev, VFIO_IOMMU_NOTIFY, &events, + &matrix_mdev->iommu_notifier); if (ret) goto out_unregister_group; return 0; out_unregister_group: - vfio_unregister_notifier(vdev->dev, VFIO_GROUP_NOTIFY, + vfio_unregister_notifier(vdev, VFIO_GROUP_NOTIFY, &matrix_mdev->group_notifier); return ret; } @@ -1430,9 +1430,9 @@ static void vfio_ap_mdev_close_device(struct vfio_device *vdev) struct ap_matrix_mdev *matrix_mdev = container_of(vdev, struct ap_matrix_mdev, vdev); - vfio_unregister_notifier(vdev->dev, VFIO_IOMMU_NOTIFY, + vfio_unregister_notifier(vdev, VFIO_IOMMU_NOTIFY, &matrix_mdev->iommu_notifier); - vfio_unregister_notifier(vdev->dev, VFIO_GROUP_NOTIFY, + vfio_unregister_notifier(vdev, VFIO_GROUP_NOTIFY, &matrix_mdev->group_notifier); vfio_ap_mdev_unset_kvm(matrix_mdev); } diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 0c766384cee0f8..13494c1e497405 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -2269,19 +2269,15 @@ static int vfio_unregister_group_notifier(struct vfio_group *group, return ret; } -int vfio_register_notifier(struct device *dev, enum vfio_notify_type type, +int vfio_register_notifier(struct vfio_device *dev, enum vfio_notify_type type, unsigned long *events, struct notifier_block *nb) { - struct vfio_group *group; + struct vfio_group *group = dev->group; int ret; - if (!dev || !nb || !events || (*events == 0)) + if (!nb || !events || (*events == 0)) return -EINVAL; - group = vfio_group_get_from_dev(dev); - if (!group) - return -ENODEV; - switch (type) { case VFIO_IOMMU_NOTIFY: ret = vfio_register_iommu_notifier(group, events, nb); @@ -2292,25 +2288,20 @@ int vfio_register_notifier(struct device *dev, enum vfio_notify_type type, default: ret = -EINVAL; } - - vfio_group_put(group); return ret; } EXPORT_SYMBOL(vfio_register_notifier); -int vfio_unregister_notifier(struct device *dev, enum vfio_notify_type type, +int vfio_unregister_notifier(struct vfio_device *dev, + enum vfio_notify_type type, struct notifier_block *nb) { - struct vfio_group *group; + struct vfio_group *group = dev->group; int ret; - if (!dev || !nb) + if (!nb) return -EINVAL; - group = vfio_group_get_from_dev(dev); - if (!group) - return -ENODEV; - switch (type) { case VFIO_IOMMU_NOTIFY: ret = vfio_unregister_iommu_notifier(group, nb); @@ -2321,8 +2312,6 @@ int vfio_unregister_notifier(struct device *dev, enum vfio_notify_type type, default: ret = -EINVAL; } - - vfio_group_put(group); return ret; } EXPORT_SYMBOL(vfio_unregister_notifier); diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 66dda06ec42d1b..748ec0e0293aea 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -178,11 +178,11 @@ enum vfio_notify_type { /* events for VFIO_GROUP_NOTIFY */ #define VFIO_GROUP_NOTIFY_SET_KVM BIT(0) -extern int vfio_register_notifier(struct device *dev, +extern int vfio_register_notifier(struct vfio_device *dev, enum vfio_notify_type type, unsigned long *required_events, struct notifier_block *nb); -extern int vfio_unregister_notifier(struct device *dev, +extern int vfio_unregister_notifier(struct vfio_device *dev, enum vfio_notify_type type, struct notifier_block *nb); From patchwork Mon May 2 17:31:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12834558 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B7FAC433F5 for ; Mon, 2 May 2022 17:31:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386662AbiEBRfM (ORCPT ); Mon, 2 May 2022 13:35:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40270 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1355294AbiEBRfK (ORCPT ); Mon, 2 May 2022 13:35:10 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2084.outbound.protection.outlook.com [40.107.220.84]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9FF4F2AE0 for ; Mon, 2 May 2022 10:31:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KWBHreAjSf4E95gJtXAjpE0sMbJ6cZACkQgKThAg07R0GCqKxPJdRlCzHSATxRlnE36UB1b5RwAH3OC4bPQZdqUKg1XTpDBa1hBkgHTUDNTytktuGLlJ6qHxWJjX2b1ekf5sIWlz4LnS6nCltTvtdWwx/KGbT9626LzMt7m874037rI1wMiIW0Lt2ckE+LmEDDTM7nP0a/RX7Ru7jFS40WgBDSSpQFlH3c9N1lW0D1SNuUVMdhXdNFlRy+UoF1/g4V5RCNGmwzxqgzrBawTpckuMFkP+VrvsaaV7oTB7exsqJ2QXAzR7xzP6C1A+fnAwAHQ++CMaex8rWWb/HrTMTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8LrDPKMVIsusxztPgCpb23x63uX0ihT59Q0QLuOEqwE=; b=gcBuxBSZIto2QJBk+PZHjeWsRHPVHmQT4IB4eg66XZy132RNSdQQXgExLWCZUW6xQj6eY45viTtkBZgWxWlnOs2kMYqnpDCOb2mfcVjC/9f9aHMhWVjEIvQX9TeTF87BtRvPq1Bymrw4uMJqQYR/83PeHpzsCD0mVxUeNA41kFIY6v7FVef3JuVenEKYcuKjXm6DAB1Htx3BjnRL9M8Mi2PICkSvshlzoWy1IuE9eRS/o/39xFhaQRoHfGQl6QhsypLIwql1jazthLl01qmNYaWtakoFaSM7q4QAxnERZULAa85nGszTQBCnb8bgJPhFwjgs5CkwDIhA9MHKN5rnAw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8LrDPKMVIsusxztPgCpb23x63uX0ihT59Q0QLuOEqwE=; b=Gk4MtPNKxVi8PjeGr3++4qjX0h5dRojwKE/U+yLxjuq7D/D3eYNAHjbCi1iwGtt9ZWRxEHvK0z3cSSePAOIoLEt5R/cZW90d7RBw4XNUjGnNYO1E0ErSdZWwrSrrHrjEuNDKhD6WT/lYcS83FmEqPaj/u67uHHrh2J9OfIUEM16xoFjcFG9+NaoZmT1O5dt/Ce0PvDQtf2H4bpjYgI31Yclk4Rh5kTtm0ZMVNgI/HQueB6QIvt0xlDR9tZxQtuJUTknRKdkbR1laYcO+0wAVcgQxGBP1ek8aXsBqIKLdQaE0foQs/RFFIRNHmFNf9QA4yQ4yoOnPodFDaBBiZAibSg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by BYAPR12MB4709.namprd12.prod.outlook.com (2603:10b6:a03:98::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5186.20; Mon, 2 May 2022 17:31:38 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5206.024; Mon, 2 May 2022 17:31:38 +0000 From: Jason Gunthorpe Cc: Tony Krowiak , Eric Farman , Christoph Hellwig , intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org, "Jason J. Herne" , "Tian, Kevin" , kvm@vger.kernel.org, "Liu, Yi L" Subject: [PATCH v3 2/7] vfio/ccw: Remove mdev from struct channel_program Date: Mon, 2 May 2022 14:31:32 -0300 Message-Id: <2-v3-e131a9b6b467+14b6-vfio_mdev_no_group_jgg@nvidia.com> In-Reply-To: <0-v3-e131a9b6b467+14b6-vfio_mdev_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR02CA0059.namprd02.prod.outlook.com (2603:10b6:207:3d::36) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d02ac7c5-2485-4335-1598-08da2c619cc3 X-MS-TrafficTypeDiagnostic: BYAPR12MB4709:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: lG6lG3RgK5FMbaRCwnPhSNRX4vXEgV+6HEU3e5XZa7GlFmMNYAfiSeFfLd98g5pd8X3ib00FZ+E2QGJOxY2HJB8noMcPZE58Z7hkbEl8vUlq/OVg2o13BMn8kWOiNnG5mGjYvYI0y+35wiWOgMxbUBYXIRKFs7jImqvBDDeWxhh9cvicU5yEuowUPSsF5+D8RKvSM8Qt/EIv2OBjCTeWrIGfTC9M0D3TY8zjsrXWjZPKQk/I0vWyHFr52E46a/BlDAB4jb56GELxPayWtYHm9GhAIbT+DNrX/Py/oF8g5cBgE/GwXoCDulHYPIccwNqzqaAv7k0nnSUlC3bquDpEyiSD90sRNvTAljiSOtXzMNnIeE3CCSUQYbOYzracev7dIuvwkIbyrc3B7ZWcjUL1y8FlmpUFNOm/+dmHKvRJrH3ftbP5PXaSlgmmxNWxb3qGVADpvXjCTlSL6tdFY/zY6C6iyBXSFuAZVtMpkDHe1+vidyFqePpeAbpxW8RAHSZoV3IlQsbBpT63Iq6w5KEVfK2gT2adNiUD/jvLFeqfoPiE8m6Rbgu3OEEzJkTmEt+yWMLJ518+JJIr8YrTp8bAtRLrY/r9QETiOwd7KsIjr4Kr/lGgly32bAuc2G6efNN/ANEJx1qasm2tG1CTP3MMTQuep96b3Vo8jv0cYrOW8y0nzEScrDm/ExctRoeFT1PYA9cnAfh4bQjGyKltbvANQJSqwzpUDTFXX73XqJWA/X4= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(5660300002)(66946007)(66556008)(66476007)(8936002)(8676002)(4326008)(316002)(6666004)(38100700002)(86362001)(6486002)(36756003)(2906002)(6506007)(2616005)(186003)(54906003)(83380400001)(508600001)(6512007)(109986005)(26005)(4216001)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 4iIdHHx5im+RJFa6Id6DxnafPwgJcPvlfY1YPaviu5VFnkD1ssu845XxAmZSx2SyxJv3Nby81r1x2UNMEbcmP/G0xIp5sFswLHtU3JAi3TYkKsH1NCztJ4jOiae12rs7JrJL5v4QXFRY+Tb3V/jLAFm1x6NPKvytGQbFtOwmz/s4YC0vCInmos2GZX7wv2dAywEjy0Qic5vkVkWYfet/HVZgKte9R+vWjcvWrEG7sawQMt/++kda/09gd1M2SWQtGa2ePuWrTr3SEO8McP58WGEmkpaq6npu4kXYCY0PxRlNn3PB2UUHpI9ELXTMUO8DUYIWRzVvl2EFserWZkKC7IwDYeXWovpmS4vOFhsPCQu101ZgC2hNbZj3gnHof9fI0SiO2vsiY8Y6dJw/O6kxro0bXJWYzmMQl9I8geJ8oZiE1RiWy8Dm2xlQiEaqu6zCCnI+out3VDgKXLm1GrtUA1blz0MGQxJS6ceBrCv8cu1DZ7uCSzN2yQVAdFXacQt0r+kqe2Fk6ZH/kuuH4d8fTZ5cfMlk0VPPzM4hReu1uCQpRzArb+KxiPcXt8BUeFeBv7LzjFhSME8zmCIu/+97MbDgUhnnP1wLRLmlioCeM3dKUcbj80vSZZOa4+WqyO+LmPT0Ybcl6BAlR9YR6+5wFioFmAR6lfQIfg9+pN4Ui6Y1zRwPk004zQkwHVvB27gnTSN80A+ETV8hWAYy4hBhxyiBGDOCWWrYpbJJ6ETVlHisLYXT1f4QPsVQQ4ISo/wxoly3WxHQJ5EgzfnidEC/+XVRgkfkAh3Zyo1UpPa8XaBMxbAzrmXMdwTYvP7X8y0hFZbRwQBnhki1CHV4z0Kc3lDitRxZQyDgWY+HgReoL+qjsV7VY78xdckLVjS0d2/42QiWMSHq/m51WpenONhOY79Nwh6UwPt/Y1x0vg73qlaMtzrhVMMYen3JP17S5YVZyeXCB7qrsLWO12agwUd74uI8c4vhWZDxnamnSuCLri2yXdzptC6MyWZMjNjw7CnekiCGkzXLC44DA3dt6wHCU94eEiM/skv+yOpnbVujJpeAYOoQUuBz9812RPi5aAbBLEFq4l8dOGsZFtH8c72RDQbEfGEGo8MEku1bUQOi7irDsDEyBenJM/1bmBICytr4fs7Zo8JJnZW+TsiMCfILLtAaBLVbAz3DaFUaoxqk0E21XWwhNabBWco8CyuaiGrdPdDaNXuk2Oy3Hc5o2XXb/E+wUpeMapzjc7+wCgkp5doDUA39hu8s/VyQRWprzzcmcTDuGTHLgRGJs0ul6khgBBwlHf0ZBQwSnrFvrsVgtIeBqgIUZNAz+yeZ47/WMcE4Xw0qJYh/53lXrmr94vA5TfzCahTwgHJnbYtPR10nVLtAjUKieFqFnNQxTwZuoSnz4sQOLfhu67aGotFTXnx3+zD162LBr9HFNuJIX5v5UbpVqLPyUju3pATwIoK/VMXkC8hHilwJBvLAiLWOa7OyVan/n4FPlfakJQR6PIkZKNriVGL2Ji1fOQYQ91f5Q4V5kk2xZ53G2ZsdZoEg/G3M5ahazIITepe27KTGTNeZazhA43vRnG5KVsVT8oxc+Ics29FFkeYg7b9G5vYQFEL51KaTb6L5rZBKrmOs2+gsdy6pxLpDmW/vCzBMMWt9annfl2Ow2LnEKHbfFjcL7BmIBs5I6fa9nd1Yh0eZJQMEwjZ87khCPvwK2mGwKWjN0O5+6lYAsID9LiZiwI3uoQN4JA== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: d02ac7c5-2485-4335-1598-08da2c619cc3 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2022 17:31:38.1076 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hxZRfP8T+nvrTroFQ8eZiJePiMGaN6GHXxXjqICgvdVs1HXoU8Fi+kKSwEAmA0cq X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB4709 To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The next patch wants the vfio_device instead. There is no reason to store a pointer here since we can container_of back to the vfio_device. Reviewed-by: Eric Farman Signed-off-by: Jason Gunthorpe --- drivers/s390/cio/vfio_ccw_cp.c | 47 ++++++++++++++++++++------------- drivers/s390/cio/vfio_ccw_cp.h | 4 +-- drivers/s390/cio/vfio_ccw_fsm.c | 3 +-- 3 files changed, 30 insertions(+), 24 deletions(-) diff --git a/drivers/s390/cio/vfio_ccw_cp.c b/drivers/s390/cio/vfio_ccw_cp.c index 8d1b2771c1aa02..7a1cf3091cd647 100644 --- a/drivers/s390/cio/vfio_ccw_cp.c +++ b/drivers/s390/cio/vfio_ccw_cp.c @@ -16,6 +16,7 @@ #include #include "vfio_ccw_cp.h" +#include "vfio_ccw_private.h" struct pfn_array { /* Starting guest physical I/O address. */ @@ -98,17 +99,17 @@ static int pfn_array_alloc(struct pfn_array *pa, u64 iova, unsigned int len) * If the pin request partially succeeds, or fails completely, * all pages are left unpinned and a negative error value is returned. */ -static int pfn_array_pin(struct pfn_array *pa, struct device *mdev) +static int pfn_array_pin(struct pfn_array *pa, struct vfio_device *vdev) { int ret = 0; - ret = vfio_pin_pages(mdev, pa->pa_iova_pfn, pa->pa_nr, + ret = vfio_pin_pages(vdev->dev, pa->pa_iova_pfn, pa->pa_nr, IOMMU_READ | IOMMU_WRITE, pa->pa_pfn); if (ret < 0) { goto err_out; } else if (ret > 0 && ret != pa->pa_nr) { - vfio_unpin_pages(mdev, pa->pa_iova_pfn, ret); + vfio_unpin_pages(vdev->dev, pa->pa_iova_pfn, ret); ret = -EINVAL; goto err_out; } @@ -122,11 +123,11 @@ static int pfn_array_pin(struct pfn_array *pa, struct device *mdev) } /* Unpin the pages before releasing the memory. */ -static void pfn_array_unpin_free(struct pfn_array *pa, struct device *mdev) +static void pfn_array_unpin_free(struct pfn_array *pa, struct vfio_device *vdev) { /* Only unpin if any pages were pinned to begin with */ if (pa->pa_nr) - vfio_unpin_pages(mdev, pa->pa_iova_pfn, pa->pa_nr); + vfio_unpin_pages(vdev->dev, pa->pa_iova_pfn, pa->pa_nr); pa->pa_nr = 0; kfree(pa->pa_iova_pfn); } @@ -190,8 +191,7 @@ static void convert_ccw0_to_ccw1(struct ccw1 *source, unsigned long len) * Within the domain (@mdev), copy @n bytes from a guest physical * address (@iova) to a host physical address (@to). */ -static long copy_from_iova(struct device *mdev, - void *to, u64 iova, +static long copy_from_iova(struct vfio_device *vdev, void *to, u64 iova, unsigned long n) { struct pfn_array pa = {0}; @@ -203,9 +203,9 @@ static long copy_from_iova(struct device *mdev, if (ret < 0) return ret; - ret = pfn_array_pin(&pa, mdev); + ret = pfn_array_pin(&pa, vdev); if (ret < 0) { - pfn_array_unpin_free(&pa, mdev); + pfn_array_unpin_free(&pa, vdev); return ret; } @@ -226,7 +226,7 @@ static long copy_from_iova(struct device *mdev, break; } - pfn_array_unpin_free(&pa, mdev); + pfn_array_unpin_free(&pa, vdev); return l; } @@ -423,11 +423,13 @@ static int ccwchain_loop_tic(struct ccwchain *chain, static int ccwchain_handle_ccw(u32 cda, struct channel_program *cp) { + struct vfio_device *vdev = + &container_of(cp, struct vfio_ccw_private, cp)->vdev; struct ccwchain *chain; int len, ret; /* Copy 2K (the most we support today) of possible CCWs */ - len = copy_from_iova(cp->mdev, cp->guest_cp, cda, + len = copy_from_iova(vdev, cp->guest_cp, cda, CCWCHAIN_LEN_MAX * sizeof(struct ccw1)); if (len) return len; @@ -508,6 +510,8 @@ static int ccwchain_fetch_direct(struct ccwchain *chain, int idx, struct channel_program *cp) { + struct vfio_device *vdev = + &container_of(cp, struct vfio_ccw_private, cp)->vdev; struct ccw1 *ccw; struct pfn_array *pa; u64 iova; @@ -526,7 +530,7 @@ static int ccwchain_fetch_direct(struct ccwchain *chain, if (ccw_is_idal(ccw)) { /* Read first IDAW to see if it's 4K-aligned or not. */ /* All subsequent IDAws will be 4K-aligned. */ - ret = copy_from_iova(cp->mdev, &iova, ccw->cda, sizeof(iova)); + ret = copy_from_iova(vdev, &iova, ccw->cda, sizeof(iova)); if (ret) return ret; } else { @@ -555,7 +559,7 @@ static int ccwchain_fetch_direct(struct ccwchain *chain, if (ccw_is_idal(ccw)) { /* Copy guest IDAL into host IDAL */ - ret = copy_from_iova(cp->mdev, idaws, ccw->cda, idal_len); + ret = copy_from_iova(vdev, idaws, ccw->cda, idal_len); if (ret) goto out_unpin; @@ -574,7 +578,7 @@ static int ccwchain_fetch_direct(struct ccwchain *chain, } if (ccw_does_data_transfer(ccw)) { - ret = pfn_array_pin(pa, cp->mdev); + ret = pfn_array_pin(pa, vdev); if (ret < 0) goto out_unpin; } else { @@ -590,7 +594,7 @@ static int ccwchain_fetch_direct(struct ccwchain *chain, return 0; out_unpin: - pfn_array_unpin_free(pa, cp->mdev); + pfn_array_unpin_free(pa, vdev); out_free_idaws: kfree(idaws); out_init: @@ -632,8 +636,10 @@ static int ccwchain_fetch_one(struct ccwchain *chain, * Returns: * %0 on success and a negative error value on failure. */ -int cp_init(struct channel_program *cp, struct device *mdev, union orb *orb) +int cp_init(struct channel_program *cp, union orb *orb) { + struct vfio_device *vdev = + &container_of(cp, struct vfio_ccw_private, cp)->vdev; /* custom ratelimit used to avoid flood during guest IPL */ static DEFINE_RATELIMIT_STATE(ratelimit_state, 5 * HZ, 1); int ret; @@ -650,11 +656,12 @@ int cp_init(struct channel_program *cp, struct device *mdev, union orb *orb) * the problem if something does break. */ if (!orb->cmd.pfch && __ratelimit(&ratelimit_state)) - dev_warn(mdev, "Prefetching channel program even though prefetch not specified in ORB"); + dev_warn( + vdev->dev, + "Prefetching channel program even though prefetch not specified in ORB"); INIT_LIST_HEAD(&cp->ccwchain_list); memcpy(&cp->orb, orb, sizeof(*orb)); - cp->mdev = mdev; /* Build a ccwchain for the first CCW segment */ ret = ccwchain_handle_ccw(orb->cmd.cpa, cp); @@ -682,6 +689,8 @@ int cp_init(struct channel_program *cp, struct device *mdev, union orb *orb) */ void cp_free(struct channel_program *cp) { + struct vfio_device *vdev = + &container_of(cp, struct vfio_ccw_private, cp)->vdev; struct ccwchain *chain, *temp; int i; @@ -691,7 +700,7 @@ void cp_free(struct channel_program *cp) cp->initialized = false; list_for_each_entry_safe(chain, temp, &cp->ccwchain_list, next) { for (i = 0; i < chain->ch_len; i++) { - pfn_array_unpin_free(chain->ch_pa + i, cp->mdev); + pfn_array_unpin_free(chain->ch_pa + i, vdev); ccwchain_cda_free(chain, i); } ccwchain_free(chain); diff --git a/drivers/s390/cio/vfio_ccw_cp.h b/drivers/s390/cio/vfio_ccw_cp.h index ba31240ce96594..e4c436199b4cda 100644 --- a/drivers/s390/cio/vfio_ccw_cp.h +++ b/drivers/s390/cio/vfio_ccw_cp.h @@ -37,13 +37,11 @@ struct channel_program { struct list_head ccwchain_list; union orb orb; - struct device *mdev; bool initialized; struct ccw1 *guest_cp; }; -extern int cp_init(struct channel_program *cp, struct device *mdev, - union orb *orb); +extern int cp_init(struct channel_program *cp, union orb *orb); extern void cp_free(struct channel_program *cp); extern int cp_prefetch(struct channel_program *cp); extern union orb *cp_get_orb(struct channel_program *cp, u32 intparm, u8 lpm); diff --git a/drivers/s390/cio/vfio_ccw_fsm.c b/drivers/s390/cio/vfio_ccw_fsm.c index e435a9cd92dacf..8483a266051c21 100644 --- a/drivers/s390/cio/vfio_ccw_fsm.c +++ b/drivers/s390/cio/vfio_ccw_fsm.c @@ -262,8 +262,7 @@ static void fsm_io_request(struct vfio_ccw_private *private, errstr = "transport mode"; goto err_out; } - io_region->ret_code = cp_init(&private->cp, mdev_dev(mdev), - orb); + io_region->ret_code = cp_init(&private->cp, orb); if (io_region->ret_code) { VFIO_CCW_MSG_EVENT(2, "%pUl (%x.%x.%04x): cp_init=%d\n", From patchwork Mon May 2 17:31:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12834562 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46BFBC4332F for ; Mon, 2 May 2022 17:31:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386678AbiEBRfT (ORCPT ); Mon, 2 May 2022 13:35:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40322 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386658AbiEBRfN (ORCPT ); Mon, 2 May 2022 13:35:13 -0400 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2063.outbound.protection.outlook.com [40.107.93.63]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C145A6268 for ; Mon, 2 May 2022 10:31:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OQ2Mk/hZ025bNu2EdpIyREOj/3b4eGmmOEQVGVCJGvganOEsxpOjHYgDXj+ZcdojhD2qBWRE1rIrZ2cle1GRqbJvA07WwvnlwGTjXqvpHYqZ++STCpnUBXA+7mbc/fOgI+tDq2f8soVC6JqpLy8SqxT/p1+UeYn6P9mPl7swVFLUEYC4e2ryrIusZRs/8L9x3gCWH7bTVoQGMEEWyinE/kFOGidqXeYfyVjdf9pJYJKOnxiw0uofM/9a9k9hM3/erXypni0/iXW+LTnXmBYfQ6gXMJKfV2fO5Tdf7XszvoSnlO4okD8J0hUfj8dD3KxHgW5vZjw/2JLKEqpe8xr/4w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CKgmevxs4ymKgofDqjFWofAStJLtOC/grEjyH3LWHdk=; b=LwlCEVyHaMhRfzuqNiVhhYNQnU41T+cUbsFHOpyWrIs8LKCqV1QsuvwN1djt+NS1i1WX1yG1471hhsYufurPNpDxq8nTiOIoaApSkhjRyD8p15LXpsD7wL4+XR6TSlDGdnSv9l6E3+/+1z2A13XFuyrwWhOsLldR+dbjJK+B4cpdoYhzd3WqWlrlzqZXDVeO1zHut82aEc0kJxyKoDd8tfP0hqv48kyt8tIOSPmpOe0FoSOXfG+y6LNYtqwK9ySUNpnHw5Ubn8iWucKmOcxhFebMZTGq7Ina3e1GbFyb9h8eVSTiagPeKDVX+jT8ac9f2TBc1feWTuJdTA3BH4e5ww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CKgmevxs4ymKgofDqjFWofAStJLtOC/grEjyH3LWHdk=; b=a2RPKlASPhrQsXFt5E+luf0zVeU1VAM143m0A1Z9EOKhajY0d8hH/A5pOGcelr1k/FnJtCTWycwK6hhP9Busmt5Ez3gVe+zqlmKGwlBqzyRWjFmV0+Ak8rSsV0IIS4igWDnx3NmnEV0G8ZDFkVc3s9SZ3epi79v0qBwGNukb+z6ECfMWgPhQFDlU4MuiKvlAV8vVp0OWL6xlQeh4Dicq5bnONkFP6Tycrl2qPL2tF6/CRXE8ja/sUICLgBpx7oOBBZ8rZFMyD5go0+1ElkR9vwNnSv5Q5F1E8Pn3xOoqdq2AyjbVR/lZsoBxGlHn9DLsbiez9R5TNhKKva4gQ84/0g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by CO6PR12MB5458.namprd12.prod.outlook.com (2603:10b6:5:35b::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.13; Mon, 2 May 2022 17:31:41 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5206.024; Mon, 2 May 2022 17:31:41 +0000 From: Jason Gunthorpe Cc: Tony Krowiak , Eric Farman , Christoph Hellwig , intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org, "Jason J. Herne" , "Tian, Kevin" , kvm@vger.kernel.org, "Liu, Yi L" Subject: [PATCH v3 3/7] vfio/mdev: Pass in a struct vfio_device * to vfio_pin/unpin_pages() Date: Mon, 2 May 2022 14:31:33 -0300 Message-Id: <3-v3-e131a9b6b467+14b6-vfio_mdev_no_group_jgg@nvidia.com> In-Reply-To: <0-v3-e131a9b6b467+14b6-vfio_mdev_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL1PR13CA0085.namprd13.prod.outlook.com (2603:10b6:208:2b8::30) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 802b1530-f908-4750-fb00-08da2c619d0c X-MS-TrafficTypeDiagnostic: CO6PR12MB5458:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: bMTylhpwUhiR/Gdn+lfYdnXgPhvmX1LvXilvrrBwULVsNDzQqSopuTkvilkpTXAueiNmvvyhxlLqPwZmjv3oPBxNJwdGi7EuiG3Cd9Km8qej647w8Kvw0I9eMXdAEEvqimbgn6DlU4y9dSG/sZc/r8TXd6vKkAi2PNz25E+DU6uZC1m2GiJFJCtx1lu0DkHsxKMgy63VJxUevVBysk/RfzDvWYOy7XhxuEf3k1tCoj2CZkbowIhkNrQMbCj/Do6gl5qr2kLS+QhamuRKoo03zkYRQzlCpvW2ia6T8OMIrMg34twtTIRMKAo2toB96+Isg2WA08IBa3HNdhDgt52ifPf/bDKo/Rv8GmZ06E8pi7tKbUfxJ7S3T/Vbl7lRfdnSFDkwmRdSVYg/CWmMt8l0D3XkbUc1HCr+7RImkuTa8eVLykSH9/2lcTCFjN1/w9tQ3NFGorYjom71tVEoGngEbShuJu5MjgtoUkmcEEXGFRE6yqY1A09wtSL7siMPvC4/yAXuCAWlwrLHtVoSqPynuVEOdJd0KMCblQimn2mrYkkaVtoc4HIXIKK72f2L7KIgRvZr+KGc7kRi7Roqytbes3gyshYRRZOs6bRPDG4X6m1l82p/soYhnoopEO4tvIrB7powvlMc+RzD14BO8ZBG4Z1xYlA8ybbyXGVw9fWVCdYuoZN4WMa6PR8lG1tAAlE9BWu3LBlQ+o53Z6y46QJHuGR058MvLUxb5Ylu7G5nzJI= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(109986005)(6486002)(508600001)(2616005)(86362001)(6512007)(26005)(6506007)(38100700002)(6666004)(316002)(66476007)(2906002)(66946007)(66556008)(186003)(36756003)(5660300002)(8936002)(54906003)(4326008)(83380400001)(8676002)(4216001)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 7ZBIwdeSQhkw1D1WG2p8XRmbIwy3Ra/l9ZUYvZXdOUs8gibSaT1gYloP8hWeRBNTvkp80z1yqtEh9E5wYkUIsW0G8W/zoLjBnw+4VFx7m6rbPY7W8sqQxsYDRZjTXQEBythsZu6N1jDE3m5vueLJHfzembF5wsVbHCo+9iwqlmP7x9r5ucMIiese1KwEMGfimemvKaS8ATklpA8zMcbQ2pJPuxTVCRnYTgYxy+NvufTI0Qe/HhmBY6YHBdt7Q3hphFzu6fCgIdGmEk/9TnEG03QTcqrQskHrjy90nRNulmf+6XX1M57mBK/6MPboGzsU7++cUUqJbWkPrSfrLu59fTiqJL3yF498zucPipFM0NkOKct5zv0aTUtl/klRMmPVNYy+y4ovINSRGVnfu/pdhSSclsFRqikeHnDksh7PpiE7ffMfIZvG+XCuo+iBCimg0FCRvgxgM0yyiPtF6Mcm9ms0cJIC7pa3FgFY4xfD3X/IV/pFOgvkbo1x2c6DgwvPuuQqxcp/MiS4/7Z2FXRYLNIz6Y98PoWXSI6Ih0lFFadJ/Y+4EdwpKp+I6HCGO1HOmmAD7tzTAs4E3hKci8rNivTMbq3KveX9+3hTAaJI+t9P12ZPktrHKsZBWAOD2y7nDv7j1ddyutQyzSlQaB+jqp6F3rKc+UFi0CVqzT/xHHu81NPuOUGNvNfUs8KxcI4lnPhCwoUNq4dz47DVOHNQ5IBBed6xWxYZpjsyBU8CoNHxWOUhefzhbARAO8RqA+/BYVmaE6WdlTBmVbMcozRKo6pmJ2hJb/f4i5ubIB44qd68Xp5NG+81UbN7MqdfSVp/w5NZNYCo3GKSSLF71QNci9Va0IEFXp1xYbuJysmYaBTroTeT9NkCiR1G2ruqgnMC5z6w5Fxgo4szQYglIeE7pJaIWuYG8ylEa0SOuCW66p6R4nzACpzj5gRuMxTfeBoQrNQS/FnqxfJtUin+aWGBit3NcPKr6RzVwcw1FIkEBCLuQ+IZ5imA93a0wMBCRhGJhz5Dzxy/7ekJydZJX/ihTaCB92IXan4qtdNvCCCyvUg/Lr89Lw9exnCvNIzPwV4g4hA5LkEbTJN6qGEQ+KRUKFMKp6bt0Pw0+FHrAZtxXMKzT16fsNzzIjC8v/yCAs35sTun0tJ81uY9vgnUy77h3mseryN9RYi9anLISAULED0zKJEWh62MbK841DUUQzpSuPwb7SW46wDFA3o8TJmNgMhNdFkLlWUrINcW+3axB/vLwD8JayhYZiMrddFc6B2pyLZf0HOJuwWdLoxj1KG8lp2R3Wy5IAydHJBAL0tMD8cB/apYEVaBGXQmOSpOvy4h1dIG9uLFXQPlD2ioynJDiPMD3361hVIdTrIYELXj5PmN1mcWb1wUlHkaAo7DiV40YdPj2TgXdLrI4goUdXpVYIreBL8m14NCMzixn+/XnFmLgS3v4ksE5w1O+VkWVKZiIBBVo9U4PnVt/+pCVyC7iJp3TALCGPdPrBsOzd3ifGSf1zlG1A37laceM/dbgQ+t6GT9hFhzjQkqAEJrdM5/59d/Ps6sPybaisoom9SruNBdkkM60NXmH210O6ZnTMcxNAz5GQNIouC8AXhoMH9BGJ8Xld3HN3vLsVUqY41eXsuNgaU/vyt1QpRtSS0WoMcBVNVFkfVFjo1XiKNfaHQebdqcvNeaqPvqFfMxhv9hVJYtdsM5NreIfuoqcGu31gheQ9VJS1pMtXvtOr+on/tb1g== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 802b1530-f908-4750-fb00-08da2c619d0c X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2022 17:31:38.6087 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: AhXYzSjIvfi5wx5H7ymmiPJZD5jwLhEDGdWZnyxMiALkipjOoX63SaykvqQyA9UM X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR12MB5458 To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Every caller has a readily available vfio_device pointer, use that instead of passing in a generic struct device. The struct vfio_device already contains the group we need so this avoids complexity, extra refcountings, and a confusing lifecycle model. Reviewed-by: Christoph Hellwig Acked-by: Eric Farman Reviewed-by: Jason J. Herne Reviewed-by: Tony Krowiak Reviewed-by: Kevin Tian Signed-off-by: Jason Gunthorpe --- .../driver-api/vfio-mediated-device.rst | 4 +- drivers/s390/cio/vfio_ccw_cp.c | 6 +-- drivers/s390/crypto/vfio_ap_ops.c | 9 ++--- drivers/vfio/vfio.c | 40 ++++++------------- include/linux/vfio.h | 4 +- 5 files changed, 24 insertions(+), 39 deletions(-) diff --git a/Documentation/driver-api/vfio-mediated-device.rst b/Documentation/driver-api/vfio-mediated-device.rst index 784bbeb22adcf5..2f9e6025eb2fe1 100644 --- a/Documentation/driver-api/vfio-mediated-device.rst +++ b/Documentation/driver-api/vfio-mediated-device.rst @@ -262,10 +262,10 @@ Translation APIs for Mediated Devices The following APIs are provided for translating user pfn to host pfn in a VFIO driver:: - extern int vfio_pin_pages(struct device *dev, unsigned long *user_pfn, + int vfio_pin_pages(struct vfio_device *vdev, unsigned long *user_pfn, int npage, int prot, unsigned long *phys_pfn); - extern int vfio_unpin_pages(struct device *dev, unsigned long *user_pfn, + int vfio_unpin_pages(struct vfio_device *vdev, unsigned long *user_pfn, int npage); These functions call back into the back-end IOMMU module by using the pin_pages diff --git a/drivers/s390/cio/vfio_ccw_cp.c b/drivers/s390/cio/vfio_ccw_cp.c index 7a1cf3091cd647..0c2be9421ab78f 100644 --- a/drivers/s390/cio/vfio_ccw_cp.c +++ b/drivers/s390/cio/vfio_ccw_cp.c @@ -103,13 +103,13 @@ static int pfn_array_pin(struct pfn_array *pa, struct vfio_device *vdev) { int ret = 0; - ret = vfio_pin_pages(vdev->dev, pa->pa_iova_pfn, pa->pa_nr, + ret = vfio_pin_pages(vdev, pa->pa_iova_pfn, pa->pa_nr, IOMMU_READ | IOMMU_WRITE, pa->pa_pfn); if (ret < 0) { goto err_out; } else if (ret > 0 && ret != pa->pa_nr) { - vfio_unpin_pages(vdev->dev, pa->pa_iova_pfn, ret); + vfio_unpin_pages(vdev, pa->pa_iova_pfn, ret); ret = -EINVAL; goto err_out; } @@ -127,7 +127,7 @@ static void pfn_array_unpin_free(struct pfn_array *pa, struct vfio_device *vdev) { /* Only unpin if any pages were pinned to begin with */ if (pa->pa_nr) - vfio_unpin_pages(vdev->dev, pa->pa_iova_pfn, pa->pa_nr); + vfio_unpin_pages(vdev, pa->pa_iova_pfn, pa->pa_nr); pa->pa_nr = 0; kfree(pa->pa_iova_pfn); } diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_ap_ops.c index bfa7ee6ef532d9..e8914024f5b1af 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -124,8 +124,7 @@ static void vfio_ap_free_aqic_resources(struct vfio_ap_queue *q) q->saved_isc = VFIO_AP_ISC_INVALID; } if (q->saved_pfn && !WARN_ON(!q->matrix_mdev)) { - vfio_unpin_pages(mdev_dev(q->matrix_mdev->mdev), - &q->saved_pfn, 1); + vfio_unpin_pages(&q->matrix_mdev->vdev, &q->saved_pfn, 1); q->saved_pfn = 0; } } @@ -258,7 +257,7 @@ static struct ap_queue_status vfio_ap_irq_enable(struct vfio_ap_queue *q, return status; } - ret = vfio_pin_pages(mdev_dev(q->matrix_mdev->mdev), &g_pfn, 1, + ret = vfio_pin_pages(&q->matrix_mdev->vdev, &g_pfn, 1, IOMMU_READ | IOMMU_WRITE, &h_pfn); switch (ret) { case 1: @@ -301,7 +300,7 @@ static struct ap_queue_status vfio_ap_irq_enable(struct vfio_ap_queue *q, break; case AP_RESPONSE_OTHERWISE_CHANGED: /* We could not modify IRQ setings: clear new configuration */ - vfio_unpin_pages(mdev_dev(q->matrix_mdev->mdev), &g_pfn, 1); + vfio_unpin_pages(&q->matrix_mdev->vdev, &g_pfn, 1); kvm_s390_gisc_unregister(kvm, isc); break; default: @@ -1250,7 +1249,7 @@ static int vfio_ap_mdev_iommu_notifier(struct notifier_block *nb, struct vfio_iommu_type1_dma_unmap *unmap = data; unsigned long g_pfn = unmap->iova >> PAGE_SHIFT; - vfio_unpin_pages(mdev_dev(matrix_mdev->mdev), &g_pfn, 1); + vfio_unpin_pages(&matrix_mdev->vdev, &g_pfn, 1); return NOTIFY_OK; } diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 13494c1e497405..dc5d40b07e2381 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -1927,32 +1927,26 @@ EXPORT_SYMBOL(vfio_set_irqs_validate_and_prepare); * @phys_pfn[out]: array of host PFNs * Return error or number of pages pinned. */ -int vfio_pin_pages(struct device *dev, unsigned long *user_pfn, int npage, +int vfio_pin_pages(struct vfio_device *vdev, unsigned long *user_pfn, int npage, int prot, unsigned long *phys_pfn) { struct vfio_container *container; - struct vfio_group *group; + struct vfio_group *group = vdev->group; struct vfio_iommu_driver *driver; int ret; - if (!dev || !user_pfn || !phys_pfn || !npage) + if (!user_pfn || !phys_pfn || !npage) return -EINVAL; if (npage > VFIO_PIN_PAGES_MAX_ENTRIES) return -E2BIG; - group = vfio_group_get_from_dev(dev); - if (!group) - return -ENODEV; - - if (group->dev_counter > 1) { - ret = -EINVAL; - goto err_pin_pages; - } + if (group->dev_counter > 1) + return -EINVAL; ret = vfio_group_add_container_user(group); if (ret) - goto err_pin_pages; + return ret; container = group->container; driver = container->iommu_driver; @@ -1965,8 +1959,6 @@ int vfio_pin_pages(struct device *dev, unsigned long *user_pfn, int npage, vfio_group_try_dissolve_container(group); -err_pin_pages: - vfio_group_put(group); return ret; } EXPORT_SYMBOL(vfio_pin_pages); @@ -1980,28 +1972,24 @@ EXPORT_SYMBOL(vfio_pin_pages); * be greater than VFIO_PIN_PAGES_MAX_ENTRIES. * Return error or number of pages unpinned. */ -int vfio_unpin_pages(struct device *dev, unsigned long *user_pfn, int npage) +int vfio_unpin_pages(struct vfio_device *vdev, unsigned long *user_pfn, + int npage) { struct vfio_container *container; - struct vfio_group *group; struct vfio_iommu_driver *driver; int ret; - if (!dev || !user_pfn || !npage) + if (!user_pfn || !npage) return -EINVAL; if (npage > VFIO_PIN_PAGES_MAX_ENTRIES) return -E2BIG; - group = vfio_group_get_from_dev(dev); - if (!group) - return -ENODEV; - - ret = vfio_group_add_container_user(group); + ret = vfio_group_add_container_user(vdev->group); if (ret) - goto err_unpin_pages; + return ret; - container = group->container; + container = vdev->group->container; driver = container->iommu_driver; if (likely(driver && driver->ops->unpin_pages)) ret = driver->ops->unpin_pages(container->iommu_data, user_pfn, @@ -2009,10 +1997,8 @@ int vfio_unpin_pages(struct device *dev, unsigned long *user_pfn, int npage) else ret = -ENOTTY; - vfio_group_try_dissolve_container(group); + vfio_group_try_dissolve_container(vdev->group); -err_unpin_pages: - vfio_group_put(group); return ret; } EXPORT_SYMBOL(vfio_unpin_pages); diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 748ec0e0293aea..8f2a09801a660b 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -150,9 +150,9 @@ extern long vfio_external_check_extension(struct vfio_group *group, #define VFIO_PIN_PAGES_MAX_ENTRIES (PAGE_SIZE/sizeof(unsigned long)) -extern int vfio_pin_pages(struct device *dev, unsigned long *user_pfn, +extern int vfio_pin_pages(struct vfio_device *vdev, unsigned long *user_pfn, int npage, int prot, unsigned long *phys_pfn); -extern int vfio_unpin_pages(struct device *dev, unsigned long *user_pfn, +extern int vfio_unpin_pages(struct vfio_device *vdev, unsigned long *user_pfn, int npage); extern int vfio_group_pin_pages(struct vfio_group *group, From patchwork Mon May 2 17:31:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12834563 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B5A5C433FE for ; Mon, 2 May 2022 17:31:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386684AbiEBRfV (ORCPT ); Mon, 2 May 2022 13:35:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40342 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348252AbiEBRfO (ORCPT ); Mon, 2 May 2022 13:35:14 -0400 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2063.outbound.protection.outlook.com [40.107.93.63]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B7D125DE for ; Mon, 2 May 2022 10:31:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cSCVC4XpYhophoCKZCu8hAkP0JjVWRhFy4BgmeCMpZDzzkKqhbVX3Xzo0/o5Rgl+YP2m2QKyRCAPmD/VSeABDZsgiX5exMU1ZRiITQc2REeHhjXDTgmrApXHvgKGfyxgMopLwdKAFc8Dc+1QFsiOFHYadQIES8mOaDPKqvGzpyR1ILrNEJ2BlW7ByrnKlHcGpA5hOwWhJWLETfwb/BMeA8ldGw90sL9qcaSv6YELx0ye89Jnd+HD/dUIMRXBCWoxSwiXDBHG9z3O4519+e78jby9vTmNjlaq2jTtzG3ODMG58HoTc/NJQAIc14DoyymKN9o0OZ8VEhtGvoA+/J5wEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VBOcVwxEzw6azq3PAlPnF8zldZZAK5HVtoqM4LTZsrw=; b=Y7rBOKWHxTcQnVviWhw3lttJy+Q+7PlrP9TDW0m20bGw+EZsrB2I4PCFPB8rqOOHR8O9xusP3s7qDmWhEiH3l7kiauNwrlpPc+MuNhtimHKnAlVSVrcll019KWSPHRgI+pLZOn5i97U1ree9JfbYo7RfQeVKw2OVIOAyrtRZS6dj0Igh0Mx8cnDmz+IEyXoK46OG3uloCq27Qb+pg+QUpx4E5GccD3sJbNzBN01GXslKaFIum+W50B7bbzpvFZY499qsPdlM3l+mF92FIlOLpuMeHuc83Z9/+cKzw9+k3cW2hzplwYnkYxbprbE4KnmRKsDg2c+v/o9zsx41n7D0zA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VBOcVwxEzw6azq3PAlPnF8zldZZAK5HVtoqM4LTZsrw=; b=EAtjCoDVCETIP564cPeuohSrcX+Fl5YXqOlqvAgPGs3pDmFJdMj1uefpFPX8Lr9oF5eS7J+8DbJHTWKrpv4o7PVTFm8ALRzK7qf4knAztiuG+EDa33SUZJu782ltFOvRQCT1vwJznyI/wganzhoegcCrstI3ncKMKIMqQlBi1QZMuM9RVwnOt9f9/pDANAfn/vCXvl5n/dy5xjMpuKWnwcTf+PxPmsAorXq8HY+DDHDDM50wylQtZclUkKgQXfwIG9pwmDkd3Xw4zmVfEiicK1rNm5ikzQxOLEJKJ9WxQVG5vhDQMxXdqcw9/yXhWDwK8xqvx9f3Ldt7kPF8m8M/0A== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by CO6PR12MB5458.namprd12.prod.outlook.com (2603:10b6:5:35b::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.13; Mon, 2 May 2022 17:31:42 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5206.024; Mon, 2 May 2022 17:31:42 +0000 From: Jason Gunthorpe Cc: Tony Krowiak , Eric Farman , Christoph Hellwig , intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org, "Jason J. Herne" , "Tian, Kevin" , kvm@vger.kernel.org, "Liu, Yi L" Subject: [PATCH v3 4/7] vfio/mdev: Pass in a struct vfio_device * to vfio_dma_rw() Date: Mon, 2 May 2022 14:31:34 -0300 Message-Id: <4-v3-e131a9b6b467+14b6-vfio_mdev_no_group_jgg@nvidia.com> In-Reply-To: <0-v3-e131a9b6b467+14b6-vfio_mdev_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL0PR02CA0042.namprd02.prod.outlook.com (2603:10b6:207:3d::19) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e8dad3b3-42f2-4dd7-9866-08da2c619d60 X-MS-TrafficTypeDiagnostic: CO6PR12MB5458:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ECFykogh1A9Ulpe7tkyQ/oXtbgEBSCXgL9uTZyqJX9qEseM7NF/4e2iYty8hQKt8KUIJ7wdq5egKnaEWAEDiXwf2hvApVaz261HkcNxJj9jYzCfJnzZNlYe/Qzs/K0hZHMPYxEMQJR9IhRn97j4mHpZvBJ5k+3ViKkrGHrZbOu4zzt2hOjbvsm3xIYwZtGDoV3/oczelTmj3qFNE77qD7rujNbAjOcOePm1hG3s8f1Qrcx7fbQRwRgKXtS+OEBpvXLV3ZybBZspNP4UXzYlMbi4okJ4tjPwpmYm6X6zD8xxhcKIQc0zWFOFAM+7sVq++zi4/UaDebcncelzr4Q3Oj1DXugsE3EDcUCNef8OisL4OowXTDwPReIO6ZtNCH+xrwcEQcI/4a8JDuap/ON50ySn2T3atdxs8oefFpdGHob9Rh6oeCWGdy7RAP2OIBMdSo/5aKdKMNUBZQ+Lw//EdSJYxZShbgFFAHg7m4l8KpdwtqzsT1DpHEiE+F6cUpaN6MmrX/0PCD0IdiO4wirCzmY2mEh71YSC6JCDjFlA2d7AEdLqTi4X70lb1Z6uqLfyGSwc01zskOre3rzRopAdL3nnlU4lPowef7UTe1mImYTfw2Y8fk9WLpWQlENLrZkpJclYNlS6P6c9QvkEwCJOdLBIHibcYjrV3d6BP2VqnjO+yktdtEiZgUtmD6a9Ny6z/tjxqrSgECVRjcGgRCbeXo28W8HgJnFF+4cBw8EDIGng= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(109986005)(6486002)(508600001)(2616005)(86362001)(6512007)(26005)(6506007)(38100700002)(6666004)(316002)(66476007)(2906002)(66946007)(66556008)(186003)(36756003)(5660300002)(8936002)(54906003)(4326008)(83380400001)(8676002)(4216001)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: kc9Ey3OgfBvQUhvzHx0rQL4A2fiRyYV2Pjt5bhssVqSv5Z93BMTxFg4jmHFGx14XD6HIuqjTeLwzHr9XXIEfKpVuxBDUXqaQhBY1vHbalhXbM6eZcY3jl9a4m9tzfcztfX4dI6T+Lqla0YevGorpfqUfR4SNlcmiVldsE3zG/lpPl3kMDW5Ruxau+f4jyY53iFztps5vWCkWuOfE1FoN0pXm3bPLTkNHc3O9sILJ34Jt03oger+0H7ynPLXetfSxVGNJ5e6z14pbhWojhGzaqUBeGxkgxNful2jFl2aycq+Jy/MnxVSfRvSykbgBi32iZnxm4FB6oASQ+27GV/NYhkDbKEi2Z0NbqO62EWcb670Krtfbx6UDDhK5ph5V51lmoaw0wOK26sJEQooSIAgQ/MdlTKRc71tLTdh51B754GqS6FfQQifyDzww6hmAeplWnl6TaeOOhooBebmTRqbzoWMFQhxFsEOgIwDZEDo0rNwl53XD+JAFCEp/qehDXeeOWSw92RdqxIGW3Bw8TGi6gOaJJeCTdfPNQXEF0LbOxCW6BinYZwIgUoXoYFlkZvQzrcw2uSy8wdcaoainCK7c/itamvFduLhFE6M/mZNfPF03WCL2jkihzBF7JkxivKdhhG7eyX8JAEIiuRNb+NMQm2bRYlLecyfEO+NOQQoopyQJjjnozbrJhwDAQXEDOh4Iv34o7iuUpAutmT6gGXGHsoBRNEsHxbRjaPOCya42V3aOTNOV8K1NtDOyoREaXAzFJPsF0CkA3RTgmJ4g2sIP60csOc7NgCT43kQMog61lt2IYwsBMU+rRKWPTdkssLUT/e9LcMmE2i6Sy+SDAE+dp/mRoC+NVJk4pLLEPCD7u+PP/V/8Hh+sqGfu7sIyeQ9clP/ixCbyTK+xKqtUkML1GY40jHIcow+e4HCWjmiL5BzooyssC4EsIoDNmPWRw2sg8y1VmL3a/luk36/8s5m1hbaxRPwF6/0g9jTz3gpRTxaRj9SnFAwvzazBjGjn54nO7zVz9VZ+UFaqYJqB6xqJAoYM34pVekazGzE9WZA+Xoo0n2SZhJfnbc5UGdewZPslUYPWbSpJ4S5D9V15zBlp6PbAPptpMXj4WLKrXe9bJ6oGPAB+gRYd9CfvmBFPoUgQ6GPzT+yXP/u7kG3V2MqNCZ4rlc3asxewECJZdZNGnqQBL8ysnyW2u74kfPew7CIz4UBtjA0dW4MsOg2J98o3XYrM94nS0fs+3/5QMbTRM98LvCxrcJ27/WcKlylucJjNP1SqJsUewkQy/qvehZCp10d+yuWxldefiktNKfuo2J5S6MzhPTItamKzXaJh3yGaJSz0pC09YIpP8LXsK0veeMvJb+7dIILkPRz5gH/prl5PFy+ZHnh4xNYJC9S6l3ssCRnJRnTYz8maKf8DyA+5yOWMT87ppZDQOg7GkyLQIEHTKDOQvCYzoDYK0T+QNMpatbPqPtbbOp/xAG39kCIoUrrUkzdCOfdvi8kDVppvGFJfnlUTxFygDaI3jmTkvhTzFNS5T2mbuL+BO+d7OONP/YyAIfVSBavQeh+wt11+e+v1v1JuAXNeDrdNcNS4inh5+WB+zNL/+hojLciucH3K3iqwfSt2f2VhVVI8a3bkFJZZRQpPuPLJMDczC5bLNdkOfxQ80U2gnvi2lml7JcaUzL5z5QXCOIQEp4fQuoA445FHqApJxL+tOgHNLMlECi2m52xBMS6lCb+60E2w/AJ5GA== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e8dad3b3-42f2-4dd7-9866-08da2c619d60 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2022 17:31:39.1389 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: FTux7DOrjIn9N3fe9g9+VkCBLRvF2mv79xk7LkVmUbp5JDloVAe3+Er/j+tsKmE9 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR12MB5458 To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Every caller has a readily available vfio_device pointer, use that instead of passing in a generic struct device. Change vfio_dma_rw() to take in the struct vfio_device and move the container users that would have been held by vfio_group_get_external_user_from_dev() to vfio_dma_rw() directly, like vfio_pin/unpin_pages(). Reviewed-by: Christoph Hellwig Reviewed-by: Kevin Tian Signed-off-by: Jason Gunthorpe --- drivers/gpu/drm/i915/gvt/gvt.h | 4 ++-- drivers/vfio/vfio.c | 24 +++++++++++------------- include/linux/vfio.h | 2 +- 3 files changed, 14 insertions(+), 16 deletions(-) diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h index 03ecffc2ba56a9..5a28ee965b7f3e 100644 --- a/drivers/gpu/drm/i915/gvt/gvt.h +++ b/drivers/gpu/drm/i915/gvt/gvt.h @@ -732,7 +732,7 @@ static inline int intel_gvt_read_gpa(struct intel_vgpu *vgpu, unsigned long gpa, { if (!vgpu->attached) return -ESRCH; - return vfio_dma_rw(vgpu->vfio_group, gpa, buf, len, false); + return vfio_dma_rw(&vgpu->vfio_device, gpa, buf, len, false); } /** @@ -750,7 +750,7 @@ static inline int intel_gvt_write_gpa(struct intel_vgpu *vgpu, { if (!vgpu->attached) return -ESRCH; - return vfio_dma_rw(vgpu->vfio_group, gpa, buf, len, true); + return vfio_dma_rw(&vgpu->vfio_device, gpa, buf, len, true); } void intel_gvt_debugfs_remove_vgpu(struct intel_vgpu *vgpu); diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index dc5d40b07e2381..7960a153879ba5 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -2108,32 +2108,28 @@ EXPORT_SYMBOL(vfio_group_unpin_pages); * As the read/write of user space memory is conducted via the CPUs and is * not a real device DMA, it is not necessary to pin the user space memory. * - * The caller needs to call vfio_group_get_external_user() or - * vfio_group_get_external_user_from_dev() prior to calling this interface, - * so as to prevent the VFIO group from disposal in the middle of the call. - * But it can keep the reference to the VFIO group for several calls into - * this interface. - * After finishing using of the VFIO group, the caller needs to release the - * VFIO group by calling vfio_group_put_external_user(). - * - * @group [in] : VFIO group + * @vdev [in] : VFIO device * @user_iova [in] : base IOVA of a user space buffer * @data [in] : pointer to kernel buffer * @len [in] : kernel buffer length * @write : indicate read or write * Return error code on failure or 0 on success. */ -int vfio_dma_rw(struct vfio_group *group, dma_addr_t user_iova, - void *data, size_t len, bool write) +int vfio_dma_rw(struct vfio_device *vdev, dma_addr_t user_iova, void *data, + size_t len, bool write) { struct vfio_container *container; struct vfio_iommu_driver *driver; int ret = 0; - if (!group || !data || len <= 0) + if (!data || len <= 0) return -EINVAL; - container = group->container; + ret = vfio_group_add_container_user(vdev->group); + if (ret) + return ret; + + container = vdev->group->container; driver = container->iommu_driver; if (likely(driver && driver->ops->dma_rw)) @@ -2142,6 +2138,8 @@ int vfio_dma_rw(struct vfio_group *group, dma_addr_t user_iova, else ret = -ENOTTY; + vfio_group_try_dissolve_container(vdev->group); + return ret; } EXPORT_SYMBOL(vfio_dma_rw); diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 8f2a09801a660b..91d46e532ca104 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -161,7 +161,7 @@ extern int vfio_group_pin_pages(struct vfio_group *group, extern int vfio_group_unpin_pages(struct vfio_group *group, unsigned long *user_iova_pfn, int npage); -extern int vfio_dma_rw(struct vfio_group *group, dma_addr_t user_iova, +extern int vfio_dma_rw(struct vfio_device *vdev, dma_addr_t user_iova, void *data, size_t len, bool write); extern struct iommu_domain *vfio_group_iommu_domain(struct vfio_group *group); From patchwork Mon May 2 17:31:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12834561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3B6AC433EF for ; Mon, 2 May 2022 17:31:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386672AbiEBRfR (ORCPT ); Mon, 2 May 2022 13:35:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40300 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352196AbiEBRfM (ORCPT ); Mon, 2 May 2022 13:35:12 -0400 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2063.outbound.protection.outlook.com [40.107.93.63]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CEEEB2AE0 for ; Mon, 2 May 2022 10:31:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=L91XIhmYw9r5ivX0nZgsIyzgHS9KB9tEhNIr196yJxJ/elFgRylD05cOfIlGsOHkLCerpwxHYo/QhJP73eiebShj9gz0lYYCOrM37+NNo85Ydqq6W2un5WiV90hH5O9SK8qEIr/nuAftAPY0/cb+rwb1qKKGDmPWHz4MdE9czE5crT98NTxfIWrhF0dNS5jX1Z1AB0P/j8JrQsj3sGCX7Bo3yrS8t7O/fKT+BsS+ZDAHGofyPoPBMallN7xcp1/f3Kbxz7o+606/YsO45QF+NMOb226TURgcL+DkCtkWOeKD0RKRJ3ACdQSZrQ5gPIeTUI07fSzybqqHmoK7y/MKpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rX3j7rOWSWO7c4hbUYMNBvD/GJno+vvgfbHyrqgd16w=; b=mt5N5Ec4lfk9ZtaAGm9hp8Oab+k6RzvvhmCs/DGoxt1E8J6XjaGyx/7XYwugzE2TWKi4PofW8DFXGfd8lBsg/DNrINwOqnMRUEoMtF5UVvHEyOGyTHrFZkv4z55b4UaQFHnUu5HZMzy426kw1/RRN2zZxL6uaQ/sQ5rh1gGDc1an2wCKWsn0Qk+Mh4iemYJXzRxSzBdmjsZb4+zkom04lrFGuDsuS/trGAMoB9R+j4+8bKwuopU3hcbZyladmigGvdFlFc+EA/cEeli3FGNKZ+L65Dj/0pzS3oW1TVxyTlOrkYpcCja0nbOL+Kt9FFYI3EdCPFDBboTMwutsZbsEtA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rX3j7rOWSWO7c4hbUYMNBvD/GJno+vvgfbHyrqgd16w=; b=ldzaEqzjrzNsqSp0FUvd1umzAxWOtAToOfJu1G4QVsgCJXWCD2Mwg0esQ7fKuQdLRr1GMQTF2Ixv0Ysj0rCOjEKIjwlUkKdM+nOI8Nu1MpqxEDZ7xrNCyokEaBiXMscmAlDJ/DQtxj8SUqDgmd8BRvBPl0r1X68qI5ock3+yROOjvNPPIVzmFAiWjrv7p5GRSQkPj+r9KsF0YgXyt8oU11Rk7TraKCyMyMpwaUtUAifCNUG2aHbHjeRt0YhsHwWdHq77p/fEMY+goEMLdy9ngj+/nI2F4PZD1gnOVRAiMVSSBLwChdeirXV4DM1++syPtZ3ZGkvewYGL1OdkNiHByA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by CO6PR12MB5458.namprd12.prod.outlook.com (2603:10b6:5:35b::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.13; Mon, 2 May 2022 17:31:40 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5206.024; Mon, 2 May 2022 17:31:40 +0000 From: Jason Gunthorpe Cc: Tony Krowiak , Eric Farman , Christoph Hellwig , intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org, "Jason J. Herne" , "Tian, Kevin" , kvm@vger.kernel.org, "Liu, Yi L" Subject: [PATCH v3 5/7] drm/i915/gvt: Change from vfio_group_(un)pin_pages to vfio_(un)pin_pages Date: Mon, 2 May 2022 14:31:35 -0300 Message-Id: <5-v3-e131a9b6b467+14b6-vfio_mdev_no_group_jgg@nvidia.com> In-Reply-To: <0-v3-e131a9b6b467+14b6-vfio_mdev_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR07CA0001.namprd07.prod.outlook.com (2603:10b6:208:1a0::11) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 026e049a-080e-408f-e90e-08da2c619cf9 X-MS-TrafficTypeDiagnostic: CO6PR12MB5458:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: wrg+/G5WbV7kvYvgZdd8HvKTxtHRUKF7r3qKNPpZdIwvFf8ys7IxfPNKqQLGsrJPiNkggK9s0UYK/axTECTpQ3irDkf/ah95I9vD6cNYg7sJx67NU/lJcP1Zg/kjKbSi+cbDYiiZY3k6uubTiYAkZvNyzd5hWrCqAVw84BbiyfvVhbdqdU2e36YptgsNFonHbM1oNfwRqVAh/K4LYFIvjNLyLEVqpUfsz9lpljZoixtjtTyKK+RxixUUCnSu9kk4QxX0aFOMHvyIb9IyIAYaJc9tV4BUcwNrhNEJdmEIRWYfdYS7hZ1LIaJr35KWvMRI6QMNw1d+0jG+VXa0vCc7bh43JE2g6el9xR25dUNsbzFBWzif2asbIUMXzJSLGn+a9VnGYzixAe3oMnsF8xvXEeUdmv00fUzuaWEv5VGlAKgqNm1FVdSikx0T4zBmWCEfNDB/P2tcin2sHHSd7yifir96RloUbH5vy9p/oBSqa2Fmi9vT2RIn0yVLQxrHaZdbDYOMv2BRLRVVJ7kVRAlE67qYUdEFbnCwNJwdX3BQs0/919W8k/YLinsHA6loZ9lSpA+7UyBv81f5iBz7ZDhlUt5BVYeUAcoA+NU/dJl/9gR8EwsEWtif1nY0zGtWf1pUmuTM4sGWpd3nXDEkPW/GaoLClvwquhDTt5Vyc6+0rOEhkkTAvOvcndBaRloRSkmyiRL87v534zdmNwaUKHLTYsptjnE/SAlpq3Gd43E6Iws= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(109986005)(6486002)(508600001)(2616005)(86362001)(6512007)(26005)(6506007)(38100700002)(6666004)(316002)(66476007)(2906002)(66946007)(66556008)(186003)(36756003)(5660300002)(8936002)(54906003)(4326008)(83380400001)(8676002)(4216001)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: WOigoFJ2xSaEYHyQ5VZ8FPJtL9XYnVvGJ5RK30CDeMMh0OSEapGjZCAUzTEhpqmtyTiMilniNkMeiQbCICj6LGJmQCdgOYH/j5AjuB7BADpCAnvOQggIQtQcMA0RpaYhCtpDdU9BjUY69lsC7ZjaA6n6gitZnyEOfjJHV+maMWgC26x3ltwq8H595GBQolUOV1w+GBQLcaovtK1WSqFJsgCEpRJhvgUIehW5FcGdJOD2XyWGxmHv5aNx4VyIKZEL1LlJ0RJP0fNYnYeWjnkFApLprGw979fm3WTQYCqXwtCsEvGeM4gcGor0C+BbPQnKfMthee+RbYmh/vOI//xZOoZuldlxsT0m+KgFueMT3tX304Cg7r2jXp7/myxGClXnMRNYy09u9XKPnA2NXJBFtOBSWLSTjYwfCXeeRk3yiCMn8ago83k8pePuPoZpA0SYwxcnX9te1uwXM9YDYY9zhNh+HsQ6mO1dNLW3/Qi8POkqQ8/kv7eqaoLQXaEyFt/bCnU72ZSXrVoTXveNYYb97G9xfFu1vHP8mhVbWbnEAOiddgoGQgtcnN1wio+q8U0m6fh/G4GUdEo6nOSj0NAZpzr9k313kSPeccnZOQp6ZvG1cC8yn2dHa9Gr68l7Yhp1awMKgZ2O+iNCNEAXjZaDQM/LWmTxb/xi/7+bJVf44W9De7woBODxmlzLyFGvf5Y+Bea9l66Soe8KcTYThKpA7yV08zpehbRhCOtDG6dEBCCAKJEouZLHg30snEP6XeXBtL7mHtmW5cZG2uq7A1dThwfj4ScPQJWOuN1RhI728KCSFQfmBpQ2Il/BoMnHqE8mrw1YWLcYgfwmPCyzC9cjZ7U1v9ULHflCHxIL56GqirWpCTErlDvWeQGSuPOYm/dE7kcy4wZekjcKT/A4glZZzocl+YlB5AOGY04emURV3qehcNnAqePMX+SqYctYbzRA+kMS/4LWG9JhJu4uunX0r0NARQ/BYkWNQy/Cl8LQeyapRmV+Op0D5jt0AIK3ZmJPfAoCUv38VVu0mwg8gG1gVzdgeHtoRTNcgPELgaVY8H+b1zS4HgJDCwjnkD4gh9ZLyT5uwduDquz2cxLjuTqWIkE1vqPoRkH6A42bayWkbzP4fMY0pR+pSTT0Q8ggi6jRZZyWHdgEz3Z5Ypy3a/Prlk1z/WWXSan05KAY1R1JYtAMQyQupFP3DOYYD+8ksAqyxHqq61ceZ4r8CilCterObA8xZOyCkKaPkYxF/OIQIRNO3PiK4Vw37PIrCYnI6Il8htZn2YZtwmEyLj+yeDdLfpACfcgVX8m1A6K+G+CL6mxO2PixjqMymr/jLpS1M+e6GcC/94ZbN9SBFsxUFBM/EEBsAoPs3slPwecy72fNXk0c2LGUt0QCxRaAJdOwgjtRewDqiHg0lXC0mtU7br4NwhS9nkUuvcT4SZ3vpB0ZTiBRBjonH1sohxuSUwvTWKyyLyT9RJ/asJDJvZKf529+1LKuTcwtdEhkcKHS1gz8jpCaLwKHH8np2oUJSQNfjKEuBPoJe1JjmCT9TiyGA6gl7izEH5nZNFGP+u241hd8zbynOQMWMJUYy5rCfzokvmN+TTdUZKY+tJ7UzgoveLCyjnXD9Y8nsIDX/b1GJA4U0Wb7tTSPL4PT13cz6qX3Oo7qNH6+tCZ/bMZ2eNDxOpSlqnkQJFW4o02j4VNq7yNSpcqeEJ4yLhWneVqcmE2UkH0xYOlawdZZRCMb1bSZgFwokw== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 026e049a-080e-408f-e90e-08da2c619cf9 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2022 17:31:38.4669 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: d7BggOxdsL/U8ESgeWqcTWtDf5ir48yWZNIpaRzV4ezJp/a/5vY+8VPFFWsmZ7Pt X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR12MB5458 To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Use the existing vfio_device versions of vfio_(un)pin_pages(). There is no reason to use a group interface here, kvmgt has easy access to a vfio_device. Delete kvmgt_vdev::vfio_group since these calls were the last users. Reviewed-by: Kevin Tian Reviewed-by: Christoph Hellwig Signed-off-by: Jason Gunthorpe --- drivers/gpu/drm/i915/gvt/gvt.h | 1 - drivers/gpu/drm/i915/gvt/kvmgt.c | 27 ++++++--------------------- 2 files changed, 6 insertions(+), 22 deletions(-) diff --git a/drivers/gpu/drm/i915/gvt/gvt.h b/drivers/gpu/drm/i915/gvt/gvt.h index 5a28ee965b7f3e..2af4c83e733c6c 100644 --- a/drivers/gpu/drm/i915/gvt/gvt.h +++ b/drivers/gpu/drm/i915/gvt/gvt.h @@ -231,7 +231,6 @@ struct intel_vgpu { struct kvm *kvm; struct work_struct release_work; atomic_t released; - struct vfio_group *vfio_group; struct kvm_page_track_notifier_node track_node; #define NR_BKT (1 << 18) diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c index 1cec4f1fdfaced..7655ffa97d5116 100644 --- a/drivers/gpu/drm/i915/gvt/kvmgt.c +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c @@ -243,7 +243,7 @@ static void gvt_unpin_guest_page(struct intel_vgpu *vgpu, unsigned long gfn, for (npage = 0; npage < total_pages; npage++) { unsigned long cur_gfn = gfn + npage; - ret = vfio_group_unpin_pages(vgpu->vfio_group, &cur_gfn, 1); + ret = vfio_unpin_pages(&vgpu->vfio_device, &cur_gfn, 1); drm_WARN_ON(&i915->drm, ret != 1); } } @@ -266,8 +266,8 @@ static int gvt_pin_guest_page(struct intel_vgpu *vgpu, unsigned long gfn, unsigned long cur_gfn = gfn + npage; unsigned long pfn; - ret = vfio_group_pin_pages(vgpu->vfio_group, &cur_gfn, 1, - IOMMU_READ | IOMMU_WRITE, &pfn); + ret = vfio_pin_pages(&vgpu->vfio_device, &cur_gfn, 1, + IOMMU_READ | IOMMU_WRITE, &pfn); if (ret != 1) { gvt_vgpu_err("vfio_pin_pages failed for gfn 0x%lx, ret %d\n", cur_gfn, ret); @@ -804,7 +804,6 @@ static int intel_vgpu_open_device(struct vfio_device *vfio_dev) struct intel_vgpu *vgpu = vfio_dev_to_vgpu(vfio_dev); unsigned long events; int ret; - struct vfio_group *vfio_group; vgpu->iommu_notifier.notifier_call = intel_vgpu_iommu_notifier; vgpu->group_notifier.notifier_call = intel_vgpu_group_notifier; @@ -827,28 +826,19 @@ static int intel_vgpu_open_device(struct vfio_device *vfio_dev) goto undo_iommu; } - vfio_group = - vfio_group_get_external_user_from_dev(vgpu->vfio_device.dev); - if (IS_ERR_OR_NULL(vfio_group)) { - ret = !vfio_group ? -EFAULT : PTR_ERR(vfio_group); - gvt_vgpu_err("vfio_group_get_external_user_from_dev failed\n"); - goto undo_register; - } - vgpu->vfio_group = vfio_group; - ret = -EEXIST; if (vgpu->attached) - goto undo_group; + goto undo_register; ret = -ESRCH; if (!vgpu->kvm || vgpu->kvm->mm != current->mm) { gvt_vgpu_err("KVM is required to use Intel vGPU\n"); - goto undo_group; + goto undo_register; } ret = -EEXIST; if (__kvmgt_vgpu_exist(vgpu)) - goto undo_group; + goto undo_register; vgpu->attached = true; kvm_get_kvm(vgpu->kvm); @@ -868,10 +858,6 @@ static int intel_vgpu_open_device(struct vfio_device *vfio_dev) atomic_set(&vgpu->released, 0); return 0; -undo_group: - vfio_group_put_external_user(vgpu->vfio_group); - vgpu->vfio_group = NULL; - undo_register: vfio_unregister_notifier(vfio_dev, VFIO_GROUP_NOTIFY, &vgpu->group_notifier); @@ -925,7 +911,6 @@ static void __intel_vgpu_release(struct intel_vgpu *vgpu) gvt_cache_destroy(vgpu); intel_vgpu_release_msi_eventfd_ctx(vgpu); - vfio_group_put_external_user(vgpu->vfio_group); vgpu->kvm = NULL; vgpu->attached = false; From patchwork Mon May 2 17:31:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12834560 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8CE58C433F5 for ; Mon, 2 May 2022 17:31:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1386670AbiEBRfP (ORCPT ); Mon, 2 May 2022 13:35:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40286 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386639AbiEBRfL (ORCPT ); Mon, 2 May 2022 13:35:11 -0400 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2063.outbound.protection.outlook.com [40.107.93.63]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D976B62C5 for ; Mon, 2 May 2022 10:31:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iQD5Ylc8iUhVn4i2odGuFqE7aocF4vXA7piqTzCXc3SANn0T1neroBy3+GboEBxiYlVBGTYRsQRy+CCW1Fp/UGh1fHeGIDD6+3G1tMc/TsyiWIhe+9hxZ9iLzW003cuBPaeVO0PyuK6y5aPL7QONQ9dGC/wzMyQrNm3MDbuVNjjo7xqZSFAQ66rkR/C8iWrTD7cXKMjvzd+9qTWR5Ztt+hXlyhHssJXAF68f/+Yy1D61mX+fUmJ+apO/S6vad1fWDzF6AaiCHf/KfQZSiu+/wGNro/BFiRNaLXTMAXheSVd2aOtkZsc50VyxTrDQORn0gXBPeNraLyPHjEoo6ofG1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ww9bH2biUzKJJodU726/qqkRUF5l4LTSNft/M4XYHG0=; b=eNOTi3xoMJwDDlkbxu8rpGRGc4zOI5RKeMBQ2GzBjitIIVQ+htKjzmxUbeWI3YbB3oB2jhJjFwzpv7Z/l0WpvZkmf1hrFQ4igSqj88IsrA4lJbRrIh/zjW6qWDw8i+2Kaw1LYeY4bd41cCPh7X7PkqHItiwAJHDivc0mPHwDfkuEGL2ztWAFCSMN2Qsti8d2uSU1ed1dPZjgqG+/uBpuedELBXRf/PdHHIlR7CdiYxz/53Ym1x6G971IH4GlP+EmCgG7PqsCUlWY7oGJrWqfr89aBpvHTp4hMN+pU9N5VpUbU4zKMYN8jiW9RI2pDi+oe2COTryFAN7eyv8MjdWEqg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ww9bH2biUzKJJodU726/qqkRUF5l4LTSNft/M4XYHG0=; b=ZDCCtphtmIqPeXZVI9zbgONpxqFvvpamcsztyEm/5BLCCrbV8P/fYnDRUJTxAtbrIIgx9eYkkZEuniQuTFbsH10uYMOaxyajl2POQEJb4ZE9M9V9QQrv4RX90IpDlq516HnDUbHR5c0Iu+q40u0qER7sfjeOxcGCcfdZj68GKh2jPq6P4OLdcqDEiKBJhTI05gktYGuWyJESCkIqXRjcYJ3ob+0TCPkKlPF67Ag8rh9o5vI3aEmC0IhBnBkWjXkgI4kkxvaZiSfFKQflXQXX/lUFOwQNGcfhRNzpWFfwqUHoziz37pjvTLuDiJUu8x8nziRdXhITJZDSHPA65mJY8A== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by CO6PR12MB5458.namprd12.prod.outlook.com (2603:10b6:5:35b::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.13; Mon, 2 May 2022 17:31:40 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5206.024; Mon, 2 May 2022 17:31:40 +0000 From: Jason Gunthorpe Cc: Tony Krowiak , Eric Farman , Christoph Hellwig , intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org, "Jason J. Herne" , "Tian, Kevin" , kvm@vger.kernel.org, "Liu, Yi L" Subject: [PATCH v3 6/7] vfio: Remove dead code Date: Mon, 2 May 2022 14:31:36 -0300 Message-Id: <6-v3-e131a9b6b467+14b6-vfio_mdev_no_group_jgg@nvidia.com> In-Reply-To: <0-v3-e131a9b6b467+14b6-vfio_mdev_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL1PR13CA0072.namprd13.prod.outlook.com (2603:10b6:208:2b8::17) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 01da8ae3-f4f5-457f-601c-08da2c619cf0 X-MS-TrafficTypeDiagnostic: CO6PR12MB5458:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zR5YygArsp/yBS93zqzoberdYk3ZISr8zPivqGxCc/kcZQ6riJfomXH1+cu7f6re0jIi5OPGjr+cawqhK+wa9EPM1dZVSATh4h6wUyIEOTC3WZoBMRLea8v6g/kjnqRnOrR8M6HbwLr4NEkdQQ0alUAXNF+p+cgXGvyCi+t2IGLgJe1WCHIVHf2U5vppVuMJMkmgekrxnkSaJC5e1KMjwE7isIGYI2E312Aeoxwk0Ot7Uz/4pgeXcdYx7sCC1E550g3DFFbnARUeOsRRF4G6lZeZfhugqtpI5h9YJnFL5jK4q24GZ9WTWEUQYK3Ldc12KY23GpmQFsEa/eY+YAyL2drEh3vcaz4R0hkMmvHGaQCuH/VEvzFmntOKSgg/cPWzrTWerGH/x3t5xc8VAxlNpAUm0oK8bK7guiS2D6O02FffIi4d09vbj6PHG5Idh6azxGlFkuviZflkd509/gVhbjgciB7p9JW+lGHbXUd/Xr9fEtCPqiaNEXyTXfMz5l6WwXVi7uvlw+bkk485mH2W/zG7TG7NUEvysb1ysFw6D+P1LjdX15nIZPeMGlVuuU11SyScSq5ZljjbdBcLdwQx+07B9XnWVjDn/xBWTw4GsWAXuJUiHRKMXLolgi2pSQU/rWMqfp/1kJl+JD1ybkpvgzOCFoCFTr4LmD+U1msIOeUF2lc51GhRB/JS3TbdWsBy+wSq/XwKiir7OmrNbIP/Ob+s6wFvrazcy/amwSL1oEc= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(109986005)(6486002)(508600001)(2616005)(86362001)(6512007)(26005)(6506007)(38100700002)(6666004)(316002)(66476007)(2906002)(66946007)(66556008)(186003)(36756003)(5660300002)(8936002)(54906003)(4326008)(83380400001)(8676002)(4216001)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: wLYwjBmv8F4/XVSq7gGH7ltPbxNPoCInpZo//9ZpznVxdnDHUIqXlfZrvK2ex8LG1qV8rTI71vGc0ePrzH1D0fhayWZDnN3yEKNTQKqsLcufgLHD9rp0hzV8MExHpIoZjXv5xNRcwkkCuOCP3z/22gTb9rugghKC3dYs6hLwsuSBZ9wl+aA/sV2yu2iVcXjOqBwwGdZF9WoII1Fk6R5K9JjpkR1IEjXPr1qGx2ZSQ4HojVlZcamUR+VjZQYPRQ9E/w7VI2PiWZ5MJmyp0nMm/68D80fiRELOhE6HuumFX2mqe07mm6h1jM2Z+bDJlUzVWJyGRHxoBnRAWOWgVRlucL8vfhkQAQat/s4AguQLjdudaIzxmr/pov0PmYs2FJAmFjDILIbcZDMk5SSMkvdP07oiFyIEsz3Ikvdy8ySXfKKX1wR6lzwO6t6GmJNrGRjK7ge0QCuJPKjlqzUJXoYegm7pS6rIjZplJb20BoZQALmfgp15kJJErmJXxKjk29DPL4C5/c+BO9fRp8ixDgHmpjowWMLnXiZe+kRID+x0r2PAFc/7VlqJdxPyeQiVPRyaNT8zvx9VDzFH2TmwG93nPNrWaKWHibCmOvRgkJb90xFdR596cw1UX/4t2r27kBESV/GEt1WY52zsKyUOyMOEhh9jYn/7Dk5K6kk3iAG+TxzsZDmTOHQ3biX13NwfJJpiurCMDaGdvaZQlrFVVrajh20hSpPgp8LQROhCiK6Rs+/JUUTQiJkXn7dsfEfyKbyBT+MmwfadhluBIb7/IC2i/inl/n/YTz0GYtRaIFwa/H4NO1Lt+YRHJRpgaKSy9/VH1gJshRa8h+g8zLIgAJfplWZAmVzZBd1R19KyhqjKxn1qzNbZEOG+Sl+EMnZd/jmJR7XNXQ/YgZU7nwh2e7pgodhP153b4ACozsiPqRrDJyUINyn+dDNOtrdLeswFrbaigkC8ELc/prgdON7Ihk+bni4f7whBrLeE7Son19QAinPdhampp7lzDyk8Dkzu7BMtUCbmQx+AnK/e0oWDdZx/0x0BmpMs7c8lDGBrdYtfm7btF7KsWWC3iQhlW7eZV9kHam9S25110iQKVINTEyBSoSOgvzo24BAoMGny0pDpxO8yuzd0w6941Bq9YkdGAatomUohCT8ZWswRLzmu7OMFrOC6/fWf7F8W93arV5hcvnrsIt6rnVZAjDkZ584x5lwk0IKT58UO9unfF1mnWW9XAd209ZHuAO+9gnoAoVkUYbuJ283GmZyBkcd3JYKqgn+Nrf6h9IpR0+VkRtTkfI30Wb1uSRamIn498eyRAeUKDMHQtc7/zFI1c2eUd9Oy/VJMQbOpuErFx8Y1UcjfnzkRMR06OWOwypWHD3Iow84+Zrs+tEGCtUwY1KC5pJDiidhlaeHKxt3t4Ax3L0LHDZWnw67vpMVHPZ04APDxP9C7Vv1eSe14reDlQsUlcL5KaSy8tp4MQ3SnrtExM/tHeQA4ci3f2L7rJ5hHw1+gDKfOMVVzVfpjE6CYQqkwEQhyaHI+ioCaZ3DT4XORV0K8Jcs+HOjIadMKWOXhqwfcCWCPJzXo1M6EnTBzqlc5G8dXVqkVd5+MS1dSnMCkoR+uZu8fRicBZXJvNEZitEyNAODbQw0xhs2BEn4fvpDgHPLG2DZRm//Bwri4DihP8Xf4Thz2CsbJv7AcBJKSA10T/tQ8gshSeFe2Z1ytXdoZ+N4VZ44zp42A/evmfbqD71JAROcDqg== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 01da8ae3-f4f5-457f-601c-08da2c619cf0 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2022 17:31:38.4357 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: u1vCd2C6mCAiFSD5KH71PE0Zvu7ASMgXWjE6l6qiKe7fZ5DJqvmSvgcFZMv+Bser X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR12MB5458 To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Now that callers have been updated to use the vfio_device APIs the driver facing group interface is no longer used, delete it: - vfio_group_get_external_user_from_dev() - vfio_group_pin_pages() - vfio_group_unpin_pages() - vfio_group_iommu_domain() Reviewed-by: Christoph Hellwig Reviewed-by: Kevin Tian Signed-off-by: Jason Gunthorpe --- drivers/vfio/vfio.c | 151 ------------------------------------------- include/linux/vfio.h | 11 ---- 2 files changed, 162 deletions(-) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 7960a153879ba5..0184d760ec1e44 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -1732,44 +1732,6 @@ struct vfio_group *vfio_group_get_external_user(struct file *filep) } EXPORT_SYMBOL_GPL(vfio_group_get_external_user); -/* - * External user API, exported by symbols to be linked dynamically. - * The external user passes in a device pointer - * to verify that: - * - A VFIO group is assiciated with the device; - * - IOMMU is set for the group. - * If both checks passed, vfio_group_get_external_user_from_dev() - * increments the container user counter to prevent the VFIO group - * from disposal before external user exits and returns the pointer - * to the VFIO group. - * - * When the external user finishes using the VFIO group, it calls - * vfio_group_put_external_user() to release the VFIO group and - * decrement the container user counter. - * - * @dev [in] : device - * Return error PTR or pointer to VFIO group. - */ - -struct vfio_group *vfio_group_get_external_user_from_dev(struct device *dev) -{ - struct vfio_group *group; - int ret; - - group = vfio_group_get_from_dev(dev); - if (!group) - return ERR_PTR(-ENODEV); - - ret = vfio_group_add_container_user(group); - if (ret) { - vfio_group_put(group); - return ERR_PTR(ret); - } - - return group; -} -EXPORT_SYMBOL_GPL(vfio_group_get_external_user_from_dev); - void vfio_group_put_external_user(struct vfio_group *group) { vfio_group_try_dissolve_container(group); @@ -2003,101 +1965,6 @@ int vfio_unpin_pages(struct vfio_device *vdev, unsigned long *user_pfn, } EXPORT_SYMBOL(vfio_unpin_pages); -/* - * Pin a set of guest IOVA PFNs and return their associated host PFNs for a - * VFIO group. - * - * The caller needs to call vfio_group_get_external_user() or - * vfio_group_get_external_user_from_dev() prior to calling this interface, - * so as to prevent the VFIO group from disposal in the middle of the call. - * But it can keep the reference to the VFIO group for several calls into - * this interface. - * After finishing using of the VFIO group, the caller needs to release the - * VFIO group by calling vfio_group_put_external_user(). - * - * @group [in] : VFIO group - * @user_iova_pfn [in] : array of user/guest IOVA PFNs to be pinned. - * @npage [in] : count of elements in user_iova_pfn array. - * This count should not be greater - * VFIO_PIN_PAGES_MAX_ENTRIES. - * @prot [in] : protection flags - * @phys_pfn [out] : array of host PFNs - * Return error or number of pages pinned. - */ -int vfio_group_pin_pages(struct vfio_group *group, - unsigned long *user_iova_pfn, int npage, - int prot, unsigned long *phys_pfn) -{ - struct vfio_container *container; - struct vfio_iommu_driver *driver; - int ret; - - if (!group || !user_iova_pfn || !phys_pfn || !npage) - return -EINVAL; - - if (group->dev_counter > 1) - return -EINVAL; - - if (npage > VFIO_PIN_PAGES_MAX_ENTRIES) - return -E2BIG; - - container = group->container; - driver = container->iommu_driver; - if (likely(driver && driver->ops->pin_pages)) - ret = driver->ops->pin_pages(container->iommu_data, - group->iommu_group, user_iova_pfn, - npage, prot, phys_pfn); - else - ret = -ENOTTY; - - return ret; -} -EXPORT_SYMBOL(vfio_group_pin_pages); - -/* - * Unpin a set of guest IOVA PFNs for a VFIO group. - * - * The caller needs to call vfio_group_get_external_user() or - * vfio_group_get_external_user_from_dev() prior to calling this interface, - * so as to prevent the VFIO group from disposal in the middle of the call. - * But it can keep the reference to the VFIO group for several calls into - * this interface. - * After finishing using of the VFIO group, the caller needs to release the - * VFIO group by calling vfio_group_put_external_user(). - * - * @group [in] : vfio group - * @user_iova_pfn [in] : array of user/guest IOVA PFNs to be unpinned. - * @npage [in] : count of elements in user_iova_pfn array. - * This count should not be greater than - * VFIO_PIN_PAGES_MAX_ENTRIES. - * Return error or number of pages unpinned. - */ -int vfio_group_unpin_pages(struct vfio_group *group, - unsigned long *user_iova_pfn, int npage) -{ - struct vfio_container *container; - struct vfio_iommu_driver *driver; - int ret; - - if (!group || !user_iova_pfn || !npage) - return -EINVAL; - - if (npage > VFIO_PIN_PAGES_MAX_ENTRIES) - return -E2BIG; - - container = group->container; - driver = container->iommu_driver; - if (likely(driver && driver->ops->unpin_pages)) - ret = driver->ops->unpin_pages(container->iommu_data, - user_iova_pfn, npage); - else - ret = -ENOTTY; - - return ret; -} -EXPORT_SYMBOL(vfio_group_unpin_pages); - - /* * This interface allows the CPUs to perform some sort of virtual DMA on * behalf of the device. @@ -2300,24 +2167,6 @@ int vfio_unregister_notifier(struct vfio_device *dev, } EXPORT_SYMBOL(vfio_unregister_notifier); -struct iommu_domain *vfio_group_iommu_domain(struct vfio_group *group) -{ - struct vfio_container *container; - struct vfio_iommu_driver *driver; - - if (!group) - return ERR_PTR(-EINVAL); - - container = group->container; - driver = container->iommu_driver; - if (likely(driver && driver->ops->group_iommu_domain)) - return driver->ops->group_iommu_domain(container->iommu_data, - group->iommu_group); - - return ERR_PTR(-ENOTTY); -} -EXPORT_SYMBOL_GPL(vfio_group_iommu_domain); - /* * Module/class support */ diff --git a/include/linux/vfio.h b/include/linux/vfio.h index 91d46e532ca104..9a9981c2622896 100644 --- a/include/linux/vfio.h +++ b/include/linux/vfio.h @@ -140,8 +140,6 @@ int vfio_mig_get_next_state(struct vfio_device *device, */ extern struct vfio_group *vfio_group_get_external_user(struct file *filep); extern void vfio_group_put_external_user(struct vfio_group *group); -extern struct vfio_group *vfio_group_get_external_user_from_dev(struct device - *dev); extern bool vfio_external_group_match_file(struct vfio_group *group, struct file *filep); extern int vfio_external_user_iommu_id(struct vfio_group *group); @@ -154,18 +152,9 @@ extern int vfio_pin_pages(struct vfio_device *vdev, unsigned long *user_pfn, int npage, int prot, unsigned long *phys_pfn); extern int vfio_unpin_pages(struct vfio_device *vdev, unsigned long *user_pfn, int npage); - -extern int vfio_group_pin_pages(struct vfio_group *group, - unsigned long *user_iova_pfn, int npage, - int prot, unsigned long *phys_pfn); -extern int vfio_group_unpin_pages(struct vfio_group *group, - unsigned long *user_iova_pfn, int npage); - extern int vfio_dma_rw(struct vfio_device *vdev, dma_addr_t user_iova, void *data, size_t len, bool write); -extern struct iommu_domain *vfio_group_iommu_domain(struct vfio_group *group); - /* each type has independent events */ enum vfio_notify_type { VFIO_IOMMU_NOTIFY = 0, From patchwork Mon May 2 17:31:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 12834564 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82B75C433F5 for ; Mon, 2 May 2022 17:31:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1348252AbiEBRfW (ORCPT ); Mon, 2 May 2022 13:35:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1386673AbiEBRfQ (ORCPT ); Mon, 2 May 2022 13:35:16 -0400 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2063.outbound.protection.outlook.com [40.107.93.63]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C9126303 for ; Mon, 2 May 2022 10:31:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gRnyYOu0ZBciM9S7DefSNVHFeWwRlxRCRjhLRw9PPnwDuH35D2b0SPIEa3uwU2W4ZLhgVOT5r77Ff844ef1fDFanvhoarCGS6aVV21D6kq76rud+Mg0Pmytb8x9e+E9qoQGc+Ml+Xk3/m8sVbb7C2MuGrtkYa6FEwEHmH61voMpTTIkgrYH6dM8O70ZRAjKWg0K+Pewy8gRZdAWEAn+jdJgEMPOLvfTenZxDBSTZLRHjUHq/wquytWU5PQ3BUo1YQe8Dyovv4oruXFyajpJmvEVz0MgvmQ22CbxBVL/MdeMtHBaPws9tCzU8GlJR4ePRBGce7cphUN/oEtqDoJfopw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kpAa7bGLl2IxIgrwhnI2gmSgYw9pFpZbZratY2IedFc=; b=dERe8ZgJ0gcH5/cYkfvYavOTAMnfmuzSDX1abViDzPTejB4l+SEW/vYTA/boxF2FCouDC5i0zeZa2GSKuuWoQToCv44G52ltApTlwIxTHqbJKUwIPYOgZ128CvYuvzyXb4aw4eDCct3p90QhQBTAbGg6etzM5aaIuzSFU469bl3HsqmTve+sY9Y2V+bA3xWgf/vbnPOCKYgPREAUvIP6t0tHmZeQjeYpItSJcdq+d3vs2nzx2MK6a+HBnN9TGpbTfWdy2VFeOIkWtYMnlf5LAXDmC8kFfu8fz2oo0ixG5MBHLeuEtp0TolOnzPN/r/ZpRD4UGksDBSDNQ48lBoYAWw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kpAa7bGLl2IxIgrwhnI2gmSgYw9pFpZbZratY2IedFc=; b=bgrMLlBCkQXqbvRQWxnjJUL2aO65JoHfXQ+8NGjybtt0ai0/YCR8QbAWHAV275JhSv8iiT4IvSDzBDCIdWafm3aUBDjTVnWPTWC+yrl1qastWuQCy2Vjkr7udB6ecqIc0D43xuQn1QGHWGgHkAwwXP+xLSWygEt7UQQxYnnFT2qS6nh5jLYgoNPJPRxCn/ebQGYx8cpSj57o3yRlkzwBM5nYU1zg3bGaGBhPf2AwxqvSNsdCV2rhaYxPYWguPjiw07aLZcEKM0AWxGHfP/epa+Y0MYXQlmv7wzi57q8HSMk/oWefspgAbDOeTo1BeZl3+ASMv4cOQq1tthozQTyOCQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by CO6PR12MB5458.namprd12.prod.outlook.com (2603:10b6:5:35b::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5206.13; Mon, 2 May 2022 17:31:42 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::ec2d:9167:1b47:2db2%6]) with mapi id 15.20.5206.024; Mon, 2 May 2022 17:31:42 +0000 From: Jason Gunthorpe Cc: Tony Krowiak , Eric Farman , Christoph Hellwig , intel-gfx@lists.freedesktop.org, intel-gvt-dev@lists.freedesktop.org, "Jason J. Herne" , "Tian, Kevin" , kvm@vger.kernel.org, "Liu, Yi L" Subject: [PATCH v3 7/7] vfio: Remove calls to vfio_group_add_container_user() Date: Mon, 2 May 2022 14:31:37 -0300 Message-Id: <7-v3-e131a9b6b467+14b6-vfio_mdev_no_group_jgg@nvidia.com> In-Reply-To: <0-v3-e131a9b6b467+14b6-vfio_mdev_no_group_jgg@nvidia.com> References: X-ClientProxiedBy: BL1PR13CA0090.namprd13.prod.outlook.com (2603:10b6:208:2b8::35) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 27245f75-04da-4695-59d3-08da2c619d8b X-MS-TrafficTypeDiagnostic: CO6PR12MB5458:EE_ X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ka+rcU4Bl5Y7GiWCLtTFjqihVG5KaXtsif3SrgHYOy+mUL74+ZYOlUI69+Jj2SMeaKiowzRI3EIsLWe0pQLDy0yIbAi4ysL+ff0kcUI2cMa7VU5+fkejpIQvSLa7TzWXB1/YpNWWbo7INIA2HBuCttNOYxliQ95mImGeZT8G1z+0U5PtR1S78QBQTdTLIcevmORxTai8ld5iCS40cmoseNjhOgjijij8WcNQcEUkgofr+qpoWYgqSbCYQC+8s8n0Hvpwix2HOL3gd7UcZ/mz3giKwq80lqyiVXhiHO8ctMXq7u2hyi48UhMCKFDZTodaTKTsPVs/HFYdPMCf+BUOlnjNUd3o5m7AExcXvKTGdsD5CduB1WMcK863047trWx1NvAd5mgSrxqbVBMRByQd8kh2RMXFDvUvYepZA868ogKyRAisLxmIHNJ7cL4THpjlxa2CyLiHPB3Rnypg++RISYQGMIJPMr31lgscuDBZVB2NXTQOi0GWBNHL0mpkoh8o0Z5iCzqDGUW4bCquvks8OvsK2vnbvD7pcvn6KwnChZmJX1nVvXy/iETRwR3Aic/4w9x0dprf8aLaUr+e9hsKJ5oa9Qtvl6lGnHxR3/gP2mSpxDzXiS/Jev2VgVUF2hzFSU2HzcvwNF9tUljgxANpBLSVyAWQYVnsSsqBNBZELoB6pl0wuFafMvUlhwQn5E+8EpUjwzIHA0ID6Gvwy31RLg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230001)(4636009)(366004)(109986005)(6486002)(508600001)(2616005)(86362001)(6512007)(26005)(6506007)(38100700002)(6666004)(316002)(66476007)(2906002)(66946007)(66556008)(186003)(36756003)(5660300002)(8936002)(54906003)(4326008)(83380400001)(8676002)(4216001)(266003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: OSiVmvsxaWgc5E//AKXcfc057e8GECn0dv4wCpwIuwBoGeRdurm9TJBGBxuyDculdD6D+LaG2XrMzmdN7zIG6+rHoZ5IHgHXn/ZUeJbzRdEd9IutoBf68aIwck6OrnXj6FBw5nGaLzda96WVlFiYWYdjQA3MqcEsZ36tqXJNjT2tIJpcnxSEhec7SQNH+qDdDFjh5+MFKEsNBf3fP8RsWe79Mb7Qc0CprQhdxmO2CGO/PA2uF4VPwoTpGku2EJsMmNwr9FnKtCm14DAfldQCYCyG4r3G8vE06CsMEXKdaR5bn8YX//8UGGaZ4H8Mu7wzuCj/znZvmyOeDbo/1TyHkxEY+Kn1cA5hB3P2gDaWHC6YMSWIciids0tgxh4ShjGCW6jGeIsbc7VOwzBz+rV2T+/W1bTexxsPgnwIJ6flI7mCwFlHlT1z6Qln9m6XdsRIH5pJdYLZpYtY234mhyQDm/Ngpw6oiFswyEUWoJcEgppfORQVgoE7IVJk/Zs5OjRxWtMksgF2ijZ/Rm8jBnNhGjxW2X4XISFVRFBCX7P43ZniFctvnKx8jry5cPxH59xGmhxzhEK7M1e+XDRxBsJqN1G3dTKcdAEvyCHZ20DmkYzUDfhn0bvINBpwkmeFO7vmNoGUdXQo7wEXJ7bQ5gHgYk8IQXL7kMVWrQuQny8jPQVsrwwciJbQP0NhuezqEPprbZ24gVp1WyU5ywXmWzzsKorEdF/hJ+8k3g/EyQzKTQmfEuL7vJ43k0IVFyROsnzVvjDy1HlpEATeCLk207XR0NZAsAPaonQ8Jjh973CISEV8jlxC9Wxg7Q455TNAX4fMydsYkBp7ca53drK5sbDghzr5mGGcbz3WVBtjpRdTUoDgr3JlEzeNeyb+1lNeD4xZFY+i5N+33T0wzjNFS/0Cv9/NBY8WlQl6lvlrmS7PUwGREFnB6z/T8uqeXk4I4QnxOBi3QQ8hADbBXiRt/Ass4HF4I/5lwqILmbkFgqAxagjw5Vrb6VtQno7dGPB8D3QQJobAH5xKGwfCAEpKn4zY4EZN/DPpZHuW0p45OOg5IQkAbADkcavW6YhSpBs5qFWP55ae7wco43OctM9cmVlVOqhfkS3o3IKojaoh6P2CjAwWSUXf3CzfRZPuwTuanlLoTfiSi/OXnihnyjR14wRydvdwqCGNsn+0IsZPHX9bevgrDGlpCv6SF4YDSQGQXhbNa3ok2IJF1raiBuoQpuZWxx970S61KD6+vz55sKvtawcJbplr1d1R/ZDHqkdjCvk1SpJLfgmFUNsm4Wpl/2P3H1oa2Vgn4z+PdG+97ejxD69dW7xIbykr7LD6Zo25rYg5cSk1CEmbx5mcylG0M/f3BospHbA6w/cjS/idCEBCVBw6ar6wdmWvndOPV4hTdUJZi5T6dFCuR2/NY58SH/14+7uJSBMlN8gt+dspFZUwuniJ0Cdf7wboy0CUPNPoRWGGyF3xc3PM0bBiGBNW0qfvrLLsoNIBzCIc7EyuhxeuK2MYdRqAf/121c+ZcRLYrZlOO+CVmHkaQjLVdGx3QC7DMp6S9Kn6iNI+ISvj15w7M5EBwxjgrhfDzNPQK85CvUIAFCzbSeB/ZEYla9GMky1nXSxKzn1JLewtDlejtpdOwgV3uQk4Um1HRmd9FKhedHWg5i+IUoxG1vGseHHHcLKZiGVsz1lMRI89HIXFdVpn+AhJ81Kz9uA/sLPP/fMz1ZYNOYBcJc3OGCXMgvS9FV5jew== X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 27245f75-04da-4695-59d3-08da2c619d8b X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2022 17:31:39.4513 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hBKeHbLcFJOzd5oN1SZwRFhRv2VbZOaTDdbRQp8SUdSfG1jn0LQVFHlPNBsi37m6 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR12MB5458 To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When the open_device() op is called the container_users is incremented and held incremented until close_device(). Thus, so long as drivers call functions within their open_device()/close_device() region they do not need to worry about the container_users. These functions can all only be called between open_device() and close_device(): vfio_pin_pages() vfio_unpin_pages() vfio_dma_rw() vfio_register_notifier() vfio_unregister_notifier() Eliminate the calls to vfio_group_add_container_user() and add vfio_assert_device_open() to detect driver mis-use. This causes the close_device() op to check device->open_count so always leave it elevated while calling the op. Reviewed-by: Christoph Hellwig Reviewed-by: Kevin Tian Signed-off-by: Jason Gunthorpe --- drivers/vfio/vfio.c | 78 +++++++++------------------------------------ 1 file changed, 15 insertions(+), 63 deletions(-) diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c index 0184d760ec1e44..f7d1898129ad1c 100644 --- a/drivers/vfio/vfio.c +++ b/drivers/vfio/vfio.c @@ -1115,6 +1115,12 @@ static int vfio_group_add_container_user(struct vfio_group *group) static const struct file_operations vfio_device_fops; +/* true if the vfio_device has open_device() called but not close_device() */ +static bool vfio_assert_device_open(struct vfio_device *device) +{ + return !WARN_ON_ONCE(!READ_ONCE(device->open_count)); +} + static int vfio_group_get_device_fd(struct vfio_group *group, char *buf) { struct vfio_device *device; @@ -1329,8 +1335,10 @@ static int vfio_device_fops_release(struct inode *inode, struct file *filep) struct vfio_device *device = filep->private_data; mutex_lock(&device->dev_set->lock); - if (!--device->open_count && device->ops->close_device) + vfio_assert_device_open(device); + if (device->open_count == 1 && device->ops->close_device) device->ops->close_device(device); + device->open_count--; mutex_unlock(&device->dev_set->lock); module_put(device->dev->driver->owner); @@ -1897,7 +1905,7 @@ int vfio_pin_pages(struct vfio_device *vdev, unsigned long *user_pfn, int npage, struct vfio_iommu_driver *driver; int ret; - if (!user_pfn || !phys_pfn || !npage) + if (!user_pfn || !phys_pfn || !npage || !vfio_assert_device_open(vdev)) return -EINVAL; if (npage > VFIO_PIN_PAGES_MAX_ENTRIES) @@ -1906,10 +1914,6 @@ int vfio_pin_pages(struct vfio_device *vdev, unsigned long *user_pfn, int npage, if (group->dev_counter > 1) return -EINVAL; - ret = vfio_group_add_container_user(group); - if (ret) - return ret; - container = group->container; driver = container->iommu_driver; if (likely(driver && driver->ops->pin_pages)) @@ -1919,8 +1923,6 @@ int vfio_pin_pages(struct vfio_device *vdev, unsigned long *user_pfn, int npage, else ret = -ENOTTY; - vfio_group_try_dissolve_container(group); - return ret; } EXPORT_SYMBOL(vfio_pin_pages); @@ -1941,16 +1943,12 @@ int vfio_unpin_pages(struct vfio_device *vdev, unsigned long *user_pfn, struct vfio_iommu_driver *driver; int ret; - if (!user_pfn || !npage) + if (!user_pfn || !npage || !vfio_assert_device_open(vdev)) return -EINVAL; if (npage > VFIO_PIN_PAGES_MAX_ENTRIES) return -E2BIG; - ret = vfio_group_add_container_user(vdev->group); - if (ret) - return ret; - container = vdev->group->container; driver = container->iommu_driver; if (likely(driver && driver->ops->unpin_pages)) @@ -1959,8 +1957,6 @@ int vfio_unpin_pages(struct vfio_device *vdev, unsigned long *user_pfn, else ret = -ENOTTY; - vfio_group_try_dissolve_container(vdev->group); - return ret; } EXPORT_SYMBOL(vfio_unpin_pages); @@ -1989,13 +1985,9 @@ int vfio_dma_rw(struct vfio_device *vdev, dma_addr_t user_iova, void *data, struct vfio_iommu_driver *driver; int ret = 0; - if (!data || len <= 0) + if (!data || len <= 0 || !vfio_assert_device_open(vdev)) return -EINVAL; - ret = vfio_group_add_container_user(vdev->group); - if (ret) - return ret; - container = vdev->group->container; driver = container->iommu_driver; @@ -2004,9 +1996,6 @@ int vfio_dma_rw(struct vfio_device *vdev, dma_addr_t user_iova, void *data, user_iova, data, len, write); else ret = -ENOTTY; - - vfio_group_try_dissolve_container(vdev->group); - return ret; } EXPORT_SYMBOL(vfio_dma_rw); @@ -2019,10 +2008,6 @@ static int vfio_register_iommu_notifier(struct vfio_group *group, struct vfio_iommu_driver *driver; int ret; - ret = vfio_group_add_container_user(group); - if (ret) - return -EINVAL; - container = group->container; driver = container->iommu_driver; if (likely(driver && driver->ops->register_notifier)) @@ -2030,9 +2015,6 @@ static int vfio_register_iommu_notifier(struct vfio_group *group, events, nb); else ret = -ENOTTY; - - vfio_group_try_dissolve_container(group); - return ret; } @@ -2043,10 +2025,6 @@ static int vfio_unregister_iommu_notifier(struct vfio_group *group, struct vfio_iommu_driver *driver; int ret; - ret = vfio_group_add_container_user(group); - if (ret) - return -EINVAL; - container = group->container; driver = container->iommu_driver; if (likely(driver && driver->ops->unregister_notifier)) @@ -2054,9 +2032,6 @@ static int vfio_unregister_iommu_notifier(struct vfio_group *group, nb); else ret = -ENOTTY; - - vfio_group_try_dissolve_container(group); - return ret; } @@ -2085,10 +2060,6 @@ static int vfio_register_group_notifier(struct vfio_group *group, if (*events) return -EINVAL; - ret = vfio_group_add_container_user(group); - if (ret) - return -EINVAL; - ret = blocking_notifier_chain_register(&group->notifier, nb); /* @@ -2098,25 +2069,6 @@ static int vfio_register_group_notifier(struct vfio_group *group, if (!ret && set_kvm && group->kvm) blocking_notifier_call_chain(&group->notifier, VFIO_GROUP_NOTIFY_SET_KVM, group->kvm); - - vfio_group_try_dissolve_container(group); - - return ret; -} - -static int vfio_unregister_group_notifier(struct vfio_group *group, - struct notifier_block *nb) -{ - int ret; - - ret = vfio_group_add_container_user(group); - if (ret) - return -EINVAL; - - ret = blocking_notifier_chain_unregister(&group->notifier, nb); - - vfio_group_try_dissolve_container(group); - return ret; } @@ -2126,7 +2078,7 @@ int vfio_register_notifier(struct vfio_device *dev, enum vfio_notify_type type, struct vfio_group *group = dev->group; int ret; - if (!nb || !events || (*events == 0)) + if (!nb || !events || (*events == 0) || !vfio_assert_device_open(dev)) return -EINVAL; switch (type) { @@ -2150,7 +2102,7 @@ int vfio_unregister_notifier(struct vfio_device *dev, struct vfio_group *group = dev->group; int ret; - if (!nb) + if (!nb || !vfio_assert_device_open(dev)) return -EINVAL; switch (type) { @@ -2158,7 +2110,7 @@ int vfio_unregister_notifier(struct vfio_device *dev, ret = vfio_unregister_iommu_notifier(group, nb); break; case VFIO_GROUP_NOTIFY: - ret = vfio_unregister_group_notifier(group, nb); + ret = blocking_notifier_chain_unregister(&group->notifier, nb); break; default: ret = -EINVAL;