From patchwork Tue Mar 9 08:33:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Gurtovoy X-Patchwork-Id: 12124381 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA7CAC43331 for ; Tue, 9 Mar 2021 08:34:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 83939652B9 for ; Tue, 9 Mar 2021 08:34:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230303AbhCIIeZ (ORCPT ); Tue, 9 Mar 2021 03:34:25 -0500 Received: from mail-co1nam11on2052.outbound.protection.outlook.com ([40.107.220.52]:59872 "EHLO NAM11-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S230112AbhCIIeO (ORCPT ); Tue, 9 Mar 2021 03:34:14 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BkxA1xutPMYvvAm+C01fTRmFmZ0LaJhuzvQGQi9+nREs7Rif5YHkKgZX8FALd05mqZ+iMyvZtrfa+1lRaawYBAiM+7pYxyR+jAxsvqKvl3HUkVbH8zvyFvVgaQkc82RXgPpXTK592IWKUgl43Vv3xH2SgZ4A1IdSoz7isrSDVwgyyiLFOOzlLlbXJs7r9U/KPlJTyRZlnK7y5SggKzkOReSsMbz9KTEv4sg5WVoLLP+mXVtl8af10cwP54afRI62yKRej7w00XqTS54EIE2PmZ8ZhVP3FFE1rfjlATfrmon+63icbaNzClzQyfHyJ6IorRdwPuCCkOlgJY0C+AYitw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VOoeO+9FPJVvbWDwQUx7tjOUVKIcCTn6cS4IiPHKSOM=; b=TeHHY+iNk8uuQPbTHRa+oRDgE9nC9mTChkDJU6JVbzpe+AUwdLRzxDso7D9nEnRAHXRnvvEcQe1rT6j029oCqP4atwUkrXZtqzdMX4uvCMEx7xwTtZoGuuD9gSg36nS1AWtxKNoBv+sThcYnAGJtZUUKL2TslN3CY0kf3bbhLEkD/WK7DVpRVSz0J7kGLtMqqHDOyfir19Uybu+GarFkFS4DtBN2/SqNRAjK5+gQDx40uSEQzhm4m1TJz4zLSlzDZSDTfRGdrEBDSuPTkCoYCrFosRPSmB03n3H4O9ubdqC8hsRQSzm2+UnB3o8x9YXsqQR8TQ2QFNM+ZHGSs08MLQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.36) smtp.rcpttodomain=linux.ibm.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VOoeO+9FPJVvbWDwQUx7tjOUVKIcCTn6cS4IiPHKSOM=; b=Oj3mqYNeMs8XATzu6g5ZDWrw5cGTLePbW1OR+/XdQ+sVvTayDZbSDRtouVuYM+qKi33e9vbKh91aSAbMew+fqIvZmiNfQG0AF7gB4WccLX1AlxHR6MXULFbNOHGZZF9I3oi+DKnlI38anoMkFV/fqqcLPXXnKTpIUEHjkC7wML3SjmG1zr7b9bWRdmDOof0OEh4XxvjaBXHDu6kuuSRzcHEbNMyWEGUNHYh2b3RrbFMhE1zFFw99ukJVUaQm7GVlJ/Z80jnr2azlil0kaatfJrPvQ/2FqRbfgrJowu3ryPbss9t8O5vT3UJ5FBH3UrwPa5Js1b5j+/WBZnVibJHLCg== Received: from DM3PR12CA0110.namprd12.prod.outlook.com (2603:10b6:0:55::30) by CY4PR12MB1301.namprd12.prod.outlook.com (2603:10b6:903:3e::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.23; Tue, 9 Mar 2021 08:34:11 +0000 Received: from DM6NAM11FT021.eop-nam11.prod.protection.outlook.com (2603:10b6:0:55:cafe::ac) by DM3PR12CA0110.outlook.office365.com (2603:10b6:0:55::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.17 via Frontend Transport; Tue, 9 Mar 2021 08:34:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.36) smtp.mailfrom=nvidia.com; linux.ibm.com; dkim=none (message not signed) header.d=none;linux.ibm.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.36 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.36; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.36) by DM6NAM11FT021.mail.protection.outlook.com (10.13.173.76) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.3912.17 via Frontend Transport; Tue, 9 Mar 2021 08:34:10 +0000 Received: from HQMAIL111.nvidia.com (172.20.187.18) by HQMAIL101.nvidia.com (172.20.187.10) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Mar 2021 08:34:09 +0000 Received: from r-nvmx02.mtr.labs.mlnx (172.20.145.6) by mail.nvidia.com (172.20.187.18) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 9 Mar 2021 08:34:04 +0000 From: Max Gurtovoy To: , , , , CC: , , , , , , , , , , , , , , , Max Gurtovoy Subject: [PATCH 1/9] vfio-pci: rename vfio_pci.c to vfio_pci_core.c Date: Tue, 9 Mar 2021 08:33:49 +0000 Message-ID: <20210309083357.65467-2-mgurtovoy@nvidia.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20210309083357.65467-1-mgurtovoy@nvidia.com> References: <20210309083357.65467-1-mgurtovoy@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 80edac84-688a-415e-9864-08d8e2d61cd9 X-MS-TrafficTypeDiagnostic: CY4PR12MB1301: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4714; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rxTFvQ/Z2w6ld0uo4j909wjLtUxroh3fRivum682i1tAIVBy1mW8eD9/J3XXg2EBC5XakkAeqU/79WSGOsrfnCGC6gIXNmAxuNONyTVy5r2nP3NtPbcF20+HRVI8+I88FL8q2OyxVcM/NKMH7/fpqgsSG2b5wf36E2CCnduJZOBo2sITzz+M0orMrA4a0vDHGQO+MZWe11BT+qSZ1Uk6FrDXhABX6NkZQ4d317o18Ju+KFH9fUQYFxE55ejSSUfJEz4bnHxErujlwLoP80J5vZ+loVDOG8P3VTZCzfSzI4/RhVZn1ImzMDl352zaXiTr39vZFmcvHRmzJ8PFC2G3fm5fS7nbevXY4M/KBAuXFGeDSkVkRduHfx65fH/UhkaUkbyoYThfAf1hWTwap6CmUNBO//YtAXeeZPtgKjkE6uboHaHUt4qKWW+c5BXZWnTyBnav+owKwVCBV6hEti5cQSVpiy0vPd7Lw/mg69aWM+0Tpd0UdFpZGtZI/g2S8TUbvI5T18FJDYV3tQBKG8sdsWr0RafOepBNCHVEYn7PAg4wuANPuW7I9jjg+6dRXLkpCIckFyHAjW9kfFP9/soH2QmYKGqSSpSV4RBZKP+BEZiO7aL+Rr2miCCt8T/L2BCvJGW7Uf22CAet6frJvM2kjh7YHdosIi5qGVHXelhiVSsasArUCmOpOwm6byw9MSC7 X-Forefront-Antispam-Report: CIP:216.228.112.36;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid05.nvidia.com;CAT:NONE;SFS:(4636009)(39860400002)(396003)(136003)(376002)(346002)(46966006)(36840700001)(47076005)(7636003)(2906002)(107886003)(6666004)(36756003)(478600001)(82740400003)(8676002)(83380400001)(8936002)(26005)(336012)(356005)(186003)(2616005)(54906003)(110136005)(34020700004)(4326008)(86362001)(426003)(5660300002)(1076003)(70586007)(70206006)(82310400003)(36860700001)(316002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Mar 2021 08:34:10.6636 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 80edac84-688a-415e-9864-08d8e2d61cd9 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.36];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT021.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1301 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This is a preparation patch for separating the vfio_pci driver to a subsystem driver and a generic pci driver. This patch doesn't change any logic. Signed-off-by: Max Gurtovoy --- drivers/vfio/pci/Makefile | 2 +- drivers/vfio/pci/{vfio_pci.c => vfio_pci_core.c} | 0 2 files changed, 1 insertion(+), 1 deletion(-) rename drivers/vfio/pci/{vfio_pci.c => vfio_pci_core.c} (100%) diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile index eff97a7cd9f1..bbf8d7c8fc45 100644 --- a/drivers/vfio/pci/Makefile +++ b/drivers/vfio/pci/Makefile @@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0-only -vfio-pci-y := vfio_pci.o vfio_pci_intrs.o vfio_pci_rdwr.o vfio_pci_config.o +vfio-pci-y := vfio_pci_core.o vfio_pci_intrs.o vfio_pci_rdwr.o vfio_pci_config.o vfio-pci-$(CONFIG_VFIO_PCI_IGD) += vfio_pci_igd.o vfio-pci-$(CONFIG_VFIO_PCI_NVLINK2) += vfio_pci_nvlink2.o vfio-pci-$(CONFIG_S390) += vfio_pci_zdev.o diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci_core.c similarity index 100% rename from drivers/vfio/pci/vfio_pci.c rename to drivers/vfio/pci/vfio_pci_core.c From patchwork Tue Mar 9 08:33:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Gurtovoy X-Patchwork-Id: 12124383 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5781C433DB for ; Tue, 9 Mar 2021 08:35:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A310965295 for ; Tue, 9 Mar 2021 08:35:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230285AbhCIIeu (ORCPT ); Tue, 9 Mar 2021 03:34:50 -0500 Received: from mail-bn8nam11on2069.outbound.protection.outlook.com ([40.107.236.69]:42879 "EHLO NAM11-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S230134AbhCIIeS (ORCPT ); Tue, 9 Mar 2021 03:34:18 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=DhUkMXaPWFJ/aZE1BhDRC4eDuA5haklIyMTwj5oDH9ctHi+bBDTyVBth2SOiykcqaWt6f7noyRHTZ1mL4cvUdXKgtq8Gdo/IA9DggA6eEaNJP7ZaWjKrYxM+hthvPm8rn8G0IkVivjXeNYXPDRV82qfFM/StC9juIU7138DruVcWw+qsIPRNHAovFsdQsPVV0IZuZeHoT51KwfdN63XZgAuPuHW4NgM9UZZjVxmYldSs7NbUJRIkma//76dKEzn7hsuoAXfIvzyuweTgr/3Okk5mpPb7v3jh8u4BGgiwhkav6/We/GtTUpJgROjXTdBAFR9lpAsr+6arvle8uHFZ4w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C5LYrJahpjqSwVw1fv3zp9zwi0fj1XXWGqQlvpTGALo=; b=YgHuQDSndnCEPA9CUs0z3VOfqfIhJO+N3QTYdTvDi0lWOWHFehNqOhe18/5ERdZpFxjdfabQE/VlB3A+hKHxyZdfqN7ruMBzbYL4wiVjldySrw4deaE1s3PM98yu1mmFeQXDosoVEFMqAwjjgDH1GJTy9CfmxcHtDySvgrzPooryDB8fszB1ho6g1cmXBn1XHwmyAyT3zB6UXKU7spebDwW5Itf2ngtK73yQjtiJxFsZzYJ0aPY+o312r7fHCcu+a9PQj66G4K5XQte++hIiNfu8JV/DrXgGszMTgRElQABS70qcpSBoBQmzViS3eOnsclVSBaxhDp+2H39slrHvOg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.36) smtp.rcpttodomain=linux.ibm.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C5LYrJahpjqSwVw1fv3zp9zwi0fj1XXWGqQlvpTGALo=; b=RG4ormKCQvVqrWNuw7r3Qv09Vt6UuM0zFsTD/qmfi2MIkt3juEm0SXJNvzfdQek17GpJ+EFTm8kSrWFG/f1oHBZ9ZsfXlr88zj/b0YrthjdBG73D94Dvdzgvuj+ehfIa+k87T2tDbYowH8+Rs7NpegY/zuEYZm2Q6Mns9t+6Tcj4v5wLC7MUbUYh9uHPmURn8UNV67LPxf51Q4meBfitdWmQrHTshiye3uPIk+IfQMWtst8QEaT0CsedNDpMDaZF3FlnfXWLishk+LlADbGW/DEjKpiAftd9uQ/MHVslssa9meyMsFjhbrUCbrpYEmv74G3oY9WpAm3Iqjt9J6Ur3Q== Received: from DS7PR03CA0183.namprd03.prod.outlook.com (2603:10b6:5:3b6::8) by DM5PR12MB1642.namprd12.prod.outlook.com (2603:10b6:4:7::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.19; Tue, 9 Mar 2021 08:34:17 +0000 Received: from DM6NAM11FT064.eop-nam11.prod.protection.outlook.com (2603:10b6:5:3b6:cafe::e1) by DS7PR03CA0183.outlook.office365.com (2603:10b6:5:3b6::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.17 via Frontend Transport; Tue, 9 Mar 2021 08:34:17 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.36) smtp.mailfrom=nvidia.com; linux.ibm.com; dkim=none (message not signed) header.d=none;linux.ibm.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.36 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.36; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.36) by DM6NAM11FT064.mail.protection.outlook.com (10.13.172.234) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.3912.17 via Frontend Transport; Tue, 9 Mar 2021 08:34:16 +0000 Received: from HQMAIL111.nvidia.com (172.20.187.18) by HQMAIL101.nvidia.com (172.20.187.10) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Mar 2021 08:34:15 +0000 Received: from r-nvmx02.mtr.labs.mlnx (172.20.145.6) by mail.nvidia.com (172.20.187.18) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 9 Mar 2021 08:34:10 +0000 From: Max Gurtovoy To: , , , , CC: , , , , , , , , , , , , , , , Max Gurtovoy Subject: [PATCH 2/9] vfio-pci: rename vfio_pci_private.h to vfio_pci_core.h Date: Tue, 9 Mar 2021 08:33:50 +0000 Message-ID: <20210309083357.65467-3-mgurtovoy@nvidia.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20210309083357.65467-1-mgurtovoy@nvidia.com> References: <20210309083357.65467-1-mgurtovoy@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 74cf09a9-430f-4aa8-964e-08d8e2d62065 X-MS-TrafficTypeDiagnostic: DM5PR12MB1642: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4125; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FI189oxNfwx3dsgtdOQeiQ7gorOuSpVm0vJZS5G6zw6iYe9eu6AHMcpK9VEsQ2hLVuR6ngdlkf3iUXutHrcuqNDKQp7oK45h/mm48Qc/a3jfRJb0mk1up1lY5JWDvwEB6/IARoXdTcJp36mxmXLSpy7wSNH40CIxEPYTb8tdRap6G1p5VmkrHIZbCnr21cltM9zScOAUkqtcmc2NMBT2XiNMYqbzIvg5UhkC2aCIQ2jFym7ZMHazt72aleEe+OwG/Leha6LLzjKMiaZ8FJEZ5IcUFBBaqe1OP45aAD7M446h5j+7Te/jxkxYpmf0SVhqEDARODBvprrWEJbmLafyvt1KAqelfjIOJJBRcPrxMBZBqTAGK/FiC/Za9qk9MYZhHT5vxzlCPgb0qLeit1Go38vVZWjinDOVIh34VX+vb7v5hm6kbyVv1+aPmpYJtO9QuWpv2U/jz6QPzwtiRNKngFWTy5fKkxXmhHiKx7McPo2EwR51gylP7ZexvcM7DOaFn1W7zinrzmNf+XxlPeXy2qunzx3vjfMD4TFzNHED2oOnOeFekeWT8/K8vnY4zFOPZHMo1u8tUm97vTEBsLgxXWdEi/Xn+5P0PnxWj4vxW9TuU+Yz7AfJab5QyIeAWcPCh29WmizUnWLAzMWkRJ7V2SyAFRThnn1NXzXqpuoFEcP7ZWUdJ9IqqA2T9oO/2QYm X-Forefront-Antispam-Report: CIP:216.228.112.36;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid05.nvidia.com;CAT:NONE;SFS:(4636009)(376002)(136003)(346002)(396003)(39860400002)(36840700001)(46966006)(2616005)(107886003)(36756003)(5660300002)(82310400003)(186003)(4326008)(70206006)(70586007)(47076005)(8936002)(2906002)(7636003)(83380400001)(8676002)(316002)(478600001)(336012)(110136005)(356005)(86362001)(6666004)(26005)(36860700001)(54906003)(34020700004)(1076003)(82740400003)(426003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Mar 2021 08:34:16.6213 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 74cf09a9-430f-4aa8-964e-08d8e2d62065 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.36];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT064.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1642 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This is a preparation patch for separating the vfio_pci driver to a subsystem driver and a generic pci driver. This patch doesn't change any logic. Signed-off-by: Max Gurtovoy --- drivers/vfio/pci/vfio_pci_config.c | 2 +- drivers/vfio/pci/vfio_pci_core.c | 2 +- drivers/vfio/pci/{vfio_pci_private.h => vfio_pci_core.h} | 6 +++--- drivers/vfio/pci/vfio_pci_igd.c | 2 +- drivers/vfio/pci/vfio_pci_intrs.c | 2 +- drivers/vfio/pci/vfio_pci_nvlink2.c | 2 +- drivers/vfio/pci/vfio_pci_rdwr.c | 2 +- drivers/vfio/pci/vfio_pci_zdev.c | 2 +- 8 files changed, 10 insertions(+), 10 deletions(-) rename drivers/vfio/pci/{vfio_pci_private.h => vfio_pci_core.h} (98%) diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c index a402adee8a21..5e9d24992207 100644 --- a/drivers/vfio/pci/vfio_pci_config.c +++ b/drivers/vfio/pci/vfio_pci_config.c @@ -26,7 +26,7 @@ #include #include -#include "vfio_pci_private.h" +#include "vfio_pci_core.h" /* Fake capability ID for standard config space */ #define PCI_CAP_ID_BASIC 0 diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index 65e7e6b44578..bd587db04625 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -28,7 +28,7 @@ #include #include -#include "vfio_pci_private.h" +#include "vfio_pci_core.h" #define DRIVER_VERSION "0.2" #define DRIVER_AUTHOR "Alex Williamson " diff --git a/drivers/vfio/pci/vfio_pci_private.h b/drivers/vfio/pci/vfio_pci_core.h similarity index 98% rename from drivers/vfio/pci/vfio_pci_private.h rename to drivers/vfio/pci/vfio_pci_core.h index 9cd1882a05af..b9ac7132b84a 100644 --- a/drivers/vfio/pci/vfio_pci_private.h +++ b/drivers/vfio/pci/vfio_pci_core.h @@ -15,8 +15,8 @@ #include #include -#ifndef VFIO_PCI_PRIVATE_H -#define VFIO_PCI_PRIVATE_H +#ifndef VFIO_PCI_CORE_H +#define VFIO_PCI_CORE_H #define VFIO_PCI_OFFSET_SHIFT 40 @@ -225,4 +225,4 @@ static inline int vfio_pci_info_zdev_add_caps(struct vfio_pci_device *vdev, } #endif -#endif /* VFIO_PCI_PRIVATE_H */ +#endif /* VFIO_PCI_CORE_H */ diff --git a/drivers/vfio/pci/vfio_pci_igd.c b/drivers/vfio/pci/vfio_pci_igd.c index 53d97f459252..0c599cd33d01 100644 --- a/drivers/vfio/pci/vfio_pci_igd.c +++ b/drivers/vfio/pci/vfio_pci_igd.c @@ -15,7 +15,7 @@ #include #include -#include "vfio_pci_private.h" +#include "vfio_pci_core.h" #define OPREGION_SIGNATURE "IntelGraphicsMem" #define OPREGION_SIZE (8 * 1024) diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index 869dce5f134d..df1e8c8c274c 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -20,7 +20,7 @@ #include #include -#include "vfio_pci_private.h" +#include "vfio_pci_core.h" /* * INTx diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c index 9adcf6a8f888..326a704c4527 100644 --- a/drivers/vfio/pci/vfio_pci_nvlink2.c +++ b/drivers/vfio/pci/vfio_pci_nvlink2.c @@ -19,7 +19,7 @@ #include #include #include -#include "vfio_pci_private.h" +#include "vfio_pci_core.h" #define CREATE_TRACE_POINTS #include "trace.h" diff --git a/drivers/vfio/pci/vfio_pci_rdwr.c b/drivers/vfio/pci/vfio_pci_rdwr.c index a0b5fc8e46f4..667e82726e75 100644 --- a/drivers/vfio/pci/vfio_pci_rdwr.c +++ b/drivers/vfio/pci/vfio_pci_rdwr.c @@ -17,7 +17,7 @@ #include #include -#include "vfio_pci_private.h" +#include "vfio_pci_core.h" #ifdef __LITTLE_ENDIAN #define vfio_ioread64 ioread64 diff --git a/drivers/vfio/pci/vfio_pci_zdev.c b/drivers/vfio/pci/vfio_pci_zdev.c index 229685634031..3e91d49fa3f0 100644 --- a/drivers/vfio/pci/vfio_pci_zdev.c +++ b/drivers/vfio/pci/vfio_pci_zdev.c @@ -19,7 +19,7 @@ #include #include -#include "vfio_pci_private.h" +#include "vfio_pci_core.h" /* * Add the Base PCI Function information to the device info region. From patchwork Tue Mar 9 08:33:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Gurtovoy X-Patchwork-Id: 12124391 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7F92C433E6 for ; Tue, 9 Mar 2021 08:35:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7CB3565295 for ; Tue, 9 Mar 2021 08:35:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230421AbhCIIew (ORCPT ); Tue, 9 Mar 2021 03:34:52 -0500 Received: from mail-bn7nam10on2089.outbound.protection.outlook.com ([40.107.92.89]:22241 "EHLO NAM10-BN7-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S230391AbhCIIe2 (ORCPT ); Tue, 9 Mar 2021 03:34:28 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cWJTK6J/jBZWsQNOhcmEgVLoxJjN1S0tkucBd78r0WBloL9n6Duo5bvIO3M+k7lLGXBPA+YWmpkI1Gmvev+8rG9SfJFSxC2NBOrsCGN9+qphhv4lNVZzz+BLdgusCo9KrrvEhsn3+HALqWIj0wnEj2xWGcW8QYHR8fU8+Y9clP23EWlyo5VsTxgUKsf/vq/qWNJWh2vuq+IgAwMgPo/Zk4brKXRiGXiYbo+KnNVsSAtAkxGVjfJVsNxhKl8c/lexjh4sJrMTOZx+W9quaWWQbGuIk4ILf7AVD6LKJpg1kQ2+g0tcFIa1Yyja/iOZhVX4a1r/QUzKmxKfkKGoJ3ZG3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vjWPBqmB7U6/noGHb2rlR42tWt1JsGf8FESaldVvzfM=; b=DaNuAM7gBMvfrYIpIO6rRzB4dsuTW/hoMiTludxir7aqqFrnlFd0xogSlo3qXdKVrKKPYukC4j8upg5Q+lmexvBCvGYleEmMk/B8a+dEwYjQU48VIHtSgo8/1J1qy8S11PtGu07u+0zNkgMTlW4BOvaJU7K9SDbywQSvx3MIQLXfLjY75HI2Ye/YvpJsTBEqEHDWuQTd+73RRdPDLsy1gYekrxB5CqHXJS6Noszk/ICSByBLvaq5Zx/OZICibGWo6xtWUw/kQGtO5PQGUQD+PJYSNtSeom0Cv03M55HAZYfxRl6boa43AHFUl6xNZgj+JFpgYDrJ5wsmo3LSK0eOSA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.36) smtp.rcpttodomain=linux.ibm.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vjWPBqmB7U6/noGHb2rlR42tWt1JsGf8FESaldVvzfM=; b=AC7fUSop7m2+3O25xvgs/olqY0srjgSn8m411St9nhydapwT3giu1hMk1xlwCW+G9mTm5WrUvozl1yk/R7YCRpAnt/s3czpGCKp03aKA4sGKyFZd0N8cOmIYBy0txpXEyG+HslQTBRsx/jrmkZmpy33JttBoc25oczE3wmf5sUzyiif9veC7odVL+ENM5TJ2QCAI+e8Su6E32YrL6nLfhXQlwfC7cBv4iXOdPljizyrEAc+9M5QmqCc+Aaj2iMM7H4HhMLXsZ1Tlp1S9PnMDmPJ0TxatY1JSElGKZ1/hHgZcFJBuDngJgWC6nI3CX+StD3JqoPLnotwwl81xvmtWzg== Received: from DM5PR12CA0018.namprd12.prod.outlook.com (2603:10b6:4:1::28) by BL0PR12MB2370.namprd12.prod.outlook.com (2603:10b6:207:47::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.18; Tue, 9 Mar 2021 08:34:23 +0000 Received: from DM6NAM11FT050.eop-nam11.prod.protection.outlook.com (2603:10b6:4:1:cafe::d3) by DM5PR12CA0018.outlook.office365.com (2603:10b6:4:1::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.17 via Frontend Transport; Tue, 9 Mar 2021 08:34:23 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.36) smtp.mailfrom=nvidia.com; linux.ibm.com; dkim=none (message not signed) header.d=none;linux.ibm.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.36 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.36; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.36) by DM6NAM11FT050.mail.protection.outlook.com (10.13.173.111) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.3912.17 via Frontend Transport; Tue, 9 Mar 2021 08:34:22 +0000 Received: from HQMAIL111.nvidia.com (172.20.187.18) by HQMAIL101.nvidia.com (172.20.187.10) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Mar 2021 08:34:21 +0000 Received: from r-nvmx02.mtr.labs.mlnx (172.20.145.6) by mail.nvidia.com (172.20.187.18) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 9 Mar 2021 08:34:16 +0000 From: Max Gurtovoy To: , , , , CC: , , , , , , , , , , , , , , , Max Gurtovoy Subject: [PATCH 3/9] vfio-pci: rename vfio_pci_device to vfio_pci_core_device Date: Tue, 9 Mar 2021 08:33:51 +0000 Message-ID: <20210309083357.65467-4-mgurtovoy@nvidia.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20210309083357.65467-1-mgurtovoy@nvidia.com> References: <20210309083357.65467-1-mgurtovoy@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bd32584f-efeb-4161-23aa-08d8e2d62402 X-MS-TrafficTypeDiagnostic: BL0PR12MB2370: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7jOurUXdMFNMCqZaigNy21q4WD6N897L9rwYhcuAnh7aMv3V09e/TL/29Ii8xHDmOLtDvhJAvOV5LyIgZHCUj7seC9Fm5C99upIYOxV6qgUEQMmgn/kfAef/jnaFROsWayNqOy07JRhIkQR9itqJwQMTwXzgkZnqjAn35GdaNSHH4qDuUzAhIknTp7hhMhvf8TjRlw+cOWCEuhqbs9x/4S5eR661JjKvVWY5Ol6DG/thqYiO1+Kk/iMG6Z96dEADF8cCWLrYr2CcD/Ql6ViTx8nxGhu+/K52ek2ffcSmrHORJIuFKvj9YB6/9GUL/ra/metD4NWedZJjdGqBc688d5yWv906MQG52V+QfpquVDlWj5b1UQboF3kKuBSQW+vz1cR9MPQ9/7jhKXaziV7y8GJmZxgPctb6IWTMj4F+V+5OKxWDMNYcFF4V3QXutznXARUlisknd1ZkuF6eH3+STeWG7rciNz1SE08DAFzM9ZK94EaZBaV26RlAVcLMPMQV4JTLWjVtcRHDmSOJvfMxpMpil4R3wwW0O1VJav3Fgxz3tvVqpbw0xjVObdTYQUNFSUQBDq95wPsacRGh1+jfgkBnxVRkW9iLI2DVnoK9xHBGDLIO0UuUHa7dnivxkuL8UWkjFh+XFcflVA8GSCDbi7dHK2ZZIo4TxOlpcdtMW1sCNOPGdVcHNW25tATRkSfR X-Forefront-Antispam-Report: CIP:216.228.112.36;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid05.nvidia.com;CAT:NONE;SFS:(4636009)(346002)(39860400002)(136003)(376002)(396003)(36840700001)(46966006)(356005)(36860700001)(2616005)(6666004)(54906003)(82740400003)(26005)(2906002)(7636003)(316002)(186003)(478600001)(47076005)(426003)(82310400003)(34020700004)(70206006)(110136005)(107886003)(4326008)(1076003)(30864003)(336012)(5660300002)(36756003)(86362001)(8936002)(8676002)(70586007)(83380400001)(579004)(559001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Mar 2021 08:34:22.6424 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: bd32584f-efeb-4161-23aa-08d8e2d62402 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.36];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT050.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB2370 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This is a preparation patch for separating the vfio_pci driver to a subsystem driver and a generic pci driver. This patch doesn't change any logic. The new vfio_pci_core_device structure will be the main structure of the core driver and later on vfio_pci_device structure will be the main structure of the generic vfio_pci driver. Signed-off-by: Max Gurtovoy --- drivers/vfio/pci/vfio_pci_config.c | 68 +++++++++++----------- drivers/vfio/pci/vfio_pci_core.c | 90 ++++++++++++++--------------- drivers/vfio/pci/vfio_pci_core.h | 76 ++++++++++++------------ drivers/vfio/pci/vfio_pci_igd.c | 14 ++--- drivers/vfio/pci/vfio_pci_intrs.c | 40 ++++++------- drivers/vfio/pci/vfio_pci_nvlink2.c | 20 +++---- drivers/vfio/pci/vfio_pci_rdwr.c | 16 ++--- drivers/vfio/pci/vfio_pci_zdev.c | 10 ++-- 8 files changed, 167 insertions(+), 167 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_config.c b/drivers/vfio/pci/vfio_pci_config.c index 5e9d24992207..122918d0f733 100644 --- a/drivers/vfio/pci/vfio_pci_config.c +++ b/drivers/vfio/pci/vfio_pci_config.c @@ -108,9 +108,9 @@ static const u16 pci_ext_cap_length[PCI_EXT_CAP_ID_MAX + 1] = { struct perm_bits { u8 *virt; /* read/write virtual data, not hw */ u8 *write; /* writeable bits */ - int (*readfn)(struct vfio_pci_device *vdev, int pos, int count, + int (*readfn)(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 *val); - int (*writefn)(struct vfio_pci_device *vdev, int pos, int count, + int (*writefn)(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 val); }; @@ -171,7 +171,7 @@ static int vfio_user_config_write(struct pci_dev *pdev, int offset, return ret; } -static int vfio_default_config_read(struct vfio_pci_device *vdev, int pos, +static int vfio_default_config_read(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 *val) { @@ -197,7 +197,7 @@ static int vfio_default_config_read(struct vfio_pci_device *vdev, int pos, return count; } -static int vfio_default_config_write(struct vfio_pci_device *vdev, int pos, +static int vfio_default_config_write(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 val) { @@ -244,7 +244,7 @@ static int vfio_default_config_write(struct vfio_pci_device *vdev, int pos, } /* Allow direct read from hardware, except for capability next pointer */ -static int vfio_direct_config_read(struct vfio_pci_device *vdev, int pos, +static int vfio_direct_config_read(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 *val) { @@ -269,7 +269,7 @@ static int vfio_direct_config_read(struct vfio_pci_device *vdev, int pos, } /* Raw access skips any kind of virtualization */ -static int vfio_raw_config_write(struct vfio_pci_device *vdev, int pos, +static int vfio_raw_config_write(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 val) { @@ -282,7 +282,7 @@ static int vfio_raw_config_write(struct vfio_pci_device *vdev, int pos, return count; } -static int vfio_raw_config_read(struct vfio_pci_device *vdev, int pos, +static int vfio_raw_config_read(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 *val) { @@ -296,7 +296,7 @@ static int vfio_raw_config_read(struct vfio_pci_device *vdev, int pos, } /* Virt access uses only virtualization */ -static int vfio_virt_config_write(struct vfio_pci_device *vdev, int pos, +static int vfio_virt_config_write(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 val) { @@ -304,7 +304,7 @@ static int vfio_virt_config_write(struct vfio_pci_device *vdev, int pos, return count; } -static int vfio_virt_config_read(struct vfio_pci_device *vdev, int pos, +static int vfio_virt_config_read(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 *val) { @@ -396,7 +396,7 @@ static inline void p_setd(struct perm_bits *p, int off, u32 virt, u32 write) } /* Caller should hold memory_lock semaphore */ -bool __vfio_pci_memory_enabled(struct vfio_pci_device *vdev) +bool __vfio_pci_memory_enabled(struct vfio_pci_core_device *vdev) { struct pci_dev *pdev = vdev->pdev; u16 cmd = le16_to_cpu(*(__le16 *)&vdev->vconfig[PCI_COMMAND]); @@ -413,7 +413,7 @@ bool __vfio_pci_memory_enabled(struct vfio_pci_device *vdev) * Restore the *real* BARs after we detect a FLR or backdoor reset. * (backdoor = some device specific technique that we didn't catch) */ -static void vfio_bar_restore(struct vfio_pci_device *vdev) +static void vfio_bar_restore(struct vfio_pci_core_device *vdev) { struct pci_dev *pdev = vdev->pdev; u32 *rbar = vdev->rbar; @@ -460,7 +460,7 @@ static __le32 vfio_generate_bar_flags(struct pci_dev *pdev, int bar) * Pretend we're hardware and tweak the values of the *virtual* PCI BARs * to reflect the hardware capabilities. This implements BAR sizing. */ -static void vfio_bar_fixup(struct vfio_pci_device *vdev) +static void vfio_bar_fixup(struct vfio_pci_core_device *vdev) { struct pci_dev *pdev = vdev->pdev; int i; @@ -514,7 +514,7 @@ static void vfio_bar_fixup(struct vfio_pci_device *vdev) vdev->bardirty = false; } -static int vfio_basic_config_read(struct vfio_pci_device *vdev, int pos, +static int vfio_basic_config_read(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 *val) { @@ -536,7 +536,7 @@ static int vfio_basic_config_read(struct vfio_pci_device *vdev, int pos, } /* Test whether BARs match the value we think they should contain */ -static bool vfio_need_bar_restore(struct vfio_pci_device *vdev) +static bool vfio_need_bar_restore(struct vfio_pci_core_device *vdev) { int i = 0, pos = PCI_BASE_ADDRESS_0, ret; u32 bar; @@ -552,7 +552,7 @@ static bool vfio_need_bar_restore(struct vfio_pci_device *vdev) return false; } -static int vfio_basic_config_write(struct vfio_pci_device *vdev, int pos, +static int vfio_basic_config_write(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 val) { @@ -692,7 +692,7 @@ static int __init init_pci_cap_basic_perm(struct perm_bits *perm) return 0; } -static int vfio_pm_config_write(struct vfio_pci_device *vdev, int pos, +static int vfio_pm_config_write(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 val) { @@ -747,7 +747,7 @@ static int __init init_pci_cap_pm_perm(struct perm_bits *perm) return 0; } -static int vfio_vpd_config_write(struct vfio_pci_device *vdev, int pos, +static int vfio_vpd_config_write(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 val) { @@ -829,7 +829,7 @@ static int __init init_pci_cap_pcix_perm(struct perm_bits *perm) return 0; } -static int vfio_exp_config_write(struct vfio_pci_device *vdev, int pos, +static int vfio_exp_config_write(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 val) { @@ -913,7 +913,7 @@ static int __init init_pci_cap_exp_perm(struct perm_bits *perm) return 0; } -static int vfio_af_config_write(struct vfio_pci_device *vdev, int pos, +static int vfio_af_config_write(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 val) { @@ -1072,7 +1072,7 @@ int __init vfio_pci_init_perm_bits(void) return ret; } -static int vfio_find_cap_start(struct vfio_pci_device *vdev, int pos) +static int vfio_find_cap_start(struct vfio_pci_core_device *vdev, int pos) { u8 cap; int base = (pos >= PCI_CFG_SPACE_SIZE) ? PCI_CFG_SPACE_SIZE : @@ -1089,7 +1089,7 @@ static int vfio_find_cap_start(struct vfio_pci_device *vdev, int pos) return pos; } -static int vfio_msi_config_read(struct vfio_pci_device *vdev, int pos, +static int vfio_msi_config_read(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 *val) { @@ -1109,7 +1109,7 @@ static int vfio_msi_config_read(struct vfio_pci_device *vdev, int pos, return vfio_default_config_read(vdev, pos, count, perm, offset, val); } -static int vfio_msi_config_write(struct vfio_pci_device *vdev, int pos, +static int vfio_msi_config_write(struct vfio_pci_core_device *vdev, int pos, int count, struct perm_bits *perm, int offset, __le32 val) { @@ -1189,7 +1189,7 @@ static int init_pci_cap_msi_perm(struct perm_bits *perm, int len, u16 flags) } /* Determine MSI CAP field length; initialize msi_perms on 1st call per vdev */ -static int vfio_msi_cap_len(struct vfio_pci_device *vdev, u8 pos) +static int vfio_msi_cap_len(struct vfio_pci_core_device *vdev, u8 pos) { struct pci_dev *pdev = vdev->pdev; int len, ret; @@ -1222,7 +1222,7 @@ static int vfio_msi_cap_len(struct vfio_pci_device *vdev, u8 pos) } /* Determine extended capability length for VC (2 & 9) and MFVC */ -static int vfio_vc_cap_len(struct vfio_pci_device *vdev, u16 pos) +static int vfio_vc_cap_len(struct vfio_pci_core_device *vdev, u16 pos) { struct pci_dev *pdev = vdev->pdev; u32 tmp; @@ -1263,7 +1263,7 @@ static int vfio_vc_cap_len(struct vfio_pci_device *vdev, u16 pos) return len; } -static int vfio_cap_len(struct vfio_pci_device *vdev, u8 cap, u8 pos) +static int vfio_cap_len(struct vfio_pci_core_device *vdev, u8 cap, u8 pos) { struct pci_dev *pdev = vdev->pdev; u32 dword; @@ -1338,7 +1338,7 @@ static int vfio_cap_len(struct vfio_pci_device *vdev, u8 cap, u8 pos) return 0; } -static int vfio_ext_cap_len(struct vfio_pci_device *vdev, u16 ecap, u16 epos) +static int vfio_ext_cap_len(struct vfio_pci_core_device *vdev, u16 ecap, u16 epos) { struct pci_dev *pdev = vdev->pdev; u8 byte; @@ -1412,7 +1412,7 @@ static int vfio_ext_cap_len(struct vfio_pci_device *vdev, u16 ecap, u16 epos) return 0; } -static int vfio_fill_vconfig_bytes(struct vfio_pci_device *vdev, +static int vfio_fill_vconfig_bytes(struct vfio_pci_core_device *vdev, int offset, int size) { struct pci_dev *pdev = vdev->pdev; @@ -1459,7 +1459,7 @@ static int vfio_fill_vconfig_bytes(struct vfio_pci_device *vdev, return ret; } -static int vfio_cap_init(struct vfio_pci_device *vdev) +static int vfio_cap_init(struct vfio_pci_core_device *vdev) { struct pci_dev *pdev = vdev->pdev; u8 *map = vdev->pci_config_map; @@ -1549,7 +1549,7 @@ static int vfio_cap_init(struct vfio_pci_device *vdev) return 0; } -static int vfio_ecap_init(struct vfio_pci_device *vdev) +static int vfio_ecap_init(struct vfio_pci_core_device *vdev) { struct pci_dev *pdev = vdev->pdev; u8 *map = vdev->pci_config_map; @@ -1669,7 +1669,7 @@ static const struct pci_device_id known_bogus_vf_intx_pin[] = { * for each area requiring emulated bits, but the array of pointers * would be comparable in size (at least for standard config space). */ -int vfio_config_init(struct vfio_pci_device *vdev) +int vfio_config_init(struct vfio_pci_core_device *vdev) { struct pci_dev *pdev = vdev->pdev; u8 *map, *vconfig; @@ -1773,7 +1773,7 @@ int vfio_config_init(struct vfio_pci_device *vdev) return pcibios_err_to_errno(ret); } -void vfio_config_free(struct vfio_pci_device *vdev) +void vfio_config_free(struct vfio_pci_core_device *vdev) { kfree(vdev->vconfig); vdev->vconfig = NULL; @@ -1790,7 +1790,7 @@ void vfio_config_free(struct vfio_pci_device *vdev) * Find the remaining number of bytes in a dword that match the given * position. Stop at either the end of the capability or the dword boundary. */ -static size_t vfio_pci_cap_remaining_dword(struct vfio_pci_device *vdev, +static size_t vfio_pci_cap_remaining_dword(struct vfio_pci_core_device *vdev, loff_t pos) { u8 cap = vdev->pci_config_map[pos]; @@ -1802,7 +1802,7 @@ static size_t vfio_pci_cap_remaining_dword(struct vfio_pci_device *vdev, return i; } -static ssize_t vfio_config_do_rw(struct vfio_pci_device *vdev, char __user *buf, +static ssize_t vfio_config_do_rw(struct vfio_pci_core_device *vdev, char __user *buf, size_t count, loff_t *ppos, bool iswrite) { struct pci_dev *pdev = vdev->pdev; @@ -1885,7 +1885,7 @@ static ssize_t vfio_config_do_rw(struct vfio_pci_device *vdev, char __user *buf, return ret; } -ssize_t vfio_pci_config_rw(struct vfio_pci_device *vdev, char __user *buf, +ssize_t vfio_pci_config_rw(struct vfio_pci_core_device *vdev, char __user *buf, size_t count, loff_t *ppos, bool iswrite) { size_t done = 0; diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index bd587db04625..557a03528dcd 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -121,7 +121,7 @@ static bool vfio_pci_is_denylisted(struct pci_dev *pdev) */ static unsigned int vfio_pci_set_vga_decode(void *opaque, bool single_vga) { - struct vfio_pci_device *vdev = opaque; + struct vfio_pci_core_device *vdev = opaque; struct pci_dev *tmp = NULL, *pdev = vdev->pdev; unsigned char max_busnr; unsigned int decodes; @@ -155,7 +155,7 @@ static inline bool vfio_pci_is_vga(struct pci_dev *pdev) return (pdev->class >> 8) == PCI_CLASS_DISPLAY_VGA; } -static void vfio_pci_probe_mmaps(struct vfio_pci_device *vdev) +static void vfio_pci_probe_mmaps(struct vfio_pci_core_device *vdev) { struct resource *res; int i; @@ -223,8 +223,8 @@ static void vfio_pci_probe_mmaps(struct vfio_pci_device *vdev) } } -static void vfio_pci_try_bus_reset(struct vfio_pci_device *vdev); -static void vfio_pci_disable(struct vfio_pci_device *vdev); +static void vfio_pci_try_bus_reset(struct vfio_pci_core_device *vdev); +static void vfio_pci_disable(struct vfio_pci_core_device *vdev); static int vfio_pci_try_zap_and_vma_lock_cb(struct pci_dev *pdev, void *data); /* @@ -258,7 +258,7 @@ static bool vfio_pci_nointx(struct pci_dev *pdev) return false; } -static void vfio_pci_probe_power_state(struct vfio_pci_device *vdev) +static void vfio_pci_probe_power_state(struct vfio_pci_core_device *vdev) { struct pci_dev *pdev = vdev->pdev; u16 pmcsr; @@ -278,7 +278,7 @@ static void vfio_pci_probe_power_state(struct vfio_pci_device *vdev) * by PM capability emulation and separately from pci_dev internal saved state * to avoid it being overwritten and consumed around other resets. */ -int vfio_pci_set_power_state(struct vfio_pci_device *vdev, pci_power_t state) +int vfio_pci_set_power_state(struct vfio_pci_core_device *vdev, pci_power_t state) { struct pci_dev *pdev = vdev->pdev; bool needs_restore = false, needs_save = false; @@ -309,7 +309,7 @@ int vfio_pci_set_power_state(struct vfio_pci_device *vdev, pci_power_t state) return ret; } -static int vfio_pci_enable(struct vfio_pci_device *vdev) +static int vfio_pci_enable(struct vfio_pci_core_device *vdev) { struct pci_dev *pdev = vdev->pdev; int ret; @@ -416,7 +416,7 @@ static int vfio_pci_enable(struct vfio_pci_device *vdev) return ret; } -static void vfio_pci_disable(struct vfio_pci_device *vdev) +static void vfio_pci_disable(struct vfio_pci_core_device *vdev) { struct pci_dev *pdev = vdev->pdev; struct vfio_pci_dummy_resource *dummy_res, *tmp; @@ -517,7 +517,7 @@ static void vfio_pci_disable(struct vfio_pci_device *vdev) static struct pci_driver vfio_pci_driver; -static struct vfio_pci_device *get_pf_vdev(struct vfio_pci_device *vdev, +static struct vfio_pci_core_device *get_pf_vdev(struct vfio_pci_core_device *vdev, struct vfio_device **pf_dev) { struct pci_dev *physfn = pci_physfn(vdev->pdev); @@ -537,10 +537,10 @@ static struct vfio_pci_device *get_pf_vdev(struct vfio_pci_device *vdev, return vfio_device_data(*pf_dev); } -static void vfio_pci_vf_token_user_add(struct vfio_pci_device *vdev, int val) +static void vfio_pci_vf_token_user_add(struct vfio_pci_core_device *vdev, int val) { struct vfio_device *pf_dev; - struct vfio_pci_device *pf_vdev = get_pf_vdev(vdev, &pf_dev); + struct vfio_pci_core_device *pf_vdev = get_pf_vdev(vdev, &pf_dev); if (!pf_vdev) return; @@ -555,7 +555,7 @@ static void vfio_pci_vf_token_user_add(struct vfio_pci_device *vdev, int val) static void vfio_pci_release(void *device_data) { - struct vfio_pci_device *vdev = device_data; + struct vfio_pci_core_device *vdev = device_data; mutex_lock(&vdev->reflck->lock); @@ -583,7 +583,7 @@ static void vfio_pci_release(void *device_data) static int vfio_pci_open(void *device_data) { - struct vfio_pci_device *vdev = device_data; + struct vfio_pci_core_device *vdev = device_data; int ret = 0; if (!try_module_get(THIS_MODULE)) @@ -607,7 +607,7 @@ static int vfio_pci_open(void *device_data) return ret; } -static int vfio_pci_get_irq_count(struct vfio_pci_device *vdev, int irq_type) +static int vfio_pci_get_irq_count(struct vfio_pci_core_device *vdev, int irq_type) { if (irq_type == VFIO_PCI_INTX_IRQ_INDEX) { u8 pin; @@ -754,7 +754,7 @@ static int vfio_pci_for_each_slot_or_bus(struct pci_dev *pdev, return walk.ret; } -static int msix_mmappable_cap(struct vfio_pci_device *vdev, +static int msix_mmappable_cap(struct vfio_pci_core_device *vdev, struct vfio_info_cap *caps) { struct vfio_info_cap_header header = { @@ -765,7 +765,7 @@ static int msix_mmappable_cap(struct vfio_pci_device *vdev, return vfio_info_add_capability(caps, &header, sizeof(header)); } -int vfio_pci_register_dev_region(struct vfio_pci_device *vdev, +int vfio_pci_register_dev_region(struct vfio_pci_core_device *vdev, unsigned int type, unsigned int subtype, const struct vfio_pci_regops *ops, size_t size, u32 flags, void *data) @@ -800,7 +800,7 @@ struct vfio_devices { static long vfio_pci_ioctl(void *device_data, unsigned int cmd, unsigned long arg) { - struct vfio_pci_device *vdev = device_data; + struct vfio_pci_core_device *vdev = device_data; unsigned long minsz; if (cmd == VFIO_DEVICE_GET_INFO) { @@ -1280,7 +1280,7 @@ static long vfio_pci_ioctl(void *device_data, goto hot_reset_release; for (; mem_idx < devs.cur_index; mem_idx++) { - struct vfio_pci_device *tmp; + struct vfio_pci_core_device *tmp; tmp = vfio_device_data(devs.devices[mem_idx]); @@ -1298,7 +1298,7 @@ static long vfio_pci_ioctl(void *device_data, hot_reset_release: for (i = 0; i < devs.cur_index; i++) { struct vfio_device *device; - struct vfio_pci_device *tmp; + struct vfio_pci_core_device *tmp; device = devs.devices[i]; tmp = vfio_device_data(device); @@ -1406,7 +1406,7 @@ static ssize_t vfio_pci_rw(void *device_data, char __user *buf, size_t count, loff_t *ppos, bool iswrite) { unsigned int index = VFIO_PCI_OFFSET_TO_INDEX(*ppos); - struct vfio_pci_device *vdev = device_data; + struct vfio_pci_core_device *vdev = device_data; if (index >= VFIO_PCI_NUM_REGIONS + vdev->num_regions) return -EINVAL; @@ -1453,7 +1453,7 @@ static ssize_t vfio_pci_write(void *device_data, const char __user *buf, } /* Return 1 on zap and vma_lock acquired, 0 on contention (only with @try) */ -static int vfio_pci_zap_and_vma_lock(struct vfio_pci_device *vdev, bool try) +static int vfio_pci_zap_and_vma_lock(struct vfio_pci_core_device *vdev, bool try) { struct vfio_pci_mmap_vma *mmap_vma, *tmp; @@ -1541,14 +1541,14 @@ static int vfio_pci_zap_and_vma_lock(struct vfio_pci_device *vdev, bool try) } } -void vfio_pci_zap_and_down_write_memory_lock(struct vfio_pci_device *vdev) +void vfio_pci_zap_and_down_write_memory_lock(struct vfio_pci_core_device *vdev) { vfio_pci_zap_and_vma_lock(vdev, false); down_write(&vdev->memory_lock); mutex_unlock(&vdev->vma_lock); } -u16 vfio_pci_memory_lock_and_enable(struct vfio_pci_device *vdev) +u16 vfio_pci_memory_lock_and_enable(struct vfio_pci_core_device *vdev) { u16 cmd; @@ -1561,14 +1561,14 @@ u16 vfio_pci_memory_lock_and_enable(struct vfio_pci_device *vdev) return cmd; } -void vfio_pci_memory_unlock_and_restore(struct vfio_pci_device *vdev, u16 cmd) +void vfio_pci_memory_unlock_and_restore(struct vfio_pci_core_device *vdev, u16 cmd) { pci_write_config_word(vdev->pdev, PCI_COMMAND, cmd); up_write(&vdev->memory_lock); } /* Caller holds vma_lock */ -static int __vfio_pci_add_vma(struct vfio_pci_device *vdev, +static int __vfio_pci_add_vma(struct vfio_pci_core_device *vdev, struct vm_area_struct *vma) { struct vfio_pci_mmap_vma *mmap_vma; @@ -1594,7 +1594,7 @@ static void vfio_pci_mmap_open(struct vm_area_struct *vma) static void vfio_pci_mmap_close(struct vm_area_struct *vma) { - struct vfio_pci_device *vdev = vma->vm_private_data; + struct vfio_pci_core_device *vdev = vma->vm_private_data; struct vfio_pci_mmap_vma *mmap_vma; mutex_lock(&vdev->vma_lock); @@ -1611,7 +1611,7 @@ static void vfio_pci_mmap_close(struct vm_area_struct *vma) static vm_fault_t vfio_pci_mmap_fault(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; - struct vfio_pci_device *vdev = vma->vm_private_data; + struct vfio_pci_core_device *vdev = vma->vm_private_data; vm_fault_t ret = VM_FAULT_NOPAGE; mutex_lock(&vdev->vma_lock); @@ -1648,7 +1648,7 @@ static const struct vm_operations_struct vfio_pci_mmap_ops = { static int vfio_pci_mmap(void *device_data, struct vm_area_struct *vma) { - struct vfio_pci_device *vdev = device_data; + struct vfio_pci_core_device *vdev = device_data; struct pci_dev *pdev = vdev->pdev; unsigned int index; u64 phys_len, req_len, pgoff, req_start; @@ -1716,7 +1716,7 @@ static int vfio_pci_mmap(void *device_data, struct vm_area_struct *vma) static void vfio_pci_request(void *device_data, unsigned int count) { - struct vfio_pci_device *vdev = device_data; + struct vfio_pci_core_device *vdev = device_data; struct pci_dev *pdev = vdev->pdev; mutex_lock(&vdev->igate); @@ -1735,7 +1735,7 @@ static void vfio_pci_request(void *device_data, unsigned int count) mutex_unlock(&vdev->igate); } -static int vfio_pci_validate_vf_token(struct vfio_pci_device *vdev, +static int vfio_pci_validate_vf_token(struct vfio_pci_core_device *vdev, bool vf_token, uuid_t *uuid) { /* @@ -1768,7 +1768,7 @@ static int vfio_pci_validate_vf_token(struct vfio_pci_device *vdev, if (vdev->pdev->is_virtfn) { struct vfio_device *pf_dev; - struct vfio_pci_device *pf_vdev = get_pf_vdev(vdev, &pf_dev); + struct vfio_pci_core_device *pf_vdev = get_pf_vdev(vdev, &pf_dev); bool match; if (!pf_vdev) { @@ -1832,7 +1832,7 @@ static int vfio_pci_validate_vf_token(struct vfio_pci_device *vdev, static int vfio_pci_match(void *device_data, char *buf) { - struct vfio_pci_device *vdev = device_data; + struct vfio_pci_core_device *vdev = device_data; bool vf_token = false; uuid_t uuid; int ret; @@ -1891,14 +1891,14 @@ static const struct vfio_device_ops vfio_pci_ops = { .match = vfio_pci_match, }; -static int vfio_pci_reflck_attach(struct vfio_pci_device *vdev); +static int vfio_pci_reflck_attach(struct vfio_pci_core_device *vdev); static void vfio_pci_reflck_put(struct vfio_pci_reflck *reflck); static int vfio_pci_bus_notifier(struct notifier_block *nb, unsigned long action, void *data) { - struct vfio_pci_device *vdev = container_of(nb, - struct vfio_pci_device, nb); + struct vfio_pci_core_device *vdev = container_of(nb, + struct vfio_pci_core_device, nb); struct device *dev = data; struct pci_dev *pdev = to_pci_dev(dev); struct pci_dev *physfn = pci_physfn(pdev); @@ -1924,7 +1924,7 @@ static int vfio_pci_bus_notifier(struct notifier_block *nb, static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) { - struct vfio_pci_device *vdev; + struct vfio_pci_core_device *vdev; struct iommu_group *group; int ret; @@ -2031,7 +2031,7 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) static void vfio_pci_remove(struct pci_dev *pdev) { - struct vfio_pci_device *vdev; + struct vfio_pci_core_device *vdev; pci_disable_sriov(pdev); @@ -2071,7 +2071,7 @@ static void vfio_pci_remove(struct pci_dev *pdev) static pci_ers_result_t vfio_pci_aer_err_detected(struct pci_dev *pdev, pci_channel_state_t state) { - struct vfio_pci_device *vdev; + struct vfio_pci_core_device *vdev; struct vfio_device *device; device = vfio_device_get_from_dev(&pdev->dev); @@ -2098,7 +2098,7 @@ static pci_ers_result_t vfio_pci_aer_err_detected(struct pci_dev *pdev, static int vfio_pci_sriov_configure(struct pci_dev *pdev, int nr_virtfn) { - struct vfio_pci_device *vdev; + struct vfio_pci_core_device *vdev; struct vfio_device *device; int ret = 0; @@ -2165,7 +2165,7 @@ static int vfio_pci_reflck_find(struct pci_dev *pdev, void *data) { struct vfio_pci_reflck **preflck = data; struct vfio_device *device; - struct vfio_pci_device *vdev; + struct vfio_pci_core_device *vdev; device = vfio_device_get_from_dev(&pdev->dev); if (!device) @@ -2189,7 +2189,7 @@ static int vfio_pci_reflck_find(struct pci_dev *pdev, void *data) return 0; } -static int vfio_pci_reflck_attach(struct vfio_pci_device *vdev) +static int vfio_pci_reflck_attach(struct vfio_pci_core_device *vdev) { bool slot = !pci_probe_reset_slot(vdev->pdev->slot); @@ -2224,7 +2224,7 @@ static int vfio_pci_get_unused_devs(struct pci_dev *pdev, void *data) { struct vfio_devices *devs = data; struct vfio_device *device; - struct vfio_pci_device *vdev; + struct vfio_pci_core_device *vdev; if (devs->cur_index == devs->max_index) return -ENOSPC; @@ -2254,7 +2254,7 @@ static int vfio_pci_try_zap_and_vma_lock_cb(struct pci_dev *pdev, void *data) { struct vfio_devices *devs = data; struct vfio_device *device; - struct vfio_pci_device *vdev; + struct vfio_pci_core_device *vdev; if (devs->cur_index == devs->max_index) return -ENOSPC; @@ -2299,12 +2299,12 @@ static int vfio_pci_try_zap_and_vma_lock_cb(struct pci_dev *pdev, void *data) * to be bound to vfio_pci since that's the only way we can be sure they * stay put. */ -static void vfio_pci_try_bus_reset(struct vfio_pci_device *vdev) +static void vfio_pci_try_bus_reset(struct vfio_pci_core_device *vdev) { struct vfio_devices devs = { .cur_index = 0 }; int i = 0, ret = -EINVAL; bool slot = false; - struct vfio_pci_device *tmp; + struct vfio_pci_core_device *tmp; if (!pci_probe_reset_slot(vdev->pdev->slot)) slot = true; diff --git a/drivers/vfio/pci/vfio_pci_core.h b/drivers/vfio/pci/vfio_pci_core.h index b9ac7132b84a..3964ca898984 100644 --- a/drivers/vfio/pci/vfio_pci_core.h +++ b/drivers/vfio/pci/vfio_pci_core.h @@ -32,15 +32,15 @@ #define VFIO_PCI_IOEVENTFD_MAX 1000 struct vfio_pci_ioeventfd { - struct list_head next; - struct vfio_pci_device *vdev; - struct virqfd *virqfd; - void __iomem *addr; - uint64_t data; - loff_t pos; - int bar; - int count; - bool test_mem; + struct list_head next; + struct vfio_pci_core_device *vdev; + struct virqfd *virqfd; + void __iomem *addr; + uint64_t data; + loff_t pos; + int bar; + int count; + bool test_mem; }; struct vfio_pci_irq_ctx { @@ -52,18 +52,18 @@ struct vfio_pci_irq_ctx { struct irq_bypass_producer producer; }; -struct vfio_pci_device; +struct vfio_pci_core_device; struct vfio_pci_region; struct vfio_pci_regops { - size_t (*rw)(struct vfio_pci_device *vdev, char __user *buf, + size_t (*rw)(struct vfio_pci_core_device *vdev, char __user *buf, size_t count, loff_t *ppos, bool iswrite); - void (*release)(struct vfio_pci_device *vdev, + void (*release)(struct vfio_pci_core_device *vdev, struct vfio_pci_region *region); - int (*mmap)(struct vfio_pci_device *vdev, + int (*mmap)(struct vfio_pci_core_device *vdev, struct vfio_pci_region *region, struct vm_area_struct *vma); - int (*add_capability)(struct vfio_pci_device *vdev, + int (*add_capability)(struct vfio_pci_core_device *vdev, struct vfio_pci_region *region, struct vfio_info_cap *caps); }; @@ -99,7 +99,7 @@ struct vfio_pci_mmap_vma { struct list_head vma_next; }; -struct vfio_pci_device { +struct vfio_pci_core_device { struct pci_dev *pdev; void __iomem *barmap[PCI_STD_NUM_BARS]; bool bar_mmap_supported[PCI_STD_NUM_BARS]; @@ -150,75 +150,75 @@ struct vfio_pci_device { #define is_irq_none(vdev) (!(is_intx(vdev) || is_msi(vdev) || is_msix(vdev))) #define irq_is(vdev, type) (vdev->irq_type == type) -extern void vfio_pci_intx_mask(struct vfio_pci_device *vdev); -extern void vfio_pci_intx_unmask(struct vfio_pci_device *vdev); +extern void vfio_pci_intx_mask(struct vfio_pci_core_device *vdev); +extern void vfio_pci_intx_unmask(struct vfio_pci_core_device *vdev); -extern int vfio_pci_set_irqs_ioctl(struct vfio_pci_device *vdev, +extern int vfio_pci_set_irqs_ioctl(struct vfio_pci_core_device *vdev, uint32_t flags, unsigned index, unsigned start, unsigned count, void *data); -extern ssize_t vfio_pci_config_rw(struct vfio_pci_device *vdev, +extern ssize_t vfio_pci_config_rw(struct vfio_pci_core_device *vdev, char __user *buf, size_t count, loff_t *ppos, bool iswrite); -extern ssize_t vfio_pci_bar_rw(struct vfio_pci_device *vdev, char __user *buf, +extern ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf, size_t count, loff_t *ppos, bool iswrite); -extern ssize_t vfio_pci_vga_rw(struct vfio_pci_device *vdev, char __user *buf, +extern ssize_t vfio_pci_vga_rw(struct vfio_pci_core_device *vdev, char __user *buf, size_t count, loff_t *ppos, bool iswrite); -extern long vfio_pci_ioeventfd(struct vfio_pci_device *vdev, loff_t offset, +extern long vfio_pci_ioeventfd(struct vfio_pci_core_device *vdev, loff_t offset, uint64_t data, int count, int fd); extern int vfio_pci_init_perm_bits(void); extern void vfio_pci_uninit_perm_bits(void); -extern int vfio_config_init(struct vfio_pci_device *vdev); -extern void vfio_config_free(struct vfio_pci_device *vdev); +extern int vfio_config_init(struct vfio_pci_core_device *vdev); +extern void vfio_config_free(struct vfio_pci_core_device *vdev); -extern int vfio_pci_register_dev_region(struct vfio_pci_device *vdev, +extern int vfio_pci_register_dev_region(struct vfio_pci_core_device *vdev, unsigned int type, unsigned int subtype, const struct vfio_pci_regops *ops, size_t size, u32 flags, void *data); -extern int vfio_pci_set_power_state(struct vfio_pci_device *vdev, +extern int vfio_pci_set_power_state(struct vfio_pci_core_device *vdev, pci_power_t state); -extern bool __vfio_pci_memory_enabled(struct vfio_pci_device *vdev); -extern void vfio_pci_zap_and_down_write_memory_lock(struct vfio_pci_device +extern bool __vfio_pci_memory_enabled(struct vfio_pci_core_device *vdev); +extern void vfio_pci_zap_and_down_write_memory_lock(struct vfio_pci_core_device *vdev); -extern u16 vfio_pci_memory_lock_and_enable(struct vfio_pci_device *vdev); -extern void vfio_pci_memory_unlock_and_restore(struct vfio_pci_device *vdev, +extern u16 vfio_pci_memory_lock_and_enable(struct vfio_pci_core_device *vdev); +extern void vfio_pci_memory_unlock_and_restore(struct vfio_pci_core_device *vdev, u16 cmd); #ifdef CONFIG_VFIO_PCI_IGD -extern int vfio_pci_igd_init(struct vfio_pci_device *vdev); +extern int vfio_pci_igd_init(struct vfio_pci_core_device *vdev); #else -static inline int vfio_pci_igd_init(struct vfio_pci_device *vdev) +static inline int vfio_pci_igd_init(struct vfio_pci_core_device *vdev) { return -ENODEV; } #endif #ifdef CONFIG_VFIO_PCI_NVLINK2 -extern int vfio_pci_nvdia_v100_nvlink2_init(struct vfio_pci_device *vdev); -extern int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev); +extern int vfio_pci_nvdia_v100_nvlink2_init(struct vfio_pci_core_device *vdev); +extern int vfio_pci_ibm_npu2_init(struct vfio_pci_core_device *vdev); #else -static inline int vfio_pci_nvdia_v100_nvlink2_init(struct vfio_pci_device *vdev) +static inline int vfio_pci_nvdia_v100_nvlink2_init(struct vfio_pci_core_device *vdev) { return -ENODEV; } -static inline int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev) +static inline int vfio_pci_ibm_npu2_init(struct vfio_pci_core_device *vdev) { return -ENODEV; } #endif #ifdef CONFIG_S390 -extern int vfio_pci_info_zdev_add_caps(struct vfio_pci_device *vdev, +extern int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_device *vdev, struct vfio_info_cap *caps); #else -static inline int vfio_pci_info_zdev_add_caps(struct vfio_pci_device *vdev, +static inline int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_device *vdev, struct vfio_info_cap *caps) { return -ENODEV; diff --git a/drivers/vfio/pci/vfio_pci_igd.c b/drivers/vfio/pci/vfio_pci_igd.c index 0c599cd33d01..2388c9722ed8 100644 --- a/drivers/vfio/pci/vfio_pci_igd.c +++ b/drivers/vfio/pci/vfio_pci_igd.c @@ -21,7 +21,7 @@ #define OPREGION_SIZE (8 * 1024) #define OPREGION_PCI_ADDR 0xfc -static size_t vfio_pci_igd_rw(struct vfio_pci_device *vdev, char __user *buf, +static size_t vfio_pci_igd_rw(struct vfio_pci_core_device *vdev, char __user *buf, size_t count, loff_t *ppos, bool iswrite) { unsigned int i = VFIO_PCI_OFFSET_TO_INDEX(*ppos) - VFIO_PCI_NUM_REGIONS; @@ -41,7 +41,7 @@ static size_t vfio_pci_igd_rw(struct vfio_pci_device *vdev, char __user *buf, return count; } -static void vfio_pci_igd_release(struct vfio_pci_device *vdev, +static void vfio_pci_igd_release(struct vfio_pci_core_device *vdev, struct vfio_pci_region *region) { memunmap(region->data); @@ -52,7 +52,7 @@ static const struct vfio_pci_regops vfio_pci_igd_regops = { .release = vfio_pci_igd_release, }; -static int vfio_pci_igd_opregion_init(struct vfio_pci_device *vdev) +static int vfio_pci_igd_opregion_init(struct vfio_pci_core_device *vdev) { __le32 *dwordp = (__le32 *)(vdev->vconfig + OPREGION_PCI_ADDR); u32 addr, size; @@ -107,7 +107,7 @@ static int vfio_pci_igd_opregion_init(struct vfio_pci_device *vdev) return ret; } -static size_t vfio_pci_igd_cfg_rw(struct vfio_pci_device *vdev, +static size_t vfio_pci_igd_cfg_rw(struct vfio_pci_core_device *vdev, char __user *buf, size_t count, loff_t *ppos, bool iswrite) { @@ -200,7 +200,7 @@ static size_t vfio_pci_igd_cfg_rw(struct vfio_pci_device *vdev, return count; } -static void vfio_pci_igd_cfg_release(struct vfio_pci_device *vdev, +static void vfio_pci_igd_cfg_release(struct vfio_pci_core_device *vdev, struct vfio_pci_region *region) { struct pci_dev *pdev = region->data; @@ -213,7 +213,7 @@ static const struct vfio_pci_regops vfio_pci_igd_cfg_regops = { .release = vfio_pci_igd_cfg_release, }; -static int vfio_pci_igd_cfg_init(struct vfio_pci_device *vdev) +static int vfio_pci_igd_cfg_init(struct vfio_pci_core_device *vdev) { struct pci_dev *host_bridge, *lpc_bridge; int ret; @@ -261,7 +261,7 @@ static int vfio_pci_igd_cfg_init(struct vfio_pci_device *vdev) return 0; } -int vfio_pci_igd_init(struct vfio_pci_device *vdev) +int vfio_pci_igd_init(struct vfio_pci_core_device *vdev) { int ret; diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_intrs.c index df1e8c8c274c..945ddbdf4d11 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -27,13 +27,13 @@ */ static void vfio_send_intx_eventfd(void *opaque, void *unused) { - struct vfio_pci_device *vdev = opaque; + struct vfio_pci_core_device *vdev = opaque; if (likely(is_intx(vdev) && !vdev->virq_disabled)) eventfd_signal(vdev->ctx[0].trigger, 1); } -void vfio_pci_intx_mask(struct vfio_pci_device *vdev) +void vfio_pci_intx_mask(struct vfio_pci_core_device *vdev) { struct pci_dev *pdev = vdev->pdev; unsigned long flags; @@ -73,7 +73,7 @@ void vfio_pci_intx_mask(struct vfio_pci_device *vdev) */ static int vfio_pci_intx_unmask_handler(void *opaque, void *unused) { - struct vfio_pci_device *vdev = opaque; + struct vfio_pci_core_device *vdev = opaque; struct pci_dev *pdev = vdev->pdev; unsigned long flags; int ret = 0; @@ -107,7 +107,7 @@ static int vfio_pci_intx_unmask_handler(void *opaque, void *unused) return ret; } -void vfio_pci_intx_unmask(struct vfio_pci_device *vdev) +void vfio_pci_intx_unmask(struct vfio_pci_core_device *vdev) { if (vfio_pci_intx_unmask_handler(vdev, NULL) > 0) vfio_send_intx_eventfd(vdev, NULL); @@ -115,7 +115,7 @@ void vfio_pci_intx_unmask(struct vfio_pci_device *vdev) static irqreturn_t vfio_intx_handler(int irq, void *dev_id) { - struct vfio_pci_device *vdev = dev_id; + struct vfio_pci_core_device *vdev = dev_id; unsigned long flags; int ret = IRQ_NONE; @@ -139,7 +139,7 @@ static irqreturn_t vfio_intx_handler(int irq, void *dev_id) return ret; } -static int vfio_intx_enable(struct vfio_pci_device *vdev) +static int vfio_intx_enable(struct vfio_pci_core_device *vdev) { if (!is_irq_none(vdev)) return -EINVAL; @@ -168,7 +168,7 @@ static int vfio_intx_enable(struct vfio_pci_device *vdev) return 0; } -static int vfio_intx_set_signal(struct vfio_pci_device *vdev, int fd) +static int vfio_intx_set_signal(struct vfio_pci_core_device *vdev, int fd) { struct pci_dev *pdev = vdev->pdev; unsigned long irqflags = IRQF_SHARED; @@ -223,7 +223,7 @@ static int vfio_intx_set_signal(struct vfio_pci_device *vdev, int fd) return 0; } -static void vfio_intx_disable(struct vfio_pci_device *vdev) +static void vfio_intx_disable(struct vfio_pci_core_device *vdev) { vfio_virqfd_disable(&vdev->ctx[0].unmask); vfio_virqfd_disable(&vdev->ctx[0].mask); @@ -244,7 +244,7 @@ static irqreturn_t vfio_msihandler(int irq, void *arg) return IRQ_HANDLED; } -static int vfio_msi_enable(struct vfio_pci_device *vdev, int nvec, bool msix) +static int vfio_msi_enable(struct vfio_pci_core_device *vdev, int nvec, bool msix) { struct pci_dev *pdev = vdev->pdev; unsigned int flag = msix ? PCI_IRQ_MSIX : PCI_IRQ_MSI; @@ -285,7 +285,7 @@ static int vfio_msi_enable(struct vfio_pci_device *vdev, int nvec, bool msix) return 0; } -static int vfio_msi_set_vector_signal(struct vfio_pci_device *vdev, +static int vfio_msi_set_vector_signal(struct vfio_pci_core_device *vdev, int vector, int fd, bool msix) { struct pci_dev *pdev = vdev->pdev; @@ -364,7 +364,7 @@ static int vfio_msi_set_vector_signal(struct vfio_pci_device *vdev, return 0; } -static int vfio_msi_set_block(struct vfio_pci_device *vdev, unsigned start, +static int vfio_msi_set_block(struct vfio_pci_core_device *vdev, unsigned start, unsigned count, int32_t *fds, bool msix) { int i, j, ret = 0; @@ -385,7 +385,7 @@ static int vfio_msi_set_block(struct vfio_pci_device *vdev, unsigned start, return ret; } -static void vfio_msi_disable(struct vfio_pci_device *vdev, bool msix) +static void vfio_msi_disable(struct vfio_pci_core_device *vdev, bool msix) { struct pci_dev *pdev = vdev->pdev; int i; @@ -417,7 +417,7 @@ static void vfio_msi_disable(struct vfio_pci_device *vdev, bool msix) /* * IOCTL support */ -static int vfio_pci_set_intx_unmask(struct vfio_pci_device *vdev, +static int vfio_pci_set_intx_unmask(struct vfio_pci_core_device *vdev, unsigned index, unsigned start, unsigned count, uint32_t flags, void *data) { @@ -444,7 +444,7 @@ static int vfio_pci_set_intx_unmask(struct vfio_pci_device *vdev, return 0; } -static int vfio_pci_set_intx_mask(struct vfio_pci_device *vdev, +static int vfio_pci_set_intx_mask(struct vfio_pci_core_device *vdev, unsigned index, unsigned start, unsigned count, uint32_t flags, void *data) { @@ -464,7 +464,7 @@ static int vfio_pci_set_intx_mask(struct vfio_pci_device *vdev, return 0; } -static int vfio_pci_set_intx_trigger(struct vfio_pci_device *vdev, +static int vfio_pci_set_intx_trigger(struct vfio_pci_core_device *vdev, unsigned index, unsigned start, unsigned count, uint32_t flags, void *data) { @@ -507,7 +507,7 @@ static int vfio_pci_set_intx_trigger(struct vfio_pci_device *vdev, return 0; } -static int vfio_pci_set_msi_trigger(struct vfio_pci_device *vdev, +static int vfio_pci_set_msi_trigger(struct vfio_pci_core_device *vdev, unsigned index, unsigned start, unsigned count, uint32_t flags, void *data) { @@ -613,7 +613,7 @@ static int vfio_pci_set_ctx_trigger_single(struct eventfd_ctx **ctx, return -EINVAL; } -static int vfio_pci_set_err_trigger(struct vfio_pci_device *vdev, +static int vfio_pci_set_err_trigger(struct vfio_pci_core_device *vdev, unsigned index, unsigned start, unsigned count, uint32_t flags, void *data) { @@ -624,7 +624,7 @@ static int vfio_pci_set_err_trigger(struct vfio_pci_device *vdev, count, flags, data); } -static int vfio_pci_set_req_trigger(struct vfio_pci_device *vdev, +static int vfio_pci_set_req_trigger(struct vfio_pci_core_device *vdev, unsigned index, unsigned start, unsigned count, uint32_t flags, void *data) { @@ -635,11 +635,11 @@ static int vfio_pci_set_req_trigger(struct vfio_pci_device *vdev, count, flags, data); } -int vfio_pci_set_irqs_ioctl(struct vfio_pci_device *vdev, uint32_t flags, +int vfio_pci_set_irqs_ioctl(struct vfio_pci_core_device *vdev, uint32_t flags, unsigned index, unsigned start, unsigned count, void *data) { - int (*func)(struct vfio_pci_device *vdev, unsigned index, + int (*func)(struct vfio_pci_core_device *vdev, unsigned index, unsigned start, unsigned count, uint32_t flags, void *data) = NULL; diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2.c index 326a704c4527..8ef2c62a9d27 100644 --- a/drivers/vfio/pci/vfio_pci_nvlink2.c +++ b/drivers/vfio/pci/vfio_pci_nvlink2.c @@ -39,7 +39,7 @@ struct vfio_pci_nvgpu_data { struct notifier_block group_notifier; }; -static size_t vfio_pci_nvgpu_rw(struct vfio_pci_device *vdev, +static size_t vfio_pci_nvgpu_rw(struct vfio_pci_core_device *vdev, char __user *buf, size_t count, loff_t *ppos, bool iswrite) { unsigned int i = VFIO_PCI_OFFSET_TO_INDEX(*ppos) - VFIO_PCI_NUM_REGIONS; @@ -89,7 +89,7 @@ static size_t vfio_pci_nvgpu_rw(struct vfio_pci_device *vdev, return count; } -static void vfio_pci_nvgpu_release(struct vfio_pci_device *vdev, +static void vfio_pci_nvgpu_release(struct vfio_pci_core_device *vdev, struct vfio_pci_region *region) { struct vfio_pci_nvgpu_data *data = region->data; @@ -136,7 +136,7 @@ static const struct vm_operations_struct vfio_pci_nvgpu_mmap_vmops = { .fault = vfio_pci_nvgpu_mmap_fault, }; -static int vfio_pci_nvgpu_mmap(struct vfio_pci_device *vdev, +static int vfio_pci_nvgpu_mmap(struct vfio_pci_core_device *vdev, struct vfio_pci_region *region, struct vm_area_struct *vma) { int ret; @@ -171,7 +171,7 @@ static int vfio_pci_nvgpu_mmap(struct vfio_pci_device *vdev, return ret; } -static int vfio_pci_nvgpu_add_capability(struct vfio_pci_device *vdev, +static int vfio_pci_nvgpu_add_capability(struct vfio_pci_core_device *vdev, struct vfio_pci_region *region, struct vfio_info_cap *caps) { struct vfio_pci_nvgpu_data *data = region->data; @@ -207,7 +207,7 @@ static int vfio_pci_nvgpu_group_notifier(struct notifier_block *nb, return NOTIFY_OK; } -int vfio_pci_nvdia_v100_nvlink2_init(struct vfio_pci_device *vdev) +int vfio_pci_nvdia_v100_nvlink2_init(struct vfio_pci_core_device *vdev) { int ret; u64 reg[2]; @@ -304,7 +304,7 @@ struct vfio_pci_npu2_data { unsigned int link_speed; /* The link speed from DT's ibm,nvlink-speed */ }; -static size_t vfio_pci_npu2_rw(struct vfio_pci_device *vdev, +static size_t vfio_pci_npu2_rw(struct vfio_pci_core_device *vdev, char __user *buf, size_t count, loff_t *ppos, bool iswrite) { unsigned int i = VFIO_PCI_OFFSET_TO_INDEX(*ppos) - VFIO_PCI_NUM_REGIONS; @@ -328,7 +328,7 @@ static size_t vfio_pci_npu2_rw(struct vfio_pci_device *vdev, return count; } -static int vfio_pci_npu2_mmap(struct vfio_pci_device *vdev, +static int vfio_pci_npu2_mmap(struct vfio_pci_core_device *vdev, struct vfio_pci_region *region, struct vm_area_struct *vma) { int ret; @@ -349,7 +349,7 @@ static int vfio_pci_npu2_mmap(struct vfio_pci_device *vdev, return ret; } -static void vfio_pci_npu2_release(struct vfio_pci_device *vdev, +static void vfio_pci_npu2_release(struct vfio_pci_core_device *vdev, struct vfio_pci_region *region) { struct vfio_pci_npu2_data *data = region->data; @@ -358,7 +358,7 @@ static void vfio_pci_npu2_release(struct vfio_pci_device *vdev, kfree(data); } -static int vfio_pci_npu2_add_capability(struct vfio_pci_device *vdev, +static int vfio_pci_npu2_add_capability(struct vfio_pci_core_device *vdev, struct vfio_pci_region *region, struct vfio_info_cap *caps) { struct vfio_pci_npu2_data *data = region->data; @@ -388,7 +388,7 @@ static const struct vfio_pci_regops vfio_pci_npu2_regops = { .add_capability = vfio_pci_npu2_add_capability, }; -int vfio_pci_ibm_npu2_init(struct vfio_pci_device *vdev) +int vfio_pci_ibm_npu2_init(struct vfio_pci_core_device *vdev) { int ret; struct vfio_pci_npu2_data *data; diff --git a/drivers/vfio/pci/vfio_pci_rdwr.c b/drivers/vfio/pci/vfio_pci_rdwr.c index 667e82726e75..8fff4689dd44 100644 --- a/drivers/vfio/pci/vfio_pci_rdwr.c +++ b/drivers/vfio/pci/vfio_pci_rdwr.c @@ -38,7 +38,7 @@ #define vfio_iowrite8 iowrite8 #define VFIO_IOWRITE(size) \ -static int vfio_pci_iowrite##size(struct vfio_pci_device *vdev, \ +static int vfio_pci_iowrite##size(struct vfio_pci_core_device *vdev, \ bool test_mem, u##size val, void __iomem *io) \ { \ if (test_mem) { \ @@ -65,7 +65,7 @@ VFIO_IOWRITE(64) #endif #define VFIO_IOREAD(size) \ -static int vfio_pci_ioread##size(struct vfio_pci_device *vdev, \ +static int vfio_pci_ioread##size(struct vfio_pci_core_device *vdev, \ bool test_mem, u##size *val, void __iomem *io) \ { \ if (test_mem) { \ @@ -94,7 +94,7 @@ VFIO_IOREAD(32) * reads with -1. This is intended for handling MSI-X vector tables and * leftover space for ROM BARs. */ -static ssize_t do_io_rw(struct vfio_pci_device *vdev, bool test_mem, +static ssize_t do_io_rw(struct vfio_pci_core_device *vdev, bool test_mem, void __iomem *io, char __user *buf, loff_t off, size_t count, size_t x_start, size_t x_end, bool iswrite) @@ -200,7 +200,7 @@ static ssize_t do_io_rw(struct vfio_pci_device *vdev, bool test_mem, return done; } -static int vfio_pci_setup_barmap(struct vfio_pci_device *vdev, int bar) +static int vfio_pci_setup_barmap(struct vfio_pci_core_device *vdev, int bar) { struct pci_dev *pdev = vdev->pdev; int ret; @@ -224,7 +224,7 @@ static int vfio_pci_setup_barmap(struct vfio_pci_device *vdev, int bar) return 0; } -ssize_t vfio_pci_bar_rw(struct vfio_pci_device *vdev, char __user *buf, +ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *buf, size_t count, loff_t *ppos, bool iswrite) { struct pci_dev *pdev = vdev->pdev; @@ -288,7 +288,7 @@ ssize_t vfio_pci_bar_rw(struct vfio_pci_device *vdev, char __user *buf, return done; } -ssize_t vfio_pci_vga_rw(struct vfio_pci_device *vdev, char __user *buf, +ssize_t vfio_pci_vga_rw(struct vfio_pci_core_device *vdev, char __user *buf, size_t count, loff_t *ppos, bool iswrite) { int ret; @@ -384,7 +384,7 @@ static void vfio_pci_ioeventfd_do_write(struct vfio_pci_ioeventfd *ioeventfd, static int vfio_pci_ioeventfd_handler(void *opaque, void *unused) { struct vfio_pci_ioeventfd *ioeventfd = opaque; - struct vfio_pci_device *vdev = ioeventfd->vdev; + struct vfio_pci_core_device *vdev = ioeventfd->vdev; if (ioeventfd->test_mem) { if (!down_read_trylock(&vdev->memory_lock)) @@ -410,7 +410,7 @@ static void vfio_pci_ioeventfd_thread(void *opaque, void *unused) vfio_pci_ioeventfd_do_write(ioeventfd, ioeventfd->test_mem); } -long vfio_pci_ioeventfd(struct vfio_pci_device *vdev, loff_t offset, +long vfio_pci_ioeventfd(struct vfio_pci_core_device *vdev, loff_t offset, uint64_t data, int count, int fd) { struct pci_dev *pdev = vdev->pdev; diff --git a/drivers/vfio/pci/vfio_pci_zdev.c b/drivers/vfio/pci/vfio_pci_zdev.c index 3e91d49fa3f0..3216925c4d41 100644 --- a/drivers/vfio/pci/vfio_pci_zdev.c +++ b/drivers/vfio/pci/vfio_pci_zdev.c @@ -24,7 +24,7 @@ /* * Add the Base PCI Function information to the device info region. */ -static int zpci_base_cap(struct zpci_dev *zdev, struct vfio_pci_device *vdev, +static int zpci_base_cap(struct zpci_dev *zdev, struct vfio_pci_core_device *vdev, struct vfio_info_cap *caps) { struct vfio_device_info_cap_zpci_base cap = { @@ -45,7 +45,7 @@ static int zpci_base_cap(struct zpci_dev *zdev, struct vfio_pci_device *vdev, /* * Add the Base PCI Function Group information to the device info region. */ -static int zpci_group_cap(struct zpci_dev *zdev, struct vfio_pci_device *vdev, +static int zpci_group_cap(struct zpci_dev *zdev, struct vfio_pci_core_device *vdev, struct vfio_info_cap *caps) { struct vfio_device_info_cap_zpci_group cap = { @@ -66,7 +66,7 @@ static int zpci_group_cap(struct zpci_dev *zdev, struct vfio_pci_device *vdev, /* * Add the device utility string to the device info region. */ -static int zpci_util_cap(struct zpci_dev *zdev, struct vfio_pci_device *vdev, +static int zpci_util_cap(struct zpci_dev *zdev, struct vfio_pci_core_device *vdev, struct vfio_info_cap *caps) { struct vfio_device_info_cap_zpci_util *cap; @@ -90,7 +90,7 @@ static int zpci_util_cap(struct zpci_dev *zdev, struct vfio_pci_device *vdev, /* * Add the function path string to the device info region. */ -static int zpci_pfip_cap(struct zpci_dev *zdev, struct vfio_pci_device *vdev, +static int zpci_pfip_cap(struct zpci_dev *zdev, struct vfio_pci_core_device *vdev, struct vfio_info_cap *caps) { struct vfio_device_info_cap_zpci_pfip *cap; @@ -114,7 +114,7 @@ static int zpci_pfip_cap(struct zpci_dev *zdev, struct vfio_pci_device *vdev, /* * Add all supported capabilities to the VFIO_DEVICE_GET_INFO capability chain. */ -int vfio_pci_info_zdev_add_caps(struct vfio_pci_device *vdev, +int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_device *vdev, struct vfio_info_cap *caps) { struct zpci_dev *zdev = to_zpci(vdev->pdev); From patchwork Tue Mar 9 08:33:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Gurtovoy X-Patchwork-Id: 12124389 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0429C43381 for ; Tue, 9 Mar 2021 08:35:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C31296527C for ; Tue, 9 Mar 2021 08:35:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230437AbhCIIey (ORCPT ); Tue, 9 Mar 2021 03:34:54 -0500 Received: from mail-bn8nam11on2081.outbound.protection.outlook.com ([40.107.236.81]:4226 "EHLO NAM11-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S230112AbhCIIec (ORCPT ); Tue, 9 Mar 2021 03:34:32 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MnNMfwMkWtACh98ATYvPowEKMEFowWkH6KXIkRIZirIUi/X9LvBvNoaUT2alUybvE6QZPauZRfoHO15WH49zmZA9Y0fYBrvACP3GhKcDVSa8pYrjX9/nYUNKtuVfQl+u3mGPmlsVuqbgIOyRIcTq6++p4TaTTZ9cEwHMlpvBgDSEV2w9oJ2ZERp6aGyvSNVHu04urctr2tfwb9+5CXyzds+hVU8NNSuKv9djfAEpyBP4AfIIMMtwGbyyQ0YNrSBPpi9cjeHNRB9XOOaqcCwArHBl01ICGnKgrQYEte2BDjODl3u+HddL1qevbQMqylAXBDbGl7MLJQzCeQuH7s0IIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wpoZVBBPjW94JYFpN28otilku2QhJFNv24oeWXA07d0=; b=ZXF+ky1C/ZUYlYwIFDC4trndbq548yPYT0sGyEJlM2lPUlBQhZAlSW5Qz+J+nD7/xRgT4lAqliLGyb00RbOYGNjqmCdTY3kydKfks82VymlNBY7VOD3WSSrnORK820WwhZO2Fqk6SsQNTe8LBIwggva4VkA62ZuqA03V+UI3FZ7r6kBR+furGdILOksJtAcccg7chz4PwqEfzO3TKj6NGGUj0ehWCyTRf/xkrXP9s2nXKv53Gt9D9wCjqifrLkvdCG7xk8Q86YsdI/TXipsSsnCZR8ihVBTW8WnzqMz1oERycY/e7M7HMb5KSmk5qHJXCO7hONAFqD6V2JFCxldijA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=linux.ibm.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wpoZVBBPjW94JYFpN28otilku2QhJFNv24oeWXA07d0=; b=qYFbNDKjimhFo8ae7Zdwfgdwgcuqn/sQEXUbCrdFDOzifBUe5BF+Qx6sOR9yNQppYnKWxJNEOZBoHKLAZL+LBk2EHAPg/OyPv4R12PzJwsQ+jCa4Zc4xO7VCm1ZuS7v15lkUIkOFqS5I92Sov82P3zqmNQ0Bj68qhAjME9CXiHA9Yxhltz/kZpgFCYfkQbjWjOcg4qggu9sZk0vx2BnruQX1xk4dL0kD/1q2SQfCSaKJbYZMXFoG8tfi9ZZg78716x+FfAXZqQ9dajO2lwPDuPFLJ0kY6nkGDqGKXcSG8CHOKthbf0Jws1duVBiAXISAJb6HJk3kTxzCIfEuigsEIg== Received: from DS7PR03CA0213.namprd03.prod.outlook.com (2603:10b6:5:3ba::8) by SN1PR12MB2432.namprd12.prod.outlook.com (2603:10b6:802:29::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.17; Tue, 9 Mar 2021 08:34:29 +0000 Received: from DM6NAM11FT012.eop-nam11.prod.protection.outlook.com (2603:10b6:5:3ba:cafe::58) by DS7PR03CA0213.outlook.office365.com (2603:10b6:5:3ba::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.17 via Frontend Transport; Tue, 9 Mar 2021 08:34:29 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; linux.ibm.com; dkim=none (message not signed) header.d=none;linux.ibm.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT012.mail.protection.outlook.com (10.13.173.109) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.3912.17 via Frontend Transport; Tue, 9 Mar 2021 08:34:28 +0000 Received: from HQMAIL111.nvidia.com (172.20.187.18) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Mar 2021 08:34:27 +0000 Received: from r-nvmx02.mtr.labs.mlnx (172.20.145.6) by mail.nvidia.com (172.20.187.18) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 9 Mar 2021 08:34:21 +0000 From: Max Gurtovoy To: , , , , CC: , , , , , , , , , , , , , , , Max Gurtovoy Subject: [PATCH 4/9] vfio-pci: introduce vfio_pci_core subsystem driver Date: Tue, 9 Mar 2021 08:33:52 +0000 Message-ID: <20210309083357.65467-5-mgurtovoy@nvidia.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20210309083357.65467-1-mgurtovoy@nvidia.com> References: <20210309083357.65467-1-mgurtovoy@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e5c22a79-0c0c-469d-9e73-08d8e2d62782 X-MS-TrafficTypeDiagnostic: SN1PR12MB2432: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:862; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4y3B+l7/fpf4qS0cTkccfws8hSmJRwN6ofBPoygyutC5VlmXnYJ4XuoU8pD1t55imdeUI2WdlsBAwtS/ao0jZIa/F7WT7sVRgd/liius+raUpvD2XFP8PbSakCvolXR77uZDJujQOysw1m4fnFjG4nyU9paOb2iXZi3+honz86xN2mFv9njbmsIDRmbsvWDgAqRbPR4pSidZog9+OIv4GBgTPIOgZxqoUlzO55WJuxQ5pJGNeKjU+ldNskZOUZA/DfzR26sSExDrcqiBH1IZR+zbIgBZeg6Ox6StWHt6prCgTCAFWnbXDTdJ/X6++Gl2fdAvlN5b6nvNkIGaDEwW7c3t6UUJuP0S69afjYM8vE4QA0qMKJ/RoQUe2DFS+ZeYwiUHBPxdefGKuPt6+NsLxOCt/WU17CXahvkNJMARI6h5oTXBQLm28A4AF6Q2AQLTcWc8s3SOqbr1gboC6AzwdbSonDcX4lhLm88OtQa42EnwvER7FEWFBbgfBNHwVM2ydLdbPyduzY/vIrQgwOUxynpEYxj+vOg/0XxP+WXhHxxSRl5hrcaoKlpWmNRcE8kYwNWAzh3u7ltvbrD/X5nhCqj5AyB9fscbmW4PqBHim2NDMVl1dGXyRh34VQ5EPryMrJWXZ8gCuvNXMi/TiMmIDLvek/zZn3NbRxM1RDbxTrMCzqNwVjvdbKzjzOTvI25b X-Forefront-Antispam-Report: CIP:216.228.112.34;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid03.nvidia.com;CAT:NONE;SFS:(4636009)(39860400002)(136003)(346002)(396003)(376002)(46966006)(36840700001)(83380400001)(82310400003)(36860700001)(4326008)(107886003)(5660300002)(34020700004)(6666004)(86362001)(2616005)(30864003)(47076005)(1076003)(8936002)(36756003)(356005)(2906002)(54906003)(8676002)(82740400003)(336012)(70586007)(26005)(70206006)(478600001)(110136005)(426003)(186003)(316002)(7636003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Mar 2021 08:34:28.5236 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e5c22a79-0c0c-469d-9e73-08d8e2d62782 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.34];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT012.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB2432 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Split the vfio_pci driver into two parts, the 'struct pci_driver' (vfio_pci) and a library of code (vfio_pci_core) that helps creating a VFIO device on top of a PCI device. As before vfio_pci.ko continues to present the same interface under sysfs and this change should have no functional impact. vfio_pci_core exposes an interface that is similar to a typical Linux subsystem, in that a pci_driver doing probe() can setup a number of details and then create the VFIO char device. Allowing another module to provide the pci_driver allows that module to customize how VFIO is setup, inject its own operations, and easily extend vendor specific functionality. This is complementary to how VFIO's mediated devices work. Instead of custome device lifecycle managmenet and a special bus drivers using this approach will rely on the normal driver core lifecycle (e.g. bind/unbind) management and this is optimized to effectively support customization that is only making small modifications to what vfio_pci would do normally. This approach is also a pluggable alternative for the hard wired CONFIG_VFIO_PCI_IGD and CONFIG_VFIO_PCI_NVLINK2 "drivers" that are built into vfio-pci. Using this work all of that code can be moved to a dedicated device-specific modules and cleanly split out of the generic vfio_pci driver. Below is an example for adding new driver to vfio pci subsystem: +-------------------------------------------------+ | | | VFIO | | | +-------------------------------------------------+ +-------------------------------------------------+ | | | VFIO_PCI_CORE | | | +-------------------------------------------------+ +--------------+ +---------------+ +--------------+ | | | | | | | VFIO_PCI | | MLX5_VFIO_PCI | | IGD_VFIO_PCI | | | | | | | +--------------+ +---------------+ +--------------+ In this way mlx5_vfio_pci will use vfio_pci_core to register to vfio subsystem and also use the generic PCI functionality exported from it. Additionally it will add the needed vendor specific logic for HW specific features such as Live Migration. Same for the igd_vfio_pci that will add special extensions for Intel Graphics cards (GVT-d). Signed-off-by: Max Gurtovoy --- drivers/vfio/pci/Kconfig | 22 ++- drivers/vfio/pci/Makefile | 13 +- drivers/vfio/pci/vfio_pci.c | 247 ++++++++++++++++++++++++ drivers/vfio/pci/vfio_pci_core.c | 318 ++++++++----------------------- drivers/vfio/pci/vfio_pci_core.h | 113 +++++++---- 5 files changed, 423 insertions(+), 290 deletions(-) create mode 100644 drivers/vfio/pci/vfio_pci.c diff --git a/drivers/vfio/pci/Kconfig b/drivers/vfio/pci/Kconfig index ac3c1dd3edef..829e90a2e5a3 100644 --- a/drivers/vfio/pci/Kconfig +++ b/drivers/vfio/pci/Kconfig @@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0-only -config VFIO_PCI - tristate "VFIO support for PCI devices" +config VFIO_PCI_CORE + tristate "VFIO core support for PCI devices" depends on VFIO && PCI && EVENTFD select VFIO_VIRQFD select IRQ_BYPASS_MANAGER @@ -10,9 +10,17 @@ config VFIO_PCI If you don't know what to do here, say N. +config VFIO_PCI + tristate "VFIO support for PCI devices" + depends on VFIO_PCI_CORE + help + This provides a generic PCI support using the VFIO framework. + + If you don't know what to do here, say N. + config VFIO_PCI_VGA bool "VFIO PCI support for VGA devices" - depends on VFIO_PCI && X86 && VGA_ARB + depends on VFIO_PCI_CORE && X86 && VGA_ARB help Support for VGA extension to VFIO PCI. This exposes an additional region on VGA devices for accessing legacy VGA addresses used by @@ -21,16 +29,16 @@ config VFIO_PCI_VGA If you don't know what to do here, say N. config VFIO_PCI_MMAP - depends on VFIO_PCI + depends on VFIO_PCI_CORE def_bool y if !S390 config VFIO_PCI_INTX - depends on VFIO_PCI + depends on VFIO_PCI_CORE def_bool y if !S390 config VFIO_PCI_IGD bool "VFIO PCI extensions for Intel graphics (GVT-d)" - depends on VFIO_PCI && X86 + depends on VFIO_PCI_CORE && X86 default y help Support for Intel IGD specific extensions to enable direct @@ -42,6 +50,6 @@ config VFIO_PCI_IGD config VFIO_PCI_NVLINK2 def_bool y - depends on VFIO_PCI && PPC_POWERNV + depends on VFIO_PCI_CORE && PPC_POWERNV help VFIO PCI support for P9 Witherspoon machine with NVIDIA V100 GPUs diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile index bbf8d7c8fc45..16e7d77d63ce 100644 --- a/drivers/vfio/pci/Makefile +++ b/drivers/vfio/pci/Makefile @@ -1,8 +1,11 @@ # SPDX-License-Identifier: GPL-2.0-only -vfio-pci-y := vfio_pci_core.o vfio_pci_intrs.o vfio_pci_rdwr.o vfio_pci_config.o -vfio-pci-$(CONFIG_VFIO_PCI_IGD) += vfio_pci_igd.o -vfio-pci-$(CONFIG_VFIO_PCI_NVLINK2) += vfio_pci_nvlink2.o -vfio-pci-$(CONFIG_S390) += vfio_pci_zdev.o - +obj-$(CONFIG_VFIO_PCI_CORE) += vfio-pci-core.o obj-$(CONFIG_VFIO_PCI) += vfio-pci.o + +vfio-pci-core-y := vfio_pci_core.o vfio_pci_intrs.o vfio_pci_rdwr.o vfio_pci_config.o +vfio-pci-core-$(CONFIG_VFIO_PCI_IGD) += vfio_pci_igd.o +vfio-pci-core-$(CONFIG_VFIO_PCI_NVLINK2) += vfio_pci_nvlink2.o +vfio-pci-core-$(CONFIG_S390) += vfio_pci_zdev.o + +vfio-pci-y := vfio_pci.o diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c new file mode 100644 index 000000000000..447c31f4e64e --- /dev/null +++ b/drivers/vfio/pci/vfio_pci.c @@ -0,0 +1,247 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2020, Mellanox Technologies, Ltd. All rights reserved. + * + * Copyright (C) 2012 Red Hat, Inc. All rights reserved. + * Author: Alex Williamson + * + * Derived from original vfio: + * Copyright 2010 Cisco Systems, Inc. All rights reserved. + * Author: Tom Lyon, pugs@cisco.com + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "vfio_pci_core.h" + +#define DRIVER_VERSION "0.2" +#define DRIVER_AUTHOR "Alex Williamson " +#define DRIVER_DESC "VFIO PCI - User Level meta-driver" + +static char ids[1024] __initdata; +module_param_string(ids, ids, sizeof(ids), 0); +MODULE_PARM_DESC(ids, "Initial PCI IDs to add to the vfio driver, format is \"vendor:device[:subvendor[:subdevice[:class[:class_mask]]]]\" and multiple comma separated entries can be specified"); + +static bool enable_sriov; +#ifdef CONFIG_PCI_IOV +module_param(enable_sriov, bool, 0644); +MODULE_PARM_DESC(enable_sriov, "Enable support for SR-IOV configuration. Enabling SR-IOV on a PF typically requires support of the userspace PF driver, enabling VFs without such support may result in non-functional VFs or PF."); +#endif + +static bool disable_denylist; +module_param(disable_denylist, bool, 0444); +MODULE_PARM_DESC(disable_denylist, "Disable use of device denylist. Disabling the denylist allows binding to devices with known errata that may lead to exploitable stability or security issues when accessed by untrusted users."); + +static bool vfio_pci_dev_in_denylist(struct pci_dev *pdev) +{ + switch (pdev->vendor) { + case PCI_VENDOR_ID_INTEL: + switch (pdev->device) { + case PCI_DEVICE_ID_INTEL_QAT_C3XXX: + case PCI_DEVICE_ID_INTEL_QAT_C3XXX_VF: + case PCI_DEVICE_ID_INTEL_QAT_C62X: + case PCI_DEVICE_ID_INTEL_QAT_C62X_VF: + case PCI_DEVICE_ID_INTEL_QAT_DH895XCC: + case PCI_DEVICE_ID_INTEL_QAT_DH895XCC_VF: + return true; + default: + return false; + } + } + + return false; +} + +static bool vfio_pci_is_denylisted(struct pci_dev *pdev) +{ + if (!vfio_pci_dev_in_denylist(pdev)) + return false; + + if (disable_denylist) { + pci_warn(pdev, + "device denylist disabled - allowing device %04x:%04x.\n", + pdev->vendor, pdev->device); + return false; + } + + pci_warn(pdev, "%04x:%04x exists in vfio-pci device denylist, driver probing disallowed.\n", + pdev->vendor, pdev->device); + + return true; +} + +static void vfio_pci_release(void *device_data) +{ + struct vfio_pci_core_device *vdev = device_data; + + mutex_lock(&vdev->reflck->lock); + if (!(--vdev->refcnt)) { + vfio_pci_vf_token_user_add(vdev, -1); + vfio_pci_core_spapr_eeh_release(vdev); + vfio_pci_core_disable(vdev); + } + mutex_unlock(&vdev->reflck->lock); + + module_put(THIS_MODULE); +} + +static int vfio_pci_open(void *device_data) +{ + struct vfio_pci_core_device *vdev = device_data; + int ret = 0; + + if (!try_module_get(THIS_MODULE)) + return -ENODEV; + + mutex_lock(&vdev->reflck->lock); + + if (!vdev->refcnt) { + ret = vfio_pci_core_enable(vdev); + if (ret) + goto error; + + vfio_pci_probe_mmaps(vdev); + vfio_pci_core_spapr_eeh_open(vdev); + vfio_pci_vf_token_user_add(vdev, 1); + } + vdev->refcnt++; +error: + mutex_unlock(&vdev->reflck->lock); + if (ret) + module_put(THIS_MODULE); + return ret; +} + +static const struct vfio_device_ops vfio_pci_ops = { + .name = "vfio-pci", + .open = vfio_pci_open, + .release = vfio_pci_release, + .ioctl = vfio_pci_core_ioctl, + .read = vfio_pci_core_read, + .write = vfio_pci_core_write, + .mmap = vfio_pci_core_mmap, + .request = vfio_pci_core_request, + .match = vfio_pci_core_match, +}; + +static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) +{ + struct vfio_pci_core_device *vdev; + + if (vfio_pci_is_denylisted(pdev)) + return -EINVAL; + + vdev = vfio_create_pci_device(pdev, &vfio_pci_ops); + if (IS_ERR(vdev)) + return PTR_ERR(vdev); + + return 0; +} + +static void vfio_pci_remove(struct pci_dev *pdev) +{ + vfio_destroy_pci_device(pdev); +} + +static int vfio_pci_sriov_configure(struct pci_dev *pdev, int nr_virtfn) +{ + might_sleep(); + + if (!enable_sriov) + return -ENOENT; + + return vfio_pci_core_sriov_configure(pdev, nr_virtfn); +} + +static struct pci_driver vfio_pci_driver = { + .name = "vfio-pci", + .id_table = NULL, /* only dynamic ids */ + .probe = vfio_pci_probe, + .remove = vfio_pci_remove, + .sriov_configure = vfio_pci_sriov_configure, + .err_handler = &vfio_pci_core_err_handlers, +}; + +static void __exit vfio_pci_cleanup(void) +{ + pci_unregister_driver(&vfio_pci_driver); +} + +static void __init vfio_pci_fill_ids(void) +{ + char *p, *id; + int rc; + + /* no ids passed actually */ + if (ids[0] == '\0') + return; + + /* add ids specified in the module parameter */ + p = ids; + while ((id = strsep(&p, ","))) { + unsigned int vendor, device, subvendor = PCI_ANY_ID, + subdevice = PCI_ANY_ID, class = 0, class_mask = 0; + int fields; + + if (!strlen(id)) + continue; + + fields = sscanf(id, "%x:%x:%x:%x:%x:%x", + &vendor, &device, &subvendor, &subdevice, + &class, &class_mask); + + if (fields < 2) { + pr_warn("invalid id string \"%s\"\n", id); + continue; + } + + rc = pci_add_dynid(&vfio_pci_driver, vendor, device, + subvendor, subdevice, class, class_mask, 0); + if (rc) + pr_warn("failed to add dynamic id [%04x:%04x[%04x:%04x]] class %#08x/%08x (%d)\n", + vendor, device, subvendor, subdevice, + class, class_mask, rc); + else + pr_info("add [%04x:%04x[%04x:%04x]] class %#08x/%08x\n", + vendor, device, subvendor, subdevice, + class, class_mask); + } +} + +static int __init vfio_pci_init(void) +{ + int ret; + + /* Register and scan for devices */ + ret = pci_register_driver(&vfio_pci_driver); + if (ret) + return ret; + + vfio_pci_fill_ids(); + + if (disable_denylist) + pr_warn("device denylist disabled.\n"); + + return 0; +} + +module_init(vfio_pci_init); +module_exit(vfio_pci_cleanup); + +MODULE_VERSION(DRIVER_VERSION); +MODULE_LICENSE("GPL v2"); +MODULE_AUTHOR(DRIVER_AUTHOR); +MODULE_DESCRIPTION(DRIVER_DESC); diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index 557a03528dcd..878a3609b916 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -23,7 +23,6 @@ #include #include #include -#include #include #include #include @@ -32,11 +31,7 @@ #define DRIVER_VERSION "0.2" #define DRIVER_AUTHOR "Alex Williamson " -#define DRIVER_DESC "VFIO PCI - User Level meta-driver" - -static char ids[1024] __initdata; -module_param_string(ids, ids, sizeof(ids), 0); -MODULE_PARM_DESC(ids, "Initial PCI IDs to add to the vfio driver, format is \"vendor:device[:subvendor[:subdevice[:class[:class_mask]]]]\" and multiple comma separated entries can be specified"); +#define DRIVER_DESC "core driver for VFIO based PCI devices" static bool nointxmask; module_param_named(nointxmask, nointxmask, bool, S_IRUGO | S_IWUSR); @@ -54,16 +49,6 @@ module_param(disable_idle_d3, bool, S_IRUGO | S_IWUSR); MODULE_PARM_DESC(disable_idle_d3, "Disable using the PCI D3 low power state for idle, unused devices"); -static bool enable_sriov; -#ifdef CONFIG_PCI_IOV -module_param(enable_sriov, bool, 0644); -MODULE_PARM_DESC(enable_sriov, "Enable support for SR-IOV configuration. Enabling SR-IOV on a PF typically requires support of the userspace PF driver, enabling VFs without such support may result in non-functional VFs or PF."); -#endif - -static bool disable_denylist; -module_param(disable_denylist, bool, 0444); -MODULE_PARM_DESC(disable_denylist, "Disable use of device denylist. Disabling the denylist allows binding to devices with known errata that may lead to exploitable stability or security issues when accessed by untrusted users."); - static inline bool vfio_vga_disabled(void) { #ifdef CONFIG_VFIO_PCI_VGA @@ -73,44 +58,6 @@ static inline bool vfio_vga_disabled(void) #endif } -static bool vfio_pci_dev_in_denylist(struct pci_dev *pdev) -{ - switch (pdev->vendor) { - case PCI_VENDOR_ID_INTEL: - switch (pdev->device) { - case PCI_DEVICE_ID_INTEL_QAT_C3XXX: - case PCI_DEVICE_ID_INTEL_QAT_C3XXX_VF: - case PCI_DEVICE_ID_INTEL_QAT_C62X: - case PCI_DEVICE_ID_INTEL_QAT_C62X_VF: - case PCI_DEVICE_ID_INTEL_QAT_DH895XCC: - case PCI_DEVICE_ID_INTEL_QAT_DH895XCC_VF: - return true; - default: - return false; - } - } - - return false; -} - -static bool vfio_pci_is_denylisted(struct pci_dev *pdev) -{ - if (!vfio_pci_dev_in_denylist(pdev)) - return false; - - if (disable_denylist) { - pci_warn(pdev, - "device denylist disabled - allowing device %04x:%04x.\n", - pdev->vendor, pdev->device); - return false; - } - - pci_warn(pdev, "%04x:%04x exists in vfio-pci device denylist, driver probing disallowed.\n", - pdev->vendor, pdev->device); - - return true; -} - /* * Our VGA arbiter participation is limited since we don't know anything * about the device itself. However, if the device is the only VGA device @@ -155,7 +102,7 @@ static inline bool vfio_pci_is_vga(struct pci_dev *pdev) return (pdev->class >> 8) == PCI_CLASS_DISPLAY_VGA; } -static void vfio_pci_probe_mmaps(struct vfio_pci_core_device *vdev) +void vfio_pci_probe_mmaps(struct vfio_pci_core_device *vdev) { struct resource *res; int i; @@ -222,6 +169,7 @@ static void vfio_pci_probe_mmaps(struct vfio_pci_core_device *vdev) vdev->bar_mmap_supported[bar] = false; } } +EXPORT_SYMBOL_GPL(vfio_pci_probe_mmaps); static void vfio_pci_try_bus_reset(struct vfio_pci_core_device *vdev); static void vfio_pci_disable(struct vfio_pci_core_device *vdev); @@ -309,7 +257,24 @@ int vfio_pci_set_power_state(struct vfio_pci_core_device *vdev, pci_power_t stat return ret; } -static int vfio_pci_enable(struct vfio_pci_core_device *vdev) +void vfio_pci_core_disable(struct vfio_pci_core_device *vdev) +{ + vfio_pci_disable(vdev); + + mutex_lock(&vdev->igate); + if (vdev->err_trigger) { + eventfd_ctx_put(vdev->err_trigger); + vdev->err_trigger = NULL; + } + if (vdev->req_trigger) { + eventfd_ctx_put(vdev->req_trigger); + vdev->req_trigger = NULL; + } + mutex_unlock(&vdev->igate); +} +EXPORT_SYMBOL_GPL(vfio_pci_core_disable); + +int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) { struct pci_dev *pdev = vdev->pdev; int ret; @@ -407,14 +372,13 @@ static int vfio_pci_enable(struct vfio_pci_core_device *vdev) } } - vfio_pci_probe_mmaps(vdev); - return 0; disable_exit: vfio_pci_disable(vdev); return ret; } +EXPORT_SYMBOL_GPL(vfio_pci_core_enable); static void vfio_pci_disable(struct vfio_pci_core_device *vdev) { @@ -515,8 +479,6 @@ static void vfio_pci_disable(struct vfio_pci_core_device *vdev) vfio_pci_set_power_state(vdev, PCI_D3hot); } -static struct pci_driver vfio_pci_driver; - static struct vfio_pci_core_device *get_pf_vdev(struct vfio_pci_core_device *vdev, struct vfio_device **pf_dev) { @@ -529,7 +491,7 @@ static struct vfio_pci_core_device *get_pf_vdev(struct vfio_pci_core_device *vde if (!*pf_dev) return NULL; - if (pci_dev_driver(physfn) != &vfio_pci_driver) { + if (pci_dev_driver(physfn) != pci_dev_driver(vdev->pdev)) { vfio_device_put(*pf_dev); return NULL; } @@ -537,7 +499,7 @@ static struct vfio_pci_core_device *get_pf_vdev(struct vfio_pci_core_device *vde return vfio_device_data(*pf_dev); } -static void vfio_pci_vf_token_user_add(struct vfio_pci_core_device *vdev, int val) +void vfio_pci_vf_token_user_add(struct vfio_pci_core_device *vdev, int val) { struct vfio_device *pf_dev; struct vfio_pci_core_device *pf_vdev = get_pf_vdev(vdev, &pf_dev); @@ -552,60 +514,19 @@ static void vfio_pci_vf_token_user_add(struct vfio_pci_core_device *vdev, int va vfio_device_put(pf_dev); } +EXPORT_SYMBOL_GPL(vfio_pci_vf_token_user_add); -static void vfio_pci_release(void *device_data) +void vfio_pci_core_spapr_eeh_open(struct vfio_pci_core_device *vdev) { - struct vfio_pci_core_device *vdev = device_data; - - mutex_lock(&vdev->reflck->lock); - - if (!(--vdev->refcnt)) { - vfio_pci_vf_token_user_add(vdev, -1); - vfio_spapr_pci_eeh_release(vdev->pdev); - vfio_pci_disable(vdev); - - mutex_lock(&vdev->igate); - if (vdev->err_trigger) { - eventfd_ctx_put(vdev->err_trigger); - vdev->err_trigger = NULL; - } - if (vdev->req_trigger) { - eventfd_ctx_put(vdev->req_trigger); - vdev->req_trigger = NULL; - } - mutex_unlock(&vdev->igate); - } - - mutex_unlock(&vdev->reflck->lock); - - module_put(THIS_MODULE); + vfio_spapr_pci_eeh_open(vdev->pdev); } +EXPORT_SYMBOL_GPL(vfio_pci_core_spapr_eeh_open); -static int vfio_pci_open(void *device_data) +void vfio_pci_core_spapr_eeh_release(struct vfio_pci_core_device *vpdev) { - struct vfio_pci_core_device *vdev = device_data; - int ret = 0; - - if (!try_module_get(THIS_MODULE)) - return -ENODEV; - - mutex_lock(&vdev->reflck->lock); - - if (!vdev->refcnt) { - ret = vfio_pci_enable(vdev); - if (ret) - goto error; - - vfio_spapr_pci_eeh_open(vdev->pdev); - vfio_pci_vf_token_user_add(vdev, 1); - } - vdev->refcnt++; -error: - mutex_unlock(&vdev->reflck->lock); - if (ret) - module_put(THIS_MODULE); - return ret; + vfio_spapr_pci_eeh_release(vpdev->pdev); } +EXPORT_SYMBOL_GPL(vfio_pci_core_spapr_eeh_release); static int vfio_pci_get_irq_count(struct vfio_pci_core_device *vdev, int irq_type) { @@ -797,8 +718,8 @@ struct vfio_devices { int max_index; }; -static long vfio_pci_ioctl(void *device_data, - unsigned int cmd, unsigned long arg) +long vfio_pci_core_ioctl(void *device_data, unsigned int cmd, + unsigned long arg) { struct vfio_pci_core_device *vdev = device_data; unsigned long minsz; @@ -1401,6 +1322,7 @@ static long vfio_pci_ioctl(void *device_data, return -ENOTTY; } +EXPORT_SYMBOL_GPL(vfio_pci_core_ioctl); static ssize_t vfio_pci_rw(void *device_data, char __user *buf, size_t count, loff_t *ppos, bool iswrite) @@ -1434,23 +1356,25 @@ static ssize_t vfio_pci_rw(void *device_data, char __user *buf, return -EINVAL; } -static ssize_t vfio_pci_read(void *device_data, char __user *buf, - size_t count, loff_t *ppos) +ssize_t vfio_pci_core_read(void *device_data, char __user *buf, size_t count, + loff_t *ppos) { if (!count) return 0; return vfio_pci_rw(device_data, buf, count, ppos, false); } +EXPORT_SYMBOL_GPL(vfio_pci_core_read); -static ssize_t vfio_pci_write(void *device_data, const char __user *buf, - size_t count, loff_t *ppos) +ssize_t vfio_pci_core_write(void *device_data, const char __user *buf, + size_t count, loff_t *ppos) { if (!count) return 0; return vfio_pci_rw(device_data, (char __user *)buf, count, ppos, true); } +EXPORT_SYMBOL_GPL(vfio_pci_core_write); /* Return 1 on zap and vma_lock acquired, 0 on contention (only with @try) */ static int vfio_pci_zap_and_vma_lock(struct vfio_pci_core_device *vdev, bool try) @@ -1646,7 +1570,7 @@ static const struct vm_operations_struct vfio_pci_mmap_ops = { .fault = vfio_pci_mmap_fault, }; -static int vfio_pci_mmap(void *device_data, struct vm_area_struct *vma) +int vfio_pci_core_mmap(void *device_data, struct vm_area_struct *vma) { struct vfio_pci_core_device *vdev = device_data; struct pci_dev *pdev = vdev->pdev; @@ -1713,8 +1637,9 @@ static int vfio_pci_mmap(void *device_data, struct vm_area_struct *vma) return 0; } +EXPORT_SYMBOL_GPL(vfio_pci_core_mmap); -static void vfio_pci_request(void *device_data, unsigned int count) +void vfio_pci_core_request(void *device_data, unsigned int count) { struct vfio_pci_core_device *vdev = device_data; struct pci_dev *pdev = vdev->pdev; @@ -1734,6 +1659,7 @@ static void vfio_pci_request(void *device_data, unsigned int count) mutex_unlock(&vdev->igate); } +EXPORT_SYMBOL_GPL(vfio_pci_core_request); static int vfio_pci_validate_vf_token(struct vfio_pci_core_device *vdev, bool vf_token, uuid_t *uuid) @@ -1830,7 +1756,7 @@ static int vfio_pci_validate_vf_token(struct vfio_pci_core_device *vdev, #define VF_TOKEN_ARG "vf_token=" -static int vfio_pci_match(void *device_data, char *buf) +int vfio_pci_core_match(void *device_data, char *buf) { struct vfio_pci_core_device *vdev = device_data; bool vf_token = false; @@ -1878,18 +1804,7 @@ static int vfio_pci_match(void *device_data, char *buf) return 1; /* Match */ } - -static const struct vfio_device_ops vfio_pci_ops = { - .name = "vfio-pci", - .open = vfio_pci_open, - .release = vfio_pci_release, - .ioctl = vfio_pci_ioctl, - .read = vfio_pci_read, - .write = vfio_pci_write, - .mmap = vfio_pci_mmap, - .request = vfio_pci_request, - .match = vfio_pci_match, -}; +EXPORT_SYMBOL_GPL(vfio_pci_core_match); static int vfio_pci_reflck_attach(struct vfio_pci_core_device *vdev); static void vfio_pci_reflck_put(struct vfio_pci_reflck *reflck); @@ -1908,12 +1823,12 @@ static int vfio_pci_bus_notifier(struct notifier_block *nb, pci_info(vdev->pdev, "Captured SR-IOV VF %s driver_override\n", pci_name(pdev)); pdev->driver_override = kasprintf(GFP_KERNEL, "%s", - vfio_pci_ops.name); + vdev->vfio_pci_ops->name); } else if (action == BUS_NOTIFY_BOUND_DRIVER && pdev->is_virtfn && physfn == vdev->pdev) { struct pci_driver *drv = pci_dev_driver(pdev); - if (drv && drv != &vfio_pci_driver) + if (drv && drv != pci_dev_driver(vdev->pdev)) pci_warn(vdev->pdev, "VF %s bound to driver %s while PF bound to vfio-pci\n", pci_name(pdev), drv->name); @@ -1922,17 +1837,15 @@ static int vfio_pci_bus_notifier(struct notifier_block *nb, return 0; } -static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) +struct vfio_pci_core_device *vfio_create_pci_device(struct pci_dev *pdev, + const struct vfio_device_ops *vfio_pci_ops) { struct vfio_pci_core_device *vdev; struct iommu_group *group; int ret; - if (vfio_pci_is_denylisted(pdev)) - return -EINVAL; - if (pdev->hdr_type != PCI_HEADER_TYPE_NORMAL) - return -EINVAL; + return ERR_PTR(-EINVAL); /* * Prevent binding to PFs with VFs enabled, the VFs might be in use @@ -1944,12 +1857,12 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) */ if (pci_num_vf(pdev)) { pci_warn(pdev, "Cannot bind to PF with SR-IOV enabled\n"); - return -EBUSY; + return ERR_PTR(-EBUSY); } group = vfio_iommu_group_get(&pdev->dev); if (!group) - return -EINVAL; + return ERR_PTR(-EINVAL); vdev = kzalloc(sizeof(*vdev), GFP_KERNEL); if (!vdev) { @@ -1958,6 +1871,7 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) } vdev->pdev = pdev; + vdev->vfio_pci_ops = vfio_pci_ops; vdev->irq_type = VFIO_PCI_NUM_IRQS; mutex_init(&vdev->igate); spin_lock_init(&vdev->irqlock); @@ -1968,7 +1882,7 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) INIT_LIST_HEAD(&vdev->vma_list); init_rwsem(&vdev->memory_lock); - ret = vfio_add_group_dev(&pdev->dev, &vfio_pci_ops, vdev); + ret = vfio_add_group_dev(&pdev->dev, vfio_pci_ops, vdev); if (ret) goto out_free; @@ -2014,7 +1928,7 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) vfio_pci_set_power_state(vdev, PCI_D3hot); } - return ret; + return vdev; out_vf_token: kfree(vdev->vf_token); @@ -2026,10 +1940,11 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) kfree(vdev); out_group_put: vfio_iommu_group_put(group, &pdev->dev); - return ret; + return ERR_PTR(ret); } +EXPORT_SYMBOL_GPL(vfio_create_pci_device); -static void vfio_pci_remove(struct pci_dev *pdev) +void vfio_destroy_pci_device(struct pci_dev *pdev) { struct vfio_pci_core_device *vdev; @@ -2067,9 +1982,10 @@ static void vfio_pci_remove(struct pci_dev *pdev) VGA_RSRC_LEGACY_IO | VGA_RSRC_LEGACY_MEM); } } +EXPORT_SYMBOL_GPL(vfio_destroy_pci_device); -static pci_ers_result_t vfio_pci_aer_err_detected(struct pci_dev *pdev, - pci_channel_state_t state) +static pci_ers_result_t vfio_pci_core_aer_err_detected(struct pci_dev *pdev, + pci_channel_state_t state) { struct vfio_pci_core_device *vdev; struct vfio_device *device; @@ -2096,7 +2012,7 @@ static pci_ers_result_t vfio_pci_aer_err_detected(struct pci_dev *pdev, return PCI_ERS_RESULT_CAN_RECOVER; } -static int vfio_pci_sriov_configure(struct pci_dev *pdev, int nr_virtfn) +int vfio_pci_core_sriov_configure(struct pci_dev *pdev, int nr_virtfn) { struct vfio_pci_core_device *vdev; struct vfio_device *device; @@ -2104,9 +2020,6 @@ static int vfio_pci_sriov_configure(struct pci_dev *pdev, int nr_virtfn) might_sleep(); - if (!enable_sriov) - return -ENOENT; - device = vfio_device_get_from_dev(&pdev->dev); if (!device) return -ENODEV; @@ -2126,19 +2039,12 @@ static int vfio_pci_sriov_configure(struct pci_dev *pdev, int nr_virtfn) return ret < 0 ? ret : nr_virtfn; } +EXPORT_SYMBOL_GPL(vfio_pci_core_sriov_configure); -static const struct pci_error_handlers vfio_err_handlers = { - .error_detected = vfio_pci_aer_err_detected, -}; - -static struct pci_driver vfio_pci_driver = { - .name = "vfio-pci", - .id_table = NULL, /* only dynamic ids */ - .probe = vfio_pci_probe, - .remove = vfio_pci_remove, - .sriov_configure = vfio_pci_sriov_configure, - .err_handler = &vfio_err_handlers, +const struct pci_error_handlers vfio_pci_core_err_handlers = { + .error_detected = vfio_pci_core_aer_err_detected, }; +EXPORT_SYMBOL_GPL(vfio_pci_core_err_handlers); static DEFINE_MUTEX(reflck_lock); @@ -2171,13 +2077,13 @@ static int vfio_pci_reflck_find(struct pci_dev *pdev, void *data) if (!device) return 0; - if (pci_dev_driver(pdev) != &vfio_pci_driver) { + vdev = vfio_device_data(device); + + if (pci_dev_driver(pdev) != pci_dev_driver(vdev->pdev)) { vfio_device_put(device); return 0; } - vdev = vfio_device_data(device); - if (vdev->reflck) { vfio_pci_reflck_get(vdev->reflck); *preflck = vdev->reflck; @@ -2233,13 +2139,13 @@ static int vfio_pci_get_unused_devs(struct pci_dev *pdev, void *data) if (!device) return -EINVAL; - if (pci_dev_driver(pdev) != &vfio_pci_driver) { + vdev = vfio_device_data(device); + + if (pci_dev_driver(pdev) != pci_dev_driver(vdev->pdev)) { vfio_device_put(device); return -EBUSY; } - vdev = vfio_device_data(device); - /* Fault if the device is not unused */ if (vdev->refcnt) { vfio_device_put(device); @@ -2263,13 +2169,13 @@ static int vfio_pci_try_zap_and_vma_lock_cb(struct pci_dev *pdev, void *data) if (!device) return -EINVAL; - if (pci_dev_driver(pdev) != &vfio_pci_driver) { + vdev = vfio_device_data(device); + + if (pci_dev_driver(pdev) != pci_dev_driver(vdev->pdev)) { vfio_device_put(device); return -EBUSY; } - vdev = vfio_device_data(device); - /* * Locking multiple devices is prone to deadlock, runaway and * unwind if we hit contention. @@ -2358,81 +2264,19 @@ static void vfio_pci_try_bus_reset(struct vfio_pci_core_device *vdev) kfree(devs.devices); } -static void __exit vfio_pci_cleanup(void) +static void __exit vfio_pci_core_cleanup(void) { - pci_unregister_driver(&vfio_pci_driver); vfio_pci_uninit_perm_bits(); } -static void __init vfio_pci_fill_ids(void) +static int __init vfio_pci_core_init(void) { - char *p, *id; - int rc; - - /* no ids passed actually */ - if (ids[0] == '\0') - return; - - /* add ids specified in the module parameter */ - p = ids; - while ((id = strsep(&p, ","))) { - unsigned int vendor, device, subvendor = PCI_ANY_ID, - subdevice = PCI_ANY_ID, class = 0, class_mask = 0; - int fields; - - if (!strlen(id)) - continue; - - fields = sscanf(id, "%x:%x:%x:%x:%x:%x", - &vendor, &device, &subvendor, &subdevice, - &class, &class_mask); - - if (fields < 2) { - pr_warn("invalid id string \"%s\"\n", id); - continue; - } - - rc = pci_add_dynid(&vfio_pci_driver, vendor, device, - subvendor, subdevice, class, class_mask, 0); - if (rc) - pr_warn("failed to add dynamic id [%04x:%04x[%04x:%04x]] class %#08x/%08x (%d)\n", - vendor, device, subvendor, subdevice, - class, class_mask, rc); - else - pr_info("add [%04x:%04x[%04x:%04x]] class %#08x/%08x\n", - vendor, device, subvendor, subdevice, - class, class_mask); - } -} - -static int __init vfio_pci_init(void) -{ - int ret; - /* Allocate shared config space permision data used by all devices */ - ret = vfio_pci_init_perm_bits(); - if (ret) - return ret; - - /* Register and scan for devices */ - ret = pci_register_driver(&vfio_pci_driver); - if (ret) - goto out_driver; - - vfio_pci_fill_ids(); - - if (disable_denylist) - pr_warn("device denylist disabled.\n"); - - return 0; - -out_driver: - vfio_pci_uninit_perm_bits(); - return ret; + return vfio_pci_init_perm_bits(); } -module_init(vfio_pci_init); -module_exit(vfio_pci_cleanup); +module_init(vfio_pci_core_init); +module_exit(vfio_pci_core_cleanup); MODULE_VERSION(DRIVER_VERSION); MODULE_LICENSE("GPL v2"); diff --git a/drivers/vfio/pci/vfio_pci_core.h b/drivers/vfio/pci/vfio_pci_core.h index 3964ca898984..a3517a9472bd 100644 --- a/drivers/vfio/pci/vfio_pci_core.h +++ b/drivers/vfio/pci/vfio_pci_core.h @@ -10,6 +10,7 @@ #include #include +#include #include #include #include @@ -100,48 +101,52 @@ struct vfio_pci_mmap_vma { }; struct vfio_pci_core_device { - struct pci_dev *pdev; - void __iomem *barmap[PCI_STD_NUM_BARS]; - bool bar_mmap_supported[PCI_STD_NUM_BARS]; - u8 *pci_config_map; - u8 *vconfig; - struct perm_bits *msi_perm; - spinlock_t irqlock; - struct mutex igate; - struct vfio_pci_irq_ctx *ctx; - int num_ctx; - int irq_type; - int num_regions; - struct vfio_pci_region *region; - u8 msi_qmax; - u8 msix_bar; - u16 msix_size; - u32 msix_offset; - u32 rbar[7]; - bool pci_2_3; - bool virq_disabled; - bool reset_works; - bool extended_caps; - bool bardirty; - bool has_vga; - bool needs_reset; - bool nointx; - bool needs_pm_restore; - struct pci_saved_state *pci_saved_state; - struct pci_saved_state *pm_save; - struct vfio_pci_reflck *reflck; - int refcnt; - int ioeventfds_nr; - struct eventfd_ctx *err_trigger; - struct eventfd_ctx *req_trigger; - struct list_head dummy_resources_list; - struct mutex ioeventfds_lock; - struct list_head ioeventfds_list; + /* below are the public fields used by vfio_pci drivers */ + struct pci_dev *pdev; + const struct vfio_device_ops *vfio_pci_ops; + struct vfio_pci_reflck *reflck; + int refcnt; + struct vfio_pci_region *region; + u8 *pci_config_map; + u8 *vconfig; + + /* below are the private internal fields used by vfio_pci_core */ + void __iomem *barmap[PCI_STD_NUM_BARS]; + bool bar_mmap_supported[PCI_STD_NUM_BARS]; + struct perm_bits *msi_perm; + spinlock_t irqlock; + struct mutex igate; + struct vfio_pci_irq_ctx *ctx; + int num_ctx; + int irq_type; + int num_regions; + u8 msi_qmax; + u8 msix_bar; + u16 msix_size; + u32 msix_offset; + u32 rbar[7]; + bool pci_2_3; + bool virq_disabled; + bool reset_works; + bool extended_caps; + bool bardirty; + bool has_vga; + bool needs_reset; + bool nointx; + bool needs_pm_restore; + struct pci_saved_state *pci_saved_state; + struct pci_saved_state *pm_save; + int ioeventfds_nr; + struct eventfd_ctx *err_trigger; + struct eventfd_ctx *req_trigger; + struct list_head dummy_resources_list; + struct mutex ioeventfds_lock; + struct list_head ioeventfds_list; struct vfio_pci_vf_token *vf_token; - struct notifier_block nb; - struct mutex vma_lock; - struct list_head vma_list; - struct rw_semaphore memory_lock; + struct notifier_block nb; + struct mutex vma_lock; + struct list_head vma_list; + struct rw_semaphore memory_lock; }; #define is_intx(vdev) (vdev->irq_type == VFIO_PCI_INTX_IRQ_INDEX) @@ -225,4 +230,30 @@ static inline int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_device *vdev, } #endif +/* Exported functions */ +struct vfio_pci_core_device *vfio_create_pci_device(struct pci_dev *pdev, + const struct vfio_device_ops *vfio_pci_ops); +void vfio_destroy_pci_device(struct pci_dev *pdev); + +long vfio_pci_core_ioctl(void *device_data, unsigned int cmd, + unsigned long arg); +ssize_t vfio_pci_core_read(void *device_data, char __user *buf, size_t count, + loff_t *ppos); +ssize_t vfio_pci_core_write(void *device_data, const char __user *buf, + size_t count, loff_t *ppos); +int vfio_pci_core_mmap(void *device_data, struct vm_area_struct *vma); +void vfio_pci_core_request(void *device_data, unsigned int count); +int vfio_pci_core_match(void *device_data, char *buf); + +void vfio_pci_core_disable(struct vfio_pci_core_device *vdev); +int vfio_pci_core_enable(struct vfio_pci_core_device *vdev); +void vfio_pci_core_spapr_eeh_open(struct vfio_pci_core_device *vdev); +void vfio_pci_core_spapr_eeh_release(struct vfio_pci_core_device *vdev); +void vfio_pci_vf_token_user_add(struct vfio_pci_core_device *vdev, int val); +void vfio_pci_probe_mmaps(struct vfio_pci_core_device *vdev); + +int vfio_pci_core_sriov_configure(struct pci_dev *pdev, int nr_virtfn); + +extern const struct pci_error_handlers vfio_pci_core_err_handlers; + #endif /* VFIO_PCI_CORE_H */ From patchwork Tue Mar 9 08:33:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Gurtovoy X-Patchwork-Id: 12124387 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B6E6C4332B for ; Tue, 9 Mar 2021 08:35:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DD20265285 for ; Tue, 9 Mar 2021 08:35:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230063AbhCIIe4 (ORCPT ); Tue, 9 Mar 2021 03:34:56 -0500 Received: from mail-dm6nam11on2079.outbound.protection.outlook.com ([40.107.223.79]:41230 "EHLO NAM11-DM6-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S230150AbhCIIeh (ORCPT ); Tue, 9 Mar 2021 03:34:37 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=akyktj/l8MiwQl46le2Y1mC96POhzbkwD6g8UQWVWZTwWufkdFWOCaSUAN6gTShXpd/LhHvvrrKKZbhLv3XGoEKNM0ylK8JppuVDDqsMWAbdaF1CbnsDLAp+eY+C7s98P+UkIQUXSUvQRF/WhlXdJbNX9E12pbG+Le4R9SL85jtkFi/RMHhwxADkIRyOuY08rO8QErwPlE5MGkeojpTYJhvmenlkQzOHSt+eT9EwG67OjN8XiZxKLNlurX4lVChp2ghRC/96XHUgU6SNaxHlyeAZoY5FDkc/3eQXS/eHREMoDZsR0pn+N32qV/ShrM7LxQeyaNmwDvEEaLRk/L4odQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vB9uqE6/X34dyaE1Dfm2VA3wxh8WZp0X3Rc2bxqdFow=; b=IQK4TuUm6Bbuh6buGHrJlxAHBuhivZFjZ2xFJg9MiakalI1cqAuN4qONEdLqbQ/Ab6VygwoFIQyX6qzJho5mMYEvU0unMZfYkTEDSVgUu5s+nv01Q/eRGpExLHMO+D/4xmc5nCKa/Nco4JjVUKWxkRGas952U6RT3j0ItVAsOtJMZx38Cu9A5dpiGc+S2/CVjxjBbFAyHWIz/o3MVmo1+xOrAN1fBdYYYDNQ0sClxQm6R9NBYRoXxU2d5AyUqrwKbsUnWMeqy+1x2i4hMdHRjrxLLwpXA0gLqDrpnglLmVFT7VBN+NOSYJYFUAUJmhb31VWv6rKx9ycA1mwgcedaQg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.35) smtp.rcpttodomain=linux.ibm.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vB9uqE6/X34dyaE1Dfm2VA3wxh8WZp0X3Rc2bxqdFow=; b=Pbn8huwfoMT6LUf9ELgp9k4Sz7ugg/3tB6m/XIvSj2cVbB+3Yldk38skYYwCWwLjCkqZBk1FD17+1af8aOz/IFDymgZil0YY7YwFyzNq7DrSmoX4t+8ecxtzS/yaoTdzXC7TCL4OYYSdqMOhB9g5zcemd9U3mYYjXA1y+26p3LASguciRRc7cbk8cJvzdhMx/auh8yl7T5uI91N3jYVznUOeMiXsv5QCFUM/jy0kaq6Qms5jA6fKocTUV0u/70ZA5UQBgeImtdwprzfUKNouOv76UsdlOA/2eBa1/cXnEDbNlsisttkrqhlMZKsNzxQ0Omc0rBsMOchdttAZ1uYU+w== Received: from DM6PR07CA0075.namprd07.prod.outlook.com (2603:10b6:5:337::8) by BL0PR12MB2337.namprd12.prod.outlook.com (2603:10b6:207:45::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.19; Tue, 9 Mar 2021 08:34:34 +0000 Received: from DM6NAM11FT020.eop-nam11.prod.protection.outlook.com (2603:10b6:5:337:cafe::50) by DM6PR07CA0075.outlook.office365.com (2603:10b6:5:337::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.17 via Frontend Transport; Tue, 9 Mar 2021 08:34:34 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.35) smtp.mailfrom=nvidia.com; linux.ibm.com; dkim=none (message not signed) header.d=none;linux.ibm.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.35 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.35; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.35) by DM6NAM11FT020.mail.protection.outlook.com (10.13.172.224) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.3912.17 via Frontend Transport; Tue, 9 Mar 2021 08:34:34 +0000 Received: from HQMAIL111.nvidia.com (172.20.187.18) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Mar 2021 08:34:33 +0000 Received: from r-nvmx02.mtr.labs.mlnx (172.20.145.6) by mail.nvidia.com (172.20.187.18) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 9 Mar 2021 08:34:27 +0000 From: Max Gurtovoy To: , , , , CC: , , , , , , , , , , , , , , , Max Gurtovoy Subject: [PATCH 5/9] vfio/pci: introduce vfio_pci_device structure Date: Tue, 9 Mar 2021 08:33:53 +0000 Message-ID: <20210309083357.65467-6-mgurtovoy@nvidia.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20210309083357.65467-1-mgurtovoy@nvidia.com> References: <20210309083357.65467-1-mgurtovoy@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b53e8bf9-86ba-47ba-97e0-08d8e2d62aca X-MS-TrafficTypeDiagnostic: BL0PR12MB2337: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3xXXwRHqZhurji66F6O5YDvxm+9MsSaFrTQnP6IkRWu9uEJCJukojBKaxULTg+Hq1AEk/Z5+2OYyOInu3kXusEKQaGhE9PraABmgjkX3igs/ZlZ9p0+gKz8PI9iYPohEgErsCzhLRoQNFrVg1o2DVCdMpUjiPq5amaWIdhArz8tUoFhIh6V5Ngl07C0Ndvkke6nIZEvZgkQgNl7/Tcw/flYUMkmEzuY9iYOw/KmVyMLkUAI3KPJEAxBYBKzAY2HPytyGPTApkHxRmm+SN94E5BYW0TAhpUwairmSbGduuJ5UBhRrg2l1ZO1dI6TL4iuR2J24HVYawItDB9zhjYr1CJEmvd4+xLbK32YHpAJ+KAvwq3GPQNvK/m+bliC7svxucN6kkU9WqjnDqDemAB3fmvD+8LFxEen8t3x2JtLQNnuwLjg9jDzEE9BKUxJh0//EDuZqqoaAvSVX0EgsSw6fQop7LRE2YIUCLJbWYVdA8DvsEZn3bijXIe7B5+lgQZK+DsyOOaSsfRMrciaBYZhPTmUjdSXJnEU+Vkc9t7exVX61t6j6wp40GrV9HTd+kcflMDeoHlabtRMgENzjKmqffqCg3ytzda9XBiegI2vHFXDLWpp6bg4dOZ1vFFEpUUjOQxDWXkNdm0KX0z7Un1H+H6hmNUwkiLSm/Qdd+3kRbMQRUI2jYGEMvM7ls0xpicmg X-Forefront-Antispam-Report: CIP:216.228.112.35;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid04.nvidia.com;CAT:NONE;SFS:(4636009)(346002)(376002)(136003)(396003)(39860400002)(36840700001)(46966006)(478600001)(2616005)(34020700004)(86362001)(186003)(4326008)(5660300002)(1076003)(107886003)(336012)(6666004)(70206006)(47076005)(70586007)(36860700001)(2906002)(83380400001)(8936002)(82740400003)(36756003)(7636003)(8676002)(82310400003)(26005)(54906003)(36906005)(110136005)(316002)(356005)(426003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Mar 2021 08:34:34.0674 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b53e8bf9-86ba-47ba-97e0-08d8e2d62aca X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.35];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT020.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB2337 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This structure will hold the specific attributes for the generic vfio_pci.ko driver. It will be allocated by the vfio_pci driver that will register the vfio subsystem using vfio_pci_core_register_device and it will unregister the using vfio_pci_core_unregister_device. In this way every vfio_pci future driver will be able to use this mechanism to set vendor specific attributes in its vendor specific structure and register to subsystem core while utilizing vfio_pci_core library. This is a standard Linux subsystem behaviour and will also ease on vfio_pci drivers to extend callbacks of vfio_device_ops and will be able to use container_of mechanism as well (instead of passing void pointers as around the stack). Signed-off-by: Max Gurtovoy --- drivers/vfio/pci/vfio_pci.c | 31 +++++++++++++++++++++---- drivers/vfio/pci/vfio_pci_core.c | 39 +++++++++++++------------------- drivers/vfio/pci/vfio_pci_core.h | 5 ++-- 3 files changed, 45 insertions(+), 30 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 447c31f4e64e..dbc0a6559914 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -31,6 +32,10 @@ #define DRIVER_AUTHOR "Alex Williamson " #define DRIVER_DESC "VFIO PCI - User Level meta-driver" +struct vfio_pci_device { + struct vfio_pci_core_device vdev; +}; + static char ids[1024] __initdata; module_param_string(ids, ids, sizeof(ids), 0); MODULE_PARM_DESC(ids, "Initial PCI IDs to add to the vfio driver, format is \"vendor:device[:subvendor[:subdevice[:class[:class_mask]]]]\" and multiple comma separated entries can be specified"); @@ -139,21 +144,37 @@ static const struct vfio_device_ops vfio_pci_ops = { static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) { - struct vfio_pci_core_device *vdev; + struct vfio_pci_device *vpdev; + int ret; if (vfio_pci_is_denylisted(pdev)) return -EINVAL; - vdev = vfio_create_pci_device(pdev, &vfio_pci_ops); - if (IS_ERR(vdev)) - return PTR_ERR(vdev); + vpdev = kzalloc(sizeof(*vpdev), GFP_KERNEL); + if (!vpdev) + return -ENOMEM; + + ret = vfio_pci_core_register_device(&vpdev->vdev, pdev, &vfio_pci_ops); + if (ret) + goto out_free; return 0; + +out_free: + kfree(vpdev); + return ret; } static void vfio_pci_remove(struct pci_dev *pdev) { - vfio_destroy_pci_device(pdev); + struct vfio_device *vdev = dev_get_drvdata(&pdev->dev); + struct vfio_pci_core_device *core_vpdev = vfio_device_data(vdev); + struct vfio_pci_device *vpdev; + + vpdev = container_of(core_vpdev, struct vfio_pci_device, vdev); + + vfio_pci_core_unregister_device(core_vpdev); + kfree(vpdev); } static int vfio_pci_sriov_configure(struct pci_dev *pdev, int nr_virtfn) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index 878a3609b916..7b6be1e4646f 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1837,15 +1837,15 @@ static int vfio_pci_bus_notifier(struct notifier_block *nb, return 0; } -struct vfio_pci_core_device *vfio_create_pci_device(struct pci_dev *pdev, +int vfio_pci_core_register_device(struct vfio_pci_core_device *vdev, + struct pci_dev *pdev, const struct vfio_device_ops *vfio_pci_ops) { - struct vfio_pci_core_device *vdev; struct iommu_group *group; int ret; if (pdev->hdr_type != PCI_HEADER_TYPE_NORMAL) - return ERR_PTR(-EINVAL); + return -EINVAL; /* * Prevent binding to PFs with VFs enabled, the VFs might be in use @@ -1857,18 +1857,12 @@ struct vfio_pci_core_device *vfio_create_pci_device(struct pci_dev *pdev, */ if (pci_num_vf(pdev)) { pci_warn(pdev, "Cannot bind to PF with SR-IOV enabled\n"); - return ERR_PTR(-EBUSY); + return -EBUSY; } group = vfio_iommu_group_get(&pdev->dev); if (!group) - return ERR_PTR(-EINVAL); - - vdev = kzalloc(sizeof(*vdev), GFP_KERNEL); - if (!vdev) { - ret = -ENOMEM; - goto out_group_put; - } + return -EINVAL; vdev->pdev = pdev; vdev->vfio_pci_ops = vfio_pci_ops; @@ -1884,7 +1878,7 @@ struct vfio_pci_core_device *vfio_create_pci_device(struct pci_dev *pdev, ret = vfio_add_group_dev(&pdev->dev, vfio_pci_ops, vdev); if (ret) - goto out_free; + goto out_group_put; ret = vfio_pci_reflck_attach(vdev); if (ret) @@ -1928,7 +1922,7 @@ struct vfio_pci_core_device *vfio_create_pci_device(struct pci_dev *pdev, vfio_pci_set_power_state(vdev, PCI_D3hot); } - return vdev; + return 0; out_vf_token: kfree(vdev->vf_token); @@ -1936,22 +1930,22 @@ struct vfio_pci_core_device *vfio_create_pci_device(struct pci_dev *pdev, vfio_pci_reflck_put(vdev->reflck); out_del_group_dev: vfio_del_group_dev(&pdev->dev); -out_free: - kfree(vdev); out_group_put: vfio_iommu_group_put(group, &pdev->dev); - return ERR_PTR(ret); + return ret; } -EXPORT_SYMBOL_GPL(vfio_create_pci_device); +EXPORT_SYMBOL_GPL(vfio_pci_core_register_device); -void vfio_destroy_pci_device(struct pci_dev *pdev) +void vfio_pci_core_unregister_device(struct vfio_pci_core_device *vdev) { - struct vfio_pci_core_device *vdev; + struct pci_dev *pdev; + struct vfio_pci_core_device *g_vdev; + pdev = vdev->pdev; pci_disable_sriov(pdev); - vdev = vfio_del_group_dev(&pdev->dev); - if (!vdev) + g_vdev = vfio_del_group_dev(&pdev->dev); + if (g_vdev != vdev) return; if (vdev->vf_token) { @@ -1973,7 +1967,6 @@ void vfio_destroy_pci_device(struct pci_dev *pdev) vfio_pci_set_power_state(vdev, PCI_D0); kfree(vdev->pm_save); - kfree(vdev); if (vfio_pci_is_vga(pdev)) { vga_client_register(pdev, NULL, NULL, NULL); @@ -1982,7 +1975,7 @@ void vfio_destroy_pci_device(struct pci_dev *pdev) VGA_RSRC_LEGACY_IO | VGA_RSRC_LEGACY_MEM); } } -EXPORT_SYMBOL_GPL(vfio_destroy_pci_device); +EXPORT_SYMBOL_GPL(vfio_pci_core_unregister_device); static pci_ers_result_t vfio_pci_core_aer_err_detected(struct pci_dev *pdev, pci_channel_state_t state) diff --git a/drivers/vfio/pci/vfio_pci_core.h b/drivers/vfio/pci/vfio_pci_core.h index a3517a9472bd..46eb3443125b 100644 --- a/drivers/vfio/pci/vfio_pci_core.h +++ b/drivers/vfio/pci/vfio_pci_core.h @@ -231,9 +231,10 @@ static inline int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_device *vdev, #endif /* Exported functions */ -struct vfio_pci_core_device *vfio_create_pci_device(struct pci_dev *pdev, +int vfio_pci_core_register_device(struct vfio_pci_core_device *vdev, + struct pci_dev *pdev, const struct vfio_device_ops *vfio_pci_ops); -void vfio_destroy_pci_device(struct pci_dev *pdev); +void vfio_pci_core_unregister_device(struct vfio_pci_core_device *vdev); long vfio_pci_core_ioctl(void *device_data, unsigned int cmd, unsigned long arg); From patchwork Tue Mar 9 08:33:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Gurtovoy X-Patchwork-Id: 12124385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26CEBC43331 for ; Tue, 9 Mar 2021 08:35:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 11C356527C for ; Tue, 9 Mar 2021 08:35:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230473AbhCIIe6 (ORCPT ); Tue, 9 Mar 2021 03:34:58 -0500 Received: from mail-dm6nam12on2087.outbound.protection.outlook.com ([40.107.243.87]:26880 "EHLO NAM12-DM6-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S230422AbhCIIep (ORCPT ); Tue, 9 Mar 2021 03:34:45 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WQtKBiNv0K6lG2AR9549zkMw6ZkxX+jjHxdCXoqy/felk4NXbY3ByGgSJZrLA63UOxxa7VoSzH6gVd7oYQydl2o3peM96FjzK8uRYjCWD6r2jniFCrl7QGacNtil8M1fwPEimWdpmtbvhZb7xMPeDXM/AIfmTX28U0uZxaUAY6XZLpHjujhxZ0PB5IuHJVJOoYtVi+A68HRAkowSsF1PXP2FwVnIgmkyIEzFrTe6wB7yYDamH2usbYSZA7BTG44RcCG1My0LXqx7raRiqb6QJapHJGxoPUqMVkjNSP4/GpWVYWJd9B97jm/P7Mn6RnWQiyJsAOEWfpiFnu4Jpf15xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=44gkFOCvrgF+8Q9lSNHWLK6fQadlxTEs2tqxzuwl8pU=; b=apJI9t5UwIKiEmBDBW1IbPGHcxpKfz8Y0jOuamgWpvfX7HIB2/GVapul5La/dbFVh+FKV/C2MTGYAXd9Y2EQr4Z0fdR+IMmeCryoIJz2FndiB/xmGJQ6yKPeUY/imNh7ass+aXOS3lB+Tarx16W586J/08T1DoYjYREm5WfGvC+o9qAzpEalVnR2YAVtblCZ4Xrst26GD6lsgkT8HKZJ3wdaEVSZ6PkhXo+RDNoI6LTcC5kzlVFeAuMzfF/aqxE4CPU7wj7Kouy3fliRfK/ZRL+JggAYb4NrHIIrjfJeHnLgsE8X6KIdKisKHUc7l9lhz/R1euBfV28Fg1sJLEFlow== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.32) smtp.rcpttodomain=linux.ibm.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=44gkFOCvrgF+8Q9lSNHWLK6fQadlxTEs2tqxzuwl8pU=; b=SNXxpB8DaUglPyhQKoGQSzpc6NwpzFVD5pcz5qPzbxuWLZulD3od5oQW34FxzBIGPuTXUvaHQaEw7om1+UQhwMx1KmoroRTAv4dy/oZibwyh05Pl+fqdgeYk1Or7bKkTkTm9dmCgm4MXLeZbfYotEPDn9UR0Q6Qmo+R51Kg79eEhIeYU+AN7aYOoQOqO0JtO9GKOEZO6+fxkhOcGYVEACfO96WXORYsC6Fp26meAxBVTdcZVaF1zeIBImtuAIhbhSEhT8qxiZFhoFc4s51CMkbDO+gLMrJ8vhBM/t0Rd1pSyPi4Ary2ByncHWqdY1zFFQMjYKmLCDYf1ri/UVMJW0g== Received: from BN6PR12CA0043.namprd12.prod.outlook.com (2603:10b6:405:70::29) by MWHPR12MB1632.namprd12.prod.outlook.com (2603:10b6:301:10::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.17; Tue, 9 Mar 2021 08:34:41 +0000 Received: from BN8NAM11FT004.eop-nam11.prod.protection.outlook.com (2603:10b6:405:70:cafe::61) by BN6PR12CA0043.outlook.office365.com (2603:10b6:405:70::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.17 via Frontend Transport; Tue, 9 Mar 2021 08:34:40 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.32) smtp.mailfrom=nvidia.com; linux.ibm.com; dkim=none (message not signed) header.d=none;linux.ibm.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.32 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.32; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.32) by BN8NAM11FT004.mail.protection.outlook.com (10.13.176.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.3912.17 via Frontend Transport; Tue, 9 Mar 2021 08:34:40 +0000 Received: from HQMAIL111.nvidia.com (172.20.187.18) by HQMAIL109.nvidia.com (172.20.187.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Mar 2021 08:34:39 +0000 Received: from r-nvmx02.mtr.labs.mlnx (172.20.145.6) by mail.nvidia.com (172.20.187.18) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 9 Mar 2021 08:34:33 +0000 From: Max Gurtovoy To: , , , , CC: , , , , , , , , , , , , , , , Max Gurtovoy Subject: [PATCH 6/9] vfio-pci-core: export vfio_pci_register_dev_region function Date: Tue, 9 Mar 2021 08:33:54 +0000 Message-ID: <20210309083357.65467-7-mgurtovoy@nvidia.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20210309083357.65467-1-mgurtovoy@nvidia.com> References: <20210309083357.65467-1-mgurtovoy@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 743bc449-4e82-4f16-6000-08d8e2d62e75 X-MS-TrafficTypeDiagnostic: MWHPR12MB1632: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kr1EHgJkl4iEHVvHbGHzGEeu95ZGpOUYTd9+BfNZvuZhYoTV5cd5fdmgcy3iPNPhmhvuFTe5e/97UH4WLhm/LZKIggYCDipsMr0gMyjkbEF1XSjbyp8lkkJ/sYqdi0vV+5HgkEhEpfHD3NahuaGngPh+TftAg7pG+cNO5x+9ySre3CJy8aqjFBursTb+L1E9FjZsU6mjfNu8k0OluRBUtGbCOwpoOvV3hx7NEpjJRnMm1QOHtIFu1bCLCVc/U0BM/k1hYNXDU7osJHn+EFFgm4QC9nansKe68rJnE/Mw1HhmhNMLenYhmUg6zC+POhVNgMxnZ6YWnVhGJ5T2RxF+ciOsjF7xrwUwAtTs3rP/Z5JyEQoMmgEOlpYPCdY5vayLLPHhI4aQFsbfVnsniUodQMy8ZWZeXYt8Z/HqVjwZRe6piHzQUcujD5qgP1ooHB5zhtwfQ601Xq/KITc6Q9iJ5+fCqJsf3nWTMTLYtrjvKHTTLFI9iMWZsVYlCXWodGb+FVJ9lsh6od1Yr6xUaBVdK2dcxTdil5O0TjPeDo2V65CHY6GwOZl2XRCC8GiWe+dwtlYFzHu7ju+N9tcjkS76pXFJ4LvxgapeViGT7a8kQYUuDSI0bbong1T8ASwI/5qrwMFSg9XJRS0Aw5b8I7AjHw3V1Edaf3ESTq+bbCUzgAD3R7egDNa7QNxDLkw+JnNF X-Forefront-Antispam-Report: CIP:216.228.112.32;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid01.nvidia.com;CAT:NONE;SFS:(4636009)(396003)(136003)(346002)(376002)(39860400002)(36840700001)(46966006)(426003)(186003)(5660300002)(336012)(1076003)(8936002)(36756003)(47076005)(86362001)(83380400001)(2616005)(6666004)(26005)(107886003)(4326008)(70206006)(356005)(82310400003)(8676002)(2906002)(478600001)(54906003)(110136005)(36860700001)(82740400003)(316002)(7636003)(34020700004)(70586007);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Mar 2021 08:34:40.1719 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 743bc449-4e82-4f16-6000-08d8e2d62e75 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.32];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT004.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR12MB1632 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This function will be used to allow vendor drivers to register regions to be used and accessed by the core subsystem driver. This way, the core will use the region ops that are vendor specific and managed by the vendor vfio-pci driver. Next step that can be made is to move the logic of igd and nvlink2 to a dedicated module instead of managing their vendor specific extensions in the core driver. Signed-off-by: Max Gurtovoy --- drivers/vfio/pci/vfio_pci_core.c | 1 + drivers/vfio/pci/vfio_pci_core.h | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index 7b6be1e4646f..ba5dd4321487 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -711,6 +711,7 @@ int vfio_pci_register_dev_region(struct vfio_pci_core_device *vdev, return 0; } +EXPORT_SYMBOL_GPL(vfio_pci_register_dev_region); struct vfio_devices { struct vfio_device **devices; diff --git a/drivers/vfio/pci/vfio_pci_core.h b/drivers/vfio/pci/vfio_pci_core.h index 46eb3443125b..60b42df6c519 100644 --- a/drivers/vfio/pci/vfio_pci_core.h +++ b/drivers/vfio/pci/vfio_pci_core.h @@ -257,4 +257,9 @@ int vfio_pci_core_sriov_configure(struct pci_dev *pdev, int nr_virtfn); extern const struct pci_error_handlers vfio_pci_core_err_handlers; +int vfio_pci_register_dev_region(struct vfio_pci_core_device *vdev, + unsigned int type, unsigned int subtype, + const struct vfio_pci_regops *ops, + size_t size, u32 flags, void *data); + #endif /* VFIO_PCI_CORE_H */ From patchwork Tue Mar 9 08:33:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Gurtovoy X-Patchwork-Id: 12124393 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLACK,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4895DC43333 for ; Tue, 9 Mar 2021 08:35:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3359F6527C for ; Tue, 9 Mar 2021 08:35:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230516AbhCIIfA (ORCPT ); Tue, 9 Mar 2021 03:35:00 -0500 Received: from mail-eopbgr690044.outbound.protection.outlook.com ([40.107.69.44]:62646 "EHLO NAM04-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S230423AbhCIIet (ORCPT ); Tue, 9 Mar 2021 03:34:49 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fok+swWHigMNYe5NMDRxV+07PLgnGBD/sO/chj0ORagLP1jAsMW6yhO2+O/sNOARVOA5cHxpbEzg68dn6tYyz6XrsM1Lt18mQYc3t4RA2UFhS4g6wgFVZM9jJo1EDbGAqsEmGlFYc0sSk6MKyYScl+lcC9CyRaV2MyGmJ4tPjPsgfCrjszLoAWdFAn+v80NvmOjLuOJYV8i1fOgfFDbeSc1U8RIqaHAHvK3FR/vKPXsVauhIpeRYdWcrZ9xYuIzv07YxcZloC84Q/jRLFuXMztdoFCo6lfW+3gFOpHLpQt2Ab6GGKs7YVOiZqeigh1Dnsp6kHiq2oHOZgwVxX4dEqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=m9KtQWql1FKBYN5QGXII2v3gvk8qDoRjBpk/llyvrrw=; b=Cb+l2p2UqOnn7yMTQRnZ0y2j9ZqhefCQKuZwKtz/SIMBYEoVPurVlZqJ9+KINCyfoQhNNIrCG4Hu1mRLf/kNTMOX0gynqggh/ZC11a92cd3RpFvVbjnqhO/PV3NFXJQmH8qY2/NPHqpjQDJJ6WM2i8qnSftMNoRT3Dzax6L4yc7stjhVUnW2RvMZPC0KzbEQfeS6fMxFrhLqu2HTFjop8CJoAU9POgAZkd90Mus3f+7G6m3F1C1a089K+SowEiB2Oi+OwL/Lz9g9NG6Bfdzd5P5RNpbzPYQ2dbfZroIzlg0ZraDXilGAPBrKwlz3bp2JEqTTX4QnB3EAkScxepfcVg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=linux.ibm.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=m9KtQWql1FKBYN5QGXII2v3gvk8qDoRjBpk/llyvrrw=; b=TB4ig42CdXC9ROuFV4l4NkQeqwvxhBjodFG8ENT3WWJENP3xfQ3G5vFpxOICZpV4eD3hkTaCOIke81Zz5jMlebAaCbxqooU5NyuqtYIGAFdM2gx/1f8S80lAh9buEZl8O1LsFOW9gIX2maeT/VBAlpUvGa/w5dd27S4/KLtV8sB/qPCd0BoJoP1RBolQ6pL/Z/8G5YQyehmQaU4zvLniWN6gpffbHbxg98qDhQYEcgX4l1i5l1uW3j2DRN3Ek6yn39N7DrEeiN29N7ATLB9NGwgxpGYWGmHKWDWQXrrzqVxrOzRuq/kTGEdSmzt4QSo+2fPCdx/hOC6x3yVS9No7aA== Received: from DS7PR05CA0022.namprd05.prod.outlook.com (2603:10b6:5:3b9::27) by BYAPR12MB3541.namprd12.prod.outlook.com (2603:10b6:a03:13c::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.26; Tue, 9 Mar 2021 08:34:46 +0000 Received: from DM6NAM11FT031.eop-nam11.prod.protection.outlook.com (2603:10b6:5:3b9:cafe::77) by DS7PR05CA0022.outlook.office365.com (2603:10b6:5:3b9::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3933.13 via Frontend Transport; Tue, 9 Mar 2021 08:34:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; linux.ibm.com; dkim=none (message not signed) header.d=none;linux.ibm.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT031.mail.protection.outlook.com (10.13.172.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.3912.17 via Frontend Transport; Tue, 9 Mar 2021 08:34:45 +0000 Received: from HQMAIL111.nvidia.com (172.20.187.18) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Mar 2021 08:34:45 +0000 Received: from r-nvmx02.mtr.labs.mlnx (172.20.145.6) by mail.nvidia.com (172.20.187.18) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 9 Mar 2021 08:34:39 +0000 From: Max Gurtovoy To: , , , , CC: , , , , , , , , , , , , , , , Max Gurtovoy Subject: [PATCH 7/9] vfio/pci_core: split nvlink2 to nvlink2gpu and npu2 Date: Tue, 9 Mar 2021 08:33:55 +0000 Message-ID: <20210309083357.65467-8-mgurtovoy@nvidia.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20210309083357.65467-1-mgurtovoy@nvidia.com> References: <20210309083357.65467-1-mgurtovoy@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f37aadff-4063-4bcb-988f-08d8e2d631a8 X-MS-TrafficTypeDiagnostic: BYAPR12MB3541: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6430; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: aJXvtTBtEIUQ6V/ujbMGy/1th455geuHAAnLNZpggkpU7lhCpAA7Zk+vu//dTVPT+z12YZYz2gQWlA3MD+V7YRHgaxioOATmx9MzJ9SLocYHO+6CyxJaO0TjNr5glQSQ8W/6TIBETA9R/g0CgJxqySQ3I4bCAR9hn6bOOxS6K0pMRdbXlpB+HuhmYK0ra99QOSbLlpY3uHxJpuGDo4bWYOYiSavjI7o+taGVoYSFeKQzguNvZcCPe2VtIrNUNEnJrKOg0ChcQ+7kKRrK/3yxpD3BTuFzjKkhWeCYJB+2Y6o/HgQojrkQ0kJJWClqOId+HaB9pMkzVbSWZ/6pnlq8sP/XxNtVuqQat2Jr9aNx51i/Hhlgt3lw4FKNY/S+tMWDtdCJJvTxttH3+o1kk7UJ5vDz4viK8VjbL4YPxGds9h78SQXUiqP5r1E5CCGh8gTm9f3Ar4+VvXc8vlS0wUO1LEOZl9xwRRJIdEIOGqsx3UvPAfeM9hKaG5RoDAQn4dZDJybvSCAq4P+DtMqs0QAEyGLmJrQDYobpUOGu3wqkeH6wrgoUBvdzFhETYLNEDIXArmrbUT0rdvBy6VODbmWMlVzusrz0QVVg+PbDMTdhermVIehXX0PrQLYPB2fq7d656JXM3Cfdtk/7Di18Ssso1NSP1ih1mROaoLDYjRV5xw5xUCBWuAyK3CPlRkRjh2SJ X-Forefront-Antispam-Report: CIP:216.228.112.34;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid03.nvidia.com;CAT:NONE;SFS:(4636009)(396003)(376002)(346002)(136003)(39860400002)(36840700001)(46966006)(5660300002)(70206006)(47076005)(478600001)(82310400003)(26005)(86362001)(2616005)(2906002)(36756003)(34020700004)(356005)(6666004)(83380400001)(54906003)(186003)(110136005)(336012)(107886003)(70586007)(8676002)(4326008)(30864003)(316002)(82740400003)(36906005)(36860700001)(426003)(1076003)(7636003)(8936002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Mar 2021 08:34:45.5517 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f37aadff-4063-4bcb-988f-08d8e2d631a8 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.34];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT031.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB3541 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org This is a preparation for moving vendor specific code from vfio_pci_core to vendor specific vfio_pci drivers. The next step will be creating a dedicated module to NVIDIA NVLINK2 devices with P9 extensions and a dedicated module for Power9 NPU NVLink2 HBAs. Signed-off-by: Max Gurtovoy --- drivers/vfio/pci/Makefile | 2 +- drivers/vfio/pci/npu2_trace.h | 50 ++++ .../vfio/pci/{trace.h => nvlink2gpu_trace.h} | 27 +-- drivers/vfio/pci/vfio_pci_core.c | 2 +- drivers/vfio/pci/vfio_pci_core.h | 4 +- drivers/vfio/pci/vfio_pci_npu2.c | 222 ++++++++++++++++++ ...io_pci_nvlink2.c => vfio_pci_nvlink2gpu.c} | 201 +--------------- 7 files changed, 280 insertions(+), 228 deletions(-) create mode 100644 drivers/vfio/pci/npu2_trace.h rename drivers/vfio/pci/{trace.h => nvlink2gpu_trace.h} (72%) create mode 100644 drivers/vfio/pci/vfio_pci_npu2.c rename drivers/vfio/pci/{vfio_pci_nvlink2.c => vfio_pci_nvlink2gpu.c} (59%) diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile index 16e7d77d63ce..f539f32c9296 100644 --- a/drivers/vfio/pci/Makefile +++ b/drivers/vfio/pci/Makefile @@ -5,7 +5,7 @@ obj-$(CONFIG_VFIO_PCI) += vfio-pci.o vfio-pci-core-y := vfio_pci_core.o vfio_pci_intrs.o vfio_pci_rdwr.o vfio_pci_config.o vfio-pci-core-$(CONFIG_VFIO_PCI_IGD) += vfio_pci_igd.o -vfio-pci-core-$(CONFIG_VFIO_PCI_NVLINK2) += vfio_pci_nvlink2.o +vfio-pci-core-$(CONFIG_VFIO_PCI_NVLINK2) += vfio_pci_nvlink2gpu.o vfio_pci_npu2.o vfio-pci-core-$(CONFIG_S390) += vfio_pci_zdev.o vfio-pci-y := vfio_pci.o diff --git a/drivers/vfio/pci/npu2_trace.h b/drivers/vfio/pci/npu2_trace.h new file mode 100644 index 000000000000..c8a1110132dc --- /dev/null +++ b/drivers/vfio/pci/npu2_trace.h @@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * VFIO PCI mmap/mmap_fault tracepoints + * + * Copyright (C) 2018 IBM Corp. All rights reserved. + * Author: Alexey Kardashevskiy + */ + +#undef TRACE_SYSTEM +#define TRACE_SYSTEM vfio_pci + +#if !defined(_TRACE_VFIO_PCI_H) || defined(TRACE_HEADER_MULTI_READ) +#define _TRACE_VFIO_PCI_H + +#include + +TRACE_EVENT(vfio_pci_npu2_mmap, + TP_PROTO(struct pci_dev *pdev, unsigned long hpa, unsigned long ua, + unsigned long size, int ret), + TP_ARGS(pdev, hpa, ua, size, ret), + + TP_STRUCT__entry( + __field(const char *, name) + __field(unsigned long, hpa) + __field(unsigned long, ua) + __field(unsigned long, size) + __field(int, ret) + ), + + TP_fast_assign( + __entry->name = dev_name(&pdev->dev), + __entry->hpa = hpa; + __entry->ua = ua; + __entry->size = size; + __entry->ret = ret; + ), + + TP_printk("%s: %lx -> %lx size=%lx ret=%d", __entry->name, __entry->hpa, + __entry->ua, __entry->size, __entry->ret) +); + +#endif /* _TRACE_VFIO_PCI_H */ + +#undef TRACE_INCLUDE_PATH +#define TRACE_INCLUDE_PATH ../../drivers/vfio/pci +#undef TRACE_INCLUDE_FILE +#define TRACE_INCLUDE_FILE npu2_trace + +/* This part must be outside protection */ +#include diff --git a/drivers/vfio/pci/trace.h b/drivers/vfio/pci/nvlink2gpu_trace.h similarity index 72% rename from drivers/vfio/pci/trace.h rename to drivers/vfio/pci/nvlink2gpu_trace.h index b2aa986ab9ed..2392b9d4c6c9 100644 --- a/drivers/vfio/pci/trace.h +++ b/drivers/vfio/pci/nvlink2gpu_trace.h @@ -62,37 +62,12 @@ TRACE_EVENT(vfio_pci_nvgpu_mmap, __entry->ua, __entry->size, __entry->ret) ); -TRACE_EVENT(vfio_pci_npu2_mmap, - TP_PROTO(struct pci_dev *pdev, unsigned long hpa, unsigned long ua, - unsigned long size, int ret), - TP_ARGS(pdev, hpa, ua, size, ret), - - TP_STRUCT__entry( - __field(const char *, name) - __field(unsigned long, hpa) - __field(unsigned long, ua) - __field(unsigned long, size) - __field(int, ret) - ), - - TP_fast_assign( - __entry->name = dev_name(&pdev->dev), - __entry->hpa = hpa; - __entry->ua = ua; - __entry->size = size; - __entry->ret = ret; - ), - - TP_printk("%s: %lx -> %lx size=%lx ret=%d", __entry->name, __entry->hpa, - __entry->ua, __entry->size, __entry->ret) -); - #endif /* _TRACE_VFIO_PCI_H */ #undef TRACE_INCLUDE_PATH #define TRACE_INCLUDE_PATH ../../drivers/vfio/pci #undef TRACE_INCLUDE_FILE -#define TRACE_INCLUDE_FILE trace +#define TRACE_INCLUDE_FILE nvlink2gpu_trace /* This part must be outside protection */ #include diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index ba5dd4321487..4de8e352df9c 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -356,7 +356,7 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) if (pdev->vendor == PCI_VENDOR_ID_NVIDIA && IS_ENABLED(CONFIG_VFIO_PCI_NVLINK2)) { - ret = vfio_pci_nvdia_v100_nvlink2_init(vdev); + ret = vfio_pci_nvidia_v100_nvlink2_init(vdev); if (ret && ret != -ENODEV) { pci_warn(pdev, "Failed to setup NVIDIA NV2 RAM region\n"); goto disable_exit; diff --git a/drivers/vfio/pci/vfio_pci_core.h b/drivers/vfio/pci/vfio_pci_core.h index 60b42df6c519..8989443c3086 100644 --- a/drivers/vfio/pci/vfio_pci_core.h +++ b/drivers/vfio/pci/vfio_pci_core.h @@ -205,10 +205,10 @@ static inline int vfio_pci_igd_init(struct vfio_pci_core_device *vdev) } #endif #ifdef CONFIG_VFIO_PCI_NVLINK2 -extern int vfio_pci_nvdia_v100_nvlink2_init(struct vfio_pci_core_device *vdev); +extern int vfio_pci_nvidia_v100_nvlink2_init(struct vfio_pci_core_device *vdev); extern int vfio_pci_ibm_npu2_init(struct vfio_pci_core_device *vdev); #else -static inline int vfio_pci_nvdia_v100_nvlink2_init(struct vfio_pci_core_device *vdev) +static inline int vfio_pci_nvidia_v100_nvlink2_init(struct vfio_pci_core_device *vdev) { return -ENODEV; } diff --git a/drivers/vfio/pci/vfio_pci_npu2.c b/drivers/vfio/pci/vfio_pci_npu2.c new file mode 100644 index 000000000000..717745256ab3 --- /dev/null +++ b/drivers/vfio/pci/vfio_pci_npu2.c @@ -0,0 +1,222 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * VFIO PCI driver for POWER9 NPU support (NVLink2 host bus adapter). + * + * Copyright (c) 2020, Mellanox Technologies, Ltd. All rights reserved. + * + * Copyright (C) 2018 IBM Corp. All rights reserved. + * Author: Alexey Kardashevskiy + * + * Register an on-GPU RAM region for cacheable access. + * + * Derived from original vfio_pci_igd.c: + * Copyright (C) 2016 Red Hat, Inc. All rights reserved. + * Author: Alex Williamson + */ + +#include +#include +#include +#include +#include +#include +#include + +#include "vfio_pci_core.h" + +#define CREATE_TRACE_POINTS +#include "npu2_trace.h" + +EXPORT_TRACEPOINT_SYMBOL_GPL(vfio_pci_npu2_mmap); + +struct vfio_pci_npu2_data { + void *base; /* ATSD register virtual address, for emulated access */ + unsigned long mmio_atsd; /* ATSD physical address */ + unsigned long gpu_tgt; /* TGT address of corresponding GPU RAM */ + unsigned int link_speed; /* The link speed from DT's ibm,nvlink-speed */ +}; + +static size_t vfio_pci_npu2_rw(struct vfio_pci_core_device *vdev, + char __user *buf, size_t count, loff_t *ppos, bool iswrite) +{ + unsigned int i = VFIO_PCI_OFFSET_TO_INDEX(*ppos) - VFIO_PCI_NUM_REGIONS; + struct vfio_pci_npu2_data *data = vdev->region[i].data; + loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK; + + if (pos >= vdev->region[i].size) + return -EINVAL; + + count = min(count, (size_t)(vdev->region[i].size - pos)); + + if (iswrite) { + if (copy_from_user(data->base + pos, buf, count)) + return -EFAULT; + } else { + if (copy_to_user(buf, data->base + pos, count)) + return -EFAULT; + } + *ppos += count; + + return count; +} + +static int vfio_pci_npu2_mmap(struct vfio_pci_core_device *vdev, + struct vfio_pci_region *region, struct vm_area_struct *vma) +{ + int ret; + struct vfio_pci_npu2_data *data = region->data; + unsigned long req_len = vma->vm_end - vma->vm_start; + + if (req_len != PAGE_SIZE) + return -EINVAL; + + vma->vm_flags |= VM_PFNMAP; + vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); + + ret = remap_pfn_range(vma, vma->vm_start, data->mmio_atsd >> PAGE_SHIFT, + req_len, vma->vm_page_prot); + trace_vfio_pci_npu2_mmap(vdev->pdev, data->mmio_atsd, vma->vm_start, + vma->vm_end - vma->vm_start, ret); + + return ret; +} + +static void vfio_pci_npu2_release(struct vfio_pci_core_device *vdev, + struct vfio_pci_region *region) +{ + struct vfio_pci_npu2_data *data = region->data; + + memunmap(data->base); + kfree(data); +} + +static int vfio_pci_npu2_add_capability(struct vfio_pci_core_device *vdev, + struct vfio_pci_region *region, struct vfio_info_cap *caps) +{ + struct vfio_pci_npu2_data *data = region->data; + struct vfio_region_info_cap_nvlink2_ssatgt captgt = { + .header.id = VFIO_REGION_INFO_CAP_NVLINK2_SSATGT, + .header.version = 1, + .tgt = data->gpu_tgt + }; + struct vfio_region_info_cap_nvlink2_lnkspd capspd = { + .header.id = VFIO_REGION_INFO_CAP_NVLINK2_LNKSPD, + .header.version = 1, + .link_speed = data->link_speed + }; + int ret; + + ret = vfio_info_add_capability(caps, &captgt.header, sizeof(captgt)); + if (ret) + return ret; + + return vfio_info_add_capability(caps, &capspd.header, sizeof(capspd)); +} + +static const struct vfio_pci_regops vfio_pci_npu2_regops = { + .rw = vfio_pci_npu2_rw, + .mmap = vfio_pci_npu2_mmap, + .release = vfio_pci_npu2_release, + .add_capability = vfio_pci_npu2_add_capability, +}; + +int vfio_pci_ibm_npu2_init(struct vfio_pci_core_device *vdev) +{ + int ret; + struct vfio_pci_npu2_data *data; + struct device_node *nvlink_dn; + u32 nvlink_index = 0, mem_phandle = 0; + struct pci_dev *npdev = vdev->pdev; + struct device_node *npu_node = pci_device_to_OF_node(npdev); + struct pci_controller *hose = pci_bus_to_host(npdev->bus); + u64 mmio_atsd = 0; + u64 tgt = 0; + u32 link_speed = 0xff; + + /* + * PCI config space does not tell us about NVLink presense but + * platform does, use this. + */ + if (!pnv_pci_get_gpu_dev(vdev->pdev)) + return -ENODEV; + + if (of_property_read_u32(npu_node, "memory-region", &mem_phandle)) + return -ENODEV; + + /* + * NPU2 normally has 8 ATSD registers (for concurrency) and 6 links + * so we can allocate one register per link, using nvlink index as + * a key. + * There is always at least one ATSD register so as long as at least + * NVLink bridge #0 is passed to the guest, ATSD will be available. + */ + nvlink_dn = of_parse_phandle(npdev->dev.of_node, "ibm,nvlink", 0); + if (WARN_ON(of_property_read_u32(nvlink_dn, "ibm,npu-link-index", + &nvlink_index))) + return -ENODEV; + + if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index, + &mmio_atsd)) { + if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0, + &mmio_atsd)) { + dev_warn(&vdev->pdev->dev, "No available ATSD found\n"); + mmio_atsd = 0; + } else { + dev_warn(&vdev->pdev->dev, + "Using fallback ibm,mmio-atsd[0] for ATSD.\n"); + } + } + + if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) { + dev_warn(&vdev->pdev->dev, "No ibm,device-tgt-addr found\n"); + return -EFAULT; + } + + if (of_property_read_u32(npu_node, "ibm,nvlink-speed", &link_speed)) { + dev_warn(&vdev->pdev->dev, "No ibm,nvlink-speed found\n"); + return -EFAULT; + } + + data = kzalloc(sizeof(*data), GFP_KERNEL); + if (!data) + return -ENOMEM; + + data->mmio_atsd = mmio_atsd; + data->gpu_tgt = tgt; + data->link_speed = link_speed; + if (data->mmio_atsd) { + data->base = memremap(data->mmio_atsd, SZ_64K, MEMREMAP_WT); + if (!data->base) { + ret = -ENOMEM; + goto free_exit; + } + } + + /* + * We want to expose the capability even if this specific NVLink + * did not get its own ATSD register because capabilities + * belong to VFIO regions and normally there will be ATSD register + * assigned to the NVLink bridge. + */ + ret = vfio_pci_register_dev_region(vdev, + PCI_VENDOR_ID_IBM | + VFIO_REGION_TYPE_PCI_VENDOR_TYPE, + VFIO_REGION_SUBTYPE_IBM_NVLINK2_ATSD, + &vfio_pci_npu2_regops, + data->mmio_atsd ? PAGE_SIZE : 0, + VFIO_REGION_INFO_FLAG_READ | + VFIO_REGION_INFO_FLAG_WRITE | + VFIO_REGION_INFO_FLAG_MMAP, + data); + if (ret) + goto free_exit; + + return 0; + +free_exit: + if (data->base) + memunmap(data->base); + kfree(data); + + return ret; +} diff --git a/drivers/vfio/pci/vfio_pci_nvlink2.c b/drivers/vfio/pci/vfio_pci_nvlink2gpu.c similarity index 59% rename from drivers/vfio/pci/vfio_pci_nvlink2.c rename to drivers/vfio/pci/vfio_pci_nvlink2gpu.c index 8ef2c62a9d27..6dce1e78ee82 100644 --- a/drivers/vfio/pci/vfio_pci_nvlink2.c +++ b/drivers/vfio/pci/vfio_pci_nvlink2gpu.c @@ -19,14 +19,14 @@ #include #include #include + #include "vfio_pci_core.h" #define CREATE_TRACE_POINTS -#include "trace.h" +#include "nvlink2gpu_trace.h" EXPORT_TRACEPOINT_SYMBOL_GPL(vfio_pci_nvgpu_mmap_fault); EXPORT_TRACEPOINT_SYMBOL_GPL(vfio_pci_nvgpu_mmap); -EXPORT_TRACEPOINT_SYMBOL_GPL(vfio_pci_npu2_mmap); struct vfio_pci_nvgpu_data { unsigned long gpu_hpa; /* GPU RAM physical address */ @@ -207,7 +207,7 @@ static int vfio_pci_nvgpu_group_notifier(struct notifier_block *nb, return NOTIFY_OK; } -int vfio_pci_nvdia_v100_nvlink2_init(struct vfio_pci_core_device *vdev) +int vfio_pci_nvidia_v100_nvlink2_init(struct vfio_pci_core_device *vdev) { int ret; u64 reg[2]; @@ -293,198 +293,3 @@ int vfio_pci_nvdia_v100_nvlink2_init(struct vfio_pci_core_device *vdev) return ret; } - -/* - * IBM NPU2 bridge - */ -struct vfio_pci_npu2_data { - void *base; /* ATSD register virtual address, for emulated access */ - unsigned long mmio_atsd; /* ATSD physical address */ - unsigned long gpu_tgt; /* TGT address of corresponding GPU RAM */ - unsigned int link_speed; /* The link speed from DT's ibm,nvlink-speed */ -}; - -static size_t vfio_pci_npu2_rw(struct vfio_pci_core_device *vdev, - char __user *buf, size_t count, loff_t *ppos, bool iswrite) -{ - unsigned int i = VFIO_PCI_OFFSET_TO_INDEX(*ppos) - VFIO_PCI_NUM_REGIONS; - struct vfio_pci_npu2_data *data = vdev->region[i].data; - loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK; - - if (pos >= vdev->region[i].size) - return -EINVAL; - - count = min(count, (size_t)(vdev->region[i].size - pos)); - - if (iswrite) { - if (copy_from_user(data->base + pos, buf, count)) - return -EFAULT; - } else { - if (copy_to_user(buf, data->base + pos, count)) - return -EFAULT; - } - *ppos += count; - - return count; -} - -static int vfio_pci_npu2_mmap(struct vfio_pci_core_device *vdev, - struct vfio_pci_region *region, struct vm_area_struct *vma) -{ - int ret; - struct vfio_pci_npu2_data *data = region->data; - unsigned long req_len = vma->vm_end - vma->vm_start; - - if (req_len != PAGE_SIZE) - return -EINVAL; - - vma->vm_flags |= VM_PFNMAP; - vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); - - ret = remap_pfn_range(vma, vma->vm_start, data->mmio_atsd >> PAGE_SHIFT, - req_len, vma->vm_page_prot); - trace_vfio_pci_npu2_mmap(vdev->pdev, data->mmio_atsd, vma->vm_start, - vma->vm_end - vma->vm_start, ret); - - return ret; -} - -static void vfio_pci_npu2_release(struct vfio_pci_core_device *vdev, - struct vfio_pci_region *region) -{ - struct vfio_pci_npu2_data *data = region->data; - - memunmap(data->base); - kfree(data); -} - -static int vfio_pci_npu2_add_capability(struct vfio_pci_core_device *vdev, - struct vfio_pci_region *region, struct vfio_info_cap *caps) -{ - struct vfio_pci_npu2_data *data = region->data; - struct vfio_region_info_cap_nvlink2_ssatgt captgt = { - .header.id = VFIO_REGION_INFO_CAP_NVLINK2_SSATGT, - .header.version = 1, - .tgt = data->gpu_tgt - }; - struct vfio_region_info_cap_nvlink2_lnkspd capspd = { - .header.id = VFIO_REGION_INFO_CAP_NVLINK2_LNKSPD, - .header.version = 1, - .link_speed = data->link_speed - }; - int ret; - - ret = vfio_info_add_capability(caps, &captgt.header, sizeof(captgt)); - if (ret) - return ret; - - return vfio_info_add_capability(caps, &capspd.header, sizeof(capspd)); -} - -static const struct vfio_pci_regops vfio_pci_npu2_regops = { - .rw = vfio_pci_npu2_rw, - .mmap = vfio_pci_npu2_mmap, - .release = vfio_pci_npu2_release, - .add_capability = vfio_pci_npu2_add_capability, -}; - -int vfio_pci_ibm_npu2_init(struct vfio_pci_core_device *vdev) -{ - int ret; - struct vfio_pci_npu2_data *data; - struct device_node *nvlink_dn; - u32 nvlink_index = 0, mem_phandle = 0; - struct pci_dev *npdev = vdev->pdev; - struct device_node *npu_node = pci_device_to_OF_node(npdev); - struct pci_controller *hose = pci_bus_to_host(npdev->bus); - u64 mmio_atsd = 0; - u64 tgt = 0; - u32 link_speed = 0xff; - - /* - * PCI config space does not tell us about NVLink presense but - * platform does, use this. - */ - if (!pnv_pci_get_gpu_dev(vdev->pdev)) - return -ENODEV; - - if (of_property_read_u32(npu_node, "memory-region", &mem_phandle)) - return -ENODEV; - - /* - * NPU2 normally has 8 ATSD registers (for concurrency) and 6 links - * so we can allocate one register per link, using nvlink index as - * a key. - * There is always at least one ATSD register so as long as at least - * NVLink bridge #0 is passed to the guest, ATSD will be available. - */ - nvlink_dn = of_parse_phandle(npdev->dev.of_node, "ibm,nvlink", 0); - if (WARN_ON(of_property_read_u32(nvlink_dn, "ibm,npu-link-index", - &nvlink_index))) - return -ENODEV; - - if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", nvlink_index, - &mmio_atsd)) { - if (of_property_read_u64_index(hose->dn, "ibm,mmio-atsd", 0, - &mmio_atsd)) { - dev_warn(&vdev->pdev->dev, "No available ATSD found\n"); - mmio_atsd = 0; - } else { - dev_warn(&vdev->pdev->dev, - "Using fallback ibm,mmio-atsd[0] for ATSD.\n"); - } - } - - if (of_property_read_u64(npu_node, "ibm,device-tgt-addr", &tgt)) { - dev_warn(&vdev->pdev->dev, "No ibm,device-tgt-addr found\n"); - return -EFAULT; - } - - if (of_property_read_u32(npu_node, "ibm,nvlink-speed", &link_speed)) { - dev_warn(&vdev->pdev->dev, "No ibm,nvlink-speed found\n"); - return -EFAULT; - } - - data = kzalloc(sizeof(*data), GFP_KERNEL); - if (!data) - return -ENOMEM; - - data->mmio_atsd = mmio_atsd; - data->gpu_tgt = tgt; - data->link_speed = link_speed; - if (data->mmio_atsd) { - data->base = memremap(data->mmio_atsd, SZ_64K, MEMREMAP_WT); - if (!data->base) { - ret = -ENOMEM; - goto free_exit; - } - } - - /* - * We want to expose the capability even if this specific NVLink - * did not get its own ATSD register because capabilities - * belong to VFIO regions and normally there will be ATSD register - * assigned to the NVLink bridge. - */ - ret = vfio_pci_register_dev_region(vdev, - PCI_VENDOR_ID_IBM | - VFIO_REGION_TYPE_PCI_VENDOR_TYPE, - VFIO_REGION_SUBTYPE_IBM_NVLINK2_ATSD, - &vfio_pci_npu2_regops, - data->mmio_atsd ? PAGE_SIZE : 0, - VFIO_REGION_INFO_FLAG_READ | - VFIO_REGION_INFO_FLAG_WRITE | - VFIO_REGION_INFO_FLAG_MMAP, - data); - if (ret) - goto free_exit; - - return 0; - -free_exit: - if (data->base) - memunmap(data->base); - kfree(data); - - return ret; -} From patchwork Tue Mar 9 08:33:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Gurtovoy X-Patchwork-Id: 12124395 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLACK,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F429C433E9 for ; Tue, 9 Mar 2021 08:35:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DBAF6650F5 for ; Tue, 9 Mar 2021 08:35:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230434AbhCIIfY (ORCPT ); Tue, 9 Mar 2021 03:35:24 -0500 Received: from mail-dm6nam10on2069.outbound.protection.outlook.com ([40.107.93.69]:33792 "EHLO NAM10-DM6-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S230483AbhCIIe7 (ORCPT ); Tue, 9 Mar 2021 03:34:59 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cfHNzXjIjrG3npM+hT7NRhgoICDsnQyhuJb5pEjwNy24MPvQ9sraDtGjcB0MHvDlpS0/+k5IMFql4O45Khio2XuOOT36ZaoIFtm58H9gQgYRVC6xSthyUVT7WtbTz6AfKR48s4JE1rkSQVATNOaeafyaJBMbMFnGD+oKw4FZqxspKWMGFoB0JYeprd5mipMoty6WGYsrsu7azBp/ZnPcDWajyt3cmhboXIJtvgoL6QavzUiRGphcvsxQRbncWGxcvTp2az/UCawuIx6xhn4PLAqp2/RlbjTUTUBMboMgRf3DbWPzUndzV2iFD0OdOQOCOvEgMlhuGhwYZI8OKxq7DA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=p4daIEL87NOMVMmKAMKowc3K3gr9XlepFv1BIZAn01Y=; b=BQot8k3km+hc7P5iSmHWBrcosbchsAid6+VdekVQ074cEb67n/Nt7Z01/x73hPzjQwqb52QALQaYsojA0wjiSMQVr3UWYgB1/B+LUDwhurbkiTBZVGJeTEdN6MsFEaDLBWIZ3MG51fTpeskWL6V4VIVHFIKPm//RqaT+pgAOG0jgT/fbCnGxFbH8vXYA8ByxqtXS/aoCGgwhVi50tiwvzSVtyS7+3Ht2AKjOx/BACLuChbh0eglOxPZksHke4E/Zm6gdTRe4ApVWzqZnea0WlmqqcVwiZZa/yXkqa2hx7L+z4oopv7g7H1rwSaxJmTxnvskD4mfy99ZYyvFBQClbZA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.34) smtp.rcpttodomain=linux.ibm.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=p4daIEL87NOMVMmKAMKowc3K3gr9XlepFv1BIZAn01Y=; b=cCnKC9EiGWlpLzCvCcP/RFh0RjTitCaYYK8sYSdh/J7CsI1VDKQY8eLRnKDTB9ZLhDQW1RVGD5ptpcXWwa2eIwNcg8Ak4fT1W7u269hAnfqqaPyTehRGmZRdYE2JtpAnm6SEDD1y0UrfzuWl/r6HHG+NAi0wMVR23J1M3vb5v+13EO1EQT8FtarH7DkQma1cRS0V+1XRgXT21vsVgJR3WZl+r58O47XusOgPD2uL+DO/V6gjnN1qwweQquGnDrZ1016bcUYL6d+jwpi5PdjfCFdK3KdbkbtlCsIYTtE7fPWpXXRl/1FBhFJ8Lch0EJBGpp4iAf9wQ/q6imQ0cCiSNA== Received: from DM5PR18CA0092.namprd18.prod.outlook.com (2603:10b6:3:3::30) by MW3PR12MB4412.namprd12.prod.outlook.com (2603:10b6:303:58::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.23; Tue, 9 Mar 2021 08:34:52 +0000 Received: from DM6NAM11FT024.eop-nam11.prod.protection.outlook.com (2603:10b6:3:3:cafe::8c) by DM5PR18CA0092.outlook.office365.com (2603:10b6:3:3::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.17 via Frontend Transport; Tue, 9 Mar 2021 08:34:52 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.34) smtp.mailfrom=nvidia.com; linux.ibm.com; dkim=none (message not signed) header.d=none;linux.ibm.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.34 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.34; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.34) by DM6NAM11FT024.mail.protection.outlook.com (10.13.172.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.3912.17 via Frontend Transport; Tue, 9 Mar 2021 08:34:51 +0000 Received: from HQMAIL111.nvidia.com (172.20.187.18) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Mar 2021 08:34:50 +0000 Received: from r-nvmx02.mtr.labs.mlnx (172.20.145.6) by mail.nvidia.com (172.20.187.18) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 9 Mar 2021 08:34:45 +0000 From: Max Gurtovoy To: , , , , CC: , , , , , , , , , , , , , , , Max Gurtovoy Subject: [PATCH 8/9] vfio/pci: export nvlink2 support into vendor vfio_pci drivers Date: Tue, 9 Mar 2021 08:33:56 +0000 Message-ID: <20210309083357.65467-9-mgurtovoy@nvidia.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20210309083357.65467-1-mgurtovoy@nvidia.com> References: <20210309083357.65467-1-mgurtovoy@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 12b3926e-4db4-4850-e361-08d8e2d63561 X-MS-TrafficTypeDiagnostic: MW3PR12MB4412: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3ytcKo31YI6UjoWPp4ZpDL4wXbP3ks/w4rYMIAH2E9wmRUxWOuKylMVhXu32mo1cOn6fdPKZN65qD/sF136TweL1MMvgGwf5t4kdkYf2mMsmIl2v0jr5ItIpLkFO8ypOXTzr9ptUov3uO0IihT9SyLGVKribIMXAL1UIERHefNqQUMfTowChgqjRLmWorZQ9m5lTkwN34tpRVYQpIFXqu+3FAm6ano/VQXSFi/RxrV9L5D5nMLVPTJSrYBdKK7pER8v1Q5P0nnqgC69tH3BaFivtz61mSK9yDnf1mdEsxnk4zmaBWkcDdJSqHe++PAgx/bNSePpuTysRHgzgawk0hfMtzcoNYKY8A5l7Pc0pu4hqRgdpScD1T76x2tarm1ysTpnYw3tNNmE3s+xDWkVwPrIooC/ztnXXVcTIFC4wkwNKwSlctwBZrQ+BuJCNdhqv61w34DGTxDEsNgJu6KTWRxX3YD/T4TIsFSLTZG2Cf69e+qZhpjZqACd5zKGRC57CtNjzmff4vk6yum6pL+KZhPQu3wBu+JnYToJtsXC1Pqls3L5Y9YhTwsaGtyU/cSxlBSV21vVmiv/GDTb7izzv2zyedE3/1Ku5PIvMJR74pbsGcWmdu4i/ayXMz3fF7KZSWY6jTBl2YS4Xtk33FMS9+RdnWpwnL7CG1FzZ/UgQoIdOf66640+vmk4iOgKPe0wg X-Forefront-Antispam-Report: CIP:216.228.112.34;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid03.nvidia.com;CAT:NONE;SFS:(4636009)(396003)(346002)(376002)(136003)(39860400002)(36840700001)(46966006)(26005)(2616005)(86362001)(186003)(70586007)(1076003)(34020700004)(30864003)(5660300002)(70206006)(2906002)(54906003)(316002)(356005)(36860700001)(110136005)(83380400001)(8936002)(7636003)(8676002)(36756003)(47076005)(4326008)(6666004)(107886003)(478600001)(336012)(82310400003)(426003)(82740400003);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Mar 2021 08:34:51.7782 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 12b3926e-4db4-4850-e361-08d8e2d63561 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.34];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT024.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4412 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The new drivers introduced are nvlink2gpu_vfio_pci.ko and npu2_vfio_pci.ko. The first will be responsible for providing special extensions for NVIDIA GPUs with NVLINK2 support for P9 platform (and others in the future). The last will be responsible for POWER9 NPU2 unit (NVLink2 host bus adapter). Also, preserve backward compatibility for users that were binding NVLINK2 devices to vfio_pci.ko. Hopefully this compatibility layer will be dropped in the future Signed-off-by: Max Gurtovoy --- drivers/vfio/pci/Kconfig | 28 +++- drivers/vfio/pci/Makefile | 7 +- .../pci/{vfio_pci_npu2.c => npu2_vfio_pci.c} | 144 ++++++++++++++++- drivers/vfio/pci/npu2_vfio_pci.h | 24 +++ ...pci_nvlink2gpu.c => nvlink2gpu_vfio_pci.c} | 149 +++++++++++++++++- drivers/vfio/pci/nvlink2gpu_vfio_pci.h | 24 +++ drivers/vfio/pci/vfio_pci.c | 61 ++++++- drivers/vfio/pci/vfio_pci_core.c | 18 --- drivers/vfio/pci/vfio_pci_core.h | 14 -- 9 files changed, 422 insertions(+), 47 deletions(-) rename drivers/vfio/pci/{vfio_pci_npu2.c => npu2_vfio_pci.c} (64%) create mode 100644 drivers/vfio/pci/npu2_vfio_pci.h rename drivers/vfio/pci/{vfio_pci_nvlink2gpu.c => nvlink2gpu_vfio_pci.c} (67%) create mode 100644 drivers/vfio/pci/nvlink2gpu_vfio_pci.h diff --git a/drivers/vfio/pci/Kconfig b/drivers/vfio/pci/Kconfig index 829e90a2e5a3..88c89863a205 100644 --- a/drivers/vfio/pci/Kconfig +++ b/drivers/vfio/pci/Kconfig @@ -48,8 +48,30 @@ config VFIO_PCI_IGD To enable Intel IGD assignment through vfio-pci, say Y. -config VFIO_PCI_NVLINK2 - def_bool y +config VFIO_PCI_NVLINK2GPU + tristate "VFIO support for NVIDIA NVLINK2 GPUs" depends on VFIO_PCI_CORE && PPC_POWERNV help - VFIO PCI support for P9 Witherspoon machine with NVIDIA V100 GPUs + VFIO PCI driver for NVIDIA NVLINK2 GPUs with specific extensions + for P9 Witherspoon machine. + +config VFIO_PCI_NPU2 + tristate "VFIO support for IBM NPU host bus adapter on P9" + depends on VFIO_PCI_NVLINK2GPU && PPC_POWERNV + help + VFIO PCI specific extensions for IBM NVLink2 host bus adapter on P9 + Witherspoon machine. + +config VFIO_PCI_DRIVER_COMPAT + bool "VFIO PCI backward compatibility for vendor specific extensions" + default y + depends on VFIO_PCI + help + Say Y here if you want to preserve VFIO PCI backward + compatibility. vfio_pci.ko will continue to automatically use + the NVLINK2, NPU2 and IGD VFIO drivers when it is attached to + a compatible device. + + When N is selected the user must bind explicity to the module + they want to handle the device and vfio_pci.ko will have no + device specific special behaviors. diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile index f539f32c9296..86fb62e271fc 100644 --- a/drivers/vfio/pci/Makefile +++ b/drivers/vfio/pci/Makefile @@ -2,10 +2,15 @@ obj-$(CONFIG_VFIO_PCI_CORE) += vfio-pci-core.o obj-$(CONFIG_VFIO_PCI) += vfio-pci.o +obj-$(CONFIG_VFIO_PCI_NPU2) += npu2-vfio-pci.o +obj-$(CONFIG_VFIO_PCI_NVLINK2GPU) += nvlink2gpu-vfio-pci.o vfio-pci-core-y := vfio_pci_core.o vfio_pci_intrs.o vfio_pci_rdwr.o vfio_pci_config.o vfio-pci-core-$(CONFIG_VFIO_PCI_IGD) += vfio_pci_igd.o -vfio-pci-core-$(CONFIG_VFIO_PCI_NVLINK2) += vfio_pci_nvlink2gpu.o vfio_pci_npu2.o vfio-pci-core-$(CONFIG_S390) += vfio_pci_zdev.o vfio-pci-y := vfio_pci.o + +npu2-vfio-pci-y := npu2_vfio_pci.o + +nvlink2gpu-vfio-pci-y := nvlink2gpu_vfio_pci.o diff --git a/drivers/vfio/pci/vfio_pci_npu2.c b/drivers/vfio/pci/npu2_vfio_pci.c similarity index 64% rename from drivers/vfio/pci/vfio_pci_npu2.c rename to drivers/vfio/pci/npu2_vfio_pci.c index 717745256ab3..7071bda0f2b6 100644 --- a/drivers/vfio/pci/vfio_pci_npu2.c +++ b/drivers/vfio/pci/npu2_vfio_pci.c @@ -14,19 +14,28 @@ * Author: Alex Williamson */ +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include #include #include #include #include +#include #include #include #include #include "vfio_pci_core.h" +#include "npu2_vfio_pci.h" #define CREATE_TRACE_POINTS #include "npu2_trace.h" +#define DRIVER_VERSION "0.1" +#define DRIVER_AUTHOR "Alexey Kardashevskiy " +#define DRIVER_DESC "NPU2 VFIO PCI - User Level meta-driver for POWER9 NPU NVLink2 HBA" + EXPORT_TRACEPOINT_SYMBOL_GPL(vfio_pci_npu2_mmap); struct vfio_pci_npu2_data { @@ -36,6 +45,10 @@ struct vfio_pci_npu2_data { unsigned int link_speed; /* The link speed from DT's ibm,nvlink-speed */ }; +struct npu2_vfio_pci_device { + struct vfio_pci_core_device vdev; +}; + static size_t vfio_pci_npu2_rw(struct vfio_pci_core_device *vdev, char __user *buf, size_t count, loff_t *ppos, bool iswrite) { @@ -120,7 +133,7 @@ static const struct vfio_pci_regops vfio_pci_npu2_regops = { .add_capability = vfio_pci_npu2_add_capability, }; -int vfio_pci_ibm_npu2_init(struct vfio_pci_core_device *vdev) +static int vfio_pci_ibm_npu2_init(struct vfio_pci_core_device *vdev) { int ret; struct vfio_pci_npu2_data *data; @@ -220,3 +233,132 @@ int vfio_pci_ibm_npu2_init(struct vfio_pci_core_device *vdev) return ret; } + +static void npu2_vfio_pci_release(void *device_data) +{ + struct vfio_pci_core_device *vdev = device_data; + + mutex_lock(&vdev->reflck->lock); + if (!(--vdev->refcnt)) { + vfio_pci_vf_token_user_add(vdev, -1); + vfio_pci_core_spapr_eeh_release(vdev); + vfio_pci_core_disable(vdev); + } + mutex_unlock(&vdev->reflck->lock); + + module_put(THIS_MODULE); +} + +static int npu2_vfio_pci_open(void *device_data) +{ + struct vfio_pci_core_device *vdev = device_data; + int ret = 0; + + if (!try_module_get(THIS_MODULE)) + return -ENODEV; + + mutex_lock(&vdev->reflck->lock); + + if (!vdev->refcnt) { + ret = vfio_pci_core_enable(vdev); + if (ret) + goto error; + + ret = vfio_pci_ibm_npu2_init(vdev); + if (ret && ret != -ENODEV) { + pci_warn(vdev->pdev, + "Failed to setup NVIDIA NV2 ATSD region\n"); + vfio_pci_core_disable(vdev); + goto error; + } + ret = 0; + vfio_pci_probe_mmaps(vdev); + vfio_pci_core_spapr_eeh_open(vdev); + vfio_pci_vf_token_user_add(vdev, 1); + } + vdev->refcnt++; +error: + mutex_unlock(&vdev->reflck->lock); + if (ret) + module_put(THIS_MODULE); + return ret; +} + +static const struct vfio_device_ops npu2_vfio_pci_ops = { + .name = "npu2-vfio-pci", + .open = npu2_vfio_pci_open, + .release = npu2_vfio_pci_release, + .ioctl = vfio_pci_core_ioctl, + .read = vfio_pci_core_read, + .write = vfio_pci_core_write, + .mmap = vfio_pci_core_mmap, + .request = vfio_pci_core_request, + .match = vfio_pci_core_match, +}; + +static int npu2_vfio_pci_probe(struct pci_dev *pdev, + const struct pci_device_id *id) +{ + struct npu2_vfio_pci_device *npvdev; + int ret; + + npvdev = kzalloc(sizeof(*npvdev), GFP_KERNEL); + if (!npvdev) + return -ENOMEM; + + ret = vfio_pci_core_register_device(&npvdev->vdev, pdev, + &npu2_vfio_pci_ops); + if (ret) + goto out_free; + + return 0; + +out_free: + kfree(npvdev); + return ret; +} + +static void npu2_vfio_pci_remove(struct pci_dev *pdev) +{ + struct vfio_device *vdev = dev_get_drvdata(&pdev->dev); + struct vfio_pci_core_device *core_vpdev = vfio_device_data(vdev); + struct npu2_vfio_pci_device *npvdev; + + npvdev = container_of(core_vpdev, struct npu2_vfio_pci_device, vdev); + + vfio_pci_core_unregister_device(core_vpdev); + kfree(npvdev); +} + +static const struct pci_device_id npu2_vfio_pci_table[] = { + { PCI_VDEVICE(IBM, 0x04ea) }, + { 0, } +}; + +static struct pci_driver npu2_vfio_pci_driver = { + .name = "npu2-vfio-pci", + .id_table = npu2_vfio_pci_table, + .probe = npu2_vfio_pci_probe, + .remove = npu2_vfio_pci_remove, +#ifdef CONFIG_PCI_IOV + .sriov_configure = vfio_pci_core_sriov_configure, +#endif + .err_handler = &vfio_pci_core_err_handlers, +}; + +#ifdef CONFIG_VFIO_PCI_DRIVER_COMPAT +struct pci_driver *get_npu2_vfio_pci_driver(struct pci_dev *pdev) +{ + if (pci_match_id(npu2_vfio_pci_driver.id_table, pdev)) + return &npu2_vfio_pci_driver; + return NULL; +} +EXPORT_SYMBOL_GPL(get_npu2_vfio_pci_driver); +#endif + +module_pci_driver(npu2_vfio_pci_driver); + +MODULE_VERSION(DRIVER_VERSION); +MODULE_LICENSE("GPL v2"); +MODULE_AUTHOR(DRIVER_AUTHOR); +MODULE_DESCRIPTION(DRIVER_DESC); diff --git a/drivers/vfio/pci/npu2_vfio_pci.h b/drivers/vfio/pci/npu2_vfio_pci.h new file mode 100644 index 000000000000..92010d340346 --- /dev/null +++ b/drivers/vfio/pci/npu2_vfio_pci.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c) 2020, Mellanox Technologies, Ltd. All rights reserved. + * Author: Max Gurtovoy + */ + +#ifndef NPU2_VFIO_PCI_H +#define NPU2_VFIO_PCI_H + +#include +#include + +#ifdef CONFIG_VFIO_PCI_DRIVER_COMPAT +#if defined(CONFIG_VFIO_PCI_NPU2) || defined(CONFIG_VFIO_PCI_NPU2_MODULE) +struct pci_driver *get_npu2_vfio_pci_driver(struct pci_dev *pdev); +#else +struct pci_driver *get_npu2_vfio_pci_driver(struct pci_dev *pdev) +{ + return NULL; +} +#endif +#endif + +#endif /* NPU2_VFIO_PCI_H */ diff --git a/drivers/vfio/pci/vfio_pci_nvlink2gpu.c b/drivers/vfio/pci/nvlink2gpu_vfio_pci.c similarity index 67% rename from drivers/vfio/pci/vfio_pci_nvlink2gpu.c rename to drivers/vfio/pci/nvlink2gpu_vfio_pci.c index 6dce1e78ee82..84a5ac1ce8ac 100644 --- a/drivers/vfio/pci/vfio_pci_nvlink2gpu.c +++ b/drivers/vfio/pci/nvlink2gpu_vfio_pci.c @@ -1,6 +1,6 @@ // SPDX-License-Identifier: GPL-2.0-only /* - * VFIO PCI NVIDIA Whitherspoon GPU support a.k.a. NVLink2. + * VFIO PCI NVIDIA NVLink2 GPUs support. * * Copyright (C) 2018 IBM Corp. All rights reserved. * Author: Alexey Kardashevskiy @@ -12,6 +12,9 @@ * Author: Alex Williamson */ +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include #include #include #include @@ -21,10 +24,15 @@ #include #include "vfio_pci_core.h" +#include "nvlink2gpu_vfio_pci.h" #define CREATE_TRACE_POINTS #include "nvlink2gpu_trace.h" +#define DRIVER_VERSION "0.1" +#define DRIVER_AUTHOR "Alexey Kardashevskiy " +#define DRIVER_DESC "NVLINK2GPU VFIO PCI - User Level meta-driver for NVIDIA NVLink2 GPUs" + EXPORT_TRACEPOINT_SYMBOL_GPL(vfio_pci_nvgpu_mmap_fault); EXPORT_TRACEPOINT_SYMBOL_GPL(vfio_pci_nvgpu_mmap); @@ -39,6 +47,10 @@ struct vfio_pci_nvgpu_data { struct notifier_block group_notifier; }; +struct nv_vfio_pci_device { + struct vfio_pci_core_device vdev; +}; + static size_t vfio_pci_nvgpu_rw(struct vfio_pci_core_device *vdev, char __user *buf, size_t count, loff_t *ppos, bool iswrite) { @@ -207,7 +219,8 @@ static int vfio_pci_nvgpu_group_notifier(struct notifier_block *nb, return NOTIFY_OK; } -int vfio_pci_nvidia_v100_nvlink2_init(struct vfio_pci_core_device *vdev) +static int +vfio_pci_nvidia_v100_nvlink2_init(struct vfio_pci_core_device *vdev) { int ret; u64 reg[2]; @@ -293,3 +306,135 @@ int vfio_pci_nvidia_v100_nvlink2_init(struct vfio_pci_core_device *vdev) return ret; } + +static void nvlink2gpu_vfio_pci_release(void *device_data) +{ + struct vfio_pci_core_device *vdev = device_data; + + mutex_lock(&vdev->reflck->lock); + if (!(--vdev->refcnt)) { + vfio_pci_vf_token_user_add(vdev, -1); + vfio_pci_core_spapr_eeh_release(vdev); + vfio_pci_core_disable(vdev); + } + mutex_unlock(&vdev->reflck->lock); + + module_put(THIS_MODULE); +} + +static int nvlink2gpu_vfio_pci_open(void *device_data) +{ + struct vfio_pci_core_device *vdev = device_data; + int ret = 0; + + if (!try_module_get(THIS_MODULE)) + return -ENODEV; + + mutex_lock(&vdev->reflck->lock); + + if (!vdev->refcnt) { + ret = vfio_pci_core_enable(vdev); + if (ret) + goto error; + + ret = vfio_pci_nvidia_v100_nvlink2_init(vdev); + if (ret && ret != -ENODEV) { + pci_warn(vdev->pdev, + "Failed to setup NVIDIA NV2 RAM region\n"); + vfio_pci_core_disable(vdev); + goto error; + } + ret = 0; + vfio_pci_probe_mmaps(vdev); + vfio_pci_core_spapr_eeh_open(vdev); + vfio_pci_vf_token_user_add(vdev, 1); + } + vdev->refcnt++; +error: + mutex_unlock(&vdev->reflck->lock); + if (ret) + module_put(THIS_MODULE); + return ret; +} + +static const struct vfio_device_ops nvlink2gpu_vfio_pci_ops = { + .name = "nvlink2gpu-vfio-pci", + .open = nvlink2gpu_vfio_pci_open, + .release = nvlink2gpu_vfio_pci_release, + .ioctl = vfio_pci_core_ioctl, + .read = vfio_pci_core_read, + .write = vfio_pci_core_write, + .mmap = vfio_pci_core_mmap, + .request = vfio_pci_core_request, + .match = vfio_pci_core_match, +}; + +static int nvlink2gpu_vfio_pci_probe(struct pci_dev *pdev, + const struct pci_device_id *id) +{ + struct nv_vfio_pci_device *nvdev; + int ret; + + nvdev = kzalloc(sizeof(*nvdev), GFP_KERNEL); + if (!nvdev) + return -ENOMEM; + + ret = vfio_pci_core_register_device(&nvdev->vdev, pdev, + &nvlink2gpu_vfio_pci_ops); + if (ret) + goto out_free; + + return 0; + +out_free: + kfree(nvdev); + return ret; +} + +static void nvlink2gpu_vfio_pci_remove(struct pci_dev *pdev) +{ + struct vfio_device *vdev = dev_get_drvdata(&pdev->dev); + struct vfio_pci_core_device *core_vpdev = vfio_device_data(vdev); + struct nv_vfio_pci_device *nvdev; + + nvdev = container_of(core_vpdev, struct nv_vfio_pci_device, vdev); + + vfio_pci_core_unregister_device(core_vpdev); + kfree(nvdev); +} + +static const struct pci_device_id nvlink2gpu_vfio_pci_table[] = { + { PCI_VDEVICE(NVIDIA, 0x1DB1) }, /* GV100GL-A NVIDIA Tesla V100-SXM2-16GB */ + { PCI_VDEVICE(NVIDIA, 0x1DB5) }, /* GV100GL-A NVIDIA Tesla V100-SXM2-32GB */ + { PCI_VDEVICE(NVIDIA, 0x1DB8) }, /* GV100GL-A NVIDIA Tesla V100-SXM3-32GB */ + { PCI_VDEVICE(NVIDIA, 0x1DF5) }, /* GV100GL-B NVIDIA Tesla V100-SXM2-16GB */ + { 0, } +}; + +static struct pci_driver nvlink2gpu_vfio_pci_driver = { + .name = "nvlink2gpu-vfio-pci", + .id_table = nvlink2gpu_vfio_pci_table, + .probe = nvlink2gpu_vfio_pci_probe, + .remove = nvlink2gpu_vfio_pci_remove, +#ifdef CONFIG_PCI_IOV + .sriov_configure = vfio_pci_core_sriov_configure, +#endif + .err_handler = &vfio_pci_core_err_handlers, +}; + +#ifdef CONFIG_VFIO_PCI_DRIVER_COMPAT +struct pci_driver *get_nvlink2gpu_vfio_pci_driver(struct pci_dev *pdev) +{ + if (pci_match_id(nvlink2gpu_vfio_pci_driver.id_table, pdev)) + return &nvlink2gpu_vfio_pci_driver; + return NULL; +} +EXPORT_SYMBOL_GPL(get_nvlink2gpu_vfio_pci_driver); +#endif + +module_pci_driver(nvlink2gpu_vfio_pci_driver); + +MODULE_VERSION(DRIVER_VERSION); +MODULE_LICENSE("GPL v2"); +MODULE_AUTHOR(DRIVER_AUTHOR); +MODULE_DESCRIPTION(DRIVER_DESC); diff --git a/drivers/vfio/pci/nvlink2gpu_vfio_pci.h b/drivers/vfio/pci/nvlink2gpu_vfio_pci.h new file mode 100644 index 000000000000..ebd5b600b190 --- /dev/null +++ b/drivers/vfio/pci/nvlink2gpu_vfio_pci.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c) 2020, Mellanox Technologies, Ltd. All rights reserved. + * Author: Max Gurtovoy + */ + +#ifndef NVLINK2GPU_VFIO_PCI_H +#define NVLINK2GPU_VFIO_PCI_H + +#include +#include + +#ifdef CONFIG_VFIO_PCI_DRIVER_COMPAT +#if defined(CONFIG_VFIO_PCI_NVLINK2GPU) || defined(CONFIG_VFIO_PCI_NVLINK2GPU_MODULE) +struct pci_driver *get_nvlink2gpu_vfio_pci_driver(struct pci_dev *pdev); +#else +struct pci_driver *get_nvlink2gpu_vfio_pci_driver(struct pci_dev *pdev) +{ + return NULL; +} +#endif +#endif + +#endif /* NVLINK2GPU_VFIO_PCI_H */ diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index dbc0a6559914..8e81ea039f31 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -27,6 +27,10 @@ #include #include "vfio_pci_core.h" +#ifdef CONFIG_VFIO_PCI_DRIVER_COMPAT +#include "npu2_vfio_pci.h" +#include "nvlink2gpu_vfio_pci.h" +#endif #define DRIVER_VERSION "0.2" #define DRIVER_AUTHOR "Alex Williamson " @@ -142,14 +146,48 @@ static const struct vfio_device_ops vfio_pci_ops = { .match = vfio_pci_core_match, }; +/* + * This layer is used for backward compatibility. Hopefully it will be + * removed in the future. + */ +static struct pci_driver *vfio_pci_get_compat_driver(struct pci_dev *pdev) +{ + switch (pdev->vendor) { + case PCI_VENDOR_ID_NVIDIA: + switch (pdev->device) { + case 0x1db1: + case 0x1db5: + case 0x1db8: + case 0x1df5: + return get_nvlink2gpu_vfio_pci_driver(pdev); + default: + return NULL; + } + case PCI_VENDOR_ID_IBM: + switch (pdev->device) { + case 0x04ea: + return get_npu2_vfio_pci_driver(pdev); + default: + return NULL; + } + } + + return NULL; +} + static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) { struct vfio_pci_device *vpdev; + struct pci_driver *driver; int ret; if (vfio_pci_is_denylisted(pdev)) return -EINVAL; + driver = vfio_pci_get_compat_driver(pdev); + if (driver) + return driver->probe(pdev, id); + vpdev = kzalloc(sizeof(*vpdev), GFP_KERNEL); if (!vpdev) return -ENOMEM; @@ -167,14 +205,21 @@ static int vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) static void vfio_pci_remove(struct pci_dev *pdev) { - struct vfio_device *vdev = dev_get_drvdata(&pdev->dev); - struct vfio_pci_core_device *core_vpdev = vfio_device_data(vdev); - struct vfio_pci_device *vpdev; - - vpdev = container_of(core_vpdev, struct vfio_pci_device, vdev); - - vfio_pci_core_unregister_device(core_vpdev); - kfree(vpdev); + struct pci_driver *driver; + + driver = vfio_pci_get_compat_driver(pdev); + if (driver) { + driver->remove(pdev); + } else { + struct vfio_device *vdev = dev_get_drvdata(&pdev->dev); + struct vfio_pci_core_device *core_vpdev; + struct vfio_pci_device *vpdev; + + core_vpdev = vfio_device_data(vdev); + vpdev = container_of(core_vpdev, struct vfio_pci_device, vdev); + vfio_pci_core_unregister_device(core_vpdev); + kfree(vpdev); + } } static int vfio_pci_sriov_configure(struct pci_dev *pdev, int nr_virtfn) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index 4de8e352df9c..f9b39abe54cb 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -354,24 +354,6 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) } } - if (pdev->vendor == PCI_VENDOR_ID_NVIDIA && - IS_ENABLED(CONFIG_VFIO_PCI_NVLINK2)) { - ret = vfio_pci_nvidia_v100_nvlink2_init(vdev); - if (ret && ret != -ENODEV) { - pci_warn(pdev, "Failed to setup NVIDIA NV2 RAM region\n"); - goto disable_exit; - } - } - - if (pdev->vendor == PCI_VENDOR_ID_IBM && - IS_ENABLED(CONFIG_VFIO_PCI_NVLINK2)) { - ret = vfio_pci_ibm_npu2_init(vdev); - if (ret && ret != -ENODEV) { - pci_warn(pdev, "Failed to setup NVIDIA NV2 ATSD region\n"); - goto disable_exit; - } - } - return 0; disable_exit: diff --git a/drivers/vfio/pci/vfio_pci_core.h b/drivers/vfio/pci/vfio_pci_core.h index 8989443c3086..31f3836e606e 100644 --- a/drivers/vfio/pci/vfio_pci_core.h +++ b/drivers/vfio/pci/vfio_pci_core.h @@ -204,20 +204,6 @@ static inline int vfio_pci_igd_init(struct vfio_pci_core_device *vdev) return -ENODEV; } #endif -#ifdef CONFIG_VFIO_PCI_NVLINK2 -extern int vfio_pci_nvidia_v100_nvlink2_init(struct vfio_pci_core_device *vdev); -extern int vfio_pci_ibm_npu2_init(struct vfio_pci_core_device *vdev); -#else -static inline int vfio_pci_nvidia_v100_nvlink2_init(struct vfio_pci_core_device *vdev) -{ - return -ENODEV; -} - -static inline int vfio_pci_ibm_npu2_init(struct vfio_pci_core_device *vdev) -{ - return -ENODEV; -} -#endif #ifdef CONFIG_S390 extern int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_device *vdev, From patchwork Tue Mar 9 08:33:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Max Gurtovoy X-Patchwork-Id: 12124397 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FFB8C433DB for ; Tue, 9 Mar 2021 08:35:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EB384652AB for ; Tue, 9 Mar 2021 08:35:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230503AbhCIIf1 (ORCPT ); Tue, 9 Mar 2021 03:35:27 -0500 Received: from mail-bn8nam12on2057.outbound.protection.outlook.com ([40.107.237.57]:50147 "EHLO NAM12-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S230526AbhCIIfC (ORCPT ); Tue, 9 Mar 2021 03:35:02 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kQ9u2fZMXQJhnkkPpgtfoDaWbT52KEeLWiFuKAqliLkTQJe35z5t6FftTQTjW8/wi3SNgJfYKZK2JHR0tW0rAbCVr9t05qdyCSXdtprILSOSZvsx8eAwHyC6zwO/zE1dDC7kw41osNd1QH6ofGSXvHu2wxdp5A2KLl2EdOJqeA8lMx66nMcbNfsbD2yRmxltudXoV9PL8OmUQjc1zBVv8PWi4p/o67dScrtNmRRJfY2qNZjz8YhRIbHp86mAeujZLBUJX2V/tCzim97tiKKZKEQ4VlX6Ko989P2l8yQyd3bTQrjRVrsBEMDQ3vd3REwclrsN5DYxtqsceAoEFu5skw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Yh7Sv51obkkSmM/XL2iq8xuUwpYOztQTlt6iapqpfBU=; b=N2s6OxM+JzXZPzOrZ24F5E1M5DLNwGQilb/F6tWvXLLMmCvWGTcDjayDaW02m1axehscJz2HrpCc+RvyAV8dhgM2RWcqT2m4bqazoaoz7P6SqWI7gTdE422IF8ZeDdqEVCwAtxr4Qx9Nuvopq3Kpe7GnF1iLRXSCviKqQp501tFpQLeHgUqgMRWOJUfQhdF1ohGNnE+SFlXGFIVkHaxffL6Bc93+wOpq++SlB+qLCUgAG+zYG6vmiNwJ20Ucf0lUks/bcj9V59UZlEdc/B9hnpJeHi6+tMLdBpxmknnxBxceoxU3TObVe1wzAe0aER/hSyXrq2kVThcGYnvUnBWKIw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.112.35) smtp.rcpttodomain=linux.ibm.com smtp.mailfrom=nvidia.com; dmarc=pass (p=none sp=none pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Yh7Sv51obkkSmM/XL2iq8xuUwpYOztQTlt6iapqpfBU=; b=a6tx8UjZpSWkGXNYJvfwzZWLAAVljygILvotg0oZpFx2InCIEyPml+5PmZP5MJ7hXCKhfoiRN04CKsFDdm9JqCfoj0D6v9Y748ewbSweIUKjSDnCYeALh/R1gtzR78D1NYtdb7hmwbRvd4B6HGUWIt+PGwbYzAJUfDXRGY/Wj5NG3i9OA/Wea5puxIF5o/uWjkJLt/f1mzVBGl2rTmdqxeLqiRQ8ItqTtyk1PVofdKtEeJZUkAu9J1Wpl/0wCnd88SXs223qnjjdYvVggCOjITt5mXxA07V+Fvzy860ILK305niyZsJKFiRNsQRMRLw3+eArdlEmBZfm97/pEom1Jg== Received: from DM6PR13CA0025.namprd13.prod.outlook.com (2603:10b6:5:bc::38) by BN6PR12MB1202.namprd12.prod.outlook.com (2603:10b6:404:1c::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.19; Tue, 9 Mar 2021 08:34:58 +0000 Received: from DM6NAM11FT037.eop-nam11.prod.protection.outlook.com (2603:10b6:5:bc:cafe::b7) by DM6PR13CA0025.outlook.office365.com (2603:10b6:5:bc::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3933.22 via Frontend Transport; Tue, 9 Mar 2021 08:34:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.112.35) smtp.mailfrom=nvidia.com; linux.ibm.com; dkim=none (message not signed) header.d=none;linux.ibm.com; dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.112.35 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.112.35; helo=mail.nvidia.com; Received: from mail.nvidia.com (216.228.112.35) by DM6NAM11FT037.mail.protection.outlook.com (10.13.172.122) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.3912.17 via Frontend Transport; Tue, 9 Mar 2021 08:34:57 +0000 Received: from HQMAIL111.nvidia.com (172.20.187.18) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 9 Mar 2021 08:34:56 +0000 Received: from r-nvmx02.mtr.labs.mlnx (172.20.145.6) by mail.nvidia.com (172.20.187.18) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Tue, 9 Mar 2021 08:34:51 +0000 From: Max Gurtovoy To: , , , , CC: , , , , , , , , , , , , , , , Max Gurtovoy Subject: [PATCH 9/9] vfio/pci: export igd support into vendor vfio_pci driver Date: Tue, 9 Mar 2021 08:33:57 +0000 Message-ID: <20210309083357.65467-10-mgurtovoy@nvidia.com> X-Mailer: git-send-email 2.25.4 In-Reply-To: <20210309083357.65467-1-mgurtovoy@nvidia.com> References: <20210309083357.65467-1-mgurtovoy@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 7c9a7aaf-db9d-4dfa-f2c9-08d8e2d638f1 X-MS-TrafficTypeDiagnostic: BN6PR12MB1202: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:268; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zZp4SJecrXzWxkAwqtDtaGMKNxPYbimvSnS5V/tomYaNp4DB67nVFuiL0emcTdXym+4BlRHwAtQdfv4+Ksm7i8NRYOzivU7USoNr/u7wH348nXK4B/gmmTMzoFaWR07+YI5QIsuFAoGDn7SEOZARcvsbVwiDAlkFAl2+OzErgQDDSUB2GkHVCUaaCrBXgjYaWx03MQpjoPxNvrP7oIOR+D0/647N4U2NOAGcaox3Gxwmzg3KEGb9SahPMzZaZBUWwCvRWSuU1qTyDwsQ84ppj8OpnvvIuE9wpFT8lktWGKMOYwGW10wFrYbkWKtMR90lEewze1ay4YDKHFwJKmCYz1HP/impMpNT4p8uyX9rbSVXrOsgj/0Ka7FUtqIyEk1SZMEKF/WW1NrDhGgR8BPO7pWtBoQYSskmSD+ybbAPWduFEjDfa4XmpxvvUuwzfx9XSjCLRdM91B+Q5LOt0zZfYtQzez2CKRpqe9er9Md7rQ4KDtuxzsvYYPBSa22CJGlhPnciDOAUvz7LUoszbIk3943EF4GaOMfOYbkLU1HpFJUMvfNQ1bTmskUgDA8V8WTdv/pm2InZw7zTPm8NPfBHIKeTVoQiEfmEeaT7vdgKZg+4BC4K0L1f2c9ZojmcbkZCy2ilhr1x6Qvl5JVQ9U03g2E+FwpzbzeYPIuMzlTD9JjIkIuUWLlZZbXslDZWsDIZ X-Forefront-Antispam-Report: CIP:216.228.112.35;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:schybrid04.nvidia.com;CAT:NONE;SFS:(4636009)(39860400002)(376002)(346002)(396003)(136003)(36840700001)(46966006)(426003)(36860700001)(70586007)(83380400001)(36756003)(107886003)(82740400003)(2906002)(70206006)(34020700004)(478600001)(82310400003)(186003)(336012)(8676002)(26005)(8936002)(356005)(2616005)(5660300002)(110136005)(86362001)(4326008)(54906003)(1076003)(7636003)(47076005)(316002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Mar 2021 08:34:57.7575 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7c9a7aaf-db9d-4dfa-f2c9-08d8e2d638f1 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.35];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT037.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1202 Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Create a new driver igd_vfio_pci.ko that will be responsible for providing special extensions for INTEL Graphics card (GVT-d). Also preserve backward compatibility with vfio_pci.ko vendor specific extensions. Signed-off-by: Max Gurtovoy --- drivers/vfio/pci/Kconfig | 5 +- drivers/vfio/pci/Makefile | 4 +- .../pci/{vfio_pci_igd.c => igd_vfio_pci.c} | 147 +++++++++++++++++- drivers/vfio/pci/igd_vfio_pci.h | 24 +++ drivers/vfio/pci/vfio_pci.c | 4 + drivers/vfio/pci/vfio_pci_core.c | 15 -- drivers/vfio/pci/vfio_pci_core.h | 9 -- 7 files changed, 176 insertions(+), 32 deletions(-) rename drivers/vfio/pci/{vfio_pci_igd.c => igd_vfio_pci.c} (62%) create mode 100644 drivers/vfio/pci/igd_vfio_pci.h diff --git a/drivers/vfio/pci/Kconfig b/drivers/vfio/pci/Kconfig index 88c89863a205..09d85ba3e5b2 100644 --- a/drivers/vfio/pci/Kconfig +++ b/drivers/vfio/pci/Kconfig @@ -37,17 +37,14 @@ config VFIO_PCI_INTX def_bool y if !S390 config VFIO_PCI_IGD - bool "VFIO PCI extensions for Intel graphics (GVT-d)" + tristate "VFIO PCI extensions for Intel graphics (GVT-d)" depends on VFIO_PCI_CORE && X86 - default y help Support for Intel IGD specific extensions to enable direct assignment to virtual machines. This includes exposing an IGD specific firmware table and read-only copies of the host bridge and LPC bridge config space. - To enable Intel IGD assignment through vfio-pci, say Y. - config VFIO_PCI_NVLINK2GPU tristate "VFIO support for NVIDIA NVLINK2 GPUs" depends on VFIO_PCI_CORE && PPC_POWERNV diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile index 86fb62e271fc..298b2fb3f075 100644 --- a/drivers/vfio/pci/Makefile +++ b/drivers/vfio/pci/Makefile @@ -4,9 +4,9 @@ obj-$(CONFIG_VFIO_PCI_CORE) += vfio-pci-core.o obj-$(CONFIG_VFIO_PCI) += vfio-pci.o obj-$(CONFIG_VFIO_PCI_NPU2) += npu2-vfio-pci.o obj-$(CONFIG_VFIO_PCI_NVLINK2GPU) += nvlink2gpu-vfio-pci.o +obj-$(CONFIG_VFIO_PCI_IGD) += igd-vfio-pci.o vfio-pci-core-y := vfio_pci_core.o vfio_pci_intrs.o vfio_pci_rdwr.o vfio_pci_config.o -vfio-pci-core-$(CONFIG_VFIO_PCI_IGD) += vfio_pci_igd.o vfio-pci-core-$(CONFIG_S390) += vfio_pci_zdev.o vfio-pci-y := vfio_pci.o @@ -14,3 +14,5 @@ vfio-pci-y := vfio_pci.o npu2-vfio-pci-y := npu2_vfio_pci.o nvlink2gpu-vfio-pci-y := nvlink2gpu_vfio_pci.o + +igd-vfio-pci-y := igd_vfio_pci.o diff --git a/drivers/vfio/pci/vfio_pci_igd.c b/drivers/vfio/pci/igd_vfio_pci.c similarity index 62% rename from drivers/vfio/pci/vfio_pci_igd.c rename to drivers/vfio/pci/igd_vfio_pci.c index 2388c9722ed8..bbbc432bca82 100644 --- a/drivers/vfio/pci/vfio_pci_igd.c +++ b/drivers/vfio/pci/igd_vfio_pci.c @@ -10,19 +10,32 @@ * address is also virtualized to prevent user modification. */ +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include #include #include +#include #include #include #include "vfio_pci_core.h" +#include "igd_vfio_pci.h" #define OPREGION_SIGNATURE "IntelGraphicsMem" #define OPREGION_SIZE (8 * 1024) #define OPREGION_PCI_ADDR 0xfc -static size_t vfio_pci_igd_rw(struct vfio_pci_core_device *vdev, char __user *buf, - size_t count, loff_t *ppos, bool iswrite) +#define DRIVER_VERSION "0.1" +#define DRIVER_AUTHOR "Alex Williamson " +#define DRIVER_DESC "IGD VFIO PCI - User Level meta-driver for Intel Graphics Processing Unit" + +struct igd_vfio_pci_device { + struct vfio_pci_core_device vdev; +}; + +static size_t vfio_pci_igd_rw(struct vfio_pci_core_device *vdev, + char __user *buf, size_t count, loff_t *ppos, bool iswrite) { unsigned int i = VFIO_PCI_OFFSET_TO_INDEX(*ppos) - VFIO_PCI_NUM_REGIONS; void *base = vdev->region[i].data; @@ -261,7 +274,7 @@ static int vfio_pci_igd_cfg_init(struct vfio_pci_core_device *vdev) return 0; } -int vfio_pci_igd_init(struct vfio_pci_core_device *vdev) +static int vfio_pci_igd_init(struct vfio_pci_core_device *vdev) { int ret; @@ -275,3 +288,131 @@ int vfio_pci_igd_init(struct vfio_pci_core_device *vdev) return 0; } + +static void igd_vfio_pci_release(void *device_data) +{ + struct vfio_pci_core_device *vdev = device_data; + + mutex_lock(&vdev->reflck->lock); + if (!(--vdev->refcnt)) { + vfio_pci_vf_token_user_add(vdev, -1); + vfio_pci_core_spapr_eeh_release(vdev); + vfio_pci_core_disable(vdev); + } + mutex_unlock(&vdev->reflck->lock); + + module_put(THIS_MODULE); +} + +static int igd_vfio_pci_open(void *device_data) +{ + struct vfio_pci_core_device *vdev = device_data; + int ret = 0; + + if (!try_module_get(THIS_MODULE)) + return -ENODEV; + + mutex_lock(&vdev->reflck->lock); + + if (!vdev->refcnt) { + ret = vfio_pci_core_enable(vdev); + if (ret) + goto error; + + ret = vfio_pci_igd_init(vdev); + if (ret && ret != -ENODEV) { + pci_warn(vdev->pdev, "Failed to setup Intel IGD regions\n"); + vfio_pci_core_disable(vdev); + goto error; + } + ret = 0; + vfio_pci_probe_mmaps(vdev); + vfio_pci_core_spapr_eeh_open(vdev); + vfio_pci_vf_token_user_add(vdev, 1); + } + vdev->refcnt++; +error: + mutex_unlock(&vdev->reflck->lock); + if (ret) + module_put(THIS_MODULE); + return ret; +} + +static const struct vfio_device_ops igd_vfio_pci_ops = { + .name = "igd-vfio-pci", + .open = igd_vfio_pci_open, + .release = igd_vfio_pci_release, + .ioctl = vfio_pci_core_ioctl, + .read = vfio_pci_core_read, + .write = vfio_pci_core_write, + .mmap = vfio_pci_core_mmap, + .request = vfio_pci_core_request, + .match = vfio_pci_core_match, +}; + +static int igd_vfio_pci_probe(struct pci_dev *pdev, + const struct pci_device_id *id) +{ + struct igd_vfio_pci_device *igvdev; + int ret; + + igvdev = kzalloc(sizeof(*igvdev), GFP_KERNEL); + if (!igvdev) + return -ENOMEM; + + ret = vfio_pci_core_register_device(&igvdev->vdev, pdev, + &igd_vfio_pci_ops); + if (ret) + goto out_free; + + return 0; + +out_free: + kfree(igvdev); + return ret; +} + +static void igd_vfio_pci_remove(struct pci_dev *pdev) +{ + struct vfio_device *vdev = dev_get_drvdata(&pdev->dev); + struct vfio_pci_core_device *core_vpdev = vfio_device_data(vdev); + struct igd_vfio_pci_device *igvdev; + + igvdev = container_of(core_vpdev, struct igd_vfio_pci_device, vdev); + + vfio_pci_core_unregister_device(core_vpdev); + kfree(igvdev); +} + +static const struct pci_device_id igd_vfio_pci_table[] = { + { PCI_VENDOR_ID_INTEL, PCI_ANY_ID, PCI_ANY_ID, PCI_ANY_ID, PCI_CLASS_DISPLAY_VGA << 8, 0xff0000, 0 }, + { 0, } +}; + +static struct pci_driver igd_vfio_pci_driver = { + .name = "igd-vfio-pci", + .id_table = igd_vfio_pci_table, + .probe = igd_vfio_pci_probe, + .remove = igd_vfio_pci_remove, +#ifdef CONFIG_PCI_IOV + .sriov_configure = vfio_pci_core_sriov_configure, +#endif + .err_handler = &vfio_pci_core_err_handlers, +}; + +#ifdef CONFIG_VFIO_PCI_DRIVER_COMPAT +struct pci_driver *get_igd_vfio_pci_driver(struct pci_dev *pdev) +{ + if (pci_match_id(igd_vfio_pci_driver.id_table, pdev)) + return &igd_vfio_pci_driver; + return NULL; +} +EXPORT_SYMBOL_GPL(get_igd_vfio_pci_driver); +#endif + +module_pci_driver(igd_vfio_pci_driver); + +MODULE_VERSION(DRIVER_VERSION); +MODULE_LICENSE("GPL v2"); +MODULE_AUTHOR(DRIVER_AUTHOR); +MODULE_DESCRIPTION(DRIVER_DESC); diff --git a/drivers/vfio/pci/igd_vfio_pci.h b/drivers/vfio/pci/igd_vfio_pci.h new file mode 100644 index 000000000000..859aeca354cb --- /dev/null +++ b/drivers/vfio/pci/igd_vfio_pci.h @@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c) 2020, Mellanox Technologies, Ltd. All rights reserved. + * Author: Max Gurtovoy + */ + +#ifndef IGD_VFIO_PCI_H +#define IGD_VFIO_PCI_H + +#include +#include + +#ifdef CONFIG_VFIO_PCI_DRIVER_COMPAT +#if defined(CONFIG_VFIO_PCI_IGD) || defined(CONFIG_VFIO_PCI_IGD_MODULE) +struct pci_driver *get_igd_vfio_pci_driver(struct pci_dev *pdev); +#else +struct pci_driver *get_igd_vfio_pci_driver(struct pci_dev *pdev) +{ + return NULL; +} +#endif +#endif + +#endif /* IGD_VFIO_PCI_H */ diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index 8e81ea039f31..1c2f6d55a243 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -30,6 +30,7 @@ #ifdef CONFIG_VFIO_PCI_DRIVER_COMPAT #include "npu2_vfio_pci.h" #include "nvlink2gpu_vfio_pci.h" +#include "igd_vfio_pci.h" #endif #define DRIVER_VERSION "0.2" @@ -170,6 +171,9 @@ static struct pci_driver *vfio_pci_get_compat_driver(struct pci_dev *pdev) default: return NULL; } + case PCI_VENDOR_ID_INTEL: + if (pdev->class == PCI_CLASS_DISPLAY_VGA << 8) + return get_igd_vfio_pci_driver(pdev); } return NULL; diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index f9b39abe54cb..59c9d0d56a0b 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -343,22 +343,7 @@ int vfio_pci_core_enable(struct vfio_pci_core_device *vdev) if (!vfio_vga_disabled() && vfio_pci_is_vga(pdev)) vdev->has_vga = true; - - if (vfio_pci_is_vga(pdev) && - pdev->vendor == PCI_VENDOR_ID_INTEL && - IS_ENABLED(CONFIG_VFIO_PCI_IGD)) { - ret = vfio_pci_igd_init(vdev); - if (ret && ret != -ENODEV) { - pci_warn(pdev, "Failed to setup Intel IGD regions\n"); - goto disable_exit; - } - } - return 0; - -disable_exit: - vfio_pci_disable(vdev); - return ret; } EXPORT_SYMBOL_GPL(vfio_pci_core_enable); diff --git a/drivers/vfio/pci/vfio_pci_core.h b/drivers/vfio/pci/vfio_pci_core.h index 31f3836e606e..2b5ea0db9284 100644 --- a/drivers/vfio/pci/vfio_pci_core.h +++ b/drivers/vfio/pci/vfio_pci_core.h @@ -196,15 +196,6 @@ extern u16 vfio_pci_memory_lock_and_enable(struct vfio_pci_core_device *vdev); extern void vfio_pci_memory_unlock_and_restore(struct vfio_pci_core_device *vdev, u16 cmd); -#ifdef CONFIG_VFIO_PCI_IGD -extern int vfio_pci_igd_init(struct vfio_pci_core_device *vdev); -#else -static inline int vfio_pci_igd_init(struct vfio_pci_core_device *vdev) -{ - return -ENODEV; -} -#endif - #ifdef CONFIG_S390 extern int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_device *vdev, struct vfio_info_cap *caps);