From patchwork Tue May 16 02:55:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13242499 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B44D53C00 for ; Tue, 16 May 2023 02:55:50 +0000 (UTC) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on20604.outbound.protection.outlook.com [IPv6:2a01:111:f400:7eaa::604]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7021F3C1F for ; Mon, 15 May 2023 19:55:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XNYFu6YCbU7V7dVz+0KWexrQCWiSus7hRn7N0RlkTV3ZSdfBtpuW88td3HNjgaRuxmDlW2Xuy4AeH9LE76G+zV0qKAtr8yZTThHIDXtXX2WKW2v2cQbLtC68FogrZ/np76LDgkjJM5OTviaLfQnfK67fHm0rpr293UPxo13XhnM2uc909X0hfjvqrfqNSedH+17zfCeuskAFe1ySaPzs2opvPXSF3CUcW5lZui9iOyW2E0AjbTSaz5c+N7Ev70BC/UKuDF0nMrH4P1uJ/ntOVL9Bc+8T0CpIsBf+NN+zRy8rqQ/UMlBrnNEes8yJqGdziLowHmetneTuACZN/7nO0Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nDIXo4UNsCDm56zbBLhiWWii7G2lJjFG3pskVzCWRQQ=; b=LXtemATJKKTqj32+r70iQEzvyLAwEnoxJt0YLVDWMPAg/VON2O8Ge7V1gm98pS6KmlYGhFzmU8Hc3JJN41fQkTS4xOZ77F84KJC7CeJUUUJTSDBj6csY5hw1mLpkdIraTAMUw51lNAAHxlENBBiT0efthZ1zbfRBBKhVm0kG6aQc4nt4vFnci3tBrHqKo0/DR+bAS5LbjIuexBkjEBd6ZuSw8NdAS6/3vKuyLqzKsKaaDA0nvgYbg55r/xkF4UhNJp2jiK3/ens+7+rfG6i+F3yFVpLDZ/WCQuO0CLNML5wvGsqoFZmMKWuo20jrg1Fd2aPJVB7kGIWy9OC1dynRGw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nDIXo4UNsCDm56zbBLhiWWii7G2lJjFG3pskVzCWRQQ=; b=X2QC3BYstywMulkMT2PWGovU4YbYk08AAzkAiebFFslms5Ez+bEc3/scsCMtZ7KLJT2qnMcaKsE0XDzItLppU6RuwNNQ2OWdwRj6vurG9os91sPeR0xM5gZt4RI5ZKu0ZTxv22PxnqVmkUxVKRgxgsAcngGkcqRTI/eCQM4vdoc= Received: from SJ0PR03CA0027.namprd03.prod.outlook.com (2603:10b6:a03:33a::32) by PH7PR12MB8056.namprd12.prod.outlook.com (2603:10b6:510:269::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30; Tue, 16 May 2023 02:55:44 +0000 Received: from DM6NAM11FT096.eop-nam11.prod.protection.outlook.com (2603:10b6:a03:33a:cafe::48) by SJ0PR03CA0027.outlook.office365.com (2603:10b6:a03:33a::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30 via Frontend Transport; Tue, 16 May 2023 02:55:44 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT096.mail.protection.outlook.com (10.13.173.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6411.15 via Frontend Transport; Tue, 16 May 2023 02:55:44 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 15 May 2023 21:55:42 -0500 From: Shannon Nelson To: , , , , , CC: , Subject: [PATCH v6 virtio 01/11] virtio: allow caller to override device id in vp_modern Date: Mon, 15 May 2023 19:55:11 -0700 Message-ID: <20230516025521.43352-2-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230516025521.43352-1-shannon.nelson@amd.com> References: <20230516025521.43352-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT096:EE_|PH7PR12MB8056:EE_ X-MS-Office365-Filtering-Correlation-Id: 04dce377-c6e9-44bb-74e2-08db55b90ad9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yE1TTDcazG4Amqw/WLPN/V/Evs+wRkGEWjGCVK5QEvXHgPTyMZ+SY79xi0l/2gWDtdq5skdCYVdbks2tSnA6vhI8CaAZzJd8I86OMINBvitVv+ECREruhEuC0TUAP/a/KL8ispcqMes41400a5rckqSLtTHGK6spKWcLH3I+6ZxF+AX+ETX/aion2jgApPmxfjYytf3F5Pk4ASmcAEM4WgwERlNr7cwiZ+uPpcSzJJTbQXWtMgCpy70aCxVdcnp4Ba/1jbVL380UPefDu1HpMnmmhp6ziC661fGsJa4X/zPg3UFL6sSR97uuFGfnqxeSZ5bCqnGv3UXArNqF3hzQc8dW9W6IU7T1KiMwdVp4SfcZL5QR/Wx2+D0Lac+cKGnxViPn4zoIPB88tsokTn+c70ie9IYC9erkcAT6313djXJfsxcbUZ7Q+51tnyF933dzfHEEV0DZD0tG8m64A9RDB1L5DbCM/FSx0knkIdsDbFnca4vpIPIatgFL/C/jZGKFMiTdg0pkvcBq83CIh85Eoof2t7UbYrDvcz2srbg96XSc1LoEG27WHJOcEReIvufPTAT2ZCf8wD9I3fZp7jJ48kWjXaONwOPrDqHDL2LWfl1lKw26wPdGkUVg3XV+TUnSupZm4wFD+p6QpGxna2+LRw+Gj/7/F8Np172p0LMAmfN6uBVV/WGXMS5hIa6wsH30O5InhGbGsbR/XYbOoMavus3FZXQSLfA4H1Dc8VE79L+sLh+lVni986jmeYyNwp9wxOfHBS4aXNXBFtOL/4ao1g== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(376002)(39860400002)(346002)(136003)(396003)(451199021)(36840700001)(46966006)(40470700004)(82310400005)(83380400001)(36860700001)(336012)(47076005)(426003)(4326008)(82740400003)(356005)(36756003)(81166007)(41300700001)(26005)(40480700001)(478600001)(1076003)(316002)(2616005)(70206006)(70586007)(8676002)(8936002)(40460700003)(5660300002)(44832011)(110136005)(86362001)(16526019)(2906002)(54906003)(186003)(6666004)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2023 02:55:44.1610 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 04dce377-c6e9-44bb-74e2-08db55b90ad9 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT096.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB8056 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,SPF_HELO_PASS, SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net To add a bit of vendor flexibility with various virtio based devices, allow the caller to check for a different device id. This adds a function pointer field to struct virtio_pci_modern_device to specify an override device id check. If defined by the driver, this function will be called to check that the PCI device is the vendor's expected device, and will return the found device id to be stored in mdev->id.device. This allows vendors with alternative vendor device ids to use this library on their own device BAR. Note: A lot of the diff in this is simply indenting the existing code into an else block. Signed-off-by: Shannon Nelson Acked-by: Jason Wang --- drivers/virtio/virtio_pci_modern_dev.c | 30 ++++++++++++++++---------- include/linux/virtio_pci_modern.h | 3 +++ 2 files changed, 22 insertions(+), 11 deletions(-) diff --git a/drivers/virtio/virtio_pci_modern_dev.c b/drivers/virtio/virtio_pci_modern_dev.c index 869cb46bef96..9b2d6614de67 100644 --- a/drivers/virtio/virtio_pci_modern_dev.c +++ b/drivers/virtio/virtio_pci_modern_dev.c @@ -218,21 +218,29 @@ int vp_modern_probe(struct virtio_pci_modern_device *mdev) int err, common, isr, notify, device; u32 notify_length; u32 notify_offset; + int devid; check_offsets(); - /* We only own devices >= 0x1000 and <= 0x107f: leave the rest. */ - if (pci_dev->device < 0x1000 || pci_dev->device > 0x107f) - return -ENODEV; - - if (pci_dev->device < 0x1040) { - /* Transitional devices: use the PCI subsystem device id as - * virtio device id, same as legacy driver always did. - */ - mdev->id.device = pci_dev->subsystem_device; + if (mdev->device_id_check) { + devid = mdev->device_id_check(pci_dev); + if (devid < 0) + return devid; + mdev->id.device = devid; } else { - /* Modern devices: simply use PCI device id, but start from 0x1040. */ - mdev->id.device = pci_dev->device - 0x1040; + /* We only own devices >= 0x1000 and <= 0x107f: leave the rest. */ + if (pci_dev->device < 0x1000 || pci_dev->device > 0x107f) + return -ENODEV; + + if (pci_dev->device < 0x1040) { + /* Transitional devices: use the PCI subsystem device id as + * virtio device id, same as legacy driver always did. + */ + mdev->id.device = pci_dev->subsystem_device; + } else { + /* Modern devices: simply use PCI device id, but start from 0x1040. */ + mdev->id.device = pci_dev->device - 0x1040; + } } mdev->id.vendor = pci_dev->subsystem_vendor; diff --git a/include/linux/virtio_pci_modern.h b/include/linux/virtio_pci_modern.h index c4eeb79b0139..e7b1db1dd0bb 100644 --- a/include/linux/virtio_pci_modern.h +++ b/include/linux/virtio_pci_modern.h @@ -38,6 +38,9 @@ struct virtio_pci_modern_device { int modern_bars; struct virtio_device_id id; + + /* optional check for vendor virtio device, returns dev_id or -ERRNO */ + int (*device_id_check)(struct pci_dev *pdev); }; /* From patchwork Tue May 16 02:55:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13242498 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 66A01256C for ; Tue, 16 May 2023 02:55:50 +0000 (UTC) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2075.outbound.protection.outlook.com [40.107.244.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3844EE58 for ; Mon, 15 May 2023 19:55:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=U3ayax8wkWATmLtOb1whI04ZJjFlWe9pg9YMrdB8+e/y+VFdZsCYBihFqKTzbyJ4iBlj8bPQE1gwVAn9nhnxmiiwl3grQYHBhoLm300cgGiqsWS79CMaLuipVe2fHzeopTGsmAqSOQ6/cGy/Eow3nCi6roPAjC+cgYdJokcbXr0uzr6gJTgPbqrZp6t0MzabXUiEmnCrm8GFwO1u9+kbgDpvZS67fKIJn4+rHEW2WMdY65snroSlhukIW5+YOX6AJdiAYScRFkaB4S9pXJSgn0qV0A+ujmCHS+78Eds3Xedq7lZ7+LdkVk70blnryJd+W9wQAYvtmcFRM9SGEiIxaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=u7M0ZM7UHOdU4Esi2UwbRjhF2g4heOLasFC79+b2hs4=; b=k3yTsmgVPoPWo9qnXGOY/c+NS/qJad5jFpgcPVOg4MjnCd4JS1647whQBR1wMZoSA0ztqo/rtwJfOFHZvXQEbKpSb2nDUgE8FFSNU8vR005huA0bZMY6y1xa2RB+zRj8xeYP6+4IIkN3OTCtKJhTYP1oUKVhVujsBvjgnNGSulCT0qs2Mk9wPPb0/71CRrq5KSYEom2Dxc6Np0qnA4vugoBbxj4rt8dhDcUkFXUMCVEYDR+CQB37rmmF4NP0EaRq/QoznSa55TPahSS5KavIfJMQrWd9L6eRo1QMXh05rRPm6Tgyqnl1y+gtItpw/Ya0wGvJI3ieFa7Vy7uXZQnTnw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=u7M0ZM7UHOdU4Esi2UwbRjhF2g4heOLasFC79+b2hs4=; b=tVKDMmoJCdSpl9+16GgNPyjMlz3y5ydfXW63fr/WoVQpabfT0Q1jjEKsfDdFMQuUsHJE+9AbegrMeKLLcuJSTHSqyXPqInh7kyRvVWtndfBeilswc7OTecIDL6RvpmrxviAImV83vhFIrauM4dEG1mQ99dtKs9hq+Qm74ljR2iQ= Received: from SJ0PR03CA0015.namprd03.prod.outlook.com (2603:10b6:a03:33a::20) by IA1PR12MB6409.namprd12.prod.outlook.com (2603:10b6:208:38b::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30; Tue, 16 May 2023 02:55:46 +0000 Received: from DM6NAM11FT096.eop-nam11.prod.protection.outlook.com (2603:10b6:a03:33a:cafe::d3) by SJ0PR03CA0015.outlook.office365.com (2603:10b6:a03:33a::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30 via Frontend Transport; Tue, 16 May 2023 02:55:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT096.mail.protection.outlook.com (10.13.173.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6411.15 via Frontend Transport; Tue, 16 May 2023 02:55:45 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 15 May 2023 21:55:43 -0500 From: Shannon Nelson To: , , , , , CC: , Subject: [PATCH v6 virtio 02/11] virtio: allow caller to override device DMA mask in vp_modern Date: Mon, 15 May 2023 19:55:12 -0700 Message-ID: <20230516025521.43352-3-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230516025521.43352-1-shannon.nelson@amd.com> References: <20230516025521.43352-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT096:EE_|IA1PR12MB6409:EE_ X-MS-Office365-Filtering-Correlation-Id: 6c462afa-738f-43b0-1541-08db55b90be2 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: UIoTKxs+uDMdlXSYFWeAHVQYXEbYeRLwg+z/eKDoJk38ZjOzcHmWFO5dewYCvZOldnLi/vjw0UAEwFn4N9vD5rVvtUfQBu01WMGnSmSIjjsnxv9ZLvuewecyfidV8oEyFZ52CRR/2InRLkLg35i0XIQc5RHdGqokKxpjA5nxvbOSNDRR3de0HvWRLjtzxLc7iUIQu6XEpvgV8j2i0w+koInJSKVyVj3uafEcU/mp9fgCLCdbKH3Q4Wl1Zr2OKWlZoHQhLJXT93dOiI2IijQTMWOwljqorKYt2v04HNnTRLwVmLKHBrgL8eB5SRXlrNcYIs1ZBJ4lUKOvUrgZ/806iqfN/z3Tkz5xTJ2qsdHvuIATLh7U2vXX0xnf8fgVFk0r6BnMBTX+dHMqvggM9cTDKSZo6vrkSwVxyS3zhooIIuLJUxMqRHY0GLGHFcskCLtiYWwhwG81tnZeduiHgutX8ROrJyNSNu1JdRyZstJuFeV5hfcbBE1K8bSurjdB+Wi3rZ6XA6U88mhjXexXHEIZR0Ij2fMvtFitQzifaB2A/f7pj+EHxPVj+rqBpK+3QFOh6WyRLPwl8c5UvD6vvFvuX1tfj1cxBrhePZ/8GBalFkiTHNSZbIFg6i+Mpw03zkKHHRbjHNV+tXirMij87v0YfSgK0sA/244XCItInL7vrYpGLwJEErdxi8QonnmTmCQT8NC0PhHnThrBkBSQTMxMEdeW9UoCO3OD2CQjo6EsWIDx0tdYpNyfi0R2B607oeNCAlKGgh5Rt/fm2liT19mbJg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(39860400002)(376002)(346002)(136003)(396003)(451199021)(40470700004)(36840700001)(46966006)(47076005)(36860700001)(186003)(16526019)(2616005)(41300700001)(6666004)(426003)(336012)(1076003)(83380400001)(26005)(478600001)(54906003)(110136005)(40460700003)(70586007)(4326008)(70206006)(82740400003)(40480700001)(81166007)(316002)(356005)(8676002)(5660300002)(36756003)(8936002)(44832011)(86362001)(82310400005)(2906002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2023 02:55:45.8952 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6c462afa-738f-43b0-1541-08db55b90be2 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT096.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6409 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net To add a bit of vendor flexibility with various virtio based devices, allow the caller to specify a different DMA mask. This adds a dma_mask field to struct virtio_pci_modern_device. If defined by the driver, this mask will be used in a call to dma_set_mask_and_coherent() instead of the traditional DMA_BIT_MASK(64). This allows limiting the DMA space on vendor devices with address limitations. Signed-off-by: Shannon Nelson Acked-by: Jason Wang --- drivers/virtio/virtio_pci_modern_dev.c | 3 ++- include/linux/virtio_pci_modern.h | 3 +++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/virtio/virtio_pci_modern_dev.c b/drivers/virtio/virtio_pci_modern_dev.c index 9b2d6614de67..aad7d9296e77 100644 --- a/drivers/virtio/virtio_pci_modern_dev.c +++ b/drivers/virtio/virtio_pci_modern_dev.c @@ -268,7 +268,8 @@ int vp_modern_probe(struct virtio_pci_modern_device *mdev) return -EINVAL; } - err = dma_set_mask_and_coherent(&pci_dev->dev, DMA_BIT_MASK(64)); + err = dma_set_mask_and_coherent(&pci_dev->dev, + mdev->dma_mask ? : DMA_BIT_MASK(64)); if (err) err = dma_set_mask_and_coherent(&pci_dev->dev, DMA_BIT_MASK(32)); diff --git a/include/linux/virtio_pci_modern.h b/include/linux/virtio_pci_modern.h index e7b1db1dd0bb..067ac1d789bc 100644 --- a/include/linux/virtio_pci_modern.h +++ b/include/linux/virtio_pci_modern.h @@ -41,6 +41,9 @@ struct virtio_pci_modern_device { /* optional check for vendor virtio device, returns dev_id or -ERRNO */ int (*device_id_check)(struct pci_dev *pdev); + + /* optional mask for devices with limited DMA space */ + u64 dma_mask; }; /* From patchwork Tue May 16 02:55:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13242502 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B64185245 for ; Tue, 16 May 2023 02:55:53 +0000 (UTC) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2084.outbound.protection.outlook.com [40.107.93.84]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0447740D0 for ; Mon, 15 May 2023 19:55:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=B6ahQTVRdh+R/kBqKwfJgJC/KYPZU3EcKrQQB8Easv2Kq6p69ERY8hPkbAi4IRMYiQNM31CSwVtZKtLmDHS8ux2k+vPO/AstmiMRbPugHPzYMgEsIJv6f2GK7Xhcl27GXSCDtQfT0j/k95I2pDlNNcRsDb6MTa6s2uoRzCV9JaHzbFaEfGhKOkEX+dtk2NTthw0XIgn6R/XW9hXpY4uO0Uh7cFYiyIeDZNiDQJVHHzWR8mArdwFXF2sN42AjsWRuQESWyt+IjeE3azVpMu3E1hRq0kpqSbbwxLBK4vDTR86OK9Ku2RrvhwWoYcMNTQUKOQcU/Od1jEBobHYNa46mvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hO1OVx/y4DC3hfqDreSo0ry+yyEDQnOq9eb9/2WQ+PA=; b=ayA0wIIZmPAQr5OHQ9pmmRmXW2G71IYmy2ziMAp++Sc+A2xQfbHQV08n+zx0tUoftM9vxTHsaDHUQk0vh11SVpUC4A+lXKlKH/lt7BdBCBgosndk4H0BXCyNAH5AOmvW8BCSKNH2+aii/9K3ad+goWzT9pZ0eH6oOwon87JntJAFqiueaqv/2V60v7suqEAEh+MQ3411qMatphkooTtq7xIUwMaduHEhG/EuWmMlJY3c/5sBsRNvQIaA5J3L7WO8QCRA5qRfIIGt4z6UjoZACTiA4nREE5teTfNXV31hyjNng4wBNZMDuC2BfZ4JlO2JF5bPqiKEDBxg1bTkfCVQIg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hO1OVx/y4DC3hfqDreSo0ry+yyEDQnOq9eb9/2WQ+PA=; b=fGkFj4cmMMisccQ0dC+IAIW2Op5g2Roxr3eUkeuOPEnWHjlehAGuLLsLnjbHzK/6T5XAWd1kvBJfSc3guFh41vTHk2HFrCISxg0kEYq10Q5AY4pAOAz23m0lVEu8OV8cFNrgve4cCPMZv1g848fjSyRNqWzUwUlpS/FqzHNktAc= Received: from SJ0PR03CA0013.namprd03.prod.outlook.com (2603:10b6:a03:33a::18) by DS7PR12MB8370.namprd12.prod.outlook.com (2603:10b6:8:eb::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30; Tue, 16 May 2023 02:55:47 +0000 Received: from DM6NAM11FT096.eop-nam11.prod.protection.outlook.com (2603:10b6:a03:33a:cafe::6a) by SJ0PR03CA0013.outlook.office365.com (2603:10b6:a03:33a::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30 via Frontend Transport; Tue, 16 May 2023 02:55:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT096.mail.protection.outlook.com (10.13.173.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6411.15 via Frontend Transport; Tue, 16 May 2023 02:55:47 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 15 May 2023 21:55:44 -0500 From: Shannon Nelson To: , , , , , CC: , Subject: [PATCH v6 virtio 03/11] pds_vdpa: Add new vDPA driver for AMD/Pensando DSC Date: Mon, 15 May 2023 19:55:13 -0700 Message-ID: <20230516025521.43352-4-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230516025521.43352-1-shannon.nelson@amd.com> References: <20230516025521.43352-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT096:EE_|DS7PR12MB8370:EE_ X-MS-Office365-Filtering-Correlation-Id: cd91705b-9d8c-4886-305d-08db55b90ca3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: SixdRkGNArDLmUPvszPw7XyQ16aP6MPqq27351IvoZIs4yFVP7TQMogKp21lu22xDKCI3f0DYD38QJUE/Ee6caccbQP2Q5Ms980d5AgqlD7QDVLNmD5nuzV8WBfHymHZP1XvFYOFH+6ojWaGYbHELwvqSYxudWp07JimQzwSyVd55S+lp1k2Ywr/9JOWopF1IIZvfDVD16+JNLehQwxS6K5WCjdRoVWDur737iY/HeahzO1gQG88oqYNZ1BqeaOhdYCAZEBVM8rKmNFkjZm3GByFiOWmoXBF36G4nIjrN7rE1kGjdIJT+LgoLxlg+wtjsxetSGqyFA6oxm/VWVuiQfJ+zcdlZsvaKDHi3LLCzma4LQoxy7dK4FVIlFjzan97VR6OeUWLYLh9M7J6V6ebW5puPXL3FNYMZSVQieYpYhvf/ZOV6JeJ/IKWMcRkl50/V90ZnPOmraYfphlY5j3+b1F7NqVvFsocFziCV82/ywYaFSK5i/canwtlY38KMCpYl0amwtaCWjJPzsU7SgxpC/DOp9dIFw+Q89Z8394v1EaeST4fmuxpdEvNyC3iilLRKGURP9qfxau7SMOW0ZBVImQ2TG3GQm9luKWiW+cnpT+W7DhL4OboWY7aSO5bri5A21x+eQ/zqp+KFkGRXUS+txwU14T7tZT+f2wuuYwEYF3QEdMzs9Du435GR7T0J1kNaESoFyTwMIidQL+xoGm5TkR41T1UKQLRSSp9rf3ApU/WlObjkHBYkA1aBKaYb1PPTjwqaetVoH7CApoVh4dHiQ== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(39860400002)(136003)(396003)(376002)(451199021)(36840700001)(40470700004)(46966006)(70206006)(4326008)(478600001)(70586007)(110136005)(54906003)(186003)(16526019)(40460700003)(83380400001)(426003)(2616005)(336012)(47076005)(36860700001)(36756003)(8936002)(8676002)(40480700001)(82310400005)(2906002)(44832011)(41300700001)(6666004)(316002)(356005)(86362001)(81166007)(5660300002)(82740400003)(26005)(1076003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2023 02:55:47.1451 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cd91705b-9d8c-4886-305d-08db55b90ca3 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT096.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB8370 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net This is the initial auxiliary driver framework for a new vDPA device driver, an auxiliary_bus client of the pds_core driver. The pds_core driver supplies the PCI services for the VF device and for accessing the adminq in the PF device. This patch adds the very basics of registering for the auxiliary device and setting up debugfs entries. Signed-off-by: Shannon Nelson Acked-by: Jason Wang --- drivers/vdpa/Makefile | 1 + drivers/vdpa/pds/Makefile | 8 ++++ drivers/vdpa/pds/aux_drv.c | 83 ++++++++++++++++++++++++++++++++++ drivers/vdpa/pds/aux_drv.h | 15 ++++++ drivers/vdpa/pds/debugfs.c | 25 ++++++++++ drivers/vdpa/pds/debugfs.h | 12 +++++ include/linux/pds/pds_common.h | 2 + 7 files changed, 146 insertions(+) create mode 100644 drivers/vdpa/pds/Makefile create mode 100644 drivers/vdpa/pds/aux_drv.c create mode 100644 drivers/vdpa/pds/aux_drv.h create mode 100644 drivers/vdpa/pds/debugfs.c create mode 100644 drivers/vdpa/pds/debugfs.h diff --git a/drivers/vdpa/Makefile b/drivers/vdpa/Makefile index 59396ff2a318..8f53c6f3cca7 100644 --- a/drivers/vdpa/Makefile +++ b/drivers/vdpa/Makefile @@ -7,3 +7,4 @@ obj-$(CONFIG_MLX5_VDPA) += mlx5/ obj-$(CONFIG_VP_VDPA) += virtio_pci/ obj-$(CONFIG_ALIBABA_ENI_VDPA) += alibaba/ obj-$(CONFIG_SNET_VDPA) += solidrun/ +obj-$(CONFIG_PDS_VDPA) += pds/ diff --git a/drivers/vdpa/pds/Makefile b/drivers/vdpa/pds/Makefile new file mode 100644 index 000000000000..a9cd2f450ae1 --- /dev/null +++ b/drivers/vdpa/pds/Makefile @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright(c) 2023 Advanced Micro Devices, Inc + +obj-$(CONFIG_PDS_VDPA) := pds_vdpa.o + +pds_vdpa-y := aux_drv.o + +pds_vdpa-$(CONFIG_DEBUG_FS) += debugfs.o diff --git a/drivers/vdpa/pds/aux_drv.c b/drivers/vdpa/pds/aux_drv.c new file mode 100644 index 000000000000..e4a0ad61ea22 --- /dev/null +++ b/drivers/vdpa/pds/aux_drv.c @@ -0,0 +1,83 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2023 Advanced Micro Devices, Inc */ + +#include +#include + +#include +#include +#include +#include + +#include "aux_drv.h" +#include "debugfs.h" + +static const struct auxiliary_device_id pds_vdpa_id_table[] = { + { .name = PDS_VDPA_DEV_NAME, }, + {}, +}; + +static int pds_vdpa_probe(struct auxiliary_device *aux_dev, + const struct auxiliary_device_id *id) + +{ + struct pds_auxiliary_dev *padev = + container_of(aux_dev, struct pds_auxiliary_dev, aux_dev); + struct pds_vdpa_aux *vdpa_aux; + + vdpa_aux = kzalloc(sizeof(*vdpa_aux), GFP_KERNEL); + if (!vdpa_aux) + return -ENOMEM; + + vdpa_aux->padev = padev; + auxiliary_set_drvdata(aux_dev, vdpa_aux); + + return 0; +} + +static void pds_vdpa_remove(struct auxiliary_device *aux_dev) +{ + struct pds_vdpa_aux *vdpa_aux = auxiliary_get_drvdata(aux_dev); + struct device *dev = &aux_dev->dev; + + kfree(vdpa_aux); + auxiliary_set_drvdata(aux_dev, NULL); + + dev_info(dev, "Removed\n"); +} + +static struct auxiliary_driver pds_vdpa_driver = { + .name = PDS_DEV_TYPE_VDPA_STR, + .probe = pds_vdpa_probe, + .remove = pds_vdpa_remove, + .id_table = pds_vdpa_id_table, +}; + +static void __exit pds_vdpa_cleanup(void) +{ + auxiliary_driver_unregister(&pds_vdpa_driver); + + pds_vdpa_debugfs_destroy(); +} +module_exit(pds_vdpa_cleanup); + +static int __init pds_vdpa_init(void) +{ + int err; + + pds_vdpa_debugfs_create(); + + err = auxiliary_driver_register(&pds_vdpa_driver); + if (err) { + pr_err("%s: aux driver register failed: %pe\n", + PDS_VDPA_DRV_NAME, ERR_PTR(err)); + pds_vdpa_debugfs_destroy(); + } + + return err; +} +module_init(pds_vdpa_init); + +MODULE_DESCRIPTION(PDS_VDPA_DRV_DESCRIPTION); +MODULE_AUTHOR("Advanced Micro Devices, Inc"); +MODULE_LICENSE("GPL"); diff --git a/drivers/vdpa/pds/aux_drv.h b/drivers/vdpa/pds/aux_drv.h new file mode 100644 index 000000000000..f1e99359424e --- /dev/null +++ b/drivers/vdpa/pds/aux_drv.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc */ + +#ifndef _AUX_DRV_H_ +#define _AUX_DRV_H_ + +#define PDS_VDPA_DRV_DESCRIPTION "AMD/Pensando vDPA VF Device Driver" +#define PDS_VDPA_DRV_NAME KBUILD_MODNAME + +struct pds_vdpa_aux { + struct pds_auxiliary_dev *padev; + + struct dentry *dentry; +}; +#endif /* _AUX_DRV_H_ */ diff --git a/drivers/vdpa/pds/debugfs.c b/drivers/vdpa/pds/debugfs.c new file mode 100644 index 000000000000..5be22fb7a76a --- /dev/null +++ b/drivers/vdpa/pds/debugfs.c @@ -0,0 +1,25 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2023 Advanced Micro Devices, Inc */ + +#include + +#include +#include +#include +#include + +#include "aux_drv.h" +#include "debugfs.h" + +static struct dentry *dbfs_dir; + +void pds_vdpa_debugfs_create(void) +{ + dbfs_dir = debugfs_create_dir(PDS_VDPA_DRV_NAME, NULL); +} + +void pds_vdpa_debugfs_destroy(void) +{ + debugfs_remove_recursive(dbfs_dir); + dbfs_dir = NULL; +} diff --git a/drivers/vdpa/pds/debugfs.h b/drivers/vdpa/pds/debugfs.h new file mode 100644 index 000000000000..658849591a99 --- /dev/null +++ b/drivers/vdpa/pds/debugfs.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc */ + +#ifndef _PDS_VDPA_DEBUGFS_H_ +#define _PDS_VDPA_DEBUGFS_H_ + +#include + +void pds_vdpa_debugfs_create(void); +void pds_vdpa_debugfs_destroy(void); + +#endif /* _PDS_VDPA_DEBUGFS_H_ */ diff --git a/include/linux/pds/pds_common.h b/include/linux/pds/pds_common.h index 060331486d50..2a0d1669cfd0 100644 --- a/include/linux/pds/pds_common.h +++ b/include/linux/pds/pds_common.h @@ -39,6 +39,8 @@ enum pds_core_vif_types { #define PDS_DEV_TYPE_RDMA_STR "RDMA" #define PDS_DEV_TYPE_LM_STR "LM" +#define PDS_VDPA_DEV_NAME PDS_CORE_DRV_NAME "." PDS_DEV_TYPE_VDPA_STR + #define PDS_CORE_IFNAMSIZ 16 /** From patchwork Tue May 16 02:55:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13242500 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0948E522E for ; Tue, 16 May 2023 02:55:52 +0000 (UTC) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2064.outbound.protection.outlook.com [40.107.92.64]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 98F5B3C1F for ; Mon, 15 May 2023 19:55:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SWhY84oyL0/4WYNem7Xx1E6W4lDRvLPUcfZqMqfYw7EnpuR286giwjKBCGWR5ksT10i+3yAjwPgfIyFiyAbWwZAMcwM5xRQBUYFwTyDP8wz2SYDFp9epiBjcmNQs5bN7BdRnhEELFcFuDAEMTNSIIgNyAv/VBSCYHokVvbO0gsPD9m9+e3/Th+CN9x4zobBPTXabJn8Cw3vzTyzzrka16leKCN4B657V4HN9xgZtvtlKqpB0JDKmstG3dZuG87GnDPV53Sq9+7DPrbrr/0itw/fj2BYk1EVCSh6oYAzUlfRtix7+edxDsGpckJo1SxUjkd+QyDmqbzjm2QdWuBr/EA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bjLTOwD8kyi9Vm6e95kDwSNjpbT8accAL9JbwovDCuo=; b=gEcpRk3quEv/zc1cRvqUaWSTX1ZXwKDSZj+pbO58ivgNxS4jIXTslJbJUMWP7EmLwcfIz/rdN6qlK/4RvP4QksPrn1D6VQNmJqw18oApYcVDnxsViCzCfg9O4sFILzFNWnScZE6p5v992udrZQP0IrwIr99bac3XOese9wtv4QjWSk9tstD/fpvXIXoXDL9isLmV9d0uYXC5zCXp8WnnfVciIdF2zYStqVRgOT5/sMcjrReibg4wrVi72U6TNz5xxDKP8nGCc+qvj+SrhTfbkBa8bR8KMs1w3+co25XPMkHn6yp3q5i7puzpyUEXLD9tPfy6r9cVmnaRTftmVTdyHQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bjLTOwD8kyi9Vm6e95kDwSNjpbT8accAL9JbwovDCuo=; b=Zm5gcfD9KeMpdD4rS/CSm6Vdj6fNN8xildEENNqxl1OgIIuZ0hJIZJIESo1dMpXvxyC2Zc/JSOQtqM3hPMkq0qquG7W/hQZeEEBrT9D2OcGNIkm/XwsKsrXHuV4U77+AnkTRFQKUSsaGSyNDN1awTnFicCErA/4OBOabOphFaUA= Received: from SJ0PR03CA0027.namprd03.prod.outlook.com (2603:10b6:a03:33a::32) by CY5PR12MB6155.namprd12.prod.outlook.com (2603:10b6:930:25::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.33; Tue, 16 May 2023 02:55:48 +0000 Received: from DM6NAM11FT096.eop-nam11.prod.protection.outlook.com (2603:10b6:a03:33a:cafe::48) by SJ0PR03CA0027.outlook.office365.com (2603:10b6:a03:33a::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30 via Frontend Transport; Tue, 16 May 2023 02:55:48 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT096.mail.protection.outlook.com (10.13.173.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6411.15 via Frontend Transport; Tue, 16 May 2023 02:55:47 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 15 May 2023 21:55:45 -0500 From: Shannon Nelson To: , , , , , CC: , Subject: [PATCH v6 virtio 04/11] pds_vdpa: move enum from common to adminq header Date: Mon, 15 May 2023 19:55:14 -0700 Message-ID: <20230516025521.43352-5-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230516025521.43352-1-shannon.nelson@amd.com> References: <20230516025521.43352-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT096:EE_|CY5PR12MB6155:EE_ X-MS-Office365-Filtering-Correlation-Id: 9feb6a4f-8841-4074-9f39-08db55b90d15 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: pE13Q+6qleRhmh6hRWWC3fZjwNyV4JFzd55fO5MOI1ceOf9i2UgsJqkel+icCcySqXbhhKHn2PEhBFYHFeFk/WdVm5quhWUXGMGRchp+ie88fonzypOcGJxZoXbpRRvqhfWkgDnGacbG9XdzA1u+26Pj77Ndbmx6GawAG5JxKEsGnLMe2b8zcRpn0frSDNwtFW46JZUiZ/yi8XDb1OHcg5maYQg7KXZxvPUeE3VrS7Xp6VBZYHD9uWcHKxRb7PIp7oUi8TsqCyAq5G6G87VecHdDoJjAcEZPJmCcUunLpAX+5AmO/+zdlAjPfV3ghWGDb+89Dxm95IKEv80jLGpAbBCkH+wNuJ1slXdl66RWkgk5QDPc/iHOFE8Oew3levPfAapRBEAnVoOkIGhrdvPzk+WiSs9VD79nmdddyg7LsBWaLSzw+XbN3nC1ME5XGPIArBPJqK/x6EjzLiGIFTS2I+91+uLLdEjqL0Hg/MkZ+qXFFZmTiZ6VyRxrsUZZ+4Fxk20QY6LRl2qcsGj9jzHm90Q54Ht3mke2A4pMemKE6+YU+74MEVDwnXi1GBcpx6XOw776nImNP2HkzcRGghznZ+iqsnKwHRY2joQpwz7soXKEQMX7XkEV8OEWIaKfVRinaQIMbIP43kGPULm7Vy0fxYWoQgTJZMMfZXOHAdkitJ66l/wXc5VSLSW6yQgNRYESn603Zi5kZB7wGQHm/W7iquTmwDhBtIUHnl0EIgCGocgK1NsXgg6q88ISdGNkSP5083l7NvwIdVomR/nFDq0uqA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(376002)(39860400002)(136003)(346002)(396003)(451199021)(36840700001)(46966006)(40470700004)(26005)(1076003)(40460700003)(47076005)(36756003)(426003)(336012)(36860700001)(40480700001)(2616005)(81166007)(86362001)(83380400001)(82310400005)(356005)(82740400003)(186003)(16526019)(54906003)(110136005)(44832011)(478600001)(2906002)(316002)(41300700001)(5660300002)(8676002)(4326008)(8936002)(70586007)(70206006)(6666004)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2023 02:55:47.9107 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9feb6a4f-8841-4074-9f39-08db55b90d15 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT096.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6155 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net The pds_core_logical_qtype enum and IFNAMSIZ are not needed in the common PDS header, only needed when working with the adminq, so move them to the adminq header. Note: This patch might conflict with pds_vfio patches that are in review, depending on which patchset gets pulled first. Signed-off-by: Shannon Nelson Acked-by: Jason Wang --- include/linux/pds/pds_adminq.h | 21 +++++++++++++++++++++ include/linux/pds/pds_common.h | 21 --------------------- 2 files changed, 21 insertions(+), 21 deletions(-) diff --git a/include/linux/pds/pds_adminq.h b/include/linux/pds/pds_adminq.h index 98a60ce87b92..61b0a8634e1a 100644 --- a/include/linux/pds/pds_adminq.h +++ b/include/linux/pds/pds_adminq.h @@ -222,6 +222,27 @@ enum pds_core_lif_type { PDS_CORE_LIF_TYPE_DEFAULT = 0, }; +#define PDS_CORE_IFNAMSIZ 16 + +/** + * enum pds_core_logical_qtype - Logical Queue Types + * @PDS_CORE_QTYPE_ADMINQ: Administrative Queue + * @PDS_CORE_QTYPE_NOTIFYQ: Notify Queue + * @PDS_CORE_QTYPE_RXQ: Receive Queue + * @PDS_CORE_QTYPE_TXQ: Transmit Queue + * @PDS_CORE_QTYPE_EQ: Event Queue + * @PDS_CORE_QTYPE_MAX: Max queue type supported + */ +enum pds_core_logical_qtype { + PDS_CORE_QTYPE_ADMINQ = 0, + PDS_CORE_QTYPE_NOTIFYQ = 1, + PDS_CORE_QTYPE_RXQ = 2, + PDS_CORE_QTYPE_TXQ = 3, + PDS_CORE_QTYPE_EQ = 4, + + PDS_CORE_QTYPE_MAX = 16 /* don't change - used in struct size */ +}; + /** * union pds_core_lif_config - LIF configuration * @state: LIF state (enum pds_core_lif_state) diff --git a/include/linux/pds/pds_common.h b/include/linux/pds/pds_common.h index 2a0d1669cfd0..435c8e8161c2 100644 --- a/include/linux/pds/pds_common.h +++ b/include/linux/pds/pds_common.h @@ -41,27 +41,6 @@ enum pds_core_vif_types { #define PDS_VDPA_DEV_NAME PDS_CORE_DRV_NAME "." PDS_DEV_TYPE_VDPA_STR -#define PDS_CORE_IFNAMSIZ 16 - -/** - * enum pds_core_logical_qtype - Logical Queue Types - * @PDS_CORE_QTYPE_ADMINQ: Administrative Queue - * @PDS_CORE_QTYPE_NOTIFYQ: Notify Queue - * @PDS_CORE_QTYPE_RXQ: Receive Queue - * @PDS_CORE_QTYPE_TXQ: Transmit Queue - * @PDS_CORE_QTYPE_EQ: Event Queue - * @PDS_CORE_QTYPE_MAX: Max queue type supported - */ -enum pds_core_logical_qtype { - PDS_CORE_QTYPE_ADMINQ = 0, - PDS_CORE_QTYPE_NOTIFYQ = 1, - PDS_CORE_QTYPE_RXQ = 2, - PDS_CORE_QTYPE_TXQ = 3, - PDS_CORE_QTYPE_EQ = 4, - - PDS_CORE_QTYPE_MAX = 16 /* don't change - used in struct size */ -}; - int pdsc_register_notify(struct notifier_block *nb); void pdsc_unregister_notify(struct notifier_block *nb); void *pdsc_get_pf_struct(struct pci_dev *vf_pdev); From patchwork Tue May 16 02:55:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13242501 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 359685230 for ; Tue, 16 May 2023 02:55:53 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2073.outbound.protection.outlook.com [40.107.220.73]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8E920E58 for ; Mon, 15 May 2023 19:55:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XO+fukNMUjos8ZF+cxZZ5fXe9pg9pzJAB9uzoaez9zbwh+dW+7cJleBt5NfjAulZqN7eJWjxwIiy3hzVTyMwgLbuWU0KTpdeRFUYBEt7BTeKW7O8XFBboxht8VmFHUTYpSBfzOhm2vW+OaP8cre0QiTqA8NUYARcmibW7mMQTpHucBY9x4stiEYF4afQoMZvP+oE6FOuD3Rp6dxY55mKsseCW+JkaSRJdKjX8sgK1G5RZoWZfGSWoPVqPSUCShVTnAnfWbhxfx0TLpQV5uhff31pSSJdUaa9SAr3TJ7NTY4BzBMlz5WM1dz9Vsvqz6y5PO/MOKqDeuBWkiWE+f0RjA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Ar/EvQMDsaGpEXWXWaFwk/dUjNqlFEk6w69bf6/a6uY=; b=JRZW1GFGQMD3UNjFTi76M4b4/yLxMLXRrNIH9ullr9w5HHABfYbJk7/OSh2v4SRaWksjibNG/OLVQLDdsImaCwjORgQ5P/nKPFjkIwceUA7CwLuXKoTc3Q3yX5a7I/GiJZ9NTtNCwLILXupSAqewu+6j935krSEfoy5LO2liWmJiFPuVuqmaQBL8bIEDeMEyaA+F1+ia3lynz0awLUvitz+spgkUfN9SxQG+k8SZ4teFolshWn5E+tTeqfEj+EG5/GMbVDlDR4xW22nWY+NebSpRlsDiz7D+A58ZPKg5f3xipzozDKSiH17aCyaaHlJLIgqTnK73WIfqgeh6Q4OTfA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ar/EvQMDsaGpEXWXWaFwk/dUjNqlFEk6w69bf6/a6uY=; b=zlTgMoGRU81umqPKdW0HoCJH65OeT3pjPQ30Lo6WR1mVlvzt3YPoagoIZ/KZTUJTiYsJLd/bvnu8wdsQhsUPQiCauCfNpTfUrUyGsn9SPbq0yi4yGFqFSSf/lmq/+UoFcZXKYoJ2R9KJNTrNCLj6QRmxXjOCS37XhYAuqtAC96s= Received: from SJ0PR03CA0028.namprd03.prod.outlook.com (2603:10b6:a03:33a::33) by LV2PR12MB5871.namprd12.prod.outlook.com (2603:10b6:408:174::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30; Tue, 16 May 2023 02:55:49 +0000 Received: from DM6NAM11FT096.eop-nam11.prod.protection.outlook.com (2603:10b6:a03:33a:cafe::f3) by SJ0PR03CA0028.outlook.office365.com (2603:10b6:a03:33a::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30 via Frontend Transport; Tue, 16 May 2023 02:55:48 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT096.mail.protection.outlook.com (10.13.173.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6411.15 via Frontend Transport; Tue, 16 May 2023 02:55:48 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 15 May 2023 21:55:46 -0500 From: Shannon Nelson To: , , , , , CC: , Subject: [PATCH v6 virtio 05/11] pds_vdpa: new adminq entries Date: Mon, 15 May 2023 19:55:15 -0700 Message-ID: <20230516025521.43352-6-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230516025521.43352-1-shannon.nelson@amd.com> References: <20230516025521.43352-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT096:EE_|LV2PR12MB5871:EE_ X-MS-Office365-Filtering-Correlation-Id: 05a3e6fd-1c5f-4a00-2281-08db55b90d7c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rTJxFxSPOiKLc3jLrqkXpnsqpTzqDKAH050BUpZVDVhEuAuaoMhoHldHhQd3NpZelWs5U/EqH3xqofbe8ywPq5Gkxj2bcqXrjsiGO8UYonWocyJkHPsZysplkXelHjsnK8FAmyZyYSciI9/2XuduQ0k2wHF//LoyScQCWklqhiEt79+mPl38KI6vBGAdi5Kgu6gjbzzqUJTc6dNN1BtFP265aC7yVD2Q422mh/xG3cyZsKnDC0FNVUfT9AAGY4tH10WNGIQ/enNjJ9lx1eeH7xbc2xpkseVUPaHgKETuCXEeD1JB54vnzDKiof5Gaht8zmiP6jnftL2bdiudJIjeY9Y2a4sQz7eHqr25qdOQy+i86Af7vDAieCdgn+8goQmAbVaycVYkRX2klV7QH3gl1UizdgzgYIla3BWIJUnJLC+4mp7a6xeZ3gU+SpwWV0GYDhWbFMoFgXOa0VADgKBWunH7v67NrhEBKaOdKDCptKrMzvGX9dlojq9vwe8sHyH3VyeBV/Rc4URTvTrGmvhYdfZGIKuqzGevdWO3uPh9MoFMFrEK0E0VRxoPqImYYZXuA/at3p45MeZGmuohv+57JvdEMjSCiYfOFb6Tza8UmKuzULe0uVrdhSx0w6gh2dShpHtwh1tRWQeR/My4wN5159/yG1RmdBQKlxTX2dMfdz0AFXdnH/H7RKEa4tgjsSsZaRJynveI6EoqKpTSxy32P4SHTKoOYWcWziDaGNqKVD/yipKDXzHHEmZeIv12B5fPa9YPh/nVs8wq+W6z4VeZYw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(39860400002)(136003)(396003)(376002)(451199021)(46966006)(36840700001)(40470700004)(16526019)(186003)(86362001)(36860700001)(2906002)(26005)(1076003)(6666004)(2616005)(47076005)(83380400001)(82310400005)(426003)(336012)(8676002)(8936002)(478600001)(36756003)(40460700003)(54906003)(110136005)(81166007)(5660300002)(82740400003)(316002)(356005)(4326008)(41300700001)(44832011)(70206006)(70586007)(40480700001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2023 02:55:48.5825 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 05a3e6fd-1c5f-4a00-2281-08db55b90d7c X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT096.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5871 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Add new adminq definitions in support for vDPA operations. Signed-off-by: Shannon Nelson Acked-by: Jason Wang --- include/linux/pds/pds_adminq.h | 226 +++++++++++++++++++++++++++++++++ 1 file changed, 226 insertions(+) diff --git a/include/linux/pds/pds_adminq.h b/include/linux/pds/pds_adminq.h index 61b0a8634e1a..bcba7fda3cc9 100644 --- a/include/linux/pds/pds_adminq.h +++ b/include/linux/pds/pds_adminq.h @@ -605,6 +605,219 @@ struct pds_core_q_init_comp { u8 color; }; +/* + * enum pds_vdpa_cmd_opcode - vDPA Device commands + */ +enum pds_vdpa_cmd_opcode { + PDS_VDPA_CMD_INIT = 48, + PDS_VDPA_CMD_IDENT = 49, + PDS_VDPA_CMD_RESET = 51, + PDS_VDPA_CMD_VQ_RESET = 52, + PDS_VDPA_CMD_VQ_INIT = 53, + PDS_VDPA_CMD_STATUS_UPDATE = 54, + PDS_VDPA_CMD_SET_FEATURES = 55, + PDS_VDPA_CMD_SET_ATTR = 56, +}; + +/** + * struct pds_vdpa_cmd - generic command + * @opcode: Opcode + * @vdpa_index: Index for vdpa subdevice + * @vf_id: VF id + */ +struct pds_vdpa_cmd { + u8 opcode; + u8 vdpa_index; + __le16 vf_id; +}; + +/** + * struct pds_vdpa_init_cmd - INIT command + * @opcode: Opcode PDS_VDPA_CMD_INIT + * @vdpa_index: Index for vdpa subdevice + * @vf_id: VF id + */ +struct pds_vdpa_init_cmd { + u8 opcode; + u8 vdpa_index; + __le16 vf_id; +}; + +/** + * struct pds_vdpa_ident - vDPA identification data + * @hw_features: vDPA features supported by device + * @max_vqs: max queues available (2 queues for a single queuepair) + * @max_qlen: log(2) of maximum number of descriptors + * @min_qlen: log(2) of minimum number of descriptors + * + * This struct is used in a DMA block that is set up for the PDS_VDPA_CMD_IDENT + * transaction. Set up the DMA block and send the address in the IDENT cmd + * data, the DSC will write the ident information, then we can remove the DMA + * block after reading the answer. If the completion status is 0, then there + * is valid information, else there was an error and the data should be invalid. + */ +struct pds_vdpa_ident { + __le64 hw_features; + __le16 max_vqs; + __le16 max_qlen; + __le16 min_qlen; +}; + +/** + * struct pds_vdpa_ident_cmd - IDENT command + * @opcode: Opcode PDS_VDPA_CMD_IDENT + * @rsvd: Word boundary padding + * @vf_id: VF id + * @len: length of ident info DMA space + * @ident_pa: address for DMA of ident info (struct pds_vdpa_ident) + * only used for this transaction, then forgotten by DSC + */ +struct pds_vdpa_ident_cmd { + u8 opcode; + u8 rsvd; + __le16 vf_id; + __le32 len; + __le64 ident_pa; +}; + +/** + * struct pds_vdpa_status_cmd - STATUS_UPDATE command + * @opcode: Opcode PDS_VDPA_CMD_STATUS_UPDATE + * @vdpa_index: Index for vdpa subdevice + * @vf_id: VF id + * @status: new status bits + */ +struct pds_vdpa_status_cmd { + u8 opcode; + u8 vdpa_index; + __le16 vf_id; + u8 status; +}; + +/** + * enum pds_vdpa_attr - List of VDPA device attributes + * @PDS_VDPA_ATTR_MAC: MAC address + * @PDS_VDPA_ATTR_MAX_VQ_PAIRS: Max virtqueue pairs + */ +enum pds_vdpa_attr { + PDS_VDPA_ATTR_MAC = 1, + PDS_VDPA_ATTR_MAX_VQ_PAIRS = 2, +}; + +/** + * struct pds_vdpa_setattr_cmd - SET_ATTR command + * @opcode: Opcode PDS_VDPA_CMD_SET_ATTR + * @vdpa_index: Index for vdpa subdevice + * @vf_id: VF id + * @attr: attribute to be changed (enum pds_vdpa_attr) + * @pad: Word boundary padding + * @mac: new mac address to be assigned as vdpa device address + * @max_vq_pairs: new limit of virtqueue pairs + */ +struct pds_vdpa_setattr_cmd { + u8 opcode; + u8 vdpa_index; + __le16 vf_id; + u8 attr; + u8 pad[3]; + union { + u8 mac[6]; + __le16 max_vq_pairs; + } __packed; +}; + +/** + * struct pds_vdpa_vq_init_cmd - queue init command + * @opcode: Opcode PDS_VDPA_CMD_VQ_INIT + * @vdpa_index: Index for vdpa subdevice + * @vf_id: VF id + * @qid: Queue id (bit0 clear = rx, bit0 set = tx, qid=N is ctrlq) + * @len: log(2) of max descriptor count + * @desc_addr: DMA address of descriptor area + * @avail_addr: DMA address of available descriptors (aka driver area) + * @used_addr: DMA address of used descriptors (aka device area) + * @intr_index: interrupt index + * @avail_index: initial device position in available ring + * @used_index: initial device position in used ring + */ +struct pds_vdpa_vq_init_cmd { + u8 opcode; + u8 vdpa_index; + __le16 vf_id; + __le16 qid; + __le16 len; + __le64 desc_addr; + __le64 avail_addr; + __le64 used_addr; + __le16 intr_index; + __le16 avail_index; + __le16 used_index; +}; + +/** + * struct pds_vdpa_vq_init_comp - queue init completion + * @status: Status of the command (enum pds_core_status_code) + * @hw_qtype: HW queue type, used in doorbell selection + * @hw_qindex: HW queue index, used in doorbell selection + * @rsvd: Word boundary padding + * @color: Color bit + */ +struct pds_vdpa_vq_init_comp { + u8 status; + u8 hw_qtype; + __le16 hw_qindex; + u8 rsvd[11]; + u8 color; +}; + +/** + * struct pds_vdpa_vq_reset_cmd - queue reset command + * @opcode: Opcode PDS_VDPA_CMD_VQ_RESET + * @vdpa_index: Index for vdpa subdevice + * @vf_id: VF id + * @qid: Queue id + */ +struct pds_vdpa_vq_reset_cmd { + u8 opcode; + u8 vdpa_index; + __le16 vf_id; + __le16 qid; +}; + +/** + * struct pds_vdpa_vq_reset_comp - queue reset completion + * @status: Status of the command (enum pds_core_status_code) + * @rsvd0: Word boundary padding + * @avail_index: current device position in available ring + * @used_index: current device position in used ring + * @rsvd: Word boundary padding + * @color: Color bit + */ +struct pds_vdpa_vq_reset_comp { + u8 status; + u8 rsvd0; + __le16 avail_index; + __le16 used_index; + u8 rsvd[9]; + u8 color; +}; + +/** + * struct pds_vdpa_set_features_cmd - set hw features + * @opcode: Opcode PDS_VDPA_CMD_SET_FEATURES + * @vdpa_index: Index for vdpa subdevice + * @vf_id: VF id + * @rsvd: Word boundary padding + * @features: Feature bit mask + */ +struct pds_vdpa_set_features_cmd { + u8 opcode; + u8 vdpa_index; + __le16 vf_id; + __le32 rsvd; + __le64 features; +}; + union pds_core_adminq_cmd { u8 opcode; u8 bytes[64]; @@ -621,6 +834,16 @@ union pds_core_adminq_cmd { struct pds_core_q_identify_cmd q_ident; struct pds_core_q_init_cmd q_init; + + struct pds_vdpa_cmd vdpa; + struct pds_vdpa_init_cmd vdpa_init; + struct pds_vdpa_ident_cmd vdpa_ident; + struct pds_vdpa_status_cmd vdpa_status; + struct pds_vdpa_setattr_cmd vdpa_setattr; + struct pds_vdpa_set_features_cmd vdpa_set_features; + struct pds_vdpa_vq_init_cmd vdpa_vq_init; + struct pds_vdpa_vq_reset_cmd vdpa_vq_reset; + }; union pds_core_adminq_comp { @@ -642,6 +865,9 @@ union pds_core_adminq_comp { struct pds_core_q_identify_comp q_ident; struct pds_core_q_init_comp q_init; + + struct pds_vdpa_vq_init_comp vdpa_vq_init; + struct pds_vdpa_vq_reset_comp vdpa_vq_reset; }; #ifndef __CHECKER__ From patchwork Tue May 16 02:55:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13242504 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 609ED7494 for ; Tue, 16 May 2023 02:55:55 +0000 (UTC) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2075.outbound.protection.outlook.com [40.107.244.75]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AE8DCE58 for ; Mon, 15 May 2023 19:55:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZgZFVyxzPmdUdbLyzRQdC9XI7SsEjXsgivO7AQXeGmEu0vyKr015+LsuLWmET9m6meJlYUPGn+lSi7ipw6aI9eF9Jkrsl0Ng2o9fshGTpxAwPiwbKfeRIr/LhZk/7GfUEw1nIF4xZqUQEnOeKOIP2aO2+U98Qr5CW2WH5qfs/Etck8BL8ikvcZgSJTXVto+EGGV0UQ1CvwjZJ7VakhLpXvZs+LVPg2w5Xp5t6wdFhE8DNNi5ZVj0HbTYwuUj4tHiuaIQWXiGMwZdJVTmuuZ5YHY6kmk8s7HnSNocEU8aYJ/Wwqkl8mL0OWckmZZvlpefoaikamWsAKcpzQJ11Db/7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=J7pl08gsZAq6M5LfC/8SmQl1rwIMMSjRMfx5i814uyA=; b=Y4YK5jnKWLePBHKccUAUwRRZpLhmtxuZg49Ulx0VtPRyOccQCx7+8G5dSNxmbv8p8YEGPONsz7BCZhi24aQN1ME1EqIu/eXbDwG57nj6U1rPFpAcJF5eS47gwowtrpVT0wv9k0Y3GC1XamZbPA7obnIlK8/YwNvtAo6OyilWV+tD7fjFmSeY2Hn9hDlEqukRdwOsSw7zovzDGJhmudXw9c4sWsfca/uEiaxZOsI+hlY/Fb3i0PPXTUWeiMFj/XS6LWLyv55O2WsgeyWYHKs4aBplr/b91kOKgk8ZGyqMO+FVqrrJw051bmK20jEkMhhD9GMjppg1DOxMjf3hfwkbsw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=J7pl08gsZAq6M5LfC/8SmQl1rwIMMSjRMfx5i814uyA=; b=kfObty4zp89IRJCy693bFLwed7CTDIEXHHZbdlOA5NQx9eB8JESNtYgvE/VxBgySPINVEKxHrbhRhdyyxOTiRUFKLlDDChRotHFFshCwbepaHxhFae9nmU6jxgyADuVhg2XvppruP/K2PbbYNiIhhCvqJBDxHSfpAR6W/XSTt3A= Received: from SJ0PR03CA0008.namprd03.prod.outlook.com (2603:10b6:a03:33a::13) by MW4PR12MB7144.namprd12.prod.outlook.com (2603:10b6:303:21b::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30; Tue, 16 May 2023 02:55:49 +0000 Received: from DM6NAM11FT096.eop-nam11.prod.protection.outlook.com (2603:10b6:a03:33a:cafe::4e) by SJ0PR03CA0008.outlook.office365.com (2603:10b6:a03:33a::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.33 via Frontend Transport; Tue, 16 May 2023 02:55:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT096.mail.protection.outlook.com (10.13.173.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6411.15 via Frontend Transport; Tue, 16 May 2023 02:55:49 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 15 May 2023 21:55:47 -0500 From: Shannon Nelson To: , , , , , CC: , Subject: [PATCH v6 virtio 06/11] pds_vdpa: get vdpa management info Date: Mon, 15 May 2023 19:55:16 -0700 Message-ID: <20230516025521.43352-7-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230516025521.43352-1-shannon.nelson@amd.com> References: <20230516025521.43352-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT096:EE_|MW4PR12MB7144:EE_ X-MS-Office365-Filtering-Correlation-Id: ae8b918b-c13a-4d77-b856-08db55b90df5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: IoCzjQEH4gov4tP1DkTidjJwmM1UZsGtYliOZmB7zk+SuogSYtFK7kVHtQGILoEii3AHM/YrHU8iupUzbj4F80a1LxKLklaZ9vfHLbJ6WNemxwyyY+AH73LJ6kqJVIegwZ4BBImQvd45uvQynScqaCRPBMIKyMetb4gL7KrwA7t4Zb2IGMTO44Ch645fotSruHSpjOIH3xBDrNFVJkylb7AtPejVTrFhhhjSb7GJXQL4Ea6LwxQBei5XSiiSXRWt8EcIkVW9WJ9GFclscckM+5cE0D5nPKcnhQ7S0WQPOqu6yFHuoud/sO6hH0LRSFjCPriQPWbgix4vnC6+u+RJoaM1OM2cQtNmPoZK+0vgdzO2YOFAqHTOLLU2DBJmhBNfZYIK11GbxOt4FrIFD2REC751ltaogtA/u4C7n7c9LS04e4oPT0Q/R0kZ/9ZJ1MsjSAUWUGkDNgIrCBzkQT6JOjzy4OB/zNOCyZQyc+HhRarpCz4yeiyX8i7rn8fLXLfslfWNb1pLAtrJGCDLg7vDOs3msrtT/RIHjpp44q1EGmjgvcPdCLnC8Ad96ug4y2XbzlvXbKEWRSss+Yc46uj5NKGfZ6PZ9P7q23/qVlzfwhEDLHUeDwqaYn+7H7vsQCE7GcBoozf5X/SezGQVFxy1gB6CpCtZ5PmzUwXaOyD4Iq+EpVfuGOcIkZBRABEq5vrOZGvhClIAQpNItQODMwuB2T2x4Y3FomlgyenlvtW4e9xM7DV/cu2X6IWJCwa1mPk93gIai9BDgq3kfKhhXqH2bw== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(39860400002)(136003)(376002)(396003)(346002)(451199021)(36840700001)(40470700004)(46966006)(478600001)(110136005)(70586007)(70206006)(6666004)(36756003)(54906003)(316002)(4326008)(86362001)(40460700003)(82310400005)(36860700001)(426003)(1076003)(82740400003)(186003)(47076005)(336012)(81166007)(356005)(26005)(2616005)(5660300002)(41300700001)(2906002)(44832011)(8936002)(8676002)(16526019)(83380400001)(40480700001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2023 02:55:49.3793 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ae8b918b-c13a-4d77-b856-08db55b90df5 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT096.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7144 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Find the vDPA management information from the DSC in order to advertise it to the vdpa subsystem. Signed-off-by: Shannon Nelson Acked-by: Jason Wang --- drivers/vdpa/pds/Makefile | 3 +- drivers/vdpa/pds/aux_drv.c | 17 ++++++ drivers/vdpa/pds/aux_drv.h | 7 +++ drivers/vdpa/pds/debugfs.c | 1 + drivers/vdpa/pds/vdpa_dev.c | 108 ++++++++++++++++++++++++++++++++++++ drivers/vdpa/pds/vdpa_dev.h | 15 +++++ 6 files changed, 150 insertions(+), 1 deletion(-) create mode 100644 drivers/vdpa/pds/vdpa_dev.c create mode 100644 drivers/vdpa/pds/vdpa_dev.h diff --git a/drivers/vdpa/pds/Makefile b/drivers/vdpa/pds/Makefile index a9cd2f450ae1..13b50394ec64 100644 --- a/drivers/vdpa/pds/Makefile +++ b/drivers/vdpa/pds/Makefile @@ -3,6 +3,7 @@ obj-$(CONFIG_PDS_VDPA) := pds_vdpa.o -pds_vdpa-y := aux_drv.o +pds_vdpa-y := aux_drv.o \ + vdpa_dev.o pds_vdpa-$(CONFIG_DEBUG_FS) += debugfs.o diff --git a/drivers/vdpa/pds/aux_drv.c b/drivers/vdpa/pds/aux_drv.c index e4a0ad61ea22..aa748cf55d2b 100644 --- a/drivers/vdpa/pds/aux_drv.c +++ b/drivers/vdpa/pds/aux_drv.c @@ -3,6 +3,7 @@ #include #include +#include #include #include @@ -11,6 +12,7 @@ #include "aux_drv.h" #include "debugfs.h" +#include "vdpa_dev.h" static const struct auxiliary_device_id pds_vdpa_id_table[] = { { .name = PDS_VDPA_DEV_NAME, }, @@ -24,15 +26,28 @@ static int pds_vdpa_probe(struct auxiliary_device *aux_dev, struct pds_auxiliary_dev *padev = container_of(aux_dev, struct pds_auxiliary_dev, aux_dev); struct pds_vdpa_aux *vdpa_aux; + int err; vdpa_aux = kzalloc(sizeof(*vdpa_aux), GFP_KERNEL); if (!vdpa_aux) return -ENOMEM; vdpa_aux->padev = padev; + vdpa_aux->vf_id = pci_iov_vf_id(padev->vf_pdev); auxiliary_set_drvdata(aux_dev, vdpa_aux); + /* Get device ident info and set up the vdpa_mgmt_dev */ + err = pds_vdpa_get_mgmt_info(vdpa_aux); + if (err) + goto err_free_mem; + return 0; + +err_free_mem: + kfree(vdpa_aux); + auxiliary_set_drvdata(aux_dev, NULL); + + return err; } static void pds_vdpa_remove(struct auxiliary_device *aux_dev) @@ -40,6 +55,8 @@ static void pds_vdpa_remove(struct auxiliary_device *aux_dev) struct pds_vdpa_aux *vdpa_aux = auxiliary_get_drvdata(aux_dev); struct device *dev = &aux_dev->dev; + pci_free_irq_vectors(vdpa_aux->padev->vf_pdev); + kfree(vdpa_aux); auxiliary_set_drvdata(aux_dev, NULL); diff --git a/drivers/vdpa/pds/aux_drv.h b/drivers/vdpa/pds/aux_drv.h index f1e99359424e..dcec782e79eb 100644 --- a/drivers/vdpa/pds/aux_drv.h +++ b/drivers/vdpa/pds/aux_drv.h @@ -10,6 +10,13 @@ struct pds_vdpa_aux { struct pds_auxiliary_dev *padev; + struct vdpa_mgmt_dev vdpa_mdev; + + struct pds_vdpa_ident ident; + + int vf_id; struct dentry *dentry; + + int nintrs; }; #endif /* _AUX_DRV_H_ */ diff --git a/drivers/vdpa/pds/debugfs.c b/drivers/vdpa/pds/debugfs.c index 5be22fb7a76a..d91dceb07380 100644 --- a/drivers/vdpa/pds/debugfs.c +++ b/drivers/vdpa/pds/debugfs.c @@ -2,6 +2,7 @@ /* Copyright(c) 2023 Advanced Micro Devices, Inc */ #include +#include #include #include diff --git a/drivers/vdpa/pds/vdpa_dev.c b/drivers/vdpa/pds/vdpa_dev.c new file mode 100644 index 000000000000..0f0f0ab8b811 --- /dev/null +++ b/drivers/vdpa/pds/vdpa_dev.c @@ -0,0 +1,108 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2023 Advanced Micro Devices, Inc */ + +#include +#include +#include + +#include +#include +#include +#include + +#include "vdpa_dev.h" +#include "aux_drv.h" + +static struct virtio_device_id pds_vdpa_id_table[] = { + {VIRTIO_ID_NET, VIRTIO_DEV_ANY_ID}, + {0}, +}; + +static int pds_vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name, + const struct vdpa_dev_set_config *add_config) +{ + return -EOPNOTSUPP; +} + +static void pds_vdpa_dev_del(struct vdpa_mgmt_dev *mdev, + struct vdpa_device *vdpa_dev) +{ +} + +static const struct vdpa_mgmtdev_ops pds_vdpa_mgmt_dev_ops = { + .dev_add = pds_vdpa_dev_add, + .dev_del = pds_vdpa_dev_del +}; + +int pds_vdpa_get_mgmt_info(struct pds_vdpa_aux *vdpa_aux) +{ + union pds_core_adminq_cmd cmd = { + .vdpa_ident.opcode = PDS_VDPA_CMD_IDENT, + .vdpa_ident.vf_id = cpu_to_le16(vdpa_aux->vf_id), + }; + union pds_core_adminq_comp comp = {}; + struct vdpa_mgmt_dev *mgmt; + struct pci_dev *pf_pdev; + struct device *pf_dev; + struct pci_dev *pdev; + dma_addr_t ident_pa; + struct device *dev; + u16 dev_intrs; + u16 max_vqs; + int err; + + dev = &vdpa_aux->padev->aux_dev.dev; + pdev = vdpa_aux->padev->vf_pdev; + mgmt = &vdpa_aux->vdpa_mdev; + + /* Get resource info through the PF's adminq. It is a block of info, + * so we need to map some memory for PF to make available to the + * firmware for writing the data. + */ + pf_pdev = pci_physfn(vdpa_aux->padev->vf_pdev); + pf_dev = &pf_pdev->dev; + ident_pa = dma_map_single(pf_dev, &vdpa_aux->ident, + sizeof(vdpa_aux->ident), DMA_FROM_DEVICE); + if (dma_mapping_error(pf_dev, ident_pa)) { + dev_err(dev, "Failed to map ident space\n"); + return -ENOMEM; + } + + cmd.vdpa_ident.ident_pa = cpu_to_le64(ident_pa); + cmd.vdpa_ident.len = cpu_to_le32(sizeof(vdpa_aux->ident)); + err = pds_client_adminq_cmd(vdpa_aux->padev, &cmd, + sizeof(cmd.vdpa_ident), &comp, 0); + dma_unmap_single(pf_dev, ident_pa, + sizeof(vdpa_aux->ident), DMA_FROM_DEVICE); + if (err) { + dev_err(dev, "Failed to ident hw, status %d: %pe\n", + comp.status, ERR_PTR(err)); + return err; + } + + max_vqs = le16_to_cpu(vdpa_aux->ident.max_vqs); + dev_intrs = pci_msix_vec_count(pdev); + dev_dbg(dev, "ident.max_vqs %d dev_intrs %d\n", max_vqs, dev_intrs); + + max_vqs = min_t(u16, dev_intrs, max_vqs); + mgmt->max_supported_vqs = min_t(u16, PDS_VDPA_MAX_QUEUES, max_vqs); + vdpa_aux->nintrs = mgmt->max_supported_vqs; + + mgmt->ops = &pds_vdpa_mgmt_dev_ops; + mgmt->id_table = pds_vdpa_id_table; + mgmt->device = dev; + mgmt->supported_features = le64_to_cpu(vdpa_aux->ident.hw_features); + mgmt->config_attr_mask = BIT_ULL(VDPA_ATTR_DEV_NET_CFG_MACADDR); + mgmt->config_attr_mask |= BIT_ULL(VDPA_ATTR_DEV_NET_CFG_MAX_VQP); + + err = pci_alloc_irq_vectors(pdev, vdpa_aux->nintrs, vdpa_aux->nintrs, + PCI_IRQ_MSIX); + if (err < 0) { + dev_err(dev, "Couldn't get %d msix vectors: %pe\n", + vdpa_aux->nintrs, ERR_PTR(err)); + return err; + } + vdpa_aux->nintrs = err; + + return 0; +} diff --git a/drivers/vdpa/pds/vdpa_dev.h b/drivers/vdpa/pds/vdpa_dev.h new file mode 100644 index 000000000000..97fab833a0aa --- /dev/null +++ b/drivers/vdpa/pds/vdpa_dev.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc */ + +#ifndef _VDPA_DEV_H_ +#define _VDPA_DEV_H_ + +#define PDS_VDPA_MAX_QUEUES 65 + +struct pds_vdpa_device { + struct vdpa_device vdpa_dev; + struct pds_vdpa_aux *vdpa_aux; +}; + +int pds_vdpa_get_mgmt_info(struct pds_vdpa_aux *vdpa_aux); +#endif /* _VDPA_DEV_H_ */ From patchwork Tue May 16 02:55:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13242503 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC007524C for ; Tue, 16 May 2023 02:55:53 +0000 (UTC) Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2045.outbound.protection.outlook.com [40.107.243.45]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B8408449E for ; Mon, 15 May 2023 19:55:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eyC0DbGHstHNuprAefXt07RnvLtXI3P+0Mhdp18cjltZx4W3HI9jcaCAzSJEYscscc5hpuok7Q4ivOcOQ+h16a39apN6ZgFgbMNaFt/dPMfB+TCboDuUCZZYuxdMqdGVj3kA/VVDSTPrYerX3QH1hA+CTdyUbY/EhxM09euf/T4DWzXH5H45K2y7zEbMgzL6O2DwXISHuStyx2IZLLivwdO9yhEQRqkllMJFJsXDUQ/lQiWAFrPz3T2RigZZcXuTGA7rQDYRdP09IxqUrcWwmLWyELWZAkxQvInEeGTX9Xwax3Wrm4ueEsCYtDME71NsgrB15sbpS7MSYvX9HRT6yQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zvdWJJ254PoiNlvH3a/tCFZju7eimX3KwS1R1DbTR2I=; b=jvFVQml0dJF5CWz7WWMygDMAAzlFwPKP/dC0TJkj8PmEaZ8YZ7GLwGBpNKcN0dHt1CpysQNX2j8N8qyOrHof4MwhZ4a+AdJt8+hvm7Iq6ll799bKTOB92Bx5Ns3yQknWrIx5EbIBiiHh221XOd6jdFxQlpnmxzxAQqugidTwz1mLuNApWf/iLYR6LnVyOgxHBNGjQjsBj0FD7mzguwin8n7rwLLZF/t4qgTIPAsY7J/Qk1NBJ/yZ7CEK5iqi4SMV4zP2MXeT8x7VxKCtadPQOlNXt7g9vPf/25SJvcBQKsIYNE0K1bmlFj+aA/YFAmJ0WL5wK9mbrfsUVYfIz/qXAw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zvdWJJ254PoiNlvH3a/tCFZju7eimX3KwS1R1DbTR2I=; b=SFEKOHDU7jIuzi6v+dgqjUfJeQ1cotxjlwXX+EylcOaE5ZW2064gd2avZV3ZZsjaowazUPjTNSNGZ0cMIUkdKvk1qHv+HH4n260c8xz3AbFzHG+cEs2633gjbdQDdMcxCcuAh41bRYegyhI7WcHjRVap+NboSpZzmuX6j6pRQ1I= Received: from SJ0PR03CA0014.namprd03.prod.outlook.com (2603:10b6:a03:33a::19) by PH0PR12MB8051.namprd12.prod.outlook.com (2603:10b6:510:26d::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30; Tue, 16 May 2023 02:55:50 +0000 Received: from DM6NAM11FT096.eop-nam11.prod.protection.outlook.com (2603:10b6:a03:33a:cafe::e8) by SJ0PR03CA0014.outlook.office365.com (2603:10b6:a03:33a::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30 via Frontend Transport; Tue, 16 May 2023 02:55:50 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT096.mail.protection.outlook.com (10.13.173.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6411.15 via Frontend Transport; Tue, 16 May 2023 02:55:50 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 15 May 2023 21:55:48 -0500 From: Shannon Nelson To: , , , , , CC: , Subject: [PATCH v6 virtio 07/11] pds_vdpa: virtio bar setup for vdpa Date: Mon, 15 May 2023 19:55:17 -0700 Message-ID: <20230516025521.43352-8-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230516025521.43352-1-shannon.nelson@amd.com> References: <20230516025521.43352-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT096:EE_|PH0PR12MB8051:EE_ X-MS-Office365-Filtering-Correlation-Id: fead1460-6ab7-4f71-12b4-08db55b90e74 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: aNf4Lt0T93m0vD+ioSlXknuf2BkFXEk6eH5Uag3HtvC0vG75uE86TzRvnnS33mnaipkuKMsVVTHCKeIrQObiF6YtOnGb9Nz9RHR3B1U5ihkambLjErv/ZAOHmgL4QQa23936YIZQD9dnWWpcpf43esCXrA1OUYqmFhwiRN6Zg+ValPfHZol1Fkk1DvwoS3liNrVkHIN2kIknRJAGznVROe7zehFvYPpN030blhCoolD7O8Tdh5EfRvcLKmPuPptvcUIk7+87PODekP9aiGBKcrhE9ZnLrgYG1XjrMGLZjZnRv3cDMC5ylO5mfX8pvf/K+n4mI00I3k0mJJItftXWwWBOmPCE4bVQQ908n+pB16I4u5pbG1J6zbnW8BeiahV22G5YAAUZFCKJwhpcD66X8reeVX0UVtW/pNG9p6dd066Hzw9sS9Tz4Bw44aiY7yOX7twguoxEdTvL6FA2ylqiFqBAVh/SNb5NzgBQ/lEKHrFVkfXz1/1B3BsTF4T1ec9He5Jmkg3Sz60h7lAAevf/FiL/f7IkuzRCdxwkde6UqgPJ7wS8z4X690D27vrrbpDApMurMzBCtFN30cvY/zAuOfUsPc1Tg5n4RI8AR/LV4dZOI2YYybyQ24WbSlHV/3DfTpvEmKzEABI56PQLUou4Mt59qUvsSO0z6rk4a4+qerdHeS9J67p6J1SfJF6MxBbbX8tTsiwiHx/L161G33AblANqzC+ESrvP4s/QXNU41foQoxLGnkZdvRmEpGvV4C0uLUTxfNY51tGii0k7b4hDqA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(376002)(136003)(396003)(39860400002)(346002)(451199021)(40470700004)(36840700001)(46966006)(36756003)(110136005)(54906003)(40460700003)(8936002)(8676002)(478600001)(356005)(44832011)(82740400003)(70206006)(40480700001)(70586007)(4326008)(316002)(41300700001)(81166007)(5660300002)(2906002)(26005)(1076003)(86362001)(16526019)(186003)(36860700001)(83380400001)(47076005)(2616005)(336012)(426003)(82310400005)(6666004)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2023 02:55:50.1918 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: fead1460-6ab7-4f71-12b4-08db55b90e74 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT096.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB8051 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Prep and use the "modern" virtio bar utilities to get our virtio config space ready. Signed-off-by: Shannon Nelson Acked-by: Jason Wang --- drivers/vdpa/pds/aux_drv.c | 25 +++++++++++++++++++++++++ drivers/vdpa/pds/aux_drv.h | 3 +++ 2 files changed, 28 insertions(+) diff --git a/drivers/vdpa/pds/aux_drv.c b/drivers/vdpa/pds/aux_drv.c index aa748cf55d2b..0c4a135b1484 100644 --- a/drivers/vdpa/pds/aux_drv.c +++ b/drivers/vdpa/pds/aux_drv.c @@ -4,6 +4,7 @@ #include #include #include +#include #include #include @@ -19,12 +20,22 @@ static const struct auxiliary_device_id pds_vdpa_id_table[] = { {}, }; +static int pds_vdpa_device_id_check(struct pci_dev *pdev) +{ + if (pdev->device != PCI_DEVICE_ID_PENSANDO_VDPA_VF || + pdev->vendor != PCI_VENDOR_ID_PENSANDO) + return -ENODEV; + + return PCI_DEVICE_ID_PENSANDO_VDPA_VF; +} + static int pds_vdpa_probe(struct auxiliary_device *aux_dev, const struct auxiliary_device_id *id) { struct pds_auxiliary_dev *padev = container_of(aux_dev, struct pds_auxiliary_dev, aux_dev); + struct device *dev = &aux_dev->dev; struct pds_vdpa_aux *vdpa_aux; int err; @@ -41,8 +52,21 @@ static int pds_vdpa_probe(struct auxiliary_device *aux_dev, if (err) goto err_free_mem; + /* Find the virtio configuration */ + vdpa_aux->vd_mdev.pci_dev = padev->vf_pdev; + vdpa_aux->vd_mdev.device_id_check = pds_vdpa_device_id_check; + vdpa_aux->vd_mdev.dma_mask = DMA_BIT_MASK(PDS_CORE_ADDR_LEN); + err = vp_modern_probe(&vdpa_aux->vd_mdev); + if (err) { + dev_err(dev, "Unable to probe for virtio configuration: %pe\n", + ERR_PTR(err)); + goto err_free_mgmt_info; + } + return 0; +err_free_mgmt_info: + pci_free_irq_vectors(padev->vf_pdev); err_free_mem: kfree(vdpa_aux); auxiliary_set_drvdata(aux_dev, NULL); @@ -55,6 +79,7 @@ static void pds_vdpa_remove(struct auxiliary_device *aux_dev) struct pds_vdpa_aux *vdpa_aux = auxiliary_get_drvdata(aux_dev); struct device *dev = &aux_dev->dev; + vp_modern_remove(&vdpa_aux->vd_mdev); pci_free_irq_vectors(vdpa_aux->padev->vf_pdev); kfree(vdpa_aux); diff --git a/drivers/vdpa/pds/aux_drv.h b/drivers/vdpa/pds/aux_drv.h index dcec782e79eb..99e0ff340bfa 100644 --- a/drivers/vdpa/pds/aux_drv.h +++ b/drivers/vdpa/pds/aux_drv.h @@ -4,6 +4,8 @@ #ifndef _AUX_DRV_H_ #define _AUX_DRV_H_ +#include + #define PDS_VDPA_DRV_DESCRIPTION "AMD/Pensando vDPA VF Device Driver" #define PDS_VDPA_DRV_NAME KBUILD_MODNAME @@ -16,6 +18,7 @@ struct pds_vdpa_aux { int vf_id; struct dentry *dentry; + struct virtio_pci_modern_device vd_mdev; int nintrs; }; From patchwork Tue May 16 02:55:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13242505 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D5E38846D for ; Tue, 16 May 2023 02:55:57 +0000 (UTC) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2046.outbound.protection.outlook.com [40.107.100.46]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 51986449E for ; Mon, 15 May 2023 19:55:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kSuyv+xf4p5xEaqc4ndaotqabGCrgd9DLfe27hcNWaI7wraJwsDhD1jXjbuayC2ybNc/KirsUOaRT7X30UJAofz0nEymCSJu58dxQAoDNmkCv8JvIiOZHjmD55kCkjiIyVcwzw7GHPUfQ6pE5u+ld4UygQXJK/VwkSb8dkBHyozz001Qb5IUBTt5kezxreOmJeeRobnoxgcsOsOJC8MfsKf92UoSyHXQs2CUjHqek2By0oF0SaZBTWFBaSyKnmLx6w9ITdSHXEaXKI0oLCrl33Rdn6e+cJgI/mZKjNpB/qtWG/Emy2VvqNbJuFmxwgMRVXuymkSN5MTm1eBIdZ7i4w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2cMQJdmdzsxJ2kM66vHCHLSEYBcgp+6lYO1eq3z2SsU=; b=gKtDsJMNBd/4oANw2rVRU+r583J+hCHsw6QOhWLhb+M0CCj44ZEZJpvJ1gljSXXkMmxsYmUScaMIXQwAa+kHrjftJnd/cS3rYZpPSj2vW4QtJnvvP9TA3xd2+DzOzXrZjWw8BnSiLsI/7gYPlxntvqdQSpWhEcwQcOfzlHRjj4mIIwSLRSDj2M5PadkIOo2HWm53Ox3xZ2EAj/vKavC4uypsRtkLMaqJtz69I4ecADH89GjZGD8CrMtC+ylybAJHS7Sc3OOAWq/N8eAUJVsW/xusysrJS5+i6IjF4weKb9gb+73mv2cUI4ETQJiyA085NXlup1t/j+n2KryvKEYpjA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2cMQJdmdzsxJ2kM66vHCHLSEYBcgp+6lYO1eq3z2SsU=; b=5qjKNvA+6jbleAfsWo/cb+roCBAfML+gFLStfik32vfxnqhlIco0jj7RSsh9YNu9UrzLNDQ0YguDyZB8jM8AQw0n54nf+2BRVYs0xdCWNniCeTJvNvR1mbuyxJS7SBa/FH68XR3MuLKx/d3cdm1FP9szSTkCKx+v2JHTmlyNDto= Received: from SJ0PR03CA0027.namprd03.prod.outlook.com (2603:10b6:a03:33a::32) by SJ0PR12MB7458.namprd12.prod.outlook.com (2603:10b6:a03:48d::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30; Tue, 16 May 2023 02:55:51 +0000 Received: from DM6NAM11FT096.eop-nam11.prod.protection.outlook.com (2603:10b6:a03:33a:cafe::48) by SJ0PR03CA0027.outlook.office365.com (2603:10b6:a03:33a::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30 via Frontend Transport; Tue, 16 May 2023 02:55:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT096.mail.protection.outlook.com (10.13.173.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6411.15 via Frontend Transport; Tue, 16 May 2023 02:55:50 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 15 May 2023 21:55:49 -0500 From: Shannon Nelson To: , , , , , CC: , Subject: [PATCH v6 virtio 08/11] pds_vdpa: add vdpa config client commands Date: Mon, 15 May 2023 19:55:18 -0700 Message-ID: <20230516025521.43352-9-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230516025521.43352-1-shannon.nelson@amd.com> References: <20230516025521.43352-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT096:EE_|SJ0PR12MB7458:EE_ X-MS-Office365-Filtering-Correlation-Id: 919d2657-e996-4b2e-f85b-08db55b90edc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: GX8hvOeqI2Gg9Z028BYbA8hpUBhjISI2CF7/lFV8JBT1c+bpllicoYK7O+X8kdzq7FeUU4tBWub3mUH2cidm3Pog8g5kaPCivHeBij1wb+AY34SGg42rEAh5q98hHddEklhWn996/KOAyYCMaR9FIUyWPaArn2xFBeTDcaWt8LZKMX7+jYWSd8IMIVB9Sm+WeyRqEy6Kpela5noY/pFn8WANpM60rUsy/utLr81CoBCZ38dkUSBQCtOAuFz10KWnl0a8ZX1PyWY9KfBByw9OsXFhoSrnUkwNqKv/hCjEdKGD/ZimUdlSCwaoBluMlqLQl8tdbjIv7AriKa8RTmSdA4b3GgArGybgn6q7BhLtByz+Re2pzqsDpwSfOs5iiOVbC9wPu/Et8MGqqgBAdzcSkjt4nCEe9ktygCxmOEXPG6V6kl3PNiujbSSeNt4MqTydDtOVvtrbmo2QTyZm++B2bAV4hpflBD+A0jDB2ZM6e7IIfC48y2RGCUoeZSGoneL7TZ70wF5K81umEBOgk2KZQljF/sHoL1EFVVcznHzNBxN1MREceCMKKaRenP8tzl52+nFhuER3k0wddQ9Lqzcr4mmlCrQC6JVo9C0Oy/uaFJLlyEtjv6/iYnbksTjKWO64wro9KU3je0UJ5/mG4WVfto+iC0BGycHOVUFxtoHImHzE9NttZGFjZYc1gR6QdPWM0QgL9nR6QW4isH+R4CoSwq0LyA4dCRkbGar5Mn7oBOctRtOrBOmYtIbcA9lodWkCGOzMblqm9MtmFa8e1llGoA== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(136003)(376002)(346002)(396003)(39860400002)(451199021)(40470700004)(36840700001)(46966006)(6666004)(36756003)(4326008)(47076005)(478600001)(54906003)(110136005)(316002)(70206006)(70586007)(81166007)(356005)(5660300002)(2616005)(41300700001)(426003)(83380400001)(336012)(36860700001)(82740400003)(16526019)(8676002)(1076003)(26005)(8936002)(44832011)(186003)(40460700003)(2906002)(86362001)(40480700001)(82310400005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2023 02:55:50.8949 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 919d2657-e996-4b2e-f85b-08db55b90edc X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT096.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB7458 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net These are the adminq commands that will be needed for setting up and using the vDPA device. There are a number of commands defined in the FW's API, but by making use of the FW's virtio BAR we only need a few of these commands for vDPA support. Signed-off-by: Shannon Nelson Acked-by: Jason Wang --- drivers/vdpa/pds/Makefile | 1 + drivers/vdpa/pds/cmds.c | 185 ++++++++++++++++++++++++++++++++++++ drivers/vdpa/pds/cmds.h | 18 ++++ drivers/vdpa/pds/vdpa_dev.h | 33 ++++++- 4 files changed, 236 insertions(+), 1 deletion(-) create mode 100644 drivers/vdpa/pds/cmds.c create mode 100644 drivers/vdpa/pds/cmds.h diff --git a/drivers/vdpa/pds/Makefile b/drivers/vdpa/pds/Makefile index 13b50394ec64..2e22418e3ab3 100644 --- a/drivers/vdpa/pds/Makefile +++ b/drivers/vdpa/pds/Makefile @@ -4,6 +4,7 @@ obj-$(CONFIG_PDS_VDPA) := pds_vdpa.o pds_vdpa-y := aux_drv.o \ + cmds.o \ vdpa_dev.o pds_vdpa-$(CONFIG_DEBUG_FS) += debugfs.o diff --git a/drivers/vdpa/pds/cmds.c b/drivers/vdpa/pds/cmds.c new file mode 100644 index 000000000000..80863a41c3cd --- /dev/null +++ b/drivers/vdpa/pds/cmds.c @@ -0,0 +1,185 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2023 Advanced Micro Devices, Inc */ + +#include +#include + +#include +#include +#include +#include + +#include "vdpa_dev.h" +#include "aux_drv.h" +#include "cmds.h" + +int pds_vdpa_init_hw(struct pds_vdpa_device *pdsv) +{ + struct pds_auxiliary_dev *padev = pdsv->vdpa_aux->padev; + struct device *dev = &padev->aux_dev.dev; + union pds_core_adminq_cmd cmd = { + .vdpa_init.opcode = PDS_VDPA_CMD_INIT, + .vdpa_init.vdpa_index = pdsv->vdpa_index, + .vdpa_init.vf_id = cpu_to_le16(pdsv->vdpa_aux->vf_id), + }; + union pds_core_adminq_comp comp = {}; + int err; + + /* Initialize the vdpa/virtio device */ + err = pds_client_adminq_cmd(padev, &cmd, sizeof(cmd.vdpa_init), + &comp, 0); + if (err) + dev_dbg(dev, "Failed to init hw, status %d: %pe\n", + comp.status, ERR_PTR(err)); + + return err; +} + +int pds_vdpa_cmd_reset(struct pds_vdpa_device *pdsv) +{ + struct pds_auxiliary_dev *padev = pdsv->vdpa_aux->padev; + struct device *dev = &padev->aux_dev.dev; + union pds_core_adminq_cmd cmd = { + .vdpa.opcode = PDS_VDPA_CMD_RESET, + .vdpa.vdpa_index = pdsv->vdpa_index, + .vdpa.vf_id = cpu_to_le16(pdsv->vdpa_aux->vf_id), + }; + union pds_core_adminq_comp comp = {}; + int err; + + err = pds_client_adminq_cmd(padev, &cmd, sizeof(cmd.vdpa), &comp, 0); + if (err) + dev_dbg(dev, "Failed to reset hw, status %d: %pe\n", + comp.status, ERR_PTR(err)); + + return err; +} + +int pds_vdpa_cmd_set_status(struct pds_vdpa_device *pdsv, u8 status) +{ + struct pds_auxiliary_dev *padev = pdsv->vdpa_aux->padev; + struct device *dev = &padev->aux_dev.dev; + union pds_core_adminq_cmd cmd = { + .vdpa_status.opcode = PDS_VDPA_CMD_STATUS_UPDATE, + .vdpa_status.vdpa_index = pdsv->vdpa_index, + .vdpa_status.vf_id = cpu_to_le16(pdsv->vdpa_aux->vf_id), + .vdpa_status.status = status, + }; + union pds_core_adminq_comp comp = {}; + int err; + + err = pds_client_adminq_cmd(padev, &cmd, sizeof(cmd.vdpa_status), &comp, 0); + if (err) + dev_dbg(dev, "Failed to set status to %#x, error status %d: %pe\n", + status, comp.status, ERR_PTR(err)); + + return err; +} + +int pds_vdpa_cmd_set_mac(struct pds_vdpa_device *pdsv, u8 *mac) +{ + struct pds_auxiliary_dev *padev = pdsv->vdpa_aux->padev; + struct device *dev = &padev->aux_dev.dev; + union pds_core_adminq_cmd cmd = { + .vdpa_setattr.opcode = PDS_VDPA_CMD_SET_ATTR, + .vdpa_setattr.vdpa_index = pdsv->vdpa_index, + .vdpa_setattr.vf_id = cpu_to_le16(pdsv->vdpa_aux->vf_id), + .vdpa_setattr.attr = PDS_VDPA_ATTR_MAC, + }; + union pds_core_adminq_comp comp = {}; + int err; + + ether_addr_copy(cmd.vdpa_setattr.mac, mac); + err = pds_client_adminq_cmd(padev, &cmd, sizeof(cmd.vdpa_setattr), + &comp, 0); + if (err) + dev_dbg(dev, "Failed to set mac address %pM, status %d: %pe\n", + mac, comp.status, ERR_PTR(err)); + + return err; +} + +int pds_vdpa_cmd_set_max_vq_pairs(struct pds_vdpa_device *pdsv, u16 max_vqp) +{ + struct pds_auxiliary_dev *padev = pdsv->vdpa_aux->padev; + struct device *dev = &padev->aux_dev.dev; + union pds_core_adminq_cmd cmd = { + .vdpa_setattr.opcode = PDS_VDPA_CMD_SET_ATTR, + .vdpa_setattr.vdpa_index = pdsv->vdpa_index, + .vdpa_setattr.vf_id = cpu_to_le16(pdsv->vdpa_aux->vf_id), + .vdpa_setattr.attr = PDS_VDPA_ATTR_MAX_VQ_PAIRS, + .vdpa_setattr.max_vq_pairs = cpu_to_le16(max_vqp), + }; + union pds_core_adminq_comp comp = {}; + int err; + + err = pds_client_adminq_cmd(padev, &cmd, sizeof(cmd.vdpa_setattr), + &comp, 0); + if (err) + dev_dbg(dev, "Failed to set max vq pairs %u, status %d: %pe\n", + max_vqp, comp.status, ERR_PTR(err)); + + return err; +} + +int pds_vdpa_cmd_init_vq(struct pds_vdpa_device *pdsv, u16 qid, u16 invert_idx, + struct pds_vdpa_vq_info *vq_info) +{ + struct pds_auxiliary_dev *padev = pdsv->vdpa_aux->padev; + struct device *dev = &padev->aux_dev.dev; + union pds_core_adminq_cmd cmd = { + .vdpa_vq_init.opcode = PDS_VDPA_CMD_VQ_INIT, + .vdpa_vq_init.vdpa_index = pdsv->vdpa_index, + .vdpa_vq_init.vf_id = cpu_to_le16(pdsv->vdpa_aux->vf_id), + .vdpa_vq_init.qid = cpu_to_le16(qid), + .vdpa_vq_init.len = cpu_to_le16(ilog2(vq_info->q_len)), + .vdpa_vq_init.desc_addr = cpu_to_le64(vq_info->desc_addr), + .vdpa_vq_init.avail_addr = cpu_to_le64(vq_info->avail_addr), + .vdpa_vq_init.used_addr = cpu_to_le64(vq_info->used_addr), + .vdpa_vq_init.intr_index = cpu_to_le16(qid), + .vdpa_vq_init.avail_index = cpu_to_le16(vq_info->avail_idx ^ invert_idx), + .vdpa_vq_init.used_index = cpu_to_le16(vq_info->used_idx ^ invert_idx), + }; + union pds_core_adminq_comp comp = {}; + int err; + + dev_dbg(dev, "%s: qid %d len %d desc_addr %#llx avail_addr %#llx used_addr %#llx\n", + __func__, qid, ilog2(vq_info->q_len), + vq_info->desc_addr, vq_info->avail_addr, vq_info->used_addr); + + err = pds_client_adminq_cmd(padev, &cmd, sizeof(cmd.vdpa_vq_init), + &comp, 0); + if (err) + dev_dbg(dev, "Failed to init vq %d, status %d: %pe\n", + qid, comp.status, ERR_PTR(err)); + + return err; +} + +int pds_vdpa_cmd_reset_vq(struct pds_vdpa_device *pdsv, u16 qid, u16 invert_idx, + struct pds_vdpa_vq_info *vq_info) +{ + struct pds_auxiliary_dev *padev = pdsv->vdpa_aux->padev; + struct device *dev = &padev->aux_dev.dev; + union pds_core_adminq_cmd cmd = { + .vdpa_vq_reset.opcode = PDS_VDPA_CMD_VQ_RESET, + .vdpa_vq_reset.vdpa_index = pdsv->vdpa_index, + .vdpa_vq_reset.vf_id = cpu_to_le16(pdsv->vdpa_aux->vf_id), + .vdpa_vq_reset.qid = cpu_to_le16(qid), + }; + union pds_core_adminq_comp comp = {}; + int err; + + err = pds_client_adminq_cmd(padev, &cmd, sizeof(cmd.vdpa_vq_reset), + &comp, 0); + if (err) { + dev_dbg(dev, "Failed to reset vq %d, status %d: %pe\n", + qid, comp.status, ERR_PTR(err)); + return err; + } + + vq_info->avail_idx = le16_to_cpu(comp.vdpa_vq_reset.avail_index) ^ invert_idx; + vq_info->used_idx = le16_to_cpu(comp.vdpa_vq_reset.used_index) ^ invert_idx; + + return 0; +} diff --git a/drivers/vdpa/pds/cmds.h b/drivers/vdpa/pds/cmds.h new file mode 100644 index 000000000000..e24d85cb8f1c --- /dev/null +++ b/drivers/vdpa/pds/cmds.h @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright(c) 2023 Advanced Micro Devices, Inc */ + +#ifndef _VDPA_CMDS_H_ +#define _VDPA_CMDS_H_ + +int pds_vdpa_init_hw(struct pds_vdpa_device *pdsv); + +int pds_vdpa_cmd_reset(struct pds_vdpa_device *pdsv); +int pds_vdpa_cmd_set_status(struct pds_vdpa_device *pdsv, u8 status); +int pds_vdpa_cmd_set_mac(struct pds_vdpa_device *pdsv, u8 *mac); +int pds_vdpa_cmd_set_max_vq_pairs(struct pds_vdpa_device *pdsv, u16 max_vqp); +int pds_vdpa_cmd_init_vq(struct pds_vdpa_device *pdsv, u16 qid, u16 invert_idx, + struct pds_vdpa_vq_info *vq_info); +int pds_vdpa_cmd_reset_vq(struct pds_vdpa_device *pdsv, u16 qid, u16 invert_idx, + struct pds_vdpa_vq_info *vq_info); +int pds_vdpa_cmd_set_features(struct pds_vdpa_device *pdsv, u64 features); +#endif /* _VDPA_CMDS_H_ */ diff --git a/drivers/vdpa/pds/vdpa_dev.h b/drivers/vdpa/pds/vdpa_dev.h index 97fab833a0aa..a21596f438c1 100644 --- a/drivers/vdpa/pds/vdpa_dev.h +++ b/drivers/vdpa/pds/vdpa_dev.h @@ -4,11 +4,42 @@ #ifndef _VDPA_DEV_H_ #define _VDPA_DEV_H_ -#define PDS_VDPA_MAX_QUEUES 65 +#include +#include + +struct pds_vdpa_vq_info { + bool ready; + u64 desc_addr; + u64 avail_addr; + u64 used_addr; + u32 q_len; + u16 qid; + int irq; + char irq_name[32]; + + void __iomem *notify; + dma_addr_t notify_pa; + + u64 doorbell; + u16 avail_idx; + u16 used_idx; + struct vdpa_callback event_cb; + struct pds_vdpa_device *pdsv; +}; + +#define PDS_VDPA_MAX_QUEUES 65 +#define PDS_VDPA_MAX_QLEN 32768 struct pds_vdpa_device { struct vdpa_device vdpa_dev; struct pds_vdpa_aux *vdpa_aux; + + struct pds_vdpa_vq_info vqs[PDS_VDPA_MAX_QUEUES]; + u64 req_features; /* features requested by vdpa */ + u64 actual_features; /* features negotiated and in use */ + u8 vdpa_index; /* rsvd for future subdevice use */ + u8 num_vqs; /* num vqs in use */ + struct vdpa_callback config_cb; }; int pds_vdpa_get_mgmt_info(struct pds_vdpa_aux *vdpa_aux); From patchwork Tue May 16 02:55:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13242506 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B25B846D for ; Tue, 16 May 2023 02:55:59 +0000 (UTC) Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2089.outbound.protection.outlook.com [40.107.244.89]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 84231E58 for ; Mon, 15 May 2023 19:55:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dTBe64Fe0r1z8mSVF/YZynAlZUJXXsf2hJMoX2jm/4BPzadhyMZkbizrK6wzKM6+JUhXX9D/Zz3o+8Y2ILB9K23mbO6FJ8/B2OSHeo8uLA3eGx93gFrMeb9m8hOKweefE/9uW7vp2aRHDnCKjpuFpYfaPBCukFMw1ehjvRQdRPRtg7DKpk00M71yEBj30FsPRtS2QEPVOCPMA4OIi/u9Y5JN+6PWH/5s+nFXI10a1p4Ej7LRKpD6//oRDO7/OF1rV7nKc+el6Bx343bmShpyDeyvl4ItoV0jF8nzY+IK5gnt1Juev2tbc2eG1zvvKc2ISmPsh6mn+fGtMa4GupnWsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zsVHtYdpRY56ZAHq2N4ViJencrC0UF5RGRXE90WK/6o=; b=jMpgkyuu534mxf9MH4s36lMKHrjC9hsPnt09jmU3stiGSSYudG7Dy8ndrCDIeNPmaxpq0a6StguU18RuueVV4oarA5T3zGmf3aK43hpIhF9RM4P/qTL74tJ72BGbLCoZYQaXx1iTZzos4Xn4aOaCqE8+liuFBG1Xxxd9Zzf/OefjzQmMDWl/6HlVvf228nWKFpk32f9kHHwYPY+Xbf7PwnREroufXdYlbaoer8ZhEJcmgyXL1AjLbKrVMRV0O8te+VtWSpiAOKMCz4gB/48Rg3HqLZM5qESUumUO+kacCumSFUbYUW5XbksMB6HaoJIP75cozLQTkjBJyJI3kGcXSQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zsVHtYdpRY56ZAHq2N4ViJencrC0UF5RGRXE90WK/6o=; b=X6iD7HpdIPYgpyCZiEtOybxN5az9K09lK+k/zKG5A2yD/jUEb2IT9/tidaLmw59vsQRDsMAH9sJ3vXJLCk1VjUEy4jqxio776R03/NL7r5nHh6vTa5bXfz68WdRIzjSTg71BgLoZsqNsAcPL5Eejw2eM0YNNqnxb44rzC4a3C+4= Received: from SJ0PR03CA0015.namprd03.prod.outlook.com (2603:10b6:a03:33a::20) by CY5PR12MB6179.namprd12.prod.outlook.com (2603:10b6:930:24::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.32; Tue, 16 May 2023 02:55:52 +0000 Received: from DM6NAM11FT096.eop-nam11.prod.protection.outlook.com (2603:10b6:a03:33a:cafe::69) by SJ0PR03CA0015.outlook.office365.com (2603:10b6:a03:33a::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30 via Frontend Transport; Tue, 16 May 2023 02:55:51 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT096.mail.protection.outlook.com (10.13.173.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6411.15 via Frontend Transport; Tue, 16 May 2023 02:55:51 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 15 May 2023 21:55:50 -0500 From: Shannon Nelson To: , , , , , CC: , Subject: [PATCH v6 virtio 09/11] pds_vdpa: add support for vdpa and vdpamgmt interfaces Date: Mon, 15 May 2023 19:55:19 -0700 Message-ID: <20230516025521.43352-10-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230516025521.43352-1-shannon.nelson@amd.com> References: <20230516025521.43352-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT096:EE_|CY5PR12MB6179:EE_ X-MS-Office365-Filtering-Correlation-Id: 33a2a449-78a7-4cb0-b70e-08db55b90f3e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Gm8cBnMZp2vNpgFRaA1ysvG88lAvmUJ1eGJ9aRtRJMjoEuXjQ4KB5EdKfhsqQkAYQrjtRMoSmFvmm/g/IV2ztAKVQ25N130brIKyw7b/pVzWH3kqFg1aojbrYfzMtLb97/Ms5zvGf79jMmnEOWuDEx/kAaGxC32biHnXnNt8tsskbqsL0pSRdiF4FY0pAVuBpQOoMhbddRdorcXMcYJH3x2GYiMh7tPeto2lhRkzvB/gFp7oxQiZpEvZRw5D85zr0Nta5r7BEAOXx2O5/cCxyYh9shMQ4asgLeaTl1KUppIdrxn5462ISQg2u7fWTS1SGLB8UcNL1PSTG5HDCLIhl4j9Q0hM4vRVfx2ApAqSkJzjf/zawd0jgerzD8rhlaomR90KJEmRL0OHwGdYSHfdKQIl3hz0rjK5N20GNArpRqCvJjcgrj8buGvzPy/ZQgZpvMPi+QYpi+e6Z461SKVMRDzGXgXmlwWgB+VPxpI7Sy7wonBQeC9TgskNJWNPX/D75CBtaRVDulryZ/yTwKbr3sXKLZS6Or1Hd33Nwt16vtC92twgTIyDfyLMX0Y3MhHq2dlfP2kZXjuB73MUu1CVKQeK62yt055aSyX9MCa7ZgV7BaGsa3J+f2TIv+tV6BHhCTPoTC3YTnVJSkpno5vaDH9yh4QZoX59d+6lVWSdZVRxkQf13mQDDncd4XwiMXoO8ho4+N5ucoeOiOEgXydLoeBTAmNt6We0/kMnyOlHIfscWiLrk/nQXxHdOdiUrCzf37fVmICeb7AFp00Fw3JcEg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(39860400002)(136003)(396003)(376002)(451199021)(40470700004)(46966006)(36840700001)(6666004)(478600001)(83380400001)(2616005)(47076005)(36860700001)(336012)(426003)(1076003)(40480700001)(26005)(16526019)(36756003)(356005)(81166007)(82310400005)(186003)(86362001)(82740400003)(40460700003)(70206006)(70586007)(4326008)(316002)(5660300002)(2906002)(30864003)(8676002)(8936002)(41300700001)(54906003)(110136005)(44832011)(66899021)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2023 02:55:51.5042 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 33a2a449-78a7-4cb0-b70e-08db55b90f3e X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT096.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6179 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net This is the vDPA device support, where we advertise that we can support the virtio queues and deal with the configuration work through the pds_core's adminq. Signed-off-by: Shannon Nelson Acked-by: Jason Wang --- drivers/vdpa/pds/aux_drv.c | 15 + drivers/vdpa/pds/aux_drv.h | 1 + drivers/vdpa/pds/debugfs.c | 263 ++++++++++++++++ drivers/vdpa/pds/debugfs.h | 5 + drivers/vdpa/pds/vdpa_dev.c | 605 +++++++++++++++++++++++++++++++++++- drivers/vdpa/pds/vdpa_dev.h | 4 +- 6 files changed, 891 insertions(+), 2 deletions(-) diff --git a/drivers/vdpa/pds/aux_drv.c b/drivers/vdpa/pds/aux_drv.c index 0c4a135b1484..186e9ee22eb1 100644 --- a/drivers/vdpa/pds/aux_drv.c +++ b/drivers/vdpa/pds/aux_drv.c @@ -63,8 +63,21 @@ static int pds_vdpa_probe(struct auxiliary_device *aux_dev, goto err_free_mgmt_info; } + /* Let vdpa know that we can provide devices */ + err = vdpa_mgmtdev_register(&vdpa_aux->vdpa_mdev); + if (err) { + dev_err(dev, "%s: Failed to initialize vdpa_mgmt interface: %pe\n", + __func__, ERR_PTR(err)); + goto err_free_virtio; + } + + pds_vdpa_debugfs_add_pcidev(vdpa_aux); + pds_vdpa_debugfs_add_ident(vdpa_aux); + return 0; +err_free_virtio: + vp_modern_remove(&vdpa_aux->vd_mdev); err_free_mgmt_info: pci_free_irq_vectors(padev->vf_pdev); err_free_mem: @@ -79,9 +92,11 @@ static void pds_vdpa_remove(struct auxiliary_device *aux_dev) struct pds_vdpa_aux *vdpa_aux = auxiliary_get_drvdata(aux_dev); struct device *dev = &aux_dev->dev; + vdpa_mgmtdev_unregister(&vdpa_aux->vdpa_mdev); vp_modern_remove(&vdpa_aux->vd_mdev); pci_free_irq_vectors(vdpa_aux->padev->vf_pdev); + pds_vdpa_debugfs_del_vdpadev(vdpa_aux); kfree(vdpa_aux); auxiliary_set_drvdata(aux_dev, NULL); diff --git a/drivers/vdpa/pds/aux_drv.h b/drivers/vdpa/pds/aux_drv.h index 99e0ff340bfa..26b75344156e 100644 --- a/drivers/vdpa/pds/aux_drv.h +++ b/drivers/vdpa/pds/aux_drv.h @@ -13,6 +13,7 @@ struct pds_vdpa_aux { struct pds_auxiliary_dev *padev; struct vdpa_mgmt_dev vdpa_mdev; + struct pds_vdpa_device *pdsv; struct pds_vdpa_ident ident; diff --git a/drivers/vdpa/pds/debugfs.c b/drivers/vdpa/pds/debugfs.c index d91dceb07380..21a0dc0cb607 100644 --- a/drivers/vdpa/pds/debugfs.c +++ b/drivers/vdpa/pds/debugfs.c @@ -10,6 +10,7 @@ #include #include "aux_drv.h" +#include "vdpa_dev.h" #include "debugfs.h" static struct dentry *dbfs_dir; @@ -24,3 +25,265 @@ void pds_vdpa_debugfs_destroy(void) debugfs_remove_recursive(dbfs_dir); dbfs_dir = NULL; } + +#define PRINT_SBIT_NAME(__seq, __f, __name) \ + do { \ + if ((__f) & (__name)) \ + seq_printf(__seq, " %s", &#__name[16]); \ + } while (0) + +static void print_status_bits(struct seq_file *seq, u8 status) +{ + seq_puts(seq, "status:"); + PRINT_SBIT_NAME(seq, status, VIRTIO_CONFIG_S_ACKNOWLEDGE); + PRINT_SBIT_NAME(seq, status, VIRTIO_CONFIG_S_DRIVER); + PRINT_SBIT_NAME(seq, status, VIRTIO_CONFIG_S_DRIVER_OK); + PRINT_SBIT_NAME(seq, status, VIRTIO_CONFIG_S_FEATURES_OK); + PRINT_SBIT_NAME(seq, status, VIRTIO_CONFIG_S_NEEDS_RESET); + PRINT_SBIT_NAME(seq, status, VIRTIO_CONFIG_S_FAILED); + seq_puts(seq, "\n"); +} + +static void print_feature_bits_all(struct seq_file *seq, u64 features) +{ + int i; + + seq_puts(seq, "features:"); + + for (i = 0; i < (sizeof(u64) * 8); i++) { + u64 mask = BIT_ULL(i); + + switch (features & mask) { + case BIT_ULL(VIRTIO_NET_F_CSUM): + seq_puts(seq, " VIRTIO_NET_F_CSUM"); + break; + case BIT_ULL(VIRTIO_NET_F_GUEST_CSUM): + seq_puts(seq, " VIRTIO_NET_F_GUEST_CSUM"); + break; + case BIT_ULL(VIRTIO_NET_F_CTRL_GUEST_OFFLOADS): + seq_puts(seq, " VIRTIO_NET_F_CTRL_GUEST_OFFLOADS"); + break; + case BIT_ULL(VIRTIO_NET_F_MTU): + seq_puts(seq, " VIRTIO_NET_F_MTU"); + break; + case BIT_ULL(VIRTIO_NET_F_MAC): + seq_puts(seq, " VIRTIO_NET_F_MAC"); + break; + case BIT_ULL(VIRTIO_NET_F_GUEST_TSO4): + seq_puts(seq, " VIRTIO_NET_F_GUEST_TSO4"); + break; + case BIT_ULL(VIRTIO_NET_F_GUEST_TSO6): + seq_puts(seq, " VIRTIO_NET_F_GUEST_TSO6"); + break; + case BIT_ULL(VIRTIO_NET_F_GUEST_ECN): + seq_puts(seq, " VIRTIO_NET_F_GUEST_ECN"); + break; + case BIT_ULL(VIRTIO_NET_F_GUEST_UFO): + seq_puts(seq, " VIRTIO_NET_F_GUEST_UFO"); + break; + case BIT_ULL(VIRTIO_NET_F_HOST_TSO4): + seq_puts(seq, " VIRTIO_NET_F_HOST_TSO4"); + break; + case BIT_ULL(VIRTIO_NET_F_HOST_TSO6): + seq_puts(seq, " VIRTIO_NET_F_HOST_TSO6"); + break; + case BIT_ULL(VIRTIO_NET_F_HOST_ECN): + seq_puts(seq, " VIRTIO_NET_F_HOST_ECN"); + break; + case BIT_ULL(VIRTIO_NET_F_HOST_UFO): + seq_puts(seq, " VIRTIO_NET_F_HOST_UFO"); + break; + case BIT_ULL(VIRTIO_NET_F_MRG_RXBUF): + seq_puts(seq, " VIRTIO_NET_F_MRG_RXBUF"); + break; + case BIT_ULL(VIRTIO_NET_F_STATUS): + seq_puts(seq, " VIRTIO_NET_F_STATUS"); + break; + case BIT_ULL(VIRTIO_NET_F_CTRL_VQ): + seq_puts(seq, " VIRTIO_NET_F_CTRL_VQ"); + break; + case BIT_ULL(VIRTIO_NET_F_CTRL_RX): + seq_puts(seq, " VIRTIO_NET_F_CTRL_RX"); + break; + case BIT_ULL(VIRTIO_NET_F_CTRL_VLAN): + seq_puts(seq, " VIRTIO_NET_F_CTRL_VLAN"); + break; + case BIT_ULL(VIRTIO_NET_F_CTRL_RX_EXTRA): + seq_puts(seq, " VIRTIO_NET_F_CTRL_RX_EXTRA"); + break; + case BIT_ULL(VIRTIO_NET_F_GUEST_ANNOUNCE): + seq_puts(seq, " VIRTIO_NET_F_GUEST_ANNOUNCE"); + break; + case BIT_ULL(VIRTIO_NET_F_MQ): + seq_puts(seq, " VIRTIO_NET_F_MQ"); + break; + case BIT_ULL(VIRTIO_NET_F_CTRL_MAC_ADDR): + seq_puts(seq, " VIRTIO_NET_F_CTRL_MAC_ADDR"); + break; + case BIT_ULL(VIRTIO_NET_F_HASH_REPORT): + seq_puts(seq, " VIRTIO_NET_F_HASH_REPORT"); + break; + case BIT_ULL(VIRTIO_NET_F_RSS): + seq_puts(seq, " VIRTIO_NET_F_RSS"); + break; + case BIT_ULL(VIRTIO_NET_F_RSC_EXT): + seq_puts(seq, " VIRTIO_NET_F_RSC_EXT"); + break; + case BIT_ULL(VIRTIO_NET_F_STANDBY): + seq_puts(seq, " VIRTIO_NET_F_STANDBY"); + break; + case BIT_ULL(VIRTIO_NET_F_SPEED_DUPLEX): + seq_puts(seq, " VIRTIO_NET_F_SPEED_DUPLEX"); + break; + case BIT_ULL(VIRTIO_F_NOTIFY_ON_EMPTY): + seq_puts(seq, " VIRTIO_F_NOTIFY_ON_EMPTY"); + break; + case BIT_ULL(VIRTIO_F_ANY_LAYOUT): + seq_puts(seq, " VIRTIO_F_ANY_LAYOUT"); + break; + case BIT_ULL(VIRTIO_F_VERSION_1): + seq_puts(seq, " VIRTIO_F_VERSION_1"); + break; + case BIT_ULL(VIRTIO_F_ACCESS_PLATFORM): + seq_puts(seq, " VIRTIO_F_ACCESS_PLATFORM"); + break; + case BIT_ULL(VIRTIO_F_RING_PACKED): + seq_puts(seq, " VIRTIO_F_RING_PACKED"); + break; + case BIT_ULL(VIRTIO_F_ORDER_PLATFORM): + seq_puts(seq, " VIRTIO_F_ORDER_PLATFORM"); + break; + case BIT_ULL(VIRTIO_F_SR_IOV): + seq_puts(seq, " VIRTIO_F_SR_IOV"); + break; + case 0: + break; + default: + seq_printf(seq, " bit_%d", i); + break; + } + } + + seq_puts(seq, "\n"); +} + +void pds_vdpa_debugfs_add_pcidev(struct pds_vdpa_aux *vdpa_aux) +{ + vdpa_aux->dentry = debugfs_create_dir(pci_name(vdpa_aux->padev->vf_pdev), dbfs_dir); +} + +static int identity_show(struct seq_file *seq, void *v) +{ + struct pds_vdpa_aux *vdpa_aux = seq->private; + struct vdpa_mgmt_dev *mgmt; + + seq_printf(seq, "aux_dev: %s\n", + dev_name(&vdpa_aux->padev->aux_dev.dev)); + + mgmt = &vdpa_aux->vdpa_mdev; + seq_printf(seq, "max_vqs: %d\n", mgmt->max_supported_vqs); + seq_printf(seq, "config_attr_mask: %#llx\n", mgmt->config_attr_mask); + seq_printf(seq, "supported_features: %#llx\n", mgmt->supported_features); + print_feature_bits_all(seq, mgmt->supported_features); + + return 0; +} +DEFINE_SHOW_ATTRIBUTE(identity); + +void pds_vdpa_debugfs_add_ident(struct pds_vdpa_aux *vdpa_aux) +{ + debugfs_create_file("identity", 0400, vdpa_aux->dentry, + vdpa_aux, &identity_fops); +} + +static int config_show(struct seq_file *seq, void *v) +{ + struct pds_vdpa_device *pdsv = seq->private; + struct virtio_net_config vc; + u64 driver_features; + u8 status; + + memcpy_fromio(&vc, pdsv->vdpa_aux->vd_mdev.device, + sizeof(struct virtio_net_config)); + + seq_printf(seq, "mac: %pM\n", vc.mac); + seq_printf(seq, "max_virtqueue_pairs: %d\n", + __virtio16_to_cpu(true, vc.max_virtqueue_pairs)); + seq_printf(seq, "mtu: %d\n", __virtio16_to_cpu(true, vc.mtu)); + seq_printf(seq, "speed: %d\n", le32_to_cpu(vc.speed)); + seq_printf(seq, "duplex: %d\n", vc.duplex); + seq_printf(seq, "rss_max_key_size: %d\n", vc.rss_max_key_size); + seq_printf(seq, "rss_max_indirection_table_length: %d\n", + le16_to_cpu(vc.rss_max_indirection_table_length)); + seq_printf(seq, "supported_hash_types: %#x\n", + le32_to_cpu(vc.supported_hash_types)); + seq_printf(seq, "vn_status: %#x\n", + __virtio16_to_cpu(true, vc.status)); + + status = vp_modern_get_status(&pdsv->vdpa_aux->vd_mdev); + seq_printf(seq, "dev_status: %#x\n", status); + print_status_bits(seq, status); + + seq_printf(seq, "req_features: %#llx\n", pdsv->req_features); + print_feature_bits_all(seq, pdsv->req_features); + driver_features = vp_modern_get_driver_features(&pdsv->vdpa_aux->vd_mdev); + seq_printf(seq, "driver_features: %#llx\n", driver_features); + print_feature_bits_all(seq, driver_features); + seq_printf(seq, "vdpa_index: %d\n", pdsv->vdpa_index); + seq_printf(seq, "num_vqs: %d\n", pdsv->num_vqs); + + return 0; +} +DEFINE_SHOW_ATTRIBUTE(config); + +static int vq_show(struct seq_file *seq, void *v) +{ + struct pds_vdpa_vq_info *vq = seq->private; + + seq_printf(seq, "ready: %d\n", vq->ready); + seq_printf(seq, "desc_addr: %#llx\n", vq->desc_addr); + seq_printf(seq, "avail_addr: %#llx\n", vq->avail_addr); + seq_printf(seq, "used_addr: %#llx\n", vq->used_addr); + seq_printf(seq, "q_len: %d\n", vq->q_len); + seq_printf(seq, "qid: %d\n", vq->qid); + + seq_printf(seq, "doorbell: %#llx\n", vq->doorbell); + seq_printf(seq, "avail_idx: %d\n", vq->avail_idx); + seq_printf(seq, "used_idx: %d\n", vq->used_idx); + seq_printf(seq, "irq: %d\n", vq->irq); + seq_printf(seq, "irq-name: %s\n", vq->irq_name); + + return 0; +} +DEFINE_SHOW_ATTRIBUTE(vq); + +void pds_vdpa_debugfs_add_vdpadev(struct pds_vdpa_aux *vdpa_aux) +{ + int i; + + debugfs_create_file("config", 0400, vdpa_aux->dentry, vdpa_aux->pdsv, &config_fops); + + for (i = 0; i < vdpa_aux->pdsv->num_vqs; i++) { + char name[8]; + + snprintf(name, sizeof(name), "vq%02d", i); + debugfs_create_file(name, 0400, vdpa_aux->dentry, + &vdpa_aux->pdsv->vqs[i], &vq_fops); + } +} + +void pds_vdpa_debugfs_del_vdpadev(struct pds_vdpa_aux *vdpa_aux) +{ + debugfs_remove_recursive(vdpa_aux->dentry); + vdpa_aux->dentry = NULL; +} + +void pds_vdpa_debugfs_reset_vdpadev(struct pds_vdpa_aux *vdpa_aux) +{ + /* we don't keep track of the entries, so remove it all + * then rebuild the basics + */ + pds_vdpa_debugfs_del_vdpadev(vdpa_aux); + pds_vdpa_debugfs_add_pcidev(vdpa_aux); + pds_vdpa_debugfs_add_ident(vdpa_aux); +} diff --git a/drivers/vdpa/pds/debugfs.h b/drivers/vdpa/pds/debugfs.h index 658849591a99..c088a4e8f1e9 100644 --- a/drivers/vdpa/pds/debugfs.h +++ b/drivers/vdpa/pds/debugfs.h @@ -8,5 +8,10 @@ void pds_vdpa_debugfs_create(void); void pds_vdpa_debugfs_destroy(void); +void pds_vdpa_debugfs_add_pcidev(struct pds_vdpa_aux *vdpa_aux); +void pds_vdpa_debugfs_add_ident(struct pds_vdpa_aux *vdpa_aux); +void pds_vdpa_debugfs_add_vdpadev(struct pds_vdpa_aux *vdpa_aux); +void pds_vdpa_debugfs_del_vdpadev(struct pds_vdpa_aux *vdpa_aux); +void pds_vdpa_debugfs_reset_vdpadev(struct pds_vdpa_aux *vdpa_aux); #endif /* _PDS_VDPA_DEBUGFS_H_ */ diff --git a/drivers/vdpa/pds/vdpa_dev.c b/drivers/vdpa/pds/vdpa_dev.c index 0f0f0ab8b811..07b98dff5701 100644 --- a/drivers/vdpa/pds/vdpa_dev.c +++ b/drivers/vdpa/pds/vdpa_dev.c @@ -4,6 +4,7 @@ #include #include #include +#include #include #include @@ -12,7 +13,464 @@ #include "vdpa_dev.h" #include "aux_drv.h" +#include "cmds.h" +#include "debugfs.h" +static u64 pds_vdpa_get_driver_features(struct vdpa_device *vdpa_dev); + +static struct pds_vdpa_device *vdpa_to_pdsv(struct vdpa_device *vdpa_dev) +{ + return container_of(vdpa_dev, struct pds_vdpa_device, vdpa_dev); +} + +static int pds_vdpa_set_vq_address(struct vdpa_device *vdpa_dev, u16 qid, + u64 desc_addr, u64 driver_addr, u64 device_addr) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + + pdsv->vqs[qid].desc_addr = desc_addr; + pdsv->vqs[qid].avail_addr = driver_addr; + pdsv->vqs[qid].used_addr = device_addr; + + return 0; +} + +static void pds_vdpa_set_vq_num(struct vdpa_device *vdpa_dev, u16 qid, u32 num) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + + pdsv->vqs[qid].q_len = num; +} + +static void pds_vdpa_kick_vq(struct vdpa_device *vdpa_dev, u16 qid) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + + iowrite16(qid, pdsv->vqs[qid].notify); +} + +static void pds_vdpa_set_vq_cb(struct vdpa_device *vdpa_dev, u16 qid, + struct vdpa_callback *cb) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + + pdsv->vqs[qid].event_cb = *cb; +} + +static irqreturn_t pds_vdpa_isr(int irq, void *data) +{ + struct pds_vdpa_vq_info *vq; + + vq = data; + if (vq->event_cb.callback) + vq->event_cb.callback(vq->event_cb.private); + + return IRQ_HANDLED; +} + +static void pds_vdpa_release_irq(struct pds_vdpa_device *pdsv, int qid) +{ + if (pdsv->vqs[qid].irq == VIRTIO_MSI_NO_VECTOR) + return; + + free_irq(pdsv->vqs[qid].irq, &pdsv->vqs[qid]); + pdsv->vqs[qid].irq = VIRTIO_MSI_NO_VECTOR; +} + +static void pds_vdpa_set_vq_ready(struct vdpa_device *vdpa_dev, u16 qid, bool ready) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + struct pci_dev *pdev = pdsv->vdpa_aux->padev->vf_pdev; + struct device *dev = &pdsv->vdpa_dev.dev; + u64 driver_features; + u16 invert_idx = 0; + int irq; + int err; + + dev_dbg(dev, "%s: qid %d ready %d => %d\n", + __func__, qid, pdsv->vqs[qid].ready, ready); + if (ready == pdsv->vqs[qid].ready) + return; + + driver_features = pds_vdpa_get_driver_features(vdpa_dev); + if (driver_features & BIT_ULL(VIRTIO_F_RING_PACKED)) + invert_idx = PDS_VDPA_PACKED_INVERT_IDX; + + if (ready) { + irq = pci_irq_vector(pdev, qid); + snprintf(pdsv->vqs[qid].irq_name, sizeof(pdsv->vqs[qid].irq_name), + "vdpa-%s-%d", dev_name(dev), qid); + + err = request_irq(irq, pds_vdpa_isr, 0, + pdsv->vqs[qid].irq_name, &pdsv->vqs[qid]); + if (err) { + dev_err(dev, "%s: no irq for qid %d: %pe\n", + __func__, qid, ERR_PTR(err)); + return; + } + pdsv->vqs[qid].irq = irq; + + /* Pass vq setup info to DSC using adminq to gather up and + * send all info at once so FW can do its full set up in + * one easy operation + */ + err = pds_vdpa_cmd_init_vq(pdsv, qid, invert_idx, &pdsv->vqs[qid]); + if (err) { + dev_err(dev, "Failed to init vq %d: %pe\n", + qid, ERR_PTR(err)); + pds_vdpa_release_irq(pdsv, qid); + ready = false; + } + } else { + err = pds_vdpa_cmd_reset_vq(pdsv, qid, invert_idx, &pdsv->vqs[qid]); + if (err) + dev_err(dev, "%s: reset_vq failed qid %d: %pe\n", + __func__, qid, ERR_PTR(err)); + pds_vdpa_release_irq(pdsv, qid); + } + + pdsv->vqs[qid].ready = ready; +} + +static bool pds_vdpa_get_vq_ready(struct vdpa_device *vdpa_dev, u16 qid) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + + return pdsv->vqs[qid].ready; +} + +static int pds_vdpa_set_vq_state(struct vdpa_device *vdpa_dev, u16 qid, + const struct vdpa_vq_state *state) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + struct pds_auxiliary_dev *padev = pdsv->vdpa_aux->padev; + struct device *dev = &padev->aux_dev.dev; + u64 driver_features; + u16 avail; + u16 used; + + if (pdsv->vqs[qid].ready) { + dev_err(dev, "Setting device position is denied while vq is enabled\n"); + return -EINVAL; + } + + driver_features = pds_vdpa_get_driver_features(vdpa_dev); + if (driver_features & BIT_ULL(VIRTIO_F_RING_PACKED)) { + avail = state->packed.last_avail_idx | + (state->packed.last_avail_counter << 15); + used = state->packed.last_used_idx | + (state->packed.last_used_counter << 15); + + /* The avail and used index are stored with the packed wrap + * counter bit inverted. This way, in case set_vq_state is + * not called, the initial value can be set to zero prior to + * feature negotiation, and it is good for both packed and + * split vq. + */ + avail ^= PDS_VDPA_PACKED_INVERT_IDX; + used ^= PDS_VDPA_PACKED_INVERT_IDX; + } else { + avail = state->split.avail_index; + /* state->split does not provide a used_index: + * the vq will be set to "empty" here, and the vq will read + * the current used index the next time the vq is kicked. + */ + used = avail; + } + + if (used != avail) { + dev_dbg(dev, "Setting used equal to avail, for interoperability\n"); + used = avail; + } + + pdsv->vqs[qid].avail_idx = avail; + pdsv->vqs[qid].used_idx = used; + + return 0; +} + +static int pds_vdpa_get_vq_state(struct vdpa_device *vdpa_dev, u16 qid, + struct vdpa_vq_state *state) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + struct pds_auxiliary_dev *padev = pdsv->vdpa_aux->padev; + struct device *dev = &padev->aux_dev.dev; + u64 driver_features; + u16 avail; + u16 used; + + if (pdsv->vqs[qid].ready) { + dev_err(dev, "Getting device position is denied while vq is enabled\n"); + return -EINVAL; + } + + avail = pdsv->vqs[qid].avail_idx; + used = pdsv->vqs[qid].used_idx; + + driver_features = pds_vdpa_get_driver_features(vdpa_dev); + if (driver_features & BIT_ULL(VIRTIO_F_RING_PACKED)) { + avail ^= PDS_VDPA_PACKED_INVERT_IDX; + used ^= PDS_VDPA_PACKED_INVERT_IDX; + + state->packed.last_avail_idx = avail & 0x7fff; + state->packed.last_avail_counter = avail >> 15; + state->packed.last_used_idx = used & 0x7fff; + state->packed.last_used_counter = used >> 15; + } else { + state->split.avail_index = avail; + /* state->split does not provide a used_index. */ + } + + return 0; +} + +static struct vdpa_notification_area +pds_vdpa_get_vq_notification(struct vdpa_device *vdpa_dev, u16 qid) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + struct virtio_pci_modern_device *vd_mdev; + struct vdpa_notification_area area; + + area.addr = pdsv->vqs[qid].notify_pa; + + vd_mdev = &pdsv->vdpa_aux->vd_mdev; + if (!vd_mdev->notify_offset_multiplier) + area.size = PDS_PAGE_SIZE; + else + area.size = vd_mdev->notify_offset_multiplier; + + return area; +} + +static int pds_vdpa_get_vq_irq(struct vdpa_device *vdpa_dev, u16 qid) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + + return pdsv->vqs[qid].irq; +} + +static u32 pds_vdpa_get_vq_align(struct vdpa_device *vdpa_dev) +{ + return PDS_PAGE_SIZE; +} + +static u32 pds_vdpa_get_vq_group(struct vdpa_device *vdpa_dev, u16 idx) +{ + return 0; +} + +static u64 pds_vdpa_get_device_features(struct vdpa_device *vdpa_dev) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + + return pdsv->supported_features; +} + +static int pds_vdpa_set_driver_features(struct vdpa_device *vdpa_dev, u64 features) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + struct device *dev = &pdsv->vdpa_dev.dev; + u64 driver_features; + u64 nego_features; + u64 missing; + + if (!(features & BIT_ULL(VIRTIO_F_ACCESS_PLATFORM)) && features) { + dev_err(dev, "VIRTIO_F_ACCESS_PLATFORM is not negotiated\n"); + return -EOPNOTSUPP; + } + + pdsv->req_features = features; + + /* Check for valid feature bits */ + nego_features = features & le64_to_cpu(pdsv->vdpa_aux->ident.hw_features); + missing = pdsv->req_features & ~nego_features; + if (missing) { + dev_err(dev, "Can't support all requested features in %#llx, missing %#llx features\n", + pdsv->req_features, missing); + return -EOPNOTSUPP; + } + + driver_features = pds_vdpa_get_driver_features(vdpa_dev); + dev_dbg(dev, "%s: %#llx => %#llx\n", + __func__, driver_features, nego_features); + + if (driver_features == nego_features) + return 0; + + vp_modern_set_features(&pdsv->vdpa_aux->vd_mdev, nego_features); + + return 0; +} + +static u64 pds_vdpa_get_driver_features(struct vdpa_device *vdpa_dev) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + + return vp_modern_get_driver_features(&pdsv->vdpa_aux->vd_mdev); +} + +static void pds_vdpa_set_config_cb(struct vdpa_device *vdpa_dev, + struct vdpa_callback *cb) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + + pdsv->config_cb.callback = cb->callback; + pdsv->config_cb.private = cb->private; +} + +static u16 pds_vdpa_get_vq_num_max(struct vdpa_device *vdpa_dev) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + + /* qemu has assert() that vq_num_max <= VIRTQUEUE_MAX_SIZE (1024) */ + return min_t(u16, 1024, BIT(le16_to_cpu(pdsv->vdpa_aux->ident.max_qlen))); +} + +static u32 pds_vdpa_get_device_id(struct vdpa_device *vdpa_dev) +{ + return VIRTIO_ID_NET; +} + +static u32 pds_vdpa_get_vendor_id(struct vdpa_device *vdpa_dev) +{ + return PCI_VENDOR_ID_PENSANDO; +} + +static u8 pds_vdpa_get_status(struct vdpa_device *vdpa_dev) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + + return vp_modern_get_status(&pdsv->vdpa_aux->vd_mdev); +} + +static void pds_vdpa_set_status(struct vdpa_device *vdpa_dev, u8 status) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + struct device *dev = &pdsv->vdpa_dev.dev; + u8 old_status; + int i; + + old_status = pds_vdpa_get_status(vdpa_dev); + dev_dbg(dev, "%s: old %#x new %#x\n", __func__, old_status, status); + + pds_vdpa_cmd_set_status(pdsv, status); + + /* Note: still working with FW on the need for this reset cmd */ + if (status == 0) { + pds_vdpa_cmd_reset(pdsv); + + for (i = 0; i < pdsv->num_vqs; i++) { + pdsv->vqs[i].avail_idx = 0; + pdsv->vqs[i].used_idx = 0; + } + } + + if (status & ~old_status & VIRTIO_CONFIG_S_FEATURES_OK) { + for (i = 0; i < pdsv->num_vqs; i++) { + pdsv->vqs[i].notify = + vp_modern_map_vq_notify(&pdsv->vdpa_aux->vd_mdev, + i, &pdsv->vqs[i].notify_pa); + } + } +} + +static int pds_vdpa_reset(struct vdpa_device *vdpa_dev) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + struct device *dev; + int err = 0; + u8 status; + int i; + + dev = &pdsv->vdpa_aux->padev->aux_dev.dev; + status = pds_vdpa_get_status(vdpa_dev); + + if (status == 0) + return 0; + + if (status & VIRTIO_CONFIG_S_DRIVER_OK) { + /* Reset the vqs */ + for (i = 0; i < pdsv->num_vqs && !err; i++) { + err = pds_vdpa_cmd_reset_vq(pdsv, i, 0, &pdsv->vqs[i]); + if (err) + dev_err(dev, "%s: reset_vq failed qid %d: %pe\n", + __func__, i, ERR_PTR(err)); + pds_vdpa_release_irq(pdsv, i); + memset(&pdsv->vqs[i], 0, sizeof(pdsv->vqs[0])); + pdsv->vqs[i].ready = false; + } + } + + pds_vdpa_set_status(vdpa_dev, 0); + + return 0; +} + +static size_t pds_vdpa_get_config_size(struct vdpa_device *vdpa_dev) +{ + return sizeof(struct virtio_net_config); +} + +static void pds_vdpa_get_config(struct vdpa_device *vdpa_dev, + unsigned int offset, + void *buf, unsigned int len) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + void __iomem *device; + + if (offset + len > sizeof(struct virtio_net_config)) { + WARN(true, "%s: bad read, offset %d len %d\n", __func__, offset, len); + return; + } + + device = pdsv->vdpa_aux->vd_mdev.device; + memcpy_fromio(buf, device + offset, len); +} + +static void pds_vdpa_set_config(struct vdpa_device *vdpa_dev, + unsigned int offset, const void *buf, + unsigned int len) +{ + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); + void __iomem *device; + + if (offset + len > sizeof(struct virtio_net_config)) { + WARN(true, "%s: bad read, offset %d len %d\n", __func__, offset, len); + return; + } + + device = pdsv->vdpa_aux->vd_mdev.device; + memcpy_toio(device + offset, buf, len); +} + +static const struct vdpa_config_ops pds_vdpa_ops = { + .set_vq_address = pds_vdpa_set_vq_address, + .set_vq_num = pds_vdpa_set_vq_num, + .kick_vq = pds_vdpa_kick_vq, + .set_vq_cb = pds_vdpa_set_vq_cb, + .set_vq_ready = pds_vdpa_set_vq_ready, + .get_vq_ready = pds_vdpa_get_vq_ready, + .set_vq_state = pds_vdpa_set_vq_state, + .get_vq_state = pds_vdpa_get_vq_state, + .get_vq_notification = pds_vdpa_get_vq_notification, + .get_vq_irq = pds_vdpa_get_vq_irq, + .get_vq_align = pds_vdpa_get_vq_align, + .get_vq_group = pds_vdpa_get_vq_group, + + .get_device_features = pds_vdpa_get_device_features, + .set_driver_features = pds_vdpa_set_driver_features, + .get_driver_features = pds_vdpa_get_driver_features, + .set_config_cb = pds_vdpa_set_config_cb, + .get_vq_num_max = pds_vdpa_get_vq_num_max, + .get_device_id = pds_vdpa_get_device_id, + .get_vendor_id = pds_vdpa_get_vendor_id, + .get_status = pds_vdpa_get_status, + .set_status = pds_vdpa_set_status, + .reset = pds_vdpa_reset, + .get_config_size = pds_vdpa_get_config_size, + .get_config = pds_vdpa_get_config, + .set_config = pds_vdpa_set_config, +}; static struct virtio_device_id pds_vdpa_id_table[] = { {VIRTIO_ID_NET, VIRTIO_DEV_ANY_ID}, {0}, @@ -21,12 +479,156 @@ static struct virtio_device_id pds_vdpa_id_table[] = { static int pds_vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name, const struct vdpa_dev_set_config *add_config) { - return -EOPNOTSUPP; + struct pds_vdpa_aux *vdpa_aux; + struct pds_vdpa_device *pdsv; + struct vdpa_mgmt_dev *mgmt; + u16 fw_max_vqs, vq_pairs; + struct device *dma_dev; + struct pci_dev *pdev; + struct device *dev; + u8 mac[ETH_ALEN]; + int err; + int i; + + vdpa_aux = container_of(mdev, struct pds_vdpa_aux, vdpa_mdev); + dev = &vdpa_aux->padev->aux_dev.dev; + mgmt = &vdpa_aux->vdpa_mdev; + + if (vdpa_aux->pdsv) { + dev_warn(dev, "Multiple vDPA devices on a VF is not supported.\n"); + return -EOPNOTSUPP; + } + + pdsv = vdpa_alloc_device(struct pds_vdpa_device, vdpa_dev, + dev, &pds_vdpa_ops, 1, 1, name, false); + if (IS_ERR(pdsv)) { + dev_err(dev, "Failed to allocate vDPA structure: %pe\n", pdsv); + return PTR_ERR(pdsv); + } + + vdpa_aux->pdsv = pdsv; + pdsv->vdpa_aux = vdpa_aux; + + pdev = vdpa_aux->padev->vf_pdev; + dma_dev = &pdev->dev; + pdsv->vdpa_dev.dma_dev = dma_dev; + + pdsv->supported_features = mgmt->supported_features; + + if (add_config->mask & BIT_ULL(VDPA_ATTR_DEV_FEATURES)) { + u64 unsupp_features = + add_config->device_features & ~mgmt->supported_features; + + if (unsupp_features) { + dev_err(dev, "Unsupported features: %#llx\n", unsupp_features); + goto err_unmap; + } + + pdsv->supported_features = add_config->device_features; + } + + err = pds_vdpa_cmd_reset(pdsv); + if (err) { + dev_err(dev, "Failed to reset hw: %pe\n", ERR_PTR(err)); + goto err_unmap; + } + + err = pds_vdpa_init_hw(pdsv); + if (err) { + dev_err(dev, "Failed to init hw: %pe\n", ERR_PTR(err)); + goto err_unmap; + } + + fw_max_vqs = le16_to_cpu(pdsv->vdpa_aux->ident.max_vqs); + vq_pairs = fw_max_vqs / 2; + + /* Make sure we have the queues being requested */ + if (add_config->mask & (1 << VDPA_ATTR_DEV_NET_CFG_MAX_VQP)) + vq_pairs = add_config->net.max_vq_pairs; + + pdsv->num_vqs = 2 * vq_pairs; + if (pdsv->supported_features & BIT_ULL(VIRTIO_NET_F_CTRL_VQ)) + pdsv->num_vqs++; + + if (pdsv->num_vqs > fw_max_vqs) { + dev_err(dev, "%s: queue count requested %u greater than max %u\n", + __func__, pdsv->num_vqs, fw_max_vqs); + err = -ENOSPC; + goto err_unmap; + } + + if (pdsv->num_vqs != fw_max_vqs) { + err = pds_vdpa_cmd_set_max_vq_pairs(pdsv, vq_pairs); + if (err) { + dev_err(dev, "Failed to set max_vq_pairs: %pe\n", + ERR_PTR(err)); + goto err_unmap; + } + } + + /* Set a mac, either from the user config if provided + * or set a random mac if default is 00:..:00 + */ + if (add_config->mask & BIT_ULL(VDPA_ATTR_DEV_NET_CFG_MACADDR)) { + ether_addr_copy(mac, add_config->net.mac); + pds_vdpa_cmd_set_mac(pdsv, mac); + } else { + struct virtio_net_config __iomem *vc; + + vc = pdsv->vdpa_aux->vd_mdev.device; + memcpy_fromio(mac, vc->mac, sizeof(mac)); + if (is_zero_ether_addr(mac)) { + eth_random_addr(mac); + dev_info(dev, "setting random mac %pM\n", mac); + pds_vdpa_cmd_set_mac(pdsv, mac); + } + } + + for (i = 0; i < pdsv->num_vqs; i++) { + pdsv->vqs[i].qid = i; + pdsv->vqs[i].pdsv = pdsv; + pdsv->vqs[i].irq = VIRTIO_MSI_NO_VECTOR; + pdsv->vqs[i].notify = vp_modern_map_vq_notify(&pdsv->vdpa_aux->vd_mdev, + i, &pdsv->vqs[i].notify_pa); + } + + pdsv->vdpa_dev.mdev = &vdpa_aux->vdpa_mdev; + + /* We use the _vdpa_register_device() call rather than the + * vdpa_register_device() to avoid a deadlock because our + * dev_add() is called with the vdpa_dev_lock already set + * by vdpa_nl_cmd_dev_add_set_doit() + */ + err = _vdpa_register_device(&pdsv->vdpa_dev, pdsv->num_vqs); + if (err) { + dev_err(dev, "Failed to register to vDPA bus: %pe\n", ERR_PTR(err)); + goto err_unmap; + } + + pds_vdpa_debugfs_add_vdpadev(vdpa_aux); + + return 0; + +err_unmap: + put_device(&pdsv->vdpa_dev.dev); + vdpa_aux->pdsv = NULL; + return err; } static void pds_vdpa_dev_del(struct vdpa_mgmt_dev *mdev, struct vdpa_device *vdpa_dev) { + struct pds_vdpa_aux *vdpa_aux; + + vdpa_aux = container_of(mdev, struct pds_vdpa_aux, vdpa_mdev); + _vdpa_unregister_device(vdpa_dev); + + pds_vdpa_cmd_reset(vdpa_aux->pdsv); + pds_vdpa_debugfs_reset_vdpadev(vdpa_aux); + + vdpa_aux->pdsv = NULL; + + dev_info(&vdpa_aux->padev->aux_dev.dev, "Removed vdpa device\n"); } static const struct vdpa_mgmtdev_ops pds_vdpa_mgmt_dev_ops = { @@ -94,6 +696,7 @@ int pds_vdpa_get_mgmt_info(struct pds_vdpa_aux *vdpa_aux) mgmt->supported_features = le64_to_cpu(vdpa_aux->ident.hw_features); mgmt->config_attr_mask = BIT_ULL(VDPA_ATTR_DEV_NET_CFG_MACADDR); mgmt->config_attr_mask |= BIT_ULL(VDPA_ATTR_DEV_NET_CFG_MAX_VQP); + mgmt->config_attr_mask |= BIT_ULL(VDPA_ATTR_DEV_FEATURES); err = pci_alloc_irq_vectors(pdev, vdpa_aux->nintrs, vdpa_aux->nintrs, PCI_IRQ_MSIX); diff --git a/drivers/vdpa/pds/vdpa_dev.h b/drivers/vdpa/pds/vdpa_dev.h index a21596f438c1..25c1d192f0ef 100644 --- a/drivers/vdpa/pds/vdpa_dev.h +++ b/drivers/vdpa/pds/vdpa_dev.h @@ -35,12 +35,14 @@ struct pds_vdpa_device { struct pds_vdpa_aux *vdpa_aux; struct pds_vdpa_vq_info vqs[PDS_VDPA_MAX_QUEUES]; + u64 supported_features; /* specified device features */ u64 req_features; /* features requested by vdpa */ - u64 actual_features; /* features negotiated and in use */ u8 vdpa_index; /* rsvd for future subdevice use */ u8 num_vqs; /* num vqs in use */ struct vdpa_callback config_cb; }; +#define PDS_VDPA_PACKED_INVERT_IDX 0x8000 + int pds_vdpa_get_mgmt_info(struct pds_vdpa_aux *vdpa_aux); #endif /* _VDPA_DEV_H_ */ From patchwork Tue May 16 02:55:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13242507 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A8678830 for ; Tue, 16 May 2023 02:56:01 +0000 (UTC) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2065.outbound.protection.outlook.com [40.107.237.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A25CD40D0 for ; Mon, 15 May 2023 19:55:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=M0otGPJ4/uKUXgJ3g4WM0t8sWgCrbanK1Eiwhe2ZlaX+uGBKRuZeq5z8518slthmZ9OBiYXrKllrx7X897mTgVkTsKy4ngP3Q3pj0BTARFWUFtRA8d0qZzb0o118J7udcHovcMpezc0XgRyQ9T5ppUsvVbaK6ccZQxh3wFEXSauL3AJFyGMDSfpGAu4n+lAgM/M/w0+LQFBJZExcLYvQurVX0/5Oy1dXIsUZjsyDRRRCJTspSxuApFXOKRxUFqCFgFV0kPRdDCMaePqlFix4J9ewwVpPgEe8Nx600S4npKznwZmWvzYaQr0wIm98YXc6DIYnc6DhHiFX7ZEb/5S9cQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UJLAmJMpBaqhUGOWtNzwNu+mPZmC2vq4ac/CQjEBuyM=; b=k1BkacBJ8jw3UBc/gMveUuFkq3iOCBMm6G5ig539HCdWAOn/wbCxEeB2gKtQBBwog/XPprAKF+Q0zK29fKPsBQ0jf1gdQo7R2XH/qhXarxeeA4z8+pMTGF4TN/hZHvupfMTu7R5YBjWCBpdNtE9dQ+Ynq8hr773Qlzlw8CKopMt77azrtdPKh6EHy+M4hDuzDXBjWUpGh82CVl0BcMx6D0O8pugTPaqf0kSM32RWHVuRTewXKV823MqH8E+aKqolPDGfIUAHzL0ijxMf3aJlgAtFk1CzM7xd7inzJWwWV/Zyni1kiRKMSmr/N0N2GsHFg12HXfQfDNy8b9BXE1uf3A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UJLAmJMpBaqhUGOWtNzwNu+mPZmC2vq4ac/CQjEBuyM=; b=lRZFOoBfuJ1tLBIn1uqHq7nu7qg4YJFjzHyE9nh8cauwqHOez+PawIetzp241Po57s7BwYj75BO9nu1o0AZKkYcYAZtaBUzAc8a3pvlAFL3OTpj6i6jikez73aS82pux4AyS1OeW9MvnjBW+YevsZmLD6HZNTsck40meTVd0kbM= Received: from SJ0PR03CA0024.namprd03.prod.outlook.com (2603:10b6:a03:33a::29) by SJ2PR12MB9244.namprd12.prod.outlook.com (2603:10b6:a03:574::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30; Tue, 16 May 2023 02:55:52 +0000 Received: from DM6NAM11FT096.eop-nam11.prod.protection.outlook.com (2603:10b6:a03:33a:cafe::15) by SJ0PR03CA0024.outlook.office365.com (2603:10b6:a03:33a::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30 via Frontend Transport; Tue, 16 May 2023 02:55:52 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT096.mail.protection.outlook.com (10.13.173.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6411.15 via Frontend Transport; Tue, 16 May 2023 02:55:52 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 15 May 2023 21:55:51 -0500 From: Shannon Nelson To: , , , , , CC: , Subject: [PATCH v6 virtio 10/11] pds_vdpa: subscribe to the pds_core events Date: Mon, 15 May 2023 19:55:20 -0700 Message-ID: <20230516025521.43352-11-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230516025521.43352-1-shannon.nelson@amd.com> References: <20230516025521.43352-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT096:EE_|SJ2PR12MB9244:EE_ X-MS-Office365-Filtering-Correlation-Id: acd7fd45-ff34-42e5-73d1-08db55b90fbf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2Ma7uHkln4dcMiZyUtgjhSKWNgmzmB2zpK+E4E1/3ne/5I/ehXvgoWq9Y/H1QcxNg0qabG4504e4eQ0xdBVncd7FYsrIEqFDjMPY6BOcmP+UxGSTzoaQUwM3qesjOUq6xBHc4D8ASZ/aq79igmeJNEG+feRjjon8WsQhrD+2ZgltVhuQxCIiEp3Qs4u7WKVqwJtJ7JK6JI8zipyn/zziid/FT+5H9/8RHiHmk+rK7lao2zfMjZ9T2+omUdpFdsRFgyMxQTCwZDEChEJL9lh8xB+n+NRiAXL8XgVYrucIoUncXoedswTDCWbMahfN8pAVl422bXO6V3XcPmUsCqlLdDVlbEcWTXL8icK/2rskIVQROVMvSJxV1gUQMfQecsqx95DhK2426d2MRPaOi6HIXkxOD+Fag2Og1Nz/+Wd+Ln17qM3PvdahRUl5lAoTpKTKIhrsA0Stf2k+remzr1q7kcHtayX/ZJsliBwY3o6PT8EarTNtoSwD6LMZWnRB5Ml5wRE9WaRJym8r/riKq0+3zKnY7JslttCpSh5Rjieatj+zKUQtoMP8gLCTFTTDkOJ9Z3YnQDO7GOpYwqSOfhtyFlRgqTjcDAI26n6dV2Lo4Nmtk9nXRIaA2Ymtbf6oGSSdrvl8erM3a6yfJVn6lGkgwo3Zd6FbCNayd75phwQhhxsFzBrzrPhJi/QyKA27VmEhlHV6g3LxdU1ckyKAthX+sIiFe7S5tc8wpQbOpAkkuUWAj3bw7t3qJizE1oRkZwqp1W5uxSgkMis1eOa3yzrm3A== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(346002)(376002)(39860400002)(396003)(136003)(451199021)(46966006)(40470700004)(36840700001)(83380400001)(82310400005)(44832011)(5660300002)(8936002)(8676002)(356005)(36860700001)(81166007)(316002)(86362001)(2906002)(47076005)(426003)(336012)(54906003)(2616005)(16526019)(186003)(110136005)(1076003)(40460700003)(478600001)(26005)(70206006)(70586007)(41300700001)(36756003)(82740400003)(6666004)(4326008)(40480700001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2023 02:55:52.3791 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: acd7fd45-ff34-42e5-73d1-08db55b90fbf X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT096.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB9244 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Register for the pds_core's notification events, primarily to find out when the FW has been reset so we can pass this on back up the chain. Signed-off-by: Shannon Nelson Acked-by: Jason Wang --- drivers/vdpa/pds/vdpa_dev.c | 59 ++++++++++++++++++++++++++++++++++++- drivers/vdpa/pds/vdpa_dev.h | 1 + 2 files changed, 59 insertions(+), 1 deletion(-) diff --git a/drivers/vdpa/pds/vdpa_dev.c b/drivers/vdpa/pds/vdpa_dev.c index 07b98dff5701..9afa803c4f21 100644 --- a/drivers/vdpa/pds/vdpa_dev.c +++ b/drivers/vdpa/pds/vdpa_dev.c @@ -23,6 +23,52 @@ static struct pds_vdpa_device *vdpa_to_pdsv(struct vdpa_device *vdpa_dev) return container_of(vdpa_dev, struct pds_vdpa_device, vdpa_dev); } +static int pds_vdpa_notify_handler(struct notifier_block *nb, + unsigned long ecode, + void *data) +{ + struct pds_vdpa_device *pdsv = container_of(nb, struct pds_vdpa_device, nb); + struct device *dev = &pdsv->vdpa_aux->padev->aux_dev.dev; + + dev_dbg(dev, "%s: event code %lu\n", __func__, ecode); + + if (ecode == PDS_EVENT_RESET || ecode == PDS_EVENT_LINK_CHANGE) { + if (pdsv->config_cb.callback) + pdsv->config_cb.callback(pdsv->config_cb.private); + } + + return 0; +} + +static int pds_vdpa_register_event_handler(struct pds_vdpa_device *pdsv) +{ + struct device *dev = &pdsv->vdpa_aux->padev->aux_dev.dev; + struct notifier_block *nb = &pdsv->nb; + int err; + + if (!nb->notifier_call) { + nb->notifier_call = pds_vdpa_notify_handler; + err = pdsc_register_notify(nb); + if (err) { + nb->notifier_call = NULL; + dev_err(dev, "failed to register pds event handler: %ps\n", + ERR_PTR(err)); + return -EINVAL; + } + dev_dbg(dev, "pds event handler registered\n"); + } + + return 0; +} + +static void pds_vdpa_unregister_event_handler(struct pds_vdpa_device *pdsv) +{ + if (pdsv->nb.notifier_call) { + pdsc_unregister_notify(&pdsv->nb); + pdsv->nb.notifier_call = NULL; + } +} + static int pds_vdpa_set_vq_address(struct vdpa_device *vdpa_dev, u16 qid, u64 desc_addr, u64 driver_addr, u64 device_addr) { @@ -594,6 +640,12 @@ static int pds_vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name, pdsv->vdpa_dev.mdev = &vdpa_aux->vdpa_mdev; + err = pds_vdpa_register_event_handler(pdsv); + if (err) { + dev_err(dev, "Failed to register for PDS events: %pe\n", ERR_PTR(err)); + goto err_unmap; + } + /* We use the _vdpa_register_device() call rather than the * vdpa_register_device() to avoid a deadlock because our * dev_add() is called with the vdpa_dev_lock already set @@ -602,13 +654,15 @@ static int pds_vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name, err = _vdpa_register_device(&pdsv->vdpa_dev, pdsv->num_vqs); if (err) { dev_err(dev, "Failed to register to vDPA bus: %pe\n", ERR_PTR(err)); - goto err_unmap; + goto err_unevent; } pds_vdpa_debugfs_add_vdpadev(vdpa_aux); return 0; +err_unevent: + pds_vdpa_unregister_event_handler(pdsv); err_unmap: put_device(&pdsv->vdpa_dev.dev); vdpa_aux->pdsv = NULL; @@ -618,8 +672,11 @@ static int pds_vdpa_dev_add(struct vdpa_mgmt_dev *mdev, const char *name, static void pds_vdpa_dev_del(struct vdpa_mgmt_dev *mdev, struct vdpa_device *vdpa_dev) { + struct pds_vdpa_device *pdsv = vdpa_to_pdsv(vdpa_dev); struct pds_vdpa_aux *vdpa_aux; + pds_vdpa_unregister_event_handler(pdsv); + vdpa_aux = container_of(mdev, struct pds_vdpa_aux, vdpa_mdev); _vdpa_unregister_device(vdpa_dev); diff --git a/drivers/vdpa/pds/vdpa_dev.h b/drivers/vdpa/pds/vdpa_dev.h index 25c1d192f0ef..a1bc37de9537 100644 --- a/drivers/vdpa/pds/vdpa_dev.h +++ b/drivers/vdpa/pds/vdpa_dev.h @@ -40,6 +40,7 @@ struct pds_vdpa_device { u8 vdpa_index; /* rsvd for future subdevice use */ u8 num_vqs; /* num vqs in use */ struct vdpa_callback config_cb; + struct notifier_block nb; }; #define PDS_VDPA_PACKED_INVERT_IDX 0x8000 From patchwork Tue May 16 02:55:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13242508 X-Patchwork-Delegate: kuba@kernel.org Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net [23.128.96.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CFD832564 for ; Tue, 16 May 2023 02:56:27 +0000 (UTC) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2078.outbound.protection.outlook.com [40.107.212.78]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 62D273C1F for ; Mon, 15 May 2023 19:56:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=C4dRzVVW/Mbbc/tGJ090vnkmhNU+E6lwXzPjnhVvhUlyhkJumJzWuvo7I4reHIF4IcNfcw3ApYFbn1sne0dnxLyu9zkTjiG2vuEeF40UmsmSUpgTtu0R10SYU2hmbm9i8aRMkIBia3DnmXAg5T/9DtB4T3kzQzuoD4aWRI+0DlWqT/k6G0cen2i0Lw2YpreiWDgmz1zWo7jhTodVbtmLS22YBu9yzp8TiRsjH12OZiOYb+uYKoaR5P6B4C+XZoCNYa5Ai1O5PxaEotXi69IrzBQVZ5wAEvAWD6XjHZANmIVH+Mi3GuJH24uMFcN6VMfVSc4Wmm9MYN60Ev4vTRx2VQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mN115WYMdXV08PljWxqUP2AhmB9hMDME0OtPNuSTKhM=; b=eIUxHmfVtJbddTHPFOjzFH3Bw88cHm7WIKoh8Zb4bYj61lsdHys26t0Mtj2nIwq6yDpK0Vn1ToNsjuLddKcQCpqLjhKd7Lu7TfcWzhQUCgSCl83qy8RFjMCDtvjubmUlNWQeSWGdkXYPU51YwjSONo0dN8ZxUgPcxnJz7HUX50FJ/WGfcrDN2j8tGb5CA7cNhXXrUQpTCLTZepWXwAotta2zyilsHPlBeL+qoUoWz7xX31YGMn4XEovD+KVpg6sC9zq2eCxWJNhiUqB17yh+ULZZA8Lbu9fHx4TjmxuK7KSc0fvLdsPt5XoSI5k/W0ruatDIIaq0HoP5g1iPYv46xg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=redhat.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mN115WYMdXV08PljWxqUP2AhmB9hMDME0OtPNuSTKhM=; b=PZE15o/Fiq0BZZrRhxD1sCEk9V1lh1RMlNwF3b8541302zQ4lv74FOxsZJgyXEN/3txM+8rkCuaKdlQdP8rvXOAuWtnECOHXGG/f7ojR3oAyhEu++ogEpJtqW4JRJRFjOEo2tbc5FHOCX7LDQuhmSd7vhqSDWa+cyd1+jhUv/kU= Received: from DM6PR02CA0102.namprd02.prod.outlook.com (2603:10b6:5:1f4::43) by DM6PR12MB4249.namprd12.prod.outlook.com (2603:10b6:5:223::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30; Tue, 16 May 2023 02:56:24 +0000 Received: from DM6NAM11FT112.eop-nam11.prod.protection.outlook.com (2603:10b6:5:1f4:cafe::ad) by DM6PR02CA0102.outlook.office365.com (2603:10b6:5:1f4::43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6387.30 via Frontend Transport; Tue, 16 May 2023 02:56:24 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT112.mail.protection.outlook.com (10.13.173.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6411.15 via Frontend Transport; Tue, 16 May 2023 02:56:23 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Mon, 15 May 2023 21:55:52 -0500 From: Shannon Nelson To: , , , , , CC: , Subject: [PATCH v6 virtio 11/11] pds_vdpa: pds_vdps.rst and Kconfig Date: Mon, 15 May 2023 19:55:21 -0700 Message-ID: <20230516025521.43352-12-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230516025521.43352-1-shannon.nelson@amd.com> References: <20230516025521.43352-1-shannon.nelson@amd.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT112:EE_|DM6PR12MB4249:EE_ X-MS-Office365-Filtering-Correlation-Id: a6577c5e-2c6f-4527-73ec-08db55b9228c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: LPjXxbFsxcnSdvmx+IG1SsRo9jm65X6bcfeeGhiO7eoQ7VsdiV7Cj6BXpJ2c3rrsXfWhKCNNe6H4FlXP8F5Mp4p51bFNSoLK7YFJnC4w+fC/o1EujkgfO8FbTIFqugoIZvjlf90B9Xg5HNFErC9gYE9kOM177HF0fxHj7kI0TdSEe3R6TFEob5kONGlNZQhOUnqNc2iEYrCMnKv4wsFNdcSyJ0wFYmFd53W/ZgfWxUwjrmnfDe8DKcua5MSD5eP2KNIH/MBmH3JFaInDeWZR4dwMfxtwhEK0z2fawQvoLDaGzK03tYBB4k/uuPZADumjt5pUyllzddWK+6TpanfMEniMfQdec2KDbVmzBXIeZ8wOA11o7Lbczw3KTjgqxpyonE87xk/4yb1iEVvRa8qMBf2zRJH3p47MccUECoegPIrMIqKumcAbGkWsUKhobMFjJD7YOVRXWku06fSJ13JsCmoz5V0PbnkQRJIfRSohQMxPHIcu3gTxZg5/22fek+r4RhRrtZTNO7s2SxxZJDAMm4JLDDB0YQUTTwvqPd6C0Iwt2ZaTJ4yN4Y7WjGlAaCbQi/tN976J6UZR4/fj3uYBf3e6qWMbxN5kSugvQVkWkZIQHPxWMBW/Wjkw2phAtb2OQJeOhjuVvYvBCcdav6JOJvxYHfKKS4KODkDpIW0QyUamftCKDINgehohyI6sGPuQUvc0aV+AVTY4p++WEowbP3DWimD9EbLi4lgtWqg4XQvzPvUnqlC1jttisOWhe8Wfr6sjV6uRC1xjRO+kyWTMCJBQ6NW5KFb493FQQdXLsqnsBGKAsFB8MQLmuC1SWSx2 X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(376002)(396003)(39860400002)(136003)(346002)(451199021)(36840700001)(46966006)(40470700004)(40460700003)(66899021)(70586007)(4326008)(70206006)(478600001)(86362001)(316002)(110136005)(54906003)(36756003)(426003)(47076005)(26005)(1076003)(186003)(83380400001)(36860700001)(336012)(43170500006)(5660300002)(8936002)(8676002)(44832011)(6666004)(2906002)(2616005)(40480700001)(81166007)(82310400005)(82740400003)(16526019)(356005)(41300700001)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 May 2023 02:56:23.9216 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a6577c5e-2c6f-4527-73ec-08db55b9228c X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT112.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4249 X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO, RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net X-Patchwork-Delegate: kuba@kernel.org Add the documentation and Kconfig entry for pds_vdpa driver. Signed-off-by: Shannon Nelson Acked-by: Jason Wang --- .../device_drivers/ethernet/amd/pds_vdpa.rst | 85 +++++++++++++++++++ .../device_drivers/ethernet/index.rst | 1 + MAINTAINERS | 4 + drivers/vdpa/Kconfig | 10 +++ 4 files changed, 100 insertions(+) create mode 100644 Documentation/networking/device_drivers/ethernet/amd/pds_vdpa.rst diff --git a/Documentation/networking/device_drivers/ethernet/amd/pds_vdpa.rst b/Documentation/networking/device_drivers/ethernet/amd/pds_vdpa.rst new file mode 100644 index 000000000000..587927d3de92 --- /dev/null +++ b/Documentation/networking/device_drivers/ethernet/amd/pds_vdpa.rst @@ -0,0 +1,85 @@ +.. SPDX-License-Identifier: GPL-2.0+ +.. note: can be edited and viewed with /usr/bin/formiko-vim + +========================================================== +PCI vDPA driver for the AMD/Pensando(R) DSC adapter family +========================================================== + +AMD/Pensando vDPA VF Device Driver + +Copyright(c) 2023 Advanced Micro Devices, Inc + +Overview +======== + +The ``pds_vdpa`` driver is an auxiliary bus driver that supplies +a vDPA device for use by the virtio network stack. It is used with +the Pensando Virtual Function devices that offer vDPA and virtio queue +services. It depends on the ``pds_core`` driver and hardware for the PF +and VF PCI handling as well as for device configuration services. + +Using the device +================ + +The ``pds_vdpa`` device is enabled via multiple configuration steps and +depends on the ``pds_core`` driver to create and enable SR-IOV Virtual +Function devices. After the VFs are enabled, we enable the vDPA service +in the ``pds_core`` device to create the auxiliary devices used by pds_vdpa. + +Example steps: + +.. code-block:: bash + + #!/bin/bash + + modprobe pds_core + modprobe vdpa + modprobe pds_vdpa + + PF_BDF=`ls /sys/module/pds_core/drivers/pci\:pds_core/*/sriov_numvfs | awk -F / '{print $7}'` + + # Enable vDPA VF auxiliary device(s) in the PF + devlink dev param set pci/$PF_BDF name enable_vnet cmode runtime value true + + # Create a VF for vDPA use + echo 1 > /sys/bus/pci/drivers/pds_core/$PF_BDF/sriov_numvfs + + # Find the vDPA services/devices available + PDS_VDPA_MGMT=`vdpa mgmtdev show | grep vDPA | head -1 | cut -d: -f1` + + # Create a vDPA device for use in virtio network configurations + vdpa dev add name vdpa1 mgmtdev $PDS_VDPA_MGMT mac 00:11:22:33:44:55 + + # Set up an ethernet interface on the vdpa device + modprobe virtio_vdpa + + + +Enabling the driver +=================== + +The driver is enabled via the standard kernel configuration system, +using the make command:: + + make oldconfig/menuconfig/etc. + +The driver is located in the menu structure at: + + -> Device Drivers + -> Network device support (NETDEVICES [=y]) + -> Ethernet driver support + -> Pensando devices + -> Pensando Ethernet PDS_VDPA Support + +Support +======= + +For general Linux networking support, please use the netdev mailing +list, which is monitored by Pensando personnel:: + + netdev@vger.kernel.org + +For more specific support needs, please use the Pensando driver support +email:: + + drivers@pensando.io diff --git a/Documentation/networking/device_drivers/ethernet/index.rst b/Documentation/networking/device_drivers/ethernet/index.rst index 417ca514a4d0..94ecb67c0885 100644 --- a/Documentation/networking/device_drivers/ethernet/index.rst +++ b/Documentation/networking/device_drivers/ethernet/index.rst @@ -15,6 +15,7 @@ Contents: amazon/ena altera/altera_tse amd/pds_core + amd/pds_vdpa aquantia/atlantic chelsio/cxgb cirrus/cs89x0 diff --git a/MAINTAINERS b/MAINTAINERS index e2fd64c2ebdc..c3f509eeaf1d 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -22296,6 +22296,10 @@ F: include/linux/vringh.h F: include/uapi/linux/virtio_*.h F: tools/virtio/ +PDS DSC VIRTIO DATA PATH ACCELERATOR +R: Shannon Nelson +F: drivers/vdpa/pds/ + VIRTIO CRYPTO DRIVER M: Gonglei L: virtualization@lists.linux-foundation.org diff --git a/drivers/vdpa/Kconfig b/drivers/vdpa/Kconfig index cd6ad92f3f05..656c1cb541de 100644 --- a/drivers/vdpa/Kconfig +++ b/drivers/vdpa/Kconfig @@ -116,4 +116,14 @@ config ALIBABA_ENI_VDPA This driver includes a HW monitor device that reads health values from the DPU. +config PDS_VDPA + tristate "vDPA driver for AMD/Pensando DSC devices" + select VIRTIO_PCI_LIB + depends on PCI_MSI + depends on PDS_CORE + help + vDPA network driver for AMD/Pensando's PDS Core devices. + With this driver, the VirtIO dataplane can be + offloaded to an AMD/Pensando DSC device. + endif # VDPA