From patchwork Thu Apr 6 23:41:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Nelson, Shannon" X-Patchwork-Id: 13204320 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 828CAC761A6 for ; Thu, 6 Apr 2023 23:42:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236901AbjDFXmX (ORCPT ); Thu, 6 Apr 2023 19:42:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44442 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232254AbjDFXmW (ORCPT ); Thu, 6 Apr 2023 19:42:22 -0400 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2062.outbound.protection.outlook.com [40.107.220.62]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4626C93EB for ; Thu, 6 Apr 2023 16:42:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=j/xXNcfq0knCUKr1YmeDwuSRI8YpQyZc2YmTOJKs7SNhSchO3wEmtBxAMpalIK1ufVjWoYAywhpfqtBUhinsnnTPk9ldVMD4ofyI9duHnhszWm9Oe+w5usmTF6IA7SOd0sWrW4Y99qBT/pYW6kVryzhZ7luz3KXaz57r9ofqhyZl/lP2kUArET4xBLUw0zVgyDGzFEcKCxrHr3ijumF+b2E+EiIxKOAEI+bi7wAaHnRQXsICVEDk217QhWqyYP4eNtDg7CS1JVWU5oq5VYqo6CXDcT9jbr+aIQ5upRoHQORX4iXX5V5WH9MK+LVVvVrb5UisWCx97tYntRu6RMXKJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=z/M0TDvBZEw7r1xo3jcOXotw3yVGVS1Ip1JuQMn6RqA=; b=UE4eYeulwCuV0KN7tuZ0yCvstcXKX1/XAwJBcpl5cm6NNNVthE3vPr80B8O9xSPBk+mdqF4ag9kkkwsBVm82mxNXm+VhQlnVGL0oyflApd8ga15Lv9SL1bXQupB9eqT9gRFLBEeDq0BGdBnf9jZvqHJwmqLyD0Ku+CbgLIlppwGt7pVcTohNYa4Fq6V/5VPENipsHEp3SswWIHyQnSofnkgFWUyABSS/JidQjTR+wBGBTWELJeDoJz8e4s+Xu1Y3kYMBjqAWcrI5VDuGwxA6IX2rN0uSHcOIqDaXae9cTXyc0HopZBiSs8CPdJEeFbHxVbMN1NbRl7OssNLI6Yqgew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=davemloft.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=z/M0TDvBZEw7r1xo3jcOXotw3yVGVS1Ip1JuQMn6RqA=; b=zG4xINkDirIlVwV8BYTA4QaQDTFOrQvGoWhgQWJX/lVlB2vFIhJXD4IJvr9HxxyYIIuvc//Zy1+hYwww36lX6goNLYWwnBx0pR1mw6HYpHTPMbcIo9zR7WBzXF2EVRa/zLYIgKEtSHVwD9g4whtjtTJQ03k9oEl/+Rd9T1+X5uE= Received: from DM6PR02CA0088.namprd02.prod.outlook.com (2603:10b6:5:1f4::29) by DM6PR12MB4943.namprd12.prod.outlook.com (2603:10b6:5:1bc::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.35; Thu, 6 Apr 2023 23:42:17 +0000 Received: from DM6NAM11FT087.eop-nam11.prod.protection.outlook.com (2603:10b6:5:1f4:cafe::39) by DM6PR02CA0088.outlook.office365.com (2603:10b6:5:1f4::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.31 via Frontend Transport; Thu, 6 Apr 2023 23:42:17 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT087.mail.protection.outlook.com (10.13.172.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6277.31 via Frontend Transport; Thu, 6 Apr 2023 23:42:16 +0000 Received: from driver-dev1.pensando.io (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Thu, 6 Apr 2023 18:42:15 -0500 From: Shannon Nelson To: , , , , CC: , , Subject: [PATCH v9 net-next 00/14] pds_core driver Date: Thu, 6 Apr 2023 16:41:29 -0700 Message-ID: <20230406234143.11318-1-shannon.nelson@amd.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT087:EE_|DM6PR12MB4943:EE_ X-MS-Office365-Filtering-Correlation-Id: be61219f-8cb5-4290-75ba-08db36f88e34 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6XKZN6CIsefcXKzDi8lgtRFAxXftV5vQ+BME3pl5rI6aewRiFROL/cHkA0PRBsv/SGkCIsgQb7zehx8IaALOaTa3pHVyiO4cgaNxlf7kpNNGw2YlifxHYmtxRoFRhggFNYD8x3tthS19Q+6ZHLOK5JGsl44mYaAPPxMT1OfDnaSxj6SuTzw884b6KL8OEmwbqtvFftNTNbF43F2eAiHGy4UI8lqbIWeNW5xqi0Lo4cVKky0O87LVLJ98gze9/S1A8uXa8Ck3WfjcreIUiCxuPcVXhG487Qz1D3BPBDkDaMZmRaOweFfZXEDasdmcj5HE9ZJrJVvhdxtX8L5AqLL8V3Y5440dNb8hWKIb1TSDX7YmEDGxFI/225tyEwTPxao3163rwd7EAOmrV+tUH00gTSw4zKGCw6l1Ubd4BNG7taKNMRXdgF54xeG2fL3xxGjtDL0i3na4snvmP/d9kbMYJODVeAfx38yv5nsr+SRZi0iTATXpxUXbzug/7jgL4Q+i5rcxWPvdg2k27sVbbWkyRIwsbiGJaqsK69/Gk8Crfz/KfA4gMtiSnMZg45lrLfYlz0hY1hDcGS56/dPB6DaGJfIo5JjoPMUzHLxuROZkDOz7HoVl9od/9kPJfqQwuogOl4J+DkghAIp/Bui/KuVQ378mQVuNg3l4XFAOHG87S1/NqX/PcjTkOqjch/xYqd+APJNp9pry0GLcjza8zMwuB1l5WB7fRfXSLWw3GKt1Y9m/TpUjzAsdsPrQ7KiNH5Pc2luPgo/gzbipSkM0j9mQgg== X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230028)(4636009)(39860400002)(376002)(396003)(346002)(136003)(451199021)(36840700001)(40470700004)(46966006)(478600001)(316002)(110136005)(70206006)(8676002)(70586007)(54906003)(8936002)(426003)(1076003)(26005)(336012)(16526019)(2616005)(83380400001)(47076005)(186003)(36756003)(82310400005)(966005)(6666004)(40460700003)(356005)(4326008)(36860700001)(86362001)(44832011)(5660300002)(41300700001)(40480700001)(81166007)(2906002)(82740400003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Apr 2023 23:42:16.7867 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: be61219f-8cb5-4290-75ba-08db36f88e34 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT087.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4943 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org Summary: -------- This patchset implements a new driver for use with the AMD/Pensando Distributed Services Card (DSC), intended to provide core configuration services through the auxiliary_bus and through a couple of EXPORTed functions for use initially in VFio and vDPA feature specific drivers. To keep this patchset to a manageable size, the pds_vdpa and pds_vfio drivers have been split out into their own patchsets to be reviewed separately. Detail: ------- AMD/Pensando is making available a new set of devices for supporting vDPA, VFio, and potentially other features in the Distributed Services Card (DSC). These features are implemented through a PF that serves as a Core device for controlling and configuring its VF devices. These VF devices have separate drivers that use the auxiliary_bus to work through the Core device as the control path. Currently, the DSC supports standard ethernet operations using the ionic driver. This is not replaced by the Core-based devices - these new devices are in addition to the existing Ethernet device. Typical DSC configurations will include both PDS devices and Ionic Eth devices. However, there is a potential future path for ethernet services to come through this device as well. The Core device is a new PCI PF/VF device managed by a new driver 'pds_core'. The PF device has access to an admin queue for configuring the services used by the VFs, and sets up auxiliary_bus devices for each vDPA VF for communicating with the drivers for the vDPA devices. The VFs may be for VFio or vDPA, and other services in the future; these VF types are selected as part of the DSC internal FW configurations, which is out of the scope of this patchset. When the vDPA support set is enabled in the core PF through its devlink param, auxiliary_bus devices are created for each VF that supports the feature. The vDPA driver then connects to and uses this auxiliary_device to do control path configuration through the PF device. This can then be used with the vdpa kernel module to provide devices for virtio_vdpa kernel module for host interfaces, or vhost_vdpa kernel module for interfaces exported into your favorite VM. A cheap ASCII diagram of a vDPA instance looks something like this: ,----------. | vdpa | '----------' | || ctl data | || .----------. || | pds_vdpa | || '----------' || | || pds_core.vDPA.1 || | || .---------------. || | pds_core | || '---------------' || || || || 09:00.0 09:00.1 == PCI ============================================ || || .----------. .----------. ,------| PF |---| VF |-------, | '----------' '----------' | | DSC | | | ------------------------------------------ Changes: v9: - change pdsc field name id to uid to clarify the unique id used for aux device - remove unnecessary pf->state and other checks in aux device creation - hardcode fw slotnames for devlink info, don't use strings from FW - handle errors from PDS_CORE_CMD_INIT devcmd call - tighten up health thread use of config_lock - remove pdsc_queue_health_check() layer over queuing health check - start pds_core.rst file in first patch, add to it incrementally - give more user interaction info in commit messages - removed a few more extraneous includes v8: Link: https://lore.kernel.org/netdev/20230330234628.14627-1-shannon.nelson@amd.com/ - fixed deadlock problem, use devl_health_reporter_destroy() when devlink is locked - don't clear client_id until after auxiliary_device_uninit() v7: Link: https://lore.kernel.org/netdev/20230330192313.62018-1-shannon.nelson@amd.com/ - use explicit devlink locking and devl_* APIs - move some of devlink setup logic into probe and remove - use debugfs_create_u{type}() for state and queue head and tail - add include for linux/vmalloc.h Reported-by: kernel test robot Link: https://lore.kernel.org/oe-kbuild-all/202303260420.Tgq0qobF-lkp@intel.com/ v6: Link: https://lore.kernel.org/netdev/20230324190243.27722-1-shannon.nelson@amd.com/ - removed version.h include noticed by kernel test robot's version check Reported-by: kernel test robot Link: https://lore.kernel.org/oe-kbuild-all/202303230742.pX3ply0t-lkp@intel.com/ - fixed up the more egregious checkpatch line length complaints - make sure pdsc_auxbus_dev_register() checks padev pointer errcode v5: Link: https://lore.kernel.org/netdev/20230322185626.38758-1-shannon.nelson@amd.com/ - added devlink health reporter for FW issues - removed asic_type, asic_rev, serial_num, fw_version from debugfs as they are available through other means - trimed OS info in pdsc_identify(), we don't need to send that much info to the FW - removed reg/unreg from auxbus client API, they are now in the core when VF is started - removed need for pdsc definition in client by simplifying the padev to only carry struct pci_dev pointers rather than full struct pdsc to the pf and vf - removed the unused pdsc argument in pdsc_notify() - moved include/linux/pds/pds_core.h to driver/../pds_core/core.h - restored a few pds_core_if.h interface values and structs that are shared with FW source - moved final config_lock unlock to before tear down of timer and workqueue to be sure there are no deadlocks while waiting for any stragglers - changed use of PAGE_SIZE to local PDS_PAGE_SIZE to keep with FW layout needs without regard to kernel PAGE_SIZE configuration - removed the redundant *adminqcq argument from pdsc_adminq_post() v4: Link: https://lore.kernel.org/netdev/20230308051310.12544-1-shannon.nelson@amd.com/ - reworked to attach to both Core PF and vDPA VF PCI devices - now creates auxiliary_device as part of each VF PCI probe, removes them on PCI remove - auxiliary devices now use simple unique id rather than PCI address for identifier - replaced home-grown event publishing with kernel-based notifier service - dropped live_migration parameter, not needed when not creating aux device for it - replaced devm_* functions with traditional interfaces - added MAINTAINERS entry - removed lingering traces of set/get_vf attribute adminq commands - trimmed some include lists - cleaned a kernel test robot complaint about a stray unused variable Link: https://lore.kernel.org/oe-kbuild-all/202302181049.yeUQMeWY-lkp@intel.com/ v3: Link: https://lore.kernel.org/netdev/20230217225558.19837-1-shannon.nelson@amd.com/ - changed names from "pensando" to "amd" and updated copyright strings - dropped the DEVLINK_PARAM_GENERIC_ID_FW_BANK for future development - changed the auxiliary device creation to be triggered by the PCI bus event BOUND_DRIVER, and torn down at UNBIND_DRIVER in order to properly handle users using the sysfs bind/unbind functions - dropped some noisy log messages - rebased to current net-next RFC to v2: Link: https://lore.kernel.org/netdev/20221207004443.33779-1-shannon.nelson@amd.com/ - added separate devlink param patches for DEVLINK_PARAM_GENERIC_ID_ENABLE_MIGRATION and DEVLINK_PARAM_GENERIC_ID_FW_BANK, and dropped the driver specific implementations - updated descriptions for the new devlink parameters - dropped netdev support - dropped vDPA patches, will followup later - separated fw update and fw bank select into their own patches RFC: Link: https://lore.kernel.org/netdev/20221118225656.48309-1-snelson@pensando.io/ Shannon Nelson (14): pds_core: initial framework for pds_core PF driver pds_core: add devcmd device interfaces pds_core: health timer and workqueue pds_core: add devlink health facilities pds_core: set up device and adminq pds_core: Add adminq processing and commands pds_core: add FW update feature to devlink pds_core: set up the VIF definitions and defaults pds_core: add initial VF device handling pds_core: add auxiliary_bus devices pds_core: devlink params for enabling VIF support pds_core: add the aux client API pds_core: publish events to the clients pds_core: Kconfig and pds_core.rst .../device_drivers/ethernet/amd/pds_core.rst | 144 ++++ .../device_drivers/ethernet/index.rst | 1 + MAINTAINERS | 9 + drivers/net/ethernet/amd/Kconfig | 12 + drivers/net/ethernet/amd/Makefile | 1 + drivers/net/ethernet/amd/pds_core/Makefile | 14 + drivers/net/ethernet/amd/pds_core/adminq.c | 291 ++++++++ drivers/net/ethernet/amd/pds_core/auxbus.c | 241 +++++++ drivers/net/ethernet/amd/pds_core/core.c | 616 +++++++++++++++++ drivers/net/ethernet/amd/pds_core/core.h | 325 +++++++++ drivers/net/ethernet/amd/pds_core/debugfs.c | 202 ++++++ drivers/net/ethernet/amd/pds_core/dev.c | 352 ++++++++++ drivers/net/ethernet/amd/pds_core/devlink.c | 180 +++++ drivers/net/ethernet/amd/pds_core/fw.c | 194 ++++++ drivers/net/ethernet/amd/pds_core/main.c | 489 +++++++++++++ include/linux/pds/pds_adminq.h | 645 ++++++++++++++++++ include/linux/pds/pds_auxbus.h | 44 ++ include/linux/pds/pds_common.h | 97 +++ include/linux/pds/pds_core_if.h | 540 +++++++++++++++ include/linux/pds/pds_intr.h | 163 +++++ 20 files changed, 4560 insertions(+) create mode 100644 Documentation/networking/device_drivers/ethernet/amd/pds_core.rst create mode 100644 drivers/net/ethernet/amd/pds_core/Makefile create mode 100644 drivers/net/ethernet/amd/pds_core/adminq.c create mode 100644 drivers/net/ethernet/amd/pds_core/auxbus.c create mode 100644 drivers/net/ethernet/amd/pds_core/core.c create mode 100644 drivers/net/ethernet/amd/pds_core/core.h create mode 100644 drivers/net/ethernet/amd/pds_core/debugfs.c create mode 100644 drivers/net/ethernet/amd/pds_core/dev.c create mode 100644 drivers/net/ethernet/amd/pds_core/devlink.c create mode 100644 drivers/net/ethernet/amd/pds_core/fw.c create mode 100644 drivers/net/ethernet/amd/pds_core/main.c create mode 100644 include/linux/pds/pds_adminq.h create mode 100644 include/linux/pds/pds_auxbus.h create mode 100644 include/linux/pds/pds_common.h create mode 100644 include/linux/pds/pds_core_if.h create mode 100644 include/linux/pds/pds_intr.h Acked-by: Jakub Kicinski