From patchwork Fri Feb 23 12:42:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Ankit Agrawal X-Patchwork-Id: 13569014 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0EE68C54798 for ; Fri, 23 Feb 2024 12:44:09 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rdUtt-0008V5-Na; Fri, 23 Feb 2024 07:43:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rdUtF-00080Q-15; Fri, 23 Feb 2024 07:42:57 -0500 Received: from mail-sn1nam02on2060f.outbound.protection.outlook.com ([2a01:111:f400:7ea9::60f] helo=NAM02-SN1-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rdUtB-0003lD-JD; Fri, 23 Feb 2024 07:42:56 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gGm5+IjFHST+gzuvSfxXouEIcmouqGDj/NK5/SJovpSsOXPBHxV72xsRaLDiuahzYTaccdowfM/AMoviOJsu/3yxsp8ZtnEelqujvm1FLwPw7DwSCTYJLjYsLXnBg+qqRE709EgDlp5NDuPk/1nXUzgqdoWUPYc9A8jK/MP9J4yoVpzvQfv7zbbcwlVw341ysLO69W63X5COU/AYLXfcEUYRTMxoGyCcJvuPFUMPf5WNfbjRGtT/7D/+QvxxtOFePtX4IyNlTZ+KuBazIeM08OFWw4hD8AdMM+z6m1Ty0/HzXTUHJeUwpDenr8MvNl8xbzwfejcjJL/mPGajQgrzpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=E0Uv2UPcA0Zmh1mLr1W65xS7uLDek7j4sFneRtYYlLU=; b=KSo7zsxo0EEo47D+nkPdRr/wryGsA9zjet85Y7ixaDWDZeCOsrLDTUUIMjBsTl6xxvaKMu8ympmigqBfuMBmF9JjMplfGCwi0gRY5KRLCsqeozscY/kLuKrtFq4pImcrCFo4hR1gqyXp5dXBVMN0lT/9tYt/ryl3SAnJediwQxP548ZDwPyMv1IZ/rYKzDS02xK8q3D4f7M78461I4tGxlwYUgryRqGwaflW2/S+OhwCl8o5rE21AyjSXdYUrbRpOMeTGNWeXUPPV5Vlkx3u0dnlYmHrt14pMj5OWkl1GnpVXadWrhFxeJFLAwmUYMdFLXomadQ44w0H97YvRJ+cjA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=E0Uv2UPcA0Zmh1mLr1W65xS7uLDek7j4sFneRtYYlLU=; b=fNCKv6FDPjRqH++xVjH2UmAUDUmlOeHZ3c8bIkZ562YHYoTc7cYuZyxasV/U225steHa5x/zrKoZ8ghHEcIMDpd9FrfWmp4GlOdZtRJSOgu+DPjDqqIY2I3Yy2YzvYU8Y7C4q8rdV7CBMA2WSD6+NZVi9Xa+i6k9/FSVXYScueUeVuRno+pRJQvf+cMYS2lI26byBuUzyDocdNXuZ11gSBTUmsLBw8q1WE/nMBZMoTPM9QvzjUbWm0uHqtrajNF6vzoYLY/dw9jRk68nQxuX+zp1s0oYK5gv6auau4eTLPsCXDIE/T86PIV/nmWf+jRJ08KDVy287KdXMcQVxcxXLg== Received: from SJ0PR13CA0174.namprd13.prod.outlook.com (2603:10b6:a03:2c7::29) by DS0PR12MB7581.namprd12.prod.outlook.com (2603:10b6:8:13d::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.22; Fri, 23 Feb 2024 12:42:48 +0000 Received: from MWH0EPF000989E5.namprd02.prod.outlook.com (2603:10b6:a03:2c7:cafe::19) by SJ0PR13CA0174.outlook.office365.com (2603:10b6:a03:2c7::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7316.11 via Frontend Transport; Fri, 23 Feb 2024 12:42:47 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by MWH0EPF000989E5.mail.protection.outlook.com (10.167.241.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7292.25 via Frontend Transport; Fri, 23 Feb 2024 12:42:47 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Fri, 23 Feb 2024 04:42:27 -0800 Received: from rnnvmail204.nvidia.com (10.129.68.6) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.12; Fri, 23 Feb 2024 04:42:26 -0800 Received: from localhost.nvc.nvidia.com (10.127.8.10) by mail.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1258.12 via Frontend Transport; Fri, 23 Feb 2024 04:42:25 -0800 From: To: , , , , , , , , , , , , , , , , , , CC: , , , , , , , , , Subject: [PATCH v7 2/2] hw/acpi: Implement the SRAT GI affinity structure Date: Fri, 23 Feb 2024 12:42:23 +0000 Message-ID: <20240223124223.800078-3-ankita@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240223124223.800078-1-ankita@nvidia.com> References: <20240223124223.800078-1-ankita@nvidia.com> MIME-Version: 1.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000989E5:EE_|DS0PR12MB7581:EE_ X-MS-Office365-Filtering-Correlation-Id: 86ceafd2-a030-42ba-2d1c-08dc346cf0af X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: pRPavDnV8x+5GdsKRauyKnvsdR9LlMkaixb5Z+qfGVhdo8D1ZHavGHqXYN5jXbkL5GMP4U9lSArWhKmpMNRJpKICLD/3d521Xhp2g0a0quyBleAlM7YTZlQLXbsqb2fNBBR7ht1g9g2+Wp9ScGyuJoWlQ72Hacs4ihpWSUr4XajxnVtd0Hr3qPFoouSZFD6WpkJs8dp1l2ndeUEd+zxl9szWPY4QiPRWqcKj+6NRtP2lIeEw7q8wU+BNi86SCLDpowxtL68HI2hJih3x5DUu7x+z0XrwRc7/tmOpFdvTWMTxPkhT60IkeFp6p910gWgIQweJPQkfkH4iJ15+A/lB7K6wmq3+AvyXSdFpjhN4lpm4653HRU6S0ZYmdT2CuwCxQwmqsrf+R1SB9tkU17wCCJ10DILp4qFOkD6KxHhEJgGay6J1h5vk8NoLX34ooZ7RgIo3okvy1GouyeJD0Vn5p4w5dH72IfrnsqKlmi77f4Rg3U0mCh/VEVtLtM/1J/Y7Ehaj6UQgkqNGRTmpACv2NFj36+jFRz26ijRkaSoLaWEauN5z/w4Nthbe5sRhyJqGEb81hyD9rqvJRYSftEj9DR1rsnEn/oWLRI5vpGkkWLpdqpm7mZMX7LTKcwPzG4Wi2NyokcRVr686LxjHgNVHpSIE/j11BGbPQ0RWs08AD0Q= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230031)(36860700004)(46966006)(40470700004)(921011); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2024 12:42:47.7486 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 86ceafd2-a030-42ba-2d1c-08dc346cf0af X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000989E5.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7581 Received-SPF: softfail client-ip=2a01:111:f400:7ea9::60f; envelope-from=ankita@nvidia.com; helo=NAM02-SN1-obe.outbound.protection.outlook.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.066, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org From: Ankit Agrawal ACPI spec provides a scheme to associate "Generic Initiators" [1] (e.g. heterogeneous processors and accelerators, GPUs, and I/O devices with integrated compute or DMA engines GPUs) with Proximity Domains. This is achieved using Generic Initiator Affinity Structure in SRAT. During bootup, Linux kernel parse the ACPI SRAT to determine the PXM ids and create a NUMA node for each unique PXM ID encountered. Qemu currently do not implement these structures while building SRAT. Add GI structures while building VM ACPI SRAT. The association between device and node are stored using acpi-generic-initiator object. Lookup presence of all such objects and use them to build these structures. The structure needs a PCI device handle [2] that consists of the device BDF. The vfio-pci device corresponding to the acpi-generic-initiator object is located to determine the BDF. [1] ACPI Spec 6.3, Section 5.2.16.6 [2] ACPI Spec 6.3, Table 5.80 Signed-off-by: Ankit Agrawal --- hw/acpi/acpi-generic-initiator.c | 84 ++++++++++++++++++++++++ hw/arm/virt-acpi-build.c | 3 + include/hw/acpi/acpi-generic-initiator.h | 26 ++++++++ 3 files changed, 113 insertions(+) diff --git a/hw/acpi/acpi-generic-initiator.c b/hw/acpi/acpi-generic-initiator.c index 1ade2f723f..d78382bc63 100644 --- a/hw/acpi/acpi-generic-initiator.c +++ b/hw/acpi/acpi-generic-initiator.c @@ -68,3 +68,87 @@ static void acpi_generic_initiator_class_init(ObjectClass *oc, void *data) object_class_property_add(oc, "node", "int", NULL, acpi_generic_initiator_set_node, NULL, NULL); } + +static int acpi_generic_initiator_list(Object *obj, void *opaque) +{ + GSList **list = opaque; + + if (object_dynamic_cast(obj, TYPE_ACPI_GENERIC_INITIATOR)) { + *list = g_slist_append(*list, ACPI_GENERIC_INITIATOR(obj)); + } + + object_child_foreach(obj, acpi_generic_initiator_list, opaque); + return 0; +} + +/* + * Identify Generic Initiator objects and link them into the list which is + * returned to the caller. + * + * Note: it is the caller's responsibility to free the list to avoid + * memory leak. + */ +static GSList *acpi_generic_initiator_get_list(void) +{ + GSList *list = NULL; + + object_child_foreach(object_get_root(), + acpi_generic_initiator_list, &list); + return list; +} + +/* + * ACPI 6.3: + * Table 5-78 Generic Initiator Affinity Structure + */ +static void +build_srat_generic_pci_initiator_affinity(GArray *table_data, int node, + PCIDeviceHandle *handle) +{ + uint8_t index; + + build_append_int_noprefix(table_data, 5, 1); /* Type */ + build_append_int_noprefix(table_data, 32, 1); /* Length */ + build_append_int_noprefix(table_data, 0, 1); /* Reserved */ + build_append_int_noprefix(table_data, 1, 1); /* Device Handle Type: PCI */ + build_append_int_noprefix(table_data, node, 4); /* Proximity Domain */ + + /* Device Handle - PCI */ + build_append_int_noprefix(table_data, handle->segment, 2); + build_append_int_noprefix(table_data, handle->bdf, 2); + for (index = 0; index < 12; index++) { + build_append_int_noprefix(table_data, 0, 1); + } + + build_append_int_noprefix(table_data, GEN_AFFINITY_ENABLED, 4); /* Flags */ + build_append_int_noprefix(table_data, 0, 4); /* Reserved */ +} + +void build_srat_generic_pci_initiator(GArray *table_data) +{ + GSList *gi_list, *list = acpi_generic_initiator_get_list(); + AcpiGenericInitiator *gi; + + for (gi_list = list; gi_list; gi_list = gi_list->next) { + PCIDeviceHandle dev_handle; + PCIDevice *pci_dev; + Object *o; + + gi = gi_list->data; + + o = object_resolve_path_type(gi->pci_dev, TYPE_PCI_DEVICE, NULL); + if (!o) { + error_printf("Specified device must be a PCI device.\n"); + exit(1); + } + pci_dev = PCI_DEVICE(o); + + dev_handle.segment = 0; + dev_handle.bdf = PCI_BUILD_BDF(pci_bus_num(pci_get_bus(pci_dev)), + pci_dev->devfn); + build_srat_generic_pci_initiator_affinity(table_data, + gi->node, &dev_handle); + } + + g_slist_free(list); +} diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index 8bc35a483c..00d77327e0 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -58,6 +58,7 @@ #include "migration/vmstate.h" #include "hw/acpi/ghes.h" #include "hw/acpi/viot.h" +#include "hw/acpi/acpi-generic-initiator.h" #define ARM_SPI_BASE 32 @@ -558,6 +559,8 @@ build_srat(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms) } } + build_srat_generic_pci_initiator(table_data); + if (ms->nvdimms_state->is_enabled) { nvdimm_build_srat(table_data); } diff --git a/include/hw/acpi/acpi-generic-initiator.h b/include/hw/acpi/acpi-generic-initiator.h index 2f183b029a..213545e614 100644 --- a/include/hw/acpi/acpi-generic-initiator.h +++ b/include/hw/acpi/acpi-generic-initiator.h @@ -29,4 +29,30 @@ typedef struct AcpiGenericInitiatorClass { ObjectClass parent_class; } AcpiGenericInitiatorClass; +/* + * ACPI 6.3: + * Table 5-81 Flags – Generic Initiator Affinity Structure + */ +typedef enum { + GEN_AFFINITY_ENABLED = (1 << 0), /* + * If clear, the OSPM ignores the contents + * of the Generic Initiator/Port Affinity + * Structure. This allows system firmware + * to populate the SRAT with a static + * number of structures, but only enable + * them as necessary. + */ +} GenericAffinityFlags; + +/* + * ACPI 6.3: + * Table 5-80 Device Handle - PCI + */ +typedef struct PCIDeviceHandle { + uint16_t segment; + uint16_t bdf; +} PCIDeviceHandle; + +void build_srat_generic_pci_initiator(GArray *table_data); + #endif