From patchwork Sat Dec 7 10:55:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhang, Julia" X-Patchwork-Id: 13898253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0ED38E7717B for ; Sat, 7 Dec 2024 10:58:31 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tJsVm-000626-Fx; Sat, 07 Dec 2024 05:58:10 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tJsVY-0005yY-SN for qemu-devel@nongnu.org; Sat, 07 Dec 2024 05:58:02 -0500 Received: from mail-mw2nam12on2062c.outbound.protection.outlook.com ([2a01:111:f403:200a::62c] helo=NAM12-MW2-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tJsVW-0007HX-P0 for qemu-devel@nongnu.org; Sat, 07 Dec 2024 05:57:56 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=DdDe97pi0tc8ZHZA4Tp6eA1fulBtz6474uH156UNLqzXOXYNDdmZCfco/pJTD6U3HwBUgty6a4/nuNhBtA3bfmFobXdz3xPs542HnZKPLgJU6tX9X2huknnvP8tNVFQqbP8meCdFpkZbEdPujDVpVjuTbVq6ZIyJLOeh+zpGrxEZ54jXg5O7FJRwl7xDDibxcKNofjChG3hq9Lc1b625yEErR/jkzeRMHyqjoFEySGEeLK3BGjl57xzaKf4OVY2KHC1zERTjuuYPBgSL6/s9OgUmC5BTo2bl/Kj9yJ633+5NOz3AM/zCbU2dZ9Epr5GJVpky6hvxNxMYQEnCDRLGmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=QB2v3p8xVBkoFTtj/OssK3eq3aeYBld67hSPE7XCRYE=; b=VrJdvEqSaFXdWuNz6YRJEcWBqb7loTgEJbBVP6387HHAqK57q16/kuoDFRxFfs/VwIQJ8Bpf1kVzh+0GO9AlZkZMUKGQvB6qupwVidBWpn+rXADBe4qHOheQrQvkaCJr+I9joePVkaCgeVczVgwd6xrbtd26dl31UAkkKxfTQyLRHNCtWPRW/4XqOPOqbJvPYZ6uBhVeKmFyn+jove30o9RRwlYF2cYaYqiVj2wRqcBw9wt8hysxgp/8uB+omWrpdo2+nVqQPnn426l/uU+nuFNbDTWXNyJZDrxV7wRiKm0iaZHQSlqM6SCOBZdtsBF7sBfKhFn/c2LDw9zjHjn5Ew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=QB2v3p8xVBkoFTtj/OssK3eq3aeYBld67hSPE7XCRYE=; b=bxMe+UBrFAPVtVlkMpBC5s3RhhrgLFmT+xrtuHSRWI2cTJIPlzB5iTjeQ0LJrT9PJbjShJ7Mi24oRfojYpq1rPmSb1yy8xAukdcFegY0/fw9pS0LOtG311cGDWCOsz+6AdZnlMqv32l8FC6v5CFQXC0UwvR3KlyTcjT5H6cC+v8= Received: from MN2PR19CA0065.namprd19.prod.outlook.com (2603:10b6:208:19b::42) by DM4PR12MB7501.namprd12.prod.outlook.com (2603:10b6:8:113::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8207.20; Sat, 7 Dec 2024 10:57:45 +0000 Received: from BN3PEPF0000B371.namprd21.prod.outlook.com (2603:10b6:208:19b:cafe::20) by MN2PR19CA0065.outlook.office365.com (2603:10b6:208:19b::42) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8207.16 via Frontend Transport; Sat, 7 Dec 2024 10:57:44 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN3PEPF0000B371.mail.protection.outlook.com (10.167.243.168) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8272.0 via Frontend Transport; Sat, 7 Dec 2024 10:57:44 +0000 Received: from jenkins-julia.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Sat, 7 Dec 2024 04:57:40 -0600 From: Julia Zhang To: Stefano Stabellini , Anthony PERARD , Paul Durrant , "Edgar E . Iglesias" , "Michael S . Tsirkin" , Marcel Apfelbaum , CC: Alex Deucher , =?utf-8?q?Christian_K=C3=B6nig?= , "Xenia Ragiadakou" , Julia Zhang , "Chen Jiqian" , Huang Rui , Penny Zheng , Zhu Lingshan , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Jan Beulich , Juergen Gross , Oleksandr Tyshchenko , Subject: [PATCH 3/3] virtio-gpu: add a new command to get p2pdma_distance Date: Sat, 7 Dec 2024 18:55:40 +0800 Message-ID: <20241207105537.542441-4-julia.zhang@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241207105537.542441-1-julia.zhang@amd.com> References: <20241207105537.542441-1-julia.zhang@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN3PEPF0000B371:EE_|DM4PR12MB7501:EE_ X-MS-Office365-Filtering-Correlation-Id: de85b411-cd99-41cf-2525-08dd16adfac7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|36860700013|376014|7416014|82310400026|1800799024; X-Microsoft-Antispam-Message-Info: +qa/yUL1d3QdPp0XvV+rfvpR0waCVpnNrP2Z6qcNRnuryjjeNRBJEERL51ujweuuOZV0jJk4gAOVI+FF9DpMav+r8yJ+jTfT7TQw2/SuNtn9p7QYq8/yLHvmgNgeUCpn/3pKsRlZZUf6AaVmaM7LR9lRITZchEKI4X64sfTT+dFwgFxymw59O0rpDGUzZEVn0nDQ22n8DfAMbM8sr2dntAAImKsn7t3B6aHXjGuwRs+ikRn6MOOjgV7CHgr8eF3YHZaMpsJPItEqNjcdl3vq9X7Q42/uQkVK6aSjsKO8n6bz3Cei0JOIkPYNfVOzzrbk5erVR0WE5CsQUDYQqX5IQkF9fCZ84+y9sorxD5DPoqLzpoGzat5dqyFcTCY+z22i41A/uNL/20g//KB1GOd2uKXb/AWpRCO+Xfh5qeVE0WARvWXwtwEdpiVOwD5j6nCt0GHos5BcfJLXRj4T2BXJ1ghU8/1V5MBB2nNQxjph0tE4wklH5qIilcCuXDPhOWROzxzuDK1Gdqru/Ne5kZCCO4F51RReGDvdwSHyE+Slk6XAbEPEu59PYZ5q9DwUjoDN6d7X4j6xNegfYcnWI2QGUU5cygLk8E4qXSnXQD6KBHAVDQxhVvp2BwauMBMObTOuGMifABQku11YFchgcOfxzidK9Iz7MVW5yY+gA4gWTYGRlbh/JwnRF6MYZXhjEec217h3xjpMGJUlqjtJskbYAO8y4mW/8llD1hGYDQPWeUfQIA0tWELTEyZ6CN9yAEucVVwShpvcvCzUM18584lDBHx9/ZOE69Uogfq8PYKUOa7zRhAQj6mgmQAhc5aLvxNNYD7a/ETNtPvbT/9TdZ5OwVNVQ4tZRKb4U7vNjzVI4tsNXasbtLCSwJ6wUEdhwQmcj7BC1ayHO8N3wZdXGVOqfw0HHcpYoxrDrLjGBWlLFwC9RYntowFneZ0u8U1DbZs+IvMdAFS/9+QOPfybWhdVH75ctPjWXSVk6t5r1DFRhMFfu3taG+umnAalNfGmOSosjeCioNmoUSqIE/L8WJJqQ03Ll8nyjoDOEmDM5jmDbokdxnxzq/lCg5oz3kqrJoIBCQzqwGOwmI+LSoIL8kfWAR0B4SE//9pGnd4/Js9ImiwK9eCxPjEUqtWlFHIX6Yn7vj4kXqWnjXKRWtFCZIHcJTHfHljBIPxa7HDthoa0FQQg5L6CgiwEzdlvr4w0lc7IRa/ugFCUwVohifqj3TeZ7ea6etVKlc+IrWC6UOOVW19zAXdx6PwTle1ZvHQB9tf3Y6Bca/rOWOxcbzN7HxhamiHRS5vh+VuQOImCMN1h5N1Jk7vqe2bY9IDf0yLFvo55zna62oJmfFJkVirK1lycmtBoutpgdnL8RodhfYcWSUeGnu8RH4/SlLcWQyXq7B3fYDilwWREqg555DavrEbW/CPxJfsryI44RcK8IQgi9OTEh6ABpiBud4DN4+tlJ+t1 X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(36860700013)(376014)(7416014)(82310400026)(1800799024); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Dec 2024 10:57:44.8361 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: de85b411-cd99-41cf-2525-08dd16adfac7 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN3PEPF0000B371.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB7501 Received-SPF: permerror client-ip=2a01:111:f403:200a::62c; envelope-from=Julia.Zhang@amd.com; helo=NAM12-MW2-obe.outbound.protection.outlook.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.996, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org To implement passthrough dGPU prime in guest, virtio-gpu need to check p2pdma_distance of two GPUs. This adds a new command for guest to pass virtual pci notations of two pci devices to host and send xen privcmd to calculate physical p2pdma_distance. Signed-off-by: Julia Zhang --- hw/display/virtio-gpu-virgl.c | 47 +++++++++++++++++++++ hw/i386/xen/xen-hvm.c | 6 +++ include/hw/virtio/virtio-gpu-bswap.h | 12 ++++++ include/hw/xen/xen.h | 3 ++ include/standard-headers/linux/virtio_gpu.h | 19 +++++++++ 5 files changed, 87 insertions(+) diff --git a/hw/display/virtio-gpu-virgl.c b/hw/display/virtio-gpu-virgl.c index 07faeb1834..eb9b193ade 100644 --- a/hw/display/virtio-gpu-virgl.c +++ b/hw/display/virtio-gpu-virgl.c @@ -16,6 +16,12 @@ #include "qemu/iov.h" #include "trace.h" #include "hw/virtio/virtio.h" +#include "hw/pci/pci.h" +#include "hw/pci/pci_bus.h" +#include "hw/virtio/virtio-pci.h" +#include "hw/virtio/virtio-bus.h" +#include "hw/xen/xen.h" + #include "hw/virtio/virtio-gpu.h" #include "hw/virtio/virtio-gpu-bswap.h" #include "hw/virtio/virtio-gpu-pixman.h" @@ -188,6 +194,44 @@ virtio_gpu_virgl_unmap_resource_blob(VirtIOGPU *g, return 0; } + +static void virgl_cmd_p2pdma_distance(VirtIOGPU *g, + struct virtio_gpu_ctrl_command *cmd) +{ + struct virtio_gpu_device_p2pdma_distance cmd_p; + struct virtio_gpu_resp_distance resp; + PCIDevice *client = NULL, *provider = NULL; + int ret; + + VIRTIO_GPU_FILL_CMD(cmd_p); + virtio_gpu_p2pdma_distance_bswap(&cmd_p); + + ret = pci_qdev_get_device(cmd_p.provider_bus, cmd_p.provider_slot, cmd_p.provider_func, &provider); + + if (ret) { + qemu_log_mask(LOG_GUEST_ERROR, "%s: virgl get physical device error: %s\n", + __func__, strerror(-ret)); + cmd->error = VIRTIO_GPU_RESP_ERR_INVALID_PARAMETER; + return; + } + + ret = pci_qdev_get_device(cmd_p.client_bus, cmd_p.client_slot, cmd_p.client_func, &client); + if (ret) { + qemu_log_mask(LOG_GUEST_ERROR, "%s: virgl get physical device error: %s\n", + __func__, strerror(-ret)); + cmd->error = VIRTIO_GPU_RESP_ERR_INVALID_PARAMETER; + return; + } + + int distance = xen_p2pdma_distance(provider->hostaddr.bus, provider->hostaddr.slot, + provider->hostaddr.function,client->hostaddr.bus, + client->hostaddr.slot, client->hostaddr.function); + + memset(&resp, 0, sizeof(resp)); + resp.hdr.type = VIRTIO_GPU_RESP_OK_P2PDMA_DISTANCE; + resp.distance = distance; + virtio_gpu_ctrl_response(g, cmd, &resp.hdr, sizeof(resp)); +} #endif static void virgl_cmd_create_resource_2d(VirtIOGPU *g, @@ -913,6 +957,9 @@ void virtio_gpu_virgl_process_cmd(VirtIOGPU *g, case VIRTIO_GPU_CMD_SUBMIT_3D: virgl_cmd_submit_3d(g, cmd); break; + case VIRTIO_GPU_CMD_P2PDMA_DISTANCE: + virgl_cmd_p2pdma_distance(g, cmd); + break; case VIRTIO_GPU_CMD_TRANSFER_TO_HOST_2D: virgl_cmd_transfer_to_host_2d(g, cmd); break; diff --git a/hw/i386/xen/xen-hvm.c b/hw/i386/xen/xen-hvm.c index d3df488c48..c8dd27dad7 100644 --- a/hw/i386/xen/xen-hvm.c +++ b/hw/i386/xen/xen-hvm.c @@ -604,6 +604,12 @@ static bool xen_check_stubdomain(struct xs_handle *xsh) return is_stubdom; } +int xen_p2pdma_distance(uint32_t bus, uint32_t slot, uint32_t func, + uint32_t c_bus, uint32_t c_slot, uint32_t c_func) +{ + return xc_physdev_p2pdma_distance(xen_xc, bus, slot, func, c_bus, c_slot, c_func); +} + void xen_hvm_init_pc(PCMachineState *pcms, MemoryRegion **ram_memory) { MachineState *ms = MACHINE(pcms); diff --git a/include/hw/virtio/virtio-gpu-bswap.h b/include/hw/virtio/virtio-gpu-bswap.h index dd1975e2d4..b5c0c0adcd 100644 --- a/include/hw/virtio/virtio-gpu-bswap.h +++ b/include/hw/virtio/virtio-gpu-bswap.h @@ -78,6 +78,18 @@ virtio_gpu_map_blob_bswap(struct virtio_gpu_resource_map_blob *mblob) le64_to_cpus(&mblob->offset); } +static inline void +virtio_gpu_p2pdma_distance_bswap(struct virtio_gpu_device_p2pdma_distance *p2p_dist) +{ + virtio_gpu_ctrl_hdr_bswap(&p2p_dist->hdr); + le32_to_cpus(&p2p_dist->provider_bus); + le32_to_cpus(&p2p_dist->provider_slot); + le32_to_cpus(&p2p_dist->provider_func); + le32_to_cpus(&p2p_dist->client_bus); + le32_to_cpus(&p2p_dist->client_bus); + le32_to_cpus(&p2p_dist->client_bus); +} + static inline void virtio_gpu_unmap_blob_bswap(struct virtio_gpu_resource_unmap_blob *ublob) { diff --git a/include/hw/xen/xen.h b/include/hw/xen/xen.h index ecb89ecfc1..fe1d628327 100644 --- a/include/hw/xen/xen.h +++ b/include/hw/xen/xen.h @@ -48,4 +48,7 @@ qemu_irq *xen_interrupt_controller_init(void); void xen_register_framebuffer(struct MemoryRegion *mr); +int xen_p2pdma_distance(uint32_t bus, uint32_t slot, uint32_t func, + uint32_t c_bus, uint32_t c_slot, uint32_t c_func); + #endif /* QEMU_HW_XEN_H */ diff --git a/include/standard-headers/linux/virtio_gpu.h b/include/standard-headers/linux/virtio_gpu.h index 6459fdb9fb..2e55dcc2fe 100644 --- a/include/standard-headers/linux/virtio_gpu.h +++ b/include/standard-headers/linux/virtio_gpu.h @@ -95,6 +95,7 @@ enum virtio_gpu_ctrl_type { VIRTIO_GPU_CMD_SUBMIT_3D, VIRTIO_GPU_CMD_RESOURCE_MAP_BLOB, VIRTIO_GPU_CMD_RESOURCE_UNMAP_BLOB, + VIRTIO_GPU_CMD_P2PDMA_DISTANCE, /* cursor commands */ VIRTIO_GPU_CMD_UPDATE_CURSOR = 0x0300, @@ -108,6 +109,7 @@ enum virtio_gpu_ctrl_type { VIRTIO_GPU_RESP_OK_EDID, VIRTIO_GPU_RESP_OK_RESOURCE_UUID, VIRTIO_GPU_RESP_OK_MAP_INFO, + VIRTIO_GPU_RESP_OK_P2PDMA_DISTANCE, /* error responses */ VIRTIO_GPU_RESP_ERR_UNSPEC = 0x1200, @@ -429,6 +431,23 @@ struct virtio_gpu_set_scanout_blob { uint32_t offsets[4]; }; +/* VIRTIO_GPU_CMD_P2PDMA_DISTANCE */ +struct virtio_gpu_device_p2pdma_distance { + struct virtio_gpu_ctrl_hdr hdr; + __le32 provider_bus; + __le32 provider_slot; + __le32 provider_func; + __le32 client_bus; + __le32 client_slot; + __le32 client_func; +}; + +/* VIRTIO_GPU_RESP_DISTANCE */ +struct virtio_gpu_resp_distance { + struct virtio_gpu_ctrl_hdr hdr; + __le32 distance; +}; + /* VIRTIO_GPU_CMD_RESOURCE_MAP_BLOB */ struct virtio_gpu_resource_map_blob { struct virtio_gpu_ctrl_hdr hdr;