From patchwork Tue Jun 13 17:20:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Elliot Berman X-Patchwork-Id: 13279065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64B2BEB64DC for ; Tue, 13 Jun 2023 17:23:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239600AbjFMRXJ (ORCPT ); Tue, 13 Jun 2023 13:23:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50560 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239121AbjFMRW3 (ORCPT ); Tue, 13 Jun 2023 13:22:29 -0400 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5EE961FCC; Tue, 13 Jun 2023 10:22:03 -0700 (PDT) Received: from pps.filterd (m0279873.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 35DGcgbv000521; Tue, 13 Jun 2023 17:21:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=qcppdkim1; bh=s6KQRXCr9HZdHYF3E6wOLJor6qVe+7NpuuAc9zL12G0=; b=Es4q54jWH0qW0CnRIdG5EdA+rEiQiacl2tPmR4XW9WFqL7IQspX6A8lNpVK+gf0ibuCs qsbAj5t7mzeslPyos3ZWWrfJz0igkSuVVvMGgSBRYds51wkshUhD9afCGQ2ASm88UV0T PB68Nt8Og8UKLA8tSGV9Aqw4G0A72BZWIPb/jdkdNThACM9oAndKzm23VYryRR2Usv3o ju4gW6r61jaFtlhnchTlZKec/FuayZE4qZAsDENeOrMXJobnKBAr7b1tfPEsGXyUGDy1 jOLUyMqwkzjngEK0fiK9suaPK0KZkWR4PivCxFN5AzHs0MQtJIowCZpr8dwF+V6XUylg XQ== Received: from nasanppmta02.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3r6a8ytbj7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:42 +0000 Received: from nasanex01b.na.qualcomm.com (nasanex01b.na.qualcomm.com [10.46.141.250]) by NASANPPMTA02.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTPS id 35DHLdcp029169 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 13 Jun 2023 17:21:39 GMT Received: from hu-eberman-lv.qualcomm.com (10.49.16.6) by nasanex01b.na.qualcomm.com (10.46.141.250) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.42; Tue, 13 Jun 2023 10:21:38 -0700 From: Elliot Berman To: Alex Elder , Srinivas Kandagatla , Elliot Berman , Prakruthi Deepak Heragu CC: Murali Nalajala , Trilok Soni , Srivatsa Vaddagiri , Carl van Schaik , Dmitry Baryshkov , Bjorn Andersson , "Konrad Dybcio" , Arnd Bergmann , "Greg Kroah-Hartman" , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Jonathan Corbet , Bagas Sanjaya , Will Deacon , Andy Gross , Catalin Marinas , Jassi Brar , , , , , Subject: [PATCH v14 13/25] samples: Add sample userspace Gunyah VM Manager Date: Tue, 13 Jun 2023 10:20:41 -0700 Message-ID: <20230613172054.3959700-14-quic_eberman@quicinc.com> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20230613172054.3959700-1-quic_eberman@quicinc.com> References: <20230613172054.3959700-1-quic_eberman@quicinc.com> MIME-Version: 1.0 X-Originating-IP: [10.49.16.6] X-ClientProxiedBy: nalasex01b.na.qualcomm.com (10.47.209.197) To nasanex01b.na.qualcomm.com (10.46.141.250) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 95HKHMLRR62cjSiN92ZRd3UEoRmTf5lP X-Proofpoint-ORIG-GUID: 95HKHMLRR62cjSiN92ZRd3UEoRmTf5lP X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.573,FMLib:17.11.176.26 definitions=2023-06-13_19,2023-06-12_02,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 lowpriorityscore=0 phishscore=0 spamscore=0 malwarescore=0 impostorscore=0 mlxscore=0 suspectscore=0 bulkscore=0 adultscore=0 priorityscore=1501 clxscore=1015 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2306130153 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org Add a sample Gunyah VMM capable of launching a non-proxy scheduled VM. Reviewed-by: Alex Elder Signed-off-by: Elliot Berman --- samples/Kconfig | 10 ++ samples/Makefile | 1 + samples/gunyah/.gitignore | 2 + samples/gunyah/Makefile | 6 + samples/gunyah/gunyah_vmm.c | 266 +++++++++++++++++++++++++++++++++++ samples/gunyah/sample_vm.dts | 68 +++++++++ 6 files changed, 353 insertions(+) create mode 100644 samples/gunyah/.gitignore create mode 100644 samples/gunyah/Makefile create mode 100644 samples/gunyah/gunyah_vmm.c create mode 100644 samples/gunyah/sample_vm.dts diff --git a/samples/Kconfig b/samples/Kconfig index b2db430bd3ff3..567c7a706c010 100644 --- a/samples/Kconfig +++ b/samples/Kconfig @@ -280,6 +280,16 @@ config SAMPLE_KMEMLEAK Build a sample program which have explicitly leaks memory to test kmemleak +config SAMPLE_GUNYAH + bool "Build example Gunyah Virtual Machine Manager" + depends on CC_CAN_LINK && HEADERS_INSTALL + depends on GUNYAH + help + Build an example Gunyah VMM userspace program capable of launching + a basic virtual machine under the Gunyah hypervisor. + This demonstrates how to create a virtual machine under the Gunyah + hypervisor. + source "samples/rust/Kconfig" endif # SAMPLES diff --git a/samples/Makefile b/samples/Makefile index 7727f1a0d6d15..e1b92dec169f3 100644 --- a/samples/Makefile +++ b/samples/Makefile @@ -37,3 +37,4 @@ obj-$(CONFIG_SAMPLE_KMEMLEAK) += kmemleak/ obj-$(CONFIG_SAMPLE_CORESIGHT_SYSCFG) += coresight/ obj-$(CONFIG_SAMPLE_FPROBE) += fprobe/ obj-$(CONFIG_SAMPLES_RUST) += rust/ +obj-$(CONFIG_SAMPLE_GUNYAH) += gunyah/ diff --git a/samples/gunyah/.gitignore b/samples/gunyah/.gitignore new file mode 100644 index 0000000000000..adc7d1589fde6 --- /dev/null +++ b/samples/gunyah/.gitignore @@ -0,0 +1,2 @@ +# SPDX-License-Identifier: GPL-2.0 +/gunyah_vmm diff --git a/samples/gunyah/Makefile b/samples/gunyah/Makefile new file mode 100644 index 0000000000000..faf14f9bb3374 --- /dev/null +++ b/samples/gunyah/Makefile @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0-only + +userprogs-always-y += gunyah_vmm +dtb-y += sample_vm.dtb + +userccflags += -I usr/include diff --git a/samples/gunyah/gunyah_vmm.c b/samples/gunyah/gunyah_vmm.c new file mode 100644 index 0000000000000..6f636ac227c69 --- /dev/null +++ b/samples/gunyah/gunyah_vmm.c @@ -0,0 +1,266 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#define __USE_GNU +#include + +#include + +#define DEFAULT_GUEST_BASE 0x80000000 +#define DEFAULT_GUEST_SIZE 0x6400000 /* 100 MiB */ +#define DEFAULT_DTB_OFFSET 0x45f0000 /* 70MiB - 64 KiB */ +#define DEFAULT_RAMDISK_OFFSET 0x4600000 /* 70MiB */ + +struct vm_config { + int image_fd; + int dtb_fd; + int ramdisk_fd; + + uint64_t guest_base; + uint64_t guest_size; + + off_t image_size; + uint64_t dtb_offset; + off_t dtb_size; + uint64_t ramdisk_offset; + off_t ramdisk_size; +}; + +static struct option options[] = { + { "help", no_argument, NULL, 'h' }, + { "image", required_argument, NULL, 'i' }, + { "dtb", required_argument, NULL, 'd' }, + { "ramdisk", optional_argument, NULL, 'r' }, + { "base", optional_argument, NULL, 'B' }, + { "size", optional_argument, NULL, 'S' }, + { "dtb_offset", optional_argument, NULL, 'D' }, + { "ramdisk_offset", optional_argument, NULL, 'R' }, + { } +}; + +static void print_help(char *cmd) +{ + printf("gunyah_vmm, a sample tool to launch Gunyah VMs\n" + "Usage: %s \n" + " --help, -h this menu\n" + " --image, -i VM image file to load (e.g. a kernel Image) [Required]\n" + " --dtb, -d Devicetree file to load [Required]\n" + " --ramdisk, -r Ramdisk file to load\n" + " --base, -B
Set the base address of guest's memory [Default: 0x%08x]\n" + " --size, -S The number of bytes large to make the guest's memory [Default: 0x%08x]\n" + " --dtb_offset, -D Offset into guest memory to load the DTB [Default: 0x%08x]\n" + " --ramdisk_offset, -R Offset into guest memory to load a ramdisk [Default: 0x%08x]\n" + , cmd, DEFAULT_GUEST_BASE, DEFAULT_GUEST_SIZE, + DEFAULT_DTB_OFFSET, DEFAULT_RAMDISK_OFFSET); +} + +int main(int argc, char **argv) +{ + int gunyah_fd, vm_fd, guest_fd; + struct gh_userspace_memory_region guest_mem_desc = { 0 }; + struct gh_vm_dtb_config dtb_config = { 0 }; + char *guest_mem; + struct vm_config config = { + /* Defaults good enough to boot static kernel and a basic ramdisk */ + .image_fd = -1, + .dtb_fd = -1, + .ramdisk_fd = -1, + .guest_base = DEFAULT_GUEST_BASE, + .guest_size = DEFAULT_GUEST_SIZE, + .dtb_offset = DEFAULT_DTB_OFFSET, + .ramdisk_offset = DEFAULT_RAMDISK_OFFSET, + }; + struct stat st; + int opt, optidx, ret = 0; + long l; + + while ((opt = getopt_long(argc, argv, "hi:d:r:B:S:D:R:c:", options, &optidx)) != -1) { + switch (opt) { + case 'i': + config.image_fd = open(optarg, O_RDONLY | O_CLOEXEC); + if (config.image_fd < 0) { + perror("Failed to open image"); + return -1; + } + if (stat(optarg, &st) < 0) { + perror("Failed to stat image"); + return -1; + } + config.image_size = st.st_size; + break; + case 'd': + config.dtb_fd = open(optarg, O_RDONLY | O_CLOEXEC); + if (config.dtb_fd < 0) { + perror("Failed to open dtb"); + return -1; + } + if (stat(optarg, &st) < 0) { + perror("Failed to stat dtb"); + return -1; + } + config.dtb_size = st.st_size; + break; + case 'r': + config.ramdisk_fd = open(optarg, O_RDONLY | O_CLOEXEC); + if (config.ramdisk_fd < 0) { + perror("Failed to open ramdisk"); + return -1; + } + if (stat(optarg, &st) < 0) { + perror("Failed to stat ramdisk"); + return -1; + } + config.ramdisk_size = st.st_size; + break; + case 'B': + l = strtol(optarg, NULL, 0); + if (l == LONG_MIN) { + perror("Failed to parse base address"); + return -1; + } + config.guest_base = l; + break; + case 'S': + l = strtol(optarg, NULL, 0); + if (l == LONG_MIN) { + perror("Failed to parse memory size"); + return -1; + } + config.guest_size = l; + break; + case 'D': + l = strtol(optarg, NULL, 0); + if (l == LONG_MIN) { + perror("Failed to parse dtb offset"); + return -1; + } + config.dtb_offset = l; + break; + case 'R': + l = strtol(optarg, NULL, 0); + if (l == LONG_MIN) { + perror("Failed to parse ramdisk offset"); + return -1; + } + config.ramdisk_offset = l; + break; + case 'h': + print_help(argv[0]); + return 0; + default: + print_help(argv[0]); + return -1; + } + } + + if (config.image_fd == -1 || config.dtb_fd == -1) { + print_help(argv[0]); + return -1; + } + + if (config.image_size > config.guest_size) { + fprintf(stderr, "Image size puts it outside guest memory. Make image smaller or increase guest memory size.\n"); + return -1; + } + + if (config.dtb_offset + config.dtb_size > config.guest_size) { + fprintf(stderr, "DTB offset and size puts it outside guest memory. Make dtb smaller or increase guest memory size.\n"); + return -1; + } + + if (config.ramdisk_fd == -1 && + config.ramdisk_offset + config.ramdisk_size > config.guest_size) { + fprintf(stderr, "Ramdisk offset and size puts it outside guest memory. Make ramdisk smaller or increase guest memory size.\n"); + return -1; + } + + gunyah_fd = open("/dev/gunyah", O_RDWR | O_CLOEXEC); + if (gunyah_fd < 0) { + perror("Failed to open /dev/gunyah"); + return -1; + } + + vm_fd = ioctl(gunyah_fd, GH_CREATE_VM, 0); + if (vm_fd < 0) { + perror("Failed to create vm"); + return -1; + } + + guest_fd = memfd_create("guest_memory", MFD_CLOEXEC); + if (guest_fd < 0) { + perror("Failed to create guest memfd"); + return -1; + } + + if (ftruncate(guest_fd, config.guest_size) < 0) { + perror("Failed to grow guest memory"); + return -1; + } + + guest_mem = mmap(NULL, config.guest_size, PROT_READ | PROT_WRITE, MAP_SHARED, guest_fd, 0); + if (guest_mem == MAP_FAILED) { + perror("Not enough memory"); + return -1; + } + + if (read(config.image_fd, guest_mem, config.image_size) < 0) { + perror("Failed to read image into guest memory"); + return -1; + } + + if (read(config.dtb_fd, guest_mem + config.dtb_offset, config.dtb_size) < 0) { + perror("Failed to read dtb into guest memory"); + return -1; + } + + if (config.ramdisk_fd > 0 && + read(config.ramdisk_fd, guest_mem + config.ramdisk_offset, + config.ramdisk_size) < 0) { + perror("Failed to read ramdisk into guest memory"); + return -1; + } + + guest_mem_desc.label = 0; + guest_mem_desc.flags = GH_MEM_ALLOW_READ | GH_MEM_ALLOW_WRITE | GH_MEM_ALLOW_EXEC; + guest_mem_desc.guest_phys_addr = config.guest_base; + guest_mem_desc.memory_size = config.guest_size; + guest_mem_desc.userspace_addr = (__u64)guest_mem; + + if (ioctl(vm_fd, GH_VM_SET_USER_MEM_REGION, &guest_mem_desc) < 0) { + perror("Failed to register guest memory with VM"); + return -1; + } + + dtb_config.guest_phys_addr = config.guest_base + config.dtb_offset; + dtb_config.size = config.dtb_size; + if (ioctl(vm_fd, GH_VM_SET_DTB_CONFIG, &dtb_config) < 0) { + perror("Failed to set DTB configuration for VM"); + return -1; + } + + ret = ioctl(vm_fd, GH_VM_START); + if (ret) { + perror("GH_VM_START failed"); + return -1; + } + + while (1) + pause(); + + return 0; +} diff --git a/samples/gunyah/sample_vm.dts b/samples/gunyah/sample_vm.dts new file mode 100644 index 0000000000000..293bbc0469c8f --- /dev/null +++ b/samples/gunyah/sample_vm.dts @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: BSD-3-Clause +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +/dts-v1/; + +/ { + #address-cells = <2>; + #size-cells = <2>; + interrupt-parent = <&intc>; + + chosen { + bootargs = "nokaslr"; + }; + + cpus { + #address-cells = <0x2>; + #size-cells = <0>; + + cpu@0 { + device_type = "cpu"; + compatible = "arm,armv8"; + reg = <0 0>; + }; + }; + + intc: interrupt-controller@3FFF0000 { + compatible = "arm,gic-v3"; + #interrupt-cells = <3>; + #address-cells = <2>; + #size-cells = <2>; + interrupt-controller; + reg = <0 0x3FFF0000 0 0x10000>, + <0 0x3FFD0000 0 0x20000>; + }; + + timer { + compatible = "arm,armv8-timer"; + always-on; + interrupts = <1 13 0x108>, + <1 14 0x108>, + <1 11 0x108>, + <1 10 0x108>; + clock-frequency = <19200000>; + }; + + gunyah-vm-config { + image-name = "linux_vm_0"; + + memory { + #address-cells = <2>; + #size-cells = <2>; + + base-address = <0 0x80000000>; + }; + + interrupts { + config = <&intc>; + }; + + vcpus { + affinity-map = < 0 >; + sched-priority = < (-1) >; + sched-timeslice = < 2000 >; + }; + }; +};