From patchwork Tue Sep 10 21:28:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregor Haas X-Patchwork-Id: 13799324 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 80C96EE01EB for ; Tue, 10 Sep 2024 21:28:59 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1so8PN-0006m3-UJ; Tue, 10 Sep 2024 17:28:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1so8PM-0006e3-2H; Tue, 10 Sep 2024 17:28:20 -0400 Received: from mail-yw1-x1130.google.com ([2607:f8b0:4864:20::1130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1so8PI-0002OX-U8; Tue, 10 Sep 2024 17:28:19 -0400 Received: by mail-yw1-x1130.google.com with SMTP id 00721157ae682-6d6e2b2fbc4so58508817b3.0; Tue, 10 Sep 2024 14:28:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726003695; x=1726608495; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RtLCCYi/8lZclH6EGbiQfc3dGIl0M7+pchzl78qCIMM=; b=CbB92+6iikMVgKzXpF27wpt0ZQ0tBIU861KnTKm668bKz6I3YH+w8xVqWNlu4kD3tW 4ZW8kLF7WxhcroU4/82S1JSBa6llKXQxfsYbJ3h1XUyHfZCRd86uuamAlkOXYXvR2yaI xijKOB/sZlqHg9ApNCQz+JaDK9dcK3t5hZjR7bHohNaLzLHB9e9080uofwevyYK3592b 0p3KXRVe64P/NxQMCZEWiNHBmY4eg44KTJpjhViLqP6dsbjQzRckvNTNQVyLMJeX9gh1 UrTTH3bq+4gBSisN2/Wa95GQ/jNGuKPMHHW1wXcSZO5Ah9V9AyshKp89wxfr9bWia1+F VPrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726003695; x=1726608495; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RtLCCYi/8lZclH6EGbiQfc3dGIl0M7+pchzl78qCIMM=; b=XTk4Naxs9FyjUZmEyL7LtvpzdMrPjRNUq1bSEX1JcMD3biVox9ia1caqSfBjwgGkGL n439IbRqpObgy3T3UBpbgihNmMoK3xt0FmlJgKFvSnkq/YdEb0EdqFlFpmG/gjNacsl8 mG6VcwN/qcrILSYqSfBSosy+1nWFN+z4WeQXVv1tPpyKJZBchwMi9LC8vcf2peBJeXgS U9txu6iQ24zCm9B6usEzKea1l1Tkq+BOx06oWzVKqXt/bgRIudXRCjQHaKYa7AowOBW+ t344ocQ/q/y8ylMQwWIy4eocWiqyHmTS1Be+Iqkrotqh51y8b55Zl7vvlsRr243arFjj Ax3A== X-Gm-Message-State: AOJu0YwEAqC6fwfMQ3B5Km/nGRWpJYSUFGTbbP/0nvPlF9Z7XB6AOfa8 rPDO50HmEEyQ36pZPs5KGKhElu9R2GyR60dq5vmgbSfgfXSV3onrk8yj62rV X-Google-Smtp-Source: AGHT+IEyxCM0AesICnV6B99pEpGxTV8qGGmkccJK+qUpcvAtns4GG8j4RFCoE2HvZ6og1cHVohlFdQ== X-Received: by 2002:a05:690c:490c:b0:6b7:3da:cffd with SMTP id 00721157ae682-6db44f164b6mr189307527b3.15.1726003695286; Tue, 10 Sep 2024 14:28:15 -0700 (PDT) Received: from localhost ([2600:1700:830:3db0::14]) by smtp.gmail.com with UTF8SMTPSA id 00721157ae682-6db96521cbfsm4495707b3.131.2024.09.10.14.28.14 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 10 Sep 2024 14:28:14 -0700 (PDT) From: Gregor Haas To: qemu-devel@nongnu.org Cc: qemu-riscv@nongnu.org, atishp@rivosinc.com, dbarboza@ventanamicro.com, alistair.francis@wdc.com, Gregor Haas Subject: [PATCH v5 1/2] hw/riscv: Add support for generating OpenSBI domains in the device tree Date: Tue, 10 Sep 2024 14:28:10 -0700 Message-ID: <20240910212811.996184-2-gregorhaas1997@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240910212811.996184-1-gregorhaas1997@gmail.com> References: <20240910212811.996184-1-gregorhaas1997@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1130; envelope-from=gregorhaas1997@gmail.com; helo=mail-yw1-x1130.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org OpenSBI has support for domains, which are partitions of CPUs and memory into isolated compartments. Domains can be specified in the device tree according to a standardized format [1], which OpenSBI parses at boot time to initialize all system domains. This patch enables simply specifying domains (and their associated memory regions) on the QEMU command line, from which these are then rendered into the machine's device tree. At machine initialization time, a new create_fdt_opensbi_domains() function walks the peripherals/peripherals-anon containers, identifies all domains and memregions, and parses them into the relevant device tree structures. [1] https://github.com/riscv-software-src/opensbi/blob/master/docs/domain_support.md Signed-off-by: Gregor Haas Reviewed-by: Daniel Henrique Barboza --- MAINTAINERS | 7 + hw/riscv/Kconfig | 4 + hw/riscv/meson.build | 1 + hw/riscv/opensbi_domain.c | 558 ++++++++++++++++++++++++++++++ hw/riscv/virt.c | 3 + include/hw/riscv/opensbi_domain.h | 69 ++++ 6 files changed, 642 insertions(+) create mode 100644 hw/riscv/opensbi_domain.c create mode 100644 include/hw/riscv/opensbi_domain.h diff --git a/MAINTAINERS b/MAINTAINERS index 0c1bc69828..3b9f5b7432 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -357,6 +357,13 @@ F: target/riscv/XVentanaCondOps.decode F: target/riscv/insn_trans/trans_xventanacondops.c.inc F: disas/riscv-xventana* +RISC-V OpenSBI domain support +M: Gregor Haas +L: qemu-riscv@nongnu.org +S: Maintained +F: hw/riscv/opensbi_domain.c +F: include/hw/riscv/opensbi_domain.h + RENESAS RX CPUs R: Yoshinori Sato S: Orphan diff --git a/hw/riscv/Kconfig b/hw/riscv/Kconfig index a2030e3a6f..db3a4d77ad 100644 --- a/hw/riscv/Kconfig +++ b/hw/riscv/Kconfig @@ -1,6 +1,9 @@ config RISCV_NUMA bool +config RISCV_OPENSBI_DOMAIN + bool + config IBEX bool @@ -40,6 +43,7 @@ config RISCV_VIRT imply TPM_TIS_SYSBUS select DEVICE_TREE select RISCV_NUMA + select RISCV_OPENSBI_DOMAIN select GOLDFISH_RTC select PCI select PCI_EXPRESS_GENERIC_BRIDGE diff --git a/hw/riscv/meson.build b/hw/riscv/meson.build index f872674093..f47626c164 100644 --- a/hw/riscv/meson.build +++ b/hw/riscv/meson.build @@ -1,6 +1,7 @@ riscv_ss = ss.source_set() riscv_ss.add(files('boot.c')) riscv_ss.add(when: 'CONFIG_RISCV_NUMA', if_true: files('numa.c')) +riscv_ss.add(when: 'CONFIG_RISCV_OPENSBI_DOMAIN', if_true: files('opensbi_domain.c')) riscv_ss.add(files('riscv_hart.c')) riscv_ss.add(when: 'CONFIG_OPENTITAN', if_true: files('opentitan.c')) riscv_ss.add(when: 'CONFIG_RISCV_VIRT', if_true: files('virt.c')) diff --git a/hw/riscv/opensbi_domain.c b/hw/riscv/opensbi_domain.c new file mode 100644 index 0000000000..da315e59b2 --- /dev/null +++ b/hw/riscv/opensbi_domain.c @@ -0,0 +1,558 @@ +/* + * OpenSBI Domains + * + * Copyright (c) 2024 Gregor Haas + * + * Generates OpenSBI domain nodes in the machine's device tree + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see . + */ + +#include "qemu/osdep.h" +#include "hw/riscv/opensbi_domain.h" +#include "hw/boards.h" +#include "hw/riscv/virt.h" +#include "qapi/error.h" +#include "qemu/cutils.h" +#include "qemu/error-report.h" +#include "sysemu/device_tree.h" + +#include + +static void create_fdt_domain_possible_harts(MachineState *ms, + OpenSBIDomainState *s, + char *path) { + unsigned long i, cpu; + unsigned long num_cpus; + + num_cpus = s->last_possible_hart - s->first_possible_hart + 1; + if (num_cpus) { + g_autofree uint32_t *phandles = g_malloc0_n(num_cpus, sizeof(uint32_t)); + + for (i = 0, cpu = s->first_possible_hart; i < num_cpus; i++, cpu++) { + g_autofree char *cpu_name = g_strdup_printf("/cpus/cpu@%li", cpu); + phandles[i] = cpu_to_fdt32(qemu_fdt_get_phandle( + ms->fdt, cpu_name)); + } + + qemu_fdt_setprop(ms->fdt, path, "possible-harts", + phandles, num_cpus * 4); + } +} + +static void create_fdt_domain_regions(MachineState *ms, + OpenSBIDomainState *s, + char *path) { + unsigned long i; + int num_regions = 0; + DeviceState *ds; + + for (i = 0; i < OPENSBI_DOMAIN_MEMREGIONS_MAX; i++) { + if (s->regions[i]) { + num_regions++; + } + } + + if (num_regions) { + g_autofree uint32_t *regions = + g_malloc0_n(num_regions, 2 * sizeof(uint32_t)); + for (i = 0; i < OPENSBI_DOMAIN_MEMREGIONS_MAX; i++) { + if (s->regions[i]) { + ds = DEVICE(s->regions[i]); + g_autofree char *region_name = g_strdup_printf( + "/chosen/opensbi-domains/%s", ds->id); + regions[2 * i] = cpu_to_fdt32(qemu_fdt_get_phandle + (ms->fdt, region_name)); + regions[2 * i + 1] = cpu_to_fdt32(s->region_perms[i]); + } + } + + qemu_fdt_setprop(ms->fdt, path, "regions", + regions, num_regions * 8); + } +} + +struct DomainFDTState { + MachineState *ms; + bool regions; +}; + +static void create_fdt_one_domain(MachineState *ms, OpenSBIDomainState *s) +{ + DeviceState *ds = DEVICE(s); + g_autofree char *path = g_strdup_printf("/chosen/opensbi-domains/%s", ds->id); + + qemu_fdt_add_subnode(ms->fdt, path); + qemu_fdt_setprop_string(ms->fdt, path, "compatible", + "opensbi,domain,instance"); + qemu_fdt_setprop_cells(ms->fdt, path, "phandle", + qemu_fdt_alloc_phandle(ms->fdt)); + + create_fdt_domain_possible_harts(ms, s, path); + create_fdt_domain_regions(ms, s, path); + + if (s->boot_hart != -1) { + g_autofree char *cpu_name = g_strdup_printf("/cpus/cpu@%i", s->boot_hart); + qemu_fdt_setprop_cell(ms->fdt, path, "boot-hart", + qemu_fdt_get_phandle(ms->fdt, cpu_name)); + if (s->assign) { + qemu_fdt_setprop_cell(ms->fdt, cpu_name, "opensbi-domain", + qemu_fdt_get_phandle(ms->fdt, path)); + } + } + + if (s->next_arg1 != -1) { + qemu_fdt_setprop_cells(ms->fdt, path, "next-arg1", + (uint64_t) s->next_arg1 >> 32, s->next_arg1); + } + + if (s->next_addr != -1) { + qemu_fdt_setprop_cells(ms->fdt, path, "next-addr", + (uint64_t) s->next_addr >> 32, s->next_addr); + } + + if (s->next_mode != -1) { + qemu_fdt_setprop_cell(ms->fdt, path, "next-mode", + s->next_mode); + } + + if (s->system_reset_allowed) { + qemu_fdt_setprop(ms->fdt, path, "system-reset-allowed", NULL, 0); + } + + if (s->system_suspend_allowed) { + qemu_fdt_setprop(ms->fdt, path, "system-suspend-allowed", NULL, 0); + } +} + +static uint32_t create_fdt_one_device(MachineState *ms, char *device) +{ + uint32_t phandle; + int offs = fdt_path_offset(ms->fdt, device); + + if (offs < 0) { + error_report("%s: Could not find device %s: %s", __func__, + device, fdt_strerror(offs)); + exit(1); + } + + phandle = fdt_get_phandle(ms->fdt, offs); + if (!phandle) { + phandle = qemu_fdt_alloc_phandle(ms->fdt); + qemu_fdt_setprop_cell(ms->fdt, device, "phandle", phandle); + } + + return phandle; +} + +static void create_fdt_one_memregion(MachineState *ms, + OpenSBIMemregionState *s) +{ + g_autofree char *path; + int i, dev, num_devices; + DeviceState *ds = DEVICE(s); + + path = g_strdup_printf("/chosen/opensbi-domains/%s", ds->id); + qemu_fdt_add_subnode(ms->fdt, path); + qemu_fdt_setprop_string(ms->fdt, path, "compatible", + "opensbi,domain,memregion"); + qemu_fdt_setprop_cells(ms->fdt, path, "base", + (uint64_t) s->base >> 32, s->base); + + qemu_fdt_setprop_cell(ms->fdt, path, "order", + (uint32_t) s->order); + + if (s->mmio) { + qemu_fdt_setprop(ms->fdt, path, "mmio", NULL, 0); + + /* Get all phandles for related devices */ + num_devices = 0; + for (i = 0; i < OPENSBI_MEMREGION_DEVICES_MAX; i++) { + if (s->devices[i]) { + num_devices++; + } + } + + if (num_devices) { + g_autofree uint32_t *devices = + g_malloc0_n(num_devices, sizeof(uint32_t)); + for (i = 0, dev = 0; i < OPENSBI_MEMREGION_DEVICES_MAX && + dev < num_devices; i++) { + if (s->devices[i]) { + devices[dev++] = create_fdt_one_device(ms, + s->devices[i]); + } + } + + qemu_fdt_setprop(ms->fdt, path, "devices", devices, + num_devices * 4); + } + } + + qemu_fdt_setprop_cells(ms->fdt, path, "phandle", + qemu_fdt_alloc_phandle(ms->fdt)); +} + +static int create_fdt_domains(Object *obj, void *opaque) +{ + struct DomainFDTState *dfs = opaque; + OpenSBIDomainState *osds; + OpenSBIMemregionState *osms; + + osds = (OpenSBIDomainState *) + object_dynamic_cast(obj, TYPE_OPENSBI_DOMAIN); + osms = (OpenSBIMemregionState *) + object_dynamic_cast(obj, TYPE_OPENSBI_MEMREGION); + + if (dfs->regions) { + if (osms) { + create_fdt_one_memregion(dfs->ms, osms); + } + } else { + if (osds) { + create_fdt_one_domain(dfs->ms, osds); + } + } + + return 0; +} + +static const char *containers[] = { + "/peripheral", "/peripheral-anon" +}; + +void create_fdt_opensbi_domains(MachineState *s) +{ + int i; + MachineState *ms = MACHINE(s); + Object *container; + + struct DomainFDTState check = { + .ms = ms, + .regions = true + }; + + /* Make sure that top-level node exists */ + qemu_fdt_add_subnode(ms->fdt, "/chosen/opensbi-domains"); + qemu_fdt_setprop_string(ms->fdt, "/chosen/opensbi-domains", + "compatible", "opensbi,domain,config"); + + /* Do a scan through regions first */ + for (i = 0; i < ARRAY_SIZE(containers); i++) { + container = container_get(OBJECT(s), containers[i]); + object_child_foreach(container, create_fdt_domains, &check); + } + + /* Then scan through domains */ + check.regions = false; + for (i = 0; i < ARRAY_SIZE(containers); i++) { + container = container_get(OBJECT(s), containers[i]); + object_child_foreach(container, create_fdt_domains, &check); + } +} + +/* OpenSBI Memregions */ + +static void set_mmio(Object *obj, bool val, Error **err) +{ + OpenSBIMemregionState *s = OPENSBI_MEMREGION(obj); + s->mmio = val; +} + +static void set_device(Object *obj, const char *val, Error **err) +{ + int i; + OpenSBIMemregionState *s = OPENSBI_MEMREGION(obj); + + for (i = 0; i < OPENSBI_DOMAIN_MEMREGIONS_MAX; i++) { + if (!s->devices[i]) { + s->devices[i] = g_strdup(val); + break; + } + } +} + +static void opensbi_memregion_instance_init(Object *obj) +{ + int i; + OpenSBIMemregionState *s = OPENSBI_MEMREGION(obj); + + s->base = -1; + object_property_add_uint64_ptr(obj, "base", &s->base, + OBJ_PROP_FLAG_WRITE); + object_property_set_description(obj, "base", + "The base address of the domain memory region. If \"order\" is also specified, " + "this property should be a 2 ^ order aligned 64 bit address"); + + s->order = -1; + object_property_add_uint32_ptr(obj, "order", &s->order, + OBJ_PROP_FLAG_WRITE); + object_property_set_description(obj, "order", + "The order of the domain memory region. This property should have a 32 bit value " + "(i.e. one DT cell) in the range 3 <= order <= __riscv_xlen."); + + s->mmio = false; + object_property_add_bool(obj, "mmio", NULL, set_mmio); + object_property_set_description(obj, "mmio", + "A boolean flag representing whether the domain memory region is a " + "memory-mapped I/O (MMIO) region."); + + for (i = 0; i < OPENSBI_DOMAIN_MEMREGIONS_MAX; i++) { + g_autofree char *propname = g_strdup_printf("device%i", i); + object_property_add_str(obj, propname, NULL, set_device); + + g_autofree char *description = g_strdup_printf( + "Device %i (out of %i) for this memregion. This property should be a device tree path to the device.", + i, OPENSBI_DOMAIN_MEMREGIONS_MAX); + object_property_set_description(obj, propname, description); + } +} + +static void opensbi_memregion_realize(DeviceState *ds, Error **errp) +{ + #if defined(TARGET_RISCV32) + int xlen = 32; + #elif defined(TARGET_RISCV64) + int xlen = 64; + #endif + + OpenSBIMemregionState *s = OPENSBI_MEMREGION(ds); + + if (s->base == -1) { + error_setg(errp, "must specify base"); + return; + } + + if (s->order == -1) { + error_setg(errp, "must specify order"); + return; + } + + /* Check order bounds */ + if (s->order < 3 || s->order > xlen) { + error_setg(errp, "order must be between 3 and %d", xlen); + return; + } + + /* Check base alignment */ + if (s->order < xlen && (s->base & (BIT(s->order) - 1))) { + error_setg(errp, "base not aligned to order"); + return; + } +} + +static void opensbi_memregion_class_init(ObjectClass *oc, void *opaque) +{ + DeviceClass *dc = DEVICE_CLASS(oc); + dc->realize = opensbi_memregion_realize; +} + +static const TypeInfo opensbi_memregion_info = { + .name = TYPE_OPENSBI_MEMREGION, + .parent = TYPE_DEVICE, + .instance_init = opensbi_memregion_instance_init, + .instance_size = sizeof(OpenSBIDomainState), + .class_init = opensbi_memregion_class_init +}; + +/* OpenSBI Domains */ + +static void set_sysreset_allowed(Object *obj, bool val, Error **err) +{ + OpenSBIDomainState *s = OPENSBI_DOMAIN(obj); + s->system_reset_allowed = val; +} + +static void set_suspend_allowed(Object *obj, bool val, Error **err) +{ + OpenSBIDomainState *s = OPENSBI_DOMAIN(obj); + s->system_suspend_allowed = val; +} + +static void set_assign(Object *obj, bool val, Error **err) +{ + OpenSBIDomainState *s = OPENSBI_DOMAIN(obj); + s->assign = val; +} + +static void set_possible_harts(Object *obj, const char *str, Error **err) +{ + OpenSBIDomainState *s = OPENSBI_DOMAIN(obj); + const char *firstcpu, *firstcpu_end, *lastcpu; + + firstcpu = str; + if (qemu_strtoul(firstcpu, &firstcpu_end, 0, + &s->first_possible_hart) < 0) { + error_setg(err, "could not convert firstcpu"); + return; + } + + lastcpu = qemu_strchrnul(str, '-'); + if (*lastcpu) { + if (lastcpu != firstcpu_end) { + error_setg(err, "could not separate firstcpu and lastcpu"); + return; + } + + lastcpu++; + if (qemu_strtoul(lastcpu, NULL, 0, + &s->last_possible_hart) < 0) { + error_setg(err, "could not convert lastcpu"); + return; + } + } else { + s->last_possible_hart = s->first_possible_hart; + } +} + +static void opensbi_domain_instance_init(Object *obj) +{ + int i; + OpenSBIDomainState *s = OPENSBI_DOMAIN(obj); + + s->boot_hart = VIRT_CPUS_MAX; + object_property_add_uint32_ptr(obj, "boot-hart", &s->boot_hart, + OBJ_PROP_FLAG_WRITE); + object_property_set_description(obj, "boot-hart", + "The HART booting the domain instance."); + + s->first_possible_hart = -1; + s->last_possible_hart = -1; + object_property_add_str(obj, "possible-harts", NULL, set_possible_harts); + object_property_set_description(obj, "possible-harts", + "The contiguous list of CPUs for the domain instance, specified as firstcpu[-lastcpu]"); + + s->next_arg1 = -1; + object_property_add_uint64_ptr(obj, "next-arg1", &s->next_arg1, + OBJ_PROP_FLAG_WRITE); + object_property_set_description(obj, "next-arg1", + "The 64 bit next booting stage arg1 for the domain instance."); + + s->next_addr = -1; + object_property_add_uint64_ptr(obj, "next-addr", &s->next_addr, + OBJ_PROP_FLAG_WRITE); + object_property_set_description(obj, "next-addr", + "The 64 bit next booting stage address for the domain instance."); + + s->next_mode = -1; + object_property_add_uint32_ptr(obj, "next-mode", &s->next_mode, + OBJ_PROP_FLAG_WRITE); + object_property_set_description(obj, "next-mode", + "The 32 bit next booting stage mode for the domain instance."); + + s->system_reset_allowed = false; + object_property_add_bool(obj, "system-reset-allowed", NULL, + set_sysreset_allowed); + object_property_set_description(obj, "system-reset-allowed", + "Whether the domain instance is allowed to do system reset."); + + s->system_suspend_allowed = false; + object_property_add_bool(obj, "system-suspend-allowed", NULL, + set_suspend_allowed); + object_property_set_description(obj, "system-suspend-allowed", + "Whether the domain instance is allowed to do system suspend."); + + for (i = 0; i < OPENSBI_DOMAIN_MEMREGIONS_MAX; i++) { + s->regions[i] = NULL; + g_autofree char *reg_propname = g_strdup_printf("region%i", i); + object_property_add_link(obj, reg_propname, TYPE_OPENSBI_MEMREGION, + (Object **) &s->regions[i], + qdev_prop_allow_set_link_before_realize, 0); + + g_autofree char *reg_description = g_strdup_printf( + "Region %i (out of %i) for this domain.", + i, OPENSBI_DOMAIN_MEMREGIONS_MAX); + object_property_set_description(obj, reg_propname, reg_description); + + s->region_perms[i] = 0; + g_autofree char *perm_propname = g_strdup_printf("perms%i", i); + object_property_add_uint32_ptr(obj, perm_propname, &s->region_perms[i], + OBJ_PROP_FLAG_WRITE); + + g_autofree char *perm_description = g_strdup_printf( + "Permissions for region %i for this domain.", i); + object_property_set_description(obj, perm_propname, perm_description); + } + + object_property_add_bool(obj, "assign", NULL, set_assign); + object_property_set_description(obj, "assign", + "Whether to assign this domain to its boot hart."); +} + +static void opensbi_domain_realize(DeviceState *ds, Error **errp) +{ + OpenSBIDomainState *s = OPENSBI_DOMAIN(ds); + + if (!ds->id) { + error_setg(errp, "must specify an id"); + return; + } + + if (s->boot_hart >= VIRT_CPUS_MAX) { + error_setg(errp, "boot hart larger than maximum number of CPUs (%d)", + VIRT_CPUS_MAX); + return; + } + + if (s->first_possible_hart == -1) { + if (s->last_possible_hart != -1) { + error_setg(errp, + "last possible hart set when first possible hart unset"); + return; + } + } else { + if (s->first_possible_hart >= VIRT_CPUS_MAX) { + error_setg(errp, + "first possible hart larger than maximum number of CPUs (%d)", + VIRT_CPUS_MAX); + return; + } + + if (s->last_possible_hart != -1) { + if (s->last_possible_hart < s->first_possible_hart) { + error_setg(errp, + "last possible hart larger than first possible hart"); + return; + } + + if (s->last_possible_hart >= VIRT_CPUS_MAX) { + error_setg(errp, + "last possible hart larger than maximum number of CPUS (%d)", + VIRT_CPUS_MAX); + return; + } + } + } +} + +static void opensbi_domain_class_init(ObjectClass *oc, void *opaque) +{ + DeviceClass *dc = DEVICE_CLASS(oc); + dc->realize = opensbi_domain_realize; +} + +static const TypeInfo opensbi_domain_info = { + .name = TYPE_OPENSBI_DOMAIN, + .parent = TYPE_DEVICE, + .instance_init = opensbi_domain_instance_init, + .instance_size = sizeof(OpenSBIDomainState), + .class_init = opensbi_domain_class_init +}; + +static void opensbi_register_types(void) +{ + type_register_static(&opensbi_domain_info); + type_register_static(&opensbi_memregion_info); +} + +type_init(opensbi_register_types) diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index cef41c150a..760d8ce4d4 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -55,6 +55,7 @@ #include "hw/acpi/aml-build.h" #include "qapi/qapi-visit-common.h" #include "hw/virtio/virtio-iommu.h" +#include "hw/riscv/opensbi_domain.h" /* KVM AIA only supports APLIC MSI. APLIC Wired is always emulated by QEMU. */ static bool virt_use_kvm_aia(RISCVVirtState *s) @@ -1050,6 +1051,8 @@ static void finalize_fdt(RISCVVirtState *s) create_fdt_uart(s, virt_memmap, irq_mmio_phandle); create_fdt_rtc(s, virt_memmap, irq_mmio_phandle); + + create_fdt_opensbi_domains(MACHINE(s)); } static void create_fdt(RISCVVirtState *s, const MemMapEntry *memmap) diff --git a/include/hw/riscv/opensbi_domain.h b/include/hw/riscv/opensbi_domain.h new file mode 100644 index 0000000000..1ec8ed8288 --- /dev/null +++ b/include/hw/riscv/opensbi_domain.h @@ -0,0 +1,69 @@ +/* + * OpenSBI Domains + * + * Copyright (c) 2024 Gregor Haas + * + * Generates OpenSBI domain nodes in the machine's device tree + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see . + */ + +#ifndef RISCV_DOMAIN_H +#define RISCV_DOMAIN_H + +#include "hw/sysbus.h" +#include "qom/object.h" +#include "cpu.h" + +#define TYPE_OPENSBI_MEMREGION "opensbi-memregion" +OBJECT_DECLARE_SIMPLE_TYPE(OpenSBIMemregionState, OPENSBI_MEMREGION) + +#define OPENSBI_MEMREGION_DEVICES_MAX 16 + +struct OpenSBIMemregionState { + /* public */ + DeviceState parent_obj; + + /* private */ + uint64_t base; + uint32_t order; + bool mmio; + char *devices[OPENSBI_MEMREGION_DEVICES_MAX]; +}; + +#define TYPE_OPENSBI_DOMAIN "opensbi-domain" +OBJECT_DECLARE_SIMPLE_TYPE(OpenSBIDomainState, OPENSBI_DOMAIN) + +#define OPENSBI_DOMAIN_MEMREGIONS_MAX 16 + +struct OpenSBIDomainState { + /* public */ + DeviceState parent_obj; + + /* private */ + OpenSBIMemregionState *regions[OPENSBI_DOMAIN_MEMREGIONS_MAX]; + unsigned int region_perms[OPENSBI_DOMAIN_MEMREGIONS_MAX]; + unsigned long first_possible_hart, last_possible_hart; + unsigned int boot_hart; + uint64_t next_arg1; + uint64_t next_addr; + uint32_t next_mode; + bool system_reset_allowed; + bool system_suspend_allowed; + + bool assign; +}; + +void create_fdt_opensbi_domains(MachineState *s); + +#endif /* RISCV_DOMAIN_H */ From patchwork Tue Sep 10 21:25:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregor Haas X-Patchwork-Id: 13799323 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 869B4EE01E9 for ; Tue, 10 Sep 2024 21:26:39 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1so8Ms-0002GG-9k; Tue, 10 Sep 2024 17:25:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1so8Mr-0002B0-4h; Tue, 10 Sep 2024 17:25:45 -0400 Received: from mail-yw1-x1130.google.com ([2607:f8b0:4864:20::1130]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1so8Mo-0002Iz-U0; Tue, 10 Sep 2024 17:25:44 -0400 Received: by mail-yw1-x1130.google.com with SMTP id 00721157ae682-6d3f4218081so59068327b3.1; Tue, 10 Sep 2024 14:25:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1726003541; x=1726608341; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Sul7/BXCsjPRlqt3G2+RLEijKUxX/bYzUBFroC3XHX8=; b=NNw/AwJWudejZh8znGmBJrK+PnlYsdROYO8YiMIOkwa9ILdkfeZHr9I+R7rWw661Aq zznGhVbvTnO/Q/0fUbIAMngKu6g8Q9tl+Vgd1zgHQRatchdy8JognfSYQa3YgihaEyNL jp4RttNWIcZHZdM/+meFaG+YCyqeW9XGzFCQ9o0Xk6tQfQ1a02wC1P28f4cMccTpbB9/ y+MAusBtm5nlWc3m1PQmD6pPI4EwGIud8lmz3JbLDoGjY86jfhxZOJCcuSWinQTiFNxr 67xbLuBfkZQvvKdiWRoBUzAI52/dpJ8rr5d7x9mk7ozzrIs49Unk83OQwVf/nuWKvCCS D6Ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726003541; x=1726608341; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Sul7/BXCsjPRlqt3G2+RLEijKUxX/bYzUBFroC3XHX8=; b=Ici0+Jq1O2UdHCYqayx8tu3n7/R8R0FX9KP9tjMmUrFKmQue0JPfj1PymRORz8d/5M 2/Xu/4/5JNcjaN2VHpl02kDAYYiMZagk5vmFaNGvjBMaPEijLtH0rUGsrtRkYRCwvRSq nGIlZH/raS2mmSLJ3BWbWnflonWzUTIz7YOPVmLzHz9VDCeQSHICUmqO/S4VpDlilpus v3VG9ioEuacvX+XP0kud9Dou4NOZOwXMRhv9bBTS8oRyHY1xcUYvhZV6LP15SqL0t8ZG YwRw5Osowkd0k7N+NzvtzbdgLoNm4XkF+S5lBF4lzfd578uK9I/YJ4a+5EfHmc1+2N9j ux6g== X-Gm-Message-State: AOJu0YyhKUdZHxxPM/Qa78hqRkYKQBkgq153rug0Y+BK+QzrpAZNYYxF sIVOaV06TZ3tMTzgYRpdm2HJT0fwwGRFgz0Yd9w2eOB4W2JVGV2CoQjavOew X-Google-Smtp-Source: AGHT+IF9uGnRMRk4MebQSBNtEVoh/GtzWzYdf0FBa4If4bNikE0RBCGI9Do70Bi5nVkaqO8pB/iRuQ== X-Received: by 2002:a05:690c:350e:b0:6af:b0cc:abda with SMTP id 00721157ae682-6db45164af1mr168531427b3.40.1726003541368; Tue, 10 Sep 2024 14:25:41 -0700 (PDT) Received: from localhost ([2600:1700:830:3db0::14]) by smtp.gmail.com with UTF8SMTPSA id 00721157ae682-6db9650df6dsm4428357b3.103.2024.09.10.14.25.40 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 10 Sep 2024 14:25:40 -0700 (PDT) From: Gregor Haas To: qemu-devel@nongnu.org Cc: qemu-riscv@nongnu.org, atishp@rivosinc.com, dbarboza@ventanamicro.com, alistair.francis@wdc.com, Gregor Haas Subject: [PATCH v5 2/2] docs/system/riscv: Add documentation for command-line OpenSBI domains Date: Tue, 10 Sep 2024 14:25:33 -0700 Message-ID: <20240910212533.986734-5-gregorhaas1997@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240910212533.986734-1-gregorhaas1997@gmail.com> References: <20240910212533.986734-1-gregorhaas1997@gmail.com> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1130; envelope-from=gregorhaas1997@gmail.com; helo=mail-yw1-x1130.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Signed-off-by: Gregor Haas --- MAINTAINERS | 1 + docs/system/riscv/opensbi_domains.rst | 156 ++++++++++++++++++++++++++ docs/system/target-riscv.rst | 10 ++ 3 files changed, 167 insertions(+) create mode 100644 docs/system/riscv/opensbi_domains.rst diff --git a/MAINTAINERS b/MAINTAINERS index 3b9f5b7432..5eb65cc499 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -361,6 +361,7 @@ RISC-V OpenSBI domain support M: Gregor Haas L: qemu-riscv@nongnu.org S: Maintained +F: docs/system/riscv/opensbi_domains.rst F: hw/riscv/opensbi_domain.c F: include/hw/riscv/opensbi_domain.h diff --git a/docs/system/riscv/opensbi_domains.rst b/docs/system/riscv/opensbi_domains.rst new file mode 100644 index 0000000000..b8bbf52738 --- /dev/null +++ b/docs/system/riscv/opensbi_domains.rst @@ -0,0 +1,156 @@ +OpenSBI Domains +=============== + +OpenSBI has support for domains, which are partitions of CPUs and memory into +isolated compartments. Domains can be specified in the device tree according to +a standardized format [1_], which OpenSBI parses at boot time to initialize all +system domains. Depending on the specific QEMU machine being used, these domain +configurations can be specified on the QEMU command line. Currently, this is +only possible for the ``virt`` machine. To enable this functionality for a new +machine, the initialization code must call ``create_fdt_opensbi_domains`` after +all device tree nodes for peripherals have been initialized. This is to ensure +that references to devices work for MMIO regions. + +There are two "devices" that are used to configure OpenSBI domains with this +mechanism: ``opensbi-memregion`` and ``opensbi-domain``. + +Memregions +---------- + +OpenSBI memregions can be added to the machine's device tree with the following +flag: + +.. code-block:: bash + + -device opensbi-memregion + +For this device flag, the following options are implemented: + +- ``id``: The name of the memregion. This name is later used to link this + region to a domain if desired, in which case this argument is required. +- ``base`` (required): The base address of this memregion. This address must + be aligned to ``2 ^ order``. +- ``order`` (required): The ``log2`` of the memregion's size, which must be + between 3 and ``__riscv_xlen`` inclusive. +- ``mmio`` (optional): A boolean indicating whether the specified physical + address range belongs to an MMIO-mapped peripheral device. +- ``deviceX`` (optional): If ``mmio`` is indicated, this is the device tree + path to the ``X``-th device corresponding to this physical address range, + where ``0 <= X < OPENSBI_MEMREGION_DEVICES_MAX`` (default 16). + +Domains +------- + +OpenSBI domains can be added to the machine's device tree with the following +flag: + +.. code-block:: bash + + - device opensbi-domain + +For this device flag, the following options are implemented: + +- ``id`` (required): The name of the domain, which becomes its identifier in + the device tree +- ``boot-hart`` (optional): The HART booting the domain instance. +- ``possible-harts`` (optional): The contiguous list of CPUs for the domain + instance, specified as ``firstcpu[-lastcpu]`` (e.g. ``0-3``). +- ``next-arg1`` (optional): The 64 bit next booting stage arg1 for the domain + instance. +- ``next-addr`` (optional): The 64 bit next booting stage address for the + domain instance. +- ``next-mode`` (optional): The 32 bit next booting stage mode for the domain + instance. +- ``system-reset-allowed`` (optional): Whether the domain instance is allowed + to do system reset. +- ``system-suspend-allowed`` (optional): Whether the domain instance is allowed + to do system suspend. + +Furthermore, memregions can be linked to domains using the following options: + +- ``regionX`` (optional): The ``id`` of the ``X``-th region for this domain, + where ``0 <= X < OPENSBI_DOMAIN_MEMREGIONS_MAX`` (default 16). +- ``permsX`` (optional): Access permissions for the ``X``-th region for this + domain, ``0 <= X < OPENSBI_DOMAIN_MEMREGIONS_MAX`` (default 16). This must be + encoded using OpenSBI's permission encoding scheme in ``sbi_domain.h``, and + copied below at the time of writing for convenience + +.. code-block:: c + + /** Flags representing memory region attributes */ + #define SBI_MEMREGION_M_READABLE (1UL << 0) + #define SBI_MEMREGION_M_WRITABLE (1UL << 1) + #define SBI_MEMREGION_M_EXECUTABLE (1UL << 2) + #define SBI_MEMREGION_SU_READABLE (1UL << 3) + #define SBI_MEMREGION_SU_WRITABLE (1UL << 4) + #define SBI_MEMREGION_SU_EXECUTABLE (1UL << 5) + +Example +------- + +A complete example command line is shown below: + +.. code-block:: bash + + $ qemu-system-riscv64 -machine virt -bios fw_jump.bin -cpu max -smp 2 -m 4G -nographic \ + -device opensbi-memregion,id=mem,base=0xBC000000,order=26,mmio=false \ + -device opensbi-memregion,id=uart,base=0x10000000,order=12,mmio=true,device0="/soc/serial@10000000" \ + -device opensbi-domain,id=domain,possible-harts=0-1,boot-hart=0x0,next-addr=0xBC000000,next-mode=1,region0=mem,perms0=0x3f,region1=uart,perms1=0x3f + +As a result of the above configuration, QEMU will add the following subnodes to +the device tree: + +.. code-block:: dts + + chosen { + opensbi-domains { + compatible = "opensbi,domain,config"; + + domain { + next-mode = <0x01>; + next-addr = <0x00 0xbc000000>; + boot-hart = <0x03>; + regions = <0x8000 0x3f 0x8002 0x3f>; + possible-harts = <0x03 0x01>; + phandle = <0x8003>; + compatible = "opensbi,domain,instance"; + }; + + uart { + phandle = <0x8002>; + devices = <0x1800000>; + mmio; + order = <0x0c>; + base = <0x00 0x10000000>; + compatible = "opensbi,domain,memregion"; + }; + + mem { + phandle = <0x8000>; + order = <0x1a>; + base = <0x00 0xbc000000>; + compatible = "opensbi,domain,memregion"; + }; + }; + }; + +This results in OpenSBI output as below, where regions 01-03 are inherited from +the root domain and regions 00 and 04 correspond to the user specified ones: + +.. code-block:: console + + Domain1 Name : domain + Domain1 Boot HART : 0 + Domain1 HARTs : 0,1 + Domain1 Region00 : 0x0000000010000000-0x0000000010000fff M: (I,R,W,X) S/U: (R,W,X) + Domain1 Region01 : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: () + Domain1 Region02 : 0x0000000080080000-0x000000008009ffff M: (R,W) S/U: () + Domain1 Region03 : 0x0000000080000000-0x000000008007ffff M: (R,X) S/U: () + Domain1 Region04 : 0x00000000bc000000-0x00000000bfffffff M: (R,W,X) S/U: (R,W,X) + Domain1 Next Address : 0x00000000bc000000 + Domain1 Next Arg1 : 0x0000000000000000 + Domain1 Next Mode : S-mode + Domain1 SysReset : no + Domain1 SysSuspend : no + +.. _1: https://github.com/riscv-software-src/opensbi/blob/master/docs/domain_support.md diff --git a/docs/system/target-riscv.rst b/docs/system/target-riscv.rst index ba195f1518..bb776ea5f3 100644 --- a/docs/system/target-riscv.rst +++ b/docs/system/target-riscv.rst @@ -92,3 +92,13 @@ the images they need. * ``-bios `` Tells QEMU to load the specified file as the firmware. + +RISC-V Devices +-------------- + +There are some RISC-V specific devices that can be specified: + +.. toctree:: + :maxdepth: 1 + + riscv/opensbi_domains