From patchwork Thu May 28 13:29:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anup Patel X-Patchwork-Id: 11575847 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7DA39912 for ; Thu, 28 May 2020 13:33:05 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 442CF20721 for ; Thu, 28 May 2020 13:33:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="Qo0o1/7M"; dkim=pass (1024-bit key) header.d=sharedspace.onmicrosoft.com header.i=@sharedspace.onmicrosoft.com header.b="h5a1HAed" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 442CF20721 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=wdc.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:56304 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jeIee-0003Ke-Gw for patchwork-qemu-devel@patchwork.kernel.org; Thu, 28 May 2020 09:33:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42236) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jeIcH-0007fp-8u; Thu, 28 May 2020 09:30:38 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:30080) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jeIcD-0001G2-EQ; Thu, 28 May 2020 09:30:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1590672634; x=1622208634; h=from:to:cc:subject:date:message-id:in-reply-to: references:content-transfer-encoding:mime-version; bh=sSqGlJPyiLOwhchFkyjStDMtmdm0MCiEZbacK8pKJE0=; b=Qo0o1/7MmLd0wvTYdN3I1BHQK+bWBI1ufO/b7Ih9o6ial4GbCibSdieN dw0pRNYeNaABbU7KUs/8b5is1/22H/FsUa/rncHPlmc9UNqc93a6BGiIf tpkZutJKghYrEor8wOmq3FBZ8vzA6g5V7TqdQp6dF1DcM5svb9o1Ze+4Q 3hJHm21BDxUW2IKuRKPwv0HzwkoTeQ5X1T2QnakbixKI16IVz5J6oUpdt HU8YwwZKwbFMLUVY0RTZe8xJICYlG6CXlER2L1rpLMDvWoOwTJRxthFlz Za9Yj67PfX7b1nRYBumUcJfN0bUDSZbd1+r/1yqocO1fFAjjQIpc7ARbh g==; IronPort-SDR: 0GRbNwkfWkKwuOLxQAqpLAKoW8F+AyA3jLoE8UqATktkF3mN1vFJOyGrN8xZHTgtOLJxSQg/Pz 2zSpyfQOIT7XpNwmAJws09Nx3HqIyjm9M99GM8WdmyBziW63BWhomJVBh3yRCZY2Yt1iFRF/rm N5f92rjUrEqnj9XQFjwomUPK3FsS4Y+TCa7OulKNK7TZVQeY8DhAkqB6XFb49ApMXhI+6g0u7Y zHXroNq/hfQNS2El0+VRUVRIGx4blAO+zkMH5hYfN2YT5zAUZgNQ33P2qBMZS9UHCOMjuxUQrV f5s= X-IronPort-AV: E=Sophos;i="5.73,444,1583164800"; d="scan'208";a="143036553" Received: from mail-bn3nam04lp2053.outbound.protection.outlook.com (HELO NAM04-BN3-obe.outbound.protection.outlook.com) ([104.47.46.53]) by ob1.hgst.iphmx.com with ESMTP; 28 May 2020 21:30:31 +0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HnKTyQY4KNt398JSe72gsOHA2tUPREg0pNWdKaM4gJN8WtQIyxIIgZm9MAqkcDLKHGjmJAHhKBVcq95EtJpT1f5ZYj2eS70YHmPVrt7H2fCRE6y6+ysYBPZJys4NGQ5eKgJ7pnlxErL6Igz/GtKWz3el9eMgnvc8CkIP02NsJCdOIwiNT/dr7kQjtVjrBn/IG77zpeXxnGK5+CBr23R9wf0X5jG9gcxEli4I+rTIX7jif4YQ8h3ykyVCe9bFj5u3gnHMtgIr3v52x3KmTz7BBNd1iGQbA2YiXb4DSEHXhEwku/laSTAGIknVduEbbVrZT8PSrIDrtr7txyqQghF+rw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=30cdLqImbqcIFjUhNWNk2GJsaBjYaHjzY9RV4JtRyPU=; b=K43SsZUtJhe5flYv6peiI8l3SsO22huEONjkf8/9K+fc9HNJIH00XU/O4sP49MMyoZ/2JklwqcIbUGikXrovl/r2d1BWKRNbdgSwlM4lieFj/d+Ak9uK7zumWmxAUtjaAudOX0Gls96EqE8u7B8fCqFxkGK0aE46+ZxSldtUUGgSR31DDSQJzHVOprl+jisYh+VfNp8bhteytYHujobUWui/ssXgnrNaJ6eVuOf4ahR7Qga+PA0QZUkIq18Si2zINOfpc7q6W51oRnSqGGWG5fcBl5aqRsYmS3l/wEBYAbhRfL4aFckAxOP4bkDEJwVZgrLV8zoHF1DgqHXtsKbvHg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wdc.com; dmarc=pass action=none header.from=wdc.com; dkim=pass header.d=wdc.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sharedspace.onmicrosoft.com; s=selector2-sharedspace-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=30cdLqImbqcIFjUhNWNk2GJsaBjYaHjzY9RV4JtRyPU=; b=h5a1HAeda5i+7jZn9dkpCrxl9Y3vcVfcWtlAnO8W1HUqE1sApP9K5845fCyePvXi73XVhizq7WR6m13efiLGYy57rkxpV8zZQ7nhKQ3yqPArYxpnn/7Evta8VCTNBuFFormHa0tCdlsCbFXB56e8jmqlKNJLDCWFAJdd+gn5fGs= Authentication-Results: linaro.org; dkim=none (message not signed) header.d=none;linaro.org; dmarc=none action=none header.from=wdc.com; Received: from DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) by DM6PR04MB6204.namprd04.prod.outlook.com (2603:10b6:5:122::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.19; Thu, 28 May 2020 13:30:29 +0000 Received: from DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::f8b3:c124:482b:52e0]) by DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::f8b3:c124:482b:52e0%5]) with mapi id 15.20.3045.018; Thu, 28 May 2020 13:30:29 +0000 From: Anup Patel To: Peter Maydell , Palmer Dabbelt , Alistair Francis , Sagar Karandikar Subject: [PATCH v4 1/4] hw/riscv: Allow creating multiple instances of CLINT Date: Thu, 28 May 2020 18:59:56 +0530 Message-Id: <20200528132959.47773-2-anup.patel@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200528132959.47773-1-anup.patel@wdc.com> References: <20200528132959.47773-1-anup.patel@wdc.com> X-ClientProxiedBy: MAXPR01CA0094.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:49::36) To DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from wdc.com (49.207.63.107) by MAXPR01CA0094.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:49::36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.18 via Frontend Transport; Thu, 28 May 2020 13:30:25 +0000 X-Mailer: git-send-email 2.25.1 X-Originating-IP: [49.207.63.107] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: f900eb52-f560-4796-2392-08d8030b49ae X-MS-TrafficTypeDiagnostic: DM6PR04MB6204: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: WDCIPOUTBOUND: EOP-TRUE X-MS-Oob-TLC-OOBClassifiers: OLM:3; X-Forefront-PRVS: 0417A3FFD2 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: A9UemQtFO+apGkBSQgc5QDZADuzwQ4W3oY5clWa8fksIKMTF5i/aMB8lk5aGgXbkVno1fmHq8RkUgHiRqb3G7IW4U1aAApYaiOsWwudHGUjIkq6pDlfevmBSGU0OKLBYS2r3vsJIcS5v1yjo7tmvuPf9snr3BaK9ZKrFNofgNjo/NQWGupF7+d6VKQOn90/Jg7EMMpVtxU4QlDvIEwVCC0YILm/Lmz4xOhZbkWbZAPABbSuxuocNIyxA3NWFUvId4oQI/+VZKhcWvBbMZHZWVKtiJUoeNUaS9LnM+g9WaOHZhYlHZeas4qQ+x2vKsBEZ/lPz6ic0DsLNi/K0U4yTAA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR04MB6201.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(396003)(346002)(366004)(376002)(39860400002)(956004)(26005)(55016002)(44832011)(478600001)(55236004)(2616005)(5660300002)(7696005)(16526019)(52116002)(186003)(8936002)(8676002)(1076003)(86362001)(8886007)(66556008)(66946007)(66476007)(316002)(83380400001)(4326008)(54906003)(110136005)(36756003)(6666004)(1006002)(2906002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: Ab12aRv3R+q28FhvNyGAZrZlKW2UhidyS4KwEPvg8M/DZoZVpZQ3T12fqjxxXs9Aq1dhigq2fmcvRRyJ1MzbVSfWWC8ZE0cTRKXGZiTza0vTxqF3c8nb3hEnY/+ivEAEfUCRyezRb+ISig7LtMGoUlJJEHZqDYgsFrfrBpStMwH9X6DGLCtozQJk44xY9mmVRpD8P7/qIYUfV3n/Fe/NCaUCfLOr8Qm2fJOJQE/ToOBM6P4BKt4gmgw3eqKJ+0sRkn88kxTE+jY4ye/hpdfZyFFbMT7gFP/JLi6JAjEJNH7Qg5goc8hMrU3mr4hUXVNj3FkFEIjHbl82nZn78L5PRF3VdObpJHUJaiQohxkwRVBfrEjt0tAg/5I+Oql9ueftE/Rn/fna7LBYq9lQ9GusSXUftmkYJ5eBQKhEHSvWTdqQ5yAuMpCbhHt+T13U1sNblqsgPbWwq+jvQSUHbf1/A5LZOMWIwJlWtgtKp5vvK6I= X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-Network-Message-Id: f900eb52-f560-4796-2392-08d8030b49ae X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 May 2020 13:30:29.1885 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: zBcnav+3qXZca2L3r/M/A5yqBD67jaXAq3Bw/SYTuO9nPC7Eb9A82GMEn0dld5PtH3r/If4TymWdz6ZNRqfBHA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR04MB6204 Received-SPF: pass client-ip=216.71.153.141; envelope-from=prvs=41059f22b=Anup.Patel@wdc.com; helo=esa3.hgst.iphmx.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/28 09:30:28 X-ACL-Warn: Detected OS = FreeBSD 9.x or newer [fuzzy] X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Palmer Dabbelt , qemu-riscv@nongnu.org, Anup Patel , Anup Patel , qemu-devel@nongnu.org, Atish Patra , Alistair Francis Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We extend CLINT emulation to allow multiple instances of CLINT in a QEMU RISC-V machine. To achieve this, we remove first HART id zero assumption from CLINT emulation. Signed-off-by: Anup Patel Reviewed-by: Alistair Francis Reviewed-by: Palmer Dabbelt --- hw/riscv/sifive_clint.c | 20 ++++++++++++-------- hw/riscv/sifive_e.c | 2 +- hw/riscv/sifive_u.c | 2 +- hw/riscv/spike.c | 6 +++--- hw/riscv/virt.c | 2 +- include/hw/riscv/sifive_clint.h | 7 ++++--- 6 files changed, 22 insertions(+), 17 deletions(-) diff --git a/hw/riscv/sifive_clint.c b/hw/riscv/sifive_clint.c index e933d35092..7d713fd743 100644 --- a/hw/riscv/sifive_clint.c +++ b/hw/riscv/sifive_clint.c @@ -78,7 +78,7 @@ static uint64_t sifive_clint_read(void *opaque, hwaddr addr, unsigned size) SiFiveCLINTState *clint = opaque; if (addr >= clint->sip_base && addr < clint->sip_base + (clint->num_harts << 2)) { - size_t hartid = (addr - clint->sip_base) >> 2; + size_t hartid = clint->hartid_base + ((addr - clint->sip_base) >> 2); CPUState *cpu = qemu_get_cpu(hartid); CPURISCVState *env = cpu ? cpu->env_ptr : NULL; if (!env) { @@ -91,7 +91,8 @@ static uint64_t sifive_clint_read(void *opaque, hwaddr addr, unsigned size) } } else if (addr >= clint->timecmp_base && addr < clint->timecmp_base + (clint->num_harts << 3)) { - size_t hartid = (addr - clint->timecmp_base) >> 3; + size_t hartid = clint->hartid_base + + ((addr - clint->timecmp_base) >> 3); CPUState *cpu = qemu_get_cpu(hartid); CPURISCVState *env = cpu ? cpu->env_ptr : NULL; if (!env) { @@ -128,7 +129,7 @@ static void sifive_clint_write(void *opaque, hwaddr addr, uint64_t value, if (addr >= clint->sip_base && addr < clint->sip_base + (clint->num_harts << 2)) { - size_t hartid = (addr - clint->sip_base) >> 2; + size_t hartid = clint->hartid_base + ((addr - clint->sip_base) >> 2); CPUState *cpu = qemu_get_cpu(hartid); CPURISCVState *env = cpu ? cpu->env_ptr : NULL; if (!env) { @@ -141,7 +142,8 @@ static void sifive_clint_write(void *opaque, hwaddr addr, uint64_t value, return; } else if (addr >= clint->timecmp_base && addr < clint->timecmp_base + (clint->num_harts << 3)) { - size_t hartid = (addr - clint->timecmp_base) >> 3; + size_t hartid = clint->hartid_base + + ((addr - clint->timecmp_base) >> 3); CPUState *cpu = qemu_get_cpu(hartid); CPURISCVState *env = cpu ? cpu->env_ptr : NULL; if (!env) { @@ -185,6 +187,7 @@ static const MemoryRegionOps sifive_clint_ops = { }; static Property sifive_clint_properties[] = { + DEFINE_PROP_UINT32("hartid-base", SiFiveCLINTState, hartid_base, 0), DEFINE_PROP_UINT32("num-harts", SiFiveCLINTState, num_harts, 0), DEFINE_PROP_UINT32("sip-base", SiFiveCLINTState, sip_base, 0), DEFINE_PROP_UINT32("timecmp-base", SiFiveCLINTState, timecmp_base, 0), @@ -226,13 +229,13 @@ type_init(sifive_clint_register_types) /* * Create CLINT device. */ -DeviceState *sifive_clint_create(hwaddr addr, hwaddr size, uint32_t num_harts, - uint32_t sip_base, uint32_t timecmp_base, uint32_t time_base, - bool provide_rdtime) +DeviceState *sifive_clint_create(hwaddr addr, hwaddr size, + uint32_t hartid_base, uint32_t num_harts, uint32_t sip_base, + uint32_t timecmp_base, uint32_t time_base, bool provide_rdtime) { int i; for (i = 0; i < num_harts; i++) { - CPUState *cpu = qemu_get_cpu(i); + CPUState *cpu = qemu_get_cpu(hartid_base + i); CPURISCVState *env = cpu ? cpu->env_ptr : NULL; if (!env) { continue; @@ -246,6 +249,7 @@ DeviceState *sifive_clint_create(hwaddr addr, hwaddr size, uint32_t num_harts, } DeviceState *dev = qdev_create(NULL, TYPE_SIFIVE_CLINT); + qdev_prop_set_uint32(dev, "hartid-base", hartid_base); qdev_prop_set_uint32(dev, "num-harts", num_harts); qdev_prop_set_uint32(dev, "sip-base", sip_base); qdev_prop_set_uint32(dev, "timecmp-base", timecmp_base); diff --git a/hw/riscv/sifive_e.c b/hw/riscv/sifive_e.c index b53109521e..1c3b37d0ba 100644 --- a/hw/riscv/sifive_e.c +++ b/hw/riscv/sifive_e.c @@ -163,7 +163,7 @@ static void riscv_sifive_e_soc_realize(DeviceState *dev, Error **errp) SIFIVE_E_PLIC_CONTEXT_STRIDE, memmap[SIFIVE_E_PLIC].size); sifive_clint_create(memmap[SIFIVE_E_CLINT].base, - memmap[SIFIVE_E_CLINT].size, ms->smp.cpus, + memmap[SIFIVE_E_CLINT].size, 0, ms->smp.cpus, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, false); create_unimplemented_device("riscv.sifive.e.aon", memmap[SIFIVE_E_AON].base, memmap[SIFIVE_E_AON].size); diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c index 4299bdf480..c193761916 100644 --- a/hw/riscv/sifive_u.c +++ b/hw/riscv/sifive_u.c @@ -602,7 +602,7 @@ static void riscv_sifive_u_soc_realize(DeviceState *dev, Error **errp) sifive_uart_create(system_memory, memmap[SIFIVE_U_UART1].base, serial_hd(1), qdev_get_gpio_in(DEVICE(s->plic), SIFIVE_U_UART1_IRQ)); sifive_clint_create(memmap[SIFIVE_U_CLINT].base, - memmap[SIFIVE_U_CLINT].size, ms->smp.cpus, + memmap[SIFIVE_U_CLINT].size, 0, ms->smp.cpus, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, false); object_property_set_bool(OBJECT(&s->prci), true, "realized", &err); diff --git a/hw/riscv/spike.c b/hw/riscv/spike.c index d0c4843712..d5e0103d89 100644 --- a/hw/riscv/spike.c +++ b/hw/riscv/spike.c @@ -253,7 +253,7 @@ static void spike_board_init(MachineState *machine) /* Core Local Interruptor (timer and IPI) */ sifive_clint_create(memmap[SPIKE_CLINT].base, memmap[SPIKE_CLINT].size, - smp_cpus, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, + 0, smp_cpus, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, false); } @@ -343,7 +343,7 @@ static void spike_v1_10_0_board_init(MachineState *machine) /* Core Local Interruptor (timer and IPI) */ sifive_clint_create(memmap[SPIKE_CLINT].base, memmap[SPIKE_CLINT].size, - smp_cpus, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, + 0, smp_cpus, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, false); } @@ -452,7 +452,7 @@ static void spike_v1_09_1_board_init(MachineState *machine) /* Core Local Interruptor (timer and IPI) */ sifive_clint_create(memmap[SPIKE_CLINT].base, memmap[SPIKE_CLINT].size, - smp_cpus, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, + 0, smp_cpus, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, false); g_free(config_string); diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index c695a44979..51afe7e23b 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -595,7 +595,7 @@ static void riscv_virt_board_init(MachineState *machine) VIRT_PLIC_CONTEXT_STRIDE, memmap[VIRT_PLIC].size); sifive_clint_create(memmap[VIRT_CLINT].base, - memmap[VIRT_CLINT].size, smp_cpus, + memmap[VIRT_CLINT].size, 0, smp_cpus, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, true); sifive_test_create(memmap[VIRT_TEST].base); diff --git a/include/hw/riscv/sifive_clint.h b/include/hw/riscv/sifive_clint.h index 4a720bfece..9f5fb3d31d 100644 --- a/include/hw/riscv/sifive_clint.h +++ b/include/hw/riscv/sifive_clint.h @@ -33,6 +33,7 @@ typedef struct SiFiveCLINTState { /*< public >*/ MemoryRegion mmio; + uint32_t hartid_base; uint32_t num_harts; uint32_t sip_base; uint32_t timecmp_base; @@ -40,9 +41,9 @@ typedef struct SiFiveCLINTState { uint32_t aperture_size; } SiFiveCLINTState; -DeviceState *sifive_clint_create(hwaddr addr, hwaddr size, uint32_t num_harts, - uint32_t sip_base, uint32_t timecmp_base, uint32_t time_base, - bool provide_rdtime); +DeviceState *sifive_clint_create(hwaddr addr, hwaddr size, + uint32_t hartid_base, uint32_t num_harts, uint32_t sip_base, + uint32_t timecmp_base, uint32_t time_base, bool provide_rdtime); enum { SIFIVE_SIP_BASE = 0x0, From patchwork Thu May 28 13:29:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anup Patel X-Patchwork-Id: 11575845 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9975B912 for ; Thu, 28 May 2020 13:31:44 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5A1FF20721 for ; Thu, 28 May 2020 13:31:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="EFa6ssGp"; dkim=pass (1024-bit key) header.d=sharedspace.onmicrosoft.com header.i=@sharedspace.onmicrosoft.com header.b="E/kQpaCN" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5A1FF20721 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=wdc.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:51866 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jeIdL-0001Oo-A8 for patchwork-qemu-devel@patchwork.kernel.org; Thu, 28 May 2020 09:31:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42244) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jeIcJ-0007gJ-8r; Thu, 28 May 2020 09:30:40 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:30070) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jeIcG-0001CO-5x; Thu, 28 May 2020 09:30:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1590672636; x=1622208636; h=from:to:cc:subject:date:message-id:in-reply-to: references:content-transfer-encoding:mime-version; bh=esvbG8mwyfuf8XW2HxsiBnuLQxTMUq0Obayx/hsdVcc=; b=EFa6ssGpsRMB82oUBx16cUu+SZl6B+eW8AGF6oAFVyk9hKKuJLNJcK2x uY3uQhsHFM6zIzWj2Vo6Z9/yf2bWab94yuIdWekl29jkiYhYziF1UWJSU TPgLDC1wxsLQfc5CCXdWd3ujxsYWNPhQ8pgi5KAdmFYlw0pujFQxZLC72 qf3bQkDFfMjSXqB7KvGVZDJfn8e2OB5PeNU2kJHXW+WAb2hg3s1E6Rkil tfuXCtL8/It9DeDqoYn6Hkb4nmp41lxrVBM2bHiAu5/GiEZufClk93728 vIq8wpF8z0oeRdamFjDJ9IA5xdWZHQv0pVoTWv4mGndWqpusKd6HL59aR Q==; IronPort-SDR: h0lHD64mLzfI/CI97oGk1VkX5vz2xwfvhNqazgpEiQIt9hs9ngsH0SUFvwQt+goRcttd+RAiwV ZEmyXcoTkNkEsxW0XiOMiQj/wdVIjSWdjXxY9La01WXob5aVTkqCjI53leDokI+TR6kRZSZCt+ PM6kS0sm4wURAEu382Ak2vwOT4bZ6ZNY01XLs9uq56WZfa3jF8wNDg+rxlGrFT4Obor4KTPbld EwWs1/dqxwiztbpKTi/GndkEvM4dSIjlgOhvB50nfmAPvRNYF4xPeWYfr2N/uKnv+wvMpBDkVP pbM= X-IronPort-AV: E=Sophos;i="5.73,444,1583164800"; d="scan'208";a="143036560" Received: from mail-bn3nam04lp2051.outbound.protection.outlook.com (HELO NAM04-BN3-obe.outbound.protection.outlook.com) ([104.47.46.51]) by ob1.hgst.iphmx.com with ESMTP; 28 May 2020 21:30:35 +0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dy2T+BnifTq3OpRk2ANUmq9VVHHdTtEEyc9eXQtg4PaTYZ1f6csOpbU7CoqohVUJi6NY/LlxYQ61dUQyThfXQGdMhOcAYqMq44p5VlmAK/IHVvLZxO8JL795mS9LIU0amPWWvWC1H8UMqltB/GZkzfaEG9uSDRTTM6Mf3v3SWaTVKbtHGfKPACPNct0ZXDG/tDyp7ePN+4l0bse658503kc/wh2LvI3xwViCvRHyXF86BlZ23qBuArYdPob0mokTIl1VIjfUKdXud5WUM2Jq4iQoq2fr5lpIzEMs1qw/0upeKbdQUCXN6Q+8BH1NlYYEvYfHpjf+UPKhAq1GcuKgLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SWS+d247xFZXDb66J8uxkh46EXespExk7s1Y15FDEf0=; b=S4ZlJHHOgt2mm+vMHuAnjCWmqUGevK3I3cVngBirnCmeyrsuvIrOPMfKXKBWmgC5nk4IHsNYjv3O7IUgjZfDPnF2MzH9J+C67inqbnD4hjJ/KXd1rm3QGbtYRZmPdYpyj9v8skJb49EOwx9HLLbgb38omWnZjDTnTblYiOk92JOv9vw/wdVkeUtJu45uw8jC05dmTmjY5LnQgbNFbshOjrgJZP/sSHwCUf5MYG3dpKZ0FPIqYDnpOKmZHPMKOLzdtbdxnyRdJKDm9vOCjW3GRf/iarVSLWjvJenLUKg1IT7P93uwj2Tpjuib5ZnkyctXV1i/FeKqlJBZ7EwtbhX/Aw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wdc.com; dmarc=pass action=none header.from=wdc.com; dkim=pass header.d=wdc.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sharedspace.onmicrosoft.com; s=selector2-sharedspace-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SWS+d247xFZXDb66J8uxkh46EXespExk7s1Y15FDEf0=; b=E/kQpaCNDIDCXFAGflyEvw/Ugohdz1Cx1Pk3PXD+MYQTkR8yuAVZUeaeu7IPkb62p0HkPB15nkVzo42GFqyGXZvonCMlztEAE3XQD4DgMghjqB+ZocWaff/T1bjXzcpHwM8z98erO5bGWKULcjwYFtE5ZRy4+1ntkpGq+35OKK4= Authentication-Results: linaro.org; dkim=none (message not signed) header.d=none;linaro.org; dmarc=none action=none header.from=wdc.com; Received: from DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) by DM6PR04MB6204.namprd04.prod.outlook.com (2603:10b6:5:122::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.19; Thu, 28 May 2020 13:30:32 +0000 Received: from DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::f8b3:c124:482b:52e0]) by DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::f8b3:c124:482b:52e0%5]) with mapi id 15.20.3045.018; Thu, 28 May 2020 13:30:32 +0000 From: Anup Patel To: Peter Maydell , Palmer Dabbelt , Alistair Francis , Sagar Karandikar Subject: [PATCH v4 2/4] hw/riscv: spike: Allow creating multiple NUMA sockets Date: Thu, 28 May 2020 18:59:57 +0530 Message-Id: <20200528132959.47773-3-anup.patel@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200528132959.47773-1-anup.patel@wdc.com> References: <20200528132959.47773-1-anup.patel@wdc.com> X-ClientProxiedBy: MAXPR01CA0094.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:49::36) To DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from wdc.com (49.207.63.107) by MAXPR01CA0094.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:49::36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.18 via Frontend Transport; Thu, 28 May 2020 13:30:29 +0000 X-Mailer: git-send-email 2.25.1 X-Originating-IP: [49.207.63.107] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: f3ccc6c3-b88a-49ce-2e3c-08d8030b4bbb X-MS-TrafficTypeDiagnostic: DM6PR04MB6204: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: WDCIPOUTBOUND: EOP-TRUE X-MS-Oob-TLC-OOBClassifiers: OLM:517; X-Forefront-PRVS: 0417A3FFD2 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: P8Guwp6ibPXCXHW+pRyIG8As6HUsNIJLPYwHZp3lW4qx6Y31Z4sLR9MGJLCUCV867OLfwL5XOeN6WtiTucX6sc3DUDCRz/me6DJWil+cBVHJ691/1f9Tcz3oJT/Y2bIQB1Zspel1Fz82lw5RwSguX+Iem19r8muOc+5zHoH45904XKT9xSGKeeuQ/iIP0gIw6NmwYYcCT72dKhJ/ONqg9ATA6D/Rf4cGhXLBaEEOnONAgZ4Q4elKEJDSutp2HR5juFkGh4WmCpDi2KTzQ5LgJL9gBB4UHmpp1wsvJPPoVxt+HbJoK8UhBBLKBly1PPEg X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR04MB6201.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(396003)(346002)(366004)(376002)(39860400002)(956004)(26005)(55016002)(44832011)(478600001)(55236004)(2616005)(5660300002)(7696005)(16526019)(52116002)(186003)(8936002)(30864003)(8676002)(1076003)(86362001)(8886007)(66556008)(66946007)(66476007)(316002)(83380400001)(4326008)(54906003)(110136005)(36756003)(6666004)(1006002)(2906002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: OayT0MCLvKPiKM/1BHQ6x531VnAe0NPyfiMsxBySv4KjHl1OvodQmkKbNPHUDJgQecc95el8Dyxo4+NQSMRy8ewD/j19DAlDhrjIbmdiF+15m8hpcG6F7WHiThy38pMK1Pbw0K80OJhhMwWH9DrHIcWL+V0LWhK5mZF/F1VmTQBcZtckAmY8CCuIm8SXphlBkoSbmUGFgxehPjo1AXumuZiu/4HABbWg5HTzKtuDbsli0MMEfOTxy5AcrRwe+AwVgnaHpwhe/eyiZ2XebBkCmnK9ceVqh0h0XiYHrWKbVpQvBLysi3kuASunuVjXAPkMkpL0gcLCWT6Pgc/lIFCdZoCzZDvYB4Yb70a26/ihLq0CzGOdaKDII/CIkEA6A/gYxMxpJkuA5t1sp3EsO/CnkFHwFps7B/IkSqJ1ZGPILzZ0zZXP2rXv3u0XJxQoE03w/Di55RmAtTNJv0Z6ipy9t8jlN2+u+EMTMTCCS0lOTlM= X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-Network-Message-Id: f3ccc6c3-b88a-49ce-2e3c-08d8030b4bbb X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 May 2020 13:30:32.7674 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hz3NIwhAMaoZf5BuHV2iP1hdXsg2wU6QjMtT0jf28kgrrApH3mPMcAEJt7mL/KhIEmDrWer16dZyYjs4Y2cPKg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR04MB6204 Received-SPF: pass client-ip=216.71.153.141; envelope-from=prvs=41059f22b=Anup.Patel@wdc.com; helo=esa3.hgst.iphmx.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/28 09:30:28 X-ACL-Warn: Detected OS = FreeBSD 9.x or newer [fuzzy] X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atish Patra , Anup Patel , qemu-riscv@nongnu.org, qemu-devel@nongnu.org, Anup Patel Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We extend RISC-V spike machine to allow creating a multi-socket machine. Each RISC-V spike machine socket is a NUMA node having a set of HARTs, a memory instance, and a CLINT instance. Other devices are shared between all RISC-V spike machine sockets. We also update the generated device tree accordingly. By default, NUMA multi-socket support is disabled for RISC-V spike machine. To enable it, users can use "-numa" command-line options of QEMU. Example1: For two NUMA nodes with 4 CPUs each, append following to command-line options: "-smp 8 -numa node -numa node" Example2: For two NUMA nodes with 3 and 5 CPUs, append following to command-line options: "-smp 8 -numa node,cpus=0-2 -numa node,3-7" Signed-off-by: Anup Patel --- hw/riscv/spike.c | 381 +++++++++++++++++++++++++++++---------- include/hw/riscv/spike.h | 15 +- 2 files changed, 296 insertions(+), 100 deletions(-) diff --git a/hw/riscv/spike.c b/hw/riscv/spike.c index d5e0103d89..b1923442b6 100644 --- a/hw/riscv/spike.c +++ b/hw/riscv/spike.c @@ -64,9 +64,14 @@ static void create_fdt(SpikeState *s, const struct MemmapEntry *memmap, uint64_t mem_size, const char *cmdline) { void *fdt; - int cpu; - uint32_t *cells; - char *nodename; + unsigned long clint_addr; + int i, j, idx, cpu, socket; + MachineState *mc = MACHINE(s); + uint32_t dist_matrix_size; + uint32_t *clint_cells, *dist_matrix; + uint32_t cpu_phandle, intc_phandle, phandle = 1; + char *name, *mem_name, *clint_name, *clust_name; + char *core_name, *cpu_name, *intc_name; fdt = s->fdt = create_device_tree(&s->fdt_size); if (!fdt) { @@ -88,68 +93,118 @@ static void create_fdt(SpikeState *s, const struct MemmapEntry *memmap, qemu_fdt_setprop_cell(fdt, "/soc", "#size-cells", 0x2); qemu_fdt_setprop_cell(fdt, "/soc", "#address-cells", 0x2); - nodename = g_strdup_printf("/memory@%lx", - (long)memmap[SPIKE_DRAM].base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_cells(fdt, nodename, "reg", - memmap[SPIKE_DRAM].base >> 32, memmap[SPIKE_DRAM].base, - mem_size >> 32, mem_size); - qemu_fdt_setprop_string(fdt, nodename, "device_type", "memory"); - g_free(nodename); - qemu_fdt_add_subnode(fdt, "/cpus"); qemu_fdt_setprop_cell(fdt, "/cpus", "timebase-frequency", SIFIVE_CLINT_TIMEBASE_FREQ); qemu_fdt_setprop_cell(fdt, "/cpus", "#size-cells", 0x0); qemu_fdt_setprop_cell(fdt, "/cpus", "#address-cells", 0x1); + qemu_fdt_add_subnode(fdt, "/cpus/cpu-map"); + + for (socket = (s->num_socs - 1); socket >= 0; socket--) { + clust_name = g_strdup_printf("/cpus/cpu-map/cluster%d", socket); + qemu_fdt_add_subnode(fdt, clust_name); + + clint_cells = g_new0(uint32_t, s->soc[socket].num_harts * 4); - for (cpu = s->soc.num_harts - 1; cpu >= 0; cpu--) { - nodename = g_strdup_printf("/cpus/cpu@%d", cpu); - char *intc = g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu); - char *isa = riscv_isa_string(&s->soc.harts[cpu]); - qemu_fdt_add_subnode(fdt, nodename); + for (cpu = s->soc[socket].num_harts - 1; cpu >= 0; cpu--) { + cpu_phandle = phandle++; + + cpu_name = g_strdup_printf("/cpus/cpu@%d", + s->soc[socket].hartid_base + cpu); + qemu_fdt_add_subnode(fdt, cpu_name); #if defined(TARGET_RISCV32) - qemu_fdt_setprop_string(fdt, nodename, "mmu-type", "riscv,sv32"); + qemu_fdt_setprop_string(fdt, cpu_name, "mmu-type", "riscv,sv32"); #else - qemu_fdt_setprop_string(fdt, nodename, "mmu-type", "riscv,sv48"); + qemu_fdt_setprop_string(fdt, cpu_name, "mmu-type", "riscv,sv48"); #endif - qemu_fdt_setprop_string(fdt, nodename, "riscv,isa", isa); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "riscv"); - qemu_fdt_setprop_string(fdt, nodename, "status", "okay"); - qemu_fdt_setprop_cell(fdt, nodename, "reg", cpu); - qemu_fdt_setprop_string(fdt, nodename, "device_type", "cpu"); - qemu_fdt_add_subnode(fdt, intc); - qemu_fdt_setprop_cell(fdt, intc, "phandle", 1); - qemu_fdt_setprop_string(fdt, intc, "compatible", "riscv,cpu-intc"); - qemu_fdt_setprop(fdt, intc, "interrupt-controller", NULL, 0); - qemu_fdt_setprop_cell(fdt, intc, "#interrupt-cells", 1); - g_free(isa); - g_free(intc); - g_free(nodename); + name = riscv_isa_string(&s->soc[socket].harts[cpu]); + qemu_fdt_setprop_string(fdt, cpu_name, "riscv,isa", name); + g_free(name); + qemu_fdt_setprop_string(fdt, cpu_name, "compatible", "riscv"); + qemu_fdt_setprop_string(fdt, cpu_name, "status", "okay"); + qemu_fdt_setprop_cell(fdt, cpu_name, "reg", + s->soc[socket].hartid_base + cpu); + qemu_fdt_setprop_string(fdt, cpu_name, "device_type", "cpu"); + if (s->numa_enabled) { + qemu_fdt_setprop_cell(fdt, cpu_name, "numa-node-id", socket); + } + qemu_fdt_setprop_cell(fdt, cpu_name, "phandle", cpu_phandle); + + intc_name = g_strdup_printf("%s/interrupt-controller", cpu_name); + qemu_fdt_add_subnode(fdt, intc_name); + intc_phandle = phandle++; + qemu_fdt_setprop_cell(fdt, intc_name, "phandle", intc_phandle); + qemu_fdt_setprop_string(fdt, intc_name, "compatible", + "riscv,cpu-intc"); + qemu_fdt_setprop(fdt, intc_name, "interrupt-controller", NULL, 0); + qemu_fdt_setprop_cell(fdt, intc_name, "#interrupt-cells", 1); + + clint_cells[cpu * 4 + 0] = cpu_to_be32(intc_phandle); + clint_cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_SOFT); + clint_cells[cpu * 4 + 2] = cpu_to_be32(intc_phandle); + clint_cells[cpu * 4 + 3] = cpu_to_be32(IRQ_M_TIMER); + + core_name = g_strdup_printf("%s/core%d", clust_name, cpu); + qemu_fdt_add_subnode(fdt, core_name); + qemu_fdt_setprop_cell(fdt, core_name, "cpu", cpu_phandle); + + g_free(core_name); + g_free(intc_name); + g_free(cpu_name); + } + + mem_name = g_strdup_printf("/memory@%lx", + (long)memmap[SPIKE_DRAM].base + s->mem_offset[socket]); + qemu_fdt_add_subnode(fdt, mem_name); + qemu_fdt_setprop_cells(fdt, mem_name, "reg", + (memmap[SPIKE_DRAM].base + s->mem_offset[socket]) >> 32, + memmap[SPIKE_DRAM].base + s->mem_offset[socket], + s->mem_size[socket] >> 32, s->mem_size[socket]); + qemu_fdt_setprop_string(fdt, mem_name, "device_type", "memory"); + if (s->numa_enabled) { + qemu_fdt_setprop_cell(fdt, mem_name, "numa-node-id", socket); + } + g_free(mem_name); + + clint_addr = memmap[SPIKE_CLINT].base + + (memmap[SPIKE_CLINT].size * socket); + clint_name = g_strdup_printf("/soc/clint@%lx", clint_addr); + qemu_fdt_add_subnode(fdt, clint_name); + qemu_fdt_setprop_string(fdt, clint_name, "compatible", "riscv,clint0"); + qemu_fdt_setprop_cells(fdt, clint_name, "reg", + 0x0, clint_addr, 0x0, memmap[SPIKE_CLINT].size); + qemu_fdt_setprop(fdt, clint_name, "interrupts-extended", + clint_cells, s->soc[socket].num_harts * sizeof(uint32_t) * 4); + if (s->numa_enabled) { + qemu_fdt_setprop_cell(fdt, clint_name, "numa-node-id", socket); + } + + g_free(clint_name); + g_free(clint_cells); + g_free(clust_name); } - cells = g_new0(uint32_t, s->soc.num_harts * 4); - for (cpu = 0; cpu < s->soc.num_harts; cpu++) { - nodename = - g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu); - uint32_t intc_phandle = qemu_fdt_get_phandle(fdt, nodename); - cells[cpu * 4 + 0] = cpu_to_be32(intc_phandle); - cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_SOFT); - cells[cpu * 4 + 2] = cpu_to_be32(intc_phandle); - cells[cpu * 4 + 3] = cpu_to_be32(IRQ_M_TIMER); - g_free(nodename); + if (s->numa_enabled && mc->numa_state->have_numa_distance) { + dist_matrix_size = s->num_socs * s->num_socs * 3 * sizeof(uint32_t); + dist_matrix = g_malloc0(dist_matrix_size); + + for (i = 0; i < s->num_socs; i++) { + for (j = 0; j < s->num_socs; j++) { + idx = (i * s->num_socs + j) * 3; + dist_matrix[idx + 0] = cpu_to_be32(i); + dist_matrix[idx + 1] = cpu_to_be32(j); + dist_matrix[idx + 2] = + cpu_to_be32(mc->numa_state->nodes[i].distance[j]); + } + } + + qemu_fdt_add_subnode(fdt, "/distance-map"); + qemu_fdt_setprop_string(fdt, "/distance-map", "compatible", + "numa-distance-map-v1"); + qemu_fdt_setprop(fdt, "/distance-map", "distance-matrix", + dist_matrix, dist_matrix_size); + g_free(dist_matrix); } - nodename = g_strdup_printf("/soc/clint@%lx", - (long)memmap[SPIKE_CLINT].base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "riscv,clint0"); - qemu_fdt_setprop_cells(fdt, nodename, "reg", - 0x0, memmap[SPIKE_CLINT].base, - 0x0, memmap[SPIKE_CLINT].size); - qemu_fdt_setprop(fdt, nodename, "interrupts-extended", - cells, s->soc.num_harts * sizeof(uint32_t) * 4); - g_free(cells); - g_free(nodename); if (cmdline) { qemu_fdt_add_subnode(fdt, "/chosen"); @@ -157,35 +212,82 @@ static void create_fdt(SpikeState *s, const struct MemmapEntry *memmap, } } -static void spike_board_init(MachineState *machine) +static void spike_board_init(MachineState *mc) { const struct MemmapEntry *memmap = spike_memmap; - - SpikeState *s = g_new0(SpikeState, 1); + SpikeState *s = SPIKE_MACHINE(mc); MemoryRegion *system_memory = get_system_memory(); MemoryRegion *main_mem = g_new(MemoryRegion, 1); MemoryRegion *mask_rom = g_new(MemoryRegion, 1); - int i; - unsigned int smp_cpus = machine->smp.cpus; + char *soc_name; + unsigned int i, j, base_hartid, hart_count; + uint64_t mem_offset; + + /* Populate socket details based on NUMA options */ + s->num_socs = mc->numa_state->num_nodes; + if (!s->num_socs) { + s->numa_enabled = false; + s->num_socs = 1; + s->mem_offset[0] = 0; + s->mem_size[0] = mc->ram_size; + } else { + s->numa_enabled = true; + s->num_socs = mc->numa_state->num_nodes; + mem_offset = 0; + for (i = 0; i < s->num_socs; i++) { + s->mem_offset[i] = mem_offset; + s->mem_size[i] = mc->numa_state->nodes[i].node_mem; + mem_offset += s->mem_size[i]; + } + } - /* Initialize SOC */ - object_initialize_child(OBJECT(machine), "soc", &s->soc, sizeof(s->soc), - TYPE_RISCV_HART_ARRAY, &error_abort, NULL); - object_property_set_str(OBJECT(&s->soc), machine->cpu_type, "cpu-type", - &error_abort); - object_property_set_int(OBJECT(&s->soc), smp_cpus, "num-harts", - &error_abort); - object_property_set_bool(OBJECT(&s->soc), true, "realized", - &error_abort); + /* Initialize sockets */ + for (i = 0; i < s->num_socs; i++) { + if (s->numa_enabled) { + base_hartid = mc->smp.cpus; + hart_count = 0; + for (j = 0; j < mc->smp.cpus; j++) { + if (mc->possible_cpus->cpus[j].props.node_id != i) { + continue; + } + if (j < base_hartid) { + base_hartid = j; + } + hart_count++; + } + } else { + base_hartid = 0; + hart_count = mc->smp.cpus; + } + + soc_name = g_strdup_printf("soc%d", i); + object_initialize_child(OBJECT(mc), soc_name, &s->soc[i], + sizeof(s->soc[i]), TYPE_RISCV_HART_ARRAY, &error_abort, NULL); + g_free(soc_name); + object_property_set_str(OBJECT(&s->soc[i]), + mc->cpu_type, "cpu-type", &error_abort); + object_property_set_int(OBJECT(&s->soc[i]), + base_hartid, "hartid-base", &error_abort); + object_property_set_int(OBJECT(&s->soc[i]), + hart_count, "num-harts", &error_abort); + object_property_set_bool(OBJECT(&s->soc[i]), + true, "realized", &error_abort); + + /* Core Local Interruptor (timer and IPI) for each socket */ + sifive_clint_create( + memmap[SPIKE_CLINT].base + i * memmap[SPIKE_CLINT].size, + memmap[SPIKE_CLINT].size, base_hartid, hart_count, + SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, false); + } /* register system main memory (actual RAM) */ memory_region_init_ram(main_mem, NULL, "riscv.spike.ram", - machine->ram_size, &error_fatal); + mc->ram_size, &error_fatal); memory_region_add_subregion(system_memory, memmap[SPIKE_DRAM].base, main_mem); /* create device tree */ - create_fdt(s, memmap, machine->ram_size, machine->kernel_cmdline); + create_fdt(s, memmap, mc->ram_size, mc->kernel_cmdline); /* boot rom */ memory_region_init_rom(mask_rom, NULL, "riscv.spike.mrom", @@ -193,18 +295,18 @@ static void spike_board_init(MachineState *machine) memory_region_add_subregion(system_memory, memmap[SPIKE_MROM].base, mask_rom); - riscv_find_and_load_firmware(machine, BIOS_FILENAME, + riscv_find_and_load_firmware(mc, BIOS_FILENAME, memmap[SPIKE_DRAM].base, htif_symbol_callback); - if (machine->kernel_filename) { - uint64_t kernel_entry = riscv_load_kernel(machine->kernel_filename, + if (mc->kernel_filename) { + uint64_t kernel_entry = riscv_load_kernel(mc->kernel_filename, htif_symbol_callback); - if (machine->initrd_filename) { + if (mc->initrd_filename) { hwaddr start; - hwaddr end = riscv_load_initrd(machine->initrd_filename, - machine->ram_size, kernel_entry, + hwaddr end = riscv_load_initrd(mc->initrd_filename, + mc->ram_size, kernel_entry, &start); qemu_fdt_setprop_cell(s->fdt, "/chosen", "linux,initrd-start", start); @@ -249,12 +351,8 @@ static void spike_board_init(MachineState *machine) &address_space_memory); /* initialize HTIF using symbols found in load_kernel */ - htif_mm_init(system_memory, mask_rom, &s->soc.harts[0].env, serial_hd(0)); - - /* Core Local Interruptor (timer and IPI) */ - sifive_clint_create(memmap[SPIKE_CLINT].base, memmap[SPIKE_CLINT].size, - 0, smp_cpus, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, - false); + htif_mm_init(system_memory, mask_rom, + &s->soc[0].harts[0].env, serial_hd(0)); } static void spike_v1_10_0_board_init(MachineState *machine) @@ -268,6 +366,12 @@ static void spike_v1_10_0_board_init(MachineState *machine) int i; unsigned int smp_cpus = machine->smp.cpus; + s->num_socs = 1; + s->numa_enabled = false; + s->num_socs = 1; + s->mem_offset[0] = 0; + s->mem_size[0] = machine->ram_size; + if (!qtest_enabled()) { info_report("The Spike v1.10.0 machine has been deprecated. " "Please use the generic spike machine and specify the ISA " @@ -275,13 +379,14 @@ static void spike_v1_10_0_board_init(MachineState *machine) } /* Initialize SOC */ - object_initialize_child(OBJECT(machine), "soc", &s->soc, sizeof(s->soc), + object_initialize_child(OBJECT(machine), "soc", + &s->soc[0], sizeof(s->soc[0]), TYPE_RISCV_HART_ARRAY, &error_abort, NULL); - object_property_set_str(OBJECT(&s->soc), SPIKE_V1_10_0_CPU, "cpu-type", + object_property_set_str(OBJECT(&s->soc[0]), SPIKE_V1_10_0_CPU, "cpu-type", &error_abort); - object_property_set_int(OBJECT(&s->soc), smp_cpus, "num-harts", + object_property_set_int(OBJECT(&s->soc[0]), smp_cpus, "num-harts", &error_abort); - object_property_set_bool(OBJECT(&s->soc), true, "realized", + object_property_set_bool(OBJECT(&s->soc[0]), true, "realized", &error_abort); /* register system main memory (actual RAM) */ @@ -339,7 +444,8 @@ static void spike_v1_10_0_board_init(MachineState *machine) &address_space_memory); /* initialize HTIF using symbols found in load_kernel */ - htif_mm_init(system_memory, mask_rom, &s->soc.harts[0].env, serial_hd(0)); + htif_mm_init(system_memory, mask_rom, + &s->soc[0].harts[0].env, serial_hd(0)); /* Core Local Interruptor (timer and IPI) */ sifive_clint_create(memmap[SPIKE_CLINT].base, memmap[SPIKE_CLINT].size, @@ -358,6 +464,12 @@ static void spike_v1_09_1_board_init(MachineState *machine) int i; unsigned int smp_cpus = machine->smp.cpus; + s->num_socs = 1; + s->numa_enabled = false; + s->num_socs = 1; + s->mem_offset[0] = 0; + s->mem_size[0] = machine->ram_size; + if (!qtest_enabled()) { info_report("The Spike v1.09.1 machine has been deprecated. " "Please use the generic spike machine and specify the ISA " @@ -365,13 +477,14 @@ static void spike_v1_09_1_board_init(MachineState *machine) } /* Initialize SOC */ - object_initialize_child(OBJECT(machine), "soc", &s->soc, sizeof(s->soc), + object_initialize_child(OBJECT(machine), "soc", + &s->soc[0], sizeof(s->soc[0]), TYPE_RISCV_HART_ARRAY, &error_abort, NULL); - object_property_set_str(OBJECT(&s->soc), SPIKE_V1_09_1_CPU, "cpu-type", + object_property_set_str(OBJECT(&s->soc[0]), SPIKE_V1_09_1_CPU, "cpu-type", &error_abort); - object_property_set_int(OBJECT(&s->soc), smp_cpus, "num-harts", + object_property_set_int(OBJECT(&s->soc[0]), smp_cpus, "num-harts", &error_abort); - object_property_set_bool(OBJECT(&s->soc), true, "realized", + object_property_set_bool(OBJECT(&s->soc[0]), true, "realized", &error_abort); /* register system main memory (actual RAM) */ @@ -425,7 +538,7 @@ static void spike_v1_09_1_board_init(MachineState *machine) "};\n"; /* build config string with supplied memory size */ - char *isa = riscv_isa_string(&s->soc.harts[0]); + char *isa = riscv_isa_string(&s->soc[0].harts[0]); char *config_string = g_strdup_printf(config_string_tmpl, (uint64_t)memmap[SPIKE_CLINT].base + SIFIVE_TIME_BASE, (uint64_t)memmap[SPIKE_DRAM].base, @@ -448,7 +561,8 @@ static void spike_v1_09_1_board_init(MachineState *machine) &address_space_memory); /* initialize HTIF using symbols found in load_kernel */ - htif_mm_init(system_memory, mask_rom, &s->soc.harts[0].env, serial_hd(0)); + htif_mm_init(system_memory, mask_rom, + &s->soc[0].harts[0].env, serial_hd(0)); /* Core Local Interruptor (timer and IPI) */ sifive_clint_create(memmap[SPIKE_CLINT].base, memmap[SPIKE_CLINT].size, @@ -472,15 +586,86 @@ static void spike_v1_10_0_machine_init(MachineClass *mc) mc->max_cpus = 1; } -static void spike_machine_init(MachineClass *mc) +DEFINE_MACHINE("spike_v1.9.1", spike_v1_09_1_machine_init) +DEFINE_MACHINE("spike_v1.10", spike_v1_10_0_machine_init) + +static CpuInstanceProperties +spike_cpu_index_to_props(MachineState *ms, unsigned cpu_index) +{ + MachineClass *mc = MACHINE_GET_CLASS(ms); + const CPUArchIdList *possible_cpus = mc->possible_cpu_arch_ids(ms); + + assert(cpu_index < possible_cpus->len); + return possible_cpus->cpus[cpu_index].props; +} + +static int64_t spike_get_default_cpu_node_id(const MachineState *ms, int idx) +{ + int64_t nidx = 0; + + if (ms->numa_state->num_nodes) { + nidx = idx / (ms->smp.cpus / ms->numa_state->num_nodes); + if (ms->numa_state->num_nodes <= nidx) { + nidx = ms->numa_state->num_nodes - 1; + } + } + + return nidx; +} + +static const CPUArchIdList *spike_possible_cpu_arch_ids(MachineState *ms) +{ + int n; + unsigned int max_cpus = ms->smp.max_cpus; + + if (ms->possible_cpus) { + assert(ms->possible_cpus->len == max_cpus); + return ms->possible_cpus; + } + + ms->possible_cpus = g_malloc0(sizeof(CPUArchIdList) + + sizeof(CPUArchId) * max_cpus); + ms->possible_cpus->len = max_cpus; + for (n = 0; n < ms->possible_cpus->len; n++) { + ms->possible_cpus->cpus[n].type = ms->cpu_type; + ms->possible_cpus->cpus[n].arch_id = n; + ms->possible_cpus->cpus[n].props.has_thread_id = true; + ms->possible_cpus->cpus[n].props.thread_id = n; + } + + return ms->possible_cpus; +} + +static void spike_machine_instance_init(Object *obj) +{ +} + +static void spike_machine_class_init(ObjectClass *oc, void *data) { - mc->desc = "RISC-V Spike Board"; + MachineClass *mc = MACHINE_CLASS(oc); + + mc->desc = "RISC-V Spike board"; mc->init = spike_board_init; - mc->max_cpus = 8; + mc->max_cpus = SPIKE_CPUS_MAX; mc->is_default = true; mc->default_cpu_type = SPIKE_V1_10_0_CPU; + mc->possible_cpu_arch_ids = spike_possible_cpu_arch_ids; + mc->cpu_index_to_instance_props = spike_cpu_index_to_props; + mc->get_default_cpu_node_id = spike_get_default_cpu_node_id; + mc->numa_mem_supported = true; } -DEFINE_MACHINE("spike_v1.9.1", spike_v1_09_1_machine_init) -DEFINE_MACHINE("spike_v1.10", spike_v1_10_0_machine_init) -DEFINE_MACHINE("spike", spike_machine_init) +static const TypeInfo spike_machine_typeinfo = { + .name = MACHINE_TYPE_NAME("spike"), + .parent = TYPE_MACHINE, + .class_init = spike_machine_class_init, + .instance_init = spike_machine_instance_init, + .instance_size = sizeof(SpikeState), +}; + +static void spike_machine_init_register_types(void) +{ + type_register_static(&spike_machine_typeinfo); +} + +type_init(spike_machine_init_register_types) diff --git a/include/hw/riscv/spike.h b/include/hw/riscv/spike.h index dc770421bc..ec4fc9f799 100644 --- a/include/hw/riscv/spike.h +++ b/include/hw/riscv/spike.h @@ -21,13 +21,24 @@ #include "hw/riscv/riscv_hart.h" #include "hw/sysbus.h" +#include "sysemu/numa.h" + +#define SPIKE_CPUS_MAX 8 + +#define TYPE_SPIKE_MACHINE MACHINE_TYPE_NAME("spike") +#define SPIKE_MACHINE(obj) \ + OBJECT_CHECK(SpikeState, (obj), TYPE_SPIKE_MACHINE) typedef struct { /*< private >*/ - SysBusDevice parent_obj; + MachineState parent; /*< public >*/ - RISCVHartArrayState soc; + bool numa_enabled; + unsigned int num_socs; + uint64_t mem_offset[MAX_NODES]; + uint64_t mem_size[MAX_NODES]; + RISCVHartArrayState soc[MAX_NODES]; void *fdt; int fdt_size; } SpikeState; From patchwork Thu May 28 13:29:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anup Patel X-Patchwork-Id: 11575849 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 31D5192A for ; Thu, 28 May 2020 13:34:21 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EBD0D20721 for ; Thu, 28 May 2020 13:34:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="gk7jU89R"; dkim=pass (1024-bit key) header.d=sharedspace.onmicrosoft.com header.i=@sharedspace.onmicrosoft.com header.b="ovfAwnLp" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EBD0D20721 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=wdc.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:58808 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jeIfs-0004GH-4M for patchwork-qemu-devel@patchwork.kernel.org; Thu, 28 May 2020 09:34:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42260) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jeIcP-0007hx-1V; Thu, 28 May 2020 09:30:45 -0400 Received: from esa3.hgst.iphmx.com ([216.71.153.141]:30080) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jeIcK-0001G2-Lj; Thu, 28 May 2020 09:30:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1590672641; x=1622208641; h=from:to:cc:subject:date:message-id:in-reply-to: references:content-transfer-encoding:mime-version; bh=fcl2/s4taLYbTTKslRBqqC8oTwWKwM22EGfD/3R2t9E=; b=gk7jU89RiCPBfXACucqn8KO/xzeNuuCuZ9QyiUyFVZ8WQB9do85SSEDm g913n7WWbKSao6cs5A87keT5eNhrE8RfaSGoWjYjGsTnu+RIUzGjZAGuI LkM6Kn/aXqdCoLYquD54ALDpPlDBlElFtj7hpcLLMSBFOm21Du+g8jRna rvT4xjUFRTAR1E9t1ybVAUFe8iAXJLdQSwWVxQZloKHEdKu/jgOouRnco g5yf2AGUveeN8tQ63xQw2bxfwGg90AwY5zwa/p5VGZ46O/9DHLnOjvk+c iLDe0N0iJlLDX+qfsqEIS/mikzz+c5PtsCxUa9rkNJQu0RgqsX7q0C9wx g==; IronPort-SDR: wep/JWX9WjFsMjPBS7xrFTy9a/WD1LqUfINHgfn7LoRSx/SEjKQ4yROrB10VAt9cbT0IKkkGz/ +2sc+akEsPQVT5D2jx8NFvMO6t+SN9XOz4yBvMd2x8BXAM8R7WJhh1Nj2XNmSzZs5WUlp/v/IK zQMRbFjW8H96d81WHNFAtATEJybascK+RkPPOEa7zc1EpW1spRHFQDK+EHu1YC8isU7XuevjHp TJreLpxY+EH2dJHn4V9RvxH7sa+b6WWjnG6zfKt3+xYfwni9tAZU7cH71Bgf8M6SM5XpYvqPho nI4= X-IronPort-AV: E=Sophos;i="5.73,444,1583164800"; d="scan'208";a="143036566" Received: from mail-bn3nam04lp2056.outbound.protection.outlook.com (HELO NAM04-BN3-obe.outbound.protection.outlook.com) ([104.47.46.56]) by ob1.hgst.iphmx.com with ESMTP; 28 May 2020 21:30:38 +0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=A3Vn2MfrESTEq6mrwtVxkXFCaItPCXlvMTEJj6/+bX8xzfGqybzGJoV45+NWHsRoUJFsGEFdNWRCvNfK3Pg7fPtVCgZ5IaoLx/lwBlyF137eHTKbcEzREfpyhp0QST1/I2XPoj3sjp9RJhfnjDIVpEIH6u0ZfoCMLcmWGaWoM3INh9lT038oPLUeBi2B3Lo+qOWntrc4NACz1iJ76W7hKEQ1WrlFTUqjxOK7KTod1gNq+XYWwC8W3CZQ3xZt+sMcoKkDAMkY0K/uzLTtUowWHutKlH+SCRbHAX8dtjRmQ2BfPt+806oUkhBMDMG6Cu2pnZ5OsFx2xmXq1Rs522Odeg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tSqZwf+oGC6GnqJHzn0BQlbEQMiydC/gkmw6w1nYSYI=; b=RYapNSoB+cVEs+ZORU7gO94mf3Uo6X42wPxATIIUVm9xBJQmozVpaCAs15O/2e/rCsbo1i8jnl2cCABdaYFsWlFM0tphak0Yy1pa4BEQDfHCDOUAo4ztSv12TJjjGlwDL2AItTG+XzzTOQl1tseaakKo/5bGzMCoqC/KIBkrjW3GirFiLN0tKnUPvzqjvI7GuMTZE6k0NLIcC1izsk8ZJj8mvQRJVFh+KX+LdoirBF/L3rZQWOmXViM+Bp2tIlnMhUywrFdFzmIybqwL/Ry5fjbhaqqhr1ZkpuesWlF0HJcH9FgvnowA5oL1uINodFjsxEj129Nt4gvob1eSJc8h7Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wdc.com; dmarc=pass action=none header.from=wdc.com; dkim=pass header.d=wdc.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sharedspace.onmicrosoft.com; s=selector2-sharedspace-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tSqZwf+oGC6GnqJHzn0BQlbEQMiydC/gkmw6w1nYSYI=; b=ovfAwnLpmoYFQmHTgZKT1KoY4ocJjfbI7eOk0ku885zJB2kSoTXWvqCkWXbq8POo4G2Rwlye/nYLAbsSAjkpR3twFvQ3KI75/RgctrJtHGzSVjmA4iA0p3Rz3rE7oDKXMDRXGb1qrnFEujpbppmdWj0zopLCA7I/u5Nd8mHMopE= Authentication-Results: linaro.org; dkim=none (message not signed) header.d=none;linaro.org; dmarc=none action=none header.from=wdc.com; Received: from DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) by DM6PR04MB6204.namprd04.prod.outlook.com (2603:10b6:5:122::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.19; Thu, 28 May 2020 13:30:36 +0000 Received: from DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::f8b3:c124:482b:52e0]) by DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::f8b3:c124:482b:52e0%5]) with mapi id 15.20.3045.018; Thu, 28 May 2020 13:30:36 +0000 From: Anup Patel To: Peter Maydell , Palmer Dabbelt , Alistair Francis , Sagar Karandikar Subject: [PATCH v4 3/4] hw/riscv: Allow creating multiple instances of PLIC Date: Thu, 28 May 2020 18:59:58 +0530 Message-Id: <20200528132959.47773-4-anup.patel@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200528132959.47773-1-anup.patel@wdc.com> References: <20200528132959.47773-1-anup.patel@wdc.com> X-ClientProxiedBy: MAXPR01CA0094.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:49::36) To DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from wdc.com (49.207.63.107) by MAXPR01CA0094.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:49::36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.18 via Frontend Transport; Thu, 28 May 2020 13:30:33 +0000 X-Mailer: git-send-email 2.25.1 X-Originating-IP: [49.207.63.107] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: a774c4b1-f8d4-4fe4-c96e-08d8030b4e24 X-MS-TrafficTypeDiagnostic: DM6PR04MB6204: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: WDCIPOUTBOUND: EOP-TRUE X-MS-Oob-TLC-OOBClassifiers: OLM:14; X-Forefront-PRVS: 0417A3FFD2 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 6AhqRxEIovqZ6/mh8Dqm5dbTHCp+eA8L96157C91MGqXtawpKbMHqZv70kGGzBS7eIYUnnVsJovAE/cTDk2pN8bNbjvzg+zhu1t81s8YSiNZUttXwAmicmckWGmePHMQ4QvgfnQtJa60uAY9Xhr/+k09Bne3Pe2CsaK/Gc+tBiX6WR0TC32jEre0pmZqTlfCa72exR6eqIQ1DyPXYXZIvCEGXzatnO6WNPuzYiPNEupGpXbWUqH36sE4aeL+dFepCLnFVVekfCQHZCBEv+kHSkZO2kWxxyqDY/MdE6ZHvZNsAiU9vY7+Lft+kNAGfE9Wzkne1gllF4V5E78TcZkVfg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR04MB6201.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(396003)(346002)(366004)(376002)(39860400002)(956004)(26005)(55016002)(44832011)(478600001)(55236004)(2616005)(5660300002)(7696005)(16526019)(52116002)(186003)(8936002)(8676002)(1076003)(86362001)(8886007)(66556008)(66946007)(66476007)(316002)(83380400001)(4326008)(54906003)(110136005)(36756003)(6666004)(1006002)(2906002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: H5pYC+A0zmO+fYeNJUUjir39OguHKpqkvLHbzBWjeQFAohbxXVleix1YibWDZDE9wdIA6d4rMOiRLd4mtHkwtwGPmkU+slnmUO4GB7zLQ6XIml6/DrNtbEcmO2Y2mP5RrGv/zuooTXo8YSiC1JrNZh/wN41DRO14cI46qcST/YTxrE1M+YMQ0DXl4LFVBrFFbuzP2Gr/d3gF1wmMoAf2VL0xW+hHm9O1EjRuWTz5vQXoTfFunLOtYUH55ysjJFJfY4ikyX9D7yF+BwTxgtBSAQXQZoNv6GMoBNfB+LjhEsWgkJ2EePzZ1AvlojbF5Oe47Y8JDSxjv6Bcyyedh+ZiBw2Sb4722elf93FUq8NRMeDRE9/lqhjEFWUIyNyBYlfNbEfE00ewQh4FPMP6o0PzIc5rjsOts82u/Qkj1U7dxV1VG1ugLkoSLWVwXFT30p5t0+iPPOOqtoxgcZ2i/xKO4OouiI+5kMghNZeUhRus6U0= X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-Network-Message-Id: a774c4b1-f8d4-4fe4-c96e-08d8030b4e24 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 May 2020 13:30:36.5771 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2DNEfbXCDcQWFDGUxA7Cz4LutU4Re3Qc48hV5epqQKpxTSyd4auua1C6GTvRi1+9S8m+L9VXByeXpg/vMr8xgw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR04MB6204 Received-SPF: pass client-ip=216.71.153.141; envelope-from=prvs=41059f22b=Anup.Patel@wdc.com; helo=esa3.hgst.iphmx.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/28 09:30:28 X-ACL-Warn: Detected OS = FreeBSD 9.x or newer [fuzzy] X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Anup Patel , qemu-riscv@nongnu.org, Anup Patel , Palmer Dabbelt , qemu-devel@nongnu.org, Atish Patra , Alistair Francis Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We extend PLIC emulation to allow multiple instances of PLIC in a QEMU RISC-V machine. To achieve this, we remove first HART id zero assumption from PLIC emulation. Signed-off-by: Anup Patel Reviewed-by: Palmer Dabbelt Reviewed-by: Alistair Francis --- hw/riscv/sifive_e.c | 2 +- hw/riscv/sifive_plic.c | 24 +++++++++++++----------- hw/riscv/sifive_u.c | 2 +- hw/riscv/virt.c | 2 +- include/hw/riscv/sifive_plic.h | 12 +++++++----- 5 files changed, 23 insertions(+), 19 deletions(-) diff --git a/hw/riscv/sifive_e.c b/hw/riscv/sifive_e.c index 1c3b37d0ba..bd122e71ae 100644 --- a/hw/riscv/sifive_e.c +++ b/hw/riscv/sifive_e.c @@ -152,7 +152,7 @@ static void riscv_sifive_e_soc_realize(DeviceState *dev, Error **errp) /* MMIO */ s->plic = sifive_plic_create(memmap[SIFIVE_E_PLIC].base, - (char *)SIFIVE_E_PLIC_HART_CONFIG, + (char *)SIFIVE_E_PLIC_HART_CONFIG, 0, SIFIVE_E_PLIC_NUM_SOURCES, SIFIVE_E_PLIC_NUM_PRIORITIES, SIFIVE_E_PLIC_PRIORITY_BASE, diff --git a/hw/riscv/sifive_plic.c b/hw/riscv/sifive_plic.c index c1e04cbb98..f88bb48053 100644 --- a/hw/riscv/sifive_plic.c +++ b/hw/riscv/sifive_plic.c @@ -352,6 +352,7 @@ static const MemoryRegionOps sifive_plic_ops = { static Property sifive_plic_properties[] = { DEFINE_PROP_STRING("hart-config", SiFivePLICState, hart_config), + DEFINE_PROP_UINT32("hartid-base", SiFivePLICState, hartid_base, 0), DEFINE_PROP_UINT32("num-sources", SiFivePLICState, num_sources, 0), DEFINE_PROP_UINT32("num-priorities", SiFivePLICState, num_priorities, 0), DEFINE_PROP_UINT32("priority-base", SiFivePLICState, priority_base, 0), @@ -400,10 +401,12 @@ static void parse_hart_config(SiFivePLICState *plic) } hartid++; - /* store hart/mode combinations */ plic->num_addrs = addrid; + plic->num_harts = hartid; + + /* store hart/mode combinations */ plic->addr_config = g_new(PLICAddr, plic->num_addrs); - addrid = 0, hartid = 0; + addrid = 0, hartid = plic->hartid_base; p = plic->hart_config; while ((c = *p++)) { if (c == ',') { @@ -429,8 +432,6 @@ static void sifive_plic_irq_request(void *opaque, int irq, int level) static void sifive_plic_realize(DeviceState *dev, Error **errp) { - MachineState *ms = MACHINE(qdev_get_machine()); - unsigned int smp_cpus = ms->smp.cpus; SiFivePLICState *plic = SIFIVE_PLIC(dev); int i; @@ -451,8 +452,8 @@ static void sifive_plic_realize(DeviceState *dev, Error **errp) * lost a interrupt in the case a PLIC is attached. The SEIP bit must be * hardware controlled when a PLIC is attached. */ - for (i = 0; i < smp_cpus; i++) { - RISCVCPU *cpu = RISCV_CPU(qemu_get_cpu(i)); + for (i = 0; i < plic->num_harts; i++) { + RISCVCPU *cpu = RISCV_CPU(qemu_get_cpu(plic->hartid_base + i)); if (riscv_cpu_claim_interrupts(cpu, MIP_SEIP) < 0) { error_report("SEIP already claimed"); exit(1); @@ -488,16 +489,17 @@ type_init(sifive_plic_register_types) * Create PLIC device. */ DeviceState *sifive_plic_create(hwaddr addr, char *hart_config, - uint32_t num_sources, uint32_t num_priorities, - uint32_t priority_base, uint32_t pending_base, - uint32_t enable_base, uint32_t enable_stride, - uint32_t context_base, uint32_t context_stride, - uint32_t aperture_size) + uint32_t hartid_base, uint32_t num_sources, + uint32_t num_priorities, uint32_t priority_base, + uint32_t pending_base, uint32_t enable_base, + uint32_t enable_stride, uint32_t context_base, + uint32_t context_stride, uint32_t aperture_size) { DeviceState *dev = qdev_create(NULL, TYPE_SIFIVE_PLIC); assert(enable_stride == (enable_stride & -enable_stride)); assert(context_stride == (context_stride & -context_stride)); qdev_prop_set_string(dev, "hart-config", hart_config); + qdev_prop_set_uint32(dev, "hartid-base", hartid_base); qdev_prop_set_uint32(dev, "num-sources", num_sources); qdev_prop_set_uint32(dev, "num-priorities", num_priorities); qdev_prop_set_uint32(dev, "priority-base", priority_base); diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c index c193761916..53e48e2ff5 100644 --- a/hw/riscv/sifive_u.c +++ b/hw/riscv/sifive_u.c @@ -586,7 +586,7 @@ static void riscv_sifive_u_soc_realize(DeviceState *dev, Error **errp) /* MMIO */ s->plic = sifive_plic_create(memmap[SIFIVE_U_PLIC].base, - plic_hart_config, + plic_hart_config, 0, SIFIVE_U_PLIC_NUM_SOURCES, SIFIVE_U_PLIC_NUM_PRIORITIES, SIFIVE_U_PLIC_PRIORITY_BASE, diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index 51afe7e23b..421815081d 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -584,7 +584,7 @@ static void riscv_virt_board_init(MachineState *machine) /* MMIO */ s->plic = sifive_plic_create(memmap[VIRT_PLIC].base, - plic_hart_config, + plic_hart_config, 0, VIRT_PLIC_NUM_SOURCES, VIRT_PLIC_NUM_PRIORITIES, VIRT_PLIC_PRIORITY_BASE, diff --git a/include/hw/riscv/sifive_plic.h b/include/hw/riscv/sifive_plic.h index 4421e81249..ace76d0f1b 100644 --- a/include/hw/riscv/sifive_plic.h +++ b/include/hw/riscv/sifive_plic.h @@ -48,6 +48,7 @@ typedef struct SiFivePLICState { /*< public >*/ MemoryRegion mmio; uint32_t num_addrs; + uint32_t num_harts; uint32_t bitfield_words; PLICAddr *addr_config; uint32_t *source_priority; @@ -58,6 +59,7 @@ typedef struct SiFivePLICState { /* config */ char *hart_config; + uint32_t hartid_base; uint32_t num_sources; uint32_t num_priorities; uint32_t priority_base; @@ -70,10 +72,10 @@ typedef struct SiFivePLICState { } SiFivePLICState; DeviceState *sifive_plic_create(hwaddr addr, char *hart_config, - uint32_t num_sources, uint32_t num_priorities, - uint32_t priority_base, uint32_t pending_base, - uint32_t enable_base, uint32_t enable_stride, - uint32_t context_base, uint32_t context_stride, - uint32_t aperture_size); + uint32_t hartid_base, uint32_t num_sources, + uint32_t num_priorities, uint32_t priority_base, + uint32_t pending_base, uint32_t enable_base, + uint32_t enable_stride, uint32_t context_base, + uint32_t context_stride, uint32_t aperture_size); #endif From patchwork Thu May 28 13:29:59 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anup Patel X-Patchwork-Id: 11575851 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C93A314C0 for ; Thu, 28 May 2020 13:35:30 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7EC76206F1 for ; Thu, 28 May 2020 13:35:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="Hb51fB4g"; dkim=pass (1024-bit key) header.d=sharedspace.onmicrosoft.com header.i=@sharedspace.onmicrosoft.com header.b="NshSjaVj" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7EC76206F1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=wdc.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:33852 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jeIgz-0005RP-HX for patchwork-qemu-devel@patchwork.kernel.org; Thu, 28 May 2020 09:35:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42280) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jeIcU-0007ob-5q; Thu, 28 May 2020 09:30:50 -0400 Received: from esa2.hgst.iphmx.com ([68.232.143.124]:21050) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jeIcP-0001OZ-7Q; Thu, 28 May 2020 09:30:49 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1590672647; x=1622208647; h=from:to:cc:subject:date:message-id:in-reply-to: references:content-transfer-encoding:mime-version; bh=ukJUlNUWRS69earel4LLjUfXHN8vYo8n1fNX0O3nFGs=; b=Hb51fB4gBRSgJhIWEf9QfncoTp1bK+MdT5yvBTEyzahcJNAo3w0JvUF4 CwzpJUskVKwisT5larj9nG2oKvBvh9fDaAAH/bWO95BadEGNdG1NOizsv OXjz93FYTVsY3UGNCcq32H2CGuYW+/TBVj/XvP8rhNo4NtLJhDnQgSKFF oy8kuUdg0BarEP0GJLCRg3LkvMvbVEB2PeLQR4onrrh2VB66/uOk/Qktk IfYK8O9XnDiRp8JYXBeqDb932GSCyisSrXIXD5mUrqem0lVHV8YWW+xgV 40Sa2gZS8KW0BfWTWfeqa7ky79RfyrQLcZgb8er+2VQMUlfrWcXfPh0Sg w==; IronPort-SDR: qKgWzuUkveUfSAy3VcWJmSqHuFhDOFCS5jLStUywD/ZCyi9UVlP4XvPmr1mSoLq9KjLBwmyzMl VudVRfh0xYF6PymdMKuuAij1LQFG87LuL0AZpdyW8wCdfGG9sEnr9u2wGLtd7RM/X584nsOZOV zMkYPF2wx3ZdH+/FFVW6NH2oMPOkTu8aeMQV4TeSYL5AZTXOlcDDc23a2cvxsO2A6/lnyNB0Wd ZqWzq0l5ZbAfuipSN/zEchnVJyHkTQUFZA6/dl46dQAU+Gbo2/efHj5XiqJELXS20H0x4s8Edo Hxc= X-IronPort-AV: E=Sophos;i="5.73,444,1583164800"; d="scan'208";a="241528825" Received: from mail-dm6nam10lp2106.outbound.protection.outlook.com (HELO NAM10-DM6-obe.outbound.protection.outlook.com) ([104.47.58.106]) by ob1.hgst.iphmx.com with ESMTP; 28 May 2020 21:31:04 +0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Ip6ccj3Vy6bo+b8ewcb839tEU6dP5eLBtPDszx19UfgccG76abJJXG7E5uTlojQ0Q4DYKMzb8XTIOKVWZUOgiKwfpVXQHIozVyxe49TQclcydXE6c90Tm5QcBJg/hDyjISpmSDxLnbvhpE+uvrknZbKxsbioDVGj73Edt3uc6hf48h/S7sY+6VFcuW6JkamfUqUHLpajJrBb2P9hsVaRNIIpVW8xpBxFoxU4Me8jYEuyLeW7inBfnqIOgYOWHujZFmRuU9kStZzhXeaTVZ0wsxwYiHrnsVGLJ+fDJvcp1Di+Y/hw7H71NswxSDrRW8WWqr3xKtUNzbqz+EyYsQRwbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nXvM8wM0A7M64eNgAmUAqC9I6Vn+xIzT3t15B/7lntA=; b=PlJTyDHNC7q+I5g8wSO3kflOuNr/I8lyofM0hpENCKzPcqRazL5HZ7waaNiUUDSm6P/4LRZaf/FmH6Oh7V1A3LkVRw/dC3mWfT7WO6cwQjdGhEwweHFrKxwJx2ed7BJIw8wu428aVbPBbvl7bYKh7lJNOCskhJoecUclcXkGjwVMbH1ON5pzAkad3W9P6zIqkUIN7EEw50u/47PGRc8cYyv9MbE4Iv5inD+sXs3c9xrSxfYD367QGxogm2ySKaK0CtsCZHHbZxkZ3pkSLceI1GJoQVhk/N2690vY0uMQG6fEp9xPnrdzEmHjc0EiXbqy0ahCpkjPs0UUYeGovpn6eQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wdc.com; dmarc=pass action=none header.from=wdc.com; dkim=pass header.d=wdc.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sharedspace.onmicrosoft.com; s=selector2-sharedspace-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nXvM8wM0A7M64eNgAmUAqC9I6Vn+xIzT3t15B/7lntA=; b=NshSjaVjHqzIACI0nth9JqlzMUGY5t6C7aVeg0SxJj0cijckJnojupCF3aKX9C6gX0onbyP3CSE05WOMm9zWkmXtq28Qd5OUX9xzCzk3LP7soPG2fTLhzAA1eVZutYX2XUM20wRf5Qq9lBlsdZnmNRlllpuSNwbdkLizJNQc3qs= Authentication-Results: linaro.org; dkim=none (message not signed) header.d=none;linaro.org; dmarc=none action=none header.from=wdc.com; Received: from DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) by DM6PR04MB6204.namprd04.prod.outlook.com (2603:10b6:5:122::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.19; Thu, 28 May 2020 13:30:40 +0000 Received: from DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::f8b3:c124:482b:52e0]) by DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::f8b3:c124:482b:52e0%5]) with mapi id 15.20.3045.018; Thu, 28 May 2020 13:30:40 +0000 From: Anup Patel To: Peter Maydell , Palmer Dabbelt , Alistair Francis , Sagar Karandikar Subject: [PATCH v4 4/4] hw/riscv: virt: Allow creating multiple NUMA sockets Date: Thu, 28 May 2020 18:59:59 +0530 Message-Id: <20200528132959.47773-5-anup.patel@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200528132959.47773-1-anup.patel@wdc.com> References: <20200528132959.47773-1-anup.patel@wdc.com> X-ClientProxiedBy: MAXPR01CA0094.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:49::36) To DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from wdc.com (49.207.63.107) by MAXPR01CA0094.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:49::36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3045.18 via Frontend Transport; Thu, 28 May 2020 13:30:37 +0000 X-Mailer: git-send-email 2.25.1 X-Originating-IP: [49.207.63.107] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: a99bae2f-b5b3-4960-462d-08d8030b5022 X-MS-TrafficTypeDiagnostic: DM6PR04MB6204: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: WDCIPOUTBOUND: EOP-TRUE X-MS-Oob-TLC-OOBClassifiers: OLM:152; X-Forefront-PRVS: 0417A3FFD2 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: LObUFWJaEGX4S2xMrq97pGXy1mEnSNPrdGlNYPLXh3ZtPpBPqE9/ab3zUO8Kxh1cvSPj03u9pBRXAZ3NntXpcz5FzGe/5Qgq4Zlj9iO5Fmyt+j0cGr0etXgyK/t5UUbiiDQYIFmAtglQ6/PRkcssabG/emoQBMMW0kpRPFUogWj/e3gDUb3q9oUD2bQLJXoN1i0/tK+1Edpvee7LDv83QdgAKsL40gKNp0t9q2cF+xRH8SxTwWgbn8YyRuDD4WeVf6z1xQjpITnII/JT6X3fWQoo9KaQT8Fs/4HZkYx+HVgWh1wlb+vCLFdKG028WJCt X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR04MB6201.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(136003)(396003)(346002)(366004)(376002)(39860400002)(956004)(26005)(55016002)(44832011)(478600001)(55236004)(2616005)(5660300002)(7696005)(16526019)(52116002)(186003)(8936002)(30864003)(8676002)(1076003)(86362001)(8886007)(66556008)(66946007)(66476007)(316002)(83380400001)(4326008)(54906003)(110136005)(36756003)(6666004)(1006002)(2906002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: Ud9KJ3Q8IWwn9u4Mt9H8L/gdd398EcVyvEqKRWU7kMYOKfubV+t3VUAVGStEHxNqApV5g+AXwo/FfvSCKD1FWctQe6rNUOtYqHKvgkFSRyyfo7IzWmV5unAIU5MXTkXvz7WDga34nln9AvxM+kxEa4Y5eBqO26uAWpfzCIQ1N5f9CN9d1qd8xFvFM+n02j0iKKPqdesbwpkOjhoEsBNYaDk/5TdkCwj9Gfe8FET5omcivKD6AOhfoXNzNIjqcep+OKppXluNuchUA792vScezJ46mpFYbmtsnAeZayEtK027Gwe87Q/52n/onm2wJ95z7eLoaakk+l2TK3jTBMY8epqahRQOdCnOXw9P7JT5mbH2cKMkPzW9xzESEkU6VEnR6j/RCNGz7b3Q7pvb0RsYP9b4vsECV4L2wNM141jSexQEOj3BjUdN2UfEsihuuqyJVgGfT97FcxE5oECbmyN6g2BQ5/NrwL0U2jvIkaRCKRA= X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-Network-Message-Id: a99bae2f-b5b3-4960-462d-08d8030b5022 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 May 2020 13:30:39.9660 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: uiyui3ooBjdu5BwInkQ2jcqAqez0ys1NGLLHuOVBxY2O3kM1NnUyO2C9Y7hYnkKubRWz8fTJ9cXfrS3fTr1PhQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR04MB6204 Received-SPF: pass client-ip=68.232.143.124; envelope-from=prvs=41059f22b=Anup.Patel@wdc.com; helo=esa2.hgst.iphmx.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/05/28 09:30:42 X-ACL-Warn: Detected OS = FreeBSD 9.x or newer [fuzzy] X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_MED=0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atish Patra , Anup Patel , qemu-riscv@nongnu.org, qemu-devel@nongnu.org, Anup Patel Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We extend RISC-V virt machine to allow creating a multi-socket machine. Each RISC-V virt machine socket is a NUMA node having a set of HARTs, a memory instance, a CLINT instance, and a PLIC instance. Other devices are shared between all RISC-V virt machine sockets. We also update the generated device tree accordingly. By default, NUMA multi-socket support is disabled for RISC-V virt machine. To enable it, users can use "-numa" command-line options of QEMU. Example1: For two NUMA nodes with 4 CPUs each, append following to command-line options: "-smp 8 -numa node -numa node" Example2: For two NUMA nodes with 3 and 5 CPUs, append following to command-line options: "-smp 8 -numa node,cpus=0-2 -numa node,3-7" Signed-off-by: Anup Patel --- hw/riscv/virt.c | 634 +++++++++++++++++++++++++--------------- include/hw/riscv/virt.h | 13 +- 2 files changed, 408 insertions(+), 239 deletions(-) diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index 421815081d..18af426be3 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -60,7 +60,7 @@ static const struct MemmapEntry { [VIRT_TEST] = { 0x100000, 0x1000 }, [VIRT_RTC] = { 0x101000, 0x1000 }, [VIRT_CLINT] = { 0x2000000, 0x10000 }, - [VIRT_PLIC] = { 0xc000000, 0x4000000 }, + [VIRT_PLIC] = { 0xc000000, VIRT_PLIC_SIZE(VIRT_CPUS_MAX * 2) }, [VIRT_UART0] = { 0x10000000, 0x100 }, [VIRT_VIRTIO] = { 0x10001000, 0x1000 }, [VIRT_FLASH] = { 0x20000000, 0x4000000 }, @@ -182,10 +182,17 @@ static void create_fdt(RISCVVirtState *s, const struct MemmapEntry *memmap, uint64_t mem_size, const char *cmdline) { void *fdt; - int cpu, i; - uint32_t *cells; - char *nodename; - uint32_t plic_phandle, test_phandle, phandle = 1; + int i, j, idx, cpu, socket; + MachineState *mc = MACHINE(s); + uint32_t dist_matrix_size; + uint32_t *clint_cells, *plic_cells, *dist_matrix; + unsigned long clint_addr, plic_addr; + uint32_t plic_phandle[MAX_NODES]; + uint32_t cpu_phandle, intc_phandle, test_phandle; + uint32_t phandle = 1, plic_mmio_phandle = 1; + uint32_t plic_pcie_phandle = 1, plic_virtio_phandle = 1; + char *mem_name, *cpu_name, *core_name, *intc_name; + char *name, *clint_name, *plic_name, *clust_name; hwaddr flashsize = virt_memmap[VIRT_FLASH].size / 2; hwaddr flashbase = virt_memmap[VIRT_FLASH].base; @@ -206,231 +213,267 @@ static void create_fdt(RISCVVirtState *s, const struct MemmapEntry *memmap, qemu_fdt_setprop_cell(fdt, "/soc", "#size-cells", 0x2); qemu_fdt_setprop_cell(fdt, "/soc", "#address-cells", 0x2); - nodename = g_strdup_printf("/memory@%lx", - (long)memmap[VIRT_DRAM].base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_cells(fdt, nodename, "reg", - memmap[VIRT_DRAM].base >> 32, memmap[VIRT_DRAM].base, - mem_size >> 32, mem_size); - qemu_fdt_setprop_string(fdt, nodename, "device_type", "memory"); - g_free(nodename); - qemu_fdt_add_subnode(fdt, "/cpus"); qemu_fdt_setprop_cell(fdt, "/cpus", "timebase-frequency", SIFIVE_CLINT_TIMEBASE_FREQ); qemu_fdt_setprop_cell(fdt, "/cpus", "#size-cells", 0x0); qemu_fdt_setprop_cell(fdt, "/cpus", "#address-cells", 0x1); + qemu_fdt_add_subnode(fdt, "/cpus/cpu-map"); + + for (socket = (s->num_socs - 1); socket >= 0; socket--) { + clust_name = g_strdup_printf("/cpus/cpu-map/cluster%d", socket); + qemu_fdt_add_subnode(fdt, clust_name); + + plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 4); + clint_cells = g_new0(uint32_t, s->soc[socket].num_harts * 4); - for (cpu = s->soc.num_harts - 1; cpu >= 0; cpu--) { - int cpu_phandle = phandle++; - int intc_phandle; - nodename = g_strdup_printf("/cpus/cpu@%d", cpu); - char *intc = g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu); - char *isa = riscv_isa_string(&s->soc.harts[cpu]); - qemu_fdt_add_subnode(fdt, nodename); + for (cpu = s->soc[socket].num_harts - 1; cpu >= 0; cpu--) { + cpu_phandle = phandle++; + + cpu_name = g_strdup_printf("/cpus/cpu@%d", + s->soc[socket].hartid_base + cpu); + qemu_fdt_add_subnode(fdt, cpu_name); #if defined(TARGET_RISCV32) - qemu_fdt_setprop_string(fdt, nodename, "mmu-type", "riscv,sv32"); + qemu_fdt_setprop_string(fdt, cpu_name, "mmu-type", "riscv,sv32"); #else - qemu_fdt_setprop_string(fdt, nodename, "mmu-type", "riscv,sv48"); + qemu_fdt_setprop_string(fdt, cpu_name, "mmu-type", "riscv,sv48"); #endif - qemu_fdt_setprop_string(fdt, nodename, "riscv,isa", isa); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "riscv"); - qemu_fdt_setprop_string(fdt, nodename, "status", "okay"); - qemu_fdt_setprop_cell(fdt, nodename, "reg", cpu); - qemu_fdt_setprop_string(fdt, nodename, "device_type", "cpu"); - qemu_fdt_setprop_cell(fdt, nodename, "phandle", cpu_phandle); - intc_phandle = phandle++; - qemu_fdt_add_subnode(fdt, intc); - qemu_fdt_setprop_cell(fdt, intc, "phandle", intc_phandle); - qemu_fdt_setprop_string(fdt, intc, "compatible", "riscv,cpu-intc"); - qemu_fdt_setprop(fdt, intc, "interrupt-controller", NULL, 0); - qemu_fdt_setprop_cell(fdt, intc, "#interrupt-cells", 1); - g_free(isa); - g_free(intc); - g_free(nodename); - } + name = riscv_isa_string(&s->soc[socket].harts[cpu]); + qemu_fdt_setprop_string(fdt, cpu_name, "riscv,isa", name); + g_free(name); + qemu_fdt_setprop_string(fdt, cpu_name, "compatible", "riscv"); + qemu_fdt_setprop_string(fdt, cpu_name, "status", "okay"); + qemu_fdt_setprop_cell(fdt, cpu_name, "reg", + s->soc[socket].hartid_base + cpu); + qemu_fdt_setprop_string(fdt, cpu_name, "device_type", "cpu"); + if (s->numa_enabled) { + qemu_fdt_setprop_cell(fdt, cpu_name, "numa-node-id", socket); + } + qemu_fdt_setprop_cell(fdt, cpu_name, "phandle", cpu_phandle); + + intc_name = g_strdup_printf("%s/interrupt-controller", cpu_name); + qemu_fdt_add_subnode(fdt, intc_name); + intc_phandle = phandle++; + qemu_fdt_setprop_cell(fdt, intc_name, "phandle", intc_phandle); + qemu_fdt_setprop_string(fdt, intc_name, "compatible", + "riscv,cpu-intc"); + qemu_fdt_setprop(fdt, intc_name, "interrupt-controller", NULL, 0); + qemu_fdt_setprop_cell(fdt, intc_name, "#interrupt-cells", 1); + + clint_cells[cpu * 4 + 0] = cpu_to_be32(intc_phandle); + clint_cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_SOFT); + clint_cells[cpu * 4 + 2] = cpu_to_be32(intc_phandle); + clint_cells[cpu * 4 + 3] = cpu_to_be32(IRQ_M_TIMER); + + plic_cells[cpu * 4 + 0] = cpu_to_be32(intc_phandle); + plic_cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_EXT); + plic_cells[cpu * 4 + 2] = cpu_to_be32(intc_phandle); + plic_cells[cpu * 4 + 3] = cpu_to_be32(IRQ_S_EXT); + + core_name = g_strdup_printf("%s/core%d", clust_name, cpu); + qemu_fdt_add_subnode(fdt, core_name); + qemu_fdt_setprop_cell(fdt, core_name, "cpu", cpu_phandle); + + g_free(core_name); + g_free(intc_name); + g_free(cpu_name); + } - /* Add cpu-topology node */ - qemu_fdt_add_subnode(fdt, "/cpus/cpu-map"); - qemu_fdt_add_subnode(fdt, "/cpus/cpu-map/cluster0"); - for (cpu = s->soc.num_harts - 1; cpu >= 0; cpu--) { - char *core_nodename = g_strdup_printf("/cpus/cpu-map/cluster0/core%d", - cpu); - char *cpu_nodename = g_strdup_printf("/cpus/cpu@%d", cpu); - uint32_t intc_phandle = qemu_fdt_get_phandle(fdt, cpu_nodename); - qemu_fdt_add_subnode(fdt, core_nodename); - qemu_fdt_setprop_cell(fdt, core_nodename, "cpu", intc_phandle); - g_free(core_nodename); - g_free(cpu_nodename); + mem_name = g_strdup_printf("/memory@%lx", + (long)memmap[VIRT_DRAM].base + s->mem_offset[socket]); + qemu_fdt_add_subnode(fdt, mem_name); + qemu_fdt_setprop_cells(fdt, mem_name, "reg", + (memmap[VIRT_DRAM].base + s->mem_offset[socket]) >> 32, + memmap[VIRT_DRAM].base + s->mem_offset[socket], + s->mem_size[socket] >> 32, s->mem_size[socket]); + qemu_fdt_setprop_string(fdt, mem_name, "device_type", "memory"); + if (s->numa_enabled) { + qemu_fdt_setprop_cell(fdt, mem_name, "numa-node-id", socket); + } + g_free(mem_name); + + clint_addr = memmap[VIRT_CLINT].base + + (memmap[VIRT_CLINT].size * socket); + clint_name = g_strdup_printf("/soc/clint@%lx", clint_addr); + qemu_fdt_add_subnode(fdt, clint_name); + qemu_fdt_setprop_string(fdt, clint_name, "compatible", "riscv,clint0"); + qemu_fdt_setprop_cells(fdt, clint_name, "reg", + 0x0, clint_addr, 0x0, memmap[VIRT_CLINT].size); + qemu_fdt_setprop(fdt, clint_name, "interrupts-extended", + clint_cells, s->soc[socket].num_harts * sizeof(uint32_t) * 4); + if (s->numa_enabled) { + qemu_fdt_setprop_cell(fdt, clint_name, "numa-node-id", socket); + } + g_free(clint_name); + + plic_phandle[socket] = phandle++; + plic_addr = memmap[VIRT_PLIC].base + (memmap[VIRT_PLIC].size * socket); + plic_name = g_strdup_printf("/soc/plic@%lx", plic_addr); + qemu_fdt_add_subnode(fdt, plic_name); + qemu_fdt_setprop_cell(fdt, plic_name, + "#address-cells", FDT_PLIC_ADDR_CELLS); + qemu_fdt_setprop_cell(fdt, plic_name, + "#interrupt-cells", FDT_PLIC_INT_CELLS); + qemu_fdt_setprop_string(fdt, plic_name, "compatible", "riscv,plic0"); + qemu_fdt_setprop(fdt, plic_name, "interrupt-controller", NULL, 0); + qemu_fdt_setprop(fdt, plic_name, "interrupts-extended", + plic_cells, s->soc[socket].num_harts * sizeof(uint32_t) * 4); + qemu_fdt_setprop_cells(fdt, plic_name, "reg", + 0x0, plic_addr, 0x0, memmap[VIRT_PLIC].size); + qemu_fdt_setprop_cell(fdt, plic_name, "riscv,ndev", VIRTIO_NDEV); + if (s->numa_enabled) { + qemu_fdt_setprop_cell(fdt, plic_name, "numa-node-id", socket); + } + qemu_fdt_setprop_cell(fdt, plic_name, "phandle", plic_phandle[socket]); + g_free(plic_name); + + g_free(clint_cells); + g_free(plic_cells); + g_free(clust_name); } - cells = g_new0(uint32_t, s->soc.num_harts * 4); - for (cpu = 0; cpu < s->soc.num_harts; cpu++) { - nodename = - g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu); - uint32_t intc_phandle = qemu_fdt_get_phandle(fdt, nodename); - cells[cpu * 4 + 0] = cpu_to_be32(intc_phandle); - cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_SOFT); - cells[cpu * 4 + 2] = cpu_to_be32(intc_phandle); - cells[cpu * 4 + 3] = cpu_to_be32(IRQ_M_TIMER); - g_free(nodename); + for (socket = 0; socket < s->num_socs; socket++) { + if (socket == 0) { + plic_mmio_phandle = plic_phandle[socket]; + plic_virtio_phandle = plic_phandle[socket]; + plic_pcie_phandle = plic_phandle[socket]; + } + if (socket == 1) { + plic_virtio_phandle = plic_phandle[socket]; + plic_pcie_phandle = plic_phandle[socket]; + } + if (socket == 2) { + plic_pcie_phandle = plic_phandle[socket]; + } } - nodename = g_strdup_printf("/soc/clint@%lx", - (long)memmap[VIRT_CLINT].base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "riscv,clint0"); - qemu_fdt_setprop_cells(fdt, nodename, "reg", - 0x0, memmap[VIRT_CLINT].base, - 0x0, memmap[VIRT_CLINT].size); - qemu_fdt_setprop(fdt, nodename, "interrupts-extended", - cells, s->soc.num_harts * sizeof(uint32_t) * 4); - g_free(cells); - g_free(nodename); - - plic_phandle = phandle++; - cells = g_new0(uint32_t, s->soc.num_harts * 4); - for (cpu = 0; cpu < s->soc.num_harts; cpu++) { - nodename = - g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu); - uint32_t intc_phandle = qemu_fdt_get_phandle(fdt, nodename); - cells[cpu * 4 + 0] = cpu_to_be32(intc_phandle); - cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_EXT); - cells[cpu * 4 + 2] = cpu_to_be32(intc_phandle); - cells[cpu * 4 + 3] = cpu_to_be32(IRQ_S_EXT); - g_free(nodename); + + if (s->numa_enabled && mc->numa_state->have_numa_distance) { + dist_matrix_size = s->num_socs * s->num_socs * 3 * sizeof(uint32_t); + dist_matrix = g_malloc0(dist_matrix_size); + + for (i = 0; i < s->num_socs; i++) { + for (j = 0; j < s->num_socs; j++) { + idx = (i * s->num_socs + j) * 3; + dist_matrix[idx + 0] = cpu_to_be32(i); + dist_matrix[idx + 1] = cpu_to_be32(j); + dist_matrix[idx + 2] = + cpu_to_be32(mc->numa_state->nodes[i].distance[j]); + } + } + + qemu_fdt_add_subnode(fdt, "/distance-map"); + qemu_fdt_setprop_string(fdt, "/distance-map", "compatible", + "numa-distance-map-v1"); + qemu_fdt_setprop(fdt, "/distance-map", "distance-matrix", + dist_matrix, dist_matrix_size); + g_free(dist_matrix); } - nodename = g_strdup_printf("/soc/interrupt-controller@%lx", - (long)memmap[VIRT_PLIC].base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_cell(fdt, nodename, "#address-cells", - FDT_PLIC_ADDR_CELLS); - qemu_fdt_setprop_cell(fdt, nodename, "#interrupt-cells", - FDT_PLIC_INT_CELLS); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "riscv,plic0"); - qemu_fdt_setprop(fdt, nodename, "interrupt-controller", NULL, 0); - qemu_fdt_setprop(fdt, nodename, "interrupts-extended", - cells, s->soc.num_harts * sizeof(uint32_t) * 4); - qemu_fdt_setprop_cells(fdt, nodename, "reg", - 0x0, memmap[VIRT_PLIC].base, - 0x0, memmap[VIRT_PLIC].size); - qemu_fdt_setprop_cell(fdt, nodename, "riscv,ndev", VIRTIO_NDEV); - qemu_fdt_setprop_cell(fdt, nodename, "phandle", plic_phandle); - plic_phandle = qemu_fdt_get_phandle(fdt, nodename); - g_free(cells); - g_free(nodename); for (i = 0; i < VIRTIO_COUNT; i++) { - nodename = g_strdup_printf("/virtio_mmio@%lx", + name = g_strdup_printf("/soc/virtio_mmio@%lx", (long)(memmap[VIRT_VIRTIO].base + i * memmap[VIRT_VIRTIO].size)); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "virtio,mmio"); - qemu_fdt_setprop_cells(fdt, nodename, "reg", + qemu_fdt_add_subnode(fdt, name); + qemu_fdt_setprop_string(fdt, name, "compatible", "virtio,mmio"); + qemu_fdt_setprop_cells(fdt, name, "reg", 0x0, memmap[VIRT_VIRTIO].base + i * memmap[VIRT_VIRTIO].size, 0x0, memmap[VIRT_VIRTIO].size); - qemu_fdt_setprop_cell(fdt, nodename, "interrupt-parent", plic_phandle); - qemu_fdt_setprop_cell(fdt, nodename, "interrupts", VIRTIO_IRQ + i); - g_free(nodename); + qemu_fdt_setprop_cell(fdt, name, "interrupt-parent", + plic_virtio_phandle); + qemu_fdt_setprop_cell(fdt, name, "interrupts", VIRTIO_IRQ + i); + g_free(name); } - nodename = g_strdup_printf("/soc/pci@%lx", + name = g_strdup_printf("/soc/pci@%lx", (long) memmap[VIRT_PCIE_ECAM].base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_cell(fdt, nodename, "#address-cells", - FDT_PCI_ADDR_CELLS); - qemu_fdt_setprop_cell(fdt, nodename, "#interrupt-cells", - FDT_PCI_INT_CELLS); - qemu_fdt_setprop_cell(fdt, nodename, "#size-cells", 0x2); - qemu_fdt_setprop_string(fdt, nodename, "compatible", - "pci-host-ecam-generic"); - qemu_fdt_setprop_string(fdt, nodename, "device_type", "pci"); - qemu_fdt_setprop_cell(fdt, nodename, "linux,pci-domain", 0); - qemu_fdt_setprop_cells(fdt, nodename, "bus-range", 0, - memmap[VIRT_PCIE_ECAM].size / - PCIE_MMCFG_SIZE_MIN - 1); - qemu_fdt_setprop(fdt, nodename, "dma-coherent", NULL, 0); - qemu_fdt_setprop_cells(fdt, nodename, "reg", 0, memmap[VIRT_PCIE_ECAM].base, - 0, memmap[VIRT_PCIE_ECAM].size); - qemu_fdt_setprop_sized_cells(fdt, nodename, "ranges", + qemu_fdt_add_subnode(fdt, name); + qemu_fdt_setprop_cell(fdt, name, "#address-cells", FDT_PCI_ADDR_CELLS); + qemu_fdt_setprop_cell(fdt, name, "#interrupt-cells", FDT_PCI_INT_CELLS); + qemu_fdt_setprop_cell(fdt, name, "#size-cells", 0x2); + qemu_fdt_setprop_string(fdt, name, "compatible", "pci-host-ecam-generic"); + qemu_fdt_setprop_string(fdt, name, "device_type", "pci"); + qemu_fdt_setprop_cell(fdt, name, "linux,pci-domain", 0); + qemu_fdt_setprop_cells(fdt, name, "bus-range", 0, + memmap[VIRT_PCIE_ECAM].size / PCIE_MMCFG_SIZE_MIN - 1); + qemu_fdt_setprop(fdt, name, "dma-coherent", NULL, 0); + qemu_fdt_setprop_cells(fdt, name, "reg", 0, + memmap[VIRT_PCIE_ECAM].base, 0, memmap[VIRT_PCIE_ECAM].size); + qemu_fdt_setprop_sized_cells(fdt, name, "ranges", 1, FDT_PCI_RANGE_IOPORT, 2, 0, 2, memmap[VIRT_PCIE_PIO].base, 2, memmap[VIRT_PCIE_PIO].size, 1, FDT_PCI_RANGE_MMIO, 2, memmap[VIRT_PCIE_MMIO].base, 2, memmap[VIRT_PCIE_MMIO].base, 2, memmap[VIRT_PCIE_MMIO].size); - create_pcie_irq_map(fdt, nodename, plic_phandle); - g_free(nodename); + create_pcie_irq_map(fdt, name, plic_pcie_phandle); + g_free(name); test_phandle = phandle++; - nodename = g_strdup_printf("/test@%lx", + name = g_strdup_printf("/soc/test@%lx", (long)memmap[VIRT_TEST].base); - qemu_fdt_add_subnode(fdt, nodename); + qemu_fdt_add_subnode(fdt, name); { const char compat[] = "sifive,test1\0sifive,test0\0syscon"; - qemu_fdt_setprop(fdt, nodename, "compatible", compat, sizeof(compat)); + qemu_fdt_setprop(fdt, name, "compatible", compat, sizeof(compat)); } - qemu_fdt_setprop_cells(fdt, nodename, "reg", + qemu_fdt_setprop_cells(fdt, name, "reg", 0x0, memmap[VIRT_TEST].base, 0x0, memmap[VIRT_TEST].size); - qemu_fdt_setprop_cell(fdt, nodename, "phandle", test_phandle); - test_phandle = qemu_fdt_get_phandle(fdt, nodename); - g_free(nodename); - - nodename = g_strdup_printf("/reboot"); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "syscon-reboot"); - qemu_fdt_setprop_cell(fdt, nodename, "regmap", test_phandle); - qemu_fdt_setprop_cell(fdt, nodename, "offset", 0x0); - qemu_fdt_setprop_cell(fdt, nodename, "value", FINISHER_RESET); - g_free(nodename); - - nodename = g_strdup_printf("/poweroff"); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "syscon-poweroff"); - qemu_fdt_setprop_cell(fdt, nodename, "regmap", test_phandle); - qemu_fdt_setprop_cell(fdt, nodename, "offset", 0x0); - qemu_fdt_setprop_cell(fdt, nodename, "value", FINISHER_PASS); - g_free(nodename); - - nodename = g_strdup_printf("/uart@%lx", - (long)memmap[VIRT_UART0].base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "ns16550a"); - qemu_fdt_setprop_cells(fdt, nodename, "reg", + qemu_fdt_setprop_cell(fdt, name, "phandle", test_phandle); + test_phandle = qemu_fdt_get_phandle(fdt, name); + g_free(name); + + name = g_strdup_printf("/soc/reboot"); + qemu_fdt_add_subnode(fdt, name); + qemu_fdt_setprop_string(fdt, name, "compatible", "syscon-reboot"); + qemu_fdt_setprop_cell(fdt, name, "regmap", test_phandle); + qemu_fdt_setprop_cell(fdt, name, "offset", 0x0); + qemu_fdt_setprop_cell(fdt, name, "value", FINISHER_RESET); + g_free(name); + + name = g_strdup_printf("/soc/poweroff"); + qemu_fdt_add_subnode(fdt, name); + qemu_fdt_setprop_string(fdt, name, "compatible", "syscon-poweroff"); + qemu_fdt_setprop_cell(fdt, name, "regmap", test_phandle); + qemu_fdt_setprop_cell(fdt, name, "offset", 0x0); + qemu_fdt_setprop_cell(fdt, name, "value", FINISHER_PASS); + g_free(name); + + name = g_strdup_printf("/soc/uart@%lx", (long)memmap[VIRT_UART0].base); + qemu_fdt_add_subnode(fdt, name); + qemu_fdt_setprop_string(fdt, name, "compatible", "ns16550a"); + qemu_fdt_setprop_cells(fdt, name, "reg", 0x0, memmap[VIRT_UART0].base, 0x0, memmap[VIRT_UART0].size); - qemu_fdt_setprop_cell(fdt, nodename, "clock-frequency", 3686400); - qemu_fdt_setprop_cell(fdt, nodename, "interrupt-parent", plic_phandle); - qemu_fdt_setprop_cell(fdt, nodename, "interrupts", UART0_IRQ); + qemu_fdt_setprop_cell(fdt, name, "clock-frequency", 3686400); + qemu_fdt_setprop_cell(fdt, name, "interrupt-parent", plic_mmio_phandle); + qemu_fdt_setprop_cell(fdt, name, "interrupts", UART0_IRQ); qemu_fdt_add_subnode(fdt, "/chosen"); - qemu_fdt_setprop_string(fdt, "/chosen", "stdout-path", nodename); + qemu_fdt_setprop_string(fdt, "/chosen", "stdout-path", name); if (cmdline) { qemu_fdt_setprop_string(fdt, "/chosen", "bootargs", cmdline); } - g_free(nodename); - - nodename = g_strdup_printf("/rtc@%lx", - (long)memmap[VIRT_RTC].base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_string(fdt, nodename, "compatible", - "google,goldfish-rtc"); - qemu_fdt_setprop_cells(fdt, nodename, "reg", + g_free(name); + + name = g_strdup_printf("/soc/rtc@%lx", (long)memmap[VIRT_RTC].base); + qemu_fdt_add_subnode(fdt, name); + qemu_fdt_setprop_string(fdt, name, "compatible", "google,goldfish-rtc"); + qemu_fdt_setprop_cells(fdt, name, "reg", 0x0, memmap[VIRT_RTC].base, 0x0, memmap[VIRT_RTC].size); - qemu_fdt_setprop_cell(fdt, nodename, "interrupt-parent", plic_phandle); - qemu_fdt_setprop_cell(fdt, nodename, "interrupts", RTC_IRQ); - g_free(nodename); - - nodename = g_strdup_printf("/flash@%" PRIx64, flashbase); - qemu_fdt_add_subnode(s->fdt, nodename); - qemu_fdt_setprop_string(s->fdt, nodename, "compatible", "cfi-flash"); - qemu_fdt_setprop_sized_cells(s->fdt, nodename, "reg", + qemu_fdt_setprop_cell(fdt, name, "interrupt-parent", plic_mmio_phandle); + qemu_fdt_setprop_cell(fdt, name, "interrupts", RTC_IRQ); + g_free(name); + + name = g_strdup_printf("/soc/flash@%" PRIx64, flashbase); + qemu_fdt_add_subnode(s->fdt, name); + qemu_fdt_setprop_string(s->fdt, name, "compatible", "cfi-flash"); + qemu_fdt_setprop_sized_cells(s->fdt, name, "reg", 2, flashbase, 2, flashsize, 2, flashbase + flashsize, 2, flashsize); - qemu_fdt_setprop_cell(s->fdt, nodename, "bank-width", 4); - g_free(nodename); + qemu_fdt_setprop_cell(s->fdt, name, "bank-width", 4); + g_free(name); } - static inline DeviceState *gpex_pcie_init(MemoryRegion *sys_mem, hwaddr ecam_base, hwaddr ecam_size, hwaddr mmio_base, hwaddr mmio_size, @@ -471,37 +514,128 @@ static inline DeviceState *gpex_pcie_init(MemoryRegion *sys_mem, return dev; } -static void riscv_virt_board_init(MachineState *machine) +static void riscv_virt_board_init(MachineState *mc) { const struct MemmapEntry *memmap = virt_memmap; - RISCVVirtState *s = RISCV_VIRT_MACHINE(machine); + RISCVVirtState *s = RISCV_VIRT_MACHINE(mc); MemoryRegion *system_memory = get_system_memory(); MemoryRegion *main_mem = g_new(MemoryRegion, 1); MemoryRegion *mask_rom = g_new(MemoryRegion, 1); - char *plic_hart_config; + char *plic_hart_config, *soc_name; size_t plic_hart_config_len; target_ulong start_addr = memmap[VIRT_DRAM].base; - int i; - unsigned int smp_cpus = machine->smp.cpus; - - /* Initialize SOC */ - object_initialize_child(OBJECT(machine), "soc", &s->soc, sizeof(s->soc), - TYPE_RISCV_HART_ARRAY, &error_abort, NULL); - object_property_set_str(OBJECT(&s->soc), machine->cpu_type, "cpu-type", - &error_abort); - object_property_set_int(OBJECT(&s->soc), smp_cpus, "num-harts", - &error_abort); - object_property_set_bool(OBJECT(&s->soc), true, "realized", - &error_abort); + DeviceState *mmio_plic, *virtio_plic, *pcie_plic; + unsigned int i, j, base_hartid, hart_count; + uint64_t mem_offset; + + /* Populate socket details based on NUMA options */ + s->num_socs = mc->numa_state->num_nodes; + if (!s->num_socs) { + s->numa_enabled = false; + s->num_socs = 1; + s->mem_offset[0] = 0; + s->mem_size[0] = mc->ram_size; + } else { + s->numa_enabled = true; + s->num_socs = mc->numa_state->num_nodes; + mem_offset = 0; + for (i = 0; i < s->num_socs; i++) { + s->mem_offset[i] = mem_offset; + s->mem_size[i] = mc->numa_state->nodes[i].node_mem; + mem_offset += s->mem_size[i]; + } + } + + /* Initialize sockets */ + mmio_plic = virtio_plic = pcie_plic = NULL; + for (i = 0; i < s->num_socs; i++) { + if (s->numa_enabled) { + base_hartid = mc->smp.cpus; + hart_count = 0; + for (j = 0; j < mc->smp.cpus; j++) { + if (mc->possible_cpus->cpus[j].props.node_id != i) { + continue; + } + if (j < base_hartid) { + base_hartid = j; + } + hart_count++; + } + } else { + base_hartid = 0; + hart_count = mc->smp.cpus; + } + + soc_name = g_strdup_printf("soc%d", i); + object_initialize_child(OBJECT(mc), soc_name, &s->soc[i], + sizeof(s->soc[i]), TYPE_RISCV_HART_ARRAY, &error_abort, NULL); + g_free(soc_name); + object_property_set_str(OBJECT(&s->soc[i]), + mc->cpu_type, "cpu-type", &error_abort); + object_property_set_int(OBJECT(&s->soc[i]), + base_hartid, "hartid-base", &error_abort); + object_property_set_int(OBJECT(&s->soc[i]), + hart_count, "num-harts", &error_abort); + object_property_set_bool(OBJECT(&s->soc[i]), + true, "realized", &error_abort); + + /* Per-socket CLINT */ + sifive_clint_create( + memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size, + memmap[VIRT_CLINT].size, base_hartid, hart_count, + SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, true); + + /* Per-socket PLIC hart topology configuration string */ + plic_hart_config_len = + (strlen(VIRT_PLIC_HART_CONFIG) + 1) * hart_count; + plic_hart_config = g_malloc0(plic_hart_config_len); + for (j = 0; j < hart_count; j++) { + if (j != 0) { + strncat(plic_hart_config, ",", plic_hart_config_len); + } + strncat(plic_hart_config, VIRT_PLIC_HART_CONFIG, + plic_hart_config_len); + plic_hart_config_len -= (strlen(VIRT_PLIC_HART_CONFIG) + 1); + } + + /* Per-socket PLIC */ + s->plic[i] = sifive_plic_create( + memmap[VIRT_PLIC].base + i * memmap[VIRT_PLIC].size, + plic_hart_config, base_hartid, + VIRT_PLIC_NUM_SOURCES, + VIRT_PLIC_NUM_PRIORITIES, + VIRT_PLIC_PRIORITY_BASE, + VIRT_PLIC_PENDING_BASE, + VIRT_PLIC_ENABLE_BASE, + VIRT_PLIC_ENABLE_STRIDE, + VIRT_PLIC_CONTEXT_BASE, + VIRT_PLIC_CONTEXT_STRIDE, + memmap[VIRT_PLIC].size); + g_free(plic_hart_config); + + /* Try to use different PLIC instance based device type */ + if (i == 0) { + mmio_plic = s->plic[i]; + virtio_plic = s->plic[i]; + pcie_plic = s->plic[i]; + } + if (i == 1) { + virtio_plic = s->plic[i]; + pcie_plic = s->plic[i]; + } + if (i == 2) { + pcie_plic = s->plic[i]; + } + } /* register system main memory (actual RAM) */ memory_region_init_ram(main_mem, NULL, "riscv_virt_board.ram", - machine->ram_size, &error_fatal); + mc->ram_size, &error_fatal); memory_region_add_subregion(system_memory, memmap[VIRT_DRAM].base, main_mem); /* create device tree */ - create_fdt(s, memmap, machine->ram_size, machine->kernel_cmdline); + create_fdt(s, memmap, mc->ram_size, mc->kernel_cmdline); /* boot rom */ memory_region_init_rom(mask_rom, NULL, "riscv_virt_board.mrom", @@ -509,17 +643,17 @@ static void riscv_virt_board_init(MachineState *machine) memory_region_add_subregion(system_memory, memmap[VIRT_MROM].base, mask_rom); - riscv_find_and_load_firmware(machine, BIOS_FILENAME, + riscv_find_and_load_firmware(mc, BIOS_FILENAME, memmap[VIRT_DRAM].base, NULL); - if (machine->kernel_filename) { - uint64_t kernel_entry = riscv_load_kernel(machine->kernel_filename, + if (mc->kernel_filename) { + uint64_t kernel_entry = riscv_load_kernel(mc->kernel_filename, NULL); - if (machine->initrd_filename) { + if (mc->initrd_filename) { hwaddr start; - hwaddr end = riscv_load_initrd(machine->initrd_filename, - machine->ram_size, kernel_entry, + hwaddr end = riscv_load_initrd(mc->initrd_filename, + mc->ram_size, kernel_entry, &start); qemu_fdt_setprop_cell(s->fdt, "/chosen", "linux,initrd-start", start); @@ -571,38 +705,14 @@ static void riscv_virt_board_init(MachineState *machine) memmap[VIRT_MROM].base + sizeof(reset_vec), &address_space_memory); - /* create PLIC hart topology configuration string */ - plic_hart_config_len = (strlen(VIRT_PLIC_HART_CONFIG) + 1) * smp_cpus; - plic_hart_config = g_malloc0(plic_hart_config_len); - for (i = 0; i < smp_cpus; i++) { - if (i != 0) { - strncat(plic_hart_config, ",", plic_hart_config_len); - } - strncat(plic_hart_config, VIRT_PLIC_HART_CONFIG, plic_hart_config_len); - plic_hart_config_len -= (strlen(VIRT_PLIC_HART_CONFIG) + 1); - } - - /* MMIO */ - s->plic = sifive_plic_create(memmap[VIRT_PLIC].base, - plic_hart_config, 0, - VIRT_PLIC_NUM_SOURCES, - VIRT_PLIC_NUM_PRIORITIES, - VIRT_PLIC_PRIORITY_BASE, - VIRT_PLIC_PENDING_BASE, - VIRT_PLIC_ENABLE_BASE, - VIRT_PLIC_ENABLE_STRIDE, - VIRT_PLIC_CONTEXT_BASE, - VIRT_PLIC_CONTEXT_STRIDE, - memmap[VIRT_PLIC].size); - sifive_clint_create(memmap[VIRT_CLINT].base, - memmap[VIRT_CLINT].size, 0, smp_cpus, - SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, true); + /* SiFive Test MMIO device */ sifive_test_create(memmap[VIRT_TEST].base); + /* VirtIO MMIO devices */ for (i = 0; i < VIRTIO_COUNT; i++) { sysbus_create_simple("virtio-mmio", memmap[VIRT_VIRTIO].base + i * memmap[VIRT_VIRTIO].size, - qdev_get_gpio_in(DEVICE(s->plic), VIRTIO_IRQ + i)); + qdev_get_gpio_in(DEVICE(virtio_plic), VIRTIO_IRQ + i)); } gpex_pcie_init(system_memory, @@ -611,14 +721,14 @@ static void riscv_virt_board_init(MachineState *machine) memmap[VIRT_PCIE_MMIO].base, memmap[VIRT_PCIE_MMIO].size, memmap[VIRT_PCIE_PIO].base, - DEVICE(s->plic), true); + DEVICE(pcie_plic), true); serial_mm_init(system_memory, memmap[VIRT_UART0].base, - 0, qdev_get_gpio_in(DEVICE(s->plic), UART0_IRQ), 399193, + 0, qdev_get_gpio_in(DEVICE(mmio_plic), UART0_IRQ), 399193, serial_hd(0), DEVICE_LITTLE_ENDIAN); sysbus_create_simple("goldfish_rtc", memmap[VIRT_RTC].base, - qdev_get_gpio_in(DEVICE(s->plic), RTC_IRQ)); + qdev_get_gpio_in(DEVICE(mmio_plic), RTC_IRQ)); virt_flash_create(s); @@ -628,8 +738,54 @@ static void riscv_virt_board_init(MachineState *machine) drive_get(IF_PFLASH, 0, i)); } virt_flash_map(s, system_memory); +} + +static CpuInstanceProperties +riscv_virt_cpu_index_to_props(MachineState *ms, unsigned cpu_index) +{ + MachineClass *mc = MACHINE_GET_CLASS(ms); + const CPUArchIdList *possible_cpus = mc->possible_cpu_arch_ids(ms); + + assert(cpu_index < possible_cpus->len); + return possible_cpus->cpus[cpu_index].props; +} + +static int64_t +riscv_virt_get_default_cpu_node_id(const MachineState *ms, int idx) +{ + int64_t nidx = 0; + + if (ms->numa_state->num_nodes) { + nidx = idx / (ms->smp.cpus / ms->numa_state->num_nodes); + if (ms->numa_state->num_nodes <= nidx) { + nidx = ms->numa_state->num_nodes - 1; + } + } + + return nidx; +} + +static const CPUArchIdList *riscv_virt_possible_cpu_arch_ids(MachineState *ms) +{ + int n; + unsigned int max_cpus = ms->smp.max_cpus; + + if (ms->possible_cpus) { + assert(ms->possible_cpus->len == max_cpus); + return ms->possible_cpus; + } + + ms->possible_cpus = g_malloc0(sizeof(CPUArchIdList) + + sizeof(CPUArchId) * max_cpus); + ms->possible_cpus->len = max_cpus; + for (n = 0; n < ms->possible_cpus->len; n++) { + ms->possible_cpus->cpus[n].type = ms->cpu_type; + ms->possible_cpus->cpus[n].arch_id = n; + ms->possible_cpus->cpus[n].props.has_thread_id = true; + ms->possible_cpus->cpus[n].props.thread_id = n; + } - g_free(plic_hart_config); + return ms->possible_cpus; } static void riscv_virt_machine_instance_init(Object *obj) @@ -642,9 +798,13 @@ static void riscv_virt_machine_class_init(ObjectClass *oc, void *data) mc->desc = "RISC-V VirtIO board"; mc->init = riscv_virt_board_init; - mc->max_cpus = 8; + mc->max_cpus = VIRT_CPUS_MAX; mc->default_cpu_type = VIRT_CPU; mc->pci_allow_0_address = true; + mc->possible_cpu_arch_ids = riscv_virt_possible_cpu_arch_ids; + mc->cpu_index_to_instance_props = riscv_virt_cpu_index_to_props; + mc->get_default_cpu_node_id = riscv_virt_get_default_cpu_node_id; + mc->numa_mem_supported = true; } static const TypeInfo riscv_virt_machine_typeinfo = { diff --git a/include/hw/riscv/virt.h b/include/hw/riscv/virt.h index e69355efaf..61e6dfd454 100644 --- a/include/hw/riscv/virt.h +++ b/include/hw/riscv/virt.h @@ -22,6 +22,9 @@ #include "hw/riscv/riscv_hart.h" #include "hw/sysbus.h" #include "hw/block/flash.h" +#include "sysemu/numa.h" + +#define VIRT_CPUS_MAX 8 #define TYPE_RISCV_VIRT_MACHINE MACHINE_TYPE_NAME("virt") #define RISCV_VIRT_MACHINE(obj) \ @@ -32,8 +35,12 @@ typedef struct { MachineState parent; /*< public >*/ - RISCVHartArrayState soc; - DeviceState *plic; + bool numa_enabled; + unsigned int num_socs; + uint64_t mem_offset[MAX_NODES]; + uint64_t mem_size[MAX_NODES]; + RISCVHartArrayState soc[MAX_NODES]; + DeviceState *plic[MAX_NODES]; PFlashCFI01 *flash[2]; void *fdt; @@ -74,6 +81,8 @@ enum { #define VIRT_PLIC_ENABLE_STRIDE 0x80 #define VIRT_PLIC_CONTEXT_BASE 0x200000 #define VIRT_PLIC_CONTEXT_STRIDE 0x1000 +#define VIRT_PLIC_SIZE(__num_context) \ + (VIRT_PLIC_CONTEXT_BASE + (__num_context) * VIRT_PLIC_CONTEXT_STRIDE) #define FDT_PCI_ADDR_CELLS 3 #define FDT_PCI_INT_CELLS 1