From patchwork Tue Jun 16 03:22:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anup Patel X-Patchwork-Id: 11606297 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 33415912 for ; Tue, 16 Jun 2020 03:23:58 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EE3EB20756 for ; Tue, 16 Jun 2020 03:23:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="pneEY6CY"; dkim=pass (1024-bit key) header.d=sharedspace.onmicrosoft.com header.i=@sharedspace.onmicrosoft.com header.b="bw22knvk" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EE3EB20756 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=wdc.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:59060 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jl2Cb-0001oi-8d for patchwork-qemu-devel@patchwork.kernel.org; Mon, 15 Jun 2020 23:23:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43414) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jl2Bo-0000SV-8t; Mon, 15 Jun 2020 23:23:08 -0400 Received: from esa4.hgst.iphmx.com ([216.71.154.42]:24493) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jl2Bi-0007be-QC; Mon, 15 Jun 2020 23:23:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1592277782; x=1623813782; h=from:to:cc:subject:date:message-id:in-reply-to: references:content-transfer-encoding:mime-version; bh=2/4gAOfhZqEKuUFJczqRipmnJlmMg873gQNyX6WrV+w=; b=pneEY6CYW/3fF1BKYwlFLK5/3veOgsZ3eHY4E34Bb2lhRV24Xag6qzqS D87VNu7d2cZpzL+Z51LOC/zh5t5Mc2bxIPCIkjcDNjBTg5yrBOEwXRdmi 2UFStc0L6DlDSxP2Q72L8XK3zIYmoMrHI7KEhLX/hOLscbQwEbRJnocUF yK04Modr/NmeQNwirZ8yFUHXenrNTBlaF1AKm976EuIMCNXWcUBQ0kmC4 FyLRvDUAqxAC7wVhhrr6o8duh+be/4mU2krIsTUwdJdzkjLrTylpKA0Ws tFnRykkuryryCZ4iNZa8pEBkwnrUKzT0riu+0avrANm2IDU3eFx3hJ/mv g==; IronPort-SDR: 23nPPYk4/Ta9VOOkq8wTT2DHMN4/SwniWIu1WbZt3ZDuRRbWFLHiW6y5hzW8Mys9/dXJP+f4M0 nv6uf//lrGgSxdkyjejmOgeuDDEKReA8LxvVz0/Nq1eNYMzP6EGl5n3H+h7HImyWt7+4ipRV1h C9LOWy4tKb6OYV3wKUnBWUaf7+NT7AO8DO+erlNdpCpKA8nTc77AtnDtioogjshluwC63lUoRv iyErkP0vXZXHfBPi816N3u59mUAZY9+VmSd61OIF6ColZKHoMxT1LyscvC01KchIWTMFEsEEK2 nyA= X-IronPort-AV: E=Sophos;i="5.73,517,1583164800"; d="scan'208";a="140088524" Received: from mail-cys01nam02lp2056.outbound.protection.outlook.com (HELO NAM02-CY1-obe.outbound.protection.outlook.com) ([104.47.37.56]) by ob1.hgst.iphmx.com with ESMTP; 16 Jun 2020 11:22:59 +0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NoMqZUaUSAyDoFw2JvTw9d0xYmgxtnV31qAGJo6AztOhrpt9ba0ryVoDOU25B9i/0jvFFJSqDu5q/Rtmys12qz8CPrplt6e3ti/QT/N/CRLKneyTqKQLUiOaJlbsKuhaWmn/X3NCu+7C/fvTG5b0Egb+tgrVdWwF0hm3HJW2cXzqfPxGcje5+QwpSOtiRtRRiZ8zr9WQvRSTiJDEZxqT06PwWwsfuhhEikIOxMSOeB5hCXz4FVfurFhuPDZWU5eerIKxz58aXO9aXCucfgtX6n6Oxu5QzgUc+BPIITVmTbZRVrawuS1HHmpk0HsGMXheDai2v9s58wtYrzVMz02Snw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BE4PtQUtuBhdFtTA7wq0v7JDakws5ugoQdoLlYDQklA=; b=TFq+uI7MgemkThGuFZTGoxxxZQXfWPLReeALwZIQrfai8ZJ/QU9C84qfMf4vJM6racCjRWKMcQd4BGw2+krVQlaZiIjmRQ4Ing6gy1WIv4Tm4NziEkFJAY5zlx/L6x2E/+QjFbj6GnQmg/acMlIkkW+PJNqSN3JRUI5mZX8TDwuteE1kI0zKtcuI4xwVcmMrBdzlXnAxu5+vwiQwPTMH0Z2g+N8in/ex38MXNjcC1z4Zj6zfG94P0a71I/TNjlAScEPcPGnvI9XtXDBKPZA2esXqPWg+qdl+ZTbYIPZ7hOYMhbVUc7z/AA6KQp68g/ifvYlwoGxJYVxyGmH6TOONsA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wdc.com; dmarc=pass action=none header.from=wdc.com; dkim=pass header.d=wdc.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sharedspace.onmicrosoft.com; s=selector2-sharedspace-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BE4PtQUtuBhdFtTA7wq0v7JDakws5ugoQdoLlYDQklA=; b=bw22knvk5JYcH4w9A5YSCg3JoNC4+E61Orxf1DmreyWGGYwYPSvoNpup85tWiThNVOU81b8a5D9Fy/yHXKWXQDPOegM8M02am7tXIiyJnRvfobbq6lIJlYrWo6VCC03L/O0gTkuD96XjEraaFqvNsTBdvVgerHpbA1gN1J4SpRE= Authentication-Results: linaro.org; dkim=none (message not signed) header.d=none;linaro.org; dmarc=none action=none header.from=wdc.com; Received: from DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) by DM6PR04MB4123.namprd04.prod.outlook.com (2603:10b6:5:98::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.21; Tue, 16 Jun 2020 03:22:58 +0000 Received: from DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::e0a4:aa82:1847:dea5]) by DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::e0a4:aa82:1847:dea5%7]) with mapi id 15.20.3088.029; Tue, 16 Jun 2020 03:22:58 +0000 From: Anup Patel To: Peter Maydell , Palmer Dabbelt , Alistair Francis , Sagar Karandikar Subject: [PATCH v6 1/5] hw/riscv: Allow creating multiple instances of CLINT Date: Tue, 16 Jun 2020 08:52:25 +0530 Message-Id: <20200616032229.766089-2-anup.patel@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200616032229.766089-1-anup.patel@wdc.com> References: <20200616032229.766089-1-anup.patel@wdc.com> X-ClientProxiedBy: MAXPR01CA0099.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:5d::17) To DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from wdc.com (122.171.201.92) by MAXPR01CA0099.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:5d::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.19 via Frontend Transport; Tue, 16 Jun 2020 03:22:54 +0000 X-Mailer: git-send-email 2.25.1 X-Originating-IP: [122.171.201.92] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: e3471103-c3f7-4ef5-7c91-08d811a49117 X-MS-TrafficTypeDiagnostic: DM6PR04MB4123: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: WDCIPOUTBOUND: EOP-TRUE X-MS-Oob-TLC-OOBClassifiers: OLM:3; X-Forefront-PRVS: 04362AC73B X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: RWjaOFRsA7ALJFbmf5cIwGL2b3G/tJ+m5W14M8aA2NOH/Sz15DCBvG59qv3N7GSXP95gv7+EfDKh7aVS+smnLAu15M77fIxiGFsBFEIBUWFBOpT+P6BMXQKLlE89ymcYaTrKegjhrmsdCEEUvosnPvCmtttTjXAbCcb2ZTRFHtFkZ/vDlDomk1DV2QOmn+2GIG21rkn38nYauxrn02qxYDOIrx1xx3maiNR5aD8omjs9WWcdzepdaVYifKe6xAKb1DHOwmIZ5JmpkJriXc+jh7cwOtj2ssKp5nogao4M2F5tIIxnqSX1yP+jnzDVYwBFQqOaTP/upvXeq2v8b9EifA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR04MB6201.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(136003)(366004)(376002)(396003)(346002)(66946007)(2616005)(66476007)(66556008)(8936002)(956004)(2906002)(316002)(54906003)(478600001)(6666004)(186003)(44832011)(55016002)(36756003)(7696005)(52116002)(83380400001)(26005)(8676002)(4326008)(86362001)(16526019)(1076003)(8886007)(110136005)(5660300002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: CbXM8B9RZ5Ltf/h2OKleSZfxfh6YLISwEpUUL2zATnXBhWIKX5yopyS0zWXshDrM9diTbXmtepHH+ofNGD7kz9qmKFpDxbu3HTqvVvKuHrKjLZ4wkN/jwR1SGp4ezaKnwZmbACiYj3KriTevxYZ/vimisjSKT1Yl9QBU0AwhK8L2OSHEg47UFCGeYYtMAWyvAakyr2p6GoRW4RGEke4SDYqpwQbi0P+v5IHU+ZK0A13+MVBL96vZZu40NTY92yU88rNxwTU/JGM0dZQdpwGN87FTjro4Ku8y/usIHMF1Ra6vvoQdY7/UmZWNwJFIDFe9Z4Ljrr7PpT2NbLkmWEHLLJdssYu/G2yeSNDnpR/IdgWruhR6x3j2gMdK5PSHy4KzL6hFUjNE+R+8Or8KLUXQ2F7SmEIzF3AjYTS/qC8sN9GEuRrVyiZWTG2W9DTNIo1IBwi0Lf6ihf31REb5nSYdTmkRboCRK0lRIvgewe6GLMs= X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-Network-Message-Id: e3471103-c3f7-4ef5-7c91-08d811a49117 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2020 03:22:58.1727 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 6q2ThMlKaB9qoXlAhvXa+CmDZG58vxxop/Wxq8IVLmua4H7/wp+jUczumTf1atWWp9oj5zypGBDNjH2tYYP+yA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR04MB4123 Received-SPF: pass client-ip=216.71.154.42; envelope-from=prvs=4298fe783=Anup.Patel@wdc.com; helo=esa4.hgst.iphmx.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/15 23:23:00 X-ACL-Warn: Detected OS = FreeBSD 9.x or newer [fuzzy] X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Palmer Dabbelt , qemu-riscv@nongnu.org, Anup Patel , Anup Patel , qemu-devel@nongnu.org, Atish Patra , Alistair Francis Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We extend CLINT emulation to allow multiple instances of CLINT in a QEMU RISC-V machine. To achieve this, we remove first HART id zero assumption from CLINT emulation. Signed-off-by: Anup Patel Reviewed-by: Alistair Francis Reviewed-by: Palmer Dabbelt --- hw/riscv/sifive_clint.c | 20 ++++++++++++-------- hw/riscv/sifive_e.c | 2 +- hw/riscv/sifive_u.c | 2 +- hw/riscv/spike.c | 2 +- hw/riscv/virt.c | 2 +- include/hw/riscv/sifive_clint.h | 7 ++++--- 6 files changed, 20 insertions(+), 15 deletions(-) diff --git a/hw/riscv/sifive_clint.c b/hw/riscv/sifive_clint.c index e933d35092..7d713fd743 100644 --- a/hw/riscv/sifive_clint.c +++ b/hw/riscv/sifive_clint.c @@ -78,7 +78,7 @@ static uint64_t sifive_clint_read(void *opaque, hwaddr addr, unsigned size) SiFiveCLINTState *clint = opaque; if (addr >= clint->sip_base && addr < clint->sip_base + (clint->num_harts << 2)) { - size_t hartid = (addr - clint->sip_base) >> 2; + size_t hartid = clint->hartid_base + ((addr - clint->sip_base) >> 2); CPUState *cpu = qemu_get_cpu(hartid); CPURISCVState *env = cpu ? cpu->env_ptr : NULL; if (!env) { @@ -91,7 +91,8 @@ static uint64_t sifive_clint_read(void *opaque, hwaddr addr, unsigned size) } } else if (addr >= clint->timecmp_base && addr < clint->timecmp_base + (clint->num_harts << 3)) { - size_t hartid = (addr - clint->timecmp_base) >> 3; + size_t hartid = clint->hartid_base + + ((addr - clint->timecmp_base) >> 3); CPUState *cpu = qemu_get_cpu(hartid); CPURISCVState *env = cpu ? cpu->env_ptr : NULL; if (!env) { @@ -128,7 +129,7 @@ static void sifive_clint_write(void *opaque, hwaddr addr, uint64_t value, if (addr >= clint->sip_base && addr < clint->sip_base + (clint->num_harts << 2)) { - size_t hartid = (addr - clint->sip_base) >> 2; + size_t hartid = clint->hartid_base + ((addr - clint->sip_base) >> 2); CPUState *cpu = qemu_get_cpu(hartid); CPURISCVState *env = cpu ? cpu->env_ptr : NULL; if (!env) { @@ -141,7 +142,8 @@ static void sifive_clint_write(void *opaque, hwaddr addr, uint64_t value, return; } else if (addr >= clint->timecmp_base && addr < clint->timecmp_base + (clint->num_harts << 3)) { - size_t hartid = (addr - clint->timecmp_base) >> 3; + size_t hartid = clint->hartid_base + + ((addr - clint->timecmp_base) >> 3); CPUState *cpu = qemu_get_cpu(hartid); CPURISCVState *env = cpu ? cpu->env_ptr : NULL; if (!env) { @@ -185,6 +187,7 @@ static const MemoryRegionOps sifive_clint_ops = { }; static Property sifive_clint_properties[] = { + DEFINE_PROP_UINT32("hartid-base", SiFiveCLINTState, hartid_base, 0), DEFINE_PROP_UINT32("num-harts", SiFiveCLINTState, num_harts, 0), DEFINE_PROP_UINT32("sip-base", SiFiveCLINTState, sip_base, 0), DEFINE_PROP_UINT32("timecmp-base", SiFiveCLINTState, timecmp_base, 0), @@ -226,13 +229,13 @@ type_init(sifive_clint_register_types) /* * Create CLINT device. */ -DeviceState *sifive_clint_create(hwaddr addr, hwaddr size, uint32_t num_harts, - uint32_t sip_base, uint32_t timecmp_base, uint32_t time_base, - bool provide_rdtime) +DeviceState *sifive_clint_create(hwaddr addr, hwaddr size, + uint32_t hartid_base, uint32_t num_harts, uint32_t sip_base, + uint32_t timecmp_base, uint32_t time_base, bool provide_rdtime) { int i; for (i = 0; i < num_harts; i++) { - CPUState *cpu = qemu_get_cpu(i); + CPUState *cpu = qemu_get_cpu(hartid_base + i); CPURISCVState *env = cpu ? cpu->env_ptr : NULL; if (!env) { continue; @@ -246,6 +249,7 @@ DeviceState *sifive_clint_create(hwaddr addr, hwaddr size, uint32_t num_harts, } DeviceState *dev = qdev_create(NULL, TYPE_SIFIVE_CLINT); + qdev_prop_set_uint32(dev, "hartid-base", hartid_base); qdev_prop_set_uint32(dev, "num-harts", num_harts); qdev_prop_set_uint32(dev, "sip-base", sip_base); qdev_prop_set_uint32(dev, "timecmp-base", timecmp_base); diff --git a/hw/riscv/sifive_e.c b/hw/riscv/sifive_e.c index b53109521e..1c3b37d0ba 100644 --- a/hw/riscv/sifive_e.c +++ b/hw/riscv/sifive_e.c @@ -163,7 +163,7 @@ static void riscv_sifive_e_soc_realize(DeviceState *dev, Error **errp) SIFIVE_E_PLIC_CONTEXT_STRIDE, memmap[SIFIVE_E_PLIC].size); sifive_clint_create(memmap[SIFIVE_E_CLINT].base, - memmap[SIFIVE_E_CLINT].size, ms->smp.cpus, + memmap[SIFIVE_E_CLINT].size, 0, ms->smp.cpus, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, false); create_unimplemented_device("riscv.sifive.e.aon", memmap[SIFIVE_E_AON].base, memmap[SIFIVE_E_AON].size); diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c index 4299bdf480..c193761916 100644 --- a/hw/riscv/sifive_u.c +++ b/hw/riscv/sifive_u.c @@ -602,7 +602,7 @@ static void riscv_sifive_u_soc_realize(DeviceState *dev, Error **errp) sifive_uart_create(system_memory, memmap[SIFIVE_U_UART1].base, serial_hd(1), qdev_get_gpio_in(DEVICE(s->plic), SIFIVE_U_UART1_IRQ)); sifive_clint_create(memmap[SIFIVE_U_CLINT].base, - memmap[SIFIVE_U_CLINT].size, ms->smp.cpus, + memmap[SIFIVE_U_CLINT].size, 0, ms->smp.cpus, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, false); object_property_set_bool(OBJECT(&s->prci), true, "realized", &err); diff --git a/hw/riscv/spike.c b/hw/riscv/spike.c index 7bbbdb5036..a3db885ffa 100644 --- a/hw/riscv/spike.c +++ b/hw/riscv/spike.c @@ -253,7 +253,7 @@ static void spike_board_init(MachineState *machine) /* Core Local Interruptor (timer and IPI) */ sifive_clint_create(memmap[SPIKE_CLINT].base, memmap[SPIKE_CLINT].size, - smp_cpus, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, + 0, smp_cpus, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, false); } diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index 7ce28895bc..041c400261 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -595,7 +595,7 @@ static void riscv_virt_board_init(MachineState *machine) VIRT_PLIC_CONTEXT_STRIDE, memmap[VIRT_PLIC].size); sifive_clint_create(memmap[VIRT_CLINT].base, - memmap[VIRT_CLINT].size, smp_cpus, + memmap[VIRT_CLINT].size, 0, smp_cpus, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, true); sifive_test_create(memmap[VIRT_TEST].base); diff --git a/include/hw/riscv/sifive_clint.h b/include/hw/riscv/sifive_clint.h index 4a720bfece..9f5fb3d31d 100644 --- a/include/hw/riscv/sifive_clint.h +++ b/include/hw/riscv/sifive_clint.h @@ -33,6 +33,7 @@ typedef struct SiFiveCLINTState { /*< public >*/ MemoryRegion mmio; + uint32_t hartid_base; uint32_t num_harts; uint32_t sip_base; uint32_t timecmp_base; @@ -40,9 +41,9 @@ typedef struct SiFiveCLINTState { uint32_t aperture_size; } SiFiveCLINTState; -DeviceState *sifive_clint_create(hwaddr addr, hwaddr size, uint32_t num_harts, - uint32_t sip_base, uint32_t timecmp_base, uint32_t time_base, - bool provide_rdtime); +DeviceState *sifive_clint_create(hwaddr addr, hwaddr size, + uint32_t hartid_base, uint32_t num_harts, uint32_t sip_base, + uint32_t timecmp_base, uint32_t time_base, bool provide_rdtime); enum { SIFIVE_SIP_BASE = 0x0, From patchwork Tue Jun 16 03:22:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anup Patel X-Patchwork-Id: 11606303 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8A34613A0 for ; Tue, 16 Jun 2020 03:25:26 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5111320756 for ; Tue, 16 Jun 2020 03:25:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="e4ov34RF"; dkim=pass (1024-bit key) header.d=sharedspace.onmicrosoft.com header.i=@sharedspace.onmicrosoft.com header.b="SfNNTmIz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5111320756 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=wdc.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:37266 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jl2E1-0004NX-KB for patchwork-qemu-devel@patchwork.kernel.org; Mon, 15 Jun 2020 23:25:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43422) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jl2Bp-0000T2-C6; Mon, 15 Jun 2020 23:23:09 -0400 Received: from esa6.hgst.iphmx.com ([216.71.154.45]:58592) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jl2Bm-0007dC-KE; Mon, 15 Jun 2020 23:23:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1592277787; x=1623813787; h=from:to:cc:subject:date:message-id:in-reply-to: references:content-transfer-encoding:mime-version; bh=itsAG0XTfmOAR7BnB4P9RjBCGIUIt6WWIMNonLcsJHs=; b=e4ov34RF9GTHAWZMomrxnIv3TmWACQr85aJ6276lFADcwwC7cLYCyYUV gIyHYlI7tioFEXCP9zlciz9tGjEysX2+Ii5NtxYPqSnO4dtfQDtNrcV+h qezpW5G6vQkaScYPrxWXSU7wywpzXSNtKHY4XQHl/jG2VxUJQ+e97eteV AxWUyjW/foCoYl07AU1Ye0AatEx9wWwOIpTXlHh+lY30mqPIkVUtSxczQ ftm0AwTXIVZjody+Y4ZPV0XWF6KmPTEaAeDRvJ8UROu51EZSVcdVc43Vm 3QkdirDTOpt+05kjumvwfL4Ljv86LHyOfufNSpOdMPuNoXNhCrei9tLD/ A==; IronPort-SDR: iUCwg690IM2VtgM0wyWWjXrv0+hXnQiiyRxCl14El2m8nz4RHxX2Ig+3n3Zc7y5h6Z628qKGJh K/XjPfa0k5pw3fHLyHp9Zj1a+ASSL3Wt8/9gtdmde26SOX6/AL2j3YK0phmJ9WZVcYbXs70qBG 7QnzuoKvPSA+CP093N+jgqqlqA9l8ycKvGYdDFibU1j7O5BRWlx0NseVJU9YNhdzEDXslAO4Qt 2FPrZIWS2SBgllSh74jbW8h+s6EbS2BFpEKZ3iniFZlE9gbbBNozzOvFLzB4vrXE8D2vf3ta+0 JhE= X-IronPort-AV: E=Sophos;i="5.73,517,1583164800"; d="scan'208";a="141476254" Received: from mail-cys01nam02lp2054.outbound.protection.outlook.com (HELO NAM02-CY1-obe.outbound.protection.outlook.com) ([104.47.37.54]) by ob1.hgst.iphmx.com with ESMTP; 16 Jun 2020 11:23:03 +0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LHNAYWXLoXy6n66OnT3A7zDhAt5C0muhH4WLPAyEG0T6ugmw+p5qu6uaS3fWd/NkwlHiebDJBXx9yq2dd7J/eTNj/mWDBrrbOb4UWHyHaXFQ7H8e+Kr1hfzMSIuKnsnrGSnsLZRvEU1JHM0ZkPhXtFKayXjHBpZbhfmsA4v5HIyHHA0ehMUjfSyXOIenfy+WVAkIXKop9+JrN9oJeGwobXoaGJJZB6ySRduQHFQIetNR/vxRsyLTquWeeK9/kjH69U7dD7HzciHLTPyXNZmAclsvr9SqAkQ5RypKiJEYxMtqdqt6bqtRAF9nQ0p07Jw/AUSbwFq8J52zsan2JjQC6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C1UoFnBMHGDFo/lhpDYAeBqAxyZVXWgqXdU7eGZ82Gc=; b=ckNWY01ajzzeT31tfR4A5IHP7lqGdh0VktqxqS6MZXMPQFZKDma5LjQ6cEQjzdLqwU/rYrVJLLunGIpnhs6nKrHJmT0saXzMq+1lyq4UyGakFhhLpND445x7K9UYI5drnvKcEQtUjwsWVEnD6OgXgoGlr25+PSt3roSzbgCyEgkeIln/YvXLl8Mz5fzaHaL2BsKKn0zIONXN9H9howBpnozs49N8mVbcK2d4VjEed2iliB9rkk+8FhCSlRZZAgiV5Ah4CDIV3VeTNxsQ5QOe9yVB/sCLTRxw7JesqtJUNQO5EKpPFi4lPAk/w6Jo+Y+Y0r1qIOlH0uvCdAc1O+ksNQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wdc.com; dmarc=pass action=none header.from=wdc.com; dkim=pass header.d=wdc.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sharedspace.onmicrosoft.com; s=selector2-sharedspace-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=C1UoFnBMHGDFo/lhpDYAeBqAxyZVXWgqXdU7eGZ82Gc=; b=SfNNTmIzoPx5PZ2ya2HI4/L9VoB1yugKGS9QVk7zq6zDYNOAheoSxR8Ea0gJsmUAqNpNQSHa39skXulLHJeP5IWTajIvbjGv/eSJFH5TPXTr6XfFrXSDU6vQeGkAj+beUI3y71LzIE8kksZcEZdYcWuBb840P6z/BUYCuqDu/YM= Authentication-Results: linaro.org; dkim=none (message not signed) header.d=none;linaro.org; dmarc=none action=none header.from=wdc.com; Received: from DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) by DM6PR04MB4123.namprd04.prod.outlook.com (2603:10b6:5:98::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.21; Tue, 16 Jun 2020 03:23:02 +0000 Received: from DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::e0a4:aa82:1847:dea5]) by DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::e0a4:aa82:1847:dea5%7]) with mapi id 15.20.3088.029; Tue, 16 Jun 2020 03:23:02 +0000 From: Anup Patel To: Peter Maydell , Palmer Dabbelt , Alistair Francis , Sagar Karandikar Subject: [PATCH v6 2/5] hw/riscv: Allow creating multiple instances of PLIC Date: Tue, 16 Jun 2020 08:52:26 +0530 Message-Id: <20200616032229.766089-3-anup.patel@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200616032229.766089-1-anup.patel@wdc.com> References: <20200616032229.766089-1-anup.patel@wdc.com> X-ClientProxiedBy: MAXPR01CA0099.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:5d::17) To DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from wdc.com (122.171.201.92) by MAXPR01CA0099.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:5d::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.19 via Frontend Transport; Tue, 16 Jun 2020 03:22:58 +0000 X-Mailer: git-send-email 2.25.1 X-Originating-IP: [122.171.201.92] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 022e3d58-5fb9-4f26-8023-08d811a49361 X-MS-TrafficTypeDiagnostic: DM6PR04MB4123: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: WDCIPOUTBOUND: EOP-TRUE X-MS-Oob-TLC-OOBClassifiers: OLM:14; X-Forefront-PRVS: 04362AC73B X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9A/aPqAQMgVorHQJuO3YBajFLUJyfnrmrP2fs593DyQcnZvabQT2sxsbrNjnsqG37uyk41m1Vf0Yony37r3ZTDgcK3LGzwGtZsrGW94RRB9iJRpZ6+/LSobx60oIcFoBNTSupXLdzUhedzGA1KCW+ff8W8I54kpXuLqWYRSlOTN0/ax3InaYYbDw6ox5z3BGGnIdAqlLaxKHBAX6fvh+asuLP0S8H94U+CjTKH3IR0bp8qtQvzqo7qUzuk8MA3yGRvSAVRm8OnlgZ+E/Okrlp5W0APbh2b1/6jQfnxpyIlJsa7nrWIYlfxNyGj6YypFOiPNFvPCRAB4XrRogxos6Kg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR04MB6201.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(136003)(366004)(376002)(396003)(346002)(66946007)(2616005)(66476007)(66556008)(8936002)(956004)(2906002)(316002)(54906003)(478600001)(6666004)(186003)(44832011)(55016002)(36756003)(7696005)(52116002)(83380400001)(26005)(8676002)(4326008)(86362001)(16526019)(1076003)(8886007)(110136005)(5660300002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: ieziY19S9e8NoXb/nX6q9uN4Ql980guah7kX65LDnRloueVoTMttFHFyCxT5S3Bd84nChsfPcrqEBNou9qs2VAWlEWOj9XKiM/7T+iQm7z8Ueki09EE9c6VKa8GZNffsvyFAnxkFwoJ1ti4TvnvhwY+g11ywe49DSy/i06P3NZGIGO7XMWRe77M24TQ8edrUgD/pbCGBztyDM8qBIl4yhInSS/5TXBGqnxbsAV+3+oc9KSw8LT8qXHI4oABcTUO/fRJknbDkMsqOc/ZmUoA7WEnCmuBBdNh4voUHzLUrw3R07I1+hX+q+x/AlFUyf+Ut5ucJnPzNZoHs+MEAiUEJcMboyaWSNwMTqU9lGaUYjo3IShtYlMADYYZaWC9PXYQFivoxl/3Ba7lJIW7VocWK0M5V6DWg9Toh6dyEMzMNG06E3Itc+Sd2n/u3nBXFTEfJygsKnq1zTSVinEJ4yAtqeLb93zGx+V/vxz4YgZbxvSE= X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-Network-Message-Id: 022e3d58-5fb9-4f26-8023-08d811a49361 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2020 03:23:02.2421 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nbfR2NDhmSdmBXnWAS8HhhTL5hrc3LFLCPfZe3niLC02ZchzpGy4r5wci9F2s9GcYg7w0hqg4iJduHZ6UeRKdA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR04MB4123 Received-SPF: pass client-ip=216.71.154.45; envelope-from=prvs=4298fe783=Anup.Patel@wdc.com; helo=esa6.hgst.iphmx.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/15 23:23:03 X-ACL-Warn: Detected OS = FreeBSD 9.x or newer [fuzzy] X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Anup Patel , qemu-riscv@nongnu.org, Anup Patel , Palmer Dabbelt , qemu-devel@nongnu.org, Atish Patra , Alistair Francis Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We extend PLIC emulation to allow multiple instances of PLIC in a QEMU RISC-V machine. To achieve this, we remove first HART id zero assumption from PLIC emulation. Signed-off-by: Anup Patel Reviewed-by: Palmer Dabbelt Reviewed-by: Alistair Francis --- hw/riscv/sifive_e.c | 2 +- hw/riscv/sifive_plic.c | 24 +++++++++++++----------- hw/riscv/sifive_u.c | 2 +- hw/riscv/virt.c | 2 +- include/hw/riscv/sifive_plic.h | 12 +++++++----- 5 files changed, 23 insertions(+), 19 deletions(-) diff --git a/hw/riscv/sifive_e.c b/hw/riscv/sifive_e.c index 1c3b37d0ba..bd122e71ae 100644 --- a/hw/riscv/sifive_e.c +++ b/hw/riscv/sifive_e.c @@ -152,7 +152,7 @@ static void riscv_sifive_e_soc_realize(DeviceState *dev, Error **errp) /* MMIO */ s->plic = sifive_plic_create(memmap[SIFIVE_E_PLIC].base, - (char *)SIFIVE_E_PLIC_HART_CONFIG, + (char *)SIFIVE_E_PLIC_HART_CONFIG, 0, SIFIVE_E_PLIC_NUM_SOURCES, SIFIVE_E_PLIC_NUM_PRIORITIES, SIFIVE_E_PLIC_PRIORITY_BASE, diff --git a/hw/riscv/sifive_plic.c b/hw/riscv/sifive_plic.c index c1e04cbb98..f88bb48053 100644 --- a/hw/riscv/sifive_plic.c +++ b/hw/riscv/sifive_plic.c @@ -352,6 +352,7 @@ static const MemoryRegionOps sifive_plic_ops = { static Property sifive_plic_properties[] = { DEFINE_PROP_STRING("hart-config", SiFivePLICState, hart_config), + DEFINE_PROP_UINT32("hartid-base", SiFivePLICState, hartid_base, 0), DEFINE_PROP_UINT32("num-sources", SiFivePLICState, num_sources, 0), DEFINE_PROP_UINT32("num-priorities", SiFivePLICState, num_priorities, 0), DEFINE_PROP_UINT32("priority-base", SiFivePLICState, priority_base, 0), @@ -400,10 +401,12 @@ static void parse_hart_config(SiFivePLICState *plic) } hartid++; - /* store hart/mode combinations */ plic->num_addrs = addrid; + plic->num_harts = hartid; + + /* store hart/mode combinations */ plic->addr_config = g_new(PLICAddr, plic->num_addrs); - addrid = 0, hartid = 0; + addrid = 0, hartid = plic->hartid_base; p = plic->hart_config; while ((c = *p++)) { if (c == ',') { @@ -429,8 +432,6 @@ static void sifive_plic_irq_request(void *opaque, int irq, int level) static void sifive_plic_realize(DeviceState *dev, Error **errp) { - MachineState *ms = MACHINE(qdev_get_machine()); - unsigned int smp_cpus = ms->smp.cpus; SiFivePLICState *plic = SIFIVE_PLIC(dev); int i; @@ -451,8 +452,8 @@ static void sifive_plic_realize(DeviceState *dev, Error **errp) * lost a interrupt in the case a PLIC is attached. The SEIP bit must be * hardware controlled when a PLIC is attached. */ - for (i = 0; i < smp_cpus; i++) { - RISCVCPU *cpu = RISCV_CPU(qemu_get_cpu(i)); + for (i = 0; i < plic->num_harts; i++) { + RISCVCPU *cpu = RISCV_CPU(qemu_get_cpu(plic->hartid_base + i)); if (riscv_cpu_claim_interrupts(cpu, MIP_SEIP) < 0) { error_report("SEIP already claimed"); exit(1); @@ -488,16 +489,17 @@ type_init(sifive_plic_register_types) * Create PLIC device. */ DeviceState *sifive_plic_create(hwaddr addr, char *hart_config, - uint32_t num_sources, uint32_t num_priorities, - uint32_t priority_base, uint32_t pending_base, - uint32_t enable_base, uint32_t enable_stride, - uint32_t context_base, uint32_t context_stride, - uint32_t aperture_size) + uint32_t hartid_base, uint32_t num_sources, + uint32_t num_priorities, uint32_t priority_base, + uint32_t pending_base, uint32_t enable_base, + uint32_t enable_stride, uint32_t context_base, + uint32_t context_stride, uint32_t aperture_size) { DeviceState *dev = qdev_create(NULL, TYPE_SIFIVE_PLIC); assert(enable_stride == (enable_stride & -enable_stride)); assert(context_stride == (context_stride & -context_stride)); qdev_prop_set_string(dev, "hart-config", hart_config); + qdev_prop_set_uint32(dev, "hartid-base", hartid_base); qdev_prop_set_uint32(dev, "num-sources", num_sources); qdev_prop_set_uint32(dev, "num-priorities", num_priorities); qdev_prop_set_uint32(dev, "priority-base", priority_base); diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c index c193761916..53e48e2ff5 100644 --- a/hw/riscv/sifive_u.c +++ b/hw/riscv/sifive_u.c @@ -586,7 +586,7 @@ static void riscv_sifive_u_soc_realize(DeviceState *dev, Error **errp) /* MMIO */ s->plic = sifive_plic_create(memmap[SIFIVE_U_PLIC].base, - plic_hart_config, + plic_hart_config, 0, SIFIVE_U_PLIC_NUM_SOURCES, SIFIVE_U_PLIC_NUM_PRIORITIES, SIFIVE_U_PLIC_PRIORITY_BASE, diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index 041c400261..35dc43d620 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -584,7 +584,7 @@ static void riscv_virt_board_init(MachineState *machine) /* MMIO */ s->plic = sifive_plic_create(memmap[VIRT_PLIC].base, - plic_hart_config, + plic_hart_config, 0, VIRT_PLIC_NUM_SOURCES, VIRT_PLIC_NUM_PRIORITIES, VIRT_PLIC_PRIORITY_BASE, diff --git a/include/hw/riscv/sifive_plic.h b/include/hw/riscv/sifive_plic.h index 4421e81249..ace76d0f1b 100644 --- a/include/hw/riscv/sifive_plic.h +++ b/include/hw/riscv/sifive_plic.h @@ -48,6 +48,7 @@ typedef struct SiFivePLICState { /*< public >*/ MemoryRegion mmio; uint32_t num_addrs; + uint32_t num_harts; uint32_t bitfield_words; PLICAddr *addr_config; uint32_t *source_priority; @@ -58,6 +59,7 @@ typedef struct SiFivePLICState { /* config */ char *hart_config; + uint32_t hartid_base; uint32_t num_sources; uint32_t num_priorities; uint32_t priority_base; @@ -70,10 +72,10 @@ typedef struct SiFivePLICState { } SiFivePLICState; DeviceState *sifive_plic_create(hwaddr addr, char *hart_config, - uint32_t num_sources, uint32_t num_priorities, - uint32_t priority_base, uint32_t pending_base, - uint32_t enable_base, uint32_t enable_stride, - uint32_t context_base, uint32_t context_stride, - uint32_t aperture_size); + uint32_t hartid_base, uint32_t num_sources, + uint32_t num_priorities, uint32_t priority_base, + uint32_t pending_base, uint32_t enable_base, + uint32_t enable_stride, uint32_t context_base, + uint32_t context_stride, uint32_t aperture_size); #endif From patchwork Tue Jun 16 03:22:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anup Patel X-Patchwork-Id: 11606301 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 90D71912 for ; Tue, 16 Jun 2020 03:25:24 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 56FCE20756 for ; Tue, 16 Jun 2020 03:25:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="GOYvy4AX"; dkim=pass (1024-bit key) header.d=sharedspace.onmicrosoft.com header.i=@sharedspace.onmicrosoft.com header.b="h2II+FjO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 56FCE20756 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=wdc.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:37124 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jl2Dz-0004KJ-Jd for patchwork-qemu-devel@patchwork.kernel.org; Mon, 15 Jun 2020 23:25:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43438) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jl2Br-0000Wj-HY; Mon, 15 Jun 2020 23:23:11 -0400 Received: from esa4.hgst.iphmx.com ([216.71.154.42]:24493) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jl2Bo-0007be-Vl; Mon, 15 Jun 2020 23:23:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1592277788; x=1623813788; h=from:to:cc:subject:date:message-id:in-reply-to: references:content-transfer-encoding:mime-version; bh=y/kHILzyxWYJxo2eU9AxUI9BvDDA0qKd2u2xd6QkxGo=; b=GOYvy4AXUP1rYIVCmG7KFsr4eGwMkwR9+2bLJSYnYNGMDW1eZJ3bqI6n yBBvhtee/ZJNsQRXrHmGDn6TmaNIRlZA4SGnPqZ9JoxnFFvr1uGZtcUc5 KF2hk/KHrYO9GKs6k2ciffELjpHU6m70founW1dQ4fIl7DKi+20YH3WjF e+zP76qREMiKIRyKdFCKF/ne+JdydQsR0tNVnhP9zur8md8X/IZYZb8Hm DxrFqcHs0MNeTzzh/RtNmiHfF+xn/z6IXIplQMgAcihceVphOB1y4209c glPDyPLrJzFvFcgpmX6N1mcMpyLgsz+gs/yZPYAcWy2RXuyOTbl4lsCB1 A==; IronPort-SDR: psfjA4/ldvKFUQZZHuEhdYPijnt9ssuXHL5GA9QvCKtQPaUyDGlpLWIsbJilqRuwuYvxrYbuoA /Za+dlaNFQboJ8Q3Y2jsLJNK4trTn+tBYFRTDocIuEHpwaBmv0aCOdXWs91/wr3ovwgmr8npsh gaD+m5tsGiEEhNndJ19n1tyzWRrHH+QB4Zb6YzBVFjha5kYHpym0gC6hOAshBBK3THynBEv71b 0+bGsNHFus1YJjmUUhr98t//e4AaxFnH7eucbHvtB4J9ADS9pcKlm5GtBnbGIyE2VsK6AnKs1j y2A= X-IronPort-AV: E=Sophos;i="5.73,517,1583164800"; d="scan'208";a="140088533" Received: from mail-cys01nam02lp2052.outbound.protection.outlook.com (HELO NAM02-CY1-obe.outbound.protection.outlook.com) ([104.47.37.52]) by ob1.hgst.iphmx.com with ESMTP; 16 Jun 2020 11:23:06 +0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Yaab4sTuq0b0AUDZ28lXazZpiq1TCwGdYbvleaiI4rQZfZCSqu1evvs8rFKXL/rUiVGjRE9cAlSwE2s0hTqsFRtv+5WVQ3uPQq1xc2jARPizzqk0pGRQcU30WgTJcB1m20bB+p+KKw8/h56aSONGlSNYsvjQH4vX3jECEP8T1a2300xHfcKC8cVfnSs7dNAGnczBRJzmLvbXYNJY5fMaK3Ji1zszfPICujyxXZZs9vV4waOgywx5+WT62fFa2YGLHqeLo8OCA6+/1VN0mu5foOWK6VuJAxcE9AxO7dk7lDXRyS9L/TkbQGUgJPrSL89yNAw2mFdKDn8MhWsaRtL0Ig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mxSFodW3VGc14NagqnOVnr5wbkK5cBAVXCAg1VdZgI4=; b=diYJm79o+cnNgQEU3SeDjpnGGlY0OBn2+0wZ1sA6eqEBM2Xcywn07cPV74jvuVyGNgbHQOXYui5QOeMZhcxmXXadHwawwZCfwOJBBsjr9R14dxeUd9OWGgPnF/wiAJvZAxRj8l2OQZXywnShNxMXFT5tRhIfx5rB1SN0VIJYmKeaYmwQWxgMRAmSOVcl8JAxElvoXe/yLNHi3f4N21eKvW8wdtdRkGn58f5Cd91XjLT9JALoZOdXO7Pfn35vl4iiflTGJRD7XPns8PQFj2z/IFCHJEYsuwfMZHc9Inu1HefbPhu6qi6HyDjc13V9xQj2buKqAeX8IyXS3MGGMia4Sw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wdc.com; dmarc=pass action=none header.from=wdc.com; dkim=pass header.d=wdc.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sharedspace.onmicrosoft.com; s=selector2-sharedspace-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mxSFodW3VGc14NagqnOVnr5wbkK5cBAVXCAg1VdZgI4=; b=h2II+FjOaGxtSOtwbK3esjZJqHgLjg8y1bRSmoSeMrmLSNCTd+6nwur2xup6l/YXBLs9D/NX7seMzCVvpKlV8SNNp7dOL6ps4hfLhV5wfi/gEIFYO6MjzAnYqOfeBwic0y5RwwpIKqGnM2bTeVsZe/4tx/Jyy1aHv6wRczpuOJs= Authentication-Results: linaro.org; dkim=none (message not signed) header.d=none;linaro.org; dmarc=none action=none header.from=wdc.com; Received: from DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) by DM6PR04MB4123.namprd04.prod.outlook.com (2603:10b6:5:98::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.21; Tue, 16 Jun 2020 03:23:05 +0000 Received: from DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::e0a4:aa82:1847:dea5]) by DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::e0a4:aa82:1847:dea5%7]) with mapi id 15.20.3088.029; Tue, 16 Jun 2020 03:23:05 +0000 From: Anup Patel To: Peter Maydell , Palmer Dabbelt , Alistair Francis , Sagar Karandikar Subject: [PATCH v6 3/5] hw/riscv: Add helpers for RISC-V multi-socket NUMA machines Date: Tue, 16 Jun 2020 08:52:27 +0530 Message-Id: <20200616032229.766089-4-anup.patel@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200616032229.766089-1-anup.patel@wdc.com> References: <20200616032229.766089-1-anup.patel@wdc.com> X-ClientProxiedBy: MAXPR01CA0099.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:5d::17) To DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from wdc.com (122.171.201.92) by MAXPR01CA0099.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:5d::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.19 via Frontend Transport; Tue, 16 Jun 2020 03:23:02 +0000 X-Mailer: git-send-email 2.25.1 X-Originating-IP: [122.171.201.92] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: d990bc4e-3b15-4e50-de14-08d811a49583 X-MS-TrafficTypeDiagnostic: DM6PR04MB4123: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: WDCIPOUTBOUND: EOP-TRUE X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-Forefront-PRVS: 04362AC73B X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FnCcG1Wr4nxXTzrHuEew3ztEJ5rqtq7qvIyEv82uzL+LLAveYWgkOGoInxl259YhTPjOyAnlOtevaTs1jrl1GeM83xOes6hVrdyW6/C2T4eVMNgW04mQigqwhkk49viBi5G+plJlraDlYi9WbUqEFillp2qG4DF9cyCH77RxQEThby8FoAlK/1fqHehu6dyR4dGj369yvb6KU5l+ICGyK6XXyF/dVfNSDPyW5GF2K7eG/GlGoBVryxTa0HTK0lwkgmfn2ErycKy6ekdxwdxfo8H4TBgvoNGM9+zXTRvH12qHPyg/UFrNtYzMi0GkyDh4/wftWfOwSxGTKg0JDSs1FatuT9etNUI1dB+msl/hCS+E5KKkMBqC2aeD9icnEKLR1n9wtjiHGBio25zYg4v6ow7y4jy7kz12dZ6RBq5UDMXhQjL5Gryg/gXnCPAsOUvCCnu0+I0NxYnwYFprP/UfnQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR04MB6201.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(136003)(366004)(376002)(396003)(346002)(66946007)(2616005)(66476007)(66556008)(8936002)(956004)(2906002)(316002)(54906003)(478600001)(6666004)(186003)(44832011)(55016002)(36756003)(7696005)(52116002)(83380400001)(26005)(8676002)(4326008)(86362001)(30864003)(16526019)(1076003)(8886007)(110136005)(5660300002)(2004002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: ST5Tt6OLzYJIHQpQNRwQkU5MSGklXkithBF3ljPudVnbjTLoqiuJ6nXjDMqVjUJmzvpsKQ+/CSYx/v5vynWlzg5gLK8VWNCz0e+KxfSJ0jpcFAHHD6zd+A4xUA8Zi+D9NQgSRfMN1kR6OJvXav3OsQuOklCEs/oeFmJVz9aKaApEACAZar9p0u9GeRzzJBY3xrlyfbWpkcshVjNsiefwDZ/z4WfFEEVcBhhszFRzNrleOxBD95yx1bIw4n5+yt9RQRVxKqgAeJPHFJMqgBWXzzC9yjLfqkugq8VmFrykyN5g3GCsMl3KBX4n7D2ODGf7AbACmDPRiye6ZafjRF1B+keguwxu3/vGv1BMWrSoKOGtt/PAeqM7lOQH53Iw4TNcWQVWAv4mDdvbMERCjs8tpQ5/HlHJxRwbbBwcH/IvvnSzCyEoJWYVuk9JNEEnG7Vmi9uH1fuOKeCksZjR4HJUlGDHrjwdsY+W406Q66GqqN8= X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-Network-Message-Id: d990bc4e-3b15-4e50-de14-08d811a49583 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2020 03:23:05.6029 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: dm9Tqd+YA7oJLww+VK4B+/FSXX2gY1kjWvcQYqhXB725Kj/LyInbQF9Y8VGkBztsZZZT0QkcgbEZQQQrHPd7Lw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR04MB4123 Received-SPF: pass client-ip=216.71.154.42; envelope-from=prvs=4298fe783=Anup.Patel@wdc.com; helo=esa4.hgst.iphmx.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/15 23:23:00 X-ACL-Warn: Detected OS = FreeBSD 9.x or newer [fuzzy] X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atish Patra , Anup Patel , qemu-riscv@nongnu.org, qemu-devel@nongnu.org, Anup Patel Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We add common helper routines which can be shared by RISC-V multi-socket NUMA machines. We have two types of helpers: 1. riscv_socket_xyz() - These helper assist managing multiple sockets irrespective whether QEMU NUMA is enabled/disabled 2. riscv_numa_xyz() - These helpers assist in providing necessary QEMU machine callbacks for QEMU NUMA emulation Signed-off-by: Anup Patel Reviewed-by: Atish Patra Reviewed-by: Alistair Francis --- hw/riscv/Makefile.objs | 1 + hw/riscv/numa.c | 242 ++++++++++++++++++++++++++++++++++++++++ include/hw/riscv/numa.h | 113 +++++++++++++++++++ 3 files changed, 356 insertions(+) create mode 100644 hw/riscv/numa.c create mode 100644 include/hw/riscv/numa.h diff --git a/hw/riscv/Makefile.objs b/hw/riscv/Makefile.objs index fc3c6dd7c8..4483e61879 100644 --- a/hw/riscv/Makefile.objs +++ b/hw/riscv/Makefile.objs @@ -1,4 +1,5 @@ obj-y += boot.o +obj-y += numa.o obj-$(CONFIG_SPIKE) += riscv_htif.o obj-$(CONFIG_HART) += riscv_hart.o obj-$(CONFIG_SIFIVE_E) += sifive_e.o diff --git a/hw/riscv/numa.c b/hw/riscv/numa.c new file mode 100644 index 0000000000..4f92307102 --- /dev/null +++ b/hw/riscv/numa.c @@ -0,0 +1,242 @@ +/* + * QEMU RISC-V NUMA Helper + * + * Copyright (c) 2020 Western Digital Corporation or its affiliates. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see . + */ + +#include "qemu/osdep.h" +#include "qemu/units.h" +#include "qemu/log.h" +#include "qemu/error-report.h" +#include "qapi/error.h" +#include "hw/boards.h" +#include "hw/qdev-properties.h" +#include "hw/riscv/numa.h" +#include "sysemu/device_tree.h" + +static bool numa_enabled(const MachineState *ms) +{ + return (ms->numa_state && ms->numa_state->num_nodes) ? true : false; +} + +int riscv_socket_count(const MachineState *ms) +{ + return (numa_enabled(ms)) ? ms->numa_state->num_nodes : 1; +} + +int riscv_socket_first_hartid(const MachineState *ms, int socket_id) +{ + int i, first_hartid = ms->smp.cpus; + + if (!numa_enabled(ms)) { + return (!socket_id) ? 0 : -1; + } + + for (i = 0; i < ms->smp.cpus; i++) { + if (ms->possible_cpus->cpus[i].props.node_id != socket_id) { + continue; + } + if (i < first_hartid) { + first_hartid = i; + } + } + + return (first_hartid < ms->smp.cpus) ? first_hartid : -1; +} + +int riscv_socket_last_hartid(const MachineState *ms, int socket_id) +{ + int i, last_hartid = -1; + + if (!numa_enabled(ms)) { + return (!socket_id) ? ms->smp.cpus - 1 : -1; + } + + for (i = 0; i < ms->smp.cpus; i++) { + if (ms->possible_cpus->cpus[i].props.node_id != socket_id) { + continue; + } + if (i > last_hartid) { + last_hartid = i; + } + } + + return (last_hartid < ms->smp.cpus) ? last_hartid : -1; +} + +int riscv_socket_hart_count(const MachineState *ms, int socket_id) +{ + int first_hartid, last_hartid; + + if (!numa_enabled(ms)) { + return (!socket_id) ? ms->smp.cpus : -1; + } + + first_hartid = riscv_socket_first_hartid(ms, socket_id); + if (first_hartid < 0) { + return -1; + } + + last_hartid = riscv_socket_last_hartid(ms, socket_id); + if (last_hartid < 0) { + return -1; + } + + if (first_hartid > last_hartid) { + return -1; + } + + return last_hartid - first_hartid + 1; +} + +bool riscv_socket_check_hartids(const MachineState *ms, int socket_id) +{ + int i, first_hartid, last_hartid; + + if (!numa_enabled(ms)) { + return (!socket_id) ? true : false; + } + + first_hartid = riscv_socket_first_hartid(ms, socket_id); + if (first_hartid < 0) { + return false; + } + + last_hartid = riscv_socket_last_hartid(ms, socket_id); + if (last_hartid < 0) { + return false; + } + + for (i = first_hartid; i <= last_hartid; i++) { + if (ms->possible_cpus->cpus[i].props.node_id != socket_id) { + return false; + } + } + + return true; +} + +uint64_t riscv_socket_mem_offset(const MachineState *ms, int socket_id) +{ + int i; + uint64_t mem_offset = 0; + + if (!numa_enabled(ms)) { + return 0; + } + + for (i = 0; i < ms->numa_state->num_nodes; i++) { + if (i == socket_id) { + break; + } + mem_offset += ms->numa_state->nodes[i].node_mem; + } + + return (i == socket_id) ? mem_offset : 0; +} + +uint64_t riscv_socket_mem_size(const MachineState *ms, int socket_id) +{ + if (!numa_enabled(ms)) { + return (!socket_id) ? ms->ram_size : 0; + } + + return (socket_id < ms->numa_state->num_nodes) ? + ms->numa_state->nodes[socket_id].node_mem : 0; +} + +void riscv_socket_fdt_write_id(const MachineState *ms, void *fdt, + const char *node_name, int socket_id) +{ + if (numa_enabled(ms)) { + qemu_fdt_setprop_cell(fdt, node_name, "numa-node-id", socket_id); + } +} + +void riscv_socket_fdt_write_distance_matrix(const MachineState *ms, void *fdt) +{ + int i, j, idx; + uint32_t *dist_matrix, dist_matrix_size; + + if (numa_enabled(ms) && ms->numa_state->have_numa_distance) { + dist_matrix_size = riscv_socket_count(ms) * riscv_socket_count(ms); + dist_matrix_size *= (3 * sizeof(uint32_t)); + dist_matrix = g_malloc0(dist_matrix_size); + + for (i = 0; i < riscv_socket_count(ms); i++) { + for (j = 0; j < riscv_socket_count(ms); j++) { + idx = (i * riscv_socket_count(ms) + j) * 3; + dist_matrix[idx + 0] = cpu_to_be32(i); + dist_matrix[idx + 1] = cpu_to_be32(j); + dist_matrix[idx + 2] = + cpu_to_be32(ms->numa_state->nodes[i].distance[j]); + } + } + + qemu_fdt_add_subnode(fdt, "/distance-map"); + qemu_fdt_setprop_string(fdt, "/distance-map", "compatible", + "numa-distance-map-v1"); + qemu_fdt_setprop(fdt, "/distance-map", "distance-matrix", + dist_matrix, dist_matrix_size); + g_free(dist_matrix); + } +} + +CpuInstanceProperties +riscv_numa_cpu_index_to_props(MachineState *ms, unsigned cpu_index) +{ + MachineClass *mc = MACHINE_GET_CLASS(ms); + const CPUArchIdList *possible_cpus = mc->possible_cpu_arch_ids(ms); + + assert(cpu_index < possible_cpus->len); + return possible_cpus->cpus[cpu_index].props; +} + +int64_t riscv_numa_get_default_cpu_node_id(const MachineState *ms, int idx) +{ + int64_t nidx = 0; + + if (ms->numa_state->num_nodes) { + nidx = idx / (ms->smp.cpus / ms->numa_state->num_nodes); + if (ms->numa_state->num_nodes <= nidx) { + nidx = ms->numa_state->num_nodes - 1; + } + } + + return nidx; +} + +const CPUArchIdList *riscv_numa_possible_cpu_arch_ids(MachineState *ms) +{ + int n; + unsigned int max_cpus = ms->smp.max_cpus; + + if (ms->possible_cpus) { + assert(ms->possible_cpus->len == max_cpus); + return ms->possible_cpus; + } + + ms->possible_cpus = g_malloc0(sizeof(CPUArchIdList) + + sizeof(CPUArchId) * max_cpus); + ms->possible_cpus->len = max_cpus; + for (n = 0; n < ms->possible_cpus->len; n++) { + ms->possible_cpus->cpus[n].type = ms->cpu_type; + ms->possible_cpus->cpus[n].arch_id = n; + ms->possible_cpus->cpus[n].props.has_core_id = true; + ms->possible_cpus->cpus[n].props.core_id = n; + } + + return ms->possible_cpus; +} diff --git a/include/hw/riscv/numa.h b/include/hw/riscv/numa.h new file mode 100644 index 0000000000..fcce942cee --- /dev/null +++ b/include/hw/riscv/numa.h @@ -0,0 +1,113 @@ +/* + * QEMU RISC-V NUMA Helper + * + * Copyright (c) 2020 Western Digital Corporation or its affiliates. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2 or later, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program. If not, see . + */ + +#ifndef RISCV_NUMA_H +#define RISCV_NUMA_H + +#include "hw/sysbus.h" +#include "sysemu/numa.h" + +/** + * riscv_socket_count: + * @ms: pointer to machine state + * + * Returns: number of sockets for a numa system and 1 for a non-numa system + */ +int riscv_socket_count(const MachineState *ms); + +/** + * riscv_socket_first_hartid: + * @ms: pointer to machine state + * @socket_id: socket index + * + * Returns: first hartid for a valid socket and -1 for an invalid socket + */ +int riscv_socket_first_hartid(const MachineState *ms, int socket_id); + +/** + * riscv_socket_last_hartid: + * @ms: pointer to machine state + * @socket_id: socket index + * + * Returns: last hartid for a valid socket and -1 for an invalid socket + */ +int riscv_socket_last_hartid(const MachineState *ms, int socket_id); + +/** + * riscv_socket_hart_count: + * @ms: pointer to machine state + * @socket_id: socket index + * + * Returns: number of harts for a valid socket and -1 for an invalid socket + */ +int riscv_socket_hart_count(const MachineState *ms, int socket_id); + +/** + * riscv_socket_mem_offset: + * @ms: pointer to machine state + * @socket_id: socket index + * + * Returns: offset of ram belonging to given socket + */ +uint64_t riscv_socket_mem_offset(const MachineState *ms, int socket_id); + +/** + * riscv_socket_mem_size: + * @ms: pointer to machine state + * @socket_id: socket index + * + * Returns: size of ram belonging to given socket + */ +uint64_t riscv_socket_mem_size(const MachineState *ms, int socket_id); + +/** + * riscv_socket_check_hartids: + * @ms: pointer to machine state + * @socket_id: socket index + * + * Returns: true if hardids belonging to given socket are contiguous else false + */ +bool riscv_socket_check_hartids(const MachineState *ms, int socket_id); + +/** + * riscv_socket_fdt_write_id: + * @ms: pointer to machine state + * @socket_id: socket index + * + * Write NUMA node-id FDT property for given FDT node + */ +void riscv_socket_fdt_write_id(const MachineState *ms, void *fdt, + const char *node_name, int socket_id); + +/** + * riscv_socket_fdt_write_distance_matrix: + * @ms: pointer to machine state + * @socket_id: socket index + * + * Write NUMA distance matrix in FDT for given machine + */ +void riscv_socket_fdt_write_distance_matrix(const MachineState *ms, void *fdt); + +CpuInstanceProperties +riscv_numa_cpu_index_to_props(MachineState *ms, unsigned cpu_index); + +int64_t riscv_numa_get_default_cpu_node_id(const MachineState *ms, int idx); + +const CPUArchIdList *riscv_numa_possible_cpu_arch_ids(MachineState *ms); + +#endif /* RISCV_NUMA_H */ From patchwork Tue Jun 16 03:22:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anup Patel X-Patchwork-Id: 11606299 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 37C6D912 for ; Tue, 16 Jun 2020 03:24:04 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F2E7A20756 for ; Tue, 16 Jun 2020 03:24:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="hi61cfIQ"; dkim=pass (1024-bit key) header.d=sharedspace.onmicrosoft.com header.i=@sharedspace.onmicrosoft.com header.b="vny72X3l" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F2E7A20756 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=wdc.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:59532 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jl2Ch-00020M-6e for patchwork-qemu-devel@patchwork.kernel.org; Mon, 15 Jun 2020 23:24:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43454) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jl2Bu-0000e9-VK; Mon, 15 Jun 2020 23:23:15 -0400 Received: from esa4.hgst.iphmx.com ([216.71.154.42]:24515) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jl2Bs-0007eJ-0k; Mon, 15 Jun 2020 23:23:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1592277791; x=1623813791; h=from:to:cc:subject:date:message-id:in-reply-to: references:content-transfer-encoding:mime-version; bh=ezECuzNS0IoOqxWV/2ws7/kITKDf8Sw3Hg7/34rQsgc=; b=hi61cfIQoBPV99LdlqI9oacg9x9307YICreC09lx/XYAmsDuQimkZCf9 hEKb4VVOhw2wZwljEePfg0GfiUNM6MItfCbfQWk/EugiwJBlCNTcleMTQ D1DqLzG4lC0yBs6EGmtDD6nzqsJ/TUgxQ98bCKTDV/A1i25IiAxritoiy 4d4+qBj7krn+SKhzh3AV/xMiPZ9vNjdR1AQLPORoVC7Bj9cOG8d90jW9O 3f7f1XFdLHZySRIaisBXsuXWeryXOkGs0ZJB+xC5Ri1jitnw8G/fBQt/h JxVYUpDofzs3We2OFMGYA1kDTcR9rAExfF2yBe8OD8LPPhEooEiLHGFQH g==; IronPort-SDR: c4UPZLOYzB+WPkLce3avfGy0vLI0UwDTjK4YbUaScZx/3FvlbqPMct8Q+Ko9x9ruHeZrMhBTBm PF8b5GgCOWsDLH18gQcoBd8EERpUBJJoTEIrCxkQ5ReUs/iAYNkbIYbJR2Zrb8G7OoUJoQFi3r VfPpOxd3FwS0jklfonCIRQOOqZR3TlH3sWGqjnnNiUWVgbXB1lgc9zYNiFyK+npLWcTSC4hhYf rFqbI53opCOIgCWOeXXnaKgzo6g4rN12upl23nduRbo65jI94/gTDXeqmRkGV5IK+/t+0a/KFi ef8= X-IronPort-AV: E=Sophos;i="5.73,517,1583164800"; d="scan'208";a="140088540" Received: from mail-cys01nam02lp2059.outbound.protection.outlook.com (HELO NAM02-CY1-obe.outbound.protection.outlook.com) ([104.47.37.59]) by ob1.hgst.iphmx.com with ESMTP; 16 Jun 2020 11:23:09 +0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=goFWR6Yj+TBnanDUgyvJTZ8jlx82+ubwl773Sfr0aXd/mGJV1Fl0ME/beibr4lZye3H0pijwmY3K5z+eNCVvaTj86djQ3N4wnLgCSUugdXZiVfU2mxJWG9OEZg9hnIpv47McHZiUUAhoKnToX4QDSRjvmeG79H5PEJn/LOuMjl6B5Qn+IhaM4ZlWQDP135U7VOZbk8pEzO1Gq9pbqajeHIuR/bowvyCZNQnNrNDBh5Xz8eoNlVCGnnuJXGQqLCsbMvpdca5JBLCDs01XsY9bqMX8GKBUzzgMZJYNSGf5ypjCAPdBMvq8B6nQ4eo79gsa6CjTOoX9OULkeyVevDsPCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vt5N3ZRIuVPuxPFlhyqyFOzvUgP2Ab2b7m19eXpTG1g=; b=oJEiU9W3Y74Xenhq9Y/DDksgCK/SuuUmfP40KHYP6o5aReBaP9s/8P++JMSzUcFGcYYXDAr7vx6otzW7MFaarhRC4MKpwDXtXvsPeVT3R03O3tu5gls10tBz4k4X2bChCSRIOFo9NwFc+QFAGROo4PSqCzKRUMLYrLbCtz+IX64R62RjvMP1/68L6WnpwqoOL8VkdjJS5fWG6x+GAyJ8qlIMatzAoYNNs5m5j9+QJWRe+eETa4RVLzNfHkcQwWCDv3V2q8MTOFpGHd8VsImF4Ff9NzgkAlZ9xNWK2VwVq9Z1DV8wY01lPVykFco63wrbR1kxSvlaWFKhcL+4d6wvZA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wdc.com; dmarc=pass action=none header.from=wdc.com; dkim=pass header.d=wdc.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sharedspace.onmicrosoft.com; s=selector2-sharedspace-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vt5N3ZRIuVPuxPFlhyqyFOzvUgP2Ab2b7m19eXpTG1g=; b=vny72X3lSsd01Vy3oX2nga2BBDfZ82ZaK8DmsrRfMeXZc5cFIgmZTg9ewtzCnekd7x8IcNSws76NuVijQlzEyJtH8A4w3Y3kpS3BKIdPrZr/7kwUnmwq1FyuElcsKfsXOVDLF0ShtfXeO/0Wr4Xmaepdd2Zi5uwSO0ORdJ7sLD4= Authentication-Results: linaro.org; dkim=none (message not signed) header.d=none;linaro.org; dmarc=none action=none header.from=wdc.com; Received: from DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) by DM6PR04MB4123.namprd04.prod.outlook.com (2603:10b6:5:98::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.21; Tue, 16 Jun 2020 03:23:09 +0000 Received: from DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::e0a4:aa82:1847:dea5]) by DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::e0a4:aa82:1847:dea5%7]) with mapi id 15.20.3088.029; Tue, 16 Jun 2020 03:23:09 +0000 From: Anup Patel To: Peter Maydell , Palmer Dabbelt , Alistair Francis , Sagar Karandikar Subject: [PATCH v6 4/5] hw/riscv: spike: Allow creating multiple NUMA sockets Date: Tue, 16 Jun 2020 08:52:28 +0530 Message-Id: <20200616032229.766089-5-anup.patel@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200616032229.766089-1-anup.patel@wdc.com> References: <20200616032229.766089-1-anup.patel@wdc.com> X-ClientProxiedBy: MAXPR01CA0099.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:5d::17) To DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from wdc.com (122.171.201.92) by MAXPR01CA0099.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:5d::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.19 via Frontend Transport; Tue, 16 Jun 2020 03:23:06 +0000 X-Mailer: git-send-email 2.25.1 X-Originating-IP: [122.171.201.92] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: b0457a30-4c58-49ee-ecb1-08d811a49783 X-MS-TrafficTypeDiagnostic: DM6PR04MB4123: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: WDCIPOUTBOUND: EOP-TRUE X-MS-Oob-TLC-OOBClassifiers: OLM:923; X-Forefront-PRVS: 04362AC73B X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: eSPsLe6rG2cBMQR209ssHx8NSfJ/mKw3ngmLC22EplgZTmpqWC7xRd96CfvGhbwcL8TPzFd4l8qU5vA7EaZUPo9bUswyCknBz/VBkdDsZl134r592im5ZW7sQaQvgj7DBAzDBRBSzhtDryg+9jKrbS91rE1sIZyNfCNlRxe97UzieFYWcEX+XQqD4ARx3kB88kaWHFhILJ2h5nLrrlliHJHesbe31nSseu7aaK9lX0HXpEyEsDdiXT9hw/4L2uouGbfUaizShC07euYYeqDOT6fSFlRzjxOskx372bsz5if9GyORphifr6I/X432rCDhG7144RcqwU4tj86lecJndg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR04MB6201.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(136003)(366004)(376002)(396003)(346002)(66946007)(2616005)(66476007)(66556008)(8936002)(956004)(2906002)(316002)(54906003)(478600001)(6666004)(186003)(44832011)(55016002)(36756003)(7696005)(52116002)(83380400001)(26005)(8676002)(4326008)(86362001)(30864003)(16526019)(1076003)(8886007)(110136005)(5660300002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: zBfwQbzVVhoSSzxlwZh5VvQ8iapCMzDoIjjVzjUzB74+W/30xId7FoDEqxHtvSDq2Zg+kIC97D895kxrcmqVHfkynyNBH7mnfxZJcfhQSVFV6XiGhdeoW4kGmQTTMJ5FXclEy43g5qt9n36hVAXNlaSpHmRPe86wS0dx50HFomV71RGa7q9lvst5WTcNTiyWoP+56CxyBvM5KsCSGVQ3AQwInk972EcerY77+N6ht1opd42rrHnMquzzyb5CqMwg/VLZxfYgwvqQpR9aUDKivz/6DcaHjeeUS3EjBFueKkFlwVeXm9Oje5mvD38fSnlDZQ1TkTAAMptwip5uK5EAL5npFu7iaJaKqaMiceE3Eg5QPL9kynHHU5a2upr1CHp7UeKoiN11c2xjBLrbrYnfs/59bUHz3I/O9nkGEEUpvQE74FBSN02NypZjIL2L7I6SqA8JBjzxyOErPQfrfLew27esUI7cVUxoFBDdsI6pMAc= X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-Network-Message-Id: b0457a30-4c58-49ee-ecb1-08d811a49783 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2020 03:23:09.1646 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ZpIul438m/90bk6jxVPOWKp/eftWk9VA57kAsWXbsJhHiKeIIiOKykwBbO0DkBU2luK0Mogws/CehbwqqdnggQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR04MB4123 Received-SPF: pass client-ip=216.71.154.42; envelope-from=prvs=4298fe783=Anup.Patel@wdc.com; helo=esa4.hgst.iphmx.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/15 23:23:00 X-ACL-Warn: Detected OS = FreeBSD 9.x or newer [fuzzy] X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atish Patra , Anup Patel , qemu-riscv@nongnu.org, qemu-devel@nongnu.org, Anup Patel Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We extend RISC-V spike machine to allow creating a multi-socket machine. Each RISC-V spike machine socket is a NUMA node having a set of HARTs, a memory instance, and a CLINT instance. Other devices are shared between all sockets. We also update the generated device tree accordingly. By default, NUMA multi-socket support is disabled for RISC-V spike machine. To enable it, users can use "-numa" command-line options of QEMU. Example1: For two NUMA nodes with 2 CPUs each, append following to command-line options: "-smp 4 -numa node -numa node" Example2: For two NUMA nodes with 1 and 3 CPUs, append following to command-line options: "-smp 4 -numa node -numa node -numa cpu,node-id=0,core-id=0 \ -numa cpu,node-id=1,core-id=1 -numa cpu,node-id=1,core-id=2 \ -numa cpu,node-id=1,core-id=3" The maximum number of sockets in a RISC-V spike machine is 8 but this limit can be changed in future. Signed-off-by: Anup Patel Reviewed-by: Atish Patra Reviewed-by: Alistair Francis --- hw/riscv/spike.c | 237 ++++++++++++++++++++++++++------------- include/hw/riscv/spike.h | 11 +- 2 files changed, 169 insertions(+), 79 deletions(-) diff --git a/hw/riscv/spike.c b/hw/riscv/spike.c index a3db885ffa..29e7270035 100644 --- a/hw/riscv/spike.c +++ b/hw/riscv/spike.c @@ -36,6 +36,7 @@ #include "hw/riscv/sifive_clint.h" #include "hw/riscv/spike.h" #include "hw/riscv/boot.h" +#include "hw/riscv/numa.h" #include "chardev/char.h" #include "sysemu/arch_init.h" #include "sysemu/device_tree.h" @@ -64,9 +65,14 @@ static void create_fdt(SpikeState *s, const struct MemmapEntry *memmap, uint64_t mem_size, const char *cmdline) { void *fdt; - int cpu; - uint32_t *cells; - char *nodename; + uint64_t addr, size; + unsigned long clint_addr; + int cpu, socket; + MachineState *mc = MACHINE(s); + uint32_t *clint_cells; + uint32_t cpu_phandle, intc_phandle, phandle = 1; + char *name, *mem_name, *clint_name, *clust_name; + char *core_name, *cpu_name, *intc_name; fdt = s->fdt = create_device_tree(&s->fdt_size); if (!fdt) { @@ -88,68 +94,91 @@ static void create_fdt(SpikeState *s, const struct MemmapEntry *memmap, qemu_fdt_setprop_cell(fdt, "/soc", "#size-cells", 0x2); qemu_fdt_setprop_cell(fdt, "/soc", "#address-cells", 0x2); - nodename = g_strdup_printf("/memory@%lx", - (long)memmap[SPIKE_DRAM].base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_cells(fdt, nodename, "reg", - memmap[SPIKE_DRAM].base >> 32, memmap[SPIKE_DRAM].base, - mem_size >> 32, mem_size); - qemu_fdt_setprop_string(fdt, nodename, "device_type", "memory"); - g_free(nodename); - qemu_fdt_add_subnode(fdt, "/cpus"); qemu_fdt_setprop_cell(fdt, "/cpus", "timebase-frequency", SIFIVE_CLINT_TIMEBASE_FREQ); qemu_fdt_setprop_cell(fdt, "/cpus", "#size-cells", 0x0); qemu_fdt_setprop_cell(fdt, "/cpus", "#address-cells", 0x1); + qemu_fdt_add_subnode(fdt, "/cpus/cpu-map"); + + for (socket = (riscv_socket_count(mc) - 1); socket >= 0; socket--) { + clust_name = g_strdup_printf("/cpus/cpu-map/cluster%d", socket); + qemu_fdt_add_subnode(fdt, clust_name); + + clint_cells = g_new0(uint32_t, s->soc[socket].num_harts * 4); - for (cpu = s->soc.num_harts - 1; cpu >= 0; cpu--) { - nodename = g_strdup_printf("/cpus/cpu@%d", cpu); - char *intc = g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu); - char *isa = riscv_isa_string(&s->soc.harts[cpu]); - qemu_fdt_add_subnode(fdt, nodename); + for (cpu = s->soc[socket].num_harts - 1; cpu >= 0; cpu--) { + cpu_phandle = phandle++; + + cpu_name = g_strdup_printf("/cpus/cpu@%d", + s->soc[socket].hartid_base + cpu); + qemu_fdt_add_subnode(fdt, cpu_name); #if defined(TARGET_RISCV32) - qemu_fdt_setprop_string(fdt, nodename, "mmu-type", "riscv,sv32"); + qemu_fdt_setprop_string(fdt, cpu_name, "mmu-type", "riscv,sv32"); #else - qemu_fdt_setprop_string(fdt, nodename, "mmu-type", "riscv,sv48"); + qemu_fdt_setprop_string(fdt, cpu_name, "mmu-type", "riscv,sv48"); #endif - qemu_fdt_setprop_string(fdt, nodename, "riscv,isa", isa); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "riscv"); - qemu_fdt_setprop_string(fdt, nodename, "status", "okay"); - qemu_fdt_setprop_cell(fdt, nodename, "reg", cpu); - qemu_fdt_setprop_string(fdt, nodename, "device_type", "cpu"); - qemu_fdt_add_subnode(fdt, intc); - qemu_fdt_setprop_cell(fdt, intc, "phandle", 1); - qemu_fdt_setprop_string(fdt, intc, "compatible", "riscv,cpu-intc"); - qemu_fdt_setprop(fdt, intc, "interrupt-controller", NULL, 0); - qemu_fdt_setprop_cell(fdt, intc, "#interrupt-cells", 1); - g_free(isa); - g_free(intc); - g_free(nodename); - } + name = riscv_isa_string(&s->soc[socket].harts[cpu]); + qemu_fdt_setprop_string(fdt, cpu_name, "riscv,isa", name); + g_free(name); + qemu_fdt_setprop_string(fdt, cpu_name, "compatible", "riscv"); + qemu_fdt_setprop_string(fdt, cpu_name, "status", "okay"); + qemu_fdt_setprop_cell(fdt, cpu_name, "reg", + s->soc[socket].hartid_base + cpu); + qemu_fdt_setprop_string(fdt, cpu_name, "device_type", "cpu"); + riscv_socket_fdt_write_id(mc, fdt, cpu_name, socket); + qemu_fdt_setprop_cell(fdt, cpu_name, "phandle", cpu_phandle); - cells = g_new0(uint32_t, s->soc.num_harts * 4); - for (cpu = 0; cpu < s->soc.num_harts; cpu++) { - nodename = - g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu); - uint32_t intc_phandle = qemu_fdt_get_phandle(fdt, nodename); - cells[cpu * 4 + 0] = cpu_to_be32(intc_phandle); - cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_SOFT); - cells[cpu * 4 + 2] = cpu_to_be32(intc_phandle); - cells[cpu * 4 + 3] = cpu_to_be32(IRQ_M_TIMER); - g_free(nodename); + intc_name = g_strdup_printf("%s/interrupt-controller", cpu_name); + qemu_fdt_add_subnode(fdt, intc_name); + intc_phandle = phandle++; + qemu_fdt_setprop_cell(fdt, intc_name, "phandle", intc_phandle); + qemu_fdt_setprop_string(fdt, intc_name, "compatible", + "riscv,cpu-intc"); + qemu_fdt_setprop(fdt, intc_name, "interrupt-controller", NULL, 0); + qemu_fdt_setprop_cell(fdt, intc_name, "#interrupt-cells", 1); + + clint_cells[cpu * 4 + 0] = cpu_to_be32(intc_phandle); + clint_cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_SOFT); + clint_cells[cpu * 4 + 2] = cpu_to_be32(intc_phandle); + clint_cells[cpu * 4 + 3] = cpu_to_be32(IRQ_M_TIMER); + + core_name = g_strdup_printf("%s/core%d", clust_name, cpu); + qemu_fdt_add_subnode(fdt, core_name); + qemu_fdt_setprop_cell(fdt, core_name, "cpu", cpu_phandle); + + g_free(core_name); + g_free(intc_name); + g_free(cpu_name); + } + + addr = memmap[SPIKE_DRAM].base + riscv_socket_mem_offset(mc, socket); + size = riscv_socket_mem_size(mc, socket); + mem_name = g_strdup_printf("/memory@%lx", (long)addr); + qemu_fdt_add_subnode(fdt, mem_name); + qemu_fdt_setprop_cells(fdt, mem_name, "reg", + addr >> 32, addr, size >> 32, size); + qemu_fdt_setprop_string(fdt, mem_name, "device_type", "memory"); + riscv_socket_fdt_write_id(mc, fdt, mem_name, socket); + g_free(mem_name); + + clint_addr = memmap[SPIKE_CLINT].base + + (memmap[SPIKE_CLINT].size * socket); + clint_name = g_strdup_printf("/soc/clint@%lx", clint_addr); + qemu_fdt_add_subnode(fdt, clint_name); + qemu_fdt_setprop_string(fdt, clint_name, "compatible", "riscv,clint0"); + qemu_fdt_setprop_cells(fdt, clint_name, "reg", + 0x0, clint_addr, 0x0, memmap[SPIKE_CLINT].size); + qemu_fdt_setprop(fdt, clint_name, "interrupts-extended", + clint_cells, s->soc[socket].num_harts * sizeof(uint32_t) * 4); + riscv_socket_fdt_write_id(mc, fdt, clint_name, socket); + + g_free(clint_name); + g_free(clint_cells); + g_free(clust_name); } - nodename = g_strdup_printf("/soc/clint@%lx", - (long)memmap[SPIKE_CLINT].base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "riscv,clint0"); - qemu_fdt_setprop_cells(fdt, nodename, "reg", - 0x0, memmap[SPIKE_CLINT].base, - 0x0, memmap[SPIKE_CLINT].size); - qemu_fdt_setprop(fdt, nodename, "interrupts-extended", - cells, s->soc.num_harts * sizeof(uint32_t) * 4); - g_free(cells); - g_free(nodename); + + riscv_socket_fdt_write_distance_matrix(mc, fdt); if (cmdline) { qemu_fdt_add_subnode(fdt, "/chosen"); @@ -160,23 +189,58 @@ static void create_fdt(SpikeState *s, const struct MemmapEntry *memmap, static void spike_board_init(MachineState *machine) { const struct MemmapEntry *memmap = spike_memmap; - - SpikeState *s = g_new0(SpikeState, 1); + SpikeState *s = SPIKE_MACHINE(machine); MemoryRegion *system_memory = get_system_memory(); MemoryRegion *main_mem = g_new(MemoryRegion, 1); MemoryRegion *mask_rom = g_new(MemoryRegion, 1); - int i; - unsigned int smp_cpus = machine->smp.cpus; - - /* Initialize SOC */ - object_initialize_child(OBJECT(machine), "soc", &s->soc, sizeof(s->soc), - TYPE_RISCV_HART_ARRAY, &error_abort, NULL); - object_property_set_str(OBJECT(&s->soc), machine->cpu_type, "cpu-type", - &error_abort); - object_property_set_int(OBJECT(&s->soc), smp_cpus, "num-harts", - &error_abort); - object_property_set_bool(OBJECT(&s->soc), true, "realized", - &error_abort); + char *soc_name; + int i, base_hartid, hart_count; + + /* Check socket count limit */ + if (SPIKE_SOCKETS_MAX < riscv_socket_count(machine)) { + error_report("number of sockets/nodes should be less than %d", + SPIKE_SOCKETS_MAX); + exit(1); + } + + /* Initialize sockets */ + for (i = 0; i < riscv_socket_count(machine); i++) { + if (!riscv_socket_check_hartids(machine, i)) { + error_report("discontinuous hartids in socket%d", i); + exit(1); + } + + base_hartid = riscv_socket_first_hartid(machine, i); + if (base_hartid < 0) { + error_report("can't find hartid base for socket%d", i); + exit(1); + } + + hart_count = riscv_socket_hart_count(machine, i); + if (hart_count < 0) { + error_report("can't find hart count for socket%d", i); + exit(1); + } + + soc_name = g_strdup_printf("soc%d", i); + object_initialize_child(OBJECT(machine), soc_name, &s->soc[i], + sizeof(s->soc[i]), TYPE_RISCV_HART_ARRAY, &error_abort, NULL); + g_free(soc_name); + object_property_set_str(OBJECT(&s->soc[i]), + machine->cpu_type, "cpu-type", &error_abort); + object_property_set_int(OBJECT(&s->soc[i]), + base_hartid, "hartid-base", &error_abort); + object_property_set_int(OBJECT(&s->soc[i]), + hart_count, "num-harts", &error_abort); + object_property_set_bool(OBJECT(&s->soc[i]), + true, "realized", &error_abort); + + /* Core Local Interruptor (timer and IPI) for each socket */ + sifive_clint_create( + memmap[SPIKE_CLINT].base + i * memmap[SPIKE_CLINT].size, + memmap[SPIKE_CLINT].size, base_hartid, hart_count, + SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, false); + } /* register system main memory (actual RAM) */ memory_region_init_ram(main_mem, NULL, "riscv.spike.ram", @@ -249,21 +313,40 @@ static void spike_board_init(MachineState *machine) &address_space_memory); /* initialize HTIF using symbols found in load_kernel */ - htif_mm_init(system_memory, mask_rom, &s->soc.harts[0].env, serial_hd(0)); + htif_mm_init(system_memory, mask_rom, + &s->soc[0].harts[0].env, serial_hd(0)); +} - /* Core Local Interruptor (timer and IPI) */ - sifive_clint_create(memmap[SPIKE_CLINT].base, memmap[SPIKE_CLINT].size, - 0, smp_cpus, SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, - false); +static void spike_machine_instance_init(Object *obj) +{ } -static void spike_machine_init(MachineClass *mc) +static void spike_machine_class_init(ObjectClass *oc, void *data) { - mc->desc = "RISC-V Spike Board"; + MachineClass *mc = MACHINE_CLASS(oc); + + mc->desc = "RISC-V Spike board"; mc->init = spike_board_init; - mc->max_cpus = 8; + mc->max_cpus = SPIKE_CPUS_MAX; mc->is_default = true; mc->default_cpu_type = SPIKE_V1_10_0_CPU; + mc->possible_cpu_arch_ids = riscv_numa_possible_cpu_arch_ids; + mc->cpu_index_to_instance_props = riscv_numa_cpu_index_to_props; + mc->get_default_cpu_node_id = riscv_numa_get_default_cpu_node_id; + mc->numa_mem_supported = true; +} + +static const TypeInfo spike_machine_typeinfo = { + .name = MACHINE_TYPE_NAME("spike"), + .parent = TYPE_MACHINE, + .class_init = spike_machine_class_init, + .instance_init = spike_machine_instance_init, + .instance_size = sizeof(SpikeState), +}; + +static void spike_machine_init_register_types(void) +{ + type_register_static(&spike_machine_typeinfo); } -DEFINE_MACHINE("spike", spike_machine_init) +type_init(spike_machine_init_register_types) diff --git a/include/hw/riscv/spike.h b/include/hw/riscv/spike.h index 1cd72b85d6..b0a18a9c94 100644 --- a/include/hw/riscv/spike.h +++ b/include/hw/riscv/spike.h @@ -22,12 +22,19 @@ #include "hw/riscv/riscv_hart.h" #include "hw/sysbus.h" +#define SPIKE_CPUS_MAX 8 +#define SPIKE_SOCKETS_MAX 8 + +#define TYPE_SPIKE_MACHINE MACHINE_TYPE_NAME("spike") +#define SPIKE_MACHINE(obj) \ + OBJECT_CHECK(SpikeState, (obj), TYPE_SPIKE_MACHINE) + typedef struct { /*< private >*/ - SysBusDevice parent_obj; + MachineState parent; /*< public >*/ - RISCVHartArrayState soc; + RISCVHartArrayState soc[SPIKE_SOCKETS_MAX]; void *fdt; int fdt_size; } SpikeState; From patchwork Tue Jun 16 03:22:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anup Patel X-Patchwork-Id: 11606307 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3BD8D912 for ; Tue, 16 Jun 2020 03:25:34 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 01EB720756 for ; Tue, 16 Jun 2020 03:25:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="fx7Mi3nD"; dkim=pass (1024-bit key) header.d=sharedspace.onmicrosoft.com header.i=@sharedspace.onmicrosoft.com header.b="i7lYRXyW" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 01EB720756 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=wdc.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:37894 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jl2E9-0004dz-7y for patchwork-qemu-devel@patchwork.kernel.org; Mon, 15 Jun 2020 23:25:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43472) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jl2C0-0000qR-KS; Mon, 15 Jun 2020 23:23:20 -0400 Received: from esa5.hgst.iphmx.com ([216.71.153.144]:40074) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jl2Bw-0007kM-NI; Mon, 15 Jun 2020 23:23:20 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1592277797; x=1623813797; h=from:to:cc:subject:date:message-id:in-reply-to: references:content-transfer-encoding:mime-version; bh=xr7bnKjI0RQgZoavkbGr4j4Ny3ibVqhFYek1YKu7LXE=; b=fx7Mi3nDNDNkJHZ8vdm28LYk67TLCPsVX6gNR5uyB4wnlYvS8U0P7xrU +wW+0AAWDcXOfRtUC8Sbj5lTTrmnEXEZ5e8FZz2JSDRCJZhPNMmU7m25o 7Nzfm6i+bAhws4JxoxL/8apbzPxJCEvhh6f2u0SEKJHiHYjtzSZPtn8iG r8qhA5g93BM3lbRp3J1N4G3EkEX5mYv1H3+UmwKb4y8xaRtLN/HqAazrU DWlbLEfZ4UPHev/b3Cl7zmmd+01SRTz905RLtskJlxXQ9WlWCtVCXE5ME 38MbyDrbxHZhzr6+ogl8OGmODJeb9oJ8LPDnvTYEFvrsFmkCqKPKyLeEP Q==; IronPort-SDR: anzOPLaUTaS7+REQEGTGbw6OBNloP2sxI7cIuXw/2JdBZcmA0UZMpWO25k0e5MuVju0IjafdMC z5D/q28ulDRC6KjOlFMrpJMuBf9vOOykTjkHXDEs1z4wZ074FKQaCbxrGVHykZup1ZoO9t3QS7 oXNdOltbTnOYYrC9Mg52+EbHBCLnDF/M1R1hVnCrqN8SjC0xrPhkx2sSD/zBncpVc3ajEGC1I3 C4J2oh9qKsbfrFIjGC6x2nAP34cOh3qVo9TwJPmglH30EhlXC65BacQj25+bRyfkucarwMJCAB Ca0= X-IronPort-AV: E=Sophos;i="5.73,517,1583164800"; d="scan'208";a="140357459" Received: from mail-cys01nam02lp2055.outbound.protection.outlook.com (HELO NAM02-CY1-obe.outbound.protection.outlook.com) ([104.47.37.55]) by ob1.hgst.iphmx.com with ESMTP; 16 Jun 2020 11:23:15 +0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BSsvTVUNN3+WSXI6AyW8sihhschTQNQz/6ihlJDJC9OH7a0uKK8ICxp3GbeMf/HWL5K/QEEsKX70kX8vfIJT9iRvF3zDHfa+d7MO7Rt7ySDWnQa0Z8Ir+yvZbpB18gh6jizJt0ZiNRPSLkhNcUKUog99CIN20ispVseqfFkNtiqlxm8M2031p1u9B0cR/a/3JscNf1jkiFUJOwANMQlZYZJos2HNjAasCtXURvql/yyXXYYj+fwoLLTvz6cVRxYwKZALRGiiD5tBlGKhHNgq2/HUTTetiIgYGfhB2eFOdDZY5KA2uc6MJJXSw7IJ6wMw3zoCsO22Rgt0gBYVSxa+bQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=57DBHhOQii8LMwvTKA/wVtkBGvE2ZIy7O0/XH1+wy1o=; b=Aes3cMiDg5bCJIYKedE8pHpf6l5hSfNTiTRXMq4TlfAY1a1AzmlNNPjGyo2/V+SxHUpv4UN5o6I455PAxrosxkU55LsqfNYqRQVi0L1pOamP/nPhh2Xc8hYg7pwHDJOBuPHIF5PSRK/KOqMU9ACJ4GQ54aKntSY1ZAhvTetqQTpTRc+ii5tQVqHYOWZKPG4GAwUgHo1MpnXo3kDttavlFNXn6cBwtCylT16xnXJgwwsaH2REFMGzcaxtwWwvbbWmO06Q9nZsrPuq9K/nDm/uxMyyktdm6fMHJOPKROCzQ0jwqb8EFVDi32VPRlu9OClfGEjoVnnc2iEwa4RnJcwSTQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wdc.com; dmarc=pass action=none header.from=wdc.com; dkim=pass header.d=wdc.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sharedspace.onmicrosoft.com; s=selector2-sharedspace-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=57DBHhOQii8LMwvTKA/wVtkBGvE2ZIy7O0/XH1+wy1o=; b=i7lYRXyW5Ggsd7+RbkQQo3d0eJSiWh/wO7kacnE9V+/q9Cv9DU0F0kdTarba9MgJ5J7KWmIjWsn1foPKQzOB3Eq8BjvmM9sjuJCMCanKLQiDTZuB4XyBOAp5Gdo3q1rsv5AlUiA3WLbP6ZujEKZOCAL37x10ZQLqOwzKuZ2qahg= Authentication-Results: linaro.org; dkim=none (message not signed) header.d=none;linaro.org; dmarc=none action=none header.from=wdc.com; Received: from DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) by DM6PR04MB4123.namprd04.prod.outlook.com (2603:10b6:5:98::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.21; Tue, 16 Jun 2020 03:23:12 +0000 Received: from DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::e0a4:aa82:1847:dea5]) by DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::e0a4:aa82:1847:dea5%7]) with mapi id 15.20.3088.029; Tue, 16 Jun 2020 03:23:12 +0000 From: Anup Patel To: Peter Maydell , Palmer Dabbelt , Alistair Francis , Sagar Karandikar Subject: [PATCH v6 5/5] hw/riscv: virt: Allow creating multiple NUMA sockets Date: Tue, 16 Jun 2020 08:52:29 +0530 Message-Id: <20200616032229.766089-6-anup.patel@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200616032229.766089-1-anup.patel@wdc.com> References: <20200616032229.766089-1-anup.patel@wdc.com> X-ClientProxiedBy: MAXPR01CA0099.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:5d::17) To DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from wdc.com (122.171.201.92) by MAXPR01CA0099.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:5d::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3088.19 via Frontend Transport; Tue, 16 Jun 2020 03:23:09 +0000 X-Mailer: git-send-email 2.25.1 X-Originating-IP: [122.171.201.92] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 23ae7205-fa63-439a-9d47-08d811a499a4 X-MS-TrafficTypeDiagnostic: DM6PR04MB4123: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: WDCIPOUTBOUND: EOP-TRUE X-MS-Oob-TLC-OOBClassifiers: OLM:265; X-Forefront-PRVS: 04362AC73B X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: F19Q0wcQhB0xpbUIAqiNaiijG8rDk3oL508cdz+PlnEs5HaWYcJAL35n4Nq5HRsgYf+o2Y7f3ZfLOfqn6hWL5i9jqAVVdFUpWVRDWYcZWCD7tq8WTdYKVAFfGApEPCNsX8w7lpbNycwU7GujRQRoWYUAEX4dHTsfP/49lKC+vYTlXCk21/IzACuFH4OnyRxvZRo+KMpBRd8uMPKwKDpAwZ9DCewXBmjYTaIS16lp/lN7aeveOo72N/Ec4zEOimSMY6PcFeKGQtojA+zk4bO6tjneVvDkusx/+Bbu4vFjGccI/zldJ8cjaMWbMZiUuAERwOHh5mMVx7yII8uqOC7ETg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DM6PR04MB6201.namprd04.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(136003)(366004)(376002)(396003)(346002)(66946007)(2616005)(66476007)(66556008)(8936002)(956004)(2906002)(316002)(54906003)(478600001)(6666004)(186003)(44832011)(55016002)(36756003)(7696005)(52116002)(83380400001)(26005)(8676002)(4326008)(86362001)(30864003)(16526019)(1076003)(8886007)(110136005)(5660300002); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData: NvJugrKUSdocJ2iTU314uESkCFc/ii4oREXD+qfbvK5xzSKnXvZce43f2tZWxDoIeDsKAhkL5KFy1YgSXfM97+xuKAEdxXBrvyH/DOGzwls5dBkTsBa1OTdVWgghCsQxyGG+ty4VVMo/Gww1lSWzO1Twb+YD1oTqo+Mi6FwvmnzRnDWPRQrlDdRuGqCHvw4dPj22gk6euAz5QZe4EdOV2s8tlOrFbILfiby35Uyu5Fw1awuSPiPWyOBdrED9A2SBL78QK+s3hbEtvmXeg5Y8O+j0dENfOi1zyj5jU5wEPCdiMXv4feK4Jk/c62snzUBRn4dyb38NJfwB0uwjoRuBOXiH5eiItdPa6cqGbRy1UJFXxnUBQuBX20drGAAmvUGgv1zQFIhEC645m8Oe7qAwpxlK4AKcQ0d4JRejgyrKmIgJZbQL2/p81JYqs8nLxAAO2uv0EYph1LG2kKITt0LW7jmmL56SFLcH8jOuaomqeGU= X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-Network-Message-Id: 23ae7205-fa63-439a-9d47-08d811a499a4 X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2020 03:23:12.5744 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SbOrZFmf3rp/C9IwcxhckgHIH/QWAveprRxx5EU3/wb8q3VlmmrZ73kTr1U7MyNgnMJFUatTQgCTGuQxJdvj+w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR04MB4123 Received-SPF: pass client-ip=216.71.153.144; envelope-from=prvs=4298fe783=Anup.Patel@wdc.com; helo=esa5.hgst.iphmx.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/15 23:23:14 X-ACL-Warn: Detected OS = FreeBSD 9.x or newer [fuzzy] X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MSGID_FROM_MTA_HEADER=0.001, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Atish Patra , Anup Patel , qemu-riscv@nongnu.org, qemu-devel@nongnu.org, Anup Patel Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" We extend RISC-V virt machine to allow creating a multi-socket machine. Each RISC-V virt machine socket is a NUMA node having a set of HARTs, a memory instance, a CLINT instance, and a PLIC instance. Other devices are shared between all sockets. We also update the generated device tree accordingly. By default, NUMA multi-socket support is disabled for RISC-V virt machine. To enable it, users can use "-numa" command-line options of QEMU. Example1: For two NUMA nodes with 2 CPUs each, append following to command-line options: "-smp 4 -numa node -numa node" Example2: For two NUMA nodes with 1 and 3 CPUs, append following to command-line options: "-smp 4 -numa node -numa node -numa cpu,node-id=0,core-id=0 \ -numa cpu,node-id=1,core-id=1 -numa cpu,node-id=1,core-id=2 \ -numa cpu,node-id=1,core-id=3" The maximum number of sockets in a RISC-V virt machine is 8 but this limit can be changed in future. Signed-off-by: Anup Patel Reviewed-by: Atish Patra Reviewed-by: Alistair Francis --- hw/riscv/virt.c | 530 +++++++++++++++++++++++----------------- include/hw/riscv/virt.h | 9 +- 2 files changed, 308 insertions(+), 231 deletions(-) diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c index 35dc43d620..e01925ff3d 100644 --- a/hw/riscv/virt.c +++ b/hw/riscv/virt.c @@ -35,6 +35,7 @@ #include "hw/riscv/sifive_test.h" #include "hw/riscv/virt.h" #include "hw/riscv/boot.h" +#include "hw/riscv/numa.h" #include "chardev/char.h" #include "sysemu/arch_init.h" #include "sysemu/device_tree.h" @@ -60,7 +61,7 @@ static const struct MemmapEntry { [VIRT_TEST] = { 0x100000, 0x1000 }, [VIRT_RTC] = { 0x101000, 0x1000 }, [VIRT_CLINT] = { 0x2000000, 0x10000 }, - [VIRT_PLIC] = { 0xc000000, 0x4000000 }, + [VIRT_PLIC] = { 0xc000000, VIRT_PLIC_SIZE(VIRT_CPUS_MAX * 2) }, [VIRT_UART0] = { 0x10000000, 0x100 }, [VIRT_VIRTIO] = { 0x10001000, 0x1000 }, [VIRT_FLASH] = { 0x20000000, 0x4000000 }, @@ -182,10 +183,17 @@ static void create_fdt(RISCVVirtState *s, const struct MemmapEntry *memmap, uint64_t mem_size, const char *cmdline) { void *fdt; - int cpu, i; - uint32_t *cells; - char *nodename; - uint32_t plic_phandle, test_phandle, phandle = 1; + int i, cpu, socket; + MachineState *mc = MACHINE(s); + uint64_t addr, size; + uint32_t *clint_cells, *plic_cells; + unsigned long clint_addr, plic_addr; + uint32_t plic_phandle[MAX_NODES]; + uint32_t cpu_phandle, intc_phandle, test_phandle; + uint32_t phandle = 1, plic_mmio_phandle = 1; + uint32_t plic_pcie_phandle = 1, plic_virtio_phandle = 1; + char *mem_name, *cpu_name, *core_name, *intc_name; + char *name, *clint_name, *plic_name, *clust_name; hwaddr flashsize = virt_memmap[VIRT_FLASH].size / 2; hwaddr flashbase = virt_memmap[VIRT_FLASH].base; @@ -206,231 +214,238 @@ static void create_fdt(RISCVVirtState *s, const struct MemmapEntry *memmap, qemu_fdt_setprop_cell(fdt, "/soc", "#size-cells", 0x2); qemu_fdt_setprop_cell(fdt, "/soc", "#address-cells", 0x2); - nodename = g_strdup_printf("/memory@%lx", - (long)memmap[VIRT_DRAM].base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_cells(fdt, nodename, "reg", - memmap[VIRT_DRAM].base >> 32, memmap[VIRT_DRAM].base, - mem_size >> 32, mem_size); - qemu_fdt_setprop_string(fdt, nodename, "device_type", "memory"); - g_free(nodename); - qemu_fdt_add_subnode(fdt, "/cpus"); qemu_fdt_setprop_cell(fdt, "/cpus", "timebase-frequency", SIFIVE_CLINT_TIMEBASE_FREQ); qemu_fdt_setprop_cell(fdt, "/cpus", "#size-cells", 0x0); qemu_fdt_setprop_cell(fdt, "/cpus", "#address-cells", 0x1); + qemu_fdt_add_subnode(fdt, "/cpus/cpu-map"); + + for (socket = (riscv_socket_count(mc) - 1); socket >= 0; socket--) { + clust_name = g_strdup_printf("/cpus/cpu-map/cluster%d", socket); + qemu_fdt_add_subnode(fdt, clust_name); + + plic_cells = g_new0(uint32_t, s->soc[socket].num_harts * 4); + clint_cells = g_new0(uint32_t, s->soc[socket].num_harts * 4); + + for (cpu = s->soc[socket].num_harts - 1; cpu >= 0; cpu--) { + cpu_phandle = phandle++; - for (cpu = s->soc.num_harts - 1; cpu >= 0; cpu--) { - int cpu_phandle = phandle++; - int intc_phandle; - nodename = g_strdup_printf("/cpus/cpu@%d", cpu); - char *intc = g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu); - char *isa = riscv_isa_string(&s->soc.harts[cpu]); - qemu_fdt_add_subnode(fdt, nodename); + cpu_name = g_strdup_printf("/cpus/cpu@%d", + s->soc[socket].hartid_base + cpu); + qemu_fdt_add_subnode(fdt, cpu_name); #if defined(TARGET_RISCV32) - qemu_fdt_setprop_string(fdt, nodename, "mmu-type", "riscv,sv32"); + qemu_fdt_setprop_string(fdt, cpu_name, "mmu-type", "riscv,sv32"); #else - qemu_fdt_setprop_string(fdt, nodename, "mmu-type", "riscv,sv48"); + qemu_fdt_setprop_string(fdt, cpu_name, "mmu-type", "riscv,sv48"); #endif - qemu_fdt_setprop_string(fdt, nodename, "riscv,isa", isa); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "riscv"); - qemu_fdt_setprop_string(fdt, nodename, "status", "okay"); - qemu_fdt_setprop_cell(fdt, nodename, "reg", cpu); - qemu_fdt_setprop_string(fdt, nodename, "device_type", "cpu"); - qemu_fdt_setprop_cell(fdt, nodename, "phandle", cpu_phandle); - intc_phandle = phandle++; - qemu_fdt_add_subnode(fdt, intc); - qemu_fdt_setprop_cell(fdt, intc, "phandle", intc_phandle); - qemu_fdt_setprop_string(fdt, intc, "compatible", "riscv,cpu-intc"); - qemu_fdt_setprop(fdt, intc, "interrupt-controller", NULL, 0); - qemu_fdt_setprop_cell(fdt, intc, "#interrupt-cells", 1); - g_free(isa); - g_free(intc); - g_free(nodename); - } + name = riscv_isa_string(&s->soc[socket].harts[cpu]); + qemu_fdt_setprop_string(fdt, cpu_name, "riscv,isa", name); + g_free(name); + qemu_fdt_setprop_string(fdt, cpu_name, "compatible", "riscv"); + qemu_fdt_setprop_string(fdt, cpu_name, "status", "okay"); + qemu_fdt_setprop_cell(fdt, cpu_name, "reg", + s->soc[socket].hartid_base + cpu); + qemu_fdt_setprop_string(fdt, cpu_name, "device_type", "cpu"); + riscv_socket_fdt_write_id(mc, fdt, cpu_name, socket); + qemu_fdt_setprop_cell(fdt, cpu_name, "phandle", cpu_phandle); + + intc_name = g_strdup_printf("%s/interrupt-controller", cpu_name); + qemu_fdt_add_subnode(fdt, intc_name); + intc_phandle = phandle++; + qemu_fdt_setprop_cell(fdt, intc_name, "phandle", intc_phandle); + qemu_fdt_setprop_string(fdt, intc_name, "compatible", + "riscv,cpu-intc"); + qemu_fdt_setprop(fdt, intc_name, "interrupt-controller", NULL, 0); + qemu_fdt_setprop_cell(fdt, intc_name, "#interrupt-cells", 1); + + clint_cells[cpu * 4 + 0] = cpu_to_be32(intc_phandle); + clint_cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_SOFT); + clint_cells[cpu * 4 + 2] = cpu_to_be32(intc_phandle); + clint_cells[cpu * 4 + 3] = cpu_to_be32(IRQ_M_TIMER); + + plic_cells[cpu * 4 + 0] = cpu_to_be32(intc_phandle); + plic_cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_EXT); + plic_cells[cpu * 4 + 2] = cpu_to_be32(intc_phandle); + plic_cells[cpu * 4 + 3] = cpu_to_be32(IRQ_S_EXT); + + core_name = g_strdup_printf("%s/core%d", clust_name, cpu); + qemu_fdt_add_subnode(fdt, core_name); + qemu_fdt_setprop_cell(fdt, core_name, "cpu", cpu_phandle); + + g_free(core_name); + g_free(intc_name); + g_free(cpu_name); + } - /* Add cpu-topology node */ - qemu_fdt_add_subnode(fdt, "/cpus/cpu-map"); - qemu_fdt_add_subnode(fdt, "/cpus/cpu-map/cluster0"); - for (cpu = s->soc.num_harts - 1; cpu >= 0; cpu--) { - char *core_nodename = g_strdup_printf("/cpus/cpu-map/cluster0/core%d", - cpu); - char *cpu_nodename = g_strdup_printf("/cpus/cpu@%d", cpu); - uint32_t intc_phandle = qemu_fdt_get_phandle(fdt, cpu_nodename); - qemu_fdt_add_subnode(fdt, core_nodename); - qemu_fdt_setprop_cell(fdt, core_nodename, "cpu", intc_phandle); - g_free(core_nodename); - g_free(cpu_nodename); + addr = memmap[VIRT_DRAM].base + riscv_socket_mem_offset(mc, socket); + size = riscv_socket_mem_size(mc, socket); + mem_name = g_strdup_printf("/memory@%lx", (long)addr); + qemu_fdt_add_subnode(fdt, mem_name); + qemu_fdt_setprop_cells(fdt, mem_name, "reg", + addr >> 32, addr, size >> 32, size); + qemu_fdt_setprop_string(fdt, mem_name, "device_type", "memory"); + riscv_socket_fdt_write_id(mc, fdt, mem_name, socket); + g_free(mem_name); + + clint_addr = memmap[VIRT_CLINT].base + + (memmap[VIRT_CLINT].size * socket); + clint_name = g_strdup_printf("/soc/clint@%lx", clint_addr); + qemu_fdt_add_subnode(fdt, clint_name); + qemu_fdt_setprop_string(fdt, clint_name, "compatible", "riscv,clint0"); + qemu_fdt_setprop_cells(fdt, clint_name, "reg", + 0x0, clint_addr, 0x0, memmap[VIRT_CLINT].size); + qemu_fdt_setprop(fdt, clint_name, "interrupts-extended", + clint_cells, s->soc[socket].num_harts * sizeof(uint32_t) * 4); + riscv_socket_fdt_write_id(mc, fdt, clint_name, socket); + g_free(clint_name); + + plic_phandle[socket] = phandle++; + plic_addr = memmap[VIRT_PLIC].base + (memmap[VIRT_PLIC].size * socket); + plic_name = g_strdup_printf("/soc/plic@%lx", plic_addr); + qemu_fdt_add_subnode(fdt, plic_name); + qemu_fdt_setprop_cell(fdt, plic_name, + "#address-cells", FDT_PLIC_ADDR_CELLS); + qemu_fdt_setprop_cell(fdt, plic_name, + "#interrupt-cells", FDT_PLIC_INT_CELLS); + qemu_fdt_setprop_string(fdt, plic_name, "compatible", "riscv,plic0"); + qemu_fdt_setprop(fdt, plic_name, "interrupt-controller", NULL, 0); + qemu_fdt_setprop(fdt, plic_name, "interrupts-extended", + plic_cells, s->soc[socket].num_harts * sizeof(uint32_t) * 4); + qemu_fdt_setprop_cells(fdt, plic_name, "reg", + 0x0, plic_addr, 0x0, memmap[VIRT_PLIC].size); + qemu_fdt_setprop_cell(fdt, plic_name, "riscv,ndev", VIRTIO_NDEV); + riscv_socket_fdt_write_id(mc, fdt, plic_name, socket); + qemu_fdt_setprop_cell(fdt, plic_name, "phandle", plic_phandle[socket]); + g_free(plic_name); + + g_free(clint_cells); + g_free(plic_cells); + g_free(clust_name); } - cells = g_new0(uint32_t, s->soc.num_harts * 4); - for (cpu = 0; cpu < s->soc.num_harts; cpu++) { - nodename = - g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu); - uint32_t intc_phandle = qemu_fdt_get_phandle(fdt, nodename); - cells[cpu * 4 + 0] = cpu_to_be32(intc_phandle); - cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_SOFT); - cells[cpu * 4 + 2] = cpu_to_be32(intc_phandle); - cells[cpu * 4 + 3] = cpu_to_be32(IRQ_M_TIMER); - g_free(nodename); - } - nodename = g_strdup_printf("/soc/clint@%lx", - (long)memmap[VIRT_CLINT].base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "riscv,clint0"); - qemu_fdt_setprop_cells(fdt, nodename, "reg", - 0x0, memmap[VIRT_CLINT].base, - 0x0, memmap[VIRT_CLINT].size); - qemu_fdt_setprop(fdt, nodename, "interrupts-extended", - cells, s->soc.num_harts * sizeof(uint32_t) * 4); - g_free(cells); - g_free(nodename); - - plic_phandle = phandle++; - cells = g_new0(uint32_t, s->soc.num_harts * 4); - for (cpu = 0; cpu < s->soc.num_harts; cpu++) { - nodename = - g_strdup_printf("/cpus/cpu@%d/interrupt-controller", cpu); - uint32_t intc_phandle = qemu_fdt_get_phandle(fdt, nodename); - cells[cpu * 4 + 0] = cpu_to_be32(intc_phandle); - cells[cpu * 4 + 1] = cpu_to_be32(IRQ_M_EXT); - cells[cpu * 4 + 2] = cpu_to_be32(intc_phandle); - cells[cpu * 4 + 3] = cpu_to_be32(IRQ_S_EXT); - g_free(nodename); + for (socket = 0; socket < riscv_socket_count(mc); socket++) { + if (socket == 0) { + plic_mmio_phandle = plic_phandle[socket]; + plic_virtio_phandle = plic_phandle[socket]; + plic_pcie_phandle = plic_phandle[socket]; + } + if (socket == 1) { + plic_virtio_phandle = plic_phandle[socket]; + plic_pcie_phandle = plic_phandle[socket]; + } + if (socket == 2) { + plic_pcie_phandle = plic_phandle[socket]; + } } - nodename = g_strdup_printf("/soc/interrupt-controller@%lx", - (long)memmap[VIRT_PLIC].base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_cell(fdt, nodename, "#address-cells", - FDT_PLIC_ADDR_CELLS); - qemu_fdt_setprop_cell(fdt, nodename, "#interrupt-cells", - FDT_PLIC_INT_CELLS); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "riscv,plic0"); - qemu_fdt_setprop(fdt, nodename, "interrupt-controller", NULL, 0); - qemu_fdt_setprop(fdt, nodename, "interrupts-extended", - cells, s->soc.num_harts * sizeof(uint32_t) * 4); - qemu_fdt_setprop_cells(fdt, nodename, "reg", - 0x0, memmap[VIRT_PLIC].base, - 0x0, memmap[VIRT_PLIC].size); - qemu_fdt_setprop_cell(fdt, nodename, "riscv,ndev", VIRTIO_NDEV); - qemu_fdt_setprop_cell(fdt, nodename, "phandle", plic_phandle); - plic_phandle = qemu_fdt_get_phandle(fdt, nodename); - g_free(cells); - g_free(nodename); + + riscv_socket_fdt_write_distance_matrix(mc, fdt); for (i = 0; i < VIRTIO_COUNT; i++) { - nodename = g_strdup_printf("/virtio_mmio@%lx", + name = g_strdup_printf("/soc/virtio_mmio@%lx", (long)(memmap[VIRT_VIRTIO].base + i * memmap[VIRT_VIRTIO].size)); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "virtio,mmio"); - qemu_fdt_setprop_cells(fdt, nodename, "reg", + qemu_fdt_add_subnode(fdt, name); + qemu_fdt_setprop_string(fdt, name, "compatible", "virtio,mmio"); + qemu_fdt_setprop_cells(fdt, name, "reg", 0x0, memmap[VIRT_VIRTIO].base + i * memmap[VIRT_VIRTIO].size, 0x0, memmap[VIRT_VIRTIO].size); - qemu_fdt_setprop_cell(fdt, nodename, "interrupt-parent", plic_phandle); - qemu_fdt_setprop_cell(fdt, nodename, "interrupts", VIRTIO_IRQ + i); - g_free(nodename); + qemu_fdt_setprop_cell(fdt, name, "interrupt-parent", + plic_virtio_phandle); + qemu_fdt_setprop_cell(fdt, name, "interrupts", VIRTIO_IRQ + i); + g_free(name); } - nodename = g_strdup_printf("/soc/pci@%lx", + name = g_strdup_printf("/soc/pci@%lx", (long) memmap[VIRT_PCIE_ECAM].base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_cell(fdt, nodename, "#address-cells", - FDT_PCI_ADDR_CELLS); - qemu_fdt_setprop_cell(fdt, nodename, "#interrupt-cells", - FDT_PCI_INT_CELLS); - qemu_fdt_setprop_cell(fdt, nodename, "#size-cells", 0x2); - qemu_fdt_setprop_string(fdt, nodename, "compatible", - "pci-host-ecam-generic"); - qemu_fdt_setprop_string(fdt, nodename, "device_type", "pci"); - qemu_fdt_setprop_cell(fdt, nodename, "linux,pci-domain", 0); - qemu_fdt_setprop_cells(fdt, nodename, "bus-range", 0, - memmap[VIRT_PCIE_ECAM].size / - PCIE_MMCFG_SIZE_MIN - 1); - qemu_fdt_setprop(fdt, nodename, "dma-coherent", NULL, 0); - qemu_fdt_setprop_cells(fdt, nodename, "reg", 0, memmap[VIRT_PCIE_ECAM].base, - 0, memmap[VIRT_PCIE_ECAM].size); - qemu_fdt_setprop_sized_cells(fdt, nodename, "ranges", + qemu_fdt_add_subnode(fdt, name); + qemu_fdt_setprop_cell(fdt, name, "#address-cells", FDT_PCI_ADDR_CELLS); + qemu_fdt_setprop_cell(fdt, name, "#interrupt-cells", FDT_PCI_INT_CELLS); + qemu_fdt_setprop_cell(fdt, name, "#size-cells", 0x2); + qemu_fdt_setprop_string(fdt, name, "compatible", "pci-host-ecam-generic"); + qemu_fdt_setprop_string(fdt, name, "device_type", "pci"); + qemu_fdt_setprop_cell(fdt, name, "linux,pci-domain", 0); + qemu_fdt_setprop_cells(fdt, name, "bus-range", 0, + memmap[VIRT_PCIE_ECAM].size / PCIE_MMCFG_SIZE_MIN - 1); + qemu_fdt_setprop(fdt, name, "dma-coherent", NULL, 0); + qemu_fdt_setprop_cells(fdt, name, "reg", 0, + memmap[VIRT_PCIE_ECAM].base, 0, memmap[VIRT_PCIE_ECAM].size); + qemu_fdt_setprop_sized_cells(fdt, name, "ranges", 1, FDT_PCI_RANGE_IOPORT, 2, 0, 2, memmap[VIRT_PCIE_PIO].base, 2, memmap[VIRT_PCIE_PIO].size, 1, FDT_PCI_RANGE_MMIO, 2, memmap[VIRT_PCIE_MMIO].base, 2, memmap[VIRT_PCIE_MMIO].base, 2, memmap[VIRT_PCIE_MMIO].size); - create_pcie_irq_map(fdt, nodename, plic_phandle); - g_free(nodename); + create_pcie_irq_map(fdt, name, plic_pcie_phandle); + g_free(name); test_phandle = phandle++; - nodename = g_strdup_printf("/test@%lx", + name = g_strdup_printf("/soc/test@%lx", (long)memmap[VIRT_TEST].base); - qemu_fdt_add_subnode(fdt, nodename); + qemu_fdt_add_subnode(fdt, name); { const char compat[] = "sifive,test1\0sifive,test0\0syscon"; - qemu_fdt_setprop(fdt, nodename, "compatible", compat, sizeof(compat)); + qemu_fdt_setprop(fdt, name, "compatible", compat, sizeof(compat)); } - qemu_fdt_setprop_cells(fdt, nodename, "reg", + qemu_fdt_setprop_cells(fdt, name, "reg", 0x0, memmap[VIRT_TEST].base, 0x0, memmap[VIRT_TEST].size); - qemu_fdt_setprop_cell(fdt, nodename, "phandle", test_phandle); - test_phandle = qemu_fdt_get_phandle(fdt, nodename); - g_free(nodename); - - nodename = g_strdup_printf("/reboot"); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "syscon-reboot"); - qemu_fdt_setprop_cell(fdt, nodename, "regmap", test_phandle); - qemu_fdt_setprop_cell(fdt, nodename, "offset", 0x0); - qemu_fdt_setprop_cell(fdt, nodename, "value", FINISHER_RESET); - g_free(nodename); - - nodename = g_strdup_printf("/poweroff"); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "syscon-poweroff"); - qemu_fdt_setprop_cell(fdt, nodename, "regmap", test_phandle); - qemu_fdt_setprop_cell(fdt, nodename, "offset", 0x0); - qemu_fdt_setprop_cell(fdt, nodename, "value", FINISHER_PASS); - g_free(nodename); - - nodename = g_strdup_printf("/uart@%lx", - (long)memmap[VIRT_UART0].base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_string(fdt, nodename, "compatible", "ns16550a"); - qemu_fdt_setprop_cells(fdt, nodename, "reg", + qemu_fdt_setprop_cell(fdt, name, "phandle", test_phandle); + test_phandle = qemu_fdt_get_phandle(fdt, name); + g_free(name); + + name = g_strdup_printf("/soc/reboot"); + qemu_fdt_add_subnode(fdt, name); + qemu_fdt_setprop_string(fdt, name, "compatible", "syscon-reboot"); + qemu_fdt_setprop_cell(fdt, name, "regmap", test_phandle); + qemu_fdt_setprop_cell(fdt, name, "offset", 0x0); + qemu_fdt_setprop_cell(fdt, name, "value", FINISHER_RESET); + g_free(name); + + name = g_strdup_printf("/soc/poweroff"); + qemu_fdt_add_subnode(fdt, name); + qemu_fdt_setprop_string(fdt, name, "compatible", "syscon-poweroff"); + qemu_fdt_setprop_cell(fdt, name, "regmap", test_phandle); + qemu_fdt_setprop_cell(fdt, name, "offset", 0x0); + qemu_fdt_setprop_cell(fdt, name, "value", FINISHER_PASS); + g_free(name); + + name = g_strdup_printf("/soc/uart@%lx", (long)memmap[VIRT_UART0].base); + qemu_fdt_add_subnode(fdt, name); + qemu_fdt_setprop_string(fdt, name, "compatible", "ns16550a"); + qemu_fdt_setprop_cells(fdt, name, "reg", 0x0, memmap[VIRT_UART0].base, 0x0, memmap[VIRT_UART0].size); - qemu_fdt_setprop_cell(fdt, nodename, "clock-frequency", 3686400); - qemu_fdt_setprop_cell(fdt, nodename, "interrupt-parent", plic_phandle); - qemu_fdt_setprop_cell(fdt, nodename, "interrupts", UART0_IRQ); + qemu_fdt_setprop_cell(fdt, name, "clock-frequency", 3686400); + qemu_fdt_setprop_cell(fdt, name, "interrupt-parent", plic_mmio_phandle); + qemu_fdt_setprop_cell(fdt, name, "interrupts", UART0_IRQ); qemu_fdt_add_subnode(fdt, "/chosen"); - qemu_fdt_setprop_string(fdt, "/chosen", "stdout-path", nodename); + qemu_fdt_setprop_string(fdt, "/chosen", "stdout-path", name); if (cmdline) { qemu_fdt_setprop_string(fdt, "/chosen", "bootargs", cmdline); } - g_free(nodename); - - nodename = g_strdup_printf("/rtc@%lx", - (long)memmap[VIRT_RTC].base); - qemu_fdt_add_subnode(fdt, nodename); - qemu_fdt_setprop_string(fdt, nodename, "compatible", - "google,goldfish-rtc"); - qemu_fdt_setprop_cells(fdt, nodename, "reg", + g_free(name); + + name = g_strdup_printf("/soc/rtc@%lx", (long)memmap[VIRT_RTC].base); + qemu_fdt_add_subnode(fdt, name); + qemu_fdt_setprop_string(fdt, name, "compatible", "google,goldfish-rtc"); + qemu_fdt_setprop_cells(fdt, name, "reg", 0x0, memmap[VIRT_RTC].base, 0x0, memmap[VIRT_RTC].size); - qemu_fdt_setprop_cell(fdt, nodename, "interrupt-parent", plic_phandle); - qemu_fdt_setprop_cell(fdt, nodename, "interrupts", RTC_IRQ); - g_free(nodename); - - nodename = g_strdup_printf("/flash@%" PRIx64, flashbase); - qemu_fdt_add_subnode(s->fdt, nodename); - qemu_fdt_setprop_string(s->fdt, nodename, "compatible", "cfi-flash"); - qemu_fdt_setprop_sized_cells(s->fdt, nodename, "reg", + qemu_fdt_setprop_cell(fdt, name, "interrupt-parent", plic_mmio_phandle); + qemu_fdt_setprop_cell(fdt, name, "interrupts", RTC_IRQ); + g_free(name); + + name = g_strdup_printf("/soc/flash@%" PRIx64, flashbase); + qemu_fdt_add_subnode(s->fdt, name); + qemu_fdt_setprop_string(s->fdt, name, "compatible", "cfi-flash"); + qemu_fdt_setprop_sized_cells(s->fdt, name, "reg", 2, flashbase, 2, flashsize, 2, flashbase + flashsize, 2, flashsize); - qemu_fdt_setprop_cell(s->fdt, nodename, "bank-width", 4); - g_free(nodename); + qemu_fdt_setprop_cell(s->fdt, name, "bank-width", 4); + g_free(name); } - static inline DeviceState *gpex_pcie_init(MemoryRegion *sys_mem, hwaddr ecam_base, hwaddr ecam_size, hwaddr mmio_base, hwaddr mmio_size, @@ -478,21 +493,100 @@ static void riscv_virt_board_init(MachineState *machine) MemoryRegion *system_memory = get_system_memory(); MemoryRegion *main_mem = g_new(MemoryRegion, 1); MemoryRegion *mask_rom = g_new(MemoryRegion, 1); - char *plic_hart_config; + char *plic_hart_config, *soc_name; size_t plic_hart_config_len; target_ulong start_addr = memmap[VIRT_DRAM].base; - int i; - unsigned int smp_cpus = machine->smp.cpus; - - /* Initialize SOC */ - object_initialize_child(OBJECT(machine), "soc", &s->soc, sizeof(s->soc), - TYPE_RISCV_HART_ARRAY, &error_abort, NULL); - object_property_set_str(OBJECT(&s->soc), machine->cpu_type, "cpu-type", - &error_abort); - object_property_set_int(OBJECT(&s->soc), smp_cpus, "num-harts", - &error_abort); - object_property_set_bool(OBJECT(&s->soc), true, "realized", - &error_abort); + DeviceState *mmio_plic, *virtio_plic, *pcie_plic; + int i, j, base_hartid, hart_count; + + /* Check socket count limit */ + if (VIRT_SOCKETS_MAX < riscv_socket_count(machine)) { + error_report("number of sockets/nodes should be less than %d", + VIRT_SOCKETS_MAX); + exit(1); + } + + /* Initialize sockets */ + mmio_plic = virtio_plic = pcie_plic = NULL; + for (i = 0; i < riscv_socket_count(machine); i++) { + if (!riscv_socket_check_hartids(machine, i)) { + error_report("discontinuous hartids in socket%d", i); + exit(1); + } + + base_hartid = riscv_socket_first_hartid(machine, i); + if (base_hartid < 0) { + error_report("can't find hartid base for socket%d", i); + exit(1); + } + + hart_count = riscv_socket_hart_count(machine, i); + if (hart_count < 0) { + error_report("can't find hart count for socket%d", i); + exit(1); + } + + soc_name = g_strdup_printf("soc%d", i); + object_initialize_child(OBJECT(machine), soc_name, &s->soc[i], + sizeof(s->soc[i]), TYPE_RISCV_HART_ARRAY, &error_abort, NULL); + g_free(soc_name); + object_property_set_str(OBJECT(&s->soc[i]), + machine->cpu_type, "cpu-type", &error_abort); + object_property_set_int(OBJECT(&s->soc[i]), + base_hartid, "hartid-base", &error_abort); + object_property_set_int(OBJECT(&s->soc[i]), + hart_count, "num-harts", &error_abort); + object_property_set_bool(OBJECT(&s->soc[i]), + true, "realized", &error_abort); + + /* Per-socket CLINT */ + sifive_clint_create( + memmap[VIRT_CLINT].base + i * memmap[VIRT_CLINT].size, + memmap[VIRT_CLINT].size, base_hartid, hart_count, + SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, true); + + /* Per-socket PLIC hart topology configuration string */ + plic_hart_config_len = + (strlen(VIRT_PLIC_HART_CONFIG) + 1) * hart_count; + plic_hart_config = g_malloc0(plic_hart_config_len); + for (j = 0; j < hart_count; j++) { + if (j != 0) { + strncat(plic_hart_config, ",", plic_hart_config_len); + } + strncat(plic_hart_config, VIRT_PLIC_HART_CONFIG, + plic_hart_config_len); + plic_hart_config_len -= (strlen(VIRT_PLIC_HART_CONFIG) + 1); + } + + /* Per-socket PLIC */ + s->plic[i] = sifive_plic_create( + memmap[VIRT_PLIC].base + i * memmap[VIRT_PLIC].size, + plic_hart_config, base_hartid, + VIRT_PLIC_NUM_SOURCES, + VIRT_PLIC_NUM_PRIORITIES, + VIRT_PLIC_PRIORITY_BASE, + VIRT_PLIC_PENDING_BASE, + VIRT_PLIC_ENABLE_BASE, + VIRT_PLIC_ENABLE_STRIDE, + VIRT_PLIC_CONTEXT_BASE, + VIRT_PLIC_CONTEXT_STRIDE, + memmap[VIRT_PLIC].size); + g_free(plic_hart_config); + + /* Try to use different PLIC instance based device type */ + if (i == 0) { + mmio_plic = s->plic[i]; + virtio_plic = s->plic[i]; + pcie_plic = s->plic[i]; + } + if (i == 1) { + virtio_plic = s->plic[i]; + pcie_plic = s->plic[i]; + } + if (i == 2) { + pcie_plic = s->plic[i]; + } + } /* register system main memory (actual RAM) */ memory_region_init_ram(main_mem, NULL, "riscv_virt_board.ram", @@ -571,38 +665,14 @@ static void riscv_virt_board_init(MachineState *machine) memmap[VIRT_MROM].base + sizeof(reset_vec), &address_space_memory); - /* create PLIC hart topology configuration string */ - plic_hart_config_len = (strlen(VIRT_PLIC_HART_CONFIG) + 1) * smp_cpus; - plic_hart_config = g_malloc0(plic_hart_config_len); - for (i = 0; i < smp_cpus; i++) { - if (i != 0) { - strncat(plic_hart_config, ",", plic_hart_config_len); - } - strncat(plic_hart_config, VIRT_PLIC_HART_CONFIG, plic_hart_config_len); - plic_hart_config_len -= (strlen(VIRT_PLIC_HART_CONFIG) + 1); - } - - /* MMIO */ - s->plic = sifive_plic_create(memmap[VIRT_PLIC].base, - plic_hart_config, 0, - VIRT_PLIC_NUM_SOURCES, - VIRT_PLIC_NUM_PRIORITIES, - VIRT_PLIC_PRIORITY_BASE, - VIRT_PLIC_PENDING_BASE, - VIRT_PLIC_ENABLE_BASE, - VIRT_PLIC_ENABLE_STRIDE, - VIRT_PLIC_CONTEXT_BASE, - VIRT_PLIC_CONTEXT_STRIDE, - memmap[VIRT_PLIC].size); - sifive_clint_create(memmap[VIRT_CLINT].base, - memmap[VIRT_CLINT].size, 0, smp_cpus, - SIFIVE_SIP_BASE, SIFIVE_TIMECMP_BASE, SIFIVE_TIME_BASE, true); + /* SiFive Test MMIO device */ sifive_test_create(memmap[VIRT_TEST].base); + /* VirtIO MMIO devices */ for (i = 0; i < VIRTIO_COUNT; i++) { sysbus_create_simple("virtio-mmio", memmap[VIRT_VIRTIO].base + i * memmap[VIRT_VIRTIO].size, - qdev_get_gpio_in(DEVICE(s->plic), VIRTIO_IRQ + i)); + qdev_get_gpio_in(DEVICE(virtio_plic), VIRTIO_IRQ + i)); } gpex_pcie_init(system_memory, @@ -611,14 +681,14 @@ static void riscv_virt_board_init(MachineState *machine) memmap[VIRT_PCIE_MMIO].base, memmap[VIRT_PCIE_MMIO].size, memmap[VIRT_PCIE_PIO].base, - DEVICE(s->plic), true); + DEVICE(pcie_plic), true); serial_mm_init(system_memory, memmap[VIRT_UART0].base, - 0, qdev_get_gpio_in(DEVICE(s->plic), UART0_IRQ), 399193, + 0, qdev_get_gpio_in(DEVICE(mmio_plic), UART0_IRQ), 399193, serial_hd(0), DEVICE_LITTLE_ENDIAN); sysbus_create_simple("goldfish_rtc", memmap[VIRT_RTC].base, - qdev_get_gpio_in(DEVICE(s->plic), RTC_IRQ)); + qdev_get_gpio_in(DEVICE(mmio_plic), RTC_IRQ)); virt_flash_create(s); @@ -628,8 +698,6 @@ static void riscv_virt_board_init(MachineState *machine) drive_get(IF_PFLASH, 0, i)); } virt_flash_map(s, system_memory); - - g_free(plic_hart_config); } static void riscv_virt_machine_instance_init(Object *obj) @@ -642,9 +710,13 @@ static void riscv_virt_machine_class_init(ObjectClass *oc, void *data) mc->desc = "RISC-V VirtIO board"; mc->init = riscv_virt_board_init; - mc->max_cpus = 8; + mc->max_cpus = VIRT_CPUS_MAX; mc->default_cpu_type = VIRT_CPU; mc->pci_allow_0_address = true; + mc->possible_cpu_arch_ids = riscv_numa_possible_cpu_arch_ids; + mc->cpu_index_to_instance_props = riscv_numa_cpu_index_to_props; + mc->get_default_cpu_node_id = riscv_numa_get_default_cpu_node_id; + mc->numa_mem_supported = true; } static const TypeInfo riscv_virt_machine_typeinfo = { diff --git a/include/hw/riscv/virt.h b/include/hw/riscv/virt.h index e69355efaf..1beacd7666 100644 --- a/include/hw/riscv/virt.h +++ b/include/hw/riscv/virt.h @@ -23,6 +23,9 @@ #include "hw/sysbus.h" #include "hw/block/flash.h" +#define VIRT_CPUS_MAX 8 +#define VIRT_SOCKETS_MAX 8 + #define TYPE_RISCV_VIRT_MACHINE MACHINE_TYPE_NAME("virt") #define RISCV_VIRT_MACHINE(obj) \ OBJECT_CHECK(RISCVVirtState, (obj), TYPE_RISCV_VIRT_MACHINE) @@ -32,8 +35,8 @@ typedef struct { MachineState parent; /*< public >*/ - RISCVHartArrayState soc; - DeviceState *plic; + RISCVHartArrayState soc[VIRT_SOCKETS_MAX]; + DeviceState *plic[VIRT_SOCKETS_MAX]; PFlashCFI01 *flash[2]; void *fdt; @@ -74,6 +77,8 @@ enum { #define VIRT_PLIC_ENABLE_STRIDE 0x80 #define VIRT_PLIC_CONTEXT_BASE 0x200000 #define VIRT_PLIC_CONTEXT_STRIDE 0x1000 +#define VIRT_PLIC_SIZE(__num_context) \ + (VIRT_PLIC_CONTEXT_BASE + (__num_context) * VIRT_PLIC_CONTEXT_STRIDE) #define FDT_PCI_ADDR_CELLS 3 #define FDT_PCI_INT_CELLS 1