From patchwork Fri Jul 10 09:00:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anup Patel X-Patchwork-Id: 11656069 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BD19513BD for ; Fri, 10 Jul 2020 09:01:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9A124206E2 for ; Fri, 10 Jul 2020 09:01:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=wdc.com header.i=@wdc.com header.b="H87+SZSt"; dkim=pass (1024-bit key) header.d=sharedspace.onmicrosoft.com header.i=@sharedspace.onmicrosoft.com header.b="ajBUWTT2" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727998AbgGJJBa (ORCPT ); Fri, 10 Jul 2020 05:01:30 -0400 Received: from esa1.hgst.iphmx.com ([68.232.141.245]:30305 "EHLO esa1.hgst.iphmx.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727972AbgGJJB3 (ORCPT ); Fri, 10 Jul 2020 05:01:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=wdc.com; i=@wdc.com; q=dns/txt; s=dkim.wdc.com; t=1594371688; x=1625907688; h=from:to:cc:subject:date:message-id:in-reply-to: references:content-transfer-encoding:mime-version; bh=/0T5Ki2Lj1DFnjbgmjys4xMNGjRVHw8B2nUHI9CqhtM=; b=H87+SZStSPoJvIT3q4LjV1q6ucHXDEM2bSaCbeRaqEUI9jV8ZOVUs4lf 3DD+dAhGYT4Obx/O27Jzic5q+RmADWm0tdoeuaYtd9XP3r0/LR47qarbi vnUzEfZLwkfta2lR+WxO2LEum294F+ICIC/RAcr3VkhZf+gtQDLh1rJL5 gOb7VUylEKDr0IgDxQjlMymRQTZJkWtaVRrD2+1mOJdsgC9xFRbIAwPfp 2VgIio6wnBmLok/AO+fCQIQdZx6M0zbmpa7gGeNR/ro7j2O9d8sJ0/h+7 tDIM8MbiwXkbSczJ5KvaZ2lkffvTgWV4mKmn0yrJwnzmCAjkwswjLDZhl w==; IronPort-SDR: 2wZSNuralpHWtG9oHi3q0vXmCRtCIbQ4EFznGgR6Oazp8yTLdgOTIC8Fv3wrjzp+Mm7cGEt/wM ECf+jIEI+kwBD+LChadcOhkIdL8C2ylYFMfNYKptlB3Xm1mzspVKHVhkYBi25lacFwVaUXVc7A LGVCIrFb/HVuZPDlyryDJ+EEW8g4cuSd/JrQo4tYkuJ3tOX+75L9Tp8+KQllg5t5krmgB0Atib ZmVGuEAl0fWdI0dhBAY3ni3V+xJmM7AkokXVJLPydKpAg8hc+KFA7kqL0QWeAVmKuGStUPcymD SSM= X-IronPort-AV: E=Sophos;i="5.75,335,1589212800"; d="scan'208";a="251358892" Received: from mail-bn8nam12lp2172.outbound.protection.outlook.com (HELO NAM12-BN8-obe.outbound.protection.outlook.com) ([104.47.55.172]) by ob1.hgst.iphmx.com with ESMTP; 10 Jul 2020 17:01:28 +0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=POCXbwquyGTkyGSI1xiK+51yiEvzcODBZ+pi4eZVkKkyxIZOBp8FkxVW92EM4/+fTMhfOjrYcjkCyNlarr41EAfwotFxHeAV7+ad6L0Tl8rh2H6tOU5BV7ylfZrczsE6kntNidmbS+XNx5T2i1GoZr7G38gkzswHLuT7x/BU97Gf4n9YYC2XLyne64j4/42D1+gPHSP2qo6ZEkqTAjqLICQgkWWuB47kcd/83a5LLPiL8jkkNyLtVbQ182LIL0yg/uA47jC++LWMaGiuC+PehlpCvNYXPAp2GxXV5EyntaZc+gQ2qEU2k+VHlcTQX+UvHPKFG/V+l/cUApcnmGl8EA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eGmImHprEx3Lq9WiCNm9xR5RxDS+H3xDJZW2fdOX5Ho=; b=EgiNJmerT4lOANIWi28Y3naHxZsylfuep/VBfaRYmRepGAaL3+xCaKRh7J1z7YK4qGb+2IpPFpFGcH/Ijag3bbB54089ZDNH4omv2ukZuS5E4wBTbjkJ0IkW1wieimWelQ8RRCkAJxa4edF+5k1qbXtrBxdcMfl1aaQsvCh7o2LtzyLygr85tt3u34cgf8ActA0Kc8ZxuOYIHWll7RCQ8fVl6UfC75y/Yk4N4/wz+pol1CP9f0ZVadafuI4EoKFPEyLh+rvA6vLu7MLJhYOfPzSNeFuYwpLhqPFtoUoPDLkTXRAfMY06IC+sDRAaj455CvKKMZhrGfkqF97bFcpo5A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=wdc.com; dmarc=pass action=none header.from=wdc.com; dkim=pass header.d=wdc.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sharedspace.onmicrosoft.com; s=selector2-sharedspace-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eGmImHprEx3Lq9WiCNm9xR5RxDS+H3xDJZW2fdOX5Ho=; b=ajBUWTT2Vg7QmvzJaSI3pUqJ3sXh60G1TLvz9HNCt/zIdC7MEgD6GXlc8pEA69uJrtEIc4XeAOU5I9e5ETxg8RleWrpDrY8OLTfvnGoeW0td0rZTZFktNr3StKdNPAY8BMn01mmmTUYEilv5adNvwuqsMVGr3+2EUA/gadXwhsU= Authentication-Results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=wdc.com; Received: from DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) by DM5PR04MB0346.namprd04.prod.outlook.com (2603:10b6:3:6f::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3174.21; Fri, 10 Jul 2020 09:01:27 +0000 Received: from DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::e0a4:aa82:1847:dea5]) by DM6PR04MB6201.namprd04.prod.outlook.com ([fe80::e0a4:aa82:1847:dea5%7]) with mapi id 15.20.3174.023; Fri, 10 Jul 2020 09:01:26 +0000 From: Anup Patel To: Will Deacon Cc: Paolo Bonzini , Atish Patra , Alistair Francis , Anup Patel , kvm@vger.kernel.org, kvm-riscv@lists.infradead.org, Anup Patel Subject: [RFC PATCH v4 5/8] riscv: Add PLIC device emulation Date: Fri, 10 Jul 2020 14:30:32 +0530 Message-Id: <20200710090035.123941-6-anup.patel@wdc.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200710090035.123941-1-anup.patel@wdc.com> References: <20200710090035.123941-1-anup.patel@wdc.com> X-ClientProxiedBy: PN1PR0101CA0029.INDPRD01.PROD.OUTLOOK.COM (2603:1096:c00:c::15) To DM6PR04MB6201.namprd04.prod.outlook.com (2603:10b6:5:127::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from wdc.com (103.15.57.207) by PN1PR0101CA0029.INDPRD01.PROD.OUTLOOK.COM (2603:1096:c00:c::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3174.21 via Frontend Transport; Fri, 10 Jul 2020 09:01:23 +0000 X-Mailer: git-send-email 2.25.1 X-Originating-IP: [103.15.57.207] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: e1891a43-93fc-4bc0-94c4-08d824afd3c6 X-MS-TrafficTypeDiagnostic: DM5PR04MB0346: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: WDCIPOUTBOUND: EOP-TRUE X-MS-Oob-TLC-OOBClassifiers: OLM:5236; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FSyLlAKMTc5+9vYi2C3c6xGpkk2JDbdjd3js5pJH8xb8L58jR5YXT42SnFH6PKGmAEooFl0BoUu4XkK2CEqEIEWRyvX4RtlEfuhgxYwSKnvLDR8buA2J4s9nvvwol4aYIJcN4Pc9dynksDWT2jDnkJxZQMOcroecoJAmSVyCPo3bFh3uvT2jA0VNxSKpF+jBLkaJyaytjdhUtEX+sMxnU4mEX5rtlL2hiHjuFibClchw/SYtJwrLEYjV0hjkJ0hz4NpMCLIlesHLmzCnnAw3TsTtooRzsabop3v/VkFWyG26uBbtJDR3VDvbNIuZZ76X X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR04MB6201.namprd04.prod.outlook.com;PTR:;CAT:NONE;SFTY:;SFS:(4636009)(396003)(346002)(136003)(366004)(376002)(39860400002)(66476007)(186003)(66556008)(16526019)(6666004)(30864003)(52116002)(8886007)(1076003)(6916009)(86362001)(7696005)(36756003)(26005)(66946007)(4326008)(5660300002)(8936002)(956004)(2906002)(2616005)(83380400001)(55016002)(478600001)(316002)(44832011)(8676002)(54906003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData: fYqTan2t4uWZmRQV2IS/8bBcdLIufkOQPgfTj71ansDNhpVoYQ3dQldFlltDAsEHVxz7wcOhwGL+yjhWdi68YXAE/RBLoS8UNZepXZRpsNMd9t7hcAEshQdFp+SbGQnNzdTt4Tdp+hb19TtYtxLnT8xY4IURDakYF9NId+91QuIMAs6rSSDbO8Crq0dOBw1m3hvyKzg5L4udwk7nCund1kptbyTmxCQAvXwk28tFi2jBYPoX6Ajb2zg9Y02oOHRvFfW3FdLuRp+LDsVNEteythXYsm1HoJC5UsNG68ZJCQhEukx31hLTtYPXieQr/d18tpEG0CcdU3xkUwC7OxwGEogxWLs10LPmnwgn+eqdLWDRmR/6XZ070ecqM42dCXVwDGYplafjQ0+xDQBCS9nhcNlYjASq9ex3teJDFi2/J/oqqtvgB83gI7RFDIveQsiMc4dLckDk/8hBfUaPQ8sHG5sRzobOrhkbbXwJ8cL+WSo= X-OriginatorOrg: wdc.com X-MS-Exchange-CrossTenant-Network-Message-Id: e1891a43-93fc-4bc0-94c4-08d824afd3c6 X-MS-Exchange-CrossTenant-AuthSource: DM6PR04MB6201.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Jul 2020 09:01:26.7956 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: b61c8803-16f3-4c35-9b17-6f65f441df86 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: u5Gbf4argVN8BqtPe3Ed2qQcw0y723xiodl2jO2dSS9GZ+/SyYMdH0Sp3E1A6d/Zxk9xL+bN+2DwXUAJIe5WFw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR04MB0346 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org The PLIC (platform level interrupt controller) manages peripheral interrupts in RISC-V world. The per-CPU interrupts are managed using CPU CSRs hence virtualized in-kernel by KVM RISC-V. This patch adds PLIC device emulation for KVMTOOL RISC-V. Signed-off-by: Anup Patel --- Makefile | 1 + riscv/include/kvm/kvm-arch.h | 2 + riscv/irq.c | 4 +- riscv/plic.c | 513 +++++++++++++++++++++++++++++++++++ 4 files changed, 518 insertions(+), 2 deletions(-) create mode 100644 riscv/plic.c diff --git a/Makefile b/Makefile index 1782cb7..e2dd39d 100644 --- a/Makefile +++ b/Makefile @@ -203,6 +203,7 @@ ifeq ($(ARCH),riscv) OBJS += riscv/irq.o OBJS += riscv/kvm.o OBJS += riscv/kvm-cpu.o + OBJS += riscv/plic.o ifeq ($(RISCV_XLEN),32) CFLAGS += -mabi=ilp32d -march=rv32gc endif diff --git a/riscv/include/kvm/kvm-arch.h b/riscv/include/kvm/kvm-arch.h index 26816f4..bb6d99d 100644 --- a/riscv/include/kvm/kvm-arch.h +++ b/riscv/include/kvm/kvm-arch.h @@ -76,4 +76,6 @@ static inline bool riscv_addr_in_ioport_region(u64 phys_addr) enum irq_type; +void plic__irq_trig(struct kvm *kvm, int irq, int level, bool edge); + #endif /* KVM__KVM_ARCH_H */ diff --git a/riscv/irq.c b/riscv/irq.c index 8e605ef..78a582d 100644 --- a/riscv/irq.c +++ b/riscv/irq.c @@ -4,10 +4,10 @@ void kvm__irq_line(struct kvm *kvm, int irq, int level) { - /* TODO: */ + plic__irq_trig(kvm, irq, level, false); } void kvm__irq_trigger(struct kvm *kvm, int irq) { - /* TODO: */ + plic__irq_trig(kvm, irq, 1, true); } diff --git a/riscv/plic.c b/riscv/plic.c new file mode 100644 index 0000000..1faa1d5 --- /dev/null +++ b/riscv/plic.c @@ -0,0 +1,513 @@ + +#include "kvm/devices.h" +#include "kvm/ioeventfd.h" +#include "kvm/ioport.h" +#include "kvm/kvm.h" +#include "kvm/kvm-cpu.h" +#include "kvm/irq.h" +#include "kvm/mutex.h" + +#include +#include +#include +#include + +/* + * From the RISC-V Privlidged Spec v1.10: + * + * Global interrupt sources are assigned small unsigned integer identifiers, + * beginning at the value 1. An interrupt ID of 0 is reserved to mean no + * interrupt. Interrupt identifiers are also used to break ties when two or + * more interrupt sources have the same assigned priority. Smaller values of + * interrupt ID take precedence over larger values of interrupt ID. + * + * While the RISC-V supervisor spec doesn't define the maximum number of + * devices supported by the PLIC, the largest number supported by devices + * marked as 'riscv,plic0' (which is the only device type this driver supports, + * and is the only extant PLIC as of now) is 1024. As mentioned above, device + * 0 is defined to be non-existant so this device really only supports 1023 + * devices. + */ + +#define MAX_DEVICES 1024 +#define MAX_CONTEXTS 15872 + +/* + * The PLIC consists of memory-mapped control registers, with a memory map as + * follows: + * + * base + 0x000000: Reserved (interrupt source 0 does not exist) + * base + 0x000004: Interrupt source 1 priority + * base + 0x000008: Interrupt source 2 priority + * ... + * base + 0x000FFC: Interrupt source 1023 priority + * base + 0x001000: Pending 0 + * base + 0x001FFF: Pending + * base + 0x002000: Enable bits for sources 0-31 on context 0 + * base + 0x002004: Enable bits for sources 32-63 on context 0 + * ... + * base + 0x0020FC: Enable bits for sources 992-1023 on context 0 + * base + 0x002080: Enable bits for sources 0-31 on context 1 + * ... + * base + 0x002100: Enable bits for sources 0-31 on context 2 + * ... + * base + 0x1F1F80: Enable bits for sources 992-1023 on context 15871 + * base + 0x1F1F84: Reserved + * ... (higher context IDs would fit here, but wouldn't fit + * inside the per-context priority vector) + * base + 0x1FFFFC: Reserved + * base + 0x200000: Priority threshold for context 0 + * base + 0x200004: Claim/complete for context 0 + * base + 0x200008: Reserved + * ... + * base + 0x200FFC: Reserved + * base + 0x201000: Priority threshold for context 1 + * base + 0x201004: Claim/complete for context 1 + * ... + * base + 0xFFE000: Priority threshold for context 15871 + * base + 0xFFE004: Claim/complete for context 15871 + * base + 0xFFE008: Reserved + * ... + * base + 0xFFFFFC: Reserved + */ + +/* Each interrupt source has a priority register associated with it. */ +#define PRIORITY_BASE 0 +#define PRIORITY_PER_ID 4 + +/* + * Each hart context has a vector of interupt enable bits associated with it. + * There's one bit for each interrupt source. + */ +#define ENABLE_BASE 0x2000 +#define ENABLE_PER_HART 0x80 + +/* + * Each hart context has a set of control registers associated with it. Right + * now there's only two: a source priority threshold over which the hart will + * take an interrupt, and a register to claim interrupts. + */ +#define CONTEXT_BASE 0x200000 +#define CONTEXT_PER_HART 0x1000 +#define CONTEXT_THRESHOLD 0 +#define CONTEXT_CLAIM 4 + +#define REG_SIZE 0x1000000 + +struct plic_state; + +struct plic_context { + /* State to which this belongs */ + struct plic_state *s; + + /* Static Configuration */ + u32 num; + struct kvm_cpu *vcpu; + + /* Local IRQ state */ + struct mutex irq_lock; + u8 irq_priority_threshold; + u32 irq_enable[MAX_DEVICES/32]; + u32 irq_pending[MAX_DEVICES/32]; + u8 irq_pending_priority[MAX_DEVICES]; + u32 irq_claimed[MAX_DEVICES/32]; + u32 irq_autoclear[MAX_DEVICES/32]; +}; + +struct plic_state { + bool ready; + struct kvm *kvm; + struct device_header dev_hdr; + + /* Static Configuration */ + u32 num_irq; + u32 num_irq_word; + u32 max_prio; + + /* Context Array */ + u32 num_context; + struct plic_context *contexts; + + /* Global IRQ state */ + struct mutex irq_lock; + u8 irq_priority[MAX_DEVICES]; + u32 irq_level[MAX_DEVICES/32]; +}; + +static struct plic_state plic; + +/* Note: Must be called with c->irq_lock held */ +static u32 __plic_context_best_pending_irq(struct plic_state *s, + struct plic_context *c) +{ + u8 best_irq_prio = 0; + u32 i, j, irq, best_irq = 0; + + for (i = 0; i < s->num_irq_word; i++) { + if (!c->irq_pending[i]) + continue; + + for (j = 0; j < 32; j++) { + irq = i * 32 + j; + if ((s->num_irq <= irq) || + !(c->irq_pending[i] & (1 << j)) || + (c->irq_claimed[i] & (1 << j))) + continue; + + if (!best_irq || + (best_irq_prio < c->irq_pending_priority[irq])) { + best_irq = irq; + best_irq_prio = c->irq_pending_priority[irq]; + } + } + } + + return best_irq; +} + +/* Note: Must be called with c->irq_lock held */ +static void __plic_context_irq_update(struct plic_state *s, + struct plic_context *c) +{ + u32 best_irq = __plic_context_best_pending_irq(s, c); + u32 virq = (best_irq) ? KVM_INTERRUPT_SET : KVM_INTERRUPT_UNSET; + + if (ioctl(c->vcpu->vcpu_fd, KVM_INTERRUPT, &virq) < 0) + pr_warning("KVM_INTERRUPT failed"); +} + +/* Note: Must be called with c->irq_lock held */ +static u32 __plic_context_irq_claim(struct plic_state *s, + struct plic_context *c) +{ + u32 virq = KVM_INTERRUPT_UNSET; + u32 best_irq = __plic_context_best_pending_irq(s, c); + u32 best_irq_word = best_irq / 32; + u32 best_irq_mask = (1 << (best_irq % 32)); + + if (ioctl(c->vcpu->vcpu_fd, KVM_INTERRUPT, &virq) < 0) + pr_warning("KVM_INTERRUPT failed"); + + if (best_irq) { + if (c->irq_autoclear[best_irq_word] & best_irq_mask) { + c->irq_pending[best_irq_word] &= ~best_irq_mask; + c->irq_pending_priority[best_irq] = 0; + c->irq_claimed[best_irq_word] &= ~best_irq_mask; + c->irq_autoclear[best_irq_word] &= ~best_irq_mask; + } else + c->irq_claimed[best_irq_word] |= best_irq_mask; + } + + __plic_context_irq_update(s, c); + + return best_irq; +} + +void plic__irq_trig(struct kvm *kvm, int irq, int level, bool edge) +{ + bool irq_marked = false; + u8 i, irq_prio, irq_word; + u32 irq_mask; + struct plic_context *c = NULL; + struct plic_state *s = &plic; + + if (!s->ready) + return; + + if (irq <= 0 || s->num_irq <= (u32)irq) + goto done; + + mutex_lock(&s->irq_lock); + + irq_prio = s->irq_priority[irq]; + irq_word = irq / 32; + irq_mask = 1 << (irq % 32); + + if (level) + s->irq_level[irq_word] |= irq_mask; + else + s->irq_level[irq_word] &= ~irq_mask; + + /* + * Note: PLIC interrupts are level-triggered. As of now, + * there is no notion of edge-triggered interrupts. To + * handle this we auto-clear edge-triggered interrupts + * when PLIC context CLAIM register is read. + */ + for (i = 0; i < s->num_context; i++) { + c = &s->contexts[i]; + + mutex_lock(&c->irq_lock); + if (c->irq_enable[irq_word] & irq_mask) { + if (level) { + c->irq_pending[irq_word] |= irq_mask; + c->irq_pending_priority[irq] = irq_prio; + if (edge) + c->irq_autoclear[irq_word] |= irq_mask; + } else { + c->irq_pending[irq_word] &= ~irq_mask; + c->irq_pending_priority[irq] = 0; + c->irq_claimed[irq_word] &= ~irq_mask; + c->irq_autoclear[irq_word] &= ~irq_mask; + } + __plic_context_irq_update(s, c); + irq_marked = true; + } + mutex_unlock(&c->irq_lock); + + if (irq_marked) + break; + } + +done: + mutex_unlock(&s->irq_lock); +} + +static void plic__priority_read(struct plic_state *s, + u64 offset, void *data) +{ + u32 irq = (offset >> 2); + + if (irq == 0 || irq >= s->num_irq) + return; + + mutex_lock(&s->irq_lock); + ioport__write32(data, s->irq_priority[irq]); + mutex_unlock(&s->irq_lock); +} + +static void plic__priority_write(struct plic_state *s, + u64 offset, void *data) +{ + u32 val, irq = (offset >> 2); + + if (irq == 0 || irq >= s->num_irq) + return; + + mutex_lock(&s->irq_lock); + val = ioport__read32(data); + val &= ((1 << PRIORITY_PER_ID) - 1); + s->irq_priority[irq] = val; + mutex_unlock(&s->irq_lock); +} + +static void plic__context_enable_read(struct plic_state *s, + struct plic_context *c, + u64 offset, void *data) +{ + u32 irq_word = offset >> 2; + + if (s->num_irq_word < irq_word) + return; + + mutex_lock(&c->irq_lock); + ioport__write32(data, c->irq_enable[irq_word]); + mutex_unlock(&c->irq_lock); +} + +static void plic__context_enable_write(struct plic_state *s, + struct plic_context *c, + u64 offset, void *data) +{ + u8 irq_prio; + u32 i, irq, irq_mask; + u32 irq_word = offset >> 2; + u32 old_val, new_val, xor_val; + + if (s->num_irq_word < irq_word) + return; + + mutex_lock(&s->irq_lock); + + mutex_lock(&c->irq_lock); + + old_val = c->irq_enable[irq_word]; + new_val = ioport__read32(data); + + if (irq_word == 0) + new_val &= ~0x1; + + c->irq_enable[irq_word] = new_val; + + xor_val = old_val ^ new_val; + for (i = 0; i < 32; i++) { + irq = irq_word * 32 + i; + irq_mask = 1 << i; + irq_prio = s->irq_priority[irq]; + if (!(xor_val & irq_mask)) + continue; + if ((new_val & irq_mask) && + (s->irq_level[irq_word] & irq_mask)) { + c->irq_pending[irq_word] |= irq_mask; + c->irq_pending_priority[irq] = irq_prio; + } else if (!(new_val & irq_mask)) { + c->irq_pending[irq_word] &= ~irq_mask; + c->irq_pending_priority[irq] = 0; + c->irq_claimed[irq_word] &= ~irq_mask; + } + } + + __plic_context_irq_update(s, c); + + mutex_unlock(&c->irq_lock); + + mutex_unlock(&s->irq_lock); +} + +static void plic__context_read(struct plic_state *s, + struct plic_context *c, + u64 offset, void *data) +{ + mutex_lock(&c->irq_lock); + + switch (offset) { + case CONTEXT_THRESHOLD: + ioport__write32(data, c->irq_priority_threshold); + break; + case CONTEXT_CLAIM: + ioport__write32(data, __plic_context_irq_claim(s, c)); + break; + default: + break; + }; + + mutex_unlock(&c->irq_lock); +} + +static void plic__context_write(struct plic_state *s, + struct plic_context *c, + u64 offset, void *data) +{ + u32 val; + bool irq_update = false; + + mutex_lock(&c->irq_lock); + + switch (offset) { + case CONTEXT_THRESHOLD: + val = ioport__read32(data); + val &= ((1 << PRIORITY_PER_ID) - 1); + if (val <= s->max_prio) + c->irq_priority_threshold = val; + else + irq_update = true; + break; + case CONTEXT_CLAIM: + break; + default: + irq_update = true; + break; + }; + + if (irq_update) + __plic_context_irq_update(s, c); + + mutex_unlock(&c->irq_lock); +} + +static void plic__mmio_callback(struct kvm_cpu *vcpu, + u64 addr, u8 *data, u32 len, + u8 is_write, void *ptr) +{ + u32 cntx; + struct plic_state *s = ptr; + + if (len != 4) + die("plic: invalid len=%d", len); + + addr &= ~0x3; + addr -= RISCV_PLIC; + + if (is_write) { + if (PRIORITY_BASE <= addr && addr < ENABLE_BASE) { + plic__priority_write(s, addr, data); + } else if (ENABLE_BASE <= addr && addr < CONTEXT_BASE) { + cntx = (addr - ENABLE_BASE) / ENABLE_PER_HART; + addr -= cntx * ENABLE_PER_HART + ENABLE_BASE; + if (cntx < s->num_context) + plic__context_enable_write(s, + &s->contexts[cntx], + addr, data); + } else if (CONTEXT_BASE <= addr && addr < REG_SIZE) { + cntx = (addr - CONTEXT_BASE) / CONTEXT_PER_HART; + addr -= cntx * CONTEXT_PER_HART + CONTEXT_BASE; + if (cntx < s->num_context) + plic__context_write(s, &s->contexts[cntx], + addr, data); + } + } else { + if (PRIORITY_BASE <= addr && addr < ENABLE_BASE) { + plic__priority_read(s, addr, data); + } else if (ENABLE_BASE <= addr && addr < CONTEXT_BASE) { + cntx = (addr - ENABLE_BASE) / ENABLE_PER_HART; + addr -= cntx * ENABLE_PER_HART + ENABLE_BASE; + if (cntx < s->num_context) + plic__context_enable_read(s, + &s->contexts[cntx], + addr, data); + } else if (CONTEXT_BASE <= addr && addr < REG_SIZE) { + cntx = (addr - CONTEXT_BASE) / CONTEXT_PER_HART; + addr -= cntx * CONTEXT_PER_HART + CONTEXT_BASE; + if (cntx < s->num_context) + plic__context_read(s, &s->contexts[cntx], + addr, data); + } + } +} + +static int plic__init(struct kvm *kvm) +{ + u32 i; + int ret; + struct plic_context *c; + + plic.kvm = kvm; + plic.dev_hdr = (struct device_header) { + .bus_type = DEVICE_BUS_MMIO, + }; + + plic.num_irq = MAX_DEVICES; + plic.num_irq_word = plic.num_irq / 32; + if ((plic.num_irq_word * 32) < plic.num_irq) + plic.num_irq_word++; + plic.max_prio = (1UL << PRIORITY_PER_ID) - 1; + + plic.num_context = kvm->nrcpus * 2; + plic.contexts = calloc(plic.num_context, sizeof(struct plic_context)); + if (!plic.contexts) + return -ENOMEM; + for (i = 0; i < plic.num_context; i++) { + c = &plic.contexts[i]; + c->s = &plic; + c->num = i; + c->vcpu = kvm->cpus[i / 2]; + mutex_init(&c->irq_lock); + } + + mutex_init(&plic.irq_lock); + + ret = kvm__register_mmio(kvm, RISCV_PLIC, RISCV_PLIC_SIZE, + false, plic__mmio_callback, &plic); + if (ret) + return ret; + + ret = device__register(&plic.dev_hdr); + if (ret) + return ret; + + plic.ready = true; + + return 0; + +} +dev_init(plic__init); + +static int plic__exit(struct kvm *kvm) +{ + plic.ready = false; + kvm__deregister_mmio(kvm, RISCV_PLIC); + free(plic.contexts); + + return 0; +} +dev_exit(plic__exit);