[v8,00/23] QEMU RISC-V AIA support

Message

Anup Patel Jan. 19, 2022, 3:25 p.m. UTC

From: Anup Patel <[e-mail]>

The advanced interrupt architecture (AIA) extends the per-HART local
interrupt support. Along with this, it also adds IMSIC (MSI contrllor)
and Advanced PLIC (wired interrupt controller).

The latest AIA draft specification can be found here:
https://github.com/riscv/riscv-aia/releases/download/0.2-draft.28/riscv-interrupts-028.pdf

This series adds RISC-V AIA support in QEMU which includes emulating all
AIA local CSRs, APLIC, and IMSIC. Only AIA local interrupt filtering is
not implemented because we don't have any local interrupt greater than 12.

To enable AIA in QEMU, use one of the following:
1) Only AIA local interrupt CSRs: Pass "x-aia=true" as CPU paramenter
   in the QEMU command-line
2) Only APLIC for virt machine: Pass "aia=aplic" as machine parameter
   in the QEMU command-line
3) Both APLIC and IMSIC for virt machine: Pass "aia=aplic-imsic" as
   machine parameter in the QEMU command-line
4) Both APLIC and IMSIC with 2 guest files for virt machine: Pass
   "aia=aplic-imsic,aia-guests=2" as machine parameter in the QEMU
   command-line

To test series, we require OpenSBI and Linux with AIA support which can
be found in:
riscv_aia_v2 branch at https://github.com/avpatel/opensbi.git
riscv_aia_v1 branch at https://github.com/avpatel/linux.git

This series can be found riscv_aia_v8 branch at:
https://github.com/avpatel/qemu.git

Changes since v7:
 - Rebased on latest riscv-to-apply.next branch of Alistair's repo
 - Improved default priority assignment in PATCH9

Changes since v6:
 - Fixed priority comparison in riscv_cpu_pending_to_irq() of PATCH9
 - Fixed typos in comments added by PATCH11
 - Added "pend = true;" for CSR_MSETEIPNUM case of rmw_xsetclreinum()
   in PATCH15
 - Handle ithreshold == 0 case in riscv_aplic_idc_topi() of PATCH18
 - Allow setting pending bit for Level0 or Level1 interrupts in
   riscv_aplic_set_pending() of PATCH18
 - Force DOMAINCFG[31:24] bits to 0x80 in riscv_aplic_read() of PATCH18
 - For APLIC direct mode, set target.iprio to 1 when zero is writtern
   in PATCH18
 - Handle eithreshold == 0 case in riscv_imsic_topei() of PATCH20

Changes since v5:
 - Moved VSTOPI_NUM_SRCS define to top of the file in PATCH13
 - Fixed typo in PATCH16

Changes since v4:
 - Changed IRQ_LOCAL_MAX to 16 in PATCH2
 - Fixed typo in PATCH10
 - Replaced TARGET_LONG_BITS with riscv_cpu_mxl_bits(env) in PATCH11
 - Replaced TARGET_LONG_BITS with riscv_cpu_mxl_bits(env) in PATCH14
 - Replaced TARGET_LONG_BITS with riscv_cpu_mxl_bits(env) in PATCH15
 - Replaced TARGET_LONG_BITS with xlen passed via ireg callback in PATCH20
 - Retrict maximum IMSIC guest files per-HART of virt machine to 7 in
   PATCH21.
 - Added separate PATCH23 to increase maximum number of allowed CPUs
   for virt machine

Changes since v3:
 - Replaced "aplic,xyz" and "imsic,xyz" DT properties with "riscv,xyz"
   DT properties because "aplic" and "imsic" are not valid vendor names
   required by Linux DT schema checker.

Changes since v2:
 - Update PATCH4 to check and inject interrupt after V=1 when
   transitioning from V=0 to V=1

Changes since v1:
 - Revamped whole series and created more granular patches
 - Added HGEIE and HGEIP CSR emulation for H-extension
 - Added APLIC emulation
 - Added IMSIC emulation

Anup Patel (23):
  target/riscv: Fix trap cause for RV32 HS-mode CSR access from RV64
    HS-mode
  target/riscv: Implement SGEIP bit in hip and hie CSRs
  target/riscv: Implement hgeie and hgeip CSRs
  target/riscv: Improve delivery of guest external interrupts
  target/riscv: Allow setting CPU feature from machine/device emulation
  target/riscv: Add AIA cpu feature
  target/riscv: Add defines for AIA CSRs
  target/riscv: Allow AIA device emulation to set ireg rmw callback
  target/riscv: Implement AIA local interrupt priorities
  target/riscv: Implement AIA CSRs for 64 local interrupts on RV32
  target/riscv: Implement AIA hvictl and hviprioX CSRs
  target/riscv: Implement AIA interrupt filtering CSRs
  target/riscv: Implement AIA mtopi, stopi, and vstopi CSRs
  target/riscv: Implement AIA xiselect and xireg CSRs
  target/riscv: Implement AIA IMSIC interface CSRs
  hw/riscv: virt: Use AIA INTC compatible string when available
  target/riscv: Allow users to force enable AIA CSRs in HART
  hw/intc: Add RISC-V AIA APLIC device emulation
  hw/riscv: virt: Add optional AIA APLIC support to virt machine
  hw/intc: Add RISC-V AIA IMSIC device emulation
  hw/riscv: virt: Add optional AIA IMSIC support to virt machine
  docs/system: riscv: Document AIA options for virt machine
  hw/riscv: virt: Increase maximum number of allowed CPUs

 docs/system/riscv/virt.rst    |   16 +
 hw/intc/Kconfig               |    6 +
 hw/intc/meson.build           |    2 +
 hw/intc/riscv_aplic.c         |  975 +++++++++++++++++++++++++
 hw/intc/riscv_imsic.c         |  448 ++++++++++++
 hw/riscv/Kconfig              |    2 +
 hw/riscv/virt.c               |  706 +++++++++++++++---
 include/hw/intc/riscv_aplic.h |   79 ++
 include/hw/intc/riscv_imsic.h |   68 ++
 include/hw/riscv/virt.h       |   41 +-
 target/riscv/cpu.c            |  103 ++-
 target/riscv/cpu.h            |   72 +-
 target/riscv/cpu_bits.h       |  123 ++++
 target/riscv/cpu_helper.c     |  343 ++++++++-
 target/riscv/csr.c            | 1280 ++++++++++++++++++++++++++++++---
 target/riscv/machine.c        |   24 +-
 16 files changed, 3988 insertions(+), 300 deletions(-)
 create mode 100644 hw/intc/riscv_aplic.c
 create mode 100644 hw/intc/riscv_imsic.c
 create mode 100644 include/hw/intc/riscv_aplic.h
 create mode 100644 include/hw/intc/riscv_imsic.h

Comments

Frank Chang Jan. 19, 2022, 3:37 p.m. UTC | #1

On Wed, Jan 19, 2022 at 11:27 PM Anup Patel <[e-mail]> wrote:

> From: Anup Patel <[e-mail]>
>
> The RISC-V AIA (Advanced Interrupt Architecture) defines a new
> interrupt controller for wired interrupts called APLIC (Advanced
> Platform Level Interrupt Controller). The APLIC is capabable of
> forwarding wired interupts to RISC-V HARTs directly or as MSIs
> (Message Signaled Interupts).
>
> This patch adds device emulation for RISC-V AIA APLIC.
>
> Signed-off-by: Anup Patel <[e-mail]>
> Signed-off-by: Anup Patel <[e-mail]>
> Reviewed-by: Frank Chang <[e-mail]>
> ---
>  hw/intc/Kconfig               |   3 +
>  hw/intc/meson.build           |   1 +
>  hw/intc/riscv_aplic.c         | 975 ++++++++++++++++++++++++++++++++++
>  include/hw/intc/riscv_aplic.h |  79 +++
>  4 files changed, 1058 insertions(+)
>  create mode 100644 hw/intc/riscv_aplic.c
>  create mode 100644 include/hw/intc/riscv_aplic.h
>
> diff --git a/hw/intc/Kconfig b/hw/intc/Kconfig
> index 010ded7eae..528e77b4a6 100644
> --- a/hw/intc/Kconfig
> +++ b/hw/intc/Kconfig
> @@ -70,6 +70,9 @@ config LOONGSON_LIOINTC
>  config RISCV_ACLINT
>      bool
>
> +config RISCV_APLIC
> +    bool
> +
>  config SIFIVE_PLIC
>      bool
>
> diff --git a/hw/intc/meson.build b/hw/intc/meson.build
> index 70080bc161..7466024402 100644
> --- a/hw/intc/meson.build
> +++ b/hw/intc/meson.build
> @@ -50,6 +50,7 @@ specific_ss.add(when: 'CONFIG_S390_FLIC', if_true:
> files('s390_flic.c'))
>  specific_ss.add(when: 'CONFIG_S390_FLIC_KVM', if_true:
> files('s390_flic_kvm.c'))
>  specific_ss.add(when: 'CONFIG_SH_INTC', if_true: files('sh_intc.c'))
>  specific_ss.add(when: 'CONFIG_RISCV_ACLINT', if_true:
> files('riscv_aclint.c'))
> +specific_ss.add(when: 'CONFIG_RISCV_APLIC', if_true:
> files('riscv_aplic.c'))
>  specific_ss.add(when: 'CONFIG_SIFIVE_PLIC', if_true:
> files('sifive_plic.c'))
>  specific_ss.add(when: 'CONFIG_XICS', if_true: files('xics.c'))
>  specific_ss.add(when: ['CONFIG_KVM', 'CONFIG_XICS'],
> diff --git a/hw/intc/riscv_aplic.c b/hw/intc/riscv_aplic.c
> new file mode 100644
> index 0000000000..885c1de2af
> --- /dev/null
> +++ b/hw/intc/riscv_aplic.c
> @@ -0,0 +1,975 @@
> +/*
> + * RISC-V APLIC (Advanced Platform Level Interrupt Controller)
> + *
> + * Copyright (c) 2021 Western Digital Corporation or its affiliates.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms and conditions of the GNU General Public License,
> + * version 2 or later, as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
> for
> + * more details.
> + *
> + * You should have received a copy of the GNU General Public License
> along with
> + * this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qapi/error.h"
> +#include "qemu/log.h"
> +#include "qemu/module.h"
> +#include "qemu/error-report.h"
> +#include "qemu/bswap.h"
> +#include "exec/address-spaces.h"
> +#include "hw/sysbus.h"
> +#include "hw/pci/msi.h"
> +#include "hw/boards.h"
> +#include "hw/qdev-properties.h"
> +#include "hw/intc/riscv_aplic.h"
> +#include "hw/irq.h"
> +#include "target/riscv/cpu.h"
> +#include "sysemu/sysemu.h"
> +#include "migration/vmstate.h"
> +
> +#define APLIC_MAX_IDC                  (1UL << 14)
> +#define APLIC_MAX_SOURCE               1024
> +#define APLIC_MIN_IPRIO_BITS           1
> +#define APLIC_MAX_IPRIO_BITS           8
> +#define APLIC_MAX_CHILDREN             1024
> +
> +#define APLIC_DOMAINCFG                0x0000
> +#define APLIC_DOMAINCFG_RDONLY         0x80000000
> +#define APLIC_DOMAINCFG_IE             (1 << 8)
> +#define APLIC_DOMAINCFG_DM             (1 << 2)
> +#define APLIC_DOMAINCFG_BE             (1 << 0)
> +
> +#define APLIC_SOURCECFG_BASE           0x0004
> +#define APLIC_SOURCECFG_D              (1 << 10)
> +#define APLIC_SOURCECFG_CHILDIDX_MASK  0x000003ff
> +#define APLIC_SOURCECFG_SM_MASK        0x00000007
> +#define APLIC_SOURCECFG_SM_INACTIVE    0x0
> +#define APLIC_SOURCECFG_SM_DETACH      0x1
> +#define APLIC_SOURCECFG_SM_EDGE_RISE   0x4
> +#define APLIC_SOURCECFG_SM_EDGE_FALL   0x5
> +#define APLIC_SOURCECFG_SM_LEVEL_HIGH  0x6
> +#define APLIC_SOURCECFG_SM_LEVEL_LOW   0x7
> +
> +#define APLIC_MMSICFGADDR              0x1bc0
> +#define APLIC_MMSICFGADDRH             0x1bc4
> +#define APLIC_SMSICFGADDR              0x1bc8
> +#define APLIC_SMSICFGADDRH             0x1bcc
> +
> +#define APLIC_xMSICFGADDRH_L           (1UL << 31)
> +#define APLIC_xMSICFGADDRH_HHXS_MASK   0x1f
> +#define APLIC_xMSICFGADDRH_HHXS_SHIFT  24
> +#define APLIC_xMSICFGADDRH_LHXS_MASK   0x7
> +#define APLIC_xMSICFGADDRH_LHXS_SHIFT  20
> +#define APLIC_xMSICFGADDRH_HHXW_MASK   0x7
> +#define APLIC_xMSICFGADDRH_HHXW_SHIFT  16
> +#define APLIC_xMSICFGADDRH_LHXW_MASK   0xf
> +#define APLIC_xMSICFGADDRH_LHXW_SHIFT  12
> +#define APLIC_xMSICFGADDRH_BAPPN_MASK  0xfff
> +
> +#define APLIC_xMSICFGADDR_PPN_SHIFT    12
> +
> +#define APLIC_xMSICFGADDR_PPN_HART(__lhxs) \
> +    ((1UL << (__lhxs)) - 1)
> +
> +#define APLIC_xMSICFGADDR_PPN_LHX_MASK(__lhxw) \
> +    ((1UL << (__lhxw)) - 1)
> +#define APLIC_xMSICFGADDR_PPN_LHX_SHIFT(__lhxs) \
> +    ((__lhxs))
> +#define APLIC_xMSICFGADDR_PPN_LHX(__lhxw, __lhxs) \
> +    (APLIC_xMSICFGADDR_PPN_LHX_MASK(__lhxw) << \
> +     APLIC_xMSICFGADDR_PPN_LHX_SHIFT(__lhxs))
> +
> +#define APLIC_xMSICFGADDR_PPN_HHX_MASK(__hhxw) \
> +    ((1UL << (__hhxw)) - 1)
> +#define APLIC_xMSICFGADDR_PPN_HHX_SHIFT(__hhxs) \
> +    ((__hhxs) + APLIC_xMSICFGADDR_PPN_SHIFT)
> +#define APLIC_xMSICFGADDR_PPN_HHX(__hhxw, __hhxs) \
> +    (APLIC_xMSICFGADDR_PPN_HHX_MASK(__hhxw) << \
> +     APLIC_xMSICFGADDR_PPN_HHX_SHIFT(__hhxs))
> +
> +#define APLIC_xMSICFGADDRH_VALID_MASK   \
> +    (APLIC_xMSICFGADDRH_L | \
> +     (APLIC_xMSICFGADDRH_HHXS_MASK << APLIC_xMSICFGADDRH_HHXS_SHIFT) | \
> +     (APLIC_xMSICFGADDRH_LHXS_MASK << APLIC_xMSICFGADDRH_LHXS_SHIFT) | \
> +     (APLIC_xMSICFGADDRH_HHXW_MASK << APLIC_xMSICFGADDRH_HHXW_SHIFT) | \
> +     (APLIC_xMSICFGADDRH_LHXW_MASK << APLIC_xMSICFGADDRH_LHXW_SHIFT) | \
> +     APLIC_xMSICFGADDRH_BAPPN_MASK)
> +
> +#define APLIC_SETIP_BASE               0x1c00
> +#define APLIC_SETIPNUM                 0x1cdc
> +
> +#define APLIC_CLRIP_BASE               0x1d00
> +#define APLIC_CLRIPNUM                 0x1ddc
> +
> +#define APLIC_SETIE_BASE               0x1e00
> +#define APLIC_SETIENUM                 0x1edc
> +
> +#define APLIC_CLRIE_BASE               0x1f00
> +#define APLIC_CLRIENUM                 0x1fdc
> +
> +#define APLIC_SETIPNUM_LE              0x2000
> +#define APLIC_SETIPNUM_BE              0x2004
> +
> +#define APLIC_ISTATE_PENDING           (1U << 0)
> +#define APLIC_ISTATE_ENABLED           (1U << 1)
> +#define APLIC_ISTATE_ENPEND            (APLIC_ISTATE_ENABLED | \
> +                                        APLIC_ISTATE_PENDING)
> +#define APLIC_ISTATE_INPUT             (1U << 8)
> +
> +#define APLIC_GENMSI                   0x3000
> +
> +#define APLIC_TARGET_BASE              0x3004
> +#define APLIC_TARGET_HART_IDX_SHIFT    18
> +#define APLIC_TARGET_HART_IDX_MASK     0x3fff
> +#define APLIC_TARGET_GUEST_IDX_SHIFT   12
> +#define APLIC_TARGET_GUEST_IDX_MASK    0x3f
> +#define APLIC_TARGET_IPRIO_MASK        0xff
> +#define APLIC_TARGET_EIID_MASK         0x7ff
> +
> +#define APLIC_IDC_BASE                 0x4000
> +#define APLIC_IDC_SIZE                 32
> +
> +#define APLIC_IDC_IDELIVERY            0x00
> +
> +#define APLIC_IDC_IFORCE               0x04
> +
> +#define APLIC_IDC_ITHRESHOLD           0x08
> +
> +#define APLIC_IDC_TOPI                 0x18
> +#define APLIC_IDC_TOPI_ID_SHIFT        16
> +#define APLIC_IDC_TOPI_ID_MASK         0x3ff
> +#define APLIC_IDC_TOPI_PRIO_MASK       0xff
> +
> +#define APLIC_IDC_CLAIMI               0x1c
> +
> +static uint32_t riscv_aplic_read_input_word(RISCVAPLICState *aplic,
> +                                            uint32_t word)
> +{
> +    uint32_t i, irq, ret = 0;
> +
> +    for (i = 0; i < 32; i++) {
> +        irq = word * 32 + i;
> +        if (!irq || aplic->num_irqs <= irq) {
> +            continue;
> +        }
> +
> +        ret |= ((aplic->state[irq] & APLIC_ISTATE_INPUT) ? 1 : 0) << i;
> +    }
> +
> +    return ret;
> +}
> +
> +static uint32_t riscv_aplic_read_pending_word(RISCVAPLICState *aplic,
> +                                              uint32_t word)
> +{
> +    uint32_t i, irq, ret = 0;
> +
> +    for (i = 0; i < 32; i++) {
> +        irq = word * 32 + i;
> +        if (!irq || aplic->num_irqs <= irq) {
> +            continue;
> +        }
> +
> +        ret |= ((aplic->state[irq] & APLIC_ISTATE_PENDING) ? 1 : 0) << i;
> +    }
> +
> +    return ret;
> +}
> +
> +static void riscv_aplic_set_pending_raw(RISCVAPLICState *aplic,
> +                                        uint32_t irq, bool pending)
> +{
> +    if (pending) {
> +        aplic->state[irq] |= APLIC_ISTATE_PENDING;
> +    } else {
> +        aplic->state[irq] &= ~APLIC_ISTATE_PENDING;
> +    }
> +}
> +
> +static void riscv_aplic_set_pending(RISCVAPLICState *aplic,
> +                                    uint32_t irq, bool pending)
> +{
> +    uint32_t sourcecfg, sm;
> +
> +    if ((irq <= 0) || (aplic->num_irqs <= irq)) {
> +        return;
> +    }
> +
> +    sourcecfg = aplic->sourcecfg[irq];
> +    if (sourcecfg & APLIC_SOURCECFG_D) {
> +        return;
> +    }
> +
> +    sm = sourcecfg & APLIC_SOURCECFG_SM_MASK;
> +    if ((sm == APLIC_SOURCECFG_SM_INACTIVE) ||
> +        ((!aplic->msimode || (aplic->msimode && !pending)) &&
> +         ((sm == APLIC_SOURCECFG_SM_LEVEL_HIGH) ||
> +          (sm == APLIC_SOURCECFG_SM_LEVEL_LOW)))) {
> +        return;
> +    }
> +
> +    riscv_aplic_set_pending_raw(aplic, irq, pending);
> +}
> +
> +static void riscv_aplic_set_pending_word(RISCVAPLICState *aplic,
> +                                         uint32_t word, uint32_t value,
> +                                         bool pending)
> +{
> +    uint32_t i, irq;
> +
> +    for (i = 0; i < 32; i++) {
> +        irq = word * 32 + i;
> +        if (!irq || aplic->num_irqs <= irq) {
> +            continue;
> +        }
> +
> +        if (value & (1U << i)) {
> +            riscv_aplic_set_pending(aplic, irq, pending);
> +        }
> +    }
> +}
> +
> +static uint32_t riscv_aplic_read_enabled_word(RISCVAPLICState *aplic,
> +                                              int word)
> +{
> +    uint32_t i, irq, ret = 0;
> +
> +    for (i = 0; i < 32; i++) {
> +        irq = word * 32 + i;
> +        if (!irq || aplic->num_irqs <= irq) {
> +            continue;
> +        }
> +
> +        ret |= ((aplic->state[irq] & APLIC_ISTATE_ENABLED) ? 1 : 0) << i;
> +    }
> +
> +    return ret;
> +}
> +
> +static void riscv_aplic_set_enabled_raw(RISCVAPLICState *aplic,
> +                                        uint32_t irq, bool enabled)
> +{
> +    if (enabled) {
> +        aplic->state[irq] |= APLIC_ISTATE_ENABLED;
> +    } else {
> +        aplic->state[irq] &= ~APLIC_ISTATE_ENABLED;
> +    }
> +}
> +
> +static void riscv_aplic_set_enabled(RISCVAPLICState *aplic,
> +                                    uint32_t irq, bool enabled)
> +{
> +    uint32_t sourcecfg, sm;
> +
> +    if ((irq <= 0) || (aplic->num_irqs <= irq)) {
> +        return;
> +    }
> +
> +    sourcecfg = aplic->sourcecfg[irq];
> +    if (sourcecfg & APLIC_SOURCECFG_D) {
> +        return;
> +    }
> +
> +    sm = sourcecfg & APLIC_SOURCECFG_SM_MASK;
> +    if (sm == APLIC_SOURCECFG_SM_INACTIVE) {
> +        return;
> +    }
> +
> +    riscv_aplic_set_enabled_raw(aplic, irq, enabled);
> +}
> +
> +static void riscv_aplic_set_enabled_word(RISCVAPLICState *aplic,
> +                                         uint32_t word, uint32_t value,
> +                                         bool enabled)
> +{
> +    uint32_t i, irq;
> +
> +    for (i = 0; i < 32; i++) {
> +        irq = word * 32 + i;
> +        if (!irq || aplic->num_irqs <= irq) {
> +            continue;
> +        }
> +
> +        if (value & (1U << i)) {
> +            riscv_aplic_set_enabled(aplic, irq, enabled);
> +        }
> +    }
> +}
> +
> +static void riscv_aplic_msi_send(RISCVAPLICState *aplic,
> +                                 uint32_t hart_idx, uint32_t guest_idx,
> +                                 uint32_t eiid)
> +{
> +    uint64_t addr;
> +    MemTxResult result;
> +    RISCVAPLICState *aplic_m;
> +    uint32_t lhxs, lhxw, hhxs, hhxw, group_idx, msicfgaddr, msicfgaddrH;
> +
> +    aplic_m = aplic;
> +    while (aplic_m && !aplic_m->mmode) {
> +        aplic_m = aplic_m->parent;
> +    }
> +    if (!aplic_m) {
> +        qemu_log_mask(LOG_GUEST_ERROR, "%s: m-level APLIC not found\n",
> +                      __func__);
> +        return;
> +    }
> +
> +    if (aplic->mmode) {
> +        msicfgaddr = aplic_m->mmsicfgaddr;
> +        msicfgaddrH = aplic_m->mmsicfgaddrH;
> +    } else {
> +        msicfgaddr = aplic_m->smsicfgaddr;
> +        msicfgaddrH = aplic_m->smsicfgaddrH;
> +    }
> +
> +    lhxs = (msicfgaddrH >> APLIC_xMSICFGADDRH_LHXS_SHIFT) &
> +            APLIC_xMSICFGADDRH_LHXS_MASK;
> +    lhxw = (msicfgaddrH >> APLIC_xMSICFGADDRH_LHXW_SHIFT) &
> +            APLIC_xMSICFGADDRH_LHXW_MASK;
> +    hhxs = (msicfgaddrH >> APLIC_xMSICFGADDRH_HHXS_SHIFT) &
> +            APLIC_xMSICFGADDRH_HHXS_MASK;
> +    hhxw = (msicfgaddrH >> APLIC_xMSICFGADDRH_HHXW_SHIFT) &
> +            APLIC_xMSICFGADDRH_HHXW_MASK;
> +
> +    group_idx = hart_idx >> lhxw;
> +    hart_idx &= APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw);
> +
> +    addr = msicfgaddr;
> +    addr |= ((uint64_t)(msicfgaddrH & APLIC_xMSICFGADDRH_BAPPN_MASK)) <<
> 32;
> +    addr |= ((uint64_t)(group_idx &
> APLIC_xMSICFGADDR_PPN_HHX_MASK(hhxw))) <<
> +             APLIC_xMSICFGADDR_PPN_HHX_SHIFT(hhxs);
> +    addr |= ((uint64_t)(hart_idx & APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw)))
> <<
> +             APLIC_xMSICFGADDR_PPN_LHX_SHIFT(lhxs);
> +    addr |= (uint64_t)(guest_idx & APLIC_xMSICFGADDR_PPN_HART(lhxs));
> +    addr <<= APLIC_xMSICFGADDR_PPN_SHIFT;
> +
> +    address_space_stl_le(&address_space_memory, addr,
> +                         eiid, MEMTXATTRS_UNSPECIFIED, &result);
> +    if (result != MEMTX_OK) {
> +        qemu_log_mask(LOG_GUEST_ERROR, "%s: MSI write failed for "
> +                      "hart_index=%d guest_index=%d eiid=%d\n",
> +                      __func__, hart_idx, guest_idx, eiid);
> +    }
> +}
> +
> +static void riscv_aplic_msi_irq_update(RISCVAPLICState *aplic, uint32_t
> irq)
> +{
> +    uint32_t hart_idx, guest_idx, eiid;
> +
> +    if (!aplic->msimode || (aplic->num_irqs <= irq) ||
> +        !(aplic->domaincfg & APLIC_DOMAINCFG_IE)) {
> +        return;
> +    }
> +
> +    if ((aplic->state[irq] & APLIC_ISTATE_ENPEND) != APLIC_ISTATE_ENPEND)
> {
> +        return;
> +    }
> +
> +    riscv_aplic_set_pending_raw(aplic, irq, false);
> +
> +    hart_idx = aplic->target[irq] >> APLIC_TARGET_HART_IDX_SHIFT;
> +    hart_idx &= APLIC_TARGET_HART_IDX_MASK;
>

Hi Anup,

I'm wondering does the hart_idx here has to use Machine-level hart index
for Supervisor-level interrupt domain?

According to AIA spec for supervisor-level domin (Section 4.8.1):
  For a supervisor-level domain, "if" MSI target addresses are determined
by the parent domain’s
  smsiaddrcfg and smsiaddrcfgh registers, then to construct the address for
an outgoing MSI for
  interrupt source i, the Hart Index from register target[i] must first be
converted into the index
  that the machine-level parent domain uses for the same hart. (Typically,
these numbers are the
  same, but they may not be.) The address for the MSI is then computed
using this "machine-level"
  hart index, the parent domain’s smsiaddrcfg and smsiaddrcfgh, and the
Guest Index value from
  target[i].

The description of converting target[i] hart Index to machine-level hart
index in the spec
is quite vague to me.

Regards,
Frank Chang


> +    if (aplic->mmode) {
> +        /* M-level APLIC ignores guest_index */
> +        guest_idx = 0;
> +    } else {
> +        guest_idx = aplic->target[irq] >> APLIC_TARGET_GUEST_IDX_SHIFT;
> +        guest_idx &= APLIC_TARGET_GUEST_IDX_MASK;
> +    }
> +    eiid = aplic->target[irq] & APLIC_TARGET_EIID_MASK;
> +    riscv_aplic_msi_send(aplic, hart_idx, guest_idx, eiid);
> +}
> +
> +static uint32_t riscv_aplic_idc_topi(RISCVAPLICState *aplic, uint32_t idc)
> +{
> +    uint32_t best_irq, best_iprio;
> +    uint32_t irq, iprio, ihartidx, ithres;
> +
> +    if (aplic->num_harts <= idc) {
> +        return 0;
> +    }
> +
> +    ithres = aplic->ithreshold[idc];
> +    best_irq = best_iprio = UINT32_MAX;
> +    for (irq = 1; irq < aplic->num_irqs; irq++) {
> +        if ((aplic->state[irq] & APLIC_ISTATE_ENPEND) !=
> +            APLIC_ISTATE_ENPEND) {
> +            continue;
> +        }
> +
> +        ihartidx = aplic->target[irq] >> APLIC_TARGET_HART_IDX_SHIFT;
> +        ihartidx &= APLIC_TARGET_HART_IDX_MASK;
> +        if (ihartidx != idc) {
> +            continue;
> +        }
> +
> +        iprio = aplic->target[irq] & aplic->iprio_mask;
> +        if (ithres && iprio >= ithres) {
> +            continue;
> +        }
> +
> +        if (iprio < best_iprio) {
> +            best_irq = irq;
> +            best_iprio = iprio;
> +        }
> +    }
> +
> +    if (best_irq < aplic->num_irqs && best_iprio <= aplic->iprio_mask) {
> +        return (best_irq << APLIC_IDC_TOPI_ID_SHIFT) | best_iprio;
> +    }
> +
> +    return 0;
> +}
> +
> +static void riscv_aplic_idc_update(RISCVAPLICState *aplic, uint32_t idc)
> +{
> +    uint32_t topi;
> +
> +    if (aplic->msimode || aplic->num_harts <= idc) {
> +        return;
> +    }
> +
> +    topi = riscv_aplic_idc_topi(aplic, idc);
> +    if ((aplic->domaincfg & APLIC_DOMAINCFG_IE) &&
> +        aplic->idelivery[idc] &&
> +        (aplic->iforce[idc] || topi)) {
> +        qemu_irq_raise(aplic->external_irqs[idc]);
> +    } else {
> +        qemu_irq_lower(aplic->external_irqs[idc]);
> +    }
> +}
> +
> +static uint32_t riscv_aplic_idc_claimi(RISCVAPLICState *aplic, uint32_t
> idc)
> +{
> +    uint32_t irq, state, sm, topi = riscv_aplic_idc_topi(aplic, idc);
> +
> +    if (!topi) {
> +        aplic->iforce[idc] = 0;
> +        return 0;
> +    }
> +
> +    irq = (topi >> APLIC_IDC_TOPI_ID_SHIFT) & APLIC_IDC_TOPI_ID_MASK;
> +    sm = aplic->sourcecfg[irq] & APLIC_SOURCECFG_SM_MASK;
> +    state = aplic->state[irq];
> +    riscv_aplic_set_pending_raw(aplic, irq, false);
> +    if ((sm == APLIC_SOURCECFG_SM_LEVEL_HIGH) &&
> +        (state & APLIC_ISTATE_INPUT)) {
> +        riscv_aplic_set_pending_raw(aplic, irq, true);
> +    } else if ((sm == APLIC_SOURCECFG_SM_LEVEL_LOW) &&
> +               !(state & APLIC_ISTATE_INPUT)) {
> +        riscv_aplic_set_pending_raw(aplic, irq, true);
> +    }
> +    riscv_aplic_idc_update(aplic, idc);
> +
> +    return topi;
> +}
> +
> +static void riscv_aplic_request(void *opaque, int irq, int level)
> +{
> +    bool update = false;
> +    RISCVAPLICState *aplic = opaque;
> +    uint32_t sourcecfg, childidx, state, idc;
> +
> +    assert((0 < irq) && (irq < aplic->num_irqs));
> +
> +    sourcecfg = aplic->sourcecfg[irq];
> +    if (sourcecfg & APLIC_SOURCECFG_D) {
> +        childidx = sourcecfg & APLIC_SOURCECFG_CHILDIDX_MASK;
> +        if (childidx < aplic->num_children) {
> +            riscv_aplic_request(aplic->children[childidx], irq, level);
> +        }
> +        return;
> +    }
> +
> +    state = aplic->state[irq];
> +    switch (sourcecfg & APLIC_SOURCECFG_SM_MASK) {
> +    case APLIC_SOURCECFG_SM_EDGE_RISE:
> +        if ((level > 0) && !(state & APLIC_ISTATE_INPUT) &&
> +            !(state & APLIC_ISTATE_PENDING)) {
> +            riscv_aplic_set_pending_raw(aplic, irq, true);
> +            update = true;
> +        }
> +        break;
> +    case APLIC_SOURCECFG_SM_EDGE_FALL:
> +        if ((level <= 0) && (state & APLIC_ISTATE_INPUT) &&
> +            !(state & APLIC_ISTATE_PENDING)) {
> +            riscv_aplic_set_pending_raw(aplic, irq, true);
> +            update = true;
> +        }
> +        break;
> +    case APLIC_SOURCECFG_SM_LEVEL_HIGH:
> +        if ((level > 0) && !(state & APLIC_ISTATE_PENDING)) {
> +            riscv_aplic_set_pending_raw(aplic, irq, true);
> +            update = true;
> +        }
> +        break;
> +    case APLIC_SOURCECFG_SM_LEVEL_LOW:
> +        if ((level <= 0) && !(state & APLIC_ISTATE_PENDING)) {
> +            riscv_aplic_set_pending_raw(aplic, irq, true);
> +            update = true;
> +        }
> +        break;
> +    default:
> +        break;
> +    }
> +
> +    if (level <= 0) {
> +        aplic->state[irq] &= ~APLIC_ISTATE_INPUT;
> +    } else {
> +        aplic->state[irq] |= APLIC_ISTATE_INPUT;
> +    }
> +
> +    if (update) {
> +        if (aplic->msimode) {
> +            riscv_aplic_msi_irq_update(aplic, irq);
> +        } else {
> +            idc = aplic->target[irq] >> APLIC_TARGET_HART_IDX_SHIFT;
> +            idc &= APLIC_TARGET_HART_IDX_MASK;
> +            riscv_aplic_idc_update(aplic, idc);
> +        }
> +    }
> +}
> +
> +static uint64_t riscv_aplic_read(void *opaque, hwaddr addr, unsigned size)
> +{
> +    uint32_t irq, word, idc;
> +    RISCVAPLICState *aplic = opaque;
> +
> +    /* Reads must be 4 byte words */
> +    if ((addr & 0x3) != 0) {
> +        goto err;
> +    }
> +
> +    if (addr == APLIC_DOMAINCFG) {
> +        return APLIC_DOMAINCFG_RDONLY | aplic->domaincfg |
> +               (aplic->msimode ? APLIC_DOMAINCFG_DM : 0);
> +    } else if ((APLIC_SOURCECFG_BASE <= addr) &&
> +            (addr < (APLIC_SOURCECFG_BASE + (aplic->num_irqs - 1) * 4))) {
> +        irq  = ((addr - APLIC_SOURCECFG_BASE) >> 2) + 1;
> +        return aplic->sourcecfg[irq];
> +    } else if (aplic->mmode && aplic->msimode &&
> +               (addr == APLIC_MMSICFGADDR)) {
> +        return aplic->mmsicfgaddr;
> +    } else if (aplic->mmode && aplic->msimode &&
> +               (addr == APLIC_MMSICFGADDRH)) {
> +        return aplic->mmsicfgaddrH;
> +    } else if (aplic->mmode && aplic->msimode &&
> +               (addr == APLIC_SMSICFGADDR)) {
> +        /* Registers SMSICFGADDR and SMSICFGADDRH are implemented only if:
> +         * (a) the interrupt domain is at machine level
> +         * (b) the domain’s harts implement supervisor mode
> +         * (c) the domain has one or more child supervisor-level domains
> +         *     that support MSI delivery mode (domaincfg.DM is not read-
> +         *     only zero in at least one of the supervisor-level child
> +         * domains).
> +         */
> +        return (aplic->num_children) ? aplic->smsicfgaddr : 0;
> +    } else if (aplic->mmode && aplic->msimode &&
> +               (addr == APLIC_SMSICFGADDRH)) {
> +        return (aplic->num_children) ? aplic->smsicfgaddrH : 0;
> +    } else if ((APLIC_SETIP_BASE <= addr) &&
> +            (addr < (APLIC_SETIP_BASE + aplic->bitfield_words * 4))) {
> +        word = (addr - APLIC_SETIP_BASE) >> 2;
> +        return riscv_aplic_read_pending_word(aplic, word);
> +    } else if (addr == APLIC_SETIPNUM) {
> +        return 0;
> +    } else if ((APLIC_CLRIP_BASE <= addr) &&
> +            (addr < (APLIC_CLRIP_BASE + aplic->bitfield_words * 4))) {
> +        word = (addr - APLIC_CLRIP_BASE) >> 2;
> +        return riscv_aplic_read_input_word(aplic, word);
> +    } else if (addr == APLIC_CLRIPNUM) {
> +        return 0;
> +    } else if ((APLIC_SETIE_BASE <= addr) &&
> +            (addr < (APLIC_SETIE_BASE + aplic->bitfield_words * 4))) {
> +        word = (addr - APLIC_SETIE_BASE) >> 2;
> +        return riscv_aplic_read_enabled_word(aplic, word);
> +    } else if (addr == APLIC_SETIENUM) {
> +        return 0;
> +    } else if ((APLIC_CLRIE_BASE <= addr) &&
> +            (addr < (APLIC_CLRIE_BASE + aplic->bitfield_words * 4))) {
> +        return 0;
> +    } else if (addr == APLIC_CLRIENUM) {
> +        return 0;
> +    } else if (addr == APLIC_SETIPNUM_LE) {
> +        return 0;
> +    } else if (addr == APLIC_SETIPNUM_BE) {
> +        return 0;
> +    } else if (addr == APLIC_GENMSI) {
> +        return (aplic->msimode) ? aplic->genmsi : 0;
> +    } else if ((APLIC_TARGET_BASE <= addr) &&
> +            (addr < (APLIC_TARGET_BASE + (aplic->num_irqs - 1) * 4))) {
> +        irq = ((addr - APLIC_TARGET_BASE) >> 2) + 1;
> +        return aplic->target[irq];
> +    } else if (!aplic->msimode && (APLIC_IDC_BASE <= addr) &&
> +            (addr < (APLIC_IDC_BASE + aplic->num_harts *
> APLIC_IDC_SIZE))) {
> +        idc = (addr - APLIC_IDC_BASE) / APLIC_IDC_SIZE;
> +        switch (addr - (APLIC_IDC_BASE + idc * APLIC_IDC_SIZE)) {
> +        case APLIC_IDC_IDELIVERY:
> +            return aplic->idelivery[idc];
> +        case APLIC_IDC_IFORCE:
> +            return aplic->iforce[idc];
> +        case APLIC_IDC_ITHRESHOLD:
> +            return aplic->ithreshold[idc];
> +        case APLIC_IDC_TOPI:
> +            return riscv_aplic_idc_topi(aplic, idc);
> +        case APLIC_IDC_CLAIMI:
> +            return riscv_aplic_idc_claimi(aplic, idc);
> +        default:
> +            goto err;
> +        };
> +    }
> +
> +err:
> +    qemu_log_mask(LOG_GUEST_ERROR,
> +                  "%s: Invalid register read 0x%" HWADDR_PRIx "\n",
> +                  __func__, addr);
> +    return 0;
> +}
> +
> +static void riscv_aplic_write(void *opaque, hwaddr addr, uint64_t value,
> +        unsigned size)
> +{
> +    RISCVAPLICState *aplic = opaque;
> +    uint32_t irq, word, idc = UINT32_MAX;
> +
> +    /* Writes must be 4 byte words */
> +    if ((addr & 0x3) != 0) {
> +        goto err;
> +    }
> +
> +    if (addr == APLIC_DOMAINCFG) {
> +        /* Only IE bit writeable at the moment */
> +        value &= APLIC_DOMAINCFG_IE;
> +        aplic->domaincfg = value;
> +    } else if ((APLIC_SOURCECFG_BASE <= addr) &&
> +            (addr < (APLIC_SOURCECFG_BASE + (aplic->num_irqs - 1) * 4))) {
> +        irq  = ((addr - APLIC_SOURCECFG_BASE) >> 2) + 1;
> +        if (!aplic->num_children && (value & APLIC_SOURCECFG_D)) {
> +            value = 0;
> +        }
> +        if (value & APLIC_SOURCECFG_D) {
> +            value &= (APLIC_SOURCECFG_D | APLIC_SOURCECFG_CHILDIDX_MASK);
> +        } else {
> +            value &= (APLIC_SOURCECFG_D | APLIC_SOURCECFG_SM_MASK);
> +        }
> +        aplic->sourcecfg[irq] = value;
> +        if ((aplic->sourcecfg[irq] & APLIC_SOURCECFG_D) ||
> +            (aplic->sourcecfg[irq] == 0)) {
> +            riscv_aplic_set_pending_raw(aplic, irq, false);
> +            riscv_aplic_set_enabled_raw(aplic, irq, false);
> +        }
> +    } else if (aplic->mmode && aplic->msimode &&
> +               (addr == APLIC_MMSICFGADDR)) {
> +        if (!(aplic->mmsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
> +            aplic->mmsicfgaddr = value;
> +        }
> +    } else if (aplic->mmode && aplic->msimode &&
> +               (addr == APLIC_MMSICFGADDRH)) {
> +        if (!(aplic->mmsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
> +            aplic->mmsicfgaddrH = value & APLIC_xMSICFGADDRH_VALID_MASK;
> +        }
> +    } else if (aplic->mmode && aplic->msimode &&
> +               (addr == APLIC_SMSICFGADDR)) {
> +        /* Registers SMSICFGADDR and SMSICFGADDRH are implemented only if:
> +         * (a) the interrupt domain is at machine level
> +         * (b) the domain’s harts implement supervisor mode
> +         * (c) the domain has one or more child supervisor-level domains
> +         *     that support MSI delivery mode (domaincfg.DM is not read-
> +         *     only zero in at least one of the supervisor-level child
> +         * domains).
> +         */
> +        if (aplic->num_children &&
> +            !(aplic->smsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
> +            aplic->smsicfgaddr = value;
> +        }
> +    } else if (aplic->mmode && aplic->msimode &&
> +               (addr == APLIC_SMSICFGADDRH)) {
> +        if (aplic->num_children &&
> +            !(aplic->smsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
> +            aplic->smsicfgaddrH = value & APLIC_xMSICFGADDRH_VALID_MASK;
> +        }
> +    } else if ((APLIC_SETIP_BASE <= addr) &&
> +            (addr < (APLIC_SETIP_BASE + aplic->bitfield_words * 4))) {
> +        word = (addr - APLIC_SETIP_BASE) >> 2;
> +        riscv_aplic_set_pending_word(aplic, word, value, true);
> +    } else if (addr == APLIC_SETIPNUM) {
> +        riscv_aplic_set_pending(aplic, value, true);
> +    } else if ((APLIC_CLRIP_BASE <= addr) &&
> +            (addr < (APLIC_CLRIP_BASE + aplic->bitfield_words * 4))) {
> +        word = (addr - APLIC_CLRIP_BASE) >> 2;
> +        riscv_aplic_set_pending_word(aplic, word, value, false);
> +    } else if (addr == APLIC_CLRIPNUM) {
> +        riscv_aplic_set_pending(aplic, value, false);
> +    } else if ((APLIC_SETIE_BASE <= addr) &&
> +            (addr < (APLIC_SETIE_BASE + aplic->bitfield_words * 4))) {
> +        word = (addr - APLIC_SETIE_BASE) >> 2;
> +        riscv_aplic_set_enabled_word(aplic, word, value, true);
> +    } else if (addr == APLIC_SETIENUM) {
> +        riscv_aplic_set_enabled(aplic, value, true);
> +    } else if ((APLIC_CLRIE_BASE <= addr) &&
> +            (addr < (APLIC_CLRIE_BASE + aplic->bitfield_words * 4))) {
> +        word = (addr - APLIC_CLRIE_BASE) >> 2;
> +        riscv_aplic_set_enabled_word(aplic, word, value, false);
> +    } else if (addr == APLIC_CLRIENUM) {
> +        riscv_aplic_set_enabled(aplic, value, false);
> +    } else if (addr == APLIC_SETIPNUM_LE) {
> +        riscv_aplic_set_pending(aplic, value, true);
> +    } else if (addr == APLIC_SETIPNUM_BE) {
> +        riscv_aplic_set_pending(aplic, bswap32(value), true);
> +    } else if (addr == APLIC_GENMSI) {
> +        if (aplic->msimode) {
> +            aplic->genmsi = value & ~(APLIC_TARGET_GUEST_IDX_MASK <<
> +                                      APLIC_TARGET_GUEST_IDX_SHIFT);
> +            riscv_aplic_msi_send(aplic,
> +                                 value >> APLIC_TARGET_HART_IDX_SHIFT,
> +                                 0,
> +                                 value & APLIC_TARGET_EIID_MASK);
> +        }
> +    } else if ((APLIC_TARGET_BASE <= addr) &&
> +            (addr < (APLIC_TARGET_BASE + (aplic->num_irqs - 1) * 4))) {
> +        irq = ((addr - APLIC_TARGET_BASE) >> 2) + 1;
> +        if (aplic->msimode) {
> +            aplic->target[irq] = value;
> +        } else {
> +            aplic->target[irq] = (value & ~APLIC_TARGET_IPRIO_MASK) |
> +                                 ((value & aplic->iprio_mask) ?
> +                                  (value & aplic->iprio_mask) : 1);
> +        }
> +    } else if (!aplic->msimode && (APLIC_IDC_BASE <= addr) &&
> +            (addr < (APLIC_IDC_BASE + aplic->num_harts *
> APLIC_IDC_SIZE))) {
> +        idc = (addr - APLIC_IDC_BASE) / APLIC_IDC_SIZE;
> +        switch (addr - (APLIC_IDC_BASE + idc * APLIC_IDC_SIZE)) {
> +        case APLIC_IDC_IDELIVERY:
> +            aplic->idelivery[idc] = value & 0x1;
> +            break;
> +        case APLIC_IDC_IFORCE:
> +            aplic->iforce[idc] = value & 0x1;
> +            break;
> +        case APLIC_IDC_ITHRESHOLD:
> +            aplic->ithreshold[idc] = value & aplic->iprio_mask;
> +            break;
> +        default:
> +            goto err;
> +        };
> +    } else {
> +        goto err;
> +    }
> +
> +    if (aplic->msimode) {
> +        for (irq = 1; irq < aplic->num_irqs; irq++) {
> +            riscv_aplic_msi_irq_update(aplic, irq);
> +        }
> +    } else {
> +        if (idc == UINT32_MAX) {
> +            for (idc = 0; idc < aplic->num_harts; idc++) {
> +                riscv_aplic_idc_update(aplic, idc);
> +            }
> +        } else {
> +            riscv_aplic_idc_update(aplic, idc);
> +        }
> +    }
> +
> +    return;
> +
> +err:
> +    qemu_log_mask(LOG_GUEST_ERROR,
> +                  "%s: Invalid register write 0x%" HWADDR_PRIx "\n",
> +                  __func__, addr);
> +}
> +
> +static const MemoryRegionOps riscv_aplic_ops = {
> +    .read = riscv_aplic_read,
> +    .write = riscv_aplic_write,
> +    .endianness = DEVICE_LITTLE_ENDIAN,
> +    .valid = {
> +        .min_access_size = 4,
> +        .max_access_size = 4
> +    }
> +};
> +
> +static void riscv_aplic_realize(DeviceState *dev, Error **errp)
> +{
> +    uint32_t i;
> +    RISCVAPLICState *aplic = RISCV_APLIC(dev);
> +
> +    aplic->bitfield_words = (aplic->num_irqs + 31) >> 5;
> +    aplic->sourcecfg = g_new0(uint32_t, aplic->num_irqs);
> +    aplic->state = g_new(uint32_t, aplic->num_irqs);
> +    aplic->target = g_new0(uint32_t, aplic->num_irqs);
> +    if (!aplic->msimode) {
> +        for (i = 0; i < aplic->num_irqs; i++) {
> +            aplic->target[i] = 1;
> +        }
> +    }
> +    aplic->idelivery = g_new0(uint32_t, aplic->num_harts);
> +    aplic->iforce = g_new0(uint32_t, aplic->num_harts);
> +    aplic->ithreshold = g_new0(uint32_t, aplic->num_harts);
> +
> +    memory_region_init_io(&aplic->mmio, OBJECT(dev), &riscv_aplic_ops,
> aplic,
> +                          TYPE_RISCV_APLIC, aplic->aperture_size);
> +    sysbus_init_mmio(SYS_BUS_DEVICE(dev), &aplic->mmio);
> +
> +    /* Only root APLICs have hardware IRQ lines. All non-root APLICs
> +     * have IRQ lines delegated by their parent APLIC.
> +     */
> +    if (!aplic->parent) {
> +        qdev_init_gpio_in(dev, riscv_aplic_request, aplic->num_irqs);
> +    }
> +
> +    /* Create output IRQ lines for non-MSI mode */
> +    if (!aplic->msimode) {
> +        aplic->external_irqs = g_malloc(sizeof(qemu_irq) *
> aplic->num_harts);
> +        qdev_init_gpio_out(dev, aplic->external_irqs, aplic->num_harts);
> +
> +        /* Claim the CPU interrupt to be triggered by this APLIC */
> +        for (i = 0; i < aplic->num_harts; i++) {
> +            RISCVCPU *cpu = RISCV_CPU(qemu_get_cpu(aplic->hartid_base +
> i));
> +            if (riscv_cpu_claim_interrupts(cpu,
> +                (aplic->mmode) ? MIP_MEIP : MIP_SEIP) < 0) {
> +                error_report("%s already claimed",
> +                             (aplic->mmode) ? "MEIP" : "SEIP");
> +                exit(1);
> +            }
> +        }
> +    }
> +
> +    msi_nonbroken = true;
> +}
> +
> +static Property riscv_aplic_properties[] = {
> +    DEFINE_PROP_UINT32("aperture-size", RISCVAPLICState, aperture_size,
> 0),
> +    DEFINE_PROP_UINT32("hartid-base", RISCVAPLICState, hartid_base, 0),
> +    DEFINE_PROP_UINT32("num-harts", RISCVAPLICState, num_harts, 0),
> +    DEFINE_PROP_UINT32("iprio-mask", RISCVAPLICState, iprio_mask, 0),
> +    DEFINE_PROP_UINT32("num-irqs", RISCVAPLICState, num_irqs, 0),
> +    DEFINE_PROP_BOOL("msimode", RISCVAPLICState, msimode, 0),
> +    DEFINE_PROP_BOOL("mmode", RISCVAPLICState, mmode, 0),
> +    DEFINE_PROP_END_OF_LIST(),
> +};
> +
> +static const VMStateDescription vmstate_riscv_aplic = {
> +    .name = "riscv_aplic",
> +    .version_id = 1,
> +    .minimum_version_id = 1,
> +    .fields = (VMStateField[]) {
> +            VMSTATE_UINT32(domaincfg, RISCVAPLICState),
> +            VMSTATE_UINT32(mmsicfgaddr, RISCVAPLICState),
> +            VMSTATE_UINT32(mmsicfgaddrH, RISCVAPLICState),
> +            VMSTATE_UINT32(smsicfgaddr, RISCVAPLICState),
> +            VMSTATE_UINT32(smsicfgaddrH, RISCVAPLICState),
> +            VMSTATE_UINT32(genmsi, RISCVAPLICState),
> +            VMSTATE_VARRAY_UINT32(sourcecfg, RISCVAPLICState,
> +                                  num_irqs, 0,
> +                                  vmstate_info_uint32, uint32_t),
> +            VMSTATE_VARRAY_UINT32(state, RISCVAPLICState,
> +                                  num_irqs, 0,
> +                                  vmstate_info_uint32, uint32_t),
> +            VMSTATE_VARRAY_UINT32(target, RISCVAPLICState,
> +                                  num_irqs, 0,
> +                                  vmstate_info_uint32, uint32_t),
> +            VMSTATE_VARRAY_UINT32(idelivery, RISCVAPLICState,
> +                                  num_harts, 0,
> +                                  vmstate_info_uint32, uint32_t),
> +            VMSTATE_VARRAY_UINT32(iforce, RISCVAPLICState,
> +                                  num_harts, 0,
> +                                  vmstate_info_uint32, uint32_t),
> +            VMSTATE_VARRAY_UINT32(ithreshold, RISCVAPLICState,
> +                                  num_harts, 0,
> +                                  vmstate_info_uint32, uint32_t),
> +            VMSTATE_END_OF_LIST()
> +        }
> +};
> +
> +static void riscv_aplic_class_init(ObjectClass *klass, void *data)
> +{
> +    DeviceClass *dc = DEVICE_CLASS(klass);
> +
> +    device_class_set_props(dc, riscv_aplic_properties);
> +    dc->realize = riscv_aplic_realize;
> +    dc->vmsd = &vmstate_riscv_aplic;
> +}
> +
> +static const TypeInfo riscv_aplic_info = {
> +    .name          = TYPE_RISCV_APLIC,
> +    .parent        = TYPE_SYS_BUS_DEVICE,
> +    .instance_size = sizeof(RISCVAPLICState),
> +    .class_init    = riscv_aplic_class_init,
> +};
> +
> +static void riscv_aplic_register_types(void)
> +{
> +    type_register_static(&riscv_aplic_info);
> +}
> +
> +type_init(riscv_aplic_register_types)
> +
> +/*
> + * Add a APLIC device to another APLIC device as child for
> + * interrupt delegation.
> + */
> +void riscv_aplic_add_child(DeviceState *parent, DeviceState *child)
> +{
> +    RISCVAPLICState *caplic, *paplic;
> +
> +    assert(parent && child);
> +    caplic = RISCV_APLIC(child);
> +    paplic = RISCV_APLIC(parent);
> +
> +    assert(paplic->num_irqs == caplic->num_irqs);
> +    assert(paplic->num_children <= QEMU_APLIC_MAX_CHILDREN);
> +
> +    caplic->parent = paplic;
> +    paplic->children[paplic->num_children] = caplic;
> +    paplic->num_children++;
> +}
> +
> +/*
> + * Create APLIC device.
> + */
> +DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size,
> +    uint32_t hartid_base, uint32_t num_harts, uint32_t num_sources,
> +    uint32_t iprio_bits, bool msimode, bool mmode, DeviceState *parent)
> +{
> +    DeviceState *dev = qdev_new(TYPE_RISCV_APLIC);
> +    uint32_t i;
> +
> +    assert(num_harts < APLIC_MAX_IDC);
> +    assert((APLIC_IDC_BASE + (num_harts * APLIC_IDC_SIZE)) <= size);
> +    assert(num_sources < APLIC_MAX_SOURCE);
> +    assert(APLIC_MIN_IPRIO_BITS <= iprio_bits);
> +    assert(iprio_bits <= APLIC_MAX_IPRIO_BITS);
> +
> +    qdev_prop_set_uint32(dev, "aperture-size", size);
> +    qdev_prop_set_uint32(dev, "hartid-base", hartid_base);
> +    qdev_prop_set_uint32(dev, "num-harts", num_harts);
> +    qdev_prop_set_uint32(dev, "iprio-mask", ((1U << iprio_bits) - 1));
> +    qdev_prop_set_uint32(dev, "num-irqs", num_sources + 1);
> +    qdev_prop_set_bit(dev, "msimode", msimode);
> +    qdev_prop_set_bit(dev, "mmode", mmode);
> +
> +    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
> +    sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
> +
> +    if (parent) {
> +        riscv_aplic_add_child(parent, dev);
> +    }
> +
> +    if (!msimode) {
> +        for (i = 0; i < num_harts; i++) {
> +            CPUState *cpu = qemu_get_cpu(hartid_base + i);
> +
> +            qdev_connect_gpio_out_named(dev, NULL, i,
> +                                        qdev_get_gpio_in(DEVICE(cpu),
> +                                            (mmode) ? IRQ_M_EXT :
> IRQ_S_EXT));
> +        }
> +    }
> +
> +    return dev;
> +}
> diff --git a/include/hw/intc/riscv_aplic.h b/include/hw/intc/riscv_aplic.h
> new file mode 100644
> index 0000000000..de8532fbc3
> --- /dev/null
> +++ b/include/hw/intc/riscv_aplic.h
> @@ -0,0 +1,79 @@
> +/*
> + * RISC-V APLIC (Advanced Platform Level Interrupt Controller) interface
> + *
> + * Copyright (c) 2021 Western Digital Corporation or its affiliates.
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms and conditions of the GNU General Public License,
> + * version 2 or later, as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope it will be useful, but WITHOUT
> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
> for
> + * more details.
> + *
> + * You should have received a copy of the GNU General Public License
> along with
> + * this program.  If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#ifndef HW_RISCV_APLIC_H
> +#define HW_RISCV_APLIC_H
> +
> +#include "hw/sysbus.h"
> +#include "qom/object.h"
> +
> +#define TYPE_RISCV_APLIC "riscv.aplic"
> +
> +typedef struct RISCVAPLICState RISCVAPLICState;
> +DECLARE_INSTANCE_CHECKER(RISCVAPLICState, RISCV_APLIC, TYPE_RISCV_APLIC)
> +
> +#define APLIC_MIN_SIZE            0x4000
> +#define APLIC_SIZE_ALIGN(__x)     (((__x) + (APLIC_MIN_SIZE - 1)) & \
> +                                   ~(APLIC_MIN_SIZE - 1))
> +#define APLIC_SIZE(__num_harts)   (APLIC_MIN_SIZE + \
> +                                   APLIC_SIZE_ALIGN(32 * (__num_harts)))
> +
> +struct RISCVAPLICState {
> +    /*< private >*/
> +    SysBusDevice parent_obj;
> +    qemu_irq *external_irqs;
> +
> +    /*< public >*/
> +    MemoryRegion mmio;
> +    uint32_t bitfield_words;
> +    uint32_t domaincfg;
> +    uint32_t mmsicfgaddr;
> +    uint32_t mmsicfgaddrH;
> +    uint32_t smsicfgaddr;
> +    uint32_t smsicfgaddrH;
> +    uint32_t genmsi;
> +    uint32_t *sourcecfg;
> +    uint32_t *state;
> +    uint32_t *target;
> +    uint32_t *idelivery;
> +    uint32_t *iforce;
> +    uint32_t *ithreshold;
> +
> +    /* topology */
> +#define QEMU_APLIC_MAX_CHILDREN        16
> +    struct RISCVAPLICState *parent;
> +    struct RISCVAPLICState *children[QEMU_APLIC_MAX_CHILDREN];
> +    uint16_t num_children;
> +
> +    /* config */
> +    uint32_t aperture_size;
> +    uint32_t hartid_base;
> +    uint32_t num_harts;
> +    uint32_t iprio_mask;
> +    uint32_t num_irqs;
> +    bool msimode;
> +    bool mmode;
> +};
> +
> +void riscv_aplic_add_child(DeviceState *parent, DeviceState *child);
> +
> +DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size,
> +    uint32_t hartid_base, uint32_t num_harts, uint32_t num_sources,
> +    uint32_t iprio_bits, bool msimode, bool mmode, DeviceState *parent);
> +
> +#endif
> --
> 2.25.1
>
>

Anup Patel Jan. 19, 2022, 4:20 p.m. UTC | #2

Hi Frank,

On Wed, Jan 19, 2022 at 9:07 PM Frank Chang <[e-mail]> wrote:
>
> On Wed, Jan 19, 2022 at 11:27 PM Anup Patel <[e-mail]> wrote:
>>
>> From: Anup Patel <[e-mail]>
>>
>> The RISC-V AIA (Advanced Interrupt Architecture) defines a new
>> interrupt controller for wired interrupts called APLIC (Advanced
>> Platform Level Interrupt Controller). The APLIC is capabable of
>> forwarding wired interupts to RISC-V HARTs directly or as MSIs
>> (Message Signaled Interupts).
>>
>> This patch adds device emulation for RISC-V AIA APLIC.
>>
>> Signed-off-by: Anup Patel <[e-mail]>
>> Signed-off-by: Anup Patel <[e-mail]>
>> Reviewed-by: Frank Chang <[e-mail]>
>> ---
>>  hw/intc/Kconfig               |   3 +
>>  hw/intc/meson.build           |   1 +
>>  hw/intc/riscv_aplic.c         | 975 ++++++++++++++++++++++++++++++++++
>>  include/hw/intc/riscv_aplic.h |  79 +++
>>  4 files changed, 1058 insertions(+)
>>  create mode 100644 hw/intc/riscv_aplic.c
>>  create mode 100644 include/hw/intc/riscv_aplic.h
>>
>> diff --git a/hw/intc/Kconfig b/hw/intc/Kconfig
>> index 010ded7eae..528e77b4a6 100644
>> --- a/hw/intc/Kconfig
>> +++ b/hw/intc/Kconfig
>> @@ -70,6 +70,9 @@ config LOONGSON_LIOINTC
>>  config RISCV_ACLINT
>>      bool
>>
>> +config RISCV_APLIC
>> +    bool
>> +
>>  config SIFIVE_PLIC
>>      bool
>>
>> diff --git a/hw/intc/meson.build b/hw/intc/meson.build
>> index 70080bc161..7466024402 100644
>> --- a/hw/intc/meson.build
>> +++ b/hw/intc/meson.build
>> @@ -50,6 +50,7 @@ specific_ss.add(when: 'CONFIG_S390_FLIC', if_true: files('s390_flic.c'))
>>  specific_ss.add(when: 'CONFIG_S390_FLIC_KVM', if_true: files('s390_flic_kvm.c'))
>>  specific_ss.add(when: 'CONFIG_SH_INTC', if_true: files('sh_intc.c'))
>>  specific_ss.add(when: 'CONFIG_RISCV_ACLINT', if_true: files('riscv_aclint.c'))
>> +specific_ss.add(when: 'CONFIG_RISCV_APLIC', if_true: files('riscv_aplic.c'))
>>  specific_ss.add(when: 'CONFIG_SIFIVE_PLIC', if_true: files('sifive_plic.c'))
>>  specific_ss.add(when: 'CONFIG_XICS', if_true: files('xics.c'))
>>  specific_ss.add(when: ['CONFIG_KVM', 'CONFIG_XICS'],
>> diff --git a/hw/intc/riscv_aplic.c b/hw/intc/riscv_aplic.c
>> new file mode 100644
>> index 0000000000..885c1de2af
>> --- /dev/null
>> +++ b/hw/intc/riscv_aplic.c
>> @@ -0,0 +1,975 @@
>> +/*
>> + * RISC-V APLIC (Advanced Platform Level Interrupt Controller)
>> + *
>> + * Copyright (c) 2021 Western Digital Corporation or its affiliates.
>> + *
>> + * This program is free software; you can redistribute it and/or modify it
>> + * under the terms and conditions of the GNU General Public License,
>> + * version 2 or later, as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope it will be useful, but WITHOUT
>> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
>> + * more details.
>> + *
>> + * You should have received a copy of the GNU General Public License along with
>> + * this program.  If not, see <http://www.gnu.org/licenses/>.
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "qapi/error.h"
>> +#include "qemu/log.h"
>> +#include "qemu/module.h"
>> +#include "qemu/error-report.h"
>> +#include "qemu/bswap.h"
>> +#include "exec/address-spaces.h"
>> +#include "hw/sysbus.h"
>> +#include "hw/pci/msi.h"
>> +#include "hw/boards.h"
>> +#include "hw/qdev-properties.h"
>> +#include "hw/intc/riscv_aplic.h"
>> +#include "hw/irq.h"
>> +#include "target/riscv/cpu.h"
>> +#include "sysemu/sysemu.h"
>> +#include "migration/vmstate.h"
>> +
>> +#define APLIC_MAX_IDC                  (1UL << 14)
>> +#define APLIC_MAX_SOURCE               1024
>> +#define APLIC_MIN_IPRIO_BITS           1
>> +#define APLIC_MAX_IPRIO_BITS           8
>> +#define APLIC_MAX_CHILDREN             1024
>> +
>> +#define APLIC_DOMAINCFG                0x0000
>> +#define APLIC_DOMAINCFG_RDONLY         0x80000000
>> +#define APLIC_DOMAINCFG_IE             (1 << 8)
>> +#define APLIC_DOMAINCFG_DM             (1 << 2)
>> +#define APLIC_DOMAINCFG_BE             (1 << 0)
>> +
>> +#define APLIC_SOURCECFG_BASE           0x0004
>> +#define APLIC_SOURCECFG_D              (1 << 10)
>> +#define APLIC_SOURCECFG_CHILDIDX_MASK  0x000003ff
>> +#define APLIC_SOURCECFG_SM_MASK        0x00000007
>> +#define APLIC_SOURCECFG_SM_INACTIVE    0x0
>> +#define APLIC_SOURCECFG_SM_DETACH      0x1
>> +#define APLIC_SOURCECFG_SM_EDGE_RISE   0x4
>> +#define APLIC_SOURCECFG_SM_EDGE_FALL   0x5
>> +#define APLIC_SOURCECFG_SM_LEVEL_HIGH  0x6
>> +#define APLIC_SOURCECFG_SM_LEVEL_LOW   0x7
>> +
>> +#define APLIC_MMSICFGADDR              0x1bc0
>> +#define APLIC_MMSICFGADDRH             0x1bc4
>> +#define APLIC_SMSICFGADDR              0x1bc8
>> +#define APLIC_SMSICFGADDRH             0x1bcc
>> +
>> +#define APLIC_xMSICFGADDRH_L           (1UL << 31)
>> +#define APLIC_xMSICFGADDRH_HHXS_MASK   0x1f
>> +#define APLIC_xMSICFGADDRH_HHXS_SHIFT  24
>> +#define APLIC_xMSICFGADDRH_LHXS_MASK   0x7
>> +#define APLIC_xMSICFGADDRH_LHXS_SHIFT  20
>> +#define APLIC_xMSICFGADDRH_HHXW_MASK   0x7
>> +#define APLIC_xMSICFGADDRH_HHXW_SHIFT  16
>> +#define APLIC_xMSICFGADDRH_LHXW_MASK   0xf
>> +#define APLIC_xMSICFGADDRH_LHXW_SHIFT  12
>> +#define APLIC_xMSICFGADDRH_BAPPN_MASK  0xfff
>> +
>> +#define APLIC_xMSICFGADDR_PPN_SHIFT    12
>> +
>> +#define APLIC_xMSICFGADDR_PPN_HART(__lhxs) \
>> +    ((1UL << (__lhxs)) - 1)
>> +
>> +#define APLIC_xMSICFGADDR_PPN_LHX_MASK(__lhxw) \
>> +    ((1UL << (__lhxw)) - 1)
>> +#define APLIC_xMSICFGADDR_PPN_LHX_SHIFT(__lhxs) \
>> +    ((__lhxs))
>> +#define APLIC_xMSICFGADDR_PPN_LHX(__lhxw, __lhxs) \
>> +    (APLIC_xMSICFGADDR_PPN_LHX_MASK(__lhxw) << \
>> +     APLIC_xMSICFGADDR_PPN_LHX_SHIFT(__lhxs))
>> +
>> +#define APLIC_xMSICFGADDR_PPN_HHX_MASK(__hhxw) \
>> +    ((1UL << (__hhxw)) - 1)
>> +#define APLIC_xMSICFGADDR_PPN_HHX_SHIFT(__hhxs) \
>> +    ((__hhxs) + APLIC_xMSICFGADDR_PPN_SHIFT)
>> +#define APLIC_xMSICFGADDR_PPN_HHX(__hhxw, __hhxs) \
>> +    (APLIC_xMSICFGADDR_PPN_HHX_MASK(__hhxw) << \
>> +     APLIC_xMSICFGADDR_PPN_HHX_SHIFT(__hhxs))
>> +
>> +#define APLIC_xMSICFGADDRH_VALID_MASK   \
>> +    (APLIC_xMSICFGADDRH_L | \
>> +     (APLIC_xMSICFGADDRH_HHXS_MASK << APLIC_xMSICFGADDRH_HHXS_SHIFT) | \
>> +     (APLIC_xMSICFGADDRH_LHXS_MASK << APLIC_xMSICFGADDRH_LHXS_SHIFT) | \
>> +     (APLIC_xMSICFGADDRH_HHXW_MASK << APLIC_xMSICFGADDRH_HHXW_SHIFT) | \
>> +     (APLIC_xMSICFGADDRH_LHXW_MASK << APLIC_xMSICFGADDRH_LHXW_SHIFT) | \
>> +     APLIC_xMSICFGADDRH_BAPPN_MASK)
>> +
>> +#define APLIC_SETIP_BASE               0x1c00
>> +#define APLIC_SETIPNUM                 0x1cdc
>> +
>> +#define APLIC_CLRIP_BASE               0x1d00
>> +#define APLIC_CLRIPNUM                 0x1ddc
>> +
>> +#define APLIC_SETIE_BASE               0x1e00
>> +#define APLIC_SETIENUM                 0x1edc
>> +
>> +#define APLIC_CLRIE_BASE               0x1f00
>> +#define APLIC_CLRIENUM                 0x1fdc
>> +
>> +#define APLIC_SETIPNUM_LE              0x2000
>> +#define APLIC_SETIPNUM_BE              0x2004
>> +
>> +#define APLIC_ISTATE_PENDING           (1U << 0)
>> +#define APLIC_ISTATE_ENABLED           (1U << 1)
>> +#define APLIC_ISTATE_ENPEND            (APLIC_ISTATE_ENABLED | \
>> +                                        APLIC_ISTATE_PENDING)
>> +#define APLIC_ISTATE_INPUT             (1U << 8)
>> +
>> +#define APLIC_GENMSI                   0x3000
>> +
>> +#define APLIC_TARGET_BASE              0x3004
>> +#define APLIC_TARGET_HART_IDX_SHIFT    18
>> +#define APLIC_TARGET_HART_IDX_MASK     0x3fff
>> +#define APLIC_TARGET_GUEST_IDX_SHIFT   12
>> +#define APLIC_TARGET_GUEST_IDX_MASK    0x3f
>> +#define APLIC_TARGET_IPRIO_MASK        0xff
>> +#define APLIC_TARGET_EIID_MASK         0x7ff
>> +
>> +#define APLIC_IDC_BASE                 0x4000
>> +#define APLIC_IDC_SIZE                 32
>> +
>> +#define APLIC_IDC_IDELIVERY            0x00
>> +
>> +#define APLIC_IDC_IFORCE               0x04
>> +
>> +#define APLIC_IDC_ITHRESHOLD           0x08
>> +
>> +#define APLIC_IDC_TOPI                 0x18
>> +#define APLIC_IDC_TOPI_ID_SHIFT        16
>> +#define APLIC_IDC_TOPI_ID_MASK         0x3ff
>> +#define APLIC_IDC_TOPI_PRIO_MASK       0xff
>> +
>> +#define APLIC_IDC_CLAIMI               0x1c
>> +
>> +static uint32_t riscv_aplic_read_input_word(RISCVAPLICState *aplic,
>> +                                            uint32_t word)
>> +{
>> +    uint32_t i, irq, ret = 0;
>> +
>> +    for (i = 0; i < 32; i++) {
>> +        irq = word * 32 + i;
>> +        if (!irq || aplic->num_irqs <= irq) {
>> +            continue;
>> +        }
>> +
>> +        ret |= ((aplic->state[irq] & APLIC_ISTATE_INPUT) ? 1 : 0) << i;
>> +    }
>> +
>> +    return ret;
>> +}
>> +
>> +static uint32_t riscv_aplic_read_pending_word(RISCVAPLICState *aplic,
>> +                                              uint32_t word)
>> +{
>> +    uint32_t i, irq, ret = 0;
>> +
>> +    for (i = 0; i < 32; i++) {
>> +        irq = word * 32 + i;
>> +        if (!irq || aplic->num_irqs <= irq) {
>> +            continue;
>> +        }
>> +
>> +        ret |= ((aplic->state[irq] & APLIC_ISTATE_PENDING) ? 1 : 0) << i;
>> +    }
>> +
>> +    return ret;
>> +}
>> +
>> +static void riscv_aplic_set_pending_raw(RISCVAPLICState *aplic,
>> +                                        uint32_t irq, bool pending)
>> +{
>> +    if (pending) {
>> +        aplic->state[irq] |= APLIC_ISTATE_PENDING;
>> +    } else {
>> +        aplic->state[irq] &= ~APLIC_ISTATE_PENDING;
>> +    }
>> +}
>> +
>> +static void riscv_aplic_set_pending(RISCVAPLICState *aplic,
>> +                                    uint32_t irq, bool pending)
>> +{
>> +    uint32_t sourcecfg, sm;
>> +
>> +    if ((irq <= 0) || (aplic->num_irqs <= irq)) {
>> +        return;
>> +    }
>> +
>> +    sourcecfg = aplic->sourcecfg[irq];
>> +    if (sourcecfg & APLIC_SOURCECFG_D) {
>> +        return;
>> +    }
>> +
>> +    sm = sourcecfg & APLIC_SOURCECFG_SM_MASK;
>> +    if ((sm == APLIC_SOURCECFG_SM_INACTIVE) ||
>> +        ((!aplic->msimode || (aplic->msimode && !pending)) &&
>> +         ((sm == APLIC_SOURCECFG_SM_LEVEL_HIGH) ||
>> +          (sm == APLIC_SOURCECFG_SM_LEVEL_LOW)))) {
>> +        return;
>> +    }
>> +
>> +    riscv_aplic_set_pending_raw(aplic, irq, pending);
>> +}
>> +
>> +static void riscv_aplic_set_pending_word(RISCVAPLICState *aplic,
>> +                                         uint32_t word, uint32_t value,
>> +                                         bool pending)
>> +{
>> +    uint32_t i, irq;
>> +
>> +    for (i = 0; i < 32; i++) {
>> +        irq = word * 32 + i;
>> +        if (!irq || aplic->num_irqs <= irq) {
>> +            continue;
>> +        }
>> +
>> +        if (value & (1U << i)) {
>> +            riscv_aplic_set_pending(aplic, irq, pending);
>> +        }
>> +    }
>> +}
>> +
>> +static uint32_t riscv_aplic_read_enabled_word(RISCVAPLICState *aplic,
>> +                                              int word)
>> +{
>> +    uint32_t i, irq, ret = 0;
>> +
>> +    for (i = 0; i < 32; i++) {
>> +        irq = word * 32 + i;
>> +        if (!irq || aplic->num_irqs <= irq) {
>> +            continue;
>> +        }
>> +
>> +        ret |= ((aplic->state[irq] & APLIC_ISTATE_ENABLED) ? 1 : 0) << i;
>> +    }
>> +
>> +    return ret;
>> +}
>> +
>> +static void riscv_aplic_set_enabled_raw(RISCVAPLICState *aplic,
>> +                                        uint32_t irq, bool enabled)
>> +{
>> +    if (enabled) {
>> +        aplic->state[irq] |= APLIC_ISTATE_ENABLED;
>> +    } else {
>> +        aplic->state[irq] &= ~APLIC_ISTATE_ENABLED;
>> +    }
>> +}
>> +
>> +static void riscv_aplic_set_enabled(RISCVAPLICState *aplic,
>> +                                    uint32_t irq, bool enabled)
>> +{
>> +    uint32_t sourcecfg, sm;
>> +
>> +    if ((irq <= 0) || (aplic->num_irqs <= irq)) {
>> +        return;
>> +    }
>> +
>> +    sourcecfg = aplic->sourcecfg[irq];
>> +    if (sourcecfg & APLIC_SOURCECFG_D) {
>> +        return;
>> +    }
>> +
>> +    sm = sourcecfg & APLIC_SOURCECFG_SM_MASK;
>> +    if (sm == APLIC_SOURCECFG_SM_INACTIVE) {
>> +        return;
>> +    }
>> +
>> +    riscv_aplic_set_enabled_raw(aplic, irq, enabled);
>> +}
>> +
>> +static void riscv_aplic_set_enabled_word(RISCVAPLICState *aplic,
>> +                                         uint32_t word, uint32_t value,
>> +                                         bool enabled)
>> +{
>> +    uint32_t i, irq;
>> +
>> +    for (i = 0; i < 32; i++) {
>> +        irq = word * 32 + i;
>> +        if (!irq || aplic->num_irqs <= irq) {
>> +            continue;
>> +        }
>> +
>> +        if (value & (1U << i)) {
>> +            riscv_aplic_set_enabled(aplic, irq, enabled);
>> +        }
>> +    }
>> +}
>> +
>> +static void riscv_aplic_msi_send(RISCVAPLICState *aplic,
>> +                                 uint32_t hart_idx, uint32_t guest_idx,
>> +                                 uint32_t eiid)
>> +{
>> +    uint64_t addr;
>> +    MemTxResult result;
>> +    RISCVAPLICState *aplic_m;
>> +    uint32_t lhxs, lhxw, hhxs, hhxw, group_idx, msicfgaddr, msicfgaddrH;
>> +
>> +    aplic_m = aplic;
>> +    while (aplic_m && !aplic_m->mmode) {
>> +        aplic_m = aplic_m->parent;
>> +    }
>> +    if (!aplic_m) {
>> +        qemu_log_mask(LOG_GUEST_ERROR, "%s: m-level APLIC not found\n",
>> +                      __func__);
>> +        return;
>> +    }
>> +
>> +    if (aplic->mmode) {
>> +        msicfgaddr = aplic_m->mmsicfgaddr;
>> +        msicfgaddrH = aplic_m->mmsicfgaddrH;
>> +    } else {
>> +        msicfgaddr = aplic_m->smsicfgaddr;
>> +        msicfgaddrH = aplic_m->smsicfgaddrH;
>> +    }
>> +
>> +    lhxs = (msicfgaddrH >> APLIC_xMSICFGADDRH_LHXS_SHIFT) &
>> +            APLIC_xMSICFGADDRH_LHXS_MASK;
>> +    lhxw = (msicfgaddrH >> APLIC_xMSICFGADDRH_LHXW_SHIFT) &
>> +            APLIC_xMSICFGADDRH_LHXW_MASK;
>> +    hhxs = (msicfgaddrH >> APLIC_xMSICFGADDRH_HHXS_SHIFT) &
>> +            APLIC_xMSICFGADDRH_HHXS_MASK;
>> +    hhxw = (msicfgaddrH >> APLIC_xMSICFGADDRH_HHXW_SHIFT) &
>> +            APLIC_xMSICFGADDRH_HHXW_MASK;
>> +
>> +    group_idx = hart_idx >> lhxw;
>> +    hart_idx &= APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw);
>> +
>> +    addr = msicfgaddr;
>> +    addr |= ((uint64_t)(msicfgaddrH & APLIC_xMSICFGADDRH_BAPPN_MASK)) << 32;
>> +    addr |= ((uint64_t)(group_idx & APLIC_xMSICFGADDR_PPN_HHX_MASK(hhxw))) <<
>> +             APLIC_xMSICFGADDR_PPN_HHX_SHIFT(hhxs);
>> +    addr |= ((uint64_t)(hart_idx & APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw))) <<
>> +             APLIC_xMSICFGADDR_PPN_LHX_SHIFT(lhxs);
>> +    addr |= (uint64_t)(guest_idx & APLIC_xMSICFGADDR_PPN_HART(lhxs));
>> +    addr <<= APLIC_xMSICFGADDR_PPN_SHIFT;
>> +
>> +    address_space_stl_le(&address_space_memory, addr,
>> +                         eiid, MEMTXATTRS_UNSPECIFIED, &result);
>> +    if (result != MEMTX_OK) {
>> +        qemu_log_mask(LOG_GUEST_ERROR, "%s: MSI write failed for "
>> +                      "hart_index=%d guest_index=%d eiid=%d\n",
>> +                      __func__, hart_idx, guest_idx, eiid);
>> +    }
>> +}
>> +
>> +static void riscv_aplic_msi_irq_update(RISCVAPLICState *aplic, uint32_t irq)
>> +{
>> +    uint32_t hart_idx, guest_idx, eiid;
>> +
>> +    if (!aplic->msimode || (aplic->num_irqs <= irq) ||
>> +        !(aplic->domaincfg & APLIC_DOMAINCFG_IE)) {
>> +        return;
>> +    }
>> +
>> +    if ((aplic->state[irq] & APLIC_ISTATE_ENPEND) != APLIC_ISTATE_ENPEND) {
>> +        return;
>> +    }
>> +
>> +    riscv_aplic_set_pending_raw(aplic, irq, false);
>> +
>> +    hart_idx = aplic->target[irq] >> APLIC_TARGET_HART_IDX_SHIFT;
>> +    hart_idx &= APLIC_TARGET_HART_IDX_MASK;
>
>
> Hi Anup,
>
> I'm wondering does the hart_idx here has to use Machine-level hart index
> for Supervisor-level interrupt domain?
>
> According to AIA spec for supervisor-level domin (Section 4.8.1):
>   For a supervisor-level domain, "if" MSI target addresses are determined by the parent domain’s
>   smsiaddrcfg and smsiaddrcfgh registers, then to construct the address for an outgoing MSI for
>   interrupt source i, the Hart Index from register target[i] must first be converted into the index
>   that the machine-level parent domain uses for the same hart. (Typically, these numbers are the
>   same, but they may not be.) The address for the MSI is then computed using this "machine-level"
>   hart index, the parent domain’s smsiaddrcfg and smsiaddrcfgh, and the Guest Index value from
>   target[i].
>
> The description of converting target[i] hart Index to machine-level hart index in the spec
> is quite vague to me.

The "Section 4.3 Hart index numbers" states the following:

    The index number a domain associates with a hart may or may not
have any relationship
    to the unique hart identifier (“hart ID”) that the RISC-V
Privileged Architecture assigns to
    the hart.

For APLIC direct mode, if N harts are associated with a given domain
then APLIC associates
hart index from 0 to N-1 which are basically APLIC IDC numbers. In
this case, the APLIC
hart index is not related to "machine-level" hart index so software
has to discover it from the
"interrupt-extended" DT property (or similar for ACPI).

For APLIC msi mode, the APLIC can basically target any HART using information in
[m|s]msiaddrcfg registers and interrupt target registers. In this
case, the APLIC hart
index in the APLIC target register is a combination of IMSIC group
index (g) and IMSIC
hart index (h) and the APLIC target register also provides IMSIC guest
index. In other
words, the APLIC hart index for msi mode is also not related to the
"machine-level" hart
index. The APLIC driver will figure-out/extract the IMSIC group index
(g), IMSIC hart
index (h), and IMSIC guest index from target MSI address when Linux
tries to write
target MSI address for a particular APLIC interrupt.

Regards,
Anup

>
> Regards,
> Frank Chang
>
>>
>> +    if (aplic->mmode) {
>> +        /* M-level APLIC ignores guest_index */
>> +        guest_idx = 0;
>> +    } else {
>> +        guest_idx = aplic->target[irq] >> APLIC_TARGET_GUEST_IDX_SHIFT;
>> +        guest_idx &= APLIC_TARGET_GUEST_IDX_MASK;
>> +    }
>> +    eiid = aplic->target[irq] & APLIC_TARGET_EIID_MASK;
>> +    riscv_aplic_msi_send(aplic, hart_idx, guest_idx, eiid);
>> +}
>> +
>> +static uint32_t riscv_aplic_idc_topi(RISCVAPLICState *aplic, uint32_t idc)
>> +{
>> +    uint32_t best_irq, best_iprio;
>> +    uint32_t irq, iprio, ihartidx, ithres;
>> +
>> +    if (aplic->num_harts <= idc) {
>> +        return 0;
>> +    }
>> +
>> +    ithres = aplic->ithreshold[idc];
>> +    best_irq = best_iprio = UINT32_MAX;
>> +    for (irq = 1; irq < aplic->num_irqs; irq++) {
>> +        if ((aplic->state[irq] & APLIC_ISTATE_ENPEND) !=
>> +            APLIC_ISTATE_ENPEND) {
>> +            continue;
>> +        }
>> +
>> +        ihartidx = aplic->target[irq] >> APLIC_TARGET_HART_IDX_SHIFT;
>> +        ihartidx &= APLIC_TARGET_HART_IDX_MASK;
>> +        if (ihartidx != idc) {
>> +            continue;
>> +        }
>> +
>> +        iprio = aplic->target[irq] & aplic->iprio_mask;
>> +        if (ithres && iprio >= ithres) {
>> +            continue;
>> +        }
>> +
>> +        if (iprio < best_iprio) {
>> +            best_irq = irq;
>> +            best_iprio = iprio;
>> +        }
>> +    }
>> +
>> +    if (best_irq < aplic->num_irqs && best_iprio <= aplic->iprio_mask) {
>> +        return (best_irq << APLIC_IDC_TOPI_ID_SHIFT) | best_iprio;
>> +    }
>> +
>> +    return 0;
>> +}
>> +
>> +static void riscv_aplic_idc_update(RISCVAPLICState *aplic, uint32_t idc)
>> +{
>> +    uint32_t topi;
>> +
>> +    if (aplic->msimode || aplic->num_harts <= idc) {
>> +        return;
>> +    }
>> +
>> +    topi = riscv_aplic_idc_topi(aplic, idc);
>> +    if ((aplic->domaincfg & APLIC_DOMAINCFG_IE) &&
>> +        aplic->idelivery[idc] &&
>> +        (aplic->iforce[idc] || topi)) {
>> +        qemu_irq_raise(aplic->external_irqs[idc]);
>> +    } else {
>> +        qemu_irq_lower(aplic->external_irqs[idc]);
>> +    }
>> +}
>> +
>> +static uint32_t riscv_aplic_idc_claimi(RISCVAPLICState *aplic, uint32_t idc)
>> +{
>> +    uint32_t irq, state, sm, topi = riscv_aplic_idc_topi(aplic, idc);
>> +
>> +    if (!topi) {
>> +        aplic->iforce[idc] = 0;
>> +        return 0;
>> +    }
>> +
>> +    irq = (topi >> APLIC_IDC_TOPI_ID_SHIFT) & APLIC_IDC_TOPI_ID_MASK;
>> +    sm = aplic->sourcecfg[irq] & APLIC_SOURCECFG_SM_MASK;
>> +    state = aplic->state[irq];
>> +    riscv_aplic_set_pending_raw(aplic, irq, false);
>> +    if ((sm == APLIC_SOURCECFG_SM_LEVEL_HIGH) &&
>> +        (state & APLIC_ISTATE_INPUT)) {
>> +        riscv_aplic_set_pending_raw(aplic, irq, true);
>> +    } else if ((sm == APLIC_SOURCECFG_SM_LEVEL_LOW) &&
>> +               !(state & APLIC_ISTATE_INPUT)) {
>> +        riscv_aplic_set_pending_raw(aplic, irq, true);
>> +    }
>> +    riscv_aplic_idc_update(aplic, idc);
>> +
>> +    return topi;
>> +}
>> +
>> +static void riscv_aplic_request(void *opaque, int irq, int level)
>> +{
>> +    bool update = false;
>> +    RISCVAPLICState *aplic = opaque;
>> +    uint32_t sourcecfg, childidx, state, idc;
>> +
>> +    assert((0 < irq) && (irq < aplic->num_irqs));
>> +
>> +    sourcecfg = aplic->sourcecfg[irq];
>> +    if (sourcecfg & APLIC_SOURCECFG_D) {
>> +        childidx = sourcecfg & APLIC_SOURCECFG_CHILDIDX_MASK;
>> +        if (childidx < aplic->num_children) {
>> +            riscv_aplic_request(aplic->children[childidx], irq, level);
>> +        }
>> +        return;
>> +    }
>> +
>> +    state = aplic->state[irq];
>> +    switch (sourcecfg & APLIC_SOURCECFG_SM_MASK) {
>> +    case APLIC_SOURCECFG_SM_EDGE_RISE:
>> +        if ((level > 0) && !(state & APLIC_ISTATE_INPUT) &&
>> +            !(state & APLIC_ISTATE_PENDING)) {
>> +            riscv_aplic_set_pending_raw(aplic, irq, true);
>> +            update = true;
>> +        }
>> +        break;
>> +    case APLIC_SOURCECFG_SM_EDGE_FALL:
>> +        if ((level <= 0) && (state & APLIC_ISTATE_INPUT) &&
>> +            !(state & APLIC_ISTATE_PENDING)) {
>> +            riscv_aplic_set_pending_raw(aplic, irq, true);
>> +            update = true;
>> +        }
>> +        break;
>> +    case APLIC_SOURCECFG_SM_LEVEL_HIGH:
>> +        if ((level > 0) && !(state & APLIC_ISTATE_PENDING)) {
>> +            riscv_aplic_set_pending_raw(aplic, irq, true);
>> +            update = true;
>> +        }
>> +        break;
>> +    case APLIC_SOURCECFG_SM_LEVEL_LOW:
>> +        if ((level <= 0) && !(state & APLIC_ISTATE_PENDING)) {
>> +            riscv_aplic_set_pending_raw(aplic, irq, true);
>> +            update = true;
>> +        }
>> +        break;
>> +    default:
>> +        break;
>> +    }
>> +
>> +    if (level <= 0) {
>> +        aplic->state[irq] &= ~APLIC_ISTATE_INPUT;
>> +    } else {
>> +        aplic->state[irq] |= APLIC_ISTATE_INPUT;
>> +    }
>> +
>> +    if (update) {
>> +        if (aplic->msimode) {
>> +            riscv_aplic_msi_irq_update(aplic, irq);
>> +        } else {
>> +            idc = aplic->target[irq] >> APLIC_TARGET_HART_IDX_SHIFT;
>> +            idc &= APLIC_TARGET_HART_IDX_MASK;
>> +            riscv_aplic_idc_update(aplic, idc);
>> +        }
>> +    }
>> +}
>> +
>> +static uint64_t riscv_aplic_read(void *opaque, hwaddr addr, unsigned size)
>> +{
>> +    uint32_t irq, word, idc;
>> +    RISCVAPLICState *aplic = opaque;
>> +
>> +    /* Reads must be 4 byte words */
>> +    if ((addr & 0x3) != 0) {
>> +        goto err;
>> +    }
>> +
>> +    if (addr == APLIC_DOMAINCFG) {
>> +        return APLIC_DOMAINCFG_RDONLY | aplic->domaincfg |
>> +               (aplic->msimode ? APLIC_DOMAINCFG_DM : 0);
>> +    } else if ((APLIC_SOURCECFG_BASE <= addr) &&
>> +            (addr < (APLIC_SOURCECFG_BASE + (aplic->num_irqs - 1) * 4))) {
>> +        irq  = ((addr - APLIC_SOURCECFG_BASE) >> 2) + 1;
>> +        return aplic->sourcecfg[irq];
>> +    } else if (aplic->mmode && aplic->msimode &&
>> +               (addr == APLIC_MMSICFGADDR)) {
>> +        return aplic->mmsicfgaddr;
>> +    } else if (aplic->mmode && aplic->msimode &&
>> +               (addr == APLIC_MMSICFGADDRH)) {
>> +        return aplic->mmsicfgaddrH;
>> +    } else if (aplic->mmode && aplic->msimode &&
>> +               (addr == APLIC_SMSICFGADDR)) {
>> +        /* Registers SMSICFGADDR and SMSICFGADDRH are implemented only if:
>> +         * (a) the interrupt domain is at machine level
>> +         * (b) the domain’s harts implement supervisor mode
>> +         * (c) the domain has one or more child supervisor-level domains
>> +         *     that support MSI delivery mode (domaincfg.DM is not read-
>> +         *     only zero in at least one of the supervisor-level child
>> +         * domains).
>> +         */
>> +        return (aplic->num_children) ? aplic->smsicfgaddr : 0;
>> +    } else if (aplic->mmode && aplic->msimode &&
>> +               (addr == APLIC_SMSICFGADDRH)) {
>> +        return (aplic->num_children) ? aplic->smsicfgaddrH : 0;
>> +    } else if ((APLIC_SETIP_BASE <= addr) &&
>> +            (addr < (APLIC_SETIP_BASE + aplic->bitfield_words * 4))) {
>> +        word = (addr - APLIC_SETIP_BASE) >> 2;
>> +        return riscv_aplic_read_pending_word(aplic, word);
>> +    } else if (addr == APLIC_SETIPNUM) {
>> +        return 0;
>> +    } else if ((APLIC_CLRIP_BASE <= addr) &&
>> +            (addr < (APLIC_CLRIP_BASE + aplic->bitfield_words * 4))) {
>> +        word = (addr - APLIC_CLRIP_BASE) >> 2;
>> +        return riscv_aplic_read_input_word(aplic, word);
>> +    } else if (addr == APLIC_CLRIPNUM) {
>> +        return 0;
>> +    } else if ((APLIC_SETIE_BASE <= addr) &&
>> +            (addr < (APLIC_SETIE_BASE + aplic->bitfield_words * 4))) {
>> +        word = (addr - APLIC_SETIE_BASE) >> 2;
>> +        return riscv_aplic_read_enabled_word(aplic, word);
>> +    } else if (addr == APLIC_SETIENUM) {
>> +        return 0;
>> +    } else if ((APLIC_CLRIE_BASE <= addr) &&
>> +            (addr < (APLIC_CLRIE_BASE + aplic->bitfield_words * 4))) {
>> +        return 0;
>> +    } else if (addr == APLIC_CLRIENUM) {
>> +        return 0;
>> +    } else if (addr == APLIC_SETIPNUM_LE) {
>> +        return 0;
>> +    } else if (addr == APLIC_SETIPNUM_BE) {
>> +        return 0;
>> +    } else if (addr == APLIC_GENMSI) {
>> +        return (aplic->msimode) ? aplic->genmsi : 0;
>> +    } else if ((APLIC_TARGET_BASE <= addr) &&
>> +            (addr < (APLIC_TARGET_BASE + (aplic->num_irqs - 1) * 4))) {
>> +        irq = ((addr - APLIC_TARGET_BASE) >> 2) + 1;
>> +        return aplic->target[irq];
>> +    } else if (!aplic->msimode && (APLIC_IDC_BASE <= addr) &&
>> +            (addr < (APLIC_IDC_BASE + aplic->num_harts * APLIC_IDC_SIZE))) {
>> +        idc = (addr - APLIC_IDC_BASE) / APLIC_IDC_SIZE;
>> +        switch (addr - (APLIC_IDC_BASE + idc * APLIC_IDC_SIZE)) {
>> +        case APLIC_IDC_IDELIVERY:
>> +            return aplic->idelivery[idc];
>> +        case APLIC_IDC_IFORCE:
>> +            return aplic->iforce[idc];
>> +        case APLIC_IDC_ITHRESHOLD:
>> +            return aplic->ithreshold[idc];
>> +        case APLIC_IDC_TOPI:
>> +            return riscv_aplic_idc_topi(aplic, idc);
>> +        case APLIC_IDC_CLAIMI:
>> +            return riscv_aplic_idc_claimi(aplic, idc);
>> +        default:
>> +            goto err;
>> +        };
>> +    }
>> +
>> +err:
>> +    qemu_log_mask(LOG_GUEST_ERROR,
>> +                  "%s: Invalid register read 0x%" HWADDR_PRIx "\n",
>> +                  __func__, addr);
>> +    return 0;
>> +}
>> +
>> +static void riscv_aplic_write(void *opaque, hwaddr addr, uint64_t value,
>> +        unsigned size)
>> +{
>> +    RISCVAPLICState *aplic = opaque;
>> +    uint32_t irq, word, idc = UINT32_MAX;
>> +
>> +    /* Writes must be 4 byte words */
>> +    if ((addr & 0x3) != 0) {
>> +        goto err;
>> +    }
>> +
>> +    if (addr == APLIC_DOMAINCFG) {
>> +        /* Only IE bit writeable at the moment */
>> +        value &= APLIC_DOMAINCFG_IE;
>> +        aplic->domaincfg = value;
>> +    } else if ((APLIC_SOURCECFG_BASE <= addr) &&
>> +            (addr < (APLIC_SOURCECFG_BASE + (aplic->num_irqs - 1) * 4))) {
>> +        irq  = ((addr - APLIC_SOURCECFG_BASE) >> 2) + 1;
>> +        if (!aplic->num_children && (value & APLIC_SOURCECFG_D)) {
>> +            value = 0;
>> +        }
>> +        if (value & APLIC_SOURCECFG_D) {
>> +            value &= (APLIC_SOURCECFG_D | APLIC_SOURCECFG_CHILDIDX_MASK);
>> +        } else {
>> +            value &= (APLIC_SOURCECFG_D | APLIC_SOURCECFG_SM_MASK);
>> +        }
>> +        aplic->sourcecfg[irq] = value;
>> +        if ((aplic->sourcecfg[irq] & APLIC_SOURCECFG_D) ||
>> +            (aplic->sourcecfg[irq] == 0)) {
>> +            riscv_aplic_set_pending_raw(aplic, irq, false);
>> +            riscv_aplic_set_enabled_raw(aplic, irq, false);
>> +        }
>> +    } else if (aplic->mmode && aplic->msimode &&
>> +               (addr == APLIC_MMSICFGADDR)) {
>> +        if (!(aplic->mmsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
>> +            aplic->mmsicfgaddr = value;
>> +        }
>> +    } else if (aplic->mmode && aplic->msimode &&
>> +               (addr == APLIC_MMSICFGADDRH)) {
>> +        if (!(aplic->mmsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
>> +            aplic->mmsicfgaddrH = value & APLIC_xMSICFGADDRH_VALID_MASK;
>> +        }
>> +    } else if (aplic->mmode && aplic->msimode &&
>> +               (addr == APLIC_SMSICFGADDR)) {
>> +        /* Registers SMSICFGADDR and SMSICFGADDRH are implemented only if:
>> +         * (a) the interrupt domain is at machine level
>> +         * (b) the domain’s harts implement supervisor mode
>> +         * (c) the domain has one or more child supervisor-level domains
>> +         *     that support MSI delivery mode (domaincfg.DM is not read-
>> +         *     only zero in at least one of the supervisor-level child
>> +         * domains).
>> +         */
>> +        if (aplic->num_children &&
>> +            !(aplic->smsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
>> +            aplic->smsicfgaddr = value;
>> +        }
>> +    } else if (aplic->mmode && aplic->msimode &&
>> +               (addr == APLIC_SMSICFGADDRH)) {
>> +        if (aplic->num_children &&
>> +            !(aplic->smsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
>> +            aplic->smsicfgaddrH = value & APLIC_xMSICFGADDRH_VALID_MASK;
>> +        }
>> +    } else if ((APLIC_SETIP_BASE <= addr) &&
>> +            (addr < (APLIC_SETIP_BASE + aplic->bitfield_words * 4))) {
>> +        word = (addr - APLIC_SETIP_BASE) >> 2;
>> +        riscv_aplic_set_pending_word(aplic, word, value, true);
>> +    } else if (addr == APLIC_SETIPNUM) {
>> +        riscv_aplic_set_pending(aplic, value, true);
>> +    } else if ((APLIC_CLRIP_BASE <= addr) &&
>> +            (addr < (APLIC_CLRIP_BASE + aplic->bitfield_words * 4))) {
>> +        word = (addr - APLIC_CLRIP_BASE) >> 2;
>> +        riscv_aplic_set_pending_word(aplic, word, value, false);
>> +    } else if (addr == APLIC_CLRIPNUM) {
>> +        riscv_aplic_set_pending(aplic, value, false);
>> +    } else if ((APLIC_SETIE_BASE <= addr) &&
>> +            (addr < (APLIC_SETIE_BASE + aplic->bitfield_words * 4))) {
>> +        word = (addr - APLIC_SETIE_BASE) >> 2;
>> +        riscv_aplic_set_enabled_word(aplic, word, value, true);
>> +    } else if (addr == APLIC_SETIENUM) {
>> +        riscv_aplic_set_enabled(aplic, value, true);
>> +    } else if ((APLIC_CLRIE_BASE <= addr) &&
>> +            (addr < (APLIC_CLRIE_BASE + aplic->bitfield_words * 4))) {
>> +        word = (addr - APLIC_CLRIE_BASE) >> 2;
>> +        riscv_aplic_set_enabled_word(aplic, word, value, false);
>> +    } else if (addr == APLIC_CLRIENUM) {
>> +        riscv_aplic_set_enabled(aplic, value, false);
>> +    } else if (addr == APLIC_SETIPNUM_LE) {
>> +        riscv_aplic_set_pending(aplic, value, true);
>> +    } else if (addr == APLIC_SETIPNUM_BE) {
>> +        riscv_aplic_set_pending(aplic, bswap32(value), true);
>> +    } else if (addr == APLIC_GENMSI) {
>> +        if (aplic->msimode) {
>> +            aplic->genmsi = value & ~(APLIC_TARGET_GUEST_IDX_MASK <<
>> +                                      APLIC_TARGET_GUEST_IDX_SHIFT);
>> +            riscv_aplic_msi_send(aplic,
>> +                                 value >> APLIC_TARGET_HART_IDX_SHIFT,
>> +                                 0,
>> +                                 value & APLIC_TARGET_EIID_MASK);
>> +        }
>> +    } else if ((APLIC_TARGET_BASE <= addr) &&
>> +            (addr < (APLIC_TARGET_BASE + (aplic->num_irqs - 1) * 4))) {
>> +        irq = ((addr - APLIC_TARGET_BASE) >> 2) + 1;
>> +        if (aplic->msimode) {
>> +            aplic->target[irq] = value;
>> +        } else {
>> +            aplic->target[irq] = (value & ~APLIC_TARGET_IPRIO_MASK) |
>> +                                 ((value & aplic->iprio_mask) ?
>> +                                  (value & aplic->iprio_mask) : 1);
>> +        }
>> +    } else if (!aplic->msimode && (APLIC_IDC_BASE <= addr) &&
>> +            (addr < (APLIC_IDC_BASE + aplic->num_harts * APLIC_IDC_SIZE))) {
>> +        idc = (addr - APLIC_IDC_BASE) / APLIC_IDC_SIZE;
>> +        switch (addr - (APLIC_IDC_BASE + idc * APLIC_IDC_SIZE)) {
>> +        case APLIC_IDC_IDELIVERY:
>> +            aplic->idelivery[idc] = value & 0x1;
>> +            break;
>> +        case APLIC_IDC_IFORCE:
>> +            aplic->iforce[idc] = value & 0x1;
>> +            break;
>> +        case APLIC_IDC_ITHRESHOLD:
>> +            aplic->ithreshold[idc] = value & aplic->iprio_mask;
>> +            break;
>> +        default:
>> +            goto err;
>> +        };
>> +    } else {
>> +        goto err;
>> +    }
>> +
>> +    if (aplic->msimode) {
>> +        for (irq = 1; irq < aplic->num_irqs; irq++) {
>> +            riscv_aplic_msi_irq_update(aplic, irq);
>> +        }
>> +    } else {
>> +        if (idc == UINT32_MAX) {
>> +            for (idc = 0; idc < aplic->num_harts; idc++) {
>> +                riscv_aplic_idc_update(aplic, idc);
>> +            }
>> +        } else {
>> +            riscv_aplic_idc_update(aplic, idc);
>> +        }
>> +    }
>> +
>> +    return;
>> +
>> +err:
>> +    qemu_log_mask(LOG_GUEST_ERROR,
>> +                  "%s: Invalid register write 0x%" HWADDR_PRIx "\n",
>> +                  __func__, addr);
>> +}
>> +
>> +static const MemoryRegionOps riscv_aplic_ops = {
>> +    .read = riscv_aplic_read,
>> +    .write = riscv_aplic_write,
>> +    .endianness = DEVICE_LITTLE_ENDIAN,
>> +    .valid = {
>> +        .min_access_size = 4,
>> +        .max_access_size = 4
>> +    }
>> +};
>> +
>> +static void riscv_aplic_realize(DeviceState *dev, Error **errp)
>> +{
>> +    uint32_t i;
>> +    RISCVAPLICState *aplic = RISCV_APLIC(dev);
>> +
>> +    aplic->bitfield_words = (aplic->num_irqs + 31) >> 5;
>> +    aplic->sourcecfg = g_new0(uint32_t, aplic->num_irqs);
>> +    aplic->state = g_new(uint32_t, aplic->num_irqs);
>> +    aplic->target = g_new0(uint32_t, aplic->num_irqs);
>> +    if (!aplic->msimode) {
>> +        for (i = 0; i < aplic->num_irqs; i++) {
>> +            aplic->target[i] = 1;
>> +        }
>> +    }
>> +    aplic->idelivery = g_new0(uint32_t, aplic->num_harts);
>> +    aplic->iforce = g_new0(uint32_t, aplic->num_harts);
>> +    aplic->ithreshold = g_new0(uint32_t, aplic->num_harts);
>> +
>> +    memory_region_init_io(&aplic->mmio, OBJECT(dev), &riscv_aplic_ops, aplic,
>> +                          TYPE_RISCV_APLIC, aplic->aperture_size);
>> +    sysbus_init_mmio(SYS_BUS_DEVICE(dev), &aplic->mmio);
>> +
>> +    /* Only root APLICs have hardware IRQ lines. All non-root APLICs
>> +     * have IRQ lines delegated by their parent APLIC.
>> +     */
>> +    if (!aplic->parent) {
>> +        qdev_init_gpio_in(dev, riscv_aplic_request, aplic->num_irqs);
>> +    }
>> +
>> +    /* Create output IRQ lines for non-MSI mode */
>> +    if (!aplic->msimode) {
>> +        aplic->external_irqs = g_malloc(sizeof(qemu_irq) * aplic->num_harts);
>> +        qdev_init_gpio_out(dev, aplic->external_irqs, aplic->num_harts);
>> +
>> +        /* Claim the CPU interrupt to be triggered by this APLIC */
>> +        for (i = 0; i < aplic->num_harts; i++) {
>> +            RISCVCPU *cpu = RISCV_CPU(qemu_get_cpu(aplic->hartid_base + i));
>> +            if (riscv_cpu_claim_interrupts(cpu,
>> +                (aplic->mmode) ? MIP_MEIP : MIP_SEIP) < 0) {
>> +                error_report("%s already claimed",
>> +                             (aplic->mmode) ? "MEIP" : "SEIP");
>> +                exit(1);
>> +            }
>> +        }
>> +    }
>> +
>> +    msi_nonbroken = true;
>> +}
>> +
>> +static Property riscv_aplic_properties[] = {
>> +    DEFINE_PROP_UINT32("aperture-size", RISCVAPLICState, aperture_size, 0),
>> +    DEFINE_PROP_UINT32("hartid-base", RISCVAPLICState, hartid_base, 0),
>> +    DEFINE_PROP_UINT32("num-harts", RISCVAPLICState, num_harts, 0),
>> +    DEFINE_PROP_UINT32("iprio-mask", RISCVAPLICState, iprio_mask, 0),
>> +    DEFINE_PROP_UINT32("num-irqs", RISCVAPLICState, num_irqs, 0),
>> +    DEFINE_PROP_BOOL("msimode", RISCVAPLICState, msimode, 0),
>> +    DEFINE_PROP_BOOL("mmode", RISCVAPLICState, mmode, 0),
>> +    DEFINE_PROP_END_OF_LIST(),
>> +};
>> +
>> +static const VMStateDescription vmstate_riscv_aplic = {
>> +    .name = "riscv_aplic",
>> +    .version_id = 1,
>> +    .minimum_version_id = 1,
>> +    .fields = (VMStateField[]) {
>> +            VMSTATE_UINT32(domaincfg, RISCVAPLICState),
>> +            VMSTATE_UINT32(mmsicfgaddr, RISCVAPLICState),
>> +            VMSTATE_UINT32(mmsicfgaddrH, RISCVAPLICState),
>> +            VMSTATE_UINT32(smsicfgaddr, RISCVAPLICState),
>> +            VMSTATE_UINT32(smsicfgaddrH, RISCVAPLICState),
>> +            VMSTATE_UINT32(genmsi, RISCVAPLICState),
>> +            VMSTATE_VARRAY_UINT32(sourcecfg, RISCVAPLICState,
>> +                                  num_irqs, 0,
>> +                                  vmstate_info_uint32, uint32_t),
>> +            VMSTATE_VARRAY_UINT32(state, RISCVAPLICState,
>> +                                  num_irqs, 0,
>> +                                  vmstate_info_uint32, uint32_t),
>> +            VMSTATE_VARRAY_UINT32(target, RISCVAPLICState,
>> +                                  num_irqs, 0,
>> +                                  vmstate_info_uint32, uint32_t),
>> +            VMSTATE_VARRAY_UINT32(idelivery, RISCVAPLICState,
>> +                                  num_harts, 0,
>> +                                  vmstate_info_uint32, uint32_t),
>> +            VMSTATE_VARRAY_UINT32(iforce, RISCVAPLICState,
>> +                                  num_harts, 0,
>> +                                  vmstate_info_uint32, uint32_t),
>> +            VMSTATE_VARRAY_UINT32(ithreshold, RISCVAPLICState,
>> +                                  num_harts, 0,
>> +                                  vmstate_info_uint32, uint32_t),
>> +            VMSTATE_END_OF_LIST()
>> +        }
>> +};
>> +
>> +static void riscv_aplic_class_init(ObjectClass *klass, void *data)
>> +{
>> +    DeviceClass *dc = DEVICE_CLASS(klass);
>> +
>> +    device_class_set_props(dc, riscv_aplic_properties);
>> +    dc->realize = riscv_aplic_realize;
>> +    dc->vmsd = &vmstate_riscv_aplic;
>> +}
>> +
>> +static const TypeInfo riscv_aplic_info = {
>> +    .name          = TYPE_RISCV_APLIC,
>> +    .parent        = TYPE_SYS_BUS_DEVICE,
>> +    .instance_size = sizeof(RISCVAPLICState),
>> +    .class_init    = riscv_aplic_class_init,
>> +};
>> +
>> +static void riscv_aplic_register_types(void)
>> +{
>> +    type_register_static(&riscv_aplic_info);
>> +}
>> +
>> +type_init(riscv_aplic_register_types)
>> +
>> +/*
>> + * Add a APLIC device to another APLIC device as child for
>> + * interrupt delegation.
>> + */
>> +void riscv_aplic_add_child(DeviceState *parent, DeviceState *child)
>> +{
>> +    RISCVAPLICState *caplic, *paplic;
>> +
>> +    assert(parent && child);
>> +    caplic = RISCV_APLIC(child);
>> +    paplic = RISCV_APLIC(parent);
>> +
>> +    assert(paplic->num_irqs == caplic->num_irqs);
>> +    assert(paplic->num_children <= QEMU_APLIC_MAX_CHILDREN);
>> +
>> +    caplic->parent = paplic;
>> +    paplic->children[paplic->num_children] = caplic;
>> +    paplic->num_children++;
>> +}
>> +
>> +/*
>> + * Create APLIC device.
>> + */
>> +DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size,
>> +    uint32_t hartid_base, uint32_t num_harts, uint32_t num_sources,
>> +    uint32_t iprio_bits, bool msimode, bool mmode, DeviceState *parent)
>> +{
>> +    DeviceState *dev = qdev_new(TYPE_RISCV_APLIC);
>> +    uint32_t i;
>> +
>> +    assert(num_harts < APLIC_MAX_IDC);
>> +    assert((APLIC_IDC_BASE + (num_harts * APLIC_IDC_SIZE)) <= size);
>> +    assert(num_sources < APLIC_MAX_SOURCE);
>> +    assert(APLIC_MIN_IPRIO_BITS <= iprio_bits);
>> +    assert(iprio_bits <= APLIC_MAX_IPRIO_BITS);
>> +
>> +    qdev_prop_set_uint32(dev, "aperture-size", size);
>> +    qdev_prop_set_uint32(dev, "hartid-base", hartid_base);
>> +    qdev_prop_set_uint32(dev, "num-harts", num_harts);
>> +    qdev_prop_set_uint32(dev, "iprio-mask", ((1U << iprio_bits) - 1));
>> +    qdev_prop_set_uint32(dev, "num-irqs", num_sources + 1);
>> +    qdev_prop_set_bit(dev, "msimode", msimode);
>> +    qdev_prop_set_bit(dev, "mmode", mmode);
>> +
>> +    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
>> +    sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
>> +
>> +    if (parent) {
>> +        riscv_aplic_add_child(parent, dev);
>> +    }
>> +
>> +    if (!msimode) {
>> +        for (i = 0; i < num_harts; i++) {
>> +            CPUState *cpu = qemu_get_cpu(hartid_base + i);
>> +
>> +            qdev_connect_gpio_out_named(dev, NULL, i,
>> +                                        qdev_get_gpio_in(DEVICE(cpu),
>> +                                            (mmode) ? IRQ_M_EXT : IRQ_S_EXT));
>> +        }
>> +    }
>> +
>> +    return dev;
>> +}
>> diff --git a/include/hw/intc/riscv_aplic.h b/include/hw/intc/riscv_aplic.h
>> new file mode 100644
>> index 0000000000..de8532fbc3
>> --- /dev/null
>> +++ b/include/hw/intc/riscv_aplic.h
>> @@ -0,0 +1,79 @@
>> +/*
>> + * RISC-V APLIC (Advanced Platform Level Interrupt Controller) interface
>> + *
>> + * Copyright (c) 2021 Western Digital Corporation or its affiliates.
>> + *
>> + * This program is free software; you can redistribute it and/or modify it
>> + * under the terms and conditions of the GNU General Public License,
>> + * version 2 or later, as published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope it will be useful, but WITHOUT
>> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
>> + * more details.
>> + *
>> + * You should have received a copy of the GNU General Public License along with
>> + * this program.  If not, see <http://www.gnu.org/licenses/>.
>> + */
>> +
>> +#ifndef HW_RISCV_APLIC_H
>> +#define HW_RISCV_APLIC_H
>> +
>> +#include "hw/sysbus.h"
>> +#include "qom/object.h"
>> +
>> +#define TYPE_RISCV_APLIC "riscv.aplic"
>> +
>> +typedef struct RISCVAPLICState RISCVAPLICState;
>> +DECLARE_INSTANCE_CHECKER(RISCVAPLICState, RISCV_APLIC, TYPE_RISCV_APLIC)
>> +
>> +#define APLIC_MIN_SIZE            0x4000
>> +#define APLIC_SIZE_ALIGN(__x)     (((__x) + (APLIC_MIN_SIZE - 1)) & \
>> +                                   ~(APLIC_MIN_SIZE - 1))
>> +#define APLIC_SIZE(__num_harts)   (APLIC_MIN_SIZE + \
>> +                                   APLIC_SIZE_ALIGN(32 * (__num_harts)))
>> +
>> +struct RISCVAPLICState {
>> +    /*< private >*/
>> +    SysBusDevice parent_obj;
>> +    qemu_irq *external_irqs;
>> +
>> +    /*< public >*/
>> +    MemoryRegion mmio;
>> +    uint32_t bitfield_words;
>> +    uint32_t domaincfg;
>> +    uint32_t mmsicfgaddr;
>> +    uint32_t mmsicfgaddrH;
>> +    uint32_t smsicfgaddr;
>> +    uint32_t smsicfgaddrH;
>> +    uint32_t genmsi;
>> +    uint32_t *sourcecfg;
>> +    uint32_t *state;
>> +    uint32_t *target;
>> +    uint32_t *idelivery;
>> +    uint32_t *iforce;
>> +    uint32_t *ithreshold;
>> +
>> +    /* topology */
>> +#define QEMU_APLIC_MAX_CHILDREN        16
>> +    struct RISCVAPLICState *parent;
>> +    struct RISCVAPLICState *children[QEMU_APLIC_MAX_CHILDREN];
>> +    uint16_t num_children;
>> +
>> +    /* config */
>> +    uint32_t aperture_size;
>> +    uint32_t hartid_base;
>> +    uint32_t num_harts;
>> +    uint32_t iprio_mask;
>> +    uint32_t num_irqs;
>> +    bool msimode;
>> +    bool mmode;
>> +};
>> +
>> +void riscv_aplic_add_child(DeviceState *parent, DeviceState *child);
>> +
>> +DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size,
>> +    uint32_t hartid_base, uint32_t num_harts, uint32_t num_sources,
>> +    uint32_t iprio_bits, bool msimode, bool mmode, DeviceState *parent);
>> +
>> +#endif
>> --
>> 2.25.1
>>

Frank Chang Jan. 20, 2022, 8:19 a.m. UTC | #3

On Thu, Jan 20, 2022 at 12:20 AM Anup Patel <[e-mail]> wrote:

> Hi Frank,
>
> On Wed, Jan 19, 2022 at 9:07 PM Frank Chang <[e-mail]>
> wrote:
> >
> > On Wed, Jan 19, 2022 at 11:27 PM Anup Patel <[e-mail]> wrote:
> >>
> >> From: Anup Patel <[e-mail]>
> >>
> >> The RISC-V AIA (Advanced Interrupt Architecture) defines a new
> >> interrupt controller for wired interrupts called APLIC (Advanced
> >> Platform Level Interrupt Controller). The APLIC is capabable of
> >> forwarding wired interupts to RISC-V HARTs directly or as MSIs
> >> (Message Signaled Interupts).
> >>
> >> This patch adds device emulation for RISC-V AIA APLIC.
> >>
> >> Signed-off-by: Anup Patel <[e-mail]>
> >> Signed-off-by: Anup Patel <[e-mail]>
> >> Reviewed-by: Frank Chang <[e-mail]>
> >> ---
> >>  hw/intc/Kconfig               |   3 +
> >>  hw/intc/meson.build           |   1 +
> >>  hw/intc/riscv_aplic.c         | 975 ++++++++++++++++++++++++++++++++++
> >>  include/hw/intc/riscv_aplic.h |  79 +++
> >>  4 files changed, 1058 insertions(+)
> >>  create mode 100644 hw/intc/riscv_aplic.c
> >>  create mode 100644 include/hw/intc/riscv_aplic.h
> >>
> >> diff --git a/hw/intc/Kconfig b/hw/intc/Kconfig
> >> index 010ded7eae..528e77b4a6 100644
> >> --- a/hw/intc/Kconfig
> >> +++ b/hw/intc/Kconfig
> >> @@ -70,6 +70,9 @@ config LOONGSON_LIOINTC
> >>  config RISCV_ACLINT
> >>      bool
> >>
> >> +config RISCV_APLIC
> >> +    bool
> >> +
> >>  config SIFIVE_PLIC
> >>      bool
> >>
> >> diff --git a/hw/intc/meson.build b/hw/intc/meson.build
> >> index 70080bc161..7466024402 100644
> >> --- a/hw/intc/meson.build
> >> +++ b/hw/intc/meson.build
> >> @@ -50,6 +50,7 @@ specific_ss.add(when: 'CONFIG_S390_FLIC', if_true:
> files('s390_flic.c'))
> >>  specific_ss.add(when: 'CONFIG_S390_FLIC_KVM', if_true:
> files('s390_flic_kvm.c'))
> >>  specific_ss.add(when: 'CONFIG_SH_INTC', if_true: files('sh_intc.c'))
> >>  specific_ss.add(when: 'CONFIG_RISCV_ACLINT', if_true:
> files('riscv_aclint.c'))
> >> +specific_ss.add(when: 'CONFIG_RISCV_APLIC', if_true:
> files('riscv_aplic.c'))
> >>  specific_ss.add(when: 'CONFIG_SIFIVE_PLIC', if_true:
> files('sifive_plic.c'))
> >>  specific_ss.add(when: 'CONFIG_XICS', if_true: files('xics.c'))
> >>  specific_ss.add(when: ['CONFIG_KVM', 'CONFIG_XICS'],
> >> diff --git a/hw/intc/riscv_aplic.c b/hw/intc/riscv_aplic.c
> >> new file mode 100644
> >> index 0000000000..885c1de2af
> >> --- /dev/null
> >> +++ b/hw/intc/riscv_aplic.c
> >> @@ -0,0 +1,975 @@
> >> +/*
> >> + * RISC-V APLIC (Advanced Platform Level Interrupt Controller)
> >> + *
> >> + * Copyright (c) 2021 Western Digital Corporation or its affiliates.
> >> + *
> >> + * This program is free software; you can redistribute it and/or
> modify it
> >> + * under the terms and conditions of the GNU General Public License,
> >> + * version 2 or later, as published by the Free Software Foundation.
> >> + *
> >> + * This program is distributed in the hope it will be useful, but
> WITHOUT
> >> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
> or
> >> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public
> License for
> >> + * more details.
> >> + *
> >> + * You should have received a copy of the GNU General Public License
> along with
> >> + * this program.  If not, see <http://www.gnu.org/licenses/>.
> >> + */
> >> +
> >> +#include "qemu/osdep.h"
> >> +#include "qapi/error.h"
> >> +#include "qemu/log.h"
> >> +#include "qemu/module.h"
> >> +#include "qemu/error-report.h"
> >> +#include "qemu/bswap.h"
> >> +#include "exec/address-spaces.h"
> >> +#include "hw/sysbus.h"
> >> +#include "hw/pci/msi.h"
> >> +#include "hw/boards.h"
> >> +#include "hw/qdev-properties.h"
> >> +#include "hw/intc/riscv_aplic.h"
> >> +#include "hw/irq.h"
> >> +#include "target/riscv/cpu.h"
> >> +#include "sysemu/sysemu.h"
> >> +#include "migration/vmstate.h"
> >> +
> >> +#define APLIC_MAX_IDC                  (1UL << 14)
> >> +#define APLIC_MAX_SOURCE               1024
> >> +#define APLIC_MIN_IPRIO_BITS           1
> >> +#define APLIC_MAX_IPRIO_BITS           8
> >> +#define APLIC_MAX_CHILDREN             1024
> >> +
> >> +#define APLIC_DOMAINCFG                0x0000
> >> +#define APLIC_DOMAINCFG_RDONLY         0x80000000
> >> +#define APLIC_DOMAINCFG_IE             (1 << 8)
> >> +#define APLIC_DOMAINCFG_DM             (1 << 2)
> >> +#define APLIC_DOMAINCFG_BE             (1 << 0)
> >> +
> >> +#define APLIC_SOURCECFG_BASE           0x0004
> >> +#define APLIC_SOURCECFG_D              (1 << 10)
> >> +#define APLIC_SOURCECFG_CHILDIDX_MASK  0x000003ff
> >> +#define APLIC_SOURCECFG_SM_MASK        0x00000007
> >> +#define APLIC_SOURCECFG_SM_INACTIVE    0x0
> >> +#define APLIC_SOURCECFG_SM_DETACH      0x1
> >> +#define APLIC_SOURCECFG_SM_EDGE_RISE   0x4
> >> +#define APLIC_SOURCECFG_SM_EDGE_FALL   0x5
> >> +#define APLIC_SOURCECFG_SM_LEVEL_HIGH  0x6
> >> +#define APLIC_SOURCECFG_SM_LEVEL_LOW   0x7
> >> +
> >> +#define APLIC_MMSICFGADDR              0x1bc0
> >> +#define APLIC_MMSICFGADDRH             0x1bc4
> >> +#define APLIC_SMSICFGADDR              0x1bc8
> >> +#define APLIC_SMSICFGADDRH             0x1bcc
> >> +
> >> +#define APLIC_xMSICFGADDRH_L           (1UL << 31)
> >> +#define APLIC_xMSICFGADDRH_HHXS_MASK   0x1f
> >> +#define APLIC_xMSICFGADDRH_HHXS_SHIFT  24
> >> +#define APLIC_xMSICFGADDRH_LHXS_MASK   0x7
> >> +#define APLIC_xMSICFGADDRH_LHXS_SHIFT  20
> >> +#define APLIC_xMSICFGADDRH_HHXW_MASK   0x7
> >> +#define APLIC_xMSICFGADDRH_HHXW_SHIFT  16
> >> +#define APLIC_xMSICFGADDRH_LHXW_MASK   0xf
> >> +#define APLIC_xMSICFGADDRH_LHXW_SHIFT  12
> >> +#define APLIC_xMSICFGADDRH_BAPPN_MASK  0xfff
> >> +
> >> +#define APLIC_xMSICFGADDR_PPN_SHIFT    12
> >> +
> >> +#define APLIC_xMSICFGADDR_PPN_HART(__lhxs) \
> >> +    ((1UL << (__lhxs)) - 1)
> >> +
> >> +#define APLIC_xMSICFGADDR_PPN_LHX_MASK(__lhxw) \
> >> +    ((1UL << (__lhxw)) - 1)
> >> +#define APLIC_xMSICFGADDR_PPN_LHX_SHIFT(__lhxs) \
> >> +    ((__lhxs))
> >> +#define APLIC_xMSICFGADDR_PPN_LHX(__lhxw, __lhxs) \
> >> +    (APLIC_xMSICFGADDR_PPN_LHX_MASK(__lhxw) << \
> >> +     APLIC_xMSICFGADDR_PPN_LHX_SHIFT(__lhxs))
> >> +
> >> +#define APLIC_xMSICFGADDR_PPN_HHX_MASK(__hhxw) \
> >> +    ((1UL << (__hhxw)) - 1)
> >> +#define APLIC_xMSICFGADDR_PPN_HHX_SHIFT(__hhxs) \
> >> +    ((__hhxs) + APLIC_xMSICFGADDR_PPN_SHIFT)
> >> +#define APLIC_xMSICFGADDR_PPN_HHX(__hhxw, __hhxs) \
> >> +    (APLIC_xMSICFGADDR_PPN_HHX_MASK(__hhxw) << \
> >> +     APLIC_xMSICFGADDR_PPN_HHX_SHIFT(__hhxs))
> >> +
> >> +#define APLIC_xMSICFGADDRH_VALID_MASK   \
> >> +    (APLIC_xMSICFGADDRH_L | \
> >> +     (APLIC_xMSICFGADDRH_HHXS_MASK << APLIC_xMSICFGADDRH_HHXS_SHIFT) |
> \
> >> +     (APLIC_xMSICFGADDRH_LHXS_MASK << APLIC_xMSICFGADDRH_LHXS_SHIFT) |
> \
> >> +     (APLIC_xMSICFGADDRH_HHXW_MASK << APLIC_xMSICFGADDRH_HHXW_SHIFT) |
> \
> >> +     (APLIC_xMSICFGADDRH_LHXW_MASK << APLIC_xMSICFGADDRH_LHXW_SHIFT) |
> \
> >> +     APLIC_xMSICFGADDRH_BAPPN_MASK)
> >> +
> >> +#define APLIC_SETIP_BASE               0x1c00
> >> +#define APLIC_SETIPNUM                 0x1cdc
> >> +
> >> +#define APLIC_CLRIP_BASE               0x1d00
> >> +#define APLIC_CLRIPNUM                 0x1ddc
> >> +
> >> +#define APLIC_SETIE_BASE               0x1e00
> >> +#define APLIC_SETIENUM                 0x1edc
> >> +
> >> +#define APLIC_CLRIE_BASE               0x1f00
> >> +#define APLIC_CLRIENUM                 0x1fdc
> >> +
> >> +#define APLIC_SETIPNUM_LE              0x2000
> >> +#define APLIC_SETIPNUM_BE              0x2004
> >> +
> >> +#define APLIC_ISTATE_PENDING           (1U << 0)
> >> +#define APLIC_ISTATE_ENABLED           (1U << 1)
> >> +#define APLIC_ISTATE_ENPEND            (APLIC_ISTATE_ENABLED | \
> >> +                                        APLIC_ISTATE_PENDING)
> >> +#define APLIC_ISTATE_INPUT             (1U << 8)
> >> +
> >> +#define APLIC_GENMSI                   0x3000
> >> +
> >> +#define APLIC_TARGET_BASE              0x3004
> >> +#define APLIC_TARGET_HART_IDX_SHIFT    18
> >> +#define APLIC_TARGET_HART_IDX_MASK     0x3fff
> >> +#define APLIC_TARGET_GUEST_IDX_SHIFT   12
> >> +#define APLIC_TARGET_GUEST_IDX_MASK    0x3f
> >> +#define APLIC_TARGET_IPRIO_MASK        0xff
> >> +#define APLIC_TARGET_EIID_MASK         0x7ff
> >> +
> >> +#define APLIC_IDC_BASE                 0x4000
> >> +#define APLIC_IDC_SIZE                 32
> >> +
> >> +#define APLIC_IDC_IDELIVERY            0x00
> >> +
> >> +#define APLIC_IDC_IFORCE               0x04
> >> +
> >> +#define APLIC_IDC_ITHRESHOLD           0x08
> >> +
> >> +#define APLIC_IDC_TOPI                 0x18
> >> +#define APLIC_IDC_TOPI_ID_SHIFT        16
> >> +#define APLIC_IDC_TOPI_ID_MASK         0x3ff
> >> +#define APLIC_IDC_TOPI_PRIO_MASK       0xff
> >> +
> >> +#define APLIC_IDC_CLAIMI               0x1c
> >> +
> >> +static uint32_t riscv_aplic_read_input_word(RISCVAPLICState *aplic,
> >> +                                            uint32_t word)
> >> +{
> >> +    uint32_t i, irq, ret = 0;
> >> +
> >> +    for (i = 0; i < 32; i++) {
> >> +        irq = word * 32 + i;
> >> +        if (!irq || aplic->num_irqs <= irq) {
> >> +            continue;
> >> +        }
> >> +
> >> +        ret |= ((aplic->state[irq] & APLIC_ISTATE_INPUT) ? 1 : 0) << i;
> >> +    }
> >> +
> >> +    return ret;
> >> +}
> >> +
> >> +static uint32_t riscv_aplic_read_pending_word(RISCVAPLICState *aplic,
> >> +                                              uint32_t word)
> >> +{
> >> +    uint32_t i, irq, ret = 0;
> >> +
> >> +    for (i = 0; i < 32; i++) {
> >> +        irq = word * 32 + i;
> >> +        if (!irq || aplic->num_irqs <= irq) {
> >> +            continue;
> >> +        }
> >> +
> >> +        ret |= ((aplic->state[irq] & APLIC_ISTATE_PENDING) ? 1 : 0) <<
> i;
> >> +    }
> >> +
> >> +    return ret;
> >> +}
> >> +
> >> +static void riscv_aplic_set_pending_raw(RISCVAPLICState *aplic,
> >> +                                        uint32_t irq, bool pending)
> >> +{
> >> +    if (pending) {
> >> +        aplic->state[irq] |= APLIC_ISTATE_PENDING;
> >> +    } else {
> >> +        aplic->state[irq] &= ~APLIC_ISTATE_PENDING;
> >> +    }
> >> +}
> >> +
> >> +static void riscv_aplic_set_pending(RISCVAPLICState *aplic,
> >> +                                    uint32_t irq, bool pending)
> >> +{
> >> +    uint32_t sourcecfg, sm;
> >> +
> >> +    if ((irq <= 0) || (aplic->num_irqs <= irq)) {
> >> +        return;
> >> +    }
> >> +
> >> +    sourcecfg = aplic->sourcecfg[irq];
> >> +    if (sourcecfg & APLIC_SOURCECFG_D) {
> >> +        return;
> >> +    }
> >> +
> >> +    sm = sourcecfg & APLIC_SOURCECFG_SM_MASK;
> >> +    if ((sm == APLIC_SOURCECFG_SM_INACTIVE) ||
> >> +        ((!aplic->msimode || (aplic->msimode && !pending)) &&
> >> +         ((sm == APLIC_SOURCECFG_SM_LEVEL_HIGH) ||
> >> +          (sm == APLIC_SOURCECFG_SM_LEVEL_LOW)))) {
> >> +        return;
> >> +    }
> >> +
> >> +    riscv_aplic_set_pending_raw(aplic, irq, pending);
> >> +}
> >> +
> >> +static void riscv_aplic_set_pending_word(RISCVAPLICState *aplic,
> >> +                                         uint32_t word, uint32_t value,
> >> +                                         bool pending)
> >> +{
> >> +    uint32_t i, irq;
> >> +
> >> +    for (i = 0; i < 32; i++) {
> >> +        irq = word * 32 + i;
> >> +        if (!irq || aplic->num_irqs <= irq) {
> >> +            continue;
> >> +        }
> >> +
> >> +        if (value & (1U << i)) {
> >> +            riscv_aplic_set_pending(aplic, irq, pending);
> >> +        }
> >> +    }
> >> +}
> >> +
> >> +static uint32_t riscv_aplic_read_enabled_word(RISCVAPLICState *aplic,
> >> +                                              int word)
> >> +{
> >> +    uint32_t i, irq, ret = 0;
> >> +
> >> +    for (i = 0; i < 32; i++) {
> >> +        irq = word * 32 + i;
> >> +        if (!irq || aplic->num_irqs <= irq) {
> >> +            continue;
> >> +        }
> >> +
> >> +        ret |= ((aplic->state[irq] & APLIC_ISTATE_ENABLED) ? 1 : 0) <<
> i;
> >> +    }
> >> +
> >> +    return ret;
> >> +}
> >> +
> >> +static void riscv_aplic_set_enabled_raw(RISCVAPLICState *aplic,
> >> +                                        uint32_t irq, bool enabled)
> >> +{
> >> +    if (enabled) {
> >> +        aplic->state[irq] |= APLIC_ISTATE_ENABLED;
> >> +    } else {
> >> +        aplic->state[irq] &= ~APLIC_ISTATE_ENABLED;
> >> +    }
> >> +}
> >> +
> >> +static void riscv_aplic_set_enabled(RISCVAPLICState *aplic,
> >> +                                    uint32_t irq, bool enabled)
> >> +{
> >> +    uint32_t sourcecfg, sm;
> >> +
> >> +    if ((irq <= 0) || (aplic->num_irqs <= irq)) {
> >> +        return;
> >> +    }
> >> +
> >> +    sourcecfg = aplic->sourcecfg[irq];
> >> +    if (sourcecfg & APLIC_SOURCECFG_D) {
> >> +        return;
> >> +    }
> >> +
> >> +    sm = sourcecfg & APLIC_SOURCECFG_SM_MASK;
> >> +    if (sm == APLIC_SOURCECFG_SM_INACTIVE) {
> >> +        return;
> >> +    }
> >> +
> >> +    riscv_aplic_set_enabled_raw(aplic, irq, enabled);
> >> +}
> >> +
> >> +static void riscv_aplic_set_enabled_word(RISCVAPLICState *aplic,
> >> +                                         uint32_t word, uint32_t value,
> >> +                                         bool enabled)
> >> +{
> >> +    uint32_t i, irq;
> >> +
> >> +    for (i = 0; i < 32; i++) {
> >> +        irq = word * 32 + i;
> >> +        if (!irq || aplic->num_irqs <= irq) {
> >> +            continue;
> >> +        }
> >> +
> >> +        if (value & (1U << i)) {
> >> +            riscv_aplic_set_enabled(aplic, irq, enabled);
> >> +        }
> >> +    }
> >> +}
> >> +
> >> +static void riscv_aplic_msi_send(RISCVAPLICState *aplic,
> >> +                                 uint32_t hart_idx, uint32_t guest_idx,
> >> +                                 uint32_t eiid)
> >> +{
> >> +    uint64_t addr;
> >> +    MemTxResult result;
> >> +    RISCVAPLICState *aplic_m;
> >> +    uint32_t lhxs, lhxw, hhxs, hhxw, group_idx, msicfgaddr,
> msicfgaddrH;
> >> +
> >> +    aplic_m = aplic;
> >> +    while (aplic_m && !aplic_m->mmode) {
> >> +        aplic_m = aplic_m->parent;
> >> +    }
> >> +    if (!aplic_m) {
> >> +        qemu_log_mask(LOG_GUEST_ERROR, "%s: m-level APLIC not found\n",
> >> +                      __func__);
> >> +        return;
> >> +    }
> >> +
> >> +    if (aplic->mmode) {
> >> +        msicfgaddr = aplic_m->mmsicfgaddr;
> >> +        msicfgaddrH = aplic_m->mmsicfgaddrH;
> >> +    } else {
> >> +        msicfgaddr = aplic_m->smsicfgaddr;
> >> +        msicfgaddrH = aplic_m->smsicfgaddrH;
> >> +    }
> >> +
> >> +    lhxs = (msicfgaddrH >> APLIC_xMSICFGADDRH_LHXS_SHIFT) &
> >> +            APLIC_xMSICFGADDRH_LHXS_MASK;
> >> +    lhxw = (msicfgaddrH >> APLIC_xMSICFGADDRH_LHXW_SHIFT) &
> >> +            APLIC_xMSICFGADDRH_LHXW_MASK;
> >> +    hhxs = (msicfgaddrH >> APLIC_xMSICFGADDRH_HHXS_SHIFT) &
> >> +            APLIC_xMSICFGADDRH_HHXS_MASK;
> >> +    hhxw = (msicfgaddrH >> APLIC_xMSICFGADDRH_HHXW_SHIFT) &
> >> +            APLIC_xMSICFGADDRH_HHXW_MASK;
> >> +
> >> +    group_idx = hart_idx >> lhxw;
> >> +    hart_idx &= APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw);
> >> +
> >> +    addr = msicfgaddr;
> >> +    addr |= ((uint64_t)(msicfgaddrH & APLIC_xMSICFGADDRH_BAPPN_MASK))
> << 32;
> >> +    addr |= ((uint64_t)(group_idx &
> APLIC_xMSICFGADDR_PPN_HHX_MASK(hhxw))) <<
> >> +             APLIC_xMSICFGADDR_PPN_HHX_SHIFT(hhxs);
> >> +    addr |= ((uint64_t)(hart_idx &
> APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw))) <<
> >> +             APLIC_xMSICFGADDR_PPN_LHX_SHIFT(lhxs);
> >> +    addr |= (uint64_t)(guest_idx & APLIC_xMSICFGADDR_PPN_HART(lhxs));
> >> +    addr <<= APLIC_xMSICFGADDR_PPN_SHIFT;
> >> +
> >> +    address_space_stl_le(&address_space_memory, addr,
> >> +                         eiid, MEMTXATTRS_UNSPECIFIED, &result);
> >> +    if (result != MEMTX_OK) {
> >> +        qemu_log_mask(LOG_GUEST_ERROR, "%s: MSI write failed for "
> >> +                      "hart_index=%d guest_index=%d eiid=%d\n",
> >> +                      __func__, hart_idx, guest_idx, eiid);
> >> +    }
> >> +}
> >> +
> >> +static void riscv_aplic_msi_irq_update(RISCVAPLICState *aplic,
> uint32_t irq)
> >> +{
> >> +    uint32_t hart_idx, guest_idx, eiid;
> >> +
> >> +    if (!aplic->msimode || (aplic->num_irqs <= irq) ||
> >> +        !(aplic->domaincfg & APLIC_DOMAINCFG_IE)) {
> >> +        return;
> >> +    }
> >> +
> >> +    if ((aplic->state[irq] & APLIC_ISTATE_ENPEND) !=
> APLIC_ISTATE_ENPEND) {
> >> +        return;
> >> +    }
> >> +
> >> +    riscv_aplic_set_pending_raw(aplic, irq, false);
> >> +
> >> +    hart_idx = aplic->target[irq] >> APLIC_TARGET_HART_IDX_SHIFT;
> >> +    hart_idx &= APLIC_TARGET_HART_IDX_MASK;
> >
> >
> > Hi Anup,
> >
> > I'm wondering does the hart_idx here has to use Machine-level hart index
> > for Supervisor-level interrupt domain?
> >
> > According to AIA spec for supervisor-level domin (Section 4.8.1):
> >   For a supervisor-level domain, "if" MSI target addresses are
> determined by the parent domain’s
> >   smsiaddrcfg and smsiaddrcfgh registers, then to construct the address
> for an outgoing MSI for
> >   interrupt source i, the Hart Index from register target[i] must first
> be converted into the index
> >   that the machine-level parent domain uses for the same hart.
> (Typically, these numbers are the
> >   same, but they may not be.) The address for the MSI is then computed
> using this "machine-level"
> >   hart index, the parent domain’s smsiaddrcfg and smsiaddrcfgh, and the
> Guest Index value from
> >   target[i].
> >
> > The description of converting target[i] hart Index to machine-level hart
> index in the spec
> > is quite vague to me.
>
> The "Section 4.3 Hart index numbers" states the following:
>
>     The index number a domain associates with a hart may or may not
> have any relationship
>     to the unique hart identifier (“hart ID”) that the RISC-V
> Privileged Architecture assigns to
>     the hart.
>
> For APLIC direct mode, if N harts are associated with a given domain
> then APLIC associates
> hart index from 0 to N-1 which are basically APLIC IDC numbers. In
> this case, the APLIC
> hart index is not related to "machine-level" hart index so software
> has to discover it from the
> "interrupt-extended" DT property (or similar for ACPI).
>
> For APLIC msi mode, the APLIC can basically target any HART using
> information in
> [m|s]msiaddrcfg registers and interrupt target registers. In this
> case, the APLIC hart
> index in the APLIC target register is a combination of IMSIC group
> index (g) and IMSIC
> hart index (h) and the APLIC target register also provides IMSIC guest
> index. In other
> words, the APLIC hart index for msi mode is also not related to the
> "machine-level" hart
> index. The APLIC driver will figure-out/extract the IMSIC group index
> (g), IMSIC hart
> index (h), and IMSIC guest index from target MSI address when Linux
> tries to write
> target MSI address for a particular APLIC interrupt.
>
> Regards,
> Anup
>

Hi Anup,

Thanks for the detailed explanation.

I think it's the formula in the AIA spec which confused me:
  g = (machine-level hart index >> LHXW) & (2^HHXW − 1)
  h = machine-level hart index & (2^LHXW − 1)
  MSI address =  (Base PPN | (g << (HHXS + 12)) | (h << LHXS) | Guest
Index) << 12

The term "machine-level hart index" makes me thought that
we should use the hart index from parent machine-level interrupt domain.

So it's driver's responsibility to figure out the correct IMSIC hart index,
guest index
it wants to forward the MSI to and configure them to the target[i] register?

From hardware (QEMU)'s perspective, it just needs to extract
the Hart index, Guest index out from *target[irq]* register in IRQ's
interrupt domain
and calculate the MSI address with the PPN from parent interrupt domain's
m/smsiaddrcfg and m/smsiaddrcfgh?

Regards,
Frank Chang


>
> >
> > Regards,
> > Frank Chang
> >
> >>
> >> +    if (aplic->mmode) {
> >> +        /* M-level APLIC ignores guest_index */
> >> +        guest_idx = 0;
> >> +    } else {
> >> +        guest_idx = aplic->target[irq] >> APLIC_TARGET_GUEST_IDX_SHIFT;
> >> +        guest_idx &= APLIC_TARGET_GUEST_IDX_MASK;
> >> +    }
> >> +    eiid = aplic->target[irq] & APLIC_TARGET_EIID_MASK;
> >> +    riscv_aplic_msi_send(aplic, hart_idx, guest_idx, eiid);
> >> +}
> >> +
> >> +static uint32_t riscv_aplic_idc_topi(RISCVAPLICState *aplic, uint32_t
> idc)
> >> +{
> >> +    uint32_t best_irq, best_iprio;
> >> +    uint32_t irq, iprio, ihartidx, ithres;
> >> +
> >> +    if (aplic->num_harts <= idc) {
> >> +        return 0;
> >> +    }
> >> +
> >> +    ithres = aplic->ithreshold[idc];
> >> +    best_irq = best_iprio = UINT32_MAX;
> >> +    for (irq = 1; irq < aplic->num_irqs; irq++) {
> >> +        if ((aplic->state[irq] & APLIC_ISTATE_ENPEND) !=
> >> +            APLIC_ISTATE_ENPEND) {
> >> +            continue;
> >> +        }
> >> +
> >> +        ihartidx = aplic->target[irq] >> APLIC_TARGET_HART_IDX_SHIFT;
> >> +        ihartidx &= APLIC_TARGET_HART_IDX_MASK;
> >> +        if (ihartidx != idc) {
> >> +            continue;
> >> +        }
> >> +
> >> +        iprio = aplic->target[irq] & aplic->iprio_mask;
> >> +        if (ithres && iprio >= ithres) {
> >> +            continue;
> >> +        }
> >> +
> >> +        if (iprio < best_iprio) {
> >> +            best_irq = irq;
> >> +            best_iprio = iprio;
> >> +        }
> >> +    }
> >> +
> >> +    if (best_irq < aplic->num_irqs && best_iprio <= aplic->iprio_mask)
> {
> >> +        return (best_irq << APLIC_IDC_TOPI_ID_SHIFT) | best_iprio;
> >> +    }
> >> +
> >> +    return 0;
> >> +}
> >> +
> >> +static void riscv_aplic_idc_update(RISCVAPLICState *aplic, uint32_t
> idc)
> >> +{
> >> +    uint32_t topi;
> >> +
> >> +    if (aplic->msimode || aplic->num_harts <= idc) {
> >> +        return;
> >> +    }
> >> +
> >> +    topi = riscv_aplic_idc_topi(aplic, idc);
> >> +    if ((aplic->domaincfg & APLIC_DOMAINCFG_IE) &&
> >> +        aplic->idelivery[idc] &&
> >> +        (aplic->iforce[idc] || topi)) {
> >> +        qemu_irq_raise(aplic->external_irqs[idc]);
> >> +    } else {
> >> +        qemu_irq_lower(aplic->external_irqs[idc]);
> >> +    }
> >> +}
> >> +
> >> +static uint32_t riscv_aplic_idc_claimi(RISCVAPLICState *aplic,
> uint32_t idc)
> >> +{
> >> +    uint32_t irq, state, sm, topi = riscv_aplic_idc_topi(aplic, idc);
> >> +
> >> +    if (!topi) {
> >> +        aplic->iforce[idc] = 0;
> >> +        return 0;
> >> +    }
> >> +
> >> +    irq = (topi >> APLIC_IDC_TOPI_ID_SHIFT) & APLIC_IDC_TOPI_ID_MASK;
> >> +    sm = aplic->sourcecfg[irq] & APLIC_SOURCECFG_SM_MASK;
> >> +    state = aplic->state[irq];
> >> +    riscv_aplic_set_pending_raw(aplic, irq, false);
> >> +    if ((sm == APLIC_SOURCECFG_SM_LEVEL_HIGH) &&
> >> +        (state & APLIC_ISTATE_INPUT)) {
> >> +        riscv_aplic_set_pending_raw(aplic, irq, true);
> >> +    } else if ((sm == APLIC_SOURCECFG_SM_LEVEL_LOW) &&
> >> +               !(state & APLIC_ISTATE_INPUT)) {
> >> +        riscv_aplic_set_pending_raw(aplic, irq, true);
> >> +    }
> >> +    riscv_aplic_idc_update(aplic, idc);
> >> +
> >> +    return topi;
> >> +}
> >> +
> >> +static void riscv_aplic_request(void *opaque, int irq, int level)
> >> +{
> >> +    bool update = false;
> >> +    RISCVAPLICState *aplic = opaque;
> >> +    uint32_t sourcecfg, childidx, state, idc;
> >> +
> >> +    assert((0 < irq) && (irq < aplic->num_irqs));
> >> +
> >> +    sourcecfg = aplic->sourcecfg[irq];
> >> +    if (sourcecfg & APLIC_SOURCECFG_D) {
> >> +        childidx = sourcecfg & APLIC_SOURCECFG_CHILDIDX_MASK;
> >> +        if (childidx < aplic->num_children) {
> >> +            riscv_aplic_request(aplic->children[childidx], irq, level);
> >> +        }
> >> +        return;
> >> +    }
> >> +
> >> +    state = aplic->state[irq];
> >> +    switch (sourcecfg & APLIC_SOURCECFG_SM_MASK) {
> >> +    case APLIC_SOURCECFG_SM_EDGE_RISE:
> >> +        if ((level > 0) && !(state & APLIC_ISTATE_INPUT) &&
> >> +            !(state & APLIC_ISTATE_PENDING)) {
> >> +            riscv_aplic_set_pending_raw(aplic, irq, true);
> >> +            update = true;
> >> +        }
> >> +        break;
> >> +    case APLIC_SOURCECFG_SM_EDGE_FALL:
> >> +        if ((level <= 0) && (state & APLIC_ISTATE_INPUT) &&
> >> +            !(state & APLIC_ISTATE_PENDING)) {
> >> +            riscv_aplic_set_pending_raw(aplic, irq, true);
> >> +            update = true;
> >> +        }
> >> +        break;
> >> +    case APLIC_SOURCECFG_SM_LEVEL_HIGH:
> >> +        if ((level > 0) && !(state & APLIC_ISTATE_PENDING)) {
> >> +            riscv_aplic_set_pending_raw(aplic, irq, true);
> >> +            update = true;
> >> +        }
> >> +        break;
> >> +    case APLIC_SOURCECFG_SM_LEVEL_LOW:
> >> +        if ((level <= 0) && !(state & APLIC_ISTATE_PENDING)) {
> >> +            riscv_aplic_set_pending_raw(aplic, irq, true);
> >> +            update = true;
> >> +        }
> >> +        break;
> >> +    default:
> >> +        break;
> >> +    }
> >> +
> >> +    if (level <= 0) {
> >> +        aplic->state[irq] &= ~APLIC_ISTATE_INPUT;
> >> +    } else {
> >> +        aplic->state[irq] |= APLIC_ISTATE_INPUT;
> >> +    }
> >> +
> >> +    if (update) {
> >> +        if (aplic->msimode) {
> >> +            riscv_aplic_msi_irq_update(aplic, irq);
> >> +        } else {
> >> +            idc = aplic->target[irq] >> APLIC_TARGET_HART_IDX_SHIFT;
> >> +            idc &= APLIC_TARGET_HART_IDX_MASK;
> >> +            riscv_aplic_idc_update(aplic, idc);
> >> +        }
> >> +    }
> >> +}
> >> +
> >> +static uint64_t riscv_aplic_read(void *opaque, hwaddr addr, unsigned
> size)
> >> +{
> >> +    uint32_t irq, word, idc;
> >> +    RISCVAPLICState *aplic = opaque;
> >> +
> >> +    /* Reads must be 4 byte words */
> >> +    if ((addr & 0x3) != 0) {
> >> +        goto err;
> >> +    }
> >> +
> >> +    if (addr == APLIC_DOMAINCFG) {
> >> +        return APLIC_DOMAINCFG_RDONLY | aplic->domaincfg |
> >> +               (aplic->msimode ? APLIC_DOMAINCFG_DM : 0);
> >> +    } else if ((APLIC_SOURCECFG_BASE <= addr) &&
> >> +            (addr < (APLIC_SOURCECFG_BASE + (aplic->num_irqs - 1) *
> 4))) {
> >> +        irq  = ((addr - APLIC_SOURCECFG_BASE) >> 2) + 1;
> >> +        return aplic->sourcecfg[irq];
> >> +    } else if (aplic->mmode && aplic->msimode &&
> >> +               (addr == APLIC_MMSICFGADDR)) {
> >> +        return aplic->mmsicfgaddr;
> >> +    } else if (aplic->mmode && aplic->msimode &&
> >> +               (addr == APLIC_MMSICFGADDRH)) {
> >> +        return aplic->mmsicfgaddrH;
> >> +    } else if (aplic->mmode && aplic->msimode &&
> >> +               (addr == APLIC_SMSICFGADDR)) {
> >> +        /* Registers SMSICFGADDR and SMSICFGADDRH are implemented only
> if:
> >> +         * (a) the interrupt domain is at machine level
> >> +         * (b) the domain’s harts implement supervisor mode
> >> +         * (c) the domain has one or more child supervisor-level
> domains
> >> +         *     that support MSI delivery mode (domaincfg.DM is not
> read-
> >> +         *     only zero in at least one of the supervisor-level child
> >> +         * domains).
> >> +         */
> >> +        return (aplic->num_children) ? aplic->smsicfgaddr : 0;
> >> +    } else if (aplic->mmode && aplic->msimode &&
> >> +               (addr == APLIC_SMSICFGADDRH)) {
> >> +        return (aplic->num_children) ? aplic->smsicfgaddrH : 0;
> >> +    } else if ((APLIC_SETIP_BASE <= addr) &&
> >> +            (addr < (APLIC_SETIP_BASE + aplic->bitfield_words * 4))) {
> >> +        word = (addr - APLIC_SETIP_BASE) >> 2;
> >> +        return riscv_aplic_read_pending_word(aplic, word);
> >> +    } else if (addr == APLIC_SETIPNUM) {
> >> +        return 0;
> >> +    } else if ((APLIC_CLRIP_BASE <= addr) &&
> >> +            (addr < (APLIC_CLRIP_BASE + aplic->bitfield_words * 4))) {
> >> +        word = (addr - APLIC_CLRIP_BASE) >> 2;
> >> +        return riscv_aplic_read_input_word(aplic, word);
> >> +    } else if (addr == APLIC_CLRIPNUM) {
> >> +        return 0;
> >> +    } else if ((APLIC_SETIE_BASE <= addr) &&
> >> +            (addr < (APLIC_SETIE_BASE + aplic->bitfield_words * 4))) {
> >> +        word = (addr - APLIC_SETIE_BASE) >> 2;
> >> +        return riscv_aplic_read_enabled_word(aplic, word);
> >> +    } else if (addr == APLIC_SETIENUM) {
> >> +        return 0;
> >> +    } else if ((APLIC_CLRIE_BASE <= addr) &&
> >> +            (addr < (APLIC_CLRIE_BASE + aplic->bitfield_words * 4))) {
> >> +        return 0;
> >> +    } else if (addr == APLIC_CLRIENUM) {
> >> +        return 0;
> >> +    } else if (addr == APLIC_SETIPNUM_LE) {
> >> +        return 0;
> >> +    } else if (addr == APLIC_SETIPNUM_BE) {
> >> +        return 0;
> >> +    } else if (addr == APLIC_GENMSI) {
> >> +        return (aplic->msimode) ? aplic->genmsi : 0;
> >> +    } else if ((APLIC_TARGET_BASE <= addr) &&
> >> +            (addr < (APLIC_TARGET_BASE + (aplic->num_irqs - 1) * 4))) {
> >> +        irq = ((addr - APLIC_TARGET_BASE) >> 2) + 1;
> >> +        return aplic->target[irq];
> >> +    } else if (!aplic->msimode && (APLIC_IDC_BASE <= addr) &&
> >> +            (addr < (APLIC_IDC_BASE + aplic->num_harts *
> APLIC_IDC_SIZE))) {
> >> +        idc = (addr - APLIC_IDC_BASE) / APLIC_IDC_SIZE;
> >> +        switch (addr - (APLIC_IDC_BASE + idc * APLIC_IDC_SIZE)) {
> >> +        case APLIC_IDC_IDELIVERY:
> >> +            return aplic->idelivery[idc];
> >> +        case APLIC_IDC_IFORCE:
> >> +            return aplic->iforce[idc];
> >> +        case APLIC_IDC_ITHRESHOLD:
> >> +            return aplic->ithreshold[idc];
> >> +        case APLIC_IDC_TOPI:
> >> +            return riscv_aplic_idc_topi(aplic, idc);
> >> +        case APLIC_IDC_CLAIMI:
> >> +            return riscv_aplic_idc_claimi(aplic, idc);
> >> +        default:
> >> +            goto err;
> >> +        };
> >> +    }
> >> +
> >> +err:
> >> +    qemu_log_mask(LOG_GUEST_ERROR,
> >> +                  "%s: Invalid register read 0x%" HWADDR_PRIx "\n",
> >> +                  __func__, addr);
> >> +    return 0;
> >> +}
> >> +
> >> +static void riscv_aplic_write(void *opaque, hwaddr addr, uint64_t
> value,
> >> +        unsigned size)
> >> +{
> >> +    RISCVAPLICState *aplic = opaque;
> >> +    uint32_t irq, word, idc = UINT32_MAX;
> >> +
> >> +    /* Writes must be 4 byte words */
> >> +    if ((addr & 0x3) != 0) {
> >> +        goto err;
> >> +    }
> >> +
> >> +    if (addr == APLIC_DOMAINCFG) {
> >> +        /* Only IE bit writeable at the moment */
> >> +        value &= APLIC_DOMAINCFG_IE;
> >> +        aplic->domaincfg = value;
> >> +    } else if ((APLIC_SOURCECFG_BASE <= addr) &&
> >> +            (addr < (APLIC_SOURCECFG_BASE + (aplic->num_irqs - 1) *
> 4))) {
> >> +        irq  = ((addr - APLIC_SOURCECFG_BASE) >> 2) + 1;
> >> +        if (!aplic->num_children && (value & APLIC_SOURCECFG_D)) {
> >> +            value = 0;
> >> +        }
> >> +        if (value & APLIC_SOURCECFG_D) {
> >> +            value &= (APLIC_SOURCECFG_D |
> APLIC_SOURCECFG_CHILDIDX_MASK);
> >> +        } else {
> >> +            value &= (APLIC_SOURCECFG_D | APLIC_SOURCECFG_SM_MASK);
> >> +        }
> >> +        aplic->sourcecfg[irq] = value;
> >> +        if ((aplic->sourcecfg[irq] & APLIC_SOURCECFG_D) ||
> >> +            (aplic->sourcecfg[irq] == 0)) {
> >> +            riscv_aplic_set_pending_raw(aplic, irq, false);
> >> +            riscv_aplic_set_enabled_raw(aplic, irq, false);
> >> +        }
> >> +    } else if (aplic->mmode && aplic->msimode &&
> >> +               (addr == APLIC_MMSICFGADDR)) {
> >> +        if (!(aplic->mmsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
> >> +            aplic->mmsicfgaddr = value;
> >> +        }
> >> +    } else if (aplic->mmode && aplic->msimode &&
> >> +               (addr == APLIC_MMSICFGADDRH)) {
> >> +        if (!(aplic->mmsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
> >> +            aplic->mmsicfgaddrH = value &
> APLIC_xMSICFGADDRH_VALID_MASK;
> >> +        }
> >> +    } else if (aplic->mmode && aplic->msimode &&
> >> +               (addr == APLIC_SMSICFGADDR)) {
> >> +        /* Registers SMSICFGADDR and SMSICFGADDRH are implemented only
> if:
> >> +         * (a) the interrupt domain is at machine level
> >> +         * (b) the domain’s harts implement supervisor mode
> >> +         * (c) the domain has one or more child supervisor-level
> domains
> >> +         *     that support MSI delivery mode (domaincfg.DM is not
> read-
> >> +         *     only zero in at least one of the supervisor-level child
> >> +         * domains).
> >> +         */
> >> +        if (aplic->num_children &&
> >> +            !(aplic->smsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
> >> +            aplic->smsicfgaddr = value;
> >> +        }
> >> +    } else if (aplic->mmode && aplic->msimode &&
> >> +               (addr == APLIC_SMSICFGADDRH)) {
> >> +        if (aplic->num_children &&
> >> +            !(aplic->smsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
> >> +            aplic->smsicfgaddrH = value &
> APLIC_xMSICFGADDRH_VALID_MASK;
> >> +        }
> >> +    } else if ((APLIC_SETIP_BASE <= addr) &&
> >> +            (addr < (APLIC_SETIP_BASE + aplic->bitfield_words * 4))) {
> >> +        word = (addr - APLIC_SETIP_BASE) >> 2;
> >> +        riscv_aplic_set_pending_word(aplic, word, value, true);
> >> +    } else if (addr == APLIC_SETIPNUM) {
> >> +        riscv_aplic_set_pending(aplic, value, true);
> >> +    } else if ((APLIC_CLRIP_BASE <= addr) &&
> >> +            (addr < (APLIC_CLRIP_BASE + aplic->bitfield_words * 4))) {
> >> +        word = (addr - APLIC_CLRIP_BASE) >> 2;
> >> +        riscv_aplic_set_pending_word(aplic, word, value, false);
> >> +    } else if (addr == APLIC_CLRIPNUM) {
> >> +        riscv_aplic_set_pending(aplic, value, false);
> >> +    } else if ((APLIC_SETIE_BASE <= addr) &&
> >> +            (addr < (APLIC_SETIE_BASE + aplic->bitfield_words * 4))) {
> >> +        word = (addr - APLIC_SETIE_BASE) >> 2;
> >> +        riscv_aplic_set_enabled_word(aplic, word, value, true);
> >> +    } else if (addr == APLIC_SETIENUM) {
> >> +        riscv_aplic_set_enabled(aplic, value, true);
> >> +    } else if ((APLIC_CLRIE_BASE <= addr) &&
> >> +            (addr < (APLIC_CLRIE_BASE + aplic->bitfield_words * 4))) {
> >> +        word = (addr - APLIC_CLRIE_BASE) >> 2;
> >> +        riscv_aplic_set_enabled_word(aplic, word, value, false);
> >> +    } else if (addr == APLIC_CLRIENUM) {
> >> +        riscv_aplic_set_enabled(aplic, value, false);
> >> +    } else if (addr == APLIC_SETIPNUM_LE) {
> >> +        riscv_aplic_set_pending(aplic, value, true);
> >> +    } else if (addr == APLIC_SETIPNUM_BE) {
> >> +        riscv_aplic_set_pending(aplic, bswap32(value), true);
> >> +    } else if (addr == APLIC_GENMSI) {
> >> +        if (aplic->msimode) {
> >> +            aplic->genmsi = value & ~(APLIC_TARGET_GUEST_IDX_MASK <<
> >> +                                      APLIC_TARGET_GUEST_IDX_SHIFT);
> >> +            riscv_aplic_msi_send(aplic,
> >> +                                 value >> APLIC_TARGET_HART_IDX_SHIFT,
> >> +                                 0,
> >> +                                 value & APLIC_TARGET_EIID_MASK);
> >> +        }
> >> +    } else if ((APLIC_TARGET_BASE <= addr) &&
> >> +            (addr < (APLIC_TARGET_BASE + (aplic->num_irqs - 1) * 4))) {
> >> +        irq = ((addr - APLIC_TARGET_BASE) >> 2) + 1;
> >> +        if (aplic->msimode) {
> >> +            aplic->target[irq] = value;
> >> +        } else {
> >> +            aplic->target[irq] = (value & ~APLIC_TARGET_IPRIO_MASK) |
> >> +                                 ((value & aplic->iprio_mask) ?
> >> +                                  (value & aplic->iprio_mask) : 1);
> >> +        }
> >> +    } else if (!aplic->msimode && (APLIC_IDC_BASE <= addr) &&
> >> +            (addr < (APLIC_IDC_BASE + aplic->num_harts *
> APLIC_IDC_SIZE))) {
> >> +        idc = (addr - APLIC_IDC_BASE) / APLIC_IDC_SIZE;
> >> +        switch (addr - (APLIC_IDC_BASE + idc * APLIC_IDC_SIZE)) {
> >> +        case APLIC_IDC_IDELIVERY:
> >> +            aplic->idelivery[idc] = value & 0x1;
> >> +            break;
> >> +        case APLIC_IDC_IFORCE:
> >> +            aplic->iforce[idc] = value & 0x1;
> >> +            break;
> >> +        case APLIC_IDC_ITHRESHOLD:
> >> +            aplic->ithreshold[idc] = value & aplic->iprio_mask;
> >> +            break;
> >> +        default:
> >> +            goto err;
> >> +        };
> >> +    } else {
> >> +        goto err;
> >> +    }
> >> +
> >> +    if (aplic->msimode) {
> >> +        for (irq = 1; irq < aplic->num_irqs; irq++) {
> >> +            riscv_aplic_msi_irq_update(aplic, irq);
> >> +        }
> >> +    } else {
> >> +        if (idc == UINT32_MAX) {
> >> +            for (idc = 0; idc < aplic->num_harts; idc++) {
> >> +                riscv_aplic_idc_update(aplic, idc);
> >> +            }
> >> +        } else {
> >> +            riscv_aplic_idc_update(aplic, idc);
> >> +        }
> >> +    }
> >> +
> >> +    return;
> >> +
> >> +err:
> >> +    qemu_log_mask(LOG_GUEST_ERROR,
> >> +                  "%s: Invalid register write 0x%" HWADDR_PRIx "\n",
> >> +                  __func__, addr);
> >> +}
> >> +
> >> +static const MemoryRegionOps riscv_aplic_ops = {
> >> +    .read = riscv_aplic_read,
> >> +    .write = riscv_aplic_write,
> >> +    .endianness = DEVICE_LITTLE_ENDIAN,
> >> +    .valid = {
> >> +        .min_access_size = 4,
> >> +        .max_access_size = 4
> >> +    }
> >> +};
> >> +
> >> +static void riscv_aplic_realize(DeviceState *dev, Error **errp)
> >> +{
> >> +    uint32_t i;
> >> +    RISCVAPLICState *aplic = RISCV_APLIC(dev);
> >> +
> >> +    aplic->bitfield_words = (aplic->num_irqs + 31) >> 5;
> >> +    aplic->sourcecfg = g_new0(uint32_t, aplic->num_irqs);
> >> +    aplic->state = g_new(uint32_t, aplic->num_irqs);
> >> +    aplic->target = g_new0(uint32_t, aplic->num_irqs);
> >> +    if (!aplic->msimode) {
> >> +        for (i = 0; i < aplic->num_irqs; i++) {
> >> +            aplic->target[i] = 1;
> >> +        }
> >> +    }
> >> +    aplic->idelivery = g_new0(uint32_t, aplic->num_harts);
> >> +    aplic->iforce = g_new0(uint32_t, aplic->num_harts);
> >> +    aplic->ithreshold = g_new0(uint32_t, aplic->num_harts);
> >> +
> >> +    memory_region_init_io(&aplic->mmio, OBJECT(dev), &riscv_aplic_ops,
> aplic,
> >> +                          TYPE_RISCV_APLIC, aplic->aperture_size);
> >> +    sysbus_init_mmio(SYS_BUS_DEVICE(dev), &aplic->mmio);
> >> +
> >> +    /* Only root APLICs have hardware IRQ lines. All non-root APLICs
> >> +     * have IRQ lines delegated by their parent APLIC.
> >> +     */
> >> +    if (!aplic->parent) {
> >> +        qdev_init_gpio_in(dev, riscv_aplic_request, aplic->num_irqs);
> >> +    }
> >> +
> >> +    /* Create output IRQ lines for non-MSI mode */
> >> +    if (!aplic->msimode) {
> >> +        aplic->external_irqs = g_malloc(sizeof(qemu_irq) *
> aplic->num_harts);
> >> +        qdev_init_gpio_out(dev, aplic->external_irqs,
> aplic->num_harts);
> >> +
> >> +        /* Claim the CPU interrupt to be triggered by this APLIC */
> >> +        for (i = 0; i < aplic->num_harts; i++) {
> >> +            RISCVCPU *cpu = RISCV_CPU(qemu_get_cpu(aplic->hartid_base
> + i));
> >> +            if (riscv_cpu_claim_interrupts(cpu,
> >> +                (aplic->mmode) ? MIP_MEIP : MIP_SEIP) < 0) {
> >> +                error_report("%s already claimed",
> >> +                             (aplic->mmode) ? "MEIP" : "SEIP");
> >> +                exit(1);
> >> +            }
> >> +        }
> >> +    }
> >> +
> >> +    msi_nonbroken = true;
> >> +}
> >> +
> >> +static Property riscv_aplic_properties[] = {
> >> +    DEFINE_PROP_UINT32("aperture-size", RISCVAPLICState,
> aperture_size, 0),
> >> +    DEFINE_PROP_UINT32("hartid-base", RISCVAPLICState, hartid_base, 0),
> >> +    DEFINE_PROP_UINT32("num-harts", RISCVAPLICState, num_harts, 0),
> >> +    DEFINE_PROP_UINT32("iprio-mask", RISCVAPLICState, iprio_mask, 0),
> >> +    DEFINE_PROP_UINT32("num-irqs", RISCVAPLICState, num_irqs, 0),
> >> +    DEFINE_PROP_BOOL("msimode", RISCVAPLICState, msimode, 0),
> >> +    DEFINE_PROP_BOOL("mmode", RISCVAPLICState, mmode, 0),
> >> +    DEFINE_PROP_END_OF_LIST(),
> >> +};
> >> +
> >> +static const VMStateDescription vmstate_riscv_aplic = {
> >> +    .name = "riscv_aplic",
> >> +    .version_id = 1,
> >> +    .minimum_version_id = 1,
> >> +    .fields = (VMStateField[]) {
> >> +            VMSTATE_UINT32(domaincfg, RISCVAPLICState),
> >> +            VMSTATE_UINT32(mmsicfgaddr, RISCVAPLICState),
> >> +            VMSTATE_UINT32(mmsicfgaddrH, RISCVAPLICState),
> >> +            VMSTATE_UINT32(smsicfgaddr, RISCVAPLICState),
> >> +            VMSTATE_UINT32(smsicfgaddrH, RISCVAPLICState),
> >> +            VMSTATE_UINT32(genmsi, RISCVAPLICState),
> >> +            VMSTATE_VARRAY_UINT32(sourcecfg, RISCVAPLICState,
> >> +                                  num_irqs, 0,
> >> +                                  vmstate_info_uint32, uint32_t),
> >> +            VMSTATE_VARRAY_UINT32(state, RISCVAPLICState,
> >> +                                  num_irqs, 0,
> >> +                                  vmstate_info_uint32, uint32_t),
> >> +            VMSTATE_VARRAY_UINT32(target, RISCVAPLICState,
> >> +                                  num_irqs, 0,
> >> +                                  vmstate_info_uint32, uint32_t),
> >> +            VMSTATE_VARRAY_UINT32(idelivery, RISCVAPLICState,
> >> +                                  num_harts, 0,
> >> +                                  vmstate_info_uint32, uint32_t),
> >> +            VMSTATE_VARRAY_UINT32(iforce, RISCVAPLICState,
> >> +                                  num_harts, 0,
> >> +                                  vmstate_info_uint32, uint32_t),
> >> +            VMSTATE_VARRAY_UINT32(ithreshold, RISCVAPLICState,
> >> +                                  num_harts, 0,
> >> +                                  vmstate_info_uint32, uint32_t),
> >> +            VMSTATE_END_OF_LIST()
> >> +        }
> >> +};
> >> +
> >> +static void riscv_aplic_class_init(ObjectClass *klass, void *data)
> >> +{
> >> +    DeviceClass *dc = DEVICE_CLASS(klass);
> >> +
> >> +    device_class_set_props(dc, riscv_aplic_properties);
> >> +    dc->realize = riscv_aplic_realize;
> >> +    dc->vmsd = &vmstate_riscv_aplic;
> >> +}
> >> +
> >> +static const TypeInfo riscv_aplic_info = {
> >> +    .name          = TYPE_RISCV_APLIC,
> >> +    .parent        = TYPE_SYS_BUS_DEVICE,
> >> +    .instance_size = sizeof(RISCVAPLICState),
> >> +    .class_init    = riscv_aplic_class_init,
> >> +};
> >> +
> >> +static void riscv_aplic_register_types(void)
> >> +{
> >> +    type_register_static(&riscv_aplic_info);
> >> +}
> >> +
> >> +type_init(riscv_aplic_register_types)
> >> +
> >> +/*
> >> + * Add a APLIC device to another APLIC device as child for
> >> + * interrupt delegation.
> >> + */
> >> +void riscv_aplic_add_child(DeviceState *parent, DeviceState *child)
> >> +{
> >> +    RISCVAPLICState *caplic, *paplic;
> >> +
> >> +    assert(parent && child);
> >> +    caplic = RISCV_APLIC(child);
> >> +    paplic = RISCV_APLIC(parent);
> >> +
> >> +    assert(paplic->num_irqs == caplic->num_irqs);
> >> +    assert(paplic->num_children <= QEMU_APLIC_MAX_CHILDREN);
> >> +
> >> +    caplic->parent = paplic;
> >> +    paplic->children[paplic->num_children] = caplic;
> >> +    paplic->num_children++;
> >> +}
> >> +
> >> +/*
> >> + * Create APLIC device.
> >> + */
> >> +DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size,
> >> +    uint32_t hartid_base, uint32_t num_harts, uint32_t num_sources,
> >> +    uint32_t iprio_bits, bool msimode, bool mmode, DeviceState *parent)
> >> +{
> >> +    DeviceState *dev = qdev_new(TYPE_RISCV_APLIC);
> >> +    uint32_t i;
> >> +
> >> +    assert(num_harts < APLIC_MAX_IDC);
> >> +    assert((APLIC_IDC_BASE + (num_harts * APLIC_IDC_SIZE)) <= size);
> >> +    assert(num_sources < APLIC_MAX_SOURCE);
> >> +    assert(APLIC_MIN_IPRIO_BITS <= iprio_bits);
> >> +    assert(iprio_bits <= APLIC_MAX_IPRIO_BITS);
> >> +
> >> +    qdev_prop_set_uint32(dev, "aperture-size", size);
> >> +    qdev_prop_set_uint32(dev, "hartid-base", hartid_base);
> >> +    qdev_prop_set_uint32(dev, "num-harts", num_harts);
> >> +    qdev_prop_set_uint32(dev, "iprio-mask", ((1U << iprio_bits) - 1));
> >> +    qdev_prop_set_uint32(dev, "num-irqs", num_sources + 1);
> >> +    qdev_prop_set_bit(dev, "msimode", msimode);
> >> +    qdev_prop_set_bit(dev, "mmode", mmode);
> >> +
> >> +    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
> >> +    sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
> >> +
> >> +    if (parent) {
> >> +        riscv_aplic_add_child(parent, dev);
> >> +    }
> >> +
> >> +    if (!msimode) {
> >> +        for (i = 0; i < num_harts; i++) {
> >> +            CPUState *cpu = qemu_get_cpu(hartid_base + i);
> >> +
> >> +            qdev_connect_gpio_out_named(dev, NULL, i,
> >> +                                        qdev_get_gpio_in(DEVICE(cpu),
> >> +                                            (mmode) ? IRQ_M_EXT :
> IRQ_S_EXT));
> >> +        }
> >> +    }
> >> +
> >> +    return dev;
> >> +}
> >> diff --git a/include/hw/intc/riscv_aplic.h
> b/include/hw/intc/riscv_aplic.h
> >> new file mode 100644
> >> index 0000000000..de8532fbc3
> >> --- /dev/null
> >> +++ b/include/hw/intc/riscv_aplic.h
> >> @@ -0,0 +1,79 @@
> >> +/*
> >> + * RISC-V APLIC (Advanced Platform Level Interrupt Controller)
> interface
> >> + *
> >> + * Copyright (c) 2021 Western Digital Corporation or its affiliates.
> >> + *
> >> + * This program is free software; you can redistribute it and/or
> modify it
> >> + * under the terms and conditions of the GNU General Public License,
> >> + * version 2 or later, as published by the Free Software Foundation.
> >> + *
> >> + * This program is distributed in the hope it will be useful, but
> WITHOUT
> >> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
> or
> >> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public
> License for
> >> + * more details.
> >> + *
> >> + * You should have received a copy of the GNU General Public License
> along with
> >> + * this program.  If not, see <http://www.gnu.org/licenses/>.
> >> + */
> >> +
> >> +#ifndef HW_RISCV_APLIC_H
> >> +#define HW_RISCV_APLIC_H
> >> +
> >> +#include "hw/sysbus.h"
> >> +#include "qom/object.h"
> >> +
> >> +#define TYPE_RISCV_APLIC "riscv.aplic"
> >> +
> >> +typedef struct RISCVAPLICState RISCVAPLICState;
> >> +DECLARE_INSTANCE_CHECKER(RISCVAPLICState, RISCV_APLIC,
> TYPE_RISCV_APLIC)
> >> +
> >> +#define APLIC_MIN_SIZE            0x4000
> >> +#define APLIC_SIZE_ALIGN(__x)     (((__x) + (APLIC_MIN_SIZE - 1)) & \
> >> +                                   ~(APLIC_MIN_SIZE - 1))
> >> +#define APLIC_SIZE(__num_harts)   (APLIC_MIN_SIZE + \
> >> +                                   APLIC_SIZE_ALIGN(32 *
> (__num_harts)))
> >> +
> >> +struct RISCVAPLICState {
> >> +    /*< private >*/
> >> +    SysBusDevice parent_obj;
> >> +    qemu_irq *external_irqs;
> >> +
> >> +    /*< public >*/
> >> +    MemoryRegion mmio;
> >> +    uint32_t bitfield_words;
> >> +    uint32_t domaincfg;
> >> +    uint32_t mmsicfgaddr;
> >> +    uint32_t mmsicfgaddrH;
> >> +    uint32_t smsicfgaddr;
> >> +    uint32_t smsicfgaddrH;
> >> +    uint32_t genmsi;
> >> +    uint32_t *sourcecfg;
> >> +    uint32_t *state;
> >> +    uint32_t *target;
> >> +    uint32_t *idelivery;
> >> +    uint32_t *iforce;
> >> +    uint32_t *ithreshold;
> >> +
> >> +    /* topology */
> >> +#define QEMU_APLIC_MAX_CHILDREN        16
> >> +    struct RISCVAPLICState *parent;
> >> +    struct RISCVAPLICState *children[QEMU_APLIC_MAX_CHILDREN];
> >> +    uint16_t num_children;
> >> +
> >> +    /* config */
> >> +    uint32_t aperture_size;
> >> +    uint32_t hartid_base;
> >> +    uint32_t num_harts;
> >> +    uint32_t iprio_mask;
> >> +    uint32_t num_irqs;
> >> +    bool msimode;
> >> +    bool mmode;
> >> +};
> >> +
> >> +void riscv_aplic_add_child(DeviceState *parent, DeviceState *child);
> >> +
> >> +DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size,
> >> +    uint32_t hartid_base, uint32_t num_harts, uint32_t num_sources,
> >> +    uint32_t iprio_bits, bool msimode, bool mmode, DeviceState
> *parent);
> >> +
> >> +#endif
> >> --
> >> 2.25.1
> >>
>

Anup Patel Jan. 20, 2022, 12:05 p.m. UTC | #4

On Thu, Jan 20, 2022 at 1:49 PM Frank Chang <[e-mail]> wrote:
>
> On Thu, Jan 20, 2022 at 12:20 AM Anup Patel <[e-mail]> wrote:
>>
>> Hi Frank,
>>
>> On Wed, Jan 19, 2022 at 9:07 PM Frank Chang <[e-mail]> wrote:
>> >
>> > On Wed, Jan 19, 2022 at 11:27 PM Anup Patel <[e-mail]> wrote:
>> >>
>> >> From: Anup Patel <[e-mail]>
>> >>
>> >> The RISC-V AIA (Advanced Interrupt Architecture) defines a new
>> >> interrupt controller for wired interrupts called APLIC (Advanced
>> >> Platform Level Interrupt Controller). The APLIC is capabable of
>> >> forwarding wired interupts to RISC-V HARTs directly or as MSIs
>> >> (Message Signaled Interupts).
>> >>
>> >> This patch adds device emulation for RISC-V AIA APLIC.
>> >>
>> >> Signed-off-by: Anup Patel <[e-mail]>
>> >> Signed-off-by: Anup Patel <[e-mail]>
>> >> Reviewed-by: Frank Chang <[e-mail]>
>> >> ---
>> >>  hw/intc/Kconfig               |   3 +
>> >>  hw/intc/meson.build           |   1 +
>> >>  hw/intc/riscv_aplic.c         | 975 ++++++++++++++++++++++++++++++++++
>> >>  include/hw/intc/riscv_aplic.h |  79 +++
>> >>  4 files changed, 1058 insertions(+)
>> >>  create mode 100644 hw/intc/riscv_aplic.c
>> >>  create mode 100644 include/hw/intc/riscv_aplic.h
>> >>
>> >> diff --git a/hw/intc/Kconfig b/hw/intc/Kconfig
>> >> index 010ded7eae..528e77b4a6 100644
>> >> --- a/hw/intc/Kconfig
>> >> +++ b/hw/intc/Kconfig
>> >> @@ -70,6 +70,9 @@ config LOONGSON_LIOINTC
>> >>  config RISCV_ACLINT
>> >>      bool
>> >>
>> >> +config RISCV_APLIC
>> >> +    bool
>> >> +
>> >>  config SIFIVE_PLIC
>> >>      bool
>> >>
>> >> diff --git a/hw/intc/meson.build b/hw/intc/meson.build
>> >> index 70080bc161..7466024402 100644
>> >> --- a/hw/intc/meson.build
>> >> +++ b/hw/intc/meson.build
>> >> @@ -50,6 +50,7 @@ specific_ss.add(when: 'CONFIG_S390_FLIC', if_true: files('s390_flic.c'))
>> >>  specific_ss.add(when: 'CONFIG_S390_FLIC_KVM', if_true: files('s390_flic_kvm.c'))
>> >>  specific_ss.add(when: 'CONFIG_SH_INTC', if_true: files('sh_intc.c'))
>> >>  specific_ss.add(when: 'CONFIG_RISCV_ACLINT', if_true: files('riscv_aclint.c'))
>> >> +specific_ss.add(when: 'CONFIG_RISCV_APLIC', if_true: files('riscv_aplic.c'))
>> >>  specific_ss.add(when: 'CONFIG_SIFIVE_PLIC', if_true: files('sifive_plic.c'))
>> >>  specific_ss.add(when: 'CONFIG_XICS', if_true: files('xics.c'))
>> >>  specific_ss.add(when: ['CONFIG_KVM', 'CONFIG_XICS'],
>> >> diff --git a/hw/intc/riscv_aplic.c b/hw/intc/riscv_aplic.c
>> >> new file mode 100644
>> >> index 0000000000..885c1de2af
>> >> --- /dev/null
>> >> +++ b/hw/intc/riscv_aplic.c
>> >> @@ -0,0 +1,975 @@
>> >> +/*
>> >> + * RISC-V APLIC (Advanced Platform Level Interrupt Controller)
>> >> + *
>> >> + * Copyright (c) 2021 Western Digital Corporation or its affiliates.
>> >> + *
>> >> + * This program is free software; you can redistribute it and/or modify it
>> >> + * under the terms and conditions of the GNU General Public License,
>> >> + * version 2 or later, as published by the Free Software Foundation.
>> >> + *
>> >> + * This program is distributed in the hope it will be useful, but WITHOUT
>> >> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> >> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
>> >> + * more details.
>> >> + *
>> >> + * You should have received a copy of the GNU General Public License along with
>> >> + * this program.  If not, see <http://www.gnu.org/licenses/>.
>> >> + */
>> >> +
>> >> +#include "qemu/osdep.h"
>> >> +#include "qapi/error.h"
>> >> +#include "qemu/log.h"
>> >> +#include "qemu/module.h"
>> >> +#include "qemu/error-report.h"
>> >> +#include "qemu/bswap.h"
>> >> +#include "exec/address-spaces.h"
>> >> +#include "hw/sysbus.h"
>> >> +#include "hw/pci/msi.h"
>> >> +#include "hw/boards.h"
>> >> +#include "hw/qdev-properties.h"
>> >> +#include "hw/intc/riscv_aplic.h"
>> >> +#include "hw/irq.h"
>> >> +#include "target/riscv/cpu.h"
>> >> +#include "sysemu/sysemu.h"
>> >> +#include "migration/vmstate.h"
>> >> +
>> >> +#define APLIC_MAX_IDC                  (1UL << 14)
>> >> +#define APLIC_MAX_SOURCE               1024
>> >> +#define APLIC_MIN_IPRIO_BITS           1
>> >> +#define APLIC_MAX_IPRIO_BITS           8
>> >> +#define APLIC_MAX_CHILDREN             1024
>> >> +
>> >> +#define APLIC_DOMAINCFG                0x0000
>> >> +#define APLIC_DOMAINCFG_RDONLY         0x80000000
>> >> +#define APLIC_DOMAINCFG_IE             (1 << 8)
>> >> +#define APLIC_DOMAINCFG_DM             (1 << 2)
>> >> +#define APLIC_DOMAINCFG_BE             (1 << 0)
>> >> +
>> >> +#define APLIC_SOURCECFG_BASE           0x0004
>> >> +#define APLIC_SOURCECFG_D              (1 << 10)
>> >> +#define APLIC_SOURCECFG_CHILDIDX_MASK  0x000003ff
>> >> +#define APLIC_SOURCECFG_SM_MASK        0x00000007
>> >> +#define APLIC_SOURCECFG_SM_INACTIVE    0x0
>> >> +#define APLIC_SOURCECFG_SM_DETACH      0x1
>> >> +#define APLIC_SOURCECFG_SM_EDGE_RISE   0x4
>> >> +#define APLIC_SOURCECFG_SM_EDGE_FALL   0x5
>> >> +#define APLIC_SOURCECFG_SM_LEVEL_HIGH  0x6
>> >> +#define APLIC_SOURCECFG_SM_LEVEL_LOW   0x7
>> >> +
>> >> +#define APLIC_MMSICFGADDR              0x1bc0
>> >> +#define APLIC_MMSICFGADDRH             0x1bc4
>> >> +#define APLIC_SMSICFGADDR              0x1bc8
>> >> +#define APLIC_SMSICFGADDRH             0x1bcc
>> >> +
>> >> +#define APLIC_xMSICFGADDRH_L           (1UL << 31)
>> >> +#define APLIC_xMSICFGADDRH_HHXS_MASK   0x1f
>> >> +#define APLIC_xMSICFGADDRH_HHXS_SHIFT  24
>> >> +#define APLIC_xMSICFGADDRH_LHXS_MASK   0x7
>> >> +#define APLIC_xMSICFGADDRH_LHXS_SHIFT  20
>> >> +#define APLIC_xMSICFGADDRH_HHXW_MASK   0x7
>> >> +#define APLIC_xMSICFGADDRH_HHXW_SHIFT  16
>> >> +#define APLIC_xMSICFGADDRH_LHXW_MASK   0xf
>> >> +#define APLIC_xMSICFGADDRH_LHXW_SHIFT  12
>> >> +#define APLIC_xMSICFGADDRH_BAPPN_MASK  0xfff
>> >> +
>> >> +#define APLIC_xMSICFGADDR_PPN_SHIFT    12
>> >> +
>> >> +#define APLIC_xMSICFGADDR_PPN_HART(__lhxs) \
>> >> +    ((1UL << (__lhxs)) - 1)
>> >> +
>> >> +#define APLIC_xMSICFGADDR_PPN_LHX_MASK(__lhxw) \
>> >> +    ((1UL << (__lhxw)) - 1)
>> >> +#define APLIC_xMSICFGADDR_PPN_LHX_SHIFT(__lhxs) \
>> >> +    ((__lhxs))
>> >> +#define APLIC_xMSICFGADDR_PPN_LHX(__lhxw, __lhxs) \
>> >> +    (APLIC_xMSICFGADDR_PPN_LHX_MASK(__lhxw) << \
>> >> +     APLIC_xMSICFGADDR_PPN_LHX_SHIFT(__lhxs))
>> >> +
>> >> +#define APLIC_xMSICFGADDR_PPN_HHX_MASK(__hhxw) \
>> >> +    ((1UL << (__hhxw)) - 1)
>> >> +#define APLIC_xMSICFGADDR_PPN_HHX_SHIFT(__hhxs) \
>> >> +    ((__hhxs) + APLIC_xMSICFGADDR_PPN_SHIFT)
>> >> +#define APLIC_xMSICFGADDR_PPN_HHX(__hhxw, __hhxs) \
>> >> +    (APLIC_xMSICFGADDR_PPN_HHX_MASK(__hhxw) << \
>> >> +     APLIC_xMSICFGADDR_PPN_HHX_SHIFT(__hhxs))
>> >> +
>> >> +#define APLIC_xMSICFGADDRH_VALID_MASK   \
>> >> +    (APLIC_xMSICFGADDRH_L | \
>> >> +     (APLIC_xMSICFGADDRH_HHXS_MASK << APLIC_xMSICFGADDRH_HHXS_SHIFT) | \
>> >> +     (APLIC_xMSICFGADDRH_LHXS_MASK << APLIC_xMSICFGADDRH_LHXS_SHIFT) | \
>> >> +     (APLIC_xMSICFGADDRH_HHXW_MASK << APLIC_xMSICFGADDRH_HHXW_SHIFT) | \
>> >> +     (APLIC_xMSICFGADDRH_LHXW_MASK << APLIC_xMSICFGADDRH_LHXW_SHIFT) | \
>> >> +     APLIC_xMSICFGADDRH_BAPPN_MASK)
>> >> +
>> >> +#define APLIC_SETIP_BASE               0x1c00
>> >> +#define APLIC_SETIPNUM                 0x1cdc
>> >> +
>> >> +#define APLIC_CLRIP_BASE               0x1d00
>> >> +#define APLIC_CLRIPNUM                 0x1ddc
>> >> +
>> >> +#define APLIC_SETIE_BASE               0x1e00
>> >> +#define APLIC_SETIENUM                 0x1edc
>> >> +
>> >> +#define APLIC_CLRIE_BASE               0x1f00
>> >> +#define APLIC_CLRIENUM                 0x1fdc
>> >> +
>> >> +#define APLIC_SETIPNUM_LE              0x2000
>> >> +#define APLIC_SETIPNUM_BE              0x2004
>> >> +
>> >> +#define APLIC_ISTATE_PENDING           (1U << 0)
>> >> +#define APLIC_ISTATE_ENABLED           (1U << 1)
>> >> +#define APLIC_ISTATE_ENPEND            (APLIC_ISTATE_ENABLED | \
>> >> +                                        APLIC_ISTATE_PENDING)
>> >> +#define APLIC_ISTATE_INPUT             (1U << 8)
>> >> +
>> >> +#define APLIC_GENMSI                   0x3000
>> >> +
>> >> +#define APLIC_TARGET_BASE              0x3004
>> >> +#define APLIC_TARGET_HART_IDX_SHIFT    18
>> >> +#define APLIC_TARGET_HART_IDX_MASK     0x3fff
>> >> +#define APLIC_TARGET_GUEST_IDX_SHIFT   12
>> >> +#define APLIC_TARGET_GUEST_IDX_MASK    0x3f
>> >> +#define APLIC_TARGET_IPRIO_MASK        0xff
>> >> +#define APLIC_TARGET_EIID_MASK         0x7ff
>> >> +
>> >> +#define APLIC_IDC_BASE                 0x4000
>> >> +#define APLIC_IDC_SIZE                 32
>> >> +
>> >> +#define APLIC_IDC_IDELIVERY            0x00
>> >> +
>> >> +#define APLIC_IDC_IFORCE               0x04
>> >> +
>> >> +#define APLIC_IDC_ITHRESHOLD           0x08
>> >> +
>> >> +#define APLIC_IDC_TOPI                 0x18
>> >> +#define APLIC_IDC_TOPI_ID_SHIFT        16
>> >> +#define APLIC_IDC_TOPI_ID_MASK         0x3ff
>> >> +#define APLIC_IDC_TOPI_PRIO_MASK       0xff
>> >> +
>> >> +#define APLIC_IDC_CLAIMI               0x1c
>> >> +
>> >> +static uint32_t riscv_aplic_read_input_word(RISCVAPLICState *aplic,
>> >> +                                            uint32_t word)
>> >> +{
>> >> +    uint32_t i, irq, ret = 0;
>> >> +
>> >> +    for (i = 0; i < 32; i++) {
>> >> +        irq = word * 32 + i;
>> >> +        if (!irq || aplic->num_irqs <= irq) {
>> >> +            continue;
>> >> +        }
>> >> +
>> >> +        ret |= ((aplic->state[irq] & APLIC_ISTATE_INPUT) ? 1 : 0) << i;
>> >> +    }
>> >> +
>> >> +    return ret;
>> >> +}
>> >> +
>> >> +static uint32_t riscv_aplic_read_pending_word(RISCVAPLICState *aplic,
>> >> +                                              uint32_t word)
>> >> +{
>> >> +    uint32_t i, irq, ret = 0;
>> >> +
>> >> +    for (i = 0; i < 32; i++) {
>> >> +        irq = word * 32 + i;
>> >> +        if (!irq || aplic->num_irqs <= irq) {
>> >> +            continue;
>> >> +        }
>> >> +
>> >> +        ret |= ((aplic->state[irq] & APLIC_ISTATE_PENDING) ? 1 : 0) << i;
>> >> +    }
>> >> +
>> >> +    return ret;
>> >> +}
>> >> +
>> >> +static void riscv_aplic_set_pending_raw(RISCVAPLICState *aplic,
>> >> +                                        uint32_t irq, bool pending)
>> >> +{
>> >> +    if (pending) {
>> >> +        aplic->state[irq] |= APLIC_ISTATE_PENDING;
>> >> +    } else {
>> >> +        aplic->state[irq] &= ~APLIC_ISTATE_PENDING;
>> >> +    }
>> >> +}
>> >> +
>> >> +static void riscv_aplic_set_pending(RISCVAPLICState *aplic,
>> >> +                                    uint32_t irq, bool pending)
>> >> +{
>> >> +    uint32_t sourcecfg, sm;
>> >> +
>> >> +    if ((irq <= 0) || (aplic->num_irqs <= irq)) {
>> >> +        return;
>> >> +    }
>> >> +
>> >> +    sourcecfg = aplic->sourcecfg[irq];
>> >> +    if (sourcecfg & APLIC_SOURCECFG_D) {
>> >> +        return;
>> >> +    }
>> >> +
>> >> +    sm = sourcecfg & APLIC_SOURCECFG_SM_MASK;
>> >> +    if ((sm == APLIC_SOURCECFG_SM_INACTIVE) ||
>> >> +        ((!aplic->msimode || (aplic->msimode && !pending)) &&
>> >> +         ((sm == APLIC_SOURCECFG_SM_LEVEL_HIGH) ||
>> >> +          (sm == APLIC_SOURCECFG_SM_LEVEL_LOW)))) {
>> >> +        return;
>> >> +    }
>> >> +
>> >> +    riscv_aplic_set_pending_raw(aplic, irq, pending);
>> >> +}
>> >> +
>> >> +static void riscv_aplic_set_pending_word(RISCVAPLICState *aplic,
>> >> +                                         uint32_t word, uint32_t value,
>> >> +                                         bool pending)
>> >> +{
>> >> +    uint32_t i, irq;
>> >> +
>> >> +    for (i = 0; i < 32; i++) {
>> >> +        irq = word * 32 + i;
>> >> +        if (!irq || aplic->num_irqs <= irq) {
>> >> +            continue;
>> >> +        }
>> >> +
>> >> +        if (value & (1U << i)) {
>> >> +            riscv_aplic_set_pending(aplic, irq, pending);
>> >> +        }
>> >> +    }
>> >> +}
>> >> +
>> >> +static uint32_t riscv_aplic_read_enabled_word(RISCVAPLICState *aplic,
>> >> +                                              int word)
>> >> +{
>> >> +    uint32_t i, irq, ret = 0;
>> >> +
>> >> +    for (i = 0; i < 32; i++) {
>> >> +        irq = word * 32 + i;
>> >> +        if (!irq || aplic->num_irqs <= irq) {
>> >> +            continue;
>> >> +        }
>> >> +
>> >> +        ret |= ((aplic->state[irq] & APLIC_ISTATE_ENABLED) ? 1 : 0) << i;
>> >> +    }
>> >> +
>> >> +    return ret;
>> >> +}
>> >> +
>> >> +static void riscv_aplic_set_enabled_raw(RISCVAPLICState *aplic,
>> >> +                                        uint32_t irq, bool enabled)
>> >> +{
>> >> +    if (enabled) {
>> >> +        aplic->state[irq] |= APLIC_ISTATE_ENABLED;
>> >> +    } else {
>> >> +        aplic->state[irq] &= ~APLIC_ISTATE_ENABLED;
>> >> +    }
>> >> +}
>> >> +
>> >> +static void riscv_aplic_set_enabled(RISCVAPLICState *aplic,
>> >> +                                    uint32_t irq, bool enabled)
>> >> +{
>> >> +    uint32_t sourcecfg, sm;
>> >> +
>> >> +    if ((irq <= 0) || (aplic->num_irqs <= irq)) {
>> >> +        return;
>> >> +    }
>> >> +
>> >> +    sourcecfg = aplic->sourcecfg[irq];
>> >> +    if (sourcecfg & APLIC_SOURCECFG_D) {
>> >> +        return;
>> >> +    }
>> >> +
>> >> +    sm = sourcecfg & APLIC_SOURCECFG_SM_MASK;
>> >> +    if (sm == APLIC_SOURCECFG_SM_INACTIVE) {
>> >> +        return;
>> >> +    }
>> >> +
>> >> +    riscv_aplic_set_enabled_raw(aplic, irq, enabled);
>> >> +}
>> >> +
>> >> +static void riscv_aplic_set_enabled_word(RISCVAPLICState *aplic,
>> >> +                                         uint32_t word, uint32_t value,
>> >> +                                         bool enabled)
>> >> +{
>> >> +    uint32_t i, irq;
>> >> +
>> >> +    for (i = 0; i < 32; i++) {
>> >> +        irq = word * 32 + i;
>> >> +        if (!irq || aplic->num_irqs <= irq) {
>> >> +            continue;
>> >> +        }
>> >> +
>> >> +        if (value & (1U << i)) {
>> >> +            riscv_aplic_set_enabled(aplic, irq, enabled);
>> >> +        }
>> >> +    }
>> >> +}
>> >> +
>> >> +static void riscv_aplic_msi_send(RISCVAPLICState *aplic,
>> >> +                                 uint32_t hart_idx, uint32_t guest_idx,
>> >> +                                 uint32_t eiid)
>> >> +{
>> >> +    uint64_t addr;
>> >> +    MemTxResult result;
>> >> +    RISCVAPLICState *aplic_m;
>> >> +    uint32_t lhxs, lhxw, hhxs, hhxw, group_idx, msicfgaddr, msicfgaddrH;
>> >> +
>> >> +    aplic_m = aplic;
>> >> +    while (aplic_m && !aplic_m->mmode) {
>> >> +        aplic_m = aplic_m->parent;
>> >> +    }
>> >> +    if (!aplic_m) {
>> >> +        qemu_log_mask(LOG_GUEST_ERROR, "%s: m-level APLIC not found\n",
>> >> +                      __func__);
>> >> +        return;
>> >> +    }
>> >> +
>> >> +    if (aplic->mmode) {
>> >> +        msicfgaddr = aplic_m->mmsicfgaddr;
>> >> +        msicfgaddrH = aplic_m->mmsicfgaddrH;
>> >> +    } else {
>> >> +        msicfgaddr = aplic_m->smsicfgaddr;
>> >> +        msicfgaddrH = aplic_m->smsicfgaddrH;
>> >> +    }
>> >> +
>> >> +    lhxs = (msicfgaddrH >> APLIC_xMSICFGADDRH_LHXS_SHIFT) &
>> >> +            APLIC_xMSICFGADDRH_LHXS_MASK;
>> >> +    lhxw = (msicfgaddrH >> APLIC_xMSICFGADDRH_LHXW_SHIFT) &
>> >> +            APLIC_xMSICFGADDRH_LHXW_MASK;
>> >> +    hhxs = (msicfgaddrH >> APLIC_xMSICFGADDRH_HHXS_SHIFT) &
>> >> +            APLIC_xMSICFGADDRH_HHXS_MASK;
>> >> +    hhxw = (msicfgaddrH >> APLIC_xMSICFGADDRH_HHXW_SHIFT) &
>> >> +            APLIC_xMSICFGADDRH_HHXW_MASK;
>> >> +
>> >> +    group_idx = hart_idx >> lhxw;
>> >> +    hart_idx &= APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw);
>> >> +
>> >> +    addr = msicfgaddr;
>> >> +    addr |= ((uint64_t)(msicfgaddrH & APLIC_xMSICFGADDRH_BAPPN_MASK)) << 32;
>> >> +    addr |= ((uint64_t)(group_idx & APLIC_xMSICFGADDR_PPN_HHX_MASK(hhxw))) <<
>> >> +             APLIC_xMSICFGADDR_PPN_HHX_SHIFT(hhxs);
>> >> +    addr |= ((uint64_t)(hart_idx & APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw))) <<
>> >> +             APLIC_xMSICFGADDR_PPN_LHX_SHIFT(lhxs);
>> >> +    addr |= (uint64_t)(guest_idx & APLIC_xMSICFGADDR_PPN_HART(lhxs));
>> >> +    addr <<= APLIC_xMSICFGADDR_PPN_SHIFT;
>> >> +
>> >> +    address_space_stl_le(&address_space_memory, addr,
>> >> +                         eiid, MEMTXATTRS_UNSPECIFIED, &result);
>> >> +    if (result != MEMTX_OK) {
>> >> +        qemu_log_mask(LOG_GUEST_ERROR, "%s: MSI write failed for "
>> >> +                      "hart_index=%d guest_index=%d eiid=%d\n",
>> >> +                      __func__, hart_idx, guest_idx, eiid);
>> >> +    }
>> >> +}
>> >> +
>> >> +static void riscv_aplic_msi_irq_update(RISCVAPLICState *aplic, uint32_t irq)
>> >> +{
>> >> +    uint32_t hart_idx, guest_idx, eiid;
>> >> +
>> >> +    if (!aplic->msimode || (aplic->num_irqs <= irq) ||
>> >> +        !(aplic->domaincfg & APLIC_DOMAINCFG_IE)) {
>> >> +        return;
>> >> +    }
>> >> +
>> >> +    if ((aplic->state[irq] & APLIC_ISTATE_ENPEND) != APLIC_ISTATE_ENPEND) {
>> >> +        return;
>> >> +    }
>> >> +
>> >> +    riscv_aplic_set_pending_raw(aplic, irq, false);
>> >> +
>> >> +    hart_idx = aplic->target[irq] >> APLIC_TARGET_HART_IDX_SHIFT;
>> >> +    hart_idx &= APLIC_TARGET_HART_IDX_MASK;
>> >
>> >
>> > Hi Anup,
>> >
>> > I'm wondering does the hart_idx here has to use Machine-level hart index
>> > for Supervisor-level interrupt domain?
>> >
>> > According to AIA spec for supervisor-level domin (Section 4.8.1):
>> >   For a supervisor-level domain, "if" MSI target addresses are determined by the parent domain’s
>> >   smsiaddrcfg and smsiaddrcfgh registers, then to construct the address for an outgoing MSI for
>> >   interrupt source i, the Hart Index from register target[i] must first be converted into the index
>> >   that the machine-level parent domain uses for the same hart. (Typically, these numbers are the
>> >   same, but they may not be.) The address for the MSI is then computed using this "machine-level"
>> >   hart index, the parent domain’s smsiaddrcfg and smsiaddrcfgh, and the Guest Index value from
>> >   target[i].
>> >
>> > The description of converting target[i] hart Index to machine-level hart index in the spec
>> > is quite vague to me.
>>
>> The "Section 4.3 Hart index numbers" states the following:
>>
>>     The index number a domain associates with a hart may or may not
>> have any relationship
>>     to the unique hart identifier (“hart ID”) that the RISC-V
>> Privileged Architecture assigns to
>>     the hart.
>>
>> For APLIC direct mode, if N harts are associated with a given domain
>> then APLIC associates
>> hart index from 0 to N-1 which are basically APLIC IDC numbers. In
>> this case, the APLIC
>> hart index is not related to "machine-level" hart index so software
>> has to discover it from the
>> "interrupt-extended" DT property (or similar for ACPI).
>>
>> For APLIC msi mode, the APLIC can basically target any HART using information in
>> [m|s]msiaddrcfg registers and interrupt target registers. In this
>> case, the APLIC hart
>> index in the APLIC target register is a combination of IMSIC group
>> index (g) and IMSIC
>> hart index (h) and the APLIC target register also provides IMSIC guest
>> index. In other
>> words, the APLIC hart index for msi mode is also not related to the
>> "machine-level" hart
>> index. The APLIC driver will figure-out/extract the IMSIC group index
>> (g), IMSIC hart
>> index (h), and IMSIC guest index from target MSI address when Linux
>> tries to write
>> target MSI address for a particular APLIC interrupt.
>>
>> Regards,
>> Anup
>
>
> Hi Anup,
>
> Thanks for the detailed explanation.
>
> I think it's the formula in the AIA spec which confused me:
>   g = (machine-level hart index >> LHXW) & (2^HHXW − 1)
>   h = machine-level hart index & (2^LHXW − 1)
>   MSI address =  (Base PPN | (g << (HHXS + 12)) | (h << LHXS) | Guest Index) << 12
>
> The term "machine-level hart index" makes me thought that
> we should use the hart index from parent machine-level interrupt domain.

Yes, the term "machine-level hart index" is confusing. In reality, it is the
"hart index" in the APLIC interrupt target registers.

>
> So it's driver's responsibility to figure out the correct IMSIC hart index, guest index
> it wants to forward the MSI to and configure them to the target[i] register?

Yes, that's correct.

>
> From hardware (QEMU)'s perspective, it just needs to extract
> the Hart index, Guest index out from target[irq] register in IRQ's interrupt domain
> and calculate the MSI address with the PPN from parent interrupt domain's
> m/smsiaddrcfg and m/smsiaddrcfgh?

Yes, that's correct.

The [m|s]msiaddrcfg registers are only available in the root APLIC domain
and M-mode runtime firmware (OpenSBI) has to initialize it correctly.

The APLIC Linux/OS driver has to ensure that APLIC interrupt target registers
are programmed properly so that APLIC can generate the correct output MSI
address.

Regards,
Anup

>
> Regards,
> Frank Chang
>
>>
>>
>> >
>> > Regards,
>> > Frank Chang
>> >
>> >>
>> >> +    if (aplic->mmode) {
>> >> +        /* M-level APLIC ignores guest_index */
>> >> +        guest_idx = 0;
>> >> +    } else {
>> >> +        guest_idx = aplic->target[irq] >> APLIC_TARGET_GUEST_IDX_SHIFT;
>> >> +        guest_idx &= APLIC_TARGET_GUEST_IDX_MASK;
>> >> +    }
>> >> +    eiid = aplic->target[irq] & APLIC_TARGET_EIID_MASK;
>> >> +    riscv_aplic_msi_send(aplic, hart_idx, guest_idx, eiid);
>> >> +}
>> >> +
>> >> +static uint32_t riscv_aplic_idc_topi(RISCVAPLICState *aplic, uint32_t idc)
>> >> +{
>> >> +    uint32_t best_irq, best_iprio;
>> >> +    uint32_t irq, iprio, ihartidx, ithres;
>> >> +
>> >> +    if (aplic->num_harts <= idc) {
>> >> +        return 0;
>> >> +    }
>> >> +
>> >> +    ithres = aplic->ithreshold[idc];
>> >> +    best_irq = best_iprio = UINT32_MAX;
>> >> +    for (irq = 1; irq < aplic->num_irqs; irq++) {
>> >> +        if ((aplic->state[irq] & APLIC_ISTATE_ENPEND) !=
>> >> +            APLIC_ISTATE_ENPEND) {
>> >> +            continue;
>> >> +        }
>> >> +
>> >> +        ihartidx = aplic->target[irq] >> APLIC_TARGET_HART_IDX_SHIFT;
>> >> +        ihartidx &= APLIC_TARGET_HART_IDX_MASK;
>> >> +        if (ihartidx != idc) {
>> >> +            continue;
>> >> +        }
>> >> +
>> >> +        iprio = aplic->target[irq] & aplic->iprio_mask;
>> >> +        if (ithres && iprio >= ithres) {
>> >> +            continue;
>> >> +        }
>> >> +
>> >> +        if (iprio < best_iprio) {
>> >> +            best_irq = irq;
>> >> +            best_iprio = iprio;
>> >> +        }
>> >> +    }
>> >> +
>> >> +    if (best_irq < aplic->num_irqs && best_iprio <= aplic->iprio_mask) {
>> >> +        return (best_irq << APLIC_IDC_TOPI_ID_SHIFT) | best_iprio;
>> >> +    }
>> >> +
>> >> +    return 0;
>> >> +}
>> >> +
>> >> +static void riscv_aplic_idc_update(RISCVAPLICState *aplic, uint32_t idc)
>> >> +{
>> >> +    uint32_t topi;
>> >> +
>> >> +    if (aplic->msimode || aplic->num_harts <= idc) {
>> >> +        return;
>> >> +    }
>> >> +
>> >> +    topi = riscv_aplic_idc_topi(aplic, idc);
>> >> +    if ((aplic->domaincfg & APLIC_DOMAINCFG_IE) &&
>> >> +        aplic->idelivery[idc] &&
>> >> +        (aplic->iforce[idc] || topi)) {
>> >> +        qemu_irq_raise(aplic->external_irqs[idc]);
>> >> +    } else {
>> >> +        qemu_irq_lower(aplic->external_irqs[idc]);
>> >> +    }
>> >> +}
>> >> +
>> >> +static uint32_t riscv_aplic_idc_claimi(RISCVAPLICState *aplic, uint32_t idc)
>> >> +{
>> >> +    uint32_t irq, state, sm, topi = riscv_aplic_idc_topi(aplic, idc);
>> >> +
>> >> +    if (!topi) {
>> >> +        aplic->iforce[idc] = 0;
>> >> +        return 0;
>> >> +    }
>> >> +
>> >> +    irq = (topi >> APLIC_IDC_TOPI_ID_SHIFT) & APLIC_IDC_TOPI_ID_MASK;
>> >> +    sm = aplic->sourcecfg[irq] & APLIC_SOURCECFG_SM_MASK;
>> >> +    state = aplic->state[irq];
>> >> +    riscv_aplic_set_pending_raw(aplic, irq, false);
>> >> +    if ((sm == APLIC_SOURCECFG_SM_LEVEL_HIGH) &&
>> >> +        (state & APLIC_ISTATE_INPUT)) {
>> >> +        riscv_aplic_set_pending_raw(aplic, irq, true);
>> >> +    } else if ((sm == APLIC_SOURCECFG_SM_LEVEL_LOW) &&
>> >> +               !(state & APLIC_ISTATE_INPUT)) {
>> >> +        riscv_aplic_set_pending_raw(aplic, irq, true);
>> >> +    }
>> >> +    riscv_aplic_idc_update(aplic, idc);
>> >> +
>> >> +    return topi;
>> >> +}
>> >> +
>> >> +static void riscv_aplic_request(void *opaque, int irq, int level)
>> >> +{
>> >> +    bool update = false;
>> >> +    RISCVAPLICState *aplic = opaque;
>> >> +    uint32_t sourcecfg, childidx, state, idc;
>> >> +
>> >> +    assert((0 < irq) && (irq < aplic->num_irqs));
>> >> +
>> >> +    sourcecfg = aplic->sourcecfg[irq];
>> >> +    if (sourcecfg & APLIC_SOURCECFG_D) {
>> >> +        childidx = sourcecfg & APLIC_SOURCECFG_CHILDIDX_MASK;
>> >> +        if (childidx < aplic->num_children) {
>> >> +            riscv_aplic_request(aplic->children[childidx], irq, level);
>> >> +        }
>> >> +        return;
>> >> +    }
>> >> +
>> >> +    state = aplic->state[irq];
>> >> +    switch (sourcecfg & APLIC_SOURCECFG_SM_MASK) {
>> >> +    case APLIC_SOURCECFG_SM_EDGE_RISE:
>> >> +        if ((level > 0) && !(state & APLIC_ISTATE_INPUT) &&
>> >> +            !(state & APLIC_ISTATE_PENDING)) {
>> >> +            riscv_aplic_set_pending_raw(aplic, irq, true);
>> >> +            update = true;
>> >> +        }
>> >> +        break;
>> >> +    case APLIC_SOURCECFG_SM_EDGE_FALL:
>> >> +        if ((level <= 0) && (state & APLIC_ISTATE_INPUT) &&
>> >> +            !(state & APLIC_ISTATE_PENDING)) {
>> >> +            riscv_aplic_set_pending_raw(aplic, irq, true);
>> >> +            update = true;
>> >> +        }
>> >> +        break;
>> >> +    case APLIC_SOURCECFG_SM_LEVEL_HIGH:
>> >> +        if ((level > 0) && !(state & APLIC_ISTATE_PENDING)) {
>> >> +            riscv_aplic_set_pending_raw(aplic, irq, true);
>> >> +            update = true;
>> >> +        }
>> >> +        break;
>> >> +    case APLIC_SOURCECFG_SM_LEVEL_LOW:
>> >> +        if ((level <= 0) && !(state & APLIC_ISTATE_PENDING)) {
>> >> +            riscv_aplic_set_pending_raw(aplic, irq, true);
>> >> +            update = true;
>> >> +        }
>> >> +        break;
>> >> +    default:
>> >> +        break;
>> >> +    }
>> >> +
>> >> +    if (level <= 0) {
>> >> +        aplic->state[irq] &= ~APLIC_ISTATE_INPUT;
>> >> +    } else {
>> >> +        aplic->state[irq] |= APLIC_ISTATE_INPUT;
>> >> +    }
>> >> +
>> >> +    if (update) {
>> >> +        if (aplic->msimode) {
>> >> +            riscv_aplic_msi_irq_update(aplic, irq);
>> >> +        } else {
>> >> +            idc = aplic->target[irq] >> APLIC_TARGET_HART_IDX_SHIFT;
>> >> +            idc &= APLIC_TARGET_HART_IDX_MASK;
>> >> +            riscv_aplic_idc_update(aplic, idc);
>> >> +        }
>> >> +    }
>> >> +}
>> >> +
>> >> +static uint64_t riscv_aplic_read(void *opaque, hwaddr addr, unsigned size)
>> >> +{
>> >> +    uint32_t irq, word, idc;
>> >> +    RISCVAPLICState *aplic = opaque;
>> >> +
>> >> +    /* Reads must be 4 byte words */
>> >> +    if ((addr & 0x3) != 0) {
>> >> +        goto err;
>> >> +    }
>> >> +
>> >> +    if (addr == APLIC_DOMAINCFG) {
>> >> +        return APLIC_DOMAINCFG_RDONLY | aplic->domaincfg |
>> >> +               (aplic->msimode ? APLIC_DOMAINCFG_DM : 0);
>> >> +    } else if ((APLIC_SOURCECFG_BASE <= addr) &&
>> >> +            (addr < (APLIC_SOURCECFG_BASE + (aplic->num_irqs - 1) * 4))) {
>> >> +        irq  = ((addr - APLIC_SOURCECFG_BASE) >> 2) + 1;
>> >> +        return aplic->sourcecfg[irq];
>> >> +    } else if (aplic->mmode && aplic->msimode &&
>> >> +               (addr == APLIC_MMSICFGADDR)) {
>> >> +        return aplic->mmsicfgaddr;
>> >> +    } else if (aplic->mmode && aplic->msimode &&
>> >> +               (addr == APLIC_MMSICFGADDRH)) {
>> >> +        return aplic->mmsicfgaddrH;
>> >> +    } else if (aplic->mmode && aplic->msimode &&
>> >> +               (addr == APLIC_SMSICFGADDR)) {
>> >> +        /* Registers SMSICFGADDR and SMSICFGADDRH are implemented only if:
>> >> +         * (a) the interrupt domain is at machine level
>> >> +         * (b) the domain’s harts implement supervisor mode
>> >> +         * (c) the domain has one or more child supervisor-level domains
>> >> +         *     that support MSI delivery mode (domaincfg.DM is not read-
>> >> +         *     only zero in at least one of the supervisor-level child
>> >> +         * domains).
>> >> +         */
>> >> +        return (aplic->num_children) ? aplic->smsicfgaddr : 0;
>> >> +    } else if (aplic->mmode && aplic->msimode &&
>> >> +               (addr == APLIC_SMSICFGADDRH)) {
>> >> +        return (aplic->num_children) ? aplic->smsicfgaddrH : 0;
>> >> +    } else if ((APLIC_SETIP_BASE <= addr) &&
>> >> +            (addr < (APLIC_SETIP_BASE + aplic->bitfield_words * 4))) {
>> >> +        word = (addr - APLIC_SETIP_BASE) >> 2;
>> >> +        return riscv_aplic_read_pending_word(aplic, word);
>> >> +    } else if (addr == APLIC_SETIPNUM) {
>> >> +        return 0;
>> >> +    } else if ((APLIC_CLRIP_BASE <= addr) &&
>> >> +            (addr < (APLIC_CLRIP_BASE + aplic->bitfield_words * 4))) {
>> >> +        word = (addr - APLIC_CLRIP_BASE) >> 2;
>> >> +        return riscv_aplic_read_input_word(aplic, word);
>> >> +    } else if (addr == APLIC_CLRIPNUM) {
>> >> +        return 0;
>> >> +    } else if ((APLIC_SETIE_BASE <= addr) &&
>> >> +            (addr < (APLIC_SETIE_BASE + aplic->bitfield_words * 4))) {
>> >> +        word = (addr - APLIC_SETIE_BASE) >> 2;
>> >> +        return riscv_aplic_read_enabled_word(aplic, word);
>> >> +    } else if (addr == APLIC_SETIENUM) {
>> >> +        return 0;
>> >> +    } else if ((APLIC_CLRIE_BASE <= addr) &&
>> >> +            (addr < (APLIC_CLRIE_BASE + aplic->bitfield_words * 4))) {
>> >> +        return 0;
>> >> +    } else if (addr == APLIC_CLRIENUM) {
>> >> +        return 0;
>> >> +    } else if (addr == APLIC_SETIPNUM_LE) {
>> >> +        return 0;
>> >> +    } else if (addr == APLIC_SETIPNUM_BE) {
>> >> +        return 0;
>> >> +    } else if (addr == APLIC_GENMSI) {
>> >> +        return (aplic->msimode) ? aplic->genmsi : 0;
>> >> +    } else if ((APLIC_TARGET_BASE <= addr) &&
>> >> +            (addr < (APLIC_TARGET_BASE + (aplic->num_irqs - 1) * 4))) {
>> >> +        irq = ((addr - APLIC_TARGET_BASE) >> 2) + 1;
>> >> +        return aplic->target[irq];
>> >> +    } else if (!aplic->msimode && (APLIC_IDC_BASE <= addr) &&
>> >> +            (addr < (APLIC_IDC_BASE + aplic->num_harts * APLIC_IDC_SIZE))) {
>> >> +        idc = (addr - APLIC_IDC_BASE) / APLIC_IDC_SIZE;
>> >> +        switch (addr - (APLIC_IDC_BASE + idc * APLIC_IDC_SIZE)) {
>> >> +        case APLIC_IDC_IDELIVERY:
>> >> +            return aplic->idelivery[idc];
>> >> +        case APLIC_IDC_IFORCE:
>> >> +            return aplic->iforce[idc];
>> >> +        case APLIC_IDC_ITHRESHOLD:
>> >> +            return aplic->ithreshold[idc];
>> >> +        case APLIC_IDC_TOPI:
>> >> +            return riscv_aplic_idc_topi(aplic, idc);
>> >> +        case APLIC_IDC_CLAIMI:
>> >> +            return riscv_aplic_idc_claimi(aplic, idc);
>> >> +        default:
>> >> +            goto err;
>> >> +        };
>> >> +    }
>> >> +
>> >> +err:
>> >> +    qemu_log_mask(LOG_GUEST_ERROR,
>> >> +                  "%s: Invalid register read 0x%" HWADDR_PRIx "\n",
>> >> +                  __func__, addr);
>> >> +    return 0;
>> >> +}
>> >> +
>> >> +static void riscv_aplic_write(void *opaque, hwaddr addr, uint64_t value,
>> >> +        unsigned size)
>> >> +{
>> >> +    RISCVAPLICState *aplic = opaque;
>> >> +    uint32_t irq, word, idc = UINT32_MAX;
>> >> +
>> >> +    /* Writes must be 4 byte words */
>> >> +    if ((addr & 0x3) != 0) {
>> >> +        goto err;
>> >> +    }
>> >> +
>> >> +    if (addr == APLIC_DOMAINCFG) {
>> >> +        /* Only IE bit writeable at the moment */
>> >> +        value &= APLIC_DOMAINCFG_IE;
>> >> +        aplic->domaincfg = value;
>> >> +    } else if ((APLIC_SOURCECFG_BASE <= addr) &&
>> >> +            (addr < (APLIC_SOURCECFG_BASE + (aplic->num_irqs - 1) * 4))) {
>> >> +        irq  = ((addr - APLIC_SOURCECFG_BASE) >> 2) + 1;
>> >> +        if (!aplic->num_children && (value & APLIC_SOURCECFG_D)) {
>> >> +            value = 0;
>> >> +        }
>> >> +        if (value & APLIC_SOURCECFG_D) {
>> >> +            value &= (APLIC_SOURCECFG_D | APLIC_SOURCECFG_CHILDIDX_MASK);
>> >> +        } else {
>> >> +            value &= (APLIC_SOURCECFG_D | APLIC_SOURCECFG_SM_MASK);
>> >> +        }
>> >> +        aplic->sourcecfg[irq] = value;
>> >> +        if ((aplic->sourcecfg[irq] & APLIC_SOURCECFG_D) ||
>> >> +            (aplic->sourcecfg[irq] == 0)) {
>> >> +            riscv_aplic_set_pending_raw(aplic, irq, false);
>> >> +            riscv_aplic_set_enabled_raw(aplic, irq, false);
>> >> +        }
>> >> +    } else if (aplic->mmode && aplic->msimode &&
>> >> +               (addr == APLIC_MMSICFGADDR)) {
>> >> +        if (!(aplic->mmsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
>> >> +            aplic->mmsicfgaddr = value;
>> >> +        }
>> >> +    } else if (aplic->mmode && aplic->msimode &&
>> >> +               (addr == APLIC_MMSICFGADDRH)) {
>> >> +        if (!(aplic->mmsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
>> >> +            aplic->mmsicfgaddrH = value & APLIC_xMSICFGADDRH_VALID_MASK;
>> >> +        }
>> >> +    } else if (aplic->mmode && aplic->msimode &&
>> >> +               (addr == APLIC_SMSICFGADDR)) {
>> >> +        /* Registers SMSICFGADDR and SMSICFGADDRH are implemented only if:
>> >> +         * (a) the interrupt domain is at machine level
>> >> +         * (b) the domain’s harts implement supervisor mode
>> >> +         * (c) the domain has one or more child supervisor-level domains
>> >> +         *     that support MSI delivery mode (domaincfg.DM is not read-
>> >> +         *     only zero in at least one of the supervisor-level child
>> >> +         * domains).
>> >> +         */
>> >> +        if (aplic->num_children &&
>> >> +            !(aplic->smsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
>> >> +            aplic->smsicfgaddr = value;
>> >> +        }
>> >> +    } else if (aplic->mmode && aplic->msimode &&
>> >> +               (addr == APLIC_SMSICFGADDRH)) {
>> >> +        if (aplic->num_children &&
>> >> +            !(aplic->smsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
>> >> +            aplic->smsicfgaddrH = value & APLIC_xMSICFGADDRH_VALID_MASK;
>> >> +        }
>> >> +    } else if ((APLIC_SETIP_BASE <= addr) &&
>> >> +            (addr < (APLIC_SETIP_BASE + aplic->bitfield_words * 4))) {
>> >> +        word = (addr - APLIC_SETIP_BASE) >> 2;
>> >> +        riscv_aplic_set_pending_word(aplic, word, value, true);
>> >> +    } else if (addr == APLIC_SETIPNUM) {
>> >> +        riscv_aplic_set_pending(aplic, value, true);
>> >> +    } else if ((APLIC_CLRIP_BASE <= addr) &&
>> >> +            (addr < (APLIC_CLRIP_BASE + aplic->bitfield_words * 4))) {
>> >> +        word = (addr - APLIC_CLRIP_BASE) >> 2;
>> >> +        riscv_aplic_set_pending_word(aplic, word, value, false);
>> >> +    } else if (addr == APLIC_CLRIPNUM) {
>> >> +        riscv_aplic_set_pending(aplic, value, false);
>> >> +    } else if ((APLIC_SETIE_BASE <= addr) &&
>> >> +            (addr < (APLIC_SETIE_BASE + aplic->bitfield_words * 4))) {
>> >> +        word = (addr - APLIC_SETIE_BASE) >> 2;
>> >> +        riscv_aplic_set_enabled_word(aplic, word, value, true);
>> >> +    } else if (addr == APLIC_SETIENUM) {
>> >> +        riscv_aplic_set_enabled(aplic, value, true);
>> >> +    } else if ((APLIC_CLRIE_BASE <= addr) &&
>> >> +            (addr < (APLIC_CLRIE_BASE + aplic->bitfield_words * 4))) {
>> >> +        word = (addr - APLIC_CLRIE_BASE) >> 2;
>> >> +        riscv_aplic_set_enabled_word(aplic, word, value, false);
>> >> +    } else if (addr == APLIC_CLRIENUM) {
>> >> +        riscv_aplic_set_enabled(aplic, value, false);
>> >> +    } else if (addr == APLIC_SETIPNUM_LE) {
>> >> +        riscv_aplic_set_pending(aplic, value, true);
>> >> +    } else if (addr == APLIC_SETIPNUM_BE) {
>> >> +        riscv_aplic_set_pending(aplic, bswap32(value), true);
>> >> +    } else if (addr == APLIC_GENMSI) {
>> >> +        if (aplic->msimode) {
>> >> +            aplic->genmsi = value & ~(APLIC_TARGET_GUEST_IDX_MASK <<
>> >> +                                      APLIC_TARGET_GUEST_IDX_SHIFT);
>> >> +            riscv_aplic_msi_send(aplic,
>> >> +                                 value >> APLIC_TARGET_HART_IDX_SHIFT,
>> >> +                                 0,
>> >> +                                 value & APLIC_TARGET_EIID_MASK);
>> >> +        }
>> >> +    } else if ((APLIC_TARGET_BASE <= addr) &&
>> >> +            (addr < (APLIC_TARGET_BASE + (aplic->num_irqs - 1) * 4))) {
>> >> +        irq = ((addr - APLIC_TARGET_BASE) >> 2) + 1;
>> >> +        if (aplic->msimode) {
>> >> +            aplic->target[irq] = value;
>> >> +        } else {
>> >> +            aplic->target[irq] = (value & ~APLIC_TARGET_IPRIO_MASK) |
>> >> +                                 ((value & aplic->iprio_mask) ?
>> >> +                                  (value & aplic->iprio_mask) : 1);
>> >> +        }
>> >> +    } else if (!aplic->msimode && (APLIC_IDC_BASE <= addr) &&
>> >> +            (addr < (APLIC_IDC_BASE + aplic->num_harts * APLIC_IDC_SIZE))) {
>> >> +        idc = (addr - APLIC_IDC_BASE) / APLIC_IDC_SIZE;
>> >> +        switch (addr - (APLIC_IDC_BASE + idc * APLIC_IDC_SIZE)) {
>> >> +        case APLIC_IDC_IDELIVERY:
>> >> +            aplic->idelivery[idc] = value & 0x1;
>> >> +            break;
>> >> +        case APLIC_IDC_IFORCE:
>> >> +            aplic->iforce[idc] = value & 0x1;
>> >> +            break;
>> >> +        case APLIC_IDC_ITHRESHOLD:
>> >> +            aplic->ithreshold[idc] = value & aplic->iprio_mask;
>> >> +            break;
>> >> +        default:
>> >> +            goto err;
>> >> +        };
>> >> +    } else {
>> >> +        goto err;
>> >> +    }
>> >> +
>> >> +    if (aplic->msimode) {
>> >> +        for (irq = 1; irq < aplic->num_irqs; irq++) {
>> >> +            riscv_aplic_msi_irq_update(aplic, irq);
>> >> +        }
>> >> +    } else {
>> >> +        if (idc == UINT32_MAX) {
>> >> +            for (idc = 0; idc < aplic->num_harts; idc++) {
>> >> +                riscv_aplic_idc_update(aplic, idc);
>> >> +            }
>> >> +        } else {
>> >> +            riscv_aplic_idc_update(aplic, idc);
>> >> +        }
>> >> +    }
>> >> +
>> >> +    return;
>> >> +
>> >> +err:
>> >> +    qemu_log_mask(LOG_GUEST_ERROR,
>> >> +                  "%s: Invalid register write 0x%" HWADDR_PRIx "\n",
>> >> +                  __func__, addr);
>> >> +}
>> >> +
>> >> +static const MemoryRegionOps riscv_aplic_ops = {
>> >> +    .read = riscv_aplic_read,
>> >> +    .write = riscv_aplic_write,
>> >> +    .endianness = DEVICE_LITTLE_ENDIAN,
>> >> +    .valid = {
>> >> +        .min_access_size = 4,
>> >> +        .max_access_size = 4
>> >> +    }
>> >> +};
>> >> +
>> >> +static void riscv_aplic_realize(DeviceState *dev, Error **errp)
>> >> +{
>> >> +    uint32_t i;
>> >> +    RISCVAPLICState *aplic = RISCV_APLIC(dev);
>> >> +
>> >> +    aplic->bitfield_words = (aplic->num_irqs + 31) >> 5;
>> >> +    aplic->sourcecfg = g_new0(uint32_t, aplic->num_irqs);
>> >> +    aplic->state = g_new(uint32_t, aplic->num_irqs);
>> >> +    aplic->target = g_new0(uint32_t, aplic->num_irqs);
>> >> +    if (!aplic->msimode) {
>> >> +        for (i = 0; i < aplic->num_irqs; i++) {
>> >> +            aplic->target[i] = 1;
>> >> +        }
>> >> +    }
>> >> +    aplic->idelivery = g_new0(uint32_t, aplic->num_harts);
>> >> +    aplic->iforce = g_new0(uint32_t, aplic->num_harts);
>> >> +    aplic->ithreshold = g_new0(uint32_t, aplic->num_harts);
>> >> +
>> >> +    memory_region_init_io(&aplic->mmio, OBJECT(dev), &riscv_aplic_ops, aplic,
>> >> +                          TYPE_RISCV_APLIC, aplic->aperture_size);
>> >> +    sysbus_init_mmio(SYS_BUS_DEVICE(dev), &aplic->mmio);
>> >> +
>> >> +    /* Only root APLICs have hardware IRQ lines. All non-root APLICs
>> >> +     * have IRQ lines delegated by their parent APLIC.
>> >> +     */
>> >> +    if (!aplic->parent) {
>> >> +        qdev_init_gpio_in(dev, riscv_aplic_request, aplic->num_irqs);
>> >> +    }
>> >> +
>> >> +    /* Create output IRQ lines for non-MSI mode */
>> >> +    if (!aplic->msimode) {
>> >> +        aplic->external_irqs = g_malloc(sizeof(qemu_irq) * aplic->num_harts);
>> >> +        qdev_init_gpio_out(dev, aplic->external_irqs, aplic->num_harts);
>> >> +
>> >> +        /* Claim the CPU interrupt to be triggered by this APLIC */
>> >> +        for (i = 0; i < aplic->num_harts; i++) {
>> >> +            RISCVCPU *cpu = RISCV_CPU(qemu_get_cpu(aplic->hartid_base + i));
>> >> +            if (riscv_cpu_claim_interrupts(cpu,
>> >> +                (aplic->mmode) ? MIP_MEIP : MIP_SEIP) < 0) {
>> >> +                error_report("%s already claimed",
>> >> +                             (aplic->mmode) ? "MEIP" : "SEIP");
>> >> +                exit(1);
>> >> +            }
>> >> +        }
>> >> +    }
>> >> +
>> >> +    msi_nonbroken = true;
>> >> +}
>> >> +
>> >> +static Property riscv_aplic_properties[] = {
>> >> +    DEFINE_PROP_UINT32("aperture-size", RISCVAPLICState, aperture_size, 0),
>> >> +    DEFINE_PROP_UINT32("hartid-base", RISCVAPLICState, hartid_base, 0),
>> >> +    DEFINE_PROP_UINT32("num-harts", RISCVAPLICState, num_harts, 0),
>> >> +    DEFINE_PROP_UINT32("iprio-mask", RISCVAPLICState, iprio_mask, 0),
>> >> +    DEFINE_PROP_UINT32("num-irqs", RISCVAPLICState, num_irqs, 0),
>> >> +    DEFINE_PROP_BOOL("msimode", RISCVAPLICState, msimode, 0),
>> >> +    DEFINE_PROP_BOOL("mmode", RISCVAPLICState, mmode, 0),
>> >> +    DEFINE_PROP_END_OF_LIST(),
>> >> +};
>> >> +
>> >> +static const VMStateDescription vmstate_riscv_aplic = {
>> >> +    .name = "riscv_aplic",
>> >> +    .version_id = 1,
>> >> +    .minimum_version_id = 1,
>> >> +    .fields = (VMStateField[]) {
>> >> +            VMSTATE_UINT32(domaincfg, RISCVAPLICState),
>> >> +            VMSTATE_UINT32(mmsicfgaddr, RISCVAPLICState),
>> >> +            VMSTATE_UINT32(mmsicfgaddrH, RISCVAPLICState),
>> >> +            VMSTATE_UINT32(smsicfgaddr, RISCVAPLICState),
>> >> +            VMSTATE_UINT32(smsicfgaddrH, RISCVAPLICState),
>> >> +            VMSTATE_UINT32(genmsi, RISCVAPLICState),
>> >> +            VMSTATE_VARRAY_UINT32(sourcecfg, RISCVAPLICState,
>> >> +                                  num_irqs, 0,
>> >> +                                  vmstate_info_uint32, uint32_t),
>> >> +            VMSTATE_VARRAY_UINT32(state, RISCVAPLICState,
>> >> +                                  num_irqs, 0,
>> >> +                                  vmstate_info_uint32, uint32_t),
>> >> +            VMSTATE_VARRAY_UINT32(target, RISCVAPLICState,
>> >> +                                  num_irqs, 0,
>> >> +                                  vmstate_info_uint32, uint32_t),
>> >> +            VMSTATE_VARRAY_UINT32(idelivery, RISCVAPLICState,
>> >> +                                  num_harts, 0,
>> >> +                                  vmstate_info_uint32, uint32_t),
>> >> +            VMSTATE_VARRAY_UINT32(iforce, RISCVAPLICState,
>> >> +                                  num_harts, 0,
>> >> +                                  vmstate_info_uint32, uint32_t),
>> >> +            VMSTATE_VARRAY_UINT32(ithreshold, RISCVAPLICState,
>> >> +                                  num_harts, 0,
>> >> +                                  vmstate_info_uint32, uint32_t),
>> >> +            VMSTATE_END_OF_LIST()
>> >> +        }
>> >> +};
>> >> +
>> >> +static void riscv_aplic_class_init(ObjectClass *klass, void *data)
>> >> +{
>> >> +    DeviceClass *dc = DEVICE_CLASS(klass);
>> >> +
>> >> +    device_class_set_props(dc, riscv_aplic_properties);
>> >> +    dc->realize = riscv_aplic_realize;
>> >> +    dc->vmsd = &vmstate_riscv_aplic;
>> >> +}
>> >> +
>> >> +static const TypeInfo riscv_aplic_info = {
>> >> +    .name          = TYPE_RISCV_APLIC,
>> >> +    .parent        = TYPE_SYS_BUS_DEVICE,
>> >> +    .instance_size = sizeof(RISCVAPLICState),
>> >> +    .class_init    = riscv_aplic_class_init,
>> >> +};
>> >> +
>> >> +static void riscv_aplic_register_types(void)
>> >> +{
>> >> +    type_register_static(&riscv_aplic_info);
>> >> +}
>> >> +
>> >> +type_init(riscv_aplic_register_types)
>> >> +
>> >> +/*
>> >> + * Add a APLIC device to another APLIC device as child for
>> >> + * interrupt delegation.
>> >> + */
>> >> +void riscv_aplic_add_child(DeviceState *parent, DeviceState *child)
>> >> +{
>> >> +    RISCVAPLICState *caplic, *paplic;
>> >> +
>> >> +    assert(parent && child);
>> >> +    caplic = RISCV_APLIC(child);
>> >> +    paplic = RISCV_APLIC(parent);
>> >> +
>> >> +    assert(paplic->num_irqs == caplic->num_irqs);
>> >> +    assert(paplic->num_children <= QEMU_APLIC_MAX_CHILDREN);
>> >> +
>> >> +    caplic->parent = paplic;
>> >> +    paplic->children[paplic->num_children] = caplic;
>> >> +    paplic->num_children++;
>> >> +}
>> >> +
>> >> +/*
>> >> + * Create APLIC device.
>> >> + */
>> >> +DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size,
>> >> +    uint32_t hartid_base, uint32_t num_harts, uint32_t num_sources,
>> >> +    uint32_t iprio_bits, bool msimode, bool mmode, DeviceState *parent)
>> >> +{
>> >> +    DeviceState *dev = qdev_new(TYPE_RISCV_APLIC);
>> >> +    uint32_t i;
>> >> +
>> >> +    assert(num_harts < APLIC_MAX_IDC);
>> >> +    assert((APLIC_IDC_BASE + (num_harts * APLIC_IDC_SIZE)) <= size);
>> >> +    assert(num_sources < APLIC_MAX_SOURCE);
>> >> +    assert(APLIC_MIN_IPRIO_BITS <= iprio_bits);
>> >> +    assert(iprio_bits <= APLIC_MAX_IPRIO_BITS);
>> >> +
>> >> +    qdev_prop_set_uint32(dev, "aperture-size", size);
>> >> +    qdev_prop_set_uint32(dev, "hartid-base", hartid_base);
>> >> +    qdev_prop_set_uint32(dev, "num-harts", num_harts);
>> >> +    qdev_prop_set_uint32(dev, "iprio-mask", ((1U << iprio_bits) - 1));
>> >> +    qdev_prop_set_uint32(dev, "num-irqs", num_sources + 1);
>> >> +    qdev_prop_set_bit(dev, "msimode", msimode);
>> >> +    qdev_prop_set_bit(dev, "mmode", mmode);
>> >> +
>> >> +    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
>> >> +    sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
>> >> +
>> >> +    if (parent) {
>> >> +        riscv_aplic_add_child(parent, dev);
>> >> +    }
>> >> +
>> >> +    if (!msimode) {
>> >> +        for (i = 0; i < num_harts; i++) {
>> >> +            CPUState *cpu = qemu_get_cpu(hartid_base + i);
>> >> +
>> >> +            qdev_connect_gpio_out_named(dev, NULL, i,
>> >> +                                        qdev_get_gpio_in(DEVICE(cpu),
>> >> +                                            (mmode) ? IRQ_M_EXT : IRQ_S_EXT));
>> >> +        }
>> >> +    }
>> >> +
>> >> +    return dev;
>> >> +}
>> >> diff --git a/include/hw/intc/riscv_aplic.h b/include/hw/intc/riscv_aplic.h
>> >> new file mode 100644
>> >> index 0000000000..de8532fbc3
>> >> --- /dev/null
>> >> +++ b/include/hw/intc/riscv_aplic.h
>> >> @@ -0,0 +1,79 @@
>> >> +/*
>> >> + * RISC-V APLIC (Advanced Platform Level Interrupt Controller) interface
>> >> + *
>> >> + * Copyright (c) 2021 Western Digital Corporation or its affiliates.
>> >> + *
>> >> + * This program is free software; you can redistribute it and/or modify it
>> >> + * under the terms and conditions of the GNU General Public License,
>> >> + * version 2 or later, as published by the Free Software Foundation.
>> >> + *
>> >> + * This program is distributed in the hope it will be useful, but WITHOUT
>> >> + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
>> >> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
>> >> + * more details.
>> >> + *
>> >> + * You should have received a copy of the GNU General Public License along with
>> >> + * this program.  If not, see <http://www.gnu.org/licenses/>.
>> >> + */
>> >> +
>> >> +#ifndef HW_RISCV_APLIC_H
>> >> +#define HW_RISCV_APLIC_H
>> >> +
>> >> +#include "hw/sysbus.h"
>> >> +#include "qom/object.h"
>> >> +
>> >> +#define TYPE_RISCV_APLIC "riscv.aplic"
>> >> +
>> >> +typedef struct RISCVAPLICState RISCVAPLICState;
>> >> +DECLARE_INSTANCE_CHECKER(RISCVAPLICState, RISCV_APLIC, TYPE_RISCV_APLIC)
>> >> +
>> >> +#define APLIC_MIN_SIZE            0x4000
>> >> +#define APLIC_SIZE_ALIGN(__x)     (((__x) + (APLIC_MIN_SIZE - 1)) & \
>> >> +                                   ~(APLIC_MIN_SIZE - 1))
>> >> +#define APLIC_SIZE(__num_harts)   (APLIC_MIN_SIZE + \
>> >> +                                   APLIC_SIZE_ALIGN(32 * (__num_harts)))
>> >> +
>> >> +struct RISCVAPLICState {
>> >> +    /*< private >*/
>> >> +    SysBusDevice parent_obj;
>> >> +    qemu_irq *external_irqs;
>> >> +
>> >> +    /*< public >*/
>> >> +    MemoryRegion mmio;
>> >> +    uint32_t bitfield_words;
>> >> +    uint32_t domaincfg;
>> >> +    uint32_t mmsicfgaddr;
>> >> +    uint32_t mmsicfgaddrH;
>> >> +    uint32_t smsicfgaddr;
>> >> +    uint32_t smsicfgaddrH;
>> >> +    uint32_t genmsi;
>> >> +    uint32_t *sourcecfg;
>> >> +    uint32_t *state;
>> >> +    uint32_t *target;
>> >> +    uint32_t *idelivery;
>> >> +    uint32_t *iforce;
>> >> +    uint32_t *ithreshold;
>> >> +
>> >> +    /* topology */
>> >> +#define QEMU_APLIC_MAX_CHILDREN        16
>> >> +    struct RISCVAPLICState *parent;
>> >> +    struct RISCVAPLICState *children[QEMU_APLIC_MAX_CHILDREN];
>> >> +    uint16_t num_children;
>> >> +
>> >> +    /* config */
>> >> +    uint32_t aperture_size;
>> >> +    uint32_t hartid_base;
>> >> +    uint32_t num_harts;
>> >> +    uint32_t iprio_mask;
>> >> +    uint32_t num_irqs;
>> >> +    bool msimode;
>> >> +    bool mmode;
>> >> +};
>> >> +
>> >> +void riscv_aplic_add_child(DeviceState *parent, DeviceState *child);
>> >> +
>> >> +DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size,
>> >> +    uint32_t hartid_base, uint32_t num_harts, uint32_t num_sources,
>> >> +    uint32_t iprio_bits, bool msimode, bool mmode, DeviceState *parent);
>> >> +
>> >> +#endif
>> >> --
>> >> 2.25.1
>> >>

Anup Patel Jan. 20, 2022, 12:11 p.m. UTC | #5

Hi Alistair,

On Wed, Jan 19, 2022 at 8:56 PM Anup Patel <[e-mail]> wrote:
>
> From: Anup Patel <[e-mail]>
>
> The advanced interrupt architecture (AIA) extends the per-HART local
> interrupt support. Along with this, it also adds IMSIC (MSI contrllor)
> and Advanced PLIC (wired interrupt controller).
>
> The latest AIA draft specification can be found here:
> https://github.com/riscv/riscv-aia/releases/download/0.2-draft.28/riscv-interrupts-028.pdf
>
> This series adds RISC-V AIA support in QEMU which includes emulating all
> AIA local CSRs, APLIC, and IMSIC. Only AIA local interrupt filtering is
> not implemented because we don't have any local interrupt greater than 12.
>
> To enable AIA in QEMU, use one of the following:
> 1) Only AIA local interrupt CSRs: Pass "x-aia=true" as CPU paramenter
>    in the QEMU command-line
> 2) Only APLIC for virt machine: Pass "aia=aplic" as machine parameter
>    in the QEMU command-line
> 3) Both APLIC and IMSIC for virt machine: Pass "aia=aplic-imsic" as
>    machine parameter in the QEMU command-line
> 4) Both APLIC and IMSIC with 2 guest files for virt machine: Pass
>    "aia=aplic-imsic,aia-guests=2" as machine parameter in the QEMU
>    command-line
>
> To test series, we require OpenSBI and Linux with AIA support which can
> be found in:
> riscv_aia_v2 branch at https://github.com/avpatel/opensbi.git
> riscv_aia_v1 branch at https://github.com/avpatel/linux.git
>
> This series can be found riscv_aia_v8 branch at:
> https://github.com/avpatel/qemu.git
>
> Changes since v7:
>  - Rebased on latest riscv-to-apply.next branch of Alistair's repo
>  - Improved default priority assignment in PATCH9
>
> Changes since v6:
>  - Fixed priority comparison in riscv_cpu_pending_to_irq() of PATCH9
>  - Fixed typos in comments added by PATCH11
>  - Added "pend = true;" for CSR_MSETEIPNUM case of rmw_xsetclreinum()
>    in PATCH15
>  - Handle ithreshold == 0 case in riscv_aplic_idc_topi() of PATCH18
>  - Allow setting pending bit for Level0 or Level1 interrupts in
>    riscv_aplic_set_pending() of PATCH18
>  - Force DOMAINCFG[31:24] bits to 0x80 in riscv_aplic_read() of PATCH18
>  - For APLIC direct mode, set target.iprio to 1 when zero is writtern
>    in PATCH18
>  - Handle eithreshold == 0 case in riscv_imsic_topei() of PATCH20
>
> Changes since v5:
>  - Moved VSTOPI_NUM_SRCS define to top of the file in PATCH13
>  - Fixed typo in PATCH16
>
> Changes since v4:
>  - Changed IRQ_LOCAL_MAX to 16 in PATCH2
>  - Fixed typo in PATCH10
>  - Replaced TARGET_LONG_BITS with riscv_cpu_mxl_bits(env) in PATCH11
>  - Replaced TARGET_LONG_BITS with riscv_cpu_mxl_bits(env) in PATCH14
>  - Replaced TARGET_LONG_BITS with riscv_cpu_mxl_bits(env) in PATCH15
>  - Replaced TARGET_LONG_BITS with xlen passed via ireg callback in PATCH20
>  - Retrict maximum IMSIC guest files per-HART of virt machine to 7 in
>    PATCH21.
>  - Added separate PATCH23 to increase maximum number of allowed CPUs
>    for virt machine
>
> Changes since v3:
>  - Replaced "aplic,xyz" and "imsic,xyz" DT properties with "riscv,xyz"
>    DT properties because "aplic" and "imsic" are not valid vendor names
>    required by Linux DT schema checker.
>
> Changes since v2:
>  - Update PATCH4 to check and inject interrupt after V=1 when
>    transitioning from V=0 to V=1
>
> Changes since v1:
>  - Revamped whole series and created more granular patches
>  - Added HGEIE and HGEIP CSR emulation for H-extension
>  - Added APLIC emulation
>  - Added IMSIC emulation
>
> Anup Patel (23):
>   target/riscv: Fix trap cause for RV32 HS-mode CSR access from RV64
>     HS-mode
>   target/riscv: Implement SGEIP bit in hip and hie CSRs
>   target/riscv: Implement hgeie and hgeip CSRs
>   target/riscv: Improve delivery of guest external interrupts
>   target/riscv: Allow setting CPU feature from machine/device emulation
>   target/riscv: Add AIA cpu feature
>   target/riscv: Add defines for AIA CSRs
>   target/riscv: Allow AIA device emulation to set ireg rmw callback
>   target/riscv: Implement AIA local interrupt priorities
>   target/riscv: Implement AIA CSRs for 64 local interrupts on RV32
>   target/riscv: Implement AIA hvictl and hviprioX CSRs
>   target/riscv: Implement AIA interrupt filtering CSRs
>   target/riscv: Implement AIA mtopi, stopi, and vstopi CSRs
>   target/riscv: Implement AIA xiselect and xireg CSRs
>   target/riscv: Implement AIA IMSIC interface CSRs
>   hw/riscv: virt: Use AIA INTC compatible string when available
>   target/riscv: Allow users to force enable AIA CSRs in HART
>   hw/intc: Add RISC-V AIA APLIC device emulation
>   hw/riscv: virt: Add optional AIA APLIC support to virt machine
>   hw/intc: Add RISC-V AIA IMSIC device emulation
>   hw/riscv: virt: Add optional AIA IMSIC support to virt machine
>   docs/system: riscv: Document AIA options for virt machine
>   hw/riscv: virt: Increase maximum number of allowed CPUs

Only PATCH19 and PATCH21 are pending for review.

Can this series be considered for riscv-to-apply.next ?

Regards,
Anup


>
>  docs/system/riscv/virt.rst    |   16 +
>  hw/intc/Kconfig               |    6 +
>  hw/intc/meson.build           |    2 +
>  hw/intc/riscv_aplic.c         |  975 +++++++++++++++++++++++++
>  hw/intc/riscv_imsic.c         |  448 ++++++++++++
>  hw/riscv/Kconfig              |    2 +
>  hw/riscv/virt.c               |  706 +++++++++++++++---
>  include/hw/intc/riscv_aplic.h |   79 ++
>  include/hw/intc/riscv_imsic.h |   68 ++
>  include/hw/riscv/virt.h       |   41 +-
>  target/riscv/cpu.c            |  103 ++-
>  target/riscv/cpu.h            |   72 +-
>  target/riscv/cpu_bits.h       |  123 ++++
>  target/riscv/cpu_helper.c     |  343 ++++++++-
>  target/riscv/csr.c            | 1280 ++++++++++++++++++++++++++++++---
>  target/riscv/machine.c        |   24 +-
>  16 files changed, 3988 insertions(+), 300 deletions(-)
>  create mode 100644 hw/intc/riscv_aplic.c
>  create mode 100644 hw/intc/riscv_imsic.c
>  create mode 100644 include/hw/intc/riscv_aplic.h
>  create mode 100644 include/hw/intc/riscv_imsic.h
>
> --
> 2.25.1
>

Frank Chang Jan. 20, 2022, 1:25 p.m. UTC | #6

On Thu, Jan 20, 2022 at 8:05 PM Anup Patel <[e-mail]> wrote:

> On Thu, Jan 20, 2022 at 1:49 PM Frank Chang <[e-mail]>
> wrote:
> >
> > On Thu, Jan 20, 2022 at 12:20 AM Anup Patel <[e-mail]> wrote:
> >>
> >> Hi Frank,
> >>
> >> On Wed, Jan 19, 2022 at 9:07 PM Frank Chang <[e-mail]>
> wrote:
> >> >
> >> > On Wed, Jan 19, 2022 at 11:27 PM Anup Patel <[e-mail]>
> wrote:
> >> >>
> >> >> From: Anup Patel <[e-mail]>
> >> >>
> >> >> The RISC-V AIA (Advanced Interrupt Architecture) defines a new
> >> >> interrupt controller for wired interrupts called APLIC (Advanced
> >> >> Platform Level Interrupt Controller). The APLIC is capabable of
> >> >> forwarding wired interupts to RISC-V HARTs directly or as MSIs
> >> >> (Message Signaled Interupts).
> >> >>
> >> >> This patch adds device emulation for RISC-V AIA APLIC.
> >> >>
> >> >> Signed-off-by: Anup Patel <[e-mail]>
> >> >> Signed-off-by: Anup Patel <[e-mail]>
> >> >> Reviewed-by: Frank Chang <[e-mail]>
> >> >> ---
> >> >>  hw/intc/Kconfig               |   3 +
> >> >>  hw/intc/meson.build           |   1 +
> >> >>  hw/intc/riscv_aplic.c         | 975
> ++++++++++++++++++++++++++++++++++
> >> >>  include/hw/intc/riscv_aplic.h |  79 +++
> >> >>  4 files changed, 1058 insertions(+)
> >> >>  create mode 100644 hw/intc/riscv_aplic.c
> >> >>  create mode 100644 include/hw/intc/riscv_aplic.h
> >> >>
> >> >> diff --git a/hw/intc/Kconfig b/hw/intc/Kconfig
> >> >> index 010ded7eae..528e77b4a6 100644
> >> >> --- a/hw/intc/Kconfig
> >> >> +++ b/hw/intc/Kconfig
> >> >> @@ -70,6 +70,9 @@ config LOONGSON_LIOINTC
> >> >>  config RISCV_ACLINT
> >> >>      bool
> >> >>
> >> >> +config RISCV_APLIC
> >> >> +    bool
> >> >> +
> >> >>  config SIFIVE_PLIC
> >> >>      bool
> >> >>
> >> >> diff --git a/hw/intc/meson.build b/hw/intc/meson.build
> >> >> index 70080bc161..7466024402 100644
> >> >> --- a/hw/intc/meson.build
> >> >> +++ b/hw/intc/meson.build
> >> >> @@ -50,6 +50,7 @@ specific_ss.add(when: 'CONFIG_S390_FLIC', if_true:
> files('s390_flic.c'))
> >> >>  specific_ss.add(when: 'CONFIG_S390_FLIC_KVM', if_true:
> files('s390_flic_kvm.c'))
> >> >>  specific_ss.add(when: 'CONFIG_SH_INTC', if_true: files('sh_intc.c'))
> >> >>  specific_ss.add(when: 'CONFIG_RISCV_ACLINT', if_true:
> files('riscv_aclint.c'))
> >> >> +specific_ss.add(when: 'CONFIG_RISCV_APLIC', if_true:
> files('riscv_aplic.c'))
> >> >>  specific_ss.add(when: 'CONFIG_SIFIVE_PLIC', if_true:
> files('sifive_plic.c'))
> >> >>  specific_ss.add(when: 'CONFIG_XICS', if_true: files('xics.c'))
> >> >>  specific_ss.add(when: ['CONFIG_KVM', 'CONFIG_XICS'],
> >> >> diff --git a/hw/intc/riscv_aplic.c b/hw/intc/riscv_aplic.c
> >> >> new file mode 100644
> >> >> index 0000000000..885c1de2af
> >> >> --- /dev/null
> >> >> +++ b/hw/intc/riscv_aplic.c
> >> >> @@ -0,0 +1,975 @@
> >> >> +/*
> >> >> + * RISC-V APLIC (Advanced Platform Level Interrupt Controller)
> >> >> + *
> >> >> + * Copyright (c) 2021 Western Digital Corporation or its affiliates.
> >> >> + *
> >> >> + * This program is free software; you can redistribute it and/or
> modify it
> >> >> + * under the terms and conditions of the GNU General Public License,
> >> >> + * version 2 or later, as published by the Free Software Foundation.
> >> >> + *
> >> >> + * This program is distributed in the hope it will be useful, but
> WITHOUT
> >> >> + * ANY WARRANTY; without even the implied warranty of
> MERCHANTABILITY or
> >> >> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public
> License for
> >> >> + * more details.
> >> >> + *
> >> >> + * You should have received a copy of the GNU General Public
> License along with
> >> >> + * this program.  If not, see <http://www.gnu.org/licenses/>.
> >> >> + */
> >> >> +
> >> >> +#include "qemu/osdep.h"
> >> >> +#include "qapi/error.h"
> >> >> +#include "qemu/log.h"
> >> >> +#include "qemu/module.h"
> >> >> +#include "qemu/error-report.h"
> >> >> +#include "qemu/bswap.h"
> >> >> +#include "exec/address-spaces.h"
> >> >> +#include "hw/sysbus.h"
> >> >> +#include "hw/pci/msi.h"
> >> >> +#include "hw/boards.h"
> >> >> +#include "hw/qdev-properties.h"
> >> >> +#include "hw/intc/riscv_aplic.h"
> >> >> +#include "hw/irq.h"
> >> >> +#include "target/riscv/cpu.h"
> >> >> +#include "sysemu/sysemu.h"
> >> >> +#include "migration/vmstate.h"
> >> >> +
> >> >> +#define APLIC_MAX_IDC                  (1UL << 14)
> >> >> +#define APLIC_MAX_SOURCE               1024
> >> >> +#define APLIC_MIN_IPRIO_BITS           1
> >> >> +#define APLIC_MAX_IPRIO_BITS           8
> >> >> +#define APLIC_MAX_CHILDREN             1024
> >> >> +
> >> >> +#define APLIC_DOMAINCFG                0x0000
> >> >> +#define APLIC_DOMAINCFG_RDONLY         0x80000000
> >> >> +#define APLIC_DOMAINCFG_IE             (1 << 8)
> >> >> +#define APLIC_DOMAINCFG_DM             (1 << 2)
> >> >> +#define APLIC_DOMAINCFG_BE             (1 << 0)
> >> >> +
> >> >> +#define APLIC_SOURCECFG_BASE           0x0004
> >> >> +#define APLIC_SOURCECFG_D              (1 << 10)
> >> >> +#define APLIC_SOURCECFG_CHILDIDX_MASK  0x000003ff
> >> >> +#define APLIC_SOURCECFG_SM_MASK        0x00000007
> >> >> +#define APLIC_SOURCECFG_SM_INACTIVE    0x0
> >> >> +#define APLIC_SOURCECFG_SM_DETACH      0x1
> >> >> +#define APLIC_SOURCECFG_SM_EDGE_RISE   0x4
> >> >> +#define APLIC_SOURCECFG_SM_EDGE_FALL   0x5
> >> >> +#define APLIC_SOURCECFG_SM_LEVEL_HIGH  0x6
> >> >> +#define APLIC_SOURCECFG_SM_LEVEL_LOW   0x7
> >> >> +
> >> >> +#define APLIC_MMSICFGADDR              0x1bc0
> >> >> +#define APLIC_MMSICFGADDRH             0x1bc4
> >> >> +#define APLIC_SMSICFGADDR              0x1bc8
> >> >> +#define APLIC_SMSICFGADDRH             0x1bcc
> >> >> +
> >> >> +#define APLIC_xMSICFGADDRH_L           (1UL << 31)
> >> >> +#define APLIC_xMSICFGADDRH_HHXS_MASK   0x1f
> >> >> +#define APLIC_xMSICFGADDRH_HHXS_SHIFT  24
> >> >> +#define APLIC_xMSICFGADDRH_LHXS_MASK   0x7
> >> >> +#define APLIC_xMSICFGADDRH_LHXS_SHIFT  20
> >> >> +#define APLIC_xMSICFGADDRH_HHXW_MASK   0x7
> >> >> +#define APLIC_xMSICFGADDRH_HHXW_SHIFT  16
> >> >> +#define APLIC_xMSICFGADDRH_LHXW_MASK   0xf
> >> >> +#define APLIC_xMSICFGADDRH_LHXW_SHIFT  12
> >> >> +#define APLIC_xMSICFGADDRH_BAPPN_MASK  0xfff
> >> >> +
> >> >> +#define APLIC_xMSICFGADDR_PPN_SHIFT    12
> >> >> +
> >> >> +#define APLIC_xMSICFGADDR_PPN_HART(__lhxs) \
> >> >> +    ((1UL << (__lhxs)) - 1)
> >> >> +
> >> >> +#define APLIC_xMSICFGADDR_PPN_LHX_MASK(__lhxw) \
> >> >> +    ((1UL << (__lhxw)) - 1)
> >> >> +#define APLIC_xMSICFGADDR_PPN_LHX_SHIFT(__lhxs) \
> >> >> +    ((__lhxs))
> >> >> +#define APLIC_xMSICFGADDR_PPN_LHX(__lhxw, __lhxs) \
> >> >> +    (APLIC_xMSICFGADDR_PPN_LHX_MASK(__lhxw) << \
> >> >> +     APLIC_xMSICFGADDR_PPN_LHX_SHIFT(__lhxs))
> >> >> +
> >> >> +#define APLIC_xMSICFGADDR_PPN_HHX_MASK(__hhxw) \
> >> >> +    ((1UL << (__hhxw)) - 1)
> >> >> +#define APLIC_xMSICFGADDR_PPN_HHX_SHIFT(__hhxs) \
> >> >> +    ((__hhxs) + APLIC_xMSICFGADDR_PPN_SHIFT)
> >> >> +#define APLIC_xMSICFGADDR_PPN_HHX(__hhxw, __hhxs) \
> >> >> +    (APLIC_xMSICFGADDR_PPN_HHX_MASK(__hhxw) << \
> >> >> +     APLIC_xMSICFGADDR_PPN_HHX_SHIFT(__hhxs))
> >> >> +
> >> >> +#define APLIC_xMSICFGADDRH_VALID_MASK   \
> >> >> +    (APLIC_xMSICFGADDRH_L | \
> >> >> +     (APLIC_xMSICFGADDRH_HHXS_MASK <<
> APLIC_xMSICFGADDRH_HHXS_SHIFT) | \
> >> >> +     (APLIC_xMSICFGADDRH_LHXS_MASK <<
> APLIC_xMSICFGADDRH_LHXS_SHIFT) | \
> >> >> +     (APLIC_xMSICFGADDRH_HHXW_MASK <<
> APLIC_xMSICFGADDRH_HHXW_SHIFT) | \
> >> >> +     (APLIC_xMSICFGADDRH_LHXW_MASK <<
> APLIC_xMSICFGADDRH_LHXW_SHIFT) | \
> >> >> +     APLIC_xMSICFGADDRH_BAPPN_MASK)
> >> >> +
> >> >> +#define APLIC_SETIP_BASE               0x1c00
> >> >> +#define APLIC_SETIPNUM                 0x1cdc
> >> >> +
> >> >> +#define APLIC_CLRIP_BASE               0x1d00
> >> >> +#define APLIC_CLRIPNUM                 0x1ddc
> >> >> +
> >> >> +#define APLIC_SETIE_BASE               0x1e00
> >> >> +#define APLIC_SETIENUM                 0x1edc
> >> >> +
> >> >> +#define APLIC_CLRIE_BASE               0x1f00
> >> >> +#define APLIC_CLRIENUM                 0x1fdc
> >> >> +
> >> >> +#define APLIC_SETIPNUM_LE              0x2000
> >> >> +#define APLIC_SETIPNUM_BE              0x2004
> >> >> +
> >> >> +#define APLIC_ISTATE_PENDING           (1U << 0)
> >> >> +#define APLIC_ISTATE_ENABLED           (1U << 1)
> >> >> +#define APLIC_ISTATE_ENPEND            (APLIC_ISTATE_ENABLED | \
> >> >> +                                        APLIC_ISTATE_PENDING)
> >> >> +#define APLIC_ISTATE_INPUT             (1U << 8)
> >> >> +
> >> >> +#define APLIC_GENMSI                   0x3000
> >> >> +
> >> >> +#define APLIC_TARGET_BASE              0x3004
> >> >> +#define APLIC_TARGET_HART_IDX_SHIFT    18
> >> >> +#define APLIC_TARGET_HART_IDX_MASK     0x3fff
> >> >> +#define APLIC_TARGET_GUEST_IDX_SHIFT   12
> >> >> +#define APLIC_TARGET_GUEST_IDX_MASK    0x3f
> >> >> +#define APLIC_TARGET_IPRIO_MASK        0xff
> >> >> +#define APLIC_TARGET_EIID_MASK         0x7ff
> >> >> +
> >> >> +#define APLIC_IDC_BASE                 0x4000
> >> >> +#define APLIC_IDC_SIZE                 32
> >> >> +
> >> >> +#define APLIC_IDC_IDELIVERY            0x00
> >> >> +
> >> >> +#define APLIC_IDC_IFORCE               0x04
> >> >> +
> >> >> +#define APLIC_IDC_ITHRESHOLD           0x08
> >> >> +
> >> >> +#define APLIC_IDC_TOPI                 0x18
> >> >> +#define APLIC_IDC_TOPI_ID_SHIFT        16
> >> >> +#define APLIC_IDC_TOPI_ID_MASK         0x3ff
> >> >> +#define APLIC_IDC_TOPI_PRIO_MASK       0xff
> >> >> +
> >> >> +#define APLIC_IDC_CLAIMI               0x1c
> >> >> +
> >> >> +static uint32_t riscv_aplic_read_input_word(RISCVAPLICState *aplic,
> >> >> +                                            uint32_t word)
> >> >> +{
> >> >> +    uint32_t i, irq, ret = 0;
> >> >> +
> >> >> +    for (i = 0; i < 32; i++) {
> >> >> +        irq = word * 32 + i;
> >> >> +        if (!irq || aplic->num_irqs <= irq) {
> >> >> +            continue;
> >> >> +        }
> >> >> +
> >> >> +        ret |= ((aplic->state[irq] & APLIC_ISTATE_INPUT) ? 1 : 0)
> << i;
> >> >> +    }
> >> >> +
> >> >> +    return ret;
> >> >> +}
> >> >> +
> >> >> +static uint32_t riscv_aplic_read_pending_word(RISCVAPLICState
> *aplic,
> >> >> +                                              uint32_t word)
> >> >> +{
> >> >> +    uint32_t i, irq, ret = 0;
> >> >> +
> >> >> +    for (i = 0; i < 32; i++) {
> >> >> +        irq = word * 32 + i;
> >> >> +        if (!irq || aplic->num_irqs <= irq) {
> >> >> +            continue;
> >> >> +        }
> >> >> +
> >> >> +        ret |= ((aplic->state[irq] & APLIC_ISTATE_PENDING) ? 1 : 0)
> << i;
> >> >> +    }
> >> >> +
> >> >> +    return ret;
> >> >> +}
> >> >> +
> >> >> +static void riscv_aplic_set_pending_raw(RISCVAPLICState *aplic,
> >> >> +                                        uint32_t irq, bool pending)
> >> >> +{
> >> >> +    if (pending) {
> >> >> +        aplic->state[irq] |= APLIC_ISTATE_PENDING;
> >> >> +    } else {
> >> >> +        aplic->state[irq] &= ~APLIC_ISTATE_PENDING;
> >> >> +    }
> >> >> +}
> >> >> +
> >> >> +static void riscv_aplic_set_pending(RISCVAPLICState *aplic,
> >> >> +                                    uint32_t irq, bool pending)
> >> >> +{
> >> >> +    uint32_t sourcecfg, sm;
> >> >> +
> >> >> +    if ((irq <= 0) || (aplic->num_irqs <= irq)) {
> >> >> +        return;
> >> >> +    }
> >> >> +
> >> >> +    sourcecfg = aplic->sourcecfg[irq];
> >> >> +    if (sourcecfg & APLIC_SOURCECFG_D) {
> >> >> +        return;
> >> >> +    }
> >> >> +
> >> >> +    sm = sourcecfg & APLIC_SOURCECFG_SM_MASK;
> >> >> +    if ((sm == APLIC_SOURCECFG_SM_INACTIVE) ||
> >> >> +        ((!aplic->msimode || (aplic->msimode && !pending)) &&
> >> >> +         ((sm == APLIC_SOURCECFG_SM_LEVEL_HIGH) ||
> >> >> +          (sm == APLIC_SOURCECFG_SM_LEVEL_LOW)))) {
> >> >> +        return;
> >> >> +    }
> >> >> +
> >> >> +    riscv_aplic_set_pending_raw(aplic, irq, pending);
> >> >> +}
> >> >> +
> >> >> +static void riscv_aplic_set_pending_word(RISCVAPLICState *aplic,
> >> >> +                                         uint32_t word, uint32_t
> value,
> >> >> +                                         bool pending)
> >> >> +{
> >> >> +    uint32_t i, irq;
> >> >> +
> >> >> +    for (i = 0; i < 32; i++) {
> >> >> +        irq = word * 32 + i;
> >> >> +        if (!irq || aplic->num_irqs <= irq) {
> >> >> +            continue;
> >> >> +        }
> >> >> +
> >> >> +        if (value & (1U << i)) {
> >> >> +            riscv_aplic_set_pending(aplic, irq, pending);
> >> >> +        }
> >> >> +    }
> >> >> +}
> >> >> +
> >> >> +static uint32_t riscv_aplic_read_enabled_word(RISCVAPLICState
> *aplic,
> >> >> +                                              int word)
> >> >> +{
> >> >> +    uint32_t i, irq, ret = 0;
> >> >> +
> >> >> +    for (i = 0; i < 32; i++) {
> >> >> +        irq = word * 32 + i;
> >> >> +        if (!irq || aplic->num_irqs <= irq) {
> >> >> +            continue;
> >> >> +        }
> >> >> +
> >> >> +        ret |= ((aplic->state[irq] & APLIC_ISTATE_ENABLED) ? 1 : 0)
> << i;
> >> >> +    }
> >> >> +
> >> >> +    return ret;
> >> >> +}
> >> >> +
> >> >> +static void riscv_aplic_set_enabled_raw(RISCVAPLICState *aplic,
> >> >> +                                        uint32_t irq, bool enabled)
> >> >> +{
> >> >> +    if (enabled) {
> >> >> +        aplic->state[irq] |= APLIC_ISTATE_ENABLED;
> >> >> +    } else {
> >> >> +        aplic->state[irq] &= ~APLIC_ISTATE_ENABLED;
> >> >> +    }
> >> >> +}
> >> >> +
> >> >> +static void riscv_aplic_set_enabled(RISCVAPLICState *aplic,
> >> >> +                                    uint32_t irq, bool enabled)
> >> >> +{
> >> >> +    uint32_t sourcecfg, sm;
> >> >> +
> >> >> +    if ((irq <= 0) || (aplic->num_irqs <= irq)) {
> >> >> +        return;
> >> >> +    }
> >> >> +
> >> >> +    sourcecfg = aplic->sourcecfg[irq];
> >> >> +    if (sourcecfg & APLIC_SOURCECFG_D) {
> >> >> +        return;
> >> >> +    }
> >> >> +
> >> >> +    sm = sourcecfg & APLIC_SOURCECFG_SM_MASK;
> >> >> +    if (sm == APLIC_SOURCECFG_SM_INACTIVE) {
> >> >> +        return;
> >> >> +    }
> >> >> +
> >> >> +    riscv_aplic_set_enabled_raw(aplic, irq, enabled);
> >> >> +}
> >> >> +
> >> >> +static void riscv_aplic_set_enabled_word(RISCVAPLICState *aplic,
> >> >> +                                         uint32_t word, uint32_t
> value,
> >> >> +                                         bool enabled)
> >> >> +{
> >> >> +    uint32_t i, irq;
> >> >> +
> >> >> +    for (i = 0; i < 32; i++) {
> >> >> +        irq = word * 32 + i;
> >> >> +        if (!irq || aplic->num_irqs <= irq) {
> >> >> +            continue;
> >> >> +        }
> >> >> +
> >> >> +        if (value & (1U << i)) {
> >> >> +            riscv_aplic_set_enabled(aplic, irq, enabled);
> >> >> +        }
> >> >> +    }
> >> >> +}
> >> >> +
> >> >> +static void riscv_aplic_msi_send(RISCVAPLICState *aplic,
> >> >> +                                 uint32_t hart_idx, uint32_t
> guest_idx,
> >> >> +                                 uint32_t eiid)
> >> >> +{
> >> >> +    uint64_t addr;
> >> >> +    MemTxResult result;
> >> >> +    RISCVAPLICState *aplic_m;
> >> >> +    uint32_t lhxs, lhxw, hhxs, hhxw, group_idx, msicfgaddr,
> msicfgaddrH;
> >> >> +
> >> >> +    aplic_m = aplic;
> >> >> +    while (aplic_m && !aplic_m->mmode) {
> >> >> +        aplic_m = aplic_m->parent;
> >> >> +    }
> >> >> +    if (!aplic_m) {
> >> >> +        qemu_log_mask(LOG_GUEST_ERROR, "%s: m-level APLIC not
> found\n",
> >> >> +                      __func__);
> >> >> +        return;
> >> >> +    }
> >> >> +
> >> >> +    if (aplic->mmode) {
> >> >> +        msicfgaddr = aplic_m->mmsicfgaddr;
> >> >> +        msicfgaddrH = aplic_m->mmsicfgaddrH;
> >> >> +    } else {
> >> >> +        msicfgaddr = aplic_m->smsicfgaddr;
> >> >> +        msicfgaddrH = aplic_m->smsicfgaddrH;
> >> >> +    }
> >> >> +
> >> >> +    lhxs = (msicfgaddrH >> APLIC_xMSICFGADDRH_LHXS_SHIFT) &
> >> >> +            APLIC_xMSICFGADDRH_LHXS_MASK;
> >> >> +    lhxw = (msicfgaddrH >> APLIC_xMSICFGADDRH_LHXW_SHIFT) &
> >> >> +            APLIC_xMSICFGADDRH_LHXW_MASK;
> >> >> +    hhxs = (msicfgaddrH >> APLIC_xMSICFGADDRH_HHXS_SHIFT) &
> >> >> +            APLIC_xMSICFGADDRH_HHXS_MASK;
> >> >> +    hhxw = (msicfgaddrH >> APLIC_xMSICFGADDRH_HHXW_SHIFT) &
> >> >> +            APLIC_xMSICFGADDRH_HHXW_MASK;
> >> >> +
> >> >> +    group_idx = hart_idx >> lhxw;
> >> >> +    hart_idx &= APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw);
> >> >> +
> >> >> +    addr = msicfgaddr;
> >> >> +    addr |= ((uint64_t)(msicfgaddrH &
> APLIC_xMSICFGADDRH_BAPPN_MASK)) << 32;
> >> >> +    addr |= ((uint64_t)(group_idx &
> APLIC_xMSICFGADDR_PPN_HHX_MASK(hhxw))) <<
> >> >> +             APLIC_xMSICFGADDR_PPN_HHX_SHIFT(hhxs);
> >> >> +    addr |= ((uint64_t)(hart_idx &
> APLIC_xMSICFGADDR_PPN_LHX_MASK(lhxw))) <<
> >> >> +             APLIC_xMSICFGADDR_PPN_LHX_SHIFT(lhxs);
> >> >> +    addr |= (uint64_t)(guest_idx &
> APLIC_xMSICFGADDR_PPN_HART(lhxs));
> >> >> +    addr <<= APLIC_xMSICFGADDR_PPN_SHIFT;
> >> >> +
> >> >> +    address_space_stl_le(&address_space_memory, addr,
> >> >> +                         eiid, MEMTXATTRS_UNSPECIFIED, &result);
> >> >> +    if (result != MEMTX_OK) {
> >> >> +        qemu_log_mask(LOG_GUEST_ERROR, "%s: MSI write failed for "
> >> >> +                      "hart_index=%d guest_index=%d eiid=%d\n",
> >> >> +                      __func__, hart_idx, guest_idx, eiid);
> >> >> +    }
> >> >> +}
> >> >> +
> >> >> +static void riscv_aplic_msi_irq_update(RISCVAPLICState *aplic,
> uint32_t irq)
> >> >> +{
> >> >> +    uint32_t hart_idx, guest_idx, eiid;
> >> >> +
> >> >> +    if (!aplic->msimode || (aplic->num_irqs <= irq) ||
> >> >> +        !(aplic->domaincfg & APLIC_DOMAINCFG_IE)) {
> >> >> +        return;
> >> >> +    }
> >> >> +
> >> >> +    if ((aplic->state[irq] & APLIC_ISTATE_ENPEND) !=
> APLIC_ISTATE_ENPEND) {
> >> >> +        return;
> >> >> +    }
> >> >> +
> >> >> +    riscv_aplic_set_pending_raw(aplic, irq, false);
> >> >> +
> >> >> +    hart_idx = aplic->target[irq] >> APLIC_TARGET_HART_IDX_SHIFT;
> >> >> +    hart_idx &= APLIC_TARGET_HART_IDX_MASK;
> >> >
> >> >
> >> > Hi Anup,
> >> >
> >> > I'm wondering does the hart_idx here has to use Machine-level hart
> index
> >> > for Supervisor-level interrupt domain?
> >> >
> >> > According to AIA spec for supervisor-level domin (Section 4.8.1):
> >> >   For a supervisor-level domain, "if" MSI target addresses are
> determined by the parent domain’s
> >> >   smsiaddrcfg and smsiaddrcfgh registers, then to construct the
> address for an outgoing MSI for
> >> >   interrupt source i, the Hart Index from register target[i] must
> first be converted into the index
> >> >   that the machine-level parent domain uses for the same hart.
> (Typically, these numbers are the
> >> >   same, but they may not be.) The address for the MSI is then
> computed using this "machine-level"
> >> >   hart index, the parent domain’s smsiaddrcfg and smsiaddrcfgh, and
> the Guest Index value from
> >> >   target[i].
> >> >
> >> > The description of converting target[i] hart Index to machine-level
> hart index in the spec
> >> > is quite vague to me.
> >>
> >> The "Section 4.3 Hart index numbers" states the following:
> >>
> >>     The index number a domain associates with a hart may or may not
> >> have any relationship
> >>     to the unique hart identifier (“hart ID”) that the RISC-V
> >> Privileged Architecture assigns to
> >>     the hart.
> >>
> >> For APLIC direct mode, if N harts are associated with a given domain
> >> then APLIC associates
> >> hart index from 0 to N-1 which are basically APLIC IDC numbers. In
> >> this case, the APLIC
> >> hart index is not related to "machine-level" hart index so software
> >> has to discover it from the
> >> "interrupt-extended" DT property (or similar for ACPI).
> >>
> >> For APLIC msi mode, the APLIC can basically target any HART using
> information in
> >> [m|s]msiaddrcfg registers and interrupt target registers. In this
> >> case, the APLIC hart
> >> index in the APLIC target register is a combination of IMSIC group
> >> index (g) and IMSIC
> >> hart index (h) and the APLIC target register also provides IMSIC guest
> >> index. In other
> >> words, the APLIC hart index for msi mode is also not related to the
> >> "machine-level" hart
> >> index. The APLIC driver will figure-out/extract the IMSIC group index
> >> (g), IMSIC hart
> >> index (h), and IMSIC guest index from target MSI address when Linux
> >> tries to write
> >> target MSI address for a particular APLIC interrupt.
> >>
> >> Regards,
> >> Anup
> >
> >
> > Hi Anup,
> >
> > Thanks for the detailed explanation.
> >
> > I think it's the formula in the AIA spec which confused me:
> >   g = (machine-level hart index >> LHXW) & (2^HHXW − 1)
> >   h = machine-level hart index & (2^LHXW − 1)
> >   MSI address =  (Base PPN | (g << (HHXS + 12)) | (h << LHXS) | Guest
> Index) << 12
> >
> > The term "machine-level hart index" makes me thought that
> > we should use the hart index from parent machine-level interrupt domain.
>
> Yes, the term "machine-level hart index" is confusing. In reality, it is
> the
> "hart index" in the APLIC interrupt target registers.
>
> >
> > So it's driver's responsibility to figure out the correct IMSIC hart
> index, guest index
> > it wants to forward the MSI to and configure them to the target[i]
> register?
>
> Yes, that's correct.
>
> >
> > From hardware (QEMU)'s perspective, it just needs to extract
> > the Hart index, Guest index out from target[irq] register in IRQ's
> interrupt domain
> > and calculate the MSI address with the PPN from parent interrupt domain's
> > m/smsiaddrcfg and m/smsiaddrcfgh?
>
> Yes, that's correct.
>
> The [m|s]msiaddrcfg registers are only available in the root APLIC domain
> and M-mode runtime firmware (OpenSBI) has to initialize it correctly.
>
> The APLIC Linux/OS driver has to ensure that APLIC interrupt target
> registers
> are programmed properly so that APLIC can generate the correct output MSI
> address.
>
> Regards,
> Anup
>

Thanks Anup,

That's really helpful :)

Regards,
Frank Chang

>
> >
> > Regards,
> > Frank Chang
> >
> >>
> >>
> >> >
> >> > Regards,
> >> > Frank Chang
> >> >
> >> >>
> >> >> +    if (aplic->mmode) {
> >> >> +        /* M-level APLIC ignores guest_index */
> >> >> +        guest_idx = 0;
> >> >> +    } else {
> >> >> +        guest_idx = aplic->target[irq] >>
> APLIC_TARGET_GUEST_IDX_SHIFT;
> >> >> +        guest_idx &= APLIC_TARGET_GUEST_IDX_MASK;
> >> >> +    }
> >> >> +    eiid = aplic->target[irq] & APLIC_TARGET_EIID_MASK;
> >> >> +    riscv_aplic_msi_send(aplic, hart_idx, guest_idx, eiid);
> >> >> +}
> >> >> +
> >> >> +static uint32_t riscv_aplic_idc_topi(RISCVAPLICState *aplic,
> uint32_t idc)
> >> >> +{
> >> >> +    uint32_t best_irq, best_iprio;
> >> >> +    uint32_t irq, iprio, ihartidx, ithres;
> >> >> +
> >> >> +    if (aplic->num_harts <= idc) {
> >> >> +        return 0;
> >> >> +    }
> >> >> +
> >> >> +    ithres = aplic->ithreshold[idc];
> >> >> +    best_irq = best_iprio = UINT32_MAX;
> >> >> +    for (irq = 1; irq < aplic->num_irqs; irq++) {
> >> >> +        if ((aplic->state[irq] & APLIC_ISTATE_ENPEND) !=
> >> >> +            APLIC_ISTATE_ENPEND) {
> >> >> +            continue;
> >> >> +        }
> >> >> +
> >> >> +        ihartidx = aplic->target[irq] >>
> APLIC_TARGET_HART_IDX_SHIFT;
> >> >> +        ihartidx &= APLIC_TARGET_HART_IDX_MASK;
> >> >> +        if (ihartidx != idc) {
> >> >> +            continue;
> >> >> +        }
> >> >> +
> >> >> +        iprio = aplic->target[irq] & aplic->iprio_mask;
> >> >> +        if (ithres && iprio >= ithres) {
> >> >> +            continue;
> >> >> +        }
> >> >> +
> >> >> +        if (iprio < best_iprio) {
> >> >> +            best_irq = irq;
> >> >> +            best_iprio = iprio;
> >> >> +        }
> >> >> +    }
> >> >> +
> >> >> +    if (best_irq < aplic->num_irqs && best_iprio <=
> aplic->iprio_mask) {
> >> >> +        return (best_irq << APLIC_IDC_TOPI_ID_SHIFT) | best_iprio;
> >> >> +    }
> >> >> +
> >> >> +    return 0;
> >> >> +}
> >> >> +
> >> >> +static void riscv_aplic_idc_update(RISCVAPLICState *aplic, uint32_t
> idc)
> >> >> +{
> >> >> +    uint32_t topi;
> >> >> +
> >> >> +    if (aplic->msimode || aplic->num_harts <= idc) {
> >> >> +        return;
> >> >> +    }
> >> >> +
> >> >> +    topi = riscv_aplic_idc_topi(aplic, idc);
> >> >> +    if ((aplic->domaincfg & APLIC_DOMAINCFG_IE) &&
> >> >> +        aplic->idelivery[idc] &&
> >> >> +        (aplic->iforce[idc] || topi)) {
> >> >> +        qemu_irq_raise(aplic->external_irqs[idc]);
> >> >> +    } else {
> >> >> +        qemu_irq_lower(aplic->external_irqs[idc]);
> >> >> +    }
> >> >> +}
> >> >> +
> >> >> +static uint32_t riscv_aplic_idc_claimi(RISCVAPLICState *aplic,
> uint32_t idc)
> >> >> +{
> >> >> +    uint32_t irq, state, sm, topi = riscv_aplic_idc_topi(aplic,
> idc);
> >> >> +
> >> >> +    if (!topi) {
> >> >> +        aplic->iforce[idc] = 0;
> >> >> +        return 0;
> >> >> +    }
> >> >> +
> >> >> +    irq = (topi >> APLIC_IDC_TOPI_ID_SHIFT) &
> APLIC_IDC_TOPI_ID_MASK;
> >> >> +    sm = aplic->sourcecfg[irq] & APLIC_SOURCECFG_SM_MASK;
> >> >> +    state = aplic->state[irq];
> >> >> +    riscv_aplic_set_pending_raw(aplic, irq, false);
> >> >> +    if ((sm == APLIC_SOURCECFG_SM_LEVEL_HIGH) &&
> >> >> +        (state & APLIC_ISTATE_INPUT)) {
> >> >> +        riscv_aplic_set_pending_raw(aplic, irq, true);
> >> >> +    } else if ((sm == APLIC_SOURCECFG_SM_LEVEL_LOW) &&
> >> >> +               !(state & APLIC_ISTATE_INPUT)) {
> >> >> +        riscv_aplic_set_pending_raw(aplic, irq, true);
> >> >> +    }
> >> >> +    riscv_aplic_idc_update(aplic, idc);
> >> >> +
> >> >> +    return topi;
> >> >> +}
> >> >> +
> >> >> +static void riscv_aplic_request(void *opaque, int irq, int level)
> >> >> +{
> >> >> +    bool update = false;
> >> >> +    RISCVAPLICState *aplic = opaque;
> >> >> +    uint32_t sourcecfg, childidx, state, idc;
> >> >> +
> >> >> +    assert((0 < irq) && (irq < aplic->num_irqs));
> >> >> +
> >> >> +    sourcecfg = aplic->sourcecfg[irq];
> >> >> +    if (sourcecfg & APLIC_SOURCECFG_D) {
> >> >> +        childidx = sourcecfg & APLIC_SOURCECFG_CHILDIDX_MASK;
> >> >> +        if (childidx < aplic->num_children) {
> >> >> +            riscv_aplic_request(aplic->children[childidx], irq,
> level);
> >> >> +        }
> >> >> +        return;
> >> >> +    }
> >> >> +
> >> >> +    state = aplic->state[irq];
> >> >> +    switch (sourcecfg & APLIC_SOURCECFG_SM_MASK) {
> >> >> +    case APLIC_SOURCECFG_SM_EDGE_RISE:
> >> >> +        if ((level > 0) && !(state & APLIC_ISTATE_INPUT) &&
> >> >> +            !(state & APLIC_ISTATE_PENDING)) {
> >> >> +            riscv_aplic_set_pending_raw(aplic, irq, true);
> >> >> +            update = true;
> >> >> +        }
> >> >> +        break;
> >> >> +    case APLIC_SOURCECFG_SM_EDGE_FALL:
> >> >> +        if ((level <= 0) && (state & APLIC_ISTATE_INPUT) &&
> >> >> +            !(state & APLIC_ISTATE_PENDING)) {
> >> >> +            riscv_aplic_set_pending_raw(aplic, irq, true);
> >> >> +            update = true;
> >> >> +        }
> >> >> +        break;
> >> >> +    case APLIC_SOURCECFG_SM_LEVEL_HIGH:
> >> >> +        if ((level > 0) && !(state & APLIC_ISTATE_PENDING)) {
> >> >> +            riscv_aplic_set_pending_raw(aplic, irq, true);
> >> >> +            update = true;
> >> >> +        }
> >> >> +        break;
> >> >> +    case APLIC_SOURCECFG_SM_LEVEL_LOW:
> >> >> +        if ((level <= 0) && !(state & APLIC_ISTATE_PENDING)) {
> >> >> +            riscv_aplic_set_pending_raw(aplic, irq, true);
> >> >> +            update = true;
> >> >> +        }
> >> >> +        break;
> >> >> +    default:
> >> >> +        break;
> >> >> +    }
> >> >> +
> >> >> +    if (level <= 0) {
> >> >> +        aplic->state[irq] &= ~APLIC_ISTATE_INPUT;
> >> >> +    } else {
> >> >> +        aplic->state[irq] |= APLIC_ISTATE_INPUT;
> >> >> +    }
> >> >> +
> >> >> +    if (update) {
> >> >> +        if (aplic->msimode) {
> >> >> +            riscv_aplic_msi_irq_update(aplic, irq);
> >> >> +        } else {
> >> >> +            idc = aplic->target[irq] >> APLIC_TARGET_HART_IDX_SHIFT;
> >> >> +            idc &= APLIC_TARGET_HART_IDX_MASK;
> >> >> +            riscv_aplic_idc_update(aplic, idc);
> >> >> +        }
> >> >> +    }
> >> >> +}
> >> >> +
> >> >> +static uint64_t riscv_aplic_read(void *opaque, hwaddr addr,
> unsigned size)
> >> >> +{
> >> >> +    uint32_t irq, word, idc;
> >> >> +    RISCVAPLICState *aplic = opaque;
> >> >> +
> >> >> +    /* Reads must be 4 byte words */
> >> >> +    if ((addr & 0x3) != 0) {
> >> >> +        goto err;
> >> >> +    }
> >> >> +
> >> >> +    if (addr == APLIC_DOMAINCFG) {
> >> >> +        return APLIC_DOMAINCFG_RDONLY | aplic->domaincfg |
> >> >> +               (aplic->msimode ? APLIC_DOMAINCFG_DM : 0);
> >> >> +    } else if ((APLIC_SOURCECFG_BASE <= addr) &&
> >> >> +            (addr < (APLIC_SOURCECFG_BASE + (aplic->num_irqs - 1) *
> 4))) {
> >> >> +        irq  = ((addr - APLIC_SOURCECFG_BASE) >> 2) + 1;
> >> >> +        return aplic->sourcecfg[irq];
> >> >> +    } else if (aplic->mmode && aplic->msimode &&
> >> >> +               (addr == APLIC_MMSICFGADDR)) {
> >> >> +        return aplic->mmsicfgaddr;
> >> >> +    } else if (aplic->mmode && aplic->msimode &&
> >> >> +               (addr == APLIC_MMSICFGADDRH)) {
> >> >> +        return aplic->mmsicfgaddrH;
> >> >> +    } else if (aplic->mmode && aplic->msimode &&
> >> >> +               (addr == APLIC_SMSICFGADDR)) {
> >> >> +        /* Registers SMSICFGADDR and SMSICFGADDRH are implemented
> only if:
> >> >> +         * (a) the interrupt domain is at machine level
> >> >> +         * (b) the domain’s harts implement supervisor mode
> >> >> +         * (c) the domain has one or more child supervisor-level
> domains
> >> >> +         *     that support MSI delivery mode (domaincfg.DM is not
> read-
> >> >> +         *     only zero in at least one of the supervisor-level
> child
> >> >> +         * domains).
> >> >> +         */
> >> >> +        return (aplic->num_children) ? aplic->smsicfgaddr : 0;
> >> >> +    } else if (aplic->mmode && aplic->msimode &&
> >> >> +               (addr == APLIC_SMSICFGADDRH)) {
> >> >> +        return (aplic->num_children) ? aplic->smsicfgaddrH : 0;
> >> >> +    } else if ((APLIC_SETIP_BASE <= addr) &&
> >> >> +            (addr < (APLIC_SETIP_BASE + aplic->bitfield_words *
> 4))) {
> >> >> +        word = (addr - APLIC_SETIP_BASE) >> 2;
> >> >> +        return riscv_aplic_read_pending_word(aplic, word);
> >> >> +    } else if (addr == APLIC_SETIPNUM) {
> >> >> +        return 0;
> >> >> +    } else if ((APLIC_CLRIP_BASE <= addr) &&
> >> >> +            (addr < (APLIC_CLRIP_BASE + aplic->bitfield_words *
> 4))) {
> >> >> +        word = (addr - APLIC_CLRIP_BASE) >> 2;
> >> >> +        return riscv_aplic_read_input_word(aplic, word);
> >> >> +    } else if (addr == APLIC_CLRIPNUM) {
> >> >> +        return 0;
> >> >> +    } else if ((APLIC_SETIE_BASE <= addr) &&
> >> >> +            (addr < (APLIC_SETIE_BASE + aplic->bitfield_words *
> 4))) {
> >> >> +        word = (addr - APLIC_SETIE_BASE) >> 2;
> >> >> +        return riscv_aplic_read_enabled_word(aplic, word);
> >> >> +    } else if (addr == APLIC_SETIENUM) {
> >> >> +        return 0;
> >> >> +    } else if ((APLIC_CLRIE_BASE <= addr) &&
> >> >> +            (addr < (APLIC_CLRIE_BASE + aplic->bitfield_words *
> 4))) {
> >> >> +        return 0;
> >> >> +    } else if (addr == APLIC_CLRIENUM) {
> >> >> +        return 0;
> >> >> +    } else if (addr == APLIC_SETIPNUM_LE) {
> >> >> +        return 0;
> >> >> +    } else if (addr == APLIC_SETIPNUM_BE) {
> >> >> +        return 0;
> >> >> +    } else if (addr == APLIC_GENMSI) {
> >> >> +        return (aplic->msimode) ? aplic->genmsi : 0;
> >> >> +    } else if ((APLIC_TARGET_BASE <= addr) &&
> >> >> +            (addr < (APLIC_TARGET_BASE + (aplic->num_irqs - 1) *
> 4))) {
> >> >> +        irq = ((addr - APLIC_TARGET_BASE) >> 2) + 1;
> >> >> +        return aplic->target[irq];
> >> >> +    } else if (!aplic->msimode && (APLIC_IDC_BASE <= addr) &&
> >> >> +            (addr < (APLIC_IDC_BASE + aplic->num_harts *
> APLIC_IDC_SIZE))) {
> >> >> +        idc = (addr - APLIC_IDC_BASE) / APLIC_IDC_SIZE;
> >> >> +        switch (addr - (APLIC_IDC_BASE + idc * APLIC_IDC_SIZE)) {
> >> >> +        case APLIC_IDC_IDELIVERY:
> >> >> +            return aplic->idelivery[idc];
> >> >> +        case APLIC_IDC_IFORCE:
> >> >> +            return aplic->iforce[idc];
> >> >> +        case APLIC_IDC_ITHRESHOLD:
> >> >> +            return aplic->ithreshold[idc];
> >> >> +        case APLIC_IDC_TOPI:
> >> >> +            return riscv_aplic_idc_topi(aplic, idc);
> >> >> +        case APLIC_IDC_CLAIMI:
> >> >> +            return riscv_aplic_idc_claimi(aplic, idc);
> >> >> +        default:
> >> >> +            goto err;
> >> >> +        };
> >> >> +    }
> >> >> +
> >> >> +err:
> >> >> +    qemu_log_mask(LOG_GUEST_ERROR,
> >> >> +                  "%s: Invalid register read 0x%" HWADDR_PRIx "\n",
> >> >> +                  __func__, addr);
> >> >> +    return 0;
> >> >> +}
> >> >> +
> >> >> +static void riscv_aplic_write(void *opaque, hwaddr addr, uint64_t
> value,
> >> >> +        unsigned size)
> >> >> +{
> >> >> +    RISCVAPLICState *aplic = opaque;
> >> >> +    uint32_t irq, word, idc = UINT32_MAX;
> >> >> +
> >> >> +    /* Writes must be 4 byte words */
> >> >> +    if ((addr & 0x3) != 0) {
> >> >> +        goto err;
> >> >> +    }
> >> >> +
> >> >> +    if (addr == APLIC_DOMAINCFG) {
> >> >> +        /* Only IE bit writeable at the moment */
> >> >> +        value &= APLIC_DOMAINCFG_IE;
> >> >> +        aplic->domaincfg = value;
> >> >> +    } else if ((APLIC_SOURCECFG_BASE <= addr) &&
> >> >> +            (addr < (APLIC_SOURCECFG_BASE + (aplic->num_irqs - 1) *
> 4))) {
> >> >> +        irq  = ((addr - APLIC_SOURCECFG_BASE) >> 2) + 1;
> >> >> +        if (!aplic->num_children && (value & APLIC_SOURCECFG_D)) {
> >> >> +            value = 0;
> >> >> +        }
> >> >> +        if (value & APLIC_SOURCECFG_D) {
> >> >> +            value &= (APLIC_SOURCECFG_D |
> APLIC_SOURCECFG_CHILDIDX_MASK);
> >> >> +        } else {
> >> >> +            value &= (APLIC_SOURCECFG_D | APLIC_SOURCECFG_SM_MASK);
> >> >> +        }
> >> >> +        aplic->sourcecfg[irq] = value;
> >> >> +        if ((aplic->sourcecfg[irq] & APLIC_SOURCECFG_D) ||
> >> >> +            (aplic->sourcecfg[irq] == 0)) {
> >> >> +            riscv_aplic_set_pending_raw(aplic, irq, false);
> >> >> +            riscv_aplic_set_enabled_raw(aplic, irq, false);
> >> >> +        }
> >> >> +    } else if (aplic->mmode && aplic->msimode &&
> >> >> +               (addr == APLIC_MMSICFGADDR)) {
> >> >> +        if (!(aplic->mmsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
> >> >> +            aplic->mmsicfgaddr = value;
> >> >> +        }
> >> >> +    } else if (aplic->mmode && aplic->msimode &&
> >> >> +               (addr == APLIC_MMSICFGADDRH)) {
> >> >> +        if (!(aplic->mmsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
> >> >> +            aplic->mmsicfgaddrH = value &
> APLIC_xMSICFGADDRH_VALID_MASK;
> >> >> +        }
> >> >> +    } else if (aplic->mmode && aplic->msimode &&
> >> >> +               (addr == APLIC_SMSICFGADDR)) {
> >> >> +        /* Registers SMSICFGADDR and SMSICFGADDRH are implemented
> only if:
> >> >> +         * (a) the interrupt domain is at machine level
> >> >> +         * (b) the domain’s harts implement supervisor mode
> >> >> +         * (c) the domain has one or more child supervisor-level
> domains
> >> >> +         *     that support MSI delivery mode (domaincfg.DM is not
> read-
> >> >> +         *     only zero in at least one of the supervisor-level
> child
> >> >> +         * domains).
> >> >> +         */
> >> >> +        if (aplic->num_children &&
> >> >> +            !(aplic->smsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
> >> >> +            aplic->smsicfgaddr = value;
> >> >> +        }
> >> >> +    } else if (aplic->mmode && aplic->msimode &&
> >> >> +               (addr == APLIC_SMSICFGADDRH)) {
> >> >> +        if (aplic->num_children &&
> >> >> +            !(aplic->smsicfgaddrH & APLIC_xMSICFGADDRH_L)) {
> >> >> +            aplic->smsicfgaddrH = value &
> APLIC_xMSICFGADDRH_VALID_MASK;
> >> >> +        }
> >> >> +    } else if ((APLIC_SETIP_BASE <= addr) &&
> >> >> +            (addr < (APLIC_SETIP_BASE + aplic->bitfield_words *
> 4))) {
> >> >> +        word = (addr - APLIC_SETIP_BASE) >> 2;
> >> >> +        riscv_aplic_set_pending_word(aplic, word, value, true);
> >> >> +    } else if (addr == APLIC_SETIPNUM) {
> >> >> +        riscv_aplic_set_pending(aplic, value, true);
> >> >> +    } else if ((APLIC_CLRIP_BASE <= addr) &&
> >> >> +            (addr < (APLIC_CLRIP_BASE + aplic->bitfield_words *
> 4))) {
> >> >> +        word = (addr - APLIC_CLRIP_BASE) >> 2;
> >> >> +        riscv_aplic_set_pending_word(aplic, word, value, false);
> >> >> +    } else if (addr == APLIC_CLRIPNUM) {
> >> >> +        riscv_aplic_set_pending(aplic, value, false);
> >> >> +    } else if ((APLIC_SETIE_BASE <= addr) &&
> >> >> +            (addr < (APLIC_SETIE_BASE + aplic->bitfield_words *
> 4))) {
> >> >> +        word = (addr - APLIC_SETIE_BASE) >> 2;
> >> >> +        riscv_aplic_set_enabled_word(aplic, word, value, true);
> >> >> +    } else if (addr == APLIC_SETIENUM) {
> >> >> +        riscv_aplic_set_enabled(aplic, value, true);
> >> >> +    } else if ((APLIC_CLRIE_BASE <= addr) &&
> >> >> +            (addr < (APLIC_CLRIE_BASE + aplic->bitfield_words *
> 4))) {
> >> >> +        word = (addr - APLIC_CLRIE_BASE) >> 2;
> >> >> +        riscv_aplic_set_enabled_word(aplic, word, value, false);
> >> >> +    } else if (addr == APLIC_CLRIENUM) {
> >> >> +        riscv_aplic_set_enabled(aplic, value, false);
> >> >> +    } else if (addr == APLIC_SETIPNUM_LE) {
> >> >> +        riscv_aplic_set_pending(aplic, value, true);
> >> >> +    } else if (addr == APLIC_SETIPNUM_BE) {
> >> >> +        riscv_aplic_set_pending(aplic, bswap32(value), true);
> >> >> +    } else if (addr == APLIC_GENMSI) {
> >> >> +        if (aplic->msimode) {
> >> >> +            aplic->genmsi = value & ~(APLIC_TARGET_GUEST_IDX_MASK <<
> >> >> +                                      APLIC_TARGET_GUEST_IDX_SHIFT);
> >> >> +            riscv_aplic_msi_send(aplic,
> >> >> +                                 value >>
> APLIC_TARGET_HART_IDX_SHIFT,
> >> >> +                                 0,
> >> >> +                                 value & APLIC_TARGET_EIID_MASK);
> >> >> +        }
> >> >> +    } else if ((APLIC_TARGET_BASE <= addr) &&
> >> >> +            (addr < (APLIC_TARGET_BASE + (aplic->num_irqs - 1) *
> 4))) {
> >> >> +        irq = ((addr - APLIC_TARGET_BASE) >> 2) + 1;
> >> >> +        if (aplic->msimode) {
> >> >> +            aplic->target[irq] = value;
> >> >> +        } else {
> >> >> +            aplic->target[irq] = (value & ~APLIC_TARGET_IPRIO_MASK)
> |
> >> >> +                                 ((value & aplic->iprio_mask) ?
> >> >> +                                  (value & aplic->iprio_mask) : 1);
> >> >> +        }
> >> >> +    } else if (!aplic->msimode && (APLIC_IDC_BASE <= addr) &&
> >> >> +            (addr < (APLIC_IDC_BASE + aplic->num_harts *
> APLIC_IDC_SIZE))) {
> >> >> +        idc = (addr - APLIC_IDC_BASE) / APLIC_IDC_SIZE;
> >> >> +        switch (addr - (APLIC_IDC_BASE + idc * APLIC_IDC_SIZE)) {
> >> >> +        case APLIC_IDC_IDELIVERY:
> >> >> +            aplic->idelivery[idc] = value & 0x1;
> >> >> +            break;
> >> >> +        case APLIC_IDC_IFORCE:
> >> >> +            aplic->iforce[idc] = value & 0x1;
> >> >> +            break;
> >> >> +        case APLIC_IDC_ITHRESHOLD:
> >> >> +            aplic->ithreshold[idc] = value & aplic->iprio_mask;
> >> >> +            break;
> >> >> +        default:
> >> >> +            goto err;
> >> >> +        };
> >> >> +    } else {
> >> >> +        goto err;
> >> >> +    }
> >> >> +
> >> >> +    if (aplic->msimode) {
> >> >> +        for (irq = 1; irq < aplic->num_irqs; irq++) {
> >> >> +            riscv_aplic_msi_irq_update(aplic, irq);
> >> >> +        }
> >> >> +    } else {
> >> >> +        if (idc == UINT32_MAX) {
> >> >> +            for (idc = 0; idc < aplic->num_harts; idc++) {
> >> >> +                riscv_aplic_idc_update(aplic, idc);
> >> >> +            }
> >> >> +        } else {
> >> >> +            riscv_aplic_idc_update(aplic, idc);
> >> >> +        }
> >> >> +    }
> >> >> +
> >> >> +    return;
> >> >> +
> >> >> +err:
> >> >> +    qemu_log_mask(LOG_GUEST_ERROR,
> >> >> +                  "%s: Invalid register write 0x%" HWADDR_PRIx "\n",
> >> >> +                  __func__, addr);
> >> >> +}
> >> >> +
> >> >> +static const MemoryRegionOps riscv_aplic_ops = {
> >> >> +    .read = riscv_aplic_read,
> >> >> +    .write = riscv_aplic_write,
> >> >> +    .endianness = DEVICE_LITTLE_ENDIAN,
> >> >> +    .valid = {
> >> >> +        .min_access_size = 4,
> >> >> +        .max_access_size = 4
> >> >> +    }
> >> >> +};
> >> >> +
> >> >> +static void riscv_aplic_realize(DeviceState *dev, Error **errp)
> >> >> +{
> >> >> +    uint32_t i;
> >> >> +    RISCVAPLICState *aplic = RISCV_APLIC(dev);
> >> >> +
> >> >> +    aplic->bitfield_words = (aplic->num_irqs + 31) >> 5;
> >> >> +    aplic->sourcecfg = g_new0(uint32_t, aplic->num_irqs);
> >> >> +    aplic->state = g_new(uint32_t, aplic->num_irqs);
> >> >> +    aplic->target = g_new0(uint32_t, aplic->num_irqs);
> >> >> +    if (!aplic->msimode) {
> >> >> +        for (i = 0; i < aplic->num_irqs; i++) {
> >> >> +            aplic->target[i] = 1;
> >> >> +        }
> >> >> +    }
> >> >> +    aplic->idelivery = g_new0(uint32_t, aplic->num_harts);
> >> >> +    aplic->iforce = g_new0(uint32_t, aplic->num_harts);
> >> >> +    aplic->ithreshold = g_new0(uint32_t, aplic->num_harts);
> >> >> +
> >> >> +    memory_region_init_io(&aplic->mmio, OBJECT(dev),
> &riscv_aplic_ops, aplic,
> >> >> +                          TYPE_RISCV_APLIC, aplic->aperture_size);
> >> >> +    sysbus_init_mmio(SYS_BUS_DEVICE(dev), &aplic->mmio);
> >> >> +
> >> >> +    /* Only root APLICs have hardware IRQ lines. All non-root APLICs
> >> >> +     * have IRQ lines delegated by their parent APLIC.
> >> >> +     */
> >> >> +    if (!aplic->parent) {
> >> >> +        qdev_init_gpio_in(dev, riscv_aplic_request,
> aplic->num_irqs);
> >> >> +    }
> >> >> +
> >> >> +    /* Create output IRQ lines for non-MSI mode */
> >> >> +    if (!aplic->msimode) {
> >> >> +        aplic->external_irqs = g_malloc(sizeof(qemu_irq) *
> aplic->num_harts);
> >> >> +        qdev_init_gpio_out(dev, aplic->external_irqs,
> aplic->num_harts);
> >> >> +
> >> >> +        /* Claim the CPU interrupt to be triggered by this APLIC */
> >> >> +        for (i = 0; i < aplic->num_harts; i++) {
> >> >> +            RISCVCPU *cpu =
> RISCV_CPU(qemu_get_cpu(aplic->hartid_base + i));
> >> >> +            if (riscv_cpu_claim_interrupts(cpu,
> >> >> +                (aplic->mmode) ? MIP_MEIP : MIP_SEIP) < 0) {
> >> >> +                error_report("%s already claimed",
> >> >> +                             (aplic->mmode) ? "MEIP" : "SEIP");
> >> >> +                exit(1);
> >> >> +            }
> >> >> +        }
> >> >> +    }
> >> >> +
> >> >> +    msi_nonbroken = true;
> >> >> +}
> >> >> +
> >> >> +static Property riscv_aplic_properties[] = {
> >> >> +    DEFINE_PROP_UINT32("aperture-size", RISCVAPLICState,
> aperture_size, 0),
> >> >> +    DEFINE_PROP_UINT32("hartid-base", RISCVAPLICState, hartid_base,
> 0),
> >> >> +    DEFINE_PROP_UINT32("num-harts", RISCVAPLICState, num_harts, 0),
> >> >> +    DEFINE_PROP_UINT32("iprio-mask", RISCVAPLICState, iprio_mask,
> 0),
> >> >> +    DEFINE_PROP_UINT32("num-irqs", RISCVAPLICState, num_irqs, 0),
> >> >> +    DEFINE_PROP_BOOL("msimode", RISCVAPLICState, msimode, 0),
> >> >> +    DEFINE_PROP_BOOL("mmode", RISCVAPLICState, mmode, 0),
> >> >> +    DEFINE_PROP_END_OF_LIST(),
> >> >> +};
> >> >> +
> >> >> +static const VMStateDescription vmstate_riscv_aplic = {
> >> >> +    .name = "riscv_aplic",
> >> >> +    .version_id = 1,
> >> >> +    .minimum_version_id = 1,
> >> >> +    .fields = (VMStateField[]) {
> >> >> +            VMSTATE_UINT32(domaincfg, RISCVAPLICState),
> >> >> +            VMSTATE_UINT32(mmsicfgaddr, RISCVAPLICState),
> >> >> +            VMSTATE_UINT32(mmsicfgaddrH, RISCVAPLICState),
> >> >> +            VMSTATE_UINT32(smsicfgaddr, RISCVAPLICState),
> >> >> +            VMSTATE_UINT32(smsicfgaddrH, RISCVAPLICState),
> >> >> +            VMSTATE_UINT32(genmsi, RISCVAPLICState),
> >> >> +            VMSTATE_VARRAY_UINT32(sourcecfg, RISCVAPLICState,
> >> >> +                                  num_irqs, 0,
> >> >> +                                  vmstate_info_uint32, uint32_t),
> >> >> +            VMSTATE_VARRAY_UINT32(state, RISCVAPLICState,
> >> >> +                                  num_irqs, 0,
> >> >> +                                  vmstate_info_uint32, uint32_t),
> >> >> +            VMSTATE_VARRAY_UINT32(target, RISCVAPLICState,
> >> >> +                                  num_irqs, 0,
> >> >> +                                  vmstate_info_uint32, uint32_t),
> >> >> +            VMSTATE_VARRAY_UINT32(idelivery, RISCVAPLICState,
> >> >> +                                  num_harts, 0,
> >> >> +                                  vmstate_info_uint32, uint32_t),
> >> >> +            VMSTATE_VARRAY_UINT32(iforce, RISCVAPLICState,
> >> >> +                                  num_harts, 0,
> >> >> +                                  vmstate_info_uint32, uint32_t),
> >> >> +            VMSTATE_VARRAY_UINT32(ithreshold, RISCVAPLICState,
> >> >> +                                  num_harts, 0,
> >> >> +                                  vmstate_info_uint32, uint32_t),
> >> >> +            VMSTATE_END_OF_LIST()
> >> >> +        }
> >> >> +};
> >> >> +
> >> >> +static void riscv_aplic_class_init(ObjectClass *klass, void *data)
> >> >> +{
> >> >> +    DeviceClass *dc = DEVICE_CLASS(klass);
> >> >> +
> >> >> +    device_class_set_props(dc, riscv_aplic_properties);
> >> >> +    dc->realize = riscv_aplic_realize;
> >> >> +    dc->vmsd = &vmstate_riscv_aplic;
> >> >> +}
> >> >> +
> >> >> +static const TypeInfo riscv_aplic_info = {
> >> >> +    .name          = TYPE_RISCV_APLIC,
> >> >> +    .parent        = TYPE_SYS_BUS_DEVICE,
> >> >> +    .instance_size = sizeof(RISCVAPLICState),
> >> >> +    .class_init    = riscv_aplic_class_init,
> >> >> +};
> >> >> +
> >> >> +static void riscv_aplic_register_types(void)
> >> >> +{
> >> >> +    type_register_static(&riscv_aplic_info);
> >> >> +}
> >> >> +
> >> >> +type_init(riscv_aplic_register_types)
> >> >> +
> >> >> +/*
> >> >> + * Add a APLIC device to another APLIC device as child for
> >> >> + * interrupt delegation.
> >> >> + */
> >> >> +void riscv_aplic_add_child(DeviceState *parent, DeviceState *child)
> >> >> +{
> >> >> +    RISCVAPLICState *caplic, *paplic;
> >> >> +
> >> >> +    assert(parent && child);
> >> >> +    caplic = RISCV_APLIC(child);
> >> >> +    paplic = RISCV_APLIC(parent);
> >> >> +
> >> >> +    assert(paplic->num_irqs == caplic->num_irqs);
> >> >> +    assert(paplic->num_children <= QEMU_APLIC_MAX_CHILDREN);
> >> >> +
> >> >> +    caplic->parent = paplic;
> >> >> +    paplic->children[paplic->num_children] = caplic;
> >> >> +    paplic->num_children++;
> >> >> +}
> >> >> +
> >> >> +/*
> >> >> + * Create APLIC device.
> >> >> + */
> >> >> +DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size,
> >> >> +    uint32_t hartid_base, uint32_t num_harts, uint32_t num_sources,
> >> >> +    uint32_t iprio_bits, bool msimode, bool mmode, DeviceState
> *parent)
> >> >> +{
> >> >> +    DeviceState *dev = qdev_new(TYPE_RISCV_APLIC);
> >> >> +    uint32_t i;
> >> >> +
> >> >> +    assert(num_harts < APLIC_MAX_IDC);
> >> >> +    assert((APLIC_IDC_BASE + (num_harts * APLIC_IDC_SIZE)) <= size);
> >> >> +    assert(num_sources < APLIC_MAX_SOURCE);
> >> >> +    assert(APLIC_MIN_IPRIO_BITS <= iprio_bits);
> >> >> +    assert(iprio_bits <= APLIC_MAX_IPRIO_BITS);
> >> >> +
> >> >> +    qdev_prop_set_uint32(dev, "aperture-size", size);
> >> >> +    qdev_prop_set_uint32(dev, "hartid-base", hartid_base);
> >> >> +    qdev_prop_set_uint32(dev, "num-harts", num_harts);
> >> >> +    qdev_prop_set_uint32(dev, "iprio-mask", ((1U << iprio_bits) -
> 1));
> >> >> +    qdev_prop_set_uint32(dev, "num-irqs", num_sources + 1);
> >> >> +    qdev_prop_set_bit(dev, "msimode", msimode);
> >> >> +    qdev_prop_set_bit(dev, "mmode", mmode);
> >> >> +
> >> >> +    sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
> >> >> +    sysbus_mmio_map(SYS_BUS_DEVICE(dev), 0, addr);
> >> >> +
> >> >> +    if (parent) {
> >> >> +        riscv_aplic_add_child(parent, dev);
> >> >> +    }
> >> >> +
> >> >> +    if (!msimode) {
> >> >> +        for (i = 0; i < num_harts; i++) {
> >> >> +            CPUState *cpu = qemu_get_cpu(hartid_base + i);
> >> >> +
> >> >> +            qdev_connect_gpio_out_named(dev, NULL, i,
> >> >> +
> qdev_get_gpio_in(DEVICE(cpu),
> >> >> +                                            (mmode) ? IRQ_M_EXT :
> IRQ_S_EXT));
> >> >> +        }
> >> >> +    }
> >> >> +
> >> >> +    return dev;
> >> >> +}
> >> >> diff --git a/include/hw/intc/riscv_aplic.h
> b/include/hw/intc/riscv_aplic.h
> >> >> new file mode 100644
> >> >> index 0000000000..de8532fbc3
> >> >> --- /dev/null
> >> >> +++ b/include/hw/intc/riscv_aplic.h
> >> >> @@ -0,0 +1,79 @@
> >> >> +/*
> >> >> + * RISC-V APLIC (Advanced Platform Level Interrupt Controller)
> interface
> >> >> + *
> >> >> + * Copyright (c) 2021 Western Digital Corporation or its affiliates.
> >> >> + *
> >> >> + * This program is free software; you can redistribute it and/or
> modify it
> >> >> + * under the terms and conditions of the GNU General Public License,
> >> >> + * version 2 or later, as published by the Free Software Foundation.
> >> >> + *
> >> >> + * This program is distributed in the hope it will be useful, but
> WITHOUT
> >> >> + * ANY WARRANTY; without even the implied warranty of
> MERCHANTABILITY or
> >> >> + * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public
> License for
> >> >> + * more details.
> >> >> + *
> >> >> + * You should have received a copy of the GNU General Public
> License along with
> >> >> + * this program.  If not, see <http://www.gnu.org/licenses/>.
> >> >> + */
> >> >> +
> >> >> +#ifndef HW_RISCV_APLIC_H
> >> >> +#define HW_RISCV_APLIC_H
> >> >> +
> >> >> +#include "hw/sysbus.h"
> >> >> +#include "qom/object.h"
> >> >> +
> >> >> +#define TYPE_RISCV_APLIC "riscv.aplic"
> >> >> +
> >> >> +typedef struct RISCVAPLICState RISCVAPLICState;
> >> >> +DECLARE_INSTANCE_CHECKER(RISCVAPLICState, RISCV_APLIC,
> TYPE_RISCV_APLIC)
> >> >> +
> >> >> +#define APLIC_MIN_SIZE            0x4000
> >> >> +#define APLIC_SIZE_ALIGN(__x)     (((__x) + (APLIC_MIN_SIZE - 1)) &
> \
> >> >> +                                   ~(APLIC_MIN_SIZE - 1))
> >> >> +#define APLIC_SIZE(__num_harts)   (APLIC_MIN_SIZE + \
> >> >> +                                   APLIC_SIZE_ALIGN(32 *
> (__num_harts)))
> >> >> +
> >> >> +struct RISCVAPLICState {
> >> >> +    /*< private >*/
> >> >> +    SysBusDevice parent_obj;
> >> >> +    qemu_irq *external_irqs;
> >> >> +
> >> >> +    /*< public >*/
> >> >> +    MemoryRegion mmio;
> >> >> +    uint32_t bitfield_words;
> >> >> +    uint32_t domaincfg;
> >> >> +    uint32_t mmsicfgaddr;
> >> >> +    uint32_t mmsicfgaddrH;
> >> >> +    uint32_t smsicfgaddr;
> >> >> +    uint32_t smsicfgaddrH;
> >> >> +    uint32_t genmsi;
> >> >> +    uint32_t *sourcecfg;
> >> >> +    uint32_t *state;
> >> >> +    uint32_t *target;
> >> >> +    uint32_t *idelivery;
> >> >> +    uint32_t *iforce;
> >> >> +    uint32_t *ithreshold;
> >> >> +
> >> >> +    /* topology */
> >> >> +#define QEMU_APLIC_MAX_CHILDREN        16
> >> >> +    struct RISCVAPLICState *parent;
> >> >> +    struct RISCVAPLICState *children[QEMU_APLIC_MAX_CHILDREN];
> >> >> +    uint16_t num_children;
> >> >> +
> >> >> +    /* config */
> >> >> +    uint32_t aperture_size;
> >> >> +    uint32_t hartid_base;
> >> >> +    uint32_t num_harts;
> >> >> +    uint32_t iprio_mask;
> >> >> +    uint32_t num_irqs;
> >> >> +    bool msimode;
> >> >> +    bool mmode;
> >> >> +};
> >> >> +
> >> >> +void riscv_aplic_add_child(DeviceState *parent, DeviceState *child);
> >> >> +
> >> >> +DeviceState *riscv_aplic_create(hwaddr addr, hwaddr size,
> >> >> +    uint32_t hartid_base, uint32_t num_harts, uint32_t num_sources,
> >> >> +    uint32_t iprio_bits, bool msimode, bool mmode, DeviceState
> *parent);
> >> >> +
> >> >> +#endif
> >> >> --
> >> >> 2.25.1
> >> >>
>

Alistair Francis Jan. 21, 2022, 6:16 a.m. UTC | #7

On Fri, Jan 21, 2022 at 2:02 AM Anup Patel <[e-mail]> wrote:
>
> Hi Alistair,
>
> On Wed, Jan 19, 2022 at 8:56 PM Anup Patel <[e-mail]> wrote:
> >
> > From: Anup Patel <[e-mail]>
> >
> > The advanced interrupt architecture (AIA) extends the per-HART local
> > interrupt support. Along with this, it also adds IMSIC (MSI contrllor)
> > and Advanced PLIC (wired interrupt controller).
> >
> > The latest AIA draft specification can be found here:
> > https://github.com/riscv/riscv-aia/releases/download/0.2-draft.28/riscv-interrupts-028.pdf
> >
> > This series adds RISC-V AIA support in QEMU which includes emulating all
> > AIA local CSRs, APLIC, and IMSIC. Only AIA local interrupt filtering is
> > not implemented because we don't have any local interrupt greater than 12.
> >
> > To enable AIA in QEMU, use one of the following:
> > 1) Only AIA local interrupt CSRs: Pass "x-aia=true" as CPU paramenter
> >    in the QEMU command-line
> > 2) Only APLIC for virt machine: Pass "aia=aplic" as machine parameter
> >    in the QEMU command-line
> > 3) Both APLIC and IMSIC for virt machine: Pass "aia=aplic-imsic" as
> >    machine parameter in the QEMU command-line
> > 4) Both APLIC and IMSIC with 2 guest files for virt machine: Pass
> >    "aia=aplic-imsic,aia-guests=2" as machine parameter in the QEMU
> >    command-line
> >
> > To test series, we require OpenSBI and Linux with AIA support which can
> > be found in:
> > riscv_aia_v2 branch at https://github.com/avpatel/opensbi.git
> > riscv_aia_v1 branch at https://github.com/avpatel/linux.git
> >
> > This series can be found riscv_aia_v8 branch at:
> > https://github.com/avpatel/qemu.git
> >
> > Changes since v7:
> >  - Rebased on latest riscv-to-apply.next branch of Alistair's repo
> >  - Improved default priority assignment in PATCH9
> >
> > Changes since v6:
> >  - Fixed priority comparison in riscv_cpu_pending_to_irq() of PATCH9
> >  - Fixed typos in comments added by PATCH11
> >  - Added "pend = true;" for CSR_MSETEIPNUM case of rmw_xsetclreinum()
> >    in PATCH15
> >  - Handle ithreshold == 0 case in riscv_aplic_idc_topi() of PATCH18
> >  - Allow setting pending bit for Level0 or Level1 interrupts in
> >    riscv_aplic_set_pending() of PATCH18
> >  - Force DOMAINCFG[31:24] bits to 0x80 in riscv_aplic_read() of PATCH18
> >  - For APLIC direct mode, set target.iprio to 1 when zero is writtern
> >    in PATCH18
> >  - Handle eithreshold == 0 case in riscv_imsic_topei() of PATCH20
> >
> > Changes since v5:
> >  - Moved VSTOPI_NUM_SRCS define to top of the file in PATCH13
> >  - Fixed typo in PATCH16
> >
> > Changes since v4:
> >  - Changed IRQ_LOCAL_MAX to 16 in PATCH2
> >  - Fixed typo in PATCH10
> >  - Replaced TARGET_LONG_BITS with riscv_cpu_mxl_bits(env) in PATCH11
> >  - Replaced TARGET_LONG_BITS with riscv_cpu_mxl_bits(env) in PATCH14
> >  - Replaced TARGET_LONG_BITS with riscv_cpu_mxl_bits(env) in PATCH15
> >  - Replaced TARGET_LONG_BITS with xlen passed via ireg callback in PATCH20
> >  - Retrict maximum IMSIC guest files per-HART of virt machine to 7 in
> >    PATCH21.
> >  - Added separate PATCH23 to increase maximum number of allowed CPUs
> >    for virt machine
> >
> > Changes since v3:
> >  - Replaced "aplic,xyz" and "imsic,xyz" DT properties with "riscv,xyz"
> >    DT properties because "aplic" and "imsic" are not valid vendor names
> >    required by Linux DT schema checker.
> >
> > Changes since v2:
> >  - Update PATCH4 to check and inject interrupt after V=1 when
> >    transitioning from V=0 to V=1
> >
> > Changes since v1:
> >  - Revamped whole series and created more granular patches
> >  - Added HGEIE and HGEIP CSR emulation for H-extension
> >  - Added APLIC emulation
> >  - Added IMSIC emulation
> >
> > Anup Patel (23):
> >   target/riscv: Fix trap cause for RV32 HS-mode CSR access from RV64
> >     HS-mode
> >   target/riscv: Implement SGEIP bit in hip and hie CSRs
> >   target/riscv: Implement hgeie and hgeip CSRs
> >   target/riscv: Improve delivery of guest external interrupts
> >   target/riscv: Allow setting CPU feature from machine/device emulation
> >   target/riscv: Add AIA cpu feature
> >   target/riscv: Add defines for AIA CSRs
> >   target/riscv: Allow AIA device emulation to set ireg rmw callback
> >   target/riscv: Implement AIA local interrupt priorities
> >   target/riscv: Implement AIA CSRs for 64 local interrupts on RV32
> >   target/riscv: Implement AIA hvictl and hviprioX CSRs
> >   target/riscv: Implement AIA interrupt filtering CSRs
> >   target/riscv: Implement AIA mtopi, stopi, and vstopi CSRs
> >   target/riscv: Implement AIA xiselect and xireg CSRs
> >   target/riscv: Implement AIA IMSIC interface CSRs
> >   hw/riscv: virt: Use AIA INTC compatible string when available
> >   target/riscv: Allow users to force enable AIA CSRs in HART
> >   hw/intc: Add RISC-V AIA APLIC device emulation
> >   hw/riscv: virt: Add optional AIA APLIC support to virt machine
> >   hw/intc: Add RISC-V AIA IMSIC device emulation
> >   hw/riscv: virt: Add optional AIA IMSIC support to virt machine
> >   docs/system: riscv: Document AIA options for virt machine
> >   hw/riscv: virt: Increase maximum number of allowed CPUs
>
> Only PATCH19 and PATCH21 are pending for review.
>
> Can this series be considered for riscv-to-apply.next ?

Yep, it should make it into the next PR.

Sorry it's taken so long, there is just a lot going on in this series
so it takes time to review.

Alistair

>
> Regards,
> Anup
>
>
> >
> >  docs/system/riscv/virt.rst    |   16 +
> >  hw/intc/Kconfig               |    6 +
> >  hw/intc/meson.build           |    2 +
> >  hw/intc/riscv_aplic.c         |  975 +++++++++++++++++++++++++
> >  hw/intc/riscv_imsic.c         |  448 ++++++++++++
> >  hw/riscv/Kconfig              |    2 +
> >  hw/riscv/virt.c               |  706 +++++++++++++++---
> >  include/hw/intc/riscv_aplic.h |   79 ++
> >  include/hw/intc/riscv_imsic.h |   68 ++
> >  include/hw/riscv/virt.h       |   41 +-
> >  target/riscv/cpu.c            |  103 ++-
> >  target/riscv/cpu.h            |   72 +-
> >  target/riscv/cpu_bits.h       |  123 ++++
> >  target/riscv/cpu_helper.c     |  343 ++++++++-
> >  target/riscv/csr.c            | 1280 ++++++++++++++++++++++++++++++---
> >  target/riscv/machine.c        |   24 +-
> >  16 files changed, 3988 insertions(+), 300 deletions(-)
> >  create mode 100644 hw/intc/riscv_aplic.c
> >  create mode 100644 hw/intc/riscv_imsic.c
> >  create mode 100644 include/hw/intc/riscv_aplic.h
> >  create mode 100644 include/hw/intc/riscv_imsic.h
> >
> > --
> > 2.25.1
> >
>