| Message ID | 20250102094116.3847894-1-vladimir.kondratiev@mobileye.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | riscv,aplic: support for hart indexes | expand |
On Thu, Jan 2, 2025 at 3:11 PM Vladimir Kondratiev <vladimir.kondratiev@mobileye.com> wrote: > > Risc-v APLIC uses "hart index" to access data per destination hart. > Current implementation assumes hart indexes are consecutive integers > starting from 0, while Risc-V documentation says it may be > arbitrary numbers, with a clue that it may be related to the hart IDs. > > In all boards I see in today's kernel, hart IDs are consecutive > integers, thus using dart IDs is the same as indexes. > > However, for the MIPS P8700, hart IDs are different from indexes, > on this SoC they encode thread number, core and cluster in bits > [0..3], [4..15], [16..19] resulting Soc consisting of 3 clusters * > 4 cores * 2 threads with hart IDs: > 0x0, 0x1, 0x10, 0x11, 0x20, 0x21, 0x30, 0x31, 0x10000 etc. > > Change default hart index to be hart ID related to the start of domain, > and add optional property to configure arbitrary indexes. We don't need any APLIC DT binding change for supporting random HART ID assignments. Please see below for a detailed explanation. > > Use of "device_property" API allows to cover both ACPI and OF in single > code > > 1-st commit adds dt-bindings, 2-nd - code > > Vladimir Kondratiev (2): > dt-bindings: interrupt-controller: add risc-v,aplic hart indexes > irqchip/riscv-aplic: add support for hart indexes The RISC-V APLIC driver does not assume any correlation between APLIC "HART Index" and the actual HART ID of various HARTs. Each APLIC domain has its own 14bit "HART Index" space and the APLIC driver determines "HART Index" based on the APLIC mode (Direct/MSI mode). The APLIC domain in direct mode requires "HART Index" to be between 0 to N - 1 where N is the number of HARTs targeted by the APLIC domain because the APLIC IDC structures are placed consecutively in the MMIO space and located based on "HART index". (Refer, first paragraph of the section "4.8 Interrupt delivery directly by the APLIC" of the ratified RISC-V AIA v1.0 specification) On the other hand, there is no constraint on the "HART Index" space for the APLIC domain in MSI mode because the "HART Index" bits are part of the MSI target address generated by APLIC. (Refer, section "4.9.1 Addresses and data for outgoing MSIs" of the ratified RISC-V AIA v1.0 specification). The RISC-V APLIC direct mode driver uses the "interrupts-extended" DT property to determine how an APLIC domain connects to a set of HARTs. Here's how this DT property is used: 1) The number of entries in the "interrupts-extended" DT property tells the number of APLIC IDC structures where the first entry is for "HART Index = 0", the second entry is for "HART Index = 1" and so on. 2) The first value (aka phandle) of each entry in the "interrupts-extended" DT property points to the target HART INTC. 3) The second value of each entry in the "interrupts-extended" DT property determines the target privilege level on the HART. For example, 11 means "M-mode external interrupt" and 9 means "S-mode external interrupt" The RISC-V APLIC MSI mode driver extracts the "HART Index" the target MSI address and the RISC-V IMSIC driver selects the target HART for handling a particular APLIC MSI-mode interrupt. Clearly, the APLIC DT binding is flexible and does not depend upon the HART IDs assigned to the HARTs. The "Example1" of the riscv,aplic.yaml already shows three different APLIC domains targeting different sets of HARTs and privilege levels. In fact, the upstream QEMU virt machine already supports creating up to 8 APLIC domains in direct mode where each targets a different set of HARTs and privilege levels. Regards, Anup
>The APLIC domain in direct mode requires "HART Index" to be >between 0 to N - 1 where N is the number of HARTs targeted by >the APLIC domain because the APLIC IDC structures are placed >consecutively in the MMIO space and located based on "HART index". >(Refer, first paragraph of the section "4.8 Interrupt delivery directly >by the APLIC" of the ratified RISC-V AIA v1.0 specification) Hi Anup, Sorry, perhaps I am reading spec the wrong way. I don't see where spec required this: >"HART Index" to be >between 0 to N - 1 where N is the number of HARTs targeted by >the APLIC domain I have a real hardware (MIPS P8700) where APLIC is in direct mode and hart indexes are same as hart IDs, masking out cluster number. These hart indexes as I mentioned, "0x00, 0x01, 0x10, 0x11" etc. For delivering IRQ, for example, to CPU2 I need to access its IDC by hart index 0x10. Current code uses IDC at index 2 and hardware don't work this way because there's no registers at this address. If spec indeed requires hart indexes to be in range as you mentioned, such hardware is not spec compliant. Is it the case? In addition, as spec says, hart index may be different than its hart ID and I don't see any provisioning in current code to supply this hart index. What am I missing?
On Thu, Jan 2, 2025 at 7:31 PM Vladimir Kondratiev <Vladimir.Kondratiev@mobileye.com> wrote: > > >The APLIC domain in direct mode requires "HART Index" to be > >between 0 to N - 1 where N is the number of HARTs targeted by > >the APLIC domain because the APLIC IDC structures are placed > >consecutively in the MMIO space and located based on "HART index". > >(Refer, first paragraph of the section "4.8 Interrupt delivery directly > >by the APLIC" of the ratified RISC-V AIA v1.0 specification) > > Hi Anup, > > Sorry, perhaps I am reading spec the wrong way. I don't see where > spec required this: > > >"HART Index" to be > >between 0 to N - 1 where N is the number of HARTs targeted by > >the APLIC domain > > I have a real hardware (MIPS P8700) where APLIC is in direct mode > and hart indexes are same as hart IDs, masking out cluster number. > These hart indexes as I mentioned, "0x00, 0x01, 0x10, 0x11" etc. > For delivering IRQ, for example, to CPU2 I need to access its IDC > by hart index 0x10. Current code uses IDC at index 2 and hardware > don't work this way because there's no registers at this address. > > If spec indeed requires hart indexes to be in range as you mentioned, > such hardware is not spec compliant. Is it the case? Here's the text from "4.8 Interrupt delivery directly by the APLIC" of the AIA specification: "When an interrupt domain is in direct delivery mode (domaincfg.DM = 0), interrupts are delivered from the APLIC to harts by a unique signal to each hart, usually a dedicated wire. In this case, the domain’s memory-mapped control region contains at the end an array of interrupt delivery control (IDC) structures, one IDC structure per potential hart index. The first IDC structure is for the domain’s hart with index 0; the second is for the hart with index 1; etc." The AIA spec clearly says that for APLIC in direct mode requires sequential "HART index" starting from 0 so that IDC structures can be located using the APLIC "HART index". The non-contiguous APLIC "HART index" assignment in MIPS P8700 is clearly a violation of the AIA specification. > > In addition, as spec says, hart index may be different than its hart ID > and I don't see any provisioning in current code to supply this > hart index. Yes, this is true for both APLIC direct-mode as well because each APLIC domain in direct-mode will have its own "HART index" space starting from 0. > > What am I missing? To me it seems MIPS P8700 is only IP and not actual silicon ?? (https://mips.com/products/hardware/p8700/) If so then it is better to fix the IP itself. If it is real silicon then I can think of some work-around for the non-compliance. Regards, Anup
>Here's the text from "4.8 Interrupt delivery directly by the APLIC" of >the AIA specification: >"When an interrupt domain is in direct delivery mode (domaincfg.DM = 0), >interrupts are delivered from the APLIC to harts by a unique signal to each >hart, usually a dedicated wire. In this case, the domain’s memory-mapped >control region contains at the end an array of interrupt delivery control (IDC) >structures, one IDC structure per potential hart index. The first IDC structure >is for the domain’s hart with index 0; the second is for the hart with >index 1; etc." >The AIA spec clearly says that for APLIC in direct mode requires >sequential "HART index" starting from 0 so that IDC structures can >be located using the APLIC "HART index". Hi Anup, I am reading same text but interpreting it differently. I understand it this way: for every hart, there's "hart index" that may be arbitrary (unique in domain) 14-bit integer. Then IDC should be accessed using this index. I don't see anything that prohibits sparse array of IDCs. To support this, #4.5 "Memory-mapped control region for an interrupt domain" says: The array of IDC structures may include some for potential hart index numbers that are not actual hart index numbers in the domain. For example, the first IDC structure is always for hart index 0, but 0 is not necessarily a valid index number for any hart in the domain. For each IDC structure in the array that does not correspond to a valid hart index number in the domain, the IDC structure’s registers may (or may not) be all read-only zeros. This suggests 0 may be not a valid hart index, so clearly some gaps are allowed. Thanks, Vladimir.
On Thu, Jan 2, 2025 at 8:37 PM Vladimir Kondratiev <Vladimir.Kondratiev@mobileye.com> wrote: > > >Here's the text from "4.8 Interrupt delivery directly by the APLIC" of > >the AIA specification: > > >"When an interrupt domain is in direct delivery mode (domaincfg.DM = 0), > >interrupts are delivered from the APLIC to harts by a unique signal to each > >hart, usually a dedicated wire. In this case, the domain’s memory-mapped > >control region contains at the end an array of interrupt delivery control (IDC) > >structures, one IDC structure per potential hart index. The first IDC structure > >is for the domain’s hart with index 0; the second is for the hart with > >index 1; etc." > > >The AIA spec clearly says that for APLIC in direct mode requires > >sequential "HART index" starting from 0 so that IDC structures can > >be located using the APLIC "HART index". > > Hi Anup, > I am reading same text but interpreting it differently. I understand it this way: > for every hart, there's "hart index" that may be arbitrary (unique in domain) > 14-bit integer. Then IDC should be accessed using this index. I don't see > anything that prohibits sparse array of IDCs. > > To support this, #4.5 "Memory-mapped control region for an interrupt domain" says: > > The array of IDC structures may include some for potential hart index numbers that are not actual > hart index numbers in the domain. For example, the first IDC structure is always for hart index 0, > but 0 is not necessarily a valid index number for any hart in the domain. For each IDC structure in > the array that does not correspond to a valid hart index number in the domain, the IDC structure’s > registers may (or may not) be all read-only zeros. > > This suggests 0 may be not a valid hart index, so clearly some gaps are allowed. I see your point and this is a fair interpretation. I will review other patches. Thanks, Anup