From patchwork Fri Jan 20 14:09:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13109906 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81594C05027 for ; Fri, 20 Jan 2023 14:11:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230237AbjATOLB (ORCPT ); Fri, 20 Jan 2023 09:11:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37542 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230250AbjATOK6 (ORCPT ); Fri, 20 Jan 2023 09:10:58 -0500 Received: from fx403.security-mail.net (smtpout140.security-mail.net [85.31.212.143]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 67DC0C79C6 for ; Fri, 20 Jan 2023 06:10:29 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by fx403.security-mail.net (Postfix) with ESMTP id 8019F46581E for ; Fri, 20 Jan 2023 15:10:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674223827; bh=Ui9r/rRdVGI+UnUa+0EUbJ6z3sjzTXQnjxpOf50PyF4=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ULbYg+FdvyY4rmWtoFqDA7u7xJ5Macwksa9Y1rcSsv3KzJ4VAcShocpwc5HD7xAit 2E5cbFl7qBWhPNxxbuWK1kd5T85Kvz6L9zKbabvszkTBeDSBJAE2x9QI0XKBcqTLdJ qWAJvsvHEm7JtcOlAondTp8b5k4EnOVB3n9CH1Gw= Received: from fx403 (localhost [127.0.0.1]) by fx403.security-mail.net (Postfix) with ESMTP id E4339465A81; Fri, 20 Jan 2023 15:10:26 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx403.security-mail.net (Postfix) with ESMTPS id 78CFD46579C; Fri, 20 Jan 2023 15:10:25 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 1E3D027E043A; Fri, 20 Jan 2023 15:10:25 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id F173727E043E; Fri, 20 Jan 2023 15:10:24 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id MEoArrao9Cfh; Fri, 20 Jan 2023 15:10:24 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 6BD6927E0437; Fri, 20 Jan 2023 15:10:24 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu F173727E043E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223825; bh=Zp0X1I0k96/5mr+EkjuIVQnXZuSe1nXK+jotNQ/BUV8=; h=From:To:Date:Message-Id:MIME-Version; b=Yfsk3WJZICFHhuNT7zKhTwUU95jAj/gl6YNlDlSuQTPlaDNqoqz5KqQhT7uUzfQAU SXPhVGN7NwXjXpgWcaPgqN23Y4/OckddkZN3F9jfhtvl+BFvjqvBJCbpFCd3vP4Y+m EeOpwIuyigWNINfUvTVmUYgFh7+BYNoNmF5VF+Zc= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 01/31] Documentation: kvx: Add basic documentation Date: Fri, 20 Jan 2023 15:09:32 +0100 Message-ID: <20230120141002.2442-2-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Add some documentation for the kvx architecture and its Linux port. Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Jules Maselbas Signed-off-by: Jules Maselbas Co-developed-by: Guillaume Thouvenin Signed-off-by: Guillaume Thouvenin Signed-off-by: Yann Sionneau Signed-off-by: Bagas Sanjaya --- Notes: V1 -> V2: - converted to .rst, typos and formatting fixes - reword few paragraphs Documentation/arch.rst | 1 + Documentation/kvx/index.rst | 17 ++ Documentation/kvx/kvx-exceptions.rst | 256 ++++++++++++++++++++++++ Documentation/kvx/kvx-iommu.rst | 191 ++++++++++++++++++ Documentation/kvx/kvx-mmu.rst | 287 +++++++++++++++++++++++++++ Documentation/kvx/kvx-smp.rst | 39 ++++ Documentation/kvx/kvx.rst | 273 +++++++++++++++++++++++++ 7 files changed, 1064 insertions(+) create mode 100644 Documentation/kvx/index.rst create mode 100644 Documentation/kvx/kvx-exceptions.rst create mode 100644 Documentation/kvx/kvx-iommu.rst create mode 100644 Documentation/kvx/kvx-mmu.rst create mode 100644 Documentation/kvx/kvx-smp.rst create mode 100644 Documentation/kvx/kvx.rst diff --git a/Documentation/arch.rst b/Documentation/arch.rst index 41a66a8b38e4..1ccda8ef6eef 100644 --- a/Documentation/arch.rst +++ b/Documentation/arch.rst @@ -13,6 +13,7 @@ implementation. arm/index arm64/index ia64/index + kvx/index loongarch/index m68k/index mips/index diff --git a/Documentation/kvx/index.rst b/Documentation/kvx/index.rst new file mode 100644 index 000000000000..0bac597b5fbe --- /dev/null +++ b/Documentation/kvx/index.rst @@ -0,0 +1,17 @@ +.. SPDX-License-Identifier: GPL-2.0 + +======================= +Kalray kvx Architecture +======================= + +.. toctree:: + :maxdepth: 2 + :numbered: + + kvx + kvx-exceptions + kvx-smp + kvx-mmu + kvx-iommu + + diff --git a/Documentation/kvx/kvx-exceptions.rst b/Documentation/kvx/kvx-exceptions.rst new file mode 100644 index 000000000000..5e01e934192f --- /dev/null +++ b/Documentation/kvx/kvx-exceptions.rst @@ -0,0 +1,256 @@ +========== +Exceptions +========== + +On kvx, handlers are set using ``$ev`` (exception vector) register which +specifies a base address. An offset is added to ``$ev`` upon exception +and the result is used as the new ``$pc``. +The offset depends on which exception vector the cpu wants to jump to: + + ============== ========= + ``$ev + 0x00`` debug + ``$ev + 0x40`` trap + ``$ev + 0x80`` interrupt + ``$ev + 0xc0`` syscall + ============== ========= + +Then, handlers are laid in the following order:: + + _____________ + | | + | Syscall | + |_____________| + | | + | Interrupts | + |_____________| + | | + | Traps | + |_____________| + | | ^ + | Debug | | Stride + BASE -> |_____________| v + + +Interrupts and traps are serviced similarly, ie: + + - Jump to handler + - Save all registers + - Prepare the call (do_IRQ or trap_handler) + - restore all registers + - return from exception + +entry.S file is (as for other architectures) the entry point into the kernel. +It contains all assembly routines related to interrupts/traps/syscall. + +Syscall handling +---------------- + +When executing a syscall, it must be done using ``scall $r6`` where ``$r6`` +contains the syscall number. This convention allows to modify and restart +a syscall from the kernel. + +Syscalls are handled differently than interrupts/exceptions. From an ABI +point of view, syscalls are like function calls: any caller-saved register +can be clobbered by the syscall. However, syscall parameters are passed +using registers r0 through r7. These registers must be preserved to avoid +clobberring them before the actual syscall function. + +On syscall from userspace (``scall`` instruction), the processor will put +the syscall number in $es.sn and switch from user to kernel privilege +mode. ``kvx_syscall_handler`` will be called in kernel mode. + +The following steps are then taken: + + 1. Switch to kernel stack + 2. Extract syscall number + 3. If the syscall number is bogus, set the syscall function to ``sys_ni_syscall`` + 4. If tracing is enabled + + - Jump to ``trace_syscall_enter`` + - Save syscall arguments (``r0`` -> ``r7``) on stack in ``pt_regs``. + - Call ``do_trace_syscall_enter`` function. + + 5. Restore syscall arguments since they have been modified by C call + 6. Call the syscall function + 7. Save ``$r0`` in ``pt_regs`` since it can be clobberred afterward + 8. If tracing is enabled, call ``trace_syscall_exit``. + 9. Call ``work_pending`` + 10. Return to user ! + +The trace call is handled out of the fast path. All slow path handling +is done in another part of code to avoid messing with the cache. + +Signals +------- + +Signals are handled when exiting kernel before returning to user. +When handling a signal, the path is the following: + + 1. User application is executing normally + Then any exception happens (syscall, interrupt, trap) + 2. The exception handling path is taken + and before returning to user, pending signals are checked + 3. Signal are handled by ``do_signal``. + Registers are saved and a special part of the stack is modified + to create a trampoline to call ``rt_sigreturn``, + ``$spc`` is modified to jump to user signal handler; + ``$ra`` is modified to jump to sigreturn trampoline directly after + returning from user signal handler. + 4. User signal handler is called after ``rfe`` from exception + when returning, ``$ra`` is retored to ``$pc``, resulting in a call + to the syscall trampoline. + 5. syscall trampoline is executed, leading to rt_sigreturn syscall + 6. ``rt_sigreturn`` syscall is executed. Previous registers are restored to + allow returning to user correctly. + 7. User application is restored at the exact point it was interrupted + before. + +:: + + +----------+ + | 1 | + | User app | @func + | (user) | + +---+------+ + | + | it/trap/scall + | + +---v-------+ + | 2 | + | exception | + | handling | + | (kernel) | + +---+-------+ + | + | Check if signal are pending, if so, handle signals + | + +---v--------+ + | 3 | + | do_signal | + | handling | + | (kernel) | + +----+-------+ + | + | Return to user signal handler + | + +----v------+ + | 4 | + | signal | + | handler | + | (user) | + +----+------+ + | + | Return to sigreturn trampoline + | + +----v-------+ + | 5 | + | syscall | + |rt_sigreturn| + | (user) | + +----+-------+ + | + | Syscall to rt_sigreturn + | + +----v-------+ + | 6 | + | sigreturn | + | handler | + | (kernel) | + +----+-------+ + | + | Modify context to return to original func + | + +----v-----+ + | 7 | + | User app | @func + | (user) | + +----------+ + + +Registers handling +------------------ + +MMU is disabled in all exceptions paths, during register save and restoration. +This will prevent from triggering MMU fault (such as TLB miss) which could +clobber the current register state. Such event can occurs when RWX mode is +enabled and the memory accessed to save register can trigger a TLB miss. +Aside from that which is common for all exceptions path, registers are saved +differently depending on the exception type. + +Interrupts and traps +-------------------- + +When interrupt and traps are triggered, only caller-saved registers are saved. +Indeed, we rely on the fact that C code will save and restore callee-saved and +hence, there is no need to save them. The path is:: + + +------------+ +-----------+ +---------------+ + IT | Save caller| C Call | Execute C | Ret | Restore caller| Ret from IT + +--->+ saved +--------->+ handler +------->+ saved +-----> + | registers | +-----------+ | registers | + +------------+ +---------------+ + +However, when returning to user, we check if there is work_pending. If a signal +is pending and there is a signal handler to be called, then all registers are +saved on the stack in ``pt_regs`` before executing the signal handler and +restored after that. Since only caller-saved registers have been saved before +checking for pending work, callee-saved registers also need to be saved to +restore everything correctly when before returning to user. +This path is the following (a bit more complicated !):: + + +------------+ + | Save caller| +-----------+ Ret +------------+ + IT | saved | C Call | Execute C | to asm | Check work | + +--->+ registers +--------->+ handler +------->+ pending | + | to pt_regs | +-----------+ +--+---+-----+ + +------------+ | | + Work pending | | No work pending + +--------------------------------------------+ | + | | + | +------------+ + v | + +------+------+ v + | Save callee | +-------+-------+ + | saved | | Restore caller| RFE from IT + | registers | | saved +-------> + | to pt_regs | | registers | + +--+-------+--+ | from pt_regs | + | | +-------+-------+ + | | +---------+ ^ + | | | Execute | | + | +-------->+ needed +-----------+ + | | work | + | +---------+ + |Signal handler ? + v + +----+----------+ RFE to user +-------------+ +--------------+ + | Copy all | handler | Execute | ret | rt_sigreturn | + | registers +------------>+ user signal +------>+ trampoline | + | from pt_regs | | handler | | to kernel | + | to user stack | +-------------+ +------+-------+ + +---------------+ | + syscall rt_sigreturn | + +-------------------------------------------------+ + | + v + +--------+-------+ +-------------+ + | Recopy all | | Restore all | RFE + | registers from +--------------------->+ saved +-------> + | user stack | Return | registers | + | to pt_regs | from sigreturn |from pt_regs | + +----------------+ (via ret_from_fork) +-------------+ + + +Syscalls +-------- + +As explained before, for syscalls, we can use whatever callee-saved registers +we want since syscall are seen as a "classic" call from ABI pov. +Only different path is the one for clone. For this path, since the child expects +to find same callee-registers content than his parent, we must save them before +executing the clone syscall and restore them after that for the child. This is +done via a redefinition of __sys_clone in assembly which will be called in place +of the standard sys_clone. This new call will save callee saved registers +in pt_regs. Parent will return using the syscall standard path. Freshly spawned +child however will be woken up via ret_from_fork which will restore all +registers (even if caller saved are not needed). diff --git a/Documentation/kvx/kvx-iommu.rst b/Documentation/kvx/kvx-iommu.rst new file mode 100644 index 000000000000..240995d315ce --- /dev/null +++ b/Documentation/kvx/kvx-iommu.rst @@ -0,0 +1,191 @@ +.. SPDX-License-Identifier: GPL-2.0 + +===== +IOMMU +===== + +General Overview +---------------- + +To exchange data between device and users through memory, the driver +has to set up a buffer by doing some kernel allocation. The buffer uses +virtual address and the physical address is obtained through the MMU. +When the device wants to access the same physical memory space it uses +the bus address which is obtained by using the DMA mapping API. The +Coolidge SoC includes several IOMMUs for clusters, PCIe peripherals, +SoC peripherals, and more; that will translate this "bus address" into +a physical one for DMA operations. + +Bus addresses are IOVA (I/O Virtual Address) or DMA addresses. These +addresses can be obtained by calling the allocation functions of the DMA APIs. +It can also be obtained through classical kernel allocation of physical +contiguous memory and then calling mapping functions of the DMA API. + +In order to be able to use the kvx IOMMU we have implemented the IOMMU DMA +interface in arch/kvx/mm/dma-mapping.c. DMA functions are registered by +implementing arch_setup_dma_ops() and generic IOMMU functions. Generic IOMMU +are calling our specific IOMMU functions that adds or remove mappings between +DMA addresses and physical addresses in the IOMMU TLB. + +Specific IOMMU functions are defined in the kvx IOMMU driver. The kvx IOMMU +driver manage two physical hardware IOMMU: one used for TX and one for RX. +In the next section we described the HW IOMMUs. + +Cluster IOMMUs +-------------- + +IOMMUs on cluster are used for DMA and cryptographic accelerators. +There are six IOMMUs connected to the: + + - cluster DMA tx + - cluster DMA rx + - first non secure cryptographic accelerator + - second non secure cryptographic accelerator + - first secure cryptographic accelerator + - second secure cryptographic accelerator + +SoC peripherals IOMMUs +---------------------- + +Since SoC peripherals are connected to an AXI bus, two IOMMUs are used: one for +each AXI channel (read and write). These two IOMMUs are shared between all master +devices and DMA. These two IOMMUs will have the same entries but need to be configured +independently. + +PCIe IOMMUs +----------- + +There is a slave IOMMU (read and write from the MPPA to the PCIe endpoint) +and a master IOMMU (read and write from a PCIe endpoint to system DDR). +The PCIe root complex and the MSI/MSI-X controller have been designed to use +the IOMMU feature when enabled. (For example for supporting endpoint that +support only 32 bits addresses and allow them to access any memory in a +64 bits address space). For security reason it is highly recommended to +activate the IOMMU for PCIe. + +IOMMU implementation +-------------------- + +The kvx is providing several IOMMUs. Here is a simplified view of all IOMMUs +and translations that occurs between memory and devices:: + + +---------------------------------------------------------------------+ + | +------------+ +---------+ | CLUSTER X | + | | Cores 0-15 +---->+ Crypto | +-----------| + | +-----+------+ +----+----+ | + | | | | + | v v | + | +-------+ +------------------------------+ | + | | MMU | +----+ IOMMU x4 (secure + insecure) | | + | +---+---+ | +------------------------------+ | + | | | | + +--------------------+ | + | | | | + v v | | + +---+--------+-+ | | + | MEMORY | | +----------+ +--------+ +-------+ | + | +<-|-----+ IOMMU Rx |<----+ DMA Rx |<----+ | | + | | | +----------+ +--------+ | | | + | | | | NoC | | + | | | +----------+ +--------+ | | | + | +--|---->| IOMMU Tx +---->| DMA Tx +---->+ | | + | | | +----------+ +--------+ +-------+ | + | | +------------------------------------------------+ + | | + | | +--------------+ +------+ + | |<--->+ IOMMU Rx/Tx +<--->+ PCIe + + | | +--------------+ +------+ + | | + | | +--------------+ +------------------------+ + | |<--->+ IOMMU Rx/Tx +<--->+ master Soc Peripherals | + | | +--------------+ +------------------------+ + +--------------+ + + +There is also an IOMMU dedicated to the crypto module but this module will not +be accessed by the operating system. + +We will provide one driver to manage IOMMUs RX/TX. All of them will be +described in the device tree to be able to get their particularities. See +the example below that describes the relation between IOMMU, DMA and NoC in +the cluster. + +IOMMU is related to a specific bus like PCIe we will be able to specify that +all peripherals will go through this IOMMU. + +IOMMU Page table +~~~~~~~~~~~~~~~~ + +We need to be able to know which IO virtual addresses (IOVA) are mapped in the +TLB in order to be able to remove entries when a device finishes a transfer and +release memory. This information could be extracted when needed by computing all +sets used by the memory and then reads all sixteen ways and compare them to the +IOVA but it won't be efficient. We also need to be able to translate an IOVA +to a physical address as required by the iova_to_phys IOMMU ops that is used +by DMA. Like previously it can be done by extracting the set from the address +and comparing the IOVA to each sixteen entries of the given set. + +A solution is to keep a page table for the IOMMU. But this method is not +efficient for reloading an entry of the TLB without the help of an hardware +page table. So to prevent the need of a refill we will update the TLB when a +device request access to memory and if there is no more slot available in the +TLB we will just fail and the device will have to try again later. It is not +efficient but at least we won't need to manage the refill of the TLB. + +This limits the total amount of memory that can be used for transfer between +device and memory (see Limitations section below). +To be able to manage bigger transfer we can implement the huge page table in +the Linux kernel and use a page table that match the size of huge page table +for a given IOMMU (typically the PCIe IOMMU). + +As we won't refill the TLB we know that we won't have more than 128*16 entries. +In this case we can simply keep a table with all possible entries. + +Maintenance interface +~~~~~~~~~~~~~~~~~~~~~ + +It is possible to have several "maintainers" for the same IOMMU. The driver is +using two of them. One that writes the TLB and another interface reads TLB. For +debug purpose it is possible to display the content of the tlb by using the +following command in gdb:: + + gdb> p kvx_iommu_dump_tlb( , 0) + +Since different management interface are used for read and write it is safe to +execute the above command at any moment. + +Interrupts +~~~~~~~~~~ + +IOMMU can have 3 kind of interrupts that corresponds to 3 different types of +errors (no mapping. protection, parity). When the IOMMU is shared between +clusters (SoC periph and PCIe) then fifteen IRQs are generated according to the +configuration of an association table. The association table is indexed by the +ASN number (9 bits) and the entry of the table is a subscription mask with one +bit per destination. Currently this is not managed by the driver. + +The driver is only managing interrupts for the cluster. The mode used is the +stall one. So when an interrupt occurs it is managed by the driver. All others +interrupts that occurs are stored and the IOMMU is stalled. When driver cleans +the first interrupt others will be managed one by one. + +ASN (Address Space Number) +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +This is also know as ASID in some other architecture. Each device will have a +given ASN that will be given through the device tree. As address space is +managed at the IOMMU domain level we will use one group and one domain per ID. +ASN are coded on 9 bits. + +Device tree +----------- + +Relationships between devices, DMAs and IOMMUs are described in the +device tree (see `Documentation/devicetree/bindings/iommu/kalray,kvx-iommu.txt` +for more details). + +Limitations +----------- + +Only supporting 4KB page size will limit the size of mapped memory to 8MB +because the IOMMU TLB can have at most 128*16 entries. diff --git a/Documentation/kvx/kvx-mmu.rst b/Documentation/kvx/kvx-mmu.rst new file mode 100644 index 000000000000..b7186331396c --- /dev/null +++ b/Documentation/kvx/kvx-mmu.rst @@ -0,0 +1,287 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=== +MMU +=== + +Virtual addresses are on 41 bits for kvx when using 64-bit mode. +To differentiate kernel from user space, we use the high order bit +(bit 40). When bit 40 is set, then the higher remaining bits must also be +set to 1. The virtual address must be extended with 1 when the bit 40 is set, +if not the address must be zero extended. Bit 40 is set for kernel space +mappings and not set for user space mappings. + +Memory Map +---------- + +In Linux physical memories are arranged into banks according to the cost of an +access in term of distance to a memory. As we are UMA architecture we only have +one bank and thus one node. + +A node is divided into several kind of zone. For example if DMA can only access +a specific area in the physical memory we will define a ZONE_DMA for this purpose. +In our case we are considering that DMA can access all DDR so we don't have a specific +zone for this. On 64 bit architecture all DDR can be mapped in virtual kernel space +so there is no need for a ZONE_HIGHMEM. That means that in our case there is +only one ZONE_NORMAL. This will be updated if DMA cannot access all memory. + +Currently, the memory mapping is the following for 4KB page: + + ======================== ======================= ====== ======= ============== + Start End Attr Size Name + ======================== ======================= ====== ======= ============== + 0000 0000 0000 0000 0000 003F FFFF FFFF --- 256GB User + 0000 0040 0000 0000 0000 007F FFFF FFFF --- 256GB MMAP + 0000 0080 0000 0000 FFFF FF7F FFFF FFFF --- --- Gap + FFFF FF80 0000 0000 FFFF FFFF FFFF FFFF --- 512GB Kernel + FFFF FF80 0000 0000 FFFF FF8F FFFF FFFF RWX 64GB Direct Map + FFFF FF90 0000 0000 FFFF FF90 3FFF FFFF RWX 1GB Vmalloc + FFFF FF90 4000 0000 FFFF FFFF FFFF FFFF RW 447GB Free area + ======================== ======================= ====== ======= ============== + +Enable the MMU +-------------- + +All kernel functions and symbols are in virtual memory except for kvx_start() +function which is loaded at 0x0 in physical memory. +To be able to switch from physical addresses to virtual addresses we choose to +setup the TLB at the very beginning of the boot process to be able to map both +pieces of code. For this we added two entries in the LTLB. The first one, +LTLB[0], contains the mapping between virtual memory and DDR. Its size is 512MB. +The second entry, LTLB[1], contains a flat mapping of the first 2MB of the SMEM. +Once those two entries are present we can enable the MMU. LTLB[1] will be +removed during paging_init() because once we are really running in virtual space +it will not be used anymore. +In order to access more than 512MB DDR memory, the remaining memory (> 512MB) is +refill using a comparison in kernel_perf_refill that does not walk the kernel +page table, thus having a faster refill time for kernel. These entries are +inserted into the LTLB for easier computation (4 LTLB entries). The drawback of +this approach is that mapped entries are using RWX protection attributes, +leading to no protection at all. + +Kernel strict RWX +----------------- + +``CONFIG_STRICT_KERNEL_RWX`` is enabled by default in defconfig. +Once booted, if ``CONFIG_STRICT_KERNEL_RWX`` is enable, the kernel text and memory +will be mapped in the init_mm page table. Once mapped, the refill routine for +the kernel is patched to always do a page table walk, bypassing the faster +comparison but enforcing page protection attributes when refilling. +Finally, the LTLB[0] entry is replaced by a 4K one, mapping only exceptions with +RX protection. It allows us to never trigger nomapping on nomapping refill +routine which would (obviously) not work... Once this is done, we can flush the +4 LTLB entries for kernel refill in order to be sure there is no stalled +entries and that new entries inserted in JTLB will apply. + +By default, the following policy is applied on vmlinux sections: + + - init_data: RW + - init_text: RX (or RWX if parameter rodata=off) + - text: RX (or RWX if parameter rodata=off) + - rodata: RW before init, RO after init + - sdata: RW + +Kernel RWX mode can then be switched on/off using /sys/kvx/kernel_rwx file. + +Privilege Level +--------------- + +Since we are using privilege levels on kvx, we make use of the virtual +spaces to be in the same space as the user. The kernel will have the +$ps.mmup set in kernel (PL1) and unset for user (PL2). +As said in kvx documentation, we have two cases when the kernel is +booted: + + - Either we have been booted by someone (bootloader, hypervisor, etc) + - Or we are alone (boot from flash) + +In both cases, we will use the virtual space 0. Indeed, if we are alone +on the core, then it means nobody is using the MMU and we can take the +first virtual space. If not alone, then when writing an entry to the tlb +using writetlb instruction, the hypervisor will catch it and change the +virtual space accordingly. + +Memblock +======== + +When the kernel starts there is no memory allocator available. One of the first +step in the kernel is to detect the amount of DDR available by getting this +information in the device tree and initialize the low-level "memblock" allocator. + +We start by reserving memory for the whole kernel. For instance with a device +tree containing 512MB of DDR you could see the following boot messages:: + + setup_bootmem: Memory : 0x100000000 - 0x120000000 + setup_bootmem: Reserved: 0x10001f000 - 0x1002d1bc0 + +During the paging init we need to set: + + - min_low_pfn that is the lowest PFN available in the system + - max_low_pfn that indicates the end if NORMAL zone + - max_pfn that is the number of pages in the system + +This setting is used for dividing memory into pages and for configuring the +zone. See the memory map section for more information about ZONE. + +Zones are configured in free_area_init_core(). During start_kernel() other +allocations are done for command line, cpu areas, PID hash table, different +caches for VFS. This allocator is used until mem_init() is called. + +mem_init() is provided by the architecture. For MPPA we just call +free_all_bootmem() that will go through all pages that are not used by the +low level allocator and mark them as not used. So physical pages that are +reserved for the kernel are still used and remain in physical memory. All pages +released will now be used by the buddy allocator. + +Peripherals +----------- + +Peripherals are mapped using standard ioremap infrastructure, therefore +mapped addresses are located in the vmalloc space. + +LTLB Usage +---------- + +LTLB is used to add resident mapping which allows for faster MMU lookup. +Currently, the LTLB is used to map some mandatory kernel pages and to allow fast +accesses to l2 cache (mailbox and registers). +When CONFIG_STRICT_KERNEL_RWX is disabled, 4 entries are reserved for kernel +TLB refill using 512MB pages. When CONFIG_STRICT_KERNEL_RWX is enabled, these +entries are unused since kernel is paginated using the same mecanism than for +user (page walking and entries in JTLB) + +Page Table +========== + +We only support three levels for the page table and 4KB for page size. + +3 levels page table +------------------- + +:: + + ...-----+--------+--------+--------+--------+--------+ + 40|39 32|31 24|23 16|15 8|7 0| + ...-----++-------+--+-----+---+----+----+---+--------+ + | | | | + | | | +---> [11:0] Offset (12 bits) + | | +-------------> [20:12] PTE offset (9 bits) + | +-----------------------> [29:21] PMD offset (9 bits) + +----------------------------------> [39:30] PGD offset (10 bits) + +Bits 40 to 64 are signed extended according to bit 39. If bit 39 is equal to 1 +we are in kernel space. + +As 10 bits are used for PGD we need to allocate 2 pages. + +PTE format +========== + +About the format of the PTE entry, as we are not forced by hardware for choices, +we choose to follow the format described in the RiscV implementation as a +starting point:: + + +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+ + | 63..23 | 22..13 | 12 | 11..10 | 9 | 8 | 7 | 6 | 5 | 4 | 3..2 | 1 | 0 | + +---------+--------+----+--------+---+---+---+---+---+---+------+---+---+ + PFN Unused S PageSZ H G X W R D CP A P + where: + P: Present + A: Accessed + CP: Cache policy + D: Dirty + R: Read + W: Write + X: Executable + G: Global + H: Huge page + PageSZ: Page size as set in TLB format (0:4KB, 1:64KB, 2:2MB, 3:512MB) + S: Soft/Special + PFN: Page frame number (depends on page size) + +Huge bit must be somewhere in the first 12 bits to be able to detect it +when reading the PMD entry. + +PageSZ must be on bit 10 and 11 because it matches the TEL.PS bits. And +by doing that it is easier in assembly to set the TEL.PS to PageSZ. + +Fast TLB refill +=============== + +kvx core does not feature a hardware page walker. This work must be done +by the core in software. In order to optimize TLB refill, a special fast +path is taken when entering in kernel space. +In order to speed up the process, the following actions are taken: + + 1. Save some registers in a per process scratchpad + 2. If the trap is a nomapping then try the fastpath + 3. Save some more registers for this fastpath + 4. Check if faulting address is a memory direct mapping one. + + * If entry is a direct mapping one and RWX is not enabled, add an entry into LTLB + * If not, continue + + 5. Try to walk the page table + + * If entry is not present, take the slowpath (do_page_fault) + + 6. Refill the tlb properly + 7. Exit by restoring only a few registers + +ASN Handling +============ + +Disclaimer: Some part of this are taken from ARC architecture. + +kvx MMU provides 9-bit ASN (Address Space Number) in order to tag TLB entries. +It allows for multiple process with the same virtual space to cohabit without +the need to flush TLB everytime we context switch. +kvx implementation to use them is based on other architectures (such as arc +or xtensa) and uses a wrapping ASN counter containing both cycle/generation and +asn. + +:: + + +---------+--------+ + |63 10|9 0| + +---------+--------+ + Cycle ASN + +This ASN counter is incremented monotonously to allocate new ASNs. When the +counter reaches 511 (9 bit), TLB is completely flushed and a new cycle is +started. A new allocation cycle, post rollover, could potentially reassign an +ASN to a different task. Thus the rule is to reassign an ASN when the current +context cycles does not match the allocation cycle. +The 64 bit @cpu_asn_cache (and mm->asn) have 9 bits MMU ASN and rest 55 bits +serve as cycle/generation indicator and natural 64 bit unsigned math +automagically increments the generation when lower 9 bits rollover. +When the counter completely wraps, we reset the counter to first cycle value +(ie cycle = 1). This allows to distinguish context without any ASN and old cycle +generated value with the same operation (XOR on cycle). + +Huge page +========= + +Currently only 3 level page table has been implemented for 4KB base page size. +So the page shift is 12 bits, the pmd shift is 21 and the pgdir shift is 30 bits. +This choice implies that for 4KB base page size if we use a PMD as a huge +page the size will be 2MB and if we use a PUD as a huge page it will be 1GB. + +To support other huge page sizes (64KB and 512MB) we need to use several +contiguous entries in the page table. For huge page of 64KB we will need to +use 16 entries in the PTE and for a huge page of 512MB it means that 256 +entries in PMD will be used. + +Debug +===== + +In order to debug the page table and tlb entries, gdb scripts contains commands +which allows to dump the page table: + +:``lx-kvx-page-table-walk``: Display the current process page table by default +:``lx-kvx-tlb-decode``: Display the content of $tel and $teh into something readable + +Other commands available in kvx-gdb are the following: + +:``mppa-dump-tlb``: Display the content of TLBs (JTLB and LTLB) +:``mppa-lookup-addr``: Find physical address matching a virtual one diff --git a/Documentation/kvx/kvx-smp.rst b/Documentation/kvx/kvx-smp.rst new file mode 100644 index 000000000000..12efddbfd1e0 --- /dev/null +++ b/Documentation/kvx/kvx-smp.rst @@ -0,0 +1,39 @@ +=== +SMP +=== + +The Coolidge SoC is comprised of 5 clusters, each organized as a group +of 17 cores: 16 application core (PE) and 1 secure core (RM). +These 17 cores have their L1 cache coherent with the local Tightly +Coupled Memory (TCM or SMEM). The L2 cache is necessary for SMP support +is and implemented with a mix of HW support and SW firmware. The L2 cache +data and meta-data are stored in the TCM. +The RM core is not meant to run Linux and is reserved for implementing +hypervisor services, thus only 16 processors are available for SMP. + +Booting +------- + +When booting the kvx processor, only the RM is woken up. This RM will +execute a portion of code located in the section named ``.rm_firmware``. +By default, a simple power off code is embedded in this section. +To avoid embedding the firmware in kernel sources, the section is patched +using external tools to add the L2 firmware (and replace the default firmware). +Before executing this firmware, the RM boots the PE0. PE0 will then enable L2 +coherency and request will be stalled until RM boots the L2 firmware. + +Locking primitives +------------------ + +spinlock/rwlock are using the kernel standard queued spinlock/rwlocks. +These primitives are based on cmpxch and xchg. More particularly, it uses xchg16 +which is implemented as a read modify write with acswap on 32bit word since +kvx does not have atomic cmpxchg instructions for less than 32 bits. + +IPI +--- + +An IPI controller allows to communicate between CPUs using a simple +memory mapped register. This register can simply be written using a +mask to trigger interrupts directly to the cores matching the mask. + diff --git a/Documentation/kvx/kvx.rst b/Documentation/kvx/kvx.rst new file mode 100644 index 000000000000..c222328aa4b2 --- /dev/null +++ b/Documentation/kvx/kvx.rst @@ -0,0 +1,273 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========= +kvx Linux +========= + +This documents will try to explain any architecture choice for the kvx +Linux port. + +Regarding the peripheral, we MUST use device tree to describe ALL +peripherals. The bindings should always start with "kalray,kvx" for all +core related peripherals (watchdog, timer, etc) + +System Architecture +------------------- + +On kvx, we have 4 levels of privilege level starting from 0 (most +privileged one) to 3 (less privilege one). A system of owners allows +to delegate ownership of resources by using specials system registers. + +The 2 main software stacks for Linux Kernel are the following:: + + +-------------+ +-------------+ + | PL0: Debug | | PL0: Debug | + +-------------+ +-------------+ + | PL1: Linux | | PL1: HyperV | + +-------------+ +-------------+ + | PL2: User | | PL2: Linux | + +-------------+ +-------------+ + | | | PL3: User | + +-------------+ +-------------+ + +In both cases, the kvx support for privileges has been designed using +only relative PL and thus should work on both configurations without +any modifications. + +When booting, the CPU is executing in PL0 and owns all the privileges. +This level is almost dedicated to the debug routines for the debugguer. +It only needs to own few privileges (breakpoint 0 and watchpoint 0) to +be able to debug a system executing in PL1 to PL3. +Debug routines are not always there for instance when the kernel is +executing alone (booted from flash). +In order to ease the load of debug routines, software convention is to +jump directly to PL1 and let PL0 for the debug. +When the kernel boots, it checks if the current privilege level is 0 +(`$ps.pl` is the only absolute value). If so, then it will delegate +almost all resources to PL1 and use a RFE to lower its execution +privilege level (see asm_delegate_pl in head.S). +If the current PL is already different from 0, then it means somebody +is above us and we need to request resource to inform it we need them. It will +then either delegate them to us directly or virtualize the delegation. +All privileges levels have their set of banked registers (ps, ea, sps, +sr, etc) which contain privilege level specific values. +`$sr` (system reserved) is banked and will hold the current task_struct. +This register is reserved and should not be touched by any other code. +For more information, refer to the kvx system level architecture manual. + +Boot +---- + +On kvx, the RM (Secure Core) of Cluster 0 will boot first. It will then be able +to boot a firmware. This firmware is stored in the rm_firmware section. +The first argument ($r0) of this firmware will be a pointer to a function with +the following prototype: void firmware_init_done(uint64_t features). This +function is responsible of describing the features supported by the firmware and +will start the first PE after that. +By default, the rm_firmware function act as the "default" firmware. This +function does nothing except calling firmware_init_done and then goes to sleep. +In order to add another firmware, the rm_firmware section is patched using +objcopy. The content of this section is then replaced by the provided firmware. +This firmware will do an init and then call firmware_init_done before running +the main loop. +When the PE boots, it will check for the firmware features to enable or disable +specific core features (L2 for instance). + +When entering the C (kvx_lowlevel_start) the kernel will look for a special +magic (``0x494C314B``) in ``$r0``. This magic tells the kernel if there is arguments +passed by a bootloader. +Currently, the following values are passed through registers: + + :``$r1``: pointer to command line setup by bootloader + :``$r2``: device tree + +If this magic is not set, then, the command line will be the one +provided in the device tree (see bootargs). The default device tree is +not builtin but will be patched by the runner used (simulator or jtag) in the +dtb section. + +A default stdout-path is desirable to allow early printk. + +Boot Memory Allocator +--------------------- + +The boot memory allocator is used to allocate memory before paging is enabled. +It is initialized with DDR and also with the shared memory. This first one is +initialized during the setup_bootmem() and the second one when calling +early_init_fdt_scan_reserved_mem(). + + +Virtual and physical memory +--------------------------- + +The mapping used and the memory management is described in +Documentation/kvx/kvx-mmu.rst. +Our Kernel is compiled using virtual addresses that starts at ``0xffffff0000000000``. +But when it is started the kernel uses physical addresses. +Before calling the first function arch_low_level_start() we configure 2 entries +of the LTLB. + +The first entry will map the first 1G of virtual address space to the first +1G of DDR:: + + TLB[0]: 0xffffff0000000000 -> 0x100000000 (size 512Mo) + +The second entry will be a flat mapping of the first 512 Ko of the SMEM. It +is required to have this flat mapping because there is still code located at +this address that needs to be executed:: + + TLB[1]: 0x0 -> 0x0 (size 512Ko) + +Once virtual space reached the second entry is removed. + +To be able to set breakpoints when MMU is enabled we added a label called +gdb_mmu_enabled. If you try to set a breakpoint on a function that is in +virtual memory before the activation of the MMU this address as no signification +for GDB. So, for example, if you want to break on the function start_kernel() +you will need to run:: + + kvx-gdb -silent path_to/vmlinux \ + -ex 'tbreak gdb_mmu_enabled' -ex 'run' \ + -ex 'break start_kernel' \ + -ex 'continue' + +We will also add an option to kvx-gdb to simplify this step. + +Timers +------ + +The free-running clock (clocksource) is based on the DSU. This clock is +not interruptible and never stops even if core go into idle. + +Regarding the tick (clockevent), we use the timer 0 available on the core. +This timer allows to set a periodic tick which will be used as the main +tick for each core. Note that this clock is percpu. + +The get_cycles implementation is based on performance counter. One of them +is used to count cycles. Note that since this is used only when the core +is running, there is no need to worry about core sleeping (which will +stop the cycle counter) + +Context switching +----------------- + +Context switching is done in entry.S. When spawning a fresh thread, +copy_thread is called. During this call, we setup callee saved register +``r20`` and ``r21`` to special values containing the function to call. + +The normal path for a kernel thread will be the following: + +1. Enter copy_thread_tls and setup callee saved registers which will + be restored in __switch_to. +2. set r20 and r21 (in thread_struct) to function and argument and + ra to ret_from_kernel_thread. + These callee saved will be restored in switch_to. +3. Call _switch_to at some point. +4. Save all callee saved register since switch_to is seen as a + standard function call by the caller. +5. Change stack pointer to the new stack +6. At the end of switch to, set sr0 to the new task and use ret to + jump to ret_from_kernel_thread (address restored from ra). +7. In ret_from_kernel_thread, execute the function with arguments by + using r20, r21 and we are done + +For more explanations, you can refer to https://lwn.net/Articles/520227/ + +User thread creation +-------------------- + +We are using almost the same path as copy_thread to create it. +The detailed path is the following: + + 1. Call start_thread which will setup user pc and stack pointer in + task regs. We also set sps and clear privilege mode bit. + When returning from exception, it will "flip" to user mode. + 2. Enter copy_thread_tls and setup callee saved registers which will + be restored in __switch_to. Also, set the "return" function to be + ret_from_fork which will be called at end of switch_to + 3. set r20 (in thread_struct) with tracing information. + (simply by lazyness to avoid computing it in assembly...) + 4. Call _switch_to at some point. + 5. The current pc will then be restored to be ret_from fork. + 6. Ret from fork calls schedule_tail and then check if tracing is + enabled. If so call syscall_trace_exit + 7. finally, instead of returning to kernel, we restore all registers + that have been setup by start_thread by restoring regs stored on + stack + +L2 handling +----------- + +On kvx, the L2 is handled by a firmware running on the RM. This firmware +needs various information to be aware of its configuration and communicate +with the kernel. In order to do that, when firmware is starting, the device +tree is given as parameter along with the "registers" zone. This zone is +simply a memory area where data are exchanged between kernel <-> L2. When +some commands are written to it, the kernel sends an interrupt using a mailbox. +If the L2 node is not present in the device tree, then, the RM will directly go +into sleeping. + +Boot diagram:: + + RM PE 0 + + + +---------+ | + | Boot | | + +----+----+ | + | | + v | + +-----+-----+ | + | Prepare | | + | L2 shared | | + | memory | | + |(registers)| | + +-----+-----+ | + | | +-----------+ + +------------------->+ Boot | + | | +-----+-----+ + v | | + +--------+---------+ | | + | L2 firmware | | | + | parameters: | | | + | r0 = registers | | | + | r1 = DTB | | | + +--------+---------+ | | + | | | + v | | + +-------+--------+ | +------+------+ + | L2 firmware | | | Wait for L2 | + | execution | | | to be ready | + +-------+--------+ | +------+------+ + | | | + +------v-------+ | v + | L2 requests | | +------+------+ + +--->+ handling | | | Enable | + | +-------+------+ | | L2 caching | + | | | +------+------+ + | | | | + +------------+ + v + + +Since this driver is started early (before SMP boot), A lot of drivers +are not yet probed (mailboxes, IOMMU, etc) and thus can not be used. + +Building +-------- + +In order to build the kernel, you will need a complete kvx toolchain. +First, setup the config using the following command line:: + + $ make ARCH=kvx O=your_directory defconfig + +Adjust any configuration option you may need and then, build the kernel:: + + $ make ARCH=kvx O=your_directory -j12 + +You will finally have a vmlinux image ready to be run:: + + $ kvx-mppa -- vmlinux + +Additionally, you may want to debug it. To do so, use kvx-gdb:: + + $ kvx-gdb vmlinux + From patchwork Fri Jan 20 14:09:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13109912 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE198C38141 for ; Fri, 20 Jan 2023 14:12:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231124AbjATOMZ (ORCPT ); Fri, 20 Jan 2023 09:12:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37668 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230313AbjATOLo (ORCPT ); Fri, 20 Jan 2023 09:11:44 -0500 Received: from fx403.security-mail.net (smtpout140.security-mail.net [85.31.212.143]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D931DC63B8 for ; Fri, 20 Jan 2023 06:10:51 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by fx403.security-mail.net (Postfix) with ESMTP id A8A16465078 for ; Fri, 20 Jan 2023 15:10:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674223827; bh=4aKV1M/EL8VkswXxboYvmLELXbn01a6LSiaOR7RanlU=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=AGJaQBDvOi776LK2rqgakPd/qioMiWy2q+y7NYpPYEMQlbrPg370D79tdz2TqUDhc S56G4mPszOasqaHxq2HES5fIEmKcuxMtf2XMbd3Kyr+QKGUnwWUsudk6xvfxJpa0pz 0rkRpijvpwiPmz1L+M3/jqQ1Qk/6k2ZrdC1gCvJQ= Received: from fx403 (localhost [127.0.0.1]) by fx403.security-mail.net (Postfix) with ESMTP id 3063A46554A; Fri, 20 Jan 2023 15:10:27 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx403.security-mail.net (Postfix) with ESMTPS id BE917465A07; Fri, 20 Jan 2023 15:10:25 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 8630F27E0442; Fri, 20 Jan 2023 15:10:25 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 6823527E043E; Fri, 20 Jan 2023 15:10:25 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id huWm_inYEYG6; Fri, 20 Jan 2023 15:10:25 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id E082F27E0430; Fri, 20 Jan 2023 15:10:24 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 6823527E043E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223825; bh=hlMcSlbG2rCqvU+ECfcxotBg/9GcrvxSjAelxbRimRU=; h=From:To:Date:Message-Id:MIME-Version; b=q18eV07c3WJmIMEDF7H3KwFnXpEamD137FAmmetmHTu/mgrGthaXokUF44+LeCpUp pJJFtVF7sOI7ZD2s+ZsEGDrm1rayzHMQTwcKThR7Iht6lL3o5jqjBUHHvgfAUfAkMP GplPjyCgRgqXTjYk1prY8ZSuYTjxlX17Xhdgw28s= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 02/31] Documentation: Add binding for kalray,kv3-1-core-intc Date: Fri, 20 Jan 2023 15:09:33 +0100 Message-ID: <20230120141002.2442-3-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC From: Jules Maselbas Add documentation for `kalray,kv3-1-core-intc` binding. Co-developed-by: Jules Maselbas Signed-off-by: Jules Maselbas Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: new patch .../kalray,kv3-1-core-intc.yaml | 46 +++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100644 Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-core-intc.yaml diff --git a/Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-core-intc.yaml b/Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-core-intc.yaml new file mode 100644 index 000000000000..1e3d0593173a --- /dev/null +++ b/Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-core-intc.yaml @@ -0,0 +1,46 @@ +# SPDX-License-Identifier: GPL-2.0 +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/interrupt-controller/kalray,kv3-1-core-intc# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Kalray kv3-1 Core Interrupt Controller + +description: | + The Kalray Core Interrupt Controller is tightly integrated in each kv3 core + present in the Coolidge SoC. + + It provides the following features: + - 32 independent interrupt sources + - 2-bit configurable priority level + - 2-bit configurable ownership level + +allOf: + - $ref: /schemas/interrupt-controller.yaml# + +properties: + compatible: + const: kalray,kv3-1-core-intc + "#interrupt-cells": + const: 1 + description: + The IRQ number. + reg: + maxItems: 0 + "kalray,intc-nr-irqs": + description: Number of irqs handled by the controller. + +required: + - compatible + - "#interrupt-cells" + - interrupt-controller + +examples: + - | + intc: interrupt-controller { + compatible = "kalray,kv3-1-core-intc"; + #interrupt-cells = <1>; + interrupt-controller; + }; + +... From patchwork Fri Jan 20 14:09:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13109901 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1507EC05027 for ; Fri, 20 Jan 2023 14:10:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230225AbjATOK5 (ORCPT ); Fri, 20 Jan 2023 09:10:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37424 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230203AbjATOKv (ORCPT ); Fri, 20 Jan 2023 09:10:51 -0500 Received: from fx306.security-mail.net (smtpout30.security-mail.net [85.31.212.36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 09571C63BF for ; Fri, 20 Jan 2023 06:10:28 -0800 (PST) Received: from localhost (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id 26A0935CEF0 for ; Fri, 20 Jan 2023 15:10:27 +0100 (CET) Received: from fx306 (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id CC3FC35CE72; Fri, 20 Jan 2023 15:10:26 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx306.security-mail.net (Postfix) with ESMTPS id 525A035CEDA; Fri, 20 Jan 2023 15:10:26 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 1E8E727E0437; Fri, 20 Jan 2023 15:10:26 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id E726C27E043A; Fri, 20 Jan 2023 15:10:25 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 4dOAwL90Msog; Fri, 20 Jan 2023 15:10:25 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 5C09A27E043D; Fri, 20 Jan 2023 15:10:25 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <15347.63caa0d2.51414.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu E726C27E043A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223826; bh=vgLPgPsCekn4OMTINrnElLpgqshf2SrzKv2ha68Lwiw=; h=From:To:Date:Message-Id:MIME-Version; b=iF0GuZaVjHO/WEe3sRM2xAOMUnUC96hdxiBF7My2AaXyJFr4L+roJaijnbpF9h9IO 1H6vugremghi1ZUfPqtrK9A1FFTsZjlCKXDiuLReDqoB7Qn5HLPMIOVI90P+sTdxNy YPpX3k1YBEiwgisqbvT/PQSDOvqlYpFQZZFyJZ1o= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 03/31] Documentation: Add binding for kalray,kv3-1-apic-gic Date: Fri, 20 Jan 2023 15:09:34 +0100 Message-ID: <20230120141002.2442-4-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC From: Jules Maselbas Add documentation for `kalray,kv3-1-apic-gic` binding. Co-developed-by: Jules Maselbas Signed-off-by: Jules Maselbas Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: new patch .../kalray,kv3-1-apic-gic.yaml | 66 +++++++++++++++++++ 1 file changed, 66 insertions(+) create mode 100644 Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-apic-gic.yaml diff --git a/Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-apic-gic.yaml b/Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-apic-gic.yaml new file mode 100644 index 000000000000..7a37f19db2fb --- /dev/null +++ b/Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-apic-gic.yaml @@ -0,0 +1,66 @@ +# SPDX-License-Identifier: GPL-2.0 +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/interrupt-controller/kalray,kv3-1-apic-gic# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Kalray kv3-1 APIC-GIC + +description: | + Each cluster in the Coolidge SoC includes an Advanced Programmable Interrupt + Controller (APIC) which is split in two part: + - a Generic Interrupt Controller (referred as APIC-GIC) + - a Mailbox Controller (referred as APIC-Mailbox) + The APIC-GIC acts as an intermediary interrupt controller, muxing/routing + incoming interrupts to output interrupts connected to kvx cores interrupts lines. + The 139 possible input interrupt lines are organized as follow: + - 128 from the mailbox controller (one it per mailboxes) + - 1 from the NoC router + - 5 from IOMMUs + - 1 from L2 cache DMA job FIFO + - 1 from cluster watchdog + - 2 for SECC, DECC + - 1 from Data NoC + The 72 possible output interrupt lines: + - 68 : 4 interrupts per cores (17 cores) + - 1 for L2 cache controller + - 3 extra that are for padding + +allOf: + - $ref: /schemas/interrupt-controller.yaml# + +properties: + compatible: + const: kalray,kv3-1-apic-gic + "#interrupt-cells": + const: 1 + description: + The IRQ number. + interrupt-controller: true + interrupt-parent: true + interrupts: + maxItems: 4 + description: | + Specifies the interrupt line(s) in the interrupt-parent controller node; + valid values depend on the type of parent interrupt controller + +required: + - compatible + - reg + - "#interrupt-cells" + - interrupt-controller + - interrupt-parent + - interrupts + +examples: + - | + apic_gic: interrupt-controller@a20000 { + compatible = "kalray,kv3-1-apic-gic"; + reg = <0 0xa20000 0 0x12000>; + #interrupt-cells = <1>; + interrupt-controller; + interrupt-parent = <&intc>; + interrups = <4 5 6 7>; + }; + +... From patchwork Fri Jan 20 14:09:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13109903 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7134C54737 for ; Fri, 20 Jan 2023 14:11:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230227AbjATOK7 (ORCPT ); Fri, 20 Jan 2023 09:10:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230213AbjATOK4 (ORCPT ); Fri, 20 Jan 2023 09:10:56 -0500 Received: from fx306.security-mail.net (smtpout30.security-mail.net [85.31.212.36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 47C0FC79C5 for ; Fri, 20 Jan 2023 06:10:29 -0800 (PST) Received: from localhost (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id C069D35CF31 for ; Fri, 20 Jan 2023 15:10:27 +0100 (CET) Received: from fx306 (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id 8C9A535CEB1; Fri, 20 Jan 2023 15:10:27 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx306.security-mail.net (Postfix) with ESMTPS id CF40835CE76; Fri, 20 Jan 2023 15:10:26 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 915E327E043E; Fri, 20 Jan 2023 15:10:26 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 76ED427E043A; Fri, 20 Jan 2023 15:10:26 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id bkqmCC_I0G-S; Fri, 20 Jan 2023 15:10:26 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id E117627E0439; Fri, 20 Jan 2023 15:10:25 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <1626f.63caa0d2.ce20a.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 76ED427E043A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223826; bh=WsdsMgF+9wdt2RCH1Xf3svKE0DGavKlRxJdrjiN4Uh8=; h=From:To:Date:Message-Id:MIME-Version; b=P23x+LKcvEP+IKZLbkuicGs3Pej5mBdo8yQYkaxT3SdD6c0GM19pKDj+ase6ViEKo uPexGp9XpE0T3huFfEVD6e4MSK+6FZ4VooRCJ6oBEHDNAQPNIZeWLmd6d5qSRBL22D of29euZL+icynxeO+ubhAeSadnZtlOlSeb48YA+Q= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 04/31] Documentation: Add binding for kalray,kv3-1-apic-mailbox Date: Fri, 20 Jan 2023 15:09:35 +0100 Message-ID: <20230120141002.2442-5-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC From: Jules Maselbas Add documentation for `kalray,kv3-1-core-intc` binding. Co-developed-by: Jules Maselbas Signed-off-by: Jules Maselbas Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: new patch .../kalray,kv3-1-apic-mailbox.yaml | 75 +++++++++++++++++++ 1 file changed, 75 insertions(+) create mode 100644 Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-apic-mailbox.yaml diff --git a/Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-apic-mailbox.yaml b/Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-apic-mailbox.yaml new file mode 100644 index 000000000000..e1eb1c9fda0d --- /dev/null +++ b/Documentation/devicetree/bindings/interrupt-controller/kalray,kv3-1-apic-mailbox.yaml @@ -0,0 +1,75 @@ +# SPDX-License-Identifier: GPL-2.0 +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/interrupt-controller/kalray,kv3-1-apic-mailbox# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Kalray kv3-1 APIC-Mailbox + +description: | + Each cluster in the Coolidge SoC includes an Advanced Programmable Interrupt + Controller (APIC) which is split in two part: + - a Generic Interrupt Controller (referred as APIC-GIC) + - a Mailbox Controller (referred as APIC-Mailbox) + The APIC-Mailbox contains 128 mailboxes of 8 bytes (size of a word), + this hardware block is basically a 1 KB of smart memory space. + Each mailbox can be independently configured with a trigger condition + and an input mode function. + + Input mode are: + - write + - bitwise OR + - add + + Interrupts are generated on a write when the mailbox content value + match the configured trigger condition. + Available conditions are: + - doorbell: always raise interruption on write + - match: when the mailbox's value equal the configured trigger value + - barrier: same as match but the mailbox's value is cleared on trigger + - threshold: when the mailbox's value is greater than, or equal to, the + configured trigger value + + Since this hardware block generates IRQs based on writes to some memory + locations, it is both an interrupt controller and an MSI controller. + +allOf: + - $ref: /schemas/interrupt-controller.yaml# + +properties: + compatible: + const: kalray,kv3-1-apic-mailbox + "#interrupt-cells": + const: 1 + description: + The IRQ number. + interrupt-controller: true + interrupt-parent: true + interrupts: + maxItems: 128 + description: | + Specifies the interrupt line(s) in the interrupt-parent controller node; + valid values depend on the type of parent interrupt controller + msi-controller: true + +required: + - compatible + - reg + - "#interrupt-cells" + - interrupt-controller + - interrupt-parent + - interrupts + - msi-controller + +examples: + - | + apic_mailbox: interrupt-controller@a00000 { + compatible = "kalray,kv3-1-apic-gic"; + reg = <0 0xa00000 0 0x0f200>; + #interrupt-cells = <1>; + interrupt-controller; + interrupt-parent = <&apic_gic>; + interrups = <0 1 2 3 4 5 6 7 8 9>; + }; + +... From patchwork Fri Jan 20 14:09:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13109902 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E55A5C38159 for ; Fri, 20 Jan 2023 14:11:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230282AbjATOLA (ORCPT ); Fri, 20 Jan 2023 09:11:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37428 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230237AbjATOK5 (ORCPT ); Fri, 20 Jan 2023 09:10:57 -0500 Received: from fx408.security-mail.net (smtpout140.security-mail.net [85.31.212.148]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 395E6C79D4 for ; Fri, 20 Jan 2023 06:10:29 -0800 (PST) Received: from localhost (fx408.security-mail.net [127.0.0.1]) by fx408.security-mail.net (Postfix) with ESMTP id 39E5A3229FA for ; Fri, 20 Jan 2023 15:10:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674223828; bh=ipV6uYEByPI5c+C5yQy7aqSsXm8TxX991WZckgEkaks=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=6pUnLhMqwMXXOzvmDvV9giVIYh+t0w3F7AmKYyY1HJrHZMdJwTkoEGbh2wTLMy0Xc KVCCILeEuOheemC7HWX6MSRgFXMXF4a9dRuebQ12jRE6IFluwTZTPvS4nW589PRRAX 0ZJ0vqMSFBlOSsPYjAuikWmZBAEwUtjJpjTTOwA4= Received: from fx408 (fx408.security-mail.net [127.0.0.1]) by fx408.security-mail.net (Postfix) with ESMTP id EEE593229D9; Fri, 20 Jan 2023 15:10:27 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx408.security-mail.net (Postfix) with ESMTPS id 60123322981; Fri, 20 Jan 2023 15:10:27 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 2086127E043D; Fri, 20 Jan 2023 15:10:27 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id EEDA527E043A; Fri, 20 Jan 2023 15:10:26 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id y4B9rLqZpZJt; Fri, 20 Jan 2023 15:10:26 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 6DE6527E0437; Fri, 20 Jan 2023 15:10:26 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <122d1.63caa0d3.5ea04.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu EEDA527E043A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223827; bh=BhrYTlway6H47k+1WH6tY+7D+lekRPB9kTC8Yr1liXc=; h=From:To:Date:Message-Id:MIME-Version; b=oFg5sEcCLpoupdKUAJVDuBV1Yxh9BZLNZVHVECdDWl/HZrggI6m//7aB/zUkRD5ML GDYlchdCVVn7Xb9cxeNZJNnf8AI3loDdEUh6uWAVMFL+FcHinJp2cSNSasIrxrWySo +2PhSeUvRBMs9MSoflMTXb8aLv5jpAfzT4Fxwp0k= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 05/31] Documentation: Add binding for kalray,coolidge-itgen Date: Fri, 20 Jan 2023 15:09:36 +0100 Message-ID: <20230120141002.2442-6-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC From: Jules Maselbas Add documentation for `kalray,coolidge-itgen` binding. Co-developed-by: Jules Maselbas Signed-off-by: Jules Maselbas Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: new patch .../kalray,coolidge-itgen.yaml | 48 +++++++++++++++++++ 1 file changed, 48 insertions(+) create mode 100644 Documentation/devicetree/bindings/interrupt-controller/kalray,coolidge-itgen.yaml diff --git a/Documentation/devicetree/bindings/interrupt-controller/kalray,coolidge-itgen.yaml b/Documentation/devicetree/bindings/interrupt-controller/kalray,coolidge-itgen.yaml new file mode 100644 index 000000000000..47b503bff1d9 --- /dev/null +++ b/Documentation/devicetree/bindings/interrupt-controller/kalray,coolidge-itgen.yaml @@ -0,0 +1,48 @@ +# SPDX-License-Identifier: GPL-2.0 +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/interrupt-controller/kalray,coolidge-itgen# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Kalray Coolidge SoC Interrupt Generator (ITGEN) + +description: | + The Interrupt Generator (ITGEN) is an interrupt controller block. + It's purpose is to convert IRQ lines coming from SoC peripherals into writes + on the AXI bus. The ITGEN intended purpose is to write into the APIC mailboxes. + +allOf: + - $ref: /schemas/interrupt-controller.yaml# + +properties: + compatible: + const: kalray,coolidge-itgen + + "#interrupt-cells": + const: 2 + description: | + - 1st cell is for the IRQ number + - 2nd cell is for the trigger type as defined dt-bindings/interrupt-controller/irq.h + + interrupt-controller: true + + msi-parent: true + +required: + - compatible + - reg + - "#interrupt-cells" + - interrupt-controller + - msi-parent + +examples: + - | + itgen: interrupt-controller@27000000 { + compatible = "kalray,coolidge-itgen"; + reg = <0 0x27000000 0 0x1104>; + #interrupt-cells = <2>; + interrupt-controller; + msi-parent = <&apic_mailbox>; + }; + +... From patchwork Fri Jan 20 14:09:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13109913 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1C1BC05027 for ; Fri, 20 Jan 2023 14:12:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230373AbjATOM2 (ORCPT ); Fri, 20 Jan 2023 09:12:28 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38814 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230449AbjATOLp (ORCPT ); Fri, 20 Jan 2023 09:11:45 -0500 Received: from fx306.security-mail.net (smtpout30.security-mail.net [85.31.212.36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3291DBCE1C for ; Fri, 20 Jan 2023 06:11:00 -0800 (PST) Received: from localhost (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id 9866E35CF1B for ; Fri, 20 Jan 2023 15:10:28 +0100 (CET) Received: from fx306 (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id 5A16335CF11; Fri, 20 Jan 2023 15:10:28 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx306.security-mail.net (Postfix) with ESMTPS id D69B735CF42; Fri, 20 Jan 2023 15:10:27 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id A1E9327E043D; Fri, 20 Jan 2023 15:10:27 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 7762E27E0430; Fri, 20 Jan 2023 15:10:27 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id JyeDd67fLWzC; Fri, 20 Jan 2023 15:10:27 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id E4DF627E0439; Fri, 20 Jan 2023 15:10:26 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 7762E27E0430 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223827; bh=pa4ScTG872xOzuRt8GX8Xf9/eL9W3gZxwoN/kQBQFss=; h=From:To:Date:Message-Id:MIME-Version; b=mCuKc4/UeROfSPy0qESb6bWw6haYw2yj/Thuaz88gp4Xo6fY7v11akHAnwnsq/SE/ l71YYm+MloYSX37ktexP6Ugc0NvI9sQziU77LnSg76PpXxXYDYmYnCLgz0FFptdvhN uCVOH2np8u2uDy77i3yNVSUCOsRcLW4hLIHMTeNw= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 06/31] Documentation: Add binding for kalray,kv3-1-ipi-ctrl Date: Fri, 20 Jan 2023 15:09:37 +0100 Message-ID: <20230120141002.2442-7-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC From: Jules Maselbas Add documentation for `kalray,kv3-1-ipi-ctrl` binding. Co-developed-by: Jules Maselbas Signed-off-by: Jules Maselbas Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: new patch .../kalray/kalray,kv3-1-ipi-ctrl.yaml | 44 +++++++++++++++++++ 1 file changed, 44 insertions(+) create mode 100644 Documentation/devicetree/bindings/kalray/kalray,kv3-1-ipi-ctrl.yaml diff --git a/Documentation/devicetree/bindings/kalray/kalray,kv3-1-ipi-ctrl.yaml b/Documentation/devicetree/bindings/kalray/kalray,kv3-1-ipi-ctrl.yaml new file mode 100644 index 000000000000..dc8026b12905 --- /dev/null +++ b/Documentation/devicetree/bindings/kalray/kalray,kv3-1-ipi-ctrl.yaml @@ -0,0 +1,44 @@ +# SPDX-License-Identifier: GPL-2.0 +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/kalray/kalray,kv3-1-ipi-ctrl# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Kalray kv3-1 Inter-Processor Interrupt Controller (IPI) + +description: | + The Inter-Processor Interrupt Controller (IPI) provides a fast synchronization + mechanism to the software. It exposes eight independent set of registers that + can be use to notify each processor in the cluster. + A set of registers contains two 32-bit registers: + - 17-bit interrupt control, one bit per core, raise an interrupt on write + - 17-bit mask, one per core, to enable interrupts + + Bit at offsets 0 to 15 selects cores in the cluster, respectively PE0 to PE15, + while bit at offset 16 is for the cluster Resource Manager (RM) core. + + The eight output interrupts are connected to each processor core interrupt + controller (intc). + +properties: + compatible: + const: kalray,kv3-1-ipi-ctrl + reg: + maxItems: 1 + +required: + - compatible + - reg + - interrupt-parent + - interrupts + +examples: + - | + ipi: inter-processor-interrupt@ad0000 { + compatible = "kalray,kv3-1-ipi-ctrl"; + reg = <0x00 0xad0000 0x00 0x1000>; + interrupt-parent = <&intc>; + interrupts = <24>; + }; + +... From patchwork Fri Jan 20 14:09:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13109904 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71A71C54EB6 for ; Fri, 20 Jan 2023 14:11:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230250AbjATOLC (ORCPT ); Fri, 20 Jan 2023 09:11:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37588 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230253AbjATOLA (ORCPT ); Fri, 20 Jan 2023 09:11:00 -0500 Received: from fx304.security-mail.net (smtpout30.security-mail.net [85.31.212.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9D1F8BCE22 for ; Fri, 20 Jan 2023 06:10:31 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by fx304.security-mail.net (Postfix) with ESMTP id 7A28D9D0BA for ; Fri, 20 Jan 2023 15:10:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674223829; bh=GCb+aZyhHGWJx4uFQ9PWJOzxH5FwUHqDkAret4C+DiA=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=cTJKwCdk2zOE4eqETytFbqd3i85IUsiNVJk9ff75Yc3gmCsCn/VF3JNOrqz44ImWR WJWwfKkreBYnHE86cXSRyUtuwv5EqLnop+Yw399B1LB0qrk83uUXGLO+aNwLAWxy6M ZtnoF15TGZCyNo2hl2QSWZCIkNzmuTUjFFQpunNM= Received: from fx304 (localhost [127.0.0.1]) by fx304.security-mail.net (Postfix) with ESMTP id 0C20B9D06A; Fri, 20 Jan 2023 15:10:29 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx304.security-mail.net (Postfix) with ESMTPS id 5784C9D04D; Fri, 20 Jan 2023 15:10:28 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 2855527E043D; Fri, 20 Jan 2023 15:10:28 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id D8A7627E0439; Fri, 20 Jan 2023 15:10:27 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id pxiL3-7zmQbs; Fri, 20 Jan 2023 15:10:27 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 5AD1127E043E; Fri, 20 Jan 2023 15:10:27 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <6ff5.63caa0d4.56592.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu D8A7627E0439 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223828; bh=8uCtiqDjPzOcwhlHBsakO1CoaBAI9UncGRE7J9HRCd8=; h=From:To:Date:Message-Id:MIME-Version; b=fgJsxrw8FZvAT6LZnY+JkVSYSXWd7ACWehLxcYh/S5JZU136kC9QWyY4QSnfgA5u5 Wk0vKyjoxIlLyuLXEBNZBi6OIR4DzL0KuNqlbHwCoM8WlaSs9d3t8vMr80S8wrF5wF qyJEcuXe25rOij+wlHaC/cdMXKDfNceRZ81JV0dA= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 07/31] Documentation: Add binding for kalray,kv3-1-pwr-ctrl Date: Fri, 20 Jan 2023 15:09:38 +0100 Message-ID: <20230120141002.2442-8-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC From: Jules Maselbas Add documentation for `kalray,kv3-1-pwr-ctrl` binding. Co-developed-by: Jules Maselbas Signed-off-by: Jules Maselbas Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: new patch .../kalray/kalray,kv3-1-pwr-ctrl.yaml | 29 +++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 Documentation/devicetree/bindings/kalray/kalray,kv3-1-pwr-ctrl.yaml diff --git a/Documentation/devicetree/bindings/kalray/kalray,kv3-1-pwr-ctrl.yaml b/Documentation/devicetree/bindings/kalray/kalray,kv3-1-pwr-ctrl.yaml new file mode 100644 index 000000000000..968674bb0c63 --- /dev/null +++ b/Documentation/devicetree/bindings/kalray/kalray,kv3-1-pwr-ctrl.yaml @@ -0,0 +1,29 @@ +# SPDX-License-Identifier: GPL-2.0 +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/kalray/kalray,kv3-1-pwr-ctrl# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Kalray cluster Power Controller (pwr-ctrl) + +description: | + The Power Controller (pwr-ctrl) control cores reset and wake-up procedure. + +properties: + compatible: + const: kalray,kv3-1-pwr-ctrl + reg: + maxItems: 1 + +required: + - compatible + - reg + +examples: + - | + pwr_ctrl: power-controller@a40000 { + compatible = "kalray,kv3-1-pwr-ctrl"; + reg = <0x00 0xa40000 0x00 0x4158>; + }; + +... From patchwork Fri Jan 20 14:09:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13109905 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E49CC27C7C for ; Fri, 20 Jan 2023 14:11:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230253AbjATOLG (ORCPT ); Fri, 20 Jan 2023 09:11:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37660 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230233AbjATOLD (ORCPT ); Fri, 20 Jan 2023 09:11:03 -0500 Received: from fx304.security-mail.net (smtpout30.security-mail.net [85.31.212.34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D5ADFC639F for ; Fri, 20 Jan 2023 06:10:31 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by fx304.security-mail.net (Postfix) with ESMTP id D59829CC7E for ; Fri, 20 Jan 2023 15:10:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674223829; bh=RQWzU2KqdgJRpX94P0CtkFxRAHBwvD2KcEBPPyxcloM=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ufs/O/3xkf51fyWn74oOGkCIXq6kZPzPk9J+Dj0aso7jNtJINHInHvgTnRApvsRVY do/Qj9iEPC2cSZi4tOTJNv+velSS+SmlwB9sIj7Og40bTlNu9VTTqRvPG687NiJn9c G0x6xxxrBVy0nG6g4R2KXoW1JMh8KGGJ5AbZVF0k= Received: from fx304 (localhost [127.0.0.1]) by fx304.security-mail.net (Postfix) with ESMTP id 9D7F39D0CD; Fri, 20 Jan 2023 15:10:29 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx304.security-mail.net (Postfix) with ESMTPS id 9A3EE9D037; Fri, 20 Jan 2023 15:10:28 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 6705027E0430; Fri, 20 Jan 2023 15:10:28 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 4F41727E043E; Fri, 20 Jan 2023 15:10:28 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id zKBDQHbhDaEG; Fri, 20 Jan 2023 15:10:28 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id CDB2C27E0442; Fri, 20 Jan 2023 15:10:27 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <6f30.63caa0d4.98f72.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 4F41727E043E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223828; bh=W0RwmQvGfQWqSUcWAC1Dc3r+zpspRVwPqWqReyFBvZA=; h=From:To:Date:Message-Id:MIME-Version; b=MHdrSbZ7EFMkFaZDXJohPgZy04uIddLyTkjn3Ka32JU97Sdek2KRS4kbc9iHt+Kfu aYZGNKJMocHa+4+1QyxX/yAipaYsbh5POzPgDYjEwI/cvCS95HL2G9aGBuX0XF0+jp Iz8iNW4Xsccsvzp27uZAJvfTCRIiUBWekPYCkaKg= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 08/31] kvx: Add ELF-related definitions Date: Fri, 20 Jan 2023 15:09:39 +0100 Message-ID: <20230120141002.2442-9-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Add ELF-related definitions for kvx, including: EM_KVX, AUDIT_ARCH_KVX and NT_KVX_TCA. Co-developed-by: Clement Leger Signed-off-by: Clement Leger Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: no changes include/uapi/linux/audit.h | 1 + include/uapi/linux/elf-em.h | 1 + include/uapi/linux/elf.h | 1 + 3 files changed, 3 insertions(+) diff --git a/include/uapi/linux/audit.h b/include/uapi/linux/audit.h index d676ed2b246e..4db7aa3f84c7 100644 --- a/include/uapi/linux/audit.h +++ b/include/uapi/linux/audit.h @@ -402,6 +402,7 @@ enum { #define AUDIT_ARCH_HEXAGON (EM_HEXAGON) #define AUDIT_ARCH_I386 (EM_386|__AUDIT_ARCH_LE) #define AUDIT_ARCH_IA64 (EM_IA_64|__AUDIT_ARCH_64BIT|__AUDIT_ARCH_LE) +#define AUDIT_ARCH_KVX (EM_KVX|__AUDIT_ARCH_64BIT|__AUDIT_ARCH_LE) #define AUDIT_ARCH_M32R (EM_M32R) #define AUDIT_ARCH_M68K (EM_68K) #define AUDIT_ARCH_MICROBLAZE (EM_MICROBLAZE) diff --git a/include/uapi/linux/elf-em.h b/include/uapi/linux/elf-em.h index ef38c2bc5ab7..9cc348be7f86 100644 --- a/include/uapi/linux/elf-em.h +++ b/include/uapi/linux/elf-em.h @@ -51,6 +51,7 @@ #define EM_RISCV 243 /* RISC-V */ #define EM_BPF 247 /* Linux BPF - in-kernel virtual machine */ #define EM_CSKY 252 /* C-SKY */ +#define EM_KVX 256 /* Kalray VLIW Architecture */ #define EM_LOONGARCH 258 /* LoongArch */ #define EM_FRV 0x5441 /* Fujitsu FR-V */ diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h index c7b056af9ef0..49094f3be06c 100644 --- a/include/uapi/linux/elf.h +++ b/include/uapi/linux/elf.h @@ -444,6 +444,7 @@ typedef struct elf64_shdr { #define NT_LOONGARCH_LSX 0xa02 /* LoongArch Loongson SIMD Extension registers */ #define NT_LOONGARCH_LASX 0xa03 /* LoongArch Loongson Advanced SIMD Extension registers */ #define NT_LOONGARCH_LBT 0xa04 /* LoongArch Loongson Binary Translation registers */ +#define NT_KVX_TCA 0x900 /* kvx TCA registers */ /* Note types with note name "GNU" */ #define NT_GNU_PROPERTY_TYPE_0 5 From patchwork Fri Jan 20 14:09:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13109915 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9030FC25B4E for ; Fri, 20 Jan 2023 14:12:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230207AbjATOMf (ORCPT ); Fri, 20 Jan 2023 09:12:35 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230189AbjATOMY (ORCPT ); Fri, 20 Jan 2023 09:12:24 -0500 Received: from fx306.security-mail.net (smtpout30.security-mail.net [85.31.212.36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5BF34C79C5 for ; Fri, 20 Jan 2023 06:11:02 -0800 (PST) Received: from localhost (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id 1AD5135CF9F for ; Fri, 20 Jan 2023 15:10:30 +0100 (CET) Received: from fx306 (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id CF8DA35CF8F; Fri, 20 Jan 2023 15:10:29 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx306.security-mail.net (Postfix) with ESMTPS id 35D4435CF76; Fri, 20 Jan 2023 15:10:29 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id E334C27E043A; Fri, 20 Jan 2023 15:10:28 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id CB0C827E0430; Fri, 20 Jan 2023 15:10:28 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Xz5sHM8fC0-H; Fri, 20 Jan 2023 15:10:28 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 4DB6D27E0439; Fri, 20 Jan 2023 15:10:28 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <1474a.63caa0d5.34a0b.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu CB0C827E0430 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223828; bh=EuQZx9FUnDiDRTk50xpDbbkQoW5rtljZYfO+Kb3RngA=; h=From:To:Date:Message-Id:MIME-Version; b=pJAWVR/O4/vNSXgI2fqCq1mzyA4nHvReixv2Rv8rbziGt64b3rKESJ6G/BTtYtP7k HN8eWKe3YhaHUmwfMymJf5y/s+APHU15usfVf0MqQsOFPf7Yaejddkwnpw9jWKu3WF Pr9oOjDjubHs4HFOGHtGjs+2tlvfCLSzf6UsEuT0= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 09/31] kvx: Add build infrastructure Date: Fri, 20 Jan 2023 15:09:40 +0100 Message-ID: <20230120141002.2442-10-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Add Kbuild, Makefile, Kconfig and link script for kvx build infrastructure. Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Guillaume Thouvenin Signed-off-by: Guillaume Thouvenin Co-developed-by: Jonathan Borne Signed-off-by: Jonathan Borne Co-developed-by: Jules Maselbas Signed-off-by: Jules Maselbas Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Marc Poulhiès Signed-off-by: Marc Poulhiès Co-developed-by: Marius Gligor Signed-off-by: Marius Gligor Co-developed-by: Samuel Jones Signed-off-by: Samuel Jones Co-developed-by: Vincent Chardon Signed-off-by: Vincent Chardon Co-developed-by: Yann Sionneau Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: - typos and formatting fixes, removed int from PGTABLE_LEVELS - renamed default_defconfig to defconfig in arch/kvx/Makefile - Fix clean target raising an error from gcc (LIBGCC) arch/kvx/Kconfig | 224 +++++++++++++++++++++++++++++++ arch/kvx/Kconfig.debug | 70 ++++++++++ arch/kvx/Makefile | 53 ++++++++ arch/kvx/include/asm/Kbuild | 20 +++ arch/kvx/include/uapi/asm/Kbuild | 1 + arch/kvx/kernel/Makefile | 15 +++ arch/kvx/kernel/kvx_ksyms.c | 24 ++++ arch/kvx/kernel/vmlinux.lds.S | 150 +++++++++++++++++++++ arch/kvx/lib/Makefile | 6 + arch/kvx/mm/Makefile | 8 ++ 10 files changed, 571 insertions(+) create mode 100644 arch/kvx/Kconfig create mode 100644 arch/kvx/Kconfig.debug create mode 100644 arch/kvx/Makefile create mode 100644 arch/kvx/include/asm/Kbuild create mode 100644 arch/kvx/include/uapi/asm/Kbuild create mode 100644 arch/kvx/kernel/Makefile create mode 100644 arch/kvx/kernel/kvx_ksyms.c create mode 100644 arch/kvx/kernel/vmlinux.lds.S create mode 100644 arch/kvx/lib/Makefile create mode 100644 arch/kvx/mm/Makefile diff --git a/arch/kvx/Kconfig b/arch/kvx/Kconfig new file mode 100644 index 000000000000..0bdba6a3b08a --- /dev/null +++ b/arch/kvx/Kconfig @@ -0,0 +1,224 @@ +# +# For a description of the syntax of this configuration file, +# see Documentation/kbuild/kconfig-language.txt. +# + +config 64BIT + def_bool y + +config GENERIC_CALIBRATE_DELAY + def_bool y + +config FIX_EARLYCON_MEM + def_bool y + +config MMU + def_bool y + +config KALLSYMS_BASE_RELATIVE + def_bool n + +config GENERIC_CSUM + def_bool y + +config RWSEM_GENERIC_SPINLOCK + def_bool y + +config GENERIC_HWEIGHT + def_bool y + +config ARCH_MMAP_RND_BITS_MAX + default 24 + +config ARCH_MMAP_RND_BITS_MIN + default 18 + +config STACKTRACE_SUPPORT + def_bool y + +config LOCKDEP_SUPPORT + def_bool y + +config GENERIC_BUG + def_bool y + depends on BUG + +config KVX_4K_PAGES + def_bool y + +config KVX + def_bool y + select ARCH_CLOCKSOURCE_DATA + select ARCH_DMA_ADDR_T_64BIT + select ARCH_HAS_DEVMEM_IS_ALLOWED + select ARCH_HAS_DMA_PREP_COHERENT + select ARCH_HAS_ELF_RANDOMIZE + select ARCH_HAS_PTE_SPECIAL + select ARCH_HAS_SETUP_DMA_OPS if IOMMU_SUPPORT + select ARCH_HAS_SYNC_DMA_FOR_DEVICE + select ARCH_HAS_SYNC_DMA_FOR_CPU + select ARCH_HAS_TEARDOWN_DMA_OPS if IOMMU_SUPPORT + select ARCH_OPTIONAL_KERNEL_RWX_DEFAULT + select ARCH_USE_QUEUED_SPINLOCKS + select ARCH_USE_QUEUED_RWLOCKS + select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT + select ARCH_WANT_FRAME_POINTERS + select CLKSRC_OF + select COMMON_CLK + select DMA_DIRECT_REMAP + select GENERIC_ALLOCATOR + select GENERIC_CLOCKEVENTS + select GENERIC_CLOCKEVENTS + select GENERIC_CPU_DEVICES + select GENERIC_IOMAP + select GENERIC_IOREMAP + select GENERIC_IRQ_CHIP + select GENERIC_IRQ_PROBE + select GENERIC_IRQ_SHOW + select GENERIC_SCHED_CLOCK + select HAVE_ARCH_AUDITSYSCALL + select HAVE_ARCH_BITREVERSE + select HAVE_ARCH_MMAP_RND_BITS + select HAVE_ASM_MODVERSIONS + select HAVE_DEBUG_KMEMLEAK + select HAVE_EFFICIENT_UNALIGNED_ACCESS + select HAVE_FUTEX_CMPXCHG if FUTEX + select HAVE_IOREMAP_PROT + select HAVE_MEMBLOCK_NODE_MAP + select HAVE_PCI + select HAVE_STACKPROTECTOR + select HAVE_SYSCALL_TRACEPOINTS + select IOMMU_DMA if IOMMU_SUPPORT + select KVX_APIC_GIC + select KVX_APIC_MAILBOX + select KVX_CORE_INTC + select KVX_ITGEN + select KVX_WATCHDOG + select MODULES_USE_ELF_RELA + select OF + select OF_EARLY_FLATTREE + select OF_RESERVED_MEM + select PCI_DOMAINS_GENERIC if PCI + select SPARSE_IRQ + select SYSCTL_EXCEPTION_TRACE + select THREAD_INFO_IN_TASK + select TIMER_OF + select TRACE_IRQFLAGS_SUPPORT + select WATCHDOG + select ZONE_DMA32 + +config PGTABLE_LEVELS + default 3 + +config HAVE_KPROBES + def_bool n + +menu "System setup" + +config POISON_INITMEM + bool "Enable to poison freed initmem" + default y + help + In order to debug initmem, using poison allows to verify if some + data/code is still using them. Enable this for debug purposes. + +config KVX_PHYS_OFFSET + hex "RAM address of memory base" + default 0x100000000 + +config KVX_PAGE_OFFSET + hex "kernel virtual address of memory base" + default 0xFFFFFF8000000000 + +config ARCH_FLATMEM_ENABLE + def_bool y + +config ARCH_SPARSEMEM_ENABLE + def_bool y + +config ARCH_SPARSEMEM_DEFAULT + def_bool ARCH_SPARSEMEM_ENABLE + +config ARCH_SELECT_MEMORY_MODEL + def_bool ARCH_SPARSEMEM_ENABLE + +config STACK_MAX_DEPTH_TO_PRINT + int "Maximum depth of stack to print" + range 1 128 + default "24" + +config SECURE_DAME_HANDLING + bool "Secure DAME handling" + default y + help + In order to securely handle Data Asynchronous Memory Errors, we need + to do a barrier upon kernel entry when coming from userspace. This + barrier guarantees us that any pending DAME will be serviced right + away. We also need to do a barrier when returning from kernel to user. + This way, if the kernel or the user triggered a DAME, it will be + serviced by knowing we are coming from kernel or user and avoid + pulling the wrong lever (panic for kernel or sigfault for user). + This can be costly but ensures that user cannot interfere with kernel. + /!\ Do not disable unless you want to open a giant breach between + user and kernel /!\ + +config CACHECTL_UNSAFE_PHYS_OPERATIONS + bool "Enable cachectl syscall unsafe physical operations" + default n + help + Enable cachectl syscall to allow writebacking/invalidating ranges + based on physical addresses. These operations requires the + CAP_SYS_ADMIN capability. + +config ENABLE_TCA + bool "Enable TCA coprocessor support" + default y + help + This option enables TCA coprocessor support. It will allow the user to + use the coprocessor and save registers on context switch if used. + Registers content will also be cleared when switching. + +config SMP + bool "Symmetric multi-processing support" + default n + select GENERIC_SMP_IDLE_THREAD + select GENERIC_IRQ_IPI + select IRQ_DOMAIN_HIERARCHY + select IRQ_DOMAIN + help + This enables support for systems with more than one CPU. If you have + a system with only one CPU, say N. If you have a system with more + than one CPU, say Y. + + If you say N here, the kernel will run on uni- and multiprocessor + machines, but will use only one CPU of a multiprocessor machine. If + you say Y here, the kernel will run on many, but not all, + uniprocessor machines. On a uniprocessor machine, the kernel + will run faster if you say N here. + +config NR_CPUS + int "Maximum number of CPUs" + range 1 16 + default "16" + depends on SMP + help + Kalray support can handle a maximum of 16 CPUs. + +config KVX_PAGE_SHIFT + int + default 12 + +config CMDLINE + string "Default kernel command string" + default "" + help + On some architectures there is currently no way for the boot loader + to pass arguments to the kernel. For these architectures, you should + supply some command-line options at build time by entering them + here. + +endmenu + +menu "Kernel Features" +source "kernel/Kconfig.hz" +endmenu diff --git a/arch/kvx/Kconfig.debug b/arch/kvx/Kconfig.debug new file mode 100644 index 000000000000..171f71288ee6 --- /dev/null +++ b/arch/kvx/Kconfig.debug @@ -0,0 +1,70 @@ +menu "KVX debugging" + +config KVX_DEBUG_ASN + bool "Check ASN before writing TLB entry" + default n + help + This option allows to check if the ASN of the current + process matches the ASN found in MMC. If it is not the + case an error will be printed. + +config KVX_DEBUG_TLB_WRITE + bool "Enable TLBs write checks" + default n + help + Enabling this option will enable TLB access checks. This is + particularly helpful when modifying the assembly code responsible + for TLB refill. If set, mmc.e will be checked each time the TLB are + written and a panic will be thrown on error. + +config KVX_DEBUG_TLB_ACCESS + bool "Enable TLBs accesses logging" + default n + help + Enabling this option will enable TLB entry manipulation logging. + Each time an entry is added to the TLBs, it is logged in an array + readable via gdb scripts. This can be useful to understand strange + crashes related to suspicious virtual/physical addresses. + +config KVX_DEBUG_TLB_ACCESS_BITS + int "Number of bits used as index of entries in log table" + default 12 + depends on KVX_DEBUG_TLB_ACCESS + help + Set the number of bits used as index of entries that will be logged + in a ring buffer called kvx_tlb_access. One entry in the table + contains registers TEL, TEH and MMC. It also logs the type of the + operations (0:read, 1:write, 2:probe). Buffer is per CPU. For one + entry 24 bytes are used. So by default it uses 96KB of memory per + CPU to store 2^12 (4096) entries. + +config KVX_MMU_STATS + bool "Register MMU stats debugfs entries" + default n + depends on DEBUG_FS + help + Enable debugfs attribute which will allow inspecting various metrics + regarding MMU: + - Number of nomapping traps handled + - avg/min/max time for nomapping refill (user/kernel) + +config DEBUG_EXCEPTION_STACK + bool "Enable exception stack debugging" + default n + help + Enable stack check debugging when entering/exiting + exception handlers. + This can be particularly helpful after modifying stack + handling to see if stack when exiting is the same as the one + when entering exception handler. + +config DEBUG_SFR_SET_MASK + bool "Enable SFR set_mask debugging" + default n + help + Verify that values written using kvx_sfr_set_mask match the mask. + This ensure that no extra bits of SFR will be overridden by some + incorrectly truncated values. This can lead to huge problems by + modifying important bits in system registers. + +endmenu diff --git a/arch/kvx/Makefile b/arch/kvx/Makefile new file mode 100644 index 000000000000..78d6d7c9c43e --- /dev/null +++ b/arch/kvx/Makefile @@ -0,0 +1,53 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Copyright (C) 2018-2023 Kalray Inc. + +ifeq ($(CROSS_COMPILE),) +CROSS_COMPILE := kvx-elf- +endif + +KBUILD_DEFCONFIG := defconfig + +LDFLAGS_vmlinux := -X +OBJCOPYFLAGS := -O binary -R .comment -R .note -R .bootloader -S + +DEFAULT_OPTS := -nostdlib -fno-builtin -march=kv3-1 + +# Link with libgcc to get __div* builtins. +LIBGCC := $(shell $(CC) $(DEFAULT_OPTS) --print-libgcc-file-name) + +KBUILD_CFLAGS += $(DEFAULT_OPTS) +KBUILD_AFLAGS += $(DEFAULT_OPTS) +KBUILD_CFLAGS_MODULE += -mfarcall + +KBUILD_LDFLAGS += -m elf64kvx + +head-y := arch/kvx/kernel/head.o +libs-y += $(LIBGCC) +libs-y += arch/kvx/lib/ +core-y += arch/kvx/kernel/ \ + arch/kvx/mm/ \ + arch/kvx/platform/ +# Final targets +all: vmlinux + +BOOT_TARGETS = bImage bImage.bin bImage.bz2 bImage.gz bImage.lzma bImage.lzo + +$(BOOT_TARGETS): vmlinux + $(Q)$(MAKE) $(build)=$(boot) $(boot)/$@ + +install: + $(Q)$(MAKE) $(build)=$(boot) BOOTIMAGE=$(KBUILD_IMAGE) install + +define archhelp + echo '* bImage - Alias to selected kernel format (bImage.gz by default)' + echo ' bImage.bin - Uncompressed Kernel-only image for barebox (arch/$(ARCH)/boot/bImage.bin)' + echo ' bImage.bz2 - Kernel-only image for barebox (arch/$(ARCH)/boot/bImage.bz2)' + echo '* bImage.gz - Kernel-only image for barebox (arch/$(ARCH)/boot/bImage.gz)' + echo ' bImage.lzma - Kernel-only image for barebox (arch/$(ARCH)/boot/bImage.lzma)' + echo ' bImage.lzo - Kernel-only image for barebox (arch/$(ARCH)/boot/bImage.lzo)' + echo ' install - Install kernel using' + echo ' (your) ~/bin/$(INSTALLKERNEL) or' + echo ' (distribution) PATH: $(INSTALLKERNEL) or' + echo ' install to $$(INSTALL_PATH)' +endef diff --git a/arch/kvx/include/asm/Kbuild b/arch/kvx/include/asm/Kbuild new file mode 100644 index 000000000000..ea73552faa10 --- /dev/null +++ b/arch/kvx/include/asm/Kbuild @@ -0,0 +1,20 @@ +generic-y += asm-offsets.h +generic-y += clkdev.h +generic-y += auxvec.h +generic-y += bpf_perf_event.h +generic-y += cmpxchg-local.h +generic-y += errno.h +generic-y += extable.h +generic-y += export.h +generic-y += kvm_para.h +generic-y += mcs_spinlock.h +generic-y += mman.h +generic-y += param.h +generic-y += qrwlock.h +generic-y += qspinlock.h +generic-y += rwsem.h +generic-y += sockios.h +generic-y += stat.h +generic-y += statfs.h +generic-y += ucontext.h +generic-y += user.h diff --git a/arch/kvx/include/uapi/asm/Kbuild b/arch/kvx/include/uapi/asm/Kbuild new file mode 100644 index 000000000000..8b137891791f --- /dev/null +++ b/arch/kvx/include/uapi/asm/Kbuild @@ -0,0 +1 @@ + diff --git a/arch/kvx/kernel/Makefile b/arch/kvx/kernel/Makefile new file mode 100644 index 000000000000..735ba3893027 --- /dev/null +++ b/arch/kvx/kernel/Makefile @@ -0,0 +1,15 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Copyright (C) 2019-2023 Kalray Inc. +# + +obj-y := head.o setup.o process.o traps.o common.o time.o prom.o kvx_ksyms.o \ + irq.o cpuinfo.o entry.o ptrace.o syscall_table.o signal.o sys_kvx.o \ + stacktrace.o dame_handler.o vdso.o debug.o break_hook.o \ + reset.o io.o + +obj-$(CONFIG_SMP) += smp.o smpboot.o +obj-$(CONFIG_MODULES) += module.o +CFLAGS_module.o += -Wstrict-overflow -fstrict-overflow + +extra-y += vmlinux.lds diff --git a/arch/kvx/kernel/kvx_ksyms.c b/arch/kvx/kernel/kvx_ksyms.c new file mode 100644 index 000000000000..18990aaf259f --- /dev/null +++ b/arch/kvx/kernel/kvx_ksyms.c @@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * derived from arch/nios2/kernel/nios2_ksyms.c + * + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Yann Sionneau + */ + +#include +#include + +/* + * libgcc functions - functions that are used internally by the + * compiler... (prototypes are not correct though, but that + * doesn't really matter since they're not versioned). + */ +#define DECLARE_EXPORT(name) extern void name(void); EXPORT_SYMBOL(name) + +DECLARE_EXPORT(__moddi3); +DECLARE_EXPORT(__umoddi3); +DECLARE_EXPORT(__divdi3); +DECLARE_EXPORT(__udivdi3); +DECLARE_EXPORT(__multi3); diff --git a/arch/kvx/kernel/vmlinux.lds.S b/arch/kvx/kernel/vmlinux.lds.S new file mode 100644 index 000000000000..e8772334bd97 --- /dev/null +++ b/arch/kvx/kernel/vmlinux.lds.S @@ -0,0 +1,150 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + * Marius Gligor + * Marc Poulhiès + * Yann Sionneau + */ + +#include +#include +#include +#include +#include +#include +#include + +#define BOOT_ENTRY 0x0 +#define DTB_DEFAULT_SIZE (64 * 1024) + +#define LOAD_OFFSET (PAGE_OFFSET - PHYS_OFFSET) +#include + +OUTPUT_FORMAT("elf64-kvx") +ENTRY(kvx_start) + +#define HANDLER_SECTION(__sec, __name) \ + __sec ## _ ## __name ## _start = .; \ + KEEP(*(.##__sec ##.## __name)); \ + . = __sec ## _ ##__name ## _start + EXCEPTION_STRIDE; + +/** + * Generate correct section positioning for exception handling + * Since we need it twice for early exception handler and normal + * exception handler, factorize it here. + */ +#define EXCEPTION_SECTIONS(__sec) \ + __ ## __sec ## _start = ABSOLUTE(.); \ + HANDLER_SECTION(__sec,debug) \ + HANDLER_SECTION(__sec,trap) \ + HANDLER_SECTION(__sec,interrupt) \ + HANDLER_SECTION(__sec,syscall) + +jiffies = jiffies_64; +SECTIONS +{ + . = BOOT_ENTRY; + .boot : + { + __kernel_smem_code_start = .; + KEEP(*(.boot.startup)); + KEEP(*(.boot.*)); + __kernel_smem_code_end = .; + } + + . = PAGE_OFFSET; + _start = .; + + _stext = .; + __init_begin = .; + __inittext_start = .; + .exit.text : AT(ADDR(.exit.text) - LOAD_OFFSET) + { + EXIT_TEXT + } + + .early_exception ALIGN(EXCEPTION_ALIGNMENT) : + AT(ADDR(.early_exception) - LOAD_OFFSET) + { + EXCEPTION_SECTIONS(early_exception) + } + + HEAD_TEXT_SECTION + INIT_TEXT_SECTION(PAGE_SIZE) + . = ALIGN(PAGE_SIZE); + __inittext_end = .; + __initdata_start = .; + INIT_DATA_SECTION(16) + + /* we have to discard exit text and such at runtime, not link time */ + .exit.data : AT(ADDR(.exit.data) - LOAD_OFFSET) + { + EXIT_DATA + } + + PERCPU_SECTION(L1_CACHE_BYTES) + . = ALIGN(PAGE_SIZE); + __initdata_end = .; + __init_end = .; + + /* Everything below this point will be mapped RO EXEC up to _etext */ + .text ALIGN(PAGE_SIZE) : AT(ADDR(.text) - LOAD_OFFSET) + { + _text = .; + EXCEPTION_SECTIONS(exception) + *(.exception.text) + . = ALIGN(PAGE_SIZE); + __exception_end = .; + TEXT_TEXT + SCHED_TEXT + CPUIDLE_TEXT + LOCK_TEXT + KPROBES_TEXT + ENTRY_TEXT + IRQENTRY_TEXT + SOFTIRQENTRY_TEXT + *(.fixup) + } + . = ALIGN(PAGE_SIZE); + _etext = .; + + /* Everything below this point will be mapped RO NOEXEC up to _sdata */ + __rodata_start = .; + RO_DATA(PAGE_SIZE) + EXCEPTION_TABLE(8) + . = ALIGN(32); + .dtb : AT(ADDR(.dtb) - LOAD_OFFSET) + { + __dtb_start = .; + . += DTB_DEFAULT_SIZE; + __dtb_end = .; + } + . = ALIGN(PAGE_SIZE); + __rodata_end = .; + + /* Everything below this point will be mapped RW NOEXEC up to _end */ + _sdata = .; + RW_DATA(L1_CACHE_BYTES, PAGE_SIZE, THREAD_SIZE) + _edata = .; + + BSS_SECTION(32, 32, 32) + . = ALIGN(PAGE_SIZE); + _end = .; + + /* This page will be mapped using a FIXMAP */ + .gdb_page ALIGN(PAGE_SIZE) : AT(ADDR(.gdb_page) - LOAD_OFFSET) + { + _debug_start = ADDR(.gdb_page) - LOAD_OFFSET; + . += PAGE_SIZE; + } + _debug_start_lma = ASM_FIX_TO_VIRT(FIX_GDB_MEM_BASE_IDX); + + /* Debugging sections */ + STABS_DEBUG + DWARF_DEBUG + + /* Sections to be discarded -- must be last */ + DISCARDS +} diff --git a/arch/kvx/lib/Makefile b/arch/kvx/lib/Makefile new file mode 100644 index 000000000000..ddb666c244e1 --- /dev/null +++ b/arch/kvx/lib/Makefile @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Copyright (C) 2017-2023 Kalray Inc. +# + +lib-y := usercopy.o clear_page.o copy_page.o memcpy.o memset.o strlen.o delay.o diff --git a/arch/kvx/mm/Makefile b/arch/kvx/mm/Makefile new file mode 100644 index 000000000000..e97662cf5fed --- /dev/null +++ b/arch/kvx/mm/Makefile @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Copyright (C) 2017-2023 Kalray Inc. +# + +obj-y := init.o mmu.o fault.o tlb.o extable.o dma-mapping.o cacheflush.o +obj-$(CONFIG_KVX_MMU_STATS) += mmu_stats.o +obj-$(CONFIG_STRICT_DEVMEM) += mmap.o From patchwork Fri Jan 20 14:09:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13109907 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF949C38159 for ; Fri, 20 Jan 2023 14:11:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230406AbjATOLo (ORCPT ); Fri, 20 Jan 2023 09:11:44 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37686 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230206AbjATOLN (ORCPT ); Fri, 20 Jan 2023 09:11:13 -0500 Received: from fx408.security-mail.net (smtpout140.security-mail.net [85.31.212.148]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 60BF1BCE2B for ; Fri, 20 Jan 2023 06:10:36 -0800 (PST) Received: from localhost (fx408.security-mail.net [127.0.0.1]) by fx408.security-mail.net (Postfix) with ESMTP id 43EDD322B95 for ; Fri, 20 Jan 2023 15:10:32 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674223832; bh=Rr9QfC52jxWXt9/L6/0rXr58eBV1kbp/PCidEQ6niXo=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=a2gvWIDYicwURek2prNsBHdw/qrKpGsVqft9DPlPFrgjW2HCGVFrGzmP4Gz6op5xz Nko1t3KK5BB/nraj3gB03N9S37TcvUpjJxkHcoZYU1QO9inUkrBkXM+uQLE1rIifYt l1HfenLWwY7FwN+G0mYN8aw90/b3ggrEQNcegqj8= Received: from fx408 (fx408.security-mail.net [127.0.0.1]) by fx408.security-mail.net (Postfix) with ESMTP id 06ADB322B21; Fri, 20 Jan 2023 15:10:32 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx408.security-mail.net (Postfix) with ESMTPS id 2F716322939; Fri, 20 Jan 2023 15:10:30 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id DEDF827E0442; Fri, 20 Jan 2023 15:10:29 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id BA21327E043E; Fri, 20 Jan 2023 15:10:29 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id IVLBnrnY8q7M; Fri, 20 Jan 2023 15:10:29 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 3038927E0439; Fri, 20 Jan 2023 15:10:29 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <8eb9.63caa0d6.2d305.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu BA21327E043E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223829; bh=VF1oksnoaKlh4VeYXhJaqi0lsOke2F/Yys7H8lI9zVo=; h=From:To:Date:Message-Id:MIME-Version; b=py7k0YutOlZqWwMG5lZJu6/TDyjQJDv+WqktPHeaionlMv9JDM0vUt7RyUtwuAvEc BAaiO7pdNQhE8prp3XxiJKJo9FO4waSZpPdSMO2dSvz3s8gvb8ZkIDzBvXuiXEwQxt BrrCzIEAI8NSGikf8xD35M5SllBU3r29he3vWp5M= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 11/31] kvx: Add atomic/locking headers Date: Fri, 20 Jan 2023 15:09:42 +0100 Message-ID: <20230120141002.2442-12-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Add common headers (atomic, bitops, barrier and locking) for basic kvx support. Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Jules Maselbas Signed-off-by: Jules Maselbas Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Julien Villette Signed-off-by: Julien Villette Co-developed-by: Yann Sionneau Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: - use {READ,WRITE}_ONCE for arch_atomic64_{read,set} - use asm-generic/bitops/atomic.h instead of __test_and_*_bit - removed duplicated includes - rewrite xchg and cmpxchg in C using builtins for acswap insn arch/kvx/include/asm/atomic.h | 104 ++++++++++++++++++++ arch/kvx/include/asm/barrier.h | 15 +++ arch/kvx/include/asm/bitops.h | 115 ++++++++++++++++++++++ arch/kvx/include/asm/bitrev.h | 32 +++++++ arch/kvx/include/asm/cmpxchg.h | 170 +++++++++++++++++++++++++++++++++ 5 files changed, 436 insertions(+) create mode 100644 arch/kvx/include/asm/atomic.h create mode 100644 arch/kvx/include/asm/barrier.h create mode 100644 arch/kvx/include/asm/bitops.h create mode 100644 arch/kvx/include/asm/bitrev.h create mode 100644 arch/kvx/include/asm/cmpxchg.h diff --git a/arch/kvx/include/asm/atomic.h b/arch/kvx/include/asm/atomic.h new file mode 100644 index 000000000000..bea3d70785b1 --- /dev/null +++ b/arch/kvx/include/asm/atomic.h @@ -0,0 +1,104 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_ATOMIC_H +#define _ASM_KVX_ATOMIC_H + +#include + +#include + +#define ATOMIC64_INIT(i) { (i) } + +#define arch_atomic64_cmpxchg(v, old, new) (arch_cmpxchg(&((v)->counter), old, new)) +#define arch_atomic64_xchg(v, new) (arch_xchg(&((v)->counter), new)) + +static inline long arch_atomic64_read(const atomic64_t *v) +{ + return READ_ONCE(v->counter); +} + +static inline void arch_atomic64_set(atomic64_t *v, long i) +{ + WRITE_ONCE(v->counter, i); +} + +#define ATOMIC64_RETURN_OP(op, c_op) \ +static inline long arch_atomic64_##op##_return(long i, atomic64_t *v) \ +{ \ + long new, old, ret; \ + \ + do { \ + old = v->counter; \ + new = old c_op i; \ + ret = arch_cmpxchg(&v->counter, old, new); \ + } while (ret != old); \ + \ + return new; \ +} + +#define ATOMIC64_OP(op, c_op) \ +static inline void arch_atomic64_##op(long i, atomic64_t *v) \ +{ \ + long new, old, ret; \ + \ + do { \ + old = v->counter; \ + new = old c_op i; \ + ret = arch_cmpxchg(&v->counter, old, new); \ + } while (ret != old); \ +} + +#define ATOMIC64_FETCH_OP(op, c_op) \ +static inline long arch_atomic64_fetch_##op(long i, atomic64_t *v) \ +{ \ + long new, old, ret; \ + \ + do { \ + old = v->counter; \ + new = old c_op i; \ + ret = arch_cmpxchg(&v->counter, old, new); \ + } while (ret != old); \ + \ + return old; \ +} + +#define ATOMIC64_OPS(op, c_op) \ + ATOMIC64_OP(op, c_op) \ + ATOMIC64_RETURN_OP(op, c_op) \ + ATOMIC64_FETCH_OP(op, c_op) + +ATOMIC64_OPS(and, &) +ATOMIC64_OPS(or, |) +ATOMIC64_OPS(xor, ^) +ATOMIC64_OPS(add, +) +ATOMIC64_OPS(sub, -) + +#undef ATOMIC64_OPS +#undef ATOMIC64_FETCH_OP +#undef ATOMIC64_OP + +static inline int arch_atomic_add_return(int i, atomic_t *v) +{ + int new, old, ret; + + do { + old = v->counter; + new = old + i; + ret = arch_cmpxchg(&v->counter, old, new); + } while (ret != old); + + return new; +} + +static inline int arch_atomic_sub_return(int i, atomic_t *v) +{ + return arch_atomic_add_return(-i, v); +} + +#include + +#endif /* _ASM_KVX_ATOMIC_H */ diff --git a/arch/kvx/include/asm/barrier.h b/arch/kvx/include/asm/barrier.h new file mode 100644 index 000000000000..371f1c70746d --- /dev/null +++ b/arch/kvx/include/asm/barrier.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_BARRIER_H +#define _ASM_KVX_BARRIER_H + +/* fence is sufficient to guarantee write ordering */ +#define mb() __builtin_kvx_fence() + +#include + +#endif /* _ASM_KVX_BARRIER_H */ diff --git a/arch/kvx/include/asm/bitops.h b/arch/kvx/include/asm/bitops.h new file mode 100644 index 000000000000..c643f4765059 --- /dev/null +++ b/arch/kvx/include/asm/bitops.h @@ -0,0 +1,115 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Yann Sionneau + */ + +#ifndef _ASM_KVX_BITOPS_H +#define _ASM_KVX_BITOPS_H + +#ifdef __KERNEL__ + +#ifndef _LINUX_BITOPS_H +#error only can be included directly +#endif + +#include + +static inline int fls(int x) +{ + return 32 - __builtin_kvx_clzw(x); +} + +static inline int fls64(__u64 x) +{ + return 64 - __builtin_kvx_clzd(x); +} + +/** + * __ffs - find first set bit in word + * @word: The word to search + * + * Undefined if no set bit exists, so code should check against 0 first. + */ +static inline unsigned long __ffs(unsigned long word) +{ + return __builtin_kvx_ctzd(word); +} + +/** + * __fls - find last set bit in word + * @word: The word to search + * + * Undefined if no set bit exists, so code should check against 0 first. + */ +static inline unsigned long __fls(unsigned long word) +{ + return 63 - __builtin_kvx_clzd(word); +} + + +/** + * ffs - find first set bit in word + * @x: the word to search + * + * This is defined the same way as the libc and compiler builtin ffs + * routines, therefore differs in spirit from the other bitops. + * + * ffs(value) returns 0 if value is 0 or the position of the first + * set bit if value is nonzero. The first (least significant) bit + * is at position 1. + */ +static inline int ffs(int x) +{ + if (!x) + return 0; + return __builtin_kvx_ctzw(x) + 1; +} + +static inline unsigned int __arch_hweight32(unsigned int w) +{ + unsigned int count; + + asm volatile ("cbsw %0 = %1\n\t;;" + : "=r" (count) + : "r" (w)); + + return count; +} + +static inline unsigned int __arch_hweight64(__u64 w) +{ + unsigned int count; + + asm volatile ("cbsd %0 = %1\n\t;;" + : "=r" (count) + : "r" (w)); + + return count; +} + +static inline unsigned int __arch_hweight16(unsigned int w) +{ + return __arch_hweight32(w & 0xffff); +} + +static inline unsigned int __arch_hweight8(unsigned int w) +{ + return __arch_hweight32(w & 0xff); +} + +#include + +#include +#include + +#include +#include +#include +#include +#include + +#endif + +#endif diff --git a/arch/kvx/include/asm/bitrev.h b/arch/kvx/include/asm/bitrev.h new file mode 100644 index 000000000000..79865081905a --- /dev/null +++ b/arch/kvx/include/asm/bitrev.h @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_BITREV_H +#define _ASM_KVX_BITREV_H + +#include + +/* Bit reversal constant for matrix multiply */ +#define BIT_REVERSE 0x0102040810204080ULL + +static __always_inline __attribute_const__ u32 __arch_bitrev32(u32 x) +{ + /* Reverse all bits for each bytes and then byte-reverse the 32 LSB */ + return swab32(__builtin_kvx_sbmm8(BIT_REVERSE, x)); +} + +static __always_inline __attribute_const__ u16 __arch_bitrev16(u16 x) +{ + /* Reverse all bits for each bytes and then byte-reverse the 16 LSB */ + return swab16(__builtin_kvx_sbmm8(BIT_REVERSE, x)); +} + +static __always_inline __attribute_const__ u8 __arch_bitrev8(u8 x) +{ + return __builtin_kvx_sbmm8(BIT_REVERSE, x); +} + +#endif diff --git a/arch/kvx/include/asm/cmpxchg.h b/arch/kvx/include/asm/cmpxchg.h new file mode 100644 index 000000000000..51ccb83757cc --- /dev/null +++ b/arch/kvx/include/asm/cmpxchg.h @@ -0,0 +1,170 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Yann Sionneau + * Jules Maselbas + */ + +#ifndef _ASM_KVX_CMPXCHG_H +#define _ASM_KVX_CMPXCHG_H + +#include +#include +#include +#include + +/* + * On kvx, we have a boolean compare and swap which means that the operation + * returns only the success of operation. + * If operation succeed, this is simple, we just need to return the provided + * old value. However, if it fails, we need to load the value to return it for + * the caller. If the loaded value is different from the "old" provided by the + * caller, we can return it since it will means it failed. + * However, if for some reason the value we read is equal to the old value + * provided by the caller, we can't simply return it or the caller will think it + * succeeded. So if the value we read is the same as the "old" provided by + * the caller, we try again until either we succeed or we fail with a different + * value than the provided one. + */ + +static inline unsigned int __cmpxchg_u32(unsigned int old, unsigned int new, + volatile unsigned int *ptr) +{ + unsigned int exp = old; + + __builtin_kvx_fence(); + while (exp == old) { + if (__builtin_kvx_acswapw((void *)ptr, new, exp)) + break; /* acswap succeed */ + exp = *ptr; + } + + return exp; +} + +static inline unsigned long __cmpxchg_u64(unsigned long old, unsigned long new, + volatile unsigned long *ptr) +{ + unsigned long exp = old; + + __builtin_kvx_fence(); + while (exp == old) { + if (__builtin_kvx_acswapd((void *)ptr, new, exp)) + break; /* acswap succeed */ + exp = *ptr; + } + + return exp; +} + +extern unsigned long __cmpxchg_called_with_bad_pointer(void) + __compiletime_error("Bad argument size for cmpxchg"); + +static __always_inline unsigned long __cmpxchg(unsigned long old, + unsigned long new, + volatile void *ptr, int size) +{ + switch (size) { + case 4: + return __cmpxchg_u32(old, new, ptr); + case 8: + return __cmpxchg_u64(old, new, ptr); + default: + return __cmpxchg_called_with_bad_pointer(); + } +} + +#define arch_cmpxchg(ptr, old, new) \ + ((__typeof__(*(ptr))) __cmpxchg( \ + (unsigned long)(old), (unsigned long)(new), \ + (ptr), sizeof(*(ptr)))) + +/* + * In order to optimize xchg for 16 byte, we can use insf/extfs if we know the + * bounds. This way, we only take one more bundle than standard xchg. + * We simply do a read modify acswap on a 32 bit word. + */ + +#define __kvx_insf(org, val, start, stop) __asm__ __volatile__( \ + "insf %[_org] = %[_val], %[_stop], %[_start]\n\t;;" \ + : [_org]"+r"(org) \ + : [_val]"r"(val), [_stop]"i"(stop), [_start]"i"(start)) + +#define __kvx_extfz(out, val, start, stop) __asm__ __volatile__( \ + "extfz %[_out] = %[_val], %[_stop], %[_start]\n\t;;" \ + : [_out]"=r"(out) \ + : [_val]"r"(val), [_stop]"i"(stop), [_start]"i"(start)) + +/* Needed for generic qspinlock implementation */ +static inline unsigned int __xchg_u16(unsigned int old, unsigned int new, + volatile unsigned int *ptr) +{ + unsigned int off = ((unsigned long)ptr) % sizeof(unsigned int); + unsigned int val; + + ptr = PTR_ALIGN_DOWN(ptr, sizeof(unsigned int)); + __builtin_kvx_fence(); + do { + old = *ptr; + val = old; + if (off == 0) + __kvx_insf(val, new, 0, 15); + else + __kvx_insf(val, new, 16, 31); + } while (!__builtin_kvx_acswapw((void *)ptr, val, old)); + + if (off == 0) + __kvx_extfz(old, old, 0, 15); + else + __kvx_extfz(old, old, 16, 31); + + return old; +} + +static inline unsigned int __xchg_u32(unsigned int old, unsigned int new, + volatile unsigned int *ptr) +{ + __builtin_kvx_fence(); + do + old = *ptr; + while (!__builtin_kvx_acswapw((void *)ptr, new, old)); + + return old; +} + +static inline unsigned long __xchg_u64(unsigned long old, unsigned long new, + volatile unsigned long *ptr) +{ + __builtin_kvx_fence(); + do + old = *ptr; + while (!__builtin_kvx_acswapd((void *)ptr, new, old)); + + return old; +} + +extern unsigned long __xchg_called_with_bad_pointer(void) + __compiletime_error("Bad argument size for xchg"); + +static __always_inline unsigned long __xchg(unsigned long val, + volatile void *ptr, int size) +{ + switch (size) { + case 2: + return __xchg_u16(0, val, ptr); + case 4: + return __xchg_u32(0, val, ptr); + case 8: + return __xchg_u64(0, val, ptr); + default: + return __xchg_called_with_bad_pointer(); + } +} + +#define arch_xchg(ptr, val) \ + ((__typeof__(*(ptr))) __xchg( \ + (unsigned long)(val), \ + (ptr), sizeof(*(ptr)))) + +#endif From patchwork Fri Jan 20 14:09:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13109914 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96E51C38141 for ; Fri, 20 Jan 2023 14:12:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231178AbjATOM3 (ORCPT ); Fri, 20 Jan 2023 09:12:29 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37942 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230472AbjATOLq (ORCPT ); Fri, 20 Jan 2023 09:11:46 -0500 Received: from fx403.security-mail.net (smtpout140.security-mail.net [85.31.212.143]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 89ED1C79ED for ; Fri, 20 Jan 2023 06:11:00 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by fx403.security-mail.net (Postfix) with ESMTP id 2C01F4650F5 for ; Fri, 20 Jan 2023 15:10:32 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674223832; bh=tjQ30Z9ergbd5QQmn2LPAl2FE9a2esEgGsVsX8Hkh0Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=a1RUVvPhJFSFfSrK+MJedITiIbeTOz6d2WXE+CgVg0vhyCZ330cex/JmdzdC9kSw/ zzl01cp2UY3crt0esWNgJaSFs/qidpM4TxmvACe/rDqYNV460xpO8sZGt8tOlyfQSE DH7MWRkN0pAddwy84zLXwKR6tDw8lLfyknEyxiAQ= Received: from fx403 (localhost [127.0.0.1]) by fx403.security-mail.net (Postfix) with ESMTP id D93FE46596D; Fri, 20 Jan 2023 15:10:31 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx403.security-mail.net (Postfix) with ESMTPS id AE5D8465448; Fri, 20 Jan 2023 15:10:30 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 4CC1A27E0440; Fri, 20 Jan 2023 15:10:30 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 2CBC627E043A; Fri, 20 Jan 2023 15:10:30 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Vglx1afFJmpx; Fri, 20 Jan 2023 15:10:30 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id B43BA27E043D; Fri, 20 Jan 2023 15:10:29 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 2CBC627E043A DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223830; bh=7ihbYuWX1g/NOZf5apPInAkKPctju2YordzLcMMvd+8=; h=From:To:Date:Message-Id:MIME-Version; b=YUYYebIRIJ5bk8n/CH4YvMdtNpTm9QVYSikQs07+iRmxxq3rubrDIth9BkffMcReC s/DZFrr3fFNMHE0WuwO9jyrorHXjMRuprOkFWk9k7sCTRKtwBpoZVgMyehGNcXP0g1 2ygHrIFYBeDLTMm7y6M6ROAh2tifNdoeEu7Qcwv4= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 12/31] kvx: Add other common headers Date: Fri, 20 Jan 2023 15:09:43 +0100 Message-ID: <20230120141002.2442-13-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Add some other common headers for basic kvx support. Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Vincent Chardon Signed-off-by: Vincent Chardon Co-developed-by: Yann Sionneau Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: no changes arch/kvx/include/asm/asm-prototypes.h | 14 ++++++++ arch/kvx/include/asm/clocksource.h | 17 +++++++++ arch/kvx/include/asm/linkage.h | 13 +++++++ arch/kvx/include/asm/pci.h | 36 +++++++++++++++++++ arch/kvx/include/asm/sections.h | 18 ++++++++++ arch/kvx/include/asm/spinlock.h | 16 +++++++++ arch/kvx/include/asm/spinlock_types.h | 17 +++++++++ arch/kvx/include/asm/stackprotector.h | 47 +++++++++++++++++++++++++ arch/kvx/include/asm/timex.h | 20 +++++++++++ arch/kvx/include/asm/types.h | 12 +++++++ arch/kvx/include/uapi/asm/bitsperlong.h | 14 ++++++++ arch/kvx/include/uapi/asm/byteorder.h | 12 +++++++ tools/include/uapi/asm/bitsperlong.h | 2 ++ 13 files changed, 238 insertions(+) create mode 100644 arch/kvx/include/asm/asm-prototypes.h create mode 100644 arch/kvx/include/asm/clocksource.h create mode 100644 arch/kvx/include/asm/linkage.h create mode 100644 arch/kvx/include/asm/pci.h create mode 100644 arch/kvx/include/asm/sections.h create mode 100644 arch/kvx/include/asm/spinlock.h create mode 100644 arch/kvx/include/asm/spinlock_types.h create mode 100644 arch/kvx/include/asm/stackprotector.h create mode 100644 arch/kvx/include/asm/timex.h create mode 100644 arch/kvx/include/asm/types.h create mode 100644 arch/kvx/include/uapi/asm/bitsperlong.h create mode 100644 arch/kvx/include/uapi/asm/byteorder.h diff --git a/arch/kvx/include/asm/asm-prototypes.h b/arch/kvx/include/asm/asm-prototypes.h new file mode 100644 index 000000000000..af032508e30c --- /dev/null +++ b/arch/kvx/include/asm/asm-prototypes.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_ASM_PROTOTYPES_H +#define _ASM_KVX_ASM_PROTOTYPES_H + +#include + +#include + +#endif /* _ASM_KVX_ASM_PROTOTYPES_H */ diff --git a/arch/kvx/include/asm/clocksource.h b/arch/kvx/include/asm/clocksource.h new file mode 100644 index 000000000000..4df7c66ffbb5 --- /dev/null +++ b/arch/kvx/include/asm/clocksource.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Yann Sionneau + * Clement Leger + */ + +#ifndef _ASM_KVX_CLOCKSOURCE_H +#define _ASM_KVX_CLOCKSOURCE_H + +#include + +struct arch_clocksource_data { + void __iomem *regs; +}; + +#endif /* _ASM_KVX_CLOCKSOURCE_H */ diff --git a/arch/kvx/include/asm/linkage.h b/arch/kvx/include/asm/linkage.h new file mode 100644 index 000000000000..84e1cacf67c2 --- /dev/null +++ b/arch/kvx/include/asm/linkage.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Yann Sionneau + */ + +#ifndef __ASM_KVX_LINKAGE_H +#define __ASM_KVX_LINKAGE_H + +#define __ALIGN .align 4 +#define __ALIGN_STR ".align 4" + +#endif diff --git a/arch/kvx/include/asm/pci.h b/arch/kvx/include/asm/pci.h new file mode 100644 index 000000000000..d5bbaaf041b5 --- /dev/null +++ b/arch/kvx/include/asm/pci.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Vincent Chardon + * Clement Leger + */ + +#ifndef __ASM_KVX_PCI_H_ +#define __ASM_KVX_PCI_H_ + +#include +#include +#include + +#define ARCH_GENERIC_PCI_MMAP_RESOURCE 1 +#define HAVE_PCI_MMAP 1 + +extern int isa_dma_bridge_buggy; + +/* Can be used to override the logic in pci_scan_bus for skipping + * already-configured bus numbers - to be used for buggy BIOSes + * or architectures with incomplete PCI setup by the loader. + */ +#define pcibios_assign_all_busses() 0 + +#define PCIBIOS_MIN_IO 0UL +#define PCIBIOS_MIN_MEM 0UL + +#ifdef CONFIG_PCI_DOMAINS +static inline int pci_proc_domain(struct pci_bus *bus) +{ + return pci_domain_nr(bus); +} +#endif /* CONFIG_PCI_DOMAINS */ + +#endif /* _ASM_KVX_PCI_H */ diff --git a/arch/kvx/include/asm/sections.h b/arch/kvx/include/asm/sections.h new file mode 100644 index 000000000000..0777675ef264 --- /dev/null +++ b/arch/kvx/include/asm/sections.h @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_SECTIONS_H +#define _ASM_KVX_SECTIONS_H + +#include + +extern char __rodata_start[], __rodata_end[]; +extern char __initdata_start[], __initdata_end[]; +extern char __inittext_start[], __inittext_end[]; +extern char __exception_start[], __exception_end[]; +extern char __rm_firmware_regs_start[]; + +#endif diff --git a/arch/kvx/include/asm/spinlock.h b/arch/kvx/include/asm/spinlock.h new file mode 100644 index 000000000000..ed32fdba1e19 --- /dev/null +++ b/arch/kvx/include/asm/spinlock.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_SPINLOCK_H +#define _ASM_KVX_SPINLOCK_H + +#include +#include + +/* See include/linux/spinlock.h */ +#define smp_mb__after_spinlock() smp_mb() + +#endif diff --git a/arch/kvx/include/asm/spinlock_types.h b/arch/kvx/include/asm/spinlock_types.h new file mode 100644 index 000000000000..929a7df16ef3 --- /dev/null +++ b/arch/kvx/include/asm/spinlock_types.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_SPINLOCK_TYPES_H +#define _ASM_KVX_SPINLOCK_TYPES_H + +#if !defined(__LINUX_SPINLOCK_TYPES_RAW_H) && !defined(__ASM_SPINLOCK_H) +# error "please don't include this file directly" +#endif + +#include +#include + +#endif /* _ASM_KVX_SPINLOCK_TYPES_H */ diff --git a/arch/kvx/include/asm/stackprotector.h b/arch/kvx/include/asm/stackprotector.h new file mode 100644 index 000000000000..2c190bbb5efc --- /dev/null +++ b/arch/kvx/include/asm/stackprotector.h @@ -0,0 +1,47 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * derived from arch/mips/include/asm/stackprotector.h + * + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +/* + * GCC stack protector support. + * + * Stack protector works by putting predefined pattern at the start of + * the stack frame and verifying that it hasn't been overwritten when + * returning from the function. The pattern is called stack canary + * and gcc expects it to be defined by a global variable called + * "__stack_chk_guard" on KVX. This unfortunately means that on SMP + * we cannot have a different canary value per task. + */ + +#ifndef __ASM_STACKPROTECTOR_H +#define __ASM_STACKPROTECTOR_H + +#include +#include + +extern unsigned long __stack_chk_guard; + +/* + * Initialize the stackprotector canary value. + * + * NOTE: this must only be called from functions that never return, + * and it must always be inlined. + */ +static __always_inline void boot_init_stack_canary(void) +{ + unsigned long canary; + + /* Try to get a semi random initial value. */ + get_random_bytes(&canary, sizeof(canary)); + canary ^= LINUX_VERSION_CODE; + canary &= CANARY_MASK; + + current->stack_canary = canary; + __stack_chk_guard = current->stack_canary; +} + +#endif /* _ASM_STACKPROTECTOR_H */ diff --git a/arch/kvx/include/asm/timex.h b/arch/kvx/include/asm/timex.h new file mode 100644 index 000000000000..51e346faa887 --- /dev/null +++ b/arch/kvx/include/asm/timex.h @@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_TIMEX_H +#define _ASM_KVX_TIMEX_H + +#define get_cycles get_cycles + +#include +#include + +static inline cycles_t get_cycles(void) +{ + return kvx_sfr_get(PM0); +} + +#endif /* _ASM_KVX_TIMEX_H */ diff --git a/arch/kvx/include/asm/types.h b/arch/kvx/include/asm/types.h new file mode 100644 index 000000000000..1e6c024ee892 --- /dev/null +++ b/arch/kvx/include/asm/types.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_TYPES_H +#define _ASM_KVX_TYPES_H + +#include + +#endif /* _ASM_KVX_TYPES_H */ diff --git a/arch/kvx/include/uapi/asm/bitsperlong.h b/arch/kvx/include/uapi/asm/bitsperlong.h new file mode 100644 index 000000000000..02a91596d567 --- /dev/null +++ b/arch/kvx/include/uapi/asm/bitsperlong.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _UAPI_ASM_KVX_BITSPERLONG_H +#define _UAPI_ASM_KVX_BITSPERLONG_H + +#define __BITS_PER_LONG 64 + +#include + +#endif /* _UAPI_ASM_KVX_BITSPERLONG_H */ diff --git a/arch/kvx/include/uapi/asm/byteorder.h b/arch/kvx/include/uapi/asm/byteorder.h new file mode 100644 index 000000000000..b7d827daec73 --- /dev/null +++ b/arch/kvx/include/uapi/asm/byteorder.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_BYTEORDER_H +#define _ASM_KVX_BYTEORDER_H + +#include + +#endif /* _ASM_KVX_BYTEORDER_H */ diff --git a/tools/include/uapi/asm/bitsperlong.h b/tools/include/uapi/asm/bitsperlong.h index da5206517158..40272ffa9c32 100644 --- a/tools/include/uapi/asm/bitsperlong.h +++ b/tools/include/uapi/asm/bitsperlong.h @@ -19,6 +19,8 @@ #include "../../../arch/alpha/include/uapi/asm/bitsperlong.h" #elif defined(__loongarch__) #include "../../../arch/loongarch/include/uapi/asm/bitsperlong.h" +#elif defined(__kvx__) +#include "../../../arch/kvx/include/uapi/asm/bitsperlong.h" #else #include #endif From patchwork Fri Jan 20 14:09:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13109909 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88E2EC5320B for ; Fri, 20 Jan 2023 14:11:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230455AbjATOLp (ORCPT ); Fri, 20 Jan 2023 09:11:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37762 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230367AbjATOL0 (ORCPT ); Fri, 20 Jan 2023 09:11:26 -0500 Received: from fx405.security-mail.net (smtpout140.security-mail.net [85.31.212.145]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 78A9EC4EAA for ; Fri, 20 Jan 2023 06:10:38 -0800 (PST) Received: from localhost (fx405.security-mail.net [127.0.0.1]) by fx405.security-mail.net (Postfix) with ESMTP id 6D7F4335E3A for ; Fri, 20 Jan 2023 15:10:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674223833; bh=DkGjeJobKP1K/Idr8J6s3hrFSj+9CDHdOSWYtT+PQEc=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=EA6X8RQHXq/W1BC7yIhoLzC7INk51IG9nlnP7Ce+JsEMdrlL9d36Qt3IwyX2AUHRf 5nryYtNQP5GNypPJ0M89jMaF5OciOC4jsnHeAxWmw+M5quvMmaYpFOlz60qD+uA9lD 6Kq2q1QB272m/Cl5XNhNFhycMmtfzXziIvXUdCK8= Received: from fx405 (fx405.security-mail.net [127.0.0.1]) by fx405.security-mail.net (Postfix) with ESMTP id 703C5335C69; Fri, 20 Jan 2023 15:10:32 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx405.security-mail.net (Postfix) with ESMTPS id 0886E335CD8; Fri, 20 Jan 2023 15:10:31 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id BF2A327E043E; Fri, 20 Jan 2023 15:10:30 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id A0E8B27E043D; Fri, 20 Jan 2023 15:10:30 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id u2XWgG7WnkmB; Fri, 20 Jan 2023 15:10:30 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 20C9827E0439; Fri, 20 Jan 2023 15:10:30 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <12bde.63caa0d7.5004.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu A0E8B27E043D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223830; bh=IOewO9S6DyRnSEgyDrfMjSJtY+brX2hbFJUBrk4tQOs=; h=From:To:Date:Message-Id:MIME-Version; b=CMFBrBJsjLdbhmW5V7Q8LXfzWHjdqw/BBxCI1U8gKNKG3ZfPzHIQcvopNQVNkZ8MM Qe4QoaxpZ/t67pd61igfUkSK3LKI1kxUOPvzZfLdy3KLO+QC3Wx4SS80rnNl59Dc43 2JzfEGzoLynfwNusHMBy6K9yMGsBsth1wy4XyWeI= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 13/31] kvx: Add boot and setup routines Date: Fri, 20 Jan 2023 15:09:44 +0100 Message-ID: <20230120141002.2442-14-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Add basic boot, setup and reset routines for kvx. Co-developed-by: Alex Michon Signed-off-by: Alex Michon Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Guillaume Missonnier Signed-off-by: Guillaume Missonnier Co-developed-by: Guillaume Thouvenin Signed-off-by: Guillaume Thouvenin Co-developed-by: Jules Maselbas Signed-off-by: Jules Maselbas Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Julien Hascoet Signed-off-by: Julien Hascoet Co-developed-by: Julien Villette Signed-off-by: Julien Villette Co-developed-by: Marc Poulhiès Signed-off-by: Marc Poulhiès Co-developed-by: Luc Michel Signed-off-by: Luc Michel Co-developed-by: Marius Gligor Signed-off-by: Marius Gligor Co-developed-by: Yann Sionneau Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: removed L2 cache firmware starting code arch/kvx/include/asm/setup.h | 29 ++ arch/kvx/kernel/common.c | 11 + arch/kvx/kernel/head.S | 568 +++++++++++++++++++++++++++++++++++ arch/kvx/kernel/prom.c | 24 ++ arch/kvx/kernel/reset.c | 37 +++ arch/kvx/kernel/setup.c | 177 +++++++++++ arch/kvx/kernel/time.c | 242 +++++++++++++++ 7 files changed, 1088 insertions(+) create mode 100644 arch/kvx/include/asm/setup.h create mode 100644 arch/kvx/kernel/common.c create mode 100644 arch/kvx/kernel/head.S create mode 100644 arch/kvx/kernel/prom.c create mode 100644 arch/kvx/kernel/reset.c create mode 100644 arch/kvx/kernel/setup.c create mode 100644 arch/kvx/kernel/time.c diff --git a/arch/kvx/include/asm/setup.h b/arch/kvx/include/asm/setup.h new file mode 100644 index 000000000000..9c27d5981442 --- /dev/null +++ b/arch/kvx/include/asm/setup.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_SETUP_H +#define _ASM_KVX_SETUP_H + +#include + +#include + +/* Magic is found in r0 when some parameters are given to kernel */ +#define LINUX_BOOT_PARAM_MAGIC ULL(0x31564752414E494C) + +#ifndef __ASSEMBLY__ + +void early_fixmap_init(void); + +void setup_device_tree(void); + +void setup_arch_memory(void); + +void kvx_init_mmu(void); + +#endif + +#endif /* _ASM_KVX_SETUP_H */ diff --git a/arch/kvx/kernel/common.c b/arch/kvx/kernel/common.c new file mode 100644 index 000000000000..322498f034fd --- /dev/null +++ b/arch/kvx/kernel/common.c @@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include +#include + +DEFINE_PER_CPU(int, __preempt_count) = INIT_PREEMPT_COUNT; +EXPORT_PER_CPU_SYMBOL(__preempt_count); diff --git a/arch/kvx/kernel/head.S b/arch/kvx/kernel/head.S new file mode 100644 index 000000000000..6badd2f6a2a6 --- /dev/null +++ b/arch/kvx/kernel/head.S @@ -0,0 +1,568 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + * Marius Gligor + * Julian Vetter + * Julien Hascoet + * Yann Sionneau + * Marc Poulhiès + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#ifdef CONFIG_SMP +#define SECONDARY_START_ADDR smp_secondary_start +#else +#define SECONDARY_START_ADDR proc_power_off +#endif + +#define PS_VAL_WFXL(__field, __val) \ + SFR_SET_VAL_WFXL(PS, __field, __val) + +#define PS_WFXL_VALUE PS_VAL_WFXL(HLE, 1) | \ + PS_VAL_WFXL(USE, 1) | \ + PS_VAL_WFXL(DCE, 1) | \ + PS_VAL_WFXL(ICE, 1) | \ + PS_VAL_WFXL(MME, 1) | \ + PS_VAL_WFXL(MMUP, 1) | \ + PS_VAL_WFXL(ET, 0) | \ + PS_VAL_WFXL(HTD, 0) | \ + PS_VAL_WFXL(PMJ, KVX_SUPPORTED_PSIZE) + +#define PCR_VAL_WFXM(__field, __val) \ + SFR_SET_VAL_WFXM(PCR, __field, __val) + +#define PCR_WFXM_VALUE PCR_VAL_WFXM(L1CE, 1) + +/* 30 sec for primary watchdog timeout */ +#define PRIMARY_WATCHDOG_VALUE (30000000000UL) + +#define TCR_WFXL_VALUE SFR_SET_VAL_WFXL(TCR, WUI, 1) | \ + SFR_SET_VAL_WFXL(TCR, WCE, 1) + +/* Enable STOP in WS */ +#define WS_ENABLE_WU2 (KVX_SFR_WS_WU2_MASK) +/* We only want to clear bits in ws */ +#define WS_WFXL_VALUE (WS_ENABLE_WU2) + +/* SMP stuff */ +#define RM_PID_MASK ((KVX_RM_ID) << KVX_SFR_PCR_PID_SHIFT) + +#define PWR_CTRL_ADDR 0xA40000 + +#define PWR_CTRL_GLOBAL_CONFIG_VALUE \ + (1 << KVX_PWR_CTRL_GLOBAL_SET_PE_EN_SHIFT) + +/* Clean error and selected buffer */ +#define MMC_CLEAR_ERROR (KVX_SFR_MMC_E_MASK) + +#define TEH_VIRTUAL_MEMORY \ + TLB_MK_TEH_ENTRY(PAGE_OFFSET, 0, TLB_G_GLOBAL, 0) + +#define TEL_VIRTUAL_MEMORY \ + TLB_MK_TEL_ENTRY(PHYS_OFFSET, TLB_PS_512M, TLB_ES_A_MODIFIED,\ + TLB_CP_W_C, TLB_PA_NA_RWX) + +/* (TEH|TEL)_SHARED_MEMORY are mapping 0x0 to 0x0 */ +#define TEH_SHARED_MEMORY \ + TLB_MK_TEH_ENTRY(0, 0, TLB_G_GLOBAL, 0) + +#define TEL_SHARED_MEMORY \ + TLB_MK_TEL_ENTRY(0, TLB_PS_2M, TLB_ES_A_MODIFIED,\ + TLB_CP_W_C, TLB_PA_NA_RWX) + +#define TEH_GDB_PAGE_MEMORY \ + TLB_MK_TEH_ENTRY(0, 0, TLB_G_GLOBAL, 0) + +#define TEL_GDB_PAGE_MEMORY \ + TLB_MK_TEL_ENTRY(0, TLB_PS_4K, TLB_ES_A_MODIFIED,\ + TLB_CP_U_U, TLB_PA_RWX_RWX) + +/** + * Macros + */ +.altmacro + +/* To select the JTLB we clear SB from MMC */ +.macro select_jtlb scratch_reg + make \scratch_reg, KVX_SFR_MMC_SB_MASK + ;; + wfxl $mmc, \scratch_reg +.endm + +/* To select the LTLB we set SB from MMC */ +.macro select_ltlb scratch_reg + make \scratch_reg, KVX_SFR_MMC_SB_MASK << 32 + ;; + wfxl $mmc, \scratch_reg +.endm + +/* Set SW of the MMC with number found in the reg register */ +.macro select_way_from_register reg scratch1 scratch2 + slld \scratch1 = \reg, KVX_SFR_MMC_SW_SHIFT + make \scratch2 = KVX_SFR_MMC_SW_MASK + ;; + slld \scratch1 = \scratch1, 32 + ;; + ord \scratch1 = \scratch1, \scratch2 + ;; + wfxl $mmc = \scratch1 +.endm + +/* Set SW of the MMC with the immediate */ +.macro select_way_from_immediate imm scratch1 scratch2 + make \scratch1 = (\imm << KVX_SFR_MMC_SW_SHIFT) << 32 + make \scratch2 = KVX_SFR_MMC_SW_MASK + ;; + ord \scratch1 = \scratch1, \scratch2 + ;; + wfxl $mmc = \scratch1 +.endm + +/* write tlb after setting teh and tel registers */ +.macro write_tlb_entry teh tel + set $teh = \teh + ;; + set $tel = \tel + ;; + tlbwrite +.endm + +/* Boot args */ +#define BOOT_ARGS_COUNT 2 +.align 16 +.section .boot.data, "aw", @progbits +rm_boot_args: +.skip BOOT_ARGS_COUNT * 8 + +/* + * This is our entry point. When entering from bootloader, + * the following registers are set: + * $r0 is a magic (LINUX_BOOT_PARAM_MAGIC) + * $r1 device tree pointer + * + * WARNING WARNING WARNING + * ! DO NOT CLOBBER THEM ! + * WARNING WARNING WARNING + * + * Try to use register above $r20 to ease parameter adding in future + */ + +__HEAD + +.align 8 +.section .boot.startup, "ax", @progbits + +ENTRY(kvx_start) + /* Setup 64 bit really early to avoid bugs */ + make $r21 = PS_VAL_WFXL(V64, 1) + ;; + wfxl $ps = $r21 + ;; + call asm_init_pl + ;; + get $r20 = $pcr + ;; + andd $r21 = $r20, RM_PID_MASK + ;; + cb.dnez $r21 ? asm_rm_cfg_pwr_ctrl + ;; +init_core: + /** + * Setup watchdog early to catch potential + * crash before watchdog driver probe + */ + make $r25 = PRIMARY_WATCHDOG_VALUE + make $r26 = TCR_WFXL_VALUE + ;; + set $wdv = $r25 + ;; + wfxl $tcr, $r26 + ;; + call asm_init_mmu + ;; + /* Setup default processor status */ + make $r25 = PS_WFXL_VALUE + make $r26 = PCR_WFXM_VALUE + ;; + /** + * There is nothing much we can do if we take a early trap since the + * kernel is not yet ready to handle them. + * Register this as the early exception handler to at least avoid + * going in a black hole. + */ + make $r27 = __early_exception_start + ;; + set $ev = $r27 + ;; + wfxm $pcr = $r26 + ;; + wfxl $ps = $r25 + ;; + /* Use as break point for debugging purpose. + See Documentation/kvx/kvx.txt for more details. */ +gdb_mmu_enabled: + /* Extract processor identifier */ + get $r24 = $pcr + ;; + extfz $r24 = $r24, KVX_SFR_END(PCR_PID), KVX_SFR_START(PCR_PID) + ;; + /* If proc 0, then go to clear bss and do normal boot */ + cb.deqz $r24? clear_bss + make $r25 = SECONDARY_START_ADDR + ;; + icall $r25 + ;; +clear_bss: + /* Copy bootloader arguments before cloberring them */ + copyd $r20 = $r0 + copyd $r21 = $r1 + ;; + /* Clear BSS */ + make $r0 = __bss_start + make $r1 = __bss_stop + call asm_memzero + ;; + /* Setup stack */ + make $r40 = init_thread_union + make $r41 = init_task + ;; + set $sr = $r41 + copyd $r0 = $r20 + copyd $r1 = $r21 + ;; + addd $sp = $r40, THREAD_SIZE + /* Clear frame pointer */ + make $fp = 0x0 + /* Setup the exception handler */ + make $r27 = __exception_start + ;; + set $ev = $r27 + /* Here we go ! start the C stuff */ + make $r20 = arch_low_level_start + ;; + icall $r20 + ;; + make $r20 = proc_power_off + ;; + igoto $r20 + ;; +ENDPROC(kvx_start) + +/** + * When PE 0 is started from the RM, arguments from the bootloaders are copied + * into rm_boot_args. It allows to give parameters from RM to PE. + * Note that the 4K alignment is required by the reset pc register... + */ +.align (4 * 1024) +ENTRY(pe_start_wrapper) + make $r0 = rm_boot_args + make $r27 = PWR_CTRL_ADDR + make $r28 = kvx_start + ;; + lq $r0r1 = 0[$r0] + ;; + /* Set reset PC back to original value for SMP start */ + sd KVX_PWR_CTRL_RESET_PC_OFFSET[$r27] = $r28 + ;; + fence + goto kvx_start + ;; +ENDPROC(pe_start_wrapper) + +/** + * asm_memzero - Clear a memory zone with zeroes + * $r0 is the start of memory zone (must be align on 32 bytes boundary) + * $r1 is the end of memory zone (must be align on 32 bytes boundary) + */ +ENTRY(asm_memzero) + sbfd $r32 = $r0, $r1 + make $r36 = 0 + make $r37 = 0 + ;; + make $r38 = 0 + make $r39 = 0 + /* Divide by 32 for hardware loop */ + srld $r32, $r32, 5 + ;; + /* Clear memory with hardware loop */ + loopdo $r32, clear_mem_done + ;; + so 0[$r0] = $r36r37r38r39 + addd $r0 = $r0, 32 + ;; + clear_mem_done: + ret + ;; +ENDPROC(asm_memzero) + +/** + * Configure the power controller to be accessible by PEs + */ +ENTRY(asm_rm_cfg_pwr_ctrl) + /* Enable hwloop for memzero */ + make $r32 = PS_VAL_WFXL(HLE, 1) + ;; + wfxl $ps = $r32 + ;; + make $r26 = PWR_CTRL_ADDR + make $r27 = PWR_CTRL_GLOBAL_CONFIG_VALUE + ;; + /* Set PE enable in power controller */ + sd PWR_CTRL_GLOBAL_CONFIG_OFFSET[$r26] = $r27 + make $r28 = rm_boot_args + ;; + /* Store parameters for PE0 */ + sq 0[$r28] = $r0r1 + make $r29 = pe_start_wrapper + ;; + /* Set PE reset PC to arguments wrapper */ + sd KVX_PWR_CTRL_RESET_PC_OFFSET[$r26] = $r29 + ;; + /* Fence to make sure parameters will be visible by PE 0 */ + fence + ;; + /* Start PE 0 (1 << cpu) */ + make $r27 = 1 + make $r26 = PWR_CTRL_ADDR + ;; + /* Wake up PE0 */ + sd PWR_CTRL_WUP_SET_OFFSET[$r26] = $r27 + ;; + /* And clear wakeup to allow PE0 to sleep */ + sd PWR_CTRL_WUP_CLEAR_OFFSET[$r26] = $r27 + ;; + make $r20 = proc_power_off + ;; + igoto $r20 + ;; +ENDPROC(asm_rm_cfg_pwr_ctrl) + +#define request_ownership(__pl) ;\ + make $r21 = SYO_WFXL_VALUE_##__pl ;\ + ;; ;\ + wfxl $syow = $r21 ;\ + ;; ;\ + make $r21 = HTO_WFXL_VALUE_##__pl ;\ + ;; ;\ + wfxl $htow = $r21 ;\ + ;; ;\ + make $r21 = MO_WFXL_VALUE_##__pl ;\ + make $r22 = MO_WFXM_VALUE_##__pl ;\ + ;; ;\ + wfxl $mow = $r21 ;\ + ;; ;\ + wfxm $mow = $r22 ;\ + ;; ;\ + make $r21 = ITO_WFXL_VALUE_##__pl ;\ + make $r22 = ITO_WFXM_VALUE_##__pl ;\ + ;; ;\ + wfxl $itow = $r21 ;\ + ;; ;\ + wfxm $itow = $r22 ;\ + ;; ;\ + make $r21 = PSO_WFXL_VALUE_##__pl ;\ + make $r22 = PSO_WFXM_VALUE_##__pl ;\ + ;; ;\ + wfxl $psow = $r21 ;\ + ;; ;\ + wfxm $psow = $r22 ;\ + ;; ;\ + make $r21 = DO_WFXL_VALUE_##__pl ;\ + ;; ;\ + wfxl $dow = $r21 ;\ + ;; + +/** + * Initialize privilege level for Kernel + */ +ENTRY(asm_init_pl) + get $r21 = $ps + ;; + /* Extract privilege level from $ps to check if we need to + * lower our privilege level + */ + extfz $r20 = $r21, KVX_SFR_END(PS_PL), KVX_SFR_START(PS_PL) + ;; + /* If our privilege level is 0, then we need to lower in execution level + * to ring 1 in order to let the debug routines be inserted at runtime + * by the JTAG. In both case, we will request the resources we need for + * linux to run. + */ + cb.deqz $r20? delegate_pl + ;; + /* + * When someone is already above us, request the resources we need to + * run the kernel. No need to request double exception or ECC traps for + * instance. When doing so, the more privileged level will trap for + * permission and delegate us the required resources. + */ + request_ownership(PL_CUR) + ;; + ret + ;; +delegate_pl: + request_ownership(PL_CUR_PLUS_1) + ;; + /* Copy our $ps into $sps for 1:1 restoration */ + get $r22 = $ps + ;; + /* We will return to $ra after rfe */ + get $r21 = $ra + /* Set privilege level to +1 is $sps */ + addd $r22 = $r22, PL_CUR_PLUS_1 + ;; + set $spc = $r21 + ;; + set $sps = $r22 + ;; + rfe + ;; +ENDPROC(asm_init_pl) + +/** + * Reset and initialize minimal tlb entries + */ +ENTRY(asm_init_mmu) + make $r20 = MMC_CLEAR_ERROR + ;; + wfxl $mmc = $r20 + ;; + /* Reset the JTLB */ + select_jtlb $r20 + ;; + make $r20 = (MMU_JTLB_SETS - 1) /* Used to select the set */ + make $r21 = 0 /* Used for shifting and as scratch register */ + ;; + set $tel = $r21 /* tel is always equal to 0 */ + ;; + clear_jtlb: + slld $r21 = $r20, KVX_SFR_TEH_PN_SHIFT + addd $r20 = $r20, -1 + ;; + set $teh = $r21 + ;; + make $r22 = (MMU_JTLB_WAYS - 1) /* Used to select the way */ + ;; + loop_jtlb_way: + select_way_from_register $r22 $r23 $r24 + ;; + tlbwrite + ;; + addd $r22 = $r22, -1 + ;; + cb.dgez $r22? loop_jtlb_way + ;; + /* loop_jtlb_way done */ + cb.dgez $r20? clear_jtlb + ;; + clear_jtlb_done: + /* Reset the LTLB */ + select_ltlb $r20 + ;; + clear_ltlb: + /* There is only one set that is 0 so we can reuse the same + values for TEH and TEL. */ + make $r20 = (MMU_LTLB_WAYS - 1) + ;; + loop_ltlb_way: + select_way_from_register $r20, $r21, $r22 + ;; + tlbwrite + ;; + addd $r20 = $r20, -1 + ;; + cb.dgez $r20? loop_ltlb_way + ;; + clear_ltlb_done: + + /* See Documentation/kvx/kvx.txt for details about the settings of + the LTLB */ + select_way_from_immediate LTLB_ENTRY_KERNEL_TEXT, $r20, $r21 + ;; + make $r20 = TEH_VIRTUAL_MEMORY + make $r21 = TEL_VIRTUAL_MEMORY + ;; + write_tlb_entry $r20, $r21 + ;; + select_way_from_immediate LTLB_ENTRY_EARLY_SMEM, $r20, $r21 + ;; + make $r20 = TEH_SHARED_MEMORY + make $r21 = TEL_SHARED_MEMORY + ;; + write_tlb_entry $r20, $r21 + ;; + select_way_from_immediate LTLB_ENTRY_GDB_PAGE, $r20, $r21 + ;; + make $r20 = _debug_start_lma + make $r21 = _debug_start + ;; + andd $r20 = $r20, KVX_SFR_TEH_PN_MASK + andd $r21 = $r21, KVX_SFR_TEL_FN_MASK + ;; + addd $r20 = $r20, TEH_GDB_PAGE_MEMORY + addd $r21 = $r21, TEL_GDB_PAGE_MEMORY + ;; + write_tlb_entry $r20, $r21 + ;; + ret + ;; +ENDPROC(asm_init_mmu) + +/** + * Entry point for secondary processors + * $r24 has been set in caller and is the proc id + */ +ENTRY(smp_secondary_start) +#ifdef CONFIG_SMP + dinval + ;; + iinval + ;; + barrier + ;; + make $r25 = __cpu_up_task_pointer + make $r26 = __cpu_up_stack_pointer + ;; + ld.xs $sp = $r24[$r26] + /* Clear frame pointer */ + make $fp = 0x0 + ;; + ld.xs $r25 = $r24[$r25] + ;; + set $sr = $r25 + make $r27 = __exception_start + ;; + set $ev = $r27 + make $r26 = start_kernel_secondary + ;; + icall $r26 + ;; +#endif +ENDPROC(smp_secondary_start) + +ENTRY(proc_power_off) + make $r1 = WS_WFXL_VALUE + ;; + /* Enable STOP */ + wfxl $ws, $r1 + ;; +1: stop + ;; + goto 1b + ;; +ENDPROC(proc_power_off) diff --git a/arch/kvx/kernel/prom.c b/arch/kvx/kernel/prom.c new file mode 100644 index 000000000000..a5241aa66903 --- /dev/null +++ b/arch/kvx/kernel/prom.c @@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include +#include +#include +#include + +void __init setup_device_tree(void) +{ + const char *name; + + name = of_flat_dt_get_machine_name(); + if (!name) + return; + + pr_info("Machine model: %s\n", name); + dump_stack_set_arch_desc("%s (DT)", name); + + unflatten_device_tree(); +} diff --git a/arch/kvx/kernel/reset.c b/arch/kvx/kernel/reset.c new file mode 100644 index 000000000000..afa0ceb9d7e9 --- /dev/null +++ b/arch/kvx/kernel/reset.c @@ -0,0 +1,37 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include +#include + +#include + +static void kvx_default_power_off(void) +{ + smp_send_stop(); + local_cpu_stop(); +} + +void (*pm_power_off)(void) = kvx_default_power_off; +EXPORT_SYMBOL(pm_power_off); + +void machine_restart(char *cmd) +{ + smp_send_stop(); + do_kernel_restart(cmd); + pr_err("Reboot failed -- System halted\n"); + local_cpu_stop(); +} + +void machine_halt(void) +{ + pm_power_off(); +} + +void machine_power_off(void) +{ + pm_power_off(); +} diff --git a/arch/kvx/kernel/setup.c b/arch/kvx/kernel/setup.c new file mode 100644 index 000000000000..e155341a37fd --- /dev/null +++ b/arch/kvx/kernel/setup.c @@ -0,0 +1,177 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +struct screen_info screen_info; + +unsigned long memory_start; +EXPORT_SYMBOL(memory_start); +unsigned long memory_end; +EXPORT_SYMBOL(memory_end); + +DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_kvx, cpu_info); +EXPORT_PER_CPU_SYMBOL(cpu_info); + +static bool use_streaming = true; +static int __init parse_kvx_streaming(char *arg) +{ + strtobool(arg, &use_streaming); + + if (!use_streaming) { + pr_info("disabling streaming\n"); + kvx_sfr_set_field(PS, USE, 0); + } + + return 0; +} +early_param("kvx.streaming", parse_kvx_streaming); + +static void __init setup_user_privilege(void) +{ + /* + * We want to let the user control various fields of ps: + * - hardware loop + * - instruction cache enable + * - streaming enable + */ + uint64_t mask = KVX_SFR_PSOW_HLE_MASK | + KVX_SFR_PSOW_ICE_MASK | + KVX_SFR_PSOW_USE_MASK; + + uint64_t value = (1 << KVX_SFR_PSOW_HLE_SHIFT) | + (1 << KVX_SFR_PSOW_ICE_SHIFT) | + (1 << KVX_SFR_PSOW_USE_SHIFT); + + kvx_sfr_set_mask(PSOW, mask, value); +} + +void __init setup_cpuinfo(void) +{ + struct cpuinfo_kvx *n = this_cpu_ptr(&cpu_info); + u64 pcr = kvx_sfr_get(PCR); + + n->copro_enable = kvx_sfr_field_val(pcr, PCR, COE); + n->arch_rev = kvx_sfr_field_val(pcr, PCR, CAR); + n->uarch_rev = kvx_sfr_field_val(pcr, PCR, CMA); +} + +/* + * Everything that needs to be setup PER cpu should be put here. + * This function will be called by per-cpu setup routine. + */ +void __init setup_processor(void) +{ + /* Clear performance monitor 0 */ + kvx_sfr_set_field(PMC, PM0C, 0); + +#ifdef CONFIG_ENABLE_TCA + /* Enable TCA (COE = Coprocessor Enable) */ + kvx_sfr_set_field(PCR, COE, 1); +#else + kvx_sfr_set_field(PCR, COE, 0); +#endif + + /* + * On kvx, we have speculative accesses which differ from normal + * accesses by the fact their trapping policy is directed by mmc.sne + * (speculative no-mapping enable) and mmc.spe (speculative protection + * enabled). + * To handle these accesses properly, we disable all traps on + * speculative accesses while in kernel and user (sne & spe) + * in order to silently discard data if fetched. + * This allows to do an effective prefetch. + */ + kvx_sfr_set_field(MMC, SNE, 0); + kvx_sfr_set_field(MMC, SPE, 0); + + if (!use_streaming) + kvx_sfr_set_field(PS, USE, 0); + + kvx_init_core_irq(); + + setup_user_privilege(); + + setup_cpuinfo(); +} + +static char builtin_cmdline[COMMAND_LINE_SIZE] __initdata = CONFIG_CMDLINE; + +void __init setup_arch(char **cmdline_p) +{ + if (builtin_cmdline[0]) { + /* append boot loader cmdline to builtin */ + strlcat(builtin_cmdline, " ", COMMAND_LINE_SIZE); + strlcat(builtin_cmdline, boot_command_line, COMMAND_LINE_SIZE); + strscpy(boot_command_line, builtin_cmdline, COMMAND_LINE_SIZE); + } + + *cmdline_p = boot_command_line; + + setup_processor(); + + /* Jump labels needs fixmap to be setup for text modifications */ + early_fixmap_init(); + + /* Parameters might set static keys */ + jump_label_init(); + /* + * Parse early param after setting up arch memory since + * we need fixmap for earlycon and fixedmap need to do + * memory allocation (fixed_range_init). + */ + parse_early_param(); + + setup_arch_memory(); + + paging_init(); + + setup_device_tree(); + + smp_init_cpus(); + +#ifdef CONFIG_VT + conswitchp = &dummy_con; +#endif +} + +asmlinkage __visible void __init arch_low_level_start(unsigned long r0, + void *dtb_ptr) +{ + void *dt = __dtb_start; + + kvx_mmu_early_setup(); + + if (r0 == LINUX_BOOT_PARAM_MAGIC) + dt = __va(dtb_ptr); + + if (!early_init_dt_scan(dt)) + panic("Missing device tree\n"); + + start_kernel(); +} diff --git a/arch/kvx/kernel/time.c b/arch/kvx/kernel/time.c new file mode 100644 index 000000000000..f27103a1a6f4 --- /dev/null +++ b/arch/kvx/kernel/time.c @@ -0,0 +1,242 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Yann Sionneau + * Guillaume Thouvenin + * Luc Michel + * Julian Vetter + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#define KVX_TIMER_MIN_DELTA 1 +#define KVX_TIMER_MAX_DELTA 0xFFFFFFFFFFFFFFFFULL +#define KVX_TIMER_MAX_VALUE 0xFFFFFFFFFFFFFFFFULL + +/* + * Clockevent + */ +static unsigned int kvx_timer_frequency; +static unsigned int kvx_periodic_timer_value; +static unsigned int kvx_timer_irq; + +static void kvx_timer_set_value(unsigned long value, unsigned long reload_value) +{ + kvx_sfr_set(T0R, reload_value); + kvx_sfr_set(T0V, value); + /* Enable timer */ + kvx_sfr_set_field(TCR, T0CE, 1); +} + +static int kvx_clkevent_set_next_event(unsigned long cycles, + struct clock_event_device *dev) +{ + /* + * Hardware does not support oneshot mode. + * In order to support it, set a really high reload value. + * Then, during the interrupt handler, disable the timer if + * in oneshot mode + */ + kvx_timer_set_value(cycles - 1, KVX_TIMER_MAX_VALUE); + + return 0; +} + +/* + * Configure the rtc to periodically tick HZ times per second + */ +static int kvx_clkevent_set_state_periodic(struct clock_event_device *dev) +{ + kvx_timer_set_value(kvx_periodic_timer_value, + kvx_periodic_timer_value); + + return 0; +} + +static int kvx_clkevent_set_state_oneshot(struct clock_event_device *dev) +{ + /* Same as for kvx_clkevent_set_next_event */ + kvx_clkevent_set_next_event(kvx_periodic_timer_value, dev); + + return 0; +} + +static int kvx_clkevent_set_state_shutdown(struct clock_event_device *dev) +{ + kvx_sfr_set_field(TCR, T0CE, 0); + + return 0; +} + +static DEFINE_PER_CPU(struct clock_event_device, kvx_clockevent_device) = { + .name = "kvx-timer-0", + .features = CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_PERIODIC, + /* arbitrary rating for this clockevent */ + .rating = 300, + .set_next_event = kvx_clkevent_set_next_event, + .set_state_periodic = kvx_clkevent_set_state_periodic, + .set_state_oneshot = kvx_clkevent_set_state_oneshot, + .set_state_shutdown = kvx_clkevent_set_state_shutdown, +}; + +irqreturn_t kvx_timer_irq_handler(int irq, void *dev_id) +{ + struct clock_event_device *evt = this_cpu_ptr(&kvx_clockevent_device); + + /* Disable timer if in oneshot mode before reloading */ + if (likely(clockevent_state_oneshot(evt))) + kvx_sfr_set_field(TCR, T0CE, 0); + + evt->event_handler(evt); + + return IRQ_HANDLED; +} + +static int kvx_timer_starting_cpu(unsigned int cpu) +{ + struct clock_event_device *evt = this_cpu_ptr(&kvx_clockevent_device); + + evt->cpumask = cpumask_of(cpu); + evt->irq = kvx_timer_irq; + + clockevents_config_and_register(evt, kvx_timer_frequency, + KVX_TIMER_MIN_DELTA, + KVX_TIMER_MAX_DELTA); + + /* Enable timer interrupt */ + kvx_sfr_set_field(TCR, T0IE, 1); + + enable_percpu_irq(kvx_timer_irq, IRQ_TYPE_NONE); + + return 0; +} + +static int kvx_timer_dying_cpu(unsigned int cpu) +{ + disable_percpu_irq(kvx_timer_irq); + + return 0; +} + +static int __init kvx_setup_core_timer(struct device_node *np) +{ + struct clock_event_device *evt = this_cpu_ptr(&kvx_clockevent_device); + struct clk *clk; + int err; + + clk = of_clk_get(np, 0); + if (IS_ERR(clk)) { + pr_err("kvx_core_timer: Failed to get CPU clock: %ld\n", + PTR_ERR(clk)); + return 1; + } + + kvx_timer_frequency = clk_get_rate(clk); + clk_put(clk); + kvx_periodic_timer_value = kvx_timer_frequency / HZ; + + kvx_timer_irq = irq_of_parse_and_map(np, 0); + if (!kvx_timer_irq) { + pr_err("kvx_core_timer: Failed to parse irq: %d\n", + kvx_timer_irq); + return -EINVAL; + } + + err = request_percpu_irq(kvx_timer_irq, kvx_timer_irq_handler, + "kvx_core_timer", evt); + if (err) { + pr_err("kvx_core_timer: can't register interrupt %d (%d)\n", + kvx_timer_irq, err); + return err; + } + + err = cpuhp_setup_state(CPUHP_AP_KVX_TIMER_STARTING, + "kvx/time:online", + kvx_timer_starting_cpu, + kvx_timer_dying_cpu); + if (err < 0) { + pr_err("kvx_core_timer: Failed to setup hotplug state"); + return err; + } + + return 0; +} + +TIMER_OF_DECLARE(kvx_core_timer, "kalray,kvx-core-timer", + kvx_setup_core_timer); + +/* + * Clocksource + */ +static u64 kvx_dsu_clocksource_read(struct clocksource *cs) +{ + return readq(cs->archdata.regs); +} + +static struct clocksource kvx_dsu_clocksource = { + .name = "kvx-dsu-clock", + .rating = 400, + .read = kvx_dsu_clocksource_read, + .mask = CLOCKSOURCE_MASK(64), + .flags = CLOCK_SOURCE_IS_CONTINUOUS, +}; + +static u64 notrace kvx_dsu_sched_read(void) +{ + return readq_relaxed(kvx_dsu_clocksource.archdata.regs); +} + +static int __init kvx_setup_dsu_clock(struct device_node *np) +{ + int ret; + struct clk *clk; + unsigned long kvx_dsu_frequency; + + kvx_dsu_clocksource.archdata.regs = of_iomap(np, 0); + + WARN_ON(!kvx_dsu_clocksource.archdata.regs); + if (!kvx_dsu_clocksource.archdata.regs) + return -ENXIO; + + clk = of_clk_get(np, 0); + if (IS_ERR(clk)) { + pr_err("Failed to get CPU clock: %ld\n", PTR_ERR(clk)); + return PTR_ERR(clk); + } + + kvx_dsu_frequency = clk_get_rate(clk); + clk_put(clk); + + ret = clocksource_register_hz(&kvx_dsu_clocksource, + kvx_dsu_frequency); + if (ret) { + pr_err("failed to register dsu clocksource"); + return ret; + } + + sched_clock_register(kvx_dsu_sched_read, 64, kvx_dsu_frequency); + return 0; +} + +TIMER_OF_DECLARE(kvx_dsu_clock, "kalray,kvx-dsu-clock", + kvx_setup_dsu_clock); + +void __init time_init(void) +{ + of_clk_init(NULL); + + timer_probe(); +} From patchwork Fri Jan 20 14:09:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13109908 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D8E9C38A23 for ; Fri, 20 Jan 2023 14:11:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230423AbjATOLp (ORCPT ); Fri, 20 Jan 2023 09:11:45 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37968 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230355AbjATOLN (ORCPT ); Fri, 20 Jan 2023 09:11:13 -0500 Received: from fx601.security-mail.net (smtpout140.security-mail.net [85.31.212.146]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8ACAAC63AC for ; Fri, 20 Jan 2023 06:10:35 -0800 (PST) Received: from localhost (fx601.security-mail.net [127.0.0.1]) by fx601.security-mail.net (Postfix) with ESMTP id 144113495F3 for ; Fri, 20 Jan 2023 15:10:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674223833; bh=vEPzE4UPjrP9RfuxZKlBQyPh+DKkjOR0sWjN6rJGm/I=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=j+nepYnkPB3bEmDAI3/P4GgKvamOIR+RPMUGVlhutO01+sL6OP2EGqPM/SeyCpU81 L4VPcpAJJu3/To42lPlwc7GKLXsqGJPuXp5GIiLTmSWCEkSXxnx18jlkPER8pkUw02 /YQfwGtuLPfsinT1GeNUf3YC0GD7AEoPNJ+aJArI= Received: from fx601 (fx601.security-mail.net [127.0.0.1]) by fx601.security-mail.net (Postfix) with ESMTP id BAA51349492; Fri, 20 Jan 2023 15:10:32 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx601.security-mail.net (Postfix) with ESMTPS id C5E7D3493C7; Fri, 20 Jan 2023 15:10:31 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 362F127E0437; Fri, 20 Jan 2023 15:10:31 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 193EE27E043D; Fri, 20 Jan 2023 15:10:31 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id jvSzDkK8KBKa; Fri, 20 Jan 2023 15:10:31 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 984F727E043A; Fri, 20 Jan 2023 15:10:30 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <1837e.63caa0d7.737ff.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 193EE27E043D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223831; bh=DmzWtNbu9q/4tdr0s78bdEDa3viNBQUf7Z5t+96KI4s=; h=From:To:Date:Message-Id:MIME-Version; b=fu0N7x6nnMzNoJzaoFf7v+P0ip4/vd3WETP3kcNxOV8lUW4FUEgIHhlGEaapCaaWz trFOuxwdKHhTQMjErede9J063YUB4llfkDR1AlG/RnHltlp+dT86cwzYEgc4+IcIMU vO/Z00O1Gtxz1AmibpXGVXxbhTaqAPWZbIWrCzPw= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 14/31] kvx: Add exception/interrupt handling Date: Fri, 20 Jan 2023 15:09:45 +0100 Message-ID: <20230120141002.2442-15-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Add the exception and interrupt handling machanism for basic kvx support. Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Guillaume Thouvenin Signed-off-by: Guillaume Thouvenin Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Luc Michel Signed-off-by: Luc Michel Co-developed-by: Marius Gligor Signed-off-by: Marius Gligor Co-developed-by: Yann Sionneau Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: - removed ipi.h headers and driver (moved into ipi driver patch) arch/kvx/include/asm/break_hook.h | 69 +++++++++ arch/kvx/include/asm/bug.h | 67 ++++++++ arch/kvx/include/asm/dame.h | 31 ++++ arch/kvx/include/asm/hardirq.h | 14 ++ arch/kvx/include/asm/hw_irq.h | 14 ++ arch/kvx/include/asm/irqflags.h | 58 +++++++ arch/kvx/include/asm/stacktrace.h | 44 ++++++ arch/kvx/include/asm/traps.h | 76 ++++++++++ arch/kvx/kernel/dame_handler.c | 113 ++++++++++++++ arch/kvx/kernel/irq.c | 78 ++++++++++ arch/kvx/kernel/traps.c | 243 ++++++++++++++++++++++++++++++ 11 files changed, 807 insertions(+) create mode 100644 arch/kvx/include/asm/break_hook.h create mode 100644 arch/kvx/include/asm/bug.h create mode 100644 arch/kvx/include/asm/dame.h create mode 100644 arch/kvx/include/asm/hardirq.h create mode 100644 arch/kvx/include/asm/hw_irq.h create mode 100644 arch/kvx/include/asm/irqflags.h create mode 100644 arch/kvx/include/asm/stacktrace.h create mode 100644 arch/kvx/include/asm/traps.h create mode 100644 arch/kvx/kernel/dame_handler.c create mode 100644 arch/kvx/kernel/irq.c create mode 100644 arch/kvx/kernel/traps.c diff --git a/arch/kvx/include/asm/break_hook.h b/arch/kvx/include/asm/break_hook.h new file mode 100644 index 000000000000..333b2c440c81 --- /dev/null +++ b/arch/kvx/include/asm/break_hook.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef __ASM_KVX_BREAK_HOOK_H_ +#define __ASM_KVX_BREAK_HOOK_H_ + +#include + +#include +#include + +/* + * The following macros define the different causes of break: + * We use the `set $vsfr0 = $rXX` instruction which will raise a trap into the + * debugger. The trapping instruction is read and decoded to extract the source + * register number. The source register number is used to differentiate the + * trap cause. + */ +#define BREAK_CAUSE_BUG KVX_REG_R1 +#define BREAK_CAUSE_KGDB_DYN KVX_REG_R2 +#define BREAK_CAUSE_KGDB_COMP KVX_REG_R3 +#define BREAK_CAUSE_BKPT KVX_REG_R63 + +/** + * enum break_ret - Break return value + * @BREAK_HOOK_HANDLED: Hook handled successfully + * @BREAK_HOOK_ERROR: Hook was not handled + */ +enum break_ret { + BREAK_HOOK_HANDLED = 0, + BREAK_HOOK_ERROR = 1, +}; + +/* + * The following macro assembles a `set` instruction targeting $vsfr0 + * using the source register whose number is __id. + */ +#define KVX_BREAK_INSN(__id) \ + KVX_INSN_SET_SYLLABLE_0(KVX_INSN_PARALLEL_EOB, KVX_SFR_VSFR0, __id) + +#define KVX_BREAK_INSN_SIZE (KVX_INSN_SET_SIZE * KVX_INSN_SYLLABLE_WIDTH) + +struct pt_regs; + +/** + * struct break_hook - Break hook description + * @node: List node + * @handler: handler called when break matches this hook + * @imm: Immediate value expected for break insn + * @mode: Hook mode (user/kernel) + */ +struct break_hook { + struct list_head node; + int (*handler)(struct break_hook *brk_hook, struct pt_regs *regs); + u8 id; + u8 mode; +}; + +void kvx_skip_break_insn(struct pt_regs *regs); + +void break_hook_register(struct break_hook *brk_hook); +void break_hook_unregister(struct break_hook *brk_hook); + +int break_hook_handler(u64 es, struct pt_regs *regs); + +#endif /* __ASM_KVX_BREAK_HOOK_H_ */ diff --git a/arch/kvx/include/asm/bug.h b/arch/kvx/include/asm/bug.h new file mode 100644 index 000000000000..62f556b00d5a --- /dev/null +++ b/arch/kvx/include/asm/bug.h @@ -0,0 +1,67 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_BUG_H +#define _ASM_KVX_BUG_H + +#include +#include +#include + +#include + +#ifdef CONFIG_GENERIC_BUG + +#define BUG_INSN KVX_BREAK_INSN(BREAK_CAUSE_BUG) + +#define __BUG_ENTRY_ADDR ".dword 1b" + +#ifdef CONFIG_DEBUG_BUGVERBOSE +#define __BUG_ENTRY_LAST_MEMBER flags +#define __BUG_ENTRY \ + __BUG_ENTRY_ADDR "\n\t" \ + ".dword %0\n\t" \ + ".short %1\n\t" +#else +#define __BUG_ENTRY_LAST_MEMBER file +#define __BUG_ENTRY \ + __BUG_ENTRY_ADDR "\n\t" +#endif + +#define BUG() \ +do { \ + __asm__ __volatile__ ( \ + "1:\n\t" \ + ".word " __stringify(BUG_INSN) "\n" \ + ".pushsection __bug_table,\"a\"\n\t" \ + "2:\n\t" \ + __BUG_ENTRY \ + ".fill 1, %2, 0\n\t" \ + ".popsection" \ + : \ + : "i" (__FILE__), "i" (__LINE__), \ + "i" (sizeof(struct bug_entry) - \ + offsetof(struct bug_entry, __BUG_ENTRY_LAST_MEMBER))); \ + unreachable(); \ +} while (0) + +#else /* CONFIG_GENERIC_BUG */ +#define BUG() \ +do { \ + __asm__ __volatile__ (".word " __stringify(BUG_INSN) "\n"); \ + unreachable(); \ +} while (0) +#endif /* CONFIG_GENERIC_BUG */ + +#define HAVE_ARCH_BUG + +struct pt_regs; + +void die(struct pt_regs *regs, unsigned long ea, const char *str); + +#include + +#endif /* _ASM_KVX_BUG_H */ diff --git a/arch/kvx/include/asm/dame.h b/arch/kvx/include/asm/dame.h new file mode 100644 index 000000000000..8befd767bbee --- /dev/null +++ b/arch/kvx/include/asm/dame.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_DAME_H +#define _ASM_KVX_DAME_H + +#include +#include + +static inline void dame_irq_check(struct pt_regs *regs) +{ +#ifdef CONFIG_SECURE_DAME_HANDLING + unsigned long ilr; + /* If we are returning to the kernel, no need to check for DAME */ + if (!user_mode(regs)) + return; + + /* Else, make sure we do a barrier to trig any pending DAME IRQ */ + __builtin_kvx_barrier(); + + /* Check if we triggered a DAME */ + ilr = kvx_sfr_get(ILR); + if (ilr & KVX_SFR_ILR_IT16_MASK) + panic("DAME error encountered while in kernel !!!!\n"); +#endif +} + +#endif /* _ASM_KVX_DAME_H */ diff --git a/arch/kvx/include/asm/hardirq.h b/arch/kvx/include/asm/hardirq.h new file mode 100644 index 000000000000..f82630f7e4e2 --- /dev/null +++ b/arch/kvx/include/asm/hardirq.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_HARDIRQ_H +#define _ASM_KVX_HARDIRQ_H + +#define __ARCH_IRQ_EXIT_IRQS_DISABLED 1 + +#include + +#endif /* _ASM_KVX_HARDIRQ_H */ diff --git a/arch/kvx/include/asm/hw_irq.h b/arch/kvx/include/asm/hw_irq.h new file mode 100644 index 000000000000..f073dba3b1c5 --- /dev/null +++ b/arch/kvx/include/asm/hw_irq.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * derived from arch/mips/include/asm/ide.h + * + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_HW_IRQ_H +#define _ASM_KVX_HW_IRQ_H + +void kvx_init_core_irq(void); + +#endif /* _ASM_KVX_HW_IRQ_H */ diff --git a/arch/kvx/include/asm/irqflags.h b/arch/kvx/include/asm/irqflags.h new file mode 100644 index 000000000000..681c890b3fcd --- /dev/null +++ b/arch/kvx/include/asm/irqflags.h @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_IRQFLAGS_H +#define _ASM_KVX_IRQFLAGS_H + +#include + +#include + +static inline notrace unsigned long arch_local_save_flags(void) +{ + return kvx_sfr_get(PS) & (1 << KVX_SFR_PS_IE_SHIFT); +} + +static inline notrace unsigned long arch_local_irq_save(void) +{ + unsigned long flags = arch_local_save_flags(); + + kvx_sfr_set_field(PS, IE, 0); + + return flags; +} + +static inline notrace void arch_local_irq_restore(unsigned long flags) +{ + /* If flags are set, interrupt are enabled), set the IE bit */ + if (flags) + kvx_sfr_set_field(PS, IE, 1); + else + kvx_sfr_set_field(PS, IE, 0); +} + +static inline notrace void arch_local_irq_enable(void) +{ + kvx_sfr_set_field(PS, IE, 1); +} + +static inline notrace void arch_local_irq_disable(void) +{ + kvx_sfr_set_field(PS, IE, 0); +} + +static inline notrace bool arch_irqs_disabled_flags(unsigned long flags) +{ + return (flags & (1 << KVX_SFR_PS_IE_SHIFT)) == 0; +} + +static inline notrace bool arch_irqs_disabled(void) +{ + return arch_irqs_disabled_flags(kvx_sfr_get(PS)); +} + + +#endif /* _ASM_KVX_IRQFLAGS_H */ diff --git a/arch/kvx/include/asm/stacktrace.h b/arch/kvx/include/asm/stacktrace.h new file mode 100644 index 000000000000..0feed6bd4424 --- /dev/null +++ b/arch/kvx/include/asm/stacktrace.h @@ -0,0 +1,44 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_STACKTRACE_H +#define _ASM_KVX_STACKTRACE_H + +#include + +/** + * Structure of a frame on the stack + */ +struct stackframe { + unsigned long fp; /* Next frame pointer */ + unsigned long ra; /* Return address */ +}; + +static inline bool on_task_stack(struct task_struct *tsk, unsigned long sp) +{ + unsigned long low = (unsigned long) task_stack_page(tsk); + unsigned long high = low + THREAD_SIZE; + + if (sp < low || sp >= high) + return false; + + return true; +} + +void show_stacktrace(struct task_struct *task, struct pt_regs *regs); + + +void walk_stackframe(struct task_struct *task, struct stackframe *frame, + bool (*fn)(unsigned long, void *), void *arg); + +static inline void start_stackframe(struct stackframe *frame, + unsigned long fp, + unsigned long pc) +{ + frame->fp = fp; + frame->ra = pc; +} +#endif /* _ASM_KVX_STACKTRACE_H */ diff --git a/arch/kvx/include/asm/traps.h b/arch/kvx/include/asm/traps.h new file mode 100644 index 000000000000..77a663968135 --- /dev/null +++ b/arch/kvx/include/asm/traps.h @@ -0,0 +1,76 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + * Marius Gligor + */ + +#ifndef _ASM_KVX_TRAPS_H +#define _ASM_KVX_TRAPS_H + +#include + +#define KVX_TRAP_RESET 0x0 +#define KVX_TRAP_OPCODE 0x1 +#define KVX_TRAP_PRIVILEGE 0x2 +#define KVX_TRAP_DMISALIGN 0x3 +#define KVX_TRAP_PSYSERROR 0x4 +#define KVX_TRAP_DSYSERROR 0x5 +#define KVX_TRAP_PDECCERROR 0x6 +#define KVX_TRAP_DDECCERROR 0x7 +#define KVX_TRAP_PPARERROR 0x8 +#define KVX_TRAP_DPARERROR 0x9 +#define KVX_TRAP_PSECERROR 0xA +#define KVX_TRAP_DSECERROR 0xB +#define KVX_TRAP_NOMAPPING 0xC +#define KVX_TRAP_PROTECTION 0xD +#define KVX_TRAP_WRITETOCLEAN 0xE +#define KVX_TRAP_ATOMICTOCLEAN 0xF +#define KVX_TRAP_TPAR 0x10 +#define KVX_TRAP_DOUBLE_ECC 0x11 +#define KVX_TRAP_VSFR 0x12 +#define KVX_TRAP_PL_OVERFLOW 0x13 + +#define KVX_TRAP_COUNT 0x14 + +#define KVX_TRAP_SFRI_NOT_BCU 0 +#define KVX_TRAP_SFRI_GET 1 +#define KVX_TRAP_SFRI_IGET 2 +#define KVX_TRAP_SFRI_SET 4 +#define KVX_TRAP_SFRI_WFXL 5 +#define KVX_TRAP_SFRI_WFXM 6 +#define KVX_TRAP_SFRI_RSWAP 7 + +/* Access type on memory trap */ +#define KVX_TRAP_RWX_FETCH 1 +#define KVX_TRAP_RWX_WRITE 2 +#define KVX_TRAP_RWX_READ 4 +#define KVX_TRAP_RWX_ATOMIC 6 + +#ifndef __ASSEMBLY__ + +typedef void (*trap_handler_func) (uint64_t es, uint64_t ea, + struct pt_regs *regs); + +#define trap_cause(__es) kvx_sfr_field_val(__es, ES, HTC) + +#define trap_sfri(__es) \ + kvx_sfr_field_val((__es), ES, SFRI) + +#define trap_gprp(__es) \ + kvx_sfr_field_val((__es), ES, GPRP) + +#define trap_sfrp(__es) \ + kvx_sfr_field_val((__es), ES, SFRP) + +#ifdef CONFIG_MMU +extern void do_page_fault(uint64_t es, uint64_t ea, struct pt_regs *regs); +extern void do_writetoclean(uint64_t es, uint64_t ea, struct pt_regs *regs); +#endif + +void user_do_sig(struct pt_regs *regs, int signo, int code, unsigned long addr); + +#endif /* __ASSEMBLY__ */ + +#endif diff --git a/arch/kvx/kernel/dame_handler.c b/arch/kvx/kernel/dame_handler.c new file mode 100644 index 000000000000..ce190bee8211 --- /dev/null +++ b/arch/kvx/kernel/dame_handler.c @@ -0,0 +1,113 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static unsigned int kvx_dame_irq; + +static const char *error_str[KVX_SFR_ES_ITI_WIDTH] = { + "PSE", + "PILSY", + "PILDE", + "PILPA", + "DSE", + "DILSY", + "DILDE", + "DILPA", + "DDEE", + "DSYE" +}; + +static irqreturn_t dame_irq_handler(int irq, void *dev_id) +{ + int bit; + struct pt_regs *regs = get_irq_regs(); + unsigned long error_status = kvx_sfr_field_val(regs->es, ES, ITI); + + if (error_status) { + pr_err("Memory Error:\n"); + for_each_set_bit(bit, &error_status, KVX_SFR_ES_ITI_WIDTH) + pr_err("- %s\n", error_str[bit]); + } + + /* + * If the DAME happened in user mode, we can handle it properly + * by killing the user process. + * Otherwise, if we are in kernel, we are fried... + */ + if (user_mode(regs)) + force_sig_fault(SIGBUS, BUS_ADRERR, (void __user *) NULL); + else + die(regs, 0, "DAME error encountered while in kernel !!!!\n"); + + return IRQ_HANDLED; +} + +static int kvx_dame_starting_cpu(unsigned int cpu) +{ + enable_percpu_irq(kvx_dame_irq, IRQ_TYPE_NONE); + + return 0; +} + +static int kvx_dame_dying_cpu(unsigned int cpu) +{ + disable_percpu_irq(kvx_dame_irq); + + return 0; +} + +static int __init dame_handler_init(void) +{ + struct device_node *dame_node; + int ret; + + dame_node = of_find_compatible_node(NULL, NULL, + "kalray,kvx-dame-handler"); + if (!dame_node) { + pr_err("Failed to find dame handler device tree node\n"); + return -ENODEV; + } + + kvx_dame_irq = irq_of_parse_and_map(dame_node, 0); + of_node_put(dame_node); + + if (!kvx_dame_irq) { + pr_err("Failed to parse dame irq\n"); + return -ENODEV; + } + + ret = request_percpu_irq(kvx_dame_irq, dame_irq_handler, "dame", + &kvx_dame_irq); + if (ret) { + pr_err("Failed to request dame irq\n"); + return -ENODEV; + } + + ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, + "kvx/dame_handler:online", + kvx_dame_starting_cpu, + kvx_dame_dying_cpu); + if (ret <= 0) { + pr_err("Failed to setup cpuhp\n"); + return ret; + } + + pr_info("DAME handler registered\n"); + + return 0; +} + +core_initcall(dame_handler_init); diff --git a/arch/kvx/kernel/irq.c b/arch/kvx/kernel/irq.c new file mode 100644 index 000000000000..5d7c8cfba1dd --- /dev/null +++ b/arch/kvx/kernel/irq.c @@ -0,0 +1,78 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include +#include +#include +#include +#include +#include + +#include + +#define IT_MASK(__it) (KVX_SFR_ILL_ ## __it ## _MASK) +#define IT_LEVEL(__it, __level) \ + (__level##ULL << KVX_SFR_ILL_ ## __it ## _SHIFT) + +void do_IRQ(unsigned long hwirq_mask, struct pt_regs *regs) +{ + struct pt_regs *old_regs = set_irq_regs(regs); + int irq; + unsigned int hwirq; + + trace_hardirqs_off(); + + irq_enter(); + + while (hwirq_mask) { + hwirq = __ffs(hwirq_mask); + irq = irq_find_mapping(NULL, hwirq); + generic_handle_irq(irq); + hwirq_mask &= ~BIT_ULL(hwirq); + } + + irq_exit(); + set_irq_regs(old_regs); + + dame_irq_check(regs); +} + +/* + * Early Hardware specific Interrupt setup + * -Called very early (start_kernel -> setup_arch -> setup_processor) + * -Needed for each CPU + */ +void kvx_init_core_irq(void) +{ + /* + * On KVX, Kernel only care about the following IT: + * - IT0: Timer 0 + * - IT2: Watchdog + * - IT4: APIC IT 1 + * - IT24: IPI + */ + uint64_t mask = IT_MASK(IT0) | IT_MASK(IT2) | IT_MASK(IT4) | + IT_MASK(IT24); + + /* + * Specific priorities for ITs: + * - Watchdog has the highest priority: 3 + * - Timer has priority 2 + * - APIC entries have lowest priority: 1 + */ + uint64_t value = IT_LEVEL(IT0, 0x2) | IT_LEVEL(IT2, 0x3) | + IT_LEVEL(IT4, 0x1) | IT_LEVEL(IT24, 0x1); + + kvx_sfr_set_mask(ILL, mask, value); + + /* Set core level to 0 */ + kvx_sfr_set_field(PS, IL, 0); +} + +void __init init_IRQ(void) +{ + irqchip_init(); +} diff --git a/arch/kvx/kernel/traps.c b/arch/kvx/kernel/traps.c new file mode 100644 index 000000000000..3a1706b666c2 --- /dev/null +++ b/arch/kvx/kernel/traps.c @@ -0,0 +1,243 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + * Marius Gligor + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +int show_unhandled_signals = 1; + +static DEFINE_SPINLOCK(die_lock); + +static trap_handler_func trap_handler_table[KVX_TRAP_COUNT] = { NULL }; + +/* Trap names associated to the trap numbers */ +static const char * const trap_name[] = { + "RESET", + "OPCODE", + "PRIVILEGE", + "DMISALIGN", + "PSYSERROR", + "DSYSERROR", + "PDECCERROR", + "DDECCERROR", + "PPARERROR", + "DPARERROR", + "PSECERROR", + "DSECERROR", + /* MMU related traps */ + "NOMAPPING", + "PROTECTION", + "WRITETOCLEAN", + "ATOMICTOCLEAN", + "TPAR", + "DOUBLE_ECC", + "VSFR", + "PL_OVERFLOW" +}; + +void die(struct pt_regs *regs, unsigned long ea, const char *str) +{ + static int die_counter; + int ret; + + oops_enter(); + + spin_lock_irq(&die_lock); + console_verbose(); + bust_spinlocks(1); + + pr_emerg("%s [#%d]\n", str, ++die_counter); + print_modules(); + show_regs(regs); + + if (!user_mode(regs)) + show_stacktrace(NULL, regs); + + ret = notify_die(DIE_OOPS, str, regs, ea, 0, SIGSEGV); + + bust_spinlocks(0); + add_taint(TAINT_DIE, LOCKDEP_NOW_UNRELIABLE); + spin_unlock_irq(&die_lock); + oops_exit(); + + if (in_interrupt()) + panic("Fatal exception in interrupt"); + if (panic_on_oops) + panic("Fatal exception"); + if (ret != NOTIFY_STOP) + make_task_dead(SIGSEGV); +} + +void user_do_sig(struct pt_regs *regs, int signo, int code, unsigned long addr) +{ + struct task_struct *tsk = current; + + if (show_unhandled_signals && unhandled_signal(tsk, signo) + && printk_ratelimit()) { + pr_info("%s[%d]: unhandled signal %d code 0x%x at 0x%lx", + tsk->comm, task_pid_nr(tsk), signo, code, addr); + print_vma_addr(KERN_CONT " in ", instruction_pointer(regs)); + pr_cont("\n"); + show_regs(regs); + } + if (signo == SIGKILL) { + force_sig(signo); + return; + } + force_sig_fault(signo, code, (void __user *) addr); +} + +static void panic_or_kill(uint64_t es, uint64_t ea, struct pt_regs *regs, + int signo, int sigcode) +{ + if (user_mode(regs)) { + user_do_sig(regs, signo, sigcode, ea); + return; + } + + pr_alert(CUT_HERE "ERROR: TRAP %s received at 0x%.16llx\n", + trap_name[trap_cause(es)], regs->spc); + die(regs, ea, "Oops"); + make_task_dead(SIGKILL); +} + +int is_valid_bugaddr(unsigned long pc) +{ + /* + * Since the bug was reported, this means that the break hook handling + * already check the faulting instruction so there is no need for + * additionnal check here. This is a BUG for sure. + */ + return 1; +} + +static int bug_break_handler(struct break_hook *brk_hook, struct pt_regs *regs) +{ + enum bug_trap_type type; + + type = report_bug(regs->spc, regs); + switch (type) { + case BUG_TRAP_TYPE_NONE: + return BREAK_HOOK_ERROR; + case BUG_TRAP_TYPE_WARN: + break; + case BUG_TRAP_TYPE_BUG: + die(regs, regs->spc, "Kernel BUG"); + break; + } + + /* Skip over break insn if we survived ! */ + kvx_skip_break_insn(regs); + + return BREAK_HOOK_HANDLED; +} + +static struct break_hook bug_break_hook = { + .handler = bug_break_handler, + .id = BREAK_CAUSE_BUG, + .mode = MODE_KERNEL, +}; + +#define GEN_TRAP_HANDLER(__name, __sig, __code) \ +static void __name ## _trap_handler(uint64_t es, uint64_t ea, \ + struct pt_regs *regs) \ +{ \ + panic_or_kill(es, ea, regs, __sig, __code); \ +} + +GEN_TRAP_HANDLER(default, SIGKILL, SI_KERNEL); +GEN_TRAP_HANDLER(privilege, SIGILL, ILL_PRVREG); +GEN_TRAP_HANDLER(dmisalign, SIGBUS, BUS_ADRALN); +GEN_TRAP_HANDLER(syserror, SIGBUS, BUS_ADRERR); +GEN_TRAP_HANDLER(opcode, SIGILL, ILL_ILLOPC); + +static void register_trap_handler(unsigned int trap_nb, trap_handler_func fn) +{ + + if (trap_nb >= KVX_TRAP_COUNT || fn == NULL) + panic("Failed to register handler #%d\n", trap_nb); + + trap_handler_table[trap_nb] = fn; +} + +static void do_vsfr_fault(uint64_t es, uint64_t ea, struct pt_regs *regs) +{ + if (break_hook_handler(es, regs) == BREAK_HOOK_HANDLED) + return; + + panic_or_kill(es, ea, regs, SIGILL, ILL_PRVREG); +} + +void __init trap_init(void) +{ + int i; + + break_hook_register(&bug_break_hook); + + for (i = 0; i < KVX_TRAP_COUNT; i++) + register_trap_handler(i, default_trap_handler); +#ifdef CONFIG_MMU + register_trap_handler(KVX_TRAP_NOMAPPING, do_page_fault); + register_trap_handler(KVX_TRAP_PROTECTION, do_page_fault); + register_trap_handler(KVX_TRAP_WRITETOCLEAN, do_writetoclean); +#endif + + register_trap_handler(KVX_TRAP_PSYSERROR, syserror_trap_handler); + register_trap_handler(KVX_TRAP_DSYSERROR, syserror_trap_handler); + register_trap_handler(KVX_TRAP_PRIVILEGE, privilege_trap_handler); + register_trap_handler(KVX_TRAP_OPCODE, opcode_trap_handler); + register_trap_handler(KVX_TRAP_DMISALIGN, dmisalign_trap_handler); + register_trap_handler(KVX_TRAP_VSFR, do_vsfr_fault); +} + +/** + * trap_handler - trap handler called by _trap_handler routine in trap_handler.S + * This handler will redirect to other trap handlers if present + * If not then it will do a generic action + * @es: Exception Syndrome register value + * @ea: Exception Address register + * @regs: pointer to registers saved when trapping + */ +void trap_handler(uint64_t es, uint64_t ea, struct pt_regs *regs) +{ + enum ctx_state prev_state = exception_enter(); + int htc = trap_cause(es); + trap_handler_func trap_func = trap_handler_table[htc]; + + trace_hardirqs_off(); + + /* Normal traps number should and must be between 0 and 15 included */ + if (unlikely(htc >= KVX_TRAP_COUNT)) { + pr_err("Invalid trap %d !\n", htc); + goto done; + } + + /* If irqs were enabled in the preempted context, reenable them */ + if (regs->sps & KVX_SFR_PS_IE_MASK) + local_irq_enable(); + + trap_func(es, ea, regs); + +done: + dame_irq_check(regs); + exception_exit(prev_state); +} From patchwork Fri Jan 20 14:09:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13109911 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0ABD5C05027 for ; Fri, 20 Jan 2023 14:12:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230513AbjATOMY (ORCPT ); Fri, 20 Jan 2023 09:12:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38764 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230397AbjATOLo (ORCPT ); Fri, 20 Jan 2023 09:11:44 -0500 Received: from fx405.security-mail.net (smtpout140.security-mail.net [85.31.212.145]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2555FC63B6 for ; Fri, 20 Jan 2023 06:10:49 -0800 (PST) Received: from localhost (fx405.security-mail.net [127.0.0.1]) by fx405.security-mail.net (Postfix) with ESMTP id C673B335E13 for ; Fri, 20 Jan 2023 15:10:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674223833; bh=YRX0Wr/1x7MjAvceUm2udvzUR1TOmF2eg1JprEHHvQ4=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=4xRbut1w4zOpBXyHSq9C5pkhrGB2DjceQ+R2LOysvay+1gezw8L0D1oZzEGuT5eCO fL7HIubM9zERQDoUuV1TXhJGx3gwyRJEBYNLKPk82ghwq9y8tFs6yYB49igdxogouD ZxqEYzVf9YcTiIEp499NxozMFXug+0BI6pCW64rc= Received: from fx405 (fx405.security-mail.net [127.0.0.1]) by fx405.security-mail.net (Postfix) with ESMTP id 53789335E38; Fri, 20 Jan 2023 15:10:33 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx405.security-mail.net (Postfix) with ESMTPS id EB02D335CEB; Fri, 20 Jan 2023 15:10:31 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id A49CE27E0442; Fri, 20 Jan 2023 15:10:31 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 8BB4A27E043D; Fri, 20 Jan 2023 15:10:31 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id U6itgih5Dbpw; Fri, 20 Jan 2023 15:10:31 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 0F29627E0439; Fri, 20 Jan 2023 15:10:31 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <16482.63caa0d7.e71bf.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 8BB4A27E043D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223831; bh=VOCUsPMJmFARQsqOvo9LjS/JXRlQuyQoyz0P2SXEDK0=; h=From:To:Date:Message-Id:MIME-Version; b=PkDAJ2mbH/dRIRz3Ev0E06+CjrkEGJWMzkHLNsc+7sKPjdsXswUAdfFC4Q8QwaCOa te2fWAMAQYkLOSuheX7j5IqaDoEo/DN8AdgKbTQ8vhcaN0lyuPH0UxU4CF3xo/U4JJ ZMNxpuEbibLGCxhjY+mfcXwuzTv0JEeR1KplNBpM= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 15/31] irqchip: Add irq-kvx-apic-gic driver Date: Fri, 20 Jan 2023 15:09:46 +0100 Message-ID: <20230120141002.2442-16-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Each Cluster of the Coolidge SoC includes an Advanced Programmable Interrupt Controller (APIC) and Generic Interrupt Controller (GIC). The APIC GIC acts as an intermediary interrupt controller, muxing/routing incoming interrupts to cores in the cluster. The 139 possible input interrupt lines are organized as follow: - 128 from the mailbox controller (one it per mailboxes) - 1 from the NoC router - 5 from IOMMUs - 1 from L2 cache DMA job FIFO - 1 from cluster watchdog - 2 for SECC, DECC - 1 from Data NoC The 72 possible output interrupt line: - 68 : 4 interrupts per cores (17 cores) - 1 for L2 cache controller - 3 extra that are for padding Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Vincent Chardon Signed-off-by: Vincent Chardon Signed-off-by: Jules Maselbas Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: - removed irq-kvx-itgen driver (moved in its own patch) - removed irq-kvx-apic-mailbox driver (moved in its own patch) - removed irq-kvx-core-intc driver (moved in its own patch) - removed print on probe success drivers/irqchip/Kconfig | 6 + drivers/irqchip/Makefile | 1 + drivers/irqchip/irq-kvx-apic-gic.c | 356 +++++++++++++++++++++++++++++ 3 files changed, 363 insertions(+) create mode 100644 drivers/irqchip/irq-kvx-apic-gic.c diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig index 7ef9f5e696d3..2433e4ba0759 100644 --- a/drivers/irqchip/Kconfig +++ b/drivers/irqchip/Kconfig @@ -334,6 +334,12 @@ config MIPS_GIC select IRQ_DOMAIN_HIERARCHY select MIPS_CM +config KVX_APIC_GIC + bool + depends on KVX + select IRQ_DOMAIN + select IRQ_DOMAIN_HIERARCHY + config INGENIC_IRQ bool depends on MACH_INGENIC diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile index 87b49a10962c..8ac1dd880420 100644 --- a/drivers/irqchip/Makefile +++ b/drivers/irqchip/Makefile @@ -69,6 +69,7 @@ obj-$(CONFIG_BCM7120_L2_IRQ) += irq-bcm7120-l2.o obj-$(CONFIG_BRCMSTB_L2_IRQ) += irq-brcmstb-l2.o obj-$(CONFIG_KEYSTONE_IRQ) += irq-keystone.o obj-$(CONFIG_MIPS_GIC) += irq-mips-gic.o +obj-$(CONFIG_KVX_APIC_GIC) += irq-kvx-apic-gic.o obj-$(CONFIG_ARCH_MEDIATEK) += irq-mtk-sysirq.o irq-mtk-cirq.o obj-$(CONFIG_ARCH_DIGICOLOR) += irq-digicolor.o obj-$(CONFIG_ARCH_SA1100) += irq-sa11x0.o diff --git a/drivers/irqchip/irq-kvx-apic-gic.c b/drivers/irqchip/irq-kvx-apic-gic.c new file mode 100644 index 000000000000..cc234a075473 --- /dev/null +++ b/drivers/irqchip/irq-kvx-apic-gic.c @@ -0,0 +1,356 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2017 - 2022 Kalray Inc. + * Author(s): Clement Leger + * Julian Vetter + */ + +#define pr_fmt(fmt) "kvx_apic_gic: " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* APIC is organized in 18 groups of 4 output lines + * However, the two upper lines are for Secure RM and DMA engine + * Thus, we do not have to use them + */ +#define KVX_GIC_PER_CPU_IT_COUNT 4 +#define KVX_GIC_INPUT_IT_COUNT 0x9D +#define KVX_GIC_OUTPUT_IT_COUNT 0x10 + +/* GIC enable register definitions */ +#define KVX_GIC_ENABLE_OFFSET 0x0 +#define KVX_GIC_ENABLE_ELEM_SIZE 0x1 +#define KVX_GIC_ELEM_SIZE 0x400 + +/* GIC status lac register definitions */ +#define KVX_GIC_STATUS_LAC_OFFSET 0x120 +#define KVX_GIC_STATUS_LAC_ELEM_SIZE 0x8 +#define KVX_GIC_STATUS_LAC_ARRAY_SIZE 0x3 + +/** + * For each CPU, there is 4 output lines coming from the apic GIC. + * We only use 1 line and this structure represent this line. + * @base Output line base address + * @cpu CPU associated to this line + */ +struct gic_out_irq_line { + void __iomem *base; + unsigned int cpu; +}; + +/** + * Input irq line. + * This structure is used to store the status of the input line and the + * associated output line. + * @enabled Boolean for line status + * @cpu CPU currently receiving this interrupt + * @it_num Interrupt number + */ +struct gic_in_irq_line { + bool enabled; + struct gic_out_irq_line *out_line; + unsigned int it_num; +}; + +/** + * struct kvx_apic_gic - kvx apic gic + * @base: Base address of the controller + * @domain Domain for this controller + * @input_nr_irqs: maximum number of supported input interrupts + * @cpus: Per cpu interrupt configuration + * @output_irq: Array of output irq lines + * @input_irq: Array of input irq lines + */ +struct kvx_apic_gic { + raw_spinlock_t lock; + void __iomem *base; + struct irq_domain *domain; + uint32_t input_nr_irqs; + /* For each cpu, there is an output IT line */ + struct gic_out_irq_line output_irq[KVX_GIC_OUTPUT_IT_COUNT]; + /* Input interrupt status */ + struct gic_in_irq_line input_irq[KVX_GIC_INPUT_IT_COUNT]; +}; + +static int gic_parent_irq; + +/** + * Enable/Disable an output irq line + * This function is used by both mask/unmask to disable/enable the line. + */ +static void irq_line_set_enable(struct gic_out_irq_line *irq_line, + struct gic_in_irq_line *in_irq_line, + int enable) +{ + void __iomem *enable_line_addr = irq_line->base + + KVX_GIC_ENABLE_OFFSET + + in_irq_line->it_num * KVX_GIC_ENABLE_ELEM_SIZE; + + writeb((uint8_t) enable ? 1 : 0, enable_line_addr); + in_irq_line->enabled = enable; +} + +static void kvx_apic_gic_set_line(struct irq_data *data, int enable) +{ + struct kvx_apic_gic *gic = irq_data_get_irq_chip_data(data); + unsigned int in_irq = irqd_to_hwirq(data); + struct gic_in_irq_line *in_line = &gic->input_irq[in_irq]; + struct gic_out_irq_line *out_line = in_line->out_line; + + raw_spin_lock(&gic->lock); + /* Set line enable on currently assigned cpu */ + irq_line_set_enable(out_line, in_line, enable); + raw_spin_unlock(&gic->lock); +} + +static void kvx_apic_gic_mask(struct irq_data *data) +{ + kvx_apic_gic_set_line(data, 0); +} + +static void kvx_apic_gic_unmask(struct irq_data *data) +{ + kvx_apic_gic_set_line(data, 1); +} + +#ifdef CONFIG_SMP + +static int kvx_apic_gic_set_affinity(struct irq_data *d, + const struct cpumask *cpumask, + bool force) +{ + struct kvx_apic_gic *gic = irq_data_get_irq_chip_data(d); + unsigned int new_cpu; + unsigned int hw_irq = irqd_to_hwirq(d); + struct gic_in_irq_line *input_line = &gic->input_irq[hw_irq]; + struct gic_out_irq_line *new_out_line; + + /* We assume there is only one cpu in the mask */ + new_cpu = cpumask_first(cpumask); + new_out_line = &gic->output_irq[new_cpu]; + + raw_spin_lock(&gic->lock); + + /* Nothing to do, line is the same */ + if (new_out_line == input_line->out_line) + goto out; + + /* If old line was enabled, enable the new one before disabling + * the old one + */ + if (input_line->enabled) + irq_line_set_enable(new_out_line, input_line, 1); + + /* Disable it on old line */ + irq_line_set_enable(input_line->out_line, input_line, 0); + + /* Assign new output line to input IRQ */ + input_line->out_line = new_out_line; + +out: + raw_spin_unlock(&gic->lock); + + irq_data_update_effective_affinity(d, cpumask_of(new_cpu)); + + return IRQ_SET_MASK_OK; +} +#endif + +static struct irq_chip kvx_apic_gic_chip = { + .name = "kvx apic gic", + .irq_mask = kvx_apic_gic_mask, + .irq_unmask = kvx_apic_gic_unmask, +#ifdef CONFIG_SMP + .irq_set_affinity = kvx_apic_gic_set_affinity, +#endif +}; + +static int kvx_apic_gic_alloc(struct irq_domain *domain, unsigned int virq, + unsigned int nr_irqs, void *args) +{ + int i; + struct irq_fwspec *fwspec = args; + int hwirq = fwspec->param[0]; + + for (i = 0; i < nr_irqs; i++) { + irq_domain_set_info(domain, virq + i, hwirq + i, + &kvx_apic_gic_chip, + domain->host_data, handle_simple_irq, + NULL, NULL); + } + + return 0; +} + +static const struct irq_domain_ops kvx_apic_gic_domain_ops = { + .alloc = kvx_apic_gic_alloc, + .free = irq_domain_free_irqs_common, +}; + +static void irq_line_get_status_lac(struct gic_out_irq_line *out_irq_line, + uint64_t status[KVX_GIC_STATUS_LAC_ARRAY_SIZE]) +{ + int i; + + for (i = 0; i < KVX_GIC_STATUS_LAC_ARRAY_SIZE; i++) { + status[i] = readq(out_irq_line->base + + KVX_GIC_STATUS_LAC_OFFSET + + i * KVX_GIC_STATUS_LAC_ELEM_SIZE); + } +} + +static void kvx_apic_gic_handle_irq(struct irq_desc *desc) +{ + struct kvx_apic_gic *gic_data = irq_desc_get_handler_data(desc); + struct gic_out_irq_line *out_line; + uint64_t status[KVX_GIC_STATUS_LAC_ARRAY_SIZE]; + unsigned long irqn, cascade_irq; + unsigned long cpu = smp_processor_id(); + + out_line = &gic_data->output_irq[cpu]; + + irq_line_get_status_lac(out_line, status); + + for_each_set_bit(irqn, (unsigned long *) status, + KVX_GIC_STATUS_LAC_ARRAY_SIZE * BITS_PER_LONG) { + + cascade_irq = irq_find_mapping(gic_data->domain, irqn); + + generic_handle_irq(cascade_irq); + } +} + +static void __init apic_gic_init(struct kvx_apic_gic *gic) +{ + unsigned int cpu, line; + struct gic_in_irq_line *input_irq_line; + struct gic_out_irq_line *output_irq_line; + uint64_t status[KVX_GIC_STATUS_LAC_ARRAY_SIZE]; + + /* Initialize all input lines (device -> )*/ + for (line = 0; line < KVX_GIC_INPUT_IT_COUNT; line++) { + input_irq_line = &gic->input_irq[line]; + input_irq_line->enabled = false; + /* All input lines map on output 0 */ + input_irq_line->out_line = &gic->output_irq[0]; + input_irq_line->it_num = line; + } + + /* Clear all output lines (-> cpus) */ + for (cpu = 0; cpu < KVX_GIC_OUTPUT_IT_COUNT; cpu++) { + output_irq_line = &gic->output_irq[cpu]; + output_irq_line->cpu = cpu; + output_irq_line->base = gic->base + + cpu * (KVX_GIC_ELEM_SIZE * KVX_GIC_PER_CPU_IT_COUNT); + + /* Disable all external lines on this core */ + for (line = 0; line < KVX_GIC_INPUT_IT_COUNT; line++) + irq_line_set_enable(output_irq_line, + &gic->input_irq[line], 0x0); + + irq_line_get_status_lac(output_irq_line, status); + } +} + +static int kvx_gic_starting_cpu(unsigned int cpu) +{ + enable_percpu_irq(gic_parent_irq, IRQ_TYPE_NONE); + + return 0; +} + +static int kvx_gic_dying_cpu(unsigned int cpu) +{ + disable_percpu_irq(gic_parent_irq); + + return 0; +} + +static int __init kvx_init_apic_gic(struct device_node *node, + struct device_node *parent) +{ + struct kvx_apic_gic *gic; + int ret; + unsigned int irq; + + if (!parent) { + pr_err("kvx apic gic does not have parent\n"); + return -EINVAL; + } + + gic = kzalloc(sizeof(*gic), GFP_KERNEL); + if (!gic) + return -ENOMEM; + + if (of_property_read_u32(node, "kalray,intc-nr-irqs", + &gic->input_nr_irqs)) + gic->input_nr_irqs = KVX_GIC_INPUT_IT_COUNT; + + if (WARN_ON(gic->input_nr_irqs > KVX_GIC_INPUT_IT_COUNT)) { + ret = -EINVAL; + goto err_kfree; + } + + gic->base = of_io_request_and_map(node, 0, node->name); + if (!gic->base) { + ret = -EINVAL; + goto err_kfree; + } + + raw_spin_lock_init(&gic->lock); + apic_gic_init(gic); + + gic->domain = irq_domain_add_linear(node, + gic->input_nr_irqs, + &kvx_apic_gic_domain_ops, + gic); + if (!gic->domain) { + pr_err("Failed to add IRQ domain\n"); + ret = -EINVAL; + goto err_iounmap; + } + + irq = irq_of_parse_and_map(node, 0); + if (irq <= 0) { + pr_err("unable to parse irq\n"); + ret = -EINVAL; + goto err_irq_domain_remove; + } + + irq_set_chained_handler_and_data(irq, kvx_apic_gic_handle_irq, + gic); + + gic_parent_irq = irq; + ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, + "kvx/gic:online", + kvx_gic_starting_cpu, + kvx_gic_dying_cpu); + if (ret < 0) { + pr_err("Failed to setup hotplug state"); + goto err_irq_unmap; + } + + return 0; + +err_irq_unmap: + irq_dispose_mapping(irq); +err_irq_domain_remove: + irq_domain_remove(gic->domain); +err_iounmap: + iounmap(gic->base); +err_kfree: + kfree(gic); + + return ret; +} + +IRQCHIP_DECLARE(kvx_apic_gic, "kalray,kvx-apic-gic", kvx_init_apic_gic); From patchwork Fri Jan 20 14:09:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13109910 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 544FFC54ED7 for ; Fri, 20 Jan 2023 14:11:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230329AbjATOLs (ORCPT ); Fri, 20 Jan 2023 09:11:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37968 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230393AbjATOLn (ORCPT ); Fri, 20 Jan 2023 09:11:43 -0500 Received: from fx601.security-mail.net (smtpout140.security-mail.net [85.31.212.146]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49C8AC639E for ; Fri, 20 Jan 2023 06:10:47 -0800 (PST) Received: from localhost (fx601.security-mail.net [127.0.0.1]) by fx601.security-mail.net (Postfix) with ESMTP id 47281349884 for ; Fri, 20 Jan 2023 15:10:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674223834; bh=gHHA/mkF4t07OYcoLZXtQVGhZaJ1ygPcvebeM7VebW0=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=qTs2OwDX0M0+TjKD2v8EmPSCKQAdXEvlbkZVgdnowHQv6RT97W4rBW9zxnxd0m7TR rNZbTEFzbQfMcrtVFO8MHuLZQE3MpEYNuvtY9wg1tIikA4An7NLsLhJBdzdUrGHPtr ZgZTHLri2IGrg/bEFeBEMKfyIbM1SKt6CNRA2MeI= Received: from fx601 (fx601.security-mail.net [127.0.0.1]) by fx601.security-mail.net (Postfix) with ESMTP id 49E1D349679; Fri, 20 Jan 2023 15:10:33 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx601.security-mail.net (Postfix) with ESMTPS id 6052E349425; Fri, 20 Jan 2023 15:10:32 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 2DDA027E043D; Fri, 20 Jan 2023 15:10:32 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 0E11627E0437; Fri, 20 Jan 2023 15:10:32 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id RaEvsddOB1Lw; Fri, 20 Jan 2023 15:10:31 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 8788727E043E; Fri, 20 Jan 2023 15:10:31 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <1461a.63caa0d8.5c064.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 0E11627E0437 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223832; bh=Mmv6n5fjDahaZPf9nXXTjzf80aTW/zpMRJ3EQiPoUdY=; h=From:To:Date:Message-Id:MIME-Version; b=p6WlefvsosVHgbuzkRjVnAAQf6mQa5rqWTmqWWRlZqKx1F/YzkuMyWptweWcwaDLB 34rlCsW27s/ilBK/ZzmIj+l9DB+Uq85phZDztQ3g2/btIzaiS3qCm1vV3BnXS+GDT4 annv82J/VXktfKBu+c/t3f6opUH247PXj8+5Ksl8= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 16/31] irqchip: Add irq-kvx-itgen driver Date: Fri, 20 Jan 2023 15:09:47 +0100 Message-ID: <20230120141002.2442-17-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC From: Jules Maselbas The Kalray Core Interrupt Controller is tightly integrated in each kv3 core present in the Coolidge SoC. It provides the following features: - 32 independent interrupt sources - 2-bit configurable priority level - 2-bit configurable ownership level Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Vincent Chardon Signed-off-by: Vincent Chardon Signed-off-by: Jules Maselbas Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: new patch - removed header include/linux/irqchip/irq-kvx-apic-gic.h - header moved to drivers/irqchip/ but in another patch - removed print on probe success drivers/irqchip/Kconfig | 8 ++ drivers/irqchip/Makefile | 1 + drivers/irqchip/irq-kvx-itgen.c | 236 ++++++++++++++++++++++++++++++++ 3 files changed, 245 insertions(+) create mode 100644 drivers/irqchip/irq-kvx-itgen.c diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig index 2433e4ba0759..546bc611f3f3 100644 --- a/drivers/irqchip/Kconfig +++ b/drivers/irqchip/Kconfig @@ -340,6 +340,14 @@ config KVX_APIC_GIC select IRQ_DOMAIN select IRQ_DOMAIN_HIERARCHY +config KVX_ITGEN + bool + depends on KVX + select GENERIC_IRQ_IPI if SMP + select GENERIC_MSI_IRQ_DOMAIN + select IRQ_DOMAIN + select IRQ_DOMAIN_HIERARCHY + config INGENIC_IRQ bool depends on MACH_INGENIC diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile index 8ac1dd880420..6b8f459d8a21 100644 --- a/drivers/irqchip/Makefile +++ b/drivers/irqchip/Makefile @@ -70,6 +70,7 @@ obj-$(CONFIG_BRCMSTB_L2_IRQ) += irq-brcmstb-l2.o obj-$(CONFIG_KEYSTONE_IRQ) += irq-keystone.o obj-$(CONFIG_MIPS_GIC) += irq-mips-gic.o obj-$(CONFIG_KVX_APIC_GIC) += irq-kvx-apic-gic.o +obj-$(CONFIG_KVX_ITGEN) += irq-kvx-itgen.o obj-$(CONFIG_ARCH_MEDIATEK) += irq-mtk-sysirq.o irq-mtk-cirq.o obj-$(CONFIG_ARCH_DIGICOLOR) += irq-digicolor.o obj-$(CONFIG_ARCH_SA1100) += irq-sa11x0.o diff --git a/drivers/irqchip/irq-kvx-itgen.c b/drivers/irqchip/irq-kvx-itgen.c new file mode 100644 index 000000000000..f6af023a689e --- /dev/null +++ b/drivers/irqchip/irq-kvx-itgen.c @@ -0,0 +1,236 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Julian Vetter + * Vincent Chardon + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* Parameters */ +#define KVX_ITGEN_PARAM_OFFSET 0x1100 +#define KVX_ITGEN_PARAM_IT_NUM_OFFSET 0x0 + +/* Target configuration */ +#define KVX_ITGEN_CFG_ENABLE_OFFSET 0x8 +#define KVX_ITGEN_CFG_ELEM_SIZE 0x10 +#define KVX_ITGEN_CFG_TARGET_OFFSET 0x0 +#define KVX_ITGEN_CFG_TARGET_MAILBOX_SHIFT 0x0 +#define KVX_ITGEN_CFG_TARGET_MAILBOX_MASK 0x7FUL +#define KVX_ITGEN_CFG_TARGET_CLUSTER_SHIFT 0x8 +#define KVX_ITGEN_CFG_TARGET_CLUSTER_MASK 0x700UL +#define KVX_ITGEN_CFG_TARGET_SELECT_BIT_SHIFT 0x18 +#define KVX_ITGEN_CFG_TARGET_SELECT_BIT_MASK 0x3F000000UL + +#define MB_ADDR_CLUSTER_SHIFT 24 +#define MB_ADDR_MAILBOX_SHIFT 9 + +/** + * struct kvx_itgen - kvx interrupt generator (MSI client) + * @base: base address of the itgen controller + * @domain: IRQ domain of the controller + * @pdev: Platform device associated to the controller + */ +struct kvx_itgen { + void __iomem *base; + struct irq_domain *domain; + struct platform_device *pdev; +}; + +static void __iomem *get_itgen_cfg_offset(struct kvx_itgen *itgen, + irq_hw_number_t hwirq) +{ + return itgen->base + KVX_ITGEN_CFG_TARGET_OFFSET + + hwirq * KVX_ITGEN_CFG_ELEM_SIZE; +} + +void __iomem *get_itgen_param_offset(struct kvx_itgen *itgen) +{ + return itgen->base + KVX_ITGEN_PARAM_OFFSET; +} + +static void kvx_itgen_enable(struct irq_data *data, u32 value) +{ + struct kvx_itgen *itgen = irq_data_get_irq_chip_data(data); + void __iomem *enable_reg = + get_itgen_cfg_offset(itgen, irqd_to_hwirq(data)) + + KVX_ITGEN_CFG_ENABLE_OFFSET; + + dev_dbg(&itgen->pdev->dev, "%sabling hwirq %d, addr %p\n", + value ? "En" : "Dis", + (int) irqd_to_hwirq(data), + enable_reg); + writel(value, enable_reg); +} + +static void kvx_itgen_mask(struct irq_data *data) +{ + kvx_itgen_enable(data, 0x0); + irq_chip_mask_parent(data); +} + +static void kvx_itgen_unmask(struct irq_data *data) +{ + kvx_itgen_enable(data, 0x1); + irq_chip_unmask_parent(data); +} + +#ifdef CONFIG_SMP +static int kvx_itgen_irq_set_affinity(struct irq_data *data, + const struct cpumask *dest, bool force) +{ + return -ENOSYS; +} +#endif + +static struct irq_chip itgen_irq_chip = { + .name = "kvx-itgen", + .irq_mask = kvx_itgen_mask, + .irq_unmask = kvx_itgen_unmask, +#ifdef CONFIG_SMP + .irq_set_affinity = kvx_itgen_irq_set_affinity, +#endif +}; + +#define ITGEN_UNSUPPORTED_TYPES (IRQ_TYPE_LEVEL_LOW | IRQ_TYPE_EDGE_FALLING) + +static int kvx_itgen_domain_alloc(struct irq_domain *domain, unsigned int virq, + unsigned int nr_irqs, void *args) +{ + int i, err; + struct irq_fwspec *fwspec = args; + int hwirq = fwspec->param[0]; + int type = IRQ_TYPE_NONE; + struct kvx_itgen *itgen; + + if (fwspec->param_count >= 2) + type = fwspec->param[1]; + + WARN_ON(type & ITGEN_UNSUPPORTED_TYPES); + + err = platform_msi_device_domain_alloc(domain, virq, nr_irqs); + if (err) + return err; + + itgen = platform_msi_get_host_data(domain); + + for (i = 0; i < nr_irqs; i++) { + irq_domain_set_hwirq_and_chip(domain, virq + i, hwirq + i, + &itgen_irq_chip, itgen); + if (type == IRQ_TYPE_LEVEL_HIGH) + irq_set_handler(virq + i, handle_level_irq); + } + + return 0; +} + +static const struct irq_domain_ops itgen_domain_ops = { + .alloc = kvx_itgen_domain_alloc, + .free = irq_domain_free_irqs_common, +}; + +static void kvx_itgen_write_msg(struct msi_desc *desc, struct msi_msg *msg) +{ + struct irq_data *d = irq_get_irq_data(desc->irq); + struct kvx_itgen *itgen = irq_data_get_irq_chip_data(d); + uint32_t cfg_val = 0; + uintptr_t dest_addr = ((uint64_t) msg->address_hi << 32) | + msg->address_lo; + void __iomem *cfg = get_itgen_cfg_offset(itgen, irqd_to_hwirq(d)); + + /* + * Address in the msi data is the address of the targeted mailbox. + * To save a few cells of hw, itgen configuration expects the target + * of the write using mppa id, cluster id and mailbox id instead + * of address. + * We extract these informations from the mailbox address. + */ + + cfg_val |= (((kvx_sfr_get(PCR) & KVX_SFR_PCR_CID_MASK) >> + KVX_SFR_PCR_CID_SHIFT) + << KVX_ITGEN_CFG_TARGET_CLUSTER_SHIFT); + cfg_val |= ((dest_addr >> MB_ADDR_MAILBOX_SHIFT) & + KVX_ITGEN_CFG_TARGET_MAILBOX_MASK) + << KVX_ITGEN_CFG_TARGET_MAILBOX_SHIFT; + + /* + * msg->data contains the bit number to be written and is included in + * the itgen config + */ + cfg_val |= ((msg->data << KVX_ITGEN_CFG_TARGET_SELECT_BIT_SHIFT) + & KVX_ITGEN_CFG_TARGET_SELECT_BIT_MASK); + + dev_dbg(&itgen->pdev->dev, + "Writing dest_addr %lx, value %x to cfg %p\n", + dest_addr, cfg_val, cfg); + + writel(cfg_val, cfg); +} + +static int +kvx_itgen_device_probe(struct platform_device *pdev) +{ + struct kvx_itgen *itgen; + u32 it_count; + struct resource *mem; + + itgen = devm_kzalloc(&pdev->dev, sizeof(*itgen), GFP_KERNEL); + if (!itgen) + return -ENOMEM; + + mem = platform_get_resource(pdev, IORESOURCE_MEM, 0); + itgen->base = devm_ioremap_resource(&pdev->dev, mem); + if (IS_ERR(itgen->base)) { + dev_err(&pdev->dev, "Failed to ioremap itgen\n"); + return PTR_ERR(itgen->base); + } + + itgen->pdev = pdev; + it_count = readl(get_itgen_param_offset(itgen) + + KVX_ITGEN_PARAM_IT_NUM_OFFSET); + + itgen->domain = platform_msi_create_device_domain(&pdev->dev, + it_count, + kvx_itgen_write_msg, + &itgen_domain_ops, + itgen); + if (!itgen->domain) { + dev_err(&pdev->dev, "Failed to create device domain\n"); + return -ENOMEM; + } + + platform_set_drvdata(pdev, itgen); + + return 0; +} + +static const struct of_device_id itgen_of_match[] = { + { .compatible = "kalray,kvx-itgen" }, + { /* END */ } +}; +MODULE_DEVICE_TABLE(of, itgen_of_match); + +static struct platform_driver itgen_platform_driver = { + .driver = { + .name = "kvx-itgen", + .of_match_table = itgen_of_match, + }, + .probe = kvx_itgen_device_probe, +}; + +static int __init kvx_itgen_init(void) +{ + return platform_driver_register(&itgen_platform_driver); +} + +arch_initcall(kvx_itgen_init); From patchwork Fri Jan 20 14:09:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13109956 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD0A3C05027 for ; Fri, 20 Jan 2023 14:17:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230193AbjATORX (ORCPT ); Fri, 20 Jan 2023 09:17:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44276 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230429AbjATORU (ORCPT ); Fri, 20 Jan 2023 09:17:20 -0500 Received: from fx405.security-mail.net (smtpout140.security-mail.net [85.31.212.145]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E9CD6C925B for ; Fri, 20 Jan 2023 06:16:32 -0800 (PST) Received: from localhost (fx405.security-mail.net [127.0.0.1]) by fx405.security-mail.net (Postfix) with ESMTP id 35C8F335EC4 for ; Fri, 20 Jan 2023 15:10:35 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674223835; bh=iujg4Ph3I/139elOqxsm2GEAdcANWZxNc2YUlfHOMPk=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=At427VLRx9KZBH1e/yv/uj8ny7ycEXrXirnyPoLQe2OWRM2ZUeLHSWZmMw3jsR+e+ 5TnrsE+Tt+dXb5kUqnLTKpoVNbR2w6GYCO8DfGPEZ+MuE1VC8GypSNw5gkW4Jjtb83 /iMIeqyIZQOtZFhaycr2kz/efqq+PLlkVDCetw6Y= Received: from fx405 (fx405.security-mail.net [127.0.0.1]) by fx405.security-mail.net (Postfix) with ESMTP id 69C64335E38; Fri, 20 Jan 2023 15:10:34 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx405.security-mail.net (Postfix) with ESMTPS id 0EDD6335DF0; Fri, 20 Jan 2023 15:10:33 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id B6A4527E0440; Fri, 20 Jan 2023 15:10:32 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 9294827E043D; Fri, 20 Jan 2023 15:10:32 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id JVyjuwXZV6nO; Fri, 20 Jan 2023 15:10:32 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 0B0F627E0439; Fri, 20 Jan 2023 15:10:32 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <16482.63caa0d9.d431.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 9294827E043D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223832; bh=j3u1+7jSeFlQ+R2vG1RMdQ6rymjVuATFjYu04HprmPk=; h=From:To:Date:Message-Id:MIME-Version; b=bJCwUqYf1v/WbS/vZFq+PAILA35tNYon5tbMQndVY/zEuvJzRPP6XNc9TdSn69NRl r1nrOQyoJ8SkIAojICaqdqM9R42vB2lEIH2sPGEYNOtks6kAZhLSXC3cg4OkeS3YSZ pUnjQCtQdqnUilobABMuo4pUzCEtBqdHlf0tvWE0= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 17/31] irqchip: Add irq-kvx-apic-mailbox driver Date: Fri, 20 Jan 2023 15:09:48 +0100 Message-ID: <20230120141002.2442-18-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC From: Jules Maselbas The APIC includes a mailbox controller, containing 128 mailboxes. Each mailbox is a word of 8 bytes in the controller internal memory. Each mailbox can be independently configured with a trigger condition and an input function. After a write to a mailbox if the mailbox's trigger condition is met then an interrupt will be generated. Since this hardware block generates IRQs based on writes at some memory locations, it is both an interrupt controller and an MSI controller. Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Jules Maselbas Signed-off-by: Jules Maselbas Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Luc Michel Signed-off-by: Luc Michel Co-developed-by: Yann Sionneau Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: new patch - removed print on probe success drivers/irqchip/Kconfig | 8 + drivers/irqchip/Makefile | 1 + drivers/irqchip/irq-kvx-apic-mailbox.c | 480 +++++++++++++++++++++++++ 3 files changed, 489 insertions(+) create mode 100644 drivers/irqchip/irq-kvx-apic-mailbox.c diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig index 546bc611f3f3..806adbc7b2a4 100644 --- a/drivers/irqchip/Kconfig +++ b/drivers/irqchip/Kconfig @@ -348,6 +348,14 @@ config KVX_ITGEN select IRQ_DOMAIN select IRQ_DOMAIN_HIERARCHY +config KVX_APIC_MAILBOX + bool + depends on KVX + select GENERIC_IRQ_IPI if SMP + select GENERIC_MSI_IRQ_DOMAIN + select IRQ_DOMAIN + select IRQ_DOMAIN_HIERARCHY + config INGENIC_IRQ bool depends on MACH_INGENIC diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile index 6b8f459d8a21..7eaea87ca9ab 100644 --- a/drivers/irqchip/Makefile +++ b/drivers/irqchip/Makefile @@ -71,6 +71,7 @@ obj-$(CONFIG_KEYSTONE_IRQ) += irq-keystone.o obj-$(CONFIG_MIPS_GIC) += irq-mips-gic.o obj-$(CONFIG_KVX_APIC_GIC) += irq-kvx-apic-gic.o obj-$(CONFIG_KVX_ITGEN) += irq-kvx-itgen.o +obj-$(CONFIG_KVX_APIC_MAILBOX) += irq-kvx-apic-mailbox.o obj-$(CONFIG_ARCH_MEDIATEK) += irq-mtk-sysirq.o irq-mtk-cirq.o obj-$(CONFIG_ARCH_DIGICOLOR) += irq-digicolor.o obj-$(CONFIG_ARCH_SA1100) += irq-sa11x0.o diff --git a/drivers/irqchip/irq-kvx-apic-mailbox.c b/drivers/irqchip/irq-kvx-apic-mailbox.c new file mode 100644 index 000000000000..33249df047b6 --- /dev/null +++ b/drivers/irqchip/irq-kvx-apic-mailbox.c @@ -0,0 +1,480 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Jules Maselbas + */ + +#define pr_fmt(fmt) "kvx_apic_mailbox: " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define KVX_MAILBOX_MODE_WRITE 0x0 +#define KVX_MAILBOX_MODE_OR 0x1 +#define KVX_MAILBOX_MODE_ADD 0x2 + +#define KVX_MAILBOX_TRIG_NO_TRIG 0x0 +#define KVX_MAILBOX_TRIG_DOORBELL 0x1 +#define KVX_MAILBOX_TRIG_MATCH 0x2 +#define KVX_MAILBOX_TRIG_BARRIER 0x3 +#define KVX_MAILBOX_TRIG_THRESHOLD 0x4 + +#define KVX_MAILBOX_OFFSET 0x0 +#define KVX_MAILBOX_ELEM_SIZE 0x200 +#define KVX_MAILBOX_MASK_OFFSET 0x10 +#define KVX_MAILBOX_FUNCT_OFFSET 0x18 +#define KVX_MAILBOX_LAC_OFFSET 0x8 +#define KVX_MAILBOX_VALUE_OFFSET 0x0 +#define KVX_MAILBOX_FUNCT_MODE_SHIFT 0x0 +#define KVX_MAILBOX_FUNCT_TRIG_SHIFT 0x8 + +#define MAILBOXES_MAX_COUNT 128 + +/* Mailboxes are 64 bits wide */ +#define MAILBOXES_BIT_SIZE 64 + +/* Maximum number of mailboxes available */ +#define MAILBOXES_MAX_BIT_COUNT (MAILBOXES_MAX_COUNT * MAILBOXES_BIT_SIZE) + +/* Mailboxes are grouped by 8 in a single page */ +#define MAILBOXES_BITS_PER_PAGE (8 * MAILBOXES_BIT_SIZE) + +/** + * struct mb_data - per mailbox data + * @cpu: CPU on which the mailbox is routed + * @parent_irq: Parent IRQ on the GIC + */ +struct mb_data { + unsigned int cpu; + unsigned int parent_irq; +}; + +/** + * struct kvx_apic_mailbox - kvx apic mailbox + * @base: base address of the controller + * @device_domain: IRQ device domain for mailboxes + * @msi_domain: platform MSI domain for MSI interface + * @domain_info: Domain information needed for the MSI domain + * @mb_count: Count of mailboxes we are handling + * @available: bitmap of availables bits in mailboxes + * @mailboxes_lock: lock for irq migration + * @mask_lock: lock for irq masking + * @mb_data: data associated to each mailbox + */ +struct kvx_apic_mailbox { + void __iomem *base; + phys_addr_t phys_base; + struct irq_domain *device_domain; + struct irq_domain *msi_domain; + struct msi_domain_info domain_info; + /* Start and count of device mailboxes */ + unsigned int mb_count; + /* Bitmap of allocated bits in mailboxes */ + DECLARE_BITMAP(available, MAILBOXES_MAX_BIT_COUNT); + spinlock_t mailboxes_lock; + raw_spinlock_t mask_lock; + struct mb_data mb_data[MAILBOXES_MAX_COUNT]; +}; + +/** + * struct kvx_irq_data - per irq data + * @mb: Mailbox structure + */ +struct kvx_irq_data { + struct kvx_apic_mailbox *mb; +}; + +static void kvx_mailbox_get_from_hwirq(unsigned int hw_irq, + unsigned int *mailbox_num, + unsigned int *mailbox_bit) +{ + *mailbox_num = hw_irq / MAILBOXES_BIT_SIZE; + *mailbox_bit = hw_irq % MAILBOXES_BIT_SIZE; +} + +static void __iomem *kvx_mailbox_get_addr(struct kvx_apic_mailbox *mb, + unsigned int num) +{ + return mb->base + (num * KVX_MAILBOX_ELEM_SIZE); +} + +static phys_addr_t kvx_mailbox_get_phys_addr(struct kvx_apic_mailbox *mb, + unsigned int num) +{ + return mb->phys_base + (num * KVX_MAILBOX_ELEM_SIZE); +} + +static void kvx_mailbox_msi_compose_msg(struct irq_data *data, + struct msi_msg *msg) +{ + struct kvx_irq_data *kd = irq_data_get_irq_chip_data(data); + struct kvx_apic_mailbox *mb = kd->mb; + unsigned int mb_num, mb_bit; + phys_addr_t mb_addr; + + kvx_mailbox_get_from_hwirq(irqd_to_hwirq(data), &mb_num, &mb_bit); + mb_addr = kvx_mailbox_get_phys_addr(mb, mb_num); + + msg->address_hi = upper_32_bits(mb_addr); + msg->address_lo = lower_32_bits(mb_addr); + msg->data = mb_bit; + + iommu_dma_compose_msi_msg(irq_data_get_msi_desc(data), msg); +} + +static void kvx_mailbox_set_irq_enable(struct irq_data *data, + bool enabled) +{ + struct kvx_irq_data *kd = irq_data_get_irq_chip_data(data); + struct kvx_apic_mailbox *mb = kd->mb; + unsigned int mb_num, mb_bit; + void __iomem *mb_addr; + u64 mask_value, mb_value; + + kvx_mailbox_get_from_hwirq(irqd_to_hwirq(data), &mb_num, &mb_bit); + mb_addr = kvx_mailbox_get_addr(mb, mb_num); + + raw_spin_lock(&mb->mask_lock); + mask_value = readq(mb_addr + KVX_MAILBOX_MASK_OFFSET); + if (enabled) + mask_value |= BIT_ULL(mb_bit); + else + mask_value &= ~BIT_ULL(mb_bit); + + writeq(mask_value, mb_addr + KVX_MAILBOX_MASK_OFFSET); + + raw_spin_unlock(&mb->mask_lock); + + /** + * Since interrupts on mailboxes are edge triggered and are only + * triggered when writing the value, we need to trigger it manually + * after updating the mask if enabled. If the interrupt was triggered by + * the device just after the mask write, we can trigger a spurious + * interrupt but that is still better than missing one... + * Moreover, the mailbox is configured in OR mode which means that even + * if we write a single bit, all other bits will be kept intact. + */ + if (enabled) { + mb_value = readq(mb_addr + KVX_MAILBOX_VALUE_OFFSET); + if (mb_value & BIT_ULL(mb_bit)) + writeq(BIT_ULL(mb_bit), + mb_addr + KVX_MAILBOX_VALUE_OFFSET); + } +} + +static void kvx_mailbox_mask(struct irq_data *data) +{ + kvx_mailbox_set_irq_enable(data, false); +} + +static void kvx_mailbox_unmask(struct irq_data *data) +{ + kvx_mailbox_set_irq_enable(data, true); +} + +static void kvx_mailbox_set_cpu(struct kvx_apic_mailbox *mb, int mb_id, + int new_cpu) +{ + irq_set_affinity(mb->mb_data[mb_id].parent_irq, cpumask_of(new_cpu)); + mb->mb_data[mb_id].cpu = new_cpu; +} + +static void kvx_mailbox_free_bit(struct kvx_apic_mailbox *mb, int hw_irq) +{ + unsigned int mb_num, mb_bit; + + kvx_mailbox_get_from_hwirq(hw_irq, &mb_num, &mb_bit); + bitmap_clear(mb->available, hw_irq, 1); + + /* If there is no more IRQ on this mailbox, reset it to CPU 0 */ + if (mb->available[mb_num] == 0) + kvx_mailbox_set_cpu(mb, mb_num, 0); +} + +struct irq_chip kvx_apic_mailbox_irq_chip = { + .name = "kvx apic mailbox", + .irq_compose_msi_msg = kvx_mailbox_msi_compose_msg, + .irq_mask = kvx_mailbox_mask, + .irq_unmask = kvx_mailbox_unmask, +}; + +static int kvx_mailbox_allocate_bits(struct kvx_apic_mailbox *mb, int num_req) +{ + int first, align_mask = 0; + + /* This must be a power of 2 for bitmap_find_next_zero_area to work */ + BUILD_BUG_ON((MAILBOXES_BITS_PER_PAGE & (MAILBOXES_BITS_PER_PAGE - 1))); + + /* + * If user requested more than 1 mailbox, we must make sure it will be + * aligned on a page size for iommu_dma_prepare_msi to be correctly + * mapped in a single page. + */ + if (num_req > 1) + align_mask = (MAILBOXES_BITS_PER_PAGE - 1); + + spin_lock(&mb->mailboxes_lock); + + first = bitmap_find_next_zero_area(mb->available, + mb->mb_count * MAILBOXES_BIT_SIZE, 0, + num_req, align_mask); + if (first >= MAILBOXES_MAX_BIT_COUNT) { + spin_unlock(&mb->mailboxes_lock); + return -ENOSPC; + } + + bitmap_set(mb->available, first, num_req); + + spin_unlock(&mb->mailboxes_lock); + + return first; +} + +static int kvx_apic_mailbox_msi_alloc(struct irq_domain *domain, + unsigned int virq, + unsigned int nr_irqs, void *args) +{ + int i, err; + int hwirq = 0; + u64 mb_addr; + struct irq_data *d; + struct kvx_irq_data *kd; + struct kvx_apic_mailbox *mb = domain->host_data; + struct msi_alloc_info *msi_info = (struct msi_alloc_info *)args; + struct msi_desc *desc = msi_info->desc; + unsigned int mb_num, mb_bit; + + /* We will not be able to guarantee page alignment ! */ + if (nr_irqs > MAILBOXES_BITS_PER_PAGE) + return -EINVAL; + + hwirq = kvx_mailbox_allocate_bits(mb, nr_irqs); + if (hwirq < 0) + return hwirq; + + kvx_mailbox_get_from_hwirq(hwirq, &mb_num, &mb_bit); + mb_addr = (u64) kvx_mailbox_get_phys_addr(mb, mb_num); + err = iommu_dma_prepare_msi(desc, mb_addr); + if (err) + goto free_mb_bits; + + for (i = 0; i < nr_irqs; i++) { + kd = kmalloc(sizeof(*kd), GFP_KERNEL); + if (!kd) { + err = -ENOMEM; + goto free_irq_data; + } + + kd->mb = mb; + irq_domain_set_info(domain, virq + i, hwirq + i, + &kvx_apic_mailbox_irq_chip, + kd, handle_simple_irq, + NULL, NULL); + } + + return 0; + +free_irq_data: + for (i--; i >= 0; i--) { + d = irq_domain_get_irq_data(domain, virq + i); + kd = irq_data_get_irq_chip_data(d); + kfree(kd); + } + +free_mb_bits: + spin_lock(&mb->mailboxes_lock); + bitmap_clear(mb->available, hwirq, nr_irqs); + spin_unlock(&mb->mailboxes_lock); + + return err; +} + +static void kvx_apic_mailbox_msi_free(struct irq_domain *domain, + unsigned int virq, + unsigned int nr_irqs) +{ + int i; + struct irq_data *d; + struct kvx_irq_data *kd; + struct kvx_apic_mailbox *mb = domain->host_data; + + spin_lock(&mb->mailboxes_lock); + + for (i = 0; i < nr_irqs; i++) { + d = irq_domain_get_irq_data(domain, virq + i); + kd = irq_data_get_irq_chip_data(d); + kfree(kd); + kvx_mailbox_free_bit(mb, d->hwirq); + } + + spin_unlock(&mb->mailboxes_lock); +} + +static const struct irq_domain_ops kvx_apic_mailbox_domain_ops = { + .alloc = kvx_apic_mailbox_msi_alloc, + .free = kvx_apic_mailbox_msi_free +}; + +static struct irq_chip kvx_msi_irq_chip = { + .name = "KVX MSI", +}; + +static void kvx_apic_mailbox_handle_irq(struct irq_desc *desc) +{ + struct irq_data *data = irq_desc_get_irq_data(desc); + struct kvx_apic_mailbox *mb = irq_desc_get_handler_data(desc); + void __iomem *mb_addr = kvx_mailbox_get_addr(mb, irqd_to_hwirq(data)); + unsigned int irqn, cascade_irq, bit; + u64 mask_value, masked_its; + u64 mb_value; + /* Since we allocate 64 interrupts for each mailbox, the scheme + * to find the hwirq associated to a mailbox irq is the + * following: + * hw_irq = mb_num * MAILBOXES_BIT_SIZE + bit + */ + unsigned int mb_hwirq = irqd_to_hwirq(data) * MAILBOXES_BIT_SIZE; + + mb_value = readq(mb_addr + KVX_MAILBOX_LAC_OFFSET); + mask_value = readq(mb_addr + KVX_MAILBOX_MASK_OFFSET); + /* Mask any disabled interrupts */ + mb_value &= mask_value; + + /** + * Write all pending ITs that are masked to process them later + * Since the mailbox is in OR mode, these bits will be merged with any + * already set bits and thus avoid losing any interrupts. + */ + masked_its = (~mask_value) & mb_value; + if (masked_its) + writeq(masked_its, mb_addr + KVX_MAILBOX_LAC_OFFSET); + + for_each_set_bit(bit, (unsigned long *) &mb_value, BITS_PER_LONG) { + irqn = bit + mb_hwirq; + cascade_irq = irq_find_mapping(mb->device_domain, irqn); + generic_handle_irq(cascade_irq); + } +} + +static void __init +apic_mailbox_reset(struct kvx_apic_mailbox *mb) +{ + unsigned int i; + unsigned int mb_end = mb->mb_count; + void __iomem *mb_addr; + u64 funct_val = (KVX_MAILBOX_MODE_OR << KVX_MAILBOX_FUNCT_MODE_SHIFT) | + (KVX_MAILBOX_TRIG_DOORBELL << KVX_MAILBOX_FUNCT_TRIG_SHIFT); + + for (i = 0; i < mb_end; i++) { + mb_addr = kvx_mailbox_get_addr(mb, i); + /* Disable all interrupts */ + writeq(0ULL, mb_addr + KVX_MAILBOX_MASK_OFFSET); + /* Set mailbox to OR mode + trigger */ + writeq(funct_val, mb_addr + KVX_MAILBOX_FUNCT_OFFSET); + /* Load & Clear mailbox value */ + readq(mb_addr + KVX_MAILBOX_LAC_OFFSET); + } +} + +static struct msi_domain_ops kvx_msi_domain_ops = { +}; + +static struct msi_domain_info kvx_msi_domain_info = { + .flags = (MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS), + .ops = &kvx_msi_domain_ops, + .chip = &kvx_msi_irq_chip, +}; + +static int __init +kvx_init_apic_mailbox(struct device_node *node, + struct device_node *parent) +{ + struct kvx_apic_mailbox *mb; + unsigned int parent_irq, irq_count; + struct resource res; + int ret, i; + + mb = kzalloc(sizeof(*mb), GFP_KERNEL); + if (!mb) + return -ENOMEM; + + ret = of_address_to_resource(node, 0, &res); + if (ret) + return -EINVAL; + + mb->phys_base = res.start; + mb->base = of_io_request_and_map(node, 0, node->name); + if (!mb->base) { + ret = -EINVAL; + goto err_kfree; + } + + spin_lock_init(&mb->mailboxes_lock); + raw_spin_lock_init(&mb->mask_lock); + + irq_count = of_irq_count(node); + if (irq_count == 0 || irq_count > MAILBOXES_MAX_COUNT) { + ret = -EINVAL; + goto err_kfree; + } + mb->mb_count = irq_count; + + apic_mailbox_reset(mb); + + mb->device_domain = irq_domain_add_tree(node, + &kvx_apic_mailbox_domain_ops, + mb); + if (!mb->device_domain) { + pr_err("Failed to setup device domain\n"); + ret = -EINVAL; + goto err_iounmap; + } + + mb->msi_domain = platform_msi_create_irq_domain(of_node_to_fwnode(node), + &kvx_msi_domain_info, + mb->device_domain); + if (!mb->msi_domain) { + ret = -EINVAL; + goto err_irq_domain_add_tree; + } + + /* Chain all interrupts from gic to mailbox */ + for (i = 0; i < irq_count; i++) { + parent_irq = irq_of_parse_and_map(node, i); + if (parent_irq == 0) { + pr_err("unable to parse irq\n"); + ret = -EINVAL; + goto err_irq_domain_msi_create; + } + mb->mb_data[i].parent_irq = parent_irq; + + irq_set_chained_handler_and_data(parent_irq, + kvx_apic_mailbox_handle_irq, + mb); + } + + return 0; + +err_irq_domain_msi_create: + irq_domain_remove(mb->msi_domain); +err_irq_domain_add_tree: + irq_domain_remove(mb->device_domain); +err_iounmap: + iounmap(mb->base); +err_kfree: + kfree(mb); + + return ret; +} + +IRQCHIP_DECLARE(kvx_apic_mailbox, "kalray,kvx-apic-mailbox", + kvx_init_apic_mailbox); From patchwork Fri Jan 20 14:09:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13110009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3436EC38159 for ; Fri, 20 Jan 2023 14:26:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230273AbjATO0h (ORCPT ); Fri, 20 Jan 2023 09:26:37 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230176AbjATO0T (ORCPT ); Fri, 20 Jan 2023 09:26:19 -0500 Received: from fx408.security-mail.net (smtpout140.security-mail.net [85.31.212.148]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 287A8CD20A for ; Fri, 20 Jan 2023 06:25:46 -0800 (PST) Received: from localhost (fx408.security-mail.net [127.0.0.1]) by fx408.security-mail.net (Postfix) with ESMTP id F3184322A22 for ; Fri, 20 Jan 2023 15:25:44 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674224745; bh=C4JX2KDfJMf+kd6dGlNUF/l2OL1SVZPZi+/ckEfH3UU=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=OEAowwf4PHI9aHodicAZw2OmPCkxNfZ9jczBNGXv9tGymDEannLiE1LP+n9nmqO7+ 1cckPFYv4q6PFzmYhvScsOqu9QykuhINoOX3RAbhMQi3QQnGdmISLPYDox5zSNMbnO qegSyV5aWJrnc3PEAdTLgoFhK+tO7nBwmqbmlamA= Received: from fx408 (fx408.security-mail.net [127.0.0.1]) by fx408.security-mail.net (Postfix) with ESMTP id B5716322547; Fri, 20 Jan 2023 15:25:44 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx408.security-mail.net (Postfix) with ESMTPS id 1C87E322929; Fri, 20 Jan 2023 15:25:44 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 2A48827E0437; Fri, 20 Jan 2023 15:10:33 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 0210427E043D; Fri, 20 Jan 2023 15:10:33 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ImWCZZGcWW5W; Fri, 20 Jan 2023 15:10:32 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 8995D27E043A; Fri, 20 Jan 2023 15:10:32 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 0210427E043D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223833; bh=5XXrgrQzCfc4vEIClRHO/5PBD+bYAS7ERJgrp4OCvF8=; h=From:To:Date:Message-Id:MIME-Version; b=FSrUM33In5SGjg9c4LwAoWf/AQZS0QSN3rkfGHBE+hEzpy/Vbg1dnPf32zDa8kKMa MP0vhwj3UfQzIUAbEwlHIAX+q89B2EfKjyUFxpR38io82dkQQDXMofBuaYRqLg7798 JotwpQrZZ+Hy6YLcj2fPGriS/JxTFf2hsIWLvBEE= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 18/31] irqchip: Add kvx-core-intc core interupt controller driver Date: Fri, 20 Jan 2023 15:09:49 +0100 Message-ID: <20230120141002.2442-19-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC From: Jules Maselbas Each kvx core includes a hardware interrupt controller (core INTC) with the following features: - 32 independent interrupt sources - 4-bit priotity level - Individual interrupt enable bit - Interrupt status bit displaying the pending interrupts - Priority management between the 32 interrupts Among those 32 interrupt sources, the first are hard-wired to hardware sources. The remaining interrupt sources can be triggered via software by directly writing to the ILR SFR. The hard-wired interrupt sources are the following: 0: Timer 0 1: Timer 1 2: Watchdog 3: Performance Monitors 4: APIC GIC line 0 5: APIC GIC line 1 6: APIC GIC line 2 7: APIC GIC line 3 12: SECC error from memory system 13: Arithmetic exception (carry and IEEE 754 flags) 16: Data Asynchronous Memory Error (DAME), raised for DECC/DSYS errors 17: CLI (Cache Line Invalidation) for L1D or L1I following DECC/DSYS/Parity errors The APIC GIC lines will be used to route interrupts coming from SoC peripherals from outside the Cluster to the kvx core. Those peripherals include USB host controller, eMMC/SD host controller, i2c, spi, PCIe, IOMMUs etc... Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Vincent Chardon Signed-off-by: Vincent Chardon Co-developed-by: Jules Maselbas Signed-off-by: Jules Maselbas Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: new patch - removed print on probe success drivers/irqchip/Kconfig | 5 ++ drivers/irqchip/Makefile | 1 + drivers/irqchip/irq-kvx-core-intc.c | 80 +++++++++++++++++++++++++++++ 3 files changed, 86 insertions(+) create mode 100644 drivers/irqchip/irq-kvx-core-intc.c diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig index 806adbc7b2a4..d242e02771e3 100644 --- a/drivers/irqchip/Kconfig +++ b/drivers/irqchip/Kconfig @@ -334,6 +334,11 @@ config MIPS_GIC select IRQ_DOMAIN_HIERARCHY select MIPS_CM +config KVX_CORE_INTC + bool + depends on KVX + select IRQ_DOMAIN + config KVX_APIC_GIC bool depends on KVX diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile index 7eaea87ca9ab..d931f2eb38b6 100644 --- a/drivers/irqchip/Makefile +++ b/drivers/irqchip/Makefile @@ -69,6 +69,7 @@ obj-$(CONFIG_BCM7120_L2_IRQ) += irq-bcm7120-l2.o obj-$(CONFIG_BRCMSTB_L2_IRQ) += irq-brcmstb-l2.o obj-$(CONFIG_KEYSTONE_IRQ) += irq-keystone.o obj-$(CONFIG_MIPS_GIC) += irq-mips-gic.o +obj-$(CONFIG_KVX_CORE_INTC) += irq-kvx-core-intc.o obj-$(CONFIG_KVX_APIC_GIC) += irq-kvx-apic-gic.o obj-$(CONFIG_KVX_ITGEN) += irq-kvx-itgen.o obj-$(CONFIG_KVX_APIC_MAILBOX) += irq-kvx-apic-mailbox.o diff --git a/drivers/irqchip/irq-kvx-core-intc.c b/drivers/irqchip/irq-kvx-core-intc.c new file mode 100644 index 000000000000..145f1248925b --- /dev/null +++ b/drivers/irqchip/irq-kvx-core-intc.c @@ -0,0 +1,80 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#define pr_fmt(fmt) "kvx_core_intc: " fmt + +#include +#include +#include +#include +#include +#include +#include + +#define KVX_CORE_INTC_IRQ 32 + + +static void kvx_irq_mask(struct irq_data *data) +{ + kvx_sfr_clear_bit(ILE, data->hwirq); +} + +static void kvx_irq_unmask(struct irq_data *data) +{ + kvx_sfr_set_bit(ILE, data->hwirq); +} + +static struct irq_chip kvx_irq_chip = { + .name = "kvx core Intc", + .irq_mask = kvx_irq_mask, + .irq_unmask = kvx_irq_unmask, +}; + +static int kvx_irq_map(struct irq_domain *d, unsigned int irq, + irq_hw_number_t hw) +{ + /* All interrupts for core are per cpu */ + irq_set_percpu_devid(irq); + irq_set_chip_and_handler(irq, &kvx_irq_chip, handle_percpu_irq); + + return 0; +} + +static const struct irq_domain_ops kvx_irq_ops = { + .xlate = irq_domain_xlate_onecell, + .map = kvx_irq_map, +}; + +static int __init +kvx_init_core_intc(struct device_node *intc, struct device_node *parent) +{ + struct irq_domain *root_domain; + uint32_t core_nr_irqs; + + if (parent) + panic("DeviceTree core intc not a root irq controller\n"); + + if (of_property_read_u32(intc, "kalray,intc-nr-irqs", &core_nr_irqs)) + core_nr_irqs = KVX_CORE_INTC_IRQ; + + /* We only have up to 32 interrupts, according to IRQ-domain.txt, + * linear is likely to be the best choice + */ + root_domain = irq_domain_add_linear(intc, core_nr_irqs, + &kvx_irq_ops, NULL); + if (!root_domain) + panic("root irq domain not avail\n"); + + /* + * Needed for primary domain lookup to succeed + * This is a primary irqchip, and can never have a parent + */ + irq_set_default_host(root_domain); + + return 0; +} + +IRQCHIP_DECLARE(kvx_core_intc, "kalray,kvx-core-intc", kvx_init_core_intc); From patchwork Fri Jan 20 14:09:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13109955 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7DA61C05027 for ; Fri, 20 Jan 2023 14:16:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230208AbjATOQr (ORCPT ); Fri, 20 Jan 2023 09:16:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43800 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230178AbjATOQq (ORCPT ); Fri, 20 Jan 2023 09:16:46 -0500 Received: from fx405.security-mail.net (smtpout140.security-mail.net [85.31.212.145]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BCB6FCB52F for ; Fri, 20 Jan 2023 06:16:04 -0800 (PST) Received: from localhost (fx405.security-mail.net [127.0.0.1]) by fx405.security-mail.net (Postfix) with ESMTP id 5D5C2335ED6 for ; Fri, 20 Jan 2023 15:10:35 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674223835; bh=s0s3ZiOaChe4IspP/f0FkJ7Hvx4yiDNnsro9YJeTTzM=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=NYJxumxS5LYrv2pTdomRPl+OVVvqaBAo4kNGq82kihiJ/5wQ7SlfSX7xOgCn4eap3 znUNdtPG5/oTviicrKfq3onCFfDdKnu0k4btzE+wMjV2ydtEmcILjWCQaRGtTMrQO8 Wps+yMH/FjQAoJiHRgTcL2AxDIyH+bzK4T1KDcTI= Received: from fx405 (fx405.security-mail.net [127.0.0.1]) by fx405.security-mail.net (Postfix) with ESMTP id AF3BC335E98; Fri, 20 Jan 2023 15:10:34 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx405.security-mail.net (Postfix) with ESMTPS id C95DC335B8A; Fri, 20 Jan 2023 15:10:33 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 884B127E0444; Fri, 20 Jan 2023 15:10:33 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 7018427E043D; Fri, 20 Jan 2023 15:10:33 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id qVvk-rOi_LrW; Fri, 20 Jan 2023 15:10:33 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 0003627E0439; Fri, 20 Jan 2023 15:10:32 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <113d2.63caa0d9.c7c9d.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 7018427E043D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223833; bh=1jVxsLsovn3fhaiWW+YOhEGV9C0R95VwjQdE5el+poY=; h=From:To:Date:Message-Id:MIME-Version; b=lAi9EcZrOVnADHSlLn0HzqUHXd/OoVYz+1r4Pur0oLkIW89a5cu55ZEBVKaE81NSr 33VCPt0tq/2HWar0ALkfU+wvbyjMTEbO3NJrHfMZKcPQcwPkZHrzHWoRLcRVOuqooS hVnD0bnQbK/OUm35vmHbIDQ8ijLOINjegpqpvtoI= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 19/31] kvx: Add process management Date: Fri, 20 Jan 2023 15:09:50 +0100 Message-ID: <20230120141002.2442-20-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Add process management support for kvx, including: thread info definition, context switch and process tracing. Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Guillaume Thouvenin Signed-off-by: Guillaume Thouvenin Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Marius Gligor Signed-off-by: Marius Gligor Co-developed-by: Vincent Chardon Signed-off-by: Vincent Chardon Co-developed-by: Yann Sionneau Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: no change arch/kvx/include/asm/current.h | 22 +++ arch/kvx/include/asm/ptrace.h | 217 +++++++++++++++++++++++ arch/kvx/include/asm/switch_to.h | 21 +++ arch/kvx/include/asm/thread_info.h | 78 ++++++++ arch/kvx/include/uapi/asm/ptrace.h | 114 ++++++++++++ arch/kvx/kernel/process.c | 203 +++++++++++++++++++++ arch/kvx/kernel/ptrace.c | 276 +++++++++++++++++++++++++++++ arch/kvx/kernel/stacktrace.c | 173 ++++++++++++++++++ 8 files changed, 1104 insertions(+) create mode 100644 arch/kvx/include/asm/current.h create mode 100644 arch/kvx/include/asm/ptrace.h create mode 100644 arch/kvx/include/asm/switch_to.h create mode 100644 arch/kvx/include/asm/thread_info.h create mode 100644 arch/kvx/include/uapi/asm/ptrace.h create mode 100644 arch/kvx/kernel/process.c create mode 100644 arch/kvx/kernel/ptrace.c create mode 100644 arch/kvx/kernel/stacktrace.c diff --git a/arch/kvx/include/asm/current.h b/arch/kvx/include/asm/current.h new file mode 100644 index 000000000000..b5fd0f076ec9 --- /dev/null +++ b/arch/kvx/include/asm/current.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_CURRENT_H +#define _ASM_KVX_CURRENT_H + +#include +#include + +struct task_struct; + +static __always_inline struct task_struct *get_current(void) +{ + return (struct task_struct *) kvx_sfr_get(SR); +} + +#define current get_current() + +#endif /* _ASM_KVX_CURRENT_H */ diff --git a/arch/kvx/include/asm/ptrace.h b/arch/kvx/include/asm/ptrace.h new file mode 100644 index 000000000000..d1b1e0975d9e --- /dev/null +++ b/arch/kvx/include/asm/ptrace.h @@ -0,0 +1,217 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Marius Gligor + * Yann Sionneau + */ + +#ifndef _ASM_KVX_PTRACE_H +#define _ASM_KVX_PTRACE_H + +#include +#include +#include + +#define GPR_COUNT 64 +#define SFR_COUNT 9 +#define VIRT_COUNT 1 + +#define ES_SYSCALL 0x3 + +#define KVX_HW_BREAKPOINT_COUNT 2 +#define KVX_HW_WATCHPOINT_COUNT 1 + +#define REG_SIZE sizeof(u64) + +/** + * When updating pt_regs structure, this size must be updated. + * This is the expected size of the pt_regs struct. + * It ensures the structure layout from gcc is the same as the one we + * expect in order to do packed load (load/store octuple) in assembly. + * Conclusion: never put sizeof(pt_regs) in here or we lose this check + * (build time check done in asm-offsets.c via BUILD_BUG_ON) + */ +#define PT_REGS_STRUCT_EXPECTED_SIZE \ + ((GPR_COUNT + SFR_COUNT + VIRT_COUNT) * REG_SIZE + \ + 2 * REG_SIZE) /* Padding for stack alignment */ + +/** + * Saved register structure. Note that we should save only the necessary + * registers. + * When you modify it, please read carefully the comment above. + * Moreover, you will need to modify user_pt_regs to match the beginning + * of this struct 1:1 + */ +struct pt_regs { + union { + struct user_pt_regs user_regs; + struct { + /* GPR */ + uint64_t r0; + uint64_t r1; + uint64_t r2; + uint64_t r3; + uint64_t r4; + uint64_t r5; + uint64_t r6; + uint64_t r7; + uint64_t r8; + uint64_t r9; + uint64_t r10; + uint64_t r11; + union { + uint64_t r12; + uint64_t sp; + }; + union { + uint64_t r13; + uint64_t tp; + }; + union { + uint64_t r14; + uint64_t fp; + }; + uint64_t r15; + uint64_t r16; + uint64_t r17; + uint64_t r18; + uint64_t r19; + uint64_t r20; + uint64_t r21; + uint64_t r22; + uint64_t r23; + uint64_t r24; + uint64_t r25; + uint64_t r26; + uint64_t r27; + uint64_t r28; + uint64_t r29; + uint64_t r30; + uint64_t r31; + uint64_t r32; + uint64_t r33; + uint64_t r34; + uint64_t r35; + uint64_t r36; + uint64_t r37; + uint64_t r38; + uint64_t r39; + uint64_t r40; + uint64_t r41; + uint64_t r42; + uint64_t r43; + uint64_t r44; + uint64_t r45; + uint64_t r46; + uint64_t r47; + uint64_t r48; + uint64_t r49; + uint64_t r50; + uint64_t r51; + uint64_t r52; + uint64_t r53; + uint64_t r54; + uint64_t r55; + uint64_t r56; + uint64_t r57; + uint64_t r58; + uint64_t r59; + uint64_t r60; + uint64_t r61; + uint64_t r62; + uint64_t r63; + + /* SFR */ + uint64_t lc; + uint64_t le; + uint64_t ls; + uint64_t ra; + + uint64_t cs; + uint64_t spc; + }; + }; + uint64_t sps; + uint64_t es; + + uint64_t ilr; + + /* "Virtual" registers */ + uint64_t orig_r0; + + /* Padding for stack alignment (see STACK_ALIGN) */ + uint64_t padding[2]; + + /** + * If you add some fields, please read carefully the comment for + * PT_REGS_STRUCT_EXPECTED_SIZE. + */ +}; + +#define pl(__reg) kvx_sfr_field_val(__reg, PS, PL) + +#define MODE_KERNEL 0 +#define MODE_USER 1 + +/* Privilege level is relative in $sps, so 1 indicates current PL + 1 */ +#define user_mode(regs) (pl((regs)->sps) == MODE_USER) +#define es_ec(regs) kvx_sfr_field_val(regs->es, ES, EC) +#define es_sysno(regs) kvx_sfr_field_val(regs->es, ES, SN) + +#define debug_dc(es) kvx_sfr_field_val((es), ES, DC) + +/* ptrace */ +#define PTRACE_GET_HW_PT_REGS 20 +#define PTRACE_SET_HW_PT_REGS 21 +#define arch_has_single_step() 1 + +#define DEBUG_CAUSE_BREAKPOINT 0 +#define DEBUG_CAUSE_WATCHPOINT 1 +#define DEBUG_CAUSE_STEPI 2 +#define DEBUG_CAUSE_DSU_BREAK 3 + +static inline void enable_single_step(struct pt_regs *regs) +{ + regs->sps |= KVX_SFR_PS_SME_MASK; +} + +static inline void disable_single_step(struct pt_regs *regs) +{ + regs->sps &= ~KVX_SFR_PS_SME_MASK; +} + +static inline bool in_syscall(struct pt_regs const *regs) +{ + return es_ec(regs) == ES_SYSCALL; +} + +int do_syscall_trace_enter(struct pt_regs *regs, unsigned long syscall); +void do_syscall_trace_exit(struct pt_regs *regs); + +static inline unsigned long get_current_sp(void) +{ + register const unsigned long current_sp __asm__ ("$r12"); + + return current_sp; +} + +extern char *user_scall_rt_sigreturn_end; +extern char *user_scall_rt_sigreturn; + +static inline unsigned long instruction_pointer(struct pt_regs *regs) +{ + return regs->spc; +} + +static inline long regs_return_value(struct pt_regs *regs) +{ + return regs->r0; +} + +static inline unsigned long user_stack_pointer(struct pt_regs *regs) +{ + return regs->sp; +} + +#endif /* _ASM_KVX_PTRACE_H */ diff --git a/arch/kvx/include/asm/switch_to.h b/arch/kvx/include/asm/switch_to.h new file mode 100644 index 000000000000..2b1fda06dea8 --- /dev/null +++ b/arch/kvx/include/asm/switch_to.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_SWITCH_TO_H +#define _ASM_KVX_SWITCH_TO_H + +struct task_struct; + +/* context switching is now performed out-of-line in switch_to.S */ +extern struct task_struct *__switch_to(struct task_struct *prev, + struct task_struct *next); + +#define switch_to(prev, next, last) \ + do { \ + ((last) = __switch_to((prev), (next))); \ + } while (0) + +#endif /* _ASM_KVX_SWITCH_TO_H */ diff --git a/arch/kvx/include/asm/thread_info.h b/arch/kvx/include/asm/thread_info.h new file mode 100644 index 000000000000..4ce0154813ef --- /dev/null +++ b/arch/kvx/include/asm/thread_info.h @@ -0,0 +1,78 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + */ + +#ifndef _ASM_KVX_THREAD_INFO_H +#define _ASM_KVX_THREAD_INFO_H + +#include + +/* + * Size of the kernel stack for each process. + */ +#define THREAD_SIZE_ORDER 2 +#define THREAD_SIZE (PAGE_SIZE << THREAD_SIZE_ORDER) + +/* + * Thread information flags + * these are process state flags that various assembly files may need to + * access + * - pending work-to-be-done flags are in LSW + * - other flags in MSW + */ +#define TIF_SYSCALL_TRACE 0 /* syscall trace active */ +#define TIF_NOTIFY_RESUME 1 /* resumption notification requested */ +#define TIF_SIGPENDING 2 /* signal pending */ +#define TIF_NEED_RESCHED 3 /* rescheduling necessary */ +#define TIF_SINGLESTEP 4 /* restore singlestep on return to user mode */ +#define TIF_UPROBE 5 +#define TIF_SYSCALL_TRACEPOINT 6 /* syscall tracepoint instrumentation */ +#define TIF_SYSCALL_AUDIT 7 /* syscall auditing active */ +#define TIF_RESTORE_SIGMASK 9 +#define TIF_NOTIFY_SIGNAL 10 /* signal notifications exist */ +#define TIF_POLLING_NRFLAG 16 /* true if poll_idle() is polling TIF_NEED_RESCHED */ +#define TIF_MEMDIE 17 + +#define _TIF_SYSCALL_TRACE (1 << TIF_SYSCALL_TRACE) +#define _TIF_SYSCALL_TRACEPOINT (1 << TIF_SYSCALL_TRACEPOINT) +#define _TIF_SYSCALL_AUDIT (1 << TIF_SYSCALL_AUDIT) +#define _TIF_POLLING_NRFLAG (1 << TIF_POLLING_NRFLAG) +#define _TIF_NOTIFY_RESUME (1 << TIF_NOTIFY_RESUME) +#define _TIF_SIGPENDING (1 << TIF_SIGPENDING) +#define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED) +#define _TIF_NOTIFY_SIGNAL (1 << TIF_NOTIFY_SIGNAL) + +#define _TIF_WORK_MASK \ + (_TIF_NOTIFY_RESUME | _TIF_SIGPENDING | _TIF_NEED_RESCHED) + +#define _TIF_SYSCALL_WORK \ + (_TIF_SYSCALL_TRACE | _TIF_SYSCALL_TRACEPOINT | _TIF_SYSCALL_AUDIT) + +#ifndef __ASSEMBLY__ +/* + * We are using THREAD_INFO_IN_TASK so this struct is almost useless + * please prefer adding fields in thread_struct (processor.h) rather + * than here. + * This struct is merely a remnant of distant times where it was placed + * on the stack to avoid large task_struct. + * + * cf https://lwn.net/Articles/700615/ + */ +struct thread_info { + unsigned long flags; /* low level flags */ + int preempt_count; +#ifdef CONFIG_SMP + u32 cpu; /* current CPU */ +#endif +}; + +#define INIT_THREAD_INFO(tsk) \ +{ \ + .flags = 0, \ + .preempt_count = INIT_PREEMPT_COUNT, \ +} +#endif /* __ASSEMBLY__*/ +#endif /* _ASM_KVX_THREAD_INFO_H */ diff --git a/arch/kvx/include/uapi/asm/ptrace.h b/arch/kvx/include/uapi/asm/ptrace.h new file mode 100644 index 000000000000..f5febe830526 --- /dev/null +++ b/arch/kvx/include/uapi/asm/ptrace.h @@ -0,0 +1,114 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Yann Sionneau + */ + +#ifndef _UAPI_ASM_KVX_PTRACE_H +#define _UAPI_ASM_KVX_PTRACE_H + +#include +/* + * User-mode register state for core dumps, ptrace, sigcontext + * + * This decouples struct pt_regs from the userspace ABI. + * The struct pt_regs must start with the same layout as struct user_pt_regs. + */ +struct user_pt_regs { + /* GPR */ + unsigned long r0; + unsigned long r1; + unsigned long r2; + unsigned long r3; + unsigned long r4; + unsigned long r5; + unsigned long r6; + unsigned long r7; + unsigned long r8; + unsigned long r9; + unsigned long r10; + unsigned long r11; + union { + unsigned long r12; + unsigned long sp; + }; + union { + unsigned long r13; + unsigned long tp; + }; + union { + unsigned long r14; + unsigned long fp; + }; + unsigned long r15; + unsigned long r16; + unsigned long r17; + unsigned long r18; + unsigned long r19; + unsigned long r20; + unsigned long r21; + unsigned long r22; + unsigned long r23; + unsigned long r24; + unsigned long r25; + unsigned long r26; + unsigned long r27; + unsigned long r28; + unsigned long r29; + unsigned long r30; + unsigned long r31; + unsigned long r32; + unsigned long r33; + unsigned long r34; + unsigned long r35; + unsigned long r36; + unsigned long r37; + unsigned long r38; + unsigned long r39; + unsigned long r40; + unsigned long r41; + unsigned long r42; + unsigned long r43; + unsigned long r44; + unsigned long r45; + unsigned long r46; + unsigned long r47; + unsigned long r48; + unsigned long r49; + unsigned long r50; + unsigned long r51; + unsigned long r52; + unsigned long r53; + unsigned long r54; + unsigned long r55; + unsigned long r56; + unsigned long r57; + unsigned long r58; + unsigned long r59; + unsigned long r60; + unsigned long r61; + unsigned long r62; + unsigned long r63; + + /* SFR */ + unsigned long lc; + unsigned long le; + unsigned long ls; + unsigned long ra; + + unsigned long cs; + unsigned long spc; +}; + +/* TCA registers structure exposed to user */ +struct user_tca_regs { + struct { + __u64 x; + __u64 y; + __u64 z; + __u64 t; + } regs[48]; +}; + +#endif /* _UAPI_ASM_KVX_PTRACE_H */ diff --git a/arch/kvx/kernel/process.c b/arch/kvx/kernel/process.c new file mode 100644 index 000000000000..2a1cd0509604 --- /dev/null +++ b/arch/kvx/kernel/process.c @@ -0,0 +1,203 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + * Marius Gligor + * Yann Sionneau + */ + +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +#if defined(CONFIG_STACKPROTECTOR) +#include +unsigned long __stack_chk_guard __read_mostly; +EXPORT_SYMBOL(__stack_chk_guard); +#endif + +#define SCALL_NUM_EXIT "0xfff" + +void arch_cpu_idle(void) +{ + wait_for_interrupt(); + local_irq_enable(); +} + +void show_regs(struct pt_regs *regs) +{ + + int in_kernel = 1; + unsigned short i, reg_offset; + void *ptr; + + show_regs_print_info(KERN_DEFAULT); + + if (user_mode(regs)) + in_kernel = 0; + + pr_info("\nmode: %s\n" + " PC: %016llx PS: %016llx\n" + " CS: %016llx RA: %016llx\n" + " LS: %016llx LE: %016llx\n" + " LC: %016llx\n\n", + in_kernel ? "kernel" : "user", + regs->spc, regs->sps, + regs->cs, regs->ra, regs->ls, regs->le, regs->lc); + + /* GPR */ + ptr = regs; + ptr += offsetof(struct pt_regs, r0); + reg_offset = offsetof(struct pt_regs, r1) - + offsetof(struct pt_regs, r0); + + /** + * Display all the 64 GPRs assuming they are ordered correctly + * in the pt_regs struct... + */ + for (i = 0; i < GPR_COUNT; i += 2) { + pr_info(" R%d: %016llx R%d: %016llx\n", + i, *(uint64_t *)ptr, + i + 1, *(uint64_t *)(ptr + reg_offset)); + ptr += reg_offset * 2; + } + + pr_info("\n\n"); +} + +/** + * Prepare a thread to return to userspace + */ +void start_thread(struct pt_regs *regs, + unsigned long pc, unsigned long sp) +{ + /* Remove MMUP bit (user is not privilege in current virtual space) */ + u64 clear_bit = KVX_SFR_PS_MMUP_MASK | KVX_SFR_PS_SME_MASK | + KVX_SFR_PS_SMR_MASK; + regs->spc = pc; + regs->sp = sp; + regs->sps = kvx_sfr_get(PS); + + regs->sps &= ~clear_bit; + + /* Set privilege level to +1 (relative) */ + regs->sps &= ~KVX_SFR_PS_PL_MASK; + regs->sps |= (1 << KVX_SFR_PS_PL_SHIFT); +} + +int copy_thread(struct task_struct *p, const struct kernel_clone_args *args) +{ + struct pt_regs *regs, *childregs = task_pt_regs(p); + unsigned long clone_flags = args->flags; + unsigned long usp = args->stack; + unsigned long tls = args->tls; + + /* p->thread holds context to be restored by __switch_to() */ + if (unlikely(args->fn)) { + /* Kernel thread */ + memset(childregs, 0, sizeof(struct pt_regs)); + + p->thread.ctx_switch.r20 = (uint64_t)args->fn; /* fn */ + p->thread.ctx_switch.r21 = (uint64_t)args->fn_arg; + p->thread.ctx_switch.ra = + (unsigned long) ret_from_kernel_thread; + } else { + regs = current_pt_regs(); + + /* Copy current process registers */ + *childregs = *regs; + + /* Store tracing status in r20 to avoid computing it + * in assembly + */ + p->thread.ctx_switch.r20 = + task_thread_info(p)->flags & _TIF_SYSCALL_WORK; + p->thread.ctx_switch.ra = (unsigned long) ret_from_fork; + + childregs->r0 = 0; /* Return value of fork() */ + /* Set stack pointer if any */ + if (usp) + childregs->sp = usp; + + /* Set a new TLS ? */ + if (clone_flags & CLONE_SETTLS) + childregs->r13 = tls; + } + p->thread.kernel_sp = + (unsigned long) (task_stack_page(p) + THREAD_SIZE); + p->thread.ctx_switch.sp = (unsigned long) childregs; + + return 0; +} + +void release_thread(struct task_struct *dead_task) +{ +} + +void flush_thread(void) +{ +} + +/* Fill in the fpu structure for a core dump. */ +int dump_fpu(struct pt_regs *regs, elf_fpregset_t *fpu) +{ + /* + * On kvx, FPU uses standard registers + $cs which is a common register + * also needed for non-fpu execution, so there is no additional + * register to dump. + */ + return 0; +} + +static bool find_wchan(unsigned long pc, void *arg) +{ + unsigned long *p = arg; + + /* + * If the pc is in a scheduler function (waiting), then, this is the + * address where the process is currently stuck. Note that scheduler + * functions also include lock functions. This functions are + * materialized using annotation to put them is special text sections. + */ + if (!in_sched_functions(pc)) { + *p = pc; + return true; + } + + return false; +} + +/* + * __get_wchan is called to obtain "schedule()" caller function address. + */ +unsigned long __get_wchan(struct task_struct *p) +{ + unsigned long pc = 0; + struct stackframe frame; + + /* + * We need to obtain the task stack since we don't want the stack to + * move under our feet. + */ + if (!try_get_task_stack(p)) + return 0; + + start_stackframe(&frame, thread_saved_reg(p, fp), + thread_saved_reg(p, ra)); + walk_stackframe(p, &frame, find_wchan, &pc); + + put_task_stack(p); + + return pc; +} + diff --git a/arch/kvx/kernel/ptrace.c b/arch/kvx/kernel/ptrace.c new file mode 100644 index 000000000000..aacae5883cb5 --- /dev/null +++ b/arch/kvx/kernel/ptrace.c @@ -0,0 +1,276 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * derived from arch/riscv/kernel/ptrace.c + * + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Marius Gligor + * Clement Leger + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include + +#define CREATE_TRACE_POINTS +#include + +#define HW_PT_CMD_GET_CAPS 0 +#define HW_PT_CMD_GET_PT 1 +#define HW_PT_CMD_SET_RESERVE 0 +#define HW_PT_CMD_SET_ENABLE 1 + +#define FROM_GDB_CMD_MASK 3 +#define FROM_GDB_HP_TYPE_SHIFT 2 +#define FROM_GDB_HP_TYPE_MASK 4 +#define FROM_GDB_WP_TYPE_SHIFT 3 +#define FROM_GDB_WP_TYPE_MASK 0x18 +#define FROM_GDB_HP_IDX_SHIFT 5 + +#define hw_pt_cmd(addr) ((addr) & FROM_GDB_CMD_MASK) +#define hw_pt_is_bkp(addr) ((((addr) & FROM_GDB_HP_TYPE_MASK) >> \ + FROM_GDB_HP_TYPE_SHIFT) == KVX_HW_BREAKPOINT_TYPE) +#define get_hw_pt_wp_type(addr) ((((addr) & FROM_GDB_WP_TYPE_MASK)) >> \ + FROM_GDB_WP_TYPE_SHIFT) +#define get_hw_pt_idx(addr) ((addr) >> FROM_GDB_HP_IDX_SHIFT) +#define get_hw_pt_addr(data) ((data)[0]) +#define get_hw_pt_len(data) ((data)[1] >> 1) +#define hw_pt_is_enabled(data) ((data)[1] & 1) + +enum kvx_regset { + REGSET_GPR, +#ifdef CONFIG_ENABLE_TCA + REGSET_TCA, +#endif +}; + +void ptrace_disable(struct task_struct *child) +{ + /* nothing to do */ +} + +static int kvx_gpr_get(struct task_struct *target, + const struct user_regset *regset, + struct membuf to) +{ + struct user_pt_regs *regs = &task_pt_regs(target)->user_regs; + + return membuf_write(&to, regs, sizeof(*regs)); +} + +static int kvx_gpr_set(struct task_struct *target, + const struct user_regset *regset, + unsigned int pos, unsigned int count, + const void *kbuf, const void __user *ubuf) +{ + struct user_pt_regs *regs = &task_pt_regs(target)->user_regs; + + return user_regset_copyin(&pos, &count, &kbuf, &ubuf, regs, 0, -1); +} + +#ifdef CONFIG_ENABLE_TCA +static int kvx_tca_reg_get(struct task_struct *target, + const struct user_regset *regset, + struct membuf to) +{ + struct ctx_switch_regs *ctx_regs = &target->thread.ctx_switch; + struct tca_reg *regs = ctx_regs->tca_regs; + int ret; + + if (!ctx_regs->tca_regs_saved) + ret = membuf_zero(&to, sizeof(*regs)); + else + ret = membuf_write(&to, regs, sizeof(*regs)); + + return ret; +} + +static int kvx_tca_reg_set(struct task_struct *target, + const struct user_regset *regset, + unsigned int pos, unsigned int count, + const void *kbuf, const void __user *ubuf) +{ + struct ctx_switch_regs *ctx_regs = &target->thread.ctx_switch; + struct tca_reg *regs = ctx_regs->tca_regs; + int ret; + + if (!ctx_regs->tca_regs_saved) + ret = user_regset_copyin_ignore(&pos, &count, &kbuf, &ubuf, + 0, -1); + else + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, regs, + 0, -1); + + return ret; +} +#endif + +static const struct user_regset kvx_user_regset[] = { + [REGSET_GPR] = { + .core_note_type = NT_PRSTATUS, + .n = ELF_NGREG, + .size = sizeof(elf_greg_t), + .align = sizeof(elf_greg_t), + .regset_get = &kvx_gpr_get, + .set = &kvx_gpr_set, + }, +#ifdef CONFIG_ENABLE_TCA + [REGSET_TCA] = { + .core_note_type = NT_KVX_TCA, + .n = TCA_REG_COUNT, + .size = sizeof(struct tca_reg), + .align = sizeof(struct tca_reg), + .regset_get = &kvx_tca_reg_get, + .set = &kvx_tca_reg_set, + }, +#endif +}; + +static const struct user_regset_view user_kvx_view = { + .name = "kvx", + .e_machine = EM_KVX, + .regsets = kvx_user_regset, + .n = ARRAY_SIZE(kvx_user_regset) +}; + +const struct user_regset_view *task_user_regset_view(struct task_struct *task) +{ + return &user_kvx_view; +} + +long arch_ptrace(struct task_struct *child, long request, + unsigned long addr, unsigned long data) +{ + return ptrace_request(child, request, addr, data); +} + +/* + * Allows PTRACE_SYSCALL to work. These are called from entry.S in + * {handle,ret_from}_syscall. + */ +int do_syscall_trace_enter(struct pt_regs *regs, unsigned long syscall) +{ + int ret = 0; + +#ifdef CONFIG_CONTEXT_TRACKING_USER + user_exit_callable(); +#endif + if (test_thread_flag(TIF_SYSCALL_TRACE)) + ret = ptrace_report_syscall_entry(regs); + +#ifdef CONFIG_HAVE_SYSCALL_TRACEPOINTS + if (test_thread_flag(TIF_SYSCALL_TRACEPOINT)) + trace_sys_enter(regs, syscall_get_nr(current, regs)); +#endif + + audit_syscall_entry(syscall, regs->r0, regs->r1, regs->r2, regs->r3); + + return ret; +} + +void do_syscall_trace_exit(struct pt_regs *regs) +{ + if (test_thread_flag(TIF_SYSCALL_TRACE)) + ptrace_report_syscall_exit(regs, 0); + + audit_syscall_exit(regs); + +#ifdef CONFIG_HAVE_SYSCALL_TRACEPOINTS + if (test_thread_flag(TIF_SYSCALL_TRACEPOINT)) + trace_sys_exit(regs, regs_return_value(regs)); +#endif + +#ifdef CONFIG_CONTEXT_TRACKING_USER + user_enter_callable(); +#endif +} + +static int kvx_bkpt_handler(struct break_hook *brk_hook, struct pt_regs *regs) +{ + /* Unexpected breakpoint */ + if (!(current->ptrace & PT_PTRACED)) + return BREAK_HOOK_ERROR; + + /* deliver the signal to userspace */ + force_sig_fault(SIGTRAP, TRAP_BRKPT, (void __user *) regs->spc); + + return BREAK_HOOK_HANDLED; +} + +static void kvx_stepi(struct pt_regs *regs) +{ + /* deliver the signal to userspace */ + force_sig_fault(SIGTRAP, TRAP_TRACE, (void __user *) regs->spc); +} + +void user_enable_single_step(struct task_struct *child) +{ + struct pt_regs *regs = task_pt_regs(child); + + enable_single_step(regs); +} + +void user_disable_single_step(struct task_struct *child) +{ + struct pt_regs *regs = task_pt_regs(child); + + disable_single_step(regs); +} + +/** + * Main debug handler called by the _debug_handler routine in entry.S + * This handler will perform the required action + * @es: Exception Syndrome register value + * @ea: Exception Address register + * @regs: pointer to registers saved when enter debug + */ +int ptrace_debug_handler(u64 ea, struct pt_regs *regs) +{ + + int debug_cause = debug_dc(regs->es); + + switch (debug_cause) { + case DEBUG_CAUSE_STEPI: + kvx_stepi(regs); + break; + default: + break; + } + + return DEBUG_HOOK_HANDLED; +} + +static struct debug_hook ptrace_debug_hook = { + .handler = ptrace_debug_handler, + .mode = MODE_USER, +}; + +static struct break_hook bkpt_break_hook = { + .id = BREAK_CAUSE_BKPT, + .handler = kvx_bkpt_handler, + .mode = MODE_USER, +}; + +static int __init arch_init_breakpoint(void) +{ + break_hook_register(&bkpt_break_hook); + debug_hook_register(&ptrace_debug_hook); + + return 0; +} + +postcore_initcall(arch_init_breakpoint); diff --git a/arch/kvx/kernel/stacktrace.c b/arch/kvx/kernel/stacktrace.c new file mode 100644 index 000000000000..85d52ba2d082 --- /dev/null +++ b/arch/kvx/kernel/stacktrace.c @@ -0,0 +1,173 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Vincent Chardon + */ + +#include +#include +#include +#include + +#include +#include + +#define STACK_SLOT_PER_LINE 4 +#define STACK_MAX_SLOT_PRINT (STACK_SLOT_PER_LINE * 8) + +static int notrace unwind_frame(struct task_struct *task, + struct stackframe *frame) +{ + unsigned long fp = frame->fp; + + /* Frame pointer must be aligned on 8 bytes */ + if (fp & 0x7) + return -EINVAL; + + if (!task) + task = current; + + if (!on_task_stack(task, fp)) + return -EINVAL; + + frame->fp = READ_ONCE_NOCHECK(*(unsigned long *)(fp)); + frame->ra = READ_ONCE_NOCHECK(*(unsigned long *)(fp + 8)); + + /* + * When starting, we set the frame pointer to 0, hence end of + * frame linked list is signal by that + */ + if (!frame->fp) + return -EINVAL; + + return 0; +} + +void notrace walk_stackframe(struct task_struct *task, struct stackframe *frame, + bool (*fn)(unsigned long, void *), void *arg) +{ + unsigned long addr; + int ret; + + while (1) { + addr = frame->ra; + + if (fn(addr, arg)) + break; + + ret = unwind_frame(task, frame); + if (ret) + break; + } +} + +#ifdef CONFIG_STACKTRACE +bool append_stack_addr(unsigned long pc, void *arg) +{ + struct stack_trace *trace; + + trace = (struct stack_trace *)arg; + if (trace->skip == 0) { + trace->entries[trace->nr_entries++] = pc; + if (trace->nr_entries == trace->max_entries) + return true; + } else { + trace->skip--; + } + + return false; +} + +/* + * Save stack-backtrace addresses into a stack_trace buffer. + */ +void save_stack_trace(struct stack_trace *trace) +{ + struct stackframe frame; + + trace->nr_entries = 0; + /* We want to skip this function and the caller */ + trace->skip += 2; + + start_stackframe(&frame, (unsigned long) __builtin_frame_address(0), + (unsigned long) save_stack_trace); + walk_stackframe(current, &frame, append_stack_addr, trace); +} +EXPORT_SYMBOL(save_stack_trace); +#endif /* CONFIG_STACKTRACE */ + +static bool print_pc(unsigned long pc, void *arg) +{ + unsigned long *skip = arg; + + if (*skip == 0) + print_ip_sym(KERN_INFO, pc); + else + (*skip)--; + + return false; +} + +void show_stacktrace(struct task_struct *task, struct pt_regs *regs) +{ + struct stackframe frame; + unsigned long skip = 0; + + /* Obviously, we can't backtrace on usermode ! */ + if (regs && user_mode(regs)) + return; + + if (!task) + task = current; + + if (!try_get_task_stack(task)) + return; + + if (regs) { + start_stackframe(&frame, regs->fp, regs->spc); + } else if (task == current) { + /* Skip current function and caller */ + skip = 2; + start_stackframe(&frame, + (unsigned long) __builtin_frame_address(0), + (unsigned long) show_stacktrace); + } else { + /* task blocked in __switch_to */ + start_stackframe(&frame, + thread_saved_reg(task, fp), + thread_saved_reg(task, ra)); + } + + pr_info("Call Trace:\n"); + walk_stackframe(task, &frame, print_pc, &skip); + + put_task_stack(task); +} + +/* + * If show_stack is called with a non-null task, then the task will have been + * claimed with try_get_task_stack by the caller. If task is NULL or current + * then there is no need to get task stack since it's our current stack... + */ +void show_stack(struct task_struct *task, unsigned long *sp) +{ + int i = 0; + + if (!sp) + sp = (unsigned long *) get_current_sp(); + + pr_info("Stack dump (@%p):\n", sp); + for (i = 0; i < STACK_MAX_SLOT_PRINT; i++) { + if (kstack_end(sp)) + break; + + if (i && (i % STACK_SLOT_PER_LINE) == 0) + pr_cont("\n\t"); + + pr_cont("%016lx ", *sp++); + } + pr_cont("\n"); + + show_stacktrace(task, NULL); +} From patchwork Fri Jan 20 14:09:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13110022 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88919C05027 for ; Fri, 20 Jan 2023 14:36:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231260AbjATOgB (ORCPT ); Fri, 20 Jan 2023 09:36:01 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38354 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231202AbjATOfv (ORCPT ); Fri, 20 Jan 2023 09:35:51 -0500 Received: from fx306.security-mail.net (smtpout30.security-mail.net [85.31.212.36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2479880892 for ; Fri, 20 Jan 2023 06:35:04 -0800 (PST) Received: from localhost (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id AE94435CF54 for ; Fri, 20 Jan 2023 15:20:46 +0100 (CET) Received: from fx306 (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id DF79B35CDF8; Fri, 20 Jan 2023 15:20:45 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx306.security-mail.net (Postfix) with ESMTPS id 274F235CD14; Fri, 20 Jan 2023 15:20:45 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 8D93B27E043D; Fri, 20 Jan 2023 15:10:34 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 525A227E0442; Fri, 20 Jan 2023 15:10:34 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id fP9ew3IpTsDK; Fri, 20 Jan 2023 15:10:34 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id D9D9227E043D; Fri, 20 Jan 2023 15:10:33 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <4023.63caa33d.26329.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 525A227E0442 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223834; bh=aXD2STzsmn5SV1hfOdG/cXdwsPHfP17MTko4dr9Gtjo=; h=From:To:Date:Message-Id:MIME-Version; b=WralGqMba+8KbYihsOnUPw91xmkTXXSm6esg3i+xiykENsFIH/hBAV3cxTbS6+AT5 blVaZ9KQFidgY4WTBZNKj7Olz36kJKylp0IdN7MePkn8hRwTd7XyN9y/C578tKPMZA dqOMB7oShxaiqCqn2vBXfoTNlsasV0Yu3mNLULJg= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 21/31] kvx: Add system call support Date: Fri, 20 Jan 2023 15:09:52 +0100 Message-ID: <20230120141002.2442-22-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Add system call support and related uaccess.h for kvx. Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Guillaume Thouvenin Signed-off-by: Guillaume Thouvenin Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Julien Villette Signed-off-by: Julien Villette Co-developed-by: Marius Gligor Signed-off-by: Marius Gligor Co-developed-by: Yann Sionneau Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: - typo fixes (clober* -> clobber*) - use generic __access_ok arch/kvx/include/asm/syscall.h | 73 ++ arch/kvx/include/asm/syscalls.h | 21 + arch/kvx/include/asm/uaccess.h | 317 +++++ arch/kvx/include/asm/unistd.h | 11 + arch/kvx/include/uapi/asm/cachectl.h | 25 + arch/kvx/include/uapi/asm/unistd.h | 16 + arch/kvx/kernel/entry.S | 1759 ++++++++++++++++++++++++++ arch/kvx/kernel/sys_kvx.c | 58 + arch/kvx/kernel/syscall_table.c | 19 + 9 files changed, 2299 insertions(+) create mode 100644 arch/kvx/include/asm/syscall.h create mode 100644 arch/kvx/include/asm/syscalls.h create mode 100644 arch/kvx/include/asm/uaccess.h create mode 100644 arch/kvx/include/asm/unistd.h create mode 100644 arch/kvx/include/uapi/asm/cachectl.h create mode 100644 arch/kvx/include/uapi/asm/unistd.h create mode 100644 arch/kvx/kernel/entry.S create mode 100644 arch/kvx/kernel/sys_kvx.c create mode 100644 arch/kvx/kernel/syscall_table.c diff --git a/arch/kvx/include/asm/syscall.h b/arch/kvx/include/asm/syscall.h new file mode 100644 index 000000000000..a3f6cef73e4a --- /dev/null +++ b/arch/kvx/include/asm/syscall.h @@ -0,0 +1,73 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_SYSCALL_H +#define _ASM_KVX_SYSCALL_H + +#include +#include + +#include + +/* The array of function pointers for syscalls. */ +extern void *sys_call_table[]; + +void scall_machine_exit(unsigned char value); + +/** + * syscall_get_nr - find which system call a task is executing + * @task: task of interest, must be blocked + * @regs: task_pt_regs() of @task + * + * If @task is executing a system call or is at system call + * tracing about to attempt one, returns the system call number. + * If @task is not executing a system call, i.e. it's blocked + * inside the kernel for a fault or signal, returns -1. + * + * Note this returns int even on 64-bit machines. Only 32 bits of + * system call number can be meaningful. If the actual arch value + * is 64 bits, this truncates to 32 bits so 0xffffffff means -1. + * + * It's only valid to call this when @task is known to be blocked. + */ +static inline int syscall_get_nr(struct task_struct *task, struct pt_regs *regs) +{ + if (!in_syscall(regs)) + return -1; + + return es_sysno(regs); +} + +static inline long syscall_get_error(struct task_struct *task, + struct pt_regs *regs) +{ + /* 0 if syscall succeeded, otherwise -Errorcode */ + return IS_ERR_VALUE(regs->r0) ? regs->r0 : 0; +} + +static inline long syscall_get_return_value(struct task_struct *task, + struct pt_regs *regs) +{ + return regs->r0; +} + +static inline int syscall_get_arch(struct task_struct *task) +{ + return AUDIT_ARCH_KVX; +} + +static inline void syscall_get_arguments(struct task_struct *task, + struct pt_regs *regs, + unsigned long *args) +{ + args[0] = regs->orig_r0; + args++; + memcpy(args, ®s->r1, 5 * sizeof(args[0])); +} + +int __init setup_syscall_sigreturn_page(void *sigpage_addr); + +#endif diff --git a/arch/kvx/include/asm/syscalls.h b/arch/kvx/include/asm/syscalls.h new file mode 100644 index 000000000000..beec95ebb97a --- /dev/null +++ b/arch/kvx/include/asm/syscalls.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_SYSCALLS_H +#define _ASM_KVX_SYSCALLS_H + +#include + +/* We redefine clone in assembly for special slowpath */ +asmlinkage long __sys_clone(unsigned long clone_flags, unsigned long newsp, + int __user *parent_tid, int __user *child_tid, int tls); + +#define sys_clone __sys_clone + +long sys_cachectl(unsigned long addr, unsigned long len, unsigned long cache, + unsigned long flags); + +#endif diff --git a/arch/kvx/include/asm/uaccess.h b/arch/kvx/include/asm/uaccess.h new file mode 100644 index 000000000000..24f91d75c1dd --- /dev/null +++ b/arch/kvx/include/asm/uaccess.h @@ -0,0 +1,317 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * derived from arch/riscv/include/asm/uaccess.h + * + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + */ + +#ifndef _ASM_KVX_UACCESS_H +#define _ASM_KVX_UACCESS_H + +#include +#include + +/** + * access_ok: - Checks if a user space pointer is valid + * @addr: User space pointer to start of block to check + * @size: Size of block to check + * + * Context: User context only. This function may sleep. + * + * Checks if a pointer to a block of memory in user space is valid. + * + * Returns true (nonzero) if the memory block may be valid, false (zero) + * if it is definitely invalid. + * + * Note that, depending on architecture, this function probably just + * checks that the pointer is in the user space range - after calling + * this function, memory access functions may still return -EFAULT. + */ +#define access_ok(addr, size) ({ \ + __chk_user_ptr(addr); \ + likely(__access_ok((addr), (size))); \ +}) + +#include + +/* + * The exception table consists of pairs of addresses: the first is the + * address of an instruction that is allowed to fault, and the second is + * the address at which the program should continue. No registers are + * modified, so it is entirely up to the continuation code to figure out + * what to do. + * + * All the routines below use bits of fixup code that are out of line + * with the main instruction path. This means when everything is well, + * we don't even have to jump over them. Further, they do not intrude + * on our cache or TLB entries. + */ + +struct exception_table_entry { + unsigned long insn, fixup; +}; + +extern int fixup_exception(struct pt_regs *regs); + +/** + * Assembly defined function (usercopy.S) + */ +extern unsigned long +raw_copy_from_user(void *to, const void __user *from, unsigned long n); + +extern unsigned long +raw_copy_to_user(void __user *to, const void *from, unsigned long n); + +extern unsigned long +asm_clear_user(void __user *to, unsigned long n); + +#define __clear_user asm_clear_user + +static inline __must_check unsigned long +clear_user(void __user *to, unsigned long n) +{ + might_fault(); + if (!access_ok(to, n)) + return n; + + return asm_clear_user(to, n); +} + +extern __must_check long strnlen_user(const char __user *str, long n); +extern long strncpy_from_user(char *dest, const char __user *src, long count); + +#define __enable_user_access() +#define __disable_user_access() + +/** + * get_user: - Get a simple variable from user space. + * @x: Variable to store result. + * @ptr: Source address, in user space. + * + * Context: User context only. This function may sleep. + * + * This macro copies a single simple variable from user space to kernel + * space. It supports simple types like char and int, but not larger + * data types like structures or arrays. + * + * @ptr must have pointer-to-simple-variable type, and the result of + * dereferencing @ptr must be assignable to @x without a cast. + * + * Returns zero on success, or -EFAULT on error. + * On error, the variable @x is set to zero. + */ +#define get_user(x, ptr) \ +({ \ + long __e = -EFAULT; \ + const __typeof__(*(ptr)) __user *__p = (ptr); \ + might_fault(); \ + if (likely(access_ok(__p, sizeof(*__p)))) { \ + __e = __get_user(x, __p); \ + } else { \ + x = 0; \ + } \ + __e; \ +}) + +/** + * __get_user: - Get a simple variable from user space, with less checking. + * @x: Variable to store result. + * @ptr: Source address, in user space. + * + * Context: User context only. This function may sleep. + * + * This macro copies a single simple variable from user space to kernel + * space. It supports simple types like char and int, but not larger + * data types like structures or arrays. + * + * @ptr must have pointer-to-simple-variable type, and the result of + * dereferencing @ptr must be assignable to @x without a cast. + * + * Caller must check the pointer with access_ok() before calling this + * function. + * + * Returns zero on success, or -EFAULT on error. + * On error, the variable @x is set to zero. + */ +#define __get_user(x, ptr) \ +({ \ + unsigned long __err = 0; \ + __chk_user_ptr(ptr); \ + \ + __enable_user_access(); \ + __get_user_nocheck(x, ptr, __err); \ + __disable_user_access(); \ + \ + __err; \ +}) + +#define __get_user_nocheck(x, ptr, err) \ +do { \ + unsigned long __gu_addr = (unsigned long)(ptr); \ + unsigned long __gu_val; \ + switch (sizeof(*(ptr))) { \ + case 1: \ + __get_user_asm("lbz", __gu_val, __gu_addr, err); \ + break; \ + case 2: \ + __get_user_asm("lhz", __gu_val, __gu_addr, err); \ + break; \ + case 4: \ + __get_user_asm("lwz", __gu_val, __gu_addr, err); \ + break; \ + case 8: \ + __get_user_asm("ld", __gu_val, __gu_addr, err); \ + break; \ + default: \ + BUILD_BUG(); \ + } \ + (x) = (__typeof__(*(ptr)))__gu_val; \ +} while (0) + +#define __get_user_asm(op, x, addr, err) \ +({ \ + __asm__ __volatile__( \ + "1: "op" %1 = 0[%2]\n" \ + " ;;\n" \ + "2:\n" \ + ".section .fixup,\"ax\"\n" \ + "3: make %0 = 2b\n" \ + " make %1 = 0\n" \ + " ;;\n" \ + " make %0 = %3\n" \ + " igoto %0\n" \ + " ;;\n" \ + ".previous\n" \ + ".section __ex_table,\"a\"\n" \ + " .align 8\n" \ + " .dword 1b,3b\n" \ + ".previous" \ + : "=r"(err), "=r"(x) \ + : "r"(addr), "i"(-EFAULT), "0"(err)); \ +}) + +/** + * put_user: - Write a simple value into user space. + * @x: Value to copy to user space. + * @ptr: Destination address, in user space. + * + * Context: User context only. This function may sleep. + * + * This macro copies a single simple value from kernel space to user + * space. It supports simple types like char and int, but not larger + * data types like structures or arrays. + * + * @ptr must have pointer-to-simple-variable type, and @x must be assignable + * to the result of dereferencing @ptr. + * + * Returns zero on success, or -EFAULT on error. + */ +#define put_user(x, ptr) \ +({ \ + long __e = -EFAULT; \ + __typeof__(*(ptr)) __user *__p = (ptr); \ + might_fault(); \ + if (likely(access_ok(__p, sizeof(*__p)))) { \ + __e = __put_user(x, __p); \ + } \ + __e; \ +}) + +/** + * __put_user: - Write a simple value into user space, with less checking. + * @x: Value to copy to user space. + * @ptr: Destination address, in user space. + * + * Context: User context only. This function may sleep. + * + * This macro copies a single simple value from kernel space to user + * space. It supports simple types like char and int, but not larger + * data types like structures or arrays. + * + * @ptr must have pointer-to-simple-variable type, and @x must be assignable + * to the result of dereferencing @ptr. + * + * Caller must check the pointer with access_ok() before calling this + * function. + * + * Returns zero on success, or -EFAULT on error. + */ +#define __put_user(x, ptr) \ +({ \ + unsigned long __err = 0; \ + __chk_user_ptr(ptr); \ + \ + __enable_user_access(); \ + __put_user_nocheck(x, ptr, __err); \ + __disable_user_access(); \ + \ + __err; \ +}) + +#define __put_user_nocheck(x, ptr, err) \ +do { \ + unsigned long __pu_addr = (unsigned long)(ptr); \ + __typeof__(*(ptr)) __pu_val = (x); \ + switch (sizeof(*(ptr))) { \ + case 1: \ + __put_user_asm("sb", __pu_val, __pu_addr, err); \ + break; \ + case 2: \ + __put_user_asm("sh", __pu_val, __pu_addr, err); \ + break; \ + case 4: \ + __put_user_asm("sw", __pu_val, __pu_addr, err); \ + break; \ + case 8: \ + __put_user_asm("sd", __pu_val, __pu_addr, err); \ + break; \ + default: \ + BUILD_BUG(); \ + } \ +} while (0) + +#define __put_user_asm(op, x, addr, err) \ +({ \ + __asm__ __volatile__( \ + "1: "op" 0[%2], %1\n" \ + " ;;\n" \ + "2:\n" \ + ".section .fixup,\"ax\"\n" \ + "3: make %0 = 2b\n" \ + " ;;\n" \ + " make %0 = %3\n" \ + " igoto %0\n" \ + " ;;\n" \ + ".previous\n" \ + ".section __ex_table,\"a\"\n" \ + " .align 8\n" \ + " .dword 1b,3b\n" \ + ".previous" \ + : "=r"(err) \ + : "r"(x), "r"(addr), "i"(-EFAULT), "0"(err)); \ +}) + +#define HAVE_GET_KERNEL_NOFAULT + +#define __get_kernel_nofault(dst, src, type, err_label) \ +do { \ + long __kr_err; \ + \ + __get_user_nocheck(*((type *)(dst)), (type *)(src), __kr_err); \ + if (unlikely(__kr_err)) \ + goto err_label; \ +} while (0) + +#define __put_kernel_nofault(dst, src, type, err_label) \ +do { \ + long __kr_err; \ + \ + __put_user_nocheck(*((type *)(src)), (type *)(dst), __kr_err); \ + if (unlikely(__kr_err)) \ + goto err_label; \ +} while (0) + + +#endif /* _ASM_KVX_UACCESS_H */ diff --git a/arch/kvx/include/asm/unistd.h b/arch/kvx/include/asm/unistd.h new file mode 100644 index 000000000000..6cd093dbf2d8 --- /dev/null +++ b/arch/kvx/include/asm/unistd.h @@ -0,0 +1,11 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#define __ARCH_WANT_SYS_CLONE + +#include + +#define NR_syscalls (__NR_syscalls) diff --git a/arch/kvx/include/uapi/asm/cachectl.h b/arch/kvx/include/uapi/asm/cachectl.h new file mode 100644 index 000000000000..be0a1aa23cf6 --- /dev/null +++ b/arch/kvx/include/uapi/asm/cachectl.h @@ -0,0 +1,25 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _UAPI_ASM_KVX_CACHECTL_H +#define _UAPI_ASM_KVX_CACHECTL_H + +/* + * Cache type for cachectl system call + */ +#define CACHECTL_CACHE_DCACHE (1 << 0) + +/* + * Flags for cachectl system call + */ +#define CACHECTL_FLAG_OP_INVAL (1 << 0) +#define CACHECTL_FLAG_OP_WB (1 << 1) +#define CACHECTL_FLAG_OP_MASK (CACHECTL_FLAG_OP_INVAL | \ + CACHECTL_FLAG_OP_WB) + +#define CACHECTL_FLAG_ADDR_PHYS (1 << 2) + +#endif /* _UAPI_ASM_KVX_CACHECTL_H */ diff --git a/arch/kvx/include/uapi/asm/unistd.h b/arch/kvx/include/uapi/asm/unistd.h new file mode 100644 index 000000000000..5f86d81dbb76 --- /dev/null +++ b/arch/kvx/include/uapi/asm/unistd.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#define __ARCH_WANT_RENAMEAT +#define __ARCH_WANT_NEW_STAT +#define __ARCH_WANT_SET_GET_RLIMIT +#define __ARCH_WANT_SYS_CLONE3 + +#include + +/* Additional KVX specific syscalls */ +#define __NR_cachectl (__NR_arch_specific_syscall) +__SYSCALL(__NR_cachectl, sys_cachectl) diff --git a/arch/kvx/kernel/entry.S b/arch/kvx/kernel/entry.S new file mode 100644 index 000000000000..e73dc2221c06 --- /dev/null +++ b/arch/kvx/kernel/entry.S @@ -0,0 +1,1759 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + * Marius Gligor + * Yann Sionneau + * Julien Villette + */ + +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define MMU_SET_MASK ((1 << KVX_SFR_MMC_SS_WIDTH) - 1) + +/* Mask to replicate data from 32bits LSB to MSBs */ +#define REPLICATE_32_MASK 0x0804020108040201 + +#define PS_HWLOOP_ENABLE (KVX_SFR_PS_HLE_MASK << 32) +#define PS_HWLOOP_DISABLE (KVX_SFR_PS_HLE_MASK) +#define PS_HWLOOP_EN_ET_CLEAR_DAUS_DIS \ + (PS_HWLOOP_ENABLE | KVX_SFR_PS_ET_MASK | SFR_SET_VAL_WFXL(PS, DAUS, 0)) +#define PS_ET_CLEAR KVX_SFR_PS_ET_MASK +#define PS_HLE_EN_IT_EN_ET_CLEAR_DAUS_DIS \ + (SFR_SET_VAL_WFXL(PS, HLE, 1) | \ + SFR_SET_VAL_WFXL(PS, IE, 1) | \ + SFR_SET_VAL_WFXL(PS, ET, 0) | \ + SFR_SET_VAL_WFXL(PS, DAUS, 0)) + +#define PS_IT_DIS KVX_SFR_PS_IE_MASK +#define PS_IL_CLEAR KVX_SFR_PS_IL_WFXL_CLEAR + +#define MMC_CLEAR_TLB_CLEAR_WAY \ + (SFR_CLEAR_WFXL(MMC, SB) | SFR_CLEAR_WFXL(MMC, SW)) + +#define MMC_SEL_TLB_CLEAR_WAY(__tlb) \ + (SFR_SET_WFXL(MMC, SB, __tlb) | MMC_CLEAR_TLB_CLEAR_WAY) +#define MMC_SEL_JTLB_CLEAR_WAY MMC_SEL_TLB_CLEAR_WAY(MMC_SB_JTLB) +#define MMC_SEL_LTLB_CLEAR_WAY MMC_SEL_TLB_CLEAR_WAY(MMC_SB_LTLB) + +#define MME_WFXL(_enable) SFR_SET_VAL_WFXL(PS, MME, _enable) + +/* Temporary scract system register for trap handling */ +#define TMP_SCRATCH_SR $sr_pl3 + +.altmacro + +#define TEL_DEFAULT_VALUE (TLB_ES_A_MODIFIED << KVX_SFR_TEL_ES_SHIFT) + +#define TASK_THREAD_SAVE_AREA_QUAD(__q) \ + (TASK_THREAD_SAVE_AREA + (__q) * QUAD_REG_SIZE) + +#ifdef CONFIG_DEBUG_EXCEPTION_STACK +.section .rodata +stack_error_panic_str_label: + .string "Stack has been messed up !" +#endif + +#ifdef CONFIG_KVX_DEBUG_TLB_WRITE +.section .rodata +mmc_error_panic_str_label: + .string "Failed to write entry to the JTLB (in assembly refill)" +#endif + +#ifdef CONFIG_KVX_DEBUG_ASN +.section .rodata +asn_error_panic_str_label: + .string "ASN mismatch !" +#endif + +/** + * call_trace_hardirqs: hardirqs tracing call + * state: State of hardirqs to be reported (on/off) + */ +.macro call_trace_hardirqs state +#ifdef CONFIG_TRACE_IRQFLAGS + call trace_hardirqs_\state + ;; +#endif +.endm + +/** + * disable_interrupts: Disable interrupts + * tmp_reg: Temporary register to use for interrupt disabling + */ +.macro disable_interrupt tmp_reg + make \tmp_reg = KVX_SFR_PS_IE_MASK + ;; + wfxl $ps, \tmp_reg + ;; +.endm + +/** + * call_do_work_pending: Call do_work_pending and set stack argument ($r0) + * NOTE: Since do_work_pending requires thread_flags in $r1, they must + * be provided in $r1 before calling this macro. + * Moreover, do_work_pending expects interrupts to be disabled. + */ +.macro call_do_work_pending + copyd $r0 = $sp + call do_work_pending + ;; +.endm + +/** + * save_quad_regs: Save quad registers in temporary thread area + * After call, the quad is saved in task_struct.thread.save_area. + * sr_swap_reg is only used as a temporary value and will be + * restored after returning from this macro + * + * quad: Quad to saved + * sr_swap_reg: Register used for sr swap and quad saving. + */ +.macro save_quad_regs quad sr_swap_reg + rswap \sr_swap_reg = $sr + ;; + /* Save registers in save_area */ + so __PA(TASK_THREAD_SAVE_AREA_QUAD(0))[\sr_swap_reg] = \quad + /* Restore register */ + rswap \sr_swap_reg = $sr + ;; +.endm + +/** + * Switch task when entering kernel if needed. + * sp_reg: register which will be used to store original stack pointer when + * entering the kernel + * task_reg: is a register containing the current task pointer + * save_regs_label: label to jump to save register immediately. This label is + * used when we are already in kernel execution context. + * NOTE: when returning from this macro, we consider that the assembly following + * it will be executed only when coming from user space + */ +.macro switch_stack sp_reg task_reg save_regs_label + get \sp_reg = $sps + ;; + /* Check if $sps.pl bit is 0 (ie in kernel mode) + * if so, then we dont have to switch stack */ + cb.even \sp_reg? \save_regs_label + /* Copy original $sp in a scratch reg */ + copyd \sp_reg = $sp + ;; + /* restore sp from kernel stack pointer */ + ld $sp = __PA(TASK_THREAD_KERNEL_SP)[\task_reg] + ;; +.endm + +/** + * Disable MMU using a specified register + * scratch_reg: Scratch register to be used for $ps modification (clobbered) + * ps_val: Additionnal $ps modifications after returning disabling MMU + */ +.macro disable_mmu scratch_reg ps_val=0 + /* Disable MME in $sps */ + make \scratch_reg = MME_WFXL(0) + ;; + wfxl $sps = \scratch_reg + ;; + /* Enable DAUS in $ps */ + make \scratch_reg = SFR_SET_VAL_WFXL(PS, DAUS, 1) | \ps_val + ;; + wfxl $ps = \scratch_reg + /* We are now accessing data with MMU disabled */ + ;; +.endm + +/** + * Disable MMU when entering exception path + * This macro does not require any register since used register will be + * restored after use. This can safely be used directly after entering + * exceptions. + */ +.macro exception_entry_disable_mmu + set TMP_SCRATCH_SR = $r0 + disable_mmu $r0 + ;; + get $r0 = TMP_SCRATCH_SR + ;; +.endm + +/* Save callee registers on stack */ +.macro save_callee + sq PT_R18R19[$sp] = $r18r19 + ;; + so PT_Q20[$sp] = $r20r21r22r23 + ;; + so PT_Q24[$sp] = $r24r25r26r27 + ;; + so PT_Q28[$sp] = $r28r29r30r31 + ;; +.endm + +/** + * Save registers for entry in kernel space. + * sp_reg should point to the original stack pointer when entering kernel space + * task_reg is a register containing the current task pointer + * both task_reg and sp_reg must be in saved_quad since they have been + * clobberred and will be restored when restoring saved_quad. + * pt_regs_sp is a single register which will be filled by pt_regs addr. + * When "returning" from this macro, hardware loop are enabled and exception + * is cleared, allowing to call kernel function without any risk. + */ +.macro save_regs_for_exception sp_reg task_reg saved_quad pt_regs_sp +.if (\saved_quad==$r4r5r6r7 || \saved_quad==$r60r61r62r63) +.error "saved_quad must not be $r4r5r6r7 or $r60r61r62r63 !" +.endif + /* make some room on stack to save registers */ + addd $sp = $sp, -(PT_SIZE_ON_STACK) + so __PA(PT_Q4-PT_SIZE_ON_STACK)[$sp] = $r4r5r6r7 + /* Compute physical address of stack to avoid using 64bits immediate */ + addd $r6 = $sp, VA_TO_PA_OFFSET - (PT_SIZE_ON_STACK) + ;; + so PT_Q60[$r6] = $r60r61r62r63 + /* Now that $r60r61r62r63 is saved, we can use it for saving + * original stack stored in scratch_reg. Note that we can not + * use the r12r13r14r15 quad to do that because it would + * modify the current $r12/sp ! This is if course not what we + * want and hence we use the freshly saved quad $r60r61r62r63. + * + * Note that we must use scratch_reg before reloading the saved + * quad since the scratch reg is contained in it, so reloading + * it before copying it would overwrite it. + */ + copyd $r60 = \sp_reg + ;; + /* Reload the saved quad registers to save correct values + * Since we use the scratch reg before that */ + lo \saved_quad = __PA(TASK_THREAD_SAVE_AREA_QUAD(0))[\task_reg] + ;; + so PT_Q8[$r6] = $r8r9r10r11 + ;; + so PT_Q0[$r6] = $r0r1r2r3 + copyd $r61 = $r13 + get $r5 = $le + ;; + sq PT_R16R17[$r6] = $r16r17 + make $r10 = 0x0 + copyd $r62 = $r14 + ;; + so PT_Q32[$r6] = $r32r33r34r35 + /* Since we are going to enable hardware loop, we must be careful + * and reset le (loop exit) to avoid any exploit and return to + * user with kernel mode */ + set $le = $r10 + copyd $r63 = $r15 + ;; + so PT_Q36[$r6] = $r36r37r38r39 + get $r0 = $cs + ;; + so PT_Q40[$r6] = $r40r41r42r43 + get $r1 = $spc + ;; + so PT_Q44[$r6] = $r44r45r46r47 + get $r2 = $sps + ;; + so PT_Q48[$r6] = $r48r49r50r51 + get $r3 = $es + ;; + so PT_Q52[$r6] = $r52r53r54r55 + get $r7 = $ra + ;; + so PT_Q56[$r6] = $r56r57r58r59 + get $r4 = $lc + ;; + so PT_Q12[$r6] = $r60r61r62r63 + copyd $r63 = $r6 + get $r6 = $ls + ;; + so PT_CS_SPC_SPS_ES[$r63] = $r0r1r2r3 + ;; + so PT_LC_LE_LS_RA[$r63] = $r4r5r6r7 + /* Clear frame pointer */ + make $fp = 0 + ;; + /* Copy regs stack pointer for macro caller */ + copyd \pt_regs_sp = $sp +#ifdef CONFIG_DEBUG_EXCEPTION_STACK + addd $sp = $sp, -STACK_REG_SIZE + ;; + sd VA_TO_PA_OFFSET[$sp] = $sp + ;; +#endif + /* Reenable hwloop, MMU and clear exception taken */ + make $r8 = PS_HWLOOP_EN_ET_CLEAR_DAUS_DIS + ;; + wfxl $ps, $r8 + ;; +.endm + +/*********************************************************************** +* Exception vectors trampolines +***********************************************************************/ +#define exception_trampoline(__type) \ +.section .exception.## __type, "ax", @progbits ;\ +ENTRY(kvx_##__type ##_handler_trampoline): ;\ + goto kvx_## __type ##_handler ;\ + ;; ;\ +ENDPROC(kvx_ ## __type ## _handler_trampoline) ;\ +.section .early_exception.## __type, "ax", @progbits ;\ +ENTRY(kvx_## __type ##_early_handler): ;\ +1: nop ;\ + ;; ;\ + goto 1b ;\ + ;; ;\ +ENDPROC(kvx_ ## __type ## _early_handler) + +exception_trampoline(debug) +exception_trampoline(trap) +exception_trampoline(interrupt) +exception_trampoline(syscall) + +#define EXCEPTION_ENTRY(__name) \ +.section .exception.text, "ax", @progbits ;\ +ENTRY(__name) + + +/*********************************************************************** +* Common exception return path +***********************************************************************/ +/** + * Restore registers after exception + * When entering this macro, $sp must be located right before regs + * storage. + */ +EXCEPTION_ENTRY(return_from_exception) +#ifdef CONFIG_DEBUG_EXCEPTION_STACK + ld $r1 = 0[$sp] + ;; + sbfd $r1 = $r1, $sp + ;; + cb.deqz $r1, _check_ok + ;; + make $r2 = panic + make $r0 = stack_error_panic_str_label + ;; + icall $r2 + ;; +_check_ok: + addd $sp = $sp, STACK_REG_SIZE + ;; +#endif + get $r11 = $sr + /* Load sps value from saved registers */ + ld $r6 = PT_SPS[$sp] + ;; + /* Disable interrupt to check task flags atomically */ + disable_interrupt $r60 + ;; + /* Check PL bit of sps, if set, then it means we are returning + * to a lower privilege level (ie to user), if so, we need to + * check work pending. If coming from kernel, directly go to + * register restoration */ + cb.even $r6? _restore_regs + ld $r1 = TASK_TI_FLAGS[$r11] + ;; + /* Do we have work pending ? */ + andd $r5 = $r1, _TIF_WORK_MASK + ;; + /** + * If we do not have work pending (ie $r5 == 0) then we can + * directly jump to _restore_regs without calling do_work_pending + */ + cb.deqz $r5? _restore_regs + ;; + /* + * Work pending can potentially call a signal handler and then return + * via rt_sigreturn. Return path will be different (restore all regs) + * and hence all registers are needed to be saved. + */ + save_callee + ;; + call_do_work_pending + ;; +#ifdef CONFIG_TRACE_IRQFLAGS + /* reload sps value from saved registers */ + ld $r6 = PT_SPS[$sp] + ;; +#endif +_restore_regs: +#ifdef CONFIG_TRACE_IRQFLAGS + /* Check if IRQs are going to be reenable in next context */ + andd $r6 = $r6, KVX_SFR_SPS_IE_MASK + ;; + cb.deqz $r6? 1f + ;; + call trace_hardirqs_on + ;; +1: +#endif + disable_mmu $r0, PS_HWLOOP_DISABLE + ;; + lo $r0r1r2r3 = __PA(PT_CS_SPC_SPS_ES)[$sp] + /* Compute physical address of stack to avoid using 64bits immediate */ + addd $r11 = $sp, VA_TO_PA_OFFSET + ;; + lo $r4r5r6r7 = PT_LC_LE_LS_RA[$r11] + ;; + lo $r60r61r62r63 = PT_Q60[$r11] + ;; + lo $r56r57r58r59 = PT_Q56[$r11] + ;; + lo $r52r53r54r55 = PT_Q52[$r11] + get $r14 = $sps + ;; + lo $r48r49r50r51 = PT_Q48[$r11] + /* Generate a mask of ones at each bit where the current $sps + * differs from the $sps to be restored + */ + xord $r14 = $r2, $r14 + /* prepare wfxl clear mask on LSBs */ + notd $r15 = $r2 + /* prepare wfxl set mask on MSBs */ + slld $r13 = $r2, 32 + ;; + lo $r44r45r46r47 = PT_Q44[$r11] + /* Replicate mask of ones on the 32 MSBs */ + sbmm8 $r14 = $r14, REPLICATE_32_MASK + /* Combine the set and clear mask for wfxl */ + insf $r13 = $r15, 31, 0 + ;; + lo $r40r41r42r43 = PT_Q40[$r11] + set $lc = $r4 + /* Mask to drop identical bits in order to avoid useless + * privilege traps + */ + andd $r13 = $r13, $r14 + ;; + lq $r16r17 = PT_R16R17[$r11] + set $le = $r5 + ;; + lo $r32r33r34r35 = PT_Q32[$r11] + set $ls = $r6 + ;; + lo $r36r37r38r39 = PT_Q36[$r11] + copyd $r14 = $r11 + set $ra = $r7 + ;; + lo $r8r9r10r11 = PT_Q8[$r14] + set $cs = $r0 + /* MME was disabled by disable_mmu, reenable it before leaving */ + ord $r13 = $r13, SFR_SET_VAL_WFXL(SPS, MME, 1) + ;; + lo $r4r5r6r7 = PT_Q4[$r14] + set $spc = $r1 + ;; + lo $r0r1r2r3 = PT_Q0[$r14] + /* Store $sps wfxl value in scratch system register */ + set TMP_SCRATCH_SR = $r13 + ;; + lo $r12r13r14r15 = PT_Q12[$r14] + rswap $r0 = TMP_SCRATCH_SR + ;; + wfxl $sps = $r0 + ;; + /* Finally, restore $r0 value */ + rswap $r0 = TMP_SCRATCH_SR + ;; + rfe + ;; +ENDPROC(return_from_exception) + +/*********************************************************************** +* Debug handling +***********************************************************************/ +EXCEPTION_ENTRY(kvx_debug_handler): + exception_entry_disable_mmu + ;; + save_quad_regs $r0r1r2r3 $r4 + ;; + get $r2 = $sr + ;; + switch_stack $r1 $r2 debug_save_regs + ;; +debug_save_regs: + save_regs_for_exception $r1 $r2 $r0r1r2r3 $r1 + ;; + get $r0 = $ea + /* tail-call for return_from_exception */ + make $r3 = return_from_exception + ;; + set $ra = $r3 + ;; + goto debug_handler + ;; +ENDPROC(kvx_debug_handler) + +/*********************************************************************** +* Traps handling +***********************************************************************/ +/* These labels will be used for instruction patching */ +.global kvx_perf_tlb_refill, kvx_std_tlb_refill +EXCEPTION_ENTRY(kvx_trap_handler): + /* Enable DAUS for physical data access */ + exception_entry_disable_mmu + ;; + /* Save r3 in a temporary system register to check if the trap is a + * nomapping or not */ + set TMP_SCRATCH_SR = $r3 + ;; + get $r3 = $es + ;; + /* Hardware trap cause */ + extfz $r3 = $r3, KVX_SFR_END(ES_HTC), KVX_SFR_START(ES_HTC) + ;; + /* Is this a nomapping trap ? */ + compd.eq $r3 = $r3, KVX_TRAP_NOMAPPING + ;; + /* if nomapping trap, try fast_refill */ + cb.even $r3? trap_slow_path + ;; + /* + * Fast TLB refill routine + * + * On kvx, we do not have hardware page walking, hence, TLB refill is + * done using the core on no-mapping traps. + * This routine must be as fast as possible to avoid wasting CPU time. + * For that purpose, it is called directly from trap_handle after saving + * only 8 registers ($r0 -> $r7) in a dedicated buffer. + * To avoid taking nomapping while accessing page tables inside this + * refill handler we switch to physical accesses using DAUS. + * Once the switch is done, we save up to 8 registers to compute + * the refill data. + * This allows to avoid computing a complete task switching in order + * to greatly reduce the refill time. + * + * We refill the JTLB which contains 128 sets with 4 way each. + * Currently, the way selection is done using a round robin algorithm. + * + * The refill is using the basic flow: + * 1 - Enable physical access using DAUS. + * 2 - Save necessary registers + * 3 - Walk the page table to find the TLB entry to add (virtual to + * physical) + * 4 - Compute the TLB entry to be written (convert PTE to TLB entry) + * 5 - Compute the target set (0 -> 127) for the new TLB entry + * This is done by extracting the 6 lsb of page number + * 6 - Get the current way to be used which is selected using a + simple round robin + * 7 - Mark PTE entry as _PAGE_ACCESSED (and optionally PAGE_DIRTY) + * 8 - Commit the new tlb entry + * 9 - Restore the virtual memory by disabling DAUS. + * + */ + /* Get current task */ + rswap $r63 = $sr + ;; +#ifdef CONFIG_KVX_MMU_STATS + get $r3 = $pm0 + ;; + sd __PA(TASK_THREAD_ENTRY_TS)[$r63] = $r3 + ;; +#endif + /* Restore $r3 from temporary system scratch register */ + get $r3 = TMP_SCRATCH_SR + /* Save registers to save $tel, $teh and $mmc */ + so __PA(TASK_THREAD_SAVE_AREA_QUAD(2))[$r63] = $r8r9r10r11 + ;; + get $r8 = $tel + /* Save register for refill handler */ + so __PA(TASK_THREAD_SAVE_AREA_QUAD(1))[$r63] = $r4r5r6r7 + ;; + /* Get exception address */ + get $r0 = $ea + /* Save more registers to be comfy */ + so __PA(TASK_THREAD_SAVE_AREA_QUAD(0))[$r63] = $r0r1r2r3 + ;; + get $r9 = $teh + ;; + get $r10 = $mmc + ;; + /* Restore $r63 value */ + rswap $r63 = $sr + /* Check kernel address range */ + addd $r4 = $r0, -KERNEL_DIRECT_MEMORY_MAP_BASE + /* Load task active mm for pgd loading in macro_tlb_refill */ + ld $r1 = __PA(TASK_ACTIVE_MM)[$r63] + ;; +kvx_perf_tlb_refill: + /* Check if the address is in the kernel direct memory mapping */ + compd.ltu $r3 = $r4, KERNEL_DIRECT_MEMORY_MAP_SIZE + /* Clear low bits of virtual address to align on page size */ + andd $r5 = $r0, ~(REFILL_PERF_PAGE_SIZE - 1) + /* Create corresponding physical address */ + addd $r2 = $r4, PHYS_OFFSET + ;; + /* If address is not a kernel one, take the standard path */ + cb.deqz $r3? kvx_std_tlb_refill + /* Prepare $teh value with virtual address and kernel value */ + ord $r7 = $r5, REFILL_PERF_TEH_VAL + ;; + /* Get $pm0 value as a pseudo random value for LTLB way to use */ + get $r4 = $pm0 + /* Clear low bits of physical address to align on page size */ + andd $r2 = $r2, ~(REFILL_PERF_PAGE_SIZE - 1) + /* Prepare value for $mmc wfxl to select LTLB and correct way */ + make $r5 = MMC_SEL_LTLB_CLEAR_WAY + ;; + /* Keep low bits of timer value */ + andw $r4 = $r4, (REFILL_PERF_ENTRIES - 1) + /* Get current task pointer for register restoration */ + get $r11 = $sr + ;; + /* Add LTLB base way number for kernel refill way */ + addw $r4 = $r4, LTLB_KERNEL_RESERVED + /* Prepare $tel value with physical address and kernel value */ + ord $r6 = $r2, REFILL_PERF_TEL_VAL + set $teh = $r7 + ;; + /* insert way in $mmc wfxl value */ + insf $r5 = $r4, KVX_SFR_END(MMC_SW) + 32, KVX_SFR_START(MMC_SW) + 32 + set $tel = $r6 + ;; + wfxl $mmc = $r5 + ;; + goto do_tlb_write + ;; +kvx_std_tlb_refill: + /* extract PGD offset */ + extfz $r3 = $r0, (ASM_PGDIR_SHIFT + ASM_PGDIR_BITS - 1), ASM_PGDIR_SHIFT + /* is mm NULL ? if so, use init_mm */ + cmoved.deqz $r1? $r1 = init_mm + ;; + get $r7 = $pcr + /* Load pgd base address into $r1 */ + ld $r1 = __PA(MM_PGD)[$r1] + ;; + /* Add offset for physical address */ + addd $r1 = $r1, VA_TO_PA_OFFSET + /* Extract processor ID to compute cpu_offset*/ + extfz $r7 = $r7, KVX_SFR_END(PCR_PID), KVX_SFR_START(PCR_PID) + ;; + /* Load PGD entry offset */ + ld.xs $r1 = $r3[$r1] + /* Load per_cpu_offset */ +#if defined(CONFIG_SMP) + make $r5 = __PA(__per_cpu_offset) +#endif + ;; + /* extract PMD offset*/ + extfz $r3 = $r0, (ASM_PMD_SHIFT + ASM_PMD_BITS - 1), ASM_PMD_SHIFT + /* If pgd entry is null -> out */ + cb.deqz $r1? refill_err_out +#if defined(CONFIG_SMP) + /* Load cpu offset */ + ld.xs $r7 = $r7[$r5] +#else + /* Force cpu offset to 0 */ + make $r7 = 0 +#endif + ;; + /* Load PMD entry offset and keep pointer to the entry for huge page */ + addx8d $r2 = $r3, $r1 + ld.xs $r1 = $r3[$r1] + ;; + /* Check if it is a huge page (2Mb or 512Mb in PMD table)*/ + andd $r6 = $r1, _PAGE_HUGE + /* If pmd entry is null -> out */ + cb.deqz $r1? refill_err_out + /* extract PTE offset */ + extfz $r3 = $r0, (PAGE_SHIFT + 8), PAGE_SHIFT + ;; + /* + * If the page is a huge one we already have set the PTE and the + * pointer to the PTE. + */ + cb.dnez $r6? is_huge_page + ;; + /* Load PTE entry */ + ld.xs $r1 = $r3[$r1] + addx8d $r2 = $r3, $r1 + ;; + /* Check if it is a huge page (64Kb in PTE table) */ + andd $r6 = $r1, _PAGE_HUGE + ;; + /* Check if PTE entry is for a huge page */ + cb.dnez $r6? is_huge_page + ;; + /* 4K: Extract set value */ + extfz $r0 = $r1, (PAGE_SHIFT + KVX_SFR_MMC_SS_WIDTH - 1), PAGE_SHIFT + /* 4K: Extract virt page from ea */ + andd $r4 = $r0, PAGE_MASK + ;; +/* + * This path expects the following: + * - $r0 = set index + * - $r1 = pte entry + * - $r2 = pte entry address + * - $r4 = virtual page address + */ +pte_prepared: + /* Compute per_cpu_offset + current way of set address */ + addd $r5 = $r0, $r7 + /* Get exception cause for access type handling (page dirtying) */ + get $r7 = $es + /* Clear way and select JTLB */ + make $r6 = MMC_SEL_JTLB_CLEAR_WAY + ;; + /* Load current way to use for current set */ + lbz $r0 = __PA(jtlb_current_set_way)[$r5] + /* Check if the access was a "write" access */ + andd $r7 = $r7, (KVX_TRAP_RWX_WRITE << KVX_SFR_ES_RWX_SHIFT) + ;; + /* If bit PRESENT of pte entry is 0, then entry is not present */ + cb.even $r1? refill_err_out + /* + * Set the JTLB way in $mmc value, add 32 bits to be in the set part. + * Since we are refilling JTLB, we must make sure we insert only + * relevant bits (ie 2 bits for ways) to avoid using nonexistent ways. + */ + insf $r6 = $r0, KVX_SFR_START(MMC_SW) + 32 + (MMU_JTLB_WAYS_SHIFT - 1),\ + KVX_SFR_START(MMC_SW) + 32 + /* Extract page global bit */ + extfz $r3 = $r1, _PAGE_GLOBAL_SHIFT, _PAGE_GLOBAL_SHIFT + /* Increment way value, note that we do not care about overflow since + * we only use the two lower byte */ + addd $r0 = $r0, 1 + ;; + /* Prepare MMC */ + wfxl $mmc, $r6 + ;; + /* Insert global bit (if any) to its position into $teh value */ + insf $r4 = $r3, KVX_SFR_TEH_G_SHIFT, KVX_SFR_TEH_G_SHIFT + /* If "write" access ($r7 != 0), then set it as dirty */ + cmoved.dnez $r7? $r7 = _PAGE_DIRTY + /* Clear bits not related to FN in the pte entry for TEL writing */ + andd $r6 = $r1, KVX_PFN_MASK + /* Store new way */ + sb __PA(jtlb_current_set_way)[$r5] = $r0 + ;; + /* Extract access perms from pte entry (discard PAGE_READ bit +1) */ + extfz $r3 = $r1, KVX_ACCESS_PERM_STOP_BIT, KVX_ACCESS_PERM_START_BIT + 1 + /* Move FN bits to their place */ + srld $r6 = $r6, KVX_PFN_SHIFT - PAGE_SHIFT + /* Extract the page size + cache policy */ + andd $r0 = $r1, (KVX_PAGE_SZ_MASK | KVX_PAGE_CP_MASK) + /* Prepare SBMM value */ + make $r5 = KVX_SBMM_BYTE_SEL + ;; + /* Add page size + cache policy to $tel value */ + ord $r6 = $r6, $r0 + /* Get $mmc to get current ASN */ + get $r0 = $mmc + /* Add _PAGE_ACCESSED bit to PTE entry for writeback */ + ord $r7 = $r7, _PAGE_ACCESSED + ;; + /* OR PTE value with accessed/dirty flags */ + ord $r1 = $r1, $r7 + /* Generate the byte selection for sbmm */ + slld $r5 = $r5, $r3 + /* Compute the mask to extract set and mask exception address */ + make $r7 = KVX_PAGE_PA_MATRIX + ;; + ord $r0 = $r6, TEL_DEFAULT_VALUE + /* Add ASN from mmc into future $teh value */ + insf $r4 = $r0, KVX_SFR_END(MMC_ASN), KVX_SFR_START(MMC_ASN) + /* Get the page permission value */ + sbmm8 $r6 = $r7, $r5 + /* Check PAGE_READ bit in PTE entry */ + andd $r3 = $r1, _PAGE_READ + ;; + /* If PAGE_READ bit is not set, set policy as NA_NA */ + cmoved.deqz $r3? $r6 = TLB_PA_NA_NA + ;; + /* Shift PA to correct position */ + slld $r6 = $r6, KVX_SFR_TEL_PA_SHIFT + set $teh = $r4 + ;; + /* Store updated pte entry */ + sd 0[$r2] = $r1 + /* Prepare tel */ + ord $r6 = $r0, $r6 + /* Get current task pointer for register restoration */ + get $r11 = $sr + ;; + set $tel = $r6 + ;; +do_tlb_write: + tlbwrite + ;; +#ifdef CONFIG_KVX_DEBUG_TLB_WRITE + goto mmc_error_check + ;; +mmc_error_check_ok: +#endif +#ifdef CONFIG_KVX_DEBUG_ASN + goto asn_check + ;; +asn_check_ok: +#endif + set $tel = $r8 + /* Restore registers */ + lo $r4r5r6r7 = __PA(TASK_THREAD_SAVE_AREA_QUAD(1))[$r11] + ;; + set $teh = $r9 + lo $r0r1r2r3 = __PA(TASK_THREAD_SAVE_AREA_QUAD(0))[$r11] + ;; + set $mmc = $r10 + ;; + lo $r8r9r10r11 = __PA(TASK_THREAD_SAVE_AREA_QUAD(2))[$r11] + ;; +#ifdef CONFIG_KVX_MMU_STATS + /* + * Fence to simulate a direct data dependency after returning from trap + * nomapping handling. This is the worst case that can happen and the + * processor will be stalled waiting for previous loads to complete. + */ + fence + ;; + get $r4 = $pm0 + ;; + get $r0 = $sr + ;; + /* Get cycles measured on trap entry */ + ld $r1 = __PA(TASK_THREAD_ENTRY_TS)[$r0] + ;; + /* Compute refill time */ + sbfd $r0 = $r1, $r4 + ;; +#ifdef CONFIG_SMP + get $r1 = $pcr + ;; + /* Extract processor ID to compute cpu_offset */ + extfz $r1 = $r1, KVX_SFR_END(PCR_PID), KVX_SFR_START(PCR_PID) + make $r2 = __PA(__per_cpu_offset) + ;; + /* Load cpu offset */ + ld.xs $r1 = $r1[$r2] + ;; + addd $r1 = $r1, __PA(mmu_stats) + ;; +#else + make $r1 = __PA(mmu_stats) + ;; +#endif + /* Load time between refill + last refill cycle */ + lq $r2r3 = MMU_STATS_CYCLES_BETWEEN_REFILL_OFF[$r1] + ;; + /* Set last update time to current if 0 */ + cmoved.deqz $r3? $r3 = $r4 + /* remove current refill time to current cycle */ + sbfd $r4 = $r0, $r4 + ;; + /* Compute time between last refill and current refill */ + sbfd $r5 = $r3, $r4 + /* Update last cycle time */ + copyd $r3 = $r4 + ;; + /* Increment total time between refill */ + addd $r2 = $r2, $r5 + ;; + sq MMU_STATS_CYCLES_BETWEEN_REFILL_OFF[$r1] = $r2r3 + /* Get exception address */ + get $r4 = $ea + ;; + /* $r2 holds refill type (user/kernel/kernel_direct) */ + make $r2 = MMU_STATS_REFILL_KERNEL_OFF + /* Check if address is a kernel direct mapping one */ + compd.ltu $r3 = $r4, (KERNEL_DIRECT_MEMORY_MAP_BASE + \ + KERNEL_DIRECT_MEMORY_MAP_SIZE) + ;; + cmoved.dnez $r3? $r2 = MMU_STATS_REFILL_KERNEL_DIRECT_OFF + /* Check if address is a user (ie below kernel) */ + compd.ltu $r3 = $r4, KERNEL_DIRECT_MEMORY_MAP_BASE + ;; + cmoved.dnez $r3? $r2 = MMU_STATS_REFILL_USER_OFF + ;; + /* Compute refill struct addr into one reg */ + addd $r1 = $r2, $r1 + /* Load refill_struct values */ + lo $r4r5r6r7 = $r2[$r1] + ;; + /* Increment count */ + addd $r4 = $r4, 1 + /* Increment total cycles count */ + addd $r5 = $r5, $r0 + ;; + /* Set min to ~0 if 0 */ + cmoved.deqz $r6? $r6 = ~0 + ;; + /* Compare min and max */ + compd.ltu $r2 = $r0, $r6 + compd.gtu $r3 = $r0, $r7 + ;; + /* Update min and max*/ + cmoved.dnez $r2? $r6 = $r0 + cmoved.dnez $r3? $r7 = $r0 + ;; + /* store back all values */ + so 0[$r1] = $r4r5r6r7 + ;; + get $r0 = $sr + ;; + /* Restore clobberred registers */ + lo $r4r5r6r7 = __PA(TASK_THREAD_SAVE_AREA_QUAD(1))[$r0] + ;; + lo $r0r1r2r3 = __PA(TASK_THREAD_SAVE_AREA_QUAD(0))[$r0] + ;; +#endif + /* Save $r4 for reenabling mmu and data cache in sps */ + set TMP_SCRATCH_SR = $r4 + /* Enable MME in $sps */ + make $r4 = MME_WFXL(1) + ;; + /* Reenable $mme in $sps */ + wfxl $sps = $r4 + ;; + get $r4 = TMP_SCRATCH_SR + ;; + rfe + ;; + +is_huge_page: + /* + * When entering this path: + * - $r0 = $ea + * - $r1 = pte entry + * - $r7 = cpu offset for tlb_current_set_way + * + * From now on, we have the pte value in $r1 so we can extract the page + * size. This value is stored as it is expected by the MMU (ie between + * 0 and 3). + * Note that page size value is located at the same place as in $tel + * and this is checked at build time so we can use TEL_PS defines. + * In this codepath, we will extract the set and mask exception address + * and align virt and phys address with what the hardware expect. + * Indeed, MMU expect lsb of the virtual and physycal address to be 0 + * according to page size. + * This means that for 4K pages, the 12 lsb must be 0, for 64K + * pages, the 16 lsb must be 0 and so on. + */ + extfz $r5 = $r1, KVX_SFR_END(TEL_PS), KVX_SFR_START(TEL_PS) + /* Compute the mask to extract set and mask exception address */ + make $r4 = KVX_PS_SHIFT_MATRIX + make $r6 = KVX_SBMM_BYTE_SEL + ;; + /* Generate the byte selection for sbmm */ + slld $r6 = $r6, $r5 + ;; + /* Get the shift value */ + sbmm8 $r5 = $r4, $r6 + make $r4 = 0xFFFFFFFFFFFFFFFF + ;; + /* extract TLB set from ea (6 lsb of virtual page) */ + srld $r5 = $r0, $r5 + /* Generate ea masking according to page shift */ + slld $r4 = $r4, $r5 + ;; + /* Mask to get the set value */ + andd $r0 = $r5, MMU_SET_MASK + /* Extract virt page from ea */ + andd $r4 = $r0, $r4 + ;; + /* Returned to fast path */ + goto pte_prepared + ;; + +#ifdef CONFIG_KVX_DEBUG_TLB_WRITE +mmc_error_check: + get $r1 = $mmc + ;; + andd $r1 = $r1, KVX_SFR_MMC_E_MASK + ;; + cb.deqz $r1? mmc_error_check_ok + ;; + make $r0 = mmc_error_panic_str_label + goto asm_panic + ;; +#endif +#ifdef CONFIG_KVX_DEBUG_ASN +/* + * When entering this path $r11 = $sr. + * WARNING: Do not clobber it here if you don't want to mess up with registers + * restoration above. + */ +asn_check: + get $r1 = $ea + ;; + /* Check if kernel address, if so, there is no ASN */ + compd.geu $r2 = $r1, PAGE_OFFSET + ;; + cb.dnez $r2? asn_check_ok + ;; + get $r2 = $pcr + /* Load active mm addr */ + ld $r3 = __PA(TASK_ACTIVE_MM)[$r11] + ;; + get $r5 = $mmc + /* Extract processor ID to compute cpu_offset*/ + extfz $r2 = $r2, KVX_SFR_END(PCR_PID), KVX_SFR_START(PCR_PID) + addd $r3 = $r3, MM_CTXT_ASN + ;; + extfz $r4 = $r5, KVX_SFR_END(MMC_ASN), KVX_SFR_START(MMC_ASN) + addd $r3 = $r3, VA_TO_PA_OFFSET + ;; + /* Load current asn from active_mm */ + ld.xs $r3 = $r2[$r3] + ;; + /* Error if ASN is not set */ + cb.deqz $r3? asn_check_err + /* Mask $r3 asn cycle part */ + andd $r5 = $r3, ((1 << KVX_SFR_MMC_ASN_WIDTH) - 1) + ;; + /* Compare asn in $mmc and asn in current task mm */ + compd.eq $r3 = $r5, $r4 + ;; + cb.dnez $r3? asn_check_ok + ;; +asn_check_err: + /* We are fried, die peacefully */ + make $r0 = asn_error_panic_str_label + goto asm_panic + ;; +#endif + +#if defined(CONFIG_KVX_DEBUG_ASN) || defined(CONFIG_KVX_DEBUG_TLB_WRITE) + +/** + * This routine calls panic from assembly after setting appropriate things + * $r0 = panic string + */ +asm_panic: + /* + * Reenable hardware loop and traps (for nomapping) since some functions + * might need it. Moreover, disable DAUS to reenable MMU accesses. + */ + make $r32 = PS_HWLOOP_EN_ET_CLEAR_DAUS_DIS + make $r33 = 0 + get $r34 = $sr + ;; + /* Clear hw loop exit to disable current loop */ + set $le = $r33 + ;; + wfxl $ps = $r32 + ;; + /* Restore kernel stack */ + ld $r12 = TASK_THREAD_KERNEL_SP[$r34] + ;; + call panic + ;; +#endif + +/* Error path for TLB refill */ +refill_err_out: + get $r2 = $sr + ;; + /* Restore clobbered registers */ + lo $r8r9r10r11 = __PA(TASK_THREAD_SAVE_AREA_QUAD(2))[$r2] + ;; + lo $r4r5r6r7 = __PA(TASK_THREAD_SAVE_AREA_QUAD(1))[$r2] + goto trap_switch_stack + ;; + +/* This path is entered only when the trap is NOT a NOMAPPING */ +trap_slow_path: + /* Restore $r3 from temporary scratch system register */ + get $r3 = TMP_SCRATCH_SR + ;; + save_quad_regs $r0r1r2r3 $r4 + ;; + get $r2 = $sr + ;; +trap_switch_stack: + switch_stack $r1 $r2 trap_save_regs + ;; +trap_save_regs: + save_regs_for_exception $r1 $r2 $r0r1r2r3 $r2 + ;; + get $r1 = $ea + /* tail-call for return_from_exception */ + make $r3 = return_from_exception + ;; + set $ra = $r3 + ;; + /* Handler call */ + get $r0 = $es + ;; + goto trap_handler + ;; +ENDPROC(kvx_trap_handler) + +/*********************************************************************** +* Interrupts handling +***********************************************************************/ +EXCEPTION_ENTRY(kvx_interrupt_handler): + exception_entry_disable_mmu + ;; + save_quad_regs $r0r1r2r3 $r4 + ;; + get $r0 = $sr + ;; + switch_stack $r1 $r0 it_save_regs + ;; +#ifdef CONFIG_SECURE_DAME_HANDLING + /** + * In order to securely Handle Data Asynchronous Memory Error, + * we need to have a correct entry point. This means we do not + * want to handle a user induced DAME interrupt when entering + * kernel. + * In order to do that, we need to do a barrier, which will + * reflect the DAME status in $ilr (if any). + */ + barrier + ;; +#endif +it_save_regs: + save_regs_for_exception $r1 $r0 $r0r1r2r3 $r1 + ;; + get $r0 = $ilr + ;; + get $r2 = $ile + ;; + /** + * When an interrupt happens, the processor automatically clears the + * corresponding bit in $ilr. However, as we are using $ilr to get the + * list of irqs we want to handle, we need to add the automatically + * cleared interrupt bit. This is done by getting the interrupt number + * from $es. + */ + get $r3 = $es + make $r4 = 1 + ;; + /* Extract interrupt number from $es */ + extfz $r3 = $r3, KVX_SFR_END(ES_ITN), KVX_SFR_START(ES_ITN) + /** + * Mask requests with enabled line since ILR will also contain disabled + * interrupt lines (ie not enabled in $ile) and we want to respect the + * current state of interrupt lines. + */ + andd $r0 = $r0, $r2 + ;; + /* Clear $ilr with bits we are going to handle */ + wfxl $ilr = $r0 + slld $r4 = $r4, $r3 + ;; + /* OR the irq mask with the current pending irq */ + ord $r0 = $r0, $r4 + call do_IRQ + ;; + /* From now on, lower the interrupt level (IL) to allow IT nesting. + * If returning to user, we will call schedule which will reenable + * interrupts by itself when ready. + * If returning to kernel and with CONFIG_PREEMPT, we will call + * preempt_schedule_irq which will do the same. + */ + make $r0 = PS_IL_CLEAR + ;; + wfxl $ps = $r0 + ;; + goto return_from_exception + ;; +ENDPROC(kvx_interrupt_handler) + +/*********************************************************************** +* Syscall handling +***********************************************************************/ +EXCEPTION_ENTRY(kvx_syscall_handler): + /** + * Syscalls are seen as standard func call from ABI POV + * We may use all caller saved register whithout causing havoc + * in the userspace. We need to save callee registers because they + * will be restored when returning from fork. + * Note that r0 -> r11 MUST not be used since they are + * containing syscall parameters ! + * During this function, $r38 is the syscall handler address. Hence, + * this register must not be clobberred during function calls (tracing + * for instance. + */ + disable_mmu $r43 + ;; + get $r43 = $es + copyd $r52 = $sp + copyd $r53 = $tp + ;; + get $r36 = $sr + copyd $r54 = $fp + copyd $r55 = $r15 + ;; + /* Extract syscall number */ + extfz $r32 = $r43, KVX_SFR_END(ES_SN), KVX_SFR_START(ES_SN) + /* Get regs to save on stack */ + get $r63 = $ra + addd $r36 = $r36, VA_TO_PA_OFFSET + ;; + ld $r39 = TASK_TI_FLAGS[$r36] + get $r41 = $spc + make $r42 = __PA(sys_call_table) + ;; + /* Check for out-of-bound syscall number */ + sbfd $r50 = $r32, __NR_syscalls + /* Compute syscall func addr (ie sys_call_table[$r32])*/ + ld.xs $r38 = $r32[$r42] + get $r42 = $sps + ;; + /* True if trace syscall enable */ + andd $r37 = $r39, _TIF_SYSCALL_WORK + /* Restore kernel stack pointer */ + ld $sp = TASK_THREAD_KERNEL_SP[$r36] + /* If the syscall number is invalid, set invalid handler */ + cmoved.dlez $r50? $r38 = sys_ni_syscall + ;; + /* Prepare space on stack */ + addd $sp = $sp, -PT_SIZE_ON_STACK + get $r40 = $cs + /* Save regs r0 -> r3 in pt_regs for restart & trace if needed */ + so __PA(PT_Q0 - PT_SIZE_ON_STACK)[$sp] = $r0r1r2r3 + ;; + /* Store user stack pointer, frame pointer, thread pointer and r15 */ + so __PA(PT_Q12)[$sp] = $r52r53r54r55 + addd $r36 = $sp, VA_TO_PA_OFFSET + get $r60 = $lc + ;; +#ifdef CONFIG_SECURE_DAME_HANDLING + get $r44 = $ilr + make $r45 = SFR_CLEAR_WFXL(ILR, IT16); + ;; + /* Extract $ilr.dame bit */ + extfz $r44 = $r44, KVX_SFR_END(ILR_IT16), KVX_SFR_START(ILR_IT16) + /* Save $ilr value */ + sd PT_ILR[$r36] = $r44 + /* Clear $ilr.dame */ + wfxl $ilr = $r45 + ;; +#endif + so PT_CS_SPC_SPS_ES[$r36] = $r40r41r42r43 + get $r61 = $le + make $r43 = 0x0 + ;; + /* Reenable hardware loops, IT, exceptions and disable DAUS */ + make $r44 = PS_HLE_EN_IT_EN_ET_CLEAR_DAUS_DIS + get $r62 = $ls + /* Save regs r4 -> r7 in pt_regs for restart & trace if needed */ + so PT_Q4[$r36] = $r4r5r6r7 + ;; + /* Clear $le on entry */ + set $le = $r43 + /* Save hw loop stuff */ + so PT_LC_LE_LS_RA[$r36] = $r60r61r62r63 + /* Clear frame pointer for kernel */ + make $fp = 0 + ;; + /* Enable hwloop and interrupts and MMU + * Note that we have to reenable interrupts after saving context + * to avoid losing registers content */ + wfxl $ps, $r44 + ;; + /* Do we have to trace the syscall ? */ + cb.dnez $r37? trace_syscall_enter + /* Stroe original r0 value */ + sd PT_ORIG_R0[$sp] = $r0 + ;; +do_syscall: + /* Call the syscall handler */ + icall $r38 + ;; + /* + * rt_sigreturn syscall will not take this exit path (see + * sys_rt_sigreturn for more information). + */ + /* Reload task flags since the syscall might have generated a signal*/ + get $r36 = $sr + ;; +#ifdef CONFIG_SECURE_DAME_HANDLING + /* Barrier to force DAME interrupt to be generated if any pending */ + barrier + ;; +#endif + /* Disable interrupt to check task flags atomically */ + disable_interrupt $r60 + ;; + ld $r38 = TASK_TI_FLAGS[$r36] + ;; + andd $r37 = $r38, _TIF_SYSCALL_WORK + ;; + /* Save r0 which was returned from do_scall previously and will be + * clobberred by do_work_pending (and potentially do_syscall_trace_exit + * if tracing is enabled) + * If do_signal is called and that syscall is restarted, + * it will be modified by handle_restart to restore original + * r0 value + */ + sd PT_Q0[$sp] = $r0 + /* used to store if trace system */ + cb.dnez $r37? trace_syscall_exit + ;; +check_work_pending: + /* Do we have work pending ? */ + andd $r1 = $r38, _TIF_WORK_MASK + ;; + /* If no work pending, directly continue to ret_to_user */ + cb.dnez $r1? call_work_pending + ;; +ret_to_user: + disable_mmu $r60, PS_HWLOOP_DISABLE + ;; + /* Compute $sp virtual address to avoid using 64 bits offset */ + addd $r15 = $sp, VA_TO_PA_OFFSET + ;; + /* Restore registers */ + lo $r60r61r62r63 = PT_LC_LE_LS_RA[$r15] + get $r33 = $sps + ;; + lo $r40r41r42r43 = PT_CS_SPC_SPS_ES[$r15]; + set $ra = $r63 + ;; + /* Restore syscall arguments since they might be needed for + * syscall restart + */ + lo $r0r1r2r3 = PT_Q0[$r15] + set $cs = $r40 + /* Generate a mask of ones at each bit where the current $sps + * differs from the $sps to be restored + */ + xord $r33 = $r42, $r33 + /* prepare wfxl clear mask on LSBs */ + notd $r34 = $r42 + /* prepare wfxl set mask on MSBs */ + slld $r35 = $r42, 32 + ;; + set $lc = $r60 + /* Replicate mask of ones on the 32 MSBs */ + sbmm8 $r33 = $r33, REPLICATE_32_MASK + /* Combine the set and clear mask for wfxl */ + insf $r35 = $r34, 31, 0 + ;; + lo $r4r5r6r7 = PT_Q4[$r15] + set $le = $r61 + /* Mask to drop identical bits in order to avoid useless + * privilege traps + */ + andd $r35 = $r35, $r33 + ;; + /* Restore $ilr */ + ld $r44 = PT_ILR[$r15] + set $ls = $r62 + /* Reenable MMU in $sps */ + ord $r35 = $r35, SFR_SET_VAL_WFXL(SPS, MME, 1) + ;; + /* Restore user pointer */ + lo $r12r13r14r15 = PT_Q12[$r15] + /* Prepare $r44 as a set mask for $ilr wfxl */ + slld $r44 = $r44, 32 + ;; + /** + * wfxl on $ilr to avoid privilege traps when restoring with set + * Note that we do that after disabling interrupt since we explicitly + * want to serve DAME itnerrupt to the user (ie not in kernel mode). + */ + wfxl $ilr = $r44 + ;; + wfxl $sps = $r35 + ;; + set $spc = $r41 + ;; + /* TODO: we might have to clear some registers to avoid leaking + * information to user space ! callee saved regs have been + * restored by called function but caller saved regs might + * have been used without being cleared */ + rfe + ;; + +/* Slow paths handling */ +call_work_pending: + /* + * Work pending can potentially call a signal handler and then return + * via rt_sigreturn. Return path will be different (restore all regs) + * and hence all registers are needed to be saved. + */ + save_callee + ;; + call_do_work_pending + ;; + /* Since we are returning to user, interrupts will be reenabled */ + call_trace_hardirqs on + ;; + goto ret_to_user + ;; + +trace_syscall_enter: + /* Save $r38 (syscall handler) which was computed above */ + sd PT_R38[$sp] = $r38 + ;; + /* do_syscall_trace_enter expect pt_regs and syscall number + * as argument */ + copyd $r0 = $sp + copyd $r1 = $r32 + ;; + call do_syscall_trace_enter + ;; + /* Restore r38 (syscall handler) which might has been clobbered by + * do_syscall_trace_enter */ + ld $r38 = PT_R38[$sp] + ;; + /* if the trace system requested to abort syscall, set $r38 to + * non implemented syscall */ + cmoved.dnez $r0? $r38 = sys_ni_syscall + ;; + /* Restore registers since the do_syscall_trace_enter call might + * have clobbered them and we need them for the actual syscall + * call */ + lo $r0r1r2r3 = PT_Q0[$sp] + ;; + lo $r4r5r6r7 = PT_Q4[$sp] + ;; + goto do_syscall + ;; +trace_syscall_exit: + copyd $r0 = $sp + call do_syscall_trace_exit + ;; + /* ptrace_notify might re-enable interrupts, disable them to be sure + * it will not mess up context restoration path */ + disable_interrupt $r36 + ;; + get $r36 = $sr + ;; + ld $r38 = TASK_TI_FLAGS[$r36] + goto check_work_pending + ;; +ENDPROC(kvx_syscall_handler) + +.text +/** + * sys_rt_sigreturn: Special handler for sigreturn + * rt_sigreturn is called after invoking a user signal handler (see + * user_scall_rt_sigreturn). Since this signal handler can be invoked after + * interrupts have been handled, this means we must restore absolutely all user + * registers. Normally, this has been done by setup_sigcontext which saved all + * user registers on the user stack. + * In restore sigcontext, they have been restored onto entry stack (stack to be + * restored). However, the standard syscall path do not restore it completely + * since only callee-saved registers are restored for fork and al. + * Here, we will restore all registers which might have been clobberred. + */ +ENTRY(sys_rt_sigreturn) + /* + * If syscall tracing is enable (stored in $r37 during syscall + * fastpath), tail-call to trace_exit. If not, tail-call to + * ret_from_kernel. + */ + cmoved.dnez $r37? $r38 = scall_trace_exit + cmoved.deqz $r37? $r38 = ret_from_kernel + ;; + set $ra = $r38 + ;; + goto _sys_rt_sigreturn + ;; +scall_trace_exit: + copyd $r0 = $sp + call do_syscall_trace_exit + ;; + goto ret_from_kernel + ;; +ENDPROC(sys_rt_sigreturn) + +/** + * __sys_clone: slow path handler for clone + * + * Save callee registers (from $r18 to $31) since they are needed for + * child process when restoring it. + * Indeed, when forking we want it to have the same registers contents + * as its parent. These registers will then be restored in + * ret_from_fork. + * This slow path saves them out of the fast path to avoid bloating all syscall + * just for one special case. + */ +ENTRY(__sys_clone) + save_callee + ;; + /* + * Use goto since we want sys_clone to "ret" to the previous caller. + * This allows to simply go back in the normal syscall fastpath + */ + goto sys_clone + ;; +ENDPROC(__sys_clone) +/*********************************************************************** +* Context switch +***********************************************************************/ + +#define SAVE_TCA_REG(__reg_num, __task_reg, __zero_reg1, __zero_reg2) \ + xso (CTX_SWITCH_TCA_REGS + (__reg_num * TCA_REG_SIZE))[__task_reg] = \ + $a##__reg_num ;\ + movetq $a##__reg_num##.lo = __zero_reg1, __zero_reg2 ;\ + movetq $a##__reg_num##.hi = __zero_reg1, __zero_reg2 ;\ + ;; + +.macro save_tca_regs task_reg zero_reg1 zero_reg2 + SAVE_TCA_REG(0, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(1, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(2, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(3, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(4, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(5, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(6, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(7, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(8, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(9, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(10, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(11, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(12, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(13, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(14, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(15, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(16, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(17, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(18, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(19, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(20, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(21, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(22, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(23, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(24, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(25, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(26, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(27, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(28, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(29, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(30, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(31, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(32, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(33, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(34, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(35, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(36, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(37, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(38, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(39, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(40, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(41, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(42, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(43, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(44, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(45, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(46, \task_reg, \zero_reg1, \zero_reg2) + SAVE_TCA_REG(47, \task_reg, \zero_reg1, \zero_reg2) +.endm + +#define RESTORE_TCA_REG(__reg_num, __task_reg) \ + xlo.u $a##__reg_num = (CTX_SWITCH_TCA_REGS + (__reg_num * TCA_REG_SIZE)) \ + [__task_reg] ;\ + ;; + +.macro restore_tca_regs task_reg + RESTORE_TCA_REG(0, \task_reg) + RESTORE_TCA_REG(1, \task_reg) + RESTORE_TCA_REG(2, \task_reg) + RESTORE_TCA_REG(3, \task_reg) + RESTORE_TCA_REG(4, \task_reg) + RESTORE_TCA_REG(5, \task_reg) + RESTORE_TCA_REG(6, \task_reg) + RESTORE_TCA_REG(7, \task_reg) + RESTORE_TCA_REG(8, \task_reg) + RESTORE_TCA_REG(9, \task_reg) + RESTORE_TCA_REG(10, \task_reg) + RESTORE_TCA_REG(11, \task_reg) + RESTORE_TCA_REG(12, \task_reg) + RESTORE_TCA_REG(13, \task_reg) + RESTORE_TCA_REG(14, \task_reg) + RESTORE_TCA_REG(15, \task_reg) + RESTORE_TCA_REG(16, \task_reg) + RESTORE_TCA_REG(17, \task_reg) + RESTORE_TCA_REG(18, \task_reg) + RESTORE_TCA_REG(19, \task_reg) + RESTORE_TCA_REG(20, \task_reg) + RESTORE_TCA_REG(21, \task_reg) + RESTORE_TCA_REG(22, \task_reg) + RESTORE_TCA_REG(23, \task_reg) + RESTORE_TCA_REG(24, \task_reg) + RESTORE_TCA_REG(25, \task_reg) + RESTORE_TCA_REG(26, \task_reg) + RESTORE_TCA_REG(27, \task_reg) + RESTORE_TCA_REG(28, \task_reg) + RESTORE_TCA_REG(29, \task_reg) + RESTORE_TCA_REG(30, \task_reg) + RESTORE_TCA_REG(31, \task_reg) + RESTORE_TCA_REG(32, \task_reg) + RESTORE_TCA_REG(33, \task_reg) + RESTORE_TCA_REG(34, \task_reg) + RESTORE_TCA_REG(35, \task_reg) + RESTORE_TCA_REG(36, \task_reg) + RESTORE_TCA_REG(37, \task_reg) + RESTORE_TCA_REG(38, \task_reg) + RESTORE_TCA_REG(39, \task_reg) + RESTORE_TCA_REG(40, \task_reg) + RESTORE_TCA_REG(41, \task_reg) + RESTORE_TCA_REG(42, \task_reg) + RESTORE_TCA_REG(43, \task_reg) + RESTORE_TCA_REG(44, \task_reg) + RESTORE_TCA_REG(45, \task_reg) + RESTORE_TCA_REG(46, \task_reg) + RESTORE_TCA_REG(47, \task_reg) +.endm + +.text +/* + * When entering in ret_from_kernel_thread, r20 and r21 where set by + * copy_thread and have been restored in switch_to. + * These registers contains the values needed to call a function + * specified by the switch_to caller (or where set by copy_thread). + */ +ENTRY(ret_from_kernel_thread) + call schedule_tail + ;; + /* Call fn(arg) */ + copyd $r0 = $r21 + ;; + icall $r20 + ;; + goto ret_from_kernel + ;; +ENDPROC(ret_from_kernel_thread) + +/** + * Return from fork. + * start_thread will set return stack and pc. Then copy_thread will + * take care of the copying logic. + * $r20 will then contains 0 if tracing disabled (set by copy_thread) + * The mechanism is almost the same as for ret_from_kernel_thread. + */ +ENTRY(ret_from_fork) + call schedule_tail + ;; + /* $r20 contains 0 if tracing disable */ + cb.deqz $r20? ret_from_kernel + ;; + copyd $r0 = $sp + call do_syscall_trace_exit + ;; +ret_from_kernel: + /* + * When returning from a fork, the child will take this path. + * Since we did not restore callee in return_from_exception, we + * must do it before. + */ + lo $r28r29r30r31 = PT_Q28[$sp] + ;; + lo $r24r25r26r27 = PT_Q24[$sp] + ;; + lo $r20r21r22r23 = PT_Q20[$sp] + ;; + lq $r18r19 = PT_R18R19[$sp] + ;; +#ifdef CONFIG_DEBUG_EXCEPTION_STACK + /** + * Debug code expect entry stack to be stored at current $sp. + * Make some room and store current $sp to avoid triggering false alarm. + */ + addd $sp = $sp, -STACK_REG_SIZE + ;; + sd 0[$sp] = $sp +#endif + ;; + goto return_from_exception + ;; +ENDPROC(ret_from_fork) + +/* + * The callee-saved registers must be saved and restored. + * When entering: + * - r0 = previous task struct + * - r1 = next task struct + * Moreover, the parameters for function call (given by copy_thread) + * are stored in: + * - r20 = Func to call + * - r21 = Argument for function + */ +ENTRY(__switch_to) + sd CTX_SWITCH_FP[$r0] = $fp + ;; + /* Save previous task context */ + so CTX_SWITCH_Q20[$r0] = $r20r21r22r23 + ;; + so CTX_SWITCH_Q24[$r0] = $r24r25r26r27 + get $r16 = $ra + ;; + so CTX_SWITCH_Q28[$r0] = $r28r29r30r31 + copyd $r17 = $sp + get $r2 = $cs + ;; + so CTX_SWITCH_RA_SP_R18_R19[$r0] = $r16r17r18r19 + /* Extract XMF bit which means coprocessor was used by user */ + andd $r3 = $r2, KVX_SFR_CS_XMF_MASK + ;; +#ifdef CONFIG_ENABLE_TCA + make $r4 = 0 + make $r5 = 0 + make $r6 = 1 + cb.dnez $r3? save_tca_registers + /* Check if next task needs TCA registers to be restored */ + ld $r7 = CTX_SWITCH_TCA_REGS_SAVED[$r1] + ;; +check_restore_tca: + cb.dnez $r7? restore_tca_registers + ;; +restore_fast_path: +#endif + /* Restore next task context */ + lo $r16r17r18r19 = CTX_SWITCH_RA_SP_R18_R19[$r1] + ;; + lo $r20r21r22r23 = CTX_SWITCH_Q20[$r1] + ;; + lo $r24r25r26r27 = CTX_SWITCH_Q24[$r1] + copyd $sp = $r17 + set $ra = $r16 + ;; + lo $r28r29r30r31 = CTX_SWITCH_Q28[$r1] + set $sr = $r1 + ;; + ld $fp = CTX_SWITCH_FP[$r1] + ;; + ret + ;; +#ifdef CONFIG_ENABLE_TCA +save_tca_registers: + save_tca_regs $r0 $r4 $r5 + ;; + /* Indicates that we saved the TCA context */ + sb CTX_SWITCH_TCA_REGS_SAVED[$r0] = $r6 + goto check_restore_tca + ;; +restore_tca_registers: + restore_tca_regs $r1 + ;; + /* Clear TCA registers saved hint */ + sb CTX_SWITCH_TCA_REGS_SAVED[$r1] = $r5 + goto restore_fast_path + ;; +#endif +ENDPROC(__switch_to) + +/*********************************************************************** +* Misc functions +***********************************************************************/ + +/** + * Avoid hardcoding trampoline for rt_sigreturn by using this code and + * copying it on user trampoline + */ +.pushsection .text +.global user_scall_rt_sigreturn_end +ENTRY(user_scall_rt_sigreturn) + make $r6 = __NR_rt_sigreturn + ;; + scall $r6 + ;; +user_scall_rt_sigreturn_end: +ENDPROC(user_scall_rt_sigreturn) +.popsection diff --git a/arch/kvx/kernel/sys_kvx.c b/arch/kvx/kernel/sys_kvx.c new file mode 100644 index 000000000000..b32a821d76c9 --- /dev/null +++ b/arch/kvx/kernel/sys_kvx.c @@ -0,0 +1,58 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + */ + +#include + +#include +#include + +SYSCALL_DEFINE6(mmap, unsigned long, addr, unsigned long, len, + unsigned long, prot, unsigned long, flags, + unsigned long, fd, off_t, off) +{ + /* offset must be a multiple of the page size */ + if (unlikely(offset_in_page(off) != 0)) + return -EINVAL; + + /* Unlike mmap2 where the offset is in PAGE_SIZE-byte units, here it + * is in bytes. So we need to use PAGE_SHIFT. + */ + return ksys_mmap_pgoff(addr, len, prot, flags, fd, off >> PAGE_SHIFT); +} + +SYSCALL_DEFINE4(cachectl, unsigned long, addr, unsigned long, len, + unsigned long, cache, unsigned long, flags) +{ + bool wb = !!(flags & CACHECTL_FLAG_OP_WB); + bool inval = !!(flags & CACHECTL_FLAG_OP_INVAL); + + if (len == 0) + return 0; + + /* Check for overflow */ + if (addr + len < addr) + return -EFAULT; + + if (cache != CACHECTL_CACHE_DCACHE) + return -EINVAL; + + if ((flags & CACHECTL_FLAG_OP_MASK) == 0) + return -EINVAL; + + if (flags & CACHECTL_FLAG_ADDR_PHYS) { + if (!IS_ENABLED(CONFIG_CACHECTL_UNSAFE_PHYS_OPERATIONS)) + return -EINVAL; + + if (!capable(CAP_SYS_ADMIN)) + return -EPERM; + + dcache_wb_inval_phys_range(addr, len, wb, inval); + return 0; + } + + return dcache_wb_inval_virt_range(addr, len, wb, inval); +} diff --git a/arch/kvx/kernel/syscall_table.c b/arch/kvx/kernel/syscall_table.c new file mode 100644 index 000000000000..7859d25e728d --- /dev/null +++ b/arch/kvx/kernel/syscall_table.c @@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * derived from arch/riscv/kernel/syscall_table.c + * + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include + +#include + +#undef __SYSCALL +#define __SYSCALL(nr, call) [nr] = (call), + +void *sys_call_table[__NR_syscalls] = { + [0 ... __NR_syscalls - 1] = sys_ni_syscall, +#include +}; From patchwork Fri Jan 20 14:09:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13110025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65B3AC05027 for ; Fri, 20 Jan 2023 14:36:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229928AbjATOg4 (ORCPT ); Fri, 20 Jan 2023 09:36:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38354 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231298AbjATOgF (ORCPT ); Fri, 20 Jan 2023 09:36:05 -0500 Received: from fx306.security-mail.net (smtpout30.security-mail.net [85.31.212.36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 00317CE894 for ; Fri, 20 Jan 2023 06:35:29 -0800 (PST) Received: from localhost (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id C597635CDC0 for ; Fri, 20 Jan 2023 15:20:45 +0100 (CET) Received: from fx306 (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id 8BE2E35CD5B; Fri, 20 Jan 2023 15:20:45 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx306.security-mail.net (Postfix) with ESMTPS id 1398E35CC96; Fri, 20 Jan 2023 15:20:45 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id E7FE727E044F; Fri, 20 Jan 2023 15:10:34 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id CFF1027E0444; Fri, 20 Jan 2023 15:10:34 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id OVJ7A2wZo_Fk; Fri, 20 Jan 2023 15:10:34 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 4201627E043A; Fri, 20 Jan 2023 15:10:34 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <218f.63caa33d.dde8.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu CFF1027E0444 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223834; bh=6QcC/grprLfAmpT4NvKC+2P+bFPaJgpCrUBOCq0uMCs=; h=From:To:Date:Message-Id:MIME-Version; b=Jsz1BDnzwLSgp1uP0SbgkLnVB9cUPgoFlKGT+0670XEFz3RHZCUP0NGX7/aoACy4s I9XN+NfDX+cpKzWEgijweRzCfuAVc1E5u2WzV3GPCUDW92Nh/OWH32rzft1y5rw49a bdhVe13SPEPF+/n/GPVITHkAbagdA8H2JMOmHuSo= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 22/31] kvx: Add signal handling support Date: Fri, 20 Jan 2023 15:09:53 +0100 Message-ID: <20230120141002.2442-23-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Add sigcontext definition and signal handling support for kvx. Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Jules Maselbas Signed-off-by: Jules Maselbas Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Yann Sionneau Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: - use read_thread_flags() as suggested by Mark Rutland arch/kvx/include/uapi/asm/sigcontext.h | 16 ++ arch/kvx/kernel/signal.c | 265 +++++++++++++++++++++++++ arch/kvx/kernel/vdso.c | 87 ++++++++ 3 files changed, 368 insertions(+) create mode 100644 arch/kvx/include/uapi/asm/sigcontext.h create mode 100644 arch/kvx/kernel/signal.c create mode 100644 arch/kvx/kernel/vdso.c diff --git a/arch/kvx/include/uapi/asm/sigcontext.h b/arch/kvx/include/uapi/asm/sigcontext.h new file mode 100644 index 000000000000..97ab4f78152a --- /dev/null +++ b/arch/kvx/include/uapi/asm/sigcontext.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _UAPI_ASM_KVX_SIGCONTEXT_H +#define _UAPI_ASM_KVX_SIGCONTEXT_H + +#include + +struct sigcontext { + struct user_pt_regs sc_regs; +}; + +#endif /* _UAPI_ASM_KVX_SIGCONTEXT_H */ diff --git a/arch/kvx/kernel/signal.c b/arch/kvx/kernel/signal.c new file mode 100644 index 000000000000..5bd63d14f43e --- /dev/null +++ b/arch/kvx/kernel/signal.c @@ -0,0 +1,265 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include +#include + +#include +#include +#include +#include + +struct rt_sigframe { + struct siginfo info; + struct ucontext uc; +}; + +int __init setup_syscall_sigreturn_page(void *sigpage_addr) +{ + unsigned int frame_size = (uintptr_t) &user_scall_rt_sigreturn_end - + (uintptr_t) &user_scall_rt_sigreturn; + + /* Copy the sigreturn scall implementation */ + memcpy(sigpage_addr, &user_scall_rt_sigreturn, frame_size); + + flush_icache_range((unsigned long) sigpage_addr, + (unsigned long) sigpage_addr + frame_size); + + return 0; +} + +static long restore_sigcontext(struct pt_regs *regs, + struct sigcontext __user *sc) +{ + long err; + + /* sc_regs is structured the same as the start of pt_regs */ + err = __copy_from_user(regs, &sc->sc_regs, sizeof(sc->sc_regs)); + + return err; +} + +long _sys_rt_sigreturn(void) +{ + struct pt_regs *regs = current_pt_regs(); + struct rt_sigframe __user *frame; + struct task_struct *task; + sigset_t set; + + current->restart_block.fn = do_no_restart_syscall; + + frame = (struct rt_sigframe __user *) user_stack_pointer(regs); + + /* + * Stack is not aligned but should be ! + * User probably did some malicious things. + */ + if (user_stack_pointer(regs) & STACK_ALIGN_MASK) + goto badframe; + + if (!access_ok(frame, sizeof(*frame))) + goto badframe; + + /* Restore sigmask */ + if (__copy_from_user(&set, &frame->uc.uc_sigmask, sizeof(set))) + goto badframe; + + set_current_blocked(&set); + + if (restore_sigcontext(regs, &frame->uc.uc_mcontext)) + goto badframe; + + if (restore_altstack(&frame->uc.uc_stack)) + goto badframe; + + return regs->r0; + +badframe: + task = current; + if (show_unhandled_signals) { + pr_info_ratelimited( + "%s[%d]: bad frame in %s: frame=%p pc=%p sp=%p\n", + task->comm, task_pid_nr(task), __func__, + frame, (void *) instruction_pointer(regs), + (void *) user_stack_pointer(regs)); + } + force_sig(SIGSEGV); + return 0; +} + + +static long setup_sigcontext(struct rt_sigframe __user *frame, + struct pt_regs *regs) +{ + struct sigcontext __user *sc = &frame->uc.uc_mcontext; + long err; + + /* sc_regs is structured the same as the start of pt_regs */ + err = __copy_to_user(&sc->sc_regs, regs, sizeof(sc->sc_regs)); + + return err; +} + +static inline void __user *get_sigframe(struct ksignal *ksig, + struct pt_regs *regs, size_t framesize) +{ + unsigned long sp; + /* Default to using normal stack */ + sp = regs->sp; + + /* + * If we are on the alternate signal stack and would overflow it, don't. + * Return an always-bogus address instead so we will die with SIGSEGV. + */ + if (on_sig_stack(sp) && !likely(on_sig_stack(sp - framesize))) + return (void __user __force *)(-1UL); + + /* This is the X/Open sanctioned signal stack switching. */ + sp = sigsp(sp, ksig) - framesize; + + /* Align the stack frame on 16bytes */ + sp &= ~STACK_ALIGN_MASK; + + return (void __user *)sp; +} + +/* TODO: Use VDSO when ready ! */ +static int setup_rt_frame(struct ksignal *ksig, sigset_t *set, + struct pt_regs *regs) +{ + unsigned long sigpage = current->mm->context.sigpage; + struct rt_sigframe __user *frame; + long err = 0; + + frame = get_sigframe(ksig, regs, sizeof(*frame)); + if (!access_ok(frame, sizeof(*frame))) + return -EFAULT; + + err |= copy_siginfo_to_user(&frame->info, &ksig->info); + + /* Create the ucontext. */ + err |= __put_user(0, &frame->uc.uc_flags); + err |= __put_user(NULL, &frame->uc.uc_link); + err |= __save_altstack(&frame->uc.uc_stack, user_stack_pointer(regs)); + err |= setup_sigcontext(frame, regs); + err |= __copy_to_user(&frame->uc.uc_sigmask, set, sizeof(*set)); + if (err) + return -EFAULT; + + /* + * When returning from the handler, we want to jump to the + * sigpage which will execute the sigreturn scall. + */ + regs->ra = sigpage; + /* Return to signal handler */ + regs->spc = (unsigned long)ksig->ka.sa.sa_handler; + regs->sp = (unsigned long) frame; + + /* Parameters for signal handler */ + regs->r0 = ksig->sig; /* r0: signal number */ + regs->r1 = (unsigned long)(&frame->info); /* r1: siginfo pointer */ + regs->r2 = (unsigned long)(&frame->uc); /* r2: ucontext pointer */ + + return 0; +} + +static void handle_signal(struct ksignal *ksig, struct pt_regs *regs) +{ + sigset_t *oldset = sigmask_to_save(); + int ret; + + /* Are we from a system call? */ + if (in_syscall(regs)) { + /* If so, check system call restarting.. */ + switch (regs->r0) { + case -ERESTART_RESTARTBLOCK: + case -ERESTARTNOHAND: + regs->r0 = -EINTR; + break; + case -ERESTARTSYS: + if (!(ksig->ka.sa.sa_flags & SA_RESTART)) { + regs->r0 = -EINTR; + break; + } + fallthrough; + case -ERESTARTNOINTR: + regs->r0 = regs->orig_r0; + regs->spc -= 0x4; + break; + } + } + + ret = setup_rt_frame(ksig, oldset, regs); + + signal_setup_done(ret, ksig, 0); +} + +asmlinkage void do_signal(struct pt_regs *regs) +{ + struct ksignal ksig; + + if (get_signal(&ksig)) { + handle_signal(&ksig, regs); + return; + } + + /* Are we from a system call? */ + if (in_syscall(regs)) { + /* + * If we are here, this means there is no handler + * present and we must restart the syscall. + */ + switch (regs->r0) { + case -ERESTART_RESTARTBLOCK: + /* Modify the syscall number in order to restart it */ + regs->r6 = __NR_restart_syscall; + fallthrough; + case -ERESTARTNOHAND: + case -ERESTARTSYS: + case -ERESTARTNOINTR: + /* We are restarting the syscall */ + regs->r0 = regs->orig_r0; + /* + * scall instruction isn't bundled with anything else, + * so we can just revert the spc to restart the syscall. + */ + regs->spc -= 0x4; + break; + } + } + + /* + * If there's no signal to deliver, we just put the saved sigmask + * back. + */ + restore_saved_sigmask(); +} + + +asmlinkage void do_work_pending(struct pt_regs *regs, + unsigned long thread_flags) +{ + /* We are called with IRQs disabled */ + trace_hardirqs_off(); + + do { + if (thread_flags & _TIF_NEED_RESCHED) { + schedule(); + } else { + local_irq_enable(); + if (thread_flags & (_TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL)) + do_signal(regs); + + if (thread_flags & _TIF_NOTIFY_RESUME) { + clear_thread_flag(TIF_NOTIFY_RESUME); + resume_user_mode_work(regs); + } + } + local_irq_disable(); + thread_flags = read_thread_flags(); + } while (thread_flags & _TIF_WORK_MASK); +} + diff --git a/arch/kvx/kernel/vdso.c b/arch/kvx/kernel/vdso.c new file mode 100644 index 000000000000..1515de15eb31 --- /dev/null +++ b/arch/kvx/kernel/vdso.c @@ -0,0 +1,87 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include +#include +#include + +#include + +static struct page *signal_page; + +static int __init init_sigreturn(void) +{ + struct page *sigpage; + void *mapped_sigpage; + int err; + + sigpage = alloc_page(GFP_KERNEL); + if (!sigpage) + panic("Cannot allocate sigreturn page"); + + mapped_sigpage = vmap(&sigpage, 1, 0, PAGE_KERNEL); + if (!mapped_sigpage) + panic("Cannot map sigreturn page"); + + clear_page(mapped_sigpage); + + err = setup_syscall_sigreturn_page(mapped_sigpage); + if (err) + panic("Cannot set signal return syscall, err: %x.", err); + + vunmap(mapped_sigpage); + + signal_page = sigpage; + + return 0; +} +arch_initcall(init_sigreturn); + +static int sigpage_mremap(const struct vm_special_mapping *sm, + struct vm_area_struct *new_vma) +{ + current->mm->context.sigpage = new_vma->vm_start; + return 0; +} + +static const struct vm_special_mapping sigpage_mapping = { + .name = "[sigpage]", + .pages = &signal_page, + .mremap = sigpage_mremap, +}; + +int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp) +{ + int ret = 0; + unsigned long addr; + struct mm_struct *mm = current->mm; + struct vm_area_struct *vma; + + mmap_write_lock(mm); + + addr = get_unmapped_area(NULL, STACK_TOP, PAGE_SIZE, 0, 0); + if (IS_ERR_VALUE(addr)) { + ret = addr; + goto up_fail; + } + + vma = _install_special_mapping( + mm, + addr, + PAGE_SIZE, + VM_READ|VM_EXEC|VM_MAYREAD|VM_MAYEXEC, + &sigpage_mapping); + if (IS_ERR(vma)) { + ret = PTR_ERR(vma); + goto up_fail; + } + + mm->context.sigpage = addr; + +up_fail: + mmap_write_unlock(mm); + return ret; +} From patchwork Fri Jan 20 14:09:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13110020 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF51DC05027 for ; Fri, 20 Jan 2023 14:35:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231129AbjATOf5 (ORCPT ); Fri, 20 Jan 2023 09:35:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229929AbjATOfr (ORCPT ); Fri, 20 Jan 2023 09:35:47 -0500 Received: from fx306.security-mail.net (smtpout30.security-mail.net [85.31.212.36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 565F9808A7 for ; Fri, 20 Jan 2023 06:35:04 -0800 (PST) Received: from localhost (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id 1A68635CF95 for ; Fri, 20 Jan 2023 15:20:47 +0100 (CET) Received: from fx306 (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id 6E26435CF13; Fri, 20 Jan 2023 15:20:46 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx306.security-mail.net (Postfix) with ESMTPS id 439BB35CD24; Fri, 20 Jan 2023 15:20:45 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 8FFB327E0453; Fri, 20 Jan 2023 15:10:35 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 7082927E044D; Fri, 20 Jan 2023 15:10:35 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id gKzfpy00nt4p; Fri, 20 Jan 2023 15:10:35 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id B1D2927E0442; Fri, 20 Jan 2023 15:10:34 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <13806.63caa33d.420ef.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 7082927E044D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223835; bh=LhIynEzDrUkn1ToTMK6562lEfg1L8AvEzULLlwkvabg=; h=From:To:Date:Message-Id:MIME-Version; b=qESPLDkcLbh15lx2jo53vtJU9kmezOiQ4qaktX2mWj3whXm7lXMlwKG1tlcmmioiC DkLD5gerDabhZyK6nOO1sqGqPBDeQAJFErD08Rs8+QYH47dXaKxyUfljhtNwBSYcV6 LmWCvm9pOb57tLGFP+/+K1rY/cB49d+jwWnIKOXY= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 23/31] kvx: Add ELF relocations and module support Date: Fri, 20 Jan 2023 15:09:54 +0100 Message-ID: <20230120141002.2442-24-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Add ELF-related definition and module relocation code for basic kvx support. Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Marius Gligor Signed-off-by: Marius Gligor Co-developed-by: Yann Sionneau Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: no changes arch/kvx/include/asm/elf.h | 155 ++++++++++++++++++++++++++++++++ arch/kvx/include/asm/vermagic.h | 12 +++ arch/kvx/kernel/module.c | 148 ++++++++++++++++++++++++++++++ 3 files changed, 315 insertions(+) create mode 100644 arch/kvx/include/asm/elf.h create mode 100644 arch/kvx/include/asm/vermagic.h create mode 100644 arch/kvx/kernel/module.c diff --git a/arch/kvx/include/asm/elf.h b/arch/kvx/include/asm/elf.h new file mode 100644 index 000000000000..38978d48221e --- /dev/null +++ b/arch/kvx/include/asm/elf.h @@ -0,0 +1,155 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Yann Sionneau + * Clement Leger + * Marius Gligor + * Guillaume Thouvenin + */ + +#ifndef _ASM_KVX_ELF_H +#define _ASM_KVX_ELF_H + +#include + +#include + +/* + * These are used to set parameters in the core dumps. + */ +#define ELF_CLASS ELFCLASS64 +#define ELF_DATA ELFDATA2LSB +#define ELF_ARCH EM_KVX + +typedef uint64_t elf_greg_t; +typedef uint64_t elf_fpregset_t; + +#define ELF_NGREG (sizeof(struct user_pt_regs) / sizeof(elf_greg_t)) +typedef elf_greg_t elf_gregset_t[ELF_NGREG]; + +/* Copy user_pt_regs from pt_regs into the elf_gregset_t */ +#define ELF_CORE_COPY_REGS(dest, regs) \ + *(struct user_pt_regs *)&(dest) = (regs)->user_regs; + +/* + * This is used to ensure we don't load something for the wrong architecture. + */ +#define elf_check_arch(x) ((x)->e_machine == EM_KVX) + +#define ELF_CORE_EFLAGS 0x1308 + +#define ELF_EXEC_PAGESIZE (PAGE_SIZE) + +/* + * This is the location that an ET_DYN program is loaded if exec'ed. Typical + * use of this is to invoke "./ld.so someprog" to test out a new version of + * the loader. We need to make sure that it is out of the way of the program + * that it will "exec", and that there is sufficient room for the brk. + */ +#define ELF_ET_DYN_BASE ((TASK_SIZE / 3) * 2) + +/* + * This yields a mask that user programs can use to figure out what + * instruction set this CPU supports. This could be done in user space, + * but it's not easy, and we've already done it here. + */ +#define ELF_HWCAP (elf_hwcap) +extern unsigned long elf_hwcap; + +/* + * This yields a string that ld.so will use to load implementation + * specific libraries for optimization. This is more specific in + * intent than poking at uname or /proc/cpuinfo. + */ +#define ELF_PLATFORM (NULL) + +#define ARCH_HAS_SETUP_ADDITIONAL_PAGES 1 +struct linux_binprm; +extern int arch_setup_additional_pages(struct linux_binprm *bprm, + int uses_interp); + +/* KVX relocs */ +#define R_KVX_NONE 0 +#define R_KVX_16 1 +#define R_KVX_32 2 +#define R_KVX_64 3 +#define R_KVX_S16_PCREL 4 +#define R_KVX_PCREL17 5 +#define R_KVX_PCREL27 6 +#define R_KVX_32_PCREL 7 +#define R_KVX_S37_PCREL_LO10 8 +#define R_KVX_S37_PCREL_UP27 9 +#define R_KVX_S43_PCREL_LO10 10 +#define R_KVX_S43_PCREL_UP27 11 +#define R_KVX_S43_PCREL_EX6 12 +#define R_KVX_S64_PCREL_LO10 13 +#define R_KVX_S64_PCREL_UP27 14 +#define R_KVX_S64_PCREL_EX27 15 +#define R_KVX_64_PCREL 16 +#define R_KVX_S16 17 +#define R_KVX_S32_LO5 18 +#define R_KVX_S32_UP27 19 +#define R_KVX_S37_LO10 20 +#define R_KVX_S37_UP27 21 +#define R_KVX_S37_GOTOFF_LO10 22 +#define R_KVX_S37_GOTOFF_UP27 23 +#define R_KVX_S43_GOTOFF_LO10 24 +#define R_KVX_S43_GOTOFF_UP27 25 +#define R_KVX_S43_GOTOFF_EX6 26 +#define R_KVX_32_GOTOFF 27 +#define R_KVX_64_GOTOFF 28 +#define R_KVX_32_GOT 29 +#define R_KVX_S37_GOT_LO10 30 +#define R_KVX_S37_GOT_UP27 31 +#define R_KVX_S43_GOT_LO10 32 +#define R_KVX_S43_GOT_UP27 33 +#define R_KVX_S43_GOT_EX6 34 +#define R_KVX_64_GOT 35 +#define R_KVX_GLOB_DAT 36 +#define R_KVX_COPY 37 +#define R_KVX_JMP_SLOT 38 +#define R_KVX_RELATIVE 39 +#define R_KVX_S43_LO10 40 +#define R_KVX_S43_UP27 41 +#define R_KVX_S43_EX6 42 +#define R_KVX_S64_LO10 43 +#define R_KVX_S64_UP27 44 +#define R_KVX_S64_EX27 45 +#define R_KVX_S37_GOTADDR_LO10 46 +#define R_KVX_S37_GOTADDR_UP27 47 +#define R_KVX_S43_GOTADDR_LO10 48 +#define R_KVX_S43_GOTADDR_UP27 49 +#define R_KVX_S43_GOTADDR_EX6 50 +#define R_KVX_S64_GOTADDR_LO10 51 +#define R_KVX_S64_GOTADDR_UP27 52 +#define R_KVX_S64_GOTADDR_EX27 53 +#define R_KVX_64_DTPMOD 54 +#define R_KVX_64_DTPOFF 55 +#define R_KVX_S37_TLS_DTPOFF_LO10 56 +#define R_KVX_S37_TLS_DTPOFF_UP27 57 +#define R_KVX_S43_TLS_DTPOFF_LO10 58 +#define R_KVX_S43_TLS_DTPOFF_UP27 59 +#define R_KVX_S43_TLS_DTPOFF_EX6 60 +#define R_KVX_S37_TLS_GD_LO10 61 +#define R_KVX_S37_TLS_GD_UP27 62 +#define R_KVX_S43_TLS_GD_LO10 63 +#define R_KVX_S43_TLS_GD_UP27 64 +#define R_KVX_S43_TLS_GD_EX6 65 +#define R_KVX_S37_TLS_LD_LO10 66 +#define R_KVX_S37_TLS_LD_UP27 67 +#define R_KVX_S43_TLS_LD_LO10 68 +#define R_KVX_S43_TLS_LD_UP27 69 +#define R_KVX_S43_TLS_LD_EX6 70 +#define R_KVX_64_TPOFF 71 +#define R_KVX_S37_TLS_IE_LO10 72 +#define R_KVX_S37_TLS_IE_UP27 73 +#define R_KVX_S43_TLS_IE_LO10 74 +#define R_KVX_S43_TLS_IE_UP27 75 +#define R_KVX_S43_TLS_IE_EX6 76 +#define R_KVX_S37_TLS_LE_LO10 77 +#define R_KVX_S37_TLS_LE_UP27 78 +#define R_KVX_S43_TLS_LE_LO10 79 +#define R_KVX_S43_TLS_LE_UP27 80 +#define R_KVX_S43_TLS_LE_EX6 81 + +#endif /* _ASM_KVX_ELF_H */ diff --git a/arch/kvx/include/asm/vermagic.h b/arch/kvx/include/asm/vermagic.h new file mode 100644 index 000000000000..fef9a33065df --- /dev/null +++ b/arch/kvx/include/asm/vermagic.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_VERMAGIC_H +#define _ASM_KVX_VERMAGIC_H + +#define MODULE_ARCH_VERMAGIC "kvx" + +#endif /* _ASM_KVX_VERMAGIC_H */ diff --git a/arch/kvx/kernel/module.c b/arch/kvx/kernel/module.c new file mode 100644 index 000000000000..b9383792ae45 --- /dev/null +++ b/arch/kvx/kernel/module.c @@ -0,0 +1,148 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Yann Sionneau + * Clement Leger + */ + +#include +#include +#include + + +static int apply_rela_bits(Elf64_Addr loc, Elf64_Addr val, + int sign, int immsize, int bits, int rshift, + int lshift, unsigned int relocnum, + struct module *me) +{ + unsigned long long umax; + long long min, max; + unsigned long long mask = GENMASK_ULL(bits + lshift - 1, lshift); + + if (sign) { + min = -(1ULL << (immsize - 1)); + max = (1ULL << (immsize - 1)) - 1; + if ((long long) val < min || (long long) val > max) + goto too_big; + val = (Elf64_Addr)(((long) val) >> rshift); + } else { + if (immsize < 64) + umax = (1ULL << immsize) - 1; + else + umax = -1ULL; + if ((unsigned long long) val > umax) + goto too_big; + val >>= rshift; + } + + val <<= lshift; + val &= mask; + if (bits <= 32) + *(u32 *) loc = (*(u32 *)loc & ~mask) | val; + else + *(u64 *) loc = (*(u64 *)loc & ~mask) | val; + + return 0; +too_big: + pr_err("%s: value %llx does not fit in %d bits for reloc %u", + me->name, val, bits, relocnum); + return -ENOEXEC; +} + +int apply_relocate_add(Elf64_Shdr *sechdrs, + const char *strtab, + unsigned int symindex, + unsigned int relsec, + struct module *me) +{ + unsigned int i; + Elf64_Addr loc; + u64 val; + s64 sval; + Elf64_Sym *sym; + Elf64_Rela *rel = (void *)sechdrs[relsec].sh_addr; + int ret = 0; + + pr_debug("Applying relocate section %u to %u\n", + relsec, sechdrs[relsec].sh_info); + + for (i = 0; i < sechdrs[relsec].sh_size / sizeof(*rel); i++) { + /* This is where to make the change */ + loc = (Elf64_Addr)sechdrs[sechdrs[relsec].sh_info].sh_addr + + rel[i].r_offset; + /* This is the symbol it is referring to. Note that all + * undefined symbols have been resolved. + */ + sym = (Elf64_Sym *)sechdrs[symindex].sh_addr + + ELF64_R_SYM(rel[i].r_info); + + pr_debug("type %d st_value %llx r_addend %llx loc %llx offset %llx\n", + (int)ELF64_R_TYPE(rel[i].r_info), + sym->st_value, rel[i].r_addend, (uint64_t)loc, + rel[i].r_offset); + + val = sym->st_value + rel[i].r_addend; + switch (ELF64_R_TYPE(rel[i].r_info)) { + case R_KVX_NONE: + break; + case R_KVX_32: + ret = apply_rela_bits(loc, val, 0, 32, 32, 0, 0, + ELF64_R_TYPE(rel[i].r_info), + me); + break; + case R_KVX_64: + ret = apply_rela_bits(loc, val, 0, 64, 64, 0, 0, + ELF64_R_TYPE(rel[i].r_info), + me); + break; + case R_KVX_S43_LO10: + ret = apply_rela_bits(loc, val, 1, 43, 10, 0, 6, + ELF64_R_TYPE(rel[i].r_info), + me); + break; + case R_KVX_S64_LO10: + ret = apply_rela_bits(loc, val, 1, 64, 10, 0, 6, + ELF64_R_TYPE(rel[i].r_info), + me); + break; + case R_KVX_S43_UP27: + ret = apply_rela_bits(loc, val, 1, 43, 27, 10, 0, + ELF64_R_TYPE(rel[i].r_info), + me); + break; + case R_KVX_S64_UP27: + ret = apply_rela_bits(loc, val, 1, 64, 27, 10, 0, + ELF64_R_TYPE(rel[i].r_info), + me); + break; + case R_KVX_S43_EX6: + ret = apply_rela_bits(loc, val, 1, 43, 6, 37, 0, + ELF64_R_TYPE(rel[i].r_info), + me); + break; + case R_KVX_S64_EX27: + ret = apply_rela_bits(loc, val, 1, 64, 27, 37, 0, + ELF64_R_TYPE(rel[i].r_info), + me); + break; + case R_KVX_PCREL27: + if (__builtin_sub_overflow(val, loc, &sval)) { + pr_err("%s: Signed integer overflow, this should not happen\n", + me->name); + return -ENOEXEC; + } + sval >>= 2; + ret = apply_rela_bits(loc, (Elf64_Addr)sval, 1, 27, 27, + 0, 0, + ELF64_R_TYPE(rel[i].r_info), + me); + break; + default: + pr_err("%s: Unknown relocation: %llu\n", + me->name, ELF64_R_TYPE(rel[i].r_info)); + ret = -ENOEXEC; + } + } + return ret; +} + From patchwork Fri Jan 20 14:09:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13110021 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FC47C25B4E for ; Fri, 20 Jan 2023 14:36:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231276AbjATOgD (ORCPT ); Fri, 20 Jan 2023 09:36:03 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39146 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230387AbjATOfw (ORCPT ); Fri, 20 Jan 2023 09:35:52 -0500 Received: from fx305.security-mail.net (smtpout30.security-mail.net [85.31.212.35]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6705072C36 for ; Fri, 20 Jan 2023 06:35:09 -0800 (PST) Received: from localhost (fx305.security-mail.net [127.0.0.1]) by fx305.security-mail.net (Postfix) with ESMTP id 2C61F30FDC1 for ; Fri, 20 Jan 2023 15:20:46 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674224446; bh=fEolq90cY5EVuj1+uEWf79Pf+IA5puQR/p9asxKN+A8=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=jcOzPkZoOJPH7kMwSoH/WiR0LAll2TQCDOUHSPjjbG0WpYdDXIhpQa0AQKYLCQMzQ Tpo14mpSAvcLxR5wL5uMkiE3p0/Cb9D82993l1CRkKqXjBwNTVBL6ySvoNVTWMZIQ9 L5ImrZw+GuHLK/3yUK2UE3PlHTSTYi0LBPPQlMWs= Received: from fx305 (fx305.security-mail.net [127.0.0.1]) by fx305.security-mail.net (Postfix) with ESMTP id 5F40930FD79; Fri, 20 Jan 2023 15:20:45 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx305.security-mail.net (Postfix) with ESMTPS id 7BD1A30FCAF; Fri, 20 Jan 2023 15:20:44 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 38E4227E0452; Fri, 20 Jan 2023 15:10:36 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 20A5E27E0454; Fri, 20 Jan 2023 15:10:36 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id g2cicOgIF5vL; Fri, 20 Jan 2023 15:10:36 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 6496127E043A; Fri, 20 Jan 2023 15:10:35 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <13de6.63caa33c.7a428.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 20A5E27E0454 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223836; bh=cF5CoaZgmR1DuYZiI1Z/oNzdyD7coVLwL5/tSu1i9wQ=; h=From:To:Date:Message-Id:MIME-Version; b=j5hX1EY3DQWjpD0GffZPmcCW1pATwP/ux72mimsi2RzoDPkPiD+pGMILaWxOeKxz4 kzuu4uIUoPtUy5Oh2rn954ACY+VEHvcoTZIwuMFqr1AvlKmOFTnMvCYAHG32kpHHgs 4VfbzprQcq0M0FO3Hlbee2Wz+8siLQIRrDpRo0Ms= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 24/31] kvx: Add misc common routines Date: Fri, 20 Jan 2023 15:09:55 +0100 Message-ID: <20230120141002.2442-25-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Add some misc common routines for kvx, including: asm-offsets routines, futex functions, i/o memory access functions. Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Guillaume Thouvenin Signed-off-by: Guillaume Thouvenin Co-developed-by: Jonathan Borne Signed-off-by: Jonathan Borne Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Julien Villette Signed-off-by: Julien Villette Co-developed-by: Yann Sionneau Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: no changes arch/kvx/include/asm/futex.h | 141 ++++++++++++++++++++++++++++++ arch/kvx/include/asm/io.h | 34 ++++++++ arch/kvx/kernel/asm-offsets.c | 157 ++++++++++++++++++++++++++++++++++ arch/kvx/kernel/io.c | 96 +++++++++++++++++++++ 4 files changed, 428 insertions(+) create mode 100644 arch/kvx/include/asm/futex.h create mode 100644 arch/kvx/include/asm/io.h create mode 100644 arch/kvx/kernel/asm-offsets.c create mode 100644 arch/kvx/kernel/io.c diff --git a/arch/kvx/include/asm/futex.h b/arch/kvx/include/asm/futex.h new file mode 100644 index 000000000000..b71b52339729 --- /dev/null +++ b/arch/kvx/include/asm/futex.h @@ -0,0 +1,141 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2018-2023 Kalray Inc. + * Authors: + * Clement Leger + * Yann Sionneau + * Jonathan Borne + * + * Part of code is taken from RiscV port + */ + +#ifndef _ASM_KVX_FUTEX_H +#define _ASM_KVX_FUTEX_H + +#ifdef __KERNEL__ + +#include +#include + +#define __futex_atomic_op(insn, ret, oldval, uaddr, oparg) \ +{ \ + __enable_user_access(); \ + __asm__ __volatile__ ( \ + " fence \n" \ + " ;;\n " \ + "1: lwz $r63 = 0[%[u]] \n" \ + " ;;\n " \ + " " insn " \n" \ + " ;;\n " \ + " acswapw 0[%[u]], $r62r63 \n" \ + " ;;\n " \ + " cb.deqz $r62? 1b \n" \ + " ;;\n " \ + " copyd %[ov] = $r63 \n" \ + " ;;\n " \ + "2: \n" \ + " .section .fixup,\"ax\" \n" \ + "3: make %[r] = 2b \n" \ + " ;;\n " \ + " make %[r] = %[e] \n" \ + " igoto %[r] \n" \ + " ;;\n " \ + " .previous \n" \ + " .section __ex_table,\"a\" \n" \ + " .align 8 \n" \ + " .dword 1b,3b \n" \ + " .dword 2b,3b \n" \ + " .previous \n" \ + : [r] "+r" (ret), [ov] "+r" (oldval) \ + : [u] "r" (uaddr), \ + [op] "r" (oparg), [e] "i" (-EFAULT) \ + : "r62", "r63", "memory"); \ + __disable_user_access(); \ +} + + +static inline int +arch_futex_atomic_op_inuser(int op, u32 oparg, int *oval, u32 __user *uaddr) +{ + int oldval = 0, ret = 0; + + if (!access_ok(uaddr, sizeof(u32))) + return -EFAULT; + switch (op) { + case FUTEX_OP_SET: /* *(int *)UADDR = OPARG; */ + __futex_atomic_op("copyd $r62 = %[op]", + ret, oldval, uaddr, oparg); + break; + case FUTEX_OP_ADD: /* *(int *)UADDR += OPARG; */ + __futex_atomic_op("addw $r62 = $r63, %[op]", + ret, oldval, uaddr, oparg); + break; + case FUTEX_OP_OR: /* *(int *)UADDR |= OPARG; */ + __futex_atomic_op("orw $r62 = $r63, %[op]", + ret, oldval, uaddr, oparg); + break; + case FUTEX_OP_ANDN: /* *(int *)UADDR &= ~OPARG; */ + __futex_atomic_op("andnw $r62 = %[op], $r63", + ret, oldval, uaddr, oparg); + break; + case FUTEX_OP_XOR: + __futex_atomic_op("xorw $r62 = $r63, %[op]", + ret, oldval, uaddr, oparg); + break; + default: + ret = -ENOSYS; + } + + if (!ret) + *oval = oldval; + + return ret; +} + +static inline int futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, + u32 oldval, u32 newval) +{ + int ret = 0; + + if (!access_ok(uaddr, sizeof(u32))) + return -EFAULT; + __enable_user_access(); + __asm__ __volatile__ ( + " fence \n"/* commit previous stores */ + " copyd $r63 = %[ov] \n"/* init "expect" with ov */ + " copyd $r62 = %[nv] \n"/* init "update" with nv */ + " ;;\n " + "1: acswapw 0[%[u]], $r62r63 \n" + " ;;\n " + " cb.dnez $r62? 3f \n"/* if acswap ok -> return */ + " ;;\n " + "2: lws $r63 = 0[%[u]] \n"/* fail -> load old value */ + " ;;\n " + " compw.ne $r62 = $r63, %[ov] \n"/* check if equal to "old" */ + " ;;\n " + " cb.deqz $r62? 1b \n"/* if not equal, try again */ + " ;;\n " + "3: \n" + " .section .fixup,\"ax\" \n" + "4: make %[r] = 3b \n" + " ;;\n " + " make %[r] = %[e] \n" + " igoto %[r] \n"/* goto 3b */ + " ;;\n " + " .previous \n" + " .section __ex_table,\"a\" \n" + " .align 8 \n" + " .dword 1b,4b \n" + " .dword 2b,4b \n" + ".previous \n" + : [r] "+r" (ret) + : [ov] "r" (oldval), [nv] "r" (newval), + [e] "i" (-EFAULT), [u] "r" (uaddr) + : "r62", "r63", "memory"); + __disable_user_access(); + *uval = oldval; + return ret; +} + +#endif +#endif /* _ASM_KVX_FUTEX_H */ diff --git a/arch/kvx/include/asm/io.h b/arch/kvx/include/asm/io.h new file mode 100644 index 000000000000..c5e458c59bbb --- /dev/null +++ b/arch/kvx/include/asm/io.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_IO_H +#define _ASM_KVX_IO_H + +#include + +#include +#include + +#define _PAGE_IOREMAP _PAGE_KERNEL_DEVICE + +/* + * String version of I/O memory access operations. + */ +extern void __memcpy_fromio(void *to, const volatile void __iomem *from, + size_t count); +extern void __memcpy_toio(volatile void __iomem *to, const void *from, + size_t count); +extern void __memset_io(volatile void __iomem *dst, int c, size_t count); + +#define memset_io(c, v, l) __memset_io((c), (v), (l)) +#define memcpy_fromio(a, c, l) __memcpy_fromio((a), (c), (l)) +#define memcpy_toio(c, a, l) __memcpy_toio((c), (a), (l)) + +#include + +extern int devmem_is_allowed(unsigned long pfn); + +#endif /* _ASM_KVX_IO_H */ diff --git a/arch/kvx/kernel/asm-offsets.c b/arch/kvx/kernel/asm-offsets.c new file mode 100644 index 000000000000..3e79b6dd13bd --- /dev/null +++ b/arch/kvx/kernel/asm-offsets.c @@ -0,0 +1,157 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + * Yann Sionneau + */ + +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +int foo(void) +{ + BUILD_BUG_ON(sizeof(struct pt_regs) != PT_REGS_STRUCT_EXPECTED_SIZE); + /* + * For stack alignment purposes we must make sure the pt_regs size is + * a mutliple of stack_align + */ + BUILD_BUG_ON(!IS_ALIGNED(sizeof(struct pt_regs), STACK_ALIGNMENT)); + + /* Check that user_pt_regs size matches the beginning of pt_regs */ + BUILD_BUG_ON((offsetof(struct user_pt_regs, spc) + sizeof(uint64_t)) != + sizeof(struct user_pt_regs)); + + DEFINE(FIX_GDB_MEM_BASE_IDX, FIX_GDB_BARE_DISPLACED_MEM_BASE); + +#ifdef CONFIG_DEBUG_EXCEPTION_STACK + DEFINE(STACK_REG_SIZE, ALIGN(sizeof(uint64_t), STACK_ALIGNMENT)); +#endif + + /* + * We allocate a pt_regs on the stack when entering the kernel. This + * ensures the alignment is sane. + */ + DEFINE(PT_SIZE_ON_STACK, sizeof(struct pt_regs)); + DEFINE(TI_FLAGS_SIZE, sizeof(unsigned long)); + DEFINE(QUAD_REG_SIZE, 4 * sizeof(uint64_t)); + + /* + * When restoring registers, we do not want to restore r12 + * right now since this is our stack pointer. Allow to save + * only $r13 by using this offset. + */ + OFFSET(PT_R12, pt_regs, r12); + OFFSET(PT_R13, pt_regs, r13); + OFFSET(PT_TP, pt_regs, tp); + OFFSET(PT_R14R15, pt_regs, r14); + OFFSET(PT_R16R17, pt_regs, r16); + OFFSET(PT_R18R19, pt_regs, r18); + OFFSET(PT_FP, pt_regs, fp); + OFFSET(PT_SPS, pt_regs, sps); + + /* Quad description */ + OFFSET(PT_Q0, pt_regs, r0); + OFFSET(PT_Q4, pt_regs, r4); + OFFSET(PT_Q8, pt_regs, r8); + OFFSET(PT_Q12, pt_regs, r12); + OFFSET(PT_Q16, pt_regs, r16); + OFFSET(PT_Q20, pt_regs, r20); + OFFSET(PT_Q24, pt_regs, r24); + OFFSET(PT_Q28, pt_regs, r28); + OFFSET(PT_Q32, pt_regs, r32); + OFFSET(PT_Q36, pt_regs, r36); + OFFSET(PT_R38, pt_regs, r38); + OFFSET(PT_Q40, pt_regs, r40); + OFFSET(PT_Q44, pt_regs, r44); + OFFSET(PT_Q48, pt_regs, r48); + OFFSET(PT_Q52, pt_regs, r52); + OFFSET(PT_Q56, pt_regs, r56); + OFFSET(PT_Q60, pt_regs, r60); + OFFSET(PT_CS_SPC_SPS_ES, pt_regs, cs); + OFFSET(PT_LC_LE_LS_RA, pt_regs, lc); + OFFSET(PT_ILR, pt_regs, ilr); + OFFSET(PT_ORIG_R0, pt_regs, orig_r0); + + /* + * Flags in thread info + */ + OFFSET(TASK_TI_FLAGS, task_struct, thread_info.flags); + + /* + * Stack pointers + */ + OFFSET(TASK_THREAD_KERNEL_SP, task_struct, thread.kernel_sp); + + /* + * Offsets to save registers in switch_to using quads + */ + OFFSET(CTX_SWITCH_RA_SP_R18_R19, task_struct, thread.ctx_switch.ra); + OFFSET(CTX_SWITCH_Q20, task_struct, thread.ctx_switch.r20); + OFFSET(CTX_SWITCH_Q24, task_struct, thread.ctx_switch.r24); + OFFSET(CTX_SWITCH_Q28, task_struct, thread.ctx_switch.r28); + OFFSET(CTX_SWITCH_FP, task_struct, thread.ctx_switch.fp); + +#ifdef CONFIG_ENABLE_TCA + OFFSET(CTX_SWITCH_TCA_REGS, task_struct, thread.ctx_switch.tca_regs[0]); + OFFSET(CTX_SWITCH_TCA_REGS_SAVED, task_struct, + thread.ctx_switch.tca_regs_saved); + DEFINE(TCA_REG_SIZE, sizeof(struct tca_reg)); +#endif + + /* Save area offset */ + OFFSET(TASK_THREAD_SAVE_AREA, task_struct, thread.save_area); + + /* Fast tlb refill defines */ + OFFSET(TASK_ACTIVE_MM, task_struct, active_mm); + OFFSET(MM_PGD, mm_struct, pgd); +#ifdef CONFIG_KVX_DEBUG_ASN + OFFSET(MM_CTXT_ASN, mm_struct, context.asn); +#endif + +#ifdef CONFIG_KVX_MMU_STATS + DEFINE(MMU_REFILL_SIZE, sizeof(struct mmu_refill_stats)); + + OFFSET(MMU_STATS_REFILL_USER_OFF, mmu_stats, + refill[MMU_REFILL_TYPE_USER]); + OFFSET(MMU_STATS_REFILL_KERNEL_OFF, mmu_stats, + refill[MMU_REFILL_TYPE_KERNEL]); + OFFSET(MMU_STATS_REFILL_KERNEL_DIRECT_OFF, mmu_stats, + refill[MMU_REFILL_TYPE_KERNEL_DIRECT]); + OFFSET(MMU_STATS_CYCLES_BETWEEN_REFILL_OFF, mmu_stats, + cycles_between_refill); + OFFSET(MMU_STATS_LAST_REFILL, mmu_stats, last_refill); + + OFFSET(TASK_THREAD_ENTRY_TS, task_struct, thread.trap_entry_ts); +#endif + + DEFINE(ASM_PGDIR_SHIFT, PGDIR_SHIFT); + DEFINE(ASM_PMD_SHIFT, PMD_SHIFT); + + DEFINE(ASM_PGDIR_BITS, PGDIR_BITS); + DEFINE(ASM_PMD_BITS, PMD_BITS); + DEFINE(ASM_PTE_BITS, PTE_BITS); + + DEFINE(ASM_PTRS_PER_PGD, PTRS_PER_PGD); + DEFINE(ASM_PTRS_PER_PMD, PTRS_PER_PMD); + DEFINE(ASM_PTRS_PER_PTE, PTRS_PER_PTE); + + DEFINE(ASM_TLB_PS, TLB_DEFAULT_PS); + + return 0; +} diff --git a/arch/kvx/kernel/io.c b/arch/kvx/kernel/io.c new file mode 100644 index 000000000000..0922c1d6d0f7 --- /dev/null +++ b/arch/kvx/kernel/io.c @@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * derived from arch/arm/kernel/io.c + * + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include +#include +#include + +#define REPLICATE_BYTE_MASK 0x0101010101010101 + +/* + * Copy data from IO memory space to "real" memory space. + */ +void __memcpy_fromio(void *to, const volatile void __iomem *from, size_t count) +{ + while (count && !IS_ALIGNED((unsigned long)from, 8)) { + *(u8 *)to = __raw_readb(from); + from++; + to++; + count--; + } + + while (count >= 8) { + *(u64 *)to = __raw_readq(from); + from += 8; + to += 8; + count -= 8; + } + + while (count) { + *(u8 *)to = __raw_readb(from); + from++; + to++; + count--; + } +} +EXPORT_SYMBOL(__memcpy_fromio); + +/* + * Copy data from "real" memory space to IO memory space. + */ +void __memcpy_toio(volatile void __iomem *to, const void *from, size_t count) +{ + while (count && !IS_ALIGNED((unsigned long)to, 8)) { + __raw_writeb(*(u8 *)from, to); + from++; + to++; + count--; + } + + while (count >= 8) { + __raw_writeq(*(u64 *)from, to); + from += 8; + to += 8; + count -= 8; + } + + while (count) { + __raw_writeb(*(u8 *)from, to); + from++; + to++; + count--; + } +} +EXPORT_SYMBOL(__memcpy_toio); + +/* + * "memset" on IO memory space. + */ +void __memset_io(volatile void __iomem *dst, int c, size_t count) +{ + u64 qc = __builtin_kvx_sbmm8(c, REPLICATE_BYTE_MASK); + + while (count && !IS_ALIGNED((unsigned long)dst, 8)) { + __raw_writeb(c, dst); + dst++; + count--; + } + + while (count >= 8) { + __raw_writeq(qc, dst); + dst += 8; + count -= 8; + } + + while (count) { + __raw_writeb(c, dst); + dst++; + count--; + } +} +EXPORT_SYMBOL(__memset_io); From patchwork Fri Jan 20 14:09:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13110018 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACFCAC05027 for ; Fri, 20 Jan 2023 14:35:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231201AbjATOfU (ORCPT ); Fri, 20 Jan 2023 09:35:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38230 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231143AbjATOfT (ORCPT ); Fri, 20 Jan 2023 09:35:19 -0500 Received: from fx601.security-mail.net (smtpout140.security-mail.net [85.31.212.146]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ABFBE79297 for ; Fri, 20 Jan 2023 06:35:03 -0800 (PST) Received: from localhost (fx601.security-mail.net [127.0.0.1]) by fx601.security-mail.net (Postfix) with ESMTP id 9CE333498BF for ; Fri, 20 Jan 2023 15:20:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674224449; bh=fw6xXMajXNjhx0jPLjaTVFvl+I1RBxPH9f2FQVoTEPI=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=CXg63U8cHU2subSM4SlqKRMbOw+fdUD1Tqo5PXK4ZogMF6452Gh9QVEInt4FG8qug oZn+36xwinF+fAz2NoY7SzGTjjRYDmZBSKy8bnai9bmZcKG9edS1DxLgrWGckMpL4g bfUbm+JKD+AM0Jk6ZM+RXY0QM619HvTBOIs21GTw= Received: from fx601 (fx601.security-mail.net [127.0.0.1]) by fx601.security-mail.net (Postfix) with ESMTP id AE8CD34991F; Fri, 20 Jan 2023 15:20:47 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx601.security-mail.net (Postfix) with ESMTPS id 306C03497EB; Fri, 20 Jan 2023 15:20:45 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id B864627E045F; Fri, 20 Jan 2023 15:10:36 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 9CEA627E0454; Fri, 20 Jan 2023 15:10:36 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 7CE7-x7Qz5OQ; Fri, 20 Jan 2023 15:10:36 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 1B83627E044D; Fri, 20 Jan 2023 15:10:36 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <13f7d.63caa33d.2a770.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 9CEA627E0454 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223836; bh=YnkI4+XhgeX78k8Q76ZoPcPy7yxNCCbWh8zxK5y9348=; h=From:To:Date:Message-Id:MIME-Version; b=kXXFp1pZMzHhPKFRYroM8/+PkRoYHECkiE7VI0nI/pcainE5mFvWs5yGxLS/FH72I QLBaLeM+ZIUYoh8/PgBah6cKyRj1t5TCW0flycsAXJljcKSjQjxt9DsZ0D4Hi0eSFd nitw2XqnaM5PMEVqFZnjDe+DcM3WsGZcWMCBSfJs= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 25/31] kvx: Add some library functions Date: Fri, 20 Jan 2023 15:09:56 +0100 Message-ID: <20230120141002.2442-26-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Add some library functions for kvx, including: delay, memset, memcpy, strlen, clear_page, copy_page, raw_copy_from/to_user, asm_clear_user. Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Jules Maselbas Signed-off-by: Jules Maselbas Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Marius Gligor Signed-off-by: Marius Gligor Co-developed-by: Yann Sionneau Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: no changes arch/kvx/include/asm/string.h | 20 ++ arch/kvx/kernel/kvx_ksyms.c | 5 + arch/kvx/lib/clear_page.S | 40 ++++ arch/kvx/lib/copy_page.S | 90 +++++++++ arch/kvx/lib/delay.c | 39 ++++ arch/kvx/lib/memcpy.c | 70 +++++++ arch/kvx/lib/memset.S | 351 ++++++++++++++++++++++++++++++++++ arch/kvx/lib/strlen.S | 122 ++++++++++++ arch/kvx/lib/usercopy.S | 90 +++++++++ 9 files changed, 827 insertions(+) create mode 100644 arch/kvx/include/asm/string.h create mode 100644 arch/kvx/lib/clear_page.S create mode 100644 arch/kvx/lib/copy_page.S create mode 100644 arch/kvx/lib/delay.c create mode 100644 arch/kvx/lib/memcpy.c create mode 100644 arch/kvx/lib/memset.S create mode 100644 arch/kvx/lib/strlen.S create mode 100644 arch/kvx/lib/usercopy.S diff --git a/arch/kvx/include/asm/string.h b/arch/kvx/include/asm/string.h new file mode 100644 index 000000000000..677c1393a5cd --- /dev/null +++ b/arch/kvx/include/asm/string.h @@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Jules Maselbas + */ + +#ifndef _ASM_KVX_STRING_H +#define _ASM_KVX_STRING_H + +#define __HAVE_ARCH_MEMSET +extern void *memset(void *s, int c, size_t n); + +#define __HAVE_ARCH_MEMCPY +extern void *memcpy(void *dest, const void *src, size_t n); + +#define __HAVE_ARCH_STRLEN +extern size_t strlen(const char *s); + +#endif /* _ASM_KVX_STRING_H */ diff --git a/arch/kvx/kernel/kvx_ksyms.c b/arch/kvx/kernel/kvx_ksyms.c index 18990aaf259f..678f81716dea 100644 --- a/arch/kvx/kernel/kvx_ksyms.c +++ b/arch/kvx/kernel/kvx_ksyms.c @@ -22,3 +22,8 @@ DECLARE_EXPORT(__umoddi3); DECLARE_EXPORT(__divdi3); DECLARE_EXPORT(__udivdi3); DECLARE_EXPORT(__multi3); + +DECLARE_EXPORT(clear_page); +DECLARE_EXPORT(copy_page); +DECLARE_EXPORT(memset); +DECLARE_EXPORT(asm_clear_user); diff --git a/arch/kvx/lib/clear_page.S b/arch/kvx/lib/clear_page.S new file mode 100644 index 000000000000..364fe0663ca2 --- /dev/null +++ b/arch/kvx/lib/clear_page.S @@ -0,0 +1,40 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Marius Gligor + * Clement Leger + */ + +#include +#include +#include + +#include +#include + +#define CLEAR_PAGE_LOOP_COUNT (PAGE_SIZE / 32) + +/* + * Clear page @dest. + * + * Parameters: + * r0 - dest page + */ +ENTRY(clear_page) + make $r1 = CLEAR_PAGE_LOOP_COUNT + ;; + make $r4 = 0 + make $r5 = 0 + make $r6 = 0 + make $r7 = 0 + ;; + + loopdo $r1, clear_page_done + ;; + so 0[$r0] = $r4r5r6r7 + addd $r0 = $r0, 32 + ;; + clear_page_done: + ret + ;; +ENDPROC(clear_page) diff --git a/arch/kvx/lib/copy_page.S b/arch/kvx/lib/copy_page.S new file mode 100644 index 000000000000..4bb82d1c964c --- /dev/null +++ b/arch/kvx/lib/copy_page.S @@ -0,0 +1,90 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include +#include + +#include + +/* We have 8 load/store octuple (32 bytes) per hardware loop */ +#define COPY_SIZE_PER_LOOP (32 * 8) +#define COPY_PAGE_LOOP_COUNT (PAGE_SIZE / COPY_SIZE_PER_LOOP) + +/* + * Copy a page from src to dest (both are page aligned) + * In order to recover from smem latency, unroll the loop to trigger multiple + * onfly loads and avoid waiting too much for them to return. + * We use 8 * 32 load even though we could use more (up to 10 loads) to simplify + * the handling using a single hardware loop + * + * Parameters: + * r0 - dest + * r1 - src + */ +ENTRY(copy_page) + make $r2 = COPY_PAGE_LOOP_COUNT + make $r3 = 0 + ;; + loopdo $r2, copy_page_done + ;; + /* + * Load 8 * 32 bytes using uncached access to avoid hitting + * the cache + */ + lo.xs $r32r33r34r35 = $r3[$r1] + /* Copy current copy index for store */ + copyd $r2 = $r3 + addd $r3 = $r3, 1 + ;; + lo.xs $r36r37r38r39 = $r3[$r1] + addd $r3 = $r3, 1 + ;; + lo.xs $r40r41r42r43 = $r3[$r1] + addd $r3 = $r3, 1 + ;; + lo.xs $r44r45r46r47 = $r3[$r1] + addd $r3 = $r3, 1 + ;; + lo.xs $r48r49r50r51 = $r3[$r1] + addd $r3 = $r3, 1 + ;; + lo.xs $r52r53r54r55 = $r3[$r1] + addd $r3 = $r3, 1 + ;; + lo.xs $r56r57r58r59 = $r3[$r1] + addd $r3 = $r3, 1 + ;; + lo.xs $r60r61r62r63 = $r3[$r1] + addd $r3 = $r3, 1 + ;; + /* And then store all of them */ + so.xs $r2[$r0] = $r32r33r34r35 + addd $r2 = $r2, 1 + ;; + so.xs $r2[$r0] = $r36r37r38r39 + addd $r2 = $r2, 1 + ;; + so.xs $r2[$r0] = $r40r41r42r43 + addd $r2 = $r2, 1 + ;; + so.xs $r2[$r0] = $r44r45r46r47 + addd $r2 = $r2, 1 + ;; + so.xs $r2[$r0] = $r48r49r50r51 + addd $r2 = $r2, 1 + ;; + so.xs $r2[$r0] = $r52r53r54r55 + addd $r2 = $r2, 1 + ;; + so.xs $r2[$r0] = $r56r57r58r59 + addd $r2 = $r2, 1 + ;; + so.xs $r2[$r0] = $r60r61r62r63 + ;; + copy_page_done: + ret + ;; +ENDPROC(copy_page) diff --git a/arch/kvx/lib/delay.c b/arch/kvx/lib/delay.c new file mode 100644 index 000000000000..11295eedc3f5 --- /dev/null +++ b/arch/kvx/lib/delay.c @@ -0,0 +1,39 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include +#include + +#include +#include + +void __delay(unsigned long loops) +{ + cycles_t target_cycle = get_cycles() + loops; + + while (get_cycles() < target_cycle); +} +EXPORT_SYMBOL(__delay); + +inline void __const_udelay(unsigned long xloops) +{ + u64 loops = (u64)xloops * (u64)loops_per_jiffy * HZ; + + __delay(loops >> 32); +} +EXPORT_SYMBOL(__const_udelay); + +void __udelay(unsigned long usecs) +{ + __const_udelay(usecs * 0x10C7UL); /* 2**32 / 1000000 (rounded up) */ +} +EXPORT_SYMBOL(__udelay); + +void __ndelay(unsigned long nsecs) +{ + __const_udelay(nsecs * 0x5UL); /* 2**32 / 1000000000 (rounded up) */ +} +EXPORT_SYMBOL(__ndelay); diff --git a/arch/kvx/lib/memcpy.c b/arch/kvx/lib/memcpy.c new file mode 100644 index 000000000000..b81f746a80ee --- /dev/null +++ b/arch/kvx/lib/memcpy.c @@ -0,0 +1,70 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Yann Sionneau + */ + +#include +#include + +void *memcpy(void *dest, const void *src, size_t n) +{ + __uint128_t *tmp128_d = dest; + const __uint128_t *tmp128_s = src; + uint64_t *tmp64_d; + const uint64_t *tmp64_s; + uint32_t *tmp32_d; + const uint32_t *tmp32_s; + uint16_t *tmp16_d; + const uint16_t *tmp16_s; + uint8_t *tmp8_d; + const uint8_t *tmp8_s; + + while (n >= 16) { + *tmp128_d = *tmp128_s; + tmp128_d++; + tmp128_s++; + n -= 16; + } + + tmp64_d = (uint64_t *) tmp128_d; + tmp64_s = (uint64_t *) tmp128_s; + while (n >= 8) { + *tmp64_d = *tmp64_s; + tmp64_d++; + tmp64_s++; + n -= 8; + } + + tmp32_d = (uint32_t *) tmp64_d; + tmp32_s = (uint32_t *) tmp64_s; + while (n >= 4) { + *tmp32_d = *tmp32_s; + tmp32_d++; + tmp32_s++; + n -= 4; + } + + tmp16_d = (uint16_t *) tmp32_d; + tmp16_s = (uint16_t *) tmp32_s; + while (n >= 2) { + *tmp16_d = *tmp16_s; + tmp16_d++; + tmp16_s++; + n -= 2; + } + + tmp8_d = (uint8_t *) tmp16_d; + tmp8_s = (uint8_t *) tmp16_s; + while (n >= 1) { + *tmp8_d = *tmp8_s; + tmp8_d++; + tmp8_s++; + n--; + } + + return dest; +} +EXPORT_SYMBOL(memcpy); + diff --git a/arch/kvx/lib/memset.S b/arch/kvx/lib/memset.S new file mode 100644 index 000000000000..9eebc28da2be --- /dev/null +++ b/arch/kvx/lib/memset.S @@ -0,0 +1,351 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Marius Gligor + */ + +#include + +#include + +#define REPLICATE_BYTE_MASK 0x0101010101010101 +#define MIN_SIZE_FOR_ALIGN 128 + +/* + * Optimized memset for kvx architecture + * + * In order to optimize memset on kvx, we can use various things: + * - conditionnal store which avoid branch penalty + * - store half/word/double/quad/octuple to store up to 16 bytes at a time + * - dzerol to zero a cacheline when the pattern is '0' (often the case) + * - hardware loop for steady cases. + * + * First, we assume that memset is mainly used for zeroing areas. In order + * to optimize this case, we consider it to be the fast path of the algorithm. + * In both cases (0 and non 0 pattern), we start by checking if the size is + * below a minimum size. If so, we skip the alignment part. Indeed, the kvx + * supports misalignment and the penalty for letting it do unaligned accesses is + * lower than trying to realigning us. So for small sizes, we don't even bother + * to realign. Minor difference is that in the memset with 0, we skip after the + * dzerol loop since dzerol must be cache-line aligned (no misalignment of + * course). + * Regarding the non 0 pattern memset, we use sbmm to replicate the pattern on + * all bits on a register in one call. + * Once alignment has been reached, we can do the hardware loop for both cases( + * store octuple/dzerol) in order to optimize throughput. Care must be taken to + * align hardware loops on at least 8 bytes for performances. + * Once the main loop has been done, we finish the copy by checking length to do + * the necessary calls to store remaining bytes. + * + * Pseudo code (applies for non 0 pattern): + * + * int memset(void *dest, char pattern, long length) + * { + * long dest_align = -((long) dest); + * long copy; + * long orig_dest = dest; + * + * uint64_t pattern = sbmm8(pattern, 0x0101010101010101); + * uint128_t pattern128 = pattern << 64 | pattern; + * uint256_t pattern128 = pattern128 << 128 | pattern128; + * + * // Keep only low bits + * dest_align &= 0x1F; + * length -= dest_align; + * + * // Byte align + * copy = align & (1 << 0); + * if (copy) + * *((u8 *) dest) = pattern; + * dest += copy; + * // Half align + * copy = align & (1 << 1); + * if (copy) + * *((u16 *) dest) = pattern; + * dest += copy; + * // Word align + * copy = align & (1 << 2); + * if (copy) + * *((u32 *) dest) = pattern; + * dest += copy; + * // Double align + * copy = align & (1 << 3); + * if (copy) + * *((u64 *) dest) = pattern; + * dest += copy; + * // Quad align + * copy = align & (1 << 4); + * if (copy) + * *((u128 *) dest) = pattern128; + * dest += copy; + * + * // We are now aligned on 256 bits + * loop_octuple_count = size >> 5; + * for (i = 0; i < loop_octuple_count; i++) { + * *((u256 *) dest) = pattern256; + * dest += 32; + * } + * + * if (length == 0) + * return orig_dest; + * + * // Copy remaining part + * remain = length & (1 << 4); + * if (copy) + * *((u128 *) dest) = pattern128; + * dest += remain; + * remain = length & (1 << 3); + * if (copy) + * *((u64 *) dest) = pattern; + * dest += remain; + * remain = length & (1 << 2); + * if (copy) + * *((u32 *) dest) = pattern; + * dest += remain; + * remain = length & (1 << 1); + * if (copy) + * *((u16 *) dest) = pattern; + * dest += remain; + * remain = length & (1 << 0); + * if (copy) + * *((u8 *) dest) = pattern; + * dest += remain; + * + * return orig_dest; + * } + */ + +.text +.align 16 +ENTRY(memset): + make $r32 = 0 + make $r33 = 0 + /* Check if length < KVX_DCACHE_LINE_SIZE */ + compd.ltu $r7 = $r2, KVX_DCACHE_LINE_SIZE + /* Jump to generic memset if pattern is != 0 */ + cb.dnez $r1? memset_non_0_pattern + ;; + /* Preserve return value */ + copyd $r3 = $r0 + /* Invert address to compute size to copy to be aligned on 32 bytes */ + negd $r5 = $r0 + /* Remaining bytes for 16 bytes store (for alignment on 64 bytes) */ + andd $r8 = $r2, (1 << 5) + copyq $r34r35 = $r32, $r33 + /* Skip loopdo with dzerol if length < KVX_DCACHE_LINE_SIZE */ + cb.dnez $r7? .Ldzerol_done + ;; + /* Compute the size that will be copied to align on 64 bytes boundary */ + andw $r6 = $r5, 0x3F + /* Check if address is aligned on 64 bytes */ + andw $r9 = $r0, 0x3F + /* Alignment */ + nop + ;; + /* If address already aligned on 64 bytes, jump to dzerol loop */ + cb.deqz $r9? .Laligned_64 + /* Remove unaligned part from length */ + sbfd $r2 = $r6, $r2 + /* Check if we need to copy 1 byte */ + andw $r4 = $r5, (1 << 0) + ;; + /* If we are not aligned, store byte */ + sb.dnez $r4? [$r0] = $r32 + /* Check if we need to copy 2 bytes */ + andw $r4 = $r5, (1 << 1) + /* Add potentially copied part for next store offset */ + addd $r0 = $r0, $r4 + ;; + sh.dnez $r4? [$r0] = $r32 + /* Check if we need to copy 4 bytes */ + andw $r4 = $r5, (1 << 2) + addd $r0 = $r0, $r4 + ;; + sw.dnez $r4? [$r0] = $r32 + /* Check if we need to copy 8 bytes */ + andw $r4 = $r5, (1 << 3) + addd $r0 = $r0, $r4 + ;; + sd.dnez $r4? [$r0] = $r32 + /* Check if we need to copy 16 bytes */ + andw $r4 = $r5, (1 << 4) + addd $r0 = $r0, $r4 + ;; + sq.dnez $r4? [$r0] = $r32r33 + /* Check if we need to copy 32 bytes */ + andw $r4 = $r5, (1 << 5) + addd $r0 = $r0, $r4 + ;; + so.dnez $r4? [$r0] = $r32r33r34r35 + addd $r0 = $r0, $r4 + ;; +.Laligned_64: + /* Prepare amount of data for dzerol */ + srld $r10 = $r2, 6 + /* Size to be handled in loopdo */ + andd $r4 = $r2, ~0x3F + make $r11 = 64 + cb.deqz $r2? .Lmemset_done + ;; + /* Remaining bytes for 16 bytes store */ + andw $r8 = $r2, (1 << 5) + /* Skip dzerol if there are not enough data for 64 bytes store */ + cb.deqz $r10? .Ldzerol_done + /* Update length to copy */ + sbfd $r2 = $r4, $r2 + ;; + loopdo $r10, .Ldzerol_done + ;; + so 0[$r0], $r32r33r34r35 + ;; + so 32[$r0], $r32r33r34r35 + addd $r0 = $r0, $r11 + ;; + .Ldzerol_done: + /* + * Now that we have handled every aligned bytes using 'dzerol', we can + * handled the remainder of length using store by decrementing size + * We also exploit the fact we are aligned to simply check remaining + * size */ + so.dnez $r8? [$r0] = $r32r33r34r35 + addd $r0 = $r0, $r8 + /* Remaining bytes for 16 bytes store */ + andw $r8 = $r2, (1 << 4) + cb.deqz $r2? .Lmemset_done + ;; + sq.dnez $r8? [$r0] = $r32r33 + addd $r0 = $r0, $r8 + /* Remaining bytes for 8 bytes store */ + andw $r8 = $r2, (1 << 3) + ;; + sd.dnez $r8? [$r0] = $r32 + addd $r0 = $r0, $r8 + /* Remaining bytes for 4 bytes store */ + andw $r8 = $r2, (1 << 2) + ;; + sw.dnez $r8? [$r0] = $r32 + addd $r0 = $r0, $r8 + /* Remaining bytes for 2 bytes store */ + andw $r8 = $r2, (1 << 1) + ;; + sh.dnez $r8? [$r0] = $r32 + addd $r0 = $r0, $r8 + ;; + sb.odd $r2? [$r0] = $r32 + /* Restore original value */ + copyd $r0 = $r3 + ret + ;; + +.align 16 +memset_non_0_pattern: + /* Preserve return value */ + copyd $r3 = $r0 + /* Replicate the first pattern byte on all bytes */ + sbmm8 $r32 = $r1, REPLICATE_BYTE_MASK + /* Check if length < MIN_SIZE_FOR_ALIGN */ + compd.geu $r7 = $r2, MIN_SIZE_FOR_ALIGN + /* Invert address to compute size to copy to be aligned on 32 bytes */ + negd $r5 = $r0 + ;; + /* Check if we are aligned on 32 bytes */ + andw $r9 = $r0, 0x1F + /* Compute the size that will be copied to align on 32 bytes boundary */ + andw $r6 = $r5, 0x1F + /* + * If size < MIN_SIZE_FOR_ALIGN bits, directly go to so, it will be done + * unaligned but that is still better that what we can do with sb + */ + cb.deqz $r7? .Laligned_32 + ;; + /* Remove unaligned part from length */ + sbfd $r2 = $r6, $r2 + /* If we are already aligned on 32 bytes, jump to main "so" loop */ + cb.deqz $r9? .Laligned_32 + /* Check if we need to copy 1 byte */ + andw $r4 = $r5, (1 << 0) + ;; + /* If we are not aligned, store byte */ + sb.dnez $r4? [$r0] = $r32 + /* Check if we need to copy 2 bytes */ + andw $r4 = $r5, (1 << 1) + /* Add potentially copied part for next store offset */ + addd $r0 = $r0, $r4 + ;; + sh.dnez $r4? [$r0] = $r32 + /* Check if we need to copy 4 bytes */ + andw $r4 = $r5, (1 << 2) + addd $r0 = $r0, $r4 + ;; + sw.dnez $r4? [$r0] = $r32 + /* Check if we need to copy 8 bytes */ + andw $r4 = $r5, (1 << 3) + addd $r0 = $r0, $r4 + /* Copy second part of pattern for sq */ + copyd $r33 = $r32 + ;; + sd.dnez $r4? [$r0] = $r32 + /* Check if we need to copy 16 bytes */ + andw $r4 = $r5, (1 << 4) + addd $r0 = $r0, $r4 + ;; + sq.dnez $r4? [$r0] = $r32r33 + addd $r0 = $r0, $r4 + ;; +.Laligned_32: + /* Copy second part of pattern for sq */ + copyd $r33 = $r32 + /* Prepare amount of data for 32 bytes store */ + srld $r10 = $r2, 5 + nop + nop + ;; + copyq $r34r35 = $r32, $r33 + /* Remaining bytes for 16 bytes store */ + andw $r8 = $r2, (1 << 4) + make $r11 = 32 + /* Check if there are enough data for 32 bytes store */ + cb.deqz $r10? .Laligned_32_done + ;; + loopdo $r10, .Laligned_32_done + ;; + so 0[$r0] = $r32r33r34r35 + addd $r0 = $r0, $r11 + ;; + .Laligned_32_done: + /* + * Now that we have handled every aligned bytes using 'so', we can + * handled the remainder of length using store by decrementing size + * We also exploit the fact we are aligned to simply check remaining + * size */ + sq.dnez $r8? [$r0] = $r32r33 + addd $r0 = $r0, $r8 + /* Remaining bytes for 8 bytes store */ + andw $r8 = $r2, (1 << 3) + cb.deqz $r2? .Lmemset_done + ;; + sd.dnez $r8? [$r0] = $r32 + addd $r0 = $r0, $r8 + /* Remaining bytes for 4 bytes store */ + andw $r8 = $r2, (1 << 2) + ;; + sw.dnez $r8? [$r0] = $r32 + addd $r0 = $r0, $r8 + /* Remaining bytes for 2 bytes store */ + andw $r8 = $r2, (1 << 1) + ;; + sh.dnez $r8? [$r0] = $r32 + addd $r0 = $r0, $r8 + ;; + sb.odd $r2? [$r0] = $r32 + /* Restore original value */ + copyd $r0 = $r3 + ret + ;; +.Lmemset_done: + /* Restore original value */ + copyd $r0 = $r3 + ret + ;; +ENDPROC(memset) diff --git a/arch/kvx/lib/strlen.S b/arch/kvx/lib/strlen.S new file mode 100644 index 000000000000..8298402a7898 --- /dev/null +++ b/arch/kvx/lib/strlen.S @@ -0,0 +1,122 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Jules Maselbas + */ +#include +#include + +/* + * kvx optimized strlen + * + * This implementation of strlen only does aligned memory accesses. + * Since we don't know the total length the idea is to do double word + * load and stop on the first null byte found. As it's always safe to + * read more up to a lower 8-bytes boundary. + * + * This implementation of strlen uses a trick to detect if a double + * word contains a null byte [1]: + * + * > #define haszero(v) (((v) - 0x01010101UL) & ~(v) & 0x80808080UL) + * > The sub-expression (v - 0x01010101UL), evaluates to a high bit set + * > in any byte whenever the corresponding byte in v is zero or greater + * > than 0x80. The sub-expression ~v & 0x80808080UL evaluates to high + * > bits set in bytes where the byte of v doesn't have its high bit set + * > (so the byte was less than 0x80). Finally, by ANDing these two sub- + * > expressions the result is the high bits set where the bytes in v + * > were zero, since the high bits set due to a value greater than 0x80 + * > in the first sub-expression are masked off by the second. + * + * [1] http://graphics.stanford.edu/~seander/bithacks.html#ZeroInWord + * + * A second trick is used to get the exact number of characters before + * the first null byte in a double word: + * + * clz(sbmmt(zero, 0x0102040810204080)) + * + * This trick uses the haszero result which maps null byte to 0x80 and + * others value to 0x00. The idea is to count the number of consecutive + * null byte in the double word (counting from less significant byte + * to most significant byte). To do so, using the bit matrix transpose + * will "pack" all high bit (0x80) to the most significant byte (MSB). + * It is not possible to count the trailing zeros in this MSB, however + * if a byte swap is done before the bit matrix transpose we still have + * all the information in the MSB but now we can count the leading zeros. + * The instruction sbmmt with the matrix 0x0102040810204080 does exactly + * what we need a byte swap followed by a bit transpose. + * + * A last trick is used to handle the first double word misalignment. + * This is done by masking off the N lower bytes (excess read) with N + * between 0 and 7. The mask is applied on haszero results and will + * force the N lower bytes to be considered not null. + * + * This is a C implementation of the algorithm described above: + * + * size_t strlen(char *s) { + * uint64_t *p = (uint64_t *)((uintptr_t)s) & ~0x7; + * uint64_t rem = ((uintptr_t)s) % 8; + * uint64_t low = -0x0101010101010101; + * uint64_t high = 0x8080808080808080; + * uint64_t dword, zero; + * uint64_t msk, len; + * + * dword = *p++; + * zero = (dword + low) & ~dword & high; + * msk = 0xffffffffffffffff << (rem * 8); + * zero &= msk; + * + * while (!zero) { + * dword = *p++; + * zero = (dword + low) & ~dword & high; + * } + * + * zero = __builtin_kvx_sbmmt8(zero, 0x0102040810204080); + * len = ((void *)p - (void *)s) - 8; + * len += __builtin_kvx_clzd(zero); + * + * return len; + * } + */ + +.text +.align 16 +ENTRY(strlen) + andd $r1 = $r0, ~0x7 + andd $r2 = $r0, 0x7 + make $r10 = -0x0101010101010101 + make $r11 = 0x8080808080808080 + ;; + ld $r4 = 0[$r1] + sllw $r2 = $r2, 3 + make $r3 = 0xffffffffffffffff + ;; + slld $r2 = $r3, $r2 + addd $r5 = $r4, $r10 + andnd $r6 = $r4, $r11 + ;; + andd $r6 = $r6, $r2 + make $r3 = 0 + ;; +.loop: + andd $r4 = $r5, $r6 + addd $r1 = $r1, 0x8 + ;; + cb.dnez $r4? .end + ld.deqz $r4? $r4 = [$r1] + ;; + addd $r5 = $r4, $r10 + andnd $r6 = $r4, $r11 + goto .loop + ;; +.end: + addd $r1 = $r1, -0x8 + sbmmt8 $r4 = $r4, 0x0102040810204080 + ;; + clzd $r4 = $r4 + sbfd $r1 = $r0, $r1 + ;; + addd $r0 = $r4, $r1 + ret + ;; +ENDPROC(strlen) +EXPORT_SYMBOL(strlen) diff --git a/arch/kvx/lib/usercopy.S b/arch/kvx/lib/usercopy.S new file mode 100644 index 000000000000..bc7e1a45e1c7 --- /dev/null +++ b/arch/kvx/lib/usercopy.S @@ -0,0 +1,90 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ +#include + +/** + * Copy from/to a user buffer + * r0 = to buffer + * r1 = from buffer + * r2 = size to copy + * This function can trapped when hitting a non-mapped page. + * It will trigger a trap NOMAPPING and the trap handler will interpret + * it and check if instruction pointer is inside __ex_table. + * The next step are described later ! + */ +.text +ENTRY(raw_copy_from_user) +ENTRY(raw_copy_to_user) + /** + * naive implementation byte per byte + */ + make $r33 = 0x0; + /* If size == 0, exit directly */ + cb.deqz $r2? copy_exit + ;; + loopdo $r2, copy_exit + ;; +0: lbz $r34 = $r33[$r1] + ;; +1: sb $r33[$r0] = $r34 + addd $r33 = $r33, 1 /* Ptr increment */ + addd $r2 = $r2, -1 /* Remaining bytes to copy */ + ;; + copy_exit: + copyd $r0 = $r2 + ret + ;; +ENDPROC(raw_copy_to_user) +ENDPROC(raw_copy_from_user) + +/** + * Exception table + * each entry correspond to the following: + * .dword trapping_addr, restore_addr + * + * On trap, the handler will try to locate if $spc is matching a + * trapping address in the exception table. If so, the restore addr + * will be put in the return address of the trap handler, allowing + * to properly finish the copy and return only the bytes copied/cleared + */ +.pushsection __ex_table,"a" +.balign 8 +.dword 0b, copy_exit +.dword 1b, copy_exit +.popsection + +/** + * Clear a user buffer + * r0 = buffer to clear + * r1 = size to clear + */ +.text +ENTRY(asm_clear_user) + /** + * naive implementation byte per byte + */ + make $r33 = 0x0; + make $r34 = 0x0; + /* If size == 0, exit directly */ + cb.deqz $r1? clear_exit + ;; + loopdo $r1, clear_exit + ;; +40: sb $r33[$r0] = $r34 + addd $r33 = $r33, 1 /* Ptr increment */ + addd $r1 = $r1, -1 /* Remaining bytes to copy */ + ;; + clear_exit: + copyd $r0 = $r1 + ret + ;; +ENDPROC(asm_clear_user) + +.pushsection __ex_table,"a" +.balign 8 +.dword 40b, clear_exit +.popsection + From patchwork Fri Jan 20 14:09:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13110023 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80D3CC54794 for ; Fri, 20 Jan 2023 14:36:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231216AbjATOgH (ORCPT ); Fri, 20 Jan 2023 09:36:07 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39312 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231262AbjATOgB (ORCPT ); Fri, 20 Jan 2023 09:36:01 -0500 Received: from fx601.security-mail.net (smtpout140.security-mail.net [85.31.212.146]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D193DCE220 for ; Fri, 20 Jan 2023 06:35:15 -0800 (PST) Received: from localhost (fx601.security-mail.net [127.0.0.1]) by fx601.security-mail.net (Postfix) with ESMTP id 3396B3497F3 for ; Fri, 20 Jan 2023 15:20:47 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674224447; bh=bNmYslkbuquxKAj+Znp8eugUmqYemSNASVZgyAJPx68=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=A+h3eu0UFyHNduTYB3NIv69sDB8fnzZ6N90iu5Dv+x2BBUTriSwjOVxpdmeKzeuJR kNPsaF7l07CFLVRfnjhH4qCydeAmZVnwiF4ryulEXhqswmMSi0VP+goU3ppX+9zjUe v+8Y+NRWEcqu5UFa9XwN0UInGcAL2sfSCu5M6lAo= Received: from fx601 (fx601.security-mail.net [127.0.0.1]) by fx601.security-mail.net (Postfix) with ESMTP id DAE483498AD; Fri, 20 Jan 2023 15:20:45 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx601.security-mail.net (Postfix) with ESMTPS id A493E3497CD; Fri, 20 Jan 2023 15:20:44 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 3AAC327E0461; Fri, 20 Jan 2023 15:10:37 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 20DDD27E0456; Fri, 20 Jan 2023 15:10:37 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id O9iQteuEJcJb; Fri, 20 Jan 2023 15:10:37 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 9550C27E0457; Fri, 20 Jan 2023 15:10:36 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <5c86.63caa33c.a2d74.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 20DDD27E0456 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223837; bh=3OWgg5n0jIy60CSKPsLXfDN5oeIG50h2WPbisKqx88I=; h=From:To:Date:Message-Id:MIME-Version; b=ha+njJPrX4jOZJ1/7CAya1KsiiCG+CEgarQekDqLqd2GgjUmheDnGnk7n4MPvjURn ZvRTniHv/MFR3PJRWqisBvTz8xd3DrHxIqcwqD8l4XW6bT/pQz3cSyEDB+1mKQh7op qOvusDQL3LvcPjBMP4WfmkcjLkmthT/TRFDXAjV0= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 26/31] kvx: Add multi-processor (SMP) support Date: Fri, 20 Jan 2023 15:09:57 +0100 Message-ID: <20230120141002.2442-27-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Coolidge v1 SoC has 5 clusters of 17 kvx cores: - 16 application cores aka PE - 1 privileged core, the Resource Manager, aka RM. Linux can run in SMP config on the 16 cores of a Cluster. Memory coherency between all cores is guarenteed by the L2 cache. Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Julien Hascoet Signed-off-by: Julien Hascoet Co-developed-by: Louis Morhet Signed-off-by: Louis Morhet Co-developed-by: Luc Michel Signed-off-by: Luc Michel Co-developed-by: Marius Gligor Signed-off-by: Marius Gligor Co-developed-by: Yann Sionneau Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: - removed L2 cache driver - removed ipi and pwr-ctrl driver (splitted in their own patch) arch/kvx/include/asm/smp.h | 42 ++++++++++++ arch/kvx/kernel/smp.c | 110 ++++++++++++++++++++++++++++++++ arch/kvx/kernel/smpboot.c | 127 +++++++++++++++++++++++++++++++++++++ include/linux/cpuhotplug.h | 2 + 4 files changed, 281 insertions(+) create mode 100644 arch/kvx/include/asm/smp.h create mode 100644 arch/kvx/kernel/smp.c create mode 100644 arch/kvx/kernel/smpboot.c diff --git a/arch/kvx/include/asm/smp.h b/arch/kvx/include/asm/smp.h new file mode 100644 index 000000000000..e4fd4d001b2c --- /dev/null +++ b/arch/kvx/include/asm/smp.h @@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_SMP_H +#define _ASM_KVX_SMP_H + +#include +#include + +#include + +#ifdef CONFIG_SMP + +/* Hook for the generic smp_call_function_many() routine. */ +void arch_send_call_function_ipi_mask(struct cpumask *mask); + +/* Hook for the generic smp_call_function_single() routine. */ +void arch_send_call_function_single_ipi(int cpu); + +void __init setup_processor(void); + +void smp_init_cpus(void); + +irqreturn_t ipi_call_interrupt(int irq, void *dev_id); + +#define raw_smp_processor_id() ((int) \ + ((kvx_sfr_get(PCR) & KVX_SFR_PCR_PID_MASK) \ + >> KVX_SFR_PCR_PID_SHIFT)) + +#define flush_cache_vmap(start, end) do { } while (0) +#define flush_cache_vunmap(start, end) do { } while (0) + +#else + +void smp_init_cpus(void) {} + +#endif /* CONFIG_SMP */ + +#endif diff --git a/arch/kvx/kernel/smp.c b/arch/kvx/kernel/smp.c new file mode 100644 index 000000000000..ed4c35a8c4bc --- /dev/null +++ b/arch/kvx/kernel/smp.c @@ -0,0 +1,110 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +enum ipi_message_type { + IPI_RESCHEDULE, + IPI_CALL_FUNC, + IPI_IRQ_WORK, + IPI_MAX +}; + +/* A collection of single bit ipi messages. */ +static struct { + unsigned long bits ____cacheline_aligned; +} ipi_data[NR_CPUS] __cacheline_aligned; + +static void send_ipi_message(const struct cpumask *mask, + enum ipi_message_type operation) +{ + unsigned long flags; + int cpu; + + /* Set operation that must be done by receiver */ + for_each_cpu(cpu, mask) + set_bit(operation, &ipi_data[cpu].bits); + + /* Commit the write before sending IPI */ + smp_wmb(); + + local_irq_save(flags); + + kvx_ipi_send(mask); + + local_irq_restore(flags); +} + +void arch_send_call_function_ipi_mask(struct cpumask *mask) +{ + send_ipi_message(mask, IPI_CALL_FUNC); +} + +void arch_send_call_function_single_ipi(int cpu) +{ + send_ipi_message(cpumask_of(cpu), IPI_CALL_FUNC); +} + +#ifdef CONFIG_IRQ_WORK +void arch_irq_work_raise(void) +{ + send_ipi_message(cpumask_of(smp_processor_id()), IPI_IRQ_WORK); +} +#endif + +static void ipi_stop(void *unused) +{ + local_cpu_stop(); +} + +void smp_send_stop(void) +{ + struct cpumask targets; + + cpumask_copy(&targets, cpu_online_mask); + cpumask_clear_cpu(smp_processor_id(), &targets); + + smp_call_function_many(&targets, ipi_stop, NULL, 0); +} + +void smp_send_reschedule(int cpu) +{ + send_ipi_message(cpumask_of(cpu), IPI_RESCHEDULE); +} + +irqreturn_t ipi_call_interrupt(int irq, void *dev_id) +{ + unsigned long *pending_ipis = &ipi_data[smp_processor_id()].bits; + + while (true) { + unsigned long ops = xchg(pending_ipis, 0); + + if (ops == 0) + return IRQ_HANDLED; + + if (ops & (1 << IPI_RESCHEDULE)) + scheduler_ipi(); + + if (ops & (1 << IPI_CALL_FUNC)) + generic_smp_call_function_interrupt(); + + if (ops & (1 << IPI_IRQ_WORK)) + irq_work_run(); + + BUG_ON((ops >> IPI_MAX) != 0); + } + + return IRQ_HANDLED; +} diff --git a/arch/kvx/kernel/smpboot.c b/arch/kvx/kernel/smpboot.c new file mode 100644 index 000000000000..987a6f014163 --- /dev/null +++ b/arch/kvx/kernel/smpboot.c @@ -0,0 +1,127 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Julian Vetter + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +void *__cpu_up_stack_pointer[NR_CPUS]; +void *__cpu_up_task_pointer[NR_CPUS]; + +void __init smp_prepare_boot_cpu(void) +{ +} + +int __cpu_up(unsigned int cpu, struct task_struct *tidle) +{ + __cpu_up_stack_pointer[cpu] = task_stack_page(tidle) + THREAD_SIZE; + __cpu_up_task_pointer[cpu] = tidle; + /* We need to be sure writes are committed */ + smp_mb(); + + kvx_pwr_ctrl_cpu_poweron(cpu); + while (!cpu_online(cpu)) + cpu_relax(); + + return 0; +} + +void __init smp_cpus_done(unsigned int max_cpus) +{ +} + +void __init smp_init_cpus(void) +{ + struct cpumask cpumask; + struct device_node *cpu; + const __be32 *reg; + u32 cpu_num; + unsigned int nr_cpus = 0; + + cpumask_clear(&cpumask); + + for_each_of_cpu_node(cpu) { + if (!of_device_is_available(cpu)) + continue; + + reg = of_get_property(cpu, "reg", NULL); + if (!reg) + continue; + + cpu_num = be32_to_cpup(reg); + if (cpu_num >= nr_cpu_ids) + continue; + + nr_cpus++; + cpumask_set_cpu(cpu_num, &cpumask); + } + + pr_info("%d possible cpus\n", nr_cpus); + init_cpu_possible(&cpumask); +} + +void __init smp_prepare_cpus(unsigned int max_cpus) +{ + if (num_present_cpus() <= 1) + init_cpu_present(cpu_possible_mask); +} + +int __init setup_smp(void) +{ + int ret; + + ret = kvx_pwr_ctrl_probe(); + if (ret) + panic("Failed to probe power controller !"); + + ret = kvx_ipi_ctrl_probe(ipi_call_interrupt); + if (ret) + panic("Failed to probe IPI controller !"); + + return 0; +} + +early_initcall(setup_smp); + +/* + * C entry point for a secondary processor. + */ +void __init start_kernel_secondary(void) +{ + struct mm_struct *mm = &init_mm; + unsigned int cpu = smp_processor_id(); + + setup_processor(); + kvx_mmu_early_setup(); + + /* All kernel threads share the same mm context. */ + mmgrab(mm); + current->active_mm = mm; + cpumask_set_cpu(cpu, mm_cpumask(mm)); + + notify_cpu_starting(cpu); + set_cpu_online(cpu, true); + trace_hardirqs_off(); + + local_flush_tlb_all(); + + local_irq_enable(); + cpu_startup_entry(CPUHP_AP_ONLINE_IDLE); +} diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index f61447913db9..f5a484547b15 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -152,6 +152,7 @@ enum cpuhp_state { CPUHP_AP_IRQ_RISCV_STARTING, CPUHP_AP_IRQ_LOONGARCH_STARTING, CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING, + CPUHP_AP_IRQ_KVX_STARTING, CPUHP_AP_ARM_MVEBU_COHERENCY, CPUHP_AP_MICROCODE_LOADER, CPUHP_AP_PERF_X86_AMD_UNCORE_STARTING, @@ -189,6 +190,7 @@ enum cpuhp_state { CPUHP_AP_KVM_ARM_VGIC_INIT_STARTING, CPUHP_AP_KVM_ARM_VGIC_STARTING, CPUHP_AP_KVM_ARM_TIMER_STARTING, + CPUHP_AP_KVX_TIMER_STARTING, /* Must be the last timer callback */ CPUHP_AP_DUMMY_TIMER_STARTING, CPUHP_AP_ARM_XEN_STARTING, From patchwork Fri Jan 20 14:09:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13110019 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3031C38141 for ; Fri, 20 Jan 2023 14:35:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231225AbjATOfw (ORCPT ); Fri, 20 Jan 2023 09:35:52 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38098 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230387AbjATOfV (ORCPT ); Fri, 20 Jan 2023 09:35:21 -0500 Received: from fx306.security-mail.net (smtpout30.security-mail.net [85.31.212.36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 241857EE7D for ; Fri, 20 Jan 2023 06:35:04 -0800 (PST) Received: from localhost (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id 460D135CEED for ; Fri, 20 Jan 2023 15:20:46 +0100 (CET) Received: from fx306 (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id A206C35CD7F; Fri, 20 Jan 2023 15:20:45 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx306.security-mail.net (Postfix) with ESMTPS id 21C5235CD0C; Fri, 20 Jan 2023 15:20:45 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 9769527E0465; Fri, 20 Jan 2023 15:10:37 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 807A927E0456; Fri, 20 Jan 2023 15:10:37 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id jR-SNzcBpINi; Fri, 20 Jan 2023 15:10:37 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 1C6A727E0458; Fri, 20 Jan 2023 15:10:37 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <1035.63caa33d.1d517.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 807A927E0456 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223837; bh=hCXuqd/Le9iT4FAC9E5Y/iYkTg6Ipsgz39vkI/jYJ6E=; h=From:To:Date:Message-Id:MIME-Version; b=d0H3rQM+qImcDnObPenUoBefyEzmnq190byge1ZFC/qL2bWg8k4nnIuIikgl7XZDc fbaEx4Uyp7pzRNsTIh/9Yf6mcfd2SFEb9m+VO/GbWj8naVs6UrcjtE/E6ppNgfaWaU eNpGDhaaNyqtF03qNdt9fBQg6exSSj2xAJXMDC3o= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 27/31] kvx: Add kvx default config file Date: Fri, 20 Jan 2023 15:09:58 +0100 Message-ID: <20230120141002.2442-28-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Add a default config file for kvx based Coolidge SoC. Co-developed-by: Ashley Lesdalons Signed-off-by: Ashley Lesdalons Co-developed-by: Benjamin Mugnier Signed-off-by: Benjamin Mugnier Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Guillaume Thouvenin Signed-off-by: Guillaume Thouvenin Co-developed-by: Jules Maselbas Signed-off-by: Jules Maselbas Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Samuel Jones Signed-off-by: Samuel Jones Co-developed-by: Thomas Costis Signed-off-by: Thomas Costis Co-developed-by: Vincent Chardon Signed-off-by: Vincent Chardon Co-developed-by: Yann Sionneau Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: default_defconfig renamed to defconfig arch/kvx/configs/defconfig | 127 +++++++++++++++++++++++++++++++++++++ 1 file changed, 127 insertions(+) create mode 100644 arch/kvx/configs/defconfig diff --git a/arch/kvx/configs/defconfig b/arch/kvx/configs/defconfig new file mode 100644 index 000000000000..960784da0b1b --- /dev/null +++ b/arch/kvx/configs/defconfig @@ -0,0 +1,127 @@ +CONFIG_DEFAULT_HOSTNAME="KVXlinux" +CONFIG_SERIAL_KVX_SCALL_COMM=y +CONFIG_CONFIGFS_FS=y +CONFIG_DEBUG_KERNEL=y +CONFIG_DEBUG_INFO=y +CONFIG_DEBUG_INFO_DWARF4=y +CONFIG_PRINTK_TIME=y +CONFIG_CONSOLE_LOGLEVEL_DEFAULT=15 +CONFIG_MESSAGE_LOGLEVEL_DEFAULT=7 +CONFIG_PANIC_TIMEOUT=-1 +CONFIG_BLK_DEV_INITRD=y +CONFIG_GDB_SCRIPTS=y +CONFIG_FRAME_POINTER=y +CONFIG_HZ_100=y +CONFIG_SERIAL_EARLYCON=y +CONFIG_HOTPLUG_PCI_PCIE=y +CONFIG_PCIEAER=y +CONFIG_PCIE_DPC=y +CONFIG_HOTPLUG_PCI=y +CONFIG_SERIAL_8250=y +CONFIG_SERIAL_8250_CONSOLE=y +CONFIG_SERIAL_8250_DW=y +CONFIG_SERIAL_8250_NR_UARTS=8 +CONFIG_SERIAL_8250_RUNTIME_UARTS=8 +CONFIG_PINCTRL=y +CONFIG_PINCTRL_SINGLE=y +CONFIG_POWER_RESET_KVX_SCALL_POWEROFF=y +CONFIG_PCI=y +CONFIG_PCI_MSI=y +CONFIG_PCIE_KVX_NWL=y +CONFIG_PCIEPORTBUS=y +# CONFIG_PCIEASPM is not set +CONFIG_PCIEAER_INJECT=y +CONFIG_TMPFS=y +CONFIG_DMADEVICES=y +CONFIG_KVX_DMA_NOC=m +CONFIG_KVX_IOMMU=y +CONFIG_KVX_OTP_NV=y +CONFIG_PACKET=y +CONFIG_NET=y +# CONFIG_WLAN is not set +CONFIG_INET=y +CONFIG_IPV6=y +CONFIG_NETDEVICES=y +CONFIG_NET_CORE=y +CONFIG_E1000E=y +CONFIG_BLK_DEV_NVME=y +CONFIG_VFAT_FS=y +CONFIG_NLS_DEFAULT="iso8859-1" +CONFIG_NLS_CODEPAGE_437=y +CONFIG_NLS_ISO8859_1=y +CONFIG_WATCHDOG=y +CONFIG_KVX_WATCHDOG=y +CONFIG_HUGETLBFS=y +CONFIG_MAILBOX=y +CONFIG_KVX_MBOX=y +CONFIG_REMOTEPROC=y +CONFIG_KVX_REMOTEPROC=y +CONFIG_VIRTIO_NET=y +CONFIG_VIRTIO_MMIO=y +CONFIG_RPMSG_VIRTIO=y +CONFIG_RPMSG_CHAR=y +CONFIG_MODULES=y +CONFIG_MODULE_UNLOAD=y +CONFIG_MODVERSIONS=y +CONFIG_MODULE_SRCVERSION_ALL=y +CONFIG_BLK_DEV=y +CONFIG_BLK_DEV_LOOP=m +CONFIG_BLK_DEV_LOOP_MIN_COUNT=8 +CONFIG_EXT4_FS=m +CONFIG_EXT4_USE_FOR_EXT2=y +CONFIG_SYSVIPC=y +CONFIG_UNIX=y +CONFIG_NET_VENDOR_KALRAY=y +CONFIG_NET_KVX_SOC=m +CONFIG_STACKPROTECTOR=y +CONFIG_GPIO_DWAPB=y +CONFIG_I2C=y +CONFIG_I2C_SLAVE=y +CONFIG_I2C_CHARDEV=y +CONFIG_I2C_DESIGNWARE_PLATFORM=y +CONFIG_I2C_DESIGNWARE_CORE=y +CONFIG_I2C_DESIGNWARE_SLAVE=y +CONFIG_I2C_SLAVE_USPACE=y +CONFIG_POWER_RESET=y +CONFIG_POWER_RESET_SYSCON=y +CONFIG_SPI=y +CONFIG_SPI_DESIGNWARE=y +CONFIG_SPI_DW_MMIO=y +CONFIG_SPI_DW_KVX=y +CONFIG_MTD=y +CONFIG_MTD_SPI_NOR=y +# CONFIG_MTD_SPI_NOR_USE_4K_SECTORS is not set +CONFIG_SQUASHFS=m +CONFIG_USB=y +CONFIG_USB_CONFIGFS=m +CONFIG_USB_CONFIGFS_ACM=y +CONFIG_USB_CONFIGFS_ECM=y +CONFIG_USB_DWC2=y +CONFIG_USB_DWC2_DUAL_ROLE=y +CONFIG_USB_GADGET=y +CONFIG_U_SERIAL_CONSOLE=y +CONFIG_USB_USBNET=m +CONFIG_USB_NET_SMSC95XX=m +# CONFIG_NOP_USB_XCEIV is not set +CONFIG_USB_PHY=y +CONFIG_GENERIC_PHY=y +CONFIG_GENERIC_PHY_USB=y +CONFIG_MMC=y +CONFIG_MMC_SDHCI=y +CONFIG_MMC_SDHCI_PLTFM=y +CONFIG_MMC_SDHCI_OF_DWCMSHC=y +CONFIG_MDIO_BITBANG=m +CONFIG_MDIO_GPIO=m +CONFIG_MARVELL_PHY=m +CONFIG_GPIO_PCA953X=y +CONFIG_NETFILTER=y +CONFIG_NF_CONNTRACK=m +CONFIG_NF_NAT=m +CONFIG_IP_NF_IPTABLES=m +CONFIG_IP_NF_NAT=m +CONFIG_LEDS_GPIO=y +CONFIG_LEDS_CLASS=y +CONFIG_LEDS_TRIGGERS=y +CONFIG_LEDS_TRIGGER_NETDEV=y +CONFIG_LEDS_TRIGGER_PATTERN=y +CONFIG_DCB=y From patchwork Fri Jan 20 14:09:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13110008 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12C4CC27C7C for ; Fri, 20 Jan 2023 14:25:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231197AbjATOZk (ORCPT ); Fri, 20 Jan 2023 09:25:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55834 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230262AbjATOZh (ORCPT ); Fri, 20 Jan 2023 09:25:37 -0500 Received: from fx303.security-mail.net (mxout.security-mail.net [85.31.212.46]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8FFEE44BC7 for ; Fri, 20 Jan 2023 06:25:14 -0800 (PST) Received: from localhost (fx303.security-mail.net [127.0.0.1]) by fx303.security-mail.net (Postfix) with ESMTP id 60D4530FAD0 for ; Fri, 20 Jan 2023 15:20:46 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674224446; bh=pOkBOIfM3JMD67ygOoVKu1eQ66cv53r7ZKUe6yP7yBY=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=QCZGQaIysX3VnwI0oLpXH0rg5WKYZkIc53LdJOnrKJmUsY2Ahi1V3e0CX95xiYv6W Vy7d+FJsAPp1Tsu7VGvpPncstW9Kf8Np3nAfzeLBn96w1BvtLuelP+vbpbNcjeU6FH pVd2VgAgtkBTukPPwGJhR4NKQKtXkZjepzX4umZM= Received: from fx303 (fx303.security-mail.net [127.0.0.1]) by fx303.security-mail.net (Postfix) with ESMTP id 2736B30F7B0; Fri, 20 Jan 2023 15:20:46 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx303.security-mail.net (Postfix) with ESMTPS id 259B630F7C0; Fri, 20 Jan 2023 15:20:44 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 05DAE27E046C; Fri, 20 Jan 2023 15:10:38 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id E1B3627E0457; Fri, 20 Jan 2023 15:10:37 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id IvWjvNbLtKjb; Fri, 20 Jan 2023 15:10:37 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 7A7EF27E045E; Fri, 20 Jan 2023 15:10:37 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <168e6.63caa33c.bea66.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu E1B3627E0457 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223837; bh=MWFUeJSXqLDohlytQZd/iLpZT+G8twsG/8uQn0QCk30=; h=From:To:Date:Message-Id:MIME-Version; b=UuqinM2S2Qepgl5gcemf89cU/+xmR/I2NOCerz0NYOkRRoqf6tMhwh0YdjfOwNPhP hbPwi7gXupTxq/rpKVzY6hyE2mFc74OosyuP0A+u0C+ldJGkF24OtxkYoTbCSVNxcn 2u/tBn13StGHmMLWcwz/mTNrZBsYy2a8sijheQtQ= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 28/31] kvx: Add debugging related support Date: Fri, 20 Jan 2023 15:09:59 +0100 Message-ID: <20230120141002.2442-29-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Add kvx support for debugging Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Guillaume Thouvenin Signed-off-by: Guillaume Thouvenin Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Marius Gligor Signed-off-by: Marius Gligor Co-developed-by: Yann Sionneau Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: no changes arch/kvx/include/asm/debug.h | 35 ++++++ arch/kvx/include/asm/insns.h | 16 +++ arch/kvx/include/asm/insns_defs.h | 197 ++++++++++++++++++++++++++++++ arch/kvx/kernel/break_hook.c | 76 ++++++++++++ arch/kvx/kernel/debug.c | 64 ++++++++++ arch/kvx/kernel/insns.c | 144 ++++++++++++++++++++++ 6 files changed, 532 insertions(+) create mode 100644 arch/kvx/include/asm/debug.h create mode 100644 arch/kvx/include/asm/insns.h create mode 100644 arch/kvx/include/asm/insns_defs.h create mode 100644 arch/kvx/kernel/break_hook.c create mode 100644 arch/kvx/kernel/debug.c create mode 100644 arch/kvx/kernel/insns.c diff --git a/arch/kvx/include/asm/debug.h b/arch/kvx/include/asm/debug.h new file mode 100644 index 000000000000..f60c632e6697 --- /dev/null +++ b/arch/kvx/include/asm/debug.h @@ -0,0 +1,35 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef __ASM_KVX_DEBUG_HOOK_H_ +#define __ASM_KVX_DEBUG_HOOK_H_ + +/** + * enum debug_ret - Break return value + * @DEBUG_HOOK_HANDLED: Hook handled successfully + * @DEBUG_HOOK_IGNORED: Hook call has been ignored + */ +enum debug_ret { + DEBUG_HOOK_HANDLED = 0, + DEBUG_HOOK_IGNORED = 1, +}; + +/** + * struct debug_hook - Debug hook description + * @node: List node + * @handler: handler called on debug entry + * @mode: Hook mode (user/kernel) + */ +struct debug_hook { + struct list_head node; + int (*handler)(u64 ea, struct pt_regs *regs); + u8 mode; +}; + +void debug_hook_register(struct debug_hook *dbg_hook); +void debug_hook_unregister(struct debug_hook *dbg_hook); + +#endif /* __ASM_KVX_DEBUG_HOOK_H_ */ diff --git a/arch/kvx/include/asm/insns.h b/arch/kvx/include/asm/insns.h new file mode 100644 index 000000000000..36a9e8335ce8 --- /dev/null +++ b/arch/kvx/include/asm/insns.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_INSNS_H +#define _ASM_KVX_INSNS_H + +int kvx_insns_write_nostop(u32 *insns, u8 insns_len, u32 *insn_addr); + +int kvx_insns_write(u32 *insns, unsigned long insns_len, u32 *addr); + +int kvx_insns_read(u32 *insns, unsigned long insns_len, u32 *addr); + +#endif diff --git a/arch/kvx/include/asm/insns_defs.h b/arch/kvx/include/asm/insns_defs.h new file mode 100644 index 000000000000..ed8d9d6f0817 --- /dev/null +++ b/arch/kvx/include/asm/insns_defs.h @@ -0,0 +1,197 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + */ + +#ifndef __ASM_KVX_INSNS_DEFS_H_ +#define __ASM_KVX_INSNS_DEFS_H_ + +#include + +#ifndef __ASSEMBLY__ +static inline int check_signed_imm(long long imm, int bits) +{ + long long min, max; + + min = -BIT_ULL(bits - 1); + max = BIT_ULL(bits - 1) - 1; + if (imm < min || imm > max) + return 1; + + return 0; +} +#endif /* __ASSEMBLY__ */ + +#define BITMASK(bits) (BIT_ULL(bits) - 1) + +#define KVX_INSN_SYLLABLE_WIDTH 4 + +#define IS_INSN(__insn, __mnemo) \ + ((__insn & KVX_INSN_ ## __mnemo ## _MASK_0) == \ + KVX_INSN_ ## __mnemo ## _OPCODE_0) + +#define INSN_SIZE(__insn) \ + (KVX_INSN_ ## __insn ## _SIZE * KVX_INSN_SYLLABLE_WIDTH) + +/* Values for general registers */ +#define KVX_REG_R0 0 +#define KVX_REG_R1 1 +#define KVX_REG_R2 2 +#define KVX_REG_R3 3 +#define KVX_REG_R4 4 +#define KVX_REG_R5 5 +#define KVX_REG_R6 6 +#define KVX_REG_R7 7 +#define KVX_REG_R8 8 +#define KVX_REG_R9 9 +#define KVX_REG_R10 10 +#define KVX_REG_R11 11 +#define KVX_REG_R12 12 +#define KVX_REG_SP 12 +#define KVX_REG_R13 13 +#define KVX_REG_TP 13 +#define KVX_REG_R14 14 +#define KVX_REG_FP 14 +#define KVX_REG_R15 15 +#define KVX_REG_R16 16 +#define KVX_REG_R17 17 +#define KVX_REG_R18 18 +#define KVX_REG_R19 19 +#define KVX_REG_R20 20 +#define KVX_REG_R21 21 +#define KVX_REG_R22 22 +#define KVX_REG_R23 23 +#define KVX_REG_R24 24 +#define KVX_REG_R25 25 +#define KVX_REG_R26 26 +#define KVX_REG_R27 27 +#define KVX_REG_R28 28 +#define KVX_REG_R29 29 +#define KVX_REG_R30 30 +#define KVX_REG_R31 31 +#define KVX_REG_R32 32 +#define KVX_REG_R33 33 +#define KVX_REG_R34 34 +#define KVX_REG_R35 35 +#define KVX_REG_R36 36 +#define KVX_REG_R37 37 +#define KVX_REG_R38 38 +#define KVX_REG_R39 39 +#define KVX_REG_R40 40 +#define KVX_REG_R41 41 +#define KVX_REG_R42 42 +#define KVX_REG_R43 43 +#define KVX_REG_R44 44 +#define KVX_REG_R45 45 +#define KVX_REG_R46 46 +#define KVX_REG_R47 47 +#define KVX_REG_R48 48 +#define KVX_REG_R49 49 +#define KVX_REG_R50 50 +#define KVX_REG_R51 51 +#define KVX_REG_R52 52 +#define KVX_REG_R53 53 +#define KVX_REG_R54 54 +#define KVX_REG_R55 55 +#define KVX_REG_R56 56 +#define KVX_REG_R57 57 +#define KVX_REG_R58 58 +#define KVX_REG_R59 59 +#define KVX_REG_R60 60 +#define KVX_REG_R61 61 +#define KVX_REG_R62 62 +#define KVX_REG_R63 63 + +/* Value for bitfield parallel */ +#define KVX_INSN_PARALLEL_EOB 0x0 +#define KVX_INSN_PARALLEL_NONE 0x1 + +#define KVX_INSN_PARALLEL(__insn) (((__insn) >> 31) & 0x1) + +#define KVX_INSN_MAKE_IMM64_SIZE 3 +#define KVX_INSN_MAKE_IMM64_W64_CHECK(__val) \ + (check_signed_imm(__val, 64)) +#define KVX_INSN_MAKE_IMM64_MASK_0 0xff030000 +#define KVX_INSN_MAKE_IMM64_OPCODE_0 0xe0000000 +#define KVX_INSN_MAKE_IMM64_SYLLABLE_0(__rw, __w64) \ + (KVX_INSN_MAKE_IMM64_OPCODE_0 | (((__rw) & 0x3f) << 18) | (((__w64) & 0x3ff) << 6)) +#define KVX_INSN_MAKE_IMM64_MASK_1 0xe0000000 +#define KVX_INSN_MAKE_IMM64_OPCODE_1 0x80000000 +#define KVX_INSN_MAKE_IMM64_SYLLABLE_1(__w64) \ + (KVX_INSN_MAKE_IMM64_OPCODE_1 | (((__w64) >> 10) & 0x7ffffff)) +#define KVX_INSN_MAKE_IMM64_SYLLABLE_2(__p, __w64) \ + (((__p) << 31) | (((__w64) >> 37) & 0x7ffffff)) +#define KVX_INSN_MAKE_IMM64(__buf, __p, __rw, __w64) \ +do { \ + (__buf)[0] = KVX_INSN_MAKE_IMM64_SYLLABLE_0(__rw, __w64); \ + (__buf)[1] = KVX_INSN_MAKE_IMM64_SYLLABLE_1(__w64); \ + (__buf)[2] = KVX_INSN_MAKE_IMM64_SYLLABLE_2(__p, __w64); \ +} while (0) + +#define KVX_INSN_ICALL_SIZE 1 +#define KVX_INSN_ICALL_MASK_0 0x7ffc0000 +#define KVX_INSN_ICALL_OPCODE_0 0xfdc0000 +#define KVX_INSN_ICALL_SYLLABLE_0(__p, __rz) \ + (KVX_INSN_ICALL_OPCODE_0 | ((__p) << 31) | ((__rz) & 0x3f)) +#define KVX_INSN_ICALL(__buf, __p, __rz) \ +do { \ + (__buf)[0] = KVX_INSN_ICALL_SYLLABLE_0(__p, __rz); \ +} while (0) + +#define KVX_INSN_IGOTO_SIZE 1 +#define KVX_INSN_IGOTO_MASK_0 0x7ffc0000 +#define KVX_INSN_IGOTO_OPCODE_0 0xfd80000 +#define KVX_INSN_IGOTO_SYLLABLE_0(__p, __rz) \ + (KVX_INSN_IGOTO_OPCODE_0 | ((__p) << 31) | ((__rz) & 0x3f)) +#define KVX_INSN_IGOTO(__buf, __p, __rz) \ +do { \ + (__buf)[0] = KVX_INSN_IGOTO_SYLLABLE_0(__p, __rz); \ +} while (0) + +#define KVX_INSN_CALL_SIZE 1 +#define KVX_INSN_CALL_PCREL27_CHECK(__val) \ + (((__val) & BITMASK(2)) || check_signed_imm((__val) >> 2, 27)) +#define KVX_INSN_CALL_MASK_0 0x78000000 +#define KVX_INSN_CALL_OPCODE_0 0x18000000 +#define KVX_INSN_CALL_SYLLABLE_0(__p, __pcrel27) \ + (KVX_INSN_CALL_OPCODE_0 | ((__p) << 31) | (((__pcrel27) >> 2) & 0x7ffffff)) +#define KVX_INSN_CALL(__buf, __p, __pcrel27) \ +do { \ + (__buf)[0] = KVX_INSN_CALL_SYLLABLE_0(__p, __pcrel27); \ +} while (0) + +#define KVX_INSN_GOTO_SIZE 1 +#define KVX_INSN_GOTO_PCREL27_CHECK(__val) \ + (((__val) & BITMASK(2)) || check_signed_imm((__val) >> 2, 27)) +#define KVX_INSN_GOTO_MASK_0 0x78000000 +#define KVX_INSN_GOTO_OPCODE_0 0x10000000 +#define KVX_INSN_GOTO_SYLLABLE_0(__p, __pcrel27) \ + (KVX_INSN_GOTO_OPCODE_0 | ((__p) << 31) | (((__pcrel27) >> 2) & 0x7ffffff)) +#define KVX_INSN_GOTO(__buf, __p, __pcrel27) \ +do { \ + (__buf)[0] = KVX_INSN_GOTO_SYLLABLE_0(__p, __pcrel27); \ +} while (0) + +#define KVX_INSN_NOP_SIZE 1 +#define KVX_INSN_NOP_MASK_0 0x7f03f000 +#define KVX_INSN_NOP_OPCODE_0 0x7f03f000 +#define KVX_INSN_NOP_SYLLABLE_0(__p) \ + (KVX_INSN_NOP_OPCODE_0 | ((__p) << 31)) +#define KVX_INSN_NOP(__buf, __p) \ +do { \ + (__buf)[0] = KVX_INSN_NOP_SYLLABLE_0(__p); \ +} while (0) + +#define KVX_INSN_SET_SIZE 1 +#define KVX_INSN_SET_MASK_0 0x7ffc0000 +#define KVX_INSN_SET_OPCODE_0 0xfc00000 +#define KVX_INSN_SET_SYLLABLE_0(__p, __systemT3, __rz) \ + (KVX_INSN_SET_OPCODE_0 | ((__p) << 31) | (((__systemT3) & 0x1ff) << 6) | ((__rz) & 0x3f)) +#define KVX_INSN_SET(__buf, __p, __systemT3, __rz) \ +do { \ + (__buf)[0] = KVX_INSN_SET_SYLLABLE_0(__p, __systemT3, __rz); \ +} while (0) + +#endif /* __ASM_KVX_INSNS_DEFS_H_ */ diff --git a/arch/kvx/kernel/break_hook.c b/arch/kvx/kernel/break_hook.c new file mode 100644 index 000000000000..da38910dab04 --- /dev/null +++ b/arch/kvx/kernel/break_hook.c @@ -0,0 +1,76 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include +#include +#include +#include + +#include +#include +#include + +static DEFINE_SPINLOCK(debug_hook_lock); +static LIST_HEAD(user_break_hook); +static LIST_HEAD(kernel_break_hook); + +void kvx_skip_break_insn(struct pt_regs *regs) +{ + regs->spc += KVX_BREAK_INSN_SIZE; +} + +int break_hook_handler(uint64_t es, struct pt_regs *regs) +{ + int (*fn)(struct break_hook *brk_hook, struct pt_regs *regs) = NULL; + struct break_hook *tmp_hook, *hook = NULL; + struct list_head *list; + unsigned long flags; + u32 idx; + + if (trap_sfri(es) != KVX_TRAP_SFRI_SET || + trap_sfrp(es) != KVX_SFR_VSFR0) + return BREAK_HOOK_ERROR; + + idx = trap_gprp(es); + list = user_mode(regs) ? &user_break_hook : &kernel_break_hook; + + local_irq_save(flags); + list_for_each_entry_rcu(tmp_hook, list, node) { + if (idx == tmp_hook->id) { + hook = tmp_hook; + break; + } + } + local_irq_restore(flags); + + if (!hook) + return BREAK_HOOK_ERROR; + + fn = hook->handler; + return fn(hook, regs); +} + +void break_hook_register(struct break_hook *brk_hook) +{ + struct list_head *list; + + if (brk_hook->mode == MODE_USER) + list = &user_break_hook; + else + list = &kernel_break_hook; + + spin_lock(&debug_hook_lock); + list_add_rcu(&brk_hook->node, list); + spin_unlock(&debug_hook_lock); +} + +void break_hook_unregister(struct break_hook *brk_hook) +{ + spin_lock(&debug_hook_lock); + list_del_rcu(&brk_hook->node); + spin_unlock(&debug_hook_lock); + synchronize_rcu(); +} diff --git a/arch/kvx/kernel/debug.c b/arch/kvx/kernel/debug.c new file mode 100644 index 000000000000..d4cde403bca9 --- /dev/null +++ b/arch/kvx/kernel/debug.c @@ -0,0 +1,64 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ +#include +#include +#include +#include + +#include +#include + +static DEFINE_SPINLOCK(debug_hook_lock); +static LIST_HEAD(user_debug_hook); +static LIST_HEAD(kernel_debug_hook); + +static struct list_head *debug_hook_list(bool user_mode) +{ + return user_mode ? &user_debug_hook : &kernel_debug_hook; +} + +static void call_debug_hook(u64 ea, struct pt_regs *regs) +{ + int ret; + struct debug_hook *hook; + struct list_head *list = debug_hook_list(user_mode(regs)); + + list_for_each_entry_rcu(hook, list, node) { + ret = hook->handler(ea, regs); + if (ret == DEBUG_HOOK_HANDLED) + return; + } + + panic("Entered debug but no requester !"); +} + +void debug_hook_register(struct debug_hook *dbg_hook) +{ + struct list_head *list = debug_hook_list(dbg_hook->mode == MODE_USER); + + spin_lock(&debug_hook_lock); + list_add_rcu(&dbg_hook->node, list); + spin_unlock(&debug_hook_lock); +} + +void debug_hook_unregister(struct debug_hook *dbg_hook) +{ + spin_lock(&debug_hook_lock); + list_del_rcu(&dbg_hook->node); + spin_unlock(&debug_hook_lock); + synchronize_rcu(); +} + +/** + * Main debug handler called by the _debug_handler routine in entry.S + * This handler will perform the required action + */ +void debug_handler(u64 ea, struct pt_regs *regs) +{ + trace_hardirqs_off(); + call_debug_hook(ea, regs); + dame_irq_check(regs); +} diff --git a/arch/kvx/kernel/insns.c b/arch/kvx/kernel/insns.c new file mode 100644 index 000000000000..361621552cd5 --- /dev/null +++ b/arch/kvx/kernel/insns.c @@ -0,0 +1,144 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Yann Sionneau + * Marius Gligor + * Guillaume Thouvenin + */ + +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +struct insns_patch { + atomic_t cpu_count; + u32 *addr; + u32 *insns; + unsigned long insns_len; +}; + +static void *insn_patch_map(void *addr) +{ + unsigned long uintaddr = (uintptr_t) addr; + bool module = !core_kernel_text(uintaddr); + struct page *page; + + if (module && IS_ENABLED(CONFIG_STRICT_MODULE_RWX)) + page = vmalloc_to_page(addr); + else + return addr; + + BUG_ON(!page); + return (void *)set_fixmap_offset(FIX_TEXT_PATCH, page_to_phys(page) + + (uintaddr & ~PAGE_MASK)); +} + +static void insn_patch_unmap(void) +{ + clear_fixmap(FIX_TEXT_PATCH); +} + +int kvx_insns_write_nostop(u32 *insns, u8 insns_len, u32 *insn_addr) +{ + unsigned long current_insn_addr = (unsigned long) insn_addr; + unsigned long len_remain = insns_len; + unsigned long next_insn_page, patch_len; + void *map_patch_addr; + int ret = 0; + + do { + /* Compute next upper page boundary */ + next_insn_page = (current_insn_addr + PAGE_SIZE) & PAGE_MASK; + + patch_len = min(next_insn_page - current_insn_addr, len_remain); + len_remain -= patch_len; + + /* Map & patch insns */ + map_patch_addr = insn_patch_map((void *) current_insn_addr); + ret = copy_to_kernel_nofault(map_patch_addr, insns, patch_len); + if (ret) + break; + + insns = (void *) insns + patch_len; + current_insn_addr = next_insn_page; + + } while (len_remain); + + insn_patch_unmap(); + + /* + * Flush & invalidate L2 + L1 icache to reload instructions from memory + * L2 wbinval is necessary because we write through DEVICE cache policy + * mapping which is uncached therefore L2 is bypassed + */ + wbinval_icache_range(virt_to_phys(insn_addr), insns_len); + + return ret; +} + +static int patch_insns_percpu(void *data) +{ + struct insns_patch *ip = data; + unsigned long insn_addr = (unsigned long) ip->addr; + int ret; + + if (atomic_inc_return(&ip->cpu_count) == 1) { + ret = kvx_insns_write_nostop(ip->insns, ip->insns_len, + ip->addr); + /* Additionnal up to release other processors */ + atomic_inc(&ip->cpu_count); + + return ret; + } + + /* Wait for first processor to update instructions */ + while (atomic_read(&ip->cpu_count) <= num_online_cpus()) + cpu_relax(); + + /* Simply invalidate L1 I-cache to reload from L2 or memory */ + l1_inval_icache_range(insn_addr, insn_addr + ip->insns_len); + return 0; +} + +/** + * kvx_insns_write() Patch instructions at a specified address + * @insns: Instructions to be written at @addr + * @insns_len: Size of instructions to patch + * @addr: Address of the first instruction to patch + */ +int kvx_insns_write(u32 *insns, unsigned long insns_len, u32 *addr) +{ + struct insns_patch ip = { + .cpu_count = ATOMIC_INIT(0), + .addr = addr, + .insns = insns, + .insns_len = insns_len + }; + + if (!insns_len) + return -EINVAL; + + if (!IS_ALIGNED((unsigned long) addr, KVX_INSN_SYLLABLE_WIDTH)) + return -EINVAL; + + /* + * Function name is a "bit" misleading. while being named + * stop_machine, this function does not stop the machine per se + * but execute the provided function on all CPU in a safe state. + */ + return stop_machine(patch_insns_percpu, &ip, cpu_online_mask); +} + +int kvx_insns_read(u32 *insns, unsigned long insns_len, u32 *addr) +{ + l1_inval_dcache_range((unsigned long)addr, insns_len); + return copy_from_kernel_nofault(insns, addr, insns_len); +} From patchwork Fri Jan 20 14:10:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13110017 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63CC5C27C7C for ; Fri, 20 Jan 2023 14:35:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230476AbjATOfR (ORCPT ); Fri, 20 Jan 2023 09:35:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37930 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229770AbjATOfO (ORCPT ); Fri, 20 Jan 2023 09:35:14 -0500 Received: from fx601.security-mail.net (smtpout140.security-mail.net [85.31.212.146]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ABB3373EEB for ; Fri, 20 Jan 2023 06:35:03 -0800 (PST) Received: from localhost (fx601.security-mail.net [127.0.0.1]) by fx601.security-mail.net (Postfix) with ESMTP id 96B0E349826 for ; Fri, 20 Jan 2023 15:20:45 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674224445; bh=ZFm4jn2qVc3XxQ8bbXqM98MO1bH7PPorSUVJmGe9zIQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=NRRvyDRZ5VwOwVQfZDqU+46NQRlyLDL0W3bsNHuZTtmWfvdYPbnL8A8JWp8oPd0br xkajvWR64O8gXlZ1cyBHGFf5LFUuYo1EbpRau8jt+s5Y6qvQmdt2CzKHwmAwdyUODQ QR27+dNxyo6iaIgN6wpI2ZaqsOcbK47b8FxkLvYg= Received: from fx601 (fx601.security-mail.net [127.0.0.1]) by fx601.security-mail.net (Postfix) with ESMTP id 2A6A73493C7; Fri, 20 Jan 2023 15:20:45 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx601.security-mail.net (Postfix) with ESMTPS id 33F923496B9; Fri, 20 Jan 2023 15:20:44 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 74EE327E045E; Fri, 20 Jan 2023 15:10:38 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 5882427E0463; Fri, 20 Jan 2023 15:10:38 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id D11lu3OG-umz; Fri, 20 Jan 2023 15:10:38 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id DE5A427E0458; Fri, 20 Jan 2023 15:10:37 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <62f4.63caa33c.3169e.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 5882427E0463 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223838; bh=r0jAYvZMgDHNmtYPIBemAGb6cgybEgHcmW3NkzN/0Bc=; h=From:To:Date:Message-Id:MIME-Version; b=JKNRyzm7+8sxXJUhLzONaUXzidzmY7QHybfABV02MMeEolEK1l0P/zyCPivr0qi3X cMk6eiWHc8wcM/3mwaZc92PFr6HmcHLhtp6SGUL9n/464kv1zJak07VGMurb1BFNSB JEfwZjhbgtlfk55C9aauCac0b4/oyWiVvpTX7xMY= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 29/31] kvx: Add support for cpuinfo Date: Fri, 20 Jan 2023 15:10:00 +0100 Message-ID: <20230120141002.2442-30-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC Add support for cpuinfo on kvx arch. Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Guillaume Thouvenin Signed-off-by: Guillaume Thouvenin Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Signed-off-by: Jules Maselbas Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: no changes arch/kvx/kernel/cpuinfo.c | 96 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 96 insertions(+) create mode 100644 arch/kvx/kernel/cpuinfo.c diff --git a/arch/kvx/kernel/cpuinfo.c b/arch/kvx/kernel/cpuinfo.c new file mode 100644 index 000000000000..f44c46c1e4ba --- /dev/null +++ b/arch/kvx/kernel/cpuinfo.c @@ -0,0 +1,96 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + */ + +#include +#include +#include +#include +#include + +unsigned long elf_hwcap __read_mostly; + +static int show_cpuinfo(struct seq_file *m, void *v) +{ + int cpu_num = *(unsigned int *)v; + struct cpuinfo_kvx *n = per_cpu_ptr(&cpu_info, cpu_num); + + seq_printf(m, "processor\t: %d\nvendor_id\t: Kalray\n", cpu_num); + + seq_printf(m, + "copro enabled\t: %s\n" + "arch revision\t: %d\n" + "uarch revision\t: %d\n", + n->copro_enable ? "yes" : "no", + n->arch_rev, + n->uarch_rev); + + seq_printf(m, + "bogomips\t: %lu.%02lu\n" + "cpu MHz\t\t: %llu.%03llu\n\n", + (loops_per_jiffy * HZ) / 500000, + ((loops_per_jiffy * HZ) / 5000) % 100, + n->freq / 1000000, (n->freq / 10000) % 100); + + return 0; +} + +static void *c_start(struct seq_file *m, loff_t *pos) +{ + if (*pos == 0) + *pos = cpumask_first(cpu_online_mask); + if (*pos >= num_online_cpus()) + return NULL; + + return pos; +} + +static void *c_next(struct seq_file *m, void *v, loff_t *pos) +{ + *pos = cpumask_next(*pos, cpu_online_mask); + + return c_start(m, pos); +} + +static void c_stop(struct seq_file *m, void *v) +{ +} + +const struct seq_operations cpuinfo_op = { + .start = c_start, + .next = c_next, + .stop = c_stop, + .show = show_cpuinfo, +}; + +static int __init setup_cpuinfo(void) +{ + int cpu; + struct clk *clk; + unsigned long cpu_freq = 1000000000; + struct device_node *node = of_get_cpu_node(0, NULL); + + clk = of_clk_get(node, 0); + if (IS_ERR(clk)) { + printk(KERN_WARNING + "Device tree missing CPU 'clock' parameter. Assuming frequency is 1GHZ"); + goto setup_cpu_freq; + } + + cpu_freq = clk_get_rate(clk); + + clk_put(clk); + +setup_cpu_freq: + of_node_put(node); + + for_each_possible_cpu(cpu) + per_cpu_ptr(&cpu_info, cpu)->freq = cpu_freq; + + return 0; +} + +late_initcall(setup_cpuinfo); From patchwork Fri Jan 20 14:10:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13110024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 266A2C27C7C for ; Fri, 20 Jan 2023 14:36:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231266AbjATOgX (ORCPT ); Fri, 20 Jan 2023 09:36:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39174 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231232AbjATOgD (ORCPT ); Fri, 20 Jan 2023 09:36:03 -0500 Received: from fx306.security-mail.net (smtpout30.security-mail.net [85.31.212.36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7E934D05DA for ; Fri, 20 Jan 2023 06:35:21 -0800 (PST) Received: from localhost (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id 4B8BD35CEF6 for ; Fri, 20 Jan 2023 15:20:46 +0100 (CET) Received: from fx306 (fx306.security-mail.net [127.0.0.1]) by fx306.security-mail.net (Postfix) with ESMTP id 9AA5535CD7A; Fri, 20 Jan 2023 15:20:45 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx306.security-mail.net (Postfix) with ESMTPS id 1B27135CCDC; Fri, 20 Jan 2023 15:20:45 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id EF33327E047D; Fri, 20 Jan 2023 15:10:38 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id D247E27E047C; Fri, 20 Jan 2023 15:10:38 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id N5fxZltrUQUh; Fri, 20 Jan 2023 15:10:38 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 5595327E0470; Fri, 20 Jan 2023 15:10:38 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <2818.63caa33d.1a14f.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu D247E27E047C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223838; bh=Ulssifik/xqyxmseDcDCHWF7GqQfVZORENj6s4m7VYQ=; h=From:To:Date:Message-Id:MIME-Version; b=c8htp1BmJN/ZqK2lC0okl9ZdmRBJrjLE0Xevwykd2j/+m7QLb63Zikx4OnmEIuo/W Di1UTV3iLdZyWQAIqMw8TerInA0DmSfQOa9dBTr2kbyPxrpdc6nurhUYWTcgOQZdlP MXGfL9njI9tpJ5KB86FGEG7jmsXmQo3UTpfRPdc0= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 30/31] kvx: Add power controller driver Date: Fri, 20 Jan 2023 15:10:01 +0100 Message-ID: <20230120141002.2442-31-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC From: Jules Maselbas The Power Controller (pwr-ctrl) control cores reset and wake-up procedure. Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Louis Morhet Signed-off-by: Louis Morhet Co-developed-by: Marius Gligor Signed-off-by: Marius Gligor Signed-off-by: Jules Maselbas Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: new patch arch/kvx/include/asm/pwr_ctrl.h | 45 ++++++++++++++++ arch/kvx/platform/Makefile | 6 +++ arch/kvx/platform/pwr_ctrl.c | 91 +++++++++++++++++++++++++++++++++ 3 files changed, 142 insertions(+) create mode 100644 arch/kvx/include/asm/pwr_ctrl.h create mode 100644 arch/kvx/platform/Makefile create mode 100644 arch/kvx/platform/pwr_ctrl.c diff --git a/arch/kvx/include/asm/pwr_ctrl.h b/arch/kvx/include/asm/pwr_ctrl.h new file mode 100644 index 000000000000..25f403ba935a --- /dev/null +++ b/arch/kvx/include/asm/pwr_ctrl.h @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Marius Gligor + */ + +#ifndef _ASM_KVX_PWR_CTRL_H +#define _ASM_KVX_PWR_CTRL_H + +#ifndef __ASSEMBLY__ + +int kvx_pwr_ctrl_probe(void); + +void kvx_pwr_ctrl_cpu_poweron(unsigned int cpu); + +#endif + +/* Power controller vector register definitions */ +#define KVX_PWR_CTRL_VEC_OFFSET 0x1000 +#define KVX_PWR_CTRL_VEC_WUP_SET_OFFSET 0x10 +#define KVX_PWR_CTRL_VEC_WUP_CLEAR_OFFSET 0x20 + +/* Power controller PE reset PC register definitions */ +#define KVX_PWR_CTRL_RESET_PC_OFFSET 0x2000 + +/* Power controller global register definitions */ +#define KVX_PWR_CTRL_GLOBAL_OFFSET 0x4040 + +#define KVX_PWR_CTRL_GLOBAL_SET_OFFSET 0x10 +#define KVX_PWR_CTRL_GLOBAL_SET_PE_EN_SHIFT 0x1 + +#define PWR_CTRL_WUP_SET_OFFSET \ + (KVX_PWR_CTRL_VEC_OFFSET + \ + KVX_PWR_CTRL_VEC_WUP_SET_OFFSET) + +#define PWR_CTRL_WUP_CLEAR_OFFSET \ + (KVX_PWR_CTRL_VEC_OFFSET + \ + KVX_PWR_CTRL_VEC_WUP_CLEAR_OFFSET) + +#define PWR_CTRL_GLOBAL_CONFIG_OFFSET \ + (KVX_PWR_CTRL_GLOBAL_OFFSET + \ + KVX_PWR_CTRL_GLOBAL_SET_OFFSET) + +#endif /* _ASM_KVX_PWR_CTRL_H */ diff --git a/arch/kvx/platform/Makefile b/arch/kvx/platform/Makefile new file mode 100644 index 000000000000..c7d0abb15c27 --- /dev/null +++ b/arch/kvx/platform/Makefile @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0-only +# +# Copyright (C) 2018-2023 Kalray Inc. +# + +obj-$(CONFIG_SMP) += pwr_ctrl.o diff --git a/arch/kvx/platform/pwr_ctrl.c b/arch/kvx/platform/pwr_ctrl.c new file mode 100644 index 000000000000..ee35d04845ae --- /dev/null +++ b/arch/kvx/platform/pwr_ctrl.c @@ -0,0 +1,91 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include +#include +#include +#include + +#include +#include + +struct kvx_pwr_ctrl { + void __iomem *regs; +}; + +static struct kvx_pwr_ctrl kvx_pwr_controller; + +/** + * kvx_pwr_ctrl_cpu_poweron() - Wakeup a cpu + * @cpu: cpu to wakeup + */ +void kvx_pwr_ctrl_cpu_poweron(unsigned int cpu) +{ + /* Set PE boot address */ + writeq((unsigned long long)kvx_start, + kvx_pwr_controller.regs + KVX_PWR_CTRL_RESET_PC_OFFSET); + /* Wake up processor ! */ + writeq(1ULL << cpu, + kvx_pwr_controller.regs + PWR_CTRL_WUP_SET_OFFSET); + /* Then clear wakeup to allow processor to sleep */ + writeq(1ULL << cpu, + kvx_pwr_controller.regs + PWR_CTRL_WUP_CLEAR_OFFSET); +} + +static struct device_node * __init get_pwr_ctrl_node(void) +{ + const phandle *ph; + struct device_node *cpu; + struct device_node *node; + + cpu = of_get_cpu_node(raw_smp_processor_id(), NULL); + if (!cpu) { + pr_err("Failed to get CPU node\n"); + return NULL; + } + + ph = of_get_property(cpu, "power-controller", NULL); + if (!ph) { + pr_err("Failed to get power-controller phandle\n"); + return NULL; + } + + node = of_find_node_by_phandle(be32_to_cpup(ph)); + if (!node) { + pr_err("Failed to get power-controller node\n"); + return NULL; + } + + return node; +} + +int __init kvx_pwr_ctrl_probe(void) +{ + struct device_node *ctrl; + + ctrl = get_pwr_ctrl_node(); + if (!ctrl) { + pr_err("Failed to get power controller node\n"); + return -EINVAL; + } + + if (!of_device_is_compatible(ctrl, "kalray,kvx-pwr-ctrl")) { + pr_err("Failed to get power controller node\n"); + return -EINVAL; + } + + kvx_pwr_controller.regs = of_iomap(ctrl, 0); + if (!kvx_pwr_controller.regs) { + pr_err("Failed ioremap\n"); + return -EINVAL; + } + + return 0; +} From patchwork Fri Jan 20 14:10:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yann Sionneau X-Patchwork-Id: 13110016 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6F44C05027 for ; Fri, 20 Jan 2023 14:30:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229640AbjATOaq (ORCPT ); Fri, 20 Jan 2023 09:30:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33726 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229831AbjATOap (ORCPT ); Fri, 20 Jan 2023 09:30:45 -0500 Received: from fx601.security-mail.net (smtpout140.security-mail.net [85.31.212.146]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A0EF72C36 for ; Fri, 20 Jan 2023 06:30:19 -0800 (PST) Received: from localhost (fx601.security-mail.net [127.0.0.1]) by fx601.security-mail.net (Postfix) with ESMTP id 5C1593498D3 for ; Fri, 20 Jan 2023 15:20:46 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1674224446; bh=EHbGeTQPzFxRydvAjCAKTJptA7pNhkmTqYf6aPqpaxA=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=wJmnhHllX2nCsMbym/2Ldf4QDA0SfvPV+yw6NU+qeu7wmFEDK4K38YGz89n3hxZyQ vdINqSTmq8IOWezZXRFzPZZgVT9jwL17EfTaVkqKHdBXhlwkvlET3a2DwJUVk7/A+w OtQLLUHnZlUtPcSq29A63CTEd/6jPjvkHF3rZ7vw= Received: from fx601 (fx601.security-mail.net [127.0.0.1]) by fx601.security-mail.net (Postfix) with ESMTP id BEEDF349869; Fri, 20 Jan 2023 15:20:45 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx601.security-mail.net (Postfix) with ESMTPS id 1F87634967F; Fri, 20 Jan 2023 15:20:44 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id 7BC3F27E0481; Fri, 20 Jan 2023 15:10:39 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 62F6427E047C; Fri, 20 Jan 2023 15:10:39 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id efK8YWmnYQbb; Fri, 20 Jan 2023 15:10:39 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id C9CA227E0474; Fri, 20 Jan 2023 15:10:38 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: <5b1f.63caa33c.1c743.0> DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 62F6427E047C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1674223839; bh=5C/8QjxVUf/DelpGCfD/Pp9BdUc2vZv56uJsfxSlXPA=; h=From:To:Date:Message-Id:MIME-Version; b=P6BtAyPM/IrloLG30cdWTPBhk6bnzZeuu0JK6hHT1AvSOjTEjV23l2HTVSkWp3YOM JxhQxnIr6MpJR9Ve5OwsqaqYSpuYPPXnuoM+q+2xD3d01z+Z0ef6cKxQqGtJg6OcUa Z5LRxJhk4HSsThMlK536n8tiuxt0/9UzyH8L0fvI= From: Yann Sionneau To: Arnd Bergmann , Jonathan Corbet , Thomas Gleixner , Marc Zyngier , Rob Herring , Krzysztof Kozlowski , Will Deacon , Peter Zijlstra , Boqun Feng , Mark Rutland , Eric Biederman , Kees Cook , Oleg Nesterov , Ingo Molnar , Waiman Long , "Aneesh Kumar K.V" , Andrew Morton , Nick Piggin , Paul Moore , Eric Paris , Christian Brauner , Paul Walmsley , Palmer Dabbelt , Albert Ou , Jules Maselbas , Yann Sionneau , Guillaume Thouvenin , Clement Leger , Vincent Chardon , Marc =?utf-8?b?UG91bGhp?= =?utf-8?b?w6hz?= , Julian Vetter , Samuel Jones , Ashley Lesdalons , Thomas Costis , Marius Gligor , Jonathan Borne , Julien Villette , Luc Michel , Louis Morhet , Julien Hascoet , Jean-Christophe Pince , Guillaume Missonnier , Alex Michon , Huacai Chen , WANG Xuerui , Shaokun Zhang , John Garry , Guangbin Huang , Bharat Bhushan , Bibo Mao , Atish Patra , "Jason A. Donenfeld" , Qi Liu , Jiaxun Yang , Catalin Marinas , Mark Brown , Janosch Frank , Alexey Dobriyan Cc: Benjamin Mugnier , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-audit@redhat.com, linux-riscv@lists.infradead.org, bpf@vger.kernel.org Subject: [RFC PATCH v2 31/31] kvx: Add IPI driver Date: Fri, 20 Jan 2023 15:10:02 +0100 Message-ID: <20230120141002.2442-32-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230120141002.2442-1-ysionneau@kalray.eu> References: <20230120141002.2442-1-ysionneau@kalray.eu> MIME-Version: 1.0 X-ALTERMIMEV2_out: done Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org X-Patchwork-State: RFC From: Jules Maselbas The Inter-Processor Interrupt Controller (IPI) provides a fast synchronization mechanism to the software. It exposes eight independent set of registers that can be use to notify each processor in the cluster. test Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Guillaume Thouvenin Signed-off-by: Guillaume Thouvenin Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Luc Michel Signed-off-by: Luc Michel Signed-off-by: Jules Maselbas Signed-off-by: Yann Sionneau --- Notes: V1 -> V2: new patch arch/kvx/include/asm/ipi.h | 16 ++++++ arch/kvx/platform/Makefile | 1 + arch/kvx/platform/ipi.c | 108 +++++++++++++++++++++++++++++++++++++ 3 files changed, 125 insertions(+) create mode 100644 arch/kvx/include/asm/ipi.h create mode 100644 arch/kvx/platform/ipi.c diff --git a/arch/kvx/include/asm/ipi.h b/arch/kvx/include/asm/ipi.h new file mode 100644 index 000000000000..137407a075e6 --- /dev/null +++ b/arch/kvx/include/asm/ipi.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_IPI_H +#define _ASM_KVX_IPI_H + +#include + +int kvx_ipi_ctrl_probe(irqreturn_t (*ipi_irq_handler)(int, void *)); + +void kvx_ipi_send(const struct cpumask *mask); + +#endif /* _ASM_KVX_IPI_H */ diff --git a/arch/kvx/platform/Makefile b/arch/kvx/platform/Makefile index c7d0abb15c27..27f0914e0de5 100644 --- a/arch/kvx/platform/Makefile +++ b/arch/kvx/platform/Makefile @@ -4,3 +4,4 @@ # obj-$(CONFIG_SMP) += pwr_ctrl.o +obj-$(CONFIG_SMP) += ipi.o diff --git a/arch/kvx/platform/ipi.c b/arch/kvx/platform/ipi.c new file mode 100644 index 000000000000..a471039b1643 --- /dev/null +++ b/arch/kvx/platform/ipi.c @@ -0,0 +1,108 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Luc Michel + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define IPI_INTERRUPT_OFFSET 0x0 +#define IPI_MASK_OFFSET 0x20 + +/* + * IPI controller can signal RM and PE0 -> 15 + * In order to restrict that to the PE, write the corresponding mask + */ +#define KVX_IPI_CPU_MASK (~0xFFFF) + +struct kvx_ipi_ctrl { + void __iomem *regs; + unsigned int ipi_irq; +}; + +static struct kvx_ipi_ctrl kvx_ipi_controller; + +/** + * @kvx_pwr_ctrl_cpu_poweron Wakeup a cpu + * + * cpu: cpu to wakeup + */ +void kvx_ipi_send(const struct cpumask *mask) +{ + const unsigned long *maskb = cpumask_bits(mask); + + WARN_ON(*maskb & KVX_IPI_CPU_MASK); + writel(*maskb, kvx_ipi_controller.regs + IPI_INTERRUPT_OFFSET); +} + +static int kvx_ipi_starting_cpu(unsigned int cpu) +{ + enable_percpu_irq(kvx_ipi_controller.ipi_irq, IRQ_TYPE_NONE); + + return 0; +} + +static int kvx_ipi_dying_cpu(unsigned int cpu) +{ + disable_percpu_irq(kvx_ipi_controller.ipi_irq); + + return 0; +} + +int __init kvx_ipi_ctrl_probe(irqreturn_t (*ipi_irq_handler)(int, void *)) +{ + struct device_node *np; + int ret; + unsigned int ipi_irq; + void __iomem *ipi_base; + + np = of_find_compatible_node(NULL, NULL, "kalray,kvx-ipi-ctrl"); + BUG_ON(!np); + + ipi_base = of_iomap(np, 0); + BUG_ON(!ipi_base); + + kvx_ipi_controller.regs = ipi_base; + + /* Init mask for interrupts to PE0 -> PE15 */ + writel(KVX_IPI_CPU_MASK, kvx_ipi_controller.regs + IPI_MASK_OFFSET); + + ipi_irq = irq_of_parse_and_map(np, 0); + of_node_put(np); + if (!ipi_irq) { + pr_err("Failed to parse irq: %d\n", ipi_irq); + return -EINVAL; + } + + ret = request_percpu_irq(ipi_irq, ipi_irq_handler, + "kvx_ipi", &kvx_ipi_controller); + if (ret) { + pr_err("can't register interrupt %d (%d)\n", + ipi_irq, ret); + return ret; + } + kvx_ipi_controller.ipi_irq = ipi_irq; + + ret = cpuhp_setup_state(CPUHP_AP_IRQ_KVX_STARTING, + "kvx/ipi:online", + kvx_ipi_starting_cpu, + kvx_ipi_dying_cpu); + if (ret < 0) { + pr_err("Failed to setup hotplug state"); + return ret; + } + + return 0; +}