From patchwork Fri Jun 16 05:59:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Alison Wang X-Patchwork-Id: 13282008 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ABC20EB64D7 for ; Fri, 16 Jun 2023 05:44:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wo6lyfjTR+4CkEEwjUFG1nMwfFHDdHzv7ZkeUjdpsyA=; b=Rmaov8H1i5ZqC8 JP/nBl1qqj6zns6LNjDDqgFUFajPtUS1dKKRXkz4yxyzuCRDtNfiZmxPJlNjD6ss7TNDpERD/ygp6 e9CWcDth6OYA3Qw14nmHKYiscE4Eon0KhMz12ThKQ6AlKOCetKpT9/lFZAHsG2BI+7m2u6/rjza5/ heJfTbymAsAmxQvyP5vGgrMJVNOV7fP/S2LKEz4jjMN2AYtOsrgMcLK4HLHplyEP6nK8hvAqp7SEO Wnf14ofArrj/dh1ZsH4MxHPaHACHxTnVoc/VmUsl47ZOysBueSnEAC/fAs9xH0ZPaTrQXNyuZOETr aRZtjSE8WMyDab/XMM6w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qA2F3-00Guu1-2a; Fri, 16 Jun 2023 05:43:25 +0000 Received: from inva020.nxp.com ([92.121.34.13]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qA2En-00Gulp-0D for linux-arm-kernel@lists.infradead.org; Fri, 16 Jun 2023 05:43:18 +0000 Received: from inva020.nxp.com (localhost [127.0.0.1]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id A0B3A1A06A2; Fri, 16 Jun 2023 07:43:03 +0200 (CEST) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id AC4A61A068F; Fri, 16 Jun 2023 07:43:02 +0200 (CEST) Received: from localhost.localdomain (mega.ap.freescale.net [10.192.208.232]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id 3BFDC180222B; Fri, 16 Jun 2023 13:43:01 +0800 (+08) From: Alison Wang To: gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: leoyang.li@nxp.com, xuelin.shi@nxp.com, xiaofeng.ren@nxp.com, feng.guo@nxp.com, Alison Wang Subject: [PATCH 1/8] ethosu: Add Arm Ethos-U driver Date: Fri, 16 Jun 2023 13:59:06 +0800 Message-Id: <20230616055913.2360-2-alison.wang@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230616055913.2360-1-alison.wang@nxp.com> References: <20230616055913.2360-1-alison.wang@nxp.com> MIME-Version: 1.0 X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230615_224309_656319_0B22467D X-CRM114-Status: GOOD ( 27.40 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Ethos-U Linux driver is to provide an example of how a rich operating system like Linux can dispatch inferences to an Arm Cortex-M subsystem, consisting of an Arm Cortex-M and an Arm Ethos-U NPU. Link: https://git.mlplatform.org/ml/ethos-u/ethos-u-linux-driver-stack.git Tag: 22.02 Signed-off-by: Kristofer Jonsson Signed-off-by: Per Astrand Signed-off-by: Jonny Svärd Signed-off-by: Lior Dekel Signed-off-by: Henrik Hoglind Signed-off-by: Davide Grohmann Signed-off-by: Alison Wang --- drivers/firmware/Kconfig | 1 + drivers/firmware/Makefile | 1 + drivers/firmware/ethosu/Kconfig | 24 + drivers/firmware/ethosu/Makefile | 28 ++ drivers/firmware/ethosu/ethosu_buffer.c | 312 ++++++++++++ drivers/firmware/ethosu/ethosu_buffer.h | 106 ++++ .../firmware/ethosu/ethosu_core_interface.h | 183 +++++++ drivers/firmware/ethosu/ethosu_device.c | 474 ++++++++++++++++++ drivers/firmware/ethosu/ethosu_device.h | 88 ++++ drivers/firmware/ethosu/ethosu_driver.c | 191 +++++++ drivers/firmware/ethosu/ethosu_inference.c | 417 +++++++++++++++ drivers/firmware/ethosu/ethosu_inference.h | 120 +++++ drivers/firmware/ethosu/ethosu_mailbox.c | 377 ++++++++++++++ drivers/firmware/ethosu/ethosu_mailbox.h | 140 ++++++ drivers/firmware/ethosu/ethosu_network.c | 201 ++++++++ drivers/firmware/ethosu/ethosu_network.h | 81 +++ drivers/firmware/ethosu/uapi/ethosu.h | 207 ++++++++ 17 files changed, 2951 insertions(+) create mode 100644 drivers/firmware/ethosu/Kconfig create mode 100644 drivers/firmware/ethosu/Makefile create mode 100644 drivers/firmware/ethosu/ethosu_buffer.c create mode 100644 drivers/firmware/ethosu/ethosu_buffer.h create mode 100644 drivers/firmware/ethosu/ethosu_core_interface.h create mode 100644 drivers/firmware/ethosu/ethosu_device.c create mode 100644 drivers/firmware/ethosu/ethosu_device.h create mode 100644 drivers/firmware/ethosu/ethosu_driver.c create mode 100644 drivers/firmware/ethosu/ethosu_inference.c create mode 100644 drivers/firmware/ethosu/ethosu_inference.h create mode 100644 drivers/firmware/ethosu/ethosu_mailbox.c create mode 100644 drivers/firmware/ethosu/ethosu_mailbox.h create mode 100644 drivers/firmware/ethosu/ethosu_network.c create mode 100644 drivers/firmware/ethosu/ethosu_network.h create mode 100644 drivers/firmware/ethosu/uapi/ethosu.h diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig index b59e3041fd62..bbad6dc81e7b 100644 --- a/drivers/firmware/Kconfig +++ b/drivers/firmware/Kconfig @@ -308,6 +308,7 @@ source "drivers/firmware/broadcom/Kconfig" source "drivers/firmware/cirrus/Kconfig" source "drivers/firmware/google/Kconfig" source "drivers/firmware/efi/Kconfig" +source "drivers/firmware/ethosu/Kconfig" source "drivers/firmware/imx/Kconfig" source "drivers/firmware/meson/Kconfig" source "drivers/firmware/psci/Kconfig" diff --git a/drivers/firmware/Makefile b/drivers/firmware/Makefile index 28fcddcd688f..6c597a78daa7 100644 --- a/drivers/firmware/Makefile +++ b/drivers/firmware/Makefile @@ -33,6 +33,7 @@ obj-y += cirrus/ obj-y += meson/ obj-$(CONFIG_GOOGLE_FIRMWARE) += google/ obj-y += efi/ +obj-$(CONFIG_ETHOSU) += ethosu/ obj-y += imx/ obj-y += psci/ obj-y += smccc/ diff --git a/drivers/firmware/ethosu/Kconfig b/drivers/firmware/ethosu/Kconfig new file mode 100644 index 000000000000..ce837f45f8e5 --- /dev/null +++ b/drivers/firmware/ethosu/Kconfig @@ -0,0 +1,24 @@ +# +# (C) COPYRIGHT 2020 ARM Limited. All rights reserved. +# +# This program is free software and is provided to you under the terms of the +# GNU General Public License version 2 as published by the Free Software +# Foundation, and any use by you of this program is subject to the terms +# of such GNU licence. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, you can access it online at +# http://www.gnu.org/licenses/gpl-2.0.html. +# +# SPDX-License-Identifier: GPL-2.0-only +# + +config ETHOSU + tristate "Arm Ethos-U NPU support" + help + Arm Ethos-U NPU driver. diff --git a/drivers/firmware/ethosu/Makefile b/drivers/firmware/ethosu/Makefile new file mode 100644 index 000000000000..933efee1b22f --- /dev/null +++ b/drivers/firmware/ethosu/Makefile @@ -0,0 +1,28 @@ +# +# (C) COPYRIGHT 2020 ARM Limited. All rights reserved. +# +# This program is free software and is provided to you under the terms of the +# GNU General Public License version 2 as published by the Free Software +# Foundation, and any use by you of this program is subject to the terms +# of such GNU licence. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, you can access it online at +# http://www.gnu.org/licenses/gpl-2.0.html. +# +# SPDX-License-Identifier: GPL-2.0-only +# + +obj-$(CONFIG_ETHOSU) = ethosu.o + +ethosu-objs := ethosu_driver.o \ + ethosu_buffer.o \ + ethosu_device.o \ + ethosu_inference.o \ + ethosu_mailbox.o \ + ethosu_network.o diff --git a/drivers/firmware/ethosu/ethosu_buffer.c b/drivers/firmware/ethosu/ethosu_buffer.c new file mode 100644 index 000000000000..43a433355f30 --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_buffer.c @@ -0,0 +1,312 @@ +/* + * (C) COPYRIGHT 2020-2021 Arm Limited. All rights reserved. + * + * This program is free software and is provided to you under the terms of the + * GNU General Public License version 2 as published by the Free Software + * Foundation, and any use by you of this program is subject to the terms + * of such GNU licence. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, you can access it online at + * http://www.gnu.org/licenses/gpl-2.0.html. + * + * SPDX-License-Identifier: GPL-2.0-only + */ + +/**************************************************************************** + * Includes + ****************************************************************************/ + +#include "ethosu_buffer.h" + +#include "ethosu_device.h" +#include "uapi/ethosu.h" + +#include +#include +#include +#include +#include +#include + +/**************************************************************************** + * Variables + ****************************************************************************/ + +static int ethosu_buffer_release(struct inode *inode, + struct file *file); + +static int ethosu_buffer_mmap(struct file *file, + struct vm_area_struct *vma); + +static long ethosu_buffer_ioctl(struct file *file, + unsigned int cmd, + unsigned long arg); + +static const struct file_operations ethosu_buffer_fops = { + .release = ðosu_buffer_release, + .mmap = ðosu_buffer_mmap, + .unlocked_ioctl = ðosu_buffer_ioctl, +#ifdef CONFIG_COMPAT + .compat_ioctl = ðosu_buffer_ioctl, +#endif +}; + +/**************************************************************************** + * Functions + ****************************************************************************/ + +/* + * The 'dma-ranges' device tree property for shared dma memory does not seem + * to be fully supported for coherent memory. Therefor we apply the DMA range + * offset ourselves. + */ +static dma_addr_t ethosu_buffer_dma_ranges(struct device *dev, + dma_addr_t dma_addr, + size_t dma_buf_size) +{ + struct device_node *node = dev->of_node; + const __be32 *ranges; + int len; + int naddr; + int nsize; + int inc; + int i; + + if (!node) + return dma_addr; + + /* Get the #address-cells and #size-cells properties */ + naddr = of_n_addr_cells(node); + nsize = of_n_size_cells(node); + + /* Read the 'dma-ranges' property */ + ranges = of_get_property(node, "dma-ranges", &len); + if (!ranges || len <= 0) + return dma_addr; + + dev_dbg(dev, "ranges=%p, len=%d, naddr=%d, nsize=%d\n", + ranges, len, naddr, nsize); + + len /= sizeof(*ranges); + inc = naddr + naddr + nsize; + + for (i = 0; (i + inc) <= len; i += inc) { + dma_addr_t daddr; + dma_addr_t paddr; + dma_addr_t size; + + daddr = of_read_number(&ranges[i], naddr); + paddr = of_read_number(&ranges[i + naddr], naddr); + size = of_read_number(&ranges[i + naddr + naddr], nsize); + + dev_dbg(dev, "daddr=0x%llx, paddr=0x%llx, size=0x%llx\n", + daddr, paddr, size); + + if (dma_addr >= paddr && + (dma_addr + dma_buf_size) < (paddr + size)) + return dma_addr + daddr - paddr; + } + + return dma_addr; +} + +static bool ethosu_buffer_verify(struct file *file) +{ + return file->f_op == ðosu_buffer_fops; +} + +static void ethosu_buffer_destroy(struct kref *kref) +{ + struct ethosu_buffer *buf = + container_of(kref, struct ethosu_buffer, kref); + + dev_info(buf->edev->dev, "Buffer destroy. handle=0x%pK\n", buf); + + dma_free_coherent(buf->edev->dev, buf->capacity, buf->cpu_addr, + buf->dma_addr_orig); + devm_kfree(buf->edev->dev, buf); +} + +static int ethosu_buffer_release(struct inode *inode, + struct file *file) +{ + struct ethosu_buffer *buf = file->private_data; + + dev_info(buf->edev->dev, "Buffer release. handle=0x%pK\n", buf); + + ethosu_buffer_put(buf); + + return 0; +} + +static int ethosu_buffer_mmap(struct file *file, + struct vm_area_struct *vma) +{ + struct ethosu_buffer *buf = file->private_data; + int ret; + + dev_info(buf->edev->dev, "Buffer mmap. handle=0x%pK\n", buf); + + ret = dma_mmap_coherent(buf->edev->dev, vma, buf->cpu_addr, + buf->dma_addr_orig, + buf->capacity); + + return ret; +} + +static long ethosu_buffer_ioctl(struct file *file, + unsigned int cmd, + unsigned long arg) +{ + struct ethosu_buffer *buf = file->private_data; + void __user *udata = (void __user *)arg; + int ret = -EINVAL; + + ret = mutex_lock_interruptible(&buf->edev->mutex); + if (ret) + return ret; + + dev_info(buf->edev->dev, "Ioctl. cmd=%u, arg=%lu\n", cmd, arg); + + switch (cmd) { + case ETHOSU_IOCTL_BUFFER_SET: { + struct ethosu_uapi_buffer uapi; + + if (copy_from_user(&uapi, udata, sizeof(uapi))) + break; + + dev_info(buf->edev->dev, + "Ioctl: Buffer set. size=%u, offset=%u\n", + uapi.size, uapi.offset); + + ret = ethosu_buffer_resize(buf, uapi.size, uapi.offset); + break; + } + case ETHOSU_IOCTL_BUFFER_GET: { + struct ethosu_uapi_buffer uapi; + + uapi.size = buf->size; + uapi.offset = buf->offset; + + dev_info(buf->edev->dev, + "Ioctl: Buffer get. size=%u, offset=%u\n", + uapi.size, uapi.offset); + + if (copy_to_user(udata, &uapi, sizeof(uapi))) + break; + + ret = 0; + break; + } + default: { + dev_err(buf->edev->dev, "Invalid ioctl. cmd=%u, arg=%lu", + cmd, arg); + break; + } + } + + mutex_unlock(&buf->edev->mutex); + + return ret; +} + +int ethosu_buffer_create(struct ethosu_device *edev, + size_t capacity) +{ + struct ethosu_buffer *buf; + int ret = -ENOMEM; + + buf = devm_kzalloc(edev->dev, sizeof(*buf), GFP_KERNEL); + if (!buf) + return -ENOMEM; + + buf->edev = edev; + buf->capacity = capacity; + buf->offset = 0; + buf->size = 0; + kref_init(&buf->kref); + + buf->cpu_addr = dma_alloc_coherent(buf->edev->dev, capacity, + &buf->dma_addr_orig, GFP_KERNEL); + if (!buf->cpu_addr) + goto free_buf; + + buf->dma_addr = ethosu_buffer_dma_ranges(buf->edev->dev, + buf->dma_addr_orig, + buf->capacity); + + ret = anon_inode_getfd("ethosu-buffer", ðosu_buffer_fops, buf, + O_RDWR | O_CLOEXEC); + if (ret < 0) + goto free_dma; + + buf->file = fget(ret); + fput(buf->file); + + dev_info(buf->edev->dev, + "Buffer create. handle=0x%pK, capacity=%zu, cpu_addr=0x%pK, dma_addr=0x%llx, dma_addr_orig=0x%llx, phys_addr=0x%llx\n", + buf, capacity, buf->cpu_addr, buf->dma_addr, + buf->dma_addr_orig, virt_to_phys(buf->cpu_addr)); + + return ret; + +free_dma: + dma_free_coherent(buf->edev->dev, buf->capacity, buf->cpu_addr, + buf->dma_addr_orig); + +free_buf: + devm_kfree(buf->edev->dev, buf); + + return ret; +} + +struct ethosu_buffer *ethosu_buffer_get_from_fd(int fd) +{ + struct ethosu_buffer *buf; + struct file *file; + + file = fget(fd); + if (!file) + return ERR_PTR(-EINVAL); + + if (!ethosu_buffer_verify(file)) { + fput(file); + + return ERR_PTR(-EINVAL); + } + + buf = file->private_data; + ethosu_buffer_get(buf); + fput(file); + + return buf; +} + +void ethosu_buffer_get(struct ethosu_buffer *buf) +{ + kref_get(&buf->kref); +} + +void ethosu_buffer_put(struct ethosu_buffer *buf) +{ + kref_put(&buf->kref, ethosu_buffer_destroy); +} + +int ethosu_buffer_resize(struct ethosu_buffer *buf, + size_t size, + size_t offset) +{ + if ((size + offset) > buf->capacity) + return -EINVAL; + + buf->size = size; + buf->offset = offset; + + return 0; +} diff --git a/drivers/firmware/ethosu/ethosu_buffer.h b/drivers/firmware/ethosu/ethosu_buffer.h new file mode 100644 index 000000000000..14f26c2b0a9d --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_buffer.h @@ -0,0 +1,106 @@ +/* + * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * + * This program is free software and is provided to you under the terms of the + * GNU General Public License version 2 as published by the Free Software + * Foundation, and any use by you of this program is subject to the terms + * of such GNU licence. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, you can access it online at + * http://www.gnu.org/licenses/gpl-2.0.html. + * + * SPDX-License-Identifier: GPL-2.0-only + */ + +#ifndef ETHOSU_BUFFER_H +#define ETHOSU_BUFFER_H + +/**************************************************************************** + * Includes + ****************************************************************************/ + +#include +#include + +/**************************************************************************** + * Types + ****************************************************************************/ + +struct ethosu_device; +struct device; + +/** + * struct ethosu_buffer - Buffer + * @dev: Device + * @file: File + * @kref: Reference counting + * @capacity: Maximum capacity of the buffer + * @offset: Offset to first byte of buffer + * @size: Size of the data in the buffer + * @cpu_addr: Kernel mapped address + * @dma_addr: DMA address + * @dma_addr_orig: Original DMA address before range mapping + * + * 'offset + size' must not be larger than 'capacity'. + */ +struct ethosu_buffer { + struct ethosu_device *edev; + struct file *file; + struct kref kref; + size_t capacity; + size_t offset; + size_t size; + void *cpu_addr; + dma_addr_t dma_addr; + dma_addr_t dma_addr_orig; +}; + +/**************************************************************************** + * Functions + ****************************************************************************/ + +/** + * ethosu_buffer_create() - Create buffer + * + * This function must be called in the context of a user space process. + * + * Return: fd on success, else error code. + */ +int ethosu_buffer_create(struct ethosu_device *edev, + size_t capacity); + +/** + * ethosu_buffer_get_from_fd() - Get buffer handle from fd + * + * This function must be called from a user space context. + * + * Return: Pointer on success, else ERR_PTR. + */ +struct ethosu_buffer *ethosu_buffer_get_from_fd(int fd); + +/** + * ethosu_buffer_get() - Put buffer + */ +void ethosu_buffer_get(struct ethosu_buffer *buf); + +/** + * ethosu_buffer_put() - Put buffer + */ +void ethosu_buffer_put(struct ethosu_buffer *buf); + +/** + * ethosu_buffer_resize() - Resize and validate buffer + * + * Return: 0 on success, else error code. + */ +int ethosu_buffer_resize(struct ethosu_buffer *buf, + size_t size, + size_t offset); + +#endif /* ETHOSU_BUFFER_H */ diff --git a/drivers/firmware/ethosu/ethosu_core_interface.h b/drivers/firmware/ethosu/ethosu_core_interface.h new file mode 100644 index 000000000000..ef63c3b55352 --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_core_interface.h @@ -0,0 +1,183 @@ +/* + * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * + * This program is free software and is provided to you under the terms of the + * GNU General Public License version 2 as published by the Free Software + * Foundation, and any use by you of this program is subject to the terms + * of such GNU licence. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, you can access it online at + * http://www.gnu.org/licenses/gpl-2.0.html. + * + * SPDX-License-Identifier: GPL-2.0-only + */ + +#ifndef ETHOSU_CORE_INTERFACE_H +#define ETHOSU_CORE_INTERFACE_H + +#ifdef __KERNEL__ +#include +#else +#include +#endif + +#ifdef __cplusplus +namespace EthosU { +#endif + +/** Maximum number of IFM/OFM buffers per inference */ +#define ETHOSU_CORE_BUFFER_MAX 16 + +/** Maximum number of PMU counters to be returned for inference */ +#define ETHOSU_CORE_PMU_MAX 8 + +#define ETHOSU_CORE_MSG_MAGIC 0x41457631 +#define ETHOSU_CORE_MSG_VERSION_MAJOR 0 +#define ETHOSU_CORE_MSG_VERSION_MINOR 2 +#define ETHOSU_CORE_MSG_VERSION_PATCH 0 + +/** + * enum ethosu_core_msg_type - Message types + * + * Types for the messages sent between the host and the core subsystem. + */ +enum ethosu_core_msg_type { + ETHOSU_CORE_MSG_ERR = 1, + ETHOSU_CORE_MSG_PING, + ETHOSU_CORE_MSG_PONG, + ETHOSU_CORE_MSG_INFERENCE_REQ, + ETHOSU_CORE_MSG_INFERENCE_RSP, + ETHOSU_CORE_MSG_VERSION_REQ, + ETHOSU_CORE_MSG_VERSION_RSP, + ETHOSU_CORE_MSG_CAPABILITIES_REQ, + ETHOSU_CORE_MSG_CAPABILITIES_RSP, + ETHOSU_CORE_MSG_MAX +}; + +/** + * struct ethosu_core_msg - Message header + */ +struct ethosu_core_msg { + uint32_t magic; + uint32_t type; + uint32_t length; +}; + +/** + * struct ethosu_core_queue_header - Message queue header + */ +struct ethosu_core_queue_header { + uint32_t size; + uint32_t read; + uint32_t write; +}; + +/** + * struct ethosu_core_queue - Message queue + * + * Dynamically sized message queue. + */ +struct ethosu_core_queue { + struct ethosu_core_queue_header header; + uint8_t data[]; +}; + +enum ethosu_core_status { + ETHOSU_CORE_STATUS_OK, + ETHOSU_CORE_STATUS_ERROR +}; + +struct ethosu_core_buffer { + uint32_t ptr; + uint32_t size; +}; + +struct ethosu_core_inference_req { + uint64_t user_arg; + uint32_t ifm_count; + struct ethosu_core_buffer ifm[ETHOSU_CORE_BUFFER_MAX]; + uint32_t ofm_count; + struct ethosu_core_buffer ofm[ETHOSU_CORE_BUFFER_MAX]; + struct ethosu_core_buffer network; + uint8_t pmu_event_config[ETHOSU_CORE_PMU_MAX]; + uint32_t pmu_cycle_counter_enable; +}; + +struct ethosu_core_inference_rsp { + uint64_t user_arg; + uint32_t ofm_count; + uint32_t ofm_size[ETHOSU_CORE_BUFFER_MAX]; + uint32_t status; + uint8_t pmu_event_config[ETHOSU_CORE_PMU_MAX]; + uint32_t pmu_event_count[ETHOSU_CORE_PMU_MAX]; + uint32_t pmu_cycle_counter_enable; + uint64_t pmu_cycle_counter_count; +}; + +/** + * struct ethosu_core_msg_verson - Message protocol version + */ +struct ethosu_core_msg_version { + uint8_t major; + uint8_t minor; + uint8_t patch; + uint8_t _reserved; +}; + +/** + * struct ethosu_core_capabilities_req - Message capabilities request + */ +struct ethosu_core_capabilities_req { + uint64_t user_arg; +}; + +/** + * struct ethosu_core_capabilities_rsp - Message capabilities response + */ +struct ethosu_core_msg_capabilities_rsp { + uint64_t user_arg; + uint32_t version_status; + uint32_t version_minor; + uint32_t version_major; + uint32_t product_major; + uint32_t arch_patch_rev; + uint32_t arch_minor_rev; + uint32_t arch_major_rev; + uint32_t driver_patch_rev; + uint32_t driver_minor_rev; + uint32_t driver_major_rev; + uint32_t macs_per_cc; + uint32_t cmd_stream_version; + uint32_t custom_dma; +}; + +/** + * enum ethosu_core_msg_err_type - Error types + */ +enum ethosu_core_msg_err_type { + ETHOSU_CORE_MSG_ERR_GENERIC = 0, + ETHOSU_CORE_MSG_ERR_UNSUPPORTED_TYPE, + ETHOSU_CORE_MSG_ERR_INVALID_PAYLOAD, + ETHOSU_CORE_MSG_ERR_INVALID_SIZE, + ETHOSU_CORE_MSG_ERR_INVALID_MAGIC, + ETHOSU_CORE_MSG_ERR_MAX +}; + +/** + * struct ethosu_core_msg_err - Error message struct + */ +struct ethosu_core_msg_err { + uint32_t type; /* optional use of extra error code */ + char msg[128]; +}; +#ifdef __cplusplus +} /*namespace EthosU */ +#endif + +#endif diff --git a/drivers/firmware/ethosu/ethosu_device.c b/drivers/firmware/ethosu/ethosu_device.c new file mode 100644 index 000000000000..e6f1e8012b06 --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_device.c @@ -0,0 +1,474 @@ +/* + * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * + * This program is free software and is provided to you under the terms of the + * GNU General Public License version 2 as published by the Free Software + * Foundation, and any use by you of this program is subject to the terms + * of such GNU licence. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, you can access it online at + * http://www.gnu.org/licenses/gpl-2.0.html. + * + * SPDX-License-Identifier: GPL-2.0-only + */ + +/**************************************************************************** + * Includes + ****************************************************************************/ + +#include "ethosu_device.h" + +#include "ethosu_buffer.h" +#include "ethosu_core_interface.h" +#include "ethosu_inference.h" +#include "ethosu_network.h" +#include "uapi/ethosu.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/**************************************************************************** + * Defines + ****************************************************************************/ + +#define DMA_ADDR_BITS 32 /* Number of address bits */ + +#define CAPABILITIES_RESP_TIMEOUT_MS 2000 + +/**************************************************************************** + * Types + ****************************************************************************/ + +/**************************************************************************** + * Functions + ****************************************************************************/ + +static void ethosu_capabilities_destroy(struct kref *kref) +{ + struct ethosu_capabilities *cap = + container_of(kref, struct ethosu_capabilities, refcount); + + list_del(&cap->list); + + devm_kfree(cap->edev->dev, cap); +} + +static int ethosu_capabilities_find(struct ethosu_capabilities *cap, + struct list_head *capabilties_list) +{ + struct ethosu_capabilities *cur; + + list_for_each_entry(cur, capabilties_list, list) { + if (cur == cap) + return 0; + } + + return -EINVAL; +} + +static int ethosu_capability_rsp(struct ethosu_device *edev, + struct ethosu_core_msg_capabilities_rsp *msg) +{ + struct ethosu_capabilities *cap; + struct ethosu_uapi_device_capabilities *capabilities; + int ret; + + cap = (struct ethosu_capabilities *)msg->user_arg; + ret = ethosu_capabilities_find(cap, &edev->capabilities_list); + if (0 != ret) { + dev_warn(edev->dev, + "Handle not found in capabilities list. handle=0x%p\n", + cap); + + /* NOTE: do not call complete or kref_put on invalid data! */ + return ret; + } + + capabilities = cap->capabilities; + + capabilities->hw_id.version_status = msg->version_status; + capabilities->hw_id.version_minor = msg->version_minor; + capabilities->hw_id.version_major = msg->version_major; + capabilities->hw_id.product_major = msg->product_major; + capabilities->hw_id.arch_patch_rev = msg->arch_patch_rev; + capabilities->hw_id.arch_minor_rev = msg->arch_minor_rev; + capabilities->hw_id.arch_major_rev = msg->arch_major_rev; + capabilities->driver_patch_rev = msg->driver_patch_rev; + capabilities->driver_minor_rev = msg->driver_minor_rev; + capabilities->driver_major_rev = msg->driver_major_rev; + capabilities->hw_cfg.macs_per_cc = msg->macs_per_cc; + capabilities->hw_cfg.cmd_stream_version = msg->cmd_stream_version; + capabilities->hw_cfg.custom_dma = msg->custom_dma; + + complete(&cap->done); + + kref_put(&cap->refcount, ethosu_capabilities_destroy); + + return 0; +} + +/* Incoming messages */ +static int ethosu_handle_msg(struct ethosu_device *edev) +{ + int ret; + struct ethosu_core_msg header; + + union { + struct ethosu_core_msg_err error; + struct ethosu_core_inference_rsp inf; + struct ethosu_core_msg_version version; + struct ethosu_core_msg_capabilities_rsp capabilities; + } data; + + /* Read message */ + ret = ethosu_mailbox_read(&edev->mailbox, &header, &data, sizeof(data)); + if (ret) + return ret; + + switch (header.type) { + case ETHOSU_CORE_MSG_ERR: + if (header.length != sizeof(data.error)) { + dev_warn(edev->dev, + "Msg: Error message of incorrect size. size=%u, expected=%zu\n", header.length, + sizeof(data.error)); + ret = -EBADMSG; + break; + } + + data.error.msg[sizeof(data.error.msg) - 1] = '\0'; + dev_warn(edev->dev, "Msg: Error. type=%u, msg=\"%s\"\n", + data.error.type, data.error.msg); + ret = -EBADMSG; + break; + case ETHOSU_CORE_MSG_PING: + dev_info(edev->dev, "Msg: Ping\n"); + ret = ethosu_mailbox_pong(&edev->mailbox); + break; + case ETHOSU_CORE_MSG_PONG: + dev_info(edev->dev, "Msg: Pong\n"); + break; + case ETHOSU_CORE_MSG_INFERENCE_RSP: + if (header.length != sizeof(data.inf)) { + dev_warn(edev->dev, + "Msg: Inference response of incorrect size. size=%u, expected=%zu\n", header.length, + sizeof(data.inf)); + ret = -EBADMSG; + break; + } + + dev_info(edev->dev, + "Msg: Inference response. user_arg=0x%llx, ofm_count=%u, status=%u\n", + data.inf.user_arg, data.inf.ofm_count, + data.inf.status); + ethosu_inference_rsp(edev, &data.inf); + break; + case ETHOSU_CORE_MSG_VERSION_RSP: + if (header.length != sizeof(data.version)) { + dev_warn(edev->dev, + "Msg: Version response of incorrect size. size=%u, expected=%zu\n", header.length, + sizeof(data.version)); + ret = -EBADMSG; + break; + } + + dev_info(edev->dev, "Msg: Version response v%u.%u.%u\n", + data.version.major, data.version.minor, + data.version.patch); + + /* Check major and minor version match, else return error */ + if (data.version.major != ETHOSU_CORE_MSG_VERSION_MAJOR || + data.version.minor != ETHOSU_CORE_MSG_VERSION_MINOR) { + dev_warn(edev->dev, "Msg: Version mismatch detected! "); + dev_warn(edev->dev, "Local version: v%u.%u.%u\n", + ETHOSU_CORE_MSG_VERSION_MAJOR, + ETHOSU_CORE_MSG_VERSION_MINOR, + ETHOSU_CORE_MSG_VERSION_PATCH); + } + + break; + case ETHOSU_CORE_MSG_CAPABILITIES_RSP: + if (header.length != sizeof(data.capabilities)) { + dev_warn(edev->dev, + "Msg: Capabilities response of incorrect size. size=%u, expected=%zu\n", header.length, + sizeof(data.capabilities)); + ret = -EBADMSG; + break; + } + + dev_info(edev->dev, + "Msg: Capabilities response ua%llx vs%hhu v%hhu.%hhu p%hhu av%hhu.%hhu.%hhu dv%hhu.%hhu.%hhu mcc%hhu csv%hhu cd%hhu\n", + data.capabilities.user_arg, + data.capabilities.version_status, + data.capabilities.version_major, + data.capabilities.version_minor, + data.capabilities.product_major, + data.capabilities.arch_major_rev, + data.capabilities.arch_minor_rev, + data.capabilities.arch_patch_rev, + data.capabilities.driver_major_rev, + data.capabilities.driver_minor_rev, + data.capabilities.driver_patch_rev, + data.capabilities.macs_per_cc, + data.capabilities.cmd_stream_version, + data.capabilities.custom_dma); + + ret = ethosu_capability_rsp(edev, &data.capabilities); + break; + default: + /* This should not happen due to version checks */ + dev_warn(edev->dev, "Msg: Protocol error\n"); + ret = -EPROTO; + break; + } + + return ret; +} + +static int ethosu_open(struct inode *inode, + struct file *file) +{ + struct ethosu_device *edev = + container_of(inode->i_cdev, struct ethosu_device, cdev); + + file->private_data = edev; + + dev_info(edev->dev, "Opening device node.\n"); + + return nonseekable_open(inode, file); +} + +static int ethosu_send_capabilities_request(struct ethosu_device *edev, + void __user *udata) +{ + struct ethosu_uapi_device_capabilities uapi; + struct ethosu_capabilities *cap; + int ret; + int timeout; + + cap = devm_kzalloc(edev->dev, sizeof(struct ethosu_capabilities), + GFP_KERNEL); + if (!cap) + return -ENOMEM; + + cap->edev = edev; + cap->capabilities = &uapi; + kref_init(&cap->refcount); + init_completion(&cap->done); + list_add(&cap->list, &edev->capabilities_list); + + ret = ethosu_mailbox_capabilities_request(&edev->mailbox, cap); + if (0 != ret) + goto put_kref; + + /* + * Increase ref counter since we sent the pointer out to + * response handler thread. That thread is responsible to + * decrease the ref counter before exiting. So the memory + * can be freed. + * + * NOTE: if no response is received back, the memory is leaked. + */ + kref_get(&cap->refcount); + /* Unlock the mutex before going to block on the condition */ + mutex_unlock(&edev->mutex); + /* wait for response to arrive back */ + timeout = wait_for_completion_timeout(&cap->done, + msecs_to_jiffies( + CAPABILITIES_RESP_TIMEOUT_MS)); + /* take back the mutex before resuming to do anything */ + ret = mutex_lock_interruptible(&edev->mutex); + if (0 != ret) + goto put_kref; + + if (0 == timeout /* timed out*/) { + dev_warn(edev->dev, + "Msg: Capabilities response lost - timeout\n"); + ret = -EIO; + goto put_kref; + } + + ret = copy_to_user(udata, &uapi, sizeof(uapi)) ? -EFAULT : 0; + +put_kref: + kref_put(&cap->refcount, ethosu_capabilities_destroy); + + return ret; +} + +static long ethosu_ioctl(struct file *file, + unsigned int cmd, + unsigned long arg) +{ + struct ethosu_device *edev = file->private_data; + void __user *udata = (void __user *)arg; + int ret = -EINVAL; + + ret = mutex_lock_interruptible(&edev->mutex); + if (ret) + return ret; + + dev_info(edev->dev, "Ioctl. cmd=%u, arg=%lu\n", cmd, arg); + + switch (cmd) { + case ETHOSU_IOCTL_VERSION_REQ: + dev_info(edev->dev, "Ioctl: Send version request\n"); + ret = ethosu_mailbox_version_request(&edev->mailbox); + break; + case ETHOSU_IOCTL_CAPABILITIES_REQ: + dev_info(edev->dev, "Ioctl: Send capabilities request\n"); + ret = ethosu_send_capabilities_request(edev, udata); + break; + case ETHOSU_IOCTL_PING: { + dev_info(edev->dev, "Ioctl: Send ping\n"); + ret = ethosu_mailbox_ping(&edev->mailbox); + break; + } + case ETHOSU_IOCTL_BUFFER_CREATE: { + struct ethosu_uapi_buffer_create uapi; + + dev_info(edev->dev, "Ioctl: Buffer create\n"); + + if (copy_from_user(&uapi, udata, sizeof(uapi))) + break; + + dev_info(edev->dev, "Ioctl: Buffer. capacity=%u\n", + uapi.capacity); + + ret = ethosu_buffer_create(edev, uapi.capacity); + break; + } + case ETHOSU_IOCTL_NETWORK_CREATE: { + struct ethosu_uapi_network_create uapi; + + if (copy_from_user(&uapi, udata, sizeof(uapi))) + break; + + dev_info(edev->dev, "Ioctl: Network. fd=%u\n", uapi.fd); + + ret = ethosu_network_create(edev, &uapi); + break; + } + default: { + dev_err(edev->dev, "Invalid ioctl. cmd=%u, arg=%lu", + cmd, arg); + break; + } + } + + mutex_unlock(&edev->mutex); + + return ret; +} + +static void ethosu_mbox_rx(void *user_arg) +{ + struct ethosu_device *edev = user_arg; + int ret; + + mutex_lock(&edev->mutex); + + do { + ret = ethosu_handle_msg(edev); + if (ret && ret != -ENOMSG) + /* Need to start over in case of error, empty the queue + * by fast-forwarding read position to write position. + * */ + ethosu_mailbox_reset(&edev->mailbox); + } while (ret == 0); + + mutex_unlock(&edev->mutex); +} + +int ethosu_dev_init(struct ethosu_device *edev, + struct device *dev, + struct class *class, + dev_t devt, + struct resource *in_queue, + struct resource *out_queue) +{ + static const struct file_operations fops = { + .owner = THIS_MODULE, + .open = ðosu_open, + .unlocked_ioctl = ðosu_ioctl, +#ifdef CONFIG_COMPAT + .compat_ioctl = ðosu_ioctl, +#endif + }; + struct device *sysdev; + int ret; + + edev->dev = dev; + edev->class = class; + edev->devt = devt; + mutex_init(&edev->mutex); + INIT_LIST_HEAD(&edev->capabilities_list); + INIT_LIST_HEAD(&edev->inference_list); + + ret = of_reserved_mem_device_init(edev->dev); + if (ret) + return ret; + + dma_set_mask_and_coherent(edev->dev, DMA_BIT_MASK(DMA_ADDR_BITS)); + + ret = ethosu_mailbox_init(&edev->mailbox, dev, in_queue, out_queue, + ethosu_mbox_rx, edev); + if (ret) + goto release_reserved_mem; + + cdev_init(&edev->cdev, &fops); + edev->cdev.owner = THIS_MODULE; + + ret = cdev_add(&edev->cdev, edev->devt, 1); + if (ret) { + dev_err(edev->dev, "Failed to add character device.\n"); + goto deinit_mailbox; + } + + sysdev = device_create(edev->class, NULL, edev->devt, edev, + "ethosu%d", MINOR(edev->devt)); + if (IS_ERR(sysdev)) { + dev_err(edev->dev, "Failed to create device.\n"); + ret = PTR_ERR(sysdev); + goto del_cdev; + } + + dev_info(edev->dev, + "Created Arm Ethos-U device. name=%s, major=%d, minor=%d\n", + dev_name(sysdev), MAJOR(edev->devt), MINOR(edev->devt)); + + return 0; + +del_cdev: + cdev_del(&edev->cdev); + +deinit_mailbox: + ethosu_mailbox_deinit(&edev->mailbox); + +release_reserved_mem: + of_reserved_mem_device_release(edev->dev); + + return ret; +} + +void ethosu_dev_deinit(struct ethosu_device *edev) +{ + ethosu_mailbox_deinit(&edev->mailbox); + device_destroy(edev->class, edev->cdev.dev); + cdev_del(&edev->cdev); + of_reserved_mem_device_release(edev->dev); + + dev_info(edev->dev, "%s\n", __FUNCTION__); +} diff --git a/drivers/firmware/ethosu/ethosu_device.h b/drivers/firmware/ethosu/ethosu_device.h new file mode 100644 index 000000000000..3afdda84862d --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_device.h @@ -0,0 +1,88 @@ +/* + * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * + * This program is free software and is provided to you under the terms of the + * GNU General Public License version 2 as published by the Free Software + * Foundation, and any use by you of this program is subject to the terms + * of such GNU licence. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, you can access it online at + * http://www.gnu.org/licenses/gpl-2.0.html. + * + * SPDX-License-Identifier: GPL-2.0-only + */ + +#ifndef ETHOSU_DEVICE_H +#define ETHOSU_DEVICE_H + +/**************************************************************************** + * Includes + ****************************************************************************/ + +#include "uapi/ethosu.h" +#include "ethosu_mailbox.h" + +#include +#include +#include +#include +#include +#include + +/**************************************************************************** + * Types + ****************************************************************************/ + +/** + * struct ethosu_device - Device structure + */ +struct ethosu_device { + struct device *dev; + struct cdev cdev; + struct class *class; + dev_t devt; + struct mutex mutex; + struct ethosu_mailbox mailbox; + struct list_head capabilities_list; + struct list_head inference_list; +}; + +/** + * struct ethosu_capabilities - Capabilities internal struct + */ +struct ethosu_capabilities { + struct ethosu_device *edev; + struct completion done; + struct kref refcount; + struct ethosu_uapi_device_capabilities *capabilities; + struct list_head list; +}; + +/**************************************************************************** + * Functions + ****************************************************************************/ + +/** + * ethosu_dev_init() - Initialize the device + * + * Return: 0 on success, else error code. + */ +int ethosu_dev_init(struct ethosu_device *edev, + struct device *dev, + struct class *class, + dev_t devt, + struct resource *in_queue, + struct resource *out_queue); + +/** + * ethosu_dev_deinit() - Initialize the device + */ +void ethosu_dev_deinit(struct ethosu_device *edev); + +#endif /* ETHOSU_DEVICE_H */ diff --git a/drivers/firmware/ethosu/ethosu_driver.c b/drivers/firmware/ethosu/ethosu_driver.c new file mode 100644 index 000000000000..9d02431d3194 --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_driver.c @@ -0,0 +1,191 @@ +/* + * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * + * This program is free software and is provided to you under the terms of the + * GNU General Public License version 2 as published by the Free Software + * Foundation, and any use by you of this program is subject to the terms + * of such GNU licence. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, you can access it online at + * http://www.gnu.org/licenses/gpl-2.0.html. + * + * SPDX-License-Identifier: GPL-2.0-only + */ + +#include +#include +#include +#include +#include +#include +#include + +#include "ethosu_device.h" + +/**************************************************************************** + * Defines + ****************************************************************************/ + +#define ETHOSU_DRIVER_VERSION "1.0" +#define ETHOSU_DRIVER_NAME "ethosu" + +#define MINOR_BASE 0 /* Minor version starts at 0 */ +#define MINOR_COUNT 64 /* Allocate minor versions */ + +/**************************************************************************** + * Variables + ****************************************************************************/ + +static struct class *ethosu_class; + +static dev_t devt; + +static DECLARE_BITMAP(minors, MINOR_COUNT); + +/**************************************************************************** + * Arm Ethos-U + ****************************************************************************/ + +static int ethosu_pdev_probe(struct platform_device *pdev) +{ + struct ethosu_device *edev; + struct resource *in_queue_res; + struct resource *out_queue_res; + int minor; + int ret; + + dev_info(&pdev->dev, "Probe\n"); + + minor = find_first_zero_bit(minors, MINOR_COUNT); + if (minor >= MINOR_COUNT) { + dev_err(&pdev->dev, "No more minor numbers.\n"); + + return -ENOMEM; + } + + /* Get path to TCM memory */ + in_queue_res = platform_get_resource_byname(pdev, IORESOURCE_MEM, + "in_queue"); + if (IS_ERR(in_queue_res)) { + dev_err(&pdev->dev, "Failed to get in_queue resource.\n"); + + return PTR_ERR(in_queue_res); + } + + out_queue_res = platform_get_resource_byname(pdev, IORESOURCE_MEM, + "out_queue"); + if (IS_ERR(out_queue_res)) { + dev_err(&pdev->dev, "Failed to get out_queue resource.\n"); + + return PTR_ERR(out_queue_res); + } + + /* Allocate memory for Arm Ethos-U device */ + edev = devm_kzalloc(&pdev->dev, sizeof(*edev), GFP_KERNEL); + if (!edev) + return -ENOMEM; + + platform_set_drvdata(pdev, edev); + + /* Initialize device */ + ret = ethosu_dev_init(edev, &pdev->dev, ethosu_class, + MKDEV(MAJOR(devt), minor), in_queue_res, + out_queue_res); + if (ret) + goto free_dev; + + set_bit(minor, minors); + + return 0; + +free_dev: + devm_kfree(&pdev->dev, edev); + + return ret; +} + +static int ethosu_pdev_remove(struct platform_device *pdev) +{ + struct ethosu_device *edev = platform_get_drvdata(pdev); + + clear_bit(MINOR(edev->devt), minors); + ethosu_dev_deinit(edev); + + return 0; +} + +static const struct of_device_id ethosu_pdev_match[] = { + { .compatible = "arm,ethosu" }, + { /* Sentinel */ }, +}; + +MODULE_DEVICE_TABLE(of, ethosu_pdev_match); + +static struct platform_driver ethosu_pdev_driver = { + .probe = ðosu_pdev_probe, + .remove = ðosu_pdev_remove, + .driver = { + .name = ETHOSU_DRIVER_NAME, + .owner = THIS_MODULE, + .of_match_table = of_match_ptr(ethosu_pdev_match), + }, +}; + +/**************************************************************************** + * Module init and exit + ****************************************************************************/ + +static int __init ethosu_init(void) +{ + int ret; + + ethosu_class = class_create(THIS_MODULE, ETHOSU_DRIVER_NAME); + if (IS_ERR(ethosu_class)) { + printk("Failed to create class '%s'.\n", ETHOSU_DRIVER_NAME); + + return PTR_ERR(ethosu_class); + } + + ret = alloc_chrdev_region(&devt, MINOR_BASE, MINOR_COUNT, + ETHOSU_DRIVER_NAME); + if (ret) { + printk("Failed to allocate chrdev region.\n"); + goto destroy_class; + } + + ret = platform_driver_register(ðosu_pdev_driver); + if (ret) { + printk("Failed to register Arm Ethos-U platform driver.\n"); + goto region_unregister; + } + + return 0; + +region_unregister: + unregister_chrdev_region(devt, MINOR_COUNT); + +destroy_class: + class_destroy(ethosu_class); + + return ret; +} + +static void __exit ethosu_exit(void) +{ + platform_driver_unregister(ðosu_pdev_driver); + unregister_chrdev_region(devt, MINOR_COUNT); + class_destroy(ethosu_class); +} + +module_init(ethosu_init) +module_exit(ethosu_exit) +MODULE_LICENSE("GPL v2"); +MODULE_AUTHOR("Arm Ltd"); +MODULE_DESCRIPTION("Arm Ethos-U NPU Driver"); +MODULE_VERSION(ETHOSU_DRIVER_VERSION); diff --git a/drivers/firmware/ethosu/ethosu_inference.c b/drivers/firmware/ethosu/ethosu_inference.c new file mode 100644 index 000000000000..6fde92c148a0 --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_inference.c @@ -0,0 +1,417 @@ +/* + * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * + * This program is free software and is provided to you under the terms of the + * GNU General Public License version 2 as published by the Free Software + * Foundation, and any use by you of this program is subject to the terms + * of such GNU licence. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, you can access it online at + * http://www.gnu.org/licenses/gpl-2.0.html. + * + * SPDX-License-Identifier: GPL-2.0-only + */ + +/**************************************************************************** + * Includes + ****************************************************************************/ + +#include "ethosu_inference.h" + +#include "ethosu_buffer.h" +#include "ethosu_core_interface.h" +#include "ethosu_device.h" +#include "ethosu_network.h" +#include "uapi/ethosu.h" + +#include +#include +#include +#include + +/**************************************************************************** + * Variables + ****************************************************************************/ + +static int ethosu_inference_release(struct inode *inode, + struct file *file); + +static unsigned int ethosu_inference_poll(struct file *file, + poll_table *wait); + +static long ethosu_inference_ioctl(struct file *file, + unsigned int cmd, + unsigned long arg); + +static const struct file_operations ethosu_inference_fops = { + .release = ðosu_inference_release, + .poll = ðosu_inference_poll, + .unlocked_ioctl = ðosu_inference_ioctl, +#ifdef CONFIG_COMPAT + .compat_ioctl = ðosu_inference_ioctl, +#endif +}; + +/**************************************************************************** + * Functions + ****************************************************************************/ + +static const char *status_to_string(const enum ethosu_uapi_status status) +{ + switch (status) { + case ETHOSU_UAPI_STATUS_OK: { + return "Ok"; + } + case ETHOSU_UAPI_STATUS_ERROR: { + return "Error"; + } + default: { + return "Unknown"; + } + } +} + +static int ethosu_inference_send(struct ethosu_inference *inf) +{ + int ret; + + if (inf->pending) + return -EINVAL; + + inf->status = ETHOSU_UAPI_STATUS_ERROR; + + ret = ethosu_mailbox_inference(&inf->edev->mailbox, inf, + inf->ifm_count, inf->ifm, + inf->ofm_count, inf->ofm, + inf->net->buf, + inf->pmu_event_config, + ETHOSU_PMU_EVENT_MAX, + inf->pmu_cycle_counter_enable); + if (ret) + return ret; + + inf->pending = true; + + ethosu_inference_get(inf); + + return 0; +} + +static int ethosu_inference_find(struct ethosu_inference *inf, + struct list_head *inference_list) +{ + struct ethosu_inference *cur; + + list_for_each_entry(cur, inference_list, list) { + if (cur == inf) + return 0; + } + + return -EINVAL; +} + +static bool ethosu_inference_verify(struct file *file) +{ + return file->f_op == ðosu_inference_fops; +} + +static void ethosu_inference_kref_destroy(struct kref *kref) +{ + struct ethosu_inference *inf = + container_of(kref, struct ethosu_inference, kref); + + dev_info(inf->edev->dev, + "Inference destroy. handle=0x%pK, status=%d\n", + inf, inf->status); + + list_del(&inf->list); + + while (inf->ifm_count-- > 0) + ethosu_buffer_put(inf->ifm[inf->ifm_count]); + + while (inf->ofm_count-- > 0) + ethosu_buffer_put(inf->ofm[inf->ofm_count]); + + ethosu_network_put(inf->net); + devm_kfree(inf->edev->dev, inf); +} + +static int ethosu_inference_release(struct inode *inode, + struct file *file) +{ + struct ethosu_inference *inf = file->private_data; + + dev_info(inf->edev->dev, + "Inference release. handle=0x%pK, status=%d\n", + inf, inf->status); + + ethosu_inference_put(inf); + + return 0; +} + +static unsigned int ethosu_inference_poll(struct file *file, + poll_table *wait) +{ + struct ethosu_inference *inf = file->private_data; + int ret = 0; + + poll_wait(file, &inf->waitq, wait); + + if (!inf->pending) + ret |= POLLIN; + + return ret; +} + +static long ethosu_inference_ioctl(struct file *file, + unsigned int cmd, + unsigned long arg) +{ + struct ethosu_inference *inf = file->private_data; + void __user *udata = (void __user *)arg; + int ret; + + ret = mutex_lock_interruptible(&inf->edev->mutex); + if (ret) + return ret; + + dev_info(inf->edev->dev, "Ioctl: cmd=%u, arg=%lu\n", cmd, arg); + + switch (cmd) { + case ETHOSU_IOCTL_INFERENCE_STATUS: { + struct ethosu_uapi_result_status uapi; + int i; + + uapi.status = inf->status; + + for (i = 0; i < ETHOSU_PMU_EVENT_MAX; i++) { + uapi.pmu_config.events[i] = + inf->pmu_event_config[i]; + uapi.pmu_count.events[i] = + inf->pmu_event_count[i]; + } + + uapi.pmu_config.cycle_count = inf->pmu_cycle_counter_enable; + uapi.pmu_count.cycle_count = inf->pmu_cycle_counter_count; + + dev_info(inf->edev->dev, + "Ioctl: Inference status. status=%s (%d)\n", + status_to_string(uapi.status), uapi.status); + + ret = copy_to_user(udata, &uapi, sizeof(uapi)) ? -EFAULT : 0; + + break; + } + default: { + dev_err(inf->edev->dev, "Invalid ioctl. cmd=%u, arg=%lu", + cmd, arg); + break; + } + } + + mutex_unlock(&inf->edev->mutex); + + return ret; +} + +int ethosu_inference_create(struct ethosu_device *edev, + struct ethosu_network *net, + struct ethosu_uapi_inference_create *uapi) +{ + struct ethosu_inference *inf; + uint32_t i; + int fd; + int ret = -ENOMEM; + + inf = devm_kzalloc(edev->dev, sizeof(*inf), GFP_KERNEL); + if (!inf) + return -ENOMEM; + + inf->edev = edev; + inf->net = net; + inf->pending = false; + inf->status = ETHOSU_UAPI_STATUS_ERROR; + kref_init(&inf->kref); + init_waitqueue_head(&inf->waitq); + + /* Get pointer to IFM buffers */ + for (i = 0; i < uapi->ifm_count; i++) { + inf->ifm[i] = ethosu_buffer_get_from_fd(uapi->ifm_fd[i]); + if (IS_ERR(inf->ifm[i])) { + ret = PTR_ERR(inf->ifm[i]); + goto put_ifm; + } + + inf->ifm_count++; + } + + /* Get pointer to OFM buffer */ + for (i = 0; i < uapi->ofm_count; i++) { + inf->ofm[i] = ethosu_buffer_get_from_fd(uapi->ofm_fd[i]); + if (IS_ERR(inf->ofm[i])) { + ret = PTR_ERR(inf->ofm[i]); + goto put_ofm; + } + + inf->ofm_count++; + } + + /* Configure PMU and cycle counter */ + dev_info(inf->edev->dev, + "Configuring events for PMU. events=[%u, %u, %u, %u]\n", + uapi->pmu_config.events[0], uapi->pmu_config.events[1], + uapi->pmu_config.events[2], uapi->pmu_config.events[3]); + + /* Configure events and reset count for all events */ + for (i = 0; i < ETHOSU_PMU_EVENT_MAX; i++) { + inf->pmu_event_config[i] = uapi->pmu_config.events[i]; + inf->pmu_event_count[i] = 0; + } + + if (uapi->pmu_config.cycle_count) + dev_info(inf->edev->dev, "Enabling cycle counter\n"); + + /* Configure cycle counter and reset any previous count */ + inf->pmu_cycle_counter_enable = uapi->pmu_config.cycle_count; + inf->pmu_cycle_counter_count = 0; + + /* Increment network reference count */ + ethosu_network_get(net); + + /* Create file descriptor */ + ret = fd = anon_inode_getfd("ethosu-inference", ðosu_inference_fops, + inf, O_RDWR | O_CLOEXEC); + if (ret < 0) + goto put_net; + + /* Store pointer to file structure */ + inf->file = fget(ret); + fput(inf->file); + + /* Add inference to inference list */ + list_add(&inf->list, &edev->inference_list); + + /* Send inference request to Arm Ethos-U subsystem */ + (void)ethosu_inference_send(inf); + + dev_info(edev->dev, "Inference create. handle=0x%pK, fd=%d", + inf, fd); + + return fd; + +put_net: + ethosu_network_put(inf->net); + +put_ofm: + while (inf->ofm_count-- > 0) + ethosu_buffer_put(inf->ofm[inf->ofm_count]); + +put_ifm: + while (inf->ifm_count-- > 0) + ethosu_buffer_put(inf->ifm[inf->ifm_count]); + + devm_kfree(edev->dev, inf); + + return ret; +} + +struct ethosu_inference *ethosu_inference_get_from_fd(int fd) +{ + struct ethosu_inference *inf; + struct file *file; + + file = fget(fd); + if (!file) + return ERR_PTR(-EINVAL); + + if (!ethosu_inference_verify(file)) { + fput(file); + + return ERR_PTR(-EINVAL); + } + + inf = file->private_data; + ethosu_inference_get(inf); + fput(file); + + return inf; +} + +void ethosu_inference_get(struct ethosu_inference *inf) +{ + kref_get(&inf->kref); +} + +void ethosu_inference_put(struct ethosu_inference *inf) +{ + kref_put(&inf->kref, ðosu_inference_kref_destroy); +} + +void ethosu_inference_rsp(struct ethosu_device *edev, + struct ethosu_core_inference_rsp *rsp) +{ + struct ethosu_inference *inf = + (struct ethosu_inference *)rsp->user_arg; + int ret; + int i; + + ret = ethosu_inference_find(inf, &edev->inference_list); + if (ret) { + dev_warn(edev->dev, + "Handle not found in inference list. handle=0x%p\n", + rsp); + + return; + } + + inf->pending = false; + + if (rsp->status == ETHOSU_CORE_STATUS_OK && + inf->ofm_count <= ETHOSU_CORE_BUFFER_MAX) { + uint32_t i; + + inf->status = ETHOSU_UAPI_STATUS_OK; + + for (i = 0; i < inf->ofm_count; i++) { + struct ethosu_buffer *ofm = inf->ofm[i]; + + ret = ethosu_buffer_resize( + ofm, ofm->size + rsp->ofm_size[i], + ofm->offset); + if (ret) + inf->status = ETHOSU_UAPI_STATUS_ERROR; + } + } else { + inf->status = ETHOSU_UAPI_STATUS_ERROR; + } + + for (i = 0; i < ETHOSU_CORE_PMU_MAX; i++) { + inf->pmu_event_config[i] = rsp->pmu_event_config[i]; + inf->pmu_event_count[i] = rsp->pmu_event_count[i]; + } + + inf->pmu_cycle_counter_enable = rsp->pmu_cycle_counter_enable; + inf->pmu_cycle_counter_count = rsp->pmu_cycle_counter_count; + + dev_info(edev->dev, + "PMU events. config=[%u, %u, %u, %u], count=[%u, %u, %u, %u]\n", + inf->pmu_event_config[0], inf->pmu_event_config[1], + inf->pmu_event_config[2], inf->pmu_event_config[3], + inf->pmu_event_count[0], inf->pmu_event_count[1], + inf->pmu_event_count[2], inf->pmu_event_count[3]); + + dev_info(edev->dev, + "PMU cycle counter. enable=%u, count=%llu\n", + inf->pmu_cycle_counter_enable, + inf->pmu_cycle_counter_count); + wake_up_interruptible(&inf->waitq); + + ethosu_inference_put(inf); +} diff --git a/drivers/firmware/ethosu/ethosu_inference.h b/drivers/firmware/ethosu/ethosu_inference.h new file mode 100644 index 000000000000..07370ca01f22 --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_inference.h @@ -0,0 +1,120 @@ +/* + * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * + * This program is free software and is provided to you under the terms of the + * GNU General Public License version 2 as published by the Free Software + * Foundation, and any use by you of this program is subject to the terms + * of such GNU licence. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, you can access it online at + * http://www.gnu.org/licenses/gpl-2.0.html. + * + * SPDX-License-Identifier: GPL-2.0-only + */ + +#ifndef ETHOSU_INFERENCE_H +#define ETHOSU_INFERENCE_H + +/**************************************************************************** + * Includes + ****************************************************************************/ + +#include "uapi/ethosu.h" + +#include +#include +#include + +/**************************************************************************** + * Types + ****************************************************************************/ + +struct ethosu_buffer; +struct ethosu_core_inference_rsp; +struct ethosu_device; +struct ethosu_network; +struct ethosu_uapi_inference_create; +struct file; + +/** + * struct ethosu_inference - Inference struct + * @edev: Arm Ethos-U device + * @file: File handle + * @kref: Reference counter + * @waitq: Wait queue + * @ifm: Pointer to IFM buffer + * @ofm: Pointer to OFM buffer + * @net: Pointer to network + * @pending: Pending response from the firmware + * @status: Inference status + * @pmu_event_config: PMU event configuration + * @pmu_event_count: PMU event count after inference + * @pmu_cycle_counter_enable: PMU cycle counter config + * @pmu_cycle_counter_count: PMU cycle counter count after inference + */ +struct ethosu_inference { + struct ethosu_device *edev; + struct file *file; + struct kref kref; + wait_queue_head_t waitq; + uint32_t ifm_count; + struct ethosu_buffer *ifm[ETHOSU_FD_MAX]; + uint32_t ofm_count; + struct ethosu_buffer *ofm[ETHOSU_FD_MAX]; + struct ethosu_network *net; + bool pending; + enum ethosu_uapi_status status; + uint8_t pmu_event_config[ETHOSU_PMU_EVENT_MAX]; + uint32_t pmu_event_count[ETHOSU_PMU_EVENT_MAX]; + uint32_t pmu_cycle_counter_enable; + uint64_t pmu_cycle_counter_count; + struct list_head list; +}; + +/**************************************************************************** + * Functions + ****************************************************************************/ + +/** + * ethosu_inference_create() - Create inference + * + * This function must be called in the context of a user space process. + * + * Return: fd on success, else error code. + */ +int ethosu_inference_create(struct ethosu_device *edev, + struct ethosu_network *net, + struct ethosu_uapi_inference_create *uapi); + +/** + * ethosu_inference_get_from_fd() - Get inference handle from fd + * + * This function must be called from a user space context. + * + * Return: Pointer on success, else ERR_PTR. + */ +struct ethosu_inference *ethosu_inference_get_from_fd(int fd); + +/** + * ethosu_inference_get() - Get inference + */ +void ethosu_inference_get(struct ethosu_inference *inf); + +/** + * ethosu_inference_put() - Put inference + */ +void ethosu_inference_put(struct ethosu_inference *inf); + +/** + * ethosu_inference_rsp() - Handle inference response + */ +void ethosu_inference_rsp(struct ethosu_device *edev, + struct ethosu_core_inference_rsp *rsp); + +#endif /* ETHOSU_INFERENCE_H */ diff --git a/drivers/firmware/ethosu/ethosu_mailbox.c b/drivers/firmware/ethosu/ethosu_mailbox.c new file mode 100644 index 000000000000..7f159f3b0a60 --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_mailbox.c @@ -0,0 +1,377 @@ +/* + * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * + * This program is free software and is provided to you under the terms of the + * GNU General Public License version 2 as published by the Free Software + * Foundation, and any use by you of this program is subject to the terms + * of such GNU licence. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, you can access it online at + * http://www.gnu.org/licenses/gpl-2.0.html. + * + * SPDX-License-Identifier: GPL-2.0-only + */ + +/**************************************************************************** + * Includes + ****************************************************************************/ + +#include "ethosu_mailbox.h" + +#include "ethosu_buffer.h" +#include "ethosu_core_interface.h" +#include "ethosu_device.h" + +#include +#include + +/**************************************************************************** + * Functions + ****************************************************************************/ + +static void ethosu_core_set_size(struct ethosu_buffer *buf, + struct ethosu_core_buffer *cbuf) +{ + cbuf->ptr = (uint32_t)buf->dma_addr + buf->offset; + cbuf->size = (uint32_t)buf->size; +} + +static void ethosu_core_set_capacity(struct ethosu_buffer *buf, + struct ethosu_core_buffer *cbuf) +{ + cbuf->ptr = (uint32_t)buf->dma_addr + buf->offset + buf->size; + cbuf->size = (uint32_t)buf->capacity - buf->offset - buf->size; +} + +static size_t ethosu_queue_available(struct ethosu_core_queue *queue) +{ + size_t size = queue->header.write - queue->header.read; + + if (queue->header.read > queue->header.write) + size += queue->header.size; + + return size; +} + +static size_t ethosu_queue_capacity(struct ethosu_core_queue *queue) +{ + return queue->header.size - ethosu_queue_available(queue); +} + +static int ethosu_queue_write(struct ethosu_mailbox *mbox, + const struct kvec *vec, + size_t length) +{ + struct ethosu_core_queue *queue = mbox->in_queue; + uint8_t *dst = &queue->data[0]; + uint32_t wpos = queue->header.write; + size_t total_size; + size_t i; + int ret; + + for (i = 0, total_size = 0; i < length; i++) + total_size += vec[i].iov_len; + + if (total_size > ethosu_queue_capacity(queue)) + return -EINVAL; + + for (i = 0; i < length; i++) { + const uint8_t *src = vec[i].iov_base; + const uint8_t *end = src + vec[i].iov_len; + + while (src < end) { + dst[wpos] = *src++; + wpos = (wpos + 1) % queue->header.size; + } + } + + queue->header.write = wpos; + + ret = mbox_send_message(mbox->tx, queue); + if (ret < 0) + return ret; + + return 0; +} + +static int ethosu_queue_write_msg(struct ethosu_mailbox *mbox, + uint32_t type, + void *data, + size_t length) +{ + struct ethosu_core_msg msg = { + .magic = ETHOSU_CORE_MSG_MAGIC, + .type = type, .length= length + }; + const struct kvec vec[2] = { + { &msg, sizeof(msg) }, + { data, length } + }; + + return ethosu_queue_write(mbox, vec, 2); +} + +static int ethosu_queue_read(struct ethosu_mailbox *mbox, + void *data, + size_t length) +{ + struct ethosu_core_queue *queue = mbox->out_queue; + uint8_t *src = &queue->data[0]; + uint8_t *dst = (uint8_t *)data; + const uint8_t *end = dst + length; + uint32_t rpos = queue->header.read; + size_t queue_avail = ethosu_queue_available(queue); + + if (length == 0) + return 0; + else if (queue_avail == 0) + return -ENOMSG; + else if (length > queue_avail) + return -EBADMSG; + + while (dst < end) { + *dst++ = src[rpos]; + rpos = (rpos + 1) % queue->header.size; + } + + queue->header.read = rpos; + + return 0; +} + +void ethosu_mailbox_reset(struct ethosu_mailbox *mbox) +{ + mbox->out_queue->header.read = mbox->out_queue->header.write; +} + +int ethosu_mailbox_read(struct ethosu_mailbox *mbox, + struct ethosu_core_msg *header, + void *data, + size_t length) +{ + int ret; + + /* Read message header magic */ + ret = ethosu_queue_read(mbox, header, sizeof(*header)); + if (ret) { + if (ret != -ENOMSG) + dev_warn(mbox->dev, + "Msg: Failed to read message header\n"); + + return ret; + } + + if (header->magic != ETHOSU_CORE_MSG_MAGIC) { + dev_warn(mbox->dev, + "Msg: Invalid magic. Got: %08X but expected %08X\n", + header->magic, ETHOSU_CORE_MSG_MAGIC); + + return -EINVAL; + } + + dev_info(mbox->dev, + "mbox: Read msg header. magic=%08X, type=%u, length=%u", + header->magic, header->type, header->length); + + /* Check that payload is not larger than allocated buffer */ + if (header->length > length) { + dev_warn(mbox->dev, + "Msg: Buffer size (%zu) too small for message (%u)\n", + sizeof(data), header->length); + + return -ENOMEM; + } + + /* Read payload data */ + ret = ethosu_queue_read(mbox, data, header->length); + if (ret) { + dev_warn(mbox->dev, "Msg: Failed to read payload data\n"); + + return -EBADMSG; + } + + return 0; +} + +int ethosu_mailbox_ping(struct ethosu_mailbox *mbox) +{ + return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_PING, NULL, 0); +} + +int ethosu_mailbox_pong(struct ethosu_mailbox *mbox) +{ + return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_PONG, NULL, 0); +} + +int ethosu_mailbox_version_request(struct ethosu_mailbox *mbox) +{ + return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_VERSION_REQ, NULL, + 0); +} + +int ethosu_mailbox_capabilities_request(struct ethosu_mailbox *mbox, + void *user_arg) +{ + struct ethosu_core_capabilities_req req = { + .user_arg = (ptrdiff_t)user_arg + }; + + return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_CAPABILITIES_REQ, + &req, + sizeof(req)); +} + +int ethosu_mailbox_inference(struct ethosu_mailbox *mbox, + void *user_arg, + uint32_t ifm_count, + struct ethosu_buffer **ifm, + uint32_t ofm_count, + struct ethosu_buffer **ofm, + struct ethosu_buffer *network, + uint8_t *pmu_event_config, + uint8_t pmu_event_config_count, + uint8_t pmu_cycle_counter_enable) +{ + struct ethosu_core_inference_req inf; + uint32_t i; + + /* Verify that the uapi and core has the same number of pmus */ + if (pmu_event_config_count != ETHOSU_CORE_PMU_MAX) { + dev_err(mbox->dev, "PMU count misconfigured.\n"); + + return -EINVAL; + } + + inf.user_arg = (ptrdiff_t)user_arg; + inf.ifm_count = ifm_count; + inf.ofm_count = ofm_count; + inf.pmu_cycle_counter_enable = pmu_cycle_counter_enable; + + for (i = 0; i < ifm_count; i++) + ethosu_core_set_size(ifm[i], &inf.ifm[i]); + + for (i = 0; i < ofm_count; i++) + ethosu_core_set_capacity(ofm[i], &inf.ofm[i]); + + for (i = 0; i < ETHOSU_CORE_PMU_MAX; i++) + inf.pmu_event_config[i] = pmu_event_config[i]; + + ethosu_core_set_size(network, &inf.network); + + return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_INFERENCE_REQ, + &inf, sizeof(inf)); +} + +static void ethosu_mailbox_rx_work(struct work_struct *work) +{ + struct ethosu_mailbox *mbox = container_of(work, typeof(*mbox), work); + + mbox->callback(mbox->user_arg); +} + +static void ethosu_mailbox_rx_callback(struct mbox_client *client, + void *message) +{ + struct ethosu_mailbox *mbox = + container_of(client, typeof(*mbox), client); + + dev_info(mbox->dev, "mbox: Received message.\n"); + + queue_work(mbox->wq, &mbox->work); +} + +static void ethosu_mailbox_tx_done(struct mbox_client *client, + void *message, + int r) +{ + if (r) + dev_warn(client->dev, "mbox: Failed sending message (%d)\n", r); + else + dev_info(client->dev, "mbox: Message sent\n"); +} + +int ethosu_mailbox_init(struct ethosu_mailbox *mbox, + struct device *dev, + struct resource *in_queue, + struct resource *out_queue, + ethosu_mailbox_cb callback, + void *user_arg) +{ + int ret; + + mbox->dev = dev; + mbox->callback = callback; + mbox->user_arg = user_arg; + + mbox->client.dev = dev; + mbox->client.rx_callback = ethosu_mailbox_rx_callback; + mbox->client.tx_prepare = NULL; /* preparation of data is handled + * through the + * queue functions */ + mbox->client.tx_done = ethosu_mailbox_tx_done; + mbox->client.tx_block = true; + mbox->client.knows_txdone = false; + mbox->client.tx_tout = 500; + + mbox->in_queue = devm_ioremap_resource(mbox->dev, in_queue); + if (IS_ERR(mbox->in_queue)) + return PTR_ERR(mbox->in_queue); + + mbox->out_queue = devm_ioremap_resource(mbox->dev, out_queue); + if (IS_ERR(mbox->out_queue)) { + ret = PTR_ERR(mbox->out_queue); + goto unmap_in_queue; + } + + mbox->wq = create_singlethread_workqueue("ethosu_workqueue"); + if (!mbox->wq) { + dev_err(mbox->dev, "Failed to create work queue\n"); + ret = -EINVAL; + goto unmap_out_queue; + } + + INIT_WORK(&mbox->work, ethosu_mailbox_rx_work); + + mbox->tx = mbox_request_channel_byname(&mbox->client, "tx"); + if (IS_ERR(mbox->tx)) { + dev_warn(mbox->dev, "mbox: Failed to request tx channel\n"); + ret = PTR_ERR(mbox->tx); + goto workqueue_destroy; + } + + mbox->rx = mbox_request_channel_byname(&mbox->client, "rx"); + if (IS_ERR(mbox->rx)) { + dev_info(dev, "mbox: Using same channel for RX and TX\n"); + mbox->rx = mbox->tx; + } + + return 0; + +workqueue_destroy: + destroy_workqueue(mbox->wq); + +unmap_out_queue: + devm_iounmap(mbox->dev, mbox->out_queue); + +unmap_in_queue: + devm_iounmap(mbox->dev, mbox->in_queue); + + return ret; +} + +void ethosu_mailbox_deinit(struct ethosu_mailbox *mbox) +{ + if (mbox->rx != mbox->tx) + mbox_free_channel(mbox->rx); + + mbox_free_channel(mbox->tx); + destroy_workqueue(mbox->wq); + devm_iounmap(mbox->dev, mbox->out_queue); + devm_iounmap(mbox->dev, mbox->in_queue); +} diff --git a/drivers/firmware/ethosu/ethosu_mailbox.h b/drivers/firmware/ethosu/ethosu_mailbox.h new file mode 100644 index 000000000000..5cd5e62198b8 --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_mailbox.h @@ -0,0 +1,140 @@ +/* + * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * + * This program is free software and is provided to you under the terms of the + * GNU General Public License version 2 as published by the Free Software + * Foundation, and any use by you of this program is subject to the terms + * of such GNU licence. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, you can access it online at + * http://www.gnu.org/licenses/gpl-2.0.html. + * + * SPDX-License-Identifier: GPL-2.0-only + */ + +#ifndef ETHOSU_MAILBOX_H +#define ETHOSU_MAILBOX_H + +/**************************************************************************** + * Includes + ****************************************************************************/ +#include "ethosu_core_interface.h" + +#include +#include +#include + +/**************************************************************************** + * Types + ****************************************************************************/ + +struct device; +struct ethosu_buffer; +struct ethosu_device; +struct ethosu_core_msg; +struct ethosu_core_queue; +struct resource; + +typedef void (*ethosu_mailbox_cb)(void *user_arg); + +struct ethosu_mailbox { + struct device *dev; + struct workqueue_struct *wq; + struct work_struct work; + struct ethosu_core_queue __iomem *in_queue; + struct ethosu_core_queue __iomem *out_queue; + struct mbox_client client; + struct mbox_chan *rx; + struct mbox_chan *tx; + ethosu_mailbox_cb callback; + void *user_arg; +}; + +/**************************************************************************** + * Functions + ****************************************************************************/ + +/** + * ethosu_mailbox_init() - Initialize mailbox + * + * Return: 0 on success, else error code. + */ +int ethosu_mailbox_init(struct ethosu_mailbox *mbox, + struct device *dev, + struct resource *in_queue, + struct resource *out_queue, + ethosu_mailbox_cb callback, + void *user_arg); + +/** + * ethosu_mailbox_deinit() - Deinitialize mailbox + */ +void ethosu_mailbox_deinit(struct ethosu_mailbox *mbox); + +/** + * ethosu_mailbox_read() - Read message from mailbox + * + * Return: 0 message read, else error code. + */ +int ethosu_mailbox_read(struct ethosu_mailbox *mbox, + struct ethosu_core_msg *header, + void *data, + size_t length); + +/** + * ethosu_mailbox_reset() - Reset to end of queue + */ +void ethosu_mailbox_reset(struct ethosu_mailbox *mbox); + +/** + * ethosu_mailbox_ping() - Send ping message + * + * Return: 0 on success, else error code. + */ +int ethosu_mailbox_ping(struct ethosu_mailbox *mbox); + +/** + * ethosu_mailbox_pong() - Send pong response + * + * Return: 0 on success, else error code. + */ +int ethosu_mailbox_pong(struct ethosu_mailbox *mbox); + +/** + * ethosu_mailbox_version_response - Send version request + * + * Return: 0 on succes, else error code + */ +int ethosu_mailbox_version_request(struct ethosu_mailbox *mbox); + +/** + * ethosu_mailbox_capabilities_request() - Send capabilities request + * + * Return: 0 on success, else error code. + */ +int ethosu_mailbox_capabilities_request(struct ethosu_mailbox *mbox, + void *user_arg); + +/** + * ethosu_mailbox_inference() - Send inference + * + * Return: 0 on success, else error code. + */ +int ethosu_mailbox_inference(struct ethosu_mailbox *mbox, + void *user_arg, + uint32_t ifm_count, + struct ethosu_buffer **ifm, + uint32_t ofm_count, + struct ethosu_buffer **ofm, + struct ethosu_buffer *network, + uint8_t *pmu_event_config, + uint8_t pmu_event_config_count, + uint8_t pmu_cycle_counter_enable); + +#endif /* ETHOSU_MAILBOX_H */ diff --git a/drivers/firmware/ethosu/ethosu_network.c b/drivers/firmware/ethosu/ethosu_network.c new file mode 100644 index 000000000000..4d68f0537e38 --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_network.c @@ -0,0 +1,201 @@ +/* + * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * + * This program is free software and is provided to you under the terms of the + * GNU General Public License version 2 as published by the Free Software + * Foundation, and any use by you of this program is subject to the terms + * of such GNU licence. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, you can access it online at + * http://www.gnu.org/licenses/gpl-2.0.html. + * + * SPDX-License-Identifier: GPL-2.0-only + */ + +/**************************************************************************** + * Includes + ****************************************************************************/ + +#include "ethosu_network.h" + +#include "ethosu_buffer.h" +#include "ethosu_device.h" +#include "ethosu_inference.h" +#include "uapi/ethosu.h" + +#include +#include +#include +#include + +/**************************************************************************** + * Variables + ****************************************************************************/ + +static int ethosu_network_release(struct inode *inode, + struct file *file); + +static long ethosu_network_ioctl(struct file *file, + unsigned int cmd, + unsigned long arg); + +static const struct file_operations ethosu_network_fops = { + .release = ðosu_network_release, + .unlocked_ioctl = ðosu_network_ioctl, +#ifdef CONFIG_COMPAT + .compat_ioctl = ðosu_network_ioctl, +#endif +}; + +/**************************************************************************** + * Functions + ****************************************************************************/ + +static bool ethosu_network_verify(struct file *file) +{ + return file->f_op == ðosu_network_fops; +} + +static void ethosu_network_destroy(struct kref *kref) +{ + struct ethosu_network *net = + container_of(kref, struct ethosu_network, kref); + + dev_info(net->edev->dev, "Network destroy. handle=0x%pK\n", net); + + ethosu_buffer_put(net->buf); + devm_kfree(net->edev->dev, net); +} + +static int ethosu_network_release(struct inode *inode, + struct file *file) +{ + struct ethosu_network *net = file->private_data; + + dev_info(net->edev->dev, "Network release. handle=0x%pK\n", net); + + ethosu_network_put(net); + + return 0; +} + +static long ethosu_network_ioctl(struct file *file, + unsigned int cmd, + unsigned long arg) +{ + struct ethosu_network *net = file->private_data; + void __user *udata = (void __user *)arg; + int ret = -EINVAL; + + ret = mutex_lock_interruptible(&net->edev->mutex); + if (ret) + return ret; + + dev_info(net->edev->dev, "Ioctl: cmd=%u, arg=%lu\n", cmd, arg); + + switch (cmd) { + case ETHOSU_IOCTL_INFERENCE_CREATE: { + struct ethosu_uapi_inference_create uapi; + + if (copy_from_user(&uapi, udata, sizeof(uapi))) + break; + + dev_info(net->edev->dev, + "Ioctl: Inference. ifm_fd=%u, ofm_fd=%u\n", + uapi.ifm_fd[0], uapi.ofm_fd[0]); + + ret = ethosu_inference_create(net->edev, net, &uapi); + break; + } + default: { + dev_err(net->edev->dev, "Invalid ioctl. cmd=%u, arg=%lu", + cmd, arg); + break; + } + } + + mutex_unlock(&net->edev->mutex); + + return ret; +} + +int ethosu_network_create(struct ethosu_device *edev, + struct ethosu_uapi_network_create *uapi) +{ + struct ethosu_buffer *buf; + struct ethosu_network *net; + int ret = -ENOMEM; + + buf = ethosu_buffer_get_from_fd(uapi->fd); + if (IS_ERR(buf)) + return PTR_ERR(buf); + + net = devm_kzalloc(edev->dev, sizeof(*net), GFP_KERNEL); + if (!net) { + ret = -ENOMEM; + goto put_buf; + } + + net->edev = edev; + net->buf = buf; + kref_init(&net->kref); + + ret = anon_inode_getfd("ethosu-network", ðosu_network_fops, net, + O_RDWR | O_CLOEXEC); + if (ret < 0) + goto free_net; + + net->file = fget(ret); + fput(net->file); + + dev_info(edev->dev, "Network create. handle=0x%pK", + net); + + return ret; + +free_net: + devm_kfree(edev->dev, net); + +put_buf: + ethosu_buffer_put(buf); + + return ret; +} + +struct ethosu_network *ethosu_network_get_from_fd(int fd) +{ + struct ethosu_network *net; + struct file *file; + + file = fget(fd); + if (!file) + return ERR_PTR(-EINVAL); + + if (!ethosu_network_verify(file)) { + fput(file); + + return ERR_PTR(-EINVAL); + } + + net = file->private_data; + ethosu_network_get(net); + fput(file); + + return net; +} + +void ethosu_network_get(struct ethosu_network *net) +{ + kref_get(&net->kref); +} + +void ethosu_network_put(struct ethosu_network *net) +{ + kref_put(&net->kref, ethosu_network_destroy); +} diff --git a/drivers/firmware/ethosu/ethosu_network.h b/drivers/firmware/ethosu/ethosu_network.h new file mode 100644 index 000000000000..bb70afcb2572 --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_network.h @@ -0,0 +1,81 @@ +/* + * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * + * This program is free software and is provided to you under the terms of the + * GNU General Public License version 2 as published by the Free Software + * Foundation, and any use by you of this program is subject to the terms + * of such GNU licence. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, you can access it online at + * http://www.gnu.org/licenses/gpl-2.0.html. + * + * SPDX-License-Identifier: GPL-2.0-only + */ + +#ifndef ETHOSU_NETWORK_H +#define ETHOSU_NETWORK_H + +/**************************************************************************** + * Includes + ****************************************************************************/ + +#include +#include + +/**************************************************************************** + * Types + ****************************************************************************/ + +struct ethosu_buffer; +struct ethosu_device; +struct ethosu_uapi_network_create; +struct device; +struct file; + +struct ethosu_network { + struct ethosu_device *edev; + struct file *file; + struct kref kref; + struct ethosu_buffer *buf; +}; + +/**************************************************************************** + * Functions + ****************************************************************************/ + +/** + * ethosu_network_create() - Create network + * + * This function must be called in the context of a user space process. + * + * Return: fd on success, else error code. + */ +int ethosu_network_create(struct ethosu_device *edev, + struct ethosu_uapi_network_create *uapi); + +/** + * ethosu_network_get_from_fd() - Get network handle from fd + * + * This function must be called from a user space context. + * + * Return: Pointer on success, else ERR_PTR. + */ +struct ethosu_network *ethosu_network_get_from_fd(int fd); + +/** + * ethosu_network_get() - Get network + */ +void ethosu_network_get(struct ethosu_network *net); + +/** + * ethosu_network_put() - Put network + */ +void ethosu_network_put(struct ethosu_network *net); + +#endif /* ETHOSU_NETWORK_H */ diff --git a/drivers/firmware/ethosu/uapi/ethosu.h b/drivers/firmware/ethosu/uapi/ethosu.h new file mode 100644 index 000000000000..903316dff7b3 --- /dev/null +++ b/drivers/firmware/ethosu/uapi/ethosu.h @@ -0,0 +1,207 @@ +/* + * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * + * This program is free software and is provided to you under the terms of the + * GNU General Public License version 2 as published by the Free Software + * Foundation, and any use by you of this program is subject to the terms + * of such GNU licence. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, you can access it online at + * http://www.gnu.org/licenses/gpl-2.0.html. + * + * SPDX-License-Identifier: GPL-2.0-only + */ + +#ifndef ETHOSU_H +#define ETHOSU_H + +/**************************************************************************** + * Includes + ****************************************************************************/ + +#include +#include + +#ifdef __cplusplus +namespace EthosU { +#endif + +/**************************************************************************** + * Defines + ****************************************************************************/ + +#define ETHOSU_IOCTL_BASE 0x01 +#define ETHOSU_IO(nr) _IO(ETHOSU_IOCTL_BASE, nr) +#define ETHOSU_IOR(nr, type) _IOR(ETHOSU_IOCTL_BASE, nr, type) +#define ETHOSU_IOW(nr, type) _IOW(ETHOSU_IOCTL_BASE, nr, type) +#define ETHOSU_IOWR(nr, type) _IOWR(ETHOSU_IOCTL_BASE, nr, type) + +#define ETHOSU_IOCTL_PING ETHOSU_IO(0x00) +#define ETHOSU_IOCTL_VERSION_REQ ETHOSU_IO(0x01) +#define ETHOSU_IOCTL_CAPABILITIES_REQ ETHOSU_IO(0x02) +#define ETHOSU_IOCTL_BUFFER_CREATE ETHOSU_IOR(0x10, \ + struct ethosu_uapi_buffer_create) +#define ETHOSU_IOCTL_BUFFER_SET ETHOSU_IOR(0x11, \ + struct ethosu_uapi_buffer) +#define ETHOSU_IOCTL_BUFFER_GET ETHOSU_IOW(0x12, \ + struct ethosu_uapi_buffer) +#define ETHOSU_IOCTL_NETWORK_CREATE ETHOSU_IOR(0x20, \ + struct ethosu_uapi_network_create) +#define ETHOSU_IOCTL_INFERENCE_CREATE ETHOSU_IOR(0x30, \ + struct ethosu_uapi_inference_create) +#define ETHOSU_IOCTL_INFERENCE_STATUS ETHOSU_IOR(0x31, \ + struct ethosu_uapi_result_status) + +/* Maximum number of IFM/OFM file descriptors per network */ +#define ETHOSU_FD_MAX 16 + +/* Maximum number of PMUs available */ +#define ETHOSU_PMU_EVENT_MAX 8 + +/**************************************************************************** + * Types + ****************************************************************************/ + +/** + * enum ethosu_uapi_status - Status + */ +enum ethosu_uapi_status { + ETHOSU_UAPI_STATUS_OK, + ETHOSU_UAPI_STATUS_ERROR +}; + +/** + * struct ethosu_uapi_buffer_create - Create buffer request + * @capacity: Maximum capacity of the buffer + */ +struct ethosu_uapi_buffer_create { + __u32 capacity; +}; + +/** + * struct ethosu_uapi_buffer - Buffer descriptor + * @offset: Offset to where the data starts + * @size: Size of the data + * + * 'offset + size' must not exceed the capacity of the buffer. + */ +struct ethosu_uapi_buffer { + __u32 offset; + __u32 size; +}; + +/** + * struct ethosu_uapi_network_create - Create network request + * @fd: Buffer file descriptor + */ +struct ethosu_uapi_network_create { + __u32 fd; +}; + +/** + * struct ethosu_uapi_pmu_config - Configure performance counters + * @events: Array of counters to configure, set to non-zero for + * each counter to enable corresponding event. + * @cycle_count: Set to enable the cycle counter. + */ +struct ethosu_uapi_pmu_config { + __u32 events[ETHOSU_PMU_EVENT_MAX]; + __u32 cycle_count; +}; + +/** + * struct ethosu_uapi_pmu_counts - Status of performance counters + * @events: Count for respective configured events. + * @cycle_count: Count for cycle counter. + */ +struct ethosu_uapi_pmu_counts { + __u32 events[ETHOSU_PMU_EVENT_MAX]; + __u64 cycle_count; +}; + +/** + * struct ethosu_uapi_device_hw_id - Device hardware identification + * @version_status: Version status + * @version_minor: Version minor + * @version_major: Version major + * @product_major: Product major + * @arch_patch_rev: Architecture version patch + * @arch_minor_rev: Architecture version minor + * @arch_major_rev: Architecture version major + */ +struct ethosu_uapi_device_hw_id { + __u32 version_status; + __u32 version_minor; + __u32 version_major; + __u32 product_major; + __u32 arch_patch_rev; + __u32 arch_minor_rev; + __u32 arch_major_rev; +}; + +/** + * struct ethosu_uapi_device_hw_cfg - Device hardware configuration + * @macs_per_cc: MACs per clock cycle + * @cmd_stream_version: NPU command stream version + * @custom_dma: Custom DMA enabled + */ +struct ethosu_uapi_device_hw_cfg { + __u32 macs_per_cc; + __u32 cmd_stream_version; + __u32 custom_dma; +}; + +/** + * struct ethosu_uapi_capabilities - Device capabilities + * @hw_id: Hardware identification + * @hw_cfg: Hardware configuration + * @driver_patch_rev: Driver version patch + * @driver_minor_rev: Driver version minor + * @driver_major_rev: Driver version major + */ +struct ethosu_uapi_device_capabilities { + struct ethosu_uapi_device_hw_id hw_id; + struct ethosu_uapi_device_hw_cfg hw_cfg; + __u32 driver_patch_rev; + __u32 driver_minor_rev; + __u32 driver_major_rev; +}; + +/** + * struct ethosu_uapi_inference_create - Create network request + * @ifm_count: Number of IFM file descriptors + * @ifm_fd: IFM buffer file descriptors + * @ofm_count: Number of OFM file descriptors + * @ofm_fd: OFM buffer file descriptors + */ +struct ethosu_uapi_inference_create { + __u32 ifm_count; + __u32 ifm_fd[ETHOSU_FD_MAX]; + __u32 ofm_count; + __u32 ofm_fd[ETHOSU_FD_MAX]; + struct ethosu_uapi_pmu_config pmu_config; +}; + +/** + * struct ethosu_uapi_result_status - Status of inference + * @status Status of run inference. + * @pmu_config Configured performance counters. + * @pmu_count Perfomance counters values, when status is + * ETHOSU_UAPI_STATUS_OK. + */ +struct ethosu_uapi_result_status { + enum ethosu_uapi_status status; + struct ethosu_uapi_pmu_config pmu_config; + struct ethosu_uapi_pmu_counts pmu_count; +}; + +#ifdef __cplusplus +} /* namespace EthosU */ +#endif +#endif From patchwork Fri Jun 16 05:59:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Wang X-Patchwork-Id: 13282005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E4297EB64D7 for ; Fri, 16 Jun 2023 05:43:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=lry0IRavHn6u2N5Yd6O3puNyj2QklYg+5fg/k24cal0=; b=YKtggYw6IqBF6Q Ggknwv+fJ398AwJNjBciKHfMR2l40mnbqHwwvakJB1xEPLOTqREIVGuh1+Bw79Mflgk9XXecih/BF OSIpd6SKKtSbbuu5naSMY7MWAQFps5PmbDZY5yN+HeAVL6nTAwHJw9Lxk6stnQKuGl/nzrw+oldof O2HQGQ/JIC61JbwcRyFOXwVac8Hsqf7q153IzUfNMNyaRZdaO/Nors+OnW6TvkompOnzxPaJ+OSkv Cu/vUvS+M1sL8QAgvF671wR6mPe0CXktzLD7pqMHPfFPE8WIc8MZInLzaLivxnlPS7RpjfRj/r0bk LQbX6Sz7IMtHBTsBncRw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qA2F2-00Gusb-0X; Fri, 16 Jun 2023 05:43:24 +0000 Received: from inva020.nxp.com ([92.121.34.13]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qA2En-00Guln-1N for linux-arm-kernel@lists.infradead.org; Fri, 16 Jun 2023 05:43:16 +0000 Received: from inva020.nxp.com (localhost [127.0.0.1]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 2CBBC1A249A; Fri, 16 Jun 2023 07:43:04 +0200 (CEST) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 928521A0696; Fri, 16 Jun 2023 07:43:03 +0200 (CEST) Received: from localhost.localdomain (mega.ap.freescale.net [10.192.208.232]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id 397E91820F57; Fri, 16 Jun 2023 13:43:02 +0800 (+08) From: Alison Wang To: gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: leoyang.li@nxp.com, xuelin.shi@nxp.com, xiaofeng.ren@nxp.com, feng.guo@nxp.com, Alison Wang Subject: [PATCH 2/8] ethosu: Use RPMsg messaging protocol based on i.MX Rpmsg implementation Date: Fri, 16 Jun 2023 13:59:07 +0800 Message-Id: <20230616055913.2360-3-alison.wang@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230616055913.2360-1-alison.wang@nxp.com> References: <20230616055913.2360-1-alison.wang@nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230615_224309_892381_94986445 X-CRM114-Status: GOOD ( 24.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org RPMsg is used for the communication mechanism between Cortex-A and Cortex-M for Ethos-U driver. Signed-off-by: Lei Xu Signed-off-by: Alison Wang Reviewed-by: Peng Fan --- drivers/firmware/ethosu/Makefile | 2 +- drivers/firmware/ethosu/ethosu_device.c | 143 ++++---- drivers/firmware/ethosu/ethosu_device.h | 8 +- drivers/firmware/ethosu/ethosu_driver.c | 23 +- drivers/firmware/ethosu/ethosu_inference.c | 2 +- drivers/firmware/ethosu/ethosu_mailbox.c | 377 --------------------- drivers/firmware/ethosu/ethosu_mailbox.h | 140 -------- drivers/firmware/ethosu/ethosu_rpmsg.c | 235 +++++++++++++ drivers/firmware/ethosu/ethosu_rpmsg.h | 73 ++++ 9 files changed, 383 insertions(+), 620 deletions(-) delete mode 100644 drivers/firmware/ethosu/ethosu_mailbox.c delete mode 100644 drivers/firmware/ethosu/ethosu_mailbox.h create mode 100644 drivers/firmware/ethosu/ethosu_rpmsg.c create mode 100644 drivers/firmware/ethosu/ethosu_rpmsg.h diff --git a/drivers/firmware/ethosu/Makefile b/drivers/firmware/ethosu/Makefile index 933efee1b22f..0b3fe72c9c4f 100644 --- a/drivers/firmware/ethosu/Makefile +++ b/drivers/firmware/ethosu/Makefile @@ -24,5 +24,5 @@ ethosu-objs := ethosu_driver.o \ ethosu_buffer.o \ ethosu_device.o \ ethosu_inference.o \ - ethosu_mailbox.o \ + ethosu_rpmsg.o \ ethosu_network.o diff --git a/drivers/firmware/ethosu/ethosu_device.c b/drivers/firmware/ethosu/ethosu_device.c index e6f1e8012b06..b2d4737080ac 100644 --- a/drivers/firmware/ethosu/ethosu_device.c +++ b/drivers/firmware/ethosu/ethosu_device.c @@ -1,5 +1,6 @@ /* * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * Copyright 2020-2022 NXP * * This program is free software and is provided to you under the terms of the * GNU General Public License version 2 as published by the Free Software @@ -121,76 +122,76 @@ static int ethosu_capability_rsp(struct ethosu_device *edev, } /* Incoming messages */ -static int ethosu_handle_msg(struct ethosu_device *edev) +static int ethosu_handle_msg(struct ethosu_device *edev, void *data) { - int ret; - struct ethosu_core_msg header; - - union { - struct ethosu_core_msg_err error; - struct ethosu_core_inference_rsp inf; - struct ethosu_core_msg_version version; - struct ethosu_core_msg_capabilities_rsp capabilities; - } data; - - /* Read message */ - ret = ethosu_mailbox_read(&edev->mailbox, &header, &data, sizeof(data)); - if (ret) - return ret; - - switch (header.type) { + int ret = 0; + struct ethosu_core_msg *header = (struct ethosu_core_msg *)data; + struct ethosu_core_msg_err *error = + (struct ethosu_core_msg_err *) + ((char *)data + sizeof(struct ethosu_core_msg)); + struct ethosu_core_inference_rsp *rsp = + (struct ethosu_core_inference_rsp *) + ((char *)data + sizeof(struct ethosu_core_msg)); + struct ethosu_core_msg_version *version = + (struct ethosu_core_msg_version *) + ((char *)data + sizeof(struct ethosu_core_msg)); + struct ethosu_core_msg_capabilities_rsp *capabilities = + (struct ethosu_core_msg_capabilities_rsp *) + ((char *)data + sizeof(struct ethosu_core_msg)); + + switch (header->type) { case ETHOSU_CORE_MSG_ERR: - if (header.length != sizeof(data.error)) { + if (header->length != sizeof(struct ethosu_core_msg_err)) { dev_warn(edev->dev, - "Msg: Error message of incorrect size. size=%u, expected=%zu\n", header.length, - sizeof(data.error)); + "Msg: Error message of incorrect size. size=%u, expected=%zu\n", header->length, + sizeof(struct ethosu_core_msg_err)); ret = -EBADMSG; break; } - data.error.msg[sizeof(data.error.msg) - 1] = '\0'; + error->msg[sizeof(error->msg) - 1] = '\0'; dev_warn(edev->dev, "Msg: Error. type=%u, msg=\"%s\"\n", - data.error.type, data.error.msg); + error->type, error->msg); ret = -EBADMSG; break; case ETHOSU_CORE_MSG_PING: dev_info(edev->dev, "Msg: Ping\n"); - ret = ethosu_mailbox_pong(&edev->mailbox); + ret = ethosu_rpmsg_pong(&edev->erp); break; case ETHOSU_CORE_MSG_PONG: dev_info(edev->dev, "Msg: Pong\n"); break; case ETHOSU_CORE_MSG_INFERENCE_RSP: - if (header.length != sizeof(data.inf)) { + if (header->length != sizeof(struct ethosu_core_inference_rsp)) { dev_warn(edev->dev, - "Msg: Inference response of incorrect size. size=%u, expected=%zu\n", header.length, - sizeof(data.inf)); + "Msg: Inference response of incorrect size. size=%u, expected=%zu\n", header->length, + sizeof(struct ethosu_core_inference_rsp)); ret = -EBADMSG; break; } dev_info(edev->dev, "Msg: Inference response. user_arg=0x%llx, ofm_count=%u, status=%u\n", - data.inf.user_arg, data.inf.ofm_count, - data.inf.status); - ethosu_inference_rsp(edev, &data.inf); + rsp->user_arg, rsp->ofm_count, + rsp->status); + ethosu_inference_rsp(edev, rsp); break; case ETHOSU_CORE_MSG_VERSION_RSP: - if (header.length != sizeof(data.version)) { + if (header->length != sizeof(struct ethosu_core_msg_version)) { dev_warn(edev->dev, - "Msg: Version response of incorrect size. size=%u, expected=%zu\n", header.length, - sizeof(data.version)); + "Msg: Version response of incorrect size. size=%u, expected=%zu\n", header->length, + sizeof(struct ethosu_core_msg_version)); ret = -EBADMSG; break; } dev_info(edev->dev, "Msg: Version response v%u.%u.%u\n", - data.version.major, data.version.minor, - data.version.patch); + version->major, version->minor, + version->patch); /* Check major and minor version match, else return error */ - if (data.version.major != ETHOSU_CORE_MSG_VERSION_MAJOR || - data.version.minor != ETHOSU_CORE_MSG_VERSION_MINOR) { + if (version->major != ETHOSU_CORE_MSG_VERSION_MAJOR || + version->minor != ETHOSU_CORE_MSG_VERSION_MINOR) { dev_warn(edev->dev, "Msg: Version mismatch detected! "); dev_warn(edev->dev, "Local version: v%u.%u.%u\n", ETHOSU_CORE_MSG_VERSION_MAJOR, @@ -200,32 +201,32 @@ static int ethosu_handle_msg(struct ethosu_device *edev) break; case ETHOSU_CORE_MSG_CAPABILITIES_RSP: - if (header.length != sizeof(data.capabilities)) { + if (header->length != sizeof(struct ethosu_core_msg_capabilities_rsp)) { dev_warn(edev->dev, - "Msg: Capabilities response of incorrect size. size=%u, expected=%zu\n", header.length, - sizeof(data.capabilities)); + "Msg: Capabilities response of incorrect size. size=%u, expected=%zu\n", header->length, + sizeof(struct ethosu_core_msg_capabilities_rsp)); ret = -EBADMSG; break; } dev_info(edev->dev, "Msg: Capabilities response ua%llx vs%hhu v%hhu.%hhu p%hhu av%hhu.%hhu.%hhu dv%hhu.%hhu.%hhu mcc%hhu csv%hhu cd%hhu\n", - data.capabilities.user_arg, - data.capabilities.version_status, - data.capabilities.version_major, - data.capabilities.version_minor, - data.capabilities.product_major, - data.capabilities.arch_major_rev, - data.capabilities.arch_minor_rev, - data.capabilities.arch_patch_rev, - data.capabilities.driver_major_rev, - data.capabilities.driver_minor_rev, - data.capabilities.driver_patch_rev, - data.capabilities.macs_per_cc, - data.capabilities.cmd_stream_version, - data.capabilities.custom_dma); - - ret = ethosu_capability_rsp(edev, &data.capabilities); + capabilities->user_arg, + capabilities->version_status, + capabilities->version_major, + capabilities->version_minor, + capabilities->product_major, + capabilities->arch_major_rev, + capabilities->arch_minor_rev, + capabilities->arch_patch_rev, + capabilities->driver_major_rev, + capabilities->driver_minor_rev, + capabilities->driver_patch_rev, + capabilities->macs_per_cc, + capabilities->cmd_stream_version, + capabilities->custom_dma); + + ret = ethosu_capability_rsp(edev, capabilities); break; default: /* This should not happen due to version checks */ @@ -269,7 +270,7 @@ static int ethosu_send_capabilities_request(struct ethosu_device *edev, init_completion(&cap->done); list_add(&cap->list, &edev->capabilities_list); - ret = ethosu_mailbox_capabilities_request(&edev->mailbox, cap); + ret = ethosu_rpmsg_capabilities_request(&edev->erp, cap); if (0 != ret) goto put_kref; @@ -325,7 +326,7 @@ static long ethosu_ioctl(struct file *file, switch (cmd) { case ETHOSU_IOCTL_VERSION_REQ: dev_info(edev->dev, "Ioctl: Send version request\n"); - ret = ethosu_mailbox_version_request(&edev->mailbox); + ret = ethosu_rpmsg_version_request(&edev->erp); break; case ETHOSU_IOCTL_CAPABILITIES_REQ: dev_info(edev->dev, "Ioctl: Send capabilities request\n"); @@ -333,7 +334,7 @@ static long ethosu_ioctl(struct file *file, break; case ETHOSU_IOCTL_PING: { dev_info(edev->dev, "Ioctl: Send ping\n"); - ret = ethosu_mailbox_ping(&edev->mailbox); + ret = ethosu_rpmsg_ping(&edev->erp); break; } case ETHOSU_IOCTL_BUFFER_CREATE: { @@ -373,7 +374,7 @@ static long ethosu_ioctl(struct file *file, return ret; } -static void ethosu_mbox_rx(void *user_arg) +static void ethosu_rpmsg_rx(void *user_arg, void *data) { struct ethosu_device *edev = user_arg; int ret; @@ -381,13 +382,8 @@ static void ethosu_mbox_rx(void *user_arg) mutex_lock(&edev->mutex); do { - ret = ethosu_handle_msg(edev); - if (ret && ret != -ENOMSG) - /* Need to start over in case of error, empty the queue - * by fast-forwarding read position to write position. - * */ - ethosu_mailbox_reset(&edev->mailbox); - } while (ret == 0); + ret = ethosu_handle_msg(edev, data); + } while (ret != 0); mutex_unlock(&edev->mutex); } @@ -395,9 +391,7 @@ static void ethosu_mbox_rx(void *user_arg) int ethosu_dev_init(struct ethosu_device *edev, struct device *dev, struct class *class, - dev_t devt, - struct resource *in_queue, - struct resource *out_queue) + dev_t devt) { static const struct file_operations fops = { .owner = THIS_MODULE, @@ -423,8 +417,7 @@ int ethosu_dev_init(struct ethosu_device *edev, dma_set_mask_and_coherent(edev->dev, DMA_BIT_MASK(DMA_ADDR_BITS)); - ret = ethosu_mailbox_init(&edev->mailbox, dev, in_queue, out_queue, - ethosu_mbox_rx, edev); + ret = ethosu_rpmsg_init(&edev->erp, ethosu_rpmsg_rx, edev); if (ret) goto release_reserved_mem; @@ -434,7 +427,7 @@ int ethosu_dev_init(struct ethosu_device *edev, ret = cdev_add(&edev->cdev, edev->devt, 1); if (ret) { dev_err(edev->dev, "Failed to add character device.\n"); - goto deinit_mailbox; + goto deinit_rpmsg; } sysdev = device_create(edev->class, NULL, edev->devt, edev, @@ -454,8 +447,8 @@ int ethosu_dev_init(struct ethosu_device *edev, del_cdev: cdev_del(&edev->cdev); -deinit_mailbox: - ethosu_mailbox_deinit(&edev->mailbox); +deinit_rpmsg: + ethosu_rpmsg_deinit(&edev->erp); release_reserved_mem: of_reserved_mem_device_release(edev->dev); @@ -465,7 +458,7 @@ int ethosu_dev_init(struct ethosu_device *edev, void ethosu_dev_deinit(struct ethosu_device *edev) { - ethosu_mailbox_deinit(&edev->mailbox); + ethosu_rpmsg_deinit(&edev->erp); device_destroy(edev->class, edev->cdev.dev); cdev_del(&edev->cdev); of_reserved_mem_device_release(edev->dev); diff --git a/drivers/firmware/ethosu/ethosu_device.h b/drivers/firmware/ethosu/ethosu_device.h index 3afdda84862d..3943c8df8f28 100644 --- a/drivers/firmware/ethosu/ethosu_device.h +++ b/drivers/firmware/ethosu/ethosu_device.h @@ -26,7 +26,7 @@ ****************************************************************************/ #include "uapi/ethosu.h" -#include "ethosu_mailbox.h" +#include "ethosu_rpmsg.h" #include #include @@ -48,7 +48,7 @@ struct ethosu_device { struct class *class; dev_t devt; struct mutex mutex; - struct ethosu_mailbox mailbox; + struct ethosu_rpmsg erp; struct list_head capabilities_list; struct list_head inference_list; }; @@ -76,9 +76,7 @@ struct ethosu_capabilities { int ethosu_dev_init(struct ethosu_device *edev, struct device *dev, struct class *class, - dev_t devt, - struct resource *in_queue, - struct resource *out_queue); + dev_t devt); /** * ethosu_dev_deinit() - Initialize the device diff --git a/drivers/firmware/ethosu/ethosu_driver.c b/drivers/firmware/ethosu/ethosu_driver.c index 9d02431d3194..27931e9aa97c 100644 --- a/drivers/firmware/ethosu/ethosu_driver.c +++ b/drivers/firmware/ethosu/ethosu_driver.c @@ -1,5 +1,6 @@ /* * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * Copyright 2020-2022 NXP * * This program is free software and is provided to you under the terms of the * GNU General Public License version 2 as published by the Free Software @@ -55,8 +56,6 @@ static DECLARE_BITMAP(minors, MINOR_COUNT); static int ethosu_pdev_probe(struct platform_device *pdev) { struct ethosu_device *edev; - struct resource *in_queue_res; - struct resource *out_queue_res; int minor; int ret; @@ -69,23 +68,6 @@ static int ethosu_pdev_probe(struct platform_device *pdev) return -ENOMEM; } - /* Get path to TCM memory */ - in_queue_res = platform_get_resource_byname(pdev, IORESOURCE_MEM, - "in_queue"); - if (IS_ERR(in_queue_res)) { - dev_err(&pdev->dev, "Failed to get in_queue resource.\n"); - - return PTR_ERR(in_queue_res); - } - - out_queue_res = platform_get_resource_byname(pdev, IORESOURCE_MEM, - "out_queue"); - if (IS_ERR(out_queue_res)) { - dev_err(&pdev->dev, "Failed to get out_queue resource.\n"); - - return PTR_ERR(out_queue_res); - } - /* Allocate memory for Arm Ethos-U device */ edev = devm_kzalloc(&pdev->dev, sizeof(*edev), GFP_KERNEL); if (!edev) @@ -95,8 +77,7 @@ static int ethosu_pdev_probe(struct platform_device *pdev) /* Initialize device */ ret = ethosu_dev_init(edev, &pdev->dev, ethosu_class, - MKDEV(MAJOR(devt), minor), in_queue_res, - out_queue_res); + MKDEV(MAJOR(devt), minor)); if (ret) goto free_dev; diff --git a/drivers/firmware/ethosu/ethosu_inference.c b/drivers/firmware/ethosu/ethosu_inference.c index 6fde92c148a0..853947a2334e 100644 --- a/drivers/firmware/ethosu/ethosu_inference.c +++ b/drivers/firmware/ethosu/ethosu_inference.c @@ -86,7 +86,7 @@ static int ethosu_inference_send(struct ethosu_inference *inf) inf->status = ETHOSU_UAPI_STATUS_ERROR; - ret = ethosu_mailbox_inference(&inf->edev->mailbox, inf, + ret = ethosu_rpmsg_inference(&inf->edev->erp, inf, inf->ifm_count, inf->ifm, inf->ofm_count, inf->ofm, inf->net->buf, diff --git a/drivers/firmware/ethosu/ethosu_mailbox.c b/drivers/firmware/ethosu/ethosu_mailbox.c deleted file mode 100644 index 7f159f3b0a60..000000000000 --- a/drivers/firmware/ethosu/ethosu_mailbox.c +++ /dev/null @@ -1,377 +0,0 @@ -/* - * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. - * - * This program is free software and is provided to you under the terms of the - * GNU General Public License version 2 as published by the Free Software - * Foundation, and any use by you of this program is subject to the terms - * of such GNU licence. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program; if not, you can access it online at - * http://www.gnu.org/licenses/gpl-2.0.html. - * - * SPDX-License-Identifier: GPL-2.0-only - */ - -/**************************************************************************** - * Includes - ****************************************************************************/ - -#include "ethosu_mailbox.h" - -#include "ethosu_buffer.h" -#include "ethosu_core_interface.h" -#include "ethosu_device.h" - -#include -#include - -/**************************************************************************** - * Functions - ****************************************************************************/ - -static void ethosu_core_set_size(struct ethosu_buffer *buf, - struct ethosu_core_buffer *cbuf) -{ - cbuf->ptr = (uint32_t)buf->dma_addr + buf->offset; - cbuf->size = (uint32_t)buf->size; -} - -static void ethosu_core_set_capacity(struct ethosu_buffer *buf, - struct ethosu_core_buffer *cbuf) -{ - cbuf->ptr = (uint32_t)buf->dma_addr + buf->offset + buf->size; - cbuf->size = (uint32_t)buf->capacity - buf->offset - buf->size; -} - -static size_t ethosu_queue_available(struct ethosu_core_queue *queue) -{ - size_t size = queue->header.write - queue->header.read; - - if (queue->header.read > queue->header.write) - size += queue->header.size; - - return size; -} - -static size_t ethosu_queue_capacity(struct ethosu_core_queue *queue) -{ - return queue->header.size - ethosu_queue_available(queue); -} - -static int ethosu_queue_write(struct ethosu_mailbox *mbox, - const struct kvec *vec, - size_t length) -{ - struct ethosu_core_queue *queue = mbox->in_queue; - uint8_t *dst = &queue->data[0]; - uint32_t wpos = queue->header.write; - size_t total_size; - size_t i; - int ret; - - for (i = 0, total_size = 0; i < length; i++) - total_size += vec[i].iov_len; - - if (total_size > ethosu_queue_capacity(queue)) - return -EINVAL; - - for (i = 0; i < length; i++) { - const uint8_t *src = vec[i].iov_base; - const uint8_t *end = src + vec[i].iov_len; - - while (src < end) { - dst[wpos] = *src++; - wpos = (wpos + 1) % queue->header.size; - } - } - - queue->header.write = wpos; - - ret = mbox_send_message(mbox->tx, queue); - if (ret < 0) - return ret; - - return 0; -} - -static int ethosu_queue_write_msg(struct ethosu_mailbox *mbox, - uint32_t type, - void *data, - size_t length) -{ - struct ethosu_core_msg msg = { - .magic = ETHOSU_CORE_MSG_MAGIC, - .type = type, .length= length - }; - const struct kvec vec[2] = { - { &msg, sizeof(msg) }, - { data, length } - }; - - return ethosu_queue_write(mbox, vec, 2); -} - -static int ethosu_queue_read(struct ethosu_mailbox *mbox, - void *data, - size_t length) -{ - struct ethosu_core_queue *queue = mbox->out_queue; - uint8_t *src = &queue->data[0]; - uint8_t *dst = (uint8_t *)data; - const uint8_t *end = dst + length; - uint32_t rpos = queue->header.read; - size_t queue_avail = ethosu_queue_available(queue); - - if (length == 0) - return 0; - else if (queue_avail == 0) - return -ENOMSG; - else if (length > queue_avail) - return -EBADMSG; - - while (dst < end) { - *dst++ = src[rpos]; - rpos = (rpos + 1) % queue->header.size; - } - - queue->header.read = rpos; - - return 0; -} - -void ethosu_mailbox_reset(struct ethosu_mailbox *mbox) -{ - mbox->out_queue->header.read = mbox->out_queue->header.write; -} - -int ethosu_mailbox_read(struct ethosu_mailbox *mbox, - struct ethosu_core_msg *header, - void *data, - size_t length) -{ - int ret; - - /* Read message header magic */ - ret = ethosu_queue_read(mbox, header, sizeof(*header)); - if (ret) { - if (ret != -ENOMSG) - dev_warn(mbox->dev, - "Msg: Failed to read message header\n"); - - return ret; - } - - if (header->magic != ETHOSU_CORE_MSG_MAGIC) { - dev_warn(mbox->dev, - "Msg: Invalid magic. Got: %08X but expected %08X\n", - header->magic, ETHOSU_CORE_MSG_MAGIC); - - return -EINVAL; - } - - dev_info(mbox->dev, - "mbox: Read msg header. magic=%08X, type=%u, length=%u", - header->magic, header->type, header->length); - - /* Check that payload is not larger than allocated buffer */ - if (header->length > length) { - dev_warn(mbox->dev, - "Msg: Buffer size (%zu) too small for message (%u)\n", - sizeof(data), header->length); - - return -ENOMEM; - } - - /* Read payload data */ - ret = ethosu_queue_read(mbox, data, header->length); - if (ret) { - dev_warn(mbox->dev, "Msg: Failed to read payload data\n"); - - return -EBADMSG; - } - - return 0; -} - -int ethosu_mailbox_ping(struct ethosu_mailbox *mbox) -{ - return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_PING, NULL, 0); -} - -int ethosu_mailbox_pong(struct ethosu_mailbox *mbox) -{ - return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_PONG, NULL, 0); -} - -int ethosu_mailbox_version_request(struct ethosu_mailbox *mbox) -{ - return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_VERSION_REQ, NULL, - 0); -} - -int ethosu_mailbox_capabilities_request(struct ethosu_mailbox *mbox, - void *user_arg) -{ - struct ethosu_core_capabilities_req req = { - .user_arg = (ptrdiff_t)user_arg - }; - - return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_CAPABILITIES_REQ, - &req, - sizeof(req)); -} - -int ethosu_mailbox_inference(struct ethosu_mailbox *mbox, - void *user_arg, - uint32_t ifm_count, - struct ethosu_buffer **ifm, - uint32_t ofm_count, - struct ethosu_buffer **ofm, - struct ethosu_buffer *network, - uint8_t *pmu_event_config, - uint8_t pmu_event_config_count, - uint8_t pmu_cycle_counter_enable) -{ - struct ethosu_core_inference_req inf; - uint32_t i; - - /* Verify that the uapi and core has the same number of pmus */ - if (pmu_event_config_count != ETHOSU_CORE_PMU_MAX) { - dev_err(mbox->dev, "PMU count misconfigured.\n"); - - return -EINVAL; - } - - inf.user_arg = (ptrdiff_t)user_arg; - inf.ifm_count = ifm_count; - inf.ofm_count = ofm_count; - inf.pmu_cycle_counter_enable = pmu_cycle_counter_enable; - - for (i = 0; i < ifm_count; i++) - ethosu_core_set_size(ifm[i], &inf.ifm[i]); - - for (i = 0; i < ofm_count; i++) - ethosu_core_set_capacity(ofm[i], &inf.ofm[i]); - - for (i = 0; i < ETHOSU_CORE_PMU_MAX; i++) - inf.pmu_event_config[i] = pmu_event_config[i]; - - ethosu_core_set_size(network, &inf.network); - - return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_INFERENCE_REQ, - &inf, sizeof(inf)); -} - -static void ethosu_mailbox_rx_work(struct work_struct *work) -{ - struct ethosu_mailbox *mbox = container_of(work, typeof(*mbox), work); - - mbox->callback(mbox->user_arg); -} - -static void ethosu_mailbox_rx_callback(struct mbox_client *client, - void *message) -{ - struct ethosu_mailbox *mbox = - container_of(client, typeof(*mbox), client); - - dev_info(mbox->dev, "mbox: Received message.\n"); - - queue_work(mbox->wq, &mbox->work); -} - -static void ethosu_mailbox_tx_done(struct mbox_client *client, - void *message, - int r) -{ - if (r) - dev_warn(client->dev, "mbox: Failed sending message (%d)\n", r); - else - dev_info(client->dev, "mbox: Message sent\n"); -} - -int ethosu_mailbox_init(struct ethosu_mailbox *mbox, - struct device *dev, - struct resource *in_queue, - struct resource *out_queue, - ethosu_mailbox_cb callback, - void *user_arg) -{ - int ret; - - mbox->dev = dev; - mbox->callback = callback; - mbox->user_arg = user_arg; - - mbox->client.dev = dev; - mbox->client.rx_callback = ethosu_mailbox_rx_callback; - mbox->client.tx_prepare = NULL; /* preparation of data is handled - * through the - * queue functions */ - mbox->client.tx_done = ethosu_mailbox_tx_done; - mbox->client.tx_block = true; - mbox->client.knows_txdone = false; - mbox->client.tx_tout = 500; - - mbox->in_queue = devm_ioremap_resource(mbox->dev, in_queue); - if (IS_ERR(mbox->in_queue)) - return PTR_ERR(mbox->in_queue); - - mbox->out_queue = devm_ioremap_resource(mbox->dev, out_queue); - if (IS_ERR(mbox->out_queue)) { - ret = PTR_ERR(mbox->out_queue); - goto unmap_in_queue; - } - - mbox->wq = create_singlethread_workqueue("ethosu_workqueue"); - if (!mbox->wq) { - dev_err(mbox->dev, "Failed to create work queue\n"); - ret = -EINVAL; - goto unmap_out_queue; - } - - INIT_WORK(&mbox->work, ethosu_mailbox_rx_work); - - mbox->tx = mbox_request_channel_byname(&mbox->client, "tx"); - if (IS_ERR(mbox->tx)) { - dev_warn(mbox->dev, "mbox: Failed to request tx channel\n"); - ret = PTR_ERR(mbox->tx); - goto workqueue_destroy; - } - - mbox->rx = mbox_request_channel_byname(&mbox->client, "rx"); - if (IS_ERR(mbox->rx)) { - dev_info(dev, "mbox: Using same channel for RX and TX\n"); - mbox->rx = mbox->tx; - } - - return 0; - -workqueue_destroy: - destroy_workqueue(mbox->wq); - -unmap_out_queue: - devm_iounmap(mbox->dev, mbox->out_queue); - -unmap_in_queue: - devm_iounmap(mbox->dev, mbox->in_queue); - - return ret; -} - -void ethosu_mailbox_deinit(struct ethosu_mailbox *mbox) -{ - if (mbox->rx != mbox->tx) - mbox_free_channel(mbox->rx); - - mbox_free_channel(mbox->tx); - destroy_workqueue(mbox->wq); - devm_iounmap(mbox->dev, mbox->out_queue); - devm_iounmap(mbox->dev, mbox->in_queue); -} diff --git a/drivers/firmware/ethosu/ethosu_mailbox.h b/drivers/firmware/ethosu/ethosu_mailbox.h deleted file mode 100644 index 5cd5e62198b8..000000000000 --- a/drivers/firmware/ethosu/ethosu_mailbox.h +++ /dev/null @@ -1,140 +0,0 @@ -/* - * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. - * - * This program is free software and is provided to you under the terms of the - * GNU General Public License version 2 as published by the Free Software - * Foundation, and any use by you of this program is subject to the terms - * of such GNU licence. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program; if not, you can access it online at - * http://www.gnu.org/licenses/gpl-2.0.html. - * - * SPDX-License-Identifier: GPL-2.0-only - */ - -#ifndef ETHOSU_MAILBOX_H -#define ETHOSU_MAILBOX_H - -/**************************************************************************** - * Includes - ****************************************************************************/ -#include "ethosu_core_interface.h" - -#include -#include -#include - -/**************************************************************************** - * Types - ****************************************************************************/ - -struct device; -struct ethosu_buffer; -struct ethosu_device; -struct ethosu_core_msg; -struct ethosu_core_queue; -struct resource; - -typedef void (*ethosu_mailbox_cb)(void *user_arg); - -struct ethosu_mailbox { - struct device *dev; - struct workqueue_struct *wq; - struct work_struct work; - struct ethosu_core_queue __iomem *in_queue; - struct ethosu_core_queue __iomem *out_queue; - struct mbox_client client; - struct mbox_chan *rx; - struct mbox_chan *tx; - ethosu_mailbox_cb callback; - void *user_arg; -}; - -/**************************************************************************** - * Functions - ****************************************************************************/ - -/** - * ethosu_mailbox_init() - Initialize mailbox - * - * Return: 0 on success, else error code. - */ -int ethosu_mailbox_init(struct ethosu_mailbox *mbox, - struct device *dev, - struct resource *in_queue, - struct resource *out_queue, - ethosu_mailbox_cb callback, - void *user_arg); - -/** - * ethosu_mailbox_deinit() - Deinitialize mailbox - */ -void ethosu_mailbox_deinit(struct ethosu_mailbox *mbox); - -/** - * ethosu_mailbox_read() - Read message from mailbox - * - * Return: 0 message read, else error code. - */ -int ethosu_mailbox_read(struct ethosu_mailbox *mbox, - struct ethosu_core_msg *header, - void *data, - size_t length); - -/** - * ethosu_mailbox_reset() - Reset to end of queue - */ -void ethosu_mailbox_reset(struct ethosu_mailbox *mbox); - -/** - * ethosu_mailbox_ping() - Send ping message - * - * Return: 0 on success, else error code. - */ -int ethosu_mailbox_ping(struct ethosu_mailbox *mbox); - -/** - * ethosu_mailbox_pong() - Send pong response - * - * Return: 0 on success, else error code. - */ -int ethosu_mailbox_pong(struct ethosu_mailbox *mbox); - -/** - * ethosu_mailbox_version_response - Send version request - * - * Return: 0 on succes, else error code - */ -int ethosu_mailbox_version_request(struct ethosu_mailbox *mbox); - -/** - * ethosu_mailbox_capabilities_request() - Send capabilities request - * - * Return: 0 on success, else error code. - */ -int ethosu_mailbox_capabilities_request(struct ethosu_mailbox *mbox, - void *user_arg); - -/** - * ethosu_mailbox_inference() - Send inference - * - * Return: 0 on success, else error code. - */ -int ethosu_mailbox_inference(struct ethosu_mailbox *mbox, - void *user_arg, - uint32_t ifm_count, - struct ethosu_buffer **ifm, - uint32_t ofm_count, - struct ethosu_buffer **ofm, - struct ethosu_buffer *network, - uint8_t *pmu_event_config, - uint8_t pmu_event_config_count, - uint8_t pmu_cycle_counter_enable); - -#endif /* ETHOSU_MAILBOX_H */ diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.c b/drivers/firmware/ethosu/ethosu_rpmsg.c new file mode 100644 index 000000000000..cb7e4556f635 --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_rpmsg.c @@ -0,0 +1,235 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright 2020-2022 NXP + */ + +#include +#include +#include +#include +#include +#include + +#include "ethosu_buffer.h" +#include "ethosu_core_interface.h" +#include "ethosu_device.h" + +struct ethosu_rpmsg *grp; + +#define MSG "ethosu say hello!" + +static void ethosu_core_set_size(struct ethosu_buffer *buf, + struct ethosu_core_buffer *cbuf) +{ + cbuf->ptr = (uint32_t)buf->dma_addr + buf->offset; + cbuf->size = (uint32_t)buf->size; +} + +static void ethosu_core_set_capacity(struct ethosu_buffer *buf, + struct ethosu_core_buffer *cbuf) +{ + cbuf->ptr = (uint32_t)buf->dma_addr + buf->offset + buf->size; + cbuf->size = (uint32_t)buf->capacity - buf->offset - buf->size; +} + +static int ethosu_rpmsg_send(struct ethosu_rpmsg *erp, uint32_t type) +{ + struct ethosu_core_msg msg; + struct rpmsg_device *rpdev = erp->rpdev; + int ret; + + msg.magic = ETHOSU_CORE_MSG_MAGIC; + msg.type = type; + msg.length = 0; + + print_hex_dump(KERN_DEBUG, __func__, DUMP_PREFIX_NONE, 16, 1, + (void *)&msg, sizeof(msg), true); + + ret = rpmsg_send(rpdev->ept, (void *)&msg, sizeof(msg)); + if (ret) { + dev_err(&rpdev->dev, "rpmsg_send failed: %d\n", ret); + return ret; + } + + return 0; +} + +int ethosu_rpmsg_ping(struct ethosu_rpmsg *erp) +{ + return ethosu_rpmsg_send(erp, ETHOSU_CORE_MSG_PING); +} + +int ethosu_rpmsg_pong(struct ethosu_rpmsg *erp) +{ + return ethosu_rpmsg_send(erp, ETHOSU_CORE_MSG_PONG); +} + +int ethosu_rpmsg_version_request(struct ethosu_rpmsg *erp) +{ + return ethosu_rpmsg_send(erp, ETHOSU_CORE_MSG_VERSION_REQ); +} + +int ethosu_rpmsg_capabilities_request(struct ethosu_rpmsg *erp, void *user_arg) +{ + struct ethosu_core_msg msg = { + .magic = ETHOSU_CORE_MSG_MAGIC, + .type = ETHOSU_CORE_MSG_CAPABILITIES_REQ, + .length = sizeof(struct ethosu_core_capabilities_req) + }; + struct ethosu_core_capabilities_req req = { + .user_arg = (uint64_t)user_arg + }; + struct rpmsg_device *rpdev = erp->rpdev; + uint8_t data[sizeof(struct ethosu_core_msg) + + sizeof(struct ethosu_core_capabilities_req)]; + int ret; + + memcpy(data, &msg, sizeof(struct ethosu_core_msg)); + memcpy(data + sizeof(struct ethosu_core_msg), &req, + sizeof(struct ethosu_core_capabilities_req)); + + ret = rpmsg_send(rpdev->ept, (void *)&data, + sizeof(struct ethosu_core_msg) + + sizeof(struct ethosu_core_capabilities_req)); + if (ret) { + dev_err(&rpdev->dev, "rpmsg_send failed: %d\n", ret); + return ret; + } + return 0; +} + +int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp, + void *user_arg, + uint32_t ifm_count, + struct ethosu_buffer **ifm, + uint32_t ofm_count, + struct ethosu_buffer **ofm, + struct ethosu_buffer *network, + uint8_t *pmu_event_config, + uint8_t pmu_event_config_count, + uint8_t pmu_cycle_counter_enable) +{ + struct ethosu_core_msg msg = { + .magic = ETHOSU_CORE_MSG_MAGIC, + .type = ETHOSU_CORE_MSG_INFERENCE_REQ, + .length = sizeof(struct ethosu_core_inference_req) + }; + struct ethosu_core_inference_req req; + struct rpmsg_device *rpdev = erp->rpdev; + uint8_t data[sizeof(struct ethosu_core_msg) + + sizeof(struct ethosu_core_inference_req)]; + int ret; + uint32_t i; + + /* Verify that the uapi and core has the same number of pmus */ + if (pmu_event_config_count != ETHOSU_CORE_PMU_MAX) { + dev_err(&rpdev->dev, "PMU count misconfigured.\n"); + + return -EINVAL; + } + + req.user_arg = (uint64_t)user_arg; + req.ifm_count = ifm_count; + req.ofm_count = ofm_count; + req.pmu_cycle_counter_enable = pmu_cycle_counter_enable; + + for (i = 0; i < ifm_count; i++) + ethosu_core_set_size(ifm[i], &req.ifm[i]); + + for (i = 0; i < ofm_count; i++) + ethosu_core_set_capacity(ofm[i], &req.ofm[i]); + + for (i = 0; i < ETHOSU_CORE_PMU_MAX; i++) + req.pmu_event_config[i] = pmu_event_config[i]; + + ethosu_core_set_size(network, &req.network); + + memcpy(data, &msg, sizeof(struct ethosu_core_msg)); + memcpy(data + sizeof(struct ethosu_core_msg), &req, + sizeof(struct ethosu_core_inference_req)); + + ret = rpmsg_send(rpdev->ept, (void *)&data, + sizeof(struct ethosu_core_msg) + + sizeof(struct ethosu_core_inference_req)); + if (ret) { + dev_err(&rpdev->dev, "rpmsg_send failed: %d\n", ret); + return ret; + } + + return 0; +} + +static int rpmsg_ethosu_cb(struct rpmsg_device *rpdev, + void *data, int len, void *priv, u32 src) +{ + struct ethosu_rpmsg *rpmsg = dev_get_drvdata(&rpdev->dev); + + if (len == 0) + return 0; + + dev_dbg(&rpdev->dev, "msg(<- src 0x%x) len %d\n", src, len); + + print_hex_dump(KERN_DEBUG, __func__, DUMP_PREFIX_NONE, 16, 1, + data, len, true); + + rpmsg->callback(rpmsg->user_arg, data); + + return 0; +} + +static int ethosu_rpmsg_probe(struct rpmsg_device *rpdev) +{ + int ret; + + grp->rpdev = rpdev; + dev_set_drvdata(&rpdev->dev, grp); + + dev_info(&rpdev->dev, "new channel: 0x%x -> 0x%x!\n", + rpdev->src, rpdev->dst); + + ret = rpmsg_send(rpdev->ept, MSG, strlen(MSG)); + if (ret) { + dev_err(&rpdev->dev, "rpmsg_send failed: %d\n", ret); + } + + return 0; +} + +static void ethosu_rpmsg_remove(struct rpmsg_device *rpdev) +{ + dev_info(&rpdev->dev, "rpmsg ethosu driver is removed\n"); +} + +static struct rpmsg_device_id rpmsg_driver_ethosu_id_table[] = { + { .name = "rpmsg-ethosu-channel" }, + { }, +}; + +static struct rpmsg_driver ethosu_rpmsg_driver = { + .drv.name = KBUILD_MODNAME, + .drv.owner = THIS_MODULE, + .id_table = rpmsg_driver_ethosu_id_table, + .probe = ethosu_rpmsg_probe, + .callback = rpmsg_ethosu_cb, + .remove = ethosu_rpmsg_remove, +}; + +int ethosu_rpmsg_init(struct ethosu_rpmsg *erp, + ethosu_rpmsg_cb callback, void *user_arg) +{ + grp = erp; + erp->callback = callback; + erp->user_arg = user_arg; + + return register_rpmsg_driver(ðosu_rpmsg_driver); +} + +int ethosu_rpmsg_deinit(struct ethosu_rpmsg *erp) +{ + erp->callback = NULL; + erp->user_arg = NULL; + erp->rpdev = NULL; + + unregister_rpmsg_driver(ðosu_rpmsg_driver); + return 0; +} diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.h b/drivers/firmware/ethosu/ethosu_rpmsg.h new file mode 100644 index 000000000000..dd08e0d7945b --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_rpmsg.h @@ -0,0 +1,73 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright 2020-2022 NXP + */ + +#ifndef ETHOSU_RPMSG_H +#define ETHOSU_RPMSG_H + +#include + +struct device; +struct ethosu_buffer; +struct ethosu_device; +struct ethosu_core_msg; + +typedef void (*ethosu_rpmsg_cb)(void *user_arg, void *data); + +struct ethosu_rpmsg { + struct rpmsg_device *rpdev; + ethosu_rpmsg_cb callback; + void *user_arg; +}; +/** + * ethosu_rpmsg_ping() - Send ping message + * + * Return: 0 on success, else error code. + */ +int ethosu_rpmsg_ping(struct ethosu_rpmsg *erp); + +/** + * ethosu_rpmsg_pong() - Send pong message + * + * Return: 0 on success, else error code. + */ +int ethosu_rpmsg_pong(struct ethosu_rpmsg *erp); + +/** + * ethosu_rpmsg_version_response - Send version request + * + * Return: 0 on success, else error code + */ +int ethosu_rpmsg_version_request(struct ethosu_rpmsg *erp); + +/** + * ethosu_rpmsg_capabilities_request - Send capabilities request + * + * Return: 0 on success, else error code + */ +int ethosu_rpmsg_capabilities_request(struct ethosu_rpmsg *erp, + void *user_arg); + +/** + * ethosu_rpmsg_inference() - Send inference + * + * Return: 0 on success, else error code. + */ +int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp, + void *user_arg, + uint32_t ifm_count, + struct ethosu_buffer **ifm, + uint32_t ofm_count, + struct ethosu_buffer **ofm, + struct ethosu_buffer *network, + uint8_t *pmu_event_config, + uint8_t pmu_event_config_count, + uint8_t pmu_cycle_counter_enable + ); + +int ethosu_rpmsg_init(struct ethosu_rpmsg *erp, + ethosu_rpmsg_cb callback, void *user_arg); + +int ethosu_rpmsg_deinit(struct ethosu_rpmsg *erp); +#endif /* ETHOSU_RPMSG_H */ From patchwork Fri Jun 16 05:59:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Wang X-Patchwork-Id: 13282003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5DC27EB64D7 for ; Fri, 16 Jun 2023 05:43:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=AmMq6IDi8lNZwPCfR0Q3ehWuc3GZGMngiBsdGM72rEQ=; b=blALX5aO2W4ivH +6DBZ6YXlochCDGJHFw6JNzpxKZKRjhnyWXkBqnfRFidDQHv7ctMRDfBWT3tsmVUPS64NcdcQMan9 4xHBNxCfUclUc0CAtGCB/zUNWqM0pCXtaOJOAizWk3pcQSQpeZG2Z5w7/B5ELxQbVO1LHR6V4+wjQ slP9JneSGHqeuSak8c69CwgFNalT0swgVbfytbvrxABIdf+lM9++wPq58Si/u56fWzkuqo9jQloih n1gwFAc3AZMoAfTofM2guRyU77Isubixu6GYFbd2E3wAOdLvaqJNGUT7D7jABVzOdz/QW5ytSvUHl GrKzg3wGZRb06JcgXToQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qA2F0-00GurT-2I; Fri, 16 Jun 2023 05:43:22 +0000 Received: from inva020.nxp.com ([92.121.34.13]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qA2Eo-00Guly-0R for linux-arm-kernel@lists.infradead.org; Fri, 16 Jun 2023 05:43:12 +0000 Received: from inva020.nxp.com (localhost [127.0.0.1]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 88E131A06B6; Fri, 16 Jun 2023 07:43:04 +0200 (CEST) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 260A41A05B8; Fri, 16 Jun 2023 07:43:04 +0200 (CEST) Received: from localhost.localdomain (mega.ap.freescale.net [10.192.208.232]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id 21477181D0C0; Fri, 16 Jun 2023 13:43:03 +0800 (+08) From: Alison Wang To: gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: leoyang.li@nxp.com, xuelin.shi@nxp.com, xiaofeng.ren@nxp.com, feng.guo@nxp.com, Alison Wang Subject: [PATCH 3/8] ethosu: Add inference type option for model and op Date: Fri, 16 Jun 2023 13:59:08 +0800 Message-Id: <20230616055913.2360-4-alison.wang@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230616055913.2360-1-alison.wang@nxp.com> References: <20230616055913.2360-1-alison.wang@nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230615_224310_463065_70B094B2 X-CRM114-Status: GOOD ( 13.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This patch adds inference type option for model and op. Signed-off-by: Feng Guo Reviewed-by: Peng Fan --- drivers/firmware/ethosu/ethosu_core_interface.h | 6 +++++- drivers/firmware/ethosu/ethosu_inference.c | 14 +++++++++++++- drivers/firmware/ethosu/ethosu_inference.h | 1 + drivers/firmware/ethosu/ethosu_rpmsg.c | 4 +++- drivers/firmware/ethosu/ethosu_rpmsg.h | 3 ++- drivers/firmware/ethosu/uapi/ethosu.h | 11 ++++++++++- 6 files changed, 34 insertions(+), 5 deletions(-) diff --git a/drivers/firmware/ethosu/ethosu_core_interface.h b/drivers/firmware/ethosu/ethosu_core_interface.h index ef63c3b55352..b60508e4792f 100644 --- a/drivers/firmware/ethosu/ethosu_core_interface.h +++ b/drivers/firmware/ethosu/ethosu_core_interface.h @@ -35,13 +35,16 @@ namespace EthosU { #define ETHOSU_CORE_BUFFER_MAX 16 /** Maximum number of PMU counters to be returned for inference */ -#define ETHOSU_CORE_PMU_MAX 8 +#define ETHOSU_CORE_PMU_MAX 4 #define ETHOSU_CORE_MSG_MAGIC 0x41457631 #define ETHOSU_CORE_MSG_VERSION_MAJOR 0 #define ETHOSU_CORE_MSG_VERSION_MINOR 2 #define ETHOSU_CORE_MSG_VERSION_PATCH 0 +#define ETHOSU_CORE_INFERENCE_MODEL 0 +#define ETHOSU_CORE_INFERENCE_OP 1 + /** * enum ethosu_core_msg_type - Message types * @@ -107,6 +110,7 @@ struct ethosu_core_inference_req { struct ethosu_core_buffer network; uint8_t pmu_event_config[ETHOSU_CORE_PMU_MAX]; uint32_t pmu_cycle_counter_enable; + uint32_t inference_type; }; struct ethosu_core_inference_rsp { diff --git a/drivers/firmware/ethosu/ethosu_inference.c b/drivers/firmware/ethosu/ethosu_inference.c index 853947a2334e..b5e94e19bce0 100644 --- a/drivers/firmware/ethosu/ethosu_inference.c +++ b/drivers/firmware/ethosu/ethosu_inference.c @@ -92,7 +92,8 @@ static int ethosu_inference_send(struct ethosu_inference *inf) inf->net->buf, inf->pmu_event_config, ETHOSU_PMU_EVENT_MAX, - inf->pmu_cycle_counter_enable); + inf->pmu_cycle_counter_enable, + inf->inference_type); if (ret) return ret; @@ -233,6 +234,17 @@ int ethosu_inference_create(struct ethosu_device *edev, inf = devm_kzalloc(edev->dev, sizeof(*inf), GFP_KERNEL); if (!inf) return -ENOMEM; + switch (uapi->inference_type) { + case ETHOSU_UAPI_INFERENCE_MODEL: + inf->inference_type = ETHOSU_CORE_INFERENCE_MODEL; + break; + case ETHOSU_UAPI_INFERENCE_OP: + inf->inference_type = ETHOSU_CORE_INFERENCE_OP; + break; + default: + inf->inference_type = ETHOSU_CORE_INFERENCE_MODEL; + break; + } inf->edev = edev; inf->net = net; diff --git a/drivers/firmware/ethosu/ethosu_inference.h b/drivers/firmware/ethosu/ethosu_inference.h index 07370ca01f22..8414eadbda92 100644 --- a/drivers/firmware/ethosu/ethosu_inference.h +++ b/drivers/firmware/ethosu/ethosu_inference.h @@ -74,6 +74,7 @@ struct ethosu_inference { uint32_t pmu_event_count[ETHOSU_PMU_EVENT_MAX]; uint32_t pmu_cycle_counter_enable; uint64_t pmu_cycle_counter_count; + uint32_t inference_type; struct list_head list; }; diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.c b/drivers/firmware/ethosu/ethosu_rpmsg.c index cb7e4556f635..e4cf398468e4 100644 --- a/drivers/firmware/ethosu/ethosu_rpmsg.c +++ b/drivers/firmware/ethosu/ethosu_rpmsg.c @@ -107,7 +107,8 @@ int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp, struct ethosu_buffer *network, uint8_t *pmu_event_config, uint8_t pmu_event_config_count, - uint8_t pmu_cycle_counter_enable) + uint8_t pmu_cycle_counter_enable, + uint32_t inference_type) { struct ethosu_core_msg msg = { .magic = ETHOSU_CORE_MSG_MAGIC, @@ -132,6 +133,7 @@ int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp, req.ifm_count = ifm_count; req.ofm_count = ofm_count; req.pmu_cycle_counter_enable = pmu_cycle_counter_enable; + req.inference_type = inference_type; for (i = 0; i < ifm_count; i++) ethosu_core_set_size(ifm[i], &req.ifm[i]); diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.h b/drivers/firmware/ethosu/ethosu_rpmsg.h index dd08e0d7945b..a4a639997a26 100644 --- a/drivers/firmware/ethosu/ethosu_rpmsg.h +++ b/drivers/firmware/ethosu/ethosu_rpmsg.h @@ -63,7 +63,8 @@ int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp, struct ethosu_buffer *network, uint8_t *pmu_event_config, uint8_t pmu_event_config_count, - uint8_t pmu_cycle_counter_enable + uint8_t pmu_cycle_counter_enable, + uint32_t inference_type ); int ethosu_rpmsg_init(struct ethosu_rpmsg *erp, diff --git a/drivers/firmware/ethosu/uapi/ethosu.h b/drivers/firmware/ethosu/uapi/ethosu.h index 903316dff7b3..c4a0df67336c 100644 --- a/drivers/firmware/ethosu/uapi/ethosu.h +++ b/drivers/firmware/ethosu/uapi/ethosu.h @@ -62,7 +62,7 @@ namespace EthosU { #define ETHOSU_FD_MAX 16 /* Maximum number of PMUs available */ -#define ETHOSU_PMU_EVENT_MAX 8 +#define ETHOSU_PMU_EVENT_MAX 4 /**************************************************************************** * Types @@ -173,6 +173,14 @@ struct ethosu_uapi_device_capabilities { __u32 driver_major_rev; }; +/** + * enum ethosu_uapi_inference_type - Inference type + */ +enum ethosu_uapi_inference_type { + ETHOSU_UAPI_INFERENCE_MODEL = 0, + ETHOSU_UAPI_INFERENCE_OP +}; + /** * struct ethosu_uapi_inference_create - Create network request * @ifm_count: Number of IFM file descriptors @@ -185,6 +193,7 @@ struct ethosu_uapi_inference_create { __u32 ifm_fd[ETHOSU_FD_MAX]; __u32 ofm_count; __u32 ofm_fd[ETHOSU_FD_MAX]; + enum ethosu_uapi_inference_type inference_type; struct ethosu_uapi_pmu_config pmu_config; }; From patchwork Fri Jun 16 05:59:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Wang X-Patchwork-Id: 13282000 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E4462EB64D7 for ; Fri, 16 Jun 2023 05:43:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=D03JiVLFS92ZcNszohVE0xMH0i/11vL2npoX9z3WSjM=; b=cWw2ewAlTYlW31 LSqhcOHpGpLgBb2feZ1LhmIHW9NxFkXwSSIKoQdbL3EQHVTZlax1ZZ2mYH8qH/gRCcky9LSB0I6Ve Zb+yoNZf5zUM/ZnKUfWRVH2uZlt4uRtKAcPKT7fPy+AlrlOrUlYojMsbVlkaeHZAIYl+XWr3WLmXD FC1xkAc1C5iaqI5wtEuZtDIXB4lvw5Y8RhRVd28qwH00tis1EXtRfkaVJO2p4GrYluflEkTs+dnDm gyjbdJZiMGYISY+fEo7QJmvsR4qGd+oFgNit+/CiMjcU1kwkifgfbnSGvNJVpKFXeKsQgokyE4NiT ECT32ftkCK0ru/eqKilQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qA2Er-00Guoj-01; Fri, 16 Jun 2023 05:43:13 +0000 Received: from inva020.nxp.com ([92.121.34.13]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qA2En-00Gulo-0H for linux-arm-kernel@lists.infradead.org; Fri, 16 Jun 2023 05:43:11 +0000 Received: from inva020.nxp.com (localhost [127.0.0.1]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 6EE931A033D; Fri, 16 Jun 2023 07:43:05 +0200 (CEST) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 0D9F61A05B8; Fri, 16 Jun 2023 07:43:05 +0200 (CEST) Received: from localhost.localdomain (mega.ap.freescale.net [10.192.208.232]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id 094A61820F58; Fri, 16 Jun 2023 13:43:03 +0800 (+08) From: Alison Wang To: gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: leoyang.li@nxp.com, xuelin.shi@nxp.com, xiaofeng.ren@nxp.com, feng.guo@nxp.com, Alison Wang Subject: [PATCH 4/8] ethosu: Add suspend/resume power management Date: Fri, 16 Jun 2023 13:59:09 +0800 Message-Id: <20230616055913.2360-5-alison.wang@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230616055913.2360-1-alison.wang@nxp.com> References: <20230616055913.2360-1-alison.wang@nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230615_224309_391902_36320281 X-CRM114-Status: GOOD ( 19.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This patch adds suspend and resume power management support. The corresponding message will be sent to Cortex-M core via RPMsg protocol. Signed-off-by: Alison Wang Signed-off-by: Feng Guo Reviewed-by: Peng Fan --- drivers/firmware/ethosu/ethosu_buffer.c | 3 ++ .../firmware/ethosu/ethosu_core_interface.h | 14 ++++++++ drivers/firmware/ethosu/ethosu_device.c | 36 +++++++++++++++++++ drivers/firmware/ethosu/ethosu_device.h | 10 ++++++ drivers/firmware/ethosu/ethosu_driver.c | 29 +++++++++++++++ drivers/firmware/ethosu/ethosu_rpmsg.c | 30 ++++++++++++++++ drivers/firmware/ethosu/ethosu_rpmsg.h | 11 ++++++ 7 files changed, 133 insertions(+) diff --git a/drivers/firmware/ethosu/ethosu_buffer.c b/drivers/firmware/ethosu/ethosu_buffer.c index 43a433355f30..c8aa8031a8de 100644 --- a/drivers/firmware/ethosu/ethosu_buffer.c +++ b/drivers/firmware/ethosu/ethosu_buffer.c @@ -222,6 +222,9 @@ int ethosu_buffer_create(struct ethosu_device *edev, struct ethosu_buffer *buf; int ret = -ENOMEM; + if (!capacity) + return -EINVAL; + buf = devm_kzalloc(edev->dev, sizeof(*buf), GFP_KERNEL); if (!buf) return -ENOMEM; diff --git a/drivers/firmware/ethosu/ethosu_core_interface.h b/drivers/firmware/ethosu/ethosu_core_interface.h index b60508e4792f..9267e96aec9b 100644 --- a/drivers/firmware/ethosu/ethosu_core_interface.h +++ b/drivers/firmware/ethosu/ethosu_core_interface.h @@ -60,6 +60,8 @@ enum ethosu_core_msg_type { ETHOSU_CORE_MSG_VERSION_RSP, ETHOSU_CORE_MSG_CAPABILITIES_REQ, ETHOSU_CORE_MSG_CAPABILITIES_RSP, + ETHOSU_CORE_MSG_POWER_REQ, + ETHOSU_CORE_MSG_POWER_RSP, ETHOSU_CORE_MSG_MAX }; @@ -161,6 +163,18 @@ struct ethosu_core_msg_capabilities_rsp { uint32_t custom_dma; }; +enum ethosu_core_power_req_type { + ETHOSU_CORE_POWER_REQ_SUSPEND = 0, + ETHOSU_CORE_POWER_REQ_RESUME +}; + +/** + * struct ethosu_core_power_req - Message power request + */ +struct ethosu_core_power_req { + enum ethosu_core_power_req_type type; +}; + /** * enum ethosu_core_msg_err_type - Error types */ diff --git a/drivers/firmware/ethosu/ethosu_device.c b/drivers/firmware/ethosu/ethosu_device.c index b2d4737080ac..1627f25c8a95 100644 --- a/drivers/firmware/ethosu/ethosu_device.c +++ b/drivers/firmware/ethosu/ethosu_device.c @@ -38,6 +38,7 @@ #include #include #include +#include #include #include @@ -49,6 +50,8 @@ #define CAPABILITIES_RESP_TIMEOUT_MS 2000 +#define ETHOSU_FIRMWARE_NAME "ethosu_firmware" + /**************************************************************************** * Types ****************************************************************************/ @@ -161,6 +164,9 @@ static int ethosu_handle_msg(struct ethosu_device *edev, void *data) case ETHOSU_CORE_MSG_PONG: dev_info(edev->dev, "Msg: Pong\n"); break; + case ETHOSU_CORE_MSG_POWER_RSP: + dev_info(edev->dev, "Msg: Power response\n"); + break; case ETHOSU_CORE_MSG_INFERENCE_RSP: if (header->length != sizeof(struct ethosu_core_inference_rsp)) { dev_warn(edev->dev, @@ -243,11 +249,41 @@ static int ethosu_open(struct inode *inode, { struct ethosu_device *edev = container_of(inode->i_cdev, struct ethosu_device, cdev); + phandle rproc_phandle; + struct rproc *rproc; + int ret = 0; file->private_data = edev; dev_info(edev->dev, "Opening device node.\n"); + if (of_property_read_u32(edev->dev->of_node, "fsl,cm33-proc", + &rproc_phandle)) { + dev_err(edev->dev, "could not get rproc phandle\n"); + return -ENODEV; + } + + rproc = rproc_get_by_phandle(rproc_phandle); + if (!rproc) { + dev_err(edev->dev, "could not get rproc handle\n"); + return -EINVAL; + } + + ret = rproc_set_firmware(rproc, ETHOSU_FIRMWARE_NAME); + + if (!ret && atomic_read(&rproc->power) == 0) { + init_completion(&edev->erp.rpmsg_ready); + ret = rproc_boot(rproc); + if (ret) + dev_err(edev->dev, "could not boot a remote processor\n"); + else + wait_for_completion_interruptible(&edev->erp.rpmsg_ready); + } else { + dev_err(edev->dev, "can't change firmware or remote processor is running\n"); + } + + edev->open = true; + return nonseekable_open(inode, file); } diff --git a/drivers/firmware/ethosu/ethosu_device.h b/drivers/firmware/ethosu/ethosu_device.h index 3943c8df8f28..da470241b6a1 100644 --- a/drivers/firmware/ethosu/ethosu_device.h +++ b/drivers/firmware/ethosu/ethosu_device.h @@ -51,6 +51,7 @@ struct ethosu_device { struct ethosu_rpmsg erp; struct list_head capabilities_list; struct list_head inference_list; + bool open; }; /** @@ -83,4 +84,13 @@ int ethosu_dev_init(struct ethosu_device *edev, */ void ethosu_dev_deinit(struct ethosu_device *edev); +/** + * ethosu_suspend() - Suspend the device + */ +int ethosu_suspend(struct device *dev); +/** + * ethosu_resume() - Resume the device + */ +int ethosu_resume(struct device *dev); + #endif /* ETHOSU_DEVICE_H */ diff --git a/drivers/firmware/ethosu/ethosu_driver.c b/drivers/firmware/ethosu/ethosu_driver.c index 27931e9aa97c..d3306ef625ae 100644 --- a/drivers/firmware/ethosu/ethosu_driver.c +++ b/drivers/firmware/ethosu/ethosu_driver.c @@ -27,6 +27,7 @@ #include #include +#include "ethosu_core_interface.h" #include "ethosu_device.h" /**************************************************************************** @@ -101,6 +102,33 @@ static int ethosu_pdev_remove(struct platform_device *pdev) return 0; } +int ethosu_suspend(struct device *dev) +{ + struct ethosu_device *edev = dev->driver_data; + int ret = 0; + + if (edev->open) + ret = ethosu_rpmsg_power_request(&edev->erp, ETHOSU_CORE_POWER_REQ_SUSPEND); + + return ret; +} + +int ethosu_resume(struct device *dev) +{ + struct ethosu_device *edev = dev->driver_data; + int ret = 0; + + if (edev->open) + ret = ethosu_rpmsg_power_request(&edev->erp, ETHOSU_CORE_POWER_REQ_RESUME); + + return ret; +} + +struct dev_pm_ops ethosu_pm_ops = { + .suspend = ethosu_suspend, + .resume = ethosu_resume, +}; + static const struct of_device_id ethosu_pdev_match[] = { { .compatible = "arm,ethosu" }, { /* Sentinel */ }, @@ -115,6 +143,7 @@ static struct platform_driver ethosu_pdev_driver = { .name = ETHOSU_DRIVER_NAME, .owner = THIS_MODULE, .of_match_table = of_match_ptr(ethosu_pdev_match), + .pm = ðosu_pm_ops, }, }; diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.c b/drivers/firmware/ethosu/ethosu_rpmsg.c index e4cf398468e4..a0320b8407d1 100644 --- a/drivers/firmware/ethosu/ethosu_rpmsg.c +++ b/drivers/firmware/ethosu/ethosu_rpmsg.c @@ -98,6 +98,35 @@ int ethosu_rpmsg_capabilities_request(struct ethosu_rpmsg *erp, void *user_arg) return 0; } +int ethosu_rpmsg_power_request(struct ethosu_rpmsg *erp, + enum ethosu_core_power_req_type power_type) +{ + struct ethosu_core_msg msg = { + .magic = ETHOSU_CORE_MSG_MAGIC, + .type = ETHOSU_CORE_MSG_POWER_REQ, + .length = sizeof(struct ethosu_core_power_req) + }; + struct ethosu_core_power_req req; + struct rpmsg_device *rpdev = erp->rpdev; + uint8_t data[sizeof(struct ethosu_core_msg) + + sizeof(struct ethosu_core_power_req)]; + int ret; + + req.type = power_type; + memcpy(data, &msg, sizeof(struct ethosu_core_msg)); + memcpy(data + sizeof(struct ethosu_core_msg), &req, + sizeof(struct ethosu_core_power_req)); + + ret = rpmsg_send(rpdev->ept, (void *)&data, + sizeof(struct ethosu_core_msg) + + sizeof(struct ethosu_core_power_req)); + if (ret) { + dev_err(&rpdev->dev, "rpmsg_send failed: %d\n", ret); + return ret; + } + return 0; +} + int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp, void *user_arg, uint32_t ifm_count, @@ -193,6 +222,7 @@ static int ethosu_rpmsg_probe(struct rpmsg_device *rpdev) if (ret) { dev_err(&rpdev->dev, "rpmsg_send failed: %d\n", ret); } + complete(&grp->rpmsg_ready); return 0; } diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.h b/drivers/firmware/ethosu/ethosu_rpmsg.h index a4a639997a26..cee3d96c7895 100644 --- a/drivers/firmware/ethosu/ethosu_rpmsg.h +++ b/drivers/firmware/ethosu/ethosu_rpmsg.h @@ -7,6 +7,8 @@ #define ETHOSU_RPMSG_H #include +#include +#include "ethosu_core_interface.h" struct device; struct ethosu_buffer; @@ -19,6 +21,7 @@ struct ethosu_rpmsg { struct rpmsg_device *rpdev; ethosu_rpmsg_cb callback; void *user_arg; + struct completion rpmsg_ready; }; /** * ethosu_rpmsg_ping() - Send ping message @@ -49,6 +52,14 @@ int ethosu_rpmsg_version_request(struct ethosu_rpmsg *erp); int ethosu_rpmsg_capabilities_request(struct ethosu_rpmsg *erp, void *user_arg); +/** + * ethosu_rpmsg_power_request - Send power request + * + * Return: 0 on success, else error code + */ +int ethosu_rpmsg_power_request(struct ethosu_rpmsg *erp, + enum ethosu_core_power_req_type power_type); + /** * ethosu_rpmsg_inference() - Send inference * From patchwork Fri Jun 16 05:59:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Wang X-Patchwork-Id: 13282007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 65DE5EB64D8 for ; Fri, 16 Jun 2023 05:44:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=kbrqnD3t23AbvPUyGrKxV79ESKaH4JsRRAj8oCyT8mI=; b=IYYDrUaq0DTBz0 00AsVow2XGi95EMtWxZZ7L8r4f8VdxRT6Lji5Qa7k+4gNEU7Jt9vgrjBuFkMh+zEWm+AFsAgGduWr O5R5i0hIPCIfqgis7GCqofYKmTE1Eu8epMcgMSYM32gDkyyAOquXbwzr0ZGDl6fIPn0nYpjdDi5+p u3N5yLjTMmkIefMvOR5+oDA/04CMZxXSB/m/1AXL1vW5SHOVyDyAjD4r7G2PUk774EtWwEw9xVuhZ mDC5bDKOeJvQKzrTZCxD69z0CuSQGZ+aWBztb50ga9upVBLbOVaGuWJNJOu2bDYFdHw0G4WgNquS5 FH4xgvYKU+XVm76l3I1g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qA2F2-00Gusz-2J; Fri, 16 Jun 2023 05:43:24 +0000 Received: from inva020.nxp.com ([92.121.34.13]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qA2Eo-00Gulz-0a for linux-arm-kernel@lists.infradead.org; Fri, 16 Jun 2023 05:43:17 +0000 Received: from inva020.nxp.com (localhost [127.0.0.1]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id A452B1A05B8; Fri, 16 Jun 2023 07:43:06 +0200 (CEST) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 13DF91A068F; Fri, 16 Jun 2023 07:43:06 +0200 (CEST) Received: from localhost.localdomain (mega.ap.freescale.net [10.192.208.232]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id E567D181D0C0; Fri, 16 Jun 2023 13:43:04 +0800 (+08) From: Alison Wang To: gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: leoyang.li@nxp.com, xuelin.shi@nxp.com, xiaofeng.ren@nxp.com, feng.guo@nxp.com, Alison Wang Subject: [PATCH 5/8] ethosu: Use ids for identifying messages sent to Ethos-U firmware Date: Fri, 16 Jun 2023 13:59:10 +0800 Message-Id: <20230616055913.2360-6-alison.wang@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230616055913.2360-1-alison.wang@nxp.com> References: <20230616055913.2360-1-alison.wang@nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230615_224310_636189_BB0405E3 X-CRM114-Status: GOOD ( 27.97 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Avoid to use pointers to kernel memory as identify messages, prior this change, if memory is reused that can lead to unexpected conflicts. Remove ref counting from capabilities since memory is freed in only one place. Finally, extract the capabilities code in its own files. Signed-off-by: Davide Grohmann Signed-off-by: Kristofer Jonsson Signed-off-by: Alison Wang --- drivers/firmware/ethosu/Kconfig | 2 +- drivers/firmware/ethosu/Makefile | 5 +- drivers/firmware/ethosu/ethosu_buffer.c | 34 +-- drivers/firmware/ethosu/ethosu_buffer.h | 2 +- drivers/firmware/ethosu/ethosu_capabilities.c | 157 ++++++++++++ drivers/firmware/ethosu/ethosu_capabilities.h | 47 ++++ .../firmware/ethosu/ethosu_core_interface.h | 20 +- drivers/firmware/ethosu/ethosu_device.c | 229 +++++------------- drivers/firmware/ethosu/ethosu_device.h | 17 +- drivers/firmware/ethosu/ethosu_driver.c | 4 +- drivers/firmware/ethosu/ethosu_inference.c | 229 ++++++++++++------ drivers/firmware/ethosu/ethosu_inference.h | 14 +- drivers/firmware/ethosu/ethosu_network.c | 22 +- drivers/firmware/ethosu/ethosu_network.h | 2 +- drivers/firmware/ethosu/ethosu_rpmsg.c | 72 +++++- drivers/firmware/ethosu/ethosu_rpmsg.h | 53 +++- drivers/firmware/ethosu/uapi/ethosu.h | 8 +- 17 files changed, 606 insertions(+), 311 deletions(-) create mode 100644 drivers/firmware/ethosu/ethosu_capabilities.c create mode 100644 drivers/firmware/ethosu/ethosu_capabilities.h diff --git a/drivers/firmware/ethosu/Kconfig b/drivers/firmware/ethosu/Kconfig index ce837f45f8e5..4d3b1b86aabe 100644 --- a/drivers/firmware/ethosu/Kconfig +++ b/drivers/firmware/ethosu/Kconfig @@ -1,5 +1,5 @@ # -# (C) COPYRIGHT 2020 ARM Limited. All rights reserved. +# (C) COPYRIGHT 2020,2022 ARM Limited. # # This program is free software and is provided to you under the terms of the # GNU General Public License version 2 as published by the Free Software diff --git a/drivers/firmware/ethosu/Makefile b/drivers/firmware/ethosu/Makefile index 0b3fe72c9c4f..fa0731adad35 100644 --- a/drivers/firmware/ethosu/Makefile +++ b/drivers/firmware/ethosu/Makefile @@ -1,5 +1,5 @@ # -# (C) COPYRIGHT 2020 ARM Limited. All rights reserved. +# Copyright (c) 2020,2022 Arm Limited. # # This program is free software and is provided to you under the terms of the # GNU General Public License version 2 as published by the Free Software @@ -25,4 +25,5 @@ ethosu-objs := ethosu_driver.o \ ethosu_device.o \ ethosu_inference.o \ ethosu_rpmsg.o \ - ethosu_network.o + ethosu_network.o \ + ethosu_capabilities.o diff --git a/drivers/firmware/ethosu/ethosu_buffer.c b/drivers/firmware/ethosu/ethosu_buffer.c index c8aa8031a8de..d71db348a240 100644 --- a/drivers/firmware/ethosu/ethosu_buffer.c +++ b/drivers/firmware/ethosu/ethosu_buffer.c @@ -1,5 +1,5 @@ /* - * (C) COPYRIGHT 2020-2021 Arm Limited. All rights reserved. + * Copyright (c) 2020-2022 Arm Limited. * * This program is free software and is provided to you under the terms of the * GNU General Public License version 2 as published by the Free Software @@ -126,7 +126,7 @@ static void ethosu_buffer_destroy(struct kref *kref) struct ethosu_buffer *buf = container_of(kref, struct ethosu_buffer, kref); - dev_info(buf->edev->dev, "Buffer destroy. handle=0x%pK\n", buf); + dev_dbg(buf->edev->dev, "Buffer destroy. buf=0x%pK\n", buf); dma_free_coherent(buf->edev->dev, buf->capacity, buf->cpu_addr, buf->dma_addr_orig); @@ -138,7 +138,8 @@ static int ethosu_buffer_release(struct inode *inode, { struct ethosu_buffer *buf = file->private_data; - dev_info(buf->edev->dev, "Buffer release. handle=0x%pK\n", buf); + dev_dbg(buf->edev->dev, "Buffer release. file=0x%pK, buf=0x%pK\n", + file, buf); ethosu_buffer_put(buf); @@ -151,7 +152,8 @@ static int ethosu_buffer_mmap(struct file *file, struct ethosu_buffer *buf = file->private_data; int ret; - dev_info(buf->edev->dev, "Buffer mmap. handle=0x%pK\n", buf); + dev_dbg(buf->edev->dev, "Buffer mmap. file=0x%pK, buf=0x%pK\n", + file, buf); ret = dma_mmap_coherent(buf->edev->dev, vma, buf->cpu_addr, buf->dma_addr_orig, @@ -172,7 +174,9 @@ static long ethosu_buffer_ioctl(struct file *file, if (ret) return ret; - dev_info(buf->edev->dev, "Ioctl. cmd=%u, arg=%lu\n", cmd, arg); + dev_dbg(buf->edev->dev, + "Buffer ioctl. file=0x%pK, buf=0x%pK, cmd=0x%x, arg=%lu\n", + file, buf, cmd, arg); switch (cmd) { case ETHOSU_IOCTL_BUFFER_SET: { @@ -181,9 +185,9 @@ static long ethosu_buffer_ioctl(struct file *file, if (copy_from_user(&uapi, udata, sizeof(uapi))) break; - dev_info(buf->edev->dev, - "Ioctl: Buffer set. size=%u, offset=%u\n", - uapi.size, uapi.offset); + dev_dbg(buf->edev->dev, + "Buffer ioctl: Buffer set. size=%u, offset=%u\n", + uapi.size, uapi.offset); ret = ethosu_buffer_resize(buf, uapi.size, uapi.offset); break; @@ -194,9 +198,9 @@ static long ethosu_buffer_ioctl(struct file *file, uapi.size = buf->size; uapi.offset = buf->offset; - dev_info(buf->edev->dev, - "Ioctl: Buffer get. size=%u, offset=%u\n", - uapi.size, uapi.offset); + dev_dbg(buf->edev->dev, + "Buffer ioctl: Buffer get. size=%u, offset=%u\n", + uapi.size, uapi.offset); if (copy_to_user(udata, &uapi, sizeof(uapi))) break; @@ -252,10 +256,10 @@ int ethosu_buffer_create(struct ethosu_device *edev, buf->file = fget(ret); fput(buf->file); - dev_info(buf->edev->dev, - "Buffer create. handle=0x%pK, capacity=%zu, cpu_addr=0x%pK, dma_addr=0x%llx, dma_addr_orig=0x%llx, phys_addr=0x%llx\n", - buf, capacity, buf->cpu_addr, buf->dma_addr, - buf->dma_addr_orig, virt_to_phys(buf->cpu_addr)); + dev_dbg(buf->edev->dev, + "Buffer create. file=0x%pK, fd=%d, buf=0x%pK, capacity=%zu, cpu_addr=0x%pK, dma_addr=0x%llx, dma_addr_orig=0x%llx, phys_addr=0x%llx\n", + buf->file, ret, buf, capacity, buf->cpu_addr, buf->dma_addr, + buf->dma_addr_orig, virt_to_phys(buf->cpu_addr)); return ret; diff --git a/drivers/firmware/ethosu/ethosu_buffer.h b/drivers/firmware/ethosu/ethosu_buffer.h index 14f26c2b0a9d..7a21fcb36b2d 100644 --- a/drivers/firmware/ethosu/ethosu_buffer.h +++ b/drivers/firmware/ethosu/ethosu_buffer.h @@ -1,5 +1,5 @@ /* - * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * Copyright (c) 2020,2022 ARM Limited. * * This program is free software and is provided to you under the terms of the * GNU General Public License version 2 as published by the Free Software diff --git a/drivers/firmware/ethosu/ethosu_capabilities.c b/drivers/firmware/ethosu/ethosu_capabilities.c new file mode 100644 index 000000000000..5d7818feafe1 --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_capabilities.c @@ -0,0 +1,157 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Arm Limited. + */ + +/**************************************************************************** + * Includes + ****************************************************************************/ + +#include "ethosu_capabilities.h" + +#include "ethosu_device.h" + +#include +#include +#include + +/**************************************************************************** + * Defines + ****************************************************************************/ + +#define CAPABILITIES_RESP_TIMEOUT_MS 2000 + +/**************************************************************************** + * Functions + ****************************************************************************/ + +static inline int ethosu_capabilities_send(struct ethosu_capabilities *cap) +{ + return ethosu_rpmsg_capabilities_request(&cap->edev->erp, + &cap->msg); +} + +static void ethosu_capabilities_fail(struct ethosu_rpmsg_msg *msg) +{ + struct ethosu_capabilities *cap = + container_of(msg, typeof(*cap), msg); + + if (completion_done(&cap->done)) + return; + + cap->errno = -EFAULT; + complete(&cap->done); +} + +static int ethosu_capabilities_resend(struct ethosu_rpmsg_msg *msg) +{ + struct ethosu_capabilities *cap = + container_of(msg, typeof(*cap), msg); + + /* Don't resend request if response has already been received */ + if (completion_done(&cap->done)) + return 0; + + /* Resend request */ + return ethosu_capabilities_send(cap); +} + +void ethosu_capability_rsp(struct ethosu_device *edev, + struct ethosu_core_msg_capabilities_rsp *rsp) +{ + int id = (int)rsp->user_arg; + struct ethosu_rpmsg_msg *msg; + struct ethosu_capabilities *cap; + + msg = ethosu_rpmsg_find(&edev->erp, id); + if (IS_ERR(msg)) { + dev_warn(edev->dev, + "Id for capabilities msg not found. id=%d\n", + id); + + return; + } + + cap = container_of(msg, typeof(*cap), msg); + + if (completion_done(&cap->done)) + return; + + cap->uapi->hw_id.version_status = rsp->version_status; + cap->uapi->hw_id.version_minor = rsp->version_minor; + cap->uapi->hw_id.version_major = rsp->version_major; + cap->uapi->hw_id.product_major = rsp->product_major; + cap->uapi->hw_id.arch_patch_rev = rsp->arch_patch_rev; + cap->uapi->hw_id.arch_minor_rev = rsp->arch_minor_rev; + cap->uapi->hw_id.arch_major_rev = rsp->arch_major_rev; + cap->uapi->driver_patch_rev = rsp->driver_patch_rev; + cap->uapi->driver_minor_rev = rsp->driver_minor_rev; + cap->uapi->driver_major_rev = rsp->driver_major_rev; + cap->uapi->hw_cfg.macs_per_cc = rsp->macs_per_cc; + cap->uapi->hw_cfg.cmd_stream_version = rsp->cmd_stream_version; + cap->uapi->hw_cfg.custom_dma = rsp->custom_dma; + + cap->errno = 0; + complete(&cap->done); +} + +int ethosu_capabilities_request(struct ethosu_device *edev, + struct ethosu_uapi_device_capabilities *uapi) +{ + struct ethosu_capabilities *cap; + int ret; + int timeout; + + cap = devm_kzalloc(edev->dev, sizeof(struct ethosu_capabilities), + GFP_KERNEL); + if (!cap) + return -ENOMEM; + + cap->edev = edev; + cap->uapi = uapi; + init_completion(&cap->done); + cap->msg.fail = ethosu_capabilities_fail; + cap->msg.resend = ethosu_capabilities_resend; + + ret = ethosu_rpmsg_register(&cap->edev->erp, &cap->msg); + if (ret < 0) + goto kfree; + + dev_dbg(edev->dev, "Capabilities create. Id=%d, handle=0x%p\n", + cap->msg.id, cap); + + ret = ethosu_capabilities_send(cap); + if (ret != 0) + goto deregister; + + /* Unlock the mutex before going to block on the condition */ + mutex_unlock(&edev->mutex); + + /* wait for response to arrive back */ + timeout = wait_for_completion_timeout(&cap->done, + msecs_to_jiffies(CAPABILITIES_RESP_TIMEOUT_MS)); + + /* take back the mutex before resuming to do anything */ + mutex_lock(&edev->mutex); + + if (timeout == 0) { + dev_warn(edev->dev, "Capabilities response timeout"); + ret = -ETIME; + goto deregister; + } + + if (cap->errno) { + ret = cap->errno; + goto deregister; + } + +deregister: + ethosu_rpmsg_deregister(&cap->edev->erp, &cap->msg); + +kfree: + dev_dbg(cap->edev->dev, "Capabilities destroy. Id=%d, handle=0x%p\n", + cap->msg.id, cap); + devm_kfree(cap->edev->dev, cap); + + return ret; +} diff --git a/drivers/firmware/ethosu/ethosu_capabilities.h b/drivers/firmware/ethosu/ethosu_capabilities.h new file mode 100644 index 000000000000..87ddac3f635c --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_capabilities.h @@ -0,0 +1,47 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Arm Limited. + */ + +#ifndef ETHOSU_CAPABILITIES_H +#define ETHOSU_CAPABILITIES_H + +/**************************************************************************** + * Includes + ****************************************************************************/ + +#include "ethosu_core_interface.h" +#include "ethosu_rpmsg.h" + +#include +#include + +/**************************************************************************** + * Types + ****************************************************************************/ + +struct ethosu_device; +struct ethosu_uapi_device_capabilities; + +/** + * struct ethosu_capabilities - Capabilities internal struct + */ +struct ethosu_capabilities { + struct ethosu_device *edev; + struct completion done; + struct ethosu_uapi_device_capabilities *uapi; + struct ethosu_rpmsg_msg msg; + int errno; +}; + +/**************************************************************************** + * Functions + ****************************************************************************/ + +int ethosu_capabilities_request(struct ethosu_device *edev, + struct ethosu_uapi_device_capabilities *uapi); + +void ethosu_capability_rsp(struct ethosu_device *edev, + struct ethosu_core_msg_capabilities_rsp *rsp); + +#endif diff --git a/drivers/firmware/ethosu/ethosu_core_interface.h b/drivers/firmware/ethosu/ethosu_core_interface.h index 9267e96aec9b..11b695a5219d 100644 --- a/drivers/firmware/ethosu/ethosu_core_interface.h +++ b/drivers/firmware/ethosu/ethosu_core_interface.h @@ -1,5 +1,5 @@ /* - * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * Copyright (c) 2020-2022 Arm Limited. * * This program is free software and is provided to you under the terms of the * GNU General Public License version 2 as published by the Free Software @@ -93,11 +93,24 @@ struct ethosu_core_queue { uint8_t data[]; }; +/** + * enum ethosu_core_status - Status + */ enum ethosu_core_status { ETHOSU_CORE_STATUS_OK, - ETHOSU_CORE_STATUS_ERROR + ETHOSU_CORE_STATUS_ERROR, + ETHOSU_CORE_STATUS_RUNNING, + ETHOSU_CORE_STATUS_REJECTED, + ETHOSU_CORE_STATUS_ABORTED, + ETHOSU_CORE_STATUS_ABORTING, }; +/** + * struct ethosu_core_buffer - Buffer descriptor + * + * Pointer and size to a buffer within the Ethos-U + * address space. + */ struct ethosu_core_buffer { uint32_t ptr; uint32_t size; @@ -115,6 +128,9 @@ struct ethosu_core_inference_req { uint32_t inference_type; }; +/** + * struct ethosu_core_inference_rsp - Inference response + */ struct ethosu_core_inference_rsp { uint64_t user_arg; uint32_t ofm_count; diff --git a/drivers/firmware/ethosu/ethosu_device.c b/drivers/firmware/ethosu/ethosu_device.c index 1627f25c8a95..6643a7aaad5b 100644 --- a/drivers/firmware/ethosu/ethosu_device.c +++ b/drivers/firmware/ethosu/ethosu_device.c @@ -1,5 +1,5 @@ /* - * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * Copyright (c) 2020-2022 Arm Limited. * Copyright 2020-2022 NXP * * This program is free software and is provided to you under the terms of the @@ -27,11 +27,11 @@ #include "ethosu_buffer.h" #include "ethosu_core_interface.h" +#include "ethosu_capabilities.h" #include "ethosu_inference.h" #include "ethosu_network.h" #include "uapi/ethosu.h" -#include #include #include #include @@ -48,8 +48,6 @@ #define DMA_ADDR_BITS 32 /* Number of address bits */ -#define CAPABILITIES_RESP_TIMEOUT_MS 2000 - #define ETHOSU_FIRMWARE_NAME "ethosu_firmware" /**************************************************************************** @@ -60,70 +58,6 @@ * Functions ****************************************************************************/ -static void ethosu_capabilities_destroy(struct kref *kref) -{ - struct ethosu_capabilities *cap = - container_of(kref, struct ethosu_capabilities, refcount); - - list_del(&cap->list); - - devm_kfree(cap->edev->dev, cap); -} - -static int ethosu_capabilities_find(struct ethosu_capabilities *cap, - struct list_head *capabilties_list) -{ - struct ethosu_capabilities *cur; - - list_for_each_entry(cur, capabilties_list, list) { - if (cur == cap) - return 0; - } - - return -EINVAL; -} - -static int ethosu_capability_rsp(struct ethosu_device *edev, - struct ethosu_core_msg_capabilities_rsp *msg) -{ - struct ethosu_capabilities *cap; - struct ethosu_uapi_device_capabilities *capabilities; - int ret; - - cap = (struct ethosu_capabilities *)msg->user_arg; - ret = ethosu_capabilities_find(cap, &edev->capabilities_list); - if (0 != ret) { - dev_warn(edev->dev, - "Handle not found in capabilities list. handle=0x%p\n", - cap); - - /* NOTE: do not call complete or kref_put on invalid data! */ - return ret; - } - - capabilities = cap->capabilities; - - capabilities->hw_id.version_status = msg->version_status; - capabilities->hw_id.version_minor = msg->version_minor; - capabilities->hw_id.version_major = msg->version_major; - capabilities->hw_id.product_major = msg->product_major; - capabilities->hw_id.arch_patch_rev = msg->arch_patch_rev; - capabilities->hw_id.arch_minor_rev = msg->arch_minor_rev; - capabilities->hw_id.arch_major_rev = msg->arch_major_rev; - capabilities->driver_patch_rev = msg->driver_patch_rev; - capabilities->driver_minor_rev = msg->driver_minor_rev; - capabilities->driver_major_rev = msg->driver_major_rev; - capabilities->hw_cfg.macs_per_cc = msg->macs_per_cc; - capabilities->hw_cfg.cmd_stream_version = msg->cmd_stream_version; - capabilities->hw_cfg.custom_dma = msg->custom_dma; - - complete(&cap->done); - - kref_put(&cap->refcount, ethosu_capabilities_destroy); - - return 0; -} - /* Incoming messages */ static int ethosu_handle_msg(struct ethosu_device *edev, void *data) { @@ -158,14 +92,14 @@ static int ethosu_handle_msg(struct ethosu_device *edev, void *data) ret = -EBADMSG; break; case ETHOSU_CORE_MSG_PING: - dev_info(edev->dev, "Msg: Ping\n"); + dev_dbg(edev->dev, "Msg: Ping\n"); ret = ethosu_rpmsg_pong(&edev->erp); break; case ETHOSU_CORE_MSG_PONG: - dev_info(edev->dev, "Msg: Pong\n"); + dev_dbg(edev->dev, "Msg: Pong\n"); break; case ETHOSU_CORE_MSG_POWER_RSP: - dev_info(edev->dev, "Msg: Power response\n"); + dev_dbg(edev->dev, "Msg: Power response\n"); break; case ETHOSU_CORE_MSG_INFERENCE_RSP: if (header->length != sizeof(struct ethosu_core_inference_rsp)) { @@ -176,10 +110,10 @@ static int ethosu_handle_msg(struct ethosu_device *edev, void *data) break; } - dev_info(edev->dev, - "Msg: Inference response. user_arg=0x%llx, ofm_count=%u, status=%u\n", - rsp->user_arg, rsp->ofm_count, - rsp->status); + dev_dbg(edev->dev, + "Msg: Inference response. user_arg=0x%llx, ofm_count=%u, status=%u\n", + rsp->user_arg, rsp->ofm_count, + rsp->status); ethosu_inference_rsp(edev, rsp); break; case ETHOSU_CORE_MSG_VERSION_RSP: @@ -191,9 +125,9 @@ static int ethosu_handle_msg(struct ethosu_device *edev, void *data) break; } - dev_info(edev->dev, "Msg: Version response v%u.%u.%u\n", - version->major, version->minor, - version->patch); + dev_dbg(edev->dev, "Msg: Version response v%u.%u.%u\n", + version->major, version->minor, + version->patch); /* Check major and minor version match, else return error */ if (version->major != ETHOSU_CORE_MSG_VERSION_MAJOR || @@ -215,24 +149,24 @@ static int ethosu_handle_msg(struct ethosu_device *edev, void *data) break; } - dev_info(edev->dev, - "Msg: Capabilities response ua%llx vs%hhu v%hhu.%hhu p%hhu av%hhu.%hhu.%hhu dv%hhu.%hhu.%hhu mcc%hhu csv%hhu cd%hhu\n", - capabilities->user_arg, - capabilities->version_status, - capabilities->version_major, - capabilities->version_minor, - capabilities->product_major, - capabilities->arch_major_rev, - capabilities->arch_minor_rev, - capabilities->arch_patch_rev, - capabilities->driver_major_rev, - capabilities->driver_minor_rev, - capabilities->driver_patch_rev, - capabilities->macs_per_cc, - capabilities->cmd_stream_version, - capabilities->custom_dma); - - ret = ethosu_capability_rsp(edev, capabilities); + dev_dbg(edev->dev, + "Msg: Capabilities response ua%llx vs%hhu v%hhu.%hhu p%hhu av%hhu.%hhu.%hhu dv%hhu.%hhu.%hhu mcc%hhu csv%hhu cd%hhu\n", + capabilities->user_arg, + capabilities->version_status, + capabilities->version_major, + capabilities->version_minor, + capabilities->product_major, + capabilities->arch_major_rev, + capabilities->arch_minor_rev, + capabilities->arch_patch_rev, + capabilities->driver_major_rev, + capabilities->driver_minor_rev, + capabilities->driver_patch_rev, + capabilities->macs_per_cc, + capabilities->cmd_stream_version, + capabilities->custom_dma); + + ethosu_capability_rsp(edev, capabilities); break; default: /* This should not happen due to version checks */ @@ -255,7 +189,7 @@ static int ethosu_open(struct inode *inode, file->private_data = edev; - dev_info(edev->dev, "Opening device node.\n"); + dev_dbg(edev->dev, "Device open. file=0x%pK\n", file); if (of_property_read_u32(edev->dev->of_node, "fsl,cm33-proc", &rproc_phandle)) { @@ -287,64 +221,6 @@ static int ethosu_open(struct inode *inode, return nonseekable_open(inode, file); } -static int ethosu_send_capabilities_request(struct ethosu_device *edev, - void __user *udata) -{ - struct ethosu_uapi_device_capabilities uapi; - struct ethosu_capabilities *cap; - int ret; - int timeout; - - cap = devm_kzalloc(edev->dev, sizeof(struct ethosu_capabilities), - GFP_KERNEL); - if (!cap) - return -ENOMEM; - - cap->edev = edev; - cap->capabilities = &uapi; - kref_init(&cap->refcount); - init_completion(&cap->done); - list_add(&cap->list, &edev->capabilities_list); - - ret = ethosu_rpmsg_capabilities_request(&edev->erp, cap); - if (0 != ret) - goto put_kref; - - /* - * Increase ref counter since we sent the pointer out to - * response handler thread. That thread is responsible to - * decrease the ref counter before exiting. So the memory - * can be freed. - * - * NOTE: if no response is received back, the memory is leaked. - */ - kref_get(&cap->refcount); - /* Unlock the mutex before going to block on the condition */ - mutex_unlock(&edev->mutex); - /* wait for response to arrive back */ - timeout = wait_for_completion_timeout(&cap->done, - msecs_to_jiffies( - CAPABILITIES_RESP_TIMEOUT_MS)); - /* take back the mutex before resuming to do anything */ - ret = mutex_lock_interruptible(&edev->mutex); - if (0 != ret) - goto put_kref; - - if (0 == timeout /* timed out*/) { - dev_warn(edev->dev, - "Msg: Capabilities response lost - timeout\n"); - ret = -EIO; - goto put_kref; - } - - ret = copy_to_user(udata, &uapi, sizeof(uapi)) ? -EFAULT : 0; - -put_kref: - kref_put(&cap->refcount, ethosu_capabilities_destroy); - - return ret; -} - static long ethosu_ioctl(struct file *file, unsigned int cmd, unsigned long arg) @@ -357,32 +233,41 @@ static long ethosu_ioctl(struct file *file, if (ret) return ret; - dev_info(edev->dev, "Ioctl. cmd=%u, arg=%lu\n", cmd, arg); + dev_dbg(edev->dev, "Device ioctl. file=0x%pK, cmd=0x%x, arg=0x%lx\n", + file, cmd, arg); switch (cmd) { case ETHOSU_IOCTL_VERSION_REQ: - dev_info(edev->dev, "Ioctl: Send version request\n"); + dev_dbg(edev->dev, "Device ioctl: Send version request\n"); ret = ethosu_rpmsg_version_request(&edev->erp); break; - case ETHOSU_IOCTL_CAPABILITIES_REQ: - dev_info(edev->dev, "Ioctl: Send capabilities request\n"); - ret = ethosu_send_capabilities_request(edev, udata); + case ETHOSU_IOCTL_CAPABILITIES_REQ: { + struct ethosu_uapi_device_capabilities uapi; + + dev_dbg(edev->dev, + "Device ioctl: Send capabilities request\n"); + + ret = ethosu_capabilities_request(edev, &uapi); + if (ret) + break; + + ret = copy_to_user(udata, &uapi, sizeof(uapi)) ? -EFAULT : 0; break; + } case ETHOSU_IOCTL_PING: { - dev_info(edev->dev, "Ioctl: Send ping\n"); + dev_dbg(edev->dev, "Device ioctl: Send ping\n"); ret = ethosu_rpmsg_ping(&edev->erp); break; } case ETHOSU_IOCTL_BUFFER_CREATE: { struct ethosu_uapi_buffer_create uapi; - dev_info(edev->dev, "Ioctl: Buffer create\n"); - if (copy_from_user(&uapi, udata, sizeof(uapi))) break; - dev_info(edev->dev, "Ioctl: Buffer. capacity=%u\n", - uapi.capacity); + dev_dbg(edev->dev, + "Device ioctl: Buffer create. capacity=%u\n", + uapi.capacity); ret = ethosu_buffer_create(edev, uapi.capacity); break; @@ -393,7 +278,9 @@ static long ethosu_ioctl(struct file *file, if (copy_from_user(&uapi, udata, sizeof(uapi))) break; - dev_info(edev->dev, "Ioctl: Network. fd=%u\n", uapi.fd); + dev_dbg(edev->dev, + "Device ioctl: Network create. fd=%u\n", + uapi.fd); ret = ethosu_network_create(edev, &uapi); break; @@ -444,8 +331,6 @@ int ethosu_dev_init(struct ethosu_device *edev, edev->class = class; edev->devt = devt; mutex_init(&edev->mutex); - INIT_LIST_HEAD(&edev->capabilities_list); - INIT_LIST_HEAD(&edev->inference_list); ret = of_reserved_mem_device_init(edev->dev); if (ret) @@ -474,9 +359,9 @@ int ethosu_dev_init(struct ethosu_device *edev, goto del_cdev; } - dev_info(edev->dev, - "Created Arm Ethos-U device. name=%s, major=%d, minor=%d\n", - dev_name(sysdev), MAJOR(edev->devt), MINOR(edev->devt)); + dev_dbg(edev->dev, + "Created Arm Ethos-U device. name=%s, major=%d, minor=%d\n", + dev_name(sysdev), MAJOR(edev->devt), MINOR(edev->devt)); return 0; @@ -498,6 +383,4 @@ void ethosu_dev_deinit(struct ethosu_device *edev) device_destroy(edev->class, edev->cdev.dev); cdev_del(&edev->cdev); of_reserved_mem_device_release(edev->dev); - - dev_info(edev->dev, "%s\n", __FUNCTION__); } diff --git a/drivers/firmware/ethosu/ethosu_device.h b/drivers/firmware/ethosu/ethosu_device.h index da470241b6a1..d027d8d9b11d 100644 --- a/drivers/firmware/ethosu/ethosu_device.h +++ b/drivers/firmware/ethosu/ethosu_device.h @@ -1,5 +1,5 @@ /* - * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * Copyright (c) 2020-2022 Arm Limited. * * This program is free software and is provided to you under the terms of the * GNU General Public License version 2 as published by the Free Software @@ -32,8 +32,6 @@ #include #include #include -#include -#include /**************************************************************************** * Types @@ -49,22 +47,9 @@ struct ethosu_device { dev_t devt; struct mutex mutex; struct ethosu_rpmsg erp; - struct list_head capabilities_list; - struct list_head inference_list; bool open; }; -/** - * struct ethosu_capabilities - Capabilities internal struct - */ -struct ethosu_capabilities { - struct ethosu_device *edev; - struct completion done; - struct kref refcount; - struct ethosu_uapi_device_capabilities *capabilities; - struct list_head list; -}; - /**************************************************************************** * Functions ****************************************************************************/ diff --git a/drivers/firmware/ethosu/ethosu_driver.c b/drivers/firmware/ethosu/ethosu_driver.c index d3306ef625ae..59ae79256564 100644 --- a/drivers/firmware/ethosu/ethosu_driver.c +++ b/drivers/firmware/ethosu/ethosu_driver.c @@ -1,5 +1,5 @@ /* - * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * Copyright (c) 2020-2022 Arm Limited. * Copyright 2020-2022 NXP * * This program is free software and is provided to you under the terms of the @@ -60,7 +60,7 @@ static int ethosu_pdev_probe(struct platform_device *pdev) int minor; int ret; - dev_info(&pdev->dev, "Probe\n"); + dev_dbg(&pdev->dev, "Probe\n"); minor = find_first_zero_bit(minors, MINOR_COUNT); if (minor >= MINOR_COUNT) { diff --git a/drivers/firmware/ethosu/ethosu_inference.c b/drivers/firmware/ethosu/ethosu_inference.c index b5e94e19bce0..947988507792 100644 --- a/drivers/firmware/ethosu/ethosu_inference.c +++ b/drivers/firmware/ethosu/ethosu_inference.c @@ -1,5 +1,5 @@ /* - * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * Copyright (c) 2020,2022 Arm Limited. * * This program is free software and is provided to you under the terms of the * GNU General Public License version 2 as published by the Free Software @@ -71,6 +71,18 @@ static const char *status_to_string(const enum ethosu_uapi_status status) case ETHOSU_UAPI_STATUS_ERROR: { return "Error"; } + case ETHOSU_UAPI_STATUS_RUNNING: { + return "Running"; + } + case ETHOSU_UAPI_STATUS_REJECTED: { + return "Rejected"; + } + case ETHOSU_UAPI_STATUS_ABORTED: { + return "Aborted"; + } + case ETHOSU_UAPI_STATUS_ABORTING: { + return "Aborting"; + } default: { return "Unknown"; } @@ -81,40 +93,82 @@ static int ethosu_inference_send(struct ethosu_inference *inf) { int ret; - if (inf->pending) - return -EINVAL; - inf->status = ETHOSU_UAPI_STATUS_ERROR; - ret = ethosu_rpmsg_inference(&inf->edev->erp, inf, - inf->ifm_count, inf->ifm, - inf->ofm_count, inf->ofm, - inf->net->buf, - inf->pmu_event_config, - ETHOSU_PMU_EVENT_MAX, - inf->pmu_cycle_counter_enable, - inf->inference_type); - if (ret) + ret = ethosu_rpmsg_inference(&inf->edev->erp, &inf->msg, + inf->ifm_count, inf->ifm, + inf->ofm_count, inf->ofm, + inf->net->buf, + inf->pmu_event_config, + ETHOSU_PMU_EVENT_MAX, + inf->pmu_cycle_counter_enable, + inf->inference_type); + if (ret) { + dev_warn(inf->edev->dev, + "Failed to send inference request. inf=0x%pK, ret=%d", + inf, ret); + return ret; + } - inf->pending = true; + inf->status = ETHOSU_UAPI_STATUS_RUNNING; ethosu_inference_get(inf); return 0; } -static int ethosu_inference_find(struct ethosu_inference *inf, - struct list_head *inference_list) +static void ethosu_inference_fail(struct ethosu_rpmsg_msg *msg) +{ + struct ethosu_inference *inf = + container_of(msg, typeof(*inf), msg); + int ret; + + if (inf->done) + return; + + /* Decrement reference count if inference was pending response */ + ret = ethosu_inference_put(inf); + if (ret) + return; + + /* Set status accordingly to the inference state */ + inf->status = inf->status == ETHOSU_UAPI_STATUS_ABORTING ? + ETHOSU_UAPI_STATUS_ABORTED : + ETHOSU_UAPI_STATUS_ERROR; + /* Mark it done and wake up the waiting process */ + inf->done = true; + wake_up_interruptible(&inf->waitq); +} + +static int ethosu_inference_resend(struct ethosu_rpmsg_msg *msg) { - struct ethosu_inference *cur; + struct ethosu_inference *inf = + container_of(msg, typeof(*inf), msg); + int ret; - list_for_each_entry(cur, inference_list, list) { - if (cur == inf) - return 0; + /* Don't resend request if response has already been received */ + if (inf->done) + return 0; + + /* If marked as ABORTING simply fail it and return */ + if (inf->status == ETHOSU_UAPI_STATUS_ABORTING) { + ethosu_inference_fail(msg); + + return 0; } - return -EINVAL; + /* Decrement reference count for pending request */ + ret = ethosu_inference_put(inf); + if (ret) + return 0; + + /* Resend request */ + ret = ethosu_inference_send(inf); + if (ret) + return ret; + + return 0; } static bool ethosu_inference_verify(struct file *file) @@ -127,11 +181,11 @@ static void ethosu_inference_kref_destroy(struct kref *kref) struct ethosu_inference *inf = container_of(kref, struct ethosu_inference, kref); - dev_info(inf->edev->dev, - "Inference destroy. handle=0x%pK, status=%d\n", - inf, inf->status); + dev_dbg(inf->edev->dev, + "Inference destroy. inf=0x%pK, status=%d, ifm_count=%u, ofm_count=%u", + inf, inf->status, inf->ifm_count, inf->ofm_count); - list_del(&inf->list); + ethosu_rpmsg_deregister(&inf->edev->erp, &inf->msg); while (inf->ifm_count-- > 0) ethosu_buffer_put(inf->ifm[inf->ifm_count]); @@ -148,9 +202,9 @@ static int ethosu_inference_release(struct inode *inode, { struct ethosu_inference *inf = file->private_data; - dev_info(inf->edev->dev, - "Inference release. handle=0x%pK, status=%d\n", - inf, inf->status); + dev_dbg(inf->edev->dev, + "Inference release. file=0x%pK, inf=0x%pK", + file, inf); ethosu_inference_put(inf); @@ -165,7 +219,7 @@ static unsigned int ethosu_inference_poll(struct file *file, poll_wait(file, &inf->waitq, wait); - if (!inf->pending) + if (inf->done) ret |= POLLIN; return ret; @@ -183,7 +237,9 @@ static long ethosu_inference_ioctl(struct file *file, if (ret) return ret; - dev_info(inf->edev->dev, "Ioctl: cmd=%u, arg=%lu\n", cmd, arg); + dev_dbg(inf->edev->dev, + "Inference ioctl: file=0x%pK, inf=0x%pK, cmd=0x%x, arg=%lu", + file, inf, cmd, arg); switch (cmd) { case ETHOSU_IOCTL_INFERENCE_STATUS: { @@ -202,16 +258,16 @@ static long ethosu_inference_ioctl(struct file *file, uapi.pmu_config.cycle_count = inf->pmu_cycle_counter_enable; uapi.pmu_count.cycle_count = inf->pmu_cycle_counter_count; - dev_info(inf->edev->dev, - "Ioctl: Inference status. status=%s (%d)\n", - status_to_string(uapi.status), uapi.status); + dev_dbg(inf->edev->dev, + "Inference ioctl: Inference status. status=%s (%d)\n", + status_to_string(uapi.status), uapi.status); ret = copy_to_user(udata, &uapi, sizeof(uapi)) ? -EFAULT : 0; break; } default: { - dev_err(inf->edev->dev, "Invalid ioctl. cmd=%u, arg=%lu", + dev_err(inf->edev->dev, "Invalid ioctl. cmd=%u, arg=%lu\n", cmd, arg); break; } @@ -231,9 +287,19 @@ int ethosu_inference_create(struct ethosu_device *edev, int fd; int ret = -ENOMEM; + if (uapi->ifm_count > ETHOSU_FD_MAX || + uapi->ofm_count > ETHOSU_FD_MAX) { + dev_warn(edev->dev, + "Too many IFM and/or OFM buffers for inference. ifm_count=%u, ofm_count=%u", + uapi->ifm_count, uapi->ofm_count); + + return -EFAULT; + } + inf = devm_kzalloc(edev->dev, sizeof(*inf), GFP_KERNEL); if (!inf) return -ENOMEM; + switch (uapi->inference_type) { case ETHOSU_UAPI_INFERENCE_MODEL: inf->inference_type = ETHOSU_CORE_INFERENCE_MODEL; @@ -248,10 +314,17 @@ int ethosu_inference_create(struct ethosu_device *edev, inf->edev = edev; inf->net = net; - inf->pending = false; + inf->done = false; inf->status = ETHOSU_UAPI_STATUS_ERROR; kref_init(&inf->kref); init_waitqueue_head(&inf->waitq); + inf->msg.fail = ethosu_inference_fail; + inf->msg.resend = ethosu_inference_resend; + + /* Add inference to pending list */ + ret = ethosu_rpmsg_register(&edev->erp, &inf->msg); + if (ret < 0) + goto kfree; /* Get pointer to IFM buffers */ for (i = 0; i < uapi->ifm_count; i++) { @@ -276,10 +349,10 @@ int ethosu_inference_create(struct ethosu_device *edev, } /* Configure PMU and cycle counter */ - dev_info(inf->edev->dev, - "Configuring events for PMU. events=[%u, %u, %u, %u]\n", - uapi->pmu_config.events[0], uapi->pmu_config.events[1], - uapi->pmu_config.events[2], uapi->pmu_config.events[3]); + dev_dbg(inf->edev->dev, + "Configuring events for PMU. events=[%u, %u, %u, %u]\n", + uapi->pmu_config.events[0], uapi->pmu_config.events[1], + uapi->pmu_config.events[2], uapi->pmu_config.events[3]); /* Configure events and reset count for all events */ for (i = 0; i < ETHOSU_PMU_EVENT_MAX; i++) { @@ -288,7 +361,7 @@ int ethosu_inference_create(struct ethosu_device *edev, } if (uapi->pmu_config.cycle_count) - dev_info(inf->edev->dev, "Enabling cycle counter\n"); + dev_dbg(inf->edev->dev, "Enabling cycle counter\n"); /* Configure cycle counter and reset any previous count */ inf->pmu_cycle_counter_enable = uapi->pmu_config.cycle_count; @@ -297,6 +370,11 @@ int ethosu_inference_create(struct ethosu_device *edev, /* Increment network reference count */ ethosu_network_get(net); + /* Send inference request to Arm Ethos-U subsystem */ + ret = ethosu_inference_send(inf); + if (ret) + goto put_net; + /* Create file descriptor */ ret = fd = anon_inode_getfd("ethosu-inference", ðosu_inference_fops, inf, O_RDWR | O_CLOEXEC); @@ -307,14 +385,9 @@ int ethosu_inference_create(struct ethosu_device *edev, inf->file = fget(ret); fput(inf->file); - /* Add inference to inference list */ - list_add(&inf->list, &edev->inference_list); - - /* Send inference request to Arm Ethos-U subsystem */ - (void)ethosu_inference_send(inf); - - dev_info(edev->dev, "Inference create. handle=0x%pK, fd=%d", - inf, fd); + dev_dbg(edev->dev, + "Inference create. file=0x%pK, fd=%d, inf=0x%p, net=0x%pK, msg.id=0x%x", + inf->file, fd, inf, inf->net, inf->msg.id); return fd; @@ -329,6 +402,7 @@ int ethosu_inference_create(struct ethosu_device *edev, while (inf->ifm_count-- > 0) ethosu_buffer_put(inf->ifm[inf->ifm_count]); +kfree: devm_kfree(edev->dev, inf); return ret; @@ -361,29 +435,30 @@ void ethosu_inference_get(struct ethosu_inference *inf) kref_get(&inf->kref); } -void ethosu_inference_put(struct ethosu_inference *inf) +int ethosu_inference_put(struct ethosu_inference *inf) { - kref_put(&inf->kref, ðosu_inference_kref_destroy); + return kref_put(&inf->kref, ðosu_inference_kref_destroy); } void ethosu_inference_rsp(struct ethosu_device *edev, struct ethosu_core_inference_rsp *rsp) { - struct ethosu_inference *inf = - (struct ethosu_inference *)rsp->user_arg; + int id = (int)rsp->user_arg; + struct ethosu_rpmsg_msg *msg; + struct ethosu_inference *inf; int ret; int i; - ret = ethosu_inference_find(inf, &edev->inference_list); - if (ret) { + msg = ethosu_rpmsg_find(&edev->erp, id); + if (IS_ERR(msg)) { dev_warn(edev->dev, - "Handle not found in inference list. handle=0x%p\n", - rsp); + "Id for inference msg not found. Id=%d\n", + id); return; } - inf->pending = false; + inf = container_of(msg, typeof(*inf), msg); if (rsp->status == ETHOSU_CORE_STATUS_OK && inf->ofm_count <= ETHOSU_CORE_BUFFER_MAX) { @@ -400,29 +475,37 @@ void ethosu_inference_rsp(struct ethosu_device *edev, if (ret) inf->status = ETHOSU_UAPI_STATUS_ERROR; } + } else if (rsp->status == ETHOSU_CORE_STATUS_REJECTED) { + inf->status = ETHOSU_UAPI_STATUS_REJECTED; + } else if (rsp->status == ETHOSU_CORE_STATUS_ABORTED) { + inf->status = ETHOSU_UAPI_STATUS_ABORTED; } else { inf->status = ETHOSU_UAPI_STATUS_ERROR; } - for (i = 0; i < ETHOSU_CORE_PMU_MAX; i++) { - inf->pmu_event_config[i] = rsp->pmu_event_config[i]; - inf->pmu_event_count[i] = rsp->pmu_event_count[i]; - } + if (inf->status == ETHOSU_UAPI_STATUS_OK) { + for (i = 0; i < ETHOSU_CORE_PMU_MAX; i++) { + inf->pmu_event_config[i] = rsp->pmu_event_config[i]; + inf->pmu_event_count[i] = rsp->pmu_event_count[i]; + } + + inf->pmu_cycle_counter_enable = rsp->pmu_cycle_counter_enable; + inf->pmu_cycle_counter_count = rsp->pmu_cycle_counter_count; - inf->pmu_cycle_counter_enable = rsp->pmu_cycle_counter_enable; - inf->pmu_cycle_counter_count = rsp->pmu_cycle_counter_count; + dev_dbg(edev->dev, + "PMU events. config=[%u, %u, %u, %u], count=[%u, %u, %u, %u]\n", + inf->pmu_event_config[0], inf->pmu_event_config[1], + inf->pmu_event_config[2], inf->pmu_event_config[3], + inf->pmu_event_count[0], inf->pmu_event_count[1], + inf->pmu_event_count[2], inf->pmu_event_count[3]); - dev_info(edev->dev, - "PMU events. config=[%u, %u, %u, %u], count=[%u, %u, %u, %u]\n", - inf->pmu_event_config[0], inf->pmu_event_config[1], - inf->pmu_event_config[2], inf->pmu_event_config[3], - inf->pmu_event_count[0], inf->pmu_event_count[1], - inf->pmu_event_count[2], inf->pmu_event_count[3]); + dev_dbg(edev->dev, + "PMU cycle counter. enable=%u, count=%llu\n", + inf->pmu_cycle_counter_enable, + inf->pmu_cycle_counter_count); + } - dev_info(edev->dev, - "PMU cycle counter. enable=%u, count=%llu\n", - inf->pmu_cycle_counter_enable, - inf->pmu_cycle_counter_count); + inf->done = true; wake_up_interruptible(&inf->waitq); ethosu_inference_put(inf); diff --git a/drivers/firmware/ethosu/ethosu_inference.h b/drivers/firmware/ethosu/ethosu_inference.h index 8414eadbda92..311c727b2897 100644 --- a/drivers/firmware/ethosu/ethosu_inference.h +++ b/drivers/firmware/ethosu/ethosu_inference.h @@ -1,5 +1,5 @@ /* - * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * Copyright (c) 2020,2022 ARM Limited. * * This program is free software and is provided to you under the terms of the * GNU General Public License version 2 as published by the Free Software @@ -25,6 +25,7 @@ * Includes ****************************************************************************/ +#include "ethosu_rpmsg.h" #include "uapi/ethosu.h" #include @@ -48,34 +49,35 @@ struct file; * @file: File handle * @kref: Reference counter * @waitq: Wait queue + * @done: Wait condition is done * @ifm: Pointer to IFM buffer * @ofm: Pointer to OFM buffer * @net: Pointer to network - * @pending: Pending response from the firmware * @status: Inference status * @pmu_event_config: PMU event configuration * @pmu_event_count: PMU event count after inference * @pmu_cycle_counter_enable: PMU cycle counter config * @pmu_cycle_counter_count: PMU cycle counter count after inference + * @msg: Rpmsg message */ struct ethosu_inference { struct ethosu_device *edev; struct file *file; struct kref kref; wait_queue_head_t waitq; + bool done; uint32_t ifm_count; struct ethosu_buffer *ifm[ETHOSU_FD_MAX]; uint32_t ofm_count; struct ethosu_buffer *ofm[ETHOSU_FD_MAX]; struct ethosu_network *net; - bool pending; enum ethosu_uapi_status status; uint8_t pmu_event_config[ETHOSU_PMU_EVENT_MAX]; uint32_t pmu_event_count[ETHOSU_PMU_EVENT_MAX]; uint32_t pmu_cycle_counter_enable; uint64_t pmu_cycle_counter_count; uint32_t inference_type; - struct list_head list; + struct ethosu_rpmsg_msg msg; }; /**************************************************************************** @@ -109,8 +111,10 @@ void ethosu_inference_get(struct ethosu_inference *inf); /** * ethosu_inference_put() - Put inference + * + * Return: 1 if object was removed, else 0. */ -void ethosu_inference_put(struct ethosu_inference *inf); +int ethosu_inference_put(struct ethosu_inference *inf); /** * ethosu_inference_rsp() - Handle inference response diff --git a/drivers/firmware/ethosu/ethosu_network.c b/drivers/firmware/ethosu/ethosu_network.c index 4d68f0537e38..a7249619291b 100644 --- a/drivers/firmware/ethosu/ethosu_network.c +++ b/drivers/firmware/ethosu/ethosu_network.c @@ -1,5 +1,5 @@ /* - * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * Copyright (c) 2020,2022 Arm Limited. * * This program is free software and is provided to you under the terms of the * GNU General Public License version 2 as published by the Free Software @@ -67,7 +67,7 @@ static void ethosu_network_destroy(struct kref *kref) struct ethosu_network *net = container_of(kref, struct ethosu_network, kref); - dev_info(net->edev->dev, "Network destroy. handle=0x%pK\n", net); + dev_dbg(net->edev->dev, "Network destroy. net=0x%pK\n", net); ethosu_buffer_put(net->buf); devm_kfree(net->edev->dev, net); @@ -78,7 +78,8 @@ static int ethosu_network_release(struct inode *inode, { struct ethosu_network *net = file->private_data; - dev_info(net->edev->dev, "Network release. handle=0x%pK\n", net); + dev_dbg(net->edev->dev, "Network release. file=0x%pK, net=0x%pK\n", + file, net); ethosu_network_put(net); @@ -97,7 +98,9 @@ static long ethosu_network_ioctl(struct file *file, if (ret) return ret; - dev_info(net->edev->dev, "Ioctl: cmd=%u, arg=%lu\n", cmd, arg); + dev_dbg(net->edev->dev, + "Network ioctl: file=0x%pK, net=0x%pK, cmd=0x%x, arg=0x%lx\n", + file, net, cmd, arg); switch (cmd) { case ETHOSU_IOCTL_INFERENCE_CREATE: { @@ -106,9 +109,9 @@ static long ethosu_network_ioctl(struct file *file, if (copy_from_user(&uapi, udata, sizeof(uapi))) break; - dev_info(net->edev->dev, - "Ioctl: Inference. ifm_fd=%u, ofm_fd=%u\n", - uapi.ifm_fd[0], uapi.ofm_fd[0]); + dev_dbg(net->edev->dev, + "Network ioctl: Inference. ifm_fd=%u, ofm_fd=%u\n", + uapi.ifm_fd[0], uapi.ofm_fd[0]); ret = ethosu_inference_create(net->edev, net, &uapi); break; @@ -154,8 +157,9 @@ int ethosu_network_create(struct ethosu_device *edev, net->file = fget(ret); fput(net->file); - dev_info(edev->dev, "Network create. handle=0x%pK", - net); + dev_dbg(edev->dev, + "Network create. file=0x%pK, fd=%d, net=0x%pK, buf=0x%pK", + net->file, ret, net, net->buf); return ret; diff --git a/drivers/firmware/ethosu/ethosu_network.h b/drivers/firmware/ethosu/ethosu_network.h index bb70afcb2572..4236e1586efb 100644 --- a/drivers/firmware/ethosu/ethosu_network.h +++ b/drivers/firmware/ethosu/ethosu_network.h @@ -1,5 +1,5 @@ /* - * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * Copyright (c) 2020,2022 Arm Limited. * * This program is free software and is provided to you under the terms of the * GNU General Public License version 2 as published by the Free Software diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.c b/drivers/firmware/ethosu/ethosu_rpmsg.c index a0320b8407d1..a92439b81c96 100644 --- a/drivers/firmware/ethosu/ethosu_rpmsg.c +++ b/drivers/firmware/ethosu/ethosu_rpmsg.c @@ -32,6 +32,61 @@ static void ethosu_core_set_capacity(struct ethosu_buffer *buf, cbuf->size = (uint32_t)buf->capacity - buf->offset - buf->size; } +int ethosu_rpmsg_register(struct ethosu_rpmsg *erp, + struct ethosu_rpmsg_msg *msg) +{ + msg->id = idr_alloc_cyclic(&erp->msg_idr, msg, 0, INT_MAX, GFP_KERNEL); + if (msg->id < 0) + return msg->id; + + return 0; +} + +void ethosu_rpmsg_deregister(struct ethosu_rpmsg *erp, + struct ethosu_rpmsg_msg *msg) +{ + idr_remove(&erp->msg_idr, msg->id); +} + +struct ethosu_rpmsg_msg *ethosu_rpmsg_find(struct ethosu_rpmsg *erp, + int msg_id) +{ + struct ethosu_rpmsg_msg *ptr = + (struct ethosu_rpmsg_msg *)idr_find(&erp->msg_idr, msg_id); + + if (!ptr) + return ERR_PTR(-EINVAL); + + return ptr; +} + +void ethosu_rpmsg_fail(struct ethosu_rpmsg *erp) +{ + struct ethosu_rpmsg_msg *cur; + int id; + + idr_for_each_entry(&erp->msg_idr, cur, id) { + cur->fail(cur); + } +} + +void ethosu_rpmsg_resend(struct ethosu_rpmsg *erp) +{ + struct ethosu_rpmsg_msg *cur; + struct rpmsg_device *rpdev = erp->rpdev; + int id; + int ret; + + idr_for_each_entry(&erp->msg_idr, cur, id) { + ret = cur->resend(cur); + if (ret) { + dev_warn(&rpdev->dev, "Failed to resend msg. ret=%d", + ret); + cur->fail(cur); + } + } +} + static int ethosu_rpmsg_send(struct ethosu_rpmsg *erp, uint32_t type) { struct ethosu_core_msg msg; @@ -69,7 +124,8 @@ int ethosu_rpmsg_version_request(struct ethosu_rpmsg *erp) return ethosu_rpmsg_send(erp, ETHOSU_CORE_MSG_VERSION_REQ); } -int ethosu_rpmsg_capabilities_request(struct ethosu_rpmsg *erp, void *user_arg) +int ethosu_rpmsg_capabilities_request(struct ethosu_rpmsg *erp, + struct ethosu_rpmsg_msg *rpmsg) { struct ethosu_core_msg msg = { .magic = ETHOSU_CORE_MSG_MAGIC, @@ -77,7 +133,7 @@ int ethosu_rpmsg_capabilities_request(struct ethosu_rpmsg *erp, void *user_arg) .length = sizeof(struct ethosu_core_capabilities_req) }; struct ethosu_core_capabilities_req req = { - .user_arg = (uint64_t)user_arg + .user_arg = rpmsg->id }; struct rpmsg_device *rpdev = erp->rpdev; uint8_t data[sizeof(struct ethosu_core_msg) + @@ -128,7 +184,7 @@ int ethosu_rpmsg_power_request(struct ethosu_rpmsg *erp, } int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp, - void *user_arg, + struct ethosu_rpmsg_msg *rpmsg, uint32_t ifm_count, struct ethosu_buffer **ifm, uint32_t ofm_count, @@ -158,7 +214,7 @@ int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp, return -EINVAL; } - req.user_arg = (uint64_t)user_arg; + req.user_arg = rpmsg->id; req.ifm_count = ifm_count; req.ofm_count = ofm_count; req.pmu_cycle_counter_enable = pmu_cycle_counter_enable; @@ -215,8 +271,8 @@ static int ethosu_rpmsg_probe(struct rpmsg_device *rpdev) grp->rpdev = rpdev; dev_set_drvdata(&rpdev->dev, grp); - dev_info(&rpdev->dev, "new channel: 0x%x -> 0x%x!\n", - rpdev->src, rpdev->dst); + dev_dbg(&rpdev->dev, "new channel: 0x%x -> 0x%x!\n", + rpdev->src, rpdev->dst); ret = rpmsg_send(rpdev->ept, MSG, strlen(MSG)); if (ret) { @@ -229,7 +285,7 @@ static int ethosu_rpmsg_probe(struct rpmsg_device *rpdev) static void ethosu_rpmsg_remove(struct rpmsg_device *rpdev) { - dev_info(&rpdev->dev, "rpmsg ethosu driver is removed\n"); + dev_dbg(&rpdev->dev, "rpmsg ethosu driver is removed\n"); } static struct rpmsg_device_id rpmsg_driver_ethosu_id_table[] = { @@ -252,6 +308,8 @@ int ethosu_rpmsg_init(struct ethosu_rpmsg *erp, grp = erp; erp->callback = callback; erp->user_arg = user_arg; + erp->ping_count = 0; + idr_init(&erp->msg_idr); return register_rpmsg_driver(ðosu_rpmsg_driver); } diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.h b/drivers/firmware/ethosu/ethosu_rpmsg.h index cee3d96c7895..74ee2fdea5cb 100644 --- a/drivers/firmware/ethosu/ethosu_rpmsg.h +++ b/drivers/firmware/ethosu/ethosu_rpmsg.h @@ -6,8 +6,11 @@ #ifndef ETHOSU_RPMSG_H #define ETHOSU_RPMSG_H +#include #include #include +#include + #include "ethosu_core_interface.h" struct device; @@ -22,7 +25,16 @@ struct ethosu_rpmsg { ethosu_rpmsg_cb callback; void *user_arg; struct completion rpmsg_ready; + struct idr msg_idr; + unsigned int ping_count; +}; + +struct ethosu_rpmsg_msg { + int id; + void (*fail)(struct ethosu_rpmsg_msg *msg); + int (*resend)(struct ethosu_rpmsg_msg *msg); }; + /** * ethosu_rpmsg_ping() - Send ping message * @@ -50,7 +62,7 @@ int ethosu_rpmsg_version_request(struct ethosu_rpmsg *erp); * Return: 0 on success, else error code */ int ethosu_rpmsg_capabilities_request(struct ethosu_rpmsg *erp, - void *user_arg); + struct ethosu_rpmsg_msg *rpmsg); /** * ethosu_rpmsg_power_request - Send power request @@ -66,7 +78,7 @@ int ethosu_rpmsg_power_request(struct ethosu_rpmsg *erp, * Return: 0 on success, else error code. */ int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp, - void *user_arg, + struct ethosu_rpmsg_msg *rpmsg, uint32_t ifm_count, struct ethosu_buffer **ifm, uint32_t ofm_count, @@ -82,4 +94,41 @@ int ethosu_rpmsg_init(struct ethosu_rpmsg *erp, ethosu_rpmsg_cb callback, void *user_arg); int ethosu_rpmsg_deinit(struct ethosu_rpmsg *erp); + +/** + * ethosu_rpmsg_register() - Register the ethosu_rpmsg_msg in ethosu_rpmsg + * + * Return: 0 on success, else error code. + */ +int ethosu_rpmsg_register(struct ethosu_rpmsg *erp, + struct ethosu_rpmsg_msg *msg); + +/** + * ethosu_rpmsg_free_id() - Free the id of the ethosu_rpmsg_msg + */ +void ethosu_rpmsg_deregister(struct ethosu_rpmsg *erp, + struct ethosu_rpmsg_msg *msg); + +/** + * ethosu_rpmsg_find() - Find rpmsg message + * + * Return: a valid pointer on success, otherwise an error ptr. + */ +struct ethosu_rpmsg_msg *ethosu_rpmsg_find(struct ethosu_rpmsg *erq, + int msg_id); + +/** + * ethosu_rpmsg_fail() - Fail rpmsg messages + * + * Call fail() callback on all messages in pending list. + */ +void ethosu_rpmsg_fail(struct ethosu_rpmsg *erp); + +/** + * ethosu_rpmsg_resend() - Resend rpmsg messages + * + * Call resend() callback on all messages in pending list. + */ +void ethosu_rpmsg_resend(struct ethosu_rpmsg *erp); + #endif /* ETHOSU_RPMSG_H */ diff --git a/drivers/firmware/ethosu/uapi/ethosu.h b/drivers/firmware/ethosu/uapi/ethosu.h index c4a0df67336c..648cec40473b 100644 --- a/drivers/firmware/ethosu/uapi/ethosu.h +++ b/drivers/firmware/ethosu/uapi/ethosu.h @@ -1,5 +1,5 @@ /* - * (C) COPYRIGHT 2020 ARM Limited. All rights reserved. + * Copyright (c) 2020-2022 Arm Limited. * * This program is free software and is provided to you under the terms of the * GNU General Public License version 2 as published by the Free Software @@ -73,7 +73,11 @@ namespace EthosU { */ enum ethosu_uapi_status { ETHOSU_UAPI_STATUS_OK, - ETHOSU_UAPI_STATUS_ERROR + ETHOSU_UAPI_STATUS_ERROR, + ETHOSU_UAPI_STATUS_RUNNING, + ETHOSU_UAPI_STATUS_REJECTED, + ETHOSU_UAPI_STATUS_ABORTED, + ETHOSU_UAPI_STATUS_ABORTING, }; /** From patchwork Fri Jun 16 05:59:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Alison Wang X-Patchwork-Id: 13282006 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CF714EB64D8 for ; Fri, 16 Jun 2023 05:44:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=a1Tal9UZaFwythO/dQ/T+Z9ZJFkxHZJ9KJ+LzIjVSKQ=; b=qs65++Phx3IbbR zCJfV5ef0HyuVAJ/PWeapgLBxmq/mpbTTd2KzphOyFVVmnrq+tS3ejlt327cCzPssG70eyDYaIspx oVxaP8mImS11P3L4ItMO97AqD/FcUuuqx0BYPKWzFdvpt+AMp7uA51dPcR6hMBG9P2FLB/5iT4JNU rSiDmsIsZLkxJCO2taUa3bQntKhgHQTw9u4wJllYiOpRdV7RdOrh+Pf/5Xmlb/K3gSJDvSRZfwVQO W9QrL2yAd+lzAa5OgKxR8sWZnacjkvMvJOeoIK0emKwk5HnPAiPScqB0NzceyPqZoUA1FI0o0wx2p JA5bOeDwnFjP9SvbOPaQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qA2F1-00GusF-2F; Fri, 16 Jun 2023 05:43:23 +0000 Received: from inva021.nxp.com ([92.121.34.21]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qA2Eo-00Gum0-0y for linux-arm-kernel@lists.infradead.org; Fri, 16 Jun 2023 05:43:15 +0000 Received: from inva021.nxp.com (localhost [127.0.0.1]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id A9E0B20029D; Fri, 16 Jun 2023 07:43:07 +0200 (CEST) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva021.eu-rdc02.nxp.com (Postfix) with ESMTP id 4AABE200231; Fri, 16 Jun 2023 07:43:07 +0200 (CEST) Received: from localhost.localdomain (mega.ap.freescale.net [10.192.208.232]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id EC4D91820F57; Fri, 16 Jun 2023 13:43:05 +0800 (+08) From: Alison Wang To: gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: leoyang.li@nxp.com, xuelin.shi@nxp.com, xiaofeng.ren@nxp.com, feng.guo@nxp.com, Alison Wang Subject: [PATCH 6/8] ethosu: Add core message about network info Date: Fri, 16 Jun 2023 13:59:11 +0800 Message-Id: <20230616055913.2360-7-alison.wang@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230616055913.2360-1-alison.wang@nxp.com> References: <20230616055913.2360-1-alison.wang@nxp.com> MIME-Version: 1.0 X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230615_224310_631103_FAFE7F2D X-CRM114-Status: GOOD ( 26.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add core message that allows user space to fetch information about network models built into the firmware. Signed-off-by: Kristofer Jonsson Signed-off-by: Per Åstrand Signed-off-by: Alison Wang Signed-off-by: Feng Guo Reviewed-by: Peng Fan --- drivers/firmware/ethosu/Makefile | 1 + .../firmware/ethosu/ethosu_core_interface.h | 62 +++++- drivers/firmware/ethosu/ethosu_device.c | 28 ++- drivers/firmware/ethosu/ethosu_inference.c | 1 + drivers/firmware/ethosu/ethosu_network.c | 62 ++++-- drivers/firmware/ethosu/ethosu_network.h | 5 +- drivers/firmware/ethosu/ethosu_network_info.c | 184 ++++++++++++++++++ drivers/firmware/ethosu/ethosu_network_info.h | 56 ++++++ drivers/firmware/ethosu/ethosu_rpmsg.c | 50 ++++- drivers/firmware/ethosu/ethosu_rpmsg.h | 11 ++ drivers/firmware/ethosu/uapi/ethosu.h | 42 +++- 11 files changed, 470 insertions(+), 32 deletions(-) create mode 100644 drivers/firmware/ethosu/ethosu_network_info.c create mode 100644 drivers/firmware/ethosu/ethosu_network_info.h diff --git a/drivers/firmware/ethosu/Makefile b/drivers/firmware/ethosu/Makefile index fa0731adad35..a162a3bd32e3 100644 --- a/drivers/firmware/ethosu/Makefile +++ b/drivers/firmware/ethosu/Makefile @@ -26,4 +26,5 @@ ethosu-objs := ethosu_driver.o \ ethosu_inference.o \ ethosu_rpmsg.o \ ethosu_network.o \ + ethosu_network_info.o \ ethosu_capabilities.o diff --git a/drivers/firmware/ethosu/ethosu_core_interface.h b/drivers/firmware/ethosu/ethosu_core_interface.h index 11b695a5219d..f804de40b004 100644 --- a/drivers/firmware/ethosu/ethosu_core_interface.h +++ b/drivers/firmware/ethosu/ethosu_core_interface.h @@ -32,7 +32,9 @@ namespace EthosU { #endif /** Maximum number of IFM/OFM buffers per inference */ -#define ETHOSU_CORE_BUFFER_MAX 16 +#define ETHOSU_CORE_BUFFER_MAX 4 +/** Maximum number of dimensions for input and output */ +#define ETHOSU_CORE_DIM_MAX 4 /** Maximum number of PMU counters to be returned for inference */ #define ETHOSU_CORE_PMU_MAX 4 @@ -60,6 +62,8 @@ enum ethosu_core_msg_type { ETHOSU_CORE_MSG_VERSION_RSP, ETHOSU_CORE_MSG_CAPABILITIES_REQ, ETHOSU_CORE_MSG_CAPABILITIES_RSP, + ETHOSU_CORE_MSG_NETWORK_INFO_REQ, + ETHOSU_CORE_MSG_NETWORK_INFO_RSP, ETHOSU_CORE_MSG_POWER_REQ, ETHOSU_CORE_MSG_POWER_RSP, ETHOSU_CORE_MSG_MAX @@ -116,13 +120,35 @@ struct ethosu_core_buffer { uint32_t size; }; +/** + * enum ethosu_core_network_type - Network buffer type + */ +enum ethosu_core_network_type { + ETHOSU_CORE_NETWORK_BUFFER = 1, + ETHOSU_CORE_NETWORK_INDEX +}; + +/** + * struct ethosu_core_network_buffer - Network buffer + */ +struct ethosu_core_network_buffer { + u32 type; + union { + struct ethosu_core_buffer buffer; + u32 index; + }; +}; + +/** + * struct ethosu_core_inference_req - Inference request + */ struct ethosu_core_inference_req { uint64_t user_arg; uint32_t ifm_count; struct ethosu_core_buffer ifm[ETHOSU_CORE_BUFFER_MAX]; uint32_t ofm_count; struct ethosu_core_buffer ofm[ETHOSU_CORE_BUFFER_MAX]; - struct ethosu_core_buffer network; + struct ethosu_core_network_buffer network; uint8_t pmu_event_config[ETHOSU_CORE_PMU_MAX]; uint32_t pmu_cycle_counter_enable; uint32_t inference_type; @@ -143,7 +169,37 @@ struct ethosu_core_inference_rsp { }; /** - * struct ethosu_core_msg_verson - Message protocol version + * struct ethosu_core_network_info_req - Network information request + */ +struct ethosu_core_network_info_req { + u64 user_arg; + struct ethosu_core_network_buffer network; +}; + +/** + * struct ethosu_core_network_info_rsp - Network information response + */ +struct ethosu_core_network_info_rsp { + u64 user_arg; + char desc[32]; + u32 is_vela; + u32 ifm_count; + u32 ifm_size[ETHOSU_CORE_BUFFER_MAX]; + u32 ifm_types[ETHOSU_CORE_BUFFER_MAX]; + u32 ifm_offset[ETHOSU_CORE_BUFFER_MAX]; + u32 ifm_dims[ETHOSU_CORE_BUFFER_MAX]; + u32 ifm_shapes[ETHOSU_CORE_BUFFER_MAX][ETHOSU_CORE_DIM_MAX]; + u32 ofm_count; + u32 ofm_size[ETHOSU_CORE_BUFFER_MAX]; + u32 ofm_types[ETHOSU_CORE_BUFFER_MAX]; + u32 ofm_offset[ETHOSU_CORE_BUFFER_MAX]; + u32 ofm_dims[ETHOSU_CORE_BUFFER_MAX]; + u32 ofm_shapes[ETHOSU_CORE_BUFFER_MAX][ETHOSU_CORE_DIM_MAX]; + u32 status; +}; + +/** + * struct ethosu_core_msg_version - Message protocol version */ struct ethosu_core_msg_version { uint8_t major; diff --git a/drivers/firmware/ethosu/ethosu_device.c b/drivers/firmware/ethosu/ethosu_device.c index 6643a7aaad5b..b0be4bcba7a1 100644 --- a/drivers/firmware/ethosu/ethosu_device.c +++ b/drivers/firmware/ethosu/ethosu_device.c @@ -30,6 +30,7 @@ #include "ethosu_capabilities.h" #include "ethosu_inference.h" #include "ethosu_network.h" +#include "ethosu_network_info.h" #include "uapi/ethosu.h" #include @@ -75,6 +76,9 @@ static int ethosu_handle_msg(struct ethosu_device *edev, void *data) struct ethosu_core_msg_capabilities_rsp *capabilities = (struct ethosu_core_msg_capabilities_rsp *) ((char *)data + sizeof(struct ethosu_core_msg)); + struct ethosu_core_network_info_rsp *network_info = + (struct ethosu_core_network_info_rsp *) + ((char *)data + sizeof(struct ethosu_core_msg)); switch (header->type) { case ETHOSU_CORE_MSG_ERR: @@ -168,6 +172,22 @@ static int ethosu_handle_msg(struct ethosu_device *edev, void *data) ethosu_capability_rsp(edev, capabilities); break; + case ETHOSU_CORE_MSG_NETWORK_INFO_RSP: + if (header->length != sizeof(struct ethosu_core_network_info_rsp)) { + dev_warn(edev->dev, + "Msg: Network info response of incorrect size. size=%u, expected=%zu\n", + header->length, sizeof(struct ethosu_core_network_info_rsp)); + ret = -EBADMSG; + break; + } + + dev_dbg(edev->dev, + "Msg: Network info response. user_arg=0x%llx, status=%u", + network_info->user_arg, + network_info->status); + + ethosu_network_info_rsp(edev, network_info); + break; default: /* This should not happen due to version checks */ dev_warn(edev->dev, "Msg: Protocol error\n"); @@ -208,10 +228,8 @@ static int ethosu_open(struct inode *inode, if (!ret && atomic_read(&rproc->power) == 0) { init_completion(&edev->erp.rpmsg_ready); ret = rproc_boot(rproc); - if (ret) + if (ret || wait_for_completion_interruptible(&edev->erp.rpmsg_ready)) dev_err(edev->dev, "could not boot a remote processor\n"); - else - wait_for_completion_interruptible(&edev->erp.rpmsg_ready); } else { dev_err(edev->dev, "can't change firmware or remote processor is running\n"); } @@ -279,8 +297,8 @@ static long ethosu_ioctl(struct file *file, break; dev_dbg(edev->dev, - "Device ioctl: Network create. fd=%u\n", - uapi.fd); + "Device ioctl: Network create. type=%u, fd/index=%u\n", + uapi.type, uapi.fd); ret = ethosu_network_create(edev, &uapi); break; diff --git a/drivers/firmware/ethosu/ethosu_inference.c b/drivers/firmware/ethosu/ethosu_inference.c index 947988507792..4e5fa214c5d6 100644 --- a/drivers/firmware/ethosu/ethosu_inference.c +++ b/drivers/firmware/ethosu/ethosu_inference.c @@ -99,6 +99,7 @@ static int ethosu_inference_send(struct ethosu_inference *inf) inf->ifm_count, inf->ifm, inf->ofm_count, inf->ofm, inf->net->buf, + inf->net->index, inf->pmu_event_config, ETHOSU_PMU_EVENT_MAX, inf->pmu_cycle_counter_enable, diff --git a/drivers/firmware/ethosu/ethosu_network.c b/drivers/firmware/ethosu/ethosu_network.c index a7249619291b..e85e79073306 100644 --- a/drivers/firmware/ethosu/ethosu_network.c +++ b/drivers/firmware/ethosu/ethosu_network.c @@ -27,6 +27,7 @@ #include "ethosu_buffer.h" #include "ethosu_device.h" #include "ethosu_inference.h" +#include "ethosu_network_info.h" #include "uapi/ethosu.h" #include @@ -69,7 +70,9 @@ static void ethosu_network_destroy(struct kref *kref) dev_dbg(net->edev->dev, "Network destroy. net=0x%pK\n", net); - ethosu_buffer_put(net->buf); + if (net->buf) + ethosu_buffer_put(net->buf); + devm_kfree(net->edev->dev, net); } @@ -103,6 +106,23 @@ static long ethosu_network_ioctl(struct file *file, file, net, cmd, arg); switch (cmd) { + case ETHOSU_IOCTL_NETWORK_INFO: { + struct ethosu_uapi_network_info uapi; + + if (copy_from_user(&uapi, udata, sizeof(uapi))) + break; + + dev_dbg(net->edev->dev, + "Network ioctl: Network info. net=0x%pK\n", + net); + + ret = ethosu_network_info_request(net, &uapi); + if (ret) + break; + + ret = copy_to_user(udata, &uapi, sizeof(uapi)) ? -EFAULT : 0; + break; + } case ETHOSU_IOCTL_INFERENCE_CREATE: { struct ethosu_uapi_inference_create uapi; @@ -131,44 +151,48 @@ static long ethosu_network_ioctl(struct file *file, int ethosu_network_create(struct ethosu_device *edev, struct ethosu_uapi_network_create *uapi) { - struct ethosu_buffer *buf; struct ethosu_network *net; int ret = -ENOMEM; - buf = ethosu_buffer_get_from_fd(uapi->fd); - if (IS_ERR(buf)) - return PTR_ERR(buf); - net = devm_kzalloc(edev->dev, sizeof(*net), GFP_KERNEL); - if (!net) { - ret = -ENOMEM; - goto put_buf; - } + if (!net) + return -ENOMEM; net->edev = edev; - net->buf = buf; + net->buf = NULL; kref_init(&net->kref); + if (uapi->type == ETHOSU_UAPI_NETWORK_BUFFER) { + net->buf = ethosu_buffer_get_from_fd(uapi->fd); + if (IS_ERR(net->buf)) { + ret = PTR_ERR(net->buf); + goto free_net; + } + } else { + net->index = uapi->index; + } + ret = anon_inode_getfd("ethosu-network", ðosu_network_fops, net, O_RDWR | O_CLOEXEC); if (ret < 0) - goto free_net; + goto put_buf; net->file = fget(ret); fput(net->file); dev_dbg(edev->dev, - "Network create. file=0x%pK, fd=%d, net=0x%pK, buf=0x%pK", - net->file, ret, net, net->buf); + "Network create. file=0x%pK, fd=%d, net=0x%pK, buf=0x%pK, index=%u", + net->file, ret, net, net->buf, net->index); return ret; +put_buf: + if (net->buf) + ethosu_buffer_put(net->buf); + free_net: devm_kfree(edev->dev, net); -put_buf: - ethosu_buffer_put(buf); - return ret; } @@ -199,7 +223,7 @@ void ethosu_network_get(struct ethosu_network *net) kref_get(&net->kref); } -void ethosu_network_put(struct ethosu_network *net) +int ethosu_network_put(struct ethosu_network *net) { - kref_put(&net->kref, ethosu_network_destroy); + return kref_put(&net->kref, ethosu_network_destroy); } diff --git a/drivers/firmware/ethosu/ethosu_network.h b/drivers/firmware/ethosu/ethosu_network.h index 4236e1586efb..8ca2dd61886e 100644 --- a/drivers/firmware/ethosu/ethosu_network.h +++ b/drivers/firmware/ethosu/ethosu_network.h @@ -43,6 +43,7 @@ struct ethosu_network { struct file *file; struct kref kref; struct ethosu_buffer *buf; + u32 index; }; /**************************************************************************** @@ -75,7 +76,9 @@ void ethosu_network_get(struct ethosu_network *net); /** * ethosu_network_put() - Put network + * + * Return: 1 if object was removed, else 0. */ -void ethosu_network_put(struct ethosu_network *net); +int ethosu_network_put(struct ethosu_network *net); #endif /* ETHOSU_NETWORK_H */ diff --git a/drivers/firmware/ethosu/ethosu_network_info.c b/drivers/firmware/ethosu/ethosu_network_info.c new file mode 100644 index 000000000000..3b8ad935505a --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_network_info.c @@ -0,0 +1,184 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Arm Limited. + */ + +/**************************************************************************** + * Includes + ****************************************************************************/ + +#include "ethosu_network_info.h" + +#include "ethosu_device.h" +#include "ethosu_network.h" +#include "ethosu_rpmsg.h" +#include "uapi/ethosu.h" + +#define NETWORK_INFO_RESP_TIMEOUT_MS 30000 + +static inline int ethosu_network_info_send(struct ethosu_network_info *info) +{ + /* Send network info request to firmware */ + return ethosu_rpmsg_network_info_request(&info->edev->erp, + &info->msg, + info->net->buf, + info->net->index); +} + +static void ethosu_network_info_fail(struct ethosu_rpmsg_msg *msg) +{ + struct ethosu_network_info *info = + container_of(msg, typeof(*info), msg); + + if (completion_done(&info->done)) + return; + + info->errno = -EFAULT; + complete(&info->done); +} + +static int ethosu_network_info_resend(struct ethosu_rpmsg_msg *msg) +{ + struct ethosu_network_info *info = + container_of(msg, typeof(*info), msg); + int ret; + + /* Don't resend request if response has already been received */ + if (completion_done(&info->done)) + return 0; + + /* Resend request */ + ret = ethosu_network_info_send(info); + if (ret) + return ret; + + return 0; +} + +int ethosu_network_info_request(struct ethosu_network *net, + struct ethosu_uapi_network_info *uapi) +{ + struct ethosu_network_info *info; + int ret; + int timeout; + + info = devm_kzalloc(net->edev->dev, sizeof(*info), GFP_KERNEL); + if (!info) + return -ENOMEM; + + info->edev = net->edev; + info->net = net; + info->uapi = uapi; + init_completion(&info->done); + info->msg.fail = ethosu_network_info_fail; + info->msg.resend = ethosu_network_info_resend; + + ret = ethosu_rpmsg_register(&info->edev->erp, &info->msg); + if (ret < 0) + goto kfree; + + /* Get reference to network */ + ethosu_network_get(info->net); + + ret = ethosu_network_info_send(info); + if (ret) + goto deregister; + + dev_dbg(info->edev->dev, + "Network info create. info=0x%pK, net=0x%pK, msg.id=0x%x\n", + info, info->net, info->msg.id); + + /* Unlock the device mutex and wait for completion */ + mutex_unlock(&info->edev->mutex); + timeout = wait_for_completion_timeout(&info->done, + msecs_to_jiffies(NETWORK_INFO_RESP_TIMEOUT_MS)); + mutex_lock(&info->edev->mutex); + + if (timeout == 0) { + dev_warn(info->edev->dev, "Network info timed out. info=0x%pK", + info); + + ret = -ETIME; + goto deregister; + } + + ret = info->errno; + +deregister: + ethosu_rpmsg_deregister(&info->edev->erp, &info->msg); + ethosu_network_put(info->net); + +kfree: + dev_dbg(info->edev->dev, + "Network info destroy. info=0x%pK, msg.id=0x%x\n", + info, info->msg.id); + devm_kfree(info->edev->dev, info); + + return ret; +} + +void ethosu_network_info_rsp(struct ethosu_device *edev, + struct ethosu_core_network_info_rsp *rsp) +{ + int ret; + int id = (int)rsp->user_arg; + struct ethosu_rpmsg_msg *msg; + struct ethosu_network_info *info; + u32 i, j; + + msg = ethosu_rpmsg_find(&edev->erp, id); + if (IS_ERR(msg)) { + dev_warn(edev->dev, + "Id for network info msg not found. msg.id=0x%x\n", + id); + + return; + } + + info = container_of(msg, typeof(*info), msg); + + if (completion_done(&info->done)) + return; + + info->errno = 0; + + if (rsp->status != ETHOSU_CORE_STATUS_OK) { + info->errno = -EBADF; + goto signal_complete; + } + + if (rsp->ifm_count > ETHOSU_CORE_BUFFER_MAX || rsp->ofm_count > ETHOSU_CORE_BUFFER_MAX) { + info->errno = -ENFILE; + goto signal_complete; + } + + ret = strscpy(info->uapi->desc, rsp->desc, sizeof(info->uapi->desc)); + if (ret < 0) { + info->errno = ret; + goto signal_complete; + } + + info->uapi->is_vela = rsp->is_vela; + info->uapi->ifm_count = rsp->ifm_count; + for (i = 0; i < rsp->ifm_count; i++) { + info->uapi->ifm_size[i] = rsp->ifm_size[i]; + info->uapi->ifm_types[i] = rsp->ifm_types[i]; + info->uapi->ifm_offset[i] = rsp->ifm_offset[i]; + info->uapi->ifm_dims[i] = rsp->ifm_dims[i]; + for (j = 0; j < rsp->ifm_dims[i]; j++) + info->uapi->ifm_shapes[i][j] = rsp->ifm_shapes[i][j]; + } + + info->uapi->ofm_count = rsp->ofm_count; + for (i = 0; i < rsp->ofm_count; i++) { + info->uapi->ofm_size[i] = rsp->ofm_size[i]; + info->uapi->ofm_types[i] = rsp->ofm_types[i]; + info->uapi->ofm_offset[i] = rsp->ofm_offset[i]; + info->uapi->ofm_dims[i] = rsp->ofm_dims[i]; + for (j = 0; j < rsp->ofm_dims[i]; j++) + info->uapi->ofm_shapes[i][j] = rsp->ofm_shapes[i][j]; + } + +signal_complete: + complete(&info->done); +} diff --git a/drivers/firmware/ethosu/ethosu_network_info.h b/drivers/firmware/ethosu/ethosu_network_info.h new file mode 100644 index 000000000000..b5bc00f8e34a --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_network_info.h @@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Arm Limited. + */ + +#ifndef ETHOSU_NETWORK_INFO_H +#define ETHOSU_NETWORK_INFO_H + +/**************************************************************************** + * Includes + ****************************************************************************/ + +#include "ethosu_core_interface.h" +#include "ethosu_rpmsg.h" + +#include +#include + +/**************************************************************************** + * Types + ****************************************************************************/ + +struct ethosu_device; +struct ethosu_network; +struct ethosu_uapi_network_info; + +struct ethosu_network_info { + struct ethosu_device *edev; + struct ethosu_network *net; + struct ethosu_uapi_network_info *uapi; + struct completion done; + int errno; + struct ethosu_rpmsg_msg msg; +}; + +/**************************************************************************** + * Functions + ****************************************************************************/ + +/** + * ethosu_network_info_request() - Send a network info request + * + * This function must be called in the context of a user space process. + * + * Return: 0 on success, . + */ +int ethosu_network_info_request(struct ethosu_network *net, + struct ethosu_uapi_network_info *uapi); + +/** + * ethosu_network_info_rsp() - Handle network info response. + */ +void ethosu_network_info_rsp(struct ethosu_device *edev, + struct ethosu_core_network_info_rsp *rsp); + +#endif /* ETHOSU_NETWORK_INFO_H */ diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.c b/drivers/firmware/ethosu/ethosu_rpmsg.c index a92439b81c96..368a519e36b1 100644 --- a/drivers/firmware/ethosu/ethosu_rpmsg.c +++ b/drivers/firmware/ethosu/ethosu_rpmsg.c @@ -190,6 +190,7 @@ int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp, uint32_t ofm_count, struct ethosu_buffer **ofm, struct ethosu_buffer *network, + u32 network_index, uint8_t *pmu_event_config, uint8_t pmu_event_config_count, uint8_t pmu_cycle_counter_enable, @@ -229,7 +230,13 @@ int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp, for (i = 0; i < ETHOSU_CORE_PMU_MAX; i++) req.pmu_event_config[i] = pmu_event_config[i]; - ethosu_core_set_size(network, &req.network); + if (network) { + req.network.type = ETHOSU_CORE_NETWORK_BUFFER; + ethosu_core_set_size(network, &req.network.buffer); + } else { + req.network.type = ETHOSU_CORE_NETWORK_INDEX; + req.network.index = network_index; + } memcpy(data, &msg, sizeof(struct ethosu_core_msg)); memcpy(data + sizeof(struct ethosu_core_msg), &req, @@ -246,6 +253,47 @@ int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp, return 0; } +int ethosu_rpmsg_network_info_request(struct ethosu_rpmsg *erp, + struct ethosu_rpmsg_msg *rpmsg, + struct ethosu_buffer *network, + uint32_t network_index) +{ + struct ethosu_core_msg msg = { + .magic = ETHOSU_CORE_MSG_MAGIC, + .type = ETHOSU_CORE_MSG_NETWORK_INFO_REQ, + .length = sizeof(struct ethosu_core_network_info_req) + }; + struct ethosu_core_network_info_req req; + struct rpmsg_device *rpdev = erp->rpdev; + u8 data[sizeof(struct ethosu_core_msg) + + sizeof(struct ethosu_core_network_info_req)]; + int ret; + + req.user_arg = rpmsg->id; + + if (network) { + req.network.type = ETHOSU_CORE_NETWORK_BUFFER; + ethosu_core_set_size(network, &req.network.buffer); + } else { + req.network.type = ETHOSU_CORE_NETWORK_INDEX; + req.network.index = network_index; + } + + memcpy(data, &msg, sizeof(struct ethosu_core_msg)); + memcpy(data + sizeof(struct ethosu_core_msg), &req, + sizeof(struct ethosu_core_network_info_req)); + + ret = rpmsg_send(rpdev->ept, (void *)&data, + sizeof(struct ethosu_core_msg) + + sizeof(struct ethosu_core_network_info_req)); + if (ret) { + dev_err(&rpdev->dev, "rpmsg_send failed: %d\n", ret); + return ret; + } + + return 0; +} + static int rpmsg_ethosu_cb(struct rpmsg_device *rpdev, void *data, int len, void *priv, u32 src) { diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.h b/drivers/firmware/ethosu/ethosu_rpmsg.h index 74ee2fdea5cb..ba6fb8fd5869 100644 --- a/drivers/firmware/ethosu/ethosu_rpmsg.h +++ b/drivers/firmware/ethosu/ethosu_rpmsg.h @@ -84,12 +84,23 @@ int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp, uint32_t ofm_count, struct ethosu_buffer **ofm, struct ethosu_buffer *network, + u32 network_index, uint8_t *pmu_event_config, uint8_t pmu_event_config_count, uint8_t pmu_cycle_counter_enable, uint32_t inference_type ); +/** + * ethosu_rpmsg_network_info_request() - Send network info request + * + * Return: 0 on success, else error code. + */ +int ethosu_rpmsg_network_info_request(struct ethosu_rpmsg *erp, + struct ethosu_rpmsg_msg *rpmsg, + struct ethosu_buffer *network, + uint32_t network_index); + int ethosu_rpmsg_init(struct ethosu_rpmsg *erp, ethosu_rpmsg_cb callback, void *user_arg); diff --git a/drivers/firmware/ethosu/uapi/ethosu.h b/drivers/firmware/ethosu/uapi/ethosu.h index 648cec40473b..64a7d9d4885e 100644 --- a/drivers/firmware/ethosu/uapi/ethosu.h +++ b/drivers/firmware/ethosu/uapi/ethosu.h @@ -44,7 +44,8 @@ namespace EthosU { #define ETHOSU_IOCTL_PING ETHOSU_IO(0x00) #define ETHOSU_IOCTL_VERSION_REQ ETHOSU_IO(0x01) -#define ETHOSU_IOCTL_CAPABILITIES_REQ ETHOSU_IO(0x02) +#define ETHOSU_IOCTL_CAPABILITIES_REQ ETHOSU_IOR(0x02, \ + struct ethosu_uapi_device_capabilities) #define ETHOSU_IOCTL_BUFFER_CREATE ETHOSU_IOR(0x10, \ struct ethosu_uapi_buffer_create) #define ETHOSU_IOCTL_BUFFER_SET ETHOSU_IOR(0x11, \ @@ -53,6 +54,8 @@ namespace EthosU { struct ethosu_uapi_buffer) #define ETHOSU_IOCTL_NETWORK_CREATE ETHOSU_IOR(0x20, \ struct ethosu_uapi_network_create) +#define ETHOSU_IOCTL_NETWORK_INFO ETHOSU_IOR(0x21, \ + struct ethosu_uapi_network_info) #define ETHOSU_IOCTL_INFERENCE_CREATE ETHOSU_IOR(0x30, \ struct ethosu_uapi_inference_create) #define ETHOSU_IOCTL_INFERENCE_STATUS ETHOSU_IOR(0x31, \ @@ -100,12 +103,45 @@ struct ethosu_uapi_buffer { __u32 size; }; +/** + * enum ethosu_uapi_network_create - Network buffer type. + * @ETHOSU_UAPI_NETWORK_BUFFER: Network is stored in a buffer handle. + * @ETHOSU_UAPI_NETWORK_INDEX: Network is built into firmware and referenced by + * index. + */ +enum ethosu_uapi_network_type { + ETHOSU_UAPI_NETWORK_BUFFER = 1, + ETHOSU_UAPI_NETWORK_INDEX +}; + /** * struct ethosu_uapi_network_create - Create network request + * @type: Buffer type. See @ethosu_uapi_network_type. * @fd: Buffer file descriptor + * @index: Buffer index compiled into firmware binary. */ struct ethosu_uapi_network_create { - __u32 fd; + u32 type; + union { + __u32 fd; + __u32 index; + }; +}; + +/** + * struct ethosu_uapi_network_info - Network info + * @desc: Network description + * @ifm_count: Number of IFM buffers + * @ifm_size: IFM buffer sizes + * @ofm_count: Number of OFM buffers + * @ofm_size: OFM buffer sizes + */ +struct ethosu_uapi_network_info { + char desc[32]; + __u32 ifm_count; + __u32 ifm_size[ETHOSU_FD_MAX]; + __u32 ofm_count; + __u32 ofm_size[ETHOSU_FD_MAX]; }; /** @@ -162,7 +198,7 @@ struct ethosu_uapi_device_hw_cfg { }; /** - * struct ethosu_uapi_capabilities - Device capabilities + * struct ethosu_uapi_device_capabilities - Device capabilities * @hw_id: Hardware identification * @hw_cfg: Hardware configuration * @driver_patch_rev: Driver version patch From patchwork Fri Jun 16 05:59:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Wang X-Patchwork-Id: 13282004 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CF40EEB64D8 for ; Fri, 16 Jun 2023 05:43:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Xr7ixZNKAXhgD7CjjF4RCk07GkQU1ClNt0Ny4YggK+8=; b=Y9Jcf7MnM39v8v qUrqW0qsOzz5+4J996r89Y4fnGIl2+FIfMTH0M+4AEwQenqM1M5+1X9GHMr6excTFYbY0nA3OrXw+ LzeYzkettZR7XF7U74C/6rhB3nGClnaBwZwLfMFZ8eQO1c6k9+rNxZYX/sokqW3vQQAvot6O3imcT fA4DR485hK6jNlORRhiUi2KjIxUTmHMzcS1D52pBu7Xuy+tNkUBN6mPeeh3KxwljadUYMMwfU2kZT JHFyk7NLqdINSHpD7ujGum5KExljYMccX+94dGY4DTMilabI5BdokDNB/ynmvBYyGg8rRdMT2DEmp PiMKqDHOIDTtOQ1y6bxg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qA2F3-00GutV-0q; Fri, 16 Jun 2023 05:43:25 +0000 Received: from inva020.nxp.com ([92.121.34.13]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qA2Eq-00Gunt-0k for linux-arm-kernel@lists.infradead.org; Fri, 16 Jun 2023 05:43:17 +0000 Received: from inva020.nxp.com (localhost [127.0.0.1]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 478071A249D; Fri, 16 Jun 2023 07:43:08 +0200 (CEST) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id D99421A068F; Fri, 16 Jun 2023 07:43:07 +0200 (CEST) Received: from localhost.localdomain (mega.ap.freescale.net [10.192.208.232]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id D56D91820F58; Fri, 16 Jun 2023 13:43:06 +0800 (+08) From: Alison Wang To: gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: leoyang.li@nxp.com, xuelin.shi@nxp.com, xiaofeng.ren@nxp.com, feng.guo@nxp.com, Alison Wang Subject: [PATCH 7/8] ethosu: Add core message about inference cancellation Date: Fri, 16 Jun 2023 13:59:12 +0800 Message-Id: <20230616055913.2360-8-alison.wang@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230616055913.2360-1-alison.wang@nxp.com> References: <20230616055913.2360-1-alison.wang@nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230615_224312_569132_8EBA8976 X-CRM114-Status: GOOD ( 29.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This patch adds core message about sending cancel inference message to Ethos-U subsystem to abort inference execution. Signed-off-by: Davide Grohmann Signed-off-by: Alison Wang Reviewed-by: Peng Fan --- drivers/firmware/ethosu/Makefile | 3 +- .../firmware/ethosu/ethosu_cancel_inference.c | 185 ++++++++++++++++++ .../firmware/ethosu/ethosu_cancel_inference.h | 55 ++++++ .../firmware/ethosu/ethosu_core_interface.h | 37 ++-- drivers/firmware/ethosu/ethosu_inference.c | 16 ++ drivers/firmware/ethosu/ethosu_rpmsg.c | 33 ++++ drivers/firmware/ethosu/ethosu_rpmsg.h | 9 + drivers/firmware/ethosu/uapi/ethosu.h | 29 +++ 8 files changed, 349 insertions(+), 18 deletions(-) create mode 100644 drivers/firmware/ethosu/ethosu_cancel_inference.c create mode 100644 drivers/firmware/ethosu/ethosu_cancel_inference.h diff --git a/drivers/firmware/ethosu/Makefile b/drivers/firmware/ethosu/Makefile index a162a3bd32e3..88a352690ddd 100644 --- a/drivers/firmware/ethosu/Makefile +++ b/drivers/firmware/ethosu/Makefile @@ -27,4 +27,5 @@ ethosu-objs := ethosu_driver.o \ ethosu_rpmsg.o \ ethosu_network.o \ ethosu_network_info.o \ - ethosu_capabilities.o + ethosu_capabilities.o \ + ethosu_cancel_inference.o diff --git a/drivers/firmware/ethosu/ethosu_cancel_inference.c b/drivers/firmware/ethosu/ethosu_cancel_inference.c new file mode 100644 index 000000000000..911955889197 --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_cancel_inference.c @@ -0,0 +1,185 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022 Arm Limited. + */ + +/**************************************************************************** + * Includes + ****************************************************************************/ + +#include "ethosu_cancel_inference.h" + +#include "ethosu_core_interface.h" +#include "ethosu_device.h" +#include "ethosu_inference.h" + +#include + +/**************************************************************************** + * Defines + ****************************************************************************/ + +#define CANCEL_INFERENCE_RESP_TIMEOUT_MS 2000 + +/**************************************************************************** + * Functions + ****************************************************************************/ + +static int ethosu_cancel_inference_send(struct ethosu_cancel_inference *cancellation) +{ + return ethosu_rpmsg_cancel_inference(&cancellation->edev->erp, + &cancellation->msg, + cancellation->inf->msg.id); +} + +static void ethosu_cancel_inference_fail(struct ethosu_rpmsg_msg *msg) +{ + struct ethosu_cancel_inference *cancellation = + container_of(msg, typeof(*cancellation), msg); + + if (completion_done(&cancellation->done)) + return; + + cancellation->errno = -EFAULT; + cancellation->uapi->status = ETHOSU_UAPI_STATUS_ERROR; + complete(&cancellation->done); +} + +static int ethosu_cancel_inference_complete(struct ethosu_rpmsg_msg *msg) +{ + struct ethosu_cancel_inference *cancellation = + container_of(msg, typeof(*cancellation), msg); + + if (completion_done(&cancellation->done)) + return 0; + + cancellation->errno = 0; + cancellation->uapi->status = + cancellation->inf->done && + cancellation->inf->status != ETHOSU_UAPI_STATUS_OK ? + ETHOSU_UAPI_STATUS_OK : + ETHOSU_UAPI_STATUS_ERROR; + complete(&cancellation->done); + + return 0; +} + +int ethosu_cancel_inference_request(struct ethosu_inference *inf, + struct ethosu_uapi_cancel_inference_status *uapi) +{ + struct ethosu_cancel_inference *cancellation; + int ret; + int timeout; + + if (inf->done) { + uapi->status = ETHOSU_UAPI_STATUS_ERROR; + + return 0; + } + + cancellation = + devm_kzalloc(inf->edev->dev, + sizeof(struct ethosu_cancel_inference), + GFP_KERNEL); + if (!cancellation) + return -ENOMEM; + + /* increase ref count on the inference we are referring to */ + ethosu_inference_get(inf); + /* mark inference ABORTING to avoid resending the inference message */ + inf->status = ETHOSU_UAPI_STATUS_ABORTING; + + cancellation->edev = inf->edev; + cancellation->inf = inf; + cancellation->uapi = uapi; + init_completion(&cancellation->done); + cancellation->msg.fail = ethosu_cancel_inference_fail; + + /* Never resend messages but always complete, since we have restart the + * whole firmware and marked the inference as aborted + */ + cancellation->msg.resend = ethosu_cancel_inference_complete; + + ret = ethosu_rpmsg_register(&cancellation->edev->erp, + &cancellation->msg); + if (ret < 0) + goto kfree; + + dev_dbg(cancellation->edev->dev, + "Inference cancellation create. cancel=0x%pK, msg.id=%d\n", + cancellation, cancellation->msg.id); + + ret = ethosu_cancel_inference_send(cancellation); + if (ret != 0) + goto deregister; + + /* Unlock the mutex before going to block on the condition */ + mutex_unlock(&cancellation->edev->mutex); + /* wait for response to arrive back */ + timeout = wait_for_completion_timeout(&cancellation->done, + msecs_to_jiffies(CANCEL_INFERENCE_RESP_TIMEOUT_MS)); + + /* take back the mutex before resuming to do anything */ + ret = mutex_lock_interruptible(&cancellation->edev->mutex); + if (ret != 0) + goto deregister; + + if (timeout == 0) { + dev_warn(inf->edev->dev, + "Msg: Cancel Inference response lost - timeout\n"); + ret = -EIO; + goto deregister; + } + + if (cancellation->errno) { + ret = cancellation->errno; + goto deregister; + } + +deregister: + ethosu_rpmsg_deregister(&cancellation->edev->erp, + &cancellation->msg); + +kfree: + dev_dbg(cancellation->edev->dev, + "Cancel inference destroy. cancel=0x%pK\n", cancellation); + /* decrease the reference on the inference we are referring to */ + ethosu_inference_put(cancellation->inf); + devm_kfree(cancellation->edev->dev, cancellation); + + return ret; +} + +void ethosu_cancel_inference_rsp(struct ethosu_device *edev, + struct ethosu_core_cancel_inference_rsp *rsp) +{ + int id = (int)rsp->user_arg; + struct ethosu_rpmsg_msg *msg; + struct ethosu_cancel_inference *cancellation; + + msg = ethosu_rpmsg_find(&edev->erp, id); + if (IS_ERR(msg)) { + dev_warn(edev->dev, + "Handle not found in cancel inference list. handle=0x%p\n", + rsp); + + return; + } + + cancellation = container_of(msg, typeof(*cancellation), msg); + + if (completion_done(&cancellation->done)) + return; + + cancellation->errno = 0; + switch (rsp->status) { + case ETHOSU_CORE_STATUS_OK: + cancellation->uapi->status = ETHOSU_UAPI_STATUS_OK; + break; + case ETHOSU_CORE_STATUS_ERROR: + cancellation->uapi->status = ETHOSU_UAPI_STATUS_ERROR; + break; + } + + complete(&cancellation->done); +} diff --git a/drivers/firmware/ethosu/ethosu_cancel_inference.h b/drivers/firmware/ethosu/ethosu_cancel_inference.h new file mode 100644 index 000000000000..940e0589959a --- /dev/null +++ b/drivers/firmware/ethosu/ethosu_cancel_inference.h @@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022 Arm Limited. + */ + +#ifndef ETHOSU_CANCEL_INFERENCE_H +#define ETHOSU_CANCEL_INFERENCE_H + +/**************************************************************************** + * Includes + ****************************************************************************/ + +#include "ethosu_rpmsg.h" +#include "uapi/ethosu.h" + +#include +#include + +/**************************************************************************** + * Types + ****************************************************************************/ + +struct ethosu_core_cancel_inference_rsp; +struct ethosu_device; +struct ethosu_uapi_cancel_inference_status; +struct ethosu_inference; + +struct ethosu_cancel_inference { + struct ethosu_device *edev; + struct ethosu_inference *inf; + struct ethosu_uapi_cancel_inference_status *uapi; + struct completion done; + struct ethosu_rpmsg_msg msg; + int errno; +}; + +/**************************************************************************** + * Functions + ****************************************************************************/ + +/** + * ethosu_cancel_inference_request() - Send cancel inference request + * + * Return: 0 on success, error code otherwise. + */ +int ethosu_cancel_inference_request(struct ethosu_inference *inf, + struct ethosu_uapi_cancel_inference_status *uapi); + +/** + * ethosu_cancel_inference_rsp() - Handle cancel inference response + */ +void ethosu_cancel_inference_rsp(struct ethosu_device *edev, + struct ethosu_core_cancel_inference_rsp *rsp); + +#endif /* ETHOSU_CANCEL_INFERENCE_H */ diff --git a/drivers/firmware/ethosu/ethosu_core_interface.h b/drivers/firmware/ethosu/ethosu_core_interface.h index f804de40b004..b1244ce28d39 100644 --- a/drivers/firmware/ethosu/ethosu_core_interface.h +++ b/drivers/firmware/ethosu/ethosu_core_interface.h @@ -1,21 +1,6 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ /* - * Copyright (c) 2020-2022 Arm Limited. - * - * This program is free software and is provided to you under the terms of the - * GNU General Public License version 2 as published by the Free Software - * Foundation, and any use by you of this program is subject to the terms - * of such GNU licence. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program; if not, you can access it online at - * http://www.gnu.org/licenses/gpl-2.0.html. - * - * SPDX-License-Identifier: GPL-2.0-only + * Copyright (c) 2022 Arm Limited. */ #ifndef ETHOSU_CORE_INTERFACE_H @@ -64,6 +49,8 @@ enum ethosu_core_msg_type { ETHOSU_CORE_MSG_CAPABILITIES_RSP, ETHOSU_CORE_MSG_NETWORK_INFO_REQ, ETHOSU_CORE_MSG_NETWORK_INFO_RSP, + ETHOSU_CORE_MSG_CANCEL_INFERENCE_REQ, + ETHOSU_CORE_MSG_CANCEL_INFERENCE_RSP, ETHOSU_CORE_MSG_POWER_REQ, ETHOSU_CORE_MSG_POWER_RSP, ETHOSU_CORE_MSG_MAX @@ -235,6 +222,22 @@ struct ethosu_core_msg_capabilities_rsp { uint32_t custom_dma; }; +/** + * struct ethosu_core_cancel_inference_req - Message cancel inference request + */ +struct ethosu_core_cancel_inference_req { + u64 user_arg; + u64 inference_handle; +}; + +/** + * struct ethosu_core_cancel_inference_rsp - Message cancel inference response + */ +struct ethosu_core_cancel_inference_rsp { + u64 user_arg; + u32 status; +}; + enum ethosu_core_power_req_type { ETHOSU_CORE_POWER_REQ_SUSPEND = 0, ETHOSU_CORE_POWER_REQ_RESUME diff --git a/drivers/firmware/ethosu/ethosu_inference.c b/drivers/firmware/ethosu/ethosu_inference.c index 4e5fa214c5d6..8c0c59242650 100644 --- a/drivers/firmware/ethosu/ethosu_inference.c +++ b/drivers/firmware/ethosu/ethosu_inference.c @@ -28,6 +28,7 @@ #include "ethosu_core_interface.h" #include "ethosu_device.h" #include "ethosu_network.h" +#include "ethosu_cancel_inference.h" #include "uapi/ethosu.h" #include @@ -267,6 +268,21 @@ static long ethosu_inference_ioctl(struct file *file, break; } + case ETHOSU_IOCTL_INFERENCE_CANCEL: { + struct ethosu_uapi_cancel_inference_status uapi; + + dev_dbg(inf->edev->dev, + "Inference ioctl: Cancel Inference. Handle=%p\n", + inf); + + ret = ethosu_cancel_inference_request(inf, &uapi); + if (ret) + break; + + ret = copy_to_user(udata, &uapi, sizeof(uapi)) ? -EFAULT : 0; + + break; + } default: { dev_err(inf->edev->dev, "Invalid ioctl. cmd=%u, arg=%lu\n", cmd, arg); diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.c b/drivers/firmware/ethosu/ethosu_rpmsg.c index 368a519e36b1..351a1046e65e 100644 --- a/drivers/firmware/ethosu/ethosu_rpmsg.c +++ b/drivers/firmware/ethosu/ethosu_rpmsg.c @@ -294,6 +294,39 @@ int ethosu_rpmsg_network_info_request(struct ethosu_rpmsg *erp, return 0; } +int ethosu_rpmsg_cancel_inference(struct ethosu_rpmsg *erp, + struct ethosu_rpmsg_msg *rpmsg, + int inference_handle) +{ + struct ethosu_core_msg msg = { + .magic = ETHOSU_CORE_MSG_MAGIC, + .type = ETHOSU_CORE_MSG_CAPABILITIES_REQ, + .length = sizeof(struct ethosu_core_cancel_inference_req) + }; + struct ethosu_core_cancel_inference_req req; + struct rpmsg_device *rpdev = erp->rpdev; + u8 data[sizeof(struct ethosu_core_msg) + + sizeof(struct ethosu_core_cancel_inference_req)]; + int ret; + + req.user_arg = rpmsg->id; + req.inference_handle = inference_handle; + + memcpy(data, &msg, sizeof(struct ethosu_core_msg)); + memcpy(data + sizeof(struct ethosu_core_msg), &req, + sizeof(struct ethosu_core_cancel_inference_req)); + + ret = rpmsg_send(rpdev->ept, (void *)&data, + sizeof(struct ethosu_core_msg) + + sizeof(struct ethosu_core_cancel_inference_req)); + if (ret) { + dev_err(&rpdev->dev, "rpmsg_send failed: %d\n", ret); + return ret; + } + + return 0; +} + static int rpmsg_ethosu_cb(struct rpmsg_device *rpdev, void *data, int len, void *priv, u32 src) { diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.h b/drivers/firmware/ethosu/ethosu_rpmsg.h index ba6fb8fd5869..283d401a0dbf 100644 --- a/drivers/firmware/ethosu/ethosu_rpmsg.h +++ b/drivers/firmware/ethosu/ethosu_rpmsg.h @@ -101,6 +101,15 @@ int ethosu_rpmsg_network_info_request(struct ethosu_rpmsg *erp, struct ethosu_buffer *network, uint32_t network_index); +/** + * ethosu_rpmsg_cancel_inference() - Send inference cancellation + * + * Return: 0 on success, else error code. + */ +int ethosu_rpmsg_cancel_inference(struct ethosu_rpmsg *erp, + struct ethosu_rpmsg_msg *rpmsg, + int inference_handle); + int ethosu_rpmsg_init(struct ethosu_rpmsg *erp, ethosu_rpmsg_cb callback, void *user_arg); diff --git a/drivers/firmware/ethosu/uapi/ethosu.h b/drivers/firmware/ethosu/uapi/ethosu.h index 64a7d9d4885e..3375b1d83ab0 100644 --- a/drivers/firmware/ethosu/uapi/ethosu.h +++ b/drivers/firmware/ethosu/uapi/ethosu.h @@ -60,10 +60,15 @@ namespace EthosU { struct ethosu_uapi_inference_create) #define ETHOSU_IOCTL_INFERENCE_STATUS ETHOSU_IOR(0x31, \ struct ethosu_uapi_result_status) +#define ETHOSU_IOCTL_INFERENCE_CANCEL ETHOSU_IOR(0x32, \ + struct ethosu_uapi_cancel_inference_status) /* Maximum number of IFM/OFM file descriptors per network */ #define ETHOSU_FD_MAX 16 +/* Maximum number of dimensions for input and output */ +#define ETHOSU_DIM_MAX 8 + /* Maximum number of PMUs available */ #define ETHOSU_PMU_EVENT_MAX 4 @@ -133,15 +138,31 @@ struct ethosu_uapi_network_create { * @desc: Network description * @ifm_count: Number of IFM buffers * @ifm_size: IFM buffer sizes + * @ifm_types: IFM data types + * @ifm_offset: IFM data offset in arena + * @ifm_dims: IFM buffer dimensions + * @ifm_shapes: IFM buffer shapes * @ofm_count: Number of OFM buffers * @ofm_size: OFM buffer sizes + * @ofm_offset: OFM data offset in arena + * @ofm_dims: OFM buffer dimensions + * @ofm_shapes: OFM buffer shapes */ struct ethosu_uapi_network_info { char desc[32]; + __u32 is_vela; __u32 ifm_count; __u32 ifm_size[ETHOSU_FD_MAX]; + __u32 ifm_types[ETHOSU_FD_MAX]; + __u32 ifm_offset[ETHOSU_FD_MAX]; + __u32 ifm_dims[ETHOSU_FD_MAX]; + __u32 ifm_shapes[ETHOSU_FD_MAX][ETHOSU_DIM_MAX]; __u32 ofm_count; __u32 ofm_size[ETHOSU_FD_MAX]; + __u32 ofm_types[ETHOSU_FD_MAX]; + __u32 ofm_offset[ETHOSU_FD_MAX]; + __u32 ofm_dims[ETHOSU_FD_MAX]; + __u32 ofm_shapes[ETHOSU_FD_MAX][ETHOSU_DIM_MAX]; }; /** @@ -250,6 +271,14 @@ struct ethosu_uapi_result_status { struct ethosu_uapi_pmu_counts pmu_count; }; +/** + * struct ethosu_uapi_cancel_status - Status of inference cancellation. + * @status OK if inference cancellation was performed, ERROR otherwise. + */ +struct ethosu_uapi_cancel_inference_status { + enum ethosu_uapi_status status; +}; + #ifdef __cplusplus } /* namespace EthosU */ #endif From patchwork Fri Jun 16 05:59:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Wang X-Patchwork-Id: 13282002 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AE5EEEB64D7 for ; Fri, 16 Jun 2023 05:43:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=UYROZqyK0RztFU9oyjYf/Js3/1gz0Plhr/G5DIlmgV4=; b=MIc8DND+SZ7K4w WCuN/oiakc6J5YbIeCJw/6H1Pm5GPbl0ilP7Lm9Z1EFg86mxJ2Tin/+gdK+34ctfTZcVIrlOYFblR 6hjN8k2TBkfKkT0ch/FOe8G3N/1/udrXlv/sjuuCQ4evZqBECc/0/ajcHV/+LGa1Zcp6Ed+9uvmvg 239/hUlkHCIQPjcCgt2wHlHjC5PPEcVOUqgpUrsYnJK/MYQXPG66b1t/mB8TaHhi+TExyu8CY5rV2 T8s0viOab3FhouSFIS83A3R+piXs2p37utJ2eXMd01TaUBFfaLQNHZ9pUzGGo++MyMPfTo16A9PHC 5imbZ+iqXMrqaN7D94DA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qA2F1-00Gurm-0a; Fri, 16 Jun 2023 05:43:23 +0000 Received: from inva020.nxp.com ([92.121.34.13]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qA2Eq-00Guns-0k for linux-arm-kernel@lists.infradead.org; Fri, 16 Jun 2023 05:43:14 +0000 Received: from inva020.nxp.com (localhost [127.0.0.1]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 8532E1A068F; Fri, 16 Jun 2023 07:43:09 +0200 (CEST) Received: from aprdc01srsp001v.ap-rdc01.nxp.com (aprdc01srsp001v.ap-rdc01.nxp.com [165.114.16.16]) by inva020.eu-rdc02.nxp.com (Postfix) with ESMTP id 234031A249B; Fri, 16 Jun 2023 07:43:09 +0200 (CEST) Received: from localhost.localdomain (mega.ap.freescale.net [10.192.208.232]) by aprdc01srsp001v.ap-rdc01.nxp.com (Postfix) with ESMTP id BC8CF181D0C0; Fri, 16 Jun 2023 13:43:07 +0800 (+08) From: Alison Wang To: gregkh@linuxfoundation.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: leoyang.li@nxp.com, xuelin.shi@nxp.com, xiaofeng.ren@nxp.com, feng.guo@nxp.com, Alison Wang Subject: [PATCH 8/8] ethosu: Add rwlock when alloc and remove msg id Date: Fri, 16 Jun 2023 13:59:13 +0800 Message-Id: <20230616055913.2360-9-alison.wang@nxp.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230616055913.2360-1-alison.wang@nxp.com> References: <20230616055913.2360-1-alison.wang@nxp.com> X-Virus-Scanned: ClamAV using ClamSMTP X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230615_224312_564856_50533008 X-CRM114-Status: GOOD ( 13.45 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The msg id will be removed unpredictable when one thread calls ethosu_rpmsg_register and another thread calls ethosu_rpmsg_deregister at the same time. This patch adds write_lock in ethosu_rpmsg_register/ethosu_rpmsg_deregister and adds read_lcok in ethosu_rpmsg_find. Signed-off-by: Feng Guo Reviewed-by: Peng Fan --- drivers/firmware/ethosu/ethosu_driver.c | 2 +- drivers/firmware/ethosu/ethosu_rpmsg.c | 20 ++++++++++++++------ drivers/firmware/ethosu/ethosu_rpmsg.h | 1 + 3 files changed, 16 insertions(+), 7 deletions(-) diff --git a/drivers/firmware/ethosu/ethosu_driver.c b/drivers/firmware/ethosu/ethosu_driver.c index 59ae79256564..0742564cfaf2 100644 --- a/drivers/firmware/ethosu/ethosu_driver.c +++ b/drivers/firmware/ethosu/ethosu_driver.c @@ -155,7 +155,7 @@ static int __init ethosu_init(void) { int ret; - ethosu_class = class_create(THIS_MODULE, ETHOSU_DRIVER_NAME); + ethosu_class = class_create(ETHOSU_DRIVER_NAME); if (IS_ERR(ethosu_class)) { printk("Failed to create class '%s'.\n", ETHOSU_DRIVER_NAME); diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.c b/drivers/firmware/ethosu/ethosu_rpmsg.c index 351a1046e65e..d3e597e82762 100644 --- a/drivers/firmware/ethosu/ethosu_rpmsg.c +++ b/drivers/firmware/ethosu/ethosu_rpmsg.c @@ -35,7 +35,9 @@ static void ethosu_core_set_capacity(struct ethosu_buffer *buf, int ethosu_rpmsg_register(struct ethosu_rpmsg *erp, struct ethosu_rpmsg_msg *msg) { + write_lock(&erp->lock); msg->id = idr_alloc_cyclic(&erp->msg_idr, msg, 0, INT_MAX, GFP_KERNEL); + write_unlock(&erp->lock); if (msg->id < 0) return msg->id; @@ -45,14 +47,19 @@ int ethosu_rpmsg_register(struct ethosu_rpmsg *erp, void ethosu_rpmsg_deregister(struct ethosu_rpmsg *erp, struct ethosu_rpmsg_msg *msg) { + write_lock(&erp->lock); idr_remove(&erp->msg_idr, msg->id); + write_unlock(&erp->lock); } struct ethosu_rpmsg_msg *ethosu_rpmsg_find(struct ethosu_rpmsg *erp, int msg_id) { - struct ethosu_rpmsg_msg *ptr = - (struct ethosu_rpmsg_msg *)idr_find(&erp->msg_idr, msg_id); + struct ethosu_rpmsg_msg *ptr; + + read_lock(&erp->lock); + ptr = (struct ethosu_rpmsg_msg *)idr_find(&erp->msg_idr, msg_id); + read_unlock(&erp->lock); if (!ptr) return ERR_PTR(-EINVAL); @@ -97,8 +104,8 @@ static int ethosu_rpmsg_send(struct ethosu_rpmsg *erp, uint32_t type) msg.type = type; msg.length = 0; - print_hex_dump(KERN_DEBUG, __func__, DUMP_PREFIX_NONE, 16, 1, - (void *)&msg, sizeof(msg), true); + print_hex_dump_debug(__func__, DUMP_PREFIX_NONE, 16, 1, (void *)&msg, + sizeof(msg), true); ret = rpmsg_send(rpdev->ept, (void *)&msg, sizeof(msg)); if (ret) { @@ -337,8 +344,8 @@ static int rpmsg_ethosu_cb(struct rpmsg_device *rpdev, dev_dbg(&rpdev->dev, "msg(<- src 0x%x) len %d\n", src, len); - print_hex_dump(KERN_DEBUG, __func__, DUMP_PREFIX_NONE, 16, 1, - data, len, true); + print_hex_dump_debug(__func__, DUMP_PREFIX_NONE, 16, 1, data, + len, true); rpmsg->callback(rpmsg->user_arg, data); @@ -391,6 +398,7 @@ int ethosu_rpmsg_init(struct ethosu_rpmsg *erp, erp->user_arg = user_arg; erp->ping_count = 0; idr_init(&erp->msg_idr); + rwlock_init(&erp->lock); return register_rpmsg_driver(ðosu_rpmsg_driver); } diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.h b/drivers/firmware/ethosu/ethosu_rpmsg.h index 283d401a0dbf..a4b879e9ef6c 100644 --- a/drivers/firmware/ethosu/ethosu_rpmsg.h +++ b/drivers/firmware/ethosu/ethosu_rpmsg.h @@ -21,6 +21,7 @@ struct ethosu_core_msg; typedef void (*ethosu_rpmsg_cb)(void *user_arg, void *data); struct ethosu_rpmsg { + rwlock_t lock; struct rpmsg_device *rpdev; ethosu_rpmsg_cb callback; void *user_arg;