| Message ID | 20230616055913.2360-1-alison.wang@nxp.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | ethosu: Add Arm Ethos-U driver | expand |
On Fri, Jun 16, 2023 at 01:59:05PM +0800, Alison Wang wrote: > This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform. > > Ethos-U Linux driver is to provide an example of how a rich operating > system like Linux can dispatch inferences to an Arm Cortex-M subsystem, > consisting of an Arm Cortex-M and an Arm Ethos-U NPU. What is an "inference"? Where are the userspace tools (if any) that are needed for this driver to work properly? thanks, greg k-h
On Fri, Jun 16, 2023 at 01:59:05PM +0800, Alison Wang wrote: > This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform. > > Ethos-U Linux driver is to provide an example of how a rich operating > system like Linux can dispatch inferences to an Arm Cortex-M > subsystem, consisting of an Arm Cortex-M and an Arm Ethos-U NPU. What is an "inference"? [Alison Wang] The inference here means machine learning inference. It is the process of running data points into a machine learning model to calculate an output such as a single numerical score. For example, we use this driver, user space application and Cortex-M firmware to complete an inference about image classification. Where are the userspace tools (if any) that are needed for this driver to work properly? [Alison Wang] You could refer to https://github.com/nxp-imx/ethos-u-driver-stack-imx.git . thanks, greg k-h
On Fri, Jun 16, 2023 at 08:26:28AM +0000, Alison Wang wrote: > > On Fri, Jun 16, 2023 at 01:59:05PM +0800, Alison Wang wrote: > > This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform. > > > > Ethos-U Linux driver is to provide an example of how a rich operating > > system like Linux can dispatch inferences to an Arm Cortex-M > > subsystem, consisting of an Arm Cortex-M and an Arm Ethos-U NPU. > > What is an "inference"? > [Alison Wang] The inference here means machine learning inference. It is the process of > running data points into a machine learning model to calculate an output such as a single > numerical score. For example, we use this driver, user space application and Cortex-M firmware > to complete an inference about image classification. Nice, so why isn't this under drivers/accel/ then? Does it not need that framework? > Where are the userspace tools (if any) that are needed for this driver to work properly? > [Alison Wang] You could refer to https://github.com/nxp-imx/ethos-u-driver-stack-imx.git . Please put that in the documentation for the commit. thanks, greg k-h
On Fri, Jun 16, 2023 at 08:26:28AM +0000, Alison Wang wrote: > > On Fri, Jun 16, 2023 at 01:59:05PM +0800, Alison Wang wrote: > > This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform. > > > > Ethos-U Linux driver is to provide an example of how a rich > > operating system like Linux can dispatch inferences to an Arm > > Cortex-M subsystem, consisting of an Arm Cortex-M and an Arm Ethos-U NPU. > > What is an "inference"? > [Alison Wang] The inference here means machine learning inference. It > is the process of running data points into a machine learning model to > calculate an output such as a single numerical score. For example, we > use this driver, user space application and Cortex-M firmware to complete an inference about image classification. Nice, so why isn't this under drivers/accel/ then? Does it not need that framework? [Alison Wang] This driver doesn't need that framework. > Where are the userspace tools (if any) that are needed for this driver to work properly? > [Alison Wang] You could refer to https://github.com/nxp-imx/ethos-u-driver-stack-imx.git . Please put that in the documentation for the commit. [Alison Wang] Sure, I will. thanks, greg k-h
On Fri, Jun 16, 2023 at 10:47:56AM +0000, Alison Wang wrote: > On Fri, Jun 16, 2023 at 08:26:28AM +0000, Alison Wang wrote: > > > > On Fri, Jun 16, 2023 at 01:59:05PM +0800, Alison Wang wrote: > > > This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform. > > > > > > Ethos-U Linux driver is to provide an example of how a rich > > > operating system like Linux can dispatch inferences to an Arm > > > Cortex-M subsystem, consisting of an Arm Cortex-M and an Arm Ethos-U NPU. > > > > What is an "inference"? > > [Alison Wang] The inference here means machine learning inference. It > > is the process of running data points into a machine learning model to > > calculate an output such as a single numerical score. For example, we > > use this driver, user space application and Cortex-M firmware to complete an inference about image classification. > > Nice, so why isn't this under drivers/accel/ then? Does it not need that framework? > [Alison Wang] This driver doesn't need that framework. Odd quoting style... Anyway, why not use that famework? That's what that subsystem is for, so you are going to have to document why this does not fit into that model at all and you need your own interface instead. thanks, greg k-h
This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform. Ethos-U Linux driver is to provide an example of how a rich operating system like Linux can dispatch inferences to an Arm Cortex-M subsystem, consisting of an Arm Cortex-M and an Arm Ethos-U NPU. ---------------------------------------------------------------- Alison Wang (8): ethosu: Add Arm Ethos-U driver ethosu: Use RPMsg messaging protocol based on i.MX Rpmsg implementation ethosu: Add inference type option for model and op ethosu: Add suspend/resume power management ethosu: Use ids for identifying messages sent to Ethos-U firmware ethosu: Add core message about network info ethosu: Add core message about inference cancellation ethosu: Add rwlock when alloc and remove msg id drivers/firmware/Kconfig | 1 + drivers/firmware/Makefile | 1 + drivers/firmware/ethosu/Kconfig | 24 ++++ drivers/firmware/ethosu/Makefile | 31 +++++ drivers/firmware/ethosu/ethosu_buffer.c | 319 +++++++++++++++++++++++++++++++++++++++++++ drivers/firmware/ethosu/ethosu_buffer.h | 106 +++++++++++++++ drivers/firmware/ethosu/ethosu_cancel_inference.c | 185 +++++++++++++++++++++++++ drivers/firmware/ethosu/ethosu_cancel_inference.h | 55 ++++++++ drivers/firmware/ethosu/ethosu_capabilities.c | 157 ++++++++++++++++++++++ drivers/firmware/ethosu/ethosu_capabilities.h | 47 +++++++ drivers/firmware/ethosu/ethosu_core_interface.h | 276 ++++++++++++++++++++++++++++++++++++++ drivers/firmware/ethosu/ethosu_device.c | 404 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ drivers/firmware/ethosu/ethosu_device.h | 81 +++++++++++ drivers/firmware/ethosu/ethosu_driver.c | 201 +++++++++++++++++++++++++++ drivers/firmware/ethosu/ethosu_inference.c | 529 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ drivers/firmware/ethosu/ethosu_inference.h | 125 +++++++++++++++++ drivers/firmware/ethosu/ethosu_network.c | 229 +++++++++++++++++++++++++++++++ drivers/firmware/ethosu/ethosu_network.h | 84 ++++++++++++ drivers/firmware/ethosu/ethosu_network_info.c | 184 +++++++++++++++++++++++++ drivers/firmware/ethosu/ethosu_network_info.h | 56 ++++++++ drivers/firmware/ethosu/ethosu_rpmsg.c | 414 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ drivers/firmware/ethosu/ethosu_rpmsg.h | 155 +++++++++++++++++++++ drivers/firmware/ethosu/uapi/ethosu.h | 285 +++++++++++++++++++++++++++++++++++++++ 23 files changed, 3949 insertions(+) create mode 100644 drivers/firmware/ethosu/Kconfig create mode 100644 drivers/firmware/ethosu/Makefile create mode 100644 drivers/firmware/ethosu/ethosu_buffer.c create mode 100644 drivers/firmware/ethosu/ethosu_buffer.h create mode 100644 drivers/firmware/ethosu/ethosu_cancel_inference.c create mode 100644 drivers/firmware/ethosu/ethosu_cancel_inference.h create mode 100644 drivers/firmware/ethosu/ethosu_capabilities.c create mode 100644 drivers/firmware/ethosu/ethosu_capabilities.h create mode 100644 drivers/firmware/ethosu/ethosu_core_interface.h create mode 100644 drivers/firmware/ethosu/ethosu_device.c create mode 100644 drivers/firmware/ethosu/ethosu_device.h create mode 100644 drivers/firmware/ethosu/ethosu_driver.c create mode 100644 drivers/firmware/ethosu/ethosu_inference.c create mode 100644 drivers/firmware/ethosu/ethosu_inference.h create mode 100644 drivers/firmware/ethosu/ethosu_network.c create mode 100644 drivers/firmware/ethosu/ethosu_network.h create mode 100644 drivers/firmware/ethosu/ethosu_network_info.c create mode 100644 drivers/firmware/ethosu/ethosu_network_info.h create mode 100644 drivers/firmware/ethosu/ethosu_rpmsg.c create mode 100644 drivers/firmware/ethosu/ethosu_rpmsg.h create mode 100644 drivers/firmware/ethosu/uapi/ethosu.h