| Message ID | 20241023053357.5261-4-yesanishhere@gmail.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | remoteproc documentation changes | expand |
On Tue, Oct 22, 2024 at 10:33:53PM -0700, anish kumar wrote: > Added following changes: > 1. Components provided by remoteproc framework. > 2. Remoteproc driver responsibilities. > 3. Remoteproc framework responsibilities. > 4. Better explanation of how to ask for resources > from the framework by the remote processor. > > Signed-off-by: anish kumar <yesanishhere@gmail.com> > --- > .../driver-api/remoteproc/remoteproc.rst | 596 +++++++----------- > 1 file changed, 243 insertions(+), 353 deletions(-) > > diff --git a/Documentation/driver-api/remoteproc/remoteproc.rst b/Documentation/driver-api/remoteproc/remoteproc.rst > index 9cccd3dd6a4b..7ca545eea153 100644 > --- a/Documentation/driver-api/remoteproc/remoteproc.rst > +++ b/Documentation/driver-api/remoteproc/remoteproc.rst > @@ -1,359 +1,249 @@ > +.. SPDX-License-Identifier: GPL-2.0 > + > ========================== > Remote Processor Framework > ========================== > > -Introduction > -============ > - > -Modern SoCs typically have heterogeneous remote processor devices in asymmetric > -multiprocessing (AMP) configurations, which may be running different instances > -of operating system, whether it's Linux or any other flavor of real-time OS. > - > -OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. > -In a typical configuration, the dual cortex-A9 is running Linux in a SMP > -configuration, and each of the other three cores (two M3 cores and a DSP) > -is running its own instance of RTOS in an AMP configuration. > - > -The remoteproc framework allows different platforms/architectures to > -control (power on, load firmware, power off) those remote processors while > -abstracting the hardware differences, so the entire driver doesn't need to be > -duplicated. In addition, this framework also adds rpmsg virtio devices > -for remote processors that supports this kind of communication. This way, > -platform-specific remoteproc drivers only need to provide a few low-level > -handlers, and then all rpmsg drivers will then just work > -(for more information about the virtio-based rpmsg bus and its drivers, > -please read Documentation/staging/rpmsg.rst). > -Registration of other types of virtio devices is now also possible. Firmwares > -just need to publish what kind of virtio devices do they support, and then > -remoteproc will add those devices. This makes it possible to reuse the > -existing virtio drivers with remote processor backends at a minimal development > -cost. > - > -User API > -======== > - > -:: > - > - int rproc_boot(struct rproc *rproc) > - > -Boot a remote processor (i.e. load its firmware, power it on, ...). > - > -If the remote processor is already powered on, this function immediately > -returns (successfully). > - > -Returns 0 on success, and an appropriate error value otherwise. > -Note: to use this function you should already have a valid rproc > -handle. There are several ways to achieve that cleanly (devres, pdata, > -the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we > -might also consider using dev_archdata for this). > - > -:: > - > - void rproc_shutdown(struct rproc *rproc) > - > -Power off a remote processor (previously booted with rproc_boot()). > -In case @rproc is still being used by an additional user(s), then > -this function will just decrement the power refcount and exit, > -without really powering off the device. > - > -Every call to rproc_boot() must (eventually) be accompanied by a call > -to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. > - > -.. note:: > - > - we're not decrementing the rproc's refcount, only the power refcount. > - which means that the @rproc handle stays valid even after > - rproc_shutdown() returns, and users can still use it with a subsequent > - rproc_boot(), if needed. > - > -:: > - > - struct rproc *rproc_get_by_phandle(phandle phandle) > - > -Find an rproc handle using a device tree phandle. Returns the rproc > -handle on success, and NULL on failure. This function increments > -the remote processor's refcount, so always use rproc_put() to > -decrement it back once rproc isn't needed anymore. > - > -Typical usage > -============= > - > -:: > - > - #include <linux/remoteproc.h> > - > - /* in case we were given a valid 'rproc' handle */ > - int dummy_rproc_example(struct rproc *my_rproc) > - { > - int ret; > - > - /* let's power on and boot our remote processor */ > - ret = rproc_boot(my_rproc); > - if (ret) { > - /* > - * something went wrong. handle it and leave. > - */ > - } > - > - /* > - * our remote processor is now powered on... give it some work > - */ > - > - /* let's shut it down now */ > - rproc_shutdown(my_rproc); > - } > - > -API for implementors > -==================== > - > -:: > - > - struct rproc *rproc_alloc(struct device *dev, const char *name, > - const struct rproc_ops *ops, > - const char *firmware, int len) > - > -Allocate a new remote processor handle, but don't register > -it yet. Required parameters are the underlying device, the > -name of this remote processor, platform-specific ops handlers, > -the name of the firmware to boot this rproc with, and the > -length of private data needed by the allocating rproc driver (in bytes). > - > -This function should be used by rproc implementations during > -initialization of the remote processor. > - > -After creating an rproc handle using this function, and when ready, > -implementations should then call rproc_add() to complete > -the registration of the remote processor. > - > -On success, the new rproc is returned, and on failure, NULL. > - > -.. note:: > - > - **never** directly deallocate @rproc, even if it was not registered > - yet. Instead, when you need to unroll rproc_alloc(), use rproc_free(). > - > -:: > - > - void rproc_free(struct rproc *rproc) > - > -Free an rproc handle that was allocated by rproc_alloc. > - > -This function essentially unrolls rproc_alloc(), by decrementing the > -rproc's refcount. It doesn't directly free rproc; that would happen > -only if there are no other references to rproc and its refcount now > -dropped to zero. > - > -:: > - > - int rproc_add(struct rproc *rproc) > - > -Register @rproc with the remoteproc framework, after it has been > -allocated with rproc_alloc(). > - > -This is called by the platform-specific rproc implementation, whenever > -a new remote processor device is probed. > - > -Returns 0 on success and an appropriate error code otherwise. > -Note: this function initiates an asynchronous firmware loading > -context, which will look for virtio devices supported by the rproc's > -firmware. > - > -If found, those virtio devices will be created and added, so as a result > -of registering this remote processor, additional virtio drivers might get > -probed. > - > -:: > - > - int rproc_del(struct rproc *rproc) > - > -Unroll rproc_add(). > - > -This function should be called when the platform specific rproc > -implementation decides to remove the rproc device. it should > -_only_ be called if a previous invocation of rproc_add() > -has completed successfully. > - > -After rproc_del() returns, @rproc is still valid, and its > -last refcount should be decremented by calling rproc_free(). > - > -Returns 0 on success and -EINVAL if @rproc isn't valid. > - > -:: > - > - void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type) > - > -Report a crash in a remoteproc > - > -This function must be called every time a crash is detected by the > -platform specific rproc implementation. This should not be called from a > -non-remoteproc driver. This function can be called from atomic/interrupt > -context. > - > -Implementation callbacks > +.. Contents: > + > + 1. Introduction > + 2. Remoteproc framework responsibilities > + 3. Remoteproc driver responsibilities > + 4. Virtio and rpmsg > + > +1. Introduction > +=============== > + > +Modern System on Chips (SoCs) typically integrate heterogeneous remote > +processor devices in asymmetric multiprocessing (AMP) configurations. > +These processors may run different operating systems, such as Linux and > +various real-time operating systems (RTOS). You are moving things around _and_ making modifications to the text in the same patch, something I specifically asked not to do. Moreover, the above conveys exactly the same information as found in [1] but using different words. I am in favour of enhancing documentation but not creating unneeded churn. I found several instances of the same rewording pattern in the sections below. As such I will not look at the other patches nor move forward with this set. Thanks, Mathieu [1]. https://elixir.bootlin.com/linux/v6.12-rc4/source/Documentation/staging/remoteproc.rst > + > +For example, the OMAP4 platform features dual Cortex-A9 cores, dual > +Cortex-M3 cores, and a C64x+ DSP. In a standard setup, the Cortex-A9 > +cores execute Linux in a symmetric multiprocessing (SMP) configuration, > +while the M3 cores and DSP run independent instances of an RTOS. > + > +The remoteproc framework allows various platforms and architectures to > +manage remote processors, including operations such as powering on, > +loading firmware, and powering off. This framework abstracts hardware > +differences, promoting code reuse and minimizing duplication. It also > +supports rpmsg virtio devices for remote processors that utilize this > +communication method. Consequently, platform-specific remoteproc drivers > +need only implement a few low-level handlers, enabling seamless operation > +of all rpmsg drivers. (For more details about the virtio-based rpmsg > +bus and its drivers, refer to rpmsg documentation.) > + > +Additionally, the framework allows for the registration of various > +virtio devices. Firmware can publish the types of virtio devices it > +supports, facilitating their addition to the remoteproc framework. This > +flexibility enables the reuse of existing virtio drivers with remote > +processor backends at minimal development cost. > + > +The primary purpose of the remoteproc framework is to download firmware > +for remote processors and manage their lifecycle. The framework consists > +of several key components: > + > +- **Character Driver**: Provides userspace access to control the remote > + processor. > +- **ELF Utility**: Offers functions for handling ELF files and managing > + resources requested by the remote processor. > +- **Remoteproc Core**: Manages firmware downloads and recovery actions > + in case of a remote processor crash. > +- **Coredump**: Provides facilities for coredumping and tracing from > + the remote processor in the event of a crash. > +- **Userspace Interaction**: Uses sysfs and debugfs to manage the > + lifecycle and status of the remote processor. > +- **Virtio Support**: Facilitates interaction with the virtio and > + rpmsg bus. > + > +From here on, references to "framework" denote the remoteproc > +framework, and "driver" refers to the remoteproc driver that utilizes > +the framework for managing remote processors. > + > +2. Remoteproc framework Responsibilities > +======================================== > + > +The framework begins by gathering information about the firmware file > +to be downloaded through the request_firmware function. It supports > +the ELF format and parses the firmware image to identify the physical > +addresses that need to be populated from the corresponding ELF sections. > +The framework also requires knowledge of the logical or I/O-mapped > +addresses in the application processor. Once this information is > +obtained from the driver, the framework transfers the data to the > +specified addresses and starts the remote, along with > +any devices physically or logically connected to it. > + > +Dependent devices, referred to as `subdevices` within the framework, > +are also managed post-registration by their respective drivers. > +Subdevices can register themselves using `rproc_(add/remove)_subdev`. > +Non-remoteproc drivers can use subdevices as a way to logically connect > +to remote and get lifecycle notifications of the remote. > + > +The framework oversees the lifecycle of the remote and > +provides the `rproc_report_crash` function, which the driver invokes > +upon receiving a crash notification from the remote. The > +notification method can differ based on the design of the remote > +processor and its communication with the application processor. For > +instance, if the remote is a DSP equipped with a watchdog, > +unresponsive behavior triggers the watchdog, generating an interrupt > +that routes to the application processor, allowing it to call > +`rproc_report_crash` in the driver's interrupt context. > + > +During crash handling, the framework performs the following actions: > + > +a. Sends a request to stop the remote and any connected or > + dependent subdevices. > +b. Generates a coredump, dumping all `resources` requested by the > + remote alongside relevant debugging information. Resources are > + explained below. > +c. Reloads the firmware and restarts the remote. > + > +If the `RPROC_FEAT_ATTACH_ON_RECOVERY` flag is set, the detach and > +attach callbacks of the driver are invoked without reloading the > +firmware. This is useful when the remote requires no > +assistance for recovery, or when the application processor can restart > +independently. After recovery, the application processor can reattach > +to the remote. > + > +The remote can request resources from the framework, which > +allocates a ".resource_table" section. During the ELF parsing phase, > +the framework identifies this section and calls the appropriate > +handler to allocate the requested resources. > + > +Resource management within the framework can accommodate any type of > +`fw_resource_type`. > + > +.. code-block:: c > + > + enum fw_resource_type { > + RSC_CARVEOUT = 0, > + RSC_DEVMEM = 1, > + RSC_TRACE = 2, > + RSC_VDEV = 3, > + RSC_LAST = 4, > + RSC_VENDOR_START = 128, > + RSC_VENDOR_END = 512, > + }; > + > + struct resource_table { > + u32 ver; > + u32 num; > + u32 reserved[2]; > + u32 offset[]; > + } __packed; > + > + struct fw_rsc_hdr { > + u32 type; > + u8 data[]; > + } __packed; > + > +For example, if the remote requests both `RSC_TRACE` and > +`RSC_CARVEOUT` for memory allocation, the ELF firmware can be structured > +as follows: > + > +.. code-block:: c > + > + #define MAX_SHARED_RESOURCE 2 > + #define LOG_BUF_SIZE 1000 > + #define CARVEOUT_DUMP_PA 0x12345678 > + #define CARVEOUT_DUMP_SIZE 2000 > + > + struct shared_resource_table { > + u32 ver; > + u32 num; > + u32 reserved[2]; > + u32 offset[MAX_SHARED_RESOURCE]; > + struct fw_rsc_trace log_trace; > + struct fw_rsc_carveout dump_carveout; > + }; > + > + volatile struct shared_resource_table table = { > + .ver = 1, > + .num = 2, > + .reserved = {0, 0}, > + .offset = { > + offsetof(struct resource_table, log_trace), > + offsetof(struct resource_table, dump_carveout), > + }, > + .log_trace = { > + RSC_TRACE, > + (u32)log_buf, LOG_BUF_SIZE, 0, "log_trace", > + }, > + .dump_carveout = { > + RSC_CARVEOUT, > + (u32)FW_RSC_ADDR_ANY, CARVEOUT_PA, 0, "carveout_dump", > + }, > + }; > + > +The framework creates a sysfs file when it encounters the `RSC_TRACE` > +type to expose log information to userspace. Other resource types are > +handled accordingly. In the example above, `CARVEOUT_DUMP_SIZE` bytes > +of DMA memory will be allocated starting from `CARVEOUT_DUMP_PA`. > + > + > +3. Remoteproc driver responsibilities > +===================================== > + > +The driver must provide the following information to the core: > + > +a. Translate device addresses (physical addresses) found in the ELF > + firmware to virtual addresses in Linux using the `da_to_va` > + callback. This allows the framework to copy ELF firmware from the > + filesystem to the addresses expected by the remote since > + the framework cannot directly access those physical addresses. > +b. Prepare/unprepare the remote prior to firmware loading, > + which may involve allocating carveout and reserved memory regions. > +c. Implement methods for starting and stopping the remote, > + whether by setting registers or sending explicit interrupts, > + depending on the hardware design. > +d. Provide attach and detach callbacks to start the remote > + without loading the firmware. This is beneficial when the remote > + processor is already loaded and running. > +e. Implement a load callback for firmware loading, typically using > + the ELF loader provided by the framework; currently, only ELF > + format is supported. > +f. Invoke the framework's crash handler API upon detecting a remote > + crash. > + > +Drivers must fill the `rproc_ops` structure and call `rproc_alloc` > +to register themselves with the framework. > + > +.. code-block:: c > + > + struct rproc_ops { > + int (*prepare)(struct rproc *rproc); > + int (*unprepare)(struct rproc *rproc); > + int (*start)(struct rproc *rproc); > + int (*stop)(struct rproc *rproc); > + int (*attach)(struct rproc *rproc); > + int (*detach)(struct rproc *rproc); > + void * (*da_to_va)(struct rproc *rproc, u64 da, size_t len, > + bool *is_iomem); > + int (*parse_fw)(struct rproc *rproc, const struct firmware *fw); > + int (*handle_rsc)(struct rproc *rproc, u32 rsc_type, > + void *rsc, int offset, int avail); > + int (*load)(struct rproc *rproc, const struct firmware *fw); > + //snip > + }; > + > + > +4. Virtio and Remoteproc > ======================== > > -These callbacks should be provided by platform-specific remoteproc > -drivers:: > - > - /** > - * struct rproc_ops - platform-specific device handlers > - * @start: power on the device and boot it > - * @stop: power off the device > - * @kick: kick a virtqueue (virtqueue id given as a parameter) > - */ > - struct rproc_ops { > - int (*start)(struct rproc *rproc); > - int (*stop)(struct rproc *rproc); > - void (*kick)(struct rproc *rproc, int vqid); > - }; > - > -Every remoteproc implementation should at least provide the ->start and ->stop > -handlers. If rpmsg/virtio functionality is also desired, then the ->kick handler > -should be provided as well. > - > -The ->start() handler takes an rproc handle and should then power on the > -device and boot it (use rproc->priv to access platform-specific private data). > -The boot address, in case needed, can be found in rproc->bootaddr (remoteproc > -core puts there the ELF entry point). > -On success, 0 should be returned, and on failure, an appropriate error code. > - > -The ->stop() handler takes an rproc handle and powers the device down. > -On success, 0 is returned, and on failure, an appropriate error code. > - > -The ->kick() handler takes an rproc handle, and an index of a virtqueue > -where new message was placed in. Implementations should interrupt the remote > -processor and let it know it has pending messages. Notifying remote processors > -the exact virtqueue index to look in is optional: it is easy (and not > -too expensive) to go through the existing virtqueues and look for new buffers > -in the used rings. > - > -Binary Firmware Structure > -========================= > - > -At this point remoteproc supports ELF32 and ELF64 firmware binaries. However, > -it is quite expected that other platforms/devices which we'd want to > -support with this framework will be based on different binary formats. > - > -When those use cases show up, we will have to decouple the binary format > -from the framework core, so we can support several binary formats without > -duplicating common code. > - > -When the firmware is parsed, its various segments are loaded to memory > -according to the specified device address (might be a physical address > -if the remote processor is accessing memory directly). > - > -In addition to the standard ELF segments, most remote processors would > -also include a special section which we call "the resource table". > - > -The resource table contains system resources that the remote processor > -requires before it should be powered on, such as allocation of physically > -contiguous memory, or iommu mapping of certain on-chip peripherals. > -Remotecore will only power up the device after all the resource table's > -requirement are met. > - > -In addition to system resources, the resource table may also contain > -resource entries that publish the existence of supported features > -or configurations by the remote processor, such as trace buffers and > -supported virtio devices (and their configurations). > - > -The resource table begins with this header:: > - > - /** > - * struct resource_table - firmware resource table header > - * @ver: version number > - * @num: number of resource entries > - * @reserved: reserved (must be zero) > - * @offset: array of offsets pointing at the various resource entries > - * > - * The header of the resource table, as expressed by this structure, > - * contains a version number (should we need to change this format in the > - * future), the number of available resource entries, and their offsets > - * in the table. > - */ > - struct resource_table { > - u32 ver; > - u32 num; > - u32 reserved[2]; > - u32 offset[0]; > - } __packed; > - > -Immediately following this header are the resource entries themselves, > -each of which begins with the following resource entry header:: > - > - /** > - * struct fw_rsc_hdr - firmware resource entry header > - * @type: resource type > - * @data: resource data > - * > - * Every resource entry begins with a 'struct fw_rsc_hdr' header providing > - * its @type. The content of the entry itself will immediately follow > - * this header, and it should be parsed according to the resource type. > - */ > - struct fw_rsc_hdr { > - u32 type; > - u8 data[0]; > - } __packed; > - > -Some resources entries are mere announcements, where the host is informed > -of specific remoteproc configuration. Other entries require the host to > -do something (e.g. allocate a system resource). Sometimes a negotiation > -is expected, where the firmware requests a resource, and once allocated, > -the host should provide back its details (e.g. address of an allocated > -memory region). > - > -Here are the various resource types that are currently supported:: > - > - /** > - * enum fw_resource_type - types of resource entries > - * > - * @RSC_CARVEOUT: request for allocation of a physically contiguous > - * memory region. > - * @RSC_DEVMEM: request to iommu_map a memory-based peripheral. > - * @RSC_TRACE: announces the availability of a trace buffer into which > - * the remote processor will be writing logs. > - * @RSC_VDEV: declare support for a virtio device, and serve as its > - * virtio header. > - * @RSC_LAST: just keep this one at the end > - * @RSC_VENDOR_START: start of the vendor specific resource types range > - * @RSC_VENDOR_END: end of the vendor specific resource types range > - * > - * Please note that these values are used as indices to the rproc_handle_rsc > - * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to > - * check the validity of an index before the lookup table is accessed, so > - * please update it as needed. > - */ > - enum fw_resource_type { > - RSC_CARVEOUT = 0, > - RSC_DEVMEM = 1, > - RSC_TRACE = 2, > - RSC_VDEV = 3, > - RSC_LAST = 4, > - RSC_VENDOR_START = 128, > - RSC_VENDOR_END = 512, > - }; > - > -For more details regarding a specific resource type, please see its > -dedicated structure in include/linux/remoteproc.h. > - > -We also expect that platform-specific resource entries will show up > -at some point. When that happens, we could easily add a new RSC_PLATFORM > -type, and hand those resources to the platform-specific rproc driver to handle. > - > -Virtio and remoteproc > -===================== > - > -The firmware should provide remoteproc information about virtio devices > -that it supports, and their configurations: a RSC_VDEV resource entry > -should specify the virtio device id (as in virtio_ids.h), virtio features, > -virtio config space, vrings information, etc. > - > -When a new remote processor is registered, the remoteproc framework > -will look for its resource table and will register the virtio devices > -it supports. A firmware may support any number of virtio devices, and > -of any type (a single remote processor can also easily support several > -rpmsg virtio devices this way, if desired). > - > -Of course, RSC_VDEV resource entries are only good enough for static > -allocation of virtio devices. Dynamic allocations will also be made possible > -using the rpmsg bus (similar to how we already do dynamic allocations of > -rpmsg channels; read more about it in rpmsg.txt). > +The firmware must provide remoteproc with information regarding the > +virtio devices it supports and their configurations: an `RSC_VDEV` > +resource entry should detail the virtio device ID (as defined in > +`virtio_ids.h`), virtio features, virtio config space, vrings > +information, etc. > + > +Upon registration of a new remote, the remoteproc framework > +searches for its resource table and registers the supported virtio > +devices. A firmware may support multiple virtio devices, of various > +types (a single remote can support multiple rpmsg virtio > +devices if required). > + > +Moreover, `RSC_VDEV` resource entries suffice for static allocation > +of virtio devices. Dynamic allocations will also be supported using > +the rpmsg bus, akin to the handling of dynamic allocations for rpmsg > +channels. For more information, refer to rpmsg documentation. > -- > 2.39.3 (Apple Git-146) >
On Thu, Oct 24, 2024 at 8:53 AM Mathieu Poirier <mathieu.poirier@linaro.org> wrote: > > On Tue, Oct 22, 2024 at 10:33:53PM -0700, anish kumar wrote: > > Added following changes: > > 1. Components provided by remoteproc framework. > > 2. Remoteproc driver responsibilities. > > 3. Remoteproc framework responsibilities. > > 4. Better explanation of how to ask for resources > > from the framework by the remote processor. > > > > Signed-off-by: anish kumar <yesanishhere@gmail.com> > > --- > > .../driver-api/remoteproc/remoteproc.rst | 596 +++++++----------- > > 1 file changed, 243 insertions(+), 353 deletions(-) > > > > diff --git a/Documentation/driver-api/remoteproc/remoteproc.rst b/Documentation/driver-api/remoteproc/remoteproc.rst > > index 9cccd3dd6a4b..7ca545eea153 100644 > > --- a/Documentation/driver-api/remoteproc/remoteproc.rst > > +++ b/Documentation/driver-api/remoteproc/remoteproc.rst > > @@ -1,359 +1,249 @@ > > +.. SPDX-License-Identifier: GPL-2.0 > > + > > ========================== > > Remote Processor Framework > > ========================== > > > > -Introduction > > -============ > > - > > -Modern SoCs typically have heterogeneous remote processor devices in asymmetric > > -multiprocessing (AMP) configurations, which may be running different instances > > -of operating system, whether it's Linux or any other flavor of real-time OS. > > - > > -OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. > > -In a typical configuration, the dual cortex-A9 is running Linux in a SMP > > -configuration, and each of the other three cores (two M3 cores and a DSP) > > -is running its own instance of RTOS in an AMP configuration. > > - > > -The remoteproc framework allows different platforms/architectures to > > -control (power on, load firmware, power off) those remote processors while > > -abstracting the hardware differences, so the entire driver doesn't need to be > > -duplicated. In addition, this framework also adds rpmsg virtio devices > > -for remote processors that supports this kind of communication. This way, > > -platform-specific remoteproc drivers only need to provide a few low-level > > -handlers, and then all rpmsg drivers will then just work > > -(for more information about the virtio-based rpmsg bus and its drivers, > > -please read Documentation/staging/rpmsg.rst). > > -Registration of other types of virtio devices is now also possible. Firmwares > > -just need to publish what kind of virtio devices do they support, and then > > -remoteproc will add those devices. This makes it possible to reuse the > > -existing virtio drivers with remote processor backends at a minimal development > > -cost. > > - > > -User API > > -======== > > - > > -:: > > - > > - int rproc_boot(struct rproc *rproc) > > - > > -Boot a remote processor (i.e. load its firmware, power it on, ...). > > - > > -If the remote processor is already powered on, this function immediately > > -returns (successfully). > > - > > -Returns 0 on success, and an appropriate error value otherwise. > > -Note: to use this function you should already have a valid rproc > > -handle. There are several ways to achieve that cleanly (devres, pdata, > > -the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we > > -might also consider using dev_archdata for this). > > - > > -:: > > - > > - void rproc_shutdown(struct rproc *rproc) > > - > > -Power off a remote processor (previously booted with rproc_boot()). > > -In case @rproc is still being used by an additional user(s), then > > -this function will just decrement the power refcount and exit, > > -without really powering off the device. > > - > > -Every call to rproc_boot() must (eventually) be accompanied by a call > > -to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. > > - > > -.. note:: > > - > > - we're not decrementing the rproc's refcount, only the power refcount. > > - which means that the @rproc handle stays valid even after > > - rproc_shutdown() returns, and users can still use it with a subsequent > > - rproc_boot(), if needed. > > - > > -:: > > - > > - struct rproc *rproc_get_by_phandle(phandle phandle) > > - > > -Find an rproc handle using a device tree phandle. Returns the rproc > > -handle on success, and NULL on failure. This function increments > > -the remote processor's refcount, so always use rproc_put() to > > -decrement it back once rproc isn't needed anymore. > > - > > -Typical usage > > -============= > > - > > -:: > > - > > - #include <linux/remoteproc.h> > > - > > - /* in case we were given a valid 'rproc' handle */ > > - int dummy_rproc_example(struct rproc *my_rproc) > > - { > > - int ret; > > - > > - /* let's power on and boot our remote processor */ > > - ret = rproc_boot(my_rproc); > > - if (ret) { > > - /* > > - * something went wrong. handle it and leave. > > - */ > > - } > > - > > - /* > > - * our remote processor is now powered on... give it some work > > - */ > > - > > - /* let's shut it down now */ > > - rproc_shutdown(my_rproc); > > - } > > - > > -API for implementors > > -==================== > > - > > -:: > > - > > - struct rproc *rproc_alloc(struct device *dev, const char *name, > > - const struct rproc_ops *ops, > > - const char *firmware, int len) > > - > > -Allocate a new remote processor handle, but don't register > > -it yet. Required parameters are the underlying device, the > > -name of this remote processor, platform-specific ops handlers, > > -the name of the firmware to boot this rproc with, and the > > -length of private data needed by the allocating rproc driver (in bytes). > > - > > -This function should be used by rproc implementations during > > -initialization of the remote processor. > > - > > -After creating an rproc handle using this function, and when ready, > > -implementations should then call rproc_add() to complete > > -the registration of the remote processor. > > - > > -On success, the new rproc is returned, and on failure, NULL. > > - > > -.. note:: > > - > > - **never** directly deallocate @rproc, even if it was not registered > > - yet. Instead, when you need to unroll rproc_alloc(), use rproc_free(). > > - > > -:: > > - > > - void rproc_free(struct rproc *rproc) > > - > > -Free an rproc handle that was allocated by rproc_alloc. > > - > > -This function essentially unrolls rproc_alloc(), by decrementing the > > -rproc's refcount. It doesn't directly free rproc; that would happen > > -only if there are no other references to rproc and its refcount now > > -dropped to zero. > > - > > -:: > > - > > - int rproc_add(struct rproc *rproc) > > - > > -Register @rproc with the remoteproc framework, after it has been > > -allocated with rproc_alloc(). > > - > > -This is called by the platform-specific rproc implementation, whenever > > -a new remote processor device is probed. > > - > > -Returns 0 on success and an appropriate error code otherwise. > > -Note: this function initiates an asynchronous firmware loading > > -context, which will look for virtio devices supported by the rproc's > > -firmware. > > - > > -If found, those virtio devices will be created and added, so as a result > > -of registering this remote processor, additional virtio drivers might get > > -probed. > > - > > -:: > > - > > - int rproc_del(struct rproc *rproc) > > - > > -Unroll rproc_add(). > > - > > -This function should be called when the platform specific rproc > > -implementation decides to remove the rproc device. it should > > -_only_ be called if a previous invocation of rproc_add() > > -has completed successfully. > > - > > -After rproc_del() returns, @rproc is still valid, and its > > -last refcount should be decremented by calling rproc_free(). > > - > > -Returns 0 on success and -EINVAL if @rproc isn't valid. > > - > > -:: > > - > > - void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type) > > - > > -Report a crash in a remoteproc > > - > > -This function must be called every time a crash is detected by the > > -platform specific rproc implementation. This should not be called from a > > -non-remoteproc driver. This function can be called from atomic/interrupt > > -context. > > - > > -Implementation callbacks > > +.. Contents: > > + > > + 1. Introduction > > + 2. Remoteproc framework responsibilities > > + 3. Remoteproc driver responsibilities > > + 4. Virtio and rpmsg > > + > > +1. Introduction > > +=============== > > + > > +Modern System on Chips (SoCs) typically integrate heterogeneous remote > > +processor devices in asymmetric multiprocessing (AMP) configurations. > > +These processors may run different operating systems, such as Linux and > > +various real-time operating systems (RTOS). > > You are moving things around _and_ making modifications to the text in the same > patch, something I specifically asked not to do. Moreover, the above conveys Sorry for doing that, I thought I moved the file first as you instructed but I didn't understand that I should also not reword the sentences that convey the same meaning. Will keep the text of below content in the same file: 1. Introduction 2. Binary Firmware Structure: will just add example as shown in this patch. 3. Virtio and remoteproc: didn't change anything in the patch for this. For below: 1. User API: 2. Typical usage: 3. API for implementors : Should I just move all of the above text as it is to remoteproc-kernel-api.rst or you prefer here as I feel it belongs there? > exactly the same information as found in [1] but using different words. I > am in favour of enhancing documentation but not creating unneeded churn. > > I found several instances of the same rewording pattern in the sections below. > As such I will not look at the other patches nor move forward with this set. > > Thanks, > Mathieu Thanks Mathieu, for reviewing and providing feedback as always. > > [1]. https://elixir.bootlin.com/linux/v6.12-rc4/source/Documentation/staging/remoteproc.rst > > > + > > +For example, the OMAP4 platform features dual Cortex-A9 cores, dual > > +Cortex-M3 cores, and a C64x+ DSP. In a standard setup, the Cortex-A9 > > +cores execute Linux in a symmetric multiprocessing (SMP) configuration, > > +while the M3 cores and DSP run independent instances of an RTOS. > > + > > +The remoteproc framework allows various platforms and architectures to > > +manage remote processors, including operations such as powering on, > > +loading firmware, and powering off. This framework abstracts hardware > > +differences, promoting code reuse and minimizing duplication. It also > > +supports rpmsg virtio devices for remote processors that utilize this > > +communication method. Consequently, platform-specific remoteproc drivers > > +need only implement a few low-level handlers, enabling seamless operation > > +of all rpmsg drivers. (For more details about the virtio-based rpmsg > > +bus and its drivers, refer to rpmsg documentation.) > > + > > +Additionally, the framework allows for the registration of various > > +virtio devices. Firmware can publish the types of virtio devices it > > +supports, facilitating their addition to the remoteproc framework. This > > +flexibility enables the reuse of existing virtio drivers with remote > > +processor backends at minimal development cost. > > + > > +The primary purpose of the remoteproc framework is to download firmware > > +for remote processors and manage their lifecycle. The framework consists > > +of several key components: > > + > > +- **Character Driver**: Provides userspace access to control the remote > > + processor. > > +- **ELF Utility**: Offers functions for handling ELF files and managing > > + resources requested by the remote processor. > > +- **Remoteproc Core**: Manages firmware downloads and recovery actions > > + in case of a remote processor crash. > > +- **Coredump**: Provides facilities for coredumping and tracing from > > + the remote processor in the event of a crash. > > +- **Userspace Interaction**: Uses sysfs and debugfs to manage the > > + lifecycle and status of the remote processor. > > +- **Virtio Support**: Facilitates interaction with the virtio and > > + rpmsg bus. > > + > > +From here on, references to "framework" denote the remoteproc > > +framework, and "driver" refers to the remoteproc driver that utilizes > > +the framework for managing remote processors. > > + > > +2. Remoteproc framework Responsibilities > > +======================================== > > + > > +The framework begins by gathering information about the firmware file > > +to be downloaded through the request_firmware function. It supports > > +the ELF format and parses the firmware image to identify the physical > > +addresses that need to be populated from the corresponding ELF sections. > > +The framework also requires knowledge of the logical or I/O-mapped > > +addresses in the application processor. Once this information is > > +obtained from the driver, the framework transfers the data to the > > +specified addresses and starts the remote, along with > > +any devices physically or logically connected to it. > > + > > +Dependent devices, referred to as `subdevices` within the framework, > > +are also managed post-registration by their respective drivers. > > +Subdevices can register themselves using `rproc_(add/remove)_subdev`. > > +Non-remoteproc drivers can use subdevices as a way to logically connect > > +to remote and get lifecycle notifications of the remote. > > + > > +The framework oversees the lifecycle of the remote and > > +provides the `rproc_report_crash` function, which the driver invokes > > +upon receiving a crash notification from the remote. The > > +notification method can differ based on the design of the remote > > +processor and its communication with the application processor. For > > +instance, if the remote is a DSP equipped with a watchdog, > > +unresponsive behavior triggers the watchdog, generating an interrupt > > +that routes to the application processor, allowing it to call > > +`rproc_report_crash` in the driver's interrupt context. > > + > > +During crash handling, the framework performs the following actions: > > + > > +a. Sends a request to stop the remote and any connected or > > + dependent subdevices. > > +b. Generates a coredump, dumping all `resources` requested by the > > + remote alongside relevant debugging information. Resources are > > + explained below. > > +c. Reloads the firmware and restarts the remote. > > + > > +If the `RPROC_FEAT_ATTACH_ON_RECOVERY` flag is set, the detach and > > +attach callbacks of the driver are invoked without reloading the > > +firmware. This is useful when the remote requires no > > +assistance for recovery, or when the application processor can restart > > +independently. After recovery, the application processor can reattach > > +to the remote. > > + > > +The remote can request resources from the framework, which > > +allocates a ".resource_table" section. During the ELF parsing phase, > > +the framework identifies this section and calls the appropriate > > +handler to allocate the requested resources. > > + > > +Resource management within the framework can accommodate any type of > > +`fw_resource_type`. > > + > > +.. code-block:: c > > + > > + enum fw_resource_type { > > + RSC_CARVEOUT = 0, > > + RSC_DEVMEM = 1, > > + RSC_TRACE = 2, > > + RSC_VDEV = 3, > > + RSC_LAST = 4, > > + RSC_VENDOR_START = 128, > > + RSC_VENDOR_END = 512, > > + }; > > + > > + struct resource_table { > > + u32 ver; > > + u32 num; > > + u32 reserved[2]; > > + u32 offset[]; > > + } __packed; > > + > > + struct fw_rsc_hdr { > > + u32 type; > > + u8 data[]; > > + } __packed; > > + > > +For example, if the remote requests both `RSC_TRACE` and > > +`RSC_CARVEOUT` for memory allocation, the ELF firmware can be structured > > +as follows: > > + > > +.. code-block:: c > > + > > + #define MAX_SHARED_RESOURCE 2 > > + #define LOG_BUF_SIZE 1000 > > + #define CARVEOUT_DUMP_PA 0x12345678 > > + #define CARVEOUT_DUMP_SIZE 2000 > > + > > + struct shared_resource_table { > > + u32 ver; > > + u32 num; > > + u32 reserved[2]; > > + u32 offset[MAX_SHARED_RESOURCE]; > > + struct fw_rsc_trace log_trace; > > + struct fw_rsc_carveout dump_carveout; > > + }; > > + > > + volatile struct shared_resource_table table = { > > + .ver = 1, > > + .num = 2, > > + .reserved = {0, 0}, > > + .offset = { > > + offsetof(struct resource_table, log_trace), > > + offsetof(struct resource_table, dump_carveout), > > + }, > > + .log_trace = { > > + RSC_TRACE, > > + (u32)log_buf, LOG_BUF_SIZE, 0, "log_trace", > > + }, > > + .dump_carveout = { > > + RSC_CARVEOUT, > > + (u32)FW_RSC_ADDR_ANY, CARVEOUT_PA, 0, "carveout_dump", > > + }, > > + }; > > + > > +The framework creates a sysfs file when it encounters the `RSC_TRACE` > > +type to expose log information to userspace. Other resource types are > > +handled accordingly. In the example above, `CARVEOUT_DUMP_SIZE` bytes > > +of DMA memory will be allocated starting from `CARVEOUT_DUMP_PA`. > > + > > + > > +3. Remoteproc driver responsibilities > > +===================================== > > + > > +The driver must provide the following information to the core: > > + > > +a. Translate device addresses (physical addresses) found in the ELF > > + firmware to virtual addresses in Linux using the `da_to_va` > > + callback. This allows the framework to copy ELF firmware from the > > + filesystem to the addresses expected by the remote since > > + the framework cannot directly access those physical addresses. > > +b. Prepare/unprepare the remote prior to firmware loading, > > + which may involve allocating carveout and reserved memory regions. > > +c. Implement methods for starting and stopping the remote, > > + whether by setting registers or sending explicit interrupts, > > + depending on the hardware design. > > +d. Provide attach and detach callbacks to start the remote > > + without loading the firmware. This is beneficial when the remote > > + processor is already loaded and running. > > +e. Implement a load callback for firmware loading, typically using > > + the ELF loader provided by the framework; currently, only ELF > > + format is supported. > > +f. Invoke the framework's crash handler API upon detecting a remote > > + crash. > > + > > +Drivers must fill the `rproc_ops` structure and call `rproc_alloc` > > +to register themselves with the framework. > > + > > +.. code-block:: c > > + > > + struct rproc_ops { > > + int (*prepare)(struct rproc *rproc); > > + int (*unprepare)(struct rproc *rproc); > > + int (*start)(struct rproc *rproc); > > + int (*stop)(struct rproc *rproc); > > + int (*attach)(struct rproc *rproc); > > + int (*detach)(struct rproc *rproc); > > + void * (*da_to_va)(struct rproc *rproc, u64 da, size_t len, > > + bool *is_iomem); > > + int (*parse_fw)(struct rproc *rproc, const struct firmware *fw); > > + int (*handle_rsc)(struct rproc *rproc, u32 rsc_type, > > + void *rsc, int offset, int avail); > > + int (*load)(struct rproc *rproc, const struct firmware *fw); > > + //snip > > + }; > > + > > + > > +4. Virtio and Remoteproc > > ======================== > > > > -These callbacks should be provided by platform-specific remoteproc > > -drivers:: > > - > > - /** > > - * struct rproc_ops - platform-specific device handlers > > - * @start: power on the device and boot it > > - * @stop: power off the device > > - * @kick: kick a virtqueue (virtqueue id given as a parameter) > > - */ > > - struct rproc_ops { > > - int (*start)(struct rproc *rproc); > > - int (*stop)(struct rproc *rproc); > > - void (*kick)(struct rproc *rproc, int vqid); > > - }; > > - > > -Every remoteproc implementation should at least provide the ->start and ->stop > > -handlers. If rpmsg/virtio functionality is also desired, then the ->kick handler > > -should be provided as well. > > - > > -The ->start() handler takes an rproc handle and should then power on the > > -device and boot it (use rproc->priv to access platform-specific private data). > > -The boot address, in case needed, can be found in rproc->bootaddr (remoteproc > > -core puts there the ELF entry point). > > -On success, 0 should be returned, and on failure, an appropriate error code. > > - > > -The ->stop() handler takes an rproc handle and powers the device down. > > -On success, 0 is returned, and on failure, an appropriate error code. > > - > > -The ->kick() handler takes an rproc handle, and an index of a virtqueue > > -where new message was placed in. Implementations should interrupt the remote > > -processor and let it know it has pending messages. Notifying remote processors > > -the exact virtqueue index to look in is optional: it is easy (and not > > -too expensive) to go through the existing virtqueues and look for new buffers > > -in the used rings. > > - > > -Binary Firmware Structure > > -========================= > > - > > -At this point remoteproc supports ELF32 and ELF64 firmware binaries. However, > > -it is quite expected that other platforms/devices which we'd want to > > -support with this framework will be based on different binary formats. > > - > > -When those use cases show up, we will have to decouple the binary format > > -from the framework core, so we can support several binary formats without > > -duplicating common code. > > - > > -When the firmware is parsed, its various segments are loaded to memory > > -according to the specified device address (might be a physical address > > -if the remote processor is accessing memory directly). > > - > > -In addition to the standard ELF segments, most remote processors would > > -also include a special section which we call "the resource table". > > - > > -The resource table contains system resources that the remote processor > > -requires before it should be powered on, such as allocation of physically > > -contiguous memory, or iommu mapping of certain on-chip peripherals. > > -Remotecore will only power up the device after all the resource table's > > -requirement are met. > > - > > -In addition to system resources, the resource table may also contain > > -resource entries that publish the existence of supported features > > -or configurations by the remote processor, such as trace buffers and > > -supported virtio devices (and their configurations). > > - > > -The resource table begins with this header:: > > - > > - /** > > - * struct resource_table - firmware resource table header > > - * @ver: version number > > - * @num: number of resource entries > > - * @reserved: reserved (must be zero) > > - * @offset: array of offsets pointing at the various resource entries > > - * > > - * The header of the resource table, as expressed by this structure, > > - * contains a version number (should we need to change this format in the > > - * future), the number of available resource entries, and their offsets > > - * in the table. > > - */ > > - struct resource_table { > > - u32 ver; > > - u32 num; > > - u32 reserved[2]; > > - u32 offset[0]; > > - } __packed; > > - > > -Immediately following this header are the resource entries themselves, > > -each of which begins with the following resource entry header:: > > - > > - /** > > - * struct fw_rsc_hdr - firmware resource entry header > > - * @type: resource type > > - * @data: resource data > > - * > > - * Every resource entry begins with a 'struct fw_rsc_hdr' header providing > > - * its @type. The content of the entry itself will immediately follow > > - * this header, and it should be parsed according to the resource type. > > - */ > > - struct fw_rsc_hdr { > > - u32 type; > > - u8 data[0]; > > - } __packed; > > - > > -Some resources entries are mere announcements, where the host is informed > > -of specific remoteproc configuration. Other entries require the host to > > -do something (e.g. allocate a system resource). Sometimes a negotiation > > -is expected, where the firmware requests a resource, and once allocated, > > -the host should provide back its details (e.g. address of an allocated > > -memory region). > > - > > -Here are the various resource types that are currently supported:: > > - > > - /** > > - * enum fw_resource_type - types of resource entries > > - * > > - * @RSC_CARVEOUT: request for allocation of a physically contiguous > > - * memory region. > > - * @RSC_DEVMEM: request to iommu_map a memory-based peripheral. > > - * @RSC_TRACE: announces the availability of a trace buffer into which > > - * the remote processor will be writing logs. > > - * @RSC_VDEV: declare support for a virtio device, and serve as its > > - * virtio header. > > - * @RSC_LAST: just keep this one at the end > > - * @RSC_VENDOR_START: start of the vendor specific resource types range > > - * @RSC_VENDOR_END: end of the vendor specific resource types range > > - * > > - * Please note that these values are used as indices to the rproc_handle_rsc > > - * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to > > - * check the validity of an index before the lookup table is accessed, so > > - * please update it as needed. > > - */ > > - enum fw_resource_type { > > - RSC_CARVEOUT = 0, > > - RSC_DEVMEM = 1, > > - RSC_TRACE = 2, > > - RSC_VDEV = 3, > > - RSC_LAST = 4, > > - RSC_VENDOR_START = 128, > > - RSC_VENDOR_END = 512, > > - }; > > - > > -For more details regarding a specific resource type, please see its > > -dedicated structure in include/linux/remoteproc.h. > > - > > -We also expect that platform-specific resource entries will show up > > -at some point. When that happens, we could easily add a new RSC_PLATFORM > > -type, and hand those resources to the platform-specific rproc driver to handle. > > - > > -Virtio and remoteproc > > -===================== > > - > > -The firmware should provide remoteproc information about virtio devices > > -that it supports, and their configurations: a RSC_VDEV resource entry > > -should specify the virtio device id (as in virtio_ids.h), virtio features, > > -virtio config space, vrings information, etc. > > - > > -When a new remote processor is registered, the remoteproc framework > > -will look for its resource table and will register the virtio devices > > -it supports. A firmware may support any number of virtio devices, and > > -of any type (a single remote processor can also easily support several > > -rpmsg virtio devices this way, if desired). > > - > > -Of course, RSC_VDEV resource entries are only good enough for static > > -allocation of virtio devices. Dynamic allocations will also be made possible > > -using the rpmsg bus (similar to how we already do dynamic allocations of > > -rpmsg channels; read more about it in rpmsg.txt). > > +The firmware must provide remoteproc with information regarding the > > +virtio devices it supports and their configurations: an `RSC_VDEV` > > +resource entry should detail the virtio device ID (as defined in > > +`virtio_ids.h`), virtio features, virtio config space, vrings > > +information, etc. > > + > > +Upon registration of a new remote, the remoteproc framework > > +searches for its resource table and registers the supported virtio > > +devices. A firmware may support multiple virtio devices, of various > > +types (a single remote can support multiple rpmsg virtio > > +devices if required). > > + > > +Moreover, `RSC_VDEV` resource entries suffice for static allocation > > +of virtio devices. Dynamic allocations will also be supported using > > +the rpmsg bus, akin to the handling of dynamic allocations for rpmsg > > +channels. For more information, refer to rpmsg documentation. > > -- > > 2.39.3 (Apple Git-146) > >
On Thu, Oct 24, 2024 at 11:17:40AM -0700, anish kumar wrote: > On Thu, Oct 24, 2024 at 8:53 AM Mathieu Poirier > <mathieu.poirier@linaro.org> wrote: > > > > On Tue, Oct 22, 2024 at 10:33:53PM -0700, anish kumar wrote: > > > Added following changes: > > > 1. Components provided by remoteproc framework. > > > 2. Remoteproc driver responsibilities. > > > 3. Remoteproc framework responsibilities. > > > 4. Better explanation of how to ask for resources > > > from the framework by the remote processor. > > > > > > Signed-off-by: anish kumar <yesanishhere@gmail.com> > > > --- > > > .../driver-api/remoteproc/remoteproc.rst | 596 +++++++----------- > > > 1 file changed, 243 insertions(+), 353 deletions(-) > > > > > > diff --git a/Documentation/driver-api/remoteproc/remoteproc.rst b/Documentation/driver-api/remoteproc/remoteproc.rst > > > index 9cccd3dd6a4b..7ca545eea153 100644 > > > --- a/Documentation/driver-api/remoteproc/remoteproc.rst > > > +++ b/Documentation/driver-api/remoteproc/remoteproc.rst > > > @@ -1,359 +1,249 @@ > > > +.. SPDX-License-Identifier: GPL-2.0 > > > + > > > ========================== > > > Remote Processor Framework > > > ========================== > > > > > > -Introduction > > > -============ > > > - > > > -Modern SoCs typically have heterogeneous remote processor devices in asymmetric > > > -multiprocessing (AMP) configurations, which may be running different instances > > > -of operating system, whether it's Linux or any other flavor of real-time OS. > > > - > > > -OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. > > > -In a typical configuration, the dual cortex-A9 is running Linux in a SMP > > > -configuration, and each of the other three cores (two M3 cores and a DSP) > > > -is running its own instance of RTOS in an AMP configuration. > > > - > > > -The remoteproc framework allows different platforms/architectures to > > > -control (power on, load firmware, power off) those remote processors while > > > -abstracting the hardware differences, so the entire driver doesn't need to be > > > -duplicated. In addition, this framework also adds rpmsg virtio devices > > > -for remote processors that supports this kind of communication. This way, > > > -platform-specific remoteproc drivers only need to provide a few low-level > > > -handlers, and then all rpmsg drivers will then just work > > > -(for more information about the virtio-based rpmsg bus and its drivers, > > > -please read Documentation/staging/rpmsg.rst). > > > -Registration of other types of virtio devices is now also possible. Firmwares > > > -just need to publish what kind of virtio devices do they support, and then > > > -remoteproc will add those devices. This makes it possible to reuse the > > > -existing virtio drivers with remote processor backends at a minimal development > > > -cost. > > > - > > > -User API > > > -======== > > > - > > > -:: > > > - > > > - int rproc_boot(struct rproc *rproc) > > > - > > > -Boot a remote processor (i.e. load its firmware, power it on, ...). > > > - > > > -If the remote processor is already powered on, this function immediately > > > -returns (successfully). > > > - > > > -Returns 0 on success, and an appropriate error value otherwise. > > > -Note: to use this function you should already have a valid rproc > > > -handle. There are several ways to achieve that cleanly (devres, pdata, > > > -the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we > > > -might also consider using dev_archdata for this). > > > - > > > -:: > > > - > > > - void rproc_shutdown(struct rproc *rproc) > > > - > > > -Power off a remote processor (previously booted with rproc_boot()). > > > -In case @rproc is still being used by an additional user(s), then > > > -this function will just decrement the power refcount and exit, > > > -without really powering off the device. > > > - > > > -Every call to rproc_boot() must (eventually) be accompanied by a call > > > -to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. > > > - > > > -.. note:: > > > - > > > - we're not decrementing the rproc's refcount, only the power refcount. > > > - which means that the @rproc handle stays valid even after > > > - rproc_shutdown() returns, and users can still use it with a subsequent > > > - rproc_boot(), if needed. > > > - > > > -:: > > > - > > > - struct rproc *rproc_get_by_phandle(phandle phandle) > > > - > > > -Find an rproc handle using a device tree phandle. Returns the rproc > > > -handle on success, and NULL on failure. This function increments > > > -the remote processor's refcount, so always use rproc_put() to > > > -decrement it back once rproc isn't needed anymore. > > > - > > > -Typical usage > > > -============= > > > - > > > -:: > > > - > > > - #include <linux/remoteproc.h> > > > - > > > - /* in case we were given a valid 'rproc' handle */ > > > - int dummy_rproc_example(struct rproc *my_rproc) > > > - { > > > - int ret; > > > - > > > - /* let's power on and boot our remote processor */ > > > - ret = rproc_boot(my_rproc); > > > - if (ret) { > > > - /* > > > - * something went wrong. handle it and leave. > > > - */ > > > - } > > > - > > > - /* > > > - * our remote processor is now powered on... give it some work > > > - */ > > > - > > > - /* let's shut it down now */ > > > - rproc_shutdown(my_rproc); > > > - } > > > - > > > -API for implementors > > > -==================== > > > - > > > -:: > > > - > > > - struct rproc *rproc_alloc(struct device *dev, const char *name, > > > - const struct rproc_ops *ops, > > > - const char *firmware, int len) > > > - > > > -Allocate a new remote processor handle, but don't register > > > -it yet. Required parameters are the underlying device, the > > > -name of this remote processor, platform-specific ops handlers, > > > -the name of the firmware to boot this rproc with, and the > > > -length of private data needed by the allocating rproc driver (in bytes). > > > - > > > -This function should be used by rproc implementations during > > > -initialization of the remote processor. > > > - > > > -After creating an rproc handle using this function, and when ready, > > > -implementations should then call rproc_add() to complete > > > -the registration of the remote processor. > > > - > > > -On success, the new rproc is returned, and on failure, NULL. > > > - > > > -.. note:: > > > - > > > - **never** directly deallocate @rproc, even if it was not registered > > > - yet. Instead, when you need to unroll rproc_alloc(), use rproc_free(). > > > - > > > -:: > > > - > > > - void rproc_free(struct rproc *rproc) > > > - > > > -Free an rproc handle that was allocated by rproc_alloc. > > > - > > > -This function essentially unrolls rproc_alloc(), by decrementing the > > > -rproc's refcount. It doesn't directly free rproc; that would happen > > > -only if there are no other references to rproc and its refcount now > > > -dropped to zero. > > > - > > > -:: > > > - > > > - int rproc_add(struct rproc *rproc) > > > - > > > -Register @rproc with the remoteproc framework, after it has been > > > -allocated with rproc_alloc(). > > > - > > > -This is called by the platform-specific rproc implementation, whenever > > > -a new remote processor device is probed. > > > - > > > -Returns 0 on success and an appropriate error code otherwise. > > > -Note: this function initiates an asynchronous firmware loading > > > -context, which will look for virtio devices supported by the rproc's > > > -firmware. > > > - > > > -If found, those virtio devices will be created and added, so as a result > > > -of registering this remote processor, additional virtio drivers might get > > > -probed. > > > - > > > -:: > > > - > > > - int rproc_del(struct rproc *rproc) > > > - > > > -Unroll rproc_add(). > > > - > > > -This function should be called when the platform specific rproc > > > -implementation decides to remove the rproc device. it should > > > -_only_ be called if a previous invocation of rproc_add() > > > -has completed successfully. > > > - > > > -After rproc_del() returns, @rproc is still valid, and its > > > -last refcount should be decremented by calling rproc_free(). > > > - > > > -Returns 0 on success and -EINVAL if @rproc isn't valid. > > > - > > > -:: > > > - > > > - void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type) > > > - > > > -Report a crash in a remoteproc > > > - > > > -This function must be called every time a crash is detected by the > > > -platform specific rproc implementation. This should not be called from a > > > -non-remoteproc driver. This function can be called from atomic/interrupt > > > -context. > > > - > > > -Implementation callbacks > > > +.. Contents: > > > + > > > + 1. Introduction > > > + 2. Remoteproc framework responsibilities > > > + 3. Remoteproc driver responsibilities > > > + 4. Virtio and rpmsg > > > + > > > +1. Introduction > > > +=============== > > > + > > > +Modern System on Chips (SoCs) typically integrate heterogeneous remote > > > +processor devices in asymmetric multiprocessing (AMP) configurations. > > > +These processors may run different operating systems, such as Linux and > > > +various real-time operating systems (RTOS). > > > > You are moving things around _and_ making modifications to the text in the same > > patch, something I specifically asked not to do. Moreover, the above conveys > > Sorry for doing that, I thought I moved the file first as you instructed but > I didn't understand that I should also not reword the sentences that convey > the same meaning. Will keep the text of below content in the same file: > 1. Introduction > 2. Binary Firmware Structure: will just add example as shown in this patch. > 3. Virtio and remoteproc: didn't change anything in the patch for this. We don't seem to understand each other - the "Virtio and remoteproc" section is the first thing I red and what made me aware of the rewording. > > For below: > 1. User API: > 2. Typical usage: > 3. API for implementors : > > Should I just move all of the above text as it is to > remoteproc-kernel-api.rst or > you prefer here as I feel it belongs there? I have looked at other subsystems documented under Documentation/driver-api and I can't find instances of the type [subsystem]-kernel-api.rst. As such I don't see what is so special about remoteproc that it would mandate adding a new file. Why can't it just be in a single file like most subsystems do? > > > exactly the same information as found in [1] but using different words. I > > am in favour of enhancing documentation but not creating unneeded churn. > > > > I found several instances of the same rewording pattern in the sections below. > > As such I will not look at the other patches nor move forward with this set. > > > > Thanks, > > Mathieu > > Thanks Mathieu, for reviewing and providing feedback as always. > > > > > [1]. https://elixir.bootlin.com/linux/v6.12-rc4/source/Documentation/staging/remoteproc.rst > > > > > + > > > +For example, the OMAP4 platform features dual Cortex-A9 cores, dual > > > +Cortex-M3 cores, and a C64x+ DSP. In a standard setup, the Cortex-A9 > > > +cores execute Linux in a symmetric multiprocessing (SMP) configuration, > > > +while the M3 cores and DSP run independent instances of an RTOS. > > > + > > > +The remoteproc framework allows various platforms and architectures to > > > +manage remote processors, including operations such as powering on, > > > +loading firmware, and powering off. This framework abstracts hardware > > > +differences, promoting code reuse and minimizing duplication. It also > > > +supports rpmsg virtio devices for remote processors that utilize this > > > +communication method. Consequently, platform-specific remoteproc drivers > > > +need only implement a few low-level handlers, enabling seamless operation > > > +of all rpmsg drivers. (For more details about the virtio-based rpmsg > > > +bus and its drivers, refer to rpmsg documentation.) > > > + > > > +Additionally, the framework allows for the registration of various > > > +virtio devices. Firmware can publish the types of virtio devices it > > > +supports, facilitating their addition to the remoteproc framework. This > > > +flexibility enables the reuse of existing virtio drivers with remote > > > +processor backends at minimal development cost. > > > + > > > +The primary purpose of the remoteproc framework is to download firmware > > > +for remote processors and manage their lifecycle. The framework consists > > > +of several key components: > > > + > > > +- **Character Driver**: Provides userspace access to control the remote > > > + processor. > > > +- **ELF Utility**: Offers functions for handling ELF files and managing > > > + resources requested by the remote processor. > > > +- **Remoteproc Core**: Manages firmware downloads and recovery actions > > > + in case of a remote processor crash. > > > +- **Coredump**: Provides facilities for coredumping and tracing from > > > + the remote processor in the event of a crash. > > > +- **Userspace Interaction**: Uses sysfs and debugfs to manage the > > > + lifecycle and status of the remote processor. > > > +- **Virtio Support**: Facilitates interaction with the virtio and > > > + rpmsg bus. > > > + > > > +From here on, references to "framework" denote the remoteproc > > > +framework, and "driver" refers to the remoteproc driver that utilizes > > > +the framework for managing remote processors. > > > + > > > +2. Remoteproc framework Responsibilities > > > +======================================== > > > + > > > +The framework begins by gathering information about the firmware file > > > +to be downloaded through the request_firmware function. It supports > > > +the ELF format and parses the firmware image to identify the physical > > > +addresses that need to be populated from the corresponding ELF sections. > > > +The framework also requires knowledge of the logical or I/O-mapped > > > +addresses in the application processor. Once this information is > > > +obtained from the driver, the framework transfers the data to the > > > +specified addresses and starts the remote, along with > > > +any devices physically or logically connected to it. > > > + > > > +Dependent devices, referred to as `subdevices` within the framework, > > > +are also managed post-registration by their respective drivers. > > > +Subdevices can register themselves using `rproc_(add/remove)_subdev`. > > > +Non-remoteproc drivers can use subdevices as a way to logically connect > > > +to remote and get lifecycle notifications of the remote. > > > + > > > +The framework oversees the lifecycle of the remote and > > > +provides the `rproc_report_crash` function, which the driver invokes > > > +upon receiving a crash notification from the remote. The > > > +notification method can differ based on the design of the remote > > > +processor and its communication with the application processor. For > > > +instance, if the remote is a DSP equipped with a watchdog, > > > +unresponsive behavior triggers the watchdog, generating an interrupt > > > +that routes to the application processor, allowing it to call > > > +`rproc_report_crash` in the driver's interrupt context. > > > + > > > +During crash handling, the framework performs the following actions: > > > + > > > +a. Sends a request to stop the remote and any connected or > > > + dependent subdevices. > > > +b. Generates a coredump, dumping all `resources` requested by the > > > + remote alongside relevant debugging information. Resources are > > > + explained below. > > > +c. Reloads the firmware and restarts the remote. > > > + > > > +If the `RPROC_FEAT_ATTACH_ON_RECOVERY` flag is set, the detach and > > > +attach callbacks of the driver are invoked without reloading the > > > +firmware. This is useful when the remote requires no > > > +assistance for recovery, or when the application processor can restart > > > +independently. After recovery, the application processor can reattach > > > +to the remote. > > > + > > > +The remote can request resources from the framework, which > > > +allocates a ".resource_table" section. During the ELF parsing phase, > > > +the framework identifies this section and calls the appropriate > > > +handler to allocate the requested resources. > > > + > > > +Resource management within the framework can accommodate any type of > > > +`fw_resource_type`. > > > + > > > +.. code-block:: c > > > + > > > + enum fw_resource_type { > > > + RSC_CARVEOUT = 0, > > > + RSC_DEVMEM = 1, > > > + RSC_TRACE = 2, > > > + RSC_VDEV = 3, > > > + RSC_LAST = 4, > > > + RSC_VENDOR_START = 128, > > > + RSC_VENDOR_END = 512, > > > + }; > > > + > > > + struct resource_table { > > > + u32 ver; > > > + u32 num; > > > + u32 reserved[2]; > > > + u32 offset[]; > > > + } __packed; > > > + > > > + struct fw_rsc_hdr { > > > + u32 type; > > > + u8 data[]; > > > + } __packed; > > > + > > > +For example, if the remote requests both `RSC_TRACE` and > > > +`RSC_CARVEOUT` for memory allocation, the ELF firmware can be structured > > > +as follows: > > > + > > > +.. code-block:: c > > > + > > > + #define MAX_SHARED_RESOURCE 2 > > > + #define LOG_BUF_SIZE 1000 > > > + #define CARVEOUT_DUMP_PA 0x12345678 > > > + #define CARVEOUT_DUMP_SIZE 2000 > > > + > > > + struct shared_resource_table { > > > + u32 ver; > > > + u32 num; > > > + u32 reserved[2]; > > > + u32 offset[MAX_SHARED_RESOURCE]; > > > + struct fw_rsc_trace log_trace; > > > + struct fw_rsc_carveout dump_carveout; > > > + }; > > > + > > > + volatile struct shared_resource_table table = { > > > + .ver = 1, > > > + .num = 2, > > > + .reserved = {0, 0}, > > > + .offset = { > > > + offsetof(struct resource_table, log_trace), > > > + offsetof(struct resource_table, dump_carveout), > > > + }, > > > + .log_trace = { > > > + RSC_TRACE, > > > + (u32)log_buf, LOG_BUF_SIZE, 0, "log_trace", > > > + }, > > > + .dump_carveout = { > > > + RSC_CARVEOUT, > > > + (u32)FW_RSC_ADDR_ANY, CARVEOUT_PA, 0, "carveout_dump", > > > + }, > > > + }; > > > + > > > +The framework creates a sysfs file when it encounters the `RSC_TRACE` > > > +type to expose log information to userspace. Other resource types are > > > +handled accordingly. In the example above, `CARVEOUT_DUMP_SIZE` bytes > > > +of DMA memory will be allocated starting from `CARVEOUT_DUMP_PA`. > > > + > > > + > > > +3. Remoteproc driver responsibilities > > > +===================================== > > > + > > > +The driver must provide the following information to the core: > > > + > > > +a. Translate device addresses (physical addresses) found in the ELF > > > + firmware to virtual addresses in Linux using the `da_to_va` > > > + callback. This allows the framework to copy ELF firmware from the > > > + filesystem to the addresses expected by the remote since > > > + the framework cannot directly access those physical addresses. > > > +b. Prepare/unprepare the remote prior to firmware loading, > > > + which may involve allocating carveout and reserved memory regions. > > > +c. Implement methods for starting and stopping the remote, > > > + whether by setting registers or sending explicit interrupts, > > > + depending on the hardware design. > > > +d. Provide attach and detach callbacks to start the remote > > > + without loading the firmware. This is beneficial when the remote > > > + processor is already loaded and running. > > > +e. Implement a load callback for firmware loading, typically using > > > + the ELF loader provided by the framework; currently, only ELF > > > + format is supported. > > > +f. Invoke the framework's crash handler API upon detecting a remote > > > + crash. > > > + > > > +Drivers must fill the `rproc_ops` structure and call `rproc_alloc` > > > +to register themselves with the framework. > > > + > > > +.. code-block:: c > > > + > > > + struct rproc_ops { > > > + int (*prepare)(struct rproc *rproc); > > > + int (*unprepare)(struct rproc *rproc); > > > + int (*start)(struct rproc *rproc); > > > + int (*stop)(struct rproc *rproc); > > > + int (*attach)(struct rproc *rproc); > > > + int (*detach)(struct rproc *rproc); > > > + void * (*da_to_va)(struct rproc *rproc, u64 da, size_t len, > > > + bool *is_iomem); > > > + int (*parse_fw)(struct rproc *rproc, const struct firmware *fw); > > > + int (*handle_rsc)(struct rproc *rproc, u32 rsc_type, > > > + void *rsc, int offset, int avail); > > > + int (*load)(struct rproc *rproc, const struct firmware *fw); > > > + //snip > > > + }; > > > + > > > + > > > +4. Virtio and Remoteproc > > > ======================== > > > > > > -These callbacks should be provided by platform-specific remoteproc > > > -drivers:: > > > - > > > - /** > > > - * struct rproc_ops - platform-specific device handlers > > > - * @start: power on the device and boot it > > > - * @stop: power off the device > > > - * @kick: kick a virtqueue (virtqueue id given as a parameter) > > > - */ > > > - struct rproc_ops { > > > - int (*start)(struct rproc *rproc); > > > - int (*stop)(struct rproc *rproc); > > > - void (*kick)(struct rproc *rproc, int vqid); > > > - }; > > > - > > > -Every remoteproc implementation should at least provide the ->start and ->stop > > > -handlers. If rpmsg/virtio functionality is also desired, then the ->kick handler > > > -should be provided as well. > > > - > > > -The ->start() handler takes an rproc handle and should then power on the > > > -device and boot it (use rproc->priv to access platform-specific private data). > > > -The boot address, in case needed, can be found in rproc->bootaddr (remoteproc > > > -core puts there the ELF entry point). > > > -On success, 0 should be returned, and on failure, an appropriate error code. > > > - > > > -The ->stop() handler takes an rproc handle and powers the device down. > > > -On success, 0 is returned, and on failure, an appropriate error code. > > > - > > > -The ->kick() handler takes an rproc handle, and an index of a virtqueue > > > -where new message was placed in. Implementations should interrupt the remote > > > -processor and let it know it has pending messages. Notifying remote processors > > > -the exact virtqueue index to look in is optional: it is easy (and not > > > -too expensive) to go through the existing virtqueues and look for new buffers > > > -in the used rings. > > > - > > > -Binary Firmware Structure > > > -========================= > > > - > > > -At this point remoteproc supports ELF32 and ELF64 firmware binaries. However, > > > -it is quite expected that other platforms/devices which we'd want to > > > -support with this framework will be based on different binary formats. > > > - > > > -When those use cases show up, we will have to decouple the binary format > > > -from the framework core, so we can support several binary formats without > > > -duplicating common code. > > > - > > > -When the firmware is parsed, its various segments are loaded to memory > > > -according to the specified device address (might be a physical address > > > -if the remote processor is accessing memory directly). > > > - > > > -In addition to the standard ELF segments, most remote processors would > > > -also include a special section which we call "the resource table". > > > - > > > -The resource table contains system resources that the remote processor > > > -requires before it should be powered on, such as allocation of physically > > > -contiguous memory, or iommu mapping of certain on-chip peripherals. > > > -Remotecore will only power up the device after all the resource table's > > > -requirement are met. > > > - > > > -In addition to system resources, the resource table may also contain > > > -resource entries that publish the existence of supported features > > > -or configurations by the remote processor, such as trace buffers and > > > -supported virtio devices (and their configurations). > > > - > > > -The resource table begins with this header:: > > > - > > > - /** > > > - * struct resource_table - firmware resource table header > > > - * @ver: version number > > > - * @num: number of resource entries > > > - * @reserved: reserved (must be zero) > > > - * @offset: array of offsets pointing at the various resource entries > > > - * > > > - * The header of the resource table, as expressed by this structure, > > > - * contains a version number (should we need to change this format in the > > > - * future), the number of available resource entries, and their offsets > > > - * in the table. > > > - */ > > > - struct resource_table { > > > - u32 ver; > > > - u32 num; > > > - u32 reserved[2]; > > > - u32 offset[0]; > > > - } __packed; > > > - > > > -Immediately following this header are the resource entries themselves, > > > -each of which begins with the following resource entry header:: > > > - > > > - /** > > > - * struct fw_rsc_hdr - firmware resource entry header > > > - * @type: resource type > > > - * @data: resource data > > > - * > > > - * Every resource entry begins with a 'struct fw_rsc_hdr' header providing > > > - * its @type. The content of the entry itself will immediately follow > > > - * this header, and it should be parsed according to the resource type. > > > - */ > > > - struct fw_rsc_hdr { > > > - u32 type; > > > - u8 data[0]; > > > - } __packed; > > > - > > > -Some resources entries are mere announcements, where the host is informed > > > -of specific remoteproc configuration. Other entries require the host to > > > -do something (e.g. allocate a system resource). Sometimes a negotiation > > > -is expected, where the firmware requests a resource, and once allocated, > > > -the host should provide back its details (e.g. address of an allocated > > > -memory region). > > > - > > > -Here are the various resource types that are currently supported:: > > > - > > > - /** > > > - * enum fw_resource_type - types of resource entries > > > - * > > > - * @RSC_CARVEOUT: request for allocation of a physically contiguous > > > - * memory region. > > > - * @RSC_DEVMEM: request to iommu_map a memory-based peripheral. > > > - * @RSC_TRACE: announces the availability of a trace buffer into which > > > - * the remote processor will be writing logs. > > > - * @RSC_VDEV: declare support for a virtio device, and serve as its > > > - * virtio header. > > > - * @RSC_LAST: just keep this one at the end > > > - * @RSC_VENDOR_START: start of the vendor specific resource types range > > > - * @RSC_VENDOR_END: end of the vendor specific resource types range > > > - * > > > - * Please note that these values are used as indices to the rproc_handle_rsc > > > - * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to > > > - * check the validity of an index before the lookup table is accessed, so > > > - * please update it as needed. > > > - */ > > > - enum fw_resource_type { > > > - RSC_CARVEOUT = 0, > > > - RSC_DEVMEM = 1, > > > - RSC_TRACE = 2, > > > - RSC_VDEV = 3, > > > - RSC_LAST = 4, > > > - RSC_VENDOR_START = 128, > > > - RSC_VENDOR_END = 512, > > > - }; > > > - > > > -For more details regarding a specific resource type, please see its > > > -dedicated structure in include/linux/remoteproc.h. > > > - > > > -We also expect that platform-specific resource entries will show up > > > -at some point. When that happens, we could easily add a new RSC_PLATFORM > > > -type, and hand those resources to the platform-specific rproc driver to handle. > > > - > > > -Virtio and remoteproc > > > -===================== > > > - > > > -The firmware should provide remoteproc information about virtio devices > > > -that it supports, and their configurations: a RSC_VDEV resource entry > > > -should specify the virtio device id (as in virtio_ids.h), virtio features, > > > -virtio config space, vrings information, etc. > > > - > > > -When a new remote processor is registered, the remoteproc framework > > > -will look for its resource table and will register the virtio devices > > > -it supports. A firmware may support any number of virtio devices, and > > > -of any type (a single remote processor can also easily support several > > > -rpmsg virtio devices this way, if desired). > > > - > > > -Of course, RSC_VDEV resource entries are only good enough for static > > > -allocation of virtio devices. Dynamic allocations will also be made possible > > > -using the rpmsg bus (similar to how we already do dynamic allocations of > > > -rpmsg channels; read more about it in rpmsg.txt). > > > +The firmware must provide remoteproc with information regarding the > > > +virtio devices it supports and their configurations: an `RSC_VDEV` > > > +resource entry should detail the virtio device ID (as defined in > > > +`virtio_ids.h`), virtio features, virtio config space, vrings > > > +information, etc. > > > + > > > +Upon registration of a new remote, the remoteproc framework > > > +searches for its resource table and registers the supported virtio > > > +devices. A firmware may support multiple virtio devices, of various > > > +types (a single remote can support multiple rpmsg virtio > > > +devices if required). > > > + > > > +Moreover, `RSC_VDEV` resource entries suffice for static allocation > > > +of virtio devices. Dynamic allocations will also be supported using > > > +the rpmsg bus, akin to the handling of dynamic allocations for rpmsg > > > +channels. For more information, refer to rpmsg documentation. > > > -- > > > 2.39.3 (Apple Git-146) > > >
On Fri, Oct 25, 2024 at 7:57 AM Mathieu Poirier <mathieu.poirier@linaro.org> wrote: > > On Thu, Oct 24, 2024 at 11:17:40AM -0700, anish kumar wrote: > > On Thu, Oct 24, 2024 at 8:53 AM Mathieu Poirier > > <mathieu.poirier@linaro.org> wrote: > > > > > > On Tue, Oct 22, 2024 at 10:33:53PM -0700, anish kumar wrote: > > > > Added following changes: > > > > 1. Components provided by remoteproc framework. > > > > 2. Remoteproc driver responsibilities. > > > > 3. Remoteproc framework responsibilities. > > > > 4. Better explanation of how to ask for resources > > > > from the framework by the remote processor. > > > > > > > > Signed-off-by: anish kumar <yesanishhere@gmail.com> > > > > --- > > > > .../driver-api/remoteproc/remoteproc.rst | 596 +++++++----------- > > > > 1 file changed, 243 insertions(+), 353 deletions(-) > > > > > > > > diff --git a/Documentation/driver-api/remoteproc/remoteproc.rst b/Documentation/driver-api/remoteproc/remoteproc.rst > > > > index 9cccd3dd6a4b..7ca545eea153 100644 > > > > --- a/Documentation/driver-api/remoteproc/remoteproc.rst > > > > +++ b/Documentation/driver-api/remoteproc/remoteproc.rst > > > > @@ -1,359 +1,249 @@ > > > > +.. SPDX-License-Identifier: GPL-2.0 > > > > + > > > > ========================== > > > > Remote Processor Framework > > > > ========================== > > > > > > > > -Introduction > > > > -============ > > > > - > > > > -Modern SoCs typically have heterogeneous remote processor devices in asymmetric > > > > -multiprocessing (AMP) configurations, which may be running different instances > > > > -of operating system, whether it's Linux or any other flavor of real-time OS. > > > > - > > > > -OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. > > > > -In a typical configuration, the dual cortex-A9 is running Linux in a SMP > > > > -configuration, and each of the other three cores (two M3 cores and a DSP) > > > > -is running its own instance of RTOS in an AMP configuration. > > > > - > > > > -The remoteproc framework allows different platforms/architectures to > > > > -control (power on, load firmware, power off) those remote processors while > > > > -abstracting the hardware differences, so the entire driver doesn't need to be > > > > -duplicated. In addition, this framework also adds rpmsg virtio devices > > > > -for remote processors that supports this kind of communication. This way, > > > > -platform-specific remoteproc drivers only need to provide a few low-level > > > > -handlers, and then all rpmsg drivers will then just work > > > > -(for more information about the virtio-based rpmsg bus and its drivers, > > > > -please read Documentation/staging/rpmsg.rst). > > > > -Registration of other types of virtio devices is now also possible. Firmwares > > > > -just need to publish what kind of virtio devices do they support, and then > > > > -remoteproc will add those devices. This makes it possible to reuse the > > > > -existing virtio drivers with remote processor backends at a minimal development > > > > -cost. > > > > - > > > > -User API > > > > -======== > > > > - > > > > -:: > > > > - > > > > - int rproc_boot(struct rproc *rproc) > > > > - > > > > -Boot a remote processor (i.e. load its firmware, power it on, ...). > > > > - > > > > -If the remote processor is already powered on, this function immediately > > > > -returns (successfully). > > > > - > > > > -Returns 0 on success, and an appropriate error value otherwise. > > > > -Note: to use this function you should already have a valid rproc > > > > -handle. There are several ways to achieve that cleanly (devres, pdata, > > > > -the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we > > > > -might also consider using dev_archdata for this). > > > > - > > > > -:: > > > > - > > > > - void rproc_shutdown(struct rproc *rproc) > > > > - > > > > -Power off a remote processor (previously booted with rproc_boot()). > > > > -In case @rproc is still being used by an additional user(s), then > > > > -this function will just decrement the power refcount and exit, > > > > -without really powering off the device. > > > > - > > > > -Every call to rproc_boot() must (eventually) be accompanied by a call > > > > -to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. > > > > - > > > > -.. note:: > > > > - > > > > - we're not decrementing the rproc's refcount, only the power refcount. > > > > - which means that the @rproc handle stays valid even after > > > > - rproc_shutdown() returns, and users can still use it with a subsequent > > > > - rproc_boot(), if needed. > > > > - > > > > -:: > > > > - > > > > - struct rproc *rproc_get_by_phandle(phandle phandle) > > > > - > > > > -Find an rproc handle using a device tree phandle. Returns the rproc > > > > -handle on success, and NULL on failure. This function increments > > > > -the remote processor's refcount, so always use rproc_put() to > > > > -decrement it back once rproc isn't needed anymore. > > > > - > > > > -Typical usage > > > > -============= > > > > - > > > > -:: > > > > - > > > > - #include <linux/remoteproc.h> > > > > - > > > > - /* in case we were given a valid 'rproc' handle */ > > > > - int dummy_rproc_example(struct rproc *my_rproc) > > > > - { > > > > - int ret; > > > > - > > > > - /* let's power on and boot our remote processor */ > > > > - ret = rproc_boot(my_rproc); > > > > - if (ret) { > > > > - /* > > > > - * something went wrong. handle it and leave. > > > > - */ > > > > - } > > > > - > > > > - /* > > > > - * our remote processor is now powered on... give it some work > > > > - */ > > > > - > > > > - /* let's shut it down now */ > > > > - rproc_shutdown(my_rproc); > > > > - } > > > > - > > > > -API for implementors > > > > -==================== > > > > - > > > > -:: > > > > - > > > > - struct rproc *rproc_alloc(struct device *dev, const char *name, > > > > - const struct rproc_ops *ops, > > > > - const char *firmware, int len) > > > > - > > > > -Allocate a new remote processor handle, but don't register > > > > -it yet. Required parameters are the underlying device, the > > > > -name of this remote processor, platform-specific ops handlers, > > > > -the name of the firmware to boot this rproc with, and the > > > > -length of private data needed by the allocating rproc driver (in bytes). > > > > - > > > > -This function should be used by rproc implementations during > > > > -initialization of the remote processor. > > > > - > > > > -After creating an rproc handle using this function, and when ready, > > > > -implementations should then call rproc_add() to complete > > > > -the registration of the remote processor. > > > > - > > > > -On success, the new rproc is returned, and on failure, NULL. > > > > - > > > > -.. note:: > > > > - > > > > - **never** directly deallocate @rproc, even if it was not registered > > > > - yet. Instead, when you need to unroll rproc_alloc(), use rproc_free(). > > > > - > > > > -:: > > > > - > > > > - void rproc_free(struct rproc *rproc) > > > > - > > > > -Free an rproc handle that was allocated by rproc_alloc. > > > > - > > > > -This function essentially unrolls rproc_alloc(), by decrementing the > > > > -rproc's refcount. It doesn't directly free rproc; that would happen > > > > -only if there are no other references to rproc and its refcount now > > > > -dropped to zero. > > > > - > > > > -:: > > > > - > > > > - int rproc_add(struct rproc *rproc) > > > > - > > > > -Register @rproc with the remoteproc framework, after it has been > > > > -allocated with rproc_alloc(). > > > > - > > > > -This is called by the platform-specific rproc implementation, whenever > > > > -a new remote processor device is probed. > > > > - > > > > -Returns 0 on success and an appropriate error code otherwise. > > > > -Note: this function initiates an asynchronous firmware loading > > > > -context, which will look for virtio devices supported by the rproc's > > > > -firmware. > > > > - > > > > -If found, those virtio devices will be created and added, so as a result > > > > -of registering this remote processor, additional virtio drivers might get > > > > -probed. > > > > - > > > > -:: > > > > - > > > > - int rproc_del(struct rproc *rproc) > > > > - > > > > -Unroll rproc_add(). > > > > - > > > > -This function should be called when the platform specific rproc > > > > -implementation decides to remove the rproc device. it should > > > > -_only_ be called if a previous invocation of rproc_add() > > > > -has completed successfully. > > > > - > > > > -After rproc_del() returns, @rproc is still valid, and its > > > > -last refcount should be decremented by calling rproc_free(). > > > > - > > > > -Returns 0 on success and -EINVAL if @rproc isn't valid. > > > > - > > > > -:: > > > > - > > > > - void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type) > > > > - > > > > -Report a crash in a remoteproc > > > > - > > > > -This function must be called every time a crash is detected by the > > > > -platform specific rproc implementation. This should not be called from a > > > > -non-remoteproc driver. This function can be called from atomic/interrupt > > > > -context. > > > > - > > > > -Implementation callbacks > > > > +.. Contents: > > > > + > > > > + 1. Introduction > > > > + 2. Remoteproc framework responsibilities > > > > + 3. Remoteproc driver responsibilities > > > > + 4. Virtio and rpmsg > > > > + > > > > +1. Introduction > > > > +=============== > > > > + > > > > +Modern System on Chips (SoCs) typically integrate heterogeneous remote > > > > +processor devices in asymmetric multiprocessing (AMP) configurations. > > > > +These processors may run different operating systems, such as Linux and > > > > +various real-time operating systems (RTOS). > > > > > > You are moving things around _and_ making modifications to the text in the same > > > patch, something I specifically asked not to do. Moreover, the above conveys > > > > Sorry for doing that, I thought I moved the file first as you instructed but > > I didn't understand that I should also not reword the sentences that convey > > the same meaning. Will keep the text of below content in the same file: > > 1. Introduction > > 2. Binary Firmware Structure: will just add example as shown in this patch. > > 3. Virtio and remoteproc: didn't change anything in the patch for this. > > We don't seem to understand each other - the "Virtio and remoteproc" section is > the first thing I red and what made me aware of the rewording. I double checked and found you are right, I initially had changed it but had no intention of changing in following patches but somehow it creeped in. Will keep everything as before. > > > > > For below: > > 1. User API: > > 2. Typical usage: > > 3. API for implementors : > > > > Should I just move all of the above text as it is to > > remoteproc-kernel-api.rst or > > you prefer here as I feel it belongs there? > > I have looked at other subsystems documented under Documentation/driver-api and > I can't find instances of the type [subsystem]-kernel-api.rst. As such I don't > see what is so special about remoteproc that it would mandate adding a new file. > Why can't it just be in a single file like most subsystems do? Certainly, I can do that. Let me keep only the main file which was the case, just add the relevant updates to the sections that needs updates. Will that be agreeable to you? > > > > > > exactly the same information as found in [1] but using different words. I > > > am in favour of enhancing documentation but not creating unneeded churn. > > > > > > I found several instances of the same rewording pattern in the sections below. > > > As such I will not look at the other patches nor move forward with this set. > > > > > > Thanks, > > > Mathieu > > > > Thanks Mathieu, for reviewing and providing feedback as always. > > > > > > > > [1]. https://elixir.bootlin.com/linux/v6.12-rc4/source/Documentation/staging/remoteproc.rst > > > > > > > + > > > > +For example, the OMAP4 platform features dual Cortex-A9 cores, dual > > > > +Cortex-M3 cores, and a C64x+ DSP. In a standard setup, the Cortex-A9 > > > > +cores execute Linux in a symmetric multiprocessing (SMP) configuration, > > > > +while the M3 cores and DSP run independent instances of an RTOS. > > > > + > > > > +The remoteproc framework allows various platforms and architectures to > > > > +manage remote processors, including operations such as powering on, > > > > +loading firmware, and powering off. This framework abstracts hardware > > > > +differences, promoting code reuse and minimizing duplication. It also > > > > +supports rpmsg virtio devices for remote processors that utilize this > > > > +communication method. Consequently, platform-specific remoteproc drivers > > > > +need only implement a few low-level handlers, enabling seamless operation > > > > +of all rpmsg drivers. (For more details about the virtio-based rpmsg > > > > +bus and its drivers, refer to rpmsg documentation.) > > > > + > > > > +Additionally, the framework allows for the registration of various > > > > +virtio devices. Firmware can publish the types of virtio devices it > > > > +supports, facilitating their addition to the remoteproc framework. This > > > > +flexibility enables the reuse of existing virtio drivers with remote > > > > +processor backends at minimal development cost. > > > > + > > > > +The primary purpose of the remoteproc framework is to download firmware > > > > +for remote processors and manage their lifecycle. The framework consists > > > > +of several key components: > > > > + > > > > +- **Character Driver**: Provides userspace access to control the remote > > > > + processor. > > > > +- **ELF Utility**: Offers functions for handling ELF files and managing > > > > + resources requested by the remote processor. > > > > +- **Remoteproc Core**: Manages firmware downloads and recovery actions > > > > + in case of a remote processor crash. > > > > +- **Coredump**: Provides facilities for coredumping and tracing from > > > > + the remote processor in the event of a crash. > > > > +- **Userspace Interaction**: Uses sysfs and debugfs to manage the > > > > + lifecycle and status of the remote processor. > > > > +- **Virtio Support**: Facilitates interaction with the virtio and > > > > + rpmsg bus. > > > > + > > > > +From here on, references to "framework" denote the remoteproc > > > > +framework, and "driver" refers to the remoteproc driver that utilizes > > > > +the framework for managing remote processors. > > > > + > > > > +2. Remoteproc framework Responsibilities > > > > +======================================== > > > > + > > > > +The framework begins by gathering information about the firmware file > > > > +to be downloaded through the request_firmware function. It supports > > > > +the ELF format and parses the firmware image to identify the physical > > > > +addresses that need to be populated from the corresponding ELF sections. > > > > +The framework also requires knowledge of the logical or I/O-mapped > > > > +addresses in the application processor. Once this information is > > > > +obtained from the driver, the framework transfers the data to the > > > > +specified addresses and starts the remote, along with > > > > +any devices physically or logically connected to it. > > > > + > > > > +Dependent devices, referred to as `subdevices` within the framework, > > > > +are also managed post-registration by their respective drivers. > > > > +Subdevices can register themselves using `rproc_(add/remove)_subdev`. > > > > +Non-remoteproc drivers can use subdevices as a way to logically connect > > > > +to remote and get lifecycle notifications of the remote. > > > > + > > > > +The framework oversees the lifecycle of the remote and > > > > +provides the `rproc_report_crash` function, which the driver invokes > > > > +upon receiving a crash notification from the remote. The > > > > +notification method can differ based on the design of the remote > > > > +processor and its communication with the application processor. For > > > > +instance, if the remote is a DSP equipped with a watchdog, > > > > +unresponsive behavior triggers the watchdog, generating an interrupt > > > > +that routes to the application processor, allowing it to call > > > > +`rproc_report_crash` in the driver's interrupt context. > > > > + > > > > +During crash handling, the framework performs the following actions: > > > > + > > > > +a. Sends a request to stop the remote and any connected or > > > > + dependent subdevices. > > > > +b. Generates a coredump, dumping all `resources` requested by the > > > > + remote alongside relevant debugging information. Resources are > > > > + explained below. > > > > +c. Reloads the firmware and restarts the remote. > > > > + > > > > +If the `RPROC_FEAT_ATTACH_ON_RECOVERY` flag is set, the detach and > > > > +attach callbacks of the driver are invoked without reloading the > > > > +firmware. This is useful when the remote requires no > > > > +assistance for recovery, or when the application processor can restart > > > > +independently. After recovery, the application processor can reattach > > > > +to the remote. > > > > + > > > > +The remote can request resources from the framework, which > > > > +allocates a ".resource_table" section. During the ELF parsing phase, > > > > +the framework identifies this section and calls the appropriate > > > > +handler to allocate the requested resources. > > > > + > > > > +Resource management within the framework can accommodate any type of > > > > +`fw_resource_type`. > > > > + > > > > +.. code-block:: c > > > > + > > > > + enum fw_resource_type { > > > > + RSC_CARVEOUT = 0, > > > > + RSC_DEVMEM = 1, > > > > + RSC_TRACE = 2, > > > > + RSC_VDEV = 3, > > > > + RSC_LAST = 4, > > > > + RSC_VENDOR_START = 128, > > > > + RSC_VENDOR_END = 512, > > > > + }; > > > > + > > > > + struct resource_table { > > > > + u32 ver; > > > > + u32 num; > > > > + u32 reserved[2]; > > > > + u32 offset[]; > > > > + } __packed; > > > > + > > > > + struct fw_rsc_hdr { > > > > + u32 type; > > > > + u8 data[]; > > > > + } __packed; > > > > + > > > > +For example, if the remote requests both `RSC_TRACE` and > > > > +`RSC_CARVEOUT` for memory allocation, the ELF firmware can be structured > > > > +as follows: > > > > + > > > > +.. code-block:: c > > > > + > > > > + #define MAX_SHARED_RESOURCE 2 > > > > + #define LOG_BUF_SIZE 1000 > > > > + #define CARVEOUT_DUMP_PA 0x12345678 > > > > + #define CARVEOUT_DUMP_SIZE 2000 > > > > + > > > > + struct shared_resource_table { > > > > + u32 ver; > > > > + u32 num; > > > > + u32 reserved[2]; > > > > + u32 offset[MAX_SHARED_RESOURCE]; > > > > + struct fw_rsc_trace log_trace; > > > > + struct fw_rsc_carveout dump_carveout; > > > > + }; > > > > + > > > > + volatile struct shared_resource_table table = { > > > > + .ver = 1, > > > > + .num = 2, > > > > + .reserved = {0, 0}, > > > > + .offset = { > > > > + offsetof(struct resource_table, log_trace), > > > > + offsetof(struct resource_table, dump_carveout), > > > > + }, > > > > + .log_trace = { > > > > + RSC_TRACE, > > > > + (u32)log_buf, LOG_BUF_SIZE, 0, "log_trace", > > > > + }, > > > > + .dump_carveout = { > > > > + RSC_CARVEOUT, > > > > + (u32)FW_RSC_ADDR_ANY, CARVEOUT_PA, 0, "carveout_dump", > > > > + }, > > > > + }; > > > > + > > > > +The framework creates a sysfs file when it encounters the `RSC_TRACE` > > > > +type to expose log information to userspace. Other resource types are > > > > +handled accordingly. In the example above, `CARVEOUT_DUMP_SIZE` bytes > > > > +of DMA memory will be allocated starting from `CARVEOUT_DUMP_PA`. > > > > + > > > > + > > > > +3. Remoteproc driver responsibilities > > > > +===================================== > > > > + > > > > +The driver must provide the following information to the core: > > > > + > > > > +a. Translate device addresses (physical addresses) found in the ELF > > > > + firmware to virtual addresses in Linux using the `da_to_va` > > > > + callback. This allows the framework to copy ELF firmware from the > > > > + filesystem to the addresses expected by the remote since > > > > + the framework cannot directly access those physical addresses. > > > > +b. Prepare/unprepare the remote prior to firmware loading, > > > > + which may involve allocating carveout and reserved memory regions. > > > > +c. Implement methods for starting and stopping the remote, > > > > + whether by setting registers or sending explicit interrupts, > > > > + depending on the hardware design. > > > > +d. Provide attach and detach callbacks to start the remote > > > > + without loading the firmware. This is beneficial when the remote > > > > + processor is already loaded and running. > > > > +e. Implement a load callback for firmware loading, typically using > > > > + the ELF loader provided by the framework; currently, only ELF > > > > + format is supported. > > > > +f. Invoke the framework's crash handler API upon detecting a remote > > > > + crash. > > > > + > > > > +Drivers must fill the `rproc_ops` structure and call `rproc_alloc` > > > > +to register themselves with the framework. > > > > + > > > > +.. code-block:: c > > > > + > > > > + struct rproc_ops { > > > > + int (*prepare)(struct rproc *rproc); > > > > + int (*unprepare)(struct rproc *rproc); > > > > + int (*start)(struct rproc *rproc); > > > > + int (*stop)(struct rproc *rproc); > > > > + int (*attach)(struct rproc *rproc); > > > > + int (*detach)(struct rproc *rproc); > > > > + void * (*da_to_va)(struct rproc *rproc, u64 da, size_t len, > > > > + bool *is_iomem); > > > > + int (*parse_fw)(struct rproc *rproc, const struct firmware *fw); > > > > + int (*handle_rsc)(struct rproc *rproc, u32 rsc_type, > > > > + void *rsc, int offset, int avail); > > > > + int (*load)(struct rproc *rproc, const struct firmware *fw); > > > > + //snip > > > > + }; > > > > + > > > > + > > > > +4. Virtio and Remoteproc > > > > ======================== > > > > > > > > -These callbacks should be provided by platform-specific remoteproc > > > > -drivers:: > > > > - > > > > - /** > > > > - * struct rproc_ops - platform-specific device handlers > > > > - * @start: power on the device and boot it > > > > - * @stop: power off the device > > > > - * @kick: kick a virtqueue (virtqueue id given as a parameter) > > > > - */ > > > > - struct rproc_ops { > > > > - int (*start)(struct rproc *rproc); > > > > - int (*stop)(struct rproc *rproc); > > > > - void (*kick)(struct rproc *rproc, int vqid); > > > > - }; > > > > - > > > > -Every remoteproc implementation should at least provide the ->start and ->stop > > > > -handlers. If rpmsg/virtio functionality is also desired, then the ->kick handler > > > > -should be provided as well. > > > > - > > > > -The ->start() handler takes an rproc handle and should then power on the > > > > -device and boot it (use rproc->priv to access platform-specific private data). > > > > -The boot address, in case needed, can be found in rproc->bootaddr (remoteproc > > > > -core puts there the ELF entry point). > > > > -On success, 0 should be returned, and on failure, an appropriate error code. > > > > - > > > > -The ->stop() handler takes an rproc handle and powers the device down. > > > > -On success, 0 is returned, and on failure, an appropriate error code. > > > > - > > > > -The ->kick() handler takes an rproc handle, and an index of a virtqueue > > > > -where new message was placed in. Implementations should interrupt the remote > > > > -processor and let it know it has pending messages. Notifying remote processors > > > > -the exact virtqueue index to look in is optional: it is easy (and not > > > > -too expensive) to go through the existing virtqueues and look for new buffers > > > > -in the used rings. > > > > - > > > > -Binary Firmware Structure > > > > -========================= > > > > - > > > > -At this point remoteproc supports ELF32 and ELF64 firmware binaries. However, > > > > -it is quite expected that other platforms/devices which we'd want to > > > > -support with this framework will be based on different binary formats. > > > > - > > > > -When those use cases show up, we will have to decouple the binary format > > > > -from the framework core, so we can support several binary formats without > > > > -duplicating common code. > > > > - > > > > -When the firmware is parsed, its various segments are loaded to memory > > > > -according to the specified device address (might be a physical address > > > > -if the remote processor is accessing memory directly). > > > > - > > > > -In addition to the standard ELF segments, most remote processors would > > > > -also include a special section which we call "the resource table". > > > > - > > > > -The resource table contains system resources that the remote processor > > > > -requires before it should be powered on, such as allocation of physically > > > > -contiguous memory, or iommu mapping of certain on-chip peripherals. > > > > -Remotecore will only power up the device after all the resource table's > > > > -requirement are met. > > > > - > > > > -In addition to system resources, the resource table may also contain > > > > -resource entries that publish the existence of supported features > > > > -or configurations by the remote processor, such as trace buffers and > > > > -supported virtio devices (and their configurations). > > > > - > > > > -The resource table begins with this header:: > > > > - > > > > - /** > > > > - * struct resource_table - firmware resource table header > > > > - * @ver: version number > > > > - * @num: number of resource entries > > > > - * @reserved: reserved (must be zero) > > > > - * @offset: array of offsets pointing at the various resource entries > > > > - * > > > > - * The header of the resource table, as expressed by this structure, > > > > - * contains a version number (should we need to change this format in the > > > > - * future), the number of available resource entries, and their offsets > > > > - * in the table. > > > > - */ > > > > - struct resource_table { > > > > - u32 ver; > > > > - u32 num; > > > > - u32 reserved[2]; > > > > - u32 offset[0]; > > > > - } __packed; > > > > - > > > > -Immediately following this header are the resource entries themselves, > > > > -each of which begins with the following resource entry header:: > > > > - > > > > - /** > > > > - * struct fw_rsc_hdr - firmware resource entry header > > > > - * @type: resource type > > > > - * @data: resource data > > > > - * > > > > - * Every resource entry begins with a 'struct fw_rsc_hdr' header providing > > > > - * its @type. The content of the entry itself will immediately follow > > > > - * this header, and it should be parsed according to the resource type. > > > > - */ > > > > - struct fw_rsc_hdr { > > > > - u32 type; > > > > - u8 data[0]; > > > > - } __packed; > > > > - > > > > -Some resources entries are mere announcements, where the host is informed > > > > -of specific remoteproc configuration. Other entries require the host to > > > > -do something (e.g. allocate a system resource). Sometimes a negotiation > > > > -is expected, where the firmware requests a resource, and once allocated, > > > > -the host should provide back its details (e.g. address of an allocated > > > > -memory region). > > > > - > > > > -Here are the various resource types that are currently supported:: > > > > - > > > > - /** > > > > - * enum fw_resource_type - types of resource entries > > > > - * > > > > - * @RSC_CARVEOUT: request for allocation of a physically contiguous > > > > - * memory region. > > > > - * @RSC_DEVMEM: request to iommu_map a memory-based peripheral. > > > > - * @RSC_TRACE: announces the availability of a trace buffer into which > > > > - * the remote processor will be writing logs. > > > > - * @RSC_VDEV: declare support for a virtio device, and serve as its > > > > - * virtio header. > > > > - * @RSC_LAST: just keep this one at the end > > > > - * @RSC_VENDOR_START: start of the vendor specific resource types range > > > > - * @RSC_VENDOR_END: end of the vendor specific resource types range > > > > - * > > > > - * Please note that these values are used as indices to the rproc_handle_rsc > > > > - * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to > > > > - * check the validity of an index before the lookup table is accessed, so > > > > - * please update it as needed. > > > > - */ > > > > - enum fw_resource_type { > > > > - RSC_CARVEOUT = 0, > > > > - RSC_DEVMEM = 1, > > > > - RSC_TRACE = 2, > > > > - RSC_VDEV = 3, > > > > - RSC_LAST = 4, > > > > - RSC_VENDOR_START = 128, > > > > - RSC_VENDOR_END = 512, > > > > - }; > > > > - > > > > -For more details regarding a specific resource type, please see its > > > > -dedicated structure in include/linux/remoteproc.h. > > > > - > > > > -We also expect that platform-specific resource entries will show up > > > > -at some point. When that happens, we could easily add a new RSC_PLATFORM > > > > -type, and hand those resources to the platform-specific rproc driver to handle. > > > > - > > > > -Virtio and remoteproc > > > > -===================== > > > > - > > > > -The firmware should provide remoteproc information about virtio devices > > > > -that it supports, and their configurations: a RSC_VDEV resource entry > > > > -should specify the virtio device id (as in virtio_ids.h), virtio features, > > > > -virtio config space, vrings information, etc. > > > > - > > > > -When a new remote processor is registered, the remoteproc framework > > > > -will look for its resource table and will register the virtio devices > > > > -it supports. A firmware may support any number of virtio devices, and > > > > -of any type (a single remote processor can also easily support several > > > > -rpmsg virtio devices this way, if desired). > > > > - > > > > -Of course, RSC_VDEV resource entries are only good enough for static > > > > -allocation of virtio devices. Dynamic allocations will also be made possible > > > > -using the rpmsg bus (similar to how we already do dynamic allocations of > > > > -rpmsg channels; read more about it in rpmsg.txt). > > > > +The firmware must provide remoteproc with information regarding the > > > > +virtio devices it supports and their configurations: an `RSC_VDEV` > > > > +resource entry should detail the virtio device ID (as defined in > > > > +`virtio_ids.h`), virtio features, virtio config space, vrings > > > > +information, etc. > > > > + > > > > +Upon registration of a new remote, the remoteproc framework > > > > +searches for its resource table and registers the supported virtio > > > > +devices. A firmware may support multiple virtio devices, of various > > > > +types (a single remote can support multiple rpmsg virtio > > > > +devices if required). > > > > + > > > > +Moreover, `RSC_VDEV` resource entries suffice for static allocation > > > > +of virtio devices. Dynamic allocations will also be supported using > > > > +the rpmsg bus, akin to the handling of dynamic allocations for rpmsg > > > > +channels. For more information, refer to rpmsg documentation. > > > > -- > > > > 2.39.3 (Apple Git-146) > > > >
On Fri, 25 Oct 2024 at 09:13, anish kumar <yesanishhere@gmail.com> wrote: > > On Fri, Oct 25, 2024 at 7:57 AM Mathieu Poirier > <mathieu.poirier@linaro.org> wrote: > > > > On Thu, Oct 24, 2024 at 11:17:40AM -0700, anish kumar wrote: > > > On Thu, Oct 24, 2024 at 8:53 AM Mathieu Poirier > > > <mathieu.poirier@linaro.org> wrote: > > > > > > > > On Tue, Oct 22, 2024 at 10:33:53PM -0700, anish kumar wrote: > > > > > Added following changes: > > > > > 1. Components provided by remoteproc framework. > > > > > 2. Remoteproc driver responsibilities. > > > > > 3. Remoteproc framework responsibilities. > > > > > 4. Better explanation of how to ask for resources > > > > > from the framework by the remote processor. > > > > > > > > > > Signed-off-by: anish kumar <yesanishhere@gmail.com> > > > > > --- > > > > > .../driver-api/remoteproc/remoteproc.rst | 596 +++++++----------- > > > > > 1 file changed, 243 insertions(+), 353 deletions(-) > > > > > > > > > > diff --git a/Documentation/driver-api/remoteproc/remoteproc.rst b/Documentation/driver-api/remoteproc/remoteproc.rst > > > > > index 9cccd3dd6a4b..7ca545eea153 100644 > > > > > --- a/Documentation/driver-api/remoteproc/remoteproc.rst > > > > > +++ b/Documentation/driver-api/remoteproc/remoteproc.rst > > > > > @@ -1,359 +1,249 @@ > > > > > +.. SPDX-License-Identifier: GPL-2.0 > > > > > + > > > > > ========================== > > > > > Remote Processor Framework > > > > > ========================== > > > > > > > > > > -Introduction > > > > > -============ > > > > > - > > > > > -Modern SoCs typically have heterogeneous remote processor devices in asymmetric > > > > > -multiprocessing (AMP) configurations, which may be running different instances > > > > > -of operating system, whether it's Linux or any other flavor of real-time OS. > > > > > - > > > > > -OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. > > > > > -In a typical configuration, the dual cortex-A9 is running Linux in a SMP > > > > > -configuration, and each of the other three cores (two M3 cores and a DSP) > > > > > -is running its own instance of RTOS in an AMP configuration. > > > > > - > > > > > -The remoteproc framework allows different platforms/architectures to > > > > > -control (power on, load firmware, power off) those remote processors while > > > > > -abstracting the hardware differences, so the entire driver doesn't need to be > > > > > -duplicated. In addition, this framework also adds rpmsg virtio devices > > > > > -for remote processors that supports this kind of communication. This way, > > > > > -platform-specific remoteproc drivers only need to provide a few low-level > > > > > -handlers, and then all rpmsg drivers will then just work > > > > > -(for more information about the virtio-based rpmsg bus and its drivers, > > > > > -please read Documentation/staging/rpmsg.rst). > > > > > -Registration of other types of virtio devices is now also possible. Firmwares > > > > > -just need to publish what kind of virtio devices do they support, and then > > > > > -remoteproc will add those devices. This makes it possible to reuse the > > > > > -existing virtio drivers with remote processor backends at a minimal development > > > > > -cost. > > > > > - > > > > > -User API > > > > > -======== > > > > > - > > > > > -:: > > > > > - > > > > > - int rproc_boot(struct rproc *rproc) > > > > > - > > > > > -Boot a remote processor (i.e. load its firmware, power it on, ...). > > > > > - > > > > > -If the remote processor is already powered on, this function immediately > > > > > -returns (successfully). > > > > > - > > > > > -Returns 0 on success, and an appropriate error value otherwise. > > > > > -Note: to use this function you should already have a valid rproc > > > > > -handle. There are several ways to achieve that cleanly (devres, pdata, > > > > > -the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we > > > > > -might also consider using dev_archdata for this). > > > > > - > > > > > -:: > > > > > - > > > > > - void rproc_shutdown(struct rproc *rproc) > > > > > - > > > > > -Power off a remote processor (previously booted with rproc_boot()). > > > > > -In case @rproc is still being used by an additional user(s), then > > > > > -this function will just decrement the power refcount and exit, > > > > > -without really powering off the device. > > > > > - > > > > > -Every call to rproc_boot() must (eventually) be accompanied by a call > > > > > -to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. > > > > > - > > > > > -.. note:: > > > > > - > > > > > - we're not decrementing the rproc's refcount, only the power refcount. > > > > > - which means that the @rproc handle stays valid even after > > > > > - rproc_shutdown() returns, and users can still use it with a subsequent > > > > > - rproc_boot(), if needed. > > > > > - > > > > > -:: > > > > > - > > > > > - struct rproc *rproc_get_by_phandle(phandle phandle) > > > > > - > > > > > -Find an rproc handle using a device tree phandle. Returns the rproc > > > > > -handle on success, and NULL on failure. This function increments > > > > > -the remote processor's refcount, so always use rproc_put() to > > > > > -decrement it back once rproc isn't needed anymore. > > > > > - > > > > > -Typical usage > > > > > -============= > > > > > - > > > > > -:: > > > > > - > > > > > - #include <linux/remoteproc.h> > > > > > - > > > > > - /* in case we were given a valid 'rproc' handle */ > > > > > - int dummy_rproc_example(struct rproc *my_rproc) > > > > > - { > > > > > - int ret; > > > > > - > > > > > - /* let's power on and boot our remote processor */ > > > > > - ret = rproc_boot(my_rproc); > > > > > - if (ret) { > > > > > - /* > > > > > - * something went wrong. handle it and leave. > > > > > - */ > > > > > - } > > > > > - > > > > > - /* > > > > > - * our remote processor is now powered on... give it some work > > > > > - */ > > > > > - > > > > > - /* let's shut it down now */ > > > > > - rproc_shutdown(my_rproc); > > > > > - } > > > > > - > > > > > -API for implementors > > > > > -==================== > > > > > - > > > > > -:: > > > > > - > > > > > - struct rproc *rproc_alloc(struct device *dev, const char *name, > > > > > - const struct rproc_ops *ops, > > > > > - const char *firmware, int len) > > > > > - > > > > > -Allocate a new remote processor handle, but don't register > > > > > -it yet. Required parameters are the underlying device, the > > > > > -name of this remote processor, platform-specific ops handlers, > > > > > -the name of the firmware to boot this rproc with, and the > > > > > -length of private data needed by the allocating rproc driver (in bytes). > > > > > - > > > > > -This function should be used by rproc implementations during > > > > > -initialization of the remote processor. > > > > > - > > > > > -After creating an rproc handle using this function, and when ready, > > > > > -implementations should then call rproc_add() to complete > > > > > -the registration of the remote processor. > > > > > - > > > > > -On success, the new rproc is returned, and on failure, NULL. > > > > > - > > > > > -.. note:: > > > > > - > > > > > - **never** directly deallocate @rproc, even if it was not registered > > > > > - yet. Instead, when you need to unroll rproc_alloc(), use rproc_free(). > > > > > - > > > > > -:: > > > > > - > > > > > - void rproc_free(struct rproc *rproc) > > > > > - > > > > > -Free an rproc handle that was allocated by rproc_alloc. > > > > > - > > > > > -This function essentially unrolls rproc_alloc(), by decrementing the > > > > > -rproc's refcount. It doesn't directly free rproc; that would happen > > > > > -only if there are no other references to rproc and its refcount now > > > > > -dropped to zero. > > > > > - > > > > > -:: > > > > > - > > > > > - int rproc_add(struct rproc *rproc) > > > > > - > > > > > -Register @rproc with the remoteproc framework, after it has been > > > > > -allocated with rproc_alloc(). > > > > > - > > > > > -This is called by the platform-specific rproc implementation, whenever > > > > > -a new remote processor device is probed. > > > > > - > > > > > -Returns 0 on success and an appropriate error code otherwise. > > > > > -Note: this function initiates an asynchronous firmware loading > > > > > -context, which will look for virtio devices supported by the rproc's > > > > > -firmware. > > > > > - > > > > > -If found, those virtio devices will be created and added, so as a result > > > > > -of registering this remote processor, additional virtio drivers might get > > > > > -probed. > > > > > - > > > > > -:: > > > > > - > > > > > - int rproc_del(struct rproc *rproc) > > > > > - > > > > > -Unroll rproc_add(). > > > > > - > > > > > -This function should be called when the platform specific rproc > > > > > -implementation decides to remove the rproc device. it should > > > > > -_only_ be called if a previous invocation of rproc_add() > > > > > -has completed successfully. > > > > > - > > > > > -After rproc_del() returns, @rproc is still valid, and its > > > > > -last refcount should be decremented by calling rproc_free(). > > > > > - > > > > > -Returns 0 on success and -EINVAL if @rproc isn't valid. > > > > > - > > > > > -:: > > > > > - > > > > > - void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type) > > > > > - > > > > > -Report a crash in a remoteproc > > > > > - > > > > > -This function must be called every time a crash is detected by the > > > > > -platform specific rproc implementation. This should not be called from a > > > > > -non-remoteproc driver. This function can be called from atomic/interrupt > > > > > -context. > > > > > - > > > > > -Implementation callbacks > > > > > +.. Contents: > > > > > + > > > > > + 1. Introduction > > > > > + 2. Remoteproc framework responsibilities > > > > > + 3. Remoteproc driver responsibilities > > > > > + 4. Virtio and rpmsg > > > > > + > > > > > +1. Introduction > > > > > +=============== > > > > > + > > > > > +Modern System on Chips (SoCs) typically integrate heterogeneous remote > > > > > +processor devices in asymmetric multiprocessing (AMP) configurations. > > > > > +These processors may run different operating systems, such as Linux and > > > > > +various real-time operating systems (RTOS). > > > > > > > > You are moving things around _and_ making modifications to the text in the same > > > > patch, something I specifically asked not to do. Moreover, the above conveys > > > > > > Sorry for doing that, I thought I moved the file first as you instructed but > > > I didn't understand that I should also not reword the sentences that convey > > > the same meaning. Will keep the text of below content in the same file: > > > 1. Introduction > > > 2. Binary Firmware Structure: will just add example as shown in this patch. > > > 3. Virtio and remoteproc: didn't change anything in the patch for this. > > > > We don't seem to understand each other - the "Virtio and remoteproc" section is > > the first thing I red and what made me aware of the rewording. > > I double checked and found you are right, I initially had changed > it but had no intention of changing in following > patches but somehow it creeped in. Will keep everything as before. > > > > > > > > > For below: > > > 1. User API: > > > 2. Typical usage: > > > 3. API for implementors : > > > > > > Should I just move all of the above text as it is to > > > remoteproc-kernel-api.rst or > > > you prefer here as I feel it belongs there? > > > > I have looked at other subsystems documented under Documentation/driver-api and > > I can't find instances of the type [subsystem]-kernel-api.rst. As such I don't > > see what is so special about remoteproc that it would mandate adding a new file. > > Why can't it just be in a single file like most subsystems do? > > Certainly, I can do that. Let me keep only the main file which > was the case, just add the relevant updates to the sections that > needs updates. Will that be agreeable to you? > That's all I'm asking for. > > > > > > > > > exactly the same information as found in [1] but using different words. I > > > > am in favour of enhancing documentation but not creating unneeded churn. > > > > > > > > I found several instances of the same rewording pattern in the sections below. > > > > As such I will not look at the other patches nor move forward with this set. > > > > > > > > Thanks, > > > > Mathieu > > > > > > Thanks Mathieu, for reviewing and providing feedback as always. > > > > > > > > > > > [1]. https://elixir.bootlin.com/linux/v6.12-rc4/source/Documentation/staging/remoteproc.rst > > > > > > > > > + > > > > > +For example, the OMAP4 platform features dual Cortex-A9 cores, dual > > > > > +Cortex-M3 cores, and a C64x+ DSP. In a standard setup, the Cortex-A9 > > > > > +cores execute Linux in a symmetric multiprocessing (SMP) configuration, > > > > > +while the M3 cores and DSP run independent instances of an RTOS. > > > > > + > > > > > +The remoteproc framework allows various platforms and architectures to > > > > > +manage remote processors, including operations such as powering on, > > > > > +loading firmware, and powering off. This framework abstracts hardware > > > > > +differences, promoting code reuse and minimizing duplication. It also > > > > > +supports rpmsg virtio devices for remote processors that utilize this > > > > > +communication method. Consequently, platform-specific remoteproc drivers > > > > > +need only implement a few low-level handlers, enabling seamless operation > > > > > +of all rpmsg drivers. (For more details about the virtio-based rpmsg > > > > > +bus and its drivers, refer to rpmsg documentation.) > > > > > + > > > > > +Additionally, the framework allows for the registration of various > > > > > +virtio devices. Firmware can publish the types of virtio devices it > > > > > +supports, facilitating their addition to the remoteproc framework. This > > > > > +flexibility enables the reuse of existing virtio drivers with remote > > > > > +processor backends at minimal development cost. > > > > > + > > > > > +The primary purpose of the remoteproc framework is to download firmware > > > > > +for remote processors and manage their lifecycle. The framework consists > > > > > +of several key components: > > > > > + > > > > > +- **Character Driver**: Provides userspace access to control the remote > > > > > + processor. > > > > > +- **ELF Utility**: Offers functions for handling ELF files and managing > > > > > + resources requested by the remote processor. > > > > > +- **Remoteproc Core**: Manages firmware downloads and recovery actions > > > > > + in case of a remote processor crash. > > > > > +- **Coredump**: Provides facilities for coredumping and tracing from > > > > > + the remote processor in the event of a crash. > > > > > +- **Userspace Interaction**: Uses sysfs and debugfs to manage the > > > > > + lifecycle and status of the remote processor. > > > > > +- **Virtio Support**: Facilitates interaction with the virtio and > > > > > + rpmsg bus. > > > > > + > > > > > +From here on, references to "framework" denote the remoteproc > > > > > +framework, and "driver" refers to the remoteproc driver that utilizes > > > > > +the framework for managing remote processors. > > > > > + > > > > > +2. Remoteproc framework Responsibilities > > > > > +======================================== > > > > > + > > > > > +The framework begins by gathering information about the firmware file > > > > > +to be downloaded through the request_firmware function. It supports > > > > > +the ELF format and parses the firmware image to identify the physical > > > > > +addresses that need to be populated from the corresponding ELF sections. > > > > > +The framework also requires knowledge of the logical or I/O-mapped > > > > > +addresses in the application processor. Once this information is > > > > > +obtained from the driver, the framework transfers the data to the > > > > > +specified addresses and starts the remote, along with > > > > > +any devices physically or logically connected to it. > > > > > + > > > > > +Dependent devices, referred to as `subdevices` within the framework, > > > > > +are also managed post-registration by their respective drivers. > > > > > +Subdevices can register themselves using `rproc_(add/remove)_subdev`. > > > > > +Non-remoteproc drivers can use subdevices as a way to logically connect > > > > > +to remote and get lifecycle notifications of the remote. > > > > > + > > > > > +The framework oversees the lifecycle of the remote and > > > > > +provides the `rproc_report_crash` function, which the driver invokes > > > > > +upon receiving a crash notification from the remote. The > > > > > +notification method can differ based on the design of the remote > > > > > +processor and its communication with the application processor. For > > > > > +instance, if the remote is a DSP equipped with a watchdog, > > > > > +unresponsive behavior triggers the watchdog, generating an interrupt > > > > > +that routes to the application processor, allowing it to call > > > > > +`rproc_report_crash` in the driver's interrupt context. > > > > > + > > > > > +During crash handling, the framework performs the following actions: > > > > > + > > > > > +a. Sends a request to stop the remote and any connected or > > > > > + dependent subdevices. > > > > > +b. Generates a coredump, dumping all `resources` requested by the > > > > > + remote alongside relevant debugging information. Resources are > > > > > + explained below. > > > > > +c. Reloads the firmware and restarts the remote. > > > > > + > > > > > +If the `RPROC_FEAT_ATTACH_ON_RECOVERY` flag is set, the detach and > > > > > +attach callbacks of the driver are invoked without reloading the > > > > > +firmware. This is useful when the remote requires no > > > > > +assistance for recovery, or when the application processor can restart > > > > > +independently. After recovery, the application processor can reattach > > > > > +to the remote. > > > > > + > > > > > +The remote can request resources from the framework, which > > > > > +allocates a ".resource_table" section. During the ELF parsing phase, > > > > > +the framework identifies this section and calls the appropriate > > > > > +handler to allocate the requested resources. > > > > > + > > > > > +Resource management within the framework can accommodate any type of > > > > > +`fw_resource_type`. > > > > > + > > > > > +.. code-block:: c > > > > > + > > > > > + enum fw_resource_type { > > > > > + RSC_CARVEOUT = 0, > > > > > + RSC_DEVMEM = 1, > > > > > + RSC_TRACE = 2, > > > > > + RSC_VDEV = 3, > > > > > + RSC_LAST = 4, > > > > > + RSC_VENDOR_START = 128, > > > > > + RSC_VENDOR_END = 512, > > > > > + }; > > > > > + > > > > > + struct resource_table { > > > > > + u32 ver; > > > > > + u32 num; > > > > > + u32 reserved[2]; > > > > > + u32 offset[]; > > > > > + } __packed; > > > > > + > > > > > + struct fw_rsc_hdr { > > > > > + u32 type; > > > > > + u8 data[]; > > > > > + } __packed; > > > > > + > > > > > +For example, if the remote requests both `RSC_TRACE` and > > > > > +`RSC_CARVEOUT` for memory allocation, the ELF firmware can be structured > > > > > +as follows: > > > > > + > > > > > +.. code-block:: c > > > > > + > > > > > + #define MAX_SHARED_RESOURCE 2 > > > > > + #define LOG_BUF_SIZE 1000 > > > > > + #define CARVEOUT_DUMP_PA 0x12345678 > > > > > + #define CARVEOUT_DUMP_SIZE 2000 > > > > > + > > > > > + struct shared_resource_table { > > > > > + u32 ver; > > > > > + u32 num; > > > > > + u32 reserved[2]; > > > > > + u32 offset[MAX_SHARED_RESOURCE]; > > > > > + struct fw_rsc_trace log_trace; > > > > > + struct fw_rsc_carveout dump_carveout; > > > > > + }; > > > > > + > > > > > + volatile struct shared_resource_table table = { > > > > > + .ver = 1, > > > > > + .num = 2, > > > > > + .reserved = {0, 0}, > > > > > + .offset = { > > > > > + offsetof(struct resource_table, log_trace), > > > > > + offsetof(struct resource_table, dump_carveout), > > > > > + }, > > > > > + .log_trace = { > > > > > + RSC_TRACE, > > > > > + (u32)log_buf, LOG_BUF_SIZE, 0, "log_trace", > > > > > + }, > > > > > + .dump_carveout = { > > > > > + RSC_CARVEOUT, > > > > > + (u32)FW_RSC_ADDR_ANY, CARVEOUT_PA, 0, "carveout_dump", > > > > > + }, > > > > > + }; > > > > > + > > > > > +The framework creates a sysfs file when it encounters the `RSC_TRACE` > > > > > +type to expose log information to userspace. Other resource types are > > > > > +handled accordingly. In the example above, `CARVEOUT_DUMP_SIZE` bytes > > > > > +of DMA memory will be allocated starting from `CARVEOUT_DUMP_PA`. > > > > > + > > > > > + > > > > > +3. Remoteproc driver responsibilities > > > > > +===================================== > > > > > + > > > > > +The driver must provide the following information to the core: > > > > > + > > > > > +a. Translate device addresses (physical addresses) found in the ELF > > > > > + firmware to virtual addresses in Linux using the `da_to_va` > > > > > + callback. This allows the framework to copy ELF firmware from the > > > > > + filesystem to the addresses expected by the remote since > > > > > + the framework cannot directly access those physical addresses. > > > > > +b. Prepare/unprepare the remote prior to firmware loading, > > > > > + which may involve allocating carveout and reserved memory regions. > > > > > +c. Implement methods for starting and stopping the remote, > > > > > + whether by setting registers or sending explicit interrupts, > > > > > + depending on the hardware design. > > > > > +d. Provide attach and detach callbacks to start the remote > > > > > + without loading the firmware. This is beneficial when the remote > > > > > + processor is already loaded and running. > > > > > +e. Implement a load callback for firmware loading, typically using > > > > > + the ELF loader provided by the framework; currently, only ELF > > > > > + format is supported. > > > > > +f. Invoke the framework's crash handler API upon detecting a remote > > > > > + crash. > > > > > + > > > > > +Drivers must fill the `rproc_ops` structure and call `rproc_alloc` > > > > > +to register themselves with the framework. > > > > > + > > > > > +.. code-block:: c > > > > > + > > > > > + struct rproc_ops { > > > > > + int (*prepare)(struct rproc *rproc); > > > > > + int (*unprepare)(struct rproc *rproc); > > > > > + int (*start)(struct rproc *rproc); > > > > > + int (*stop)(struct rproc *rproc); > > > > > + int (*attach)(struct rproc *rproc); > > > > > + int (*detach)(struct rproc *rproc); > > > > > + void * (*da_to_va)(struct rproc *rproc, u64 da, size_t len, > > > > > + bool *is_iomem); > > > > > + int (*parse_fw)(struct rproc *rproc, const struct firmware *fw); > > > > > + int (*handle_rsc)(struct rproc *rproc, u32 rsc_type, > > > > > + void *rsc, int offset, int avail); > > > > > + int (*load)(struct rproc *rproc, const struct firmware *fw); > > > > > + //snip > > > > > + }; > > > > > + > > > > > + > > > > > +4. Virtio and Remoteproc > > > > > ======================== > > > > > > > > > > -These callbacks should be provided by platform-specific remoteproc > > > > > -drivers:: > > > > > - > > > > > - /** > > > > > - * struct rproc_ops - platform-specific device handlers > > > > > - * @start: power on the device and boot it > > > > > - * @stop: power off the device > > > > > - * @kick: kick a virtqueue (virtqueue id given as a parameter) > > > > > - */ > > > > > - struct rproc_ops { > > > > > - int (*start)(struct rproc *rproc); > > > > > - int (*stop)(struct rproc *rproc); > > > > > - void (*kick)(struct rproc *rproc, int vqid); > > > > > - }; > > > > > - > > > > > -Every remoteproc implementation should at least provide the ->start and ->stop > > > > > -handlers. If rpmsg/virtio functionality is also desired, then the ->kick handler > > > > > -should be provided as well. > > > > > - > > > > > -The ->start() handler takes an rproc handle and should then power on the > > > > > -device and boot it (use rproc->priv to access platform-specific private data). > > > > > -The boot address, in case needed, can be found in rproc->bootaddr (remoteproc > > > > > -core puts there the ELF entry point). > > > > > -On success, 0 should be returned, and on failure, an appropriate error code. > > > > > - > > > > > -The ->stop() handler takes an rproc handle and powers the device down. > > > > > -On success, 0 is returned, and on failure, an appropriate error code. > > > > > - > > > > > -The ->kick() handler takes an rproc handle, and an index of a virtqueue > > > > > -where new message was placed in. Implementations should interrupt the remote > > > > > -processor and let it know it has pending messages. Notifying remote processors > > > > > -the exact virtqueue index to look in is optional: it is easy (and not > > > > > -too expensive) to go through the existing virtqueues and look for new buffers > > > > > -in the used rings. > > > > > - > > > > > -Binary Firmware Structure > > > > > -========================= > > > > > - > > > > > -At this point remoteproc supports ELF32 and ELF64 firmware binaries. However, > > > > > -it is quite expected that other platforms/devices which we'd want to > > > > > -support with this framework will be based on different binary formats. > > > > > - > > > > > -When those use cases show up, we will have to decouple the binary format > > > > > -from the framework core, so we can support several binary formats without > > > > > -duplicating common code. > > > > > - > > > > > -When the firmware is parsed, its various segments are loaded to memory > > > > > -according to the specified device address (might be a physical address > > > > > -if the remote processor is accessing memory directly). > > > > > - > > > > > -In addition to the standard ELF segments, most remote processors would > > > > > -also include a special section which we call "the resource table". > > > > > - > > > > > -The resource table contains system resources that the remote processor > > > > > -requires before it should be powered on, such as allocation of physically > > > > > -contiguous memory, or iommu mapping of certain on-chip peripherals. > > > > > -Remotecore will only power up the device after all the resource table's > > > > > -requirement are met. > > > > > - > > > > > -In addition to system resources, the resource table may also contain > > > > > -resource entries that publish the existence of supported features > > > > > -or configurations by the remote processor, such as trace buffers and > > > > > -supported virtio devices (and their configurations). > > > > > - > > > > > -The resource table begins with this header:: > > > > > - > > > > > - /** > > > > > - * struct resource_table - firmware resource table header > > > > > - * @ver: version number > > > > > - * @num: number of resource entries > > > > > - * @reserved: reserved (must be zero) > > > > > - * @offset: array of offsets pointing at the various resource entries > > > > > - * > > > > > - * The header of the resource table, as expressed by this structure, > > > > > - * contains a version number (should we need to change this format in the > > > > > - * future), the number of available resource entries, and their offsets > > > > > - * in the table. > > > > > - */ > > > > > - struct resource_table { > > > > > - u32 ver; > > > > > - u32 num; > > > > > - u32 reserved[2]; > > > > > - u32 offset[0]; > > > > > - } __packed; > > > > > - > > > > > -Immediately following this header are the resource entries themselves, > > > > > -each of which begins with the following resource entry header:: > > > > > - > > > > > - /** > > > > > - * struct fw_rsc_hdr - firmware resource entry header > > > > > - * @type: resource type > > > > > - * @data: resource data > > > > > - * > > > > > - * Every resource entry begins with a 'struct fw_rsc_hdr' header providing > > > > > - * its @type. The content of the entry itself will immediately follow > > > > > - * this header, and it should be parsed according to the resource type. > > > > > - */ > > > > > - struct fw_rsc_hdr { > > > > > - u32 type; > > > > > - u8 data[0]; > > > > > - } __packed; > > > > > - > > > > > -Some resources entries are mere announcements, where the host is informed > > > > > -of specific remoteproc configuration. Other entries require the host to > > > > > -do something (e.g. allocate a system resource). Sometimes a negotiation > > > > > -is expected, where the firmware requests a resource, and once allocated, > > > > > -the host should provide back its details (e.g. address of an allocated > > > > > -memory region). > > > > > - > > > > > -Here are the various resource types that are currently supported:: > > > > > - > > > > > - /** > > > > > - * enum fw_resource_type - types of resource entries > > > > > - * > > > > > - * @RSC_CARVEOUT: request for allocation of a physically contiguous > > > > > - * memory region. > > > > > - * @RSC_DEVMEM: request to iommu_map a memory-based peripheral. > > > > > - * @RSC_TRACE: announces the availability of a trace buffer into which > > > > > - * the remote processor will be writing logs. > > > > > - * @RSC_VDEV: declare support for a virtio device, and serve as its > > > > > - * virtio header. > > > > > - * @RSC_LAST: just keep this one at the end > > > > > - * @RSC_VENDOR_START: start of the vendor specific resource types range > > > > > - * @RSC_VENDOR_END: end of the vendor specific resource types range > > > > > - * > > > > > - * Please note that these values are used as indices to the rproc_handle_rsc > > > > > - * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to > > > > > - * check the validity of an index before the lookup table is accessed, so > > > > > - * please update it as needed. > > > > > - */ > > > > > - enum fw_resource_type { > > > > > - RSC_CARVEOUT = 0, > > > > > - RSC_DEVMEM = 1, > > > > > - RSC_TRACE = 2, > > > > > - RSC_VDEV = 3, > > > > > - RSC_LAST = 4, > > > > > - RSC_VENDOR_START = 128, > > > > > - RSC_VENDOR_END = 512, > > > > > - }; > > > > > - > > > > > -For more details regarding a specific resource type, please see its > > > > > -dedicated structure in include/linux/remoteproc.h. > > > > > - > > > > > -We also expect that platform-specific resource entries will show up > > > > > -at some point. When that happens, we could easily add a new RSC_PLATFORM > > > > > -type, and hand those resources to the platform-specific rproc driver to handle. > > > > > - > > > > > -Virtio and remoteproc > > > > > -===================== > > > > > - > > > > > -The firmware should provide remoteproc information about virtio devices > > > > > -that it supports, and their configurations: a RSC_VDEV resource entry > > > > > -should specify the virtio device id (as in virtio_ids.h), virtio features, > > > > > -virtio config space, vrings information, etc. > > > > > - > > > > > -When a new remote processor is registered, the remoteproc framework > > > > > -will look for its resource table and will register the virtio devices > > > > > -it supports. A firmware may support any number of virtio devices, and > > > > > -of any type (a single remote processor can also easily support several > > > > > -rpmsg virtio devices this way, if desired). > > > > > - > > > > > -Of course, RSC_VDEV resource entries are only good enough for static > > > > > -allocation of virtio devices. Dynamic allocations will also be made possible > > > > > -using the rpmsg bus (similar to how we already do dynamic allocations of > > > > > -rpmsg channels; read more about it in rpmsg.txt). > > > > > +The firmware must provide remoteproc with information regarding the > > > > > +virtio devices it supports and their configurations: an `RSC_VDEV` > > > > > +resource entry should detail the virtio device ID (as defined in > > > > > +`virtio_ids.h`), virtio features, virtio config space, vrings > > > > > +information, etc. > > > > > + > > > > > +Upon registration of a new remote, the remoteproc framework > > > > > +searches for its resource table and registers the supported virtio > > > > > +devices. A firmware may support multiple virtio devices, of various > > > > > +types (a single remote can support multiple rpmsg virtio > > > > > +devices if required). > > > > > + > > > > > +Moreover, `RSC_VDEV` resource entries suffice for static allocation > > > > > +of virtio devices. Dynamic allocations will also be supported using > > > > > +the rpmsg bus, akin to the handling of dynamic allocations for rpmsg > > > > > +channels. For more information, refer to rpmsg documentation. > > > > > -- > > > > > 2.39.3 (Apple Git-146) > > > > >
diff --git a/Documentation/driver-api/remoteproc/remoteproc.rst b/Documentation/driver-api/remoteproc/remoteproc.rst index 9cccd3dd6a4b..7ca545eea153 100644 --- a/Documentation/driver-api/remoteproc/remoteproc.rst +++ b/Documentation/driver-api/remoteproc/remoteproc.rst @@ -1,359 +1,249 @@ +.. SPDX-License-Identifier: GPL-2.0 + ========================== Remote Processor Framework ========================== -Introduction -============ - -Modern SoCs typically have heterogeneous remote processor devices in asymmetric -multiprocessing (AMP) configurations, which may be running different instances -of operating system, whether it's Linux or any other flavor of real-time OS. - -OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP. -In a typical configuration, the dual cortex-A9 is running Linux in a SMP -configuration, and each of the other three cores (two M3 cores and a DSP) -is running its own instance of RTOS in an AMP configuration. - -The remoteproc framework allows different platforms/architectures to -control (power on, load firmware, power off) those remote processors while -abstracting the hardware differences, so the entire driver doesn't need to be -duplicated. In addition, this framework also adds rpmsg virtio devices -for remote processors that supports this kind of communication. This way, -platform-specific remoteproc drivers only need to provide a few low-level -handlers, and then all rpmsg drivers will then just work -(for more information about the virtio-based rpmsg bus and its drivers, -please read Documentation/staging/rpmsg.rst). -Registration of other types of virtio devices is now also possible. Firmwares -just need to publish what kind of virtio devices do they support, and then -remoteproc will add those devices. This makes it possible to reuse the -existing virtio drivers with remote processor backends at a minimal development -cost. - -User API -======== - -:: - - int rproc_boot(struct rproc *rproc) - -Boot a remote processor (i.e. load its firmware, power it on, ...). - -If the remote processor is already powered on, this function immediately -returns (successfully). - -Returns 0 on success, and an appropriate error value otherwise. -Note: to use this function you should already have a valid rproc -handle. There are several ways to achieve that cleanly (devres, pdata, -the way remoteproc_rpmsg.c does this, or, if this becomes prevalent, we -might also consider using dev_archdata for this). - -:: - - void rproc_shutdown(struct rproc *rproc) - -Power off a remote processor (previously booted with rproc_boot()). -In case @rproc is still being used by an additional user(s), then -this function will just decrement the power refcount and exit, -without really powering off the device. - -Every call to rproc_boot() must (eventually) be accompanied by a call -to rproc_shutdown(). Calling rproc_shutdown() redundantly is a bug. - -.. note:: - - we're not decrementing the rproc's refcount, only the power refcount. - which means that the @rproc handle stays valid even after - rproc_shutdown() returns, and users can still use it with a subsequent - rproc_boot(), if needed. - -:: - - struct rproc *rproc_get_by_phandle(phandle phandle) - -Find an rproc handle using a device tree phandle. Returns the rproc -handle on success, and NULL on failure. This function increments -the remote processor's refcount, so always use rproc_put() to -decrement it back once rproc isn't needed anymore. - -Typical usage -============= - -:: - - #include <linux/remoteproc.h> - - /* in case we were given a valid 'rproc' handle */ - int dummy_rproc_example(struct rproc *my_rproc) - { - int ret; - - /* let's power on and boot our remote processor */ - ret = rproc_boot(my_rproc); - if (ret) { - /* - * something went wrong. handle it and leave. - */ - } - - /* - * our remote processor is now powered on... give it some work - */ - - /* let's shut it down now */ - rproc_shutdown(my_rproc); - } - -API for implementors -==================== - -:: - - struct rproc *rproc_alloc(struct device *dev, const char *name, - const struct rproc_ops *ops, - const char *firmware, int len) - -Allocate a new remote processor handle, but don't register -it yet. Required parameters are the underlying device, the -name of this remote processor, platform-specific ops handlers, -the name of the firmware to boot this rproc with, and the -length of private data needed by the allocating rproc driver (in bytes). - -This function should be used by rproc implementations during -initialization of the remote processor. - -After creating an rproc handle using this function, and when ready, -implementations should then call rproc_add() to complete -the registration of the remote processor. - -On success, the new rproc is returned, and on failure, NULL. - -.. note:: - - **never** directly deallocate @rproc, even if it was not registered - yet. Instead, when you need to unroll rproc_alloc(), use rproc_free(). - -:: - - void rproc_free(struct rproc *rproc) - -Free an rproc handle that was allocated by rproc_alloc. - -This function essentially unrolls rproc_alloc(), by decrementing the -rproc's refcount. It doesn't directly free rproc; that would happen -only if there are no other references to rproc and its refcount now -dropped to zero. - -:: - - int rproc_add(struct rproc *rproc) - -Register @rproc with the remoteproc framework, after it has been -allocated with rproc_alloc(). - -This is called by the platform-specific rproc implementation, whenever -a new remote processor device is probed. - -Returns 0 on success and an appropriate error code otherwise. -Note: this function initiates an asynchronous firmware loading -context, which will look for virtio devices supported by the rproc's -firmware. - -If found, those virtio devices will be created and added, so as a result -of registering this remote processor, additional virtio drivers might get -probed. - -:: - - int rproc_del(struct rproc *rproc) - -Unroll rproc_add(). - -This function should be called when the platform specific rproc -implementation decides to remove the rproc device. it should -_only_ be called if a previous invocation of rproc_add() -has completed successfully. - -After rproc_del() returns, @rproc is still valid, and its -last refcount should be decremented by calling rproc_free(). - -Returns 0 on success and -EINVAL if @rproc isn't valid. - -:: - - void rproc_report_crash(struct rproc *rproc, enum rproc_crash_type type) - -Report a crash in a remoteproc - -This function must be called every time a crash is detected by the -platform specific rproc implementation. This should not be called from a -non-remoteproc driver. This function can be called from atomic/interrupt -context. - -Implementation callbacks +.. Contents: + + 1. Introduction + 2. Remoteproc framework responsibilities + 3. Remoteproc driver responsibilities + 4. Virtio and rpmsg + +1. Introduction +=============== + +Modern System on Chips (SoCs) typically integrate heterogeneous remote +processor devices in asymmetric multiprocessing (AMP) configurations. +These processors may run different operating systems, such as Linux and +various real-time operating systems (RTOS). + +For example, the OMAP4 platform features dual Cortex-A9 cores, dual +Cortex-M3 cores, and a C64x+ DSP. In a standard setup, the Cortex-A9 +cores execute Linux in a symmetric multiprocessing (SMP) configuration, +while the M3 cores and DSP run independent instances of an RTOS. + +The remoteproc framework allows various platforms and architectures to +manage remote processors, including operations such as powering on, +loading firmware, and powering off. This framework abstracts hardware +differences, promoting code reuse and minimizing duplication. It also +supports rpmsg virtio devices for remote processors that utilize this +communication method. Consequently, platform-specific remoteproc drivers +need only implement a few low-level handlers, enabling seamless operation +of all rpmsg drivers. (For more details about the virtio-based rpmsg +bus and its drivers, refer to rpmsg documentation.) + +Additionally, the framework allows for the registration of various +virtio devices. Firmware can publish the types of virtio devices it +supports, facilitating their addition to the remoteproc framework. This +flexibility enables the reuse of existing virtio drivers with remote +processor backends at minimal development cost. + +The primary purpose of the remoteproc framework is to download firmware +for remote processors and manage their lifecycle. The framework consists +of several key components: + +- **Character Driver**: Provides userspace access to control the remote + processor. +- **ELF Utility**: Offers functions for handling ELF files and managing + resources requested by the remote processor. +- **Remoteproc Core**: Manages firmware downloads and recovery actions + in case of a remote processor crash. +- **Coredump**: Provides facilities for coredumping and tracing from + the remote processor in the event of a crash. +- **Userspace Interaction**: Uses sysfs and debugfs to manage the + lifecycle and status of the remote processor. +- **Virtio Support**: Facilitates interaction with the virtio and + rpmsg bus. + +From here on, references to "framework" denote the remoteproc +framework, and "driver" refers to the remoteproc driver that utilizes +the framework for managing remote processors. + +2. Remoteproc framework Responsibilities +======================================== + +The framework begins by gathering information about the firmware file +to be downloaded through the request_firmware function. It supports +the ELF format and parses the firmware image to identify the physical +addresses that need to be populated from the corresponding ELF sections. +The framework also requires knowledge of the logical or I/O-mapped +addresses in the application processor. Once this information is +obtained from the driver, the framework transfers the data to the +specified addresses and starts the remote, along with +any devices physically or logically connected to it. + +Dependent devices, referred to as `subdevices` within the framework, +are also managed post-registration by their respective drivers. +Subdevices can register themselves using `rproc_(add/remove)_subdev`. +Non-remoteproc drivers can use subdevices as a way to logically connect +to remote and get lifecycle notifications of the remote. + +The framework oversees the lifecycle of the remote and +provides the `rproc_report_crash` function, which the driver invokes +upon receiving a crash notification from the remote. The +notification method can differ based on the design of the remote +processor and its communication with the application processor. For +instance, if the remote is a DSP equipped with a watchdog, +unresponsive behavior triggers the watchdog, generating an interrupt +that routes to the application processor, allowing it to call +`rproc_report_crash` in the driver's interrupt context. + +During crash handling, the framework performs the following actions: + +a. Sends a request to stop the remote and any connected or + dependent subdevices. +b. Generates a coredump, dumping all `resources` requested by the + remote alongside relevant debugging information. Resources are + explained below. +c. Reloads the firmware and restarts the remote. + +If the `RPROC_FEAT_ATTACH_ON_RECOVERY` flag is set, the detach and +attach callbacks of the driver are invoked without reloading the +firmware. This is useful when the remote requires no +assistance for recovery, or when the application processor can restart +independently. After recovery, the application processor can reattach +to the remote. + +The remote can request resources from the framework, which +allocates a ".resource_table" section. During the ELF parsing phase, +the framework identifies this section and calls the appropriate +handler to allocate the requested resources. + +Resource management within the framework can accommodate any type of +`fw_resource_type`. + +.. code-block:: c + + enum fw_resource_type { + RSC_CARVEOUT = 0, + RSC_DEVMEM = 1, + RSC_TRACE = 2, + RSC_VDEV = 3, + RSC_LAST = 4, + RSC_VENDOR_START = 128, + RSC_VENDOR_END = 512, + }; + + struct resource_table { + u32 ver; + u32 num; + u32 reserved[2]; + u32 offset[]; + } __packed; + + struct fw_rsc_hdr { + u32 type; + u8 data[]; + } __packed; + +For example, if the remote requests both `RSC_TRACE` and +`RSC_CARVEOUT` for memory allocation, the ELF firmware can be structured +as follows: + +.. code-block:: c + + #define MAX_SHARED_RESOURCE 2 + #define LOG_BUF_SIZE 1000 + #define CARVEOUT_DUMP_PA 0x12345678 + #define CARVEOUT_DUMP_SIZE 2000 + + struct shared_resource_table { + u32 ver; + u32 num; + u32 reserved[2]; + u32 offset[MAX_SHARED_RESOURCE]; + struct fw_rsc_trace log_trace; + struct fw_rsc_carveout dump_carveout; + }; + + volatile struct shared_resource_table table = { + .ver = 1, + .num = 2, + .reserved = {0, 0}, + .offset = { + offsetof(struct resource_table, log_trace), + offsetof(struct resource_table, dump_carveout), + }, + .log_trace = { + RSC_TRACE, + (u32)log_buf, LOG_BUF_SIZE, 0, "log_trace", + }, + .dump_carveout = { + RSC_CARVEOUT, + (u32)FW_RSC_ADDR_ANY, CARVEOUT_PA, 0, "carveout_dump", + }, + }; + +The framework creates a sysfs file when it encounters the `RSC_TRACE` +type to expose log information to userspace. Other resource types are +handled accordingly. In the example above, `CARVEOUT_DUMP_SIZE` bytes +of DMA memory will be allocated starting from `CARVEOUT_DUMP_PA`. + + +3. Remoteproc driver responsibilities +===================================== + +The driver must provide the following information to the core: + +a. Translate device addresses (physical addresses) found in the ELF + firmware to virtual addresses in Linux using the `da_to_va` + callback. This allows the framework to copy ELF firmware from the + filesystem to the addresses expected by the remote since + the framework cannot directly access those physical addresses. +b. Prepare/unprepare the remote prior to firmware loading, + which may involve allocating carveout and reserved memory regions. +c. Implement methods for starting and stopping the remote, + whether by setting registers or sending explicit interrupts, + depending on the hardware design. +d. Provide attach and detach callbacks to start the remote + without loading the firmware. This is beneficial when the remote + processor is already loaded and running. +e. Implement a load callback for firmware loading, typically using + the ELF loader provided by the framework; currently, only ELF + format is supported. +f. Invoke the framework's crash handler API upon detecting a remote + crash. + +Drivers must fill the `rproc_ops` structure and call `rproc_alloc` +to register themselves with the framework. + +.. code-block:: c + + struct rproc_ops { + int (*prepare)(struct rproc *rproc); + int (*unprepare)(struct rproc *rproc); + int (*start)(struct rproc *rproc); + int (*stop)(struct rproc *rproc); + int (*attach)(struct rproc *rproc); + int (*detach)(struct rproc *rproc); + void * (*da_to_va)(struct rproc *rproc, u64 da, size_t len, + bool *is_iomem); + int (*parse_fw)(struct rproc *rproc, const struct firmware *fw); + int (*handle_rsc)(struct rproc *rproc, u32 rsc_type, + void *rsc, int offset, int avail); + int (*load)(struct rproc *rproc, const struct firmware *fw); + //snip + }; + + +4. Virtio and Remoteproc ======================== -These callbacks should be provided by platform-specific remoteproc -drivers:: - - /** - * struct rproc_ops - platform-specific device handlers - * @start: power on the device and boot it - * @stop: power off the device - * @kick: kick a virtqueue (virtqueue id given as a parameter) - */ - struct rproc_ops { - int (*start)(struct rproc *rproc); - int (*stop)(struct rproc *rproc); - void (*kick)(struct rproc *rproc, int vqid); - }; - -Every remoteproc implementation should at least provide the ->start and ->stop -handlers. If rpmsg/virtio functionality is also desired, then the ->kick handler -should be provided as well. - -The ->start() handler takes an rproc handle and should then power on the -device and boot it (use rproc->priv to access platform-specific private data). -The boot address, in case needed, can be found in rproc->bootaddr (remoteproc -core puts there the ELF entry point). -On success, 0 should be returned, and on failure, an appropriate error code. - -The ->stop() handler takes an rproc handle and powers the device down. -On success, 0 is returned, and on failure, an appropriate error code. - -The ->kick() handler takes an rproc handle, and an index of a virtqueue -where new message was placed in. Implementations should interrupt the remote -processor and let it know it has pending messages. Notifying remote processors -the exact virtqueue index to look in is optional: it is easy (and not -too expensive) to go through the existing virtqueues and look for new buffers -in the used rings. - -Binary Firmware Structure -========================= - -At this point remoteproc supports ELF32 and ELF64 firmware binaries. However, -it is quite expected that other platforms/devices which we'd want to -support with this framework will be based on different binary formats. - -When those use cases show up, we will have to decouple the binary format -from the framework core, so we can support several binary formats without -duplicating common code. - -When the firmware is parsed, its various segments are loaded to memory -according to the specified device address (might be a physical address -if the remote processor is accessing memory directly). - -In addition to the standard ELF segments, most remote processors would -also include a special section which we call "the resource table". - -The resource table contains system resources that the remote processor -requires before it should be powered on, such as allocation of physically -contiguous memory, or iommu mapping of certain on-chip peripherals. -Remotecore will only power up the device after all the resource table's -requirement are met. - -In addition to system resources, the resource table may also contain -resource entries that publish the existence of supported features -or configurations by the remote processor, such as trace buffers and -supported virtio devices (and their configurations). - -The resource table begins with this header:: - - /** - * struct resource_table - firmware resource table header - * @ver: version number - * @num: number of resource entries - * @reserved: reserved (must be zero) - * @offset: array of offsets pointing at the various resource entries - * - * The header of the resource table, as expressed by this structure, - * contains a version number (should we need to change this format in the - * future), the number of available resource entries, and their offsets - * in the table. - */ - struct resource_table { - u32 ver; - u32 num; - u32 reserved[2]; - u32 offset[0]; - } __packed; - -Immediately following this header are the resource entries themselves, -each of which begins with the following resource entry header:: - - /** - * struct fw_rsc_hdr - firmware resource entry header - * @type: resource type - * @data: resource data - * - * Every resource entry begins with a 'struct fw_rsc_hdr' header providing - * its @type. The content of the entry itself will immediately follow - * this header, and it should be parsed according to the resource type. - */ - struct fw_rsc_hdr { - u32 type; - u8 data[0]; - } __packed; - -Some resources entries are mere announcements, where the host is informed -of specific remoteproc configuration. Other entries require the host to -do something (e.g. allocate a system resource). Sometimes a negotiation -is expected, where the firmware requests a resource, and once allocated, -the host should provide back its details (e.g. address of an allocated -memory region). - -Here are the various resource types that are currently supported:: - - /** - * enum fw_resource_type - types of resource entries - * - * @RSC_CARVEOUT: request for allocation of a physically contiguous - * memory region. - * @RSC_DEVMEM: request to iommu_map a memory-based peripheral. - * @RSC_TRACE: announces the availability of a trace buffer into which - * the remote processor will be writing logs. - * @RSC_VDEV: declare support for a virtio device, and serve as its - * virtio header. - * @RSC_LAST: just keep this one at the end - * @RSC_VENDOR_START: start of the vendor specific resource types range - * @RSC_VENDOR_END: end of the vendor specific resource types range - * - * Please note that these values are used as indices to the rproc_handle_rsc - * lookup table, so please keep them sane. Moreover, @RSC_LAST is used to - * check the validity of an index before the lookup table is accessed, so - * please update it as needed. - */ - enum fw_resource_type { - RSC_CARVEOUT = 0, - RSC_DEVMEM = 1, - RSC_TRACE = 2, - RSC_VDEV = 3, - RSC_LAST = 4, - RSC_VENDOR_START = 128, - RSC_VENDOR_END = 512, - }; - -For more details regarding a specific resource type, please see its -dedicated structure in include/linux/remoteproc.h. - -We also expect that platform-specific resource entries will show up -at some point. When that happens, we could easily add a new RSC_PLATFORM -type, and hand those resources to the platform-specific rproc driver to handle. - -Virtio and remoteproc -===================== - -The firmware should provide remoteproc information about virtio devices -that it supports, and their configurations: a RSC_VDEV resource entry -should specify the virtio device id (as in virtio_ids.h), virtio features, -virtio config space, vrings information, etc. - -When a new remote processor is registered, the remoteproc framework -will look for its resource table and will register the virtio devices -it supports. A firmware may support any number of virtio devices, and -of any type (a single remote processor can also easily support several -rpmsg virtio devices this way, if desired). - -Of course, RSC_VDEV resource entries are only good enough for static -allocation of virtio devices. Dynamic allocations will also be made possible -using the rpmsg bus (similar to how we already do dynamic allocations of -rpmsg channels; read more about it in rpmsg.txt). +The firmware must provide remoteproc with information regarding the +virtio devices it supports and their configurations: an `RSC_VDEV` +resource entry should detail the virtio device ID (as defined in +`virtio_ids.h`), virtio features, virtio config space, vrings +information, etc. + +Upon registration of a new remote, the remoteproc framework +searches for its resource table and registers the supported virtio +devices. A firmware may support multiple virtio devices, of various +types (a single remote can support multiple rpmsg virtio +devices if required). + +Moreover, `RSC_VDEV` resource entries suffice for static allocation +of virtio devices. Dynamic allocations will also be supported using +the rpmsg bus, akin to the handling of dynamic allocations for rpmsg +channels. For more information, refer to rpmsg documentation.
Added following changes: 1. Components provided by remoteproc framework. 2. Remoteproc driver responsibilities. 3. Remoteproc framework responsibilities. 4. Better explanation of how to ask for resources from the framework by the remote processor. Signed-off-by: anish kumar <yesanishhere@gmail.com> --- .../driver-api/remoteproc/remoteproc.rst | 596 +++++++----------- 1 file changed, 243 insertions(+), 353 deletions(-)