| Message ID | 20230807140247.956255-1-nm@ti.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | remoteproc: core: Honor device tree /alias entries when assigning IDs | expand |
On 8/7/23 09:02, Nishanth Menon wrote: > Assign remoteproc device IDs based on device tree /aliases entries if > present, falling back to the existing numbering scheme if there is no > /aliases entry (which includes when the system isn't booted using DT) > or a numbering conflict. If the alias node is not present, the driver > behaves as before. > > Cc: Robert Nelson<robertcnelson@gmail.com> > Reported-by: Kevin Cahalan<kevinacahalan@gmail.com> > Signed-off-by: Nishanth Menon<nm@ti.com> > --- Reviewed-by: Hari Nagalla <hnagalla@ti.com>
Hi Nishanth, On Mon, Aug 07, 2023 at 09:02:47AM -0500, Nishanth Menon wrote: > On many platforms, such as Beaglebone-AI64 with many remote > processors, firmware configurations provided by the distributions can > vary substantially depending on the distribution build's functionality > and the specific remote cores enabled in that variant. Ensuring > consistent udev rules mapping remoteproc nodes to constant remote > proc device indices across distributions (yocto, ubuntu, debian and > it's variants, ...) on a board basis can be challenging due to the > various functions of these distributions. Varied device node paths > create challenges for applications that operate on remote processors, > especially in minimal embedded systems(initrd like) that may not > have udev-like capabilities and rely on a more straightforward bare > filesystem. This challenge is similar to that faced by I2C, RTC or the > GPIO subsystems. > I'm puzzled by this patch. I can see how using an alias can help in boards with various HW configuration. That said, and as written above, FW files for remote processors can vary based on the build's functionality. As such "remoteproc3" will reference the same HW device on all distributions but the functionality enacted by the FW may be different. As such I don't see how an alias can help here. Can you provide a concrete example that highlights the benefits? Thanks, Mathieu > Assign remoteproc device IDs based on device tree /aliases entries if > present, falling back to the existing numbering scheme if there is no > /aliases entry (which includes when the system isn't booted using DT) > or a numbering conflict. If the alias node is not present, the driver > behaves as before. > > Cc: Robert Nelson <robertcnelson@gmail.com> > Reported-by: Kevin Cahalan <kevinacahalan@gmail.com> > Signed-off-by: Nishanth Menon <nm@ti.com> > --- > Test log: Beaglebone-AI64 > https://gist.github.com/nmenon/365cf80d6c0685dd9be7c2cb18d7c937 (along > with a test change to force the sequence of initialization) > > The report occurred on Beagle discord channel - so I am not > sure how to share the logs of the report. > > drivers/remoteproc/remoteproc_core.c | 22 +++++++++++++++++++++- > 1 file changed, 21 insertions(+), 1 deletion(-) > > diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c > index 695cce218e8c..a12f3d37b8de 100644 > --- a/drivers/remoteproc/remoteproc_core.c > +++ b/drivers/remoteproc/remoteproc_core.c > @@ -19,6 +19,7 @@ > #include <linux/delay.h> > #include <linux/kernel.h> > #include <linux/module.h> > +#include <linux/of.h> > #include <linux/device.h> > #include <linux/panic_notifier.h> > #include <linux/slab.h> > @@ -2417,6 +2418,25 @@ static int rproc_alloc_ops(struct rproc *rproc, const struct rproc_ops *ops) > return 0; > } > > +static int rproc_device_get_id(struct device *dev) > +{ > + int of_id = -1, id = -1; > + > + if (dev->of_node) > + of_id = of_alias_get_id(dev->of_node, "remoteproc"); > + > + if (of_id >= 0) { > + id = ida_simple_get(&rproc_dev_index, of_id, of_id + 1, GFP_KERNEL); > + if (id < 0) > + dev_warn(dev, "/aliases ID %d not available\n", of_id); > + } > + > + if (id < 0) > + id = ida_alloc(&rproc_dev_index, GFP_KERNEL); > + > + return id; > +} > + > /** > * rproc_alloc() - allocate a remote processor handle > * @dev: the underlying device > @@ -2476,7 +2496,7 @@ struct rproc *rproc_alloc(struct device *dev, const char *name, > goto put_device; > > /* Assign a unique device index and name */ > - rproc->index = ida_alloc(&rproc_dev_index, GFP_KERNEL); > + rproc->index = rproc_device_get_id(dev); > if (rproc->index < 0) { > dev_err(dev, "ida_alloc failed: %d\n", rproc->index); > goto put_device; > -- > 2.40.0 >
On 13:25-20230822, Mathieu Poirier wrote: > Hi Nishanth, > > On Mon, Aug 07, 2023 at 09:02:47AM -0500, Nishanth Menon wrote: > > On many platforms, such as Beaglebone-AI64 with many remote > > processors, firmware configurations provided by the distributions can > > vary substantially depending on the distribution build's functionality > > and the specific remote cores enabled in that variant. Ensuring > > consistent udev rules mapping remoteproc nodes to constant remote > > proc device indices across distributions (yocto, ubuntu, debian and > > it's variants, ...) on a board basis can be challenging due to the > > various functions of these distributions. Varied device node paths > > create challenges for applications that operate on remote processors, > > especially in minimal embedded systems(initrd like) that may not > > have udev-like capabilities and rely on a more straightforward bare > > filesystem. This challenge is similar to that faced by I2C, RTC or the > > GPIO subsystems. > > > > I'm puzzled by this patch. I can see how using an alias can help in boards with > various HW configuration. That said, and as written above, FW files for remote > processors can vary based on the build's functionality. As such "remoteproc3" > will reference the same HW device on all distributions but the functionality > enacted by the FW may be different. As such I don't see how an alias can help > here. Can you provide a concrete example that highlights the benefits? Correct - *if* remoteproc3 is the constant node reference. To take a trivial example: We ran into this issue with: https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28 remoteproc18 apparently changed numbering in a different build. If remoteproc18 remained the same between different distro builds that would have probably kept the userspace constant. but it does'nt. it dependent purely on probe order, which does'nt let userspace remain consistent. Same reason and motivation to do the following: https://git.beagleboard.org/beagleboard/repos-arm64/-/blob/main/bb-customizations/suite/bookworm/debian/86-remoteproc-noroot.rules in one technique to do it - but that only works if all the distros follow the same udev rules - and there is no reasonable way to enforce that across distributions.
On 8/22/23 3:12 PM, Nishanth Menon wrote: > On 13:25-20230822, Mathieu Poirier wrote: >> Hi Nishanth, >> >> On Mon, Aug 07, 2023 at 09:02:47AM -0500, Nishanth Menon wrote: >>> On many platforms, such as Beaglebone-AI64 with many remote >>> processors, firmware configurations provided by the distributions can >>> vary substantially depending on the distribution build's functionality >>> and the specific remote cores enabled in that variant. Ensuring >>> consistent udev rules mapping remoteproc nodes to constant remote >>> proc device indices across distributions (yocto, ubuntu, debian and >>> it's variants, ...) on a board basis can be challenging due to the >>> various functions of these distributions. Varied device node paths >>> create challenges for applications that operate on remote processors, >>> especially in minimal embedded systems(initrd like) that may not >>> have udev-like capabilities and rely on a more straightforward bare >>> filesystem. This challenge is similar to that faced by I2C, RTC or the >>> GPIO subsystems. >>> >> >> I'm puzzled by this patch. I can see how using an alias can help in boards with >> various HW configuration. That said, and as written above, FW files for remote >> processors can vary based on the build's functionality. As such "remoteproc3" >> will reference the same HW device on all distributions but the functionality >> enacted by the FW may be different. As such I don't see how an alias can help >> here. Can you provide a concrete example that highlights the benefits? > > Correct - *if* remoteproc3 is the constant node reference. > > To take a trivial example: We ran into this issue with: > https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28 > > remoteproc18 apparently changed numbering in a different build. > > If remoteproc18 remained the same between different distro builds that > would have probably kept the userspace constant. but it does'nt. it > dependent purely on probe order, which does'nt let userspace remain > consistent. > > Same reason and motivation to do the following: > https://git.beagleboard.org/beagleboard/repos-arm64/-/blob/main/bb-customizations/suite/bookworm/debian/86-remoteproc-noroot.rules > in one technique to do it - but that only works if all the distros > follow the same udev rules - and there is no reasonable way to enforce > that across distributions. > Enforcing distros to behave the same isn't the job of Device Tree, udev rules seems like a reasonable place. Anyone dealing with Linux should know they should not rely on kernel provided device names/numbers (like with disks, network interfaces, etc.). If you want to have a path that will always work you could use: /sys/devices/platform/bus@f4000/bus@f4000\:r5fss@78400000/78400000.r5f/remoteproc/ for the same. I don't like that it makes an ABI out of node names, but better than putting any more Linux configuration in DT IMHO. Andrew
On 16:45-20230822, Andrew Davis wrote: > On 8/22/23 3:12 PM, Nishanth Menon wrote: > > On 13:25-20230822, Mathieu Poirier wrote: > > > Hi Nishanth, > > > > > > On Mon, Aug 07, 2023 at 09:02:47AM -0500, Nishanth Menon wrote: > > > > On many platforms, such as Beaglebone-AI64 with many remote > > > > processors, firmware configurations provided by the distributions can > > > > vary substantially depending on the distribution build's functionality > > > > and the specific remote cores enabled in that variant. Ensuring > > > > consistent udev rules mapping remoteproc nodes to constant remote > > > > proc device indices across distributions (yocto, ubuntu, debian and > > > > it's variants, ...) on a board basis can be challenging due to the > > > > various functions of these distributions. Varied device node paths > > > > create challenges for applications that operate on remote processors, > > > > especially in minimal embedded systems(initrd like) that may not > > > > have udev-like capabilities and rely on a more straightforward bare > > > > filesystem. This challenge is similar to that faced by I2C, RTC or the > > > > GPIO subsystems. > > > > > > > > > > I'm puzzled by this patch. I can see how using an alias can help in boards with > > > various HW configuration. That said, and as written above, FW files for remote > > > processors can vary based on the build's functionality. As such "remoteproc3" > > > will reference the same HW device on all distributions but the functionality > > > enacted by the FW may be different. As such I don't see how an alias can help > > > here. Can you provide a concrete example that highlights the benefits? > > > > Correct - *if* remoteproc3 is the constant node reference. > > > > To take a trivial example: We ran into this issue with: > > https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28 > > > > remoteproc18 apparently changed numbering in a different build. > > > > If remoteproc18 remained the same between different distro builds that > > would have probably kept the userspace constant. but it does'nt. it > > dependent purely on probe order, which does'nt let userspace remain > > consistent. > > > > Same reason and motivation to do the following: > > https://git.beagleboard.org/beagleboard/repos-arm64/-/blob/main/bb-customizations/suite/bookworm/debian/86-remoteproc-noroot.rules > > in one technique to do it - but that only works if all the distros > > follow the same udev rules - and there is no reasonable way to enforce > > that across distributions. > > > > Enforcing distros to behave the same isn't the job of Device Tree, udev > rules seems like a reasonable place. Anyone dealing with Linux should know > they should not rely on kernel provided device names/numbers > (like with disks, network interfaces, etc.). > > If you want to have a path that will always work you could use: > > /sys/devices/platform/bus@f4000/bus@f4000\:r5fss@78400000/78400000.r5f/remoteproc/ > > for the same. I don't like that it makes an ABI out of node names, > but better than putting any more Linux configuration in DT IMHO. aliases are there for real reasons. So lets not confuse the two. End of the day being able to help userspace is what it does.
On Tue, Aug 22, 2023 at 03:12:05PM -0500, Nishanth Menon wrote: > On 13:25-20230822, Mathieu Poirier wrote: > > Hi Nishanth, > > > > On Mon, Aug 07, 2023 at 09:02:47AM -0500, Nishanth Menon wrote: > > > On many platforms, such as Beaglebone-AI64 with many remote > > > processors, firmware configurations provided by the distributions can > > > vary substantially depending on the distribution build's functionality > > > and the specific remote cores enabled in that variant. Ensuring > > > consistent udev rules mapping remoteproc nodes to constant remote > > > proc device indices across distributions (yocto, ubuntu, debian and > > > it's variants, ...) on a board basis can be challenging due to the > > > various functions of these distributions. Varied device node paths > > > create challenges for applications that operate on remote processors, > > > especially in minimal embedded systems(initrd like) that may not > > > have udev-like capabilities and rely on a more straightforward bare > > > filesystem. This challenge is similar to that faced by I2C, RTC or the > > > GPIO subsystems. > > > > > > > I'm puzzled by this patch. I can see how using an alias can help in boards with > > various HW configuration. That said, and as written above, FW files for remote > > processors can vary based on the build's functionality. As such "remoteproc3" > > will reference the same HW device on all distributions but the functionality > > enacted by the FW may be different. As such I don't see how an alias can help > > here. Can you provide a concrete example that highlights the benefits? > > Correct - *if* remoteproc3 is the constant node reference. > > To take a trivial example: We ran into this issue with: > https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28 > > remoteproc18 apparently changed numbering in a different build. > We are going around in circles. In the above link using an alias will guarantee that "remoteproc18" is available but won't guarantee the functionality enacted by the FW loaded in that remote processor, which is distro dependent. > If remoteproc18 remained the same between different distro builds that > would have probably kept the userspace constant. but it does'nt. it > dependent purely on probe order, which does'nt let userspace remain > consistent. > > Same reason and motivation to do the following: > https://git.beagleboard.org/beagleboard/repos-arm64/-/blob/main/bb-customizations/suite/bookworm/debian/86-remoteproc-noroot.rules > in one technique to do it - but that only works if all the distros > follow the same udev rules - and there is no reasonable way to enforce > that across distributions. > > -- > Regards, > Nishanth Menon > Key (0xDDB5849D1736249D) / Fingerprint: F8A2 8693 54EB 8232 17A3 1A34 DDB5 849D 1736 249D
On 09:23-20230823, Mathieu Poirier wrote: > On Tue, Aug 22, 2023 at 03:12:05PM -0500, Nishanth Menon wrote: > > On 13:25-20230822, Mathieu Poirier wrote: > > > Hi Nishanth, > > > > > > On Mon, Aug 07, 2023 at 09:02:47AM -0500, Nishanth Menon wrote: > > > > On many platforms, such as Beaglebone-AI64 with many remote > > > > processors, firmware configurations provided by the distributions can > > > > vary substantially depending on the distribution build's functionality > > > > and the specific remote cores enabled in that variant. Ensuring > > > > consistent udev rules mapping remoteproc nodes to constant remote > > > > proc device indices across distributions (yocto, ubuntu, debian and > > > > it's variants, ...) on a board basis can be challenging due to the > > > > various functions of these distributions. Varied device node paths > > > > create challenges for applications that operate on remote processors, > > > > especially in minimal embedded systems(initrd like) that may not > > > > have udev-like capabilities and rely on a more straightforward bare > > > > filesystem. This challenge is similar to that faced by I2C, RTC or the > > > > GPIO subsystems. > > > > > > > > > > I'm puzzled by this patch. I can see how using an alias can help in boards with > > > various HW configuration. That said, and as written above, FW files for remote > > > processors can vary based on the build's functionality. As such "remoteproc3" > > > will reference the same HW device on all distributions but the functionality > > > enacted by the FW may be different. As such I don't see how an alias can help > > > here. Can you provide a concrete example that highlights the benefits? > > > > Correct - *if* remoteproc3 is the constant node reference. > > > > To take a trivial example: We ran into this issue with: > > https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28 > > > > remoteproc18 apparently changed numbering in a different build. > > > > We are going around in circles. In the above link using an alias will > guarantee that "remoteproc18" is available but won't guarantee the > functionality enacted by the FW loaded in that remote processor, which is distro > dependent. Apologies, but I am trying to comprehend the relationship and probably am failing to see the linkage. Let me try: If I understand you correctly, you are concerned that distros do not have a mechanism to provide consistent firmware to the correct remote proc for a specific functionality.. if so, distro loads / provides the requisite firmware. How the package distribution scheme works to distribute the firmware and versioning provided varies - One typical pattern has been to use linux-firmware repo[1] (at least in other domains - say GPU, wlink or the likes) and provide package distribution. The other pattern could be build and deploy based on tag (this would be no different from any other package deployment). On the other hand, If we are looking at the fact that there can be different types of firmware that could be loaded to a remoteproc providing different functionality - that is correct, and at least in case of TI processors very valid - something like openAMP endpoint solutions probably help? Let me know if I am off-track here.. [1] https://git.ti.com/cgit/processor-firmware/ti-linux-firmware/tree/ti-ipc?h=ti-linux-firmware
On Wed, Aug 23, 2023 at 10:51:33AM -0500, Nishanth Menon wrote: > On 09:23-20230823, Mathieu Poirier wrote: > > On Tue, Aug 22, 2023 at 03:12:05PM -0500, Nishanth Menon wrote: > > > On 13:25-20230822, Mathieu Poirier wrote: > > > > Hi Nishanth, > > > > > > > > On Mon, Aug 07, 2023 at 09:02:47AM -0500, Nishanth Menon wrote: > > > > > On many platforms, such as Beaglebone-AI64 with many remote > > > > > processors, firmware configurations provided by the distributions can > > > > > vary substantially depending on the distribution build's functionality > > > > > and the specific remote cores enabled in that variant. Ensuring > > > > > consistent udev rules mapping remoteproc nodes to constant remote > > > > > proc device indices across distributions (yocto, ubuntu, debian and > > > > > it's variants, ...) on a board basis can be challenging due to the > > > > > various functions of these distributions. Varied device node paths > > > > > create challenges for applications that operate on remote processors, > > > > > especially in minimal embedded systems(initrd like) that may not > > > > > have udev-like capabilities and rely on a more straightforward bare > > > > > filesystem. This challenge is similar to that faced by I2C, RTC or the > > > > > GPIO subsystems. > > > > > > > > > > > > > I'm puzzled by this patch. I can see how using an alias can help in boards with > > > > various HW configuration. That said, and as written above, FW files for remote > > > > processors can vary based on the build's functionality. As such "remoteproc3" > > > > will reference the same HW device on all distributions but the functionality > > > > enacted by the FW may be different. As such I don't see how an alias can help > > > > here. Can you provide a concrete example that highlights the benefits? > > > > > > Correct - *if* remoteproc3 is the constant node reference. > > > > > > To take a trivial example: We ran into this issue with: > > > https://github.com/kaofishy/bbai64_cortex-r5_example/blob/main/Makefile#L28 > > > > > > remoteproc18 apparently changed numbering in a different build. > > > > > > > We are going around in circles. In the above link using an alias will > > guarantee that "remoteproc18" is available but won't guarantee the > > functionality enacted by the FW loaded in that remote processor, which is distro > > dependent. > > Apologies, but I am trying to comprehend the relationship and probably > am failing to see the linkage. Let me try: > > If I understand you correctly, you are concerned that distros do not > have a mechanism to provide consistent firmware to the correct remote > proc for a specific functionality.. > The point is that aliases will guarantee a naming convention for remote processors but won't guarantee their functionality. Sure, we can add aliases but it won't solve all your problems. > if so, distro loads / provides the requisite firmware. How > the package distribution scheme works to distribute the firmware > and versioning provided varies - One typical pattern has been to use > linux-firmware repo[1] (at least in other domains - say GPU, wlink or > the likes) and provide package distribution. The other pattern could > be build and deploy based on tag (this would be no different from any > other package deployment). > > On the other hand, If we are looking at the fact that there can be > different types of firmware that could be loaded to a remoteproc > providing different functionality - that is correct, and at least in > case of TI processors very valid That is exactly what I am referring to. >- something like openAMP endpoint > solutions probably help? I am not familiar with openAmP endpoints but certainly willing to consider it as an option. > > Let me know if I am off-track here.. > You are on track. > [1] https://git.ti.com/cgit/processor-firmware/ti-linux-firmware/tree/ti-ipc?h=ti-linux-firmware > -- > Regards, > Nishanth Menon > Key (0xDDB5849D1736249D) / Fingerprint: F8A2 8693 54EB 8232 17A3 1A34 DDB5 849D 1736 249D
diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c index 695cce218e8c..a12f3d37b8de 100644 --- a/drivers/remoteproc/remoteproc_core.c +++ b/drivers/remoteproc/remoteproc_core.c @@ -19,6 +19,7 @@ #include <linux/delay.h> #include <linux/kernel.h> #include <linux/module.h> +#include <linux/of.h> #include <linux/device.h> #include <linux/panic_notifier.h> #include <linux/slab.h> @@ -2417,6 +2418,25 @@ static int rproc_alloc_ops(struct rproc *rproc, const struct rproc_ops *ops) return 0; } +static int rproc_device_get_id(struct device *dev) +{ + int of_id = -1, id = -1; + + if (dev->of_node) + of_id = of_alias_get_id(dev->of_node, "remoteproc"); + + if (of_id >= 0) { + id = ida_simple_get(&rproc_dev_index, of_id, of_id + 1, GFP_KERNEL); + if (id < 0) + dev_warn(dev, "/aliases ID %d not available\n", of_id); + } + + if (id < 0) + id = ida_alloc(&rproc_dev_index, GFP_KERNEL); + + return id; +} + /** * rproc_alloc() - allocate a remote processor handle * @dev: the underlying device @@ -2476,7 +2496,7 @@ struct rproc *rproc_alloc(struct device *dev, const char *name, goto put_device; /* Assign a unique device index and name */ - rproc->index = ida_alloc(&rproc_dev_index, GFP_KERNEL); + rproc->index = rproc_device_get_id(dev); if (rproc->index < 0) { dev_err(dev, "ida_alloc failed: %d\n", rproc->index); goto put_device;
On many platforms, such as Beaglebone-AI64 with many remote processors, firmware configurations provided by the distributions can vary substantially depending on the distribution build's functionality and the specific remote cores enabled in that variant. Ensuring consistent udev rules mapping remoteproc nodes to constant remote proc device indices across distributions (yocto, ubuntu, debian and it's variants, ...) on a board basis can be challenging due to the various functions of these distributions. Varied device node paths create challenges for applications that operate on remote processors, especially in minimal embedded systems(initrd like) that may not have udev-like capabilities and rely on a more straightforward bare filesystem. This challenge is similar to that faced by I2C, RTC or the GPIO subsystems. Assign remoteproc device IDs based on device tree /aliases entries if present, falling back to the existing numbering scheme if there is no /aliases entry (which includes when the system isn't booted using DT) or a numbering conflict. If the alias node is not present, the driver behaves as before. Cc: Robert Nelson <robertcnelson@gmail.com> Reported-by: Kevin Cahalan <kevinacahalan@gmail.com> Signed-off-by: Nishanth Menon <nm@ti.com> --- Test log: Beaglebone-AI64 https://gist.github.com/nmenon/365cf80d6c0685dd9be7c2cb18d7c937 (along with a test change to force the sequence of initialization) The report occurred on Beagle discord channel - so I am not sure how to share the logs of the report. drivers/remoteproc/remoteproc_core.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-)