[RFC,v0,7/8] Input: ims-pcu: use firmware_stat instead of completion

Commit Message

Daniel Wagner July 28, 2016, 7:55 a.m. UTC

From: Daniel Wagner <[e-mail]>

Loading firmware is an operation many drivers implement in various ways
around the completion API. And most of them do it almost in the same
way. Let's reuse the firmware_stat API which is used also by the
firmware_class loader. Apart of streamlining the firmware loading states
we also document it slightly better.

Signed-off-by: Daniel Wagner <[e-mail]>
---
 drivers/input/misc/ims-pcu.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

Comments

Dmitry Torokhov July 28, 2016, 6:33 p.m. UTC | #1

On Thu, Jul 28, 2016 at 09:55:11AM +0200, Daniel Wagner wrote:
> From: Daniel Wagner <[e-mail]>
> 
> Loading firmware is an operation many drivers implement in various ways
> around the completion API. And most of them do it almost in the same
> way. Let's reuse the firmware_stat API which is used also by the
> firmware_class loader. Apart of streamlining the firmware loading states
> we also document it slightly better.
> 
> Signed-off-by: Daniel Wagner <[e-mail]>
> ---
>  drivers/input/misc/ims-pcu.c | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/input/misc/ims-pcu.c b/drivers/input/misc/ims-pcu.c
> index 9c0ea36..cda1fbf 100644
> --- a/drivers/input/misc/ims-pcu.c
> +++ b/drivers/input/misc/ims-pcu.c
> @@ -109,7 +109,7 @@ struct ims_pcu {
>  
>  	u32 fw_start_addr;
>  	u32 fw_end_addr;
> -	struct completion async_firmware_done;
> +	struct firmware_stat fw_st;
>  
>  	struct ims_pcu_buttons buttons;
>  	struct ims_pcu_gamepad *gamepad;
> @@ -940,7 +940,7 @@ static void ims_pcu_process_async_firmware(const struct firmware *fw,
>  	release_firmware(fw);
>  
>  out:
> -	complete(&pcu->async_firmware_done);
> +	fw_loading_done(pcu->fw_st);

Why does the driver have to do it? If firmware loader manages this, then
it should let waiters know when callback finishes. 

>  }
>  
>  /*********************************************************************
> @@ -1967,7 +1967,7 @@ static int ims_pcu_init_bootloader_mode(struct ims_pcu *pcu)
>  					ims_pcu_process_async_firmware);
>  	if (error) {
>  		/* This error is not fatal, let userspace have another chance */
> -		complete(&pcu->async_firmware_done);
> +		fw_loading_abort(pcu->fw_st);

Why should the driver signal abort if it does not manage completion in
this case?

>  	}
>  
>  	return 0;
> @@ -1976,7 +1976,7 @@ static int ims_pcu_init_bootloader_mode(struct ims_pcu *pcu)
>  static void ims_pcu_destroy_bootloader_mode(struct ims_pcu *pcu)
>  {
>  	/* Make sure our initial firmware request has completed */
> -	wait_for_completion(&pcu->async_firmware_done);
> +	fw_loading_wait(pcu->fw_st);
>  }
>  
>  #define IMS_PCU_APPLICATION_MODE	0
> @@ -2000,7 +2000,7 @@ static int ims_pcu_probe(struct usb_interface *intf,
>  	pcu->bootloader_mode = id->driver_info == IMS_PCU_BOOTLOADER_MODE;
>  	mutex_init(&pcu->cmd_mutex);
>  	init_completion(&pcu->cmd_done);
> -	init_completion(&pcu->async_firmware_done);
> +	firmware_stat_init(&pcu->fw_st);

Do not quite like it... I'd rather asynchronous request give out a
firmware status pointer that could be used later on.

	pcu->fw_st = request_firmware_async(IMS_PCU_FIRMWARE_NAME,
					    pcu,
					    ims_pcu_process_async_firmware);
	if (IS_ERR(pcu->fw_st))
		return PTR_ERR(pcu->fw_st);

	....

	fw_loading_wait(pcu->fw_st);

Thanks.

Bjorn Andersson July 28, 2016, 7:01 p.m. UTC | #2

On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:

> On Thu, Jul 28, 2016 at 09:55:11AM +0200, Daniel Wagner wrote:
> > From: Daniel Wagner <[e-mail]>
> > 
[..]
> 
> Do not quite like it... I'd rather asynchronous request give out a
> firmware status pointer that could be used later on.
> 
> 	pcu->fw_st = request_firmware_async(IMS_PCU_FIRMWARE_NAME,
> 					    pcu,
> 					    ims_pcu_process_async_firmware);
> 	if (IS_ERR(pcu->fw_st))
> 		return PTR_ERR(pcu->fw_st);
> 
> 	....
> 
> 	fw_loading_wait(pcu->fw_st);
> 

In the remoteproc case (patch 6) this would clean up the code, rather
than replacing the completion API 1 to 1. I like it!

Regards,
Bjorn
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Daniel Wagner July 29, 2016, 6:13 a.m. UTC | #3

On 07/28/2016 09:01 PM, Bjorn Andersson wrote:
> On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:
>
>> On Thu, Jul 28, 2016 at 09:55:11AM +0200, Daniel Wagner wrote:
>>> From: Daniel Wagner <[e-mail]>
>>>
> [..]
>>
>> Do not quite like it... I'd rather asynchronous request give out a
>> firmware status pointer that could be used later on.
>>
>> 	pcu->fw_st = request_firmware_async(IMS_PCU_FIRMWARE_NAME,
>> 					    pcu,
>> 					    ims_pcu_process_async_firmware);
>> 	if (IS_ERR(pcu->fw_st))
>> 		return PTR_ERR(pcu->fw_st);
>>
>> 	....
>>
>> 	fw_loading_wait(pcu->fw_st);
>>
>
> In the remoteproc case (patch 6) this would clean up the code, rather
> than replacing the completion API 1 to 1. I like it!

IIRC most drivers do it the same way. So request_firmware_async() indeed 
would be good thing to have. Let me try that.

Thanks for the excellent feedback.

cheers,
daniel
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Arend Van Spriel July 30, 2016, 12:42 p.m. UTC | #4

+ Luis (again) ;-)

On 29-07-16 08:13, Daniel Wagner wrote:
> On 07/28/2016 09:01 PM, Bjorn Andersson wrote:
>> On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:
>>
>>> On Thu, Jul 28, 2016 at 09:55:11AM +0200, Daniel Wagner wrote:
>>>> From: Daniel Wagner <[e-mail]>
>>>>
>> [..]
>>>
>>> Do not quite like it... I'd rather asynchronous request give out a
>>> firmware status pointer that could be used later on.

Excellent. Why not get rid of the callback function as well and have
fw_loading_wait() return result (0 = firmware available, < 0 = fail).
Just to confirm, you are proposing a new API function next to
request_firmware_nowait(), right?

>>>     pcu->fw_st = request_firmware_async(IMS_PCU_FIRMWARE_NAME,
>>> -                       pcu,
>>> -                       ims_pcu_process_async_firmware);
    +			    pcu);
>>>     if (IS_ERR(pcu->fw_st))
>>>         return PTR_ERR(pcu->fw_st);
>>>
>>>     ....
>>>
>>>     err = fw_loading_wait(pcu->fw_st);
	if (err)
		return err;

	fw = fwstat_get_firmware(pcu->fw_st);

Or whatever consistent prefix it is going to be.

>>>
>>
>> In the remoteproc case (patch 6) this would clean up the code, rather
>> than replacing the completion API 1 to 1. I like it!
> 
> IIRC most drivers do it the same way. So request_firmware_async() indeed
> would be good thing to have. Let me try that.

While the idea behind this series is a good one I am wondering about the
need for these drivers to use the asynchronous API. The historic reason
might be to avoid timeout caused by user-mode helper, but that may no
longer apply and these drivers could be better off using
request_firmware_direct().

There have been numerous discussions about the firmware API. Here most
recent one:

http://www.spinics.net/lists/linux-wireless/index.html#152755

Regards,
Arend

> Thanks for the excellent feedback.
> 
> cheers,
> daniel
> -- 
> To unsubscribe from this list: send the line "unsubscribe
> linux-wireless" in
> the body of a message to [e-mail]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Luis Chamberlain July 30, 2016, 4:58 p.m. UTC | #5

On Sat, Jul 30, 2016 at 02:42:41PM +0200, Arend van Spriel wrote:
> + Luis (again) ;-)
> 
> On 29-07-16 08:13, Daniel Wagner wrote:
> > On 07/28/2016 09:01 PM, Bjorn Andersson wrote:
> >> On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:
> >>
> >>> On Thu, Jul 28, 2016 at 09:55:11AM +0200, Daniel Wagner wrote:
> >>>> From: Daniel Wagner <[e-mail]>
> >>>>
> >> [..]
> >>>
> >>> Do not quite like it... I'd rather asynchronous request give out a
> >>> firmware status pointer that could be used later on.
> 
> Excellent. Why not get rid of the callback function as well and have
> fw_loading_wait() return result (0 = firmware available, < 0 = fail).
> Just to confirm, you are proposing a new API function next to
> request_firmware_nowait(), right?

If proposing new firmware_class patches please bounce / Cc me, I've
recently asked for me to be added to MAINTAINERS so I get these
e-mails as I'm working on a new flexible API which would allow us
to extend the firmware API without having to care about the old
stupid usermode helper at all.

> >>>     pcu->fw_st = request_firmware_async(IMS_PCU_FIRMWARE_NAME,
> >>> -                       pcu,
> >>> -                       ims_pcu_process_async_firmware);
>     +			    pcu);
> >>>     if (IS_ERR(pcu->fw_st))
> >>>         return PTR_ERR(pcu->fw_st);
> >>>
> >>>     ....
> >>>
> >>>     err = fw_loading_wait(pcu->fw_st);
> 	if (err)
> 		return err;
> 
> 	fw = fwstat_get_firmware(pcu->fw_st);
> 
> Or whatever consistent prefix it is going to be.
> 
> >>>
> >>
> >> In the remoteproc case (patch 6) this would clean up the code, rather
> >> than replacing the completion API 1 to 1. I like it!
> > 
> > IIRC most drivers do it the same way. So request_firmware_async() indeed
> > would be good thing to have. Let me try that.
> 
> While the idea behind this series is a good one I am wondering about the
> need for these drivers to use the asynchronous API. The historic reason
> might be to avoid timeout caused by user-mode helper, but that may no
> longer apply and these drivers could be better off using
> request_firmware_direct().

BTW I have in my queue for the sysdata API something like firmware_request_direct()
but with async support. The only thing left to do I think is just add the devm
helpers so drivers no longer need to worry about the release of the firmware.

> There have been numerous discussions about the firmware API. Here most
> recent one:
> 
> http://www.spinics.net/lists/linux-wireless/index.html#152755

And more importantly, the sysdata API queue:

https://git.kernel.org/cgit/linux/kernel/git/mcgrof/linux-next.git/log/?h=20160616-sysdata-v2

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Dmitry Torokhov July 31, 2016, 7:17 a.m. UTC | #6

On July 30, 2016 5:42:41 AM PDT, Arend van Spriel <[e-mail]> wrote:
>+ Luis (again) ;-)
>
>On 29-07-16 08:13, Daniel Wagner wrote:
>> On 07/28/2016 09:01 PM, Bjorn Andersson wrote:
>>> On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:
>>>
>>>> On Thu, Jul 28, 2016 at 09:55:11AM +0200, Daniel Wagner wrote:
>>>>> From: Daniel Wagner <[e-mail]>
>>>>>
>>> [..]
>>>>
>>>> Do not quite like it... I'd rather asynchronous request give out a
>>>> firmware status pointer that could be used later on.
>
>Excellent. Why not get rid of the callback function as well and have
>fw_loading_wait() return result (0 = firmware available, < 0 = fail).
>Just to confirm, you are proposing a new API function next to
>request_firmware_nowait(), right?

Yes, that would be a new API call. Maybe we could replace old API with the new at some point.


>>>>     pcu->fw_st = request_firmware_async(IMS_PCU_FIRMWARE_NAME,
>>>> -                       pcu,
>>>> -                       ims_pcu_process_async_firmware);
>    +			    pcu);
>>>>     if (IS_ERR(pcu->fw_st))
>>>>         return PTR_ERR(pcu->fw_st);
>>>>
>>>>     ....
>>>>
>>>>     err = fw_loading_wait(pcu->fw_st);
>	if (err)
>		return err;
>
>	fw = fwstat_get_firmware(pcu->fw_st);
>
>Or whatever consistent prefix it is going to be.
>
>>>>
>>>
>>> In the remoteproc case (patch 6) this would clean up the code,
>rather
>>> than replacing the completion API 1 to 1. I like it!
>> 
>> IIRC most drivers do it the same way. So request_firmware_async()
>indeed
>> would be good thing to have. Let me try that.
>
>While the idea behind this series is a good one I am wondering about
>the
>need for these drivers to use the asynchronous API. The historic reason
>might be to avoid timeout caused by user-mode helper, but that may no
>longer apply and these drivers could be better off using
>request_firmware_direct().

Actually systems using this driver rely on usermode helper to provide necessary delay and load the firmware from storage once root partition is mounted. Converting to request_firmware_direct() would break them.


Thanks.

Dmitry Torokhov July 31, 2016, 7:23 a.m. UTC | #7

On July 30, 2016 9:58:17 AM PDT, "Luis R. Rodriguez" <[e-mail]> wrote:
>On Sat, Jul 30, 2016 at 02:42:41PM +0200, Arend van Spriel wrote:
>> + Luis (again) ;-)
>> 
>> On 29-07-16 08:13, Daniel Wagner wrote:
>> > On 07/28/2016 09:01 PM, Bjorn Andersson wrote:
>> >> On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:
>> >>
>> >>> On Thu, Jul 28, 2016 at 09:55:11AM +0200, Daniel Wagner wrote:
>> >>>> From: Daniel Wagner <[e-mail]>
>> >>>>
>> >> [..]
>> >>>
>> >>> Do not quite like it... I'd rather asynchronous request give out
>a
>> >>> firmware status pointer that could be used later on.
>> 
>> Excellent. Why not get rid of the callback function as well and have
>> fw_loading_wait() return result (0 = firmware available, < 0 = fail).
>> Just to confirm, you are proposing a new API function next to
>> request_firmware_nowait(), right?
>
>If proposing new firmware_class patches please bounce / Cc me, I've
>recently asked for me to be added to MAINTAINERS so I get these
>e-mails as I'm working on a new flexible API which would allow us
>to extend the firmware API without having to care about the old
>stupid usermode helper at all.

I am not sure why we started calling usermode helper "stupid". We only had to implement direct kernel firmware loading because udev/stsremd folks had "interesting" ideas how events should be handled; but having userspace to feed us data is not stupid.

If we want to overhaul firmware loading support we need to figure out how to support case when a driver want to [asynchronously] request firmware/config/blob and the rest of the system is not ready. Even if we want kernel to do read/load the data we need userspace to tell kernel when firmware partition is available, until then the kernel should not fail the request.


Thanks.

Daniel Wagner Aug. 1, 2016, 12:26 p.m. UTC | #8

On 07/31/2016 09:23 AM, Dmitry Torokhov wrote:
> On July 30, 2016 9:58:17 AM PDT, "Luis R. Rodriguez" <[e-mail]> wrote:
>> On Sat, Jul 30, 2016 at 02:42:41PM +0200, Arend van Spriel wrote:
>>> On 29-07-16 08:13, Daniel Wagner wrote:
>>>> On 07/28/2016 09:01 PM, Bjorn Andersson wrote:
>>>>> On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:

>>> + Luis (again) ;-)

That was not on purpose :) My attempt to keep the Cc list a bit shorter 
was a failure.

>>>>>> Do not quite like it... I'd rather asynchronous request give out
>>>>>> firmware status pointer that could be used later on.
>>>
>>> Excellent. Why not get rid of the callback function as well and have
>>> fw_loading_wait() return result (0 = firmware available, < 0 = fail).
>>> Just to confirm, you are proposing a new API function next to
>>> request_firmware_nowait(), right?
>>
>> If proposing new firmware_class patches please bounce / Cc me, I've
>> recently asked for me to be added to MAINTAINERS so I get these
>> e-mails as I'm working on a new flexible API which would allow us
>> to extend the firmware API without having to care about the old
>> stupid usermode helper at all.

These patches here are a first attempt to clean up a bit of the code 
around the completion API. As Dmitry correctly pointed out, it makes 
more sense to go bit further and make the async loading a bit more 
convenient for the drivers.

> I am not sure why we started calling usermode helper "stupid". We
> only  had to implement direct kernel firmware loading because udev/stsremd
> folks had "interesting" ideas how events should be handled; but having
> userspace to feed us data is not stupid.

I was ignorant on all the nasty details around the firmware loading. If 
I parse Luis' patches correctly they introduce an API which calls 
kernel_read_file_from_path() asynchronously:

sysdata_file_request_async(..., &cookie)
   *coookie = async_schedule_domain(request_sysdata_file_work_func(), ..)

   request_sysdata_file_work_fun()
     _sysdata_file_request()
       fw_get_filesystem_firmware()
	kernel_read_file_from_path()

sysdata_synchronize_request(&cookie);

Doesn't look like what your asking for.

> If we want to overhaul firmware loading support we need to figure
> out  how to support case when a driver want to [asynchronously] request
> firmware/config/blob and the rest of the system is not ready. Even if we
> want kernel to do read/load the data we need userspace to tell kernel
> when firmware partition is available, until then the kernel should not
> fail the request.

I gather from Luis' blog post and comments that he is on the quest on 
removing userspace support completely.

Maybe this attempt here could be a step before. Step 1 would be changing 
request_firmware_nowait() to request_firmware_async so drivers don't 
have to come up with their own sync primitives, e.g.

   cookie = request_firmware_async()
   fw_load_wait(cookie)

Step 2 could be something more towards sysdata approach.

cheers,
daniel
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Bjorn Andersson Aug. 1, 2016, 5:19 p.m. UTC | #9

On Sat 30 Jul 09:58 PDT 2016, Luis R. Rodriguez wrote:

> On Sat, Jul 30, 2016 at 02:42:41PM +0200, Arend van Spriel wrote:
> > + Luis (again) ;-)
> > 
> > On 29-07-16 08:13, Daniel Wagner wrote:
> > > On 07/28/2016 09:01 PM, Bjorn Andersson wrote:
> > >> On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:
> > >>
> > >>> On Thu, Jul 28, 2016 at 09:55:11AM +0200, Daniel Wagner wrote:
> > >>>> From: Daniel Wagner <[e-mail]>
> > >>>>
> > >> [..]
> > >>>
> > >>> Do not quite like it... I'd rather asynchronous request give out a
> > >>> firmware status pointer that could be used later on.
> > 
> > Excellent. Why not get rid of the callback function as well and have
> > fw_loading_wait() return result (0 = firmware available, < 0 = fail).
> > Just to confirm, you are proposing a new API function next to
> > request_firmware_nowait(), right?
> 
> If proposing new firmware_class patches please bounce / Cc me, I've
> recently asked for me to be added to MAINTAINERS so I get these
> e-mails as I'm working on a new flexible API which would allow us
> to extend the firmware API without having to care about the old
> stupid usermode helper at all.
> 

In the remoteproc world there are several systems where we see 100+MB of
firmware being loaded. It's unfeasible to have these files in an
initramfs, so we need a way to indicate to the kernel that the
second/primary (or a dedicated firmware partition) is mounted.

We're currently loading these files with request_firmware_nowait(), so
either one can either use kernel modules or the user-helper fallback to
delay the loading until the files are available. (I don't like to
enforce the usage of kernel modules)

I'm open to alternative ways of having firmware loading wait on
secondary filesystems to be mounted, but have not yet tried to tackle
this problem. I do believe something that waits and retries the firmware
load as additional file systems gets mounted would be prettier than
forcing user space to tell us it's time to move on.



Due to the size of these firmware files we release the firmware as soon
as we have copied the content into the appropriate memory segments, so
we're not utilizing the caching mechanisms of the current fw loader.

Regards,
Bjorn
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Luis Chamberlain Aug. 1, 2016, 7:36 p.m. UTC | #10

On Sun, Jul 31, 2016 at 12:23:09AM -0700, Dmitry Torokhov wrote:
> On July 30, 2016 9:58:17 AM PDT, "Luis R. Rodriguez" <[e-mail]> wrote:
> >On Sat, Jul 30, 2016 at 02:42:41PM +0200, Arend van Spriel wrote:
> >> + Luis (again) ;-)
> >> 
> >> On 29-07-16 08:13, Daniel Wagner wrote:
> >> > On 07/28/2016 09:01 PM, Bjorn Andersson wrote:
> >> >> On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:
> >> >>
> >> >>> On Thu, Jul 28, 2016 at 09:55:11AM +0200, Daniel Wagner wrote:
> >> >>>> From: Daniel Wagner <[e-mail]>
> >> >>>>
> >> >> [..]
> >> >>>
> >> >>> Do not quite like it... I'd rather asynchronous request give out
> >a
> >> >>> firmware status pointer that could be used later on.
> >> 
> >> Excellent. Why not get rid of the callback function as well and have
> >> fw_loading_wait() return result (0 = firmware available, < 0 = fail).
> >> Just to confirm, you are proposing a new API function next to
> >> request_firmware_nowait(), right?
> >
> >If proposing new firmware_class patches please bounce / Cc me, I've
> >recently asked for me to be added to MAINTAINERS so I get these
> >e-mails as I'm working on a new flexible API which would allow us
> >to extend the firmware API without having to care about the old
> >stupid usermode helper at all.
> 
> I am not sure why we started calling usermode helper "stupid".

OK Fair, how systemd implemented kmod timeout for the usermode helper
was stupid and it affected tons of systems.

> We only had to
> implement direct kernel firmware loading because udev/stsremd folks had
> "interesting" ideas how events should be handled; but having userspace to
> feed us data is not stupid.

It really should just be an option. The problem was the collateral due to the
way it was handled in userspace.

> If we want to overhaul firmware loading support we need to figure out how to
> support case when a driver want to [asynchronously] request
> firmware/config/blob and the rest of the system is not ready.

There's a lot of issues. So let's break them down.

1) First the sysdata API came about to help avoid the required set of collateral
evolutions whenever the firmware API was expanded. A new argument added meant
requiring to modify all drivers with the new argument, or a new exported
symbol. The sysdata API makes the API flexible, enabling extensions by just
expanding on the descriptor passed. The few features I've added to sysdata
(like avoiding drivers having to declare their own completions, waits) are
just small enhancements, but it seems now supporting devm may be desirable
as well.

2) The usermode helper cannot be removed, however we can compartamentalize it.
The sysdata API aims at helping with that, it doesn't touch it. There are
only 2 explicit users of the usermode helper now in the kernel. If there are
further users that really want it, I'd like to hear about them.

3) The firmware API having its own kernel read thing was fine but there were
other places in the kernel doing the same, this begged sharing that code
and also allowing then two LSM hooks to take care of handling doing whatever
with a kernel read, a pre-read hook and post-read hook. Mimi completed this
work and we now have the firmware API using kernel_read_file_from_path().

4) The asynchronous firmware loading issue you describe above is just *one*
issue, but it becomes more of an generic issue if you consider 3) above...
because naturally there could potentially be other users of kernel_read_file()
or kernel_read_file_from_path() and whereby a race also can happen. We may
decide that this is up to the subsystem/user to figure out. If that's the
case lets discuss that. It however occurs to me that it could just be as
simple as adding another fs/exec.c helper like kernel_read_file_from_path_wait()
which passes a completion and fs/exec.c just signals back when its ready.
If we have this both the old firmware_class and new sysdata API could benefit
from this behind the scenes -- no new API extension would be needed, this
would just be a firmware_class fix to use a more deterministic kernel
read API.

> Even if we want kernel to do read/load the data we need userspace to tell
> kernel when firmware partition is available, until then the kernel should not
> fail the request.

pivot_root() is possible as well, so exactly when the *real* partition is
ready is very system specific -- best I think we can do is perhaps just
see when the *first* file path becomes available and signal back. Problem
with this is we would still need a way to know -- *everything is ready*
as a limit condition for waiting.

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Luis Chamberlain Aug. 1, 2016, 7:44 p.m. UTC | #11

On Mon, Aug 01, 2016 at 02:26:04PM +0200, Daniel Wagner wrote:
> On 07/31/2016 09:23 AM, Dmitry Torokhov wrote:
> >On July 30, 2016 9:58:17 AM PDT, "Luis R. Rodriguez" <[e-mail]> wrote:
> >>On Sat, Jul 30, 2016 at 02:42:41PM +0200, Arend van Spriel wrote:
> >>>On 29-07-16 08:13, Daniel Wagner wrote:
> >>>>On 07/28/2016 09:01 PM, Bjorn Andersson wrote:
> >>>>>On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:
> 
> >>>+ Luis (again) ;-)
> 
> That was not on purpose :) My attempt to keep the Cc list a bit
> shorter was a failure.
> 
> >>>>>>Do not quite like it... I'd rather asynchronous request give out
> >>>>>>firmware status pointer that could be used later on.
> >>>
> >>>Excellent. Why not get rid of the callback function as well and have
> >>>fw_loading_wait() return result (0 = firmware available, < 0 = fail).
> >>>Just to confirm, you are proposing a new API function next to
> >>>request_firmware_nowait(), right?
> >>
> >>If proposing new firmware_class patches please bounce / Cc me, I've
> >>recently asked for me to be added to MAINTAINERS so I get these
> >>e-mails as I'm working on a new flexible API which would allow us
> >>to extend the firmware API without having to care about the old
> >>stupid usermode helper at all.
> 
> These patches here are a first attempt to clean up a bit of the code
> around the completion API. As Dmitry correctly pointed out, it makes
> more sense to go bit further and make the async loading a bit more
> convenient for the drivers.
> 
> >I am not sure why we started calling usermode helper "stupid". We
> >only  had to implement direct kernel firmware loading because udev/stsremd
> >folks had "interesting" ideas how events should be handled; but having
> >userspace to feed us data is not stupid.
> 
> I was ignorant on all the nasty details around the firmware loading.
> If I parse Luis' patches correctly they introduce an API which calls
> kernel_read_file_from_path() asynchronously:
> 
> sysdata_file_request_async(..., &cookie)
>   *coookie = async_schedule_domain(request_sysdata_file_work_func(), ..)
> 
>   request_sysdata_file_work_fun()
>     _sysdata_file_request()
>       fw_get_filesystem_firmware()
> 	kernel_read_file_from_path()
> 
> sysdata_synchronize_request(&cookie);
> 
> Doesn't look like what your asking for.

No, but its also a generic kernel read issue as I noted in my last
reply.

> >If we want to overhaul firmware loading support we need to figure
> >out  how to support case when a driver want to [asynchronously] request
> >firmware/config/blob and the rest of the system is not ready. Even if we
> >want kernel to do read/load the data we need userspace to tell kernel
> >when firmware partition is available, until then the kernel should not
> >fail the request.
> 
> I gather from Luis' blog post and comments that he is on the quest
> on removing userspace support completely.

No, I explained in my last proposed documentation patch series that we cannot
get rid of the usermode helper. Its not well understood why so I explained and
documented why. Best we can do is compartamentalize its uses.

The sysdata API's main goal rather is to provide a flexible API first,
compartamentalizing the usermode helper was secondary. But now it seems
I may just also add devm support too to help simplify code further.

What Dmitry notes is an existential issue with kernel_read_file_from_path()
and we need a common solution for it.

> Maybe this attempt here could be a step before. Step 1 would be
> changing request_firmware_nowait() to request_firmware_async so
> drivers don't have to come up with their own sync primitives, e.g.
> 
>   cookie = request_firmware_async()
>   fw_load_wait(cookie)

That's one of the features already part of async mechanism of the sysdata API :)

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Luis Chamberlain Aug. 1, 2016, 7:56 p.m. UTC | #12

On Mon, Aug 01, 2016 at 10:19:51AM -0700, Bjorn Andersson wrote:
> On Sat 30 Jul 09:58 PDT 2016, Luis R. Rodriguez wrote:
> 
> > On Sat, Jul 30, 2016 at 02:42:41PM +0200, Arend van Spriel wrote:
> > > + Luis (again) ;-)
> > > 
> > > On 29-07-16 08:13, Daniel Wagner wrote:
> > > > On 07/28/2016 09:01 PM, Bjorn Andersson wrote:
> > > >> On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:
> > > >>
> > > >>> On Thu, Jul 28, 2016 at 09:55:11AM +0200, Daniel Wagner wrote:
> > > >>>> From: Daniel Wagner <[e-mail]>
> > > >>>>
> > > >> [..]
> > > >>>
> > > >>> Do not quite like it... I'd rather asynchronous request give out a
> > > >>> firmware status pointer that could be used later on.
> > > 
> > > Excellent. Why not get rid of the callback function as well and have
> > > fw_loading_wait() return result (0 = firmware available, < 0 = fail).
> > > Just to confirm, you are proposing a new API function next to
> > > request_firmware_nowait(), right?
> > 
> > If proposing new firmware_class patches please bounce / Cc me, I've
> > recently asked for me to be added to MAINTAINERS so I get these
> > e-mails as I'm working on a new flexible API which would allow us
> > to extend the firmware API without having to care about the old
> > stupid usermode helper at all.
> > 
> 
> In the remoteproc world there are several systems where we see 100+MB of
> firmware being loaded. It's unfeasible to have these files in an
> initramfs, so we need a way to indicate to the kernel that the
> second/primary (or a dedicated firmware partition) is mounted.
> 
> We're currently loading these files with request_firmware_nowait(), so
> either one can either use kernel modules or the user-helper fallback to
> delay the loading until the files are available. (I don't like to
> enforce the usage of kernel modules)

Now that the firmware API is sharing the same API call to read files
the existential issue of the file is not unique issue of firmware,
but also to any other caller of it.

> I'm open to alternative ways of having firmware loading wait on
> secondary filesystems to be mounted, but have not yet tried to tackle
> this problem. I do believe something that waits and retries the firmware
> load as additional file systems gets mounted would be prettier than
> forcing user space to tell us it's time to move on.

Agreed. We simply have not addressed this problem yet. Let's discuss a
possible solution on the other reply thread I provided with more details
to Dmitry.

> Due to the size of these firmware files we release the firmware as soon
> as we have copied the content into the appropriate memory segments, so
> we're not utilizing the caching mechanisms of the current fw loader.

BTW my goal with the sysdata API is to automatically free this for you too :)

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Dmitry Torokhov Aug. 1, 2016, 9:34 p.m. UTC | #13

On Mon, Aug 01, 2016 at 10:19:51AM -0700, Bjorn Andersson wrote:
> On Sat 30 Jul 09:58 PDT 2016, Luis R. Rodriguez wrote:
> 
> > On Sat, Jul 30, 2016 at 02:42:41PM +0200, Arend van Spriel wrote:
> > > + Luis (again) ;-)
> > > 
> > > On 29-07-16 08:13, Daniel Wagner wrote:
> > > > On 07/28/2016 09:01 PM, Bjorn Andersson wrote:
> > > >> On Thu 28 Jul 11:33 PDT 2016, Dmitry Torokhov wrote:
> > > >>
> > > >>> On Thu, Jul 28, 2016 at 09:55:11AM +0200, Daniel Wagner wrote:
> > > >>>> From: Daniel Wagner <[e-mail]>
> > > >>>>
> > > >> [..]
> > > >>>
> > > >>> Do not quite like it... I'd rather asynchronous request give out a
> > > >>> firmware status pointer that could be used later on.
> > > 
> > > Excellent. Why not get rid of the callback function as well and have
> > > fw_loading_wait() return result (0 = firmware available, < 0 = fail).
> > > Just to confirm, you are proposing a new API function next to
> > > request_firmware_nowait(), right?
> > 
> > If proposing new firmware_class patches please bounce / Cc me, I've
> > recently asked for me to be added to MAINTAINERS so I get these
> > e-mails as I'm working on a new flexible API which would allow us
> > to extend the firmware API without having to care about the old
> > stupid usermode helper at all.
> > 
> 
> In the remoteproc world there are several systems where we see 100+MB of
> firmware being loaded. It's unfeasible to have these files in an
> initramfs, so we need a way to indicate to the kernel that the
> second/primary (or a dedicated firmware partition) is mounted.
> 
> We're currently loading these files with request_firmware_nowait(), so
> either one can either use kernel modules or the user-helper fallback to
> delay the loading until the files are available. (I don't like to
> enforce the usage of kernel modules)
> 
> I'm open to alternative ways of having firmware loading wait on
> secondary filesystems to be mounted, but have not yet tried to tackle
> this problem. I do believe something that waits and retries the firmware
> load as additional file systems gets mounted would be prettier than
> forcing user space to tell us it's time to move on.

Hmm, but when do you stop? How do you know that you are not going to get
yet another fs mounted and that one will finally have the firmware you
are looking for? The kernel does now, but distribution/product
integrator does. That is why I think the most reliable way is to see if
firmware is built-in, otherwise wait and let userspace do its thing.

Thanks.

Daniel Wagner Aug. 2, 2016, 5:49 a.m. UTC | #14

Hi Luis,

>> I was ignorant on all the nasty details around the firmware loading.
>> If I parse Luis' patches correctly they introduce an API which calls
>> kernel_read_file_from_path() asynchronously:
>>
>> sysdata_file_request_async(..., &cookie)
>>   *coookie = async_schedule_domain(request_sysdata_file_work_func(), ..)
>>
>>   request_sysdata_file_work_fun()
>>     _sysdata_file_request()
>>       fw_get_filesystem_firmware()
>> 	kernel_read_file_from_path()
>>
>> sysdata_synchronize_request(&cookie);
>>
>> Doesn't look like what your asking for.
>
> No, but its also a generic kernel read issue as I noted in my last
> reply.

Okay, got it.

>>> If we want to overhaul firmware loading support we need to figure
>>> out  how to support case when a driver want to [asynchronously] request
>>> firmware/config/blob and the rest of the system is not ready. Even if we
>>> want kernel to do read/load the data we need userspace to tell kernel
>>> when firmware partition is available, until then the kernel should not
>>> fail the request.
>>
>> I gather from Luis' blog post and comments that he is on the quest
>> on removing userspace support completely.
>
> No, I explained in my last proposed documentation patch series that we cannot
> get rid of the usermode helper.

I stand corrected.

> Its not well understood why so I explained and documented why.

Obviously, I got lost somewhere there :)

> Best we can do is compartamentalize its uses.

Sounds like a plan.

> The sysdata API's main goal rather is to provide a flexible API first,
> compartamentalizing the usermode helper was secondary. But now it seems
> I may just also add devm support too to help simplify code further.

I missed the point that you plan to add usermode helper support to the 
sysdata API.

> What Dmitry notes is an existential issue with kernel_read_file_from_path()
> and we need a common solution for it.

Understood. I guess best thing to keep that discussion in the other 
subthread.

>> Maybe this attempt here could be a step before. Step 1 would be
>> changing request_firmware_nowait() to request_firmware_async so
>> drivers don't have to come up with their own sync primitives, e.g.
>>
>>   cookie = request_firmware_async()
>>   fw_load_wait(cookie)
>
> That's one of the features already part of async mechanism of the sysdata API :)

Yes, I realized that too :)

cheers,
daniel

Thanks for the feedback.
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Luis Chamberlain Aug. 2, 2016, 6:34 a.m. UTC | #15

On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
> >The sysdata API's main goal rather is to provide a flexible API first,
> >compartamentalizing the usermode helper was secondary. But now it seems
> >I may just also add devm support too to help simplify code further.
> 
> I missed the point that you plan to add usermode helper support to
> the sysdata API.

I had no such plans, when I have asked folks so far about "hey are you
really in need for it, OK what for? " and "what extended uses do you
envision?" so I far I have not gotten any replies at all. So -- instead
sysdata currently ignores it.

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Daniel Wagner Aug. 2, 2016, 6:53 a.m. UTC | #16

On 08/02/2016 08:34 AM, Luis R. Rodriguez wrote:
> On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
>>> The sysdata API's main goal rather is to provide a flexible API first,
>>> compartamentalizing the usermode helper was secondary. But now it seems
>>> I may just also add devm support too to help simplify code further.
>>
>> I missed the point that you plan to add usermode helper support to
>> the sysdata API.
>
> I had no such plans, when I have asked folks so far about "hey are you
> really in need for it, OK what for? " and "what extended uses do you
> envision?" so I far I have not gotten any replies at all. So -- instead
> sysdata currently ignores it.

So you argue for the remoteproc use case with 100+ MB firmware that if 
there is a way to load after pivot_root() (or other additional firmware 
partition shows up) then there is no need at all for usermode helper?
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Luis Chamberlain Aug. 2, 2016, 7:41 a.m. UTC | #17

On Tue, Aug 02, 2016 at 08:53:55AM +0200, Daniel Wagner wrote:
> On 08/02/2016 08:34 AM, Luis R. Rodriguez wrote:
> >On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
> >>>The sysdata API's main goal rather is to provide a flexible API first,
> >>>compartamentalizing the usermode helper was secondary. But now it seems
> >>>I may just also add devm support too to help simplify code further.
> >>
> >>I missed the point that you plan to add usermode helper support to
> >>the sysdata API.
> >
> >I had no such plans, when I have asked folks so far about "hey are you
> >really in need for it, OK what for? " and "what extended uses do you
> >envision?" so I far I have not gotten any replies at all. So -- instead
> >sysdata currently ignores it.
> 
> So you argue for the remoteproc use case with 100+ MB firmware that
> if there is a way to load after pivot_root() (or other additional
> firmware partition shows up) then there is no need at all for
> usermode helper?

No, I'm saying I'd like to hear valid uses cases for the usermode helper and so
far I have only found using coccinelle grammar 2 explicit users, that's it. My
patch series (not yet merge) then annotates these as valid as I've verified
through their documentation they have some quirky requirement.

Other than these two drivers I'd like hear to valid requirements for it.

The existential issue is a real issue but it does not look impossible to
resolve. It may be a solution to bloat up the kernel with 100+ MB size just to
stuff built-in firmware to avoid this issue, but it does not mean a solution
is not possible.

Remind me -- why can remoteproc not stuff the firmware in initramfs ?

Anyway, here's a simple suggestion: fs/exec.c gets a sentinel file monitor
support per enum kernel_read_file_id. For instance we'd have one for
READING_FIRMWARE, one for READING_KEXEC_IMAGE, perhaps READING_POLICY, and this
would in turn be used as the system configurable deterministic file for
which to wait for to be present before enabling each enum kernel_read_file_id
type read.

Thoughts ?

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Daniel Wagner Aug. 3, 2016, 6:57 a.m. UTC | #18

On 08/02/2016 09:41 AM, Luis R. Rodriguez wrote:
> On Tue, Aug 02, 2016 at 08:53:55AM +0200, Daniel Wagner wrote:
>> On 08/02/2016 08:34 AM, Luis R. Rodriguez wrote:
>>> On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
>> So you argue for the remoteproc use case with 100+ MB firmware that
>> if there is a way to load after pivot_root() (or other additional
>> firmware partition shows up) then there is no need at all for
>> usermode helper?
>
> No, I'm saying I'd like to hear valid uses cases for the usermode helper and so
> far I have only found using coccinelle grammar 2 explicit users, that's it. My
> patch series (not yet merge) then annotates these as valid as I've verified
> through their documentation they have some quirky requirement.

I got that question wrong. It should read something like 'for the 
remoteproc 100+MB there is no need for the user help?'. I've gone 
through your patches and they make perfectly sense too. Maybe I can 
convince you to take a better version of my patch 3 into your queue. And 
I help you converting the exiting drivers. Obviously if you like my help 
at all.

> Other than these two drivers I'd like hear to valid requirements for it.
>
> The existential issue is a real issue but it does not look impossible to
> resolve. It may be a solution to bloat up the kernel with 100+ MB size just to
> stuff built-in firmware to avoid this issue, but it does not mean a solution
> is not possible.
>
> Remind me -- why can remoteproc not stuff the firmware in initramfs ?

I don't know. I was just bringing it up with the hope that Bjorn will 
defend it. It seems my tactics didn't work out :)

> Anyway, here's a simple suggestion: fs/exec.c gets a sentinel file monitor
> support per enum kernel_read_file_id. For instance we'd have one for
> READING_FIRMWARE, one for READING_KEXEC_IMAGE, perhaps READING_POLICY, and this
> would in turn be used as the system configurable deterministic file for
> which to wait for to be present before enabling each enum kernel_read_file_id
> type read.
>
> Thoughts ?

Not sure if I get you here correctly. Is the 'system configurable 
deterministic file' is a knob which controlled by user space? Or it this 
something you define at compile time?

Hmm, so it would allow to decided to ask a userspace helper or load the 
firmware directly (to be more precised the kernel_read_file_id type). If 
yes, than it is what currently already have just integrated nicely into 
the new sysdata API.

cheers,
daniel

--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Dmitry Torokhov Aug. 3, 2016, 7:42 a.m. UTC | #19

On Tue, Aug 2, 2016 at 12:41 AM, Luis R. Rodriguez <[e-mail]> wrote:
> On Tue, Aug 02, 2016 at 08:53:55AM +0200, Daniel Wagner wrote:
>> On 08/02/2016 08:34 AM, Luis R. Rodriguez wrote:
>> >On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
>> >>>The sysdata API's main goal rather is to provide a flexible API first,
>> >>>compartamentalizing the usermode helper was secondary. But now it seems
>> >>>I may just also add devm support too to help simplify code further.
>> >>
>> >>I missed the point that you plan to add usermode helper support to
>> >>the sysdata API.
>> >
>> >I had no such plans, when I have asked folks so far about "hey are you
>> >really in need for it, OK what for? " and "what extended uses do you
>> >envision?" so I far I have not gotten any replies at all. So -- instead
>> >sysdata currently ignores it.
>>
>> So you argue for the remoteproc use case with 100+ MB firmware that
>> if there is a way to load after pivot_root() (or other additional
>> firmware partition shows up) then there is no need at all for
>> usermode helper?
>
> No, I'm saying I'd like to hear valid uses cases for the usermode helper and so
> far I have only found using coccinelle grammar 2 explicit users, that's it. My
> patch series (not yet merge) then annotates these as valid as I've verified
> through their documentation they have some quirky requirement.

In certain configurations (embedded) people do not want to use
initramfs nor modules nor embed firmware into the kernel. In this case
usermode helper + firmware calss timeout handling provides necessary
wait for the root filesystem to be mounted.

If we solve waiting for rootfs (or something else that may contain
firmware) then these cases will not need to use usermode helper.

Thanks.

Arend Van Spriel Aug. 3, 2016, 11:43 a.m. UTC | #20

On 03-08-16 09:42, Dmitry Torokhov wrote:
> On Tue, Aug 2, 2016 at 12:41 AM, Luis R. Rodriguez <[e-mail]> wrote:
>> On Tue, Aug 02, 2016 at 08:53:55AM +0200, Daniel Wagner wrote:
>>> On 08/02/2016 08:34 AM, Luis R. Rodriguez wrote:
>>>> On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
>>>>>> The sysdata API's main goal rather is to provide a flexible API first,
>>>>>> compartamentalizing the usermode helper was secondary. But now it seems
>>>>>> I may just also add devm support too to help simplify code further.
>>>>>
>>>>> I missed the point that you plan to add usermode helper support to
>>>>> the sysdata API.
>>>>
>>>> I had no such plans, when I have asked folks so far about "hey are you
>>>> really in need for it, OK what for? " and "what extended uses do you
>>>> envision?" so I far I have not gotten any replies at all. So -- instead
>>>> sysdata currently ignores it.
>>>
>>> So you argue for the remoteproc use case with 100+ MB firmware that
>>> if there is a way to load after pivot_root() (or other additional
>>> firmware partition shows up) then there is no need at all for
>>> usermode helper?
>>
>> No, I'm saying I'd like to hear valid uses cases for the usermode helper and so
>> far I have only found using coccinelle grammar 2 explicit users, that's it. My
>> patch series (not yet merge) then annotates these as valid as I've verified
>> through their documentation they have some quirky requirement.
> 
> In certain configurations (embedded) people do not want to use
> initramfs nor modules nor embed firmware into the kernel. In this case
> usermode helper + firmware calss timeout handling provides necessary
> wait for the root filesystem to be mounted.

And there are people who don't have a usermode helper running at all in
their configuration, but I guess they should disable the helper.

In my opinion the kernel should provide functionality to user-space and
user-space providing functionality to the kernel should be avoided.

> If we solve waiting for rootfs (or something else that may contain
> firmware) then these cases will not need to use usermode helper.

If firmware (or whatever) API could get notification of mount syscall it
could be used to retry firmware loading instead of periodic polling.
That leaves the question raised by you about when to stop trying. The
initlevel stuff is probably a user-space only concept, right? So no
ideas how the kernel itself could decide except for a "long" timeout.

Regards,
Arend
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Luis Chamberlain Aug. 3, 2016, 3:18 p.m. UTC | #21

On Wed, Aug 03, 2016 at 01:43:31PM +0200, Arend van Spriel wrote:
> On 03-08-16 09:42, Dmitry Torokhov wrote:
> > On Tue, Aug 2, 2016 at 12:41 AM, Luis R. Rodriguez <[e-mail]> wrote:
> >> On Tue, Aug 02, 2016 at 08:53:55AM +0200, Daniel Wagner wrote:
> >>> On 08/02/2016 08:34 AM, Luis R. Rodriguez wrote:
> >>>> On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
> >>>>>> The sysdata API's main goal rather is to provide a flexible API first,
> >>>>>> compartamentalizing the usermode helper was secondary. But now it seems
> >>>>>> I may just also add devm support too to help simplify code further.
> >>>>>
> >>>>> I missed the point that you plan to add usermode helper support to
> >>>>> the sysdata API.
> >>>>
> >>>> I had no such plans, when I have asked folks so far about "hey are you
> >>>> really in need for it, OK what for? " and "what extended uses do you
> >>>> envision?" so I far I have not gotten any replies at all. So -- instead
> >>>> sysdata currently ignores it.
> >>>
> >>> So you argue for the remoteproc use case with 100+ MB firmware that
> >>> if there is a way to load after pivot_root() (or other additional
> >>> firmware partition shows up) then there is no need at all for
> >>> usermode helper?
> >>
> >> No, I'm saying I'd like to hear valid uses cases for the usermode helper and so
> >> far I have only found using coccinelle grammar 2 explicit users, that's it. My
> >> patch series (not yet merge) then annotates these as valid as I've verified
> >> through their documentation they have some quirky requirement.
> > 
> > In certain configurations (embedded) people do not want to use
> > initramfs nor modules nor embed firmware into the kernel. In this case
> > usermode helper + firmware calss timeout handling provides necessary
> > wait for the root filesystem to be mounted.
> 
> And there are people who don't have a usermode helper running at all 

Correction: most Linux distributions these days have
CONFIG_FW_LOADER_USER_HELPER but disable CONFIG_FW_LOADER_USER_HELPER_FALLBACK.

And then let me re-iterate from my patches then the implications:

+When CONFIG_FW_LOADER_USER_HELPER_FALLBACK is enabled request_firmware()
+*always* calls the usermode helpers. When CONFIG_FW_LOADER_USER_HELPER is
+enabled, only if you specifically ask for it will the usermode helper be
+called. If CONFIG_FW_LOADER_USER_HELPER_FALLBACK is disabled drivers can
+still require the usermode helper by using request_firmware_nowait().

The issues with CONFIG_FW_LOADER_USER_HELPER_FALLBACK are so much so
(the kmod timeout was an engineering mishap refer to my post with
details about it [0]) that these days most distributions disable it
and there have been huge efforts to ensure its not even called
explicitly, to the point now we only have TWO explicit users for it.
Keep in mind as well that in my series of patches to help clarify the
situation with the usermode helper I also proposed a fix to the firmware_class
to avoid the usermode helper completely when the firmware cache, this is
because the firmware cache purposely *kills* any pending usermode helpers
on the way to suspend anyway, so calling a new instance would be an error
and just delay suspend further.

[0] http://www.do-not-panic.com/2015/12/linux-asynchronous-probe.html

> in
> their configuration, but I guess they should disable the helper.

Yes. I have gone to good lengths to ask why the hell its needed and so 
far I have not gotten any technical reason for it.

> > If we solve waiting for rootfs (or something else that may contain
> > firmware) then these cases will not need to use usermode helper.
> 
> If firmware (or whatever) API could get notification of mount syscall it
> could be used to retry firmware loading instead of periodic polling.
> That leaves the question raised by you about when to stop trying. The
> initlevel stuff is probably a user-space only concept, right?

No init levels here refer to include/linux/init.h init calls that one
wraps the kernel module inits, or subsystem inits. If it wasn't for
pivot_root() existing and also the fact that systems can be complex
I would have suggested simply that we make firmware_class move to
late_initcall() -- this however would not solve all races given how
complex systems could be set up with a partition.

> So no
> ideas how the kernel itself could decide except for a "long" timeout.

If we wanted to learn from history -- the timeout for this would be a bad idea
so this makes it a bit more complex.  I proposed a technical idea here which
would avoid this and I think enable making this deterministic as well for new
systems that want that.

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Dmitry Torokhov Aug. 3, 2016, 3:35 p.m. UTC | #22

On Wed, Aug 03, 2016 at 01:43:31PM +0200, Arend van Spriel wrote:
> On 03-08-16 09:42, Dmitry Torokhov wrote:
> > On Tue, Aug 2, 2016 at 12:41 AM, Luis R. Rodriguez <[e-mail]> wrote:
> >> On Tue, Aug 02, 2016 at 08:53:55AM +0200, Daniel Wagner wrote:
> >>> On 08/02/2016 08:34 AM, Luis R. Rodriguez wrote:
> >>>> On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
> >>>>>> The sysdata API's main goal rather is to provide a flexible API first,
> >>>>>> compartamentalizing the usermode helper was secondary. But now it seems
> >>>>>> I may just also add devm support too to help simplify code further.
> >>>>>
> >>>>> I missed the point that you plan to add usermode helper support to
> >>>>> the sysdata API.
> >>>>
> >>>> I had no such plans, when I have asked folks so far about "hey are you
> >>>> really in need for it, OK what for? " and "what extended uses do you
> >>>> envision?" so I far I have not gotten any replies at all. So -- instead
> >>>> sysdata currently ignores it.
> >>>
> >>> So you argue for the remoteproc use case with 100+ MB firmware that
> >>> if there is a way to load after pivot_root() (or other additional
> >>> firmware partition shows up) then there is no need at all for
> >>> usermode helper?
> >>
> >> No, I'm saying I'd like to hear valid uses cases for the usermode helper and so
> >> far I have only found using coccinelle grammar 2 explicit users, that's it. My
> >> patch series (not yet merge) then annotates these as valid as I've verified
> >> through their documentation they have some quirky requirement.
> > 
> > In certain configurations (embedded) people do not want to use
> > initramfs nor modules nor embed firmware into the kernel. In this case
> > usermode helper + firmware calss timeout handling provides necessary
> > wait for the root filesystem to be mounted.
> 
> And there are people who don't have a usermode helper running at all in
> their configuration, but I guess they should disable the helper.

Right, if they don't that means their system is misconfigured.

> 
> In my opinion the kernel should provide functionality to user-space and
> user-space providing functionality to the kernel should be avoided.

Why? We have bunch of stuff running in userspace for the kernel. Fuse
for example. I am sure there are more.

> 
> > If we solve waiting for rootfs (or something else that may contain
> > firmware) then these cases will not need to use usermode helper.
> 
> If firmware (or whatever) API could get notification of mount syscall it
> could be used to retry firmware loading instead of periodic polling.
> That leaves the question raised by you about when to stop trying. The
> initlevel stuff is probably a user-space only concept, right? So no
> ideas how the kernel itself could decide except for a "long" timeout.

The kernel really does not know, it can only guess. The firmware may get
delivered by motorized carrier pidgeons. But distribution does know how
they set up, so they are in position to tell the kernel "go" or "give
up".

Thanks.

Luis Chamberlain Aug. 3, 2016, 3:55 p.m. UTC | #23

On Wed, Aug 03, 2016 at 08:57:09AM +0200, Daniel Wagner wrote:
> On 08/02/2016 09:41 AM, Luis R. Rodriguez wrote:
> >On Tue, Aug 02, 2016 at 08:53:55AM +0200, Daniel Wagner wrote:
> >>On 08/02/2016 08:34 AM, Luis R. Rodriguez wrote:
> >>>On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
> >>So you argue for the remoteproc use case with 100+ MB firmware that
> >>if there is a way to load after pivot_root() (or other additional
> >>firmware partition shows up) then there is no need at all for
> >>usermode helper?
> >
> >No, I'm saying I'd like to hear valid uses cases for the usermode helper and so
> >far I have only found using coccinelle grammar 2 explicit users, that's it. My
> >patch series (not yet merge) then annotates these as valid as I've verified
> >through their documentation they have some quirky requirement.
> 
> I got that question wrong. It should read something like 'for the
> remoteproc 100+MB there is no need for the user help?'.

That's not a question for me but for those who say that the usermode helper
is needed for remoteproc, so far from what folks are saying it seems the only
reason for the usermodehelper was to try to avoid the deterministic issue,
but I suggested a way to resolve that without the usermode helper now so
would be curious to hear if there are any more reasons for it.

> I've gone
> through your patches and they make perfectly sense too. Maybe I can
> convince you to take a better version of my patch 3 into your queue.
> And I help you converting the exiting drivers. Obviously if you like
> my help at all.

I accept all help and would be glad to make enhancements instead of
the old API through new API. The biggest thing here first I think is
adding devm support, that I think should address what seemed to be
the need to add more code for a transformation into the API. I'd
personally only want to add that and be done with an introduction
of the sysdata API. Further changes IMHO are best done atomically
after that on top of it, but I'm happy to queue in the changes.

> >Other than these two drivers I'd like hear to valid requirements for it.
> >
> >The existential issue is a real issue but it does not look impossible to
> >resolve. It may be a solution to bloat up the kernel with 100+ MB size just to
> >stuff built-in firmware to avoid this issue, but it does not mean a solution
> >is not possible.
> >
> >Remind me -- why can remoteproc not stuff the firmware in initramfs ?
> 
> I don't know. I was just bringing it up with the hope that Bjorn
> will defend it. It seems my tactics didn't work out :)

OK.

> >Anyway, here's a simple suggestion: fs/exec.c gets a sentinel file monitor
> >support per enum kernel_read_file_id. For instance we'd have one for
> >READING_FIRMWARE, one for READING_KEXEC_IMAGE, perhaps READING_POLICY, and this
> >would in turn be used as the system configurable deterministic file for
> >which to wait for to be present before enabling each enum kernel_read_file_id
> >type read.
> >
> >Thoughts ?
> 
> Not sure if I get you here correctly. Is the 'system configurable
> deterministic file' is a knob which controlled by user space? Or it
> this something you define at compile time?

I meant at compile time on the kernel. So CONFIG_READ_READY_SENTINEL
or something like this, and it be a string, which if set then when
the kernel read APIs are used, then a new API could be introduced
that would *only* enable reading through once that sentinel has
been detected by the kernel to allowed through reads. Doing this
per mount / target filesystem is rather cumbersome given possible
overlaps in mounts and also pivot_root() being possible, so instead
targeting simply the fs/exec.c enum kernel_read_file_id would seem
more efficient and clean but we would need a decided upon set of
paths per enum kernel_read_file_id as base (or just one path per
enum kernel_read_file_id). For number of paths I mean the number
of target directories to look for the sentinel per enum kernel_read_file_id,
so for instance for READING_FIRMWARE perhaps just deciding on /lib/firmware/
would suffice, but if this supported multiple paths another option may be
for the sentinel to also be looked for in /lib/firmware/updates/,
/lib/firmware/" UTS_RELEASE -- etc. It would *stop* after finding one
sentinel on any of these paths.

If a system has has CONFIG_READ_READY_SENTINEL it would mean an agreed upon
system configuration has been decided so that at any point in time reads
against READING_FIRMWARE using a new kernel_read_file_from_path_sentinel()
(or something like it) would only allow the read to go through once
the sentinel has been found for READING_FIRMWARE on the agreed upon
paths.

The benefit of the sentintel approach is it avoids complexities with
pivot_root(), and makes the deterministic aspect of the target left
only to a system-configuration enabled target path / file.

This is just an idea. I'd like some FS folks to review.

> Hmm, so it would allow to decided to ask a userspace helper or load
> the firmware directly (to be more precised the kernel_read_file_id
> type). If yes, than it is what currently already have just
> integrated nicely into the new sysdata API.

Sorry, no, the above description is better of what I meant. This
actually would not need to go into the sysdata API, unless of
course we wanted it just as a new "feature" of it, but I don't
think that's needed unless it has some implications behind the
scenes. Given that firmware_class now uses a common core kernel
API for reading files kernel_read_file_from_path() we could
for instance add kernel_read_file_from_sentintel() and only
if CONFIG_READ_READY_SENTINEL() would it block and wait until
the sentinel clears. This should mean being able to make the
change for both the old API and the new proposed sysdata API.
Likewise for other kernel_read_file*() users -- they'd benefit
from it as well.

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Luis Chamberlain Aug. 3, 2016, 4:03 p.m. UTC | #24

On Wed, Aug 03, 2016 at 12:42:14AM -0700, Dmitry Torokhov wrote:
> On Tue, Aug 2, 2016 at 12:41 AM, Luis R. Rodriguez <[e-mail]> wrote:
> > On Tue, Aug 02, 2016 at 08:53:55AM +0200, Daniel Wagner wrote:
> >> On 08/02/2016 08:34 AM, Luis R. Rodriguez wrote:
> >> >On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
> >> >>>The sysdata API's main goal rather is to provide a flexible API first,
> >> >>>compartamentalizing the usermode helper was secondary. But now it seems
> >> >>>I may just also add devm support too to help simplify code further.
> >> >>
> >> >>I missed the point that you plan to add usermode helper support to
> >> >>the sysdata API.
> >> >
> >> >I had no such plans, when I have asked folks so far about "hey are you
> >> >really in need for it, OK what for? " and "what extended uses do you
> >> >envision?" so I far I have not gotten any replies at all. So -- instead
> >> >sysdata currently ignores it.
> >>
> >> So you argue for the remoteproc use case with 100+ MB firmware that
> >> if there is a way to load after pivot_root() (or other additional
> >> firmware partition shows up) then there is no need at all for
> >> usermode helper?
> >
> > No, I'm saying I'd like to hear valid uses cases for the usermode helper and so
> > far I have only found using coccinelle grammar 2 explicit users, that's it. My
> > patch series (not yet merge) then annotates these as valid as I've verified
> > through their documentation they have some quirky requirement.
> 
> In certain configurations (embedded) people do not want to use
> initramfs nor modules nor embed firmware into the kernel. In this case
> usermode helper + firmware calss timeout handling provides necessary
> wait for the root filesystem to be mounted.
> 
> If we solve waiting for rootfs (or something else that may contain
> firmware) then these cases will not need to use usermode helper.

Given most distributions already disable FW_LOADER_USER_HELPER_FALLBACK and
grammar shows we only have 2 explicit users of the usermode helper I'd
prefer if we indeed could just compartamentalize the usermode helper
and not rely on it further. Furthermore I think its possible address
this issue, and suggested at least one idea how for now. With a bit
further review I'm in hope we can address this well, not only
for the firmware API but for all other kernel_read_file*() users.

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Dmitry Torokhov Aug. 3, 2016, 4:18 p.m. UTC | #25

On Wed, Aug 03, 2016 at 05:55:40PM +0200, Luis R. Rodriguez wrote:
> 
> I accept all help and would be glad to make enhancements instead of
> the old API through new API. The biggest thing here first I think is
> adding devm support, that I think should address what seemed to be
> the need to add more code for a transformation into the API. I'd

I am confused. Why do we need devm support, given that devm is only
valid in probe() paths[*] and we do know that we do not want to load
firmware in probe() paths because it may cause blocking?

[*] Yes, I know there are calls to devm* outside of probe() but I am
pretty sure they are buggy unless they explicitly freed with devm* as
well and then there is no point. IN all other cases it is likely wrong
as it messes up with order of freeing resources.

Thanks.

Luis Chamberlain Aug. 3, 2016, 5:37 p.m. UTC | #26

On Wed, Aug 03, 2016 at 09:18:21AM -0700, Dmitry Torokhov wrote:
> On Wed, Aug 03, 2016 at 05:55:40PM +0200, Luis R. Rodriguez wrote:
> > 
> > I accept all help and would be glad to make enhancements instead of
> > the old API through new API. The biggest thing here first I think is
> > adding devm support, that I think should address what seemed to be
> > the need to add more code for a transformation into the API. I'd
> 
> I am confused. Why do we need devm support, given that devm is only
> valid in probe() paths[*] and we do know that we do not want to load
> firmware in probe() paths because it may cause blocking?

Its a good point, I hadn't gone on to implement devm support on the sysdata API
yet here so this requirement was not known to me. This certainly would put a
limitation to the idea of using devm then to deal with the firmware for you,
given that not all users of firmware are on probe, and as you note we want to
by default avoid firmware calls on probe since init+probe are called serially
by default unless a driver is using the new async probe. Nevertheless, even if
we had userspace or the driver always asking for async probe, most users of the
firmware API are not on probe anyway, so the gains of using devm to help with
freeing the firmware for the driver on probe would be very limited.

With that in mind, in retrospect then the current sysdata approach to require a
callback for synchronous calls would seem to work around this issue and
generalize a solution given we'd have:

For the sync case:

const struct sysdata_file_desc sysdata_desc = {
	SYSDATA_DEFAULT_SYNC(driver_sync_req_cb, dev),
	.keep = false, /* not explicitly needed as default is false */
};
ret = sysdata_file_request();
...

Behind the scenes firmware_class would call driver_sync_req_cb(),
since that's where we know the firmware will be consumed and since
the driver has explicitly asked that it no longer needs to keep the
firmware around (keep == false), it will free it on behalf of the
driver.

Since current synchronous calls for firmware do not have a callback
this would mean a driver changing to the sysdata API if it wanted
to take advantage of this feature of letting firmware_class free
the firmware for you, you'd need a bit more code than before.

For the asynchronous case this is a bit different given that the
current async firmware API requires a callback, so if keep == false
on the async sysdata API we just remove the release_firmware()
calls when converting over.

Given this, other than the bikeshedding aspects [0] ("sysdata", "driver data",
"firmware), perhaps the sysdata API is done then.

[0] http://phk.freebsd.dk/sagas/bikeshed.html

> [*] Yes, I know there are calls to devm* outside of probe() but I am
> pretty sure they are buggy unless they explicitly freed with devm* as
> well and then there is no point.

Really ? If so that's good to know.. and it should mean grammar could
be used to hunt this down, specially since we have now some grammar
basics to help us check for calls on probe or init. On the grammar
we'd just only complain if a call was used not in a probe path.

> IN all other cases it is likely wrong
> as it messes up with order of freeing resources.

Good to know, thanks. Hopefully the above semantics of the driver
using keep should suffice. Which gets me to think, what if devm
had something similar to white-list uses outside of probe so that
if a keep (or another flag name) was set then its vetting that
the order of freeing of resources is understood and fine.

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Bjorn Andersson Aug. 3, 2016, 5:39 p.m. UTC | #27

On Tue 02 Aug 00:41 PDT 2016, Luis R. Rodriguez wrote:

> On Tue, Aug 02, 2016 at 08:53:55AM +0200, Daniel Wagner wrote:
> > On 08/02/2016 08:34 AM, Luis R. Rodriguez wrote:
> > >On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
> > >>>The sysdata API's main goal rather is to provide a flexible API first,
> > >>>compartamentalizing the usermode helper was secondary. But now it seems
> > >>>I may just also add devm support too to help simplify code further.
> > >>
> > >>I missed the point that you plan to add usermode helper support to
> > >>the sysdata API.
> > >
> > >I had no such plans, when I have asked folks so far about "hey are you
> > >really in need for it, OK what for? " and "what extended uses do you
> > >envision?" so I far I have not gotten any replies at all. So -- instead
> > >sysdata currently ignores it.
> > 
> > So you argue for the remoteproc use case with 100+ MB firmware that
> > if there is a way to load after pivot_root() (or other additional
> > firmware partition shows up) then there is no need at all for
> > usermode helper?
> 
> No, I'm saying I'd like to hear valid uses cases for the usermode helper and so
> far I have only found using coccinelle grammar 2 explicit users, that's it. My
> patch series (not yet merge) then annotates these as valid as I've verified
> through their documentation they have some quirky requirement.
> 

I think we're on the same page, but just to make sure; I do not want the
usermode helper, I only want a way to wait for the firmware files to
become available.

> Other than these two drivers I'd like hear to valid requirements for it.
> 
> The existential issue is a real issue but it does not look impossible to
> resolve. It may be a solution to bloat up the kernel with 100+ MB size just to
> stuff built-in firmware to avoid this issue, but it does not mean a solution
> is not possible.
> 
> Remind me -- why can remoteproc not stuff the firmware in initramfs ?
> 

RAM usage:
Storing the files in initramfs would consume 100MB RAM, we would then
allocate 100MB RAM for buffers during firmware loading and then we have
the reserved 100MB for the peripherals. The buffers could be easily be
removed with a mechanism for providing a buffer to the load operation,
but we would still double the RAM consumption.

Boot time:
Enlarging the kernel by 100MB will give noticeable addition to boot
times.

Development issues:
I have numerous concerns related to this, e.g. not being able to side
load the firmware files without rebuilding the initramfs. But most of
these are not technical issues, but rather a matter of convenience.

One large issue would be how to figure out how large to make the boot
partition in your Android phone, to cope with potential future growth in
firmware size - which has already proven to be a mess.

Legal matters:
Some of these firmware files are not redistributable, making it
impossible for end users to rebuild their kernel without loosing
functionality. There are even cases where these files are not allowed to
share partition with GPL binaries.


Most of these are just a major inconveniences to the developer but some
are show stoppers; especially the legal matters. So if we wave this off
as something people can live without then every downstream will sit on
their own solution to reimplement it.

> Anyway, here's a simple suggestion: fs/exec.c gets a sentinel file monitor
> support per enum kernel_read_file_id. For instance we'd have one for
> READING_FIRMWARE, one for READING_KEXEC_IMAGE, perhaps READING_POLICY, and this
> would in turn be used as the system configurable deterministic file for
> which to wait for to be present before enabling each enum kernel_read_file_id
> type read.
> 
> Thoughts ?

That does sound like a good generic solution for our problem and for the
other types of files as well. Do you have any ideas (patches?) on how
each sentinel would be triggered?

The only concern I can think of right now is that the
firmware_class.path might point to a separate partition; but based on
how the signaling of the sentinels are implemented this might not be an
issue.

Regards,
Bjorn
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Luis Chamberlain Aug. 3, 2016, 6:08 p.m. UTC | #28

On Wed, Aug 03, 2016 at 10:39:55AM -0700, Bjorn Andersson wrote:
> On Tue 02 Aug 00:41 PDT 2016, Luis R. Rodriguez wrote:
> 
> > On Tue, Aug 02, 2016 at 08:53:55AM +0200, Daniel Wagner wrote:
> > > On 08/02/2016 08:34 AM, Luis R. Rodriguez wrote:
> > > >On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
> > > >>>The sysdata API's main goal rather is to provide a flexible API first,
> > > >>>compartamentalizing the usermode helper was secondary. But now it seems
> > > >>>I may just also add devm support too to help simplify code further.
> > > >>
> > > >>I missed the point that you plan to add usermode helper support to
> > > >>the sysdata API.
> > > >
> > > >I had no such plans, when I have asked folks so far about "hey are you
> > > >really in need for it, OK what for? " and "what extended uses do you
> > > >envision?" so I far I have not gotten any replies at all. So -- instead
> > > >sysdata currently ignores it.
> > > 
> > > So you argue for the remoteproc use case with 100+ MB firmware that
> > > if there is a way to load after pivot_root() (or other additional
> > > firmware partition shows up) then there is no need at all for
> > > usermode helper?
> > 
> > No, I'm saying I'd like to hear valid uses cases for the usermode helper and so
> > far I have only found using coccinelle grammar 2 explicit users, that's it. My
> > patch series (not yet merge) then annotates these as valid as I've verified
> > through their documentation they have some quirky requirement.
> > 
> 
> I think we're on the same page, but just to make sure; I do not want the
> usermode helper,

Yay.

> I only want a way to wait for the firmware files to
> become available.

Sure.

> > Other than these two drivers I'd like hear to valid requirements for it.
> > 
> > The existential issue is a real issue but it does not look impossible to
> > resolve. It may be a solution to bloat up the kernel with 100+ MB size just to
> > stuff built-in firmware to avoid this issue, but it does not mean a solution
> > is not possible.
> > 
> > Remind me -- why can remoteproc not stuff the firmware in initramfs ?
> > 
> 
> RAM usage:
> Storing the files in initramfs would consume 100MB RAM, we would then
> allocate 100MB RAM for buffers during firmware loading and then we have
> the reserved 100MB for the peripherals. The buffers could be easily be
> removed with a mechanism for providing a buffer to the load operation,
> but we would still double the RAM consumption.
> 
> Boot time:
> Enlarging the kernel by 100MB will give noticeable addition to boot
> times.

Right I see.. Since we read the full kernel...

> Development issues:
> I have numerous concerns related to this, e.g. not being able to side
> load the firmware files without rebuilding the initramfs. But most of
> these are not technical issues, but rather a matter of convenience.
> 
> One large issue would be how to figure out how large to make the boot
> partition in your Android phone, to cope with potential future growth in
> firmware size - which has already proven to be a mess.
> 
> Legal matters:
> Some of these firmware files are not redistributable, making it
> impossible for end users to rebuild their kernel without loosing
> functionality. There are even cases where these files are not allowed to
> share partition with GPL binaries.
> 
> 
> Most of these are just a major inconveniences to the developer but some
> are show stoppers; especially the legal matters. So if we wave this off
> as something people can live without then every downstream will sit on
> their own solution to reimplement it.

Thanks I'll document these into the firmware_class.

> > Anyway, here's a simple suggestion: fs/exec.c gets a sentinel file monitor
> > support per enum kernel_read_file_id. For instance we'd have one for
> > READING_FIRMWARE, one for READING_KEXEC_IMAGE, perhaps READING_POLICY, and this
> > would in turn be used as the system configurable deterministic file for
> > which to wait for to be present before enabling each enum kernel_read_file_id
> > type read.
> > 
> > Thoughts ?
> 
> That does sound like a good generic solution for our problem and for the
> other types of files as well. Do you have any ideas (patches?) on how
> each sentinel would be triggered?
> 
> The only concern I can think of right now is that the
> firmware_class.path might point to a separate partition; but based on
> how the signaling of the sentinels are implemented this might not be an
> issue.

There's another simpler suggestion I'm getting too, will post in the other
thread.

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Luis Chamberlain Aug. 3, 2016, 6:40 p.m. UTC | #29

On Wed, Aug 03, 2016 at 05:55:40PM +0200, Luis R. Rodriguez wrote:
> On Wed, Aug 03, 2016 at 08:57:09AM +0200, Daniel Wagner wrote:
> > On 08/02/2016 09:41 AM, Luis R. Rodriguez wrote:
> > >On Tue, Aug 02, 2016 at 08:53:55AM +0200, Daniel Wagner wrote:
> > >>On 08/02/2016 08:34 AM, Luis R. Rodriguez wrote:
> > >>>On Tue, Aug 02, 2016 at 07:49:19AM +0200, Daniel Wagner wrote:
> > >>So you argue for the remoteproc use case with 100+ MB firmware that
> > >>if there is a way to load after pivot_root() (or other additional
> > >>firmware partition shows up) then there is no need at all for
> > >>usermode helper?
> > >
> > >No, I'm saying I'd like to hear valid uses cases for the usermode helper and so
> > >far I have only found using coccinelle grammar 2 explicit users, that's it. My
> > >patch series (not yet merge) then annotates these as valid as I've verified
> > >through their documentation they have some quirky requirement.
> > 
> > I got that question wrong. It should read something like 'for the
> > remoteproc 100+MB there is no need for the user help?'.
> 
> That's not a question for me but for those who say that the usermode helper
> is needed for remoteproc, so far from what folks are saying it seems the only
> reason for the usermodehelper was to try to avoid the deterministic issue,
> but I suggested a way to resolve that without the usermode helper now so
> would be curious to hear if there are any more reasons for it.
> 
> > I've gone
> > through your patches and they make perfectly sense too. Maybe I can
> > convince you to take a better version of my patch 3 into your queue.
> > And I help you converting the exiting drivers. Obviously if you like
> > my help at all.
> 
> I accept all help and would be glad to make enhancements instead of
> the old API through new API. The biggest thing here first I think is
> adding devm support, that I think should address what seemed to be
> the need to add more code for a transformation into the API. I'd
> personally only want to add that and be done with an introduction
> of the sysdata API. Further changes IMHO are best done atomically
> after that on top of it, but I'm happy to queue in the changes.
> 
> > >Other than these two drivers I'd like hear to valid requirements for it.
> > >
> > >The existential issue is a real issue but it does not look impossible to
> > >resolve. It may be a solution to bloat up the kernel with 100+ MB size just to
> > >stuff built-in firmware to avoid this issue, but it does not mean a solution
> > >is not possible.
> > >
> > >Remind me -- why can remoteproc not stuff the firmware in initramfs ?
> > 
> > I don't know. I was just bringing it up with the hope that Bjorn
> > will defend it. It seems my tactics didn't work out :)
> 
> OK.
> 
> > >Anyway, here's a simple suggestion: fs/exec.c gets a sentinel file monitor
> > >support per enum kernel_read_file_id. For instance we'd have one for
> > >READING_FIRMWARE, one for READING_KEXEC_IMAGE, perhaps READING_POLICY, and this
> > >would in turn be used as the system configurable deterministic file for
> > >which to wait for to be present before enabling each enum kernel_read_file_id
> > >type read.
> > >
> > >Thoughts ?
> > 
> > Not sure if I get you here correctly. Is the 'system configurable
> > deterministic file' is a knob which controlled by user space? Or it
> > this something you define at compile time?
> 
> I meant at compile time on the kernel. So CONFIG_READ_READY_SENTINEL
> or something like this, and it be a string, which if set then when
> the kernel read APIs are used, then a new API could be introduced
> that would *only* enable reading through once that sentinel has
> been detected by the kernel to allowed through reads. Doing this
> per mount / target filesystem is rather cumbersome given possible
> overlaps in mounts and also pivot_root() being possible, so instead
> targeting simply the fs/exec.c enum kernel_read_file_id would seem
> more efficient and clean but we would need a decided upon set of
> paths per enum kernel_read_file_id as base (or just one path per
> enum kernel_read_file_id). For number of paths I mean the number
> of target directories to look for the sentinel per enum kernel_read_file_id,
> so for instance for READING_FIRMWARE perhaps just deciding on /lib/firmware/
> would suffice, but if this supported multiple paths another option may be
> for the sentinel to also be looked for in /lib/firmware/updates/,
> /lib/firmware/" UTS_RELEASE -- etc. It would *stop* after finding one
> sentinel on any of these paths.
> 
> If a system has has CONFIG_READ_READY_SENTINEL it would mean an agreed upon
> system configuration has been decided so that at any point in time reads
> against READING_FIRMWARE using a new kernel_read_file_from_path_sentinel()
> (or something like it) would only allow the read to go through once
> the sentinel has been found for READING_FIRMWARE on the agreed upon
> paths.
> 
> The benefit of the sentintel approach is it avoids complexities with
> pivot_root(), and makes the deterministic aspect of the target left
> only to a system-configuration enabled target path / file.
> 
> This is just an idea. I'd like some FS folks to review.
> 
> > Hmm, so it would allow to decided to ask a userspace helper or load
> > the firmware directly (to be more precised the kernel_read_file_id
> > type). If yes, than it is what currently already have just
> > integrated nicely into the new sysdata API.
> 
> Sorry, no, the above description is better of what I meant. This
> actually would not need to go into the sysdata API, unless of
> course we wanted it just as a new "feature" of it, but I don't
> think that's needed unless it has some implications behind the
> scenes. Given that firmware_class now uses a common core kernel
> API for reading files kernel_read_file_from_path() we could
> for instance add kernel_read_file_from_sentintel() and only
> if CONFIG_READ_READY_SENTINEL() would it block and wait until
> the sentinel clears. This should mean being able to make the
> change for both the old API and the new proposed sysdata API.
> Likewise for other kernel_read_file*() users -- they'd benefit
> from it as well.

A file sentinel would implicate a file namespace thing being used on the
filesystem -- to me this just means the Linux distribution / system integrator
would add this per filesystem, but agree this is pretty hacky.  Furthermore
we'd wait forever if the Linux distribution / system integrator forgot to set
the sentinel file. That's not good. To avoid that a generic "root fs ready"
event could be sent from userspace to know when to clear stale reads... but if
that's going to be done best just replace all sentintels with a simple "root fs
ready" which would mean all reads from the kernel are ready. If we wanted
further granularity I suppose we could further have one event per enum
kernel_read_file_id, and a generic all-is-ready one.

To start off with then a simple event from userspace should suffice. But do keep
in mind that granularity might help given that a big iron system might have some
large array of disks to mount during bootup and that may take a long while, and
you likely want to read /lib/firmware way before that filesystem is ready.

Not sure if granularity fixated by enum kernel_read_file_id should suffice,
perhaps given its also enough for LSMs...

This indeed would mean a kernelspace and userspace change, but it would mean 
not having to deal with the usermode helper crap anymore.

Anyway -- these are just ideas, patches welcomed !

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Arend Van Spriel Aug. 3, 2016, 8:42 p.m. UTC | #30

On 03-08-16 17:35, Dmitry Torokhov wrote:
>> In my opinion the kernel should provide functionality to user-space and
>> > user-space providing functionality to the kernel should be avoided.
> Why? We have bunch of stuff running in userspace for the kernel. Fuse
> for example. I am sure there are more.

To me "running in user-space" is not the same as providing
functionality, but I see your point given below.

>> > 
>>> > > If we solve waiting for rootfs (or something else that may contain
>>> > > firmware) then these cases will not need to use usermode helper.
>> > 
>> > If firmware (or whatever) API could get notification of mount syscall it
>> > could be used to retry firmware loading instead of periodic polling.
>> > That leaves the question raised by you about when to stop trying. The
>> > initlevel stuff is probably a user-space only concept, right? So no
>> > ideas how the kernel itself could decide except for a "long" timeout.
> The kernel really does not know, it can only guess. The firmware may get
> delivered by motorized carrier pidgeons. But distribution does know how
> they set up, so they are in position to tell the kernel "go" or "give
> up".

What distro employs pidgeons. Like to give it a spin ;-)

Maybe the latest idea from Luis is a viable option.

Regards,
Arend
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Bjorn Andersson Aug. 3, 2016, 10:26 p.m. UTC | #31

On Wed 03 Aug 08:55 PDT 2016, Luis R. Rodriguez wrote:

> On Wed, Aug 03, 2016 at 08:57:09AM +0200, Daniel Wagner wrote:
> > On 08/02/2016 09:41 AM, Luis R. Rodriguez wrote:
[..]
> > Not sure if I get you here correctly. Is the 'system configurable
> > deterministic file' is a knob which controlled by user space? Or it
> > this something you define at compile time?
> 
> I meant at compile time on the kernel. So CONFIG_READ_READY_SENTINEL
> or something like this, and it be a string, which if set then when
> the kernel read APIs are used, then a new API could be introduced
> that would *only* enable reading through once that sentinel has
> been detected by the kernel to allowed through reads. Doing this
> per mount / target filesystem is rather cumbersome given possible
> overlaps in mounts and also pivot_root() being possible, so instead
> targeting simply the fs/exec.c enum kernel_read_file_id would seem
> more efficient and clean but we would need a decided upon set of
> paths per enum kernel_read_file_id as base (or just one path per
> enum kernel_read_file_id). For number of paths I mean the number
> of target directories to look for the sentinel per enum kernel_read_file_id,
> so for instance for READING_FIRMWARE perhaps just deciding on /lib/firmware/
> would suffice, but if this supported multiple paths another option may be
> for the sentinel to also be looked for in /lib/firmware/updates/,
> /lib/firmware/" UTS_RELEASE -- etc. It would *stop* after finding one
> sentinel on any of these paths.
> 
> If a system has has CONFIG_READ_READY_SENTINEL it would mean an agreed upon
> system configuration has been decided so that at any point in time reads
> against READING_FIRMWARE using a new kernel_read_file_from_path_sentinel()
> (or something like it) would only allow the read to go through once
> the sentinel has been found for READING_FIRMWARE on the agreed upon
> paths.
> 
> The benefit of the sentintel approach is it avoids complexities with
> pivot_root(), and makes the deterministic aspect of the target left
> only to a system-configuration enabled target path / file.
> 

This sounds reasonable, it could be configured to wait for a certain
static file or userspace could generate this file once it reaches some
checkpoint.


Just to provide some additional input to "will rootfs mounted be enough
of a sentinel".

In an Android device you have a initramfs that will read an fstab file
and mount /system that holds most firmware, for some platforms
additional firmware will come from a third partition (in the Qualcomm
case mounted in /persist by the same mechanism).

With the sentinel approach one could configure the system either point
it at a file in the last file system to be mounted or have init generate
a file once its done with this; or in the generic configuration just
wait for /lib/firmware to show up.

I like this approach.

> This is just an idea. I'd like some FS folks to review.
> 

Regards,
Bjorn
--
To unsubscribe from this list: send the line "unsubscribe linux-wireless" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch

diff --git a/drivers/input/misc/ims-pcu.c b/drivers/input/misc/ims-pcu.c
index 9c0ea36..cda1fbf 100644
--- a/drivers/input/misc/ims-pcu.c
+++ b/drivers/input/misc/ims-pcu.c
@@ -109,7 +109,7 @@  struct ims_pcu {
 
 	u32 fw_start_addr;
 	u32 fw_end_addr;
-	struct completion async_firmware_done;
+	struct firmware_stat fw_st;
 
 	struct ims_pcu_buttons buttons;
 	struct ims_pcu_gamepad *gamepad;
@@ -940,7 +940,7 @@  static void ims_pcu_process_async_firmware(const struct firmware *fw,
 	release_firmware(fw);
 
 out:
-	complete(&pcu->async_firmware_done);
+	fw_loading_done(pcu->fw_st);
 }
 
 /*********************************************************************
@@ -1967,7 +1967,7 @@  static int ims_pcu_init_bootloader_mode(struct ims_pcu *pcu)
 					ims_pcu_process_async_firmware);
 	if (error) {
 		/* This error is not fatal, let userspace have another chance */
-		complete(&pcu->async_firmware_done);
+		fw_loading_abort(pcu->fw_st);
 	}
 
 	return 0;
@@ -1976,7 +1976,7 @@  static int ims_pcu_init_bootloader_mode(struct ims_pcu *pcu)
 static void ims_pcu_destroy_bootloader_mode(struct ims_pcu *pcu)
 {
 	/* Make sure our initial firmware request has completed */
-	wait_for_completion(&pcu->async_firmware_done);
+	fw_loading_wait(pcu->fw_st);
 }
 
 #define IMS_PCU_APPLICATION_MODE	0
@@ -2000,7 +2000,7 @@  static int ims_pcu_probe(struct usb_interface *intf,
 	pcu->bootloader_mode = id->driver_info == IMS_PCU_BOOTLOADER_MODE;
 	mutex_init(&pcu->cmd_mutex);
 	init_completion(&pcu->cmd_done);
-	init_completion(&pcu->async_firmware_done);
+	firmware_stat_init(&pcu->fw_st);
 
 	error = ims_pcu_parse_cdc_data(intf, pcu);
 	if (error)