[1/2] block: export __blk_complete_request
diff mbox

Message ID CACVXFVMPAdFbX3x-KUjHCQhb_rT6p9ROYHg63aUtEqdbitEaKQ@mail.gmail.com
State New
Headers show

Commit Message

Ming Lei June 15, 2018, 3:20 a.m. UTC
On Fri, Jun 15, 2018 at 11:04 AM, jianchao.wang
<jianchao.w.wang@oracle.com> wrote:
> Hi Ming
>
> Thanks for your kindly response.
>
> On 06/15/2018 10:56 AM, Ming Lei wrote:
>>>> IMO, ref-counter is just to fix the blk-mq req life recycle issue.
>>>> It cannot replace the blk_mark_rq_complete which could avoid the race between
>>>> timeout and io completion path.
>>> The .timeout return BLK_EH_DONE doesn't always mean the request has been completed.
>>> Such as scsi-mid layer, its .timeout callback return BLK_EH_DONE but the timed out
>>> request is still in abort or eh process. What if a completion irq come during that ?
>> For blk-mq, it is avoided by the atomic state change in
>> __blk_mq_complete_request(),
>> that is why I mentioned the question in my last reply.
>>
>
> but blk_mq_check_expired doesn't do that.
> do I miss anything ?

Right, that is the difference between blk-mq and legacy now, then if scsi-mq
drivers can work well, they should work well with the following change in the
non-mq mode:

                *next_set = 1;


Thanks,
Ming Lei

Comments

jianchao.wang June 15, 2018, 3:26 a.m. UTC | #1
Hi Ming

On 06/15/2018 11:20 AM, Ming Lei wrote:
> On Fri, Jun 15, 2018 at 11:04 AM, jianchao.wang
> <jianchao.w.wang@oracle.com> wrote:
>> Hi Ming
>>
>> Thanks for your kindly response.
>>
>> On 06/15/2018 10:56 AM, Ming Lei wrote:
>>>>> IMO, ref-counter is just to fix the blk-mq req life recycle issue.
>>>>> It cannot replace the blk_mark_rq_complete which could avoid the race between
>>>>> timeout and io completion path.
>>>> The .timeout return BLK_EH_DONE doesn't always mean the request has been completed.
>>>> Such as scsi-mid layer, its .timeout callback return BLK_EH_DONE but the timed out
>>>> request is still in abort or eh process. What if a completion irq come during that ?
>>> For blk-mq, it is avoided by the atomic state change in
>>> __blk_mq_complete_request(),
>>> that is why I mentioned the question in my last reply.
>>>
>>
>> but blk_mq_check_expired doesn't do that.
>> do I miss anything ?
> 
> Right, that is the difference between blk-mq and legacy now, 

Sorry, I cannot follow your point.
blk_mq_check_expired doesn't do a atmoc state change from IN-FLIGHT to COMPLETE.
__blk_mq_complete_request could still proceed to complete a timed out request
which is in scsi abort or eh process. Is it really OK ?


Thanks
Jianchao


> then if scsi-mq
> drivers can work well, they should work well with the following change in the
> non-mq mode:
> 
> diff --git a/block/blk-timeout.c b/block/blk-timeout.c
> index 4b8a48d48ba1..9fce09d55652 100644
> --- a/block/blk-timeout.c
> +++ b/block/blk-timeout.c
> @@ -88,7 +88,6 @@ static void blk_rq_timed_out(struct request *req)
>         switch (ret) {
>         case BLK_EH_RESET_TIMER:
>                 blk_add_timer(req);
> -               blk_clear_rq_complete(req);
>                 break;
>         case BLK_EH_DONE:
>                 /*
> @@ -115,8 +114,7 @@ static void blk_rq_check_expired(struct request
> *rq, unsigned long *next_timeout
>                 /*
>                  * Check if we raced with end io completion
>                  */
> -               if (!blk_mark_rq_complete(rq))
> -                       blk_rq_timed_out(rq);
> +               blk_rq_timed_out(rq);
>         } else if (!*next_set || time_after(*next_timeout, deadline)) {
>                 *next_timeout = deadline;
>                 *next_set = 1;
> 
> 
> Thanks,
> Ming Lei
>
Ming Lei June 15, 2018, 4:03 a.m. UTC | #2
On Fri, Jun 15, 2018 at 11:26 AM, jianchao.wang
<jianchao.w.wang@oracle.com> wrote:
> Hi Ming
>
> On 06/15/2018 11:20 AM, Ming Lei wrote:
>> On Fri, Jun 15, 2018 at 11:04 AM, jianchao.wang
>> <jianchao.w.wang@oracle.com> wrote:
>>> Hi Ming
>>>
>>> Thanks for your kindly response.
>>>
>>> On 06/15/2018 10:56 AM, Ming Lei wrote:
>>>>>> IMO, ref-counter is just to fix the blk-mq req life recycle issue.
>>>>>> It cannot replace the blk_mark_rq_complete which could avoid the race between
>>>>>> timeout and io completion path.
>>>>> The .timeout return BLK_EH_DONE doesn't always mean the request has been completed.
>>>>> Such as scsi-mid layer, its .timeout callback return BLK_EH_DONE but the timed out
>>>>> request is still in abort or eh process. What if a completion irq come during that ?
>>>> For blk-mq, it is avoided by the atomic state change in
>>>> __blk_mq_complete_request(),
>>>> that is why I mentioned the question in my last reply.
>>>>
>>>
>>> but blk_mq_check_expired doesn't do that.
>>> do I miss anything ?
>>
>> Right, that is the difference between blk-mq and legacy now,
>
> Sorry, I cannot follow your point.
> blk_mq_check_expired doesn't do a atmoc state change from IN-FLIGHT to COMPLETE.
> __blk_mq_complete_request could still proceed to complete a timed out request
> which is in scsi abort or eh process. Is it really OK ?

That is the idea of Christoph's patchset of 'complete requests from ->timeout',
then drivers need to cover race between timeout and normal completeion.

But at least the request won't be completed twice because of the atomic
state change in __blk_mq_complete_request().

So what is your real concern about blk-mq's timeout?

Thanks,
Ming
jianchao.wang June 15, 2018, 5:10 a.m. UTC | #3
Hi Ming

Thanks for your kindly response

On 06/15/2018 12:03 PM, Ming Lei wrote:
> On Fri, Jun 15, 2018 at 11:26 AM, jianchao.wang
> <jianchao.w.wang@oracle.com> wrote:
>> Hi Ming
>>
>> On 06/15/2018 11:20 AM, Ming Lei wrote:
>>> On Fri, Jun 15, 2018 at 11:04 AM, jianchao.wang
>>> <jianchao.w.wang@oracle.com> wrote:
>>>> Hi Ming
>>>>
>>>> Thanks for your kindly response.
>>>>
>>>> On 06/15/2018 10:56 AM, Ming Lei wrote:
>>>>>>> IMO, ref-counter is just to fix the blk-mq req life recycle issue.
>>>>>>> It cannot replace the blk_mark_rq_complete which could avoid the race between
>>>>>>> timeout and io completion path.
>>>>>> The .timeout return BLK_EH_DONE doesn't always mean the request has been completed.
>>>>>> Such as scsi-mid layer, its .timeout callback return BLK_EH_DONE but the timed out
>>>>>> request is still in abort or eh process. What if a completion irq come during that ?
>>>>> For blk-mq, it is avoided by the atomic state change in
>>>>> __blk_mq_complete_request(),
>>>>> that is why I mentioned the question in my last reply.
>>>>>
>>>>
>>>> but blk_mq_check_expired doesn't do that.
>>>> do I miss anything ?
>>>
>>> Right, that is the difference between blk-mq and legacy now,
>>
>> Sorry, I cannot follow your point.
>> blk_mq_check_expired doesn't do a atmoc state change from IN-FLIGHT to COMPLETE.
>> __blk_mq_complete_request could still proceed to complete a timed out request
>> which is in scsi abort or eh process. Is it really OK ?
> > That is the idea of Christoph's patchset of 'complete requests from ->timeout',

Yes, I used to read that mail thread.

> then drivers need to cover race between timeout and normal completeion.
> 
> But at least the request won't be completed twice because of the atomic
> state change in __blk_mq_complete_request().

Yes

> 
> So what is your real concern about blk-mq's timeout?
I concern whether the current drivers have bee ready for taking this task currently.
At least, for scsi, if I try to trigger timeout and completion path concurrently, system would crash.
4.17.rc7 or 4.18 with a patch that change state in blk_mq_check_expired will survive.

Thanks
jianchao 

> 
> Thanks,
> Ming
>
Christoph Hellwig June 15, 2018, 11:58 a.m. UTC | #4
On Fri, Jun 15, 2018 at 11:20:40AM +0800, Ming Lei wrote:
> > but blk_mq_check_expired doesn't do that.
> > do I miss anything ?
> 
> Right, that is the difference between blk-mq and legacy now, then if scsi-mq
> drivers can work well, they should work well with the following change in the
> non-mq mode:

We'll still need referene counting against reuse and/or premature
freeing of requests.

Patch
diff mbox

diff --git a/block/blk-timeout.c b/block/blk-timeout.c
index 4b8a48d48ba1..9fce09d55652 100644
--- a/block/blk-timeout.c
+++ b/block/blk-timeout.c
@@ -88,7 +88,6 @@  static void blk_rq_timed_out(struct request *req)
        switch (ret) {
        case BLK_EH_RESET_TIMER:
                blk_add_timer(req);
-               blk_clear_rq_complete(req);
                break;
        case BLK_EH_DONE:
                /*
@@ -115,8 +114,7 @@  static void blk_rq_check_expired(struct request
*rq, unsigned long *next_timeout
                /*
                 * Check if we raced with end io completion
                 */
-               if (!blk_mark_rq_complete(rq))
-                       blk_rq_timed_out(rq);
+               blk_rq_timed_out(rq);
        } else if (!*next_set || time_after(*next_timeout, deadline)) {
                *next_timeout = deadline;