diff mbox series

lightnvm: pblk: adjust the position of the lock

Message ID 20181127015309.2374-1-suhua.tanke@gmail.com
State Superseded
Headers show
Series lightnvm: pblk: adjust the position of the lock | expand

Commit Message

Hua Su Nov. 27, 2018, 1:53 a.m. UTC
Add lock protection for list operations.

Signed-off-by: Hua Su <suhua.tanke@gmail.com>
---
 drivers/lightnvm/pblk-core.c | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

Comments

Javier Gonzalez Nov. 27, 2018, 12:57 p.m. UTC | #1
> On 27 Nov 2018, at 02.53, Hua Su <suhua.tanke@gmail.com> wrote:
> 
> Add lock protection for list operations.
> Signed-off-by: Hua Su <suhua.tanke@gmail.com>
> ---
> drivers/lightnvm/pblk-core.c | 17 ++++++++++-------
> 1 file changed, 10 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/lightnvm/pblk-core.c b/drivers/lightnvm/pblk-core.c
> index 6944aac43b01..e490df217dac 100644
> --- a/drivers/lightnvm/pblk-core.c
> +++ b/drivers/lightnvm/pblk-core.c
> @@ -1286,24 +1286,27 @@ int pblk_line_recov_alloc(struct pblk *pblk, struct pblk_line *line)
> 	list_del(&line->list);
> 
> 	ret = pblk_line_prepare(pblk, line);
> -	if (ret) {
> -		list_add(&line->list, &l_mg->free_list);
> -		spin_unlock(&l_mg->free_lock);
> -		return ret;
> -	}
> -	spin_unlock(&l_mg->free_lock);
> +	if (ret)
> +		goto out;
> 
> 	ret = pblk_line_alloc_bitmaps(pblk, line);
> 	if (ret)
> -		return ret;
> +		goto out;
> 
> 	if (!pblk_line_init_bb(pblk, line, 0)) {
> 		list_add(&line->list, &l_mg->free_list);
> +		spin_unlock(&l_mg->free_lock);
> 		return -EINTR;
> 	}
> +	spin_unlock(&l_mg->free_lock);
> 
> 	pblk_rl_free_lines_dec(&pblk->rl, line, true);
> 	return 0;
> +
> +out:
> +	list_add(&line->list, &l_mg->free_list);
> +	spin_unlock(&l_mg->free_lock);
> +	return ret;
> }
> 
> void pblk_line_recov_close(struct pblk *pblk, struct pblk_line *line)
> --
> 2.19.1

This path is only touched by the recovery path, which is single
threaded, so there is no race condition as is. Also, if recovery fails,
pblk will not create the instance at all. This said, it would be
good to protect the list_add on the pblk_line_init_bb() error path in
case this code is used for some other purpose in the future.

However, the implementation you propose has some issues due to the
extended critical zone, which covers pblk_line_alloc_bitmaps(). Here,
allocations are intentionally non-atomic (i.e., GFP_KERNEL).

You can instead do the following:

diff --git i/drivers/lightnvm/pblk-core.c w/drivers/lightnvm/pblk-core.c
index 615817bf97e3..02c4c1708c45 100644
--- i/drivers/lightnvm/pblk-core.c
+++ w/drivers/lightnvm/pblk-core.c
@@ -1301,15 +1301,22 @@ int pblk_line_recov_alloc(struct pblk *pblk, struct pblk_line *line)

        ret = pblk_line_alloc_bitmaps(pblk, line);
        if (ret)
-               return ret;
+               goto fail;

        if (!pblk_line_init_bb(pblk, line, 0)) {
-               list_add(&line->list, &l_mg->free_list);
-               return -EINTR;
+               ret = -EINTR;
+               goto fail;
        }

        pblk_rl_free_lines_dec(&pblk->rl, line, true);
        return 0;
+
+fail:
+       spin_lock(&l_mg->free_lock);
+       list_add(&line->list, &l_mg->free_list);
+       spin_unlock(&l_mg->free_lock);
+
+       return ret;
 }

 void pblk_line_recov_close(struct pblk *pblk, struct pblk_line *line)

Javier
Matias Bjørling Nov. 27, 2018, 2:22 p.m. UTC | #2
On 11/27/2018 01:57 PM, Javier Gonzalez wrote:
> 
>> On 27 Nov 2018, at 02.53, Hua Su <suhua.tanke@gmail.com> wrote:
>>
>> Add lock protection for list operations.
>> Signed-off-by: Hua Su <suhua.tanke@gmail.com>
>> ---
>> drivers/lightnvm/pblk-core.c | 17 ++++++++++-------
>> 1 file changed, 10 insertions(+), 7 deletions(-)
>>
>> diff --git a/drivers/lightnvm/pblk-core.c b/drivers/lightnvm/pblk-core.c
>> index 6944aac43b01..e490df217dac 100644
>> --- a/drivers/lightnvm/pblk-core.c
>> +++ b/drivers/lightnvm/pblk-core.c
>> @@ -1286,24 +1286,27 @@ int pblk_line_recov_alloc(struct pblk *pblk, struct pblk_line *line)
>> 	list_del(&line->list);
>>
>> 	ret = pblk_line_prepare(pblk, line);
>> -	if (ret) {
>> -		list_add(&line->list, &l_mg->free_list);
>> -		spin_unlock(&l_mg->free_lock);
>> -		return ret;
>> -	}
>> -	spin_unlock(&l_mg->free_lock);
>> +	if (ret)
>> +		goto out;
>>
>> 	ret = pblk_line_alloc_bitmaps(pblk, line);
>> 	if (ret)
>> -		return ret;
>> +		goto out;
>>
>> 	if (!pblk_line_init_bb(pblk, line, 0)) {
>> 		list_add(&line->list, &l_mg->free_list);
>> +		spin_unlock(&l_mg->free_lock);
>> 		return -EINTR;
>> 	}
>> +	spin_unlock(&l_mg->free_lock);
>>
>> 	pblk_rl_free_lines_dec(&pblk->rl, line, true);
>> 	return 0;
>> +
>> +out:
>> +	list_add(&line->list, &l_mg->free_list);
>> +	spin_unlock(&l_mg->free_lock);
>> +	return ret;
>> }
>>
>> void pblk_line_recov_close(struct pblk *pblk, struct pblk_line *line)
>> --
>> 2.19.1
> 
> This path is only touched by the recovery path, which is single
> threaded, so there is no race condition as is. Also, if recovery fails,
> pblk will not create the instance at all. This said, it would be
> good to protect the list_add on the pblk_line_init_bb() error path in
> case this code is used for some other purpose in the future.

I like your explanation here. Another option is that we could add a 
comment to notify the developer that it safe in this context?

> 
> However, the implementation you propose has some issues due to the
> extended critical zone, which covers pblk_line_alloc_bitmaps(). Here,
> allocations are intentionally non-atomic (i.e., GFP_KERNEL).
> 
> You can instead do the following:
> 
> diff --git i/drivers/lightnvm/pblk-core.c w/drivers/lightnvm/pblk-core.c
> index 615817bf97e3..02c4c1708c45 100644
> --- i/drivers/lightnvm/pblk-core.c
> +++ w/drivers/lightnvm/pblk-core.c
> @@ -1301,15 +1301,22 @@ int pblk_line_recov_alloc(struct pblk *pblk, struct pblk_line *line)
> 
>          ret = pblk_line_alloc_bitmaps(pblk, line);
>          if (ret)
> -               return ret;
> +               goto fail;
> 
>          if (!pblk_line_init_bb(pblk, line, 0)) {
> -               list_add(&line->list, &l_mg->free_list);
> -               return -EINTR;
> +               ret = -EINTR;
> +               goto fail;
>          }
> 
>          pblk_rl_free_lines_dec(&pblk->rl, line, true);
>          return 0;
> +
> +fail:
> +       spin_lock(&l_mg->free_lock);
> +       list_add(&line->list, &l_mg->free_list);
> +       spin_unlock(&l_mg->free_lock);
> +
> +       return ret;
>   }
> 
>   void pblk_line_recov_close(struct pblk *pblk, struct pblk_line *line)
> 
> Javier
> 
>
Javier Gonzalez Nov. 27, 2018, 2:24 p.m. UTC | #3
> On 27 Nov 2018, at 15.22, Matias Bjørling <mb@lightnvm.io> wrote:
> 
> On 11/27/2018 01:57 PM, Javier Gonzalez wrote:
>>> On 27 Nov 2018, at 02.53, Hua Su <suhua.tanke@gmail.com> wrote:
>>> 
>>> Add lock protection for list operations.
>>> Signed-off-by: Hua Su <suhua.tanke@gmail.com>
>>> ---
>>> drivers/lightnvm/pblk-core.c | 17 ++++++++++-------
>>> 1 file changed, 10 insertions(+), 7 deletions(-)
>>> 
>>> diff --git a/drivers/lightnvm/pblk-core.c b/drivers/lightnvm/pblk-core.c
>>> index 6944aac43b01..e490df217dac 100644
>>> --- a/drivers/lightnvm/pblk-core.c
>>> +++ b/drivers/lightnvm/pblk-core.c
>>> @@ -1286,24 +1286,27 @@ int pblk_line_recov_alloc(struct pblk *pblk, struct pblk_line *line)
>>>    list_del(&line->list);
>>> 
>>>    ret = pblk_line_prepare(pblk, line);
>>> -    if (ret) {
>>> -        list_add(&line->list, &l_mg->free_list);
>>> -        spin_unlock(&l_mg->free_lock);
>>> -        return ret;
>>> -    }
>>> -    spin_unlock(&l_mg->free_lock);
>>> +    if (ret)
>>> +        goto out;
>>> 
>>>    ret = pblk_line_alloc_bitmaps(pblk, line);
>>>    if (ret)
>>> -        return ret;
>>> +        goto out;
>>> 
>>>    if (!pblk_line_init_bb(pblk, line, 0)) {
>>>        list_add(&line->list, &l_mg->free_list);
>>> +        spin_unlock(&l_mg->free_lock);
>>>        return -EINTR;
>>>    }
>>> +    spin_unlock(&l_mg->free_lock);
>>> 
>>>    pblk_rl_free_lines_dec(&pblk->rl, line, true);
>>>    return 0;
>>> +
>>> +out:
>>> +    list_add(&line->list, &l_mg->free_list);
>>> +    spin_unlock(&l_mg->free_lock);
>>> +    return ret;
>>> }
>>> 
>>> void pblk_line_recov_close(struct pblk *pblk, struct pblk_line *line)
>>> --
>>> 2.19.1
>> This path is only touched by the recovery path, which is single
>> threaded, so there is no race condition as is. Also, if recovery fails,
>> pblk will not create the instance at all. This said, it would be
>> good to protect the list_add on the pblk_line_init_bb() error path in
>> case this code is used for some other purpose in the future.
> 
> I like your explanation here. Another option is that we could add a comment to notify the developer that it safe in this context?

Sure. Do you want to add it? Or should I send it?

Javier
Matias Bjørling Nov. 27, 2018, 2:26 p.m. UTC | #4
On 11/27/2018 03:24 PM, Javier Gonzalez wrote:
> 
>> On 27 Nov 2018, at 15.22, Matias Bjørling <mb@lightnvm.io> wrote:
>>
>> On 11/27/2018 01:57 PM, Javier Gonzalez wrote:
>>>> On 27 Nov 2018, at 02.53, Hua Su <suhua.tanke@gmail.com> wrote:
>>>>
>>>> Add lock protection for list operations.
>>>> Signed-off-by: Hua Su <suhua.tanke@gmail.com>
>>>> ---
>>>> drivers/lightnvm/pblk-core.c | 17 ++++++++++-------
>>>> 1 file changed, 10 insertions(+), 7 deletions(-)
>>>>
>>>> diff --git a/drivers/lightnvm/pblk-core.c b/drivers/lightnvm/pblk-core.c
>>>> index 6944aac43b01..e490df217dac 100644
>>>> --- a/drivers/lightnvm/pblk-core.c
>>>> +++ b/drivers/lightnvm/pblk-core.c
>>>> @@ -1286,24 +1286,27 @@ int pblk_line_recov_alloc(struct pblk *pblk, struct pblk_line *line)
>>>>     list_del(&line->list);
>>>>
>>>>     ret = pblk_line_prepare(pblk, line);
>>>> -    if (ret) {
>>>> -        list_add(&line->list, &l_mg->free_list);
>>>> -        spin_unlock(&l_mg->free_lock);
>>>> -        return ret;
>>>> -    }
>>>> -    spin_unlock(&l_mg->free_lock);
>>>> +    if (ret)
>>>> +        goto out;
>>>>
>>>>     ret = pblk_line_alloc_bitmaps(pblk, line);
>>>>     if (ret)
>>>> -        return ret;
>>>> +        goto out;
>>>>
>>>>     if (!pblk_line_init_bb(pblk, line, 0)) {
>>>>         list_add(&line->list, &l_mg->free_list);
>>>> +        spin_unlock(&l_mg->free_lock);
>>>>         return -EINTR;
>>>>     }
>>>> +    spin_unlock(&l_mg->free_lock);
>>>>
>>>>     pblk_rl_free_lines_dec(&pblk->rl, line, true);
>>>>     return 0;
>>>> +
>>>> +out:
>>>> +    list_add(&line->list, &l_mg->free_list);
>>>> +    spin_unlock(&l_mg->free_lock);
>>>> +    return ret;
>>>> }
>>>>
>>>> void pblk_line_recov_close(struct pblk *pblk, struct pblk_line *line)
>>>> --
>>>> 2.19.1
>>> This path is only touched by the recovery path, which is single
>>> threaded, so there is no race condition as is. Also, if recovery fails,
>>> pblk will not create the instance at all. This said, it would be
>>> good to protect the list_add on the pblk_line_init_bb() error path in
>>> case this code is used for some other purpose in the future.
>>
>> I like your explanation here. Another option is that we could add a comment to notify the developer that it safe in this context?
> 
> Sure. Do you want to add it? Or should I send it?
> 
> Javier
> 

Please send it. Thanks!
Javier Gonzalez Nov. 28, 2018, 7:58 a.m. UTC | #5
> On 28 Nov 2018, at 05.26, HUA SU <suhua.tanke@gmail.com> wrote:
> 
> Thank you for your kind correction, I ignored some of the content, it will be perfected by Javier.
> 

You are welcome to re-send the patch with the code I proposed. I'll move
a couple of things around to better comment the single-threaded recovery
path afterwards.

Javier
diff mbox series

Patch

diff --git a/drivers/lightnvm/pblk-core.c b/drivers/lightnvm/pblk-core.c
index 6944aac43b01..e490df217dac 100644
--- a/drivers/lightnvm/pblk-core.c
+++ b/drivers/lightnvm/pblk-core.c
@@ -1286,24 +1286,27 @@  int pblk_line_recov_alloc(struct pblk *pblk, struct pblk_line *line)
 	list_del(&line->list);
 
 	ret = pblk_line_prepare(pblk, line);
-	if (ret) {
-		list_add(&line->list, &l_mg->free_list);
-		spin_unlock(&l_mg->free_lock);
-		return ret;
-	}
-	spin_unlock(&l_mg->free_lock);
+	if (ret)
+		goto out;
 
 	ret = pblk_line_alloc_bitmaps(pblk, line);
 	if (ret)
-		return ret;
+		goto out;
 
 	if (!pblk_line_init_bb(pblk, line, 0)) {
 		list_add(&line->list, &l_mg->free_list);
+		spin_unlock(&l_mg->free_lock);
 		return -EINTR;
 	}
+	spin_unlock(&l_mg->free_lock);
 
 	pblk_rl_free_lines_dec(&pblk->rl, line, true);
 	return 0;
+
+out:
+	list_add(&line->list, &l_mg->free_list);
+	spin_unlock(&l_mg->free_lock);
+	return ret;
 }
 
 void pblk_line_recov_close(struct pblk *pblk, struct pblk_line *line)