| Message ID | 20221202131837.375341-1-urezki@gmail.com (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | 9615ca28bc2c94cf2e092f06baace941fbf3b7e8 |
| Headers | show |
| Series | [1/1] rcu/kvfree: Use READ_ONCE() when access to krcp->head | expand |
On Fri, Dec 02, 2022 at 02:18:37PM +0100, Uladzislau Rezki (Sony) wrote: > A need_offload_krc() function is now lock-free. A compiler > can optimize readers in way that they see an old value even > though writers already updated the krcp->head from another > path. > > Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com> Queued, thank you!!! With the usual wordsmithing, so please let me know if I messed something up. Thanx, Paul ------------------------------------------------------------------------ commit 9615ca28bc2c94cf2e092f06baace941fbf3b7e8 Author: Uladzislau Rezki (Sony) <urezki@gmail.com> Date: Fri Dec 2 14:18:37 2022 +0100 rcu/kvfree: Use READ_ONCE() when access to krcp->head The need_offload_krc() function is now lock-free, which gives the compiler freedom to load old values from plain C-language loads from the kfree_rcu_cpu struture's ->head pointer. This commit therefore applied READ_ONCE() to these loads. Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com> Signed-off-by: Paul E. McKenney <paulmck@kernel.org> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index 2d16a0eee792d..ee8a6a711719a 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -3029,7 +3029,7 @@ need_offload_krc(struct kfree_rcu_cpu *krcp) if (!list_empty(&krcp->bulk_head[i])) return true; - return !!krcp->head; + return !!READ_ONCE(krcp->head); } static void @@ -3070,7 +3070,7 @@ static void kfree_rcu_monitor(struct work_struct *work) // in that case the monitor work is rearmed. if ((!list_empty(&krcp->bulk_head[0]) && list_empty(&krwp->bulk_head_free[0])) || (!list_empty(&krcp->bulk_head[1]) && list_empty(&krwp->bulk_head_free[1])) || - (krcp->head && !krwp->head_free)) { + (READ_ONCE(krcp->head) && !krwp->head_free)) { // Channel 1 corresponds to the SLAB-pointer bulk path. // Channel 2 corresponds to vmalloc-pointer bulk path.
diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c index efc79f942b30..d155f2594317 100644 --- a/kernel/rcu/tree.c +++ b/kernel/rcu/tree.c @@ -3029,7 +3029,7 @@ need_offload_krc(struct kfree_rcu_cpu *krcp) if (!list_empty(&krcp->bulk_head[i])) return true; - return !!krcp->head; + return !!READ_ONCE(krcp->head); } static void @@ -3070,7 +3070,7 @@ static void kfree_rcu_monitor(struct work_struct *work) // in that case the monitor work is rearmed. if ((!list_empty(&krcp->bulk_head[0]) && list_empty(&krwp->bulk_head_free[0])) || (!list_empty(&krcp->bulk_head[1]) && list_empty(&krwp->bulk_head_free[1])) || - (krcp->head && !krwp->head_free)) { + (READ_ONCE(krcp->head) && !krwp->head_free)) { // Channel 1 corresponds to the SLAB-pointer bulk path. // Channel 2 corresponds to vmalloc-pointer bulk path.
A need_offload_krc() function is now lock-free. A compiler can optimize readers in way that they see an old value even though writers already updated the krcp->head from another path. Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com> --- kernel/rcu/tree.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)