| Message ID | 20220714163301.67794-1-ubizjak@gmail.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | io_uring: Use atomic_long_try_cmpxchg in __io_account_mem | expand |
On 7/14/22 10:33 AM, Uros Bizjak wrote: > Use atomic_long_try_cmpxchg instead of > atomic_long_cmpxchg (*ptr, old, new) == old in __io_account_mem. > x86 CMPXCHG instruction returns success in ZF flag, so this > change saves a compare after cmpxchg (and related move > instruction in front of cmpxchg). > > Also, atomic_long_try_cmpxchg implicitly assigns old *ptr value > to "old" when cmpxchg fails, enabling further code simplifications. > > No functional change intended. This will be io_uring/rsrc.c for the for-next branches, but it'll apply directly as that with a slight offset: checking file io_uring/rsrc.c Hunk #1 succeeded at 56 (offset -10448 lines). I'll do that, thanks.
diff --git a/fs/io_uring.c b/fs/io_uring.c index a01ea49f3017..9f82904dcdae 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -10504,14 +10504,13 @@ static inline int __io_account_mem(struct user_struct *user, /* Don't allow more pages than we can safely lock */ page_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; + cur_pages = atomic_long_read(&user->locked_vm); do { - cur_pages = atomic_long_read(&user->locked_vm); new_pages = cur_pages + nr_pages; if (new_pages > page_limit) return -ENOMEM; - } while (atomic_long_cmpxchg(&user->locked_vm, cur_pages, - new_pages) != cur_pages); - + } while (!atomic_long_try_cmpxchg(&user->locked_vm, + &cur_pages, new_pages)); return 0; }
Use atomic_long_try_cmpxchg instead of atomic_long_cmpxchg (*ptr, old, new) == old in __io_account_mem. x86 CMPXCHG instruction returns success in ZF flag, so this change saves a compare after cmpxchg (and related move instruction in front of cmpxchg). Also, atomic_long_try_cmpxchg implicitly assigns old *ptr value to "old" when cmpxchg fails, enabling further code simplifications. No functional change intended. Signed-off-by: Uros Bizjak <ubizjak@gmail.com> Cc: Jens Axboe <axboe@kernel.dk> Cc: Pavel Begunkov <asml.silence@gmail.com> --- fs/io_uring.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)