| Message ID | 654332b3765fa728e1e149523bf7fb59c563a186.1650891417.git.asml.silence@gmail.com (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | 0a8afd9f026a7f6c835be0fed2ab709d4133797f |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | UDP sock_wfree optimisations | expand |
| Context | Check | Description |
|---|---|---|
| netdev/tree_selection | success | Clearly marked for net-next |
| netdev/fixes_present | success | Fixes tag not required for -next series |
| netdev/subject_prefix | success | Link |
| netdev/cover_letter | success | Series has a cover letter |
| netdev/patch_count | success | Link |
| netdev/header_inline | success | No static functions without inline keyword in header files |
| netdev/build_32bit | success | Errors and warnings before: 6 this patch: 6 |
| netdev/cc_maintainers | warning | 1 maintainers not CCed: pabeni@redhat.com |
| netdev/build_clang | success | Errors and warnings before: 9 this patch: 9 |
| netdev/module_param | success | Was 0 now: 0 |
| netdev/verify_signedoff | success | Signed-off-by tag matches author and committer |
| netdev/verify_fixes | success | No Fixes tag |
| netdev/build_allmodconfig_warn | success | Errors and warnings before: 6 this patch: 6 |
| netdev/checkpatch | warning | WARNING: waitqueue_active without comment |
| netdev/kdoc | success | Errors and warnings before: 0 this patch: 0 |
| netdev/source_inline | success | Was 0 now: 0 |
diff --git a/net/core/sock.c b/net/core/sock.c index 4ad4d6dd940e..85d0b04c7bc5 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -146,6 +146,7 @@ static DEFINE_MUTEX(proto_list_mutex); static LIST_HEAD(proto_list); +static void sock_def_write_space_wfree(struct sock *sk); static void sock_def_write_space(struct sock *sk); /** @@ -2329,7 +2330,7 @@ void sock_wfree(struct sk_buff *skb) sk->sk_write_space == sock_def_write_space) { rcu_read_lock(); free = refcount_sub_and_test(len, &sk->sk_wmem_alloc); - sock_def_write_space(sk); + sock_def_write_space_wfree(sk); rcu_read_unlock(); if (unlikely(free)) __sk_free(sk); @@ -3221,6 +3222,29 @@ static void sock_def_write_space(struct sock *sk) rcu_read_unlock(); } +/* An optimised version of sock_def_write_space(), should only be called + * for SOCK_RCU_FREE sockets under RCU read section and after putting + * ->sk_wmem_alloc. + */ +static void sock_def_write_space_wfree(struct sock *sk) +{ + /* Do not wake up a writer until he can make "significant" + * progress. --DaveM + */ + if (sock_writeable(sk)) { + struct socket_wq *wq = rcu_dereference(sk->sk_wq); + + /* rely on refcount_sub from sock_wfree() */ + smp_mb__after_atomic(); + if (wq && waitqueue_active(&wq->wait)) + wake_up_interruptible_sync_poll(&wq->wait, EPOLLOUT | + EPOLLWRNORM | EPOLLWRBAND); + + /* Should agree with poll, otherwise some programs break */ + sk_wake_async(sk, SOCK_WAKE_SPACE, POLL_OUT); + } +} + static void sock_def_destruct(struct sock *sk) { }
Now we have a separate path for sock_def_write_space() and can go one step further. When it's called from sock_wfree() we know that there is a preceding atomic for putting down ->sk_wmem_alloc. We can use it to replace to replace smb_mb() with a less expensive smp_mb__after_atomic(). It also removes an extra RCU read lock/unlock as a small bonus. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> --- net/core/sock.c | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-)