| Message ID | 20240409205300.1346681-1-zijianzhang@bytedance.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | net: socket sendmsg MSG_ZEROCOPY_UARG | expand |
On Tue, 2024-04-09 at 20:52 +0000, zijianzhang@bytedance.com wrote: > From: Zijian Zhang <zijianzhang@bytedance.com> > > Original notification mechanism needs poll + recvmmsg which is not > easy for applcations to accommodate. And, it also incurs unignorable > overhead including extra system calls and usage of optmem. > > While making maximum reuse of the existing MSG_ZEROCOPY related code, > this patch set introduces zerocopy socket send flag MSG_ZEROCOPY_UARG. > It provides a new notification method. Users of sendmsg pass a control > message as a placeholder for the incoming notifications. Upon returning, > kernel embeds notifications directly into user arguments passed in. By > doing so, we can significantly reduce the complexity and overhead for > managing notifications. In an ideal pattern, the user will keep calling > sendmsg with MSG_ZEROCOPY_UARG flag, and the notification will be > delivered as soon as possible. > > MSG_ZEROCOPY_UARG does not need to queue skb into errqueue. Thus, > skbuffs allocated from optmem are not a must. In theory, a new struct > carrying the zcopy information should be defined along with its memory > management code. However, existing zcopy generic code assumes the > information is skbuff. Given the very limited performance gain or maybe > no gain of this method, and the need to change a lot of existing code, > we still use skbuffs allocated from optmem to carry zcopy information. > > * Performance > > I extend the selftests/msg_zerocopy.c to accommodate the new flag, test > result is as follows, the new flag performs 7% better in TCP and 4% > better in UDP. > > cfg_notification_limit = 8 > +---------------------+---------+---------+---------+---------+ > > Test Type / Protocol| TCP v4 | TCP v6 | UDP v4 | UDP v6 | > +---------------------+---------+---------+---------+---------+ > > Copy | 5328 | 5159 | 8581 | 8457 | > +---------------------+---------+---------+---------+---------+ > > ZCopy | 5877 | 5568 | 10314 | 10091 | > +---------------------+---------+---------+---------+---------+ > > New ZCopy | 6254 | 5901 | 10674 | 10293 | > +---------------------+---------+---------+---------+---------+ > > ZCopy / Copy | 110.30% | 107.93% | 120.20% | 119.32% | > +---------------------+---------+---------+---------+---------+ > > New ZCopy / Copy | 117.38% | 114.38% | 124.39% | 121.71% | > +---------------------+---------+---------+---------+---------+ Minor nit for a future revision: the relevant part here is the 'New ZCopy/ ZCopy' direct comparison, which is missing - even if inferable from the above. You should provide such data, and you could drop the 'ZCopy / Copy' 'New ZCopy / Copy' and possibly even 'Copy' lines. Thanks, Paolo
On 4/10/24 1:46 AM, Paolo Abeni wrote: > On Tue, 2024-04-09 at 20:52 +0000, zijianzhang@bytedance.com wrote: >> From: Zijian Zhang <zijianzhang@bytedance.com> >> >> Original notification mechanism needs poll + recvmmsg which is not >> easy for applcations to accommodate. And, it also incurs unignorable >> overhead including extra system calls and usage of optmem. >> >> While making maximum reuse of the existing MSG_ZEROCOPY related code, >> this patch set introduces zerocopy socket send flag MSG_ZEROCOPY_UARG. >> It provides a new notification method. Users of sendmsg pass a control >> message as a placeholder for the incoming notifications. Upon returning, >> kernel embeds notifications directly into user arguments passed in. By >> doing so, we can significantly reduce the complexity and overhead for >> managing notifications. In an ideal pattern, the user will keep calling >> sendmsg with MSG_ZEROCOPY_UARG flag, and the notification will be >> delivered as soon as possible. >> >> MSG_ZEROCOPY_UARG does not need to queue skb into errqueue. Thus, >> skbuffs allocated from optmem are not a must. In theory, a new struct >> carrying the zcopy information should be defined along with its memory >> management code. However, existing zcopy generic code assumes the >> information is skbuff. Given the very limited performance gain or maybe >> no gain of this method, and the need to change a lot of existing code, >> we still use skbuffs allocated from optmem to carry zcopy information. >> >> * Performance >> >> I extend the selftests/msg_zerocopy.c to accommodate the new flag, test >> result is as follows, the new flag performs 7% better in TCP and 4% >> better in UDP. >> >> cfg_notification_limit = 8 >> +---------------------+---------+---------+---------+---------+ >>> Test Type / Protocol| TCP v4 | TCP v6 | UDP v4 | UDP v6 | >> +---------------------+---------+---------+---------+---------+ >>> Copy | 5328 | 5159 | 8581 | 8457 | >> +---------------------+---------+---------+---------+---------+ >>> ZCopy | 5877 | 5568 | 10314 | 10091 | >> +---------------------+---------+---------+---------+---------+ >>> New ZCopy | 6254 | 5901 | 10674 | 10293 | >> +---------------------+---------+---------+---------+---------+ >>> ZCopy / Copy | 110.30% | 107.93% | 120.20% | 119.32% | >> +---------------------+---------+---------+---------+---------+ >>> New ZCopy / Copy | 117.38% | 114.38% | 124.39% | 121.71% | >> +---------------------+---------+---------+---------+---------+ > > Minor nit for a future revision: the relevant part here is the 'New > ZCopy/ ZCopy' direct comparison, which is missing - even if inferable > from the above. You should provide such data, and you could drop the > 'ZCopy / Copy' 'New ZCopy / Copy' and possibly even 'Copy' lines. > > Thanks, > > Paolo > Thanks for the advice, I will update in the next version :)
From: Zijian Zhang <zijianzhang@bytedance.com> Original notification mechanism needs poll + recvmmsg which is not easy for applcations to accommodate. And, it also incurs unignorable overhead including extra system calls and usage of optmem. While making maximum reuse of the existing MSG_ZEROCOPY related code, this patch set introduces zerocopy socket send flag MSG_ZEROCOPY_UARG. It provides a new notification method. Users of sendmsg pass a control message as a placeholder for the incoming notifications. Upon returning, kernel embeds notifications directly into user arguments passed in. By doing so, we can significantly reduce the complexity and overhead for managing notifications. In an ideal pattern, the user will keep calling sendmsg with MSG_ZEROCOPY_UARG flag, and the notification will be delivered as soon as possible. MSG_ZEROCOPY_UARG does not need to queue skb into errqueue. Thus, skbuffs allocated from optmem are not a must. In theory, a new struct carrying the zcopy information should be defined along with its memory management code. However, existing zcopy generic code assumes the information is skbuff. Given the very limited performance gain or maybe no gain of this method, and the need to change a lot of existing code, we still use skbuffs allocated from optmem to carry zcopy information. * Performance I extend the selftests/msg_zerocopy.c to accommodate the new flag, test result is as follows, the new flag performs 7% better in TCP and 4% better in UDP. cfg_notification_limit = 8 +---------------------+---------+---------+---------+---------+ | Test Type / Protocol| TCP v4 | TCP v6 | UDP v4 | UDP v6 | +---------------------+---------+---------+---------+---------+ | Copy | 5328 | 5159 | 8581 | 8457 | +---------------------+---------+---------+---------+---------+ | ZCopy | 5877 | 5568 | 10314 | 10091 | +---------------------+---------+---------+---------+---------+ | New ZCopy | 6254 | 5901 | 10674 | 10293 | +---------------------+---------+---------+---------+---------+ | ZCopy / Copy | 110.30% | 107.93% | 120.20% | 119.32% | +---------------------+---------+---------+---------+---------+ | New ZCopy / Copy | 117.38% | 114.38% | 124.39% | 121.71% | +---------------------+---------+---------+---------+---------+ Zijian Zhang (3): sock: add MSG_ZEROCOPY_UARG selftests: fix OOM problem in msg_zerocopy selftest selftests: add msg_zerocopy_uarg test include/linux/skbuff.h | 7 +- include/linux/socket.h | 1 + include/linux/tcp.h | 3 + include/linux/udp.h | 3 + include/net/sock.h | 17 +++ include/net/udp.h | 1 + include/uapi/asm-generic/socket.h | 2 + include/uapi/linux/socket.h | 17 +++ net/core/skbuff.c | 137 ++++++++++++++++-- net/core/sock.c | 50 +++++++ net/ipv4/ip_output.c | 6 +- net/ipv4/tcp.c | 7 +- net/ipv4/udp.c | 9 ++ net/ipv6/ip6_output.c | 5 +- net/ipv6/udp.c | 9 ++ net/vmw_vsock/virtio_transport_common.c | 2 +- tools/testing/selftests/net/msg_zerocopy.c | 150 ++++++++++++++++++-- tools/testing/selftests/net/msg_zerocopy.sh | 1 + 18 files changed, 397 insertions(+), 30 deletions(-)