From patchwork Fri Jun 9 08:27:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Abel Wu X-Patchwork-Id: 13273470 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9219C8300C for ; Fri, 9 Jun 2023 08:28:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7944C8E0002; Fri, 9 Jun 2023 04:28:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 71D058E0001; Fri, 9 Jun 2023 04:28:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 596798E0002; Fri, 9 Jun 2023 04:28:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 46FAD8E0001 for ; Fri, 9 Jun 2023 04:28:14 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 11FC0140186 for ; Fri, 9 Jun 2023 08:28:14 +0000 (UTC) X-FDA: 80882532108.30.08995FE Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) by imf05.hostedemail.com (Postfix) with ESMTP id 471E7100015 for ; Fri, 9 Jun 2023 08:28:11 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=jsw3R6na; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf05.hostedemail.com: domain of wuyun.abel@bytedance.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=wuyun.abel@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686299292; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=3Mnn0NJnMdo5oou5LijiWQTnrZb/85o7p3nvJNaAHoM=; b=Dxb1SWjkYUYplHG3wKRDhHPw3s8YYeSigZmdiOzw8PnkswHgQRMDiv/B6S34UlrYOPrTOJ ReBcrydf9t+qt3CYmh0LSzeA41iyejbfNjRt7S0cwQyopblVGAP9vMrxvR/cqvtRPLw8n3 6NOhVpfL/4NbLpIWYiAwd3tiSBCvhHY= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=jsw3R6na; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf05.hostedemail.com: domain of wuyun.abel@bytedance.com designates 209.85.214.177 as permitted sender) smtp.mailfrom=wuyun.abel@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686299292; a=rsa-sha256; cv=none; b=F14crCHoVqh11z9esxSRP1Wz1Kig6GyqZR8UoCgH+OIZy5DksqbQ9707Qjh5+ezLmGmxom ZbZuYaPXUAxz3axx6g8H9BoQbB8aTUtNeqZ3uWl5abY6AfxGknQPMVXfo5B/pZpydkYeEw liiA3V1NonI7yZKI+OvFYsNpCQzQYdc= Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1b011cffe7fso4487815ad.1 for ; Fri, 09 Jun 2023 01:28:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1686299290; x=1688891290; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=3Mnn0NJnMdo5oou5LijiWQTnrZb/85o7p3nvJNaAHoM=; b=jsw3R6na+7LqrydPc0FvDpVQhH7T6IEkvxDv7igYNc4VoPgzY5x138m6Ufz7zAWNyh ka3dcEilI6dmem3GetyUB7wv/wXMY57kojWZ9cxIb5EnhQmcOOsuWZxBdMsPaoh99AqB B6XUvC6+N8caa3EoRa7tgVqcb9x+qj5WLqDC4KkAXMDsH/Newo4Z4/LuOJsgQBye6Ocs EnU3Cp3MVnMyFXpLqMsh77rVJmlIsEdTIQxqr+tJ3JQITj8H3Z4jcCqQoao5hD162/rM XKHOW/Glgi2xoYiE19QmBByld9VX0RLsq/IHjNR1Q+uXH+CvZE03RlvQAkruEzxL6kzf oC5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686299290; x=1688891290; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=3Mnn0NJnMdo5oou5LijiWQTnrZb/85o7p3nvJNaAHoM=; b=CTsxtEN3GJdKFKiNH1xY4CX/HLwiRY3OKQ2yDsNzRtTZOJdIUSR91s31IKSa6ebWG8 H9czGZonjAuV78sEyQb8jn4uYpGASUIZy4AisI1vi/y7XeUPn0VBkH9Dst6/CYWY5BrN DWkQCcwfGvO3WlxTei3VC4v54QXte5NFNA096l4PACiIHzbExQi4zIq5torlg+qmSdSe iPhfzlsPlKQF9u64tK5YXaWRA3sY6vSmeQtVhazB1yI52GACy2rra3UAQW5oBrQtxOQ+ hwThRjzDxBpJCzm6jxj45kcD2kARkQryCEsOcYPjAy+JnjrjjffCLouSMYRI491phRlj +F4w== X-Gm-Message-State: AC+VfDyYw8sttMqZ3bs/h7hr0BLYmkt9JLd96RpCcDyBNH1WWCjCAyDe DzYQBx20wwxytgNCpnwURhoVRQ== X-Google-Smtp-Source: ACHHUZ4vyQNHvw2dp1onWfxhxBw2KSsxsWbMp6td9W0bKDrPtVX1Xcit67MBSHQY1dlOFrCHm8jLTg== X-Received: by 2002:a17:903:496:b0:1af:f8a8:5ba4 with SMTP id jj22-20020a170903049600b001aff8a85ba4mr432039plb.4.1686299289673; Fri, 09 Jun 2023 01:28:09 -0700 (PDT) Received: from C02DV8HUMD6R.bytedance.net ([139.177.225.255]) by smtp.gmail.com with ESMTPSA id g7-20020a1709026b4700b001ab2b415bdbsm2692437plt.45.2023.06.09.01.28.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 09 Jun 2023 01:28:08 -0700 (PDT) From: Abel Wu To: Tejun Heo , Christian Warloe , Wei Wang , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , David Ahern , Yosry Ahmed , "Matthew Wilcox (Oracle)" , Yu Zhao , Vasily Averin , Kuniyuki Iwashima , Martin KaFai Lau , Abel Wu , Xin Long , Jason Xing Cc: Michal Hocko , Alexei Starovoitov , linux-kernel@vger.kernel.org (open list), netdev@vger.kernel.org (open list:NETWORKING [GENERAL]), cgroups@vger.kernel.org (open list:CONTROL GROUP - MEMORY RESOURCE CONTROLLER (MEMCG)), linux-mm@kvack.org (open list:CONTROL GROUP - MEMORY RESOURCE CONTROLLER (MEMCG)) Subject: [RFC PATCH net-next] sock: Propose socket.urgent for sockmem isolation Date: Fri, 9 Jun 2023 16:27:03 +0800 Message-Id: <20230609082712.34889-1-wuyun.abel@bytedance.com> X-Mailer: git-send-email 2.37.3 MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 471E7100015 X-Stat-Signature: eafm1x84gwp3bkhgid5yr8uwjnohj5ct X-Rspam-User: X-HE-Tag: 1686299291-153721 X-HE-Meta: U2FsdGVkX19K9y545OWKe7HSZLAHNIQcWk8bNH9JK0L7Kc8ncIiXJ4cw4QHzk4abM5vz03IacZzzYEHlLF/SIIKUYnpkBPUvYn0oQh1HyRvT8wCshTbUuPlrWjdMnqGSfGh+oej1q1x0izS7j0JPnMliPXWGBgAJiWBO9hvtL9SYvL2xL/m7IhmtEWQ5JqKBtaBPCo51xGUF7h7gNuQWouuYNiM6X2IWJRUjbytVn1+jxrE0MRoo6Bg8Bw5LASY1EDHdBxci9sJmRgbZZf1WCCCWNiNYz4zfIcOLoHIOYVVFmBUyTaRjDTkmWm3UtAFSm0VDEKGPGv0g/turQQFF73WQb4FvxPkWn/bSM3KQnp74IvV2Gn7q9aZMcH832Zi5rpAnx+z+GkajZcWFGctvSVVEkKw+/CH9KI1s2vkZKo33sY3zojJf1GMuniql3KJMDbZf4iYeeh2jtERwXz/F4wHkETy/6oR608C+BtCKx3cLk9FvTYCYdTetW+R7iRmb1PbqwmL1ftc9AWs5ADI/LVAyDjFRgMfTh14buwdwrDtaWsEbF5N5+XmGnqD/uaI0NmU70GxiVliiSy28oan5KGU190ffXIfcskJtMi34ZoU8VsBoHZGPf8yZKUVM9Z2pnBzLIDfXlLKfZfp9wiEoonkN9OPC/l+NLSeJr7EQ9CmeG79taYm73MQuSYYB3EDWrO1eYTWeFYJTXYuVFgbkufN2wFUqtEvE63Wg2W9IIcou6ZitolXxuAd02nuomZJQFdUYJj5fIVH6bFfWVOWW5HECUK2xFFO2hoFxAy7p2SbNryQ4dhr6x6DYXPv1yOSLKG1Abtoqjd1DFjWtC/zGcbGcImIggNNU56tM06LksaR+Fi+rZviFbSxTrBQMBH5HjqdUfv+awhI3z54U3+AnYJtyYU0Wi76nzMTyb/qNZBPEDd1zu1ZuMfsAzLOenDCe7ufIePKRyMroJ2E9QrW 4Bi6Ilpy jLlJblDqLGNtOCK68AXTsHLGDLrsvc7854wiLzjf4LE3d+R6JH2LpehU65IJndwvm4iGwzJJDQ0Oaujj6zTtnjkRdeTokp0hjAesl7ecQF6RIP7wjP824vu7sV/CCSoJ5W+I1lh1dELHaWkQ4utPWcCfT2NE02qX5Yr/U6b7PXzcfVy3Odc7vd2OEZXs+tnQ1+48UvVmDIeclCq7rDjiWIO6h7MhBlKdYNAqi0B/eqJaH0sy+MwzDzkS28CZx5kv8tRIp0wzJrCxArV8Quwmcm07mAJmDDielE11qItdm/zyUZWmUiMSCsWBC+sHSYT6h2D3qAQN414RHAVnIs6FpjwJRxN2ddORxclmhJ2AJ5hSRUZOGur8S7f/4JiYZtXY61ZatDLUM84GE3lalply2Qvpt3LWijRoZOPqSyjMBMLJlk4cn7Rn8FygBuiKCkk6PnFUIpdDJnh047F1e3xqOer4fd8SZ9hwzcWJVPWiS9r99aet9J0c3BV16vtKis6X79U4luJdwiCF/AcL7hRodlAwhCMvhU8yS+V2Gi1nFNtAbl1r90rLuFGomppwWWVXA/Uphk6Jwm5YKz/sp3tq9075WsTyMNQcS82ktaoZi7SqrhiZg0f3m/uccBdN81YjIqXPVCQtbAh9C8S9W4Xe+K7JQiQcku3LaOnXN69NmCIRZPdq1niVgg5OsMI5oyUdRyus9n1L6CCEQrdAZSMncyL7C191qdjhPJZxKgrU5uiEyukhwOIlQ8tJhtLyiksATpVAWoKsD7daY3kqWxd/6JxW43f5s/g6fS0c7oAXmt4h8VERPtLfbU1Lt0klNn5Jlm8pEqwiFZ+YTAS2oxKXegIlEjshcQ3osWxPTJjnEq23saCKI0UmWtFL0U4acIgaxF6wVB2l3F4OjwLC24oNzCSjNJN6N1hDoYqA1X/8o//q1bGauKhrm8vLO0VvN+I4TzX2p0xL0RAxK5urNf3Gr0JF04F6L l/JvCPyJ lSzsGJQFptEqpaGtwUtUXfDySe3bPPzlCUB5/tRpfSV6fpiFfvgmUEK9QNIUkRDF+S0n/iRZ3YYw5ZRQ5gIaqSkh0jeicF2a7EXztAVui6lgqMAApRdFjRkCxA9GyRbLQT9QilnfGOPb89eGZXNqMdRpyngAwczsI/Ig/VW/sHCikI31jP4qtmFVMO2LY4iIPZr28jxufKAu+XtjLxQAyp5KCcE7X/UM2TwDRBrfMla9PvrNtUgC5kYoy7feCeUvOLrjZS4M+FLmO/6fPeobojVbEWDWy1cictNmVoLJoRL913+wUGmB5UbVUWWpOgKnh9Wikl2NFg8U8OMEoPlgA7A2JSkxcixrfYIfSUyxO2C1q4/gbd/Ha85yxuJrJ6xW0wnZ9LCxJQfQ2hbwtaf+2A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This is just a PoC patch intended to resume the discussion about tcpmem isolation opened by Google in LPC'22 [1]. We are facing the same problem that the global shared threshold can cause isolation issues. Low priority jobs can hog TCP memory and adversely impact higher priority jobs. What's worse is that these low priority jobs usually have smaller cpu weights leading to poor ability to consume rx data. To tackle this problem, an interface for non-root cgroup memory controller named 'socket.urgent' is proposed. It determines whether the sockets of this cgroup and its descendants can escape from the constrains or not under global socket memory pressure. The 'urgent' semantics will not take effect under memcg pressure in order to protect against worse memstalls, thus will be the same as before without this patch. This proposal doesn't remove protocal's threshold as we found it useful in restraining memory defragment. As aforementioned the low priority jobs can hog lots of memory, which is unreclaimable and unmovable, for some time due to small cpu weight. So in practice we allow high priority jobs with net-memcg accounting enabled to escape the global constrains if the net-memcg itselt is not under pressure. While for lower priority jobs, the budget will be tightened as the memory usage of 'urgent' jobs increases. In this way we can finally achieve: - Important jobs won't be priority inversed by the background jobs in terms of socket memory pressure/limit. - Global constrains are still effective, but only on non-urgent jobs, useful for admins on policy decision on defrag. Comments/Ideas are welcomed, thanks! [1] https://lpc.events/event/16/contributions/1212/ Signed-off-by: Abel Wu --- include/linux/memcontrol.h | 15 +++++++++++++-- include/net/sock.h | 11 ++++++++--- include/net/tcp.h | 26 ++++++++++++++++++++------ mm/memcontrol.c | 35 +++++++++++++++++++++++++++++++++++ net/core/sock.c | 22 ++++++++++++++++++---- net/ipv4/tcp_input.c | 10 +++++++--- 6 files changed, 101 insertions(+), 18 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 222d7370134c..f8c1c108aa28 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -284,7 +284,13 @@ struct mem_cgroup { atomic_long_t memory_events[MEMCG_NR_MEMORY_EVENTS]; atomic_long_t memory_events_local[MEMCG_NR_MEMORY_EVENTS]; + /* + * Urgent sockets can escape from the contrains under global memory + * pressure/limit iff !socket_pressure. So this two variables are + * always used together, make sure they are in same cacheline. + */ unsigned long socket_pressure; + bool socket_urgent; /* Legacy tcp memory accounting */ bool tcpmem_active; @@ -1741,13 +1747,17 @@ extern struct static_key_false memcg_sockets_enabled_key; #define mem_cgroup_sockets_enabled static_branch_unlikely(&memcg_sockets_enabled_key) void mem_cgroup_sk_alloc(struct sock *sk); void mem_cgroup_sk_free(struct sock *sk); -static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *memcg) + +static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *memcg, + bool *is_urgent) { if (!cgroup_subsys_on_dfl(memory_cgrp_subsys) && memcg->tcpmem_pressure) return true; do { if (time_before(jiffies, READ_ONCE(memcg->socket_pressure))) return true; + if (is_urgent && !*is_urgent && READ_ONCE(memcg->socket_urgent)) + *is_urgent = true; } while ((memcg = parent_mem_cgroup(memcg))); return false; } @@ -1760,7 +1770,8 @@ void reparent_shrinker_deferred(struct mem_cgroup *memcg); #define mem_cgroup_sockets_enabled 0 static inline void mem_cgroup_sk_alloc(struct sock *sk) { }; static inline void mem_cgroup_sk_free(struct sock *sk) { }; -static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *memcg) +static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *memcg, + bool *is_urgent) { return false; } diff --git a/include/net/sock.h b/include/net/sock.h index 656ea89f60ff..80e1240ffc35 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -1414,9 +1414,14 @@ static inline bool sk_under_memory_pressure(const struct sock *sk) if (!sk->sk_prot->memory_pressure) return false; - if (mem_cgroup_sockets_enabled && sk->sk_memcg && - mem_cgroup_under_socket_pressure(sk->sk_memcg)) - return true; + if (mem_cgroup_sockets_enabled && sk->sk_memcg) { + bool urgent; + + if (mem_cgroup_under_socket_pressure(sk->sk_memcg, &urgent)) + return true; + if (urgent) + return false; + } return !!*sk->sk_prot->memory_pressure; } diff --git a/include/net/tcp.h b/include/net/tcp.h index 14fa716cac50..9fa8d8fcb992 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -259,9 +259,14 @@ extern unsigned long tcp_memory_pressure; /* optimized version of sk_under_memory_pressure() for TCP sockets */ static inline bool tcp_under_memory_pressure(const struct sock *sk) { - if (mem_cgroup_sockets_enabled && sk->sk_memcg && - mem_cgroup_under_socket_pressure(sk->sk_memcg)) - return true; + if (mem_cgroup_sockets_enabled && sk->sk_memcg) { + bool urgent; + + if (mem_cgroup_under_socket_pressure(sk->sk_memcg, &urgent)) + return true; + if (urgent) + return false; + } return READ_ONCE(tcp_memory_pressure); } @@ -284,9 +289,18 @@ static inline bool between(__u32 seq1, __u32 seq2, __u32 seq3) static inline bool tcp_out_of_memory(struct sock *sk) { - if (sk->sk_wmem_queued > SOCK_MIN_SNDBUF && - sk_memory_allocated(sk) > sk_prot_mem_limits(sk, 2)) - return true; + if (sk->sk_wmem_queued > SOCK_MIN_SNDBUF) { + bool urgent = false; + + if (mem_cgroup_sockets_enabled && sk->sk_memcg && + !mem_cgroup_under_socket_pressure(sk->sk_memcg, &urgent) && + urgent) + return false; + + if (sk_memory_allocated(sk) > sk_prot_mem_limits(sk, 2)) + return true; + } + return false; } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 4b27e245a055..d620c4d9b2cc 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -6753,6 +6753,35 @@ static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, return nbytes; } +static int memory_sock_urgent_show(struct seq_file *m, void *v) +{ + struct mem_cgroup *memcg = mem_cgroup_from_seq(m); + + seq_printf(m, "%d\n", READ_ONCE(memcg->socket_urgent)); + + return 0; +} + +static ssize_t memory_sock_urgent_write(struct kernfs_open_file *of, + char *buf, size_t nbytes, loff_t off) +{ + struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); + bool socket_urgent; + int ret; + + buf = strstrip(buf); + if (!buf) + return -EINVAL; + + ret = kstrtobool(buf, &socket_urgent); + if (ret) + return ret; + + WRITE_ONCE(memcg->socket_urgent, socket_urgent); + + return nbytes; +} + static struct cftype memory_files[] = { { .name = "current", @@ -6821,6 +6850,12 @@ static struct cftype memory_files[] = { .flags = CFTYPE_NS_DELEGATABLE, .write = memory_reclaim, }, + { + .name = "socket.urgent", + .flags = CFTYPE_NOT_ON_ROOT | CFTYPE_NS_DELEGATABLE, + .seq_show = memory_sock_urgent_show, + .write = memory_sock_urgent_write, + }, { } /* terminate */ }; diff --git a/net/core/sock.c b/net/core/sock.c index 5440e67bcfe3..29d2b03595cf 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -2982,10 +2982,24 @@ int __sk_mem_raise_allocated(struct sock *sk, int size, int amt, int kind) sk_memory_allocated_add(sk, amt); allocated = sk_memory_allocated(sk); - if (memcg_charge && - !(charged = mem_cgroup_charge_skmem(sk->sk_memcg, amt, - gfp_memcg_charge()))) - goto suppress_allocation; + + if (memcg_charge) { + bool urgent; + + charged = mem_cgroup_charge_skmem(sk->sk_memcg, amt, + gfp_memcg_charge()); + if (!charged) + goto suppress_allocation; + + if (!mem_cgroup_under_socket_pressure(sk->sk_memcg, &urgent)) { + /* Urgent sockets by design escape from the constrains + * under global memory pressure/limit iff there is no + * pressure in the net-memcg to avoid priority inversion. + */ + if (urgent) + return 1; + } + } /* Under limit. */ if (allocated <= sk_prot_mem_limits(sk, 0)) { diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index 61b6710f337a..7d5d4b4e17b4 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -5439,6 +5439,7 @@ static int tcp_prune_queue(struct sock *sk, const struct sk_buff *in_skb) static bool tcp_should_expand_sndbuf(struct sock *sk) { const struct tcp_sock *tp = tcp_sk(sk); + bool under_pressure, urgent = false; /* If the user specified a specific send buffer setting, do * not modify it. @@ -5446,8 +5447,11 @@ static bool tcp_should_expand_sndbuf(struct sock *sk) if (sk->sk_userlocks & SOCK_SNDBUF_LOCK) return false; - /* If we are under global TCP memory pressure, do not expand. */ - if (tcp_under_memory_pressure(sk)) { + under_pressure = mem_cgroup_sockets_enabled && sk->sk_memcg && + mem_cgroup_under_socket_pressure(sk->sk_memcg, &urgent); + + /* If we are under net-memcg/TCP memory pressure, do not expand. */ + if (under_pressure || (!urgent && READ_ONCE(tcp_memory_pressure))) { int unused_mem = sk_unused_reserved_mem(sk); /* Adjust sndbuf according to reserved mem. But make sure @@ -5461,7 +5465,7 @@ static bool tcp_should_expand_sndbuf(struct sock *sk) } /* If we are under soft global TCP memory pressure, do not expand. */ - if (sk_memory_allocated(sk) >= sk_prot_mem_limits(sk, 0)) + if (!urgent && sk_memory_allocated(sk) >= sk_prot_mem_limits(sk, 0)) return false; /* If we filled the congestion window, do not expand. */