From patchwork Fri Sep 1 06:21:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Abel Wu X-Patchwork-Id: 13372077 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48FACCA0FE6 for ; Fri, 1 Sep 2023 06:23:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5F7788D001A; Fri, 1 Sep 2023 02:23:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5A7238D0002; Fri, 1 Sep 2023 02:23:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 46F698D001A; Fri, 1 Sep 2023 02:23:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 35C738D0002 for ; Fri, 1 Sep 2023 02:23:52 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0375A14013B for ; Fri, 1 Sep 2023 06:23:51 +0000 (UTC) X-FDA: 81187037904.08.294C65F Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) by imf18.hostedemail.com (Postfix) with ESMTP id 337B71C002A for ; Fri, 1 Sep 2023 06:23:49 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=J0XMr6sO; spf=pass (imf18.hostedemail.com: domain of wuyun.abel@bytedance.com designates 209.85.210.172 as permitted sender) smtp.mailfrom=wuyun.abel@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693549430; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1QWm8cF6bkJper87bKNGM7pspfj+5pss/cTqDVG6VeI=; b=PZTp2WvtlsaUFr8UX2ASRpRHtp+TC4GzQTbiTauArz0x5UVfJZA8kDm+1qoNQ9JSxSLStD m3LPE0yVaFMERxrB2OElQl/CqOe1LTAh3eQorj4nzrKSzAVfkO4v87cIOXtcItj0m4DLfZ F+NmIv/8e405jQJxo5WOQid7ph4smm4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1693549430; a=rsa-sha256; cv=none; b=dLaVxvwTk5Hhuj9yiDWpA3SfibUqDvAHAiuPmNZ8J7NkBF6003rPl5023/tw6ZK8nAQVjY MPIZUkfsUjGUvLLk38EhpBtFBXKE3bnOkHSFZlchHyeoDRQOGnH0hxGXsqayd4uPOMBPnL J90xLkSbI6ERdf79t0KcGzBBLlv1jH4= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=J0XMr6sO; spf=pass (imf18.hostedemail.com: domain of wuyun.abel@bytedance.com designates 209.85.210.172 as permitted sender) smtp.mailfrom=wuyun.abel@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-68c0d262933so1373265b3a.0 for ; Thu, 31 Aug 2023 23:23:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1693549429; x=1694154229; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1QWm8cF6bkJper87bKNGM7pspfj+5pss/cTqDVG6VeI=; b=J0XMr6sO8WzyXpvwDsX5JdRnXmlU6aP6ayqD8Y3kOW3LQY5/AWEYzEdFkHdKWCETUY ZMM44VTdgzyhIUzovIZkRRPSWh7YhwDsELEZVNTb+wBIxOH1J0Ysn2ttnnz6Drx5K80Y g1y5zDddm/l4rqwpOPlAWTEv13lmvnQX37/yk6COsYJBheOtO7bbzZ/p9ARFs3Mbi1kq Nn0RwQavkRUjntQdBbegIoa0WByNIkNVl5biuTMbq3yn2HgpdEHABkLDgZ13kY9FGVnx n1Tkkn1FU4WEmAVB1X86U+vVq25yxbF9qkqnPX7/StjTdLxT/337oO04rnAiiGZOd6wS 1VTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693549429; x=1694154229; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1QWm8cF6bkJper87bKNGM7pspfj+5pss/cTqDVG6VeI=; b=WVz+bJf8o0Wekdz/GyTqev4+0sqjCOvHBl9ZsIF3GQ6CV0ipLqY54bYMWWJpyNtKv/ Retw94yKj7yuafdCOrCikuK1YZtPocFbJmfoRb/qJeJInkRW7G8F9BGIZE7o3oDUtW5r dGXrMjtFQJc1Im+D41p4zSc+VTMFPBRPh+8cNqzvUrus+OYoqaZnJBAG/FSn3mYRjgp7 Pdt3BrL5YqBGjl6awqLx1MOn3mqgcdOpCF54imWs4RXaB87VFTFOX6/LymjzWd6KIP24 wJ/L4epJhys0T0a1cPGj8hVwy0xbL3rkKLj8yBDOtGW3Jd+PsaQKZxBxALXIznPhSxAq 8KOQ== X-Gm-Message-State: AOJu0YzFxAjVCV2YOkoxwvMGIzSGNh83/7OyKE/MlqVUFMoJhixkBkWc MHfvWgnXzbzc2M+rgP5mvKMynQ== X-Google-Smtp-Source: AGHT+IH317txsPuj8MBF7exknVZRvNNzVT02jeLJY4AHeJlruKWwnuwX34akxA/Zq54dUriHM27EIg== X-Received: by 2002:a05:6a20:1019:b0:148:1e47:345e with SMTP id gs25-20020a056a20101900b001481e47345emr1722651pzc.49.1693549428921; Thu, 31 Aug 2023 23:23:48 -0700 (PDT) Received: from C02DV8HUMD6R.bytedance.net ([203.208.167.146]) by smtp.gmail.com with ESMTPSA id fm19-20020a056a002f9300b0068c1ac1784csm2223265pfb.59.2023.08.31.23.23.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Aug 2023 23:23:48 -0700 (PDT) From: Abel Wu To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Andrew Morton , Shakeel Butt , Roman Gushchin , Michal Hocko , Johannes Weiner , Yosry Ahmed , Yu Zhao , "Matthew Wilcox (Oracle)" , Abel Wu , Yafang Shao , Kefeng Wang , Kuniyuki Iwashima , Martin KaFai Lau , Breno Leitao , Alexander Mikhalitsyn , David Howells , Jason Xing Cc: linux-kernel@vger.kernel.org (open list), netdev@vger.kernel.org (open list:NETWORKING [GENERAL]), linux-mm@kvack.org (open list:MEMORY MANAGEMENT) Subject: [RFC PATCH net-next 1/3] sock: Code cleanup on __sk_mem_raise_allocated() Date: Fri, 1 Sep 2023 14:21:26 +0800 Message-Id: <20230901062141.51972-2-wuyun.abel@bytedance.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20230901062141.51972-1-wuyun.abel@bytedance.com> References: <20230901062141.51972-1-wuyun.abel@bytedance.com> MIME-Version: 1.0 X-Stat-Signature: oukn3w4gwn1h7bdftm4kgar65zbujop1 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 337B71C002A X-Rspam-User: X-HE-Tag: 1693549429-161443 X-HE-Meta: U2FsdGVkX1/rskRH3DGTr6etqyA2NlAeidIUrOMqEXIxqj+wsfjBHSCl6lRinUnKyjtsI3K1J9YS5g7+vc3uPdx+OPeyGH6ztbu9W/gQDBG3T6nKrtbuodqoH240sqmVWon6nAbpZh56NKk54gy5KMtrPzQLrepTo3El8oivQ7Gkd+tcHR+GJru/RoXryiRJrkqDl1BqD4W0hYCX8oFIYez9R2tggpZWAU451aYKLCOYMlFfhW3yyWjcWqY8mGKMedG61JGpP6jQzkKn2jwdqWaXrblKdmMdroduS+YHHqvzJ6xnB71gY+uigmyGv7+P2e56OFx51WBrLyv+Pf7wzsUB65bvXZg/drGrOy0BTizHIgPaVkbRh2pgmRw+3Jhn8D6qssPInn9vjd0JkfGIJ65b+eHFNvRGlyPGDHYzdZKw3AlYsD0FnWynf/4GO28lvBR4XwCkZqLIbGqA2ZtpUSr0K/yPljfREPYDIQBTaNExY/ttLDx0+2ss57bs1F+ZKIuRnnKECU2ZSVrRNGm3tOM5iihSzyQpW6el+UYYriJRBznvELZ9SMJ9k8kBh/OALjeHOb0QdmdsSVlnqOB0EYvDaKWOvrwqDdQqf9oBVDg3H1j47jLQ3ty21pnp1MKJVOeoJG0H+mGrir8M4mN7HkTYED2a3BmS9Zr54wkRnDmw5pwlACrU+5txBkjzzSH1vJZRinaKA2nr3LDz9wUP0+ZIZ9727b+8dAcEtemcfgStVnkR5Y0Njg6qvhFo9afIylphVC/zPmQkiAV4LrapYIj6zNW4Fs2ohzYvCIqp2qQnJ3DU+lOi1nnNGFCMFgMcSdXuZ4OsVFkrBKLyT5hLAxHApbP6Yi/hgyEq/rYhTNqbEMySwOFtWo5DQmUBt5OWEti2ftLbyxw8mX8L3So2WKjx67AYN+cognd6/QNY5mCcDC8XxV7hqVNCx8nYi3znKXYrISA+v6jS86eUMtP FeHoFsxZ lTxLwhOPD5nGyRhJLpDIXPrZYQA4teW5ubOkJxUOHf0SUPfJd0FqHLgxvDJDCtMNuIOfyWU1d++GckHOX3XV5r66NIQozoXaIfGekOMBJOWnoXQRX1Blyq7JETbNMfLsKYl/HwRVeqD3gjI5rGc8ib1AsEjoJd4a6kbiQIGnBJH9XNFMdJ6k8ntlwE+Psd6I/DRH0VmaElPqekrabzKaubDW5cYT0emXZBwMrTG6F7k3h8q0kPeNTEgStH0ovOmfVupBBxfs+i3Yf4BH5qLkwydXGe1LYtrN2hp5VQoRxm//F5R78QrinGVo8SR5dmB6uuq3BZXB28f+qA0OrFAek1KAoKcuTTxmbWm8LeIBIDbH4JzDDrAk1mR1RWdXjlRD/+G+/tB2Q+Mp/2LtOC8Ty0SIZWSVV2L7w50ai17rBlwdGN6RewtfPX4jUf/nvDpcLK/fuEmYMNyMIkCi0B4yeCoAomEcT5WWgHzXAwhGruM6EqKtkm1BVQ3O+P12H/5l5BJwwJtwQMn+aVCGt9Bxozsbe0acRoehGuFMfBqrq67Gs24cYIj8rEBfwIxkqfxcNkaMb X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Code cleanup for both better simplicity and readability. No functional change intended. Signed-off-by: Abel Wu Acked-by: Shakeel Butt --- net/core/sock.c | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/net/core/sock.c b/net/core/sock.c index 666a17cab4f5..af778fc60a4d 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -3040,17 +3040,19 @@ EXPORT_SYMBOL(sk_wait_data); */ int __sk_mem_raise_allocated(struct sock *sk, int size, int amt, int kind) { - bool memcg_charge = mem_cgroup_sockets_enabled && sk->sk_memcg; + struct mem_cgroup *memcg = mem_cgroup_sockets_enabled ? sk->sk_memcg : NULL; struct proto *prot = sk->sk_prot; - bool charged = true; + bool charged = false; long allocated; sk_memory_allocated_add(sk, amt); allocated = sk_memory_allocated(sk); - if (memcg_charge && - !(charged = mem_cgroup_charge_skmem(sk->sk_memcg, amt, - gfp_memcg_charge()))) - goto suppress_allocation; + + if (memcg) { + if (!mem_cgroup_charge_skmem(memcg, amt, gfp_memcg_charge())) + goto suppress_allocation; + charged = true; + } /* Under limit. */ if (allocated <= sk_prot_mem_limits(sk, 0)) { @@ -3105,8 +3107,8 @@ int __sk_mem_raise_allocated(struct sock *sk, int size, int amt, int kind) */ if (sk->sk_wmem_queued + size >= sk->sk_sndbuf) { /* Force charge with __GFP_NOFAIL */ - if (memcg_charge && !charged) { - mem_cgroup_charge_skmem(sk->sk_memcg, amt, + if (memcg && !charged) { + mem_cgroup_charge_skmem(memcg, amt, gfp_memcg_charge() | __GFP_NOFAIL); } return 1; @@ -3118,8 +3120,8 @@ int __sk_mem_raise_allocated(struct sock *sk, int size, int amt, int kind) sk_memory_allocated_sub(sk, amt); - if (memcg_charge && charged) - mem_cgroup_uncharge_skmem(sk->sk_memcg, amt); + if (charged) + mem_cgroup_uncharge_skmem(memcg, amt); return 0; } From patchwork Fri Sep 1 06:21:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Abel Wu X-Patchwork-Id: 13372078 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43D8ACA0FE1 for ; Fri, 1 Sep 2023 06:24:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C5F898D001E; Fri, 1 Sep 2023 02:24:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C36AC8D0002; Fri, 1 Sep 2023 02:24:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B30968D001E; Fri, 1 Sep 2023 02:24:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A2EC28D0002 for ; Fri, 1 Sep 2023 02:24:10 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 7780CA0146 for ; Fri, 1 Sep 2023 06:24:10 +0000 (UTC) X-FDA: 81187038660.18.802F015 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) by imf29.hostedemail.com (Postfix) with ESMTP id AD460120012 for ; Fri, 1 Sep 2023 06:24:08 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="LoAjBs/W"; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf29.hostedemail.com: domain of wuyun.abel@bytedance.com designates 209.85.210.181 as permitted sender) smtp.mailfrom=wuyun.abel@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1693549448; a=rsa-sha256; cv=none; b=PBmiJFqN4V8QjJwO9Z3mNutV5jgObQuB/jlICEmFpJM5hl6r45Z77UctT9pL0qoezAAdLb 3smyY9ICKwkMdQs3QUnWzaOyPbA8BuXZlVeCEumbKbwwrXTFRgnm8P7llkKPZJBHBTC8o6 WShyXGPTvQfXRY0qzIeqEnM9kA61UC8= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="LoAjBs/W"; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf29.hostedemail.com: domain of wuyun.abel@bytedance.com designates 209.85.210.181 as permitted sender) smtp.mailfrom=wuyun.abel@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693549448; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZQ6PDv993Rp2fl4j4E1LU63DQukXqNgX8LZTNx5xQ8Y=; b=GXbc3ihB1QkkJNJaLvJpsRiK0UjkB5FdAngKPaxIOYPtsIec6i6xCI2yUJQa9zH6w51H/q uo47Wu8D5i53Y1VOhq8KRWCIzWWgSyITRN9sbgM2vVjttrtF8V+PMs1g/o3UmXWlAOYvJ/ Rh3t/XeD4hafyRhjgVoGGFsDlCVXV50= Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-68a3b66f350so1389129b3a.3 for ; Thu, 31 Aug 2023 23:24:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1693549447; x=1694154247; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZQ6PDv993Rp2fl4j4E1LU63DQukXqNgX8LZTNx5xQ8Y=; b=LoAjBs/W/JDRV2+RlrQwMhDK4UxyIAPcsckN8Br2JY123rqDVzHVgwxSMlKCCT31/s SOiO3Xkf1joTQVBCUkisn/Uj1kX2pUt5/DMK4Gn1hmB43rh0MulR5Prxcaq6bQ8xjDfP 2b9/82VDaWPrDJDqLHOilbpqgcrCP8XEe6zIhWzC/0dCrK0topbSzoWjsLaRiLbPqoO+ yNHcNy4tFxJzivBvobSu3m5hUjlZMUb2bqJ+ZSETzIEC3PsOLSGvATI37LHvJpY4kNcw XkUVXNkpuEPeWvDDJ0laYlekwQr7hPyBdnalcDXWRK+BNgQWBof4YjUhU+mD1fRg75gI IP5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693549447; x=1694154247; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZQ6PDv993Rp2fl4j4E1LU63DQukXqNgX8LZTNx5xQ8Y=; b=DULe+iMqDqEda4AWLcWXJcGoc9J7vHpGdaZywnEH7FQ2IDrMngFg+1frFbpoPqffw8 uRQLMcOtuy9F8+D9IiMqBa+Im61LnqbwJQE0RvEA4QyPGZiHMeK1ryxuR5+Ncc8aKYE6 lidh5klRFbLoznr20d8jKDbXUnygPDpWfsi2g5/iZI4ooL/Z1zdvkMA79fUrpndiaOl0 l8xDkJjTgJA657JOk+gibsIXtOY86JPNr8ZoC4NGd5SUByD1fmTNRlBy0ExAbsD7Q4BO naMoTfLah5S+3FABKobrPtguHj5SOx78zudX9Hy4koX5oGg39Lm0cNrwTWgKK4NNP91V /a4Q== X-Gm-Message-State: AOJu0YwlieKyzNju1kZS0me1dbQOvm4r0NtMrshrgvYS+yPxrN27CIu6 UUnITFLx7Yclvp6x5epTTz9YHg== X-Google-Smtp-Source: AGHT+IFMqQT61UT4Tg0IU3qJAIzxQXgO4DhEf5zuPhELzVvTcr8/BRgzyzW20KePzBL0PUk/x6gtKA== X-Received: by 2002:a05:6a00:17a8:b0:68a:66b2:3d15 with SMTP id s40-20020a056a0017a800b0068a66b23d15mr2442799pfg.7.1693549447179; Thu, 31 Aug 2023 23:24:07 -0700 (PDT) Received: from C02DV8HUMD6R.bytedance.net ([203.208.167.146]) by smtp.gmail.com with ESMTPSA id fm19-20020a056a002f9300b0068c1ac1784csm2223265pfb.59.2023.08.31.23.23.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Aug 2023 23:24:06 -0700 (PDT) From: Abel Wu To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Andrew Morton , Shakeel Butt , Roman Gushchin , Michal Hocko , Johannes Weiner , Yosry Ahmed , "Matthew Wilcox (Oracle)" , Yu Zhao , Abel Wu , Yafang Shao , Kefeng Wang , Kuniyuki Iwashima , Martin KaFai Lau , Breno Leitao , Alexander Mikhalitsyn , David Howells , Jason Xing Cc: linux-kernel@vger.kernel.org (open list), netdev@vger.kernel.org (open list:NETWORKING [GENERAL]), linux-mm@kvack.org (open list:MEMORY MANAGEMENT) Subject: [RFC PATCH net-next 2/3] net-memcg: Record pressure level when under pressure Date: Fri, 1 Sep 2023 14:21:27 +0800 Message-Id: <20230901062141.51972-3-wuyun.abel@bytedance.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20230901062141.51972-1-wuyun.abel@bytedance.com> References: <20230901062141.51972-1-wuyun.abel@bytedance.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: AD460120012 X-Stat-Signature: pp1psjbe1c1ngwd8ng1tsqdbn6xtmosj X-HE-Tag: 1693549448-109911 X-HE-Meta: U2FsdGVkX18TXL7yNYhf7Wo3p6d3UqhA63kjO5HeDzBI+UIRDFLkCukgASIruJjjdpHWKHS+OvPuPo0PZ73BEbHoSUXQejUcSDjBQRkuWrfU0gPhG7ljRIeZcsHsWyTujSFppElQ41mPK/yOgO/tCNvLPF/UVvjQUPI7Hrl7eFxq0bEmLXvGfkvM3Ub6ky6/R/hB+ViknxLhtalPEzAed+0pQt2966w/rR4s9SS6pvxkFgL6nJtUzkqROZPM1U6RaBH//KQYLXeIi1GS1tqCH+sF0B56C3UCM+9AQj2JrbiQaSDvuYKWKVoTnjhZaKywFpmscdVMWvhDvqVWTof+SK0W1ncIyQCkQ5DOeylCdl6c4LgBIjOoBXWcj6ipfBZHGvrBoASpUbjj4A3r8P7hSFDdrQa97g7ghyNI8JD2b6+Ju5msrfgJgK0InTe0h8RSxig1hbjXdetJwNG4BIp5TEhu5A1dAKJQKUNSBUSW1cH4SkSEhDLU44RSZ6Xb5ul9Ejs51cQcl6e7GSPww7mfFs/uG6SKJ+xLEdCVEsaRlW/0E2/ubUbSXZnLZvYnZKxfI1O4//2Aieh70Pk7Aj7unl0qcoA6ZOjMNocTBOCNoRd5jo2C0izJ1LilWaaNdXTM0+CSTouZX3zAK4IIlE9ngmaW7y3KZijPRiXG2qbO5qpIV5+bgsy0ugPR0XGsqCWlMvYSRaST1CNVGMqpuueGi52idZbvNp013k1LhMAXXrPFROwzgf/5q170z1KANAJNH3DMczQPsE6OhGcRrmgx/d3bBJF3ACPJ131o0ggcGfySsxnzpqSn4KCcDkcwG+nDYY1ol4WoCJ845FhdCVeSWfprJhxijsePDK35V+FsUbYyq9Nh2UpAOh2uYxAauf2bAE2HvwxsY0OiPT4SIZTISjRzo1kiDWeKvhhW0jlx2RQe4eY22r0CFm8OZ4yhgEyBhW267FqdX9K58zxZVbg JFb1oake lic47aTbLz1MLPQHvFK4yLMZ6q2mG8kD29J26G4Qfswn9s/pVPtxr3NBqmvd3FH8woi7716k3zAUFZ4Yu//uy3AhuPTWuWNvh2jO70oaW/rvYzcgqNnJDfrFSZ0eV+Lufdl2vbG85hZUVbifBOaaaZXEYHVFImIKNTrg8U9It10u3zyGo0nDDq3X1IcmtyZ64JLSOaEkz8/Vvatz+NW6sIcguihfvCCNwaJDjbxOs8vTw84GRErnWRkXW/gnIT1WUyK1I/tX0F32Sk2cEeVrkklh6OVcQytiyqfY8RMpwd5VegnQHBXwM4AW8MaAnQq+QZQfnRocqfbfqWl/6f6slruy23+AEowPvGmJEl+HHZAwKwgxdjpDxJvofrI8Qt62uNkCnB1B38DVOyzebs7f33AK/1BczGrlCFpJrDAhKYd1I3yPfZw2qS1GbIRGBVaRjBSYSm/lztKk6vu8LbJi9xsXzaicCw3Dh5gMoxyxLPZdD+mc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: For now memcg->socket_pressure is used for judging whether there is memory reclaim pressure in this memcg. As different reclaim efficiencies require different strategies, recording the level of pressure would help do fine-grained control inside networking where performance matters a lot. The vmpressure infrastructure classifies pressure into 3 levels: low, medium and critical. It would be too much conservative if constraining socket memory usage at "low" level, so now only the other two are taken into consideration and the least significant bit of socket_pressure is enough to record this information. Signed-off-by: Abel Wu --- include/linux/memcontrol.h | 39 +++++++++++++++++++++++++++++++++----- include/net/sock.h | 2 +- include/net/tcp.h | 2 +- mm/vmpressure.c | 9 ++++++++- 4 files changed, 44 insertions(+), 8 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index dbf26bc89dd4..a24047bf7722 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -288,6 +288,9 @@ struct mem_cgroup { * Hint of reclaim pressure for socket memroy management. Note * that this indicator should NOT be used in legacy cgroup mode * where socket memory is accounted/charged separately. + * + * The least significant bit is used for indicating the level of + * pressure, 1 for 'critical' and 0 otherwise. */ unsigned long socket_pressure; @@ -1730,15 +1733,40 @@ extern struct static_key_false memcg_sockets_enabled_key; #define mem_cgroup_sockets_enabled static_branch_unlikely(&memcg_sockets_enabled_key) void mem_cgroup_sk_alloc(struct sock *sk); void mem_cgroup_sk_free(struct sock *sk); -static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *memcg) + +static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *memcg, + bool *critical) { + bool under_pressure = false; + + /* + * When cgroup is in legacy mode where tcpmem is separately + * charged, we have no idea about memcg reclaim pressure from + * here and actually no need to, so just ignore the pressure + * level info. + */ if (!cgroup_subsys_on_dfl(memory_cgrp_subsys)) return !!memcg->tcpmem_pressure; + + if (critical) + *critical = false; + do { - if (time_before(jiffies, READ_ONCE(memcg->socket_pressure))) - return true; + unsigned long expire = READ_ONCE(memcg->socket_pressure); + + if (time_before(jiffies, expire)) { + if (!under_pressure) + under_pressure = true; + if (!critical) + break; + if (expire & 1) { + *critical = true; + break; + } + } } while ((memcg = parent_mem_cgroup(memcg))); - return false; + + return under_pressure; } int alloc_shrinker_info(struct mem_cgroup *memcg); @@ -1749,7 +1777,8 @@ void reparent_shrinker_deferred(struct mem_cgroup *memcg); #define mem_cgroup_sockets_enabled 0 static inline void mem_cgroup_sk_alloc(struct sock *sk) { }; static inline void mem_cgroup_sk_free(struct sock *sk) { }; -static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *memcg) +static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *memcg, + bool *critical) { return false; } diff --git a/include/net/sock.h b/include/net/sock.h index 11d503417591..079bbee5c400 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -1434,7 +1434,7 @@ static inline bool sk_under_memory_pressure(const struct sock *sk) return false; if (mem_cgroup_sockets_enabled && sk->sk_memcg && - mem_cgroup_under_socket_pressure(sk->sk_memcg)) + mem_cgroup_under_socket_pressure(sk->sk_memcg, NULL)) return true; return !!READ_ONCE(*sk->sk_prot->memory_pressure); diff --git a/include/net/tcp.h b/include/net/tcp.h index 07b21d9a9620..81e1f9c90a94 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -261,7 +261,7 @@ extern unsigned long tcp_memory_pressure; static inline bool tcp_under_memory_pressure(const struct sock *sk) { if (mem_cgroup_sockets_enabled && sk->sk_memcg && - mem_cgroup_under_socket_pressure(sk->sk_memcg)) + mem_cgroup_under_socket_pressure(sk->sk_memcg, NULL)) return true; return READ_ONCE(tcp_memory_pressure); diff --git a/mm/vmpressure.c b/mm/vmpressure.c index 22c6689d9302..5a3ac3768c0f 100644 --- a/mm/vmpressure.c +++ b/mm/vmpressure.c @@ -308,6 +308,13 @@ void vmpressure(gfp_t gfp, struct mem_cgroup *memcg, bool tree, level = vmpressure_calc_level(scanned, reclaimed); if (level > VMPRESSURE_LOW) { + unsigned long expire = jiffies + HZ; + + if (level == VMPRESSURE_CRITICAL) + expire |= 1UL; + else + expire &= ~1UL; + /* * Let the socket buffer allocator know that * we are having trouble reclaiming LRU pages. @@ -316,7 +323,7 @@ void vmpressure(gfp_t gfp, struct mem_cgroup *memcg, bool tree, * asserted for a second in which subsequent * pressure events can occur. */ - WRITE_ONCE(memcg->socket_pressure, jiffies + HZ); + WRITE_ONCE(memcg->socket_pressure, expire); } } } From patchwork Fri Sep 1 06:21:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Abel Wu X-Patchwork-Id: 13372079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AE85CA0FEB for ; Fri, 1 Sep 2023 06:24:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 29C198D0020; Fri, 1 Sep 2023 02:24:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 24CBF8D0002; Fri, 1 Sep 2023 02:24:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 13B148D0020; Fri, 1 Sep 2023 02:24:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 043338D0002 for ; Fri, 1 Sep 2023 02:24:28 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id C9026C0138 for ; Fri, 1 Sep 2023 06:24:27 +0000 (UTC) X-FDA: 81187039374.01.63984F7 Received: from mail-oi1-f177.google.com (mail-oi1-f177.google.com [209.85.167.177]) by imf08.hostedemail.com (Postfix) with ESMTP id 041C9160006 for ; Fri, 1 Sep 2023 06:24:25 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=VheQXM6t; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf08.hostedemail.com: domain of wuyun.abel@bytedance.com designates 209.85.167.177 as permitted sender) smtp.mailfrom=wuyun.abel@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1693549466; a=rsa-sha256; cv=none; b=Tl1Vwc88rPSibT4ShPehHcTGQzTWjmtFTIRQib7U8yJ+L70R9JknxQPTNNwsj+BP0oFM0y Bv8820CPw+LNJSjvOpyi1Pktr7WJwc89EO3iPr+CpX7NljOtcEPZXj68IVSVvVnCH+TLOm 7CAyn7I8sqrYfq1IaHQQEDotcdsQlhI= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=VheQXM6t; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf08.hostedemail.com: domain of wuyun.abel@bytedance.com designates 209.85.167.177 as permitted sender) smtp.mailfrom=wuyun.abel@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693549466; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0JmFLfbSmI2emm6GyT4GHqCHxeUZuXTxsBg8j/3SD1Q=; b=Pogc9TDGfbMODSL1s96OHSCgbfAi8DgfozGvbIoNOIY7lhvMEZ6n3j93LMuvHU+9n5j1OS QpUY8U2qgo4LZcjNJn2xMuhsey28dA9sNvZy0p27aAnrEil8IctlIvhjyrA2xRxkl0xUOs 8xw6HxwAPuEpQZrFwNrAP7gHtxlhTSY= Received: by mail-oi1-f177.google.com with SMTP id 5614622812f47-3a7e68f4214so1058962b6e.1 for ; Thu, 31 Aug 2023 23:24:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1693549465; x=1694154265; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0JmFLfbSmI2emm6GyT4GHqCHxeUZuXTxsBg8j/3SD1Q=; b=VheQXM6tDLWJupwWkDF/f1KwPzAqEluvBL7U9w6/X5ogvqKWr9oiBHYeTHHwqFm8Hu yNI1o+LHgPYo+7S7MwsMFsipDeJsjirv4OGcSATwJ2Zo/yc0j+paESh7XM00oXJYXZj/ 2UABripYY+UYberfRtVdo0hdPCP4pRmU5chRZtdZEBEc7c7scXgtZy3Sk/M/2qMOMyDb ju9IMaENYRaFj5kpiawW49QWjxPz2kL25Rih2Ch9nCL/Upl8yogUt7GR/Kd2fWyUzuoU GzQgC03QtIR8rQ3ZH7K78S3ZOW1yN5QjYghTX50SFCJ4zpfWJSTzhcENWC7Di2VdY8T4 1qdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693549465; x=1694154265; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0JmFLfbSmI2emm6GyT4GHqCHxeUZuXTxsBg8j/3SD1Q=; b=XwrUqPg6KT4O2vcqfbrSeiPBkr6iDuEaAfRI7A+ipMrvNHZluvsnjuRHTNTneHHHPv RtUBxeVz4PgIfxHQVwMPSgCQw7hJ1WyZ+uT35vHD7vLKeVFROWMik9SHapZxRZunKfRW ckDPLUIL2nNncnUdnNKZpTNudgYe9ba98WPxoTZi23sBDMH1/2KCVwVvRVjB1263OUj6 Eo9tP6xvRU4Y3aFgjxPs1RFuPOWkKn6jWGbfYj+CVR1QIHshA4e7m1S0DUBiJPhVUCuQ oCVdeOgvjZN1ZiWwawsy+ozY2jAiTOD2WXx33Pasp64sqcNBOpQKzoiLYpLND6ZPDQ2y vDVg== X-Gm-Message-State: AOJu0YzIEWYMz3pQ/nk6MoL+nu/A3MXz6xDBB44mi0kQFkxelzh6PtB+ 904k3jPIb+hJx6nEETm8rvuAcQ== X-Google-Smtp-Source: AGHT+IFA+ZQPRlE/tLVc4UK8tlm+PIJyTz7163uxOg1QtvjOk/HKCDfhxw5xWkpOivUFycx5AMQukQ== X-Received: by 2002:aca:2b16:0:b0:3a3:eab8:8c40 with SMTP id i22-20020aca2b16000000b003a3eab88c40mr1685121oik.54.1693549465089; Thu, 31 Aug 2023 23:24:25 -0700 (PDT) Received: from C02DV8HUMD6R.bytedance.net ([203.208.167.146]) by smtp.gmail.com with ESMTPSA id fm19-20020a056a002f9300b0068c1ac1784csm2223265pfb.59.2023.08.31.23.24.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 31 Aug 2023 23:24:24 -0700 (PDT) From: Abel Wu To: "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Andrew Morton , Shakeel Butt , Roman Gushchin , Michal Hocko , Johannes Weiner , Yosry Ahmed , Yu Zhao , "Matthew Wilcox (Oracle)" , Kefeng Wang , Abel Wu , Yafang Shao , Kuniyuki Iwashima , Martin KaFai Lau , Breno Leitao , Alexander Mikhalitsyn , David Howells , Jason Xing Cc: linux-kernel@vger.kernel.org (open list), netdev@vger.kernel.org (open list:NETWORKING [GENERAL]), linux-mm@kvack.org (open list:MEMORY MANAGEMENT) Subject: [RFC PATCH net-next 3/3] sock: Throttle pressure-aware sockets under pressure Date: Fri, 1 Sep 2023 14:21:28 +0800 Message-Id: <20230901062141.51972-4-wuyun.abel@bytedance.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: <20230901062141.51972-1-wuyun.abel@bytedance.com> References: <20230901062141.51972-1-wuyun.abel@bytedance.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 041C9160006 X-Stat-Signature: 8qjdnsj9foh9g787rd1gikkcfiqqk3nd X-HE-Tag: 1693549465-697011 X-HE-Meta: U2FsdGVkX1/qc94Yy23itUw8rOfxMFbYn9dO//tFQU48HXuqnx4kj6U8QJCfokemzrQEgxoZPtKHerZ3I9rfxlkFIOCH2HV3kJwy+lflH/bOVVSRELOlKOZCf98PDW7xnMmgrZ+JI0gn8DMKJ5ZvdS0ogbtAmCkAQrWdSA/9gSV0GSD6F7tFrHXke1AlaZBlfBq4VpSV8WdZSmAl/7tpBPweW5tC+jyUHTy6EDS0yTTfNoJ26cpKSLFs3b8ajHH4Dgr+UPF8fkV/e5SuTcPCg+nIDxSogXGWkcvlrlPEaoyLcppUW5TZBjQ4FOcCc2Uk75mEeWwp5BL+NuMahaT/alUYhqJTlPXY1e3aSPmq4a3UKgbXRgc4wLL0f1G1xiWO7jSDr3vha2feBekKTZHdvTzHD9Iryo84Kbc6cy5EYBePRKA5+CQnyyICykT8ZylqCdJfTiA9YCB/h/W6ciTlDUyy3oDqgBmss80W10XHv1H5aOPr/Nr+7wdi4/zXudytA7Qu97cnbcV0GWrHAP2ey+WFRekGSj6PHAqN+sApHNptRaAO3TyBttAPicvnv12riHXQ2WJv6BzDnU9dulW1ruiiOEicOUEL3UWoCpBRpurM6LQGmt6tOxwWW+WDz/GOmMeLQEdh10EWMY7Ov2hwyFWSOhCn1cuvZk7/RUjEBlXhjNQ9W2aBd4Rx3vJ4AXaSbX6Py733aLfjbS0khhs9N3cD+nY9o2ibWuE16Xl7pAUJeyq/YPQr3iwm7kAEBgmiuREYj3NQCeUJa+q+dVMg+j4+XxMcw10HcsIXuuqif4RzjDc+7JOEvIzMVOfdjfLDMyQspVyu/RhC0aYWhY8Uyf3np+QgyQOg2fVQFFGU7MG9Os3WG2wZGMVz08vMRg+w8KTmiI/EBsaP8Ot46qG29MAUL4o9Tw5bF4ZZUYzTUz4gkoAT+XFSEMpyqXEz0FdDbRx7mqZSRJ9ufhUbF8m yvQYZsC2 zk0albD7KjLGGkJRy8YDMYsDlnMNlm+hCTqWoNgSUiqPlqwVdR9c4zceRywJ6panhooh0Xwpio77AKdPimY6juxYtMafkiO93UCW68g79+YIGcXFpWls5vaPJJad/OaugCPDAMYkhH5I4OHzlzL7Wt3nc1SV3csBmczVIRhwzZX5SdMo/5QMEubjNRA3dr4obrwovKQtWSyWE/UJH6AohLZXWeApuZokAY27c6mnsxB2HQMQfgZKsLiALXxroZHkDMjgqSUpSu0YZnS7h5/CxeNg1oxFEECmZ5pw/14KI64c9SvinONZwx7chPniTIezs1KRckO7ysrWjCrRG+sFOv2NYmqk0dkmEEUr9sHWFTqC12YxOsJSVQ+MWg/ViPEWn5JDOzUWCQqe7ODVBGnHIBwTHaefoN8MX0zW8fERfV3VXj/d0R5/NQeCigvQNcTnaIYOlS3doDtUC58UZ3yfpoPepfozgXGr+J4HXpty9fM2SbzQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A socket is pressure-aware when its protocol has pressure defined, that is sk_has_memory_pressure(sk) != NULL, e.g. TCP. These protocols might want to limit the usage of socket memory depending on both the state of global & memcg pressure through sk_under_memory_pressure(sk). While for allocation, memcg pressure will be simply ignored when usage is under global limit (sysctl_mem[0]). This behavior has different impacts on different cgroup modes. In cgroupv2 socket and other purposes share a same memory limit, thus allowing sockmem to burst under memcg reclaiming pressure could lead to longer stall, sometimes even OOM. While cgroupv1 has no such worries. As a cloud service provider, we encountered a problem in our production environment during the transition from cgroup v1 to v2 (partly due to the heavy taxes of accounting socket memory in v1). Say one workload behaves fine in cgroupv1 with memcg limit configured to 10GB memory and another 1GB tcpmem, but will suck (or even be OOM-killed) in v2 with 11GB memory due to burst memory usage on socket, since there is no specific limit for socket memory in cgroupv2 and relies largely on workloads doing traffic control themselves. It's rational for the workloads to build some traffic control to better utilize the resources they bought, but from kernel's point of view it's also reasonable to suppress the allocation of socket memory once there is a shortage of free memory, given that performance degradation is better than failure. As per the above, this patch aims to be more conservative on allocation for the pressure-aware sockets under global and/or memcg pressure. While OTOH throttling on incoming traffic could hurt latency badly possibly due to SACKed segs get dropped from the OFO queue. See a related commit 720ca52bcef22 ("net-memcg: avoid stalls when under memory pressure"). This patch preserves this decision by throttling RX allocation only at critical pressure level when it hardly makes sense to continue receive data. No functional change intended for pressure-unaware protocols. Signed-off-by: Abel Wu --- net/core/sock.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/net/core/sock.c b/net/core/sock.c index af778fc60a4d..6c1d13547f1b 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -3041,6 +3041,7 @@ EXPORT_SYMBOL(sk_wait_data); int __sk_mem_raise_allocated(struct sock *sk, int size, int amt, int kind) { struct mem_cgroup *memcg = mem_cgroup_sockets_enabled ? sk->sk_memcg : NULL; + bool under_memcg_pressure = false; struct proto *prot = sk->sk_prot; bool charged = false; long allocated; @@ -3051,13 +3052,25 @@ int __sk_mem_raise_allocated(struct sock *sk, int size, int amt, int kind) if (memcg) { if (!mem_cgroup_charge_skmem(memcg, amt, gfp_memcg_charge())) goto suppress_allocation; + + /* Get pressure info from net-memcg. But consider the memcg + * to be under pressure for incoming traffic iff at 'critical' + * level, see commit 720ca52bcef22 ("net-memcg: avoid stalls + * when under memory pressure"). + */ + if (sk_has_memory_pressure(sk) && + mem_cgroup_under_socket_pressure(memcg, &under_memcg_pressure) && + !in_softirq()) + under_memcg_pressure = true; + charged = true; } /* Under limit. */ if (allocated <= sk_prot_mem_limits(sk, 0)) { sk_leave_memory_pressure(sk); - return 1; + if (!under_memcg_pressure) + return 1; } /* Under pressure. */ @@ -3087,8 +3100,20 @@ int __sk_mem_raise_allocated(struct sock *sk, int size, int amt, int kind) if (sk_has_memory_pressure(sk)) { u64 alloc; - if (!sk_under_memory_pressure(sk)) + /* Be more conservative if the socket's memcg (or its + * parents) is under reclaim pressure, try to possibly + * avoid further memstall. + */ + if (under_memcg_pressure) + goto suppress_allocation; + + if (!sk_under_global_memory_pressure(sk)) return 1; + + /* Trying to be fair among all the sockets of same + * protocal under global memory pressure, by allowing + * the ones that under average usage to raise. + */ alloc = sk_sockets_allocated_read_positive(sk); if (sk_prot_mem_limits(sk, 2) > alloc * sk_mem_pages(sk->sk_wmem_queued +