From patchwork Fri Nov 8 21:29:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joshua Hahn X-Patchwork-Id: 13868973 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E02A3D6408E for ; Fri, 8 Nov 2024 21:29:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5E5E28D0001; Fri, 8 Nov 2024 16:29:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4FB59900007; Fri, 8 Nov 2024 16:29:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D7718D0001; Fri, 8 Nov 2024 16:29:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id F24496B00CA for ; Fri, 8 Nov 2024 16:29:52 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A79911A0A89 for ; Fri, 8 Nov 2024 21:29:52 +0000 (UTC) X-FDA: 82764219048.10.B04DD14 Received: from mail-yw1-f178.google.com (mail-yw1-f178.google.com [209.85.128.178]) by imf12.hostedemail.com (Postfix) with ESMTP id E75064000B for ; Fri, 8 Nov 2024 21:29:33 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=dTMX9tUR; spf=pass (imf12.hostedemail.com: domain of joshua.hahnjy@gmail.com designates 209.85.128.178 as permitted sender) smtp.mailfrom=joshua.hahnjy@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731101202; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SlQ3sjJT81bsTmwr1uay84sXlsybFvKefjGpPF85ZWQ=; b=Vf60yLGgoRzvjZ8Lz9kLCy1JYNxkLG4aII/xNLCwWfBSge37p8K79VOjle3fBlN6ZChIgF 9c2bGkYjanxfaUBNvveUn+LfQ6ynDx6mHZnca1vd/BmrKG96N0G3i4a4hcnzT59/PUh9hG mKTTN6IMomRChmglDyRAyQqhLNU953M= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=dTMX9tUR; spf=pass (imf12.hostedemail.com: domain of joshua.hahnjy@gmail.com designates 209.85.128.178 as permitted sender) smtp.mailfrom=joshua.hahnjy@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731101202; a=rsa-sha256; cv=none; b=UVX/z5Cc5tJfBXxB0s9lfFIhGeUlEfDfUHX4FMrE+AH6HHcOlECDrTxEt+KbE+wqWxQuaN jBQmeixcmPn3mWzHV4ondzlLPtfCOyX6L27iMnt8S/lY86PzqL8iVy692bGq0BMxSgXnMT KuDUpK+am58o6FTPrg8kxkvPPnULRy8= Received: by mail-yw1-f178.google.com with SMTP id 00721157ae682-6e38fc62b9fso23438587b3.2 for ; Fri, 08 Nov 2024 13:29:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731101390; x=1731706190; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SlQ3sjJT81bsTmwr1uay84sXlsybFvKefjGpPF85ZWQ=; b=dTMX9tUR/sOzHWLgwM8mVKW8wl66HtiTSoXH6Vcs3v76oz9HWte1e25G+bgKFwfixU IQRMwq3qhNPgmPWWpQbmngFXrihK6oxvJ1Wu3/w/HYpzv27u//2Jx74tuCrAcrr+Z4ef z56hfLAfSwA4PalIZkMW0dzOTFdq/mVxxfNr0+SyBoqamJjIN4PtWEihTzQ0tIdOBAXL WPkztx6Ji+iyd1NP5bRz8pqLFvcNTMOhiLr6JfzXL+DVylz6+ED+onWId5SlR8Ecaju9 zGp33qiXjQEV/n7d5Ws53KNfOI3tlu506kOYceupFxjLjWE/3j3+L4E05dzGDpsgqf/9 xtQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731101390; x=1731706190; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SlQ3sjJT81bsTmwr1uay84sXlsybFvKefjGpPF85ZWQ=; b=B+MANEaNVbtyVfjCeXdEEbxZvMqtUjZkpaZfDWimL/OrWN6ikpX0I5++RRpeNRGDcI uVNneX3UrmIul5eyY3Hm6TvycPeNmyZv+HfOYb09XZNPYkY6JitLkFQx7tRKz71rOIV0 6YsQObA5MXLuIBX/IaaXa87eWuQBh7PgmNTIXxFLkJAMTYazKTerIm2p2X/IZ4EJ58Jk n96OnRAcS1vv1Ni3AeoKxcp/AF8wQtz9KacxUyUeyDOaFMPBeWgFGBojaSkgKnDzss2S wuNv6wi/loGC7yHqV6i4KvGXtUJHin+7jbO0cTVNq5jxIuVMSGTSMQMqsssCRjakvgsv u8lQ== X-Forwarded-Encrypted: i=1; AJvYcCVCHTb5ng3V6x/VrmohfhQ63+d0x27HxA45oerjJ8WLBm7AgzE/nJwZN+UM5/2QEPwtnfFUlSNGYA==@kvack.org X-Gm-Message-State: AOJu0Yymq/SQ/fX4qJ+aWCM6pugaF0g/pN6VmSDpi9MZexItArNiFWNL NV+HyHSbZBRy+j0RUBTdqXMwcvPg8iUVIj4N/Y1/rmcIcD/0qO57 X-Google-Smtp-Source: AGHT+IFYGeqK9WHN6tzmVfn8RrbnYZR+fESmtaRa7GS9uP5CtuQHP1PNtDJrSMzfXRi9Cqzl4/l5eQ== X-Received: by 2002:a05:690c:7308:b0:6e3:3521:88ff with SMTP id 00721157ae682-6eaddd9507amr55794997b3.18.1731101389891; Fri, 08 Nov 2024 13:29:49 -0800 (PST) Received: from localhost (fwdproxy-frc-005.fbsv.net. [2a03:2880:21ff:5::face:b00c]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6eace8ee1c0sm8850617b3.28.2024.11.08.13.29.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 Nov 2024 13:29:49 -0800 (PST) From: Joshua Hahn To: shakeel.butt@linux.dev Cc: hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, muchun.song@linux.dev, akpm@linux-foundation.org, cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@meta.com Subject: [PATCH 2/3] memcg/hugetlb: Introduce mem_cgroup_charge_hugetlb Date: Fri, 8 Nov 2024 13:29:45 -0800 Message-ID: <20241108212946.2642085-3-joshua.hahnjy@gmail.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241108212946.2642085-1-joshua.hahnjy@gmail.com> References: <20241108212946.2642085-1-joshua.hahnjy@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: E75064000B X-Stat-Signature: k4fs3ndnc4bae5rrsi986g8yco79hjhg X-Rspam-User: X-HE-Tag: 1731101373-403313 X-HE-Meta: U2FsdGVkX1/wjd6NTTt8fKcKZGWy6+tLai0l70ffV1EjavOtLB3s7uBgKA/Hb+7ZdGVJqcmiXpjPnMyKMsUCG0qkIifvOplUlyTXGt7elNale+gcvqDnH+jsp/QZZsK5deXi5s2DrNJZGae3P5TONi2S4Y/YYwz1+rBvBf2auPD6o1RrUbcgeezOxDvxe5JVmfKR6JAHsYieS2dGh+lsJUHVRUHvwBKA9B/sG+yNKjBZ9/KmsTXmlYkv4/T+0tx7wrCXdLrJ17FabLuIoiJ6BtFG+OyUer5qo854TZsm0n4/rChEDol4gDY9Cylsx9pZHsTZstU8m31wibchGc8+7/6n6P81iQPIo6fLwCV08IQ7TAd3Di7qni6mubuYb+YIYFRTHfOz+jOiWiZgB6hnfNPtT4LU1i3NHw5pYROoyO7liNFE0HDtAVL8duXal6xlVNFhL0qSE37VMqYkRdbpraxr3MLrA4L9g0tNJf9w5Xkk3I3IT2VCYl+Zt7HCRE2CZRT9nJZWm6/XDPPr4UtoRl8WUBVQUnPhjlOhbWcHPPp556j4WCK6cgO09DSa9AkaROh+qg4UhRG79dMwQ3uTLwpd8SphI92skpDITD3d+pbLdZ5Wx+4Rew7klyVaC5x3O0+/iYgcgjPuq4Y5rGYPuiKPMizsqoscstzBld6jJ/wceAODD0eLaNirdsJygWA9tY6zjYf95iqUavBT2NKhJPHR/GdtSSp5Sk3td/Z3/HSc5AKbGkaUeggHFPu9ro55FcyJfUHYs5nI2GzEXYVl5LQlENVNdx7joWuMYQIDMX0LtDdmZGOjdVCS/z6Qd3AiwOX6153scc9JnCfqj/LgB9mjVb36jZvCEDz9C0yEngKK3O2edN+h3Fx2NrBPmo9SmmxErx6zfExxVdFhdylIJL/mPVVO9aJtoGAFd3xPaOXS7dESsrLpl/12sC0IKLB1H/qBynF7ahukzFe/iQh v3/HMonG fnsk3HFbtwSr4igQcjdSY8nZk/UOqc+6vj9A2ZXELlFjlCHLR5lv8lwHZxCasj54XLuz8Lcz6LVaEDsevqqWqZfF0McyIM86HbbyvBxh7kKGdqgL0rf159k2E0lemC3AdiLpfdbOol8rhIc/t655lB6/fLl4+3MW/LkQ9tPKXfV7/v2RZsHv9gVeBwut1skjBhDK81Dp9g1Hh9YwNNWlu2OAaDSf2TkhUI/Awl8/4H4v6AR0LxJKsg6fEfsfHP2jQpoJ/rYAauJ4vAi6HlxMeU2YSai98NDLgBvWTkaN57gPSTkOuA0ExFc8xfkoPdpwIslSKi+BUtdL1fCm+MX6UNA8pjBeJhouPF/3roZglToxLZSlbrJpQY86+8MS0ROKxIgmPsMCNArl6tchT/LtRYa3RCgmaArmKgOREAaY5fu2EF6DzUebV5vsKNw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This patch introduces mem_cgroup_charge_hugetlb, which combines the logic of mem_cgroup{try,commit}_hugetlb. This reduces the footprint of memcg in hugetlb code, and also consolidates the error path that memcg can take into just one point. Signed-off-by: Joshua Hahn --- include/linux/memcontrol.h | 2 ++ mm/hugetlb.c | 34 ++++++++++++---------------------- mm/memcontrol.c | 19 +++++++++++++++++++ 3 files changed, 33 insertions(+), 22 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index bb49e0d4b377..d90c1ac791f1 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -678,6 +678,8 @@ static inline int mem_cgroup_charge(struct folio *folio, struct mm_struct *mm, int mem_cgroup_hugetlb_try_charge(struct mem_cgroup *memcg, gfp_t gfp, long nr_pages); +int mem_cgroup_charge_hugetlb(struct folio* folio, gfp_t gfp); + int mem_cgroup_swapin_charge_folio(struct folio *folio, struct mm_struct *mm, gfp_t gfp, swp_entry_t entry); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index fbb10e52d7ea..6f6841483039 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -2967,21 +2967,13 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, struct hugepage_subpool *spool = subpool_vma(vma); struct hstate *h = hstate_vma(vma); struct folio *folio; - long map_chg, map_commit, nr_pages = pages_per_huge_page(h); + long map_chg, map_commit; long gbl_chg; - int memcg_charge_ret, ret, idx; + int ret, idx; struct hugetlb_cgroup *h_cg = NULL; - struct mem_cgroup *memcg; bool deferred_reserve; gfp_t gfp = htlb_alloc_mask(h) | __GFP_RETRY_MAYFAIL; - memcg = get_mem_cgroup_from_current(); - memcg_charge_ret = mem_cgroup_hugetlb_try_charge(memcg, gfp, nr_pages); - if (memcg_charge_ret == -ENOMEM) { - mem_cgroup_put(memcg); - return ERR_PTR(-ENOMEM); - } - idx = hstate_index(h); /* * Examine the region/reserve map to determine if the process @@ -2989,12 +2981,8 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, * code of zero indicates a reservation exists (no change). */ map_chg = gbl_chg = vma_needs_reservation(h, vma, addr); - if (map_chg < 0) { - if (!memcg_charge_ret) - mem_cgroup_cancel_charge(memcg, nr_pages); - mem_cgroup_put(memcg); + if (map_chg < 0) return ERR_PTR(-ENOMEM); - } /* * Processes that did not create the mapping will have no @@ -3092,10 +3080,15 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, } } - if (!memcg_charge_ret) - mem_cgroup_commit_charge(folio, memcg); - lruvec_stat_mod_folio(folio, NR_HUGETLB, pages_per_huge_page(h)); - mem_cgroup_put(memcg); + ret = mem_cgroup_charge_hugetlb(folio, gfp); + if (ret == -ENOMEM) { + spin_unlock_irq(&hugetlb_lock); + free_huge_folio(folio); + return ERR_PTR(-ENOMEM); + } + else if (!ret) + lruvec_stat_mod_folio(folio, NR_HUGETLB, + pages_per_huge_page(h)); return folio; @@ -3110,9 +3103,6 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, hugepage_subpool_put_pages(spool, 1); out_end_reservation: vma_end_reservation(h, vma, addr); - if (!memcg_charge_ret) - mem_cgroup_cancel_charge(memcg, nr_pages); - mem_cgroup_put(memcg); return ERR_PTR(-ENOSPC); } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 97f63ec9c9fb..95ee77fe27af 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -4529,6 +4529,25 @@ int mem_cgroup_hugetlb_try_charge(struct mem_cgroup *memcg, gfp_t gfp, return 0; } +int mem_cgroup_charge_hugetlb(struct folio *folio, gfp_t gfp) +{ + struct mem_cgroup *memcg = get_mem_cgroup_from_current(); + int ret = 0; + + if (mem_cgroup_disabled() || !memcg_accounts_hugetlb() || + !memcg || !cgroup_subsys_on_dfl(memory_cgrp_subsys)) { + ret = -EOPNOTSUPP; + goto out; + } + + if (charge_memcg(folio, memcg, gfp)) + ret = -ENOMEM; + +out: + mem_cgroup_put(memcg); + return ret; +} + /** * mem_cgroup_swapin_charge_folio - Charge a newly allocated folio for swapin. * @folio: folio to charge.