From patchwork Wed Nov 6 22:14:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joshua Hahn X-Patchwork-Id: 13865577 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B214D59F7D for ; Wed, 6 Nov 2024 22:14:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE5126B00A9; Wed, 6 Nov 2024 17:14:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D91E76B00AC; Wed, 6 Nov 2024 17:14:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B98296B00AD; Wed, 6 Nov 2024 17:14:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 9A5076B00A9 for ; Wed, 6 Nov 2024 17:14:40 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 53FA4120150 for ; Wed, 6 Nov 2024 22:14:40 +0000 (UTC) X-FDA: 82757073924.08.85B18B8 Received: from mail-yw1-f176.google.com (mail-yw1-f176.google.com [209.85.128.176]) by imf14.hostedemail.com (Postfix) with ESMTP id B8DF3100004 for ; Wed, 6 Nov 2024 22:13:59 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="NPXhX/2o"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf14.hostedemail.com: domain of joshua.hahnjy@gmail.com designates 209.85.128.176 as permitted sender) smtp.mailfrom=joshua.hahnjy@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1730931219; a=rsa-sha256; cv=none; b=tk82FAIALBkrexQ8fUYPVIhtwXXDTrlLN0h7kUX/w+fC7sVyolSXceHtKAoVEx3Faoqi0e RtXKRXylz+vBqd6xI4e1+zDoGHZWvM72NUViAYv8HfVDBiBfFtpRRpTgRV/HzSPgT3OR7J MZbPBXsSO9RPPvs5aB8t6y4ISt7lqEU= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="NPXhX/2o"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf14.hostedemail.com: domain of joshua.hahnjy@gmail.com designates 209.85.128.176 as permitted sender) smtp.mailfrom=joshua.hahnjy@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1730931219; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cbER8uQRgMc3lGFT4u7VFeg6pXjRf9IXs/IQf6vmq6s=; b=4bJU+At3gCV0A6W0xtVYWj69pVzfS31WI46MRe7VHLcSl7180XH5dOJV3v9OdDLyJ/SCsq ShdetASRpI5wTvdvyLZjeoq31QpCRosJkA+8TUgVuPpUPkjjpczDQ7xi/5MNm5qNk4GV09 wJFh11+1G0Xw3FvkFODCU5LVp+jvEEM= Received: by mail-yw1-f176.google.com with SMTP id 00721157ae682-6e35bf59cf6so14280787b3.0 for ; Wed, 06 Nov 2024 14:14:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1730931277; x=1731536077; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cbER8uQRgMc3lGFT4u7VFeg6pXjRf9IXs/IQf6vmq6s=; b=NPXhX/2ox2p51O8Y8qhbTISaoSKvIKfeInu8QT1Yqk98d1xGbPrB6BMToWU3sGd8OC SsIPlMePfnBvmXr9z22H58A7QCN1K37eXcu/hw0zxIlTuYAcHP/mkDCoPWGUEz31Arf/ OHx9WAaXG6YpI7wQ1KIJ2Z9wWg1poMj+ZWy28N2RECIvBHRyKao4TpyMxVeBF0zNLkif rR7ZUzMc8hk7IIQcJiGSew4ikvhVaQ9iDN9IvgGUvl+dwA2ZeOea5CjiIl2Ep19IcyTb QQedlYNPawXHcsHJ6tDvMGpjGJeEvKCnZNkQVOZi3qMlwGvy+NJMiNBLMoh5UurTCYma oZyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730931277; x=1731536077; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cbER8uQRgMc3lGFT4u7VFeg6pXjRf9IXs/IQf6vmq6s=; b=h++aFogyDz3LhDfKpWWv5sg+JLhXYnJ1hEmnC/yvFJQF6cweAsghqMq4NwPwiS3INb fBe7Tq1qfREpyOvICujmAKUktudPJZCgpzma4+oEhbSAPDVVW1pDQzYCdzaHBnQThPC6 Cq8fx8KHyHlysSNZMAvHqVnwoRYfIKIbl0KOkkUj55+gR9ix9GGLOzbkT7utLVP0Qp9s KC2PMKjDUOJp/FB8vcfFbce6/ULSIlXrjDo0ZKJCy6oA/FpPvSFNvhXVvpPyxKRqJI6J zi7DOcKEz28TrHt2Ks3xQI8TiTjUHFvMwazOQ8Q0QZF1FbYFgx+FAbmNLQYIbRHQLmkq A5zw== X-Forwarded-Encrypted: i=1; AJvYcCVlF8Y4mXzRTc41VNPCCiHByiGR1nRWIxYiR9NIpmA/Zv2cpg1U9xd5SGaN98fHG/ShxHnFKkqUcw==@kvack.org X-Gm-Message-State: AOJu0YysCPYIJqdkQflkI0uYkSxyh4qhG3x6gbZq8stC+12R4iSjdsYN QR4lvG9Ytu0KWRZ25HFKOgvhn1GhcM8C2PerYWuLeAOdGbOMijsf X-Google-Smtp-Source: AGHT+IGszx+ho52yOVYH5yhv/l6mOK19o5HKqSOg2vLBuohvAZ7xFIBSVJm6ut1NbuJt/3OD0+/bdw== X-Received: by 2002:a05:690c:23c5:b0:6db:e1e0:bf6a with SMTP id 00721157ae682-6eacbff145fmr10678587b3.7.1730931277584; Wed, 06 Nov 2024 14:14:37 -0800 (PST) Received: from localhost (fwdproxy-frc-002.fbsv.net. [2a03:2880:21ff:2::face:b00c]) by smtp.gmail.com with ESMTPSA id 00721157ae682-6eaceb7a578sm185057b3.102.2024.11.06.14.14.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Nov 2024 14:14:36 -0800 (PST) From: Joshua Hahn To: shakeel.butt@linux.dev Cc: hannes@cmpxchg.org, mhocko@kernel.org, roman.gushchin@linux.dev, muchun.song@linux.dev, akpm@linux-foundation.org, cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@meta.com Subject: [PATCH 2/2] memcg/hugetlb: Deprecate hugetlb memcg try-commit-cancel charging Date: Wed, 6 Nov 2024 14:14:34 -0800 Message-ID: <20241106221434.2029328-3-joshua.hahnjy@gmail.com> X-Mailer: git-send-email 2.43.5 In-Reply-To: <20241106221434.2029328-1-joshua.hahnjy@gmail.com> References: <20241106221434.2029328-1-joshua.hahnjy@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: B8DF3100004 X-Rspamd-Server: rspam01 X-Stat-Signature: ntdgzpq384iq5o6en94r3gtm4qqpoimu X-HE-Tag: 1730931239-807957 X-HE-Meta: U2FsdGVkX18CK+34Ud2RMk0kDsqB032UcrIEhsp1qI1J31malHLKKXcpTnLU6VGmKD+3Rex8BwgnRjDmuuLxyGMQpcpYlL2Wh5ErlzTOXC64NzGFCF6kDQ0RhmLyEqdEdjj/cA8eZRIPf8lqy80SP1G87o2wmZ0gBGfYbpnS6sYlMVLpHzWxsIn5PQakUncdqGGLHrzMqNkgysVTthPl5EoPTmxto22dqorKSjWBl99tKMEfjdyjF81b3UyOlXc8X0RRUiVcORtRrzBYASGb70z1pI0nDD5Mdm3k68AiLTwFv5Ngk3QU4Occ2faC6476gMuWBGpE68u5BkYDVoDW/AvzAYI7ftuFZzKI6dyUCVFKvx6GO5hQwBAxr6k0Syw9Rx7JizNGtCVRwii3rWjzrdcTRnWFEXd6twu5u4Yp0/xIwUA04JMteBJ4/JkhwcBtaNTF5BAHWHbs+yOLmrDRVVVMhP4nOZQg25IwRR0g/Wi09vTa0Qr6R/Diky8Wu5QxxjUycR6Q0m5K+haU8bD/QcH93FQ+2kE3s3uoNXfT51LrMHPkaWUPXHeE3SpiVt5M4c047Nj3dtlfrQN9k9G+BtoQZsm6cN3XFrwR6oJdhvAkYWBbovq3Rl7uknjt0yMrIcKmuzdUNLdbr7tLVrjrgZP7jE+EJ2RyLwIh2f+KBqTsZNf3O11T0SMuetG6XuFu9s1pATbzg//zOLdb8FSuc0uMjlCHeudh6I+xaj6QMgfs6ugbVQbku9mj7Od1fmYvA/Z2yjOZP0LdUDh9BMtCIoVLQ7K4oiYQCfFq//0QYQGgHJTZynJ9ovxeeAIK0s2I8nmfCvGY4uxcU7D+P+GKSj+O3EXRepiLzFokgHs5Y6pMm+BEVq4Uqfg12YVvMVfTqjbYPYw8ILs610n4aeehHG0nCa0jIaC5DAr3uOmDkhlpP3Qc1tGlz+oXun0t9oEZLd0GMtQnqDMhajXeGDv 2at80/7o s+ElbYZ3jW1lNkSolDbhjddcRZuo1YktYXYJV/4LAlgGKCFYwDVBnv1OxmOZ+KdF/zKCXT6h3LELi6XJ+6y5MaEfRglcp3NDG18Cm8Q7/4uP+wXX4+NKMTa3MEMcZAcGPbg4riuj7U0UKSHTRm2d/99P46Lbjppla0P1ud0ZwgKEqaVcUs+f6mTUnmJzTMGe/HDfcTQfWY4qlStB6BlATU7olmyYuiI8Cafi9oXCqMqRIJ/UbdGhUMK0F6G1OqdSQmRo1zVAP2gsefhikwyPCYlAw8Hmslmy9fn/1KtMnYCHt6aN0jahn7w7h6JJlQ1a07U7R9ka8slmr0wfaTKHTXgPQ49Au3ttNELhbAz0MLrKDyYUTSmzq/WJ3sU/q3jN34lWN1c7qrY61W8s= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This patch deprecates the memcg try-{commit,cancel} logic used in hugetlb. Instead of having three points of error for memcg accounting, the error patch is reduced to just one point at the end, and shares the same path with the hugeTLB controller as well. Please note that the hugeTLB controller still uses the try_charge to {commit/cancel} protocol. Signed-off-by: Joshua Hahn --- include/linux/memcontrol.h | 3 +-- mm/hugetlb.c | 35 ++++++++++++----------------------- mm/memcontrol.c | 37 +++++++++---------------------------- 3 files changed, 22 insertions(+), 53 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 25761d55799e..0024634d161f 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -696,8 +696,7 @@ static inline int mem_cgroup_charge(struct folio *folio, struct mm_struct *mm, bool memcg_accounts_hugetlb(void); -int mem_cgroup_hugetlb_try_charge(struct mem_cgroup *memcg, gfp_t gfp, - long nr_pages); +int mem_cgroup_charge_hugetlb(struct folio *folio, gfp_t gfp); int mem_cgroup_swapin_charge_folio(struct folio *folio, struct mm_struct *mm, gfp_t gfp, swp_entry_t entry); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index fbb10e52d7ea..db9801b16d13 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -2967,21 +2967,13 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, struct hugepage_subpool *spool = subpool_vma(vma); struct hstate *h = hstate_vma(vma); struct folio *folio; - long map_chg, map_commit, nr_pages = pages_per_huge_page(h); + long map_chg, map_commit; long gbl_chg; - int memcg_charge_ret, ret, idx; + int ret, idx; struct hugetlb_cgroup *h_cg = NULL; - struct mem_cgroup *memcg; bool deferred_reserve; gfp_t gfp = htlb_alloc_mask(h) | __GFP_RETRY_MAYFAIL; - memcg = get_mem_cgroup_from_current(); - memcg_charge_ret = mem_cgroup_hugetlb_try_charge(memcg, gfp, nr_pages); - if (memcg_charge_ret == -ENOMEM) { - mem_cgroup_put(memcg); - return ERR_PTR(-ENOMEM); - } - idx = hstate_index(h); /* * Examine the region/reserve map to determine if the process @@ -2989,12 +2981,8 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, * code of zero indicates a reservation exists (no change). */ map_chg = gbl_chg = vma_needs_reservation(h, vma, addr); - if (map_chg < 0) { - if (!memcg_charge_ret) - mem_cgroup_cancel_charge(memcg, nr_pages); - mem_cgroup_put(memcg); + if (map_chg < 0) return ERR_PTR(-ENOMEM); - } /* * Processes that did not create the mapping will have no @@ -3056,6 +3044,12 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, /* Fall through */ } + ret = mem_cgroup_charge_hugetlb(folio, gfp); + if (ret == -ENOMEM) + goto free_folio; + else if (!ret) + lruvec_stat_mod_folio(folio, NR_HUGETLB, pages_per_huge_page(h)); + hugetlb_cgroup_commit_charge(idx, pages_per_huge_page(h), h_cg, folio); /* If allocation is not consuming a reservation, also store the * hugetlb_cgroup pointer on the page. @@ -3092,13 +3086,11 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, } } - if (!memcg_charge_ret) - mem_cgroup_commit_charge(folio, memcg); - lruvec_stat_mod_folio(folio, NR_HUGETLB, pages_per_huge_page(h)); - mem_cgroup_put(memcg); - return folio; +free_folio: + spin_unlock_irq(&hugetlb_lock); + free_huge_folio(folio); out_uncharge_cgroup: hugetlb_cgroup_uncharge_cgroup(idx, pages_per_huge_page(h), h_cg); out_uncharge_cgroup_reservation: @@ -3110,9 +3102,6 @@ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, hugepage_subpool_put_pages(spool, 1); out_end_reservation: vma_end_reservation(h, vma, addr); - if (!memcg_charge_ret) - mem_cgroup_cancel_charge(memcg, nr_pages); - mem_cgroup_put(memcg); return ERR_PTR(-ENOSPC); } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 59dea0122579..3b728635d6aa 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1448,8 +1448,7 @@ static void memcg_stat_format(struct mem_cgroup *memcg, struct seq_buf *s) u64 size; #ifdef CONFIG_HUGETLB_PAGE - if (unlikely(memory_stats[i].idx == NR_HUGETLB) && - !(cgrp_dfl_root.flags & CGRP_ROOT_MEMORY_HUGETLB_ACCOUNTING)) + if (unlikely(memory_stats[i].idx == NR_HUGETLB) && !memcg_accounts_hugetlb()) continue; #endif size = memcg_page_state_output(memcg, memory_stats[i].idx); @@ -4506,37 +4505,19 @@ bool memcg_accounts_hugetlb(void) #endif } -/** - * mem_cgroup_hugetlb_try_charge - try to charge the memcg for a hugetlb folio - * @memcg: memcg to charge. - * @gfp: reclaim mode. - * @nr_pages: number of pages to charge. - * - * This function is called when allocating a huge page folio to determine if - * the memcg has the capacity for it. It does not commit the charge yet, - * as the hugetlb folio itself has not been obtained from the hugetlb pool. - * - * Once we have obtained the hugetlb folio, we can call - * mem_cgroup_commit_charge() to commit the charge. If we fail to obtain the - * folio, we should instead call mem_cgroup_cancel_charge() to undo the effect - * of try_charge(). - * - * Returns 0 on success. Otherwise, an error code is returned. - */ -int mem_cgroup_hugetlb_try_charge(struct mem_cgroup *memcg, gfp_t gfp, - long nr_pages) +int mem_cgroup_charge_hugetlb(struct folio *folio, gfp_t gfp) { - /* - * If hugetlb memcg charging is not enabled, do not fail hugetlb allocation, - * but do not attempt to commit charge later (or cancel on error) either. - */ - if (mem_cgroup_disabled() || !memcg || - !cgroup_subsys_on_dfl(memory_cgrp_subsys) || !memcg_accounts_hugetlb()) + struct mem_cgroup *memcg = get_mem_cgroup_from_current(); + + if (mem_cgroup_disabled() || !memcg_accounts_hugetlb() || + !memcg || !cgroup_subsys_on_dfl(memory_cgrp_subsys)) return -EOPNOTSUPP; - if (try_charge(memcg, gfp, nr_pages)) + if (charge_memcg(folio, memcg, gfp)) return -ENOMEM; + mem_cgroup_put(memcg); + return 0; }