From patchwork Sat Feb 1 23:18:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13956397 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9F9F5C0218A for ; Sat, 1 Feb 2025 23:18:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE76D6B007B; Sat, 1 Feb 2025 18:18:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C97566B0083; Sat, 1 Feb 2025 18:18:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B5EDB6B0085; Sat, 1 Feb 2025 18:18:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 976006B007B for ; Sat, 1 Feb 2025 18:18:09 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 39C7B121C85 for ; Sat, 1 Feb 2025 23:18:09 +0000 (UTC) X-FDA: 83072941098.25.9468E97 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf25.hostedemail.com (Postfix) with ESMTP id 82316A0007 for ; Sat, 1 Feb 2025 23:18:07 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=fY4sGA2e; spf=pass (imf25.hostedemail.com: domain of 3rqueZwYKCMM130nwkpxxpun.lxvurw36-vvt4jlt.x0p@flex--surenb.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3rqueZwYKCMM130nwkpxxpun.lxvurw36-vvt4jlt.x0p@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738451887; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=/Kvraf4w4weerjiWyvf6hTFiVHSzsydM/sRRmUt4qn4=; b=u79Lb9Q8OdkccZwPmNQG0eOXq2bA22J+wEO1z71RDd3nT9COFXP/xE82W4sejfL500Uq8F WvhTS098Im86NS8CFp/Xyl4R8nIsX+orq5Qmhuj9eKp+5L8N6mI1KLKo7GhcluZVj9rqZv jVs4dKKYg3rT5VgpI5WRZ4qKTE14cZU= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=fY4sGA2e; spf=pass (imf25.hostedemail.com: domain of 3rqueZwYKCMM130nwkpxxpun.lxvurw36-vvt4jlt.x0p@flex--surenb.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3rqueZwYKCMM130nwkpxxpun.lxvurw36-vvt4jlt.x0p@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738451887; a=rsa-sha256; cv=none; b=Qa62M0q7+tW8DYlO61W2QIsfbQJdmHE0IU5Fd8bueZuVEnCnGQQxexyKL8BbG1VlmZOGds ulGxT+BEYx81ORANbNrYtFu+y3U5WmUhhK4XTWECoLKjYD5P3j7dyFzCYvO4OgzeZUbB9Y IzZqSeC+J2HrbTn+elFimfKGRBqViGY= Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-21661949f23so97945125ad.3 for ; Sat, 01 Feb 2025 15:18:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1738451886; x=1739056686; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=/Kvraf4w4weerjiWyvf6hTFiVHSzsydM/sRRmUt4qn4=; b=fY4sGA2e+z5xsInyLnD1ZWLzpe72yxxPlqJwO649YGukm7jma8/I4IBaYUFkqkv/gk jZyQo7N1zMPy4EdHiI+ols26TLjUjZCNupn/XLvgZm9cWohnZi5MnAVABtDhNePAtL2p zBhO4GDgI2k7GINQtk/l6ZV/QeMjCCiUzhHXVe8DX0BXHPkfwN5EC8F/QDKscos+2OQK thIp6M9PNVfVIMzqxosGkA8NuSe5VVWEzwqqD/SqoKIg2CNoZOgVJil5/03mahnoV28j AoX516M5Dxpzh2MP6HOmc72YZHDq0AXC32F36ejDzd26ZW64cBGojiqCmSv5MF1WyRlt i66Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738451886; x=1739056686; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=/Kvraf4w4weerjiWyvf6hTFiVHSzsydM/sRRmUt4qn4=; b=IKoJKyNM3uDubh0qbPExQT4xODHUfMlpbBlXYC9kD9Jh4x2k9V6UO6Rmh6uqiFmczx QpxBYnGiw0qthge9/U+oOQUtGhUuwLeD25cdKtZeBN98W0yOQLEOAcTj8QIml3XD9XT/ fRZIJC09J9C9r7MhsYuLLe/SKfcuMTxXF4/VTGiOXNrTzlXPYcdimE4SEfDrP+hA+Jqb xWqB3bivyHOnct/ychg5KEuxgjP9SBMzhBoK+vGCq25gsHP9eAYOvBLeOY+kK5H4x+5g cwF96KtM2TOgcZEErSjT6VE2YLRohIaVGtqp/HQigT8d91Y5b0UmdHQDL+56oKa34tIv JIRQ== X-Forwarded-Encrypted: i=1; AJvYcCXDdMGleOaaliMVxYUMsS3fhpYziJa/OHusFmHI/0VpyyrVas8AbdNvYUTMqA6jWXKsVGBZz2t44w==@kvack.org X-Gm-Message-State: AOJu0YxIvWI9oGyCEINsqI4SQLxzsX3/GKuQGI2ZxFxI/2uNmfXGVS95 R6BCTLYLWSQOwQwQ9CfhoIAQOVLmA12+1b2mcV7MeEiUlKW9awZkFVsie8WG3+iPZf0Rd5eXxxc fNA== X-Google-Smtp-Source: AGHT+IF7rl+0kYYhaXTekrdVVwheczJIWDrZLww791nC4RwD+CIsfaA9QQuUOCf1vuXCQHD3GZRrfqGvVg0= X-Received: from plse12.prod.google.com ([2002:a17:902:b78c:b0:216:44e0:4ac0]) (user=surenb job=prod-delivery.src-stubby-dispatcher) by 2002:a17:903:244d:b0:215:6816:6345 with SMTP id d9443c01a7336-21dd7c65450mr249942315ad.16.1738451886129; Sat, 01 Feb 2025 15:18:06 -0800 (PST) Date: Sat, 1 Feb 2025 15:18:00 -0800 Mime-Version: 1.0 X-Mailer: git-send-email 2.48.1.362.g079036d154-goog Message-ID: <20250201231803.2661189-1-surenb@google.com> Subject: [PATCH v2 1/3] mm: avoid extra mem_alloc_profiling_enabled() checks From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: kent.overstreet@linux.dev, vbabka@suse.cz, rostedt@goodmis.org, peterz@infradead.org, yuzhao@google.com, minchan@google.com, shakeel.butt@linux.dev, souravpanda@google.com, pasha.tatashin@soleen.com, 00107082@163.com, quic_zhenhuah@quicinc.com, surenb@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 82316A0007 X-Stat-Signature: zr83xtzu3prj58pzrxs6urgc7hkjmgbs X-Rspam-User: X-HE-Tag: 1738451887-407693 X-HE-Meta: U2FsdGVkX19MVuz1hwzvoR4pnN3TDwHvSWZsBhFIb4EN9KV1nLf6qoQnQoEiyTY5nkRoYL+IqDFH1TAbhjI+J9HSbkKBrMxFSO+/ohw4OG41erVWLGZuxulImmm/QVvLXpzekifka7AalzeVwtQ3Y2wDQ3DPvCB1P1NFmvY8spTqTn4p9T7IxcBf8VYcNHo/GugxDk2YI4JE+hJqs4PHLtCUN5wcxKLdBcJqrrZBItUT+gUrMHLJDH9e4EkdAUZc1fFTHDMzH9I8Xmh8aNNzryl+IeJW//C/8nHQheBT7ShOq/lYspO+P+QNX4oDCAm5yu4cO3IEdIRipTFDEkMKYW1sOmOKSnAtfXOqVJPcagB4WyXK6iUCKa1X3nPDkOfwzmaUXPfwkOavVgo0dfsdgzEnfyuCflnCNH2IrCNOSb0fJggnwvhJnkn/hknHWwBOQEqu8tKgf7+AOdQSvRnfbHXElhIhVY2MHzL8HaSv3wQLyuM4yP/bBPbYU5bDjWIhCbMNVR/d+5VGbhzHYdgow5bggVR3qhpKqV/TEkVvy8aSZYlvNR+2rWNvuYtDaSWMmbHf3ohHCbcagYo7VbYdIj2NdTxiruTpwKCAdOo34QtBUJjZ+bKknsT7GVNvJQpqGIcvh9v41+MtUhz2zfoe3iDQHnwAeWbc2ufL0eE+ncXNJE9ZInFRMvSxAW4WlC2h/omdDaOul6T+m4Ie/oDHwHf2vuR9HpIM9EORkTw66nPHKDlbuAyyYamOmdMFDlBQ4sFCr6Zyi7udlCAs24pUTEVwrC/NMYeMIri/BJWhdScuw9U+Sw0vzN8wgf0VuKQ2Iul8nSmQJIXHJ5hkSpK9tV78CLTLmxPyEJDkkVROG7ivhNlvBviXdGkT/9Tfb3+vANw7BYBbmM0h36fW7YzJNg1VogH799ZMF8d4UxVTWNmxlNMT9VOj2vhQtA3aEdp3LFSlAMy9ZgKfogm+pjB 3l9B458/ 4xwKMeUfmKV8Eu+IVWJ30ZUsqPbuYb2DZ/nd+qGGY8QIe3pV8Hq47IqXhpdrIID0K9cfyRfOoT/x5ssOU734QirZP27hRs8AjzeltRGnjLA4sE8Z6huXUk1PSoHo2NZvL326DGaQgPKYKW0wi9RHM73ep+Kx0BaWgM0F+bOuE7yBpbCGG8xb/2r+7+uEK8r56tKcexscgA8PSLbmoZ1cWBumjQaI8Ifq1Pkb0KyDzOdfRfoScC96qH5IVW2oIX2HduNb2OxT16v52u4tjHpXe19uM7O2OBL3GS8p3qagsTfPtgda2kBhHbe+wjc7xfhDJPTv57XArEkyrVEid3VjHrKP7euVuTEG+uH1fwJHVGV+YnyVcxS4slqlUkbbO//m9dyfjeKkxA9VOsx3Tn3Cj7HOkS0KqeidMPObtVw9Kf+5NUzwh0bQ7nW41LehnyYI5xZgXH5jG2ej4iQ3Tv2E/60pdgFA654Z84MgpY71rTRO/3sAcNenFARMOGRy8mky4UCIr9ybYjpYuYP8bKjXNIpzy7jjWps+r8PGkiGfuamruLWk1lz5KDLfpwS9pEpYSsO37e2VNCn0EWLsZJNsNNM/10w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.005519, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Refactor code to avoid extra mem_alloc_profiling_enabled() checks inside pgalloc_tag_get() function which is often called after that check was already done. Signed-off-by: Suren Baghdasaryan Reviewed-by: Shakeel Butt --- include/linux/pgalloc_tag.h | 35 +++++++++++++++++++---------------- lib/alloc_tag.c | 6 +++--- mm/page_alloc.c | 3 +-- 3 files changed, 23 insertions(+), 21 deletions(-) base-commit: 7de6fd8ab65003f050aa58e705592745717ed318 diff --git a/include/linux/pgalloc_tag.h b/include/linux/pgalloc_tag.h index 3469c4b20105..4a82b6b4820e 100644 --- a/include/linux/pgalloc_tag.h +++ b/include/linux/pgalloc_tag.h @@ -205,28 +205,32 @@ static inline void pgalloc_tag_sub(struct page *page, unsigned int nr) } } -static inline struct alloc_tag *pgalloc_tag_get(struct page *page) +/* Should be called only if mem_alloc_profiling_enabled() */ +static inline struct alloc_tag *__pgalloc_tag_get(struct page *page) { struct alloc_tag *tag = NULL; - - if (mem_alloc_profiling_enabled()) { - union pgtag_ref_handle handle; - union codetag_ref ref; - - if (get_page_tag_ref(page, &ref, &handle)) { - alloc_tag_sub_check(&ref); - if (ref.ct) - tag = ct_to_alloc_tag(ref.ct); - put_page_tag_ref(handle); - } + union pgtag_ref_handle handle; + union codetag_ref ref; + + if (get_page_tag_ref(page, &ref, &handle)) { + alloc_tag_sub_check(&ref); + if (ref.ct) + tag = ct_to_alloc_tag(ref.ct); + put_page_tag_ref(handle); } return tag; } -static inline void pgalloc_tag_sub_pages(struct alloc_tag *tag, unsigned int nr) +static inline void pgalloc_tag_sub_pages(struct page *page, unsigned int nr) { - if (mem_alloc_profiling_enabled() && tag) + struct alloc_tag *tag; + + if (!mem_alloc_profiling_enabled()) + return; + + tag = __pgalloc_tag_get(page); + if (tag) this_cpu_sub(tag->counters->bytes, PAGE_SIZE * nr); } @@ -241,8 +245,7 @@ static inline void clear_page_tag_ref(struct page *page) {} static inline void pgalloc_tag_add(struct page *page, struct task_struct *task, unsigned int nr) {} static inline void pgalloc_tag_sub(struct page *page, unsigned int nr) {} -static inline struct alloc_tag *pgalloc_tag_get(struct page *page) { return NULL; } -static inline void pgalloc_tag_sub_pages(struct alloc_tag *tag, unsigned int nr) {} +static inline void pgalloc_tag_sub_pages(struct page *page, unsigned int nr) {} static inline void alloc_tag_sec_init(void) {} static inline void pgalloc_tag_split(struct folio *folio, int old_order, int new_order) {} static inline void pgalloc_tag_swap(struct folio *new, struct folio *old) {} diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c index 19b45617bdcf..1d893e313614 100644 --- a/lib/alloc_tag.c +++ b/lib/alloc_tag.c @@ -174,7 +174,7 @@ void pgalloc_tag_split(struct folio *folio, int old_order, int new_order) if (!mem_alloc_profiling_enabled()) return; - tag = pgalloc_tag_get(&folio->page); + tag = __pgalloc_tag_get(&folio->page); if (!tag) return; @@ -200,10 +200,10 @@ void pgalloc_tag_swap(struct folio *new, struct folio *old) if (!mem_alloc_profiling_enabled()) return; - tag_old = pgalloc_tag_get(&old->page); + tag_old = __pgalloc_tag_get(&old->page); if (!tag_old) return; - tag_new = pgalloc_tag_get(&new->page); + tag_new = __pgalloc_tag_get(&new->page); if (!tag_new) return; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 579789600a3c..b7e3b45183ed 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -4832,12 +4832,11 @@ void __free_pages(struct page *page, unsigned int order) { /* get PageHead before we drop reference */ int head = PageHead(page); - struct alloc_tag *tag = pgalloc_tag_get(page); if (put_page_testzero(page)) free_frozen_pages(page, order); else if (!head) { - pgalloc_tag_sub_pages(tag, (1 << order) - 1); + pgalloc_tag_sub_pages(page, (1 << order) - 1); while (order-- > 0) free_frozen_pages(page + (1 << order), order); }