From patchwork Sun Jan 26 07:02:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13950574 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A37D2C02181 for ; Sun, 26 Jan 2025 07:02:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B53322800A7; Sun, 26 Jan 2025 02:02:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id ADC782800A6; Sun, 26 Jan 2025 02:02:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A38A2800A7; Sun, 26 Jan 2025 02:02:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 795D02800A6 for ; Sun, 26 Jan 2025 02:02:13 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D9D5FC1312 for ; Sun, 26 Jan 2025 07:02:12 +0000 (UTC) X-FDA: 83048708904.29.8D32258 Received: from mail-oo1-f74.google.com (mail-oo1-f74.google.com [209.85.161.74]) by imf12.hostedemail.com (Postfix) with ESMTP id 2D7384000D for ; Sun, 26 Jan 2025 07:02:10 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="0Rv/kO1w"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf12.hostedemail.com: domain of 38d2VZwYKCEg241oxlqyyqvo.mywvsx47-wwu5kmu.y1q@flex--surenb.bounces.google.com designates 209.85.161.74 as permitted sender) smtp.mailfrom=38d2VZwYKCEg241oxlqyyqvo.mywvsx47-wwu5kmu.y1q@flex--surenb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737874931; a=rsa-sha256; cv=none; b=P0/lqnGkP/5eDiN61/3I5zWaKHW7hkkjvxRh1ODKAD3fR9q+BxJ/fOaqkIQFiE727gyFO+ bcWY1rCrdNYQO9+uLytlo/5b5gyOMlNxxeDXFVa9s0VS3ZXN7tvP2FK2eFzQo9r9NbDyb/ LFlMne5EM1SuvkX1ENLxVf6JTWP8dxo= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="0Rv/kO1w"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf12.hostedemail.com: domain of 38d2VZwYKCEg241oxlqyyqvo.mywvsx47-wwu5kmu.y1q@flex--surenb.bounces.google.com designates 209.85.161.74 as permitted sender) smtp.mailfrom=38d2VZwYKCEg241oxlqyyqvo.mywvsx47-wwu5kmu.y1q@flex--surenb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737874931; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=Gjibp4HHI09y13Cb+jMTPU7WntuA24yRPhNo2MxmLn0=; b=H6XtuVDUbAoRbhVrV5x0++B9EDwsr0Gn8TOoo2FQw0oQb3RzuabC4AaC4obu6zrUv3hapm gchQlizlyf0C+yFodWgCiYB89qt6w5xsIr/DTvEExd2tI/LJHaoIKdersyXONmcOeTUjuM j/XheSZrdQltomfKBHtdXaTlSfRZYpA= Received: by mail-oo1-f74.google.com with SMTP id 006d021491bc7-5f333a71d91so1252074eaf.0 for ; Sat, 25 Jan 2025 23:02:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1737874930; x=1738479730; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=Gjibp4HHI09y13Cb+jMTPU7WntuA24yRPhNo2MxmLn0=; b=0Rv/kO1wPvD2mP+5mWCSRjWISxtUWdnd4nn6KrEZmavJvHL24lC2jsdp+oMDuNZfpS Nthqvzojl7UXDl1/PLRk1DTHZMv2gykaIM9iD6c/FHPgQb0k3Us6fhKRZkWkp0GJP1K+ esMKW3QcBETDrZy0OIhEdh+hazCXccScQBoWVf8NIDNblmoB6hnyxkKzUL6Cjx81yXKP fKa5xLxT06EVa7wHg30lO8kxbBnTnyl02a4N4rlsh9dkLLM6XYBs7m1XnLr5yEfSREMM hvuV62h1lUyfcY4riaC+yk0QwmMTD6z3qwPNdcFQGd71KrK1M2YzKppr0m+yPd1Jt4HT GqHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737874930; x=1738479730; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Gjibp4HHI09y13Cb+jMTPU7WntuA24yRPhNo2MxmLn0=; b=oQrAYESSdeqds4u5akb9bDghTdCfOm8n/XgPGW3rj5pgR2O8kXnwfyk9f9reDoV0R4 PmHujft2+29JwFadwUpKQluPZHHN3lzrW/WOtsYOdsTuGEgrLN31a+LooImQ5gZ6OmrG 0JzZt8cEl3X3B57VqBFNhY0bZq74uLUHBSaatSYhaj8HTu4WbwntMi6XxpjLQIh/2nSO j+0uA1MHa/+HFIQc/cGB9tjT6rvf+2FU96GnTb0sqjnlwzIEhbZPmBR5S6Cgy9+wDTSp HB8j65QXQ62At9hcrj9yiVB0rSkbyfXusv9x3eWaZHNBdw4WXwa+lMLLUJKwxOziBzrg 7sNA== X-Forwarded-Encrypted: i=1; AJvYcCXcleCtGxPQgqx0IDnbJVsWql6EGhpHugqO6S4QKZy3cl+zfht95gUkNRBCFcqOezVi9W6r5msbEQ==@kvack.org X-Gm-Message-State: AOJu0YwpiuVC1lQv25Vw24j9eJsK09sBTNdNIEWVqjJZfmNA3uPUuGA+ spUEOfl3RGL8raRznNcusY7CvFopkd+6adiZY/AJ+xVUKJoL6TZiJtQBFOQKtLB5xex83gVxusv sCQ== X-Google-Smtp-Source: AGHT+IFx729Tm/RHbxsPZVc8EhlfE/xBJUd+ZMgbbEChjn50lBiPKyouSgWjqxGyD7UokRCh6tCkDMiH8jU= X-Received: from oacpz5.prod.google.com ([2002:a05:6871:e485:b0:29e:147:e258]) (user=surenb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6870:4e0d:b0:29e:2f3a:761a with SMTP id 586e51a60fabf-2b1c0a349f5mr20640727fac.20.1737874929840; Sat, 25 Jan 2025 23:02:09 -0800 (PST) Date: Sat, 25 Jan 2025 23:02:04 -0800 Mime-Version: 1.0 X-Mailer: git-send-email 2.48.1.262.g85cc9f2d1e-goog Message-ID: <20250126070206.381302-1-surenb@google.com> Subject: [PATCH 1/3] mm: avoid extra mem_alloc_profiling_enabled() checks From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: kent.overstreet@linux.dev, vbabka@suse.cz, yuzhao@google.com, minchan@google.com, shakeel.butt@linux.dev, souravpanda@google.com, pasha.tatashin@soleen.com, 00107082@163.com, quic_zhenhuah@quicinc.com, surenb@google.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 2D7384000D X-Stat-Signature: rudppqqr6oh8ijmt5tapgpxps4n4ym98 X-Rspam-User: X-HE-Tag: 1737874930-37603 X-HE-Meta: U2FsdGVkX19a3opnVCmGRGqKHyPVjD2T8nxUfVxi9kGebGME5jOUh+nyS1KnM7U9MhUHplrMgAz9unS7IUusM3lOqu7o/XZ5V83iQ5IOAwG0JCbEt8HjxyApQhdIyEbEs7hyPbyTDun8HLIvuiCY+wX0VS/R40ydR2njOzxqt81cUNqFCfanLt4QCw4+fTiVPFUOxHdNuVRByjXETA0foYrObVY+Gb3bep+GQ409xqVlT0Ef4LzQShC0PIDR83qLB0vNCv13+OKzTbd+JkemgwOUoHJqUXNhxxeexWXdxJ7f8UTrRA1LDuKgZvjaoGUe7YaRr4tf/IMYIVQcEfvmy1GOBbrtv0fqo/V327QyuK1isgqtkuqaj7XzSf0kGLFD2aKJocE/aeHvYa8L6BEy+UgNPk6vnjFtUyJ0hjPIq4j5ron/4ORbQlihCtFEp8jtE/nLRGccHixJSNpzXX2DpAjqkYk2Hnl2pBsK90f1sFdhUPd1KRSUlTAhhK/2nfKITHlJJY/GvJLhP8EeIFpDH4tbKJzUuLBmPk52XT4FwgqCPQejsGRYYdTGVB7cv48Uu4MEU4JrZrnIasyz51Kc7oj7iy0RTCrZ/Y35WTE+p2bbwiOKT7QGAq17bT6vyfyWdTs/6WoSgkoivxEEjfQJh+c25pVu2dSjChO+FJxP9pUnAJspR/Wpnw3QzuTquSNKXAFnCoaBqonR3RYjgVNIYmc2Ij2QT2ifRv4TXdhIpuERt3TDLnzPCNSp1Z5OWYKxdbZ6TNome6KwhfhlSEM9yB62F+AJV72PwOsWYqHwETz50NniFMToO2XkFHStRzh1OVTBySf6f6Kdb9cIXtjjdJx177YFzkeseZsjlWX2bRh1vu1X0RbaQ3OcUV0httIJOUmzb5n7IvDBRAk3prmSSQR7raUy6YS7WO9/8qCzZu7DjbKsnzjpcYdIPsMq8f7Jv+kOdGyS8TpTas5wpnj e53QfDem pyRl4fFgsKfH95qOR9ZTElN5b2UAyJ+9lTsP0vR6OJM1mzEIhSJWmB8+J+G5mA5r2rqwhBNgz8IoKozdOGA3U2bi2Bsdc7Mp7StPlYDJTNwmCPqvgPz2PEI7iYRb1ZwCu0A9kBxDO1zESIDwj71+1EwZCHnBAnK4l2XaaWkv2fgHHlfpn+cSgU64Wk3W3vIpSeekb2Nr0L3NwBvDt25L43IrWF815xwArLOMm0DbEOmuK82jnoSj3MtoAqKOna+0SbhCZjUTKisTRtSlHX0A3AZHqnrq95dIAY5dEWt0gdjtjArrq4BQUzFn+3erPtjofWwUG28gY2yo0O7dGV67l6xLoJ4Gk+j/jIgposFIjw9kLLJ8DHcVnrGl8guZivMAR8lV2QY73/PhBxo31ASQ+w4UYoWxQSo5xMJogDVmIOdTFE43Xe4D0fsyK4C00ePGktvaiFw9qYzj+1NWcFwAf5cK769w7tNT0CQvUwvovsNxMPGhH1irRiqrMubDELjP3UaL0Ss0YK77MTqBvfmBuHhCeapTJCIST31KggFZWpUBtZFf9KGRGTe18YWxOTVLLV2Wed7x1lKjgIeM0++9vjkWkJQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000083, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Refactor code to avoid extra mem_alloc_profiling_enabled() checks inside pgalloc_tag_get() function which is often called after that check was already done. Signed-off-by: Suren Baghdasaryan --- include/linux/pgalloc_tag.h | 35 +++++++++++++++++++---------------- lib/alloc_tag.c | 6 +++--- mm/page_alloc.c | 3 +-- 3 files changed, 23 insertions(+), 21 deletions(-) base-commit: a227a25fe3fdd08f743a20a6bf6367a47b20c125 diff --git a/include/linux/pgalloc_tag.h b/include/linux/pgalloc_tag.h index 3469c4b20105..4a82b6b4820e 100644 --- a/include/linux/pgalloc_tag.h +++ b/include/linux/pgalloc_tag.h @@ -205,28 +205,32 @@ static inline void pgalloc_tag_sub(struct page *page, unsigned int nr) } } -static inline struct alloc_tag *pgalloc_tag_get(struct page *page) +/* Should be called only if mem_alloc_profiling_enabled() */ +static inline struct alloc_tag *__pgalloc_tag_get(struct page *page) { struct alloc_tag *tag = NULL; - - if (mem_alloc_profiling_enabled()) { - union pgtag_ref_handle handle; - union codetag_ref ref; - - if (get_page_tag_ref(page, &ref, &handle)) { - alloc_tag_sub_check(&ref); - if (ref.ct) - tag = ct_to_alloc_tag(ref.ct); - put_page_tag_ref(handle); - } + union pgtag_ref_handle handle; + union codetag_ref ref; + + if (get_page_tag_ref(page, &ref, &handle)) { + alloc_tag_sub_check(&ref); + if (ref.ct) + tag = ct_to_alloc_tag(ref.ct); + put_page_tag_ref(handle); } return tag; } -static inline void pgalloc_tag_sub_pages(struct alloc_tag *tag, unsigned int nr) +static inline void pgalloc_tag_sub_pages(struct page *page, unsigned int nr) { - if (mem_alloc_profiling_enabled() && tag) + struct alloc_tag *tag; + + if (!mem_alloc_profiling_enabled()) + return; + + tag = __pgalloc_tag_get(page); + if (tag) this_cpu_sub(tag->counters->bytes, PAGE_SIZE * nr); } @@ -241,8 +245,7 @@ static inline void clear_page_tag_ref(struct page *page) {} static inline void pgalloc_tag_add(struct page *page, struct task_struct *task, unsigned int nr) {} static inline void pgalloc_tag_sub(struct page *page, unsigned int nr) {} -static inline struct alloc_tag *pgalloc_tag_get(struct page *page) { return NULL; } -static inline void pgalloc_tag_sub_pages(struct alloc_tag *tag, unsigned int nr) {} +static inline void pgalloc_tag_sub_pages(struct page *page, unsigned int nr) {} static inline void alloc_tag_sec_init(void) {} static inline void pgalloc_tag_split(struct folio *folio, int old_order, int new_order) {} static inline void pgalloc_tag_swap(struct folio *new, struct folio *old) {} diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c index 19b45617bdcf..1d893e313614 100644 --- a/lib/alloc_tag.c +++ b/lib/alloc_tag.c @@ -174,7 +174,7 @@ void pgalloc_tag_split(struct folio *folio, int old_order, int new_order) if (!mem_alloc_profiling_enabled()) return; - tag = pgalloc_tag_get(&folio->page); + tag = __pgalloc_tag_get(&folio->page); if (!tag) return; @@ -200,10 +200,10 @@ void pgalloc_tag_swap(struct folio *new, struct folio *old) if (!mem_alloc_profiling_enabled()) return; - tag_old = pgalloc_tag_get(&old->page); + tag_old = __pgalloc_tag_get(&old->page); if (!tag_old) return; - tag_new = pgalloc_tag_get(&new->page); + tag_new = __pgalloc_tag_get(&new->page); if (!tag_new) return; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 6e469c7ef9a4..55ed2f245f80 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -4832,12 +4832,11 @@ void __free_pages(struct page *page, unsigned int order) { /* get PageHead before we drop reference */ int head = PageHead(page); - struct alloc_tag *tag = pgalloc_tag_get(page); if (put_page_testzero(page)) free_frozen_pages(page, order); else if (!head) { - pgalloc_tag_sub_pages(tag, (1 << order) - 1); + pgalloc_tag_sub_pages(page, (1 << order) - 1); while (order-- > 0) free_frozen_pages(page + (1 << order), order); }