From patchwork Fri Sep 6 04:21:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13793200 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3753BCE7AAE for ; Fri, 6 Sep 2024 04:21:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 76B1C6B007B; Fri, 6 Sep 2024 00:21:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 71B306B0082; Fri, 6 Sep 2024 00:21:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6097A6B0085; Fri, 6 Sep 2024 00:21:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 4429F6B007B for ; Fri, 6 Sep 2024 00:21:15 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id D6FAE140E12 for ; Fri, 6 Sep 2024 04:21:14 +0000 (UTC) X-FDA: 82533013668.12.F3A943A Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf15.hostedemail.com (Postfix) with ESMTP id 34DD5A0002 for ; Fri, 6 Sep 2024 04:21:13 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=AZ2jEptt; spf=pass (imf15.hostedemail.com: domain of 3OIPaZgYKCOUfbgOHVNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--yuzhao.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3OIPaZgYKCOUfbgOHVNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725596448; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=6rMHBDk6aZCVvFepOF3gEumvZVHdTWOkgoEnOWibS94=; b=LmAXbx+juvWXBL+4h43ij440vXEH2F2cr6Z7/8tw8lvF2Wv5PJqfbmipMUV6nZKucrwSo7 7KWIgZZPnLovcLcsk0JX1kT8DaOt8RPwCiEz/9WyOiZfgmzAuKw8o+khgcYHAVA6VWOnxp 3+Q14tuJTP9RyJ0RxMoW6ffVYg6Ucfk= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=AZ2jEptt; spf=pass (imf15.hostedemail.com: domain of 3OIPaZgYKCOUfbgOHVNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--yuzhao.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3OIPaZgYKCOUfbgOHVNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725596448; a=rsa-sha256; cv=none; b=5WNFNC1U/hegOX60OG7ubytNJ1+f9bcU3ounCZrwaXRvDd50P8W7dzKsrCXb4QMlZOconH r5ZI4BYk3ggx4alnOunKLjfbTTa+vVazNHVVF9VqwcWDLreHtNBTMtRfYBRPTDpjECcAJ9 2ZGqAPD23Fv+Lo0ZTNpSXpKMsvsZxGE= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6d6bb05f2e9so44231177b3.0 for ; Thu, 05 Sep 2024 21:21:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725596472; x=1726201272; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=6rMHBDk6aZCVvFepOF3gEumvZVHdTWOkgoEnOWibS94=; b=AZ2jEpttox5npS9u+AWu1yf8GXxtwnUQv/mKfv/TdfZ+f0ygFxh8PRxZhJjgY2Z2eg wHtkRn86miV3LKAMJtihfXExUz2Csj1N5WSbbEF1O8DU9Q/pmdV/K5e5YANddjaLR1QN fsHeUjwvEEN7fMxPLW1iFqQ3NSN+Vq2zMftBtPj9NMIWcZasWR/+/aVdsDhmgHHwQjsY 12vFdsFlLNRqQpT53b1j5AA1nfn7IqxVO9YsBbZiOQbYZZTfG1AdYakqgOHcx7ZRJy98 9OpTMZWd2p64TAxkfhcD8XNEAu/at89P/pGAc7RNcwikf04gUoULVHxzMtg0R0jxRudD hVsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725596472; x=1726201272; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=6rMHBDk6aZCVvFepOF3gEumvZVHdTWOkgoEnOWibS94=; b=H8X/zZZXdOxQjslwKvHEX9cma2AU33OoqT1wsgLw7y9bY/Nv8ZO7y7DK8NvrJxCBqG 3NrrjyUvFav4xltbMayP6l3AFBViezH+p6Mdqpa0hvIxbfItv5p4WCAA8fAoQloow48y GjHvvCtaX2TYliFE3joTgkcqUkJsqZPdKRWdHyUWsqJ9NvSSv29K6awCcfdhfcsV+y2o dxgualPR+zv5aNg/bHmoPHKD7k0IfSijE2++Hn8/W3LvPhruvnNdcUygLgaNSfpdQ6dv VpYc/aWRMDdfrRnWZZ7uVNDKym/V9yjNpGKucbs0YWyXahPsbM/hF9vlf6kWHFMHFp95 /tmQ== X-Forwarded-Encrypted: i=1; AJvYcCU0AFWev544A0sU80o2G8xfEBqm2Yw+t3oHDo75QFy7JG75SAXJQBrdPZUEX9l54gJIaiZ2c+ukzg==@kvack.org X-Gm-Message-State: AOJu0Yxa7qVSZmB7gAHfXdE+8iDv+6laBw/VhZaJFrXlgRaaZxMWFMBK RsoPOJpT4sclMYCNphdM7AhT+tAWHklEafNPZEOw5LRwpsoK7owdPaSDcb8+PGBivzAgNSMk/Wf 1tQ== X-Google-Smtp-Source: AGHT+IH6Q0BKXGClpmZdeK87d3hTRQYTLHXHkCn11z/zfigFOkJw6sxJj5jJwGXGw7e9+20J7sDOfg5ifv8= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:5b7a:cdaf:9b3d:354a]) (user=yuzhao job=sendgmr) by 2002:a05:690c:d91:b0:62c:f6fd:5401 with SMTP id 00721157ae682-6db451533ecmr738287b3.6.1725596472083; Thu, 05 Sep 2024 21:21:12 -0700 (PDT) Date: Thu, 5 Sep 2024 22:21:06 -0600 Mime-Version: 1.0 X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240906042108.1150526-1-yuzhao@google.com> Subject: [PATCH mm-unstable v2 1/3] mm/codetag: fix a typo From: Yu Zhao To: Andrew Morton , Kent Overstreet , Suren Baghdasaryan Cc: Muchun Song , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao X-Rspam-User: X-Stat-Signature: 977g5xoppxzsrxisqfupeenkoo78u799 X-Rspamd-Queue-Id: 34DD5A0002 X-Rspamd-Server: rspam11 X-HE-Tag: 1725596473-183762 X-HE-Meta: U2FsdGVkX1+qaDzbS3Z9AstHVCdAilF+Qnh888FG3DjjvOkTMYtjFwFaxtuFoMtaxAC1nhMG9Uhnfy/yvqE+ZnkD0pYP+SzVP2ipEES4xTNln/Ye934LnN1EX1VPk+E9DyyAb/bfZ0jWr4gCByOI8lfbnQs4ETnLEUNyhX2s7l51BY3Z2Khu1UCo5wUMPqGd9c4EErVu93+H9/PUVzMyMyYyeY2HhhLhpbuBjCvJX9jeHW2O/7m2WU1GtMB7V7Vdj4nYraLh9JgioZxPZ5WBRsgl3JfIiTmkwi53wvIr7NsunJb5+bS+ofP5gOEXFoyYpNF1ePLtrcbHvwylrALH/m5WCkiWvSURop7l7JfDEsVcOi6cOTsB2Dys8ckDlNbll9VuAYUftqXsv/25YAgCoXB6v1CYlNurrXyKaj5/CvvY3h/+nF8lADExw2wRa22mcMS04Vzstf+TOcBIUlLupWNsrt73/a7Cy36VCEPu1qK6Bv8qXkYUwfD4yR+V3fn+48S5smhtUafmry6p+swJ68Kw7Ur1BIwrs9zDAHIyBn3yM/+fkOpP+C8M3smavMAibDtZ+oU+2EJ75Aigkpc8L9oSrmaEBF7l0PWw2YnNSTdgwASVFLGTAi2soaFCHFf8p+tWfXX3HN2Y2njTTWdDEXP2+/stW7Ld8whBkxYH97Bp47uCAjfMyKipFDGg4DkARnmzyqNAuBK6nDK3cU+2K6buaZBstqLm10mcV+g8lBn8HWqCPqFjduzvqaiOKK3IMlebIE+22QMoa40ACZwkoZ7JcOSpR8vlr48caC/ufZPvJI83M64axPVqGb2r9hSadURzCWTblr+TnktPINep/UtryWP2xdktXyzLSu9HrCMKa3BmUbc63VXnncyppZ2c+Vq0+tXMbmSof+cKncrLzDJ5GtsyFcW7NBXukkYp6Ipb1ewafozvnRp7xY+u5Sizvc9dcJzjx5XQGpKyY6t KyjpGKjF AY9l7n90CztzMqKdZau0sX9cEX93yf4ms4e60cvx5pF84weqCXmmXz7sG+NCvPIq7t/unDhbgkvJTZI/YeWKAT9wsAvSaq3mZ0djQTIFZxtZ/DAXnZn/To1Uc64hsk7fVH4tX9bGrO0/10QLVDvvbzk71LTOP3UCcNg5mc8x4dC/jc5eYKXS1HlCKEC4C22eU+Go/aFntdXj2eOZDFzOCaCCF3NONPC+NLT1qxBCcz8XmSbgk/wbS93av0oMfgawjJOLgpH0YW7iDdT29xu1tO1xcQGcCJ1axPkEb4OfuZ6syXBYJCkq4/sSWzvpfnp/zsujWNyo5lNdEpxOOuVdIcuETmpahSb9p04zEiIqAonDyPb5At8pZCk1Zwqijg9juFzzqQf3CosE+9Lk5EXcsnkFM3ck8A6ZDOjRnyPPkR0yvE4wDtMWPUEu26GxIFF9YJogMhshZj0ThmPGqyAxRY9Hn6gip/kZ5f2494qPAiocorRkvyWECSd7CG8olctGMvlVztQMygHSoktYXgRF0jjJS0Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.007688, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Fixes: 22d407b164ff ("lib: add allocation tagging support for memory allocation profiling") Signed-off-by: Yu Zhao Acked-by: Suren Baghdasaryan Acked-by: Muchun Song --- include/linux/alloc_tag.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/include/linux/alloc_tag.h b/include/linux/alloc_tag.h index 8c61ccd161ba..896491d9ebe8 100644 --- a/include/linux/alloc_tag.h +++ b/include/linux/alloc_tag.h @@ -70,7 +70,7 @@ static inline struct alloc_tag *ct_to_alloc_tag(struct codetag *ct) /* * When percpu variables are required to be defined as weak, static percpu * variables can't be used inside a function (see comments for DECLARE_PER_CPU_SECTION). - * Instead we will accound all module allocations to a single counter. + * Instead we will account all module allocations to a single counter. */ DECLARE_PER_CPU(struct alloc_tag_counters, _shared_alloc_tag); From patchwork Fri Sep 6 04:21:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13793201 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 09EBCCE7AAE for ; Fri, 6 Sep 2024 04:21:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9B4A76B0082; Fri, 6 Sep 2024 00:21:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 965506B0088; Fri, 6 Sep 2024 00:21:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 82C266B0089; Fri, 6 Sep 2024 00:21:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 642686B0082 for ; Fri, 6 Sep 2024 00:21:18 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0DA43140E12 for ; Fri, 6 Sep 2024 04:21:18 +0000 (UTC) X-FDA: 82533013836.23.4ABBEA1 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf11.hostedemail.com (Postfix) with ESMTP id 4940D40011 for ; Fri, 6 Sep 2024 04:21:16 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=HnJpyWOD; spf=pass (imf11.hostedemail.com: domain of 3O4PaZgYKCOgiejRKYQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--yuzhao.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3O4PaZgYKCOgiejRKYQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725596427; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=t8z7JhTPd5a0ay4gHmViQDV8ynpAlUlJ3UT1hxWkVyQ=; b=w4maggSjamimjUqcwUCABldGjwF9PRWXv+Xi80jEeoau3YMSqBs2wFH+qQl6dP1aBZaWTr XCuMjA6ZJQazd0bFZ8EemqfhzyYyQTeipGHIPrMpE5C7GDPZSxco/L+yZFWi/gR8qKbUYO 94Cq6W/sKJy87ij4cu2/3y5cd5P5Sc8= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=HnJpyWOD; spf=pass (imf11.hostedemail.com: domain of 3O4PaZgYKCOgiejRKYQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--yuzhao.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3O4PaZgYKCOgiejRKYQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725596427; a=rsa-sha256; cv=none; b=6Hjw5XnIcr1N31M0HbWSCLFK3Ou6ZcaihfVpBVBB8S4g2Is1dpw9f0fin+EeRAz89hcH0x 79kQ6jREe2jDXRzWYlm3U7845FqPeEQlB9cu7JhtElaTog8pwNpX9PQoyhSpDhc7Lmrbnc cExmW+k2ass2WWXd5Tbi12DQ4SahzN4= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-e1d3255f852so1304731276.3 for ; Thu, 05 Sep 2024 21:21:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725596475; x=1726201275; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=t8z7JhTPd5a0ay4gHmViQDV8ynpAlUlJ3UT1hxWkVyQ=; b=HnJpyWODiE0C8HocmIIPO5jdGgZk2p11F7zbQ/QinUpPLRwqhuZFiAyqAfFV8gw8df vozgZ72ZJ86OH6QYwR+nkbcTjeKQVJUFAdDVZ8PuGlliqsG34DWBqaLG7bvvMp7B0CDB CU0KFBHMszU0zwYaCe5YGJR53m3/Ol3ZRsx+MsDthQ5COK6hcepwTzPva+DCG4l5vco2 HcRjcAacn5jrkmSpYQ8SN5duaAVeDnFpd5kinHxtzWYRR303Z7QFxRRvrbFxv5KnjN3s CzIChp+26zjUZEEwNpITfyqmz2f54pa5oH4HfzRXFbU403ykNqAilZqRwqXcQVQSMxXL kEUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725596475; x=1726201275; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=t8z7JhTPd5a0ay4gHmViQDV8ynpAlUlJ3UT1hxWkVyQ=; b=KlCpcCKe2d40qA+S37OlFJDzVwEneAvW9qmNX7OVhT9DDE1Aw7yPAAdcf7crejYFbi tmODaNCl6X55BoQQ93OgnLPq+yheglhjuc/lUP9ZjwqjBhB8oeZkjl11kUYBmRZofC1l 8sPzmVE7Lqc1UMAD1fx0qS2DV9u4dnL3UjtDoiEvYz1h4Me1pUEJxRPtnpFq3C2ZuTHs pP8UJtyCKILZpk0revmuD5dd59xEY+pU7g3r/kI1JI+CZTpOWcT2DExTNpDjOjH+jZNX iH803OlVYFSNBWLtJLa2LPG9UgzkCFCn7F9ATFMqXDBymH8Q3/VFV78pKCjhMCiP3LZH BsnQ== X-Forwarded-Encrypted: i=1; AJvYcCV76UU2aHCOUj/tNQ+T3IQtss7ly4qufQTd4pjnH5DlYG87fa20xvVgKlhwn4j7Mw1Axb2CWgAU1w==@kvack.org X-Gm-Message-State: AOJu0YzsTP55xg9AwbfOzm0RPZWCIAlMdCe49aGpbd7crO4V7cG1+A4h 8+n4tmyMMuMJ/9/HqBE//cwOUGGhqiyy3gbZ54XyEstnj8tbscDrO8EEWoMSlQP8ug4zd+hwMeB ikA== X-Google-Smtp-Source: AGHT+IHnLZW3xX/3Rs6MjEAzTLo2oLLdlVk4kTRKTlbd52QR9KUnODr0nzvAIBIuE8r0TUpZnBgXzd5k3bg= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:5b7a:cdaf:9b3d:354a]) (user=yuzhao job=sendgmr) by 2002:a05:6902:2d43:b0:e16:4e62:8a17 with SMTP id 3f1490d57ef6-e1d3485d5cemr5514276.2.1725596475174; Thu, 05 Sep 2024 21:21:15 -0700 (PDT) Date: Thu, 5 Sep 2024 22:21:07 -0600 In-Reply-To: <20240906042108.1150526-1-yuzhao@google.com> Mime-Version: 1.0 References: <20240906042108.1150526-1-yuzhao@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240906042108.1150526-2-yuzhao@google.com> Subject: [PATCH mm-unstable v2 2/3] mm/codetag: fix pgalloc_tag_split() From: Yu Zhao To: Andrew Morton , Kent Overstreet , Suren Baghdasaryan Cc: Muchun Song , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao , stable@vger.kernel.org X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 4940D40011 X-Stat-Signature: aagfm38pustoxwdczz6a5a9w8k4pzr4b X-HE-Tag: 1725596476-403832 X-HE-Meta: U2FsdGVkX18BWnZsU+l3spZ76pjOBtz2ObT8h3gUSH1za67Uo+57GGrfKbCfBleemcIT/9If9ik5NLJ/cb2VPrZkqXoc1MpdguCLiPwbdl7IP+KTlPmeHqnN1wwG7QMSSKWoKm7Bb1rKlp595oyKAjQb5hcie53t2a7IY3raocNt+X7wW+wuS7fQmFf687NjpmbU3pDz5XtWFEQS8K/EwsHJEGtKHDa19uLN2/BYt+lZ6GBxvlOWBPomMuDlwlKR+TIB+etxlK7ZdDN3uCWsCw49qBJeBsHCf3/VvUK76Yv8FJRyyDgBvQ5zQvOXUqo32oW1T0xLNR9F4n83RqyYHe+iiiJmd1cLv72tVOCTQydUgpmw/Ckqzbe8uwK0Z6Hc7O8WfLVWOEHR7BvUMOZ44B+S4gzV4LJHt9JwlykDSFwurx5mgQFHw3IjCdKKaXWPEISKl0v9G1nTI4V7HirBh/wOJehScumZZArtIX2749HSwkw1rYFBhwt/xHLnt52RxZcmUNsoM7ifZIvSVnBk9szVlZCWn3QFLRuNa3AF+UzO0LAxmNW3rdZyz3pC7eVjgQgDAcnzMrewpgdVtkLhR6GsnH+RY/QzTQPIfwEKAkdGpiqJ7Q8y48tie5vP3xY44zb2prGrak1ePKIfmltGEK2rPjvOODG3VirPBgWDHARflnK11gXP8lGtdY2WHWNnXDKNhdpFKGdgR4eIEG84MNu7opgS0fcswh2Ws0wbrO6rhkmZ5X66DkZnUU6lrjvzEYl7UGUFrrceaV+uAUDVvTF7u4TnF+c848LRjNCYch76i21mVQfTXooh9J6mfo783auappwMZAppStFZpyyYDk8R1lnBIxnfHzTs5sCyfR890C9aVb1xocB86KdPb/8xsksow+oLB/TWD2zXnQ8BAtAMwTbL5jP7y68SDUd49MHXHT+AIktzgz3LlwGwUOnNpuZAsuk6bbWIyg0JGc3 792FAKLH 4uESqaErUUMc4uxayhUhwH5wErqpi7xk7v/RJZGgdtahl9yVgFMTIywXiHfHD4A1zj34gFkqIUk98Cr44kxTvFTVe0v9UBGUI4e6fVsTJ6l/lrCV2fmeO5EjHf2phwcU5YQHu8kCOei21CJr26i89BViGQUKw9jPeAWbwyb9HeRZRmcsj5FTq1GrG7IIFcQ+svEbQW2zRrfXUDnXvzRG6KVpAhyyCKUpCpajJuWxlaxU9lCWXNTukYTOUvtAcNufvFRXsLJVZ2EOcKPtNqogOxwLzPowKmOjyIvPY520BcbQKTF7hzmLqLP4mMgPwKfrHydnfQAUt9+e4MAaS+V61vlqGeSsNzsNT2Z0p1Q/O+Me9pBqdTxNiuyG0UkxCO+nqfrQznD07TLyk9lMLyv1cpagCAFrHxrsdvxN2ybcaGHaAZ6wOV5pOqAKKmhoKAGYyDh79NhU2emm/81oM/E0GlQZADCupOuKFUBJeU5Dwc0YvOrzApTCJNzFY1gl3bQpQRyhilcwta/LR7yEfT4pEDrc7kBPvZYxYZKrpUb1LOgJpQ3XMudD5ja39OA+spDWiHIARLhbPLi264aAGL0K+iURBmiM1zpRTm72CyH3hEDOmdSs= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The current assumption is that a large folio can only be split into order-0 folios. That is not the case for hugeTLB demotion, nor for THP split: see commit c010d47f107f ("mm: thp: split huge page to any lower order pages"). When a large folio is split into ones of a lower non-zero order, only the new head pages should be tagged. Tagging tail pages can cause imbalanced "calls" counters, since only head pages are untagged by pgalloc_tag_sub() and the "calls" counts on tail pages are leaked, e.g., # echo 2048kB >/sys/kernel/mm/hugepages/hugepages-1048576kB/demote_size # echo 700 >/sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages # time echo 700 >/sys/kernel/mm/hugepages/hugepages-1048576kB/demote # echo 0 >/sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages # grep alloc_gigantic_folio /proc/allocinfo Before this patch: 0 549427200 mm/hugetlb.c:1549 func:alloc_gigantic_folio real 0m2.057s user 0m0.000s sys 0m2.051s After this patch: 0 0 mm/hugetlb.c:1549 func:alloc_gigantic_folio real 0m1.711s user 0m0.000s sys 0m1.704s Not tagging tail pages also improves the splitting time, e.g., by about 15% when demoting 1GB hugeTLB folios to 2MB ones, as shown above. Fixes: be25d1d4e822 ("mm: create new codetag references during page splitting") Signed-off-by: Yu Zhao Acked-by: Suren Baghdasaryan Cc: --- include/linux/mm.h | 30 ++++++++++++++++++++++++++++++ include/linux/pgalloc_tag.h | 31 ------------------------------- mm/huge_memory.c | 2 +- mm/hugetlb.c | 2 +- mm/page_alloc.c | 4 ++-- 5 files changed, 34 insertions(+), 35 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index b31d4bdd65ad..a07e93adb8ad 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4137,4 +4137,34 @@ void vma_pgtable_walk_end(struct vm_area_struct *vma); int reserve_mem_find_by_name(const char *name, phys_addr_t *start, phys_addr_t *size); +#ifdef CONFIG_MEM_ALLOC_PROFILING +static inline void pgalloc_tag_split(struct folio *folio, int old_order, int new_order) +{ + int i; + struct alloc_tag *tag; + unsigned int nr_pages = 1 << new_order; + + if (!mem_alloc_profiling_enabled()) + return; + + tag = pgalloc_tag_get(&folio->page); + if (!tag) + return; + + for (i = nr_pages; i < (1 << old_order); i += nr_pages) { + union codetag_ref *ref = get_page_tag_ref(folio_page(folio, i)); + + if (ref) { + /* Set new reference to point to the original tag */ + alloc_tag_ref_set(ref, tag); + put_page_tag_ref(ref); + } + } +} +#else /* !CONFIG_MEM_ALLOC_PROFILING */ +static inline void pgalloc_tag_split(struct folio *folio, int old_order, int new_order) +{ +} +#endif /* CONFIG_MEM_ALLOC_PROFILING */ + #endif /* _LINUX_MM_H */ diff --git a/include/linux/pgalloc_tag.h b/include/linux/pgalloc_tag.h index 207f0c83c8e9..59a3deb792a8 100644 --- a/include/linux/pgalloc_tag.h +++ b/include/linux/pgalloc_tag.h @@ -80,36 +80,6 @@ static inline void pgalloc_tag_sub(struct page *page, unsigned int nr) } } -static inline void pgalloc_tag_split(struct page *page, unsigned int nr) -{ - int i; - struct page_ext *first_page_ext; - struct page_ext *page_ext; - union codetag_ref *ref; - struct alloc_tag *tag; - - if (!mem_alloc_profiling_enabled()) - return; - - first_page_ext = page_ext = page_ext_get(page); - if (unlikely(!page_ext)) - return; - - ref = codetag_ref_from_page_ext(page_ext); - if (!ref->ct) - goto out; - - tag = ct_to_alloc_tag(ref->ct); - page_ext = page_ext_next(page_ext); - for (i = 1; i < nr; i++) { - /* Set new reference to point to the original tag */ - alloc_tag_ref_set(codetag_ref_from_page_ext(page_ext), tag); - page_ext = page_ext_next(page_ext); - } -out: - page_ext_put(first_page_ext); -} - static inline struct alloc_tag *pgalloc_tag_get(struct page *page) { struct alloc_tag *tag = NULL; @@ -142,7 +112,6 @@ static inline void clear_page_tag_ref(struct page *page) {} static inline void pgalloc_tag_add(struct page *page, struct task_struct *task, unsigned int nr) {} static inline void pgalloc_tag_sub(struct page *page, unsigned int nr) {} -static inline void pgalloc_tag_split(struct page *page, unsigned int nr) {} static inline struct alloc_tag *pgalloc_tag_get(struct page *page) { return NULL; } static inline void pgalloc_tag_sub_pages(struct alloc_tag *tag, unsigned int nr) {} diff --git a/mm/huge_memory.c b/mm/huge_memory.c index fdc83b0c9e71..2a73efea02d7 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3242,7 +3242,7 @@ static void __split_huge_page(struct page *page, struct list_head *list, /* Caller disabled irqs, so they are still disabled here */ split_page_owner(head, order, new_order); - pgalloc_tag_split(head, 1 << order); + pgalloc_tag_split(folio, order, new_order); /* See comment in __split_huge_page_tail() */ if (folio_test_anon(folio)) { diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 2a73753ecf9e..5c77defad295 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3795,7 +3795,7 @@ static long demote_free_hugetlb_folios(struct hstate *src, struct hstate *dst, list_del(&folio->lru); split_page_owner(&folio->page, huge_page_order(src), huge_page_order(dst)); - pgalloc_tag_split(&folio->page, 1 << huge_page_order(src)); + pgalloc_tag_split(folio, huge_page_order(src), huge_page_order(dst)); for (i = 0; i < pages_per_huge_page(src); i += pages_per_huge_page(dst)) { struct page *page = folio_page(folio, i); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 6b003f57965d..88113fdba956 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2783,7 +2783,7 @@ void split_page(struct page *page, unsigned int order) for (i = 1; i < (1 << order); i++) set_page_refcounted(page + i); split_page_owner(page, order, 0); - pgalloc_tag_split(page, 1 << order); + pgalloc_tag_split(page_folio(page), order, 0); split_page_memcg(page, order, 0); } EXPORT_SYMBOL_GPL(split_page); @@ -4981,7 +4981,7 @@ static void *make_alloc_exact(unsigned long addr, unsigned int order, struct page *last = page + nr; split_page_owner(page, order, 0); - pgalloc_tag_split(page, 1 << order); + pgalloc_tag_split(page_folio(page), order, 0); split_page_memcg(page, order, 0); while (page < --last) set_page_refcounted(last); From patchwork Fri Sep 6 04:21:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13793202 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50BF1CE7AAE for ; Fri, 6 Sep 2024 04:21:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CC2C86B0089; Fri, 6 Sep 2024 00:21:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C71986B008C; Fri, 6 Sep 2024 00:21:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B12E46B008A; Fri, 6 Sep 2024 00:21:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 931CF6B0088 for ; Fri, 6 Sep 2024 00:21:21 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 1FD28160EC8 for ; Fri, 6 Sep 2024 04:21:21 +0000 (UTC) X-FDA: 82533013962.15.89D7FD8 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf20.hostedemail.com (Postfix) with ESMTP id 62DEC1C0009 for ; Fri, 6 Sep 2024 04:21:19 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=pW95mU8K; spf=pass (imf20.hostedemail.com: domain of 3PYPaZgYKCOokglTMaSaaSXQ.OaYXUZgj-YYWhMOW.adS@flex--yuzhao.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3PYPaZgYKCOokglTMaSaaSXQ.OaYXUZgj-YYWhMOW.adS@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725596454; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=50wP6OBJW621Z1lTCrGAHD/paOiHcQB2J19sjWWdS44=; b=C2qvpUQqJsf5L6+PoKYfUoBVSAL8zTwbypkMibGrLDQNo6Ka9r73Z1vOwJsS9VCadOqlRM zIfzIUkVg77FuyTeel8zme88Smc4srDlG2Lb3KXNlMVdyrNbX3pxaA/OfQ1DOVzh0OjzMU ZMli0JCPpZx+0WKbJzf3GF4c4xkf6iQ= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=pW95mU8K; spf=pass (imf20.hostedemail.com: domain of 3PYPaZgYKCOokglTMaSaaSXQ.OaYXUZgj-YYWhMOW.adS@flex--yuzhao.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3PYPaZgYKCOokglTMaSaaSXQ.OaYXUZgj-YYWhMOW.adS@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725596454; a=rsa-sha256; cv=none; b=rcxS6iqlCqsah9GP/LnMQ6aS5sCk14l+ASxDKDPNLCWi/q8jsBBuGZxH/QTFPTDoNGAkTQ cf7sQL6LruR6zWDttcKxXF7C5e17aEGGdOM/2SA9/Zx3e+dZZQO51oWlgL3a2ovfrgd9lU BdvvH62WEc1Y0JS926Q4pT8Rwv/WM1o= Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-e03623b24ddso3881313276.1 for ; Thu, 05 Sep 2024 21:21:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1725596478; x=1726201278; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=50wP6OBJW621Z1lTCrGAHD/paOiHcQB2J19sjWWdS44=; b=pW95mU8KEFIHUoxNUkGfQFOfz0Fp1FGTIE45dnBxUQ50bCWFxxJsZZ25UMKAWrOYnZ OGbcThTEl8v5QkwcIDHSPYmElBGbVroUvaaHMPeBB9UDZDrztgWhibdibDPuki40NRUa S3BViweJYSJotOd2OBzlCR42QczfwlAOngha62L9Tbpuqzj3hyQaJPpsTvoWVEo5OQ9E XLSSZ9zB3+9dzAwgaG2Vw3+V+S8PKlEmtavJkqkeGG7BvVs/i5f5pDO2DZvLvOEjWZGD kyB/ap0LqNpa4sy08tjbVoRShXL+jqOHI/Y+gUOsnQ4Sz9LI4CK7c6mE5UPutIfm1uzn ZsFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725596478; x=1726201278; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=50wP6OBJW621Z1lTCrGAHD/paOiHcQB2J19sjWWdS44=; b=MJT7t7IJsCnaLNMVLuRjajdekHhnL4qCfAa80dsjsv+CKIbMK+bifsW7M3vuEUT5P2 geaKxwb0SyDTzGdGWkHEjXpHbEUb72ejGP1VPXiZSjxMAW7aDwlgbR1w9oY/4uq2heVu AmLay2xZBB1GdYOPgfDuq52qGGExFkTrr2t5XCM3L4nVnLs9ZKWHkrz3gx5I1ONm44Vu V8DLmiGqMtNM2rQ+ZMVuGHHy3gaVuH8sKykM2+DUDS+pXTm5H71IzfElMVKK6NXPO96q Odjx8Cd27C9mlNjaExlL6L3SkoQNsDkcYcbulwKzc0l05VAsx4ZQud/A+CbGseGDRgYA twSQ== X-Forwarded-Encrypted: i=1; AJvYcCXXEBVzPFeU1V0nNDUg78pciOgOJDZr3rn/5ypdDRcbPLn+xKt9rvqeaEzUnBjSHAY5YVsqsUqKng==@kvack.org X-Gm-Message-State: AOJu0YyeQbD8xLTUYyjnCD7tPaH+qMgqQUdzzbmVYgv/W3Ivtd6SM5CL fErpL2Rf1qbGPjZPkPOT5scdaV6Swb0FTO09b8+cUIiAmMDwE5R/eHu5msIUUHqNnVELkN2vpsM B9g== X-Google-Smtp-Source: AGHT+IF4EfUu9keMzscep5B8o/9Ylkdf8A/cgbCOdPnpDmV6X9nqkqfVEQVj/bbPL42JtNauX/vBJjc/UKI= X-Received: from yuzhao2.bld.corp.google.com ([2a00:79e0:2e28:6:5b7a:cdaf:9b3d:354a]) (user=yuzhao job=sendgmr) by 2002:a25:d84e:0:b0:e11:5da7:337 with SMTP id 3f1490d57ef6-e1d3487f694mr2558276.3.1725596477951; Thu, 05 Sep 2024 21:21:17 -0700 (PDT) Date: Thu, 5 Sep 2024 22:21:08 -0600 In-Reply-To: <20240906042108.1150526-1-yuzhao@google.com> Mime-Version: 1.0 References: <20240906042108.1150526-1-yuzhao@google.com> X-Mailer: git-send-email 2.46.0.469.g59c65b2a67-goog Message-ID: <20240906042108.1150526-3-yuzhao@google.com> Subject: [PATCH mm-unstable v2 3/3] mm/codetag: add pgalloc_tag_copy() From: Yu Zhao To: Andrew Morton , Kent Overstreet , Suren Baghdasaryan Cc: Muchun Song , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao , stable@vger.kernel.org X-Rspam-User: X-Stat-Signature: bbgufshobypi1to4qwx89qi6gzah86tz X-Rspamd-Queue-Id: 62DEC1C0009 X-Rspamd-Server: rspam11 X-HE-Tag: 1725596479-448835 X-HE-Meta: U2FsdGVkX1+rGBhpxQzNdwVP8ZV9jpybc6bD6YpXzfiUYJaae6A2JjvJIqgqfwqgps+jcL8fD/4OYVSFrxL3m23pKdQxWs9BbwHumDFucMxwSkxlJQarY1B8inXvubpRRGq6MYRnkgbesq7njt++bMK9L6uTfVdvH39/uFwI/0U3Dxe79zEX+RMSLddzpf1B6rL1D1yaICfIhDLihV4tVUjO0iofVbzwPgOA540A3x3Akq2EPzIK0WzfapCVITCim4A8rxYKmzFv3u2Iklp/0gzPJ5hw3X+nyCLFF5cSM/jYpSaEGICYSNNMkTGgr0bB2x9F+Rt3WDs83qJ226xEUIxP9jaCKozq5YfiTVJTn9JqKV7y+FvGKVDMQvWME/HEEsNMMNhAkrArkA3x9GHnp5n/XGQd/JTfZFcz2duFFGWZQZE/URtLwUAuo8aVetOy+EYKT9y0au71Z6O+WKzLGIRCZ7y+rifv0J07NVGhkCz/7KfATXQf0LyE3/4fe/73D9JQKtLpoizBv5xCfGCTq1giQzCVz0ScKAQGMVnZh7lZZhNfyZYyKYsR867D7wNOsUhmrBi7QUbsemR3zJ6WQ0nvfzLQnhuKAsbieDAuV+T3Jjp5swTBaFKIvUpx2fpXb0e6udhmzjLfaTzV2lXK4vue2aBc8b45CWrR8NkG8bdDwo/rk16tyEu/HPmr5eg6hYunKo9PoRC/Xo0t+bun8EJK2SImSbGpVWY/oMjygfxrRozqOThhyj1nCZ69QmdelfK6OnBbVB2hUBXmDXUnqeuPTOhnbNOWtdvb3N+wRJtarP32Hh0WkxWhD603A6L41cHoVUXS4QqRhfXLJNuqOLQa1z3UFkvEdVf9z01SkKaOodOOfDpzv7KSdrCDndE+4uDod0LDNoyxpvXCIxu1S9ciCQX0mw5JJLBeDqK2NtqSFCWP+YyMXjAhM7yX+P8MeumAaYoM6ZTbdJGxOF3 utJxRZGo 1t0UvVYf6z6pKQocpxMl9cRSigi32m0/nm8gbRyanZtNFrm56FS0RaWpceLP6lIAq3UxYITQGwxWe2aFha86n0hel1Jdx00iqjc+1PR0Z/aMmR1TkiJ6w/vNbL0N3RHue9AhkRFq6o6J1Sme/UJMIu/xN312CQWB+cEnWrt9X/t1IGMpHsL2teJavOO5WHXv/MwnIOQW5xZ69dfEqSeqlDdi1UeFK4K2vk3ZHnILrtWCte7txiHIBcXy1l7RwUHxia3IBjpuLHWqrYdup/02hZkjXxErzBkWd450w3dAf/w7u/NUQEnbAAXSks7B2Ja99m6a6GTz5aZhHrsuV6ACaOukuncI/aUS6vyFAHphLq/bSVHwNTGdCQWqOBm4E1JNsOtdyYP2v95NNSZNdx+XnJfW2bXXw/EMRZzY8VXrW5ZXXviYzWXFP8FNnSlV5ECx4SjBxyfxOmZLUv4yYOCeVUBIsb2uTodwDWYwTx5mD0dMyJaU/CX1PQ3cuyFvUXLPo96nyhHrlQxj4hgYFq0Pr7yncCcYpW9ln4gPK5oZ5w6CmjPhQuOmAdimrZPBbzVkp2zte7ux+nUhIDqwP0OfXdAZ8Nc0lP154yKrGJWvdotKrpnU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000004, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add pgalloc_tag_copy() to transfer the codetag from the old folio to the new one during migration. This makes original allocation sites persist cross migration rather than lump into the get_new_folio callbacks passed into migrate_pages(), e.g., compaction_alloc(): # echo 1 >/proc/sys/vm/compact_memory # grep compaction_alloc /proc/allocinfo Before this patch: 132968448 32463 mm/compaction.c:1880 func:compaction_alloc After this patch: 0 0 mm/compaction.c:1880 func:compaction_alloc Fixes: dcfe378c81f7 ("lib: introduce support for page allocation tagging") Signed-off-by: Yu Zhao Acked-by: Suren Baghdasaryan Cc: --- include/linux/alloc_tag.h | 24 ++++++++++-------------- include/linux/mm.h | 27 +++++++++++++++++++++++++++ mm/migrate.c | 1 + 3 files changed, 38 insertions(+), 14 deletions(-) diff --git a/include/linux/alloc_tag.h b/include/linux/alloc_tag.h index 896491d9ebe8..1f0a9ff23a2c 100644 --- a/include/linux/alloc_tag.h +++ b/include/linux/alloc_tag.h @@ -137,7 +137,16 @@ static inline void alloc_tag_sub_check(union codetag_ref *ref) {} /* Caller should verify both ref and tag to be valid */ static inline void __alloc_tag_ref_set(union codetag_ref *ref, struct alloc_tag *tag) { + alloc_tag_add_check(ref, tag); + if (!ref || !tag) + return; + ref->ct = &tag->ct; +} + +static inline void alloc_tag_ref_set(union codetag_ref *ref, struct alloc_tag *tag) +{ + __alloc_tag_ref_set(ref, tag); /* * We need in increment the call counter every time we have a new * allocation or when we split a large allocation into smaller ones. @@ -147,22 +156,9 @@ static inline void __alloc_tag_ref_set(union codetag_ref *ref, struct alloc_tag this_cpu_inc(tag->counters->calls); } -static inline void alloc_tag_ref_set(union codetag_ref *ref, struct alloc_tag *tag) -{ - alloc_tag_add_check(ref, tag); - if (!ref || !tag) - return; - - __alloc_tag_ref_set(ref, tag); -} - static inline void alloc_tag_add(union codetag_ref *ref, struct alloc_tag *tag, size_t bytes) { - alloc_tag_add_check(ref, tag); - if (!ref || !tag) - return; - - __alloc_tag_ref_set(ref, tag); + alloc_tag_ref_set(ref, tag); this_cpu_add(tag->counters->bytes, bytes); } diff --git a/include/linux/mm.h b/include/linux/mm.h index a07e93adb8ad..d750be768121 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4161,10 +4161,37 @@ static inline void pgalloc_tag_split(struct folio *folio, int old_order, int new } } } + +static inline void pgalloc_tag_copy(struct folio *new, struct folio *old) +{ + struct alloc_tag *tag; + union codetag_ref *ref; + + tag = pgalloc_tag_get(&old->page); + if (!tag) + return; + + ref = get_page_tag_ref(&new->page); + if (!ref) + return; + + /* Clear the old ref to the original allocation tag. */ + clear_page_tag_ref(&old->page); + /* Decrement the counters of the tag on get_new_folio. */ + alloc_tag_sub(ref, folio_nr_pages(new)); + + __alloc_tag_ref_set(ref, tag); + + put_page_tag_ref(ref); +} #else /* !CONFIG_MEM_ALLOC_PROFILING */ static inline void pgalloc_tag_split(struct folio *folio, int old_order, int new_order) { } + +static inline void pgalloc_tag_copy(struct folio *new, struct folio *old) +{ +} #endif /* CONFIG_MEM_ALLOC_PROFILING */ #endif /* _LINUX_MM_H */ diff --git a/mm/migrate.c b/mm/migrate.c index 0f6b78fd73aa..dfdb3a136bf8 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -743,6 +743,7 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio) folio_set_readahead(newfolio); folio_copy_owner(newfolio, folio); + pgalloc_tag_copy(newfolio, folio); mem_cgroup_migrate(folio, newfolio); }