From patchwork Sat Feb 3 00:34:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13543646 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A34EC48295 for ; Sat, 3 Feb 2024 00:34:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4D4416B006E; Fri, 2 Feb 2024 19:34:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 484D96B0071; Fri, 2 Feb 2024 19:34:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 34B406B0072; Fri, 2 Feb 2024 19:34:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 232BE6B006E for ; Fri, 2 Feb 2024 19:34:19 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id BF93480E0D for ; Sat, 3 Feb 2024 00:34:18 +0000 (UTC) X-FDA: 81748620996.01.A400F02 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf09.hostedemail.com (Postfix) with ESMTP id 2589D140015 for ; Sat, 3 Feb 2024 00:34:16 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Nqlwx71T; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf09.hostedemail.com: domain of 3CIq9ZQoKCEc7x107jqvnmpxxpun.lxvurw36-vvt4jlt.x0p@flex--yosryahmed.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3CIq9ZQoKCEc7x107jqvnmpxxpun.lxvurw36-vvt4jlt.x0p@flex--yosryahmed.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706920457; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=BCDzdkXKmcCm4b3zn1uKLp7BZ6kePV4TL8d+mF4c+RU=; b=m1utW0dEqWosY3pCDRATGyAEdimRER63DkqsfdYR/oDYR0BG1Azd4ce0rjSmfW7MB1Wghx WF2bJTe501W+aQeaR77q7dIeSJ+6kyAtbuz+q+g+K6VTFSHAXXQoWgWMjVRO4TAxFHZuZe GX9W/qE9+KG3YInkEzdGpkmnsY8gBS0= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=Nqlwx71T; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf09.hostedemail.com: domain of 3CIq9ZQoKCEc7x107jqvnmpxxpun.lxvurw36-vvt4jlt.x0p@flex--yosryahmed.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3CIq9ZQoKCEc7x107jqvnmpxxpun.lxvurw36-vvt4jlt.x0p@flex--yosryahmed.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706920457; a=rsa-sha256; cv=none; b=FTJr889IFiU8f2eyKAeP4EdzKwSUjzh0L1dpK1SGqFjIuWQNha+UfFmxzoOrF9OSC3LUJM C68pouIjMb8YfBYe3vq0wUxbqENABm2uxHftNer32rDtMJfxQo+4U+FZDf7ytUehfp8+oC G4bVNVShIzr0NyYQP8WjiXwcTr2/onU= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-dc693399655so4242825276.1 for ; Fri, 02 Feb 2024 16:34:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706920456; x=1707525256; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=BCDzdkXKmcCm4b3zn1uKLp7BZ6kePV4TL8d+mF4c+RU=; b=Nqlwx71TmZrpZ0cRAxlC+ByDHzInPooSBhJNktR6S+4pZxuL1Cwn2RjvFFYfmWksFQ ZphJTXGXgS9hNkre8zl6PXsdcdtR1Y3FOquWnVTZMYy3nvwAcqF+tmGQRC58zTZGVZKL x4T/T8sO1X3cDOPX88qnq45pVbypOUq3pK/otlRqCEWH1jdoxZx67eDrrqm7XpJFEAee vXXIzoTp+RNuV5rn5fwG6e2k1xzr5xs6fhG2tWZRhF5Wwe/uIyvw7tytPkwEvEj2Ujsh LIfEh2tI+lX5+TsJvxE59l9XycdG5zScojrhy2YCHlcB+eB7Hi/NAun3sv3UWbLn26V1 vVaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706920456; x=1707525256; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=BCDzdkXKmcCm4b3zn1uKLp7BZ6kePV4TL8d+mF4c+RU=; b=mmm3tK3McpFTy44+3vy1O4/UcRYtJwtQ8UgYrzzzFSNcNQBTOMwgxLqNCDYe4S54a4 Pl/Qqi2J9H/as+xhIfNXxp61cC+H8fSvLvgORIQUctQGira3mM68v5u5UpYbo3Q0hfH/ hICCiXVSz2awmzl8ialAByjsYL/v0Ng3FIhyZ4+p+N/9EwodNP1+ZRXkc2UvMHK89tql tngKkTcYF3YDBnyJhZyifkbsGUXsWnCTSz62nxWevMUZKmxG07Hu8hqo6BiC911FB4x2 dkTuqFXNctw6fZ1SPWPryCpPoQ2yTl+4ak/5s9K9JDaxU+8ANQMxWiFqP1iI1ojWWctQ +2uQ== X-Gm-Message-State: AOJu0Yxe6UDAEZ3ggRpvNNp5Rja6TUdWpwnqqhhPskdHej/NE2Iwi9lr Rcp3xppOgQSoV915qWiN59kOme1Y4+0adl2WC1J/eomQu0nrkR/VelLxjy8UpnQ07gRaMF68Zk3 bB0yGBUAEB+3eWHSYbg== X-Google-Smtp-Source: AGHT+IFD+IjFOQHibdEZhu5cMU8oPxcS9RMVwqQhE2xvOJKOdLFApGCusY2fdfTuFLUq9U4BvkSau2GMSgc0xy0p X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a05:6902:2192:b0:dc6:fec4:1c26 with SMTP id dl18-20020a056902219200b00dc6fec41c26mr37502ybb.1.1706920456138; Fri, 02 Feb 2024 16:34:16 -0800 (PST) Date: Sat, 3 Feb 2024 00:34:13 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.43.0.594.gd9cf4e227d-goog Message-ID: <20240203003414.1067730-1-yosryahmed@google.com> Subject: [PATCH mm-hotfixes-unstable] mm: memcg: fix struct memcg_vmstats_percpu size and alignment From: Yosry Ahmed To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yosry Ahmed , Greg Thelen X-Rspamd-Queue-Id: 2589D140015 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 4cxxr3ra7jsqh6ba1saho5yudhko75ma X-HE-Tag: 1706920456-985320 X-HE-Meta: U2FsdGVkX186D3ggDPQnfPGWkiyt6tLsWuf7UcBO7BIAhqF7RnzMaq3ptE5W8PW2otPtgjfKEMqfYHO4YUHbrs6ONyiOcfagn0wI+K/GOb3Tf08KHJfZhQFiS2213OhFa5Kr2Ek1fnj4UbYZVXigHy+xBcJ3r+Qf5Rmlr3J6/MAXSnTrKntdHGa0JDHvAtVwH8cbivHaCLTQXqrFzGVxW2VkeRSNxmv1AO3ZWzltfAUd2H2Cc9hvJUXlOZF1Wc/0ZhWkq30waFng3GCNxxxvQVvmenQEuwS734HpF8ejEHI9+yHDndTN8O5rhBZ9aPYQCZDfhEe++dBrIcll+lnnth0Ibmh4mruZa9CrbdQFB/TP2J3wvdphPnkMdgIRg9F/zRObRJqUaqNnL83oSXnEPrkP4ONPPsvaLYg4P73kEKe8iygdryb1XWLU32x2z14Gj2GgEgkMzOYmgtmNedXbJp3UjEQVPqSSgBmWXWq8vVy10VGUoFNZi6UPceZhCicuodifrtGcwKIWO9XY36MAIoNMym4gqHbLlNTMfZcXiDvEJa10UV0EyzzNRRJgSHcRvHvLmg9QkxhAj2u7gheiEbpXqjc/kL/Ad/2XL5DZXDX6QWYcAxC3dtHQ74aJIEY7oxWqe3YCbR5Gp+MJ1cKnphPdTOqqynalRH3p2tDZiIQYxJvfmxzV1hZSaENoZrmVQfIfvpzx5whXX/tcjMKioV07HCI+7ref3s0p6Ao2gyoL8WkJhT0x5HuewSpP9AaCGLattX7BCKmbjNU8JFKeI+taFU90lj/8ulK9HKTzPetBcmdqoTg3WGc9x4GUG+gUueoThUzvQ2cEf6FdAAiqQxCdVoPZO9EKKAmIPnOylGKEUMaJb8EHu+YeOjNFRKyBNUXsZOPYEH8dIKtoLFrLyZRb9+OnVJHpKJ2N5x5QeJKfaBpN9ZEGKBwOuygTOFHJ2+UzycaNt0hRtgdy9VO Qx6EWg1A YpVyLMh4PtEwjTFCMZir0uTVGdqVvxWaUPZFOx9CQdSOFU2bVxSuiq5lKTF1jUVBu0oGviJiJzi7ALZPTnjhL0qo0/3XxHxBIVXcfu1Jh2ZlHXHQIxHPkewoMNDluO0sqtKJ5R8gWAA+hOdr50BkZSug/3Lk3MuYLTdWrvVIuqRIZspJl7JMPuwC2624arixCFwedXsjBQsEl09zSw4qxiRMcSpL0bP9JvnuSCzGoAlzXC1cosA3VLgYf+mhZzWr3M0IAG5XE5BwThknpZLiUSl3GtTzxk9L5QsyP7WEMditUg0i9S9m8hAH6gNQpi4uGKa3AFxa2ykOXMhAT+yhZaEVZvI1Qc7lvHtNiPsvSBtwb8bkRkoMRXynscrZYYKAfDH+rupr7/LZWG7yZ46DQF84VheGqjSOtJEr9h/EVUiCA8cP73802keZduV8FQEzUNnB8CMVt5xJlA1QtXzjUliSuhN2xq83DK/2ikE8+pw0rVyLCxzaEo89XOZ12OdtTuFFNqrKkHyZFm1Nj7UsbL3nvAo+ZrppukKqlaSmPGCzBxK5Ua7U7wur011mQT9RzVkxev+84WO9u0k7dvg6tzNqVTpl6UHzCgeTGDcgsgpG5F7ZCW42pz3L17fNx4OM04RvepQSor1UELrdhcIHmYvxFxuCp6vRCelqgxw/Ptuov1w7yMhHYAVEAXQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Commit da10d7e140196 ("mm: memcg: optimize parent iteration in memcg_rstat_updated()") added two additional pointers to the end of struct memcg_vmstats_percpu with CACHELINE_PADDING to put them in a separate cacheline. This caused the struct size to increase from 1200 to 1280 on my config (80 extra bytes instead of 16). Upon revisiting, the relevant struct members do not need to be on a separate cacheline, they just need to fit in a single one. This is a percpu struct, so there shouldn't be any contention on that cacheline anyway. Move the members to the beginning of the struct and cachealign the first member. Add a comment about the members that need to fit together in a cacheline. The struct size is now 1216 on my config with this change. Fixes: da10d7e140196 ("mm: memcg: optimize parent iteration in memcg_rstat_updated()") Reported-by: Greg Thelen Signed-off-by: Yosry Ahmed --- mm/memcontrol.c | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index d9ca0fdbe4ab0..09f09f37e397e 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -621,6 +621,15 @@ static inline int memcg_events_index(enum vm_event_item idx) } struct memcg_vmstats_percpu { + /* Stats updates since the last flush */ + unsigned int stats_updates ____cacheline_aligned; + + /* Cached pointers for fast iteration in memcg_rstat_updated() */ + struct memcg_vmstats_percpu *parent; + struct memcg_vmstats *vmstats; + + /* The above should fit a single cacheline for memcg_rstat_updated() */ + /* Local (CPU and cgroup) page state & events */ long state[MEMCG_NR_STAT]; unsigned long events[NR_MEMCG_EVENTS]; @@ -632,16 +641,6 @@ struct memcg_vmstats_percpu { /* Cgroup1: threshold notifications & softlimit tree updates */ unsigned long nr_page_events; unsigned long targets[MEM_CGROUP_NTARGETS]; - - /* Fit members below in a single cacheline for memcg_rstat_updated() */ - CACHELINE_PADDING(_pad1_); - - /* Stats updates since the last flush */ - unsigned int stats_updates; - - /* Cached pointers for fast iteration in memcg_rstat_updated() */ - struct memcg_vmstats_percpu *parent; - struct memcg_vmstats *vmstats; }; struct memcg_vmstats {