From patchwork Wed Jul 3 01:53:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13720529 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 074B1C30658 for ; Wed, 3 Jul 2024 01:54:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9A1CC6B009B; Tue, 2 Jul 2024 21:54:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 951FB6B009C; Tue, 2 Jul 2024 21:54:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81A046B009D; Tue, 2 Jul 2024 21:54:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 6447A6B009B for ; Tue, 2 Jul 2024 21:54:01 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id E4981A1A68 for ; Wed, 3 Jul 2024 01:54:00 +0000 (UTC) X-FDA: 82296770640.27.9170769 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf27.hostedemail.com (Postfix) with ESMTP id 4AA5A40008 for ; Wed, 3 Jul 2024 01:53:59 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=f5maakDV; spf=pass (imf27.hostedemail.com: domain of 3Nq-EZgYKCOEVXUHQEJRRJOH.FRPOLQXa-PPNYDFN.RUJ@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3Nq-EZgYKCOEVXUHQEJRRJOH.FRPOLQXa-PPNYDFN.RUJ@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719971628; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=4BVLSgqw94eTxc2ajevJhP5Tdjoc+6UHkyHk7q4GUNA=; b=8Kn4N3QEWNmgImClMTnPAa+DFspu/B6ZLVftP4AUm4+uv4YZjzSic15IPgRXe/FAvSDgVN NV3OgSF/A/l5uANytsB15FbYLkEg8ZVW39o3tjLophMx+Xm3nW0RWmuUdpgT8uQ5V1WBr9 cAAi5ZR6lAs0sA/ur4gnJ3H2oRzqSPo= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=f5maakDV; spf=pass (imf27.hostedemail.com: domain of 3Nq-EZgYKCOEVXUHQEJRRJOH.FRPOLQXa-PPNYDFN.RUJ@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3Nq-EZgYKCOEVXUHQEJRRJOH.FRPOLQXa-PPNYDFN.RUJ@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719971628; a=rsa-sha256; cv=none; b=CJvJ+3A2ZnS2C57LuNUq8p+wLq0NurXuBF5Mrg+0pmOdTThFNfnHOyp+wSLcyafma+vVZ8 FmFJU17Peho6Gy/JOnksZlkjsA1PBqHYBJ8OzNAkcMHDsX4BkiTPskhurEm251e/7plY9d DUL+8G8TcveBfqixU0bhzLosL9Lf020= Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6499b840395so1783657b3.1 for ; Tue, 02 Jul 2024 18:53:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1719971638; x=1720576438; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=4BVLSgqw94eTxc2ajevJhP5Tdjoc+6UHkyHk7q4GUNA=; b=f5maakDV1l6dEAUytuu9QaHMNbbb0A28r5tCVnFIgF/IWCna2Sz51hP53hjcBu2Ouu qgqJSrgjs/MzPw9syFtfZ6tFkC2tnOxVZWjq26bjQYGZdtmLTCyhDAptsTgvRHaAUF/9 jSh4ufRleYvBzJiS1pGJz5ma+ziVfutR5ie2mYvB5lZl3o7Q959/5jUvWvq2e57YLE/d lw9J0xItO9hU9qZ+IbVvbvnvoFhjoTYfJhe5rwA1yvx1hAyQkrC+Cfr3kflHUonLQyWg rXVz6hq93xzBOHIWe6kVjigrcJFuABnF9ls48xJYIWiFRAaX09bRd9FP/ZV1y6vQllZS SCJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719971638; x=1720576438; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=4BVLSgqw94eTxc2ajevJhP5Tdjoc+6UHkyHk7q4GUNA=; b=DJ8fb/CV7VroUlycwYUJJr/QknQAOSbr/eyrOX57GJ3DRTPyKob+2FtcVoyZothE3H xc2K7JKaaQek2roI2Z8eJYb75tMSMjItj2ILnt96RzMhFM32H30t5hxWU7T6EY+ZHL+G /HX3diLr7BO4+lLs+VKZkBqUo3GmvMYKwyyWOgJ/Gn4Mb7moHZR/drSgre4a10UaWhRh e4AiCxaie0bZigucXXhvyjLwlD7eud/TIXvVKt3rklccm9CzwkaSPf9b72bnRw833gMY vXYh535s8IC1nukQPTNTDb8AAin3Pc3hpWpHbuU6KL6XV5OEBDHF4VPK88QXrFgjnfHS dCKw== X-Forwarded-Encrypted: i=1; AJvYcCWhuxtIxhSdkj4z5wIiuN2H5kVtWM9aNc83Zz5fy70VP9DIjCkFRAmrvy+q/sHjqNxDBglpH7tB1zPViHELWQUkDGk= X-Gm-Message-State: AOJu0YyynwWLwWZ1ZhxA9vBGzPVUQzTMafMFreY4C9nQh/lu6QloDjoI hXXgXO06lTYFwQEKEnSSffqf0AiZHmt2S3Ua64V7Oo7MtPTjix/e8kSbulugCWp3sOFD/VI2gL1 wgA== X-Google-Smtp-Source: AGHT+IEeZbkreIAhFbscB4qIouzrsQvPZrbQM5suKCIYA5b5GLlh+nCoMvlGLTKiL6KPIeANce5Idc9gJMc= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:857f:8d15:eb07:82ef]) (user=surenb job=sendgmr) by 2002:a05:6902:2b89:b0:dfd:da3f:ad1c with SMTP id 3f1490d57ef6-e03ad92b77fmr20518276.4.1719971638333; Tue, 02 Jul 2024 18:53:58 -0700 (PDT) Date: Tue, 2 Jul 2024 18:53:54 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.45.2.803.g4e1b14247a-goog Message-ID: <20240703015354.3370503-1-surenb@google.com> Subject: [PATCH 1/1] mm, slab: move allocation tagging code in the alloc path into a hook From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: vbabka@suse.cz, kent.overstreet@linux.dev, cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, roman.gushchin@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, surenb@google.com X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 4AA5A40008 X-Stat-Signature: 3opt9wz8rsib8zzefphxt6we5as1a3sg X-HE-Tag: 1719971639-696415 X-HE-Meta: U2FsdGVkX1/PdEOf8fgzLJSlS5duVAgMF7gmMPS9yU144xm9bEAfvLXZCkr/EpnHC9O4uFiVm7hmJ0HpzP9U36+c2GlScMdIM2bPkg4iku3YqCusIt1z8ezR+mPQV3dOywmrLy54uXKP+QjyhBdJpszT1HTHWMc7GOlQbq2jrw5iFoPLaWBFf7SkL1bQcBCf6rVHCOv4YOwAR7X1A+TIgzsY3KUMA3aCpCmwUclVL5IwaCVU6xfTQXkI1MiFlNLpd5pQpXJqPMJcDym97YcgatzjXVR+RFyXcAiINDFCw8JDJDj6NArIGC45QDpGMJU2aQPWoVtjsInkj0hQy6UHYxG1fNHq5zlBDra8AhjIHUL2j80gp8HiA/j07usmmRPzYqikcBe2ND1gyiVJMtPDYHmVOx6yjMUOQJMkKgv5tLI1QezNTVjX3NGDe/DWa3sPex0DO59jtmDzsFlRgOl6JxrwOYc/d8vRVKZIZqktVo517UtPBwjQsC1yL5OTXAaNuQwI2F049se5D+fODLf9MYSrQbcKLuI/D96U0r59S1tQ3KMbG4ZG91I0CKTfCpgeDJPndNeIcZ2A5Gh83G4BZJZ6/aTVfB/Cq0HsN+y/ZrqUSKPbFUoW4BffOTCPd6a4ROy3alEu3FirjIcqXZ689YskyS2tzmaWARKT7n1elhyPtqzf3A1JLj5wTM1uyH9Rgb5HddiiV5eoWBHel9rDRN4jajGX7v2a0g0cW4uMidFMVZ4hS9NVh+bWWgWkoxv6IkJg6ZJZkqJM3YjR7PXnpjGFW0007vCjIr3zV1aqIpmHcUFRqNdpLTG5PXS2L46hnsSm6dX9SCU0WEzBlBvtc3lnJsKM+DO401D0CRNypOoPzHTJmzbqewjQ7519QQbEKTMdgzxkzVWrmtpFOMRbaZmyTAPt20Z/JXLH5PKRu1Zs/Bm/JsSBy/kSu+Lkq56OoqNveG5GK7vl27YBPPy Y06xabD1 MPJPfORzERWIqrBO3q0vv3sCRf+wwL/H1JgApaS0MWI66Irj16JtBj0zUpM9z4ywNWVggtOgs4iF785Z9Y1LJ6+k3i3V8/RqqQT06Whp7P9xGm1Qx/RFsozYrDbOSljj3HVRTCdB3kdE+O/dLrHAaGVOjVhEYQDOfBo2DFzXBBc8+Haa6Kl/R7gRtnf07K1yGvRQn4+0TOlw2iHZPdBWrLqAt/6wJT9RL23YSIn4/k2QUznIIxXlYEhYEvXxtnBSBGhtnifxcLJiR7L10K/0YM2K9UKL90i3Eqpo+xq9pDYkoYuur2ji/kdifePV3xeZoSr0+fpZ2dBixqI8zb9dMUVKchqOpIKqettPHO6hzI2FQ0NWekGt16D571bkR6he30q8784e4PsILS72LvdFMN55mIFyvHAHYq5f0YmJZHEvlPiQ1JeZ9eDxKwQnhzvTVXXNwyeafMrWKvpk9nsCQDla/FeVTgVazEKH2claSpeNIzWu+Y4CUdAig00IYutqX24Llq1JThHSMbQM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.003948, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Move allocation tagging specific code in the allocation path into alloc_tagging_slab_alloc_hook, similar to how freeing path uses alloc_tagging_slab_free_hook. No functional changes, just code cleanup. Suggested-by: Vlastimil Babka Signed-off-by: Suren Baghdasaryan --- mm/slub.c | 34 +++++++++++++++++++++++++++++----- 1 file changed, 29 insertions(+), 5 deletions(-) base-commit: e9d22f7a6655941fc8b2b942ed354ec780936b3e diff --git a/mm/slub.c b/mm/slub.c index 4927edec6a8c..99d53190cfcf 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2033,11 +2033,18 @@ prepare_slab_obj_exts_hook(struct kmem_cache *s, gfp_t flags, void *p) return slab_obj_exts(slab) + obj_to_index(s, slab, p); } +#ifdef CONFIG_MEM_ALLOC_PROFILING + +static inline void +alloc_tagging_slab_alloc_hook(struct slabobj_ext *obj_exts, unsigned int size) +{ + alloc_tag_add(&obj_exts->ref, current->alloc_tag, size); +} + static inline void alloc_tagging_slab_free_hook(struct kmem_cache *s, struct slab *slab, void **p, int objects) { -#ifdef CONFIG_MEM_ALLOC_PROFILING struct slabobj_ext *obj_exts; int i; @@ -2053,9 +2060,23 @@ alloc_tagging_slab_free_hook(struct kmem_cache *s, struct slab *slab, void **p, alloc_tag_sub(&obj_exts[off].ref, s->size); } -#endif } +#else /* CONFIG_MEM_ALLOC_PROFILING */ + +static inline void +alloc_tagging_slab_alloc_hook(struct slabobj_ext *obj_exts, unsigned int size) +{ +} + +static inline void +alloc_tagging_slab_free_hook(struct kmem_cache *s, struct slab *slab, void **p, + int objects) +{ +} + +#endif /* CONFIG_MEM_ALLOC_PROFILING*/ + #else /* CONFIG_SLAB_OBJ_EXT */ static int alloc_slab_obj_exts(struct slab *slab, struct kmem_cache *s, @@ -2079,6 +2100,11 @@ prepare_slab_obj_exts_hook(struct kmem_cache *s, gfp_t flags, void *p) return NULL; } +static inline void +alloc_tagging_slab_alloc_hook(struct slabobj_ext *obj_exts, unsigned int size) +{ +} + static inline void alloc_tagging_slab_free_hook(struct kmem_cache *s, struct slab *slab, void **p, int objects) @@ -3944,7 +3970,6 @@ bool slab_post_alloc_hook(struct kmem_cache *s, struct list_lru *lru, kmemleak_alloc_recursive(p[i], s->object_size, 1, s->flags, init_flags); kmsan_slab_alloc(s, p[i], init_flags); -#ifdef CONFIG_MEM_ALLOC_PROFILING if (need_slab_obj_ext()) { struct slabobj_ext *obj_exts; @@ -3955,9 +3980,8 @@ bool slab_post_alloc_hook(struct kmem_cache *s, struct list_lru *lru, * check should be added before alloc_tag_add(). */ if (likely(obj_exts)) - alloc_tag_add(&obj_exts->ref, current->alloc_tag, s->size); + alloc_tagging_slab_alloc_hook(obj_exts, s->size); } -#endif } return memcg_slab_post_alloc_hook(s, lru, flags, size, p);