diff mbox series

[mm-unstable,v1,3/3] mm/codetag: add pgalloc_tag_copy()

Message ID 20240903213649.3566695-3-yuzhao@google.com (mailing list archive)
State New
Headers show
Series [mm-unstable,v1,1/3] mm/codetag: fix a typo | expand

Commit Message

Yu Zhao Sept. 3, 2024, 9:36 p.m. UTC
Add pgalloc_tag_copy() to transfer the codetag from the old folio to
the new one during migration. This makes original allocation sites
persist cross migration rather than lump into compaction_alloc, e.g.,
  # echo 1 >/proc/sys/vm/compact_memory
  # grep compaction_alloc /proc/allocinfo

Before this patch:
  132968448  32463  mm/compaction.c:1880 func:compaction_alloc

After this patch:
          0      0  mm/compaction.c:1880 func:compaction_alloc

Signed-off-by: Yu Zhao <yuzhao@google.com>
---
 include/linux/alloc_tag.h | 24 ++++++++++--------------
 include/linux/mm.h        | 25 +++++++++++++++++++++++++
 mm/migrate.c              |  1 +
 3 files changed, 36 insertions(+), 14 deletions(-)

Comments

Andrew Morton Sept. 5, 2024, 11:25 p.m. UTC | #1
On Tue,  3 Sep 2024 15:36:49 -0600 Yu Zhao <yuzhao@google.com> wrote:

> Add pgalloc_tag_copy() to transfer the codetag from the old folio to
> the new one during migration. This makes original allocation sites
> persist cross migration rather than lump into compaction_alloc, e.g.,
>   # echo 1 >/proc/sys/vm/compact_memory
>   # grep compaction_alloc /proc/allocinfo
> 
> Before this patch:
>   132968448  32463  mm/compaction.c:1880 func:compaction_alloc
> 
> After this patch:
>           0      0  mm/compaction.c:1880 func:compaction_alloc
> 

I'm thinking that [2/3] should be backported?

And possibly this one, but for that we should identify a Fixes:, please.
Suren Baghdasaryan Sept. 6, 2024, 12:03 a.m. UTC | #2
On Thu, Sep 5, 2024 at 4:25 PM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> On Tue,  3 Sep 2024 15:36:49 -0600 Yu Zhao <yuzhao@google.com> wrote:
>
> > Add pgalloc_tag_copy() to transfer the codetag from the old folio to
> > the new one during migration. This makes original allocation sites
> > persist cross migration rather than lump into compaction_alloc, e.g.,
> >   # echo 1 >/proc/sys/vm/compact_memory
> >   # grep compaction_alloc /proc/allocinfo
> >
> > Before this patch:
> >   132968448  32463  mm/compaction.c:1880 func:compaction_alloc
> >
> > After this patch:
> >           0      0  mm/compaction.c:1880 func:compaction_alloc
> >
>
> I'm thinking that [2/3] should be backported?

Yes, should be CC'ed to stable #6.10

>
> And possibly this one, but for that we should identify a Fixes:, please.

I think for this one Fixes: dcfe378c81f7 ("lib: introduce support for
page allocation tagging") would be the most appropriate.

I'll review all 3 fixes later today.
Thanks!
Suren Baghdasaryan Sept. 6, 2024, 3:27 a.m. UTC | #3
On Thu, Sep 5, 2024 at 5:03 PM Suren Baghdasaryan <surenb@google.com> wrote:
>
> On Thu, Sep 5, 2024 at 4:25 PM Andrew Morton <akpm@linux-foundation.org> wrote:
> >
> > On Tue,  3 Sep 2024 15:36:49 -0600 Yu Zhao <yuzhao@google.com> wrote:
> >
> > > Add pgalloc_tag_copy() to transfer the codetag from the old folio to
> > > the new one during migration. This makes original allocation sites
> > > persist cross migration rather than lump into compaction_alloc, e.g.,
> > >   # echo 1 >/proc/sys/vm/compact_memory
> > >   # grep compaction_alloc /proc/allocinfo
> > >
> > > Before this patch:
> > >   132968448  32463  mm/compaction.c:1880 func:compaction_alloc
> > >
> > > After this patch:
> > >           0      0  mm/compaction.c:1880 func:compaction_alloc
> > >
> >
> > I'm thinking that [2/3] should be backported?
>
> Yes, should be CC'ed to stable #6.10
>
> >
> > And possibly this one, but for that we should identify a Fixes:, please.
>
> I think for this one Fixes: dcfe378c81f7 ("lib: introduce support for
> page allocation tagging") would be the most appropriate.
>
> I'll review all 3 fixes later today.

Looks good to me. Both [2/3] and [3/3] should be sent to stable 6.10.

Acked-by: Suren Baghdasaryan <surenb@google.com>

> Thanks!
diff mbox series

Patch

diff --git a/include/linux/alloc_tag.h b/include/linux/alloc_tag.h
index 896491d9ebe8..1f0a9ff23a2c 100644
--- a/include/linux/alloc_tag.h
+++ b/include/linux/alloc_tag.h
@@ -137,7 +137,16 @@  static inline void alloc_tag_sub_check(union codetag_ref *ref) {}
 /* Caller should verify both ref and tag to be valid */
 static inline void __alloc_tag_ref_set(union codetag_ref *ref, struct alloc_tag *tag)
 {
+	alloc_tag_add_check(ref, tag);
+	if (!ref || !tag)
+		return;
+
 	ref->ct = &tag->ct;
+}
+
+static inline void alloc_tag_ref_set(union codetag_ref *ref, struct alloc_tag *tag)
+{
+	__alloc_tag_ref_set(ref, tag);
 	/*
 	 * We need in increment the call counter every time we have a new
 	 * allocation or when we split a large allocation into smaller ones.
@@ -147,22 +156,9 @@  static inline void __alloc_tag_ref_set(union codetag_ref *ref, struct alloc_tag
 	this_cpu_inc(tag->counters->calls);
 }
 
-static inline void alloc_tag_ref_set(union codetag_ref *ref, struct alloc_tag *tag)
-{
-	alloc_tag_add_check(ref, tag);
-	if (!ref || !tag)
-		return;
-
-	__alloc_tag_ref_set(ref, tag);
-}
-
 static inline void alloc_tag_add(union codetag_ref *ref, struct alloc_tag *tag, size_t bytes)
 {
-	alloc_tag_add_check(ref, tag);
-	if (!ref || !tag)
-		return;
-
-	__alloc_tag_ref_set(ref, tag);
+	alloc_tag_ref_set(ref, tag);
 	this_cpu_add(tag->counters->bytes, bytes);
 }
 
diff --git a/include/linux/mm.h b/include/linux/mm.h
index a07e93adb8ad..1b98d843a5e9 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -4161,10 +4161,35 @@  static inline void pgalloc_tag_split(struct folio *folio, int old_order, int new
 		}
 	}
 }
+
+static inline void pgalloc_tag_copy(struct folio *new, struct folio *old)
+{
+	struct alloc_tag *tag;
+	union codetag_ref *ref;
+
+	tag = pgalloc_tag_get(&old->page);
+	if (!tag)
+		return;
+
+	ref = get_page_tag_ref(&new->page);
+	if (!ref)
+		return;
+
+	/* Clear the old ref to the original allocation site. */
+	clear_page_tag_ref(&old->page);
+	/* Decrement the counters of the tag on get_new_folio. */
+	alloc_tag_sub(ref, folio_nr_pages(new));
+	__alloc_tag_ref_set(ref, tag);
+	put_page_tag_ref(ref);
+}
 #else /* !CONFIG_MEM_ALLOC_PROFILING */
 static inline void pgalloc_tag_split(struct folio *folio, int old_order, int new_order)
 {
 }
+
+static inline void pgalloc_tag_copy(struct folio *new, struct folio *old)
+{
+}
 #endif /* CONFIG_MEM_ALLOC_PROFILING */
 
 #endif /* _LINUX_MM_H */
diff --git a/mm/migrate.c b/mm/migrate.c
index 35cc9d35064b..0b24021d5fee 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -750,6 +750,7 @@  void folio_migrate_flags(struct folio *newfolio, struct folio *folio)
 		folio_set_readahead(newfolio);
 
 	folio_copy_owner(newfolio, folio);
+	pgalloc_tag_copy(newfolio, folio);
 
 	mem_cgroup_migrate(folio, newfolio);
 }