| Message ID | 20240409192301.907377-2-david@redhat.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | mm: mapcount for large folios + page_mapcount() cleanups | expand |
On 9 Apr 2024, at 15:22, David Hildenbrand wrote: > Commit 53277bcf126d ("mm: support page_mapcount() on page_has_type() > pages") made it impossible to detect mapcount underflows by treating > any negative raw mapcount value as a mapcount of 0. > > We perform such underflow checks in zap_present_folio_ptes() and > zap_huge_pmd(), which would currently no longer trigger. > > Let's check against PAGE_MAPCOUNT_RESERVE instead by using > page_type_has_type(), like page_has_type() would, so we can still catch > some underflows. > > Fixes: 53277bcf126d ("mm: support page_mapcount() on page_has_type() pages") > Signed-off-by: David Hildenbrand <david@redhat.com> > --- > include/linux/mm.h | 5 ++--- > 1 file changed, 2 insertions(+), 3 deletions(-) > > diff --git a/include/linux/mm.h b/include/linux/mm.h > index ef34cf54c14f..0fb8a40f82dd 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -1229,11 +1229,10 @@ static inline void page_mapcount_reset(struct page *page) > */ > static inline int page_mapcount(struct page *page) > { > - int mapcount = atomic_read(&page->_mapcount) + 1; > + int mapcount = atomic_read(&page->_mapcount); > > /* Handle page_has_type() pages */ > - if (mapcount < 0) > - mapcount = 0; > + mapcount = page_type_has_type(mapcount) ? 0 : mapcount + 1; > if (unlikely(PageCompound(page))) > mapcount += folio_entire_mapcount(page_folio(page)); LGTM. This could be picked up separately. Reviewed-by: Zi Yan <ziy@nvidia.com> -- Best Regards, Yan, Zi
On Tue, Apr 09, 2024 at 09:22:44PM +0200, David Hildenbrand wrote: > Commit 53277bcf126d ("mm: support page_mapcount() on page_has_type() > pages") made it impossible to detect mapcount underflows by treating > any negative raw mapcount value as a mapcount of 0. Yes, but I don't think this is the right place to check for underflow. We should be checking for that on modification, not on read. I think it's more important for page_mapcount() to be fast than a debugging aid.
On 09.04.24 23:42, Matthew Wilcox wrote: > On Tue, Apr 09, 2024 at 09:22:44PM +0200, David Hildenbrand wrote: >> Commit 53277bcf126d ("mm: support page_mapcount() on page_has_type() >> pages") made it impossible to detect mapcount underflows by treating >> any negative raw mapcount value as a mapcount of 0. > > Yes, but I don't think this is the right place to check for underflow. > We should be checking for that on modification, not on read. While I don't disagree (and we'd check more instances that way, for example deferred rmap removal), that requires a bit more churn and figuring out of if losing some information we would have printed in print_bad_pte() is worth that change. > I think > it's more important for page_mapcount() to be fast than a debugging aid. I really don't think page_mapcount() is a good use of time for micro-optimizations, but let's investigate: A big hunk of code in page_mapcount() seems to be the compound handling. The code before that (reading mapcount, checking for the condition, conditionally setting it to 0), would generate right now: 177: 8b 42 30 mov 0x30(%rdx),%eax 17a: b9 00 00 00 00 mov $0x0,%ecx 17f: 83 c0 01 add $0x1,%eax 182: 0f 48 c1 cmovs %ecx,%eax My variant is longer: 17b: 8b 4a 30 mov 0x30(%rdx),%ecx 17e: 81 f9 7f ff ff ff cmp $0xffffff7f,%ecx 184: 8d 41 01 lea 0x1(%rcx),%eax 187: b9 00 00 00 00 mov $0x0,%ecx 18c: 0f 4e c1 cmovle %ecx,%eax 18f: 48 8b 0a mov (%rdx),%rcx The compiler does not seem to do the smart thing, which would be rearranging the code to effectively be: diff --git a/include/linux/mm.h b/include/linux/mm.h index ef34cf54c14f..7392596882ae 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1232,7 +1232,7 @@ static inline int page_mapcount(struct page *page) int mapcount = atomic_read(&page->_mapcount) + 1; /* Handle page_has_type() pages */ - if (mapcount < 0) + if (mapcount < PAGE_MAPCOUNT_RESERVE + 1) mapcount = 0; if (unlikely(PageCompound(page))) mapcount += folio_entire_mapcount(page_folio(page)); Which would result in: 177: 8b 42 30 mov 0x30(%rdx),%eax 17a: 31 c9 xor %ecx,%ecx 17c: 83 c0 01 add $0x1,%eax 17f: 83 f8 80 cmp $0xffffff80,%eax 182: 0f 4e c1 cmovle %ecx,%eax Same code length, one more instruction. No jumps. I can switch to the above (essentially inlining page_type_has_type()) for now and look into different sanity checks -- and extending the documentation around page_mapcount() behavior for underflows -- separately. ... unless you insist that we really have to change that immediately. Thanks!
On 09.04.24 21:22, David Hildenbrand wrote: > Commit 53277bcf126d ("mm: support page_mapcount() on page_has_type() > pages") made it impossible to detect mapcount underflows by treating > any negative raw mapcount value as a mapcount of 0. > > We perform such underflow checks in zap_present_folio_ptes() and > zap_huge_pmd(), which would currently no longer trigger. > > Let's check against PAGE_MAPCOUNT_RESERVE instead by using > page_type_has_type(), like page_has_type() would, so we can still catch > some underflows. > > Fixes: 53277bcf126d ("mm: support page_mapcount() on page_has_type() pages") > Signed-off-by: David Hildenbrand <david@redhat.com> > --- > include/linux/mm.h | 5 ++--- > 1 file changed, 2 insertions(+), 3 deletions(-) > > diff --git a/include/linux/mm.h b/include/linux/mm.h > index ef34cf54c14f..0fb8a40f82dd 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -1229,11 +1229,10 @@ static inline void page_mapcount_reset(struct page *page) > */ > static inline int page_mapcount(struct page *page) > { > - int mapcount = atomic_read(&page->_mapcount) + 1; > + int mapcount = atomic_read(&page->_mapcount); > > /* Handle page_has_type() pages */ > - if (mapcount < 0) > - mapcount = 0; > + mapcount = page_type_has_type(mapcount) ? 0 : mapcount + 1; > if (unlikely(PageCompound(page))) > mapcount += folio_entire_mapcount(page_folio(page)); > From b49849001f3d2aad0af93cf2098065d7cbd9a959 Mon Sep 17 00:00:00 2001 From: David Hildenbrand <david@redhat.com> Date: Wed, 24 Apr 2024 10:50:09 +0200 Subject: [PATCH] !fixup: mm: allow for detecting underflows with page_mapcount() again Let's make page_mapcount() slighly more efficient by inlining the page_type_has_type() check. Signed-off-by: David Hildenbrand <david@redhat.com> --- include/linux/mm.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index dc33f8269fb52..cf700c5cdd58b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1229,10 +1229,11 @@ static inline void page_mapcount_reset(struct page *page) */ static inline int page_mapcount(struct page *page) { - int mapcount = atomic_read(&page->_mapcount); + int mapcount = atomic_read(&page->_mapcount) + 1; /* Handle page_has_type() pages */ - mapcount = page_type_has_type(mapcount) ? 0 : mapcount + 1; + if (mapcount < PAGE_MAPCOUNT_RESERVE + 1) + mapcount = 0; if (unlikely(PageCompound(page))) mapcount += folio_entire_mapcount(page_folio(page));
diff --git a/include/linux/mm.h b/include/linux/mm.h index ef34cf54c14f..0fb8a40f82dd 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1229,11 +1229,10 @@ static inline void page_mapcount_reset(struct page *page) */ static inline int page_mapcount(struct page *page) { - int mapcount = atomic_read(&page->_mapcount) + 1; + int mapcount = atomic_read(&page->_mapcount); /* Handle page_has_type() pages */ - if (mapcount < 0) - mapcount = 0; + mapcount = page_type_has_type(mapcount) ? 0 : mapcount + 1; if (unlikely(PageCompound(page))) mapcount += folio_entire_mapcount(page_folio(page));
Commit 53277bcf126d ("mm: support page_mapcount() on page_has_type() pages") made it impossible to detect mapcount underflows by treating any negative raw mapcount value as a mapcount of 0. We perform such underflow checks in zap_present_folio_ptes() and zap_huge_pmd(), which would currently no longer trigger. Let's check against PAGE_MAPCOUNT_RESERVE instead by using page_type_has_type(), like page_has_type() would, so we can still catch some underflows. Fixes: 53277bcf126d ("mm: support page_mapcount() on page_has_type() pages") Signed-off-by: David Hildenbrand <david@redhat.com> --- include/linux/mm.h | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)