| Message ID | 20240129143221.263763-3-david@redhat.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | mm/memory: optimize unmap/zap with PTE-mapped THP | expand |
On 29/01/2024 14:32, David Hildenbrand wrote: > We don't need uptodate accessed/dirty bits, so in theory we could > replace ptep_get_and_clear_full() by an optimized ptep_clear_full() > function. Let's rely on the provided pte. > > Further, there is no scenario where we would have to insert uffd-wp > markers when zapping something that is not a normal page (i.e., zeropage). > Add a sanity check to make sure this remains true. > > should_zap_folio() no longer has to handle NULL pointers. This change > replaces 2/3 "!page/!folio" checks by a single "!page" one. > > Signed-off-by: David Hildenbrand <david@redhat.com> Reviewed-by: Ryan Roberts <ryan.roberts@arm.com> > --- > mm/memory.c | 20 ++++++++++---------- > 1 file changed, 10 insertions(+), 10 deletions(-) > > diff --git a/mm/memory.c b/mm/memory.c > index 50a6c79c78fc..69502cdc0a7d 100644 > --- a/mm/memory.c > +++ b/mm/memory.c > @@ -1497,10 +1497,6 @@ static inline bool should_zap_folio(struct zap_details *details, > if (should_zap_cows(details)) > return true; > > - /* E.g. the caller passes NULL for the case of a zero folio */ > - if (!folio) > - return true; > - > /* Otherwise we should only zap non-anon folios */ > return !folio_test_anon(folio); > } > @@ -1543,19 +1539,23 @@ static inline void zap_present_pte(struct mmu_gather *tlb, > struct page *page; > > page = vm_normal_page(vma, addr, ptent); > - if (page) > - folio = page_folio(page); > + if (!page) { > + /* We don't need up-to-date accessed/dirty bits. */ > + ptep_get_and_clear_full(mm, addr, pte, tlb->fullmm); > + arch_check_zapped_pte(vma, ptent); > + tlb_remove_tlb_entry(tlb, pte, addr); > + VM_WARN_ON_ONCE(userfaultfd_wp(vma)); > + ksm_might_unmap_zero_page(mm, ptent); > + return; > + } > > + folio = page_folio(page); > if (unlikely(!should_zap_folio(details, folio))) > return; > ptent = ptep_get_and_clear_full(mm, addr, pte, tlb->fullmm); > arch_check_zapped_pte(vma, ptent); > tlb_remove_tlb_entry(tlb, pte, addr); > zap_install_uffd_wp_if_needed(vma, addr, pte, details, ptent); > - if (unlikely(!page)) { > - ksm_might_unmap_zero_page(mm, ptent); > - return; > - } > > if (!folio_test_anon(folio)) { > if (pte_dirty(ptent)) {
diff --git a/mm/memory.c b/mm/memory.c index 50a6c79c78fc..69502cdc0a7d 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1497,10 +1497,6 @@ static inline bool should_zap_folio(struct zap_details *details, if (should_zap_cows(details)) return true; - /* E.g. the caller passes NULL for the case of a zero folio */ - if (!folio) - return true; - /* Otherwise we should only zap non-anon folios */ return !folio_test_anon(folio); } @@ -1543,19 +1539,23 @@ static inline void zap_present_pte(struct mmu_gather *tlb, struct page *page; page = vm_normal_page(vma, addr, ptent); - if (page) - folio = page_folio(page); + if (!page) { + /* We don't need up-to-date accessed/dirty bits. */ + ptep_get_and_clear_full(mm, addr, pte, tlb->fullmm); + arch_check_zapped_pte(vma, ptent); + tlb_remove_tlb_entry(tlb, pte, addr); + VM_WARN_ON_ONCE(userfaultfd_wp(vma)); + ksm_might_unmap_zero_page(mm, ptent); + return; + } + folio = page_folio(page); if (unlikely(!should_zap_folio(details, folio))) return; ptent = ptep_get_and_clear_full(mm, addr, pte, tlb->fullmm); arch_check_zapped_pte(vma, ptent); tlb_remove_tlb_entry(tlb, pte, addr); zap_install_uffd_wp_if_needed(vma, addr, pte, details, ptent); - if (unlikely(!page)) { - ksm_might_unmap_zero_page(mm, ptent); - return; - } if (!folio_test_anon(folio)) { if (pte_dirty(ptent)) {
We don't need uptodate accessed/dirty bits, so in theory we could replace ptep_get_and_clear_full() by an optimized ptep_clear_full() function. Let's rely on the provided pte. Further, there is no scenario where we would have to insert uffd-wp markers when zapping something that is not a normal page (i.e., zeropage). Add a sanity check to make sure this remains true. should_zap_folio() no longer has to handle NULL pointers. This change replaces 2/3 "!page/!folio" checks by a single "!page" one. Signed-off-by: David Hildenbrand <david@redhat.com> --- mm/memory.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)