| Message ID | 20250210193801.781278-16-david@redhat.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | mm: fixes for device-exclusive entries (hmm) | expand |
On Mon, 10 Feb 2025 20:37:57 +0100 David Hildenbrand <david@redhat.com> wrote: > Ever since commit b756a3b5e7ea ("mm: device exclusive memory access") > we can return with a device-exclusive entry from page_vma_mapped_walk(). > > damon_folio_mkold_one() is not prepared for that and calls > damon_ptep_mkold() with PFN swap PTEs. Teach damon_ptep_mkold() to deal > with these PFN swap PTEs. Note that device-private entries are so far not > applicable on that path, as damon_get_folio() filters out non-lru > folios. > > Should we just skip PFN swap PTEs completely? Possible, but it seems > straight forward to just handle it correctly. > > Note that we could currently only run into this case with > device-exclusive entries on THPs. We still adjust the mapcount on > conversion to device-exclusive; this makes the rmap walk > abort early for small folios, because we'll always have > !folio_mapped() with a single device-exclusive entry. We'll adjust the > mapcount logic once all page_vma_mapped_walk() users can properly > handle device-exclusive entries. > > Signed-off-by: David Hildenbrand <david@redhat.com> Reviewed-by: SeongJae Park <sj@kernel.org> Thanks, SJ [...]
diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c index d25d99cb5f2bb..86a50e8fbc806 100644 --- a/mm/damon/ops-common.c +++ b/mm/damon/ops-common.c @@ -9,6 +9,8 @@ #include <linux/page_idle.h> #include <linux/pagemap.h> #include <linux/rmap.h> +#include <linux/swap.h> +#include <linux/swapops.h> #include "ops-common.h" @@ -39,12 +41,29 @@ struct folio *damon_get_folio(unsigned long pfn) void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *vma, unsigned long addr) { - struct folio *folio = damon_get_folio(pte_pfn(ptep_get(pte))); + pte_t pteval = ptep_get(pte); + struct folio *folio; + bool young = false; + unsigned long pfn; + + if (likely(pte_present(pteval))) + pfn = pte_pfn(pteval); + else + pfn = swp_offset_pfn(pte_to_swp_entry(pteval)); + folio = damon_get_folio(pfn); if (!folio) return; - if (ptep_clear_young_notify(vma, addr, pte)) + /* + * PFN swap PTEs, such as device-exclusive ones, that actually map pages + * are "old" from a CPU perspective. The MMU notifier takes care of any + * device aspects. + */ + if (likely(pte_present(pteval))) + young |= ptep_test_and_clear_young(vma, addr, pte); + young |= mmu_notifier_clear_young(vma->vm_mm, addr, addr + PAGE_SIZE); + if (young) folio_set_young(folio); folio_set_idle(folio);
Ever since commit b756a3b5e7ea ("mm: device exclusive memory access") we can return with a device-exclusive entry from page_vma_mapped_walk(). damon_folio_mkold_one() is not prepared for that and calls damon_ptep_mkold() with PFN swap PTEs. Teach damon_ptep_mkold() to deal with these PFN swap PTEs. Note that device-private entries are so far not applicable on that path, as damon_get_folio() filters out non-lru folios. Should we just skip PFN swap PTEs completely? Possible, but it seems straight forward to just handle it correctly. Note that we could currently only run into this case with device-exclusive entries on THPs. We still adjust the mapcount on conversion to device-exclusive; this makes the rmap walk abort early for small folios, because we'll always have !folio_mapped() with a single device-exclusive entry. We'll adjust the mapcount logic once all page_vma_mapped_walk() users can properly handle device-exclusive entries. Signed-off-by: David Hildenbrand <david@redhat.com> --- mm/damon/ops-common.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-)