| Message ID | 20230618065824.1365750-1-yosryahmed@google.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | mm/mlock: new mlock_count tracking scheme | expand |
Hi, Yosry, Yosry Ahmed <yosryahmed@google.com> writes: > This reverts commit c3096e6782b733158bf34f6bbb4567808d4e0740. > > That commit made sure we immediately add the new page to the LRU before > remove_migration_ptes() is called in migrate_move_folio() (used to be > __unmap_and_move() back then), such that the rmap walk will rebuild the > correct mlock_count for the page again. This was needed because the > mlock_count was lost when the page is isolated. This is no longer the > case since mlock_count no longer overlays page->lru. > > Revert the commit (the code was foliated afterward the commit, so the > revert is updated as such). > > Signed-off-by: Yosry Ahmed <yosryahmed@google.com> > --- > mm/migrate.c | 24 +++++++++--------------- > 1 file changed, 9 insertions(+), 15 deletions(-) > > diff --git a/mm/migrate.c b/mm/migrate.c > index 01cac26a3127..68f693731865 100644 > --- a/mm/migrate.c > +++ b/mm/migrate.c > @@ -1279,19 +1279,6 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, > if (unlikely(!is_lru)) > goto out_unlock_both; The patch itself looks good to me! Thanks! Reviewed-by: "Huang, Ying" <ying.huang@intel.com> And, it seems that we can remove the above 2 lines and "out_unlock_both" label now. That can make the code simpler a little. Right? Best Regards, Huang, Ying > - /* > - * When successful, push dst to LRU immediately: so that if it > - * turns out to be an mlocked page, remove_migration_ptes() will > - * automatically build up the correct dst->mlock_count for it. > - * > - * We would like to do something similar for the old page, when > - * unsuccessful, and other cases when a page has been temporarily > - * isolated from the unevictable LRU: but this case is the easiest. > - */ > - folio_add_lru(dst); > - if (page_was_mapped) > - lru_add_drain(); > - > if (page_was_mapped) > remove_migration_ptes(src, dst, false); > > @@ -1301,9 +1288,16 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, > /* > * If migration is successful, decrease refcount of dst, > * which will not free the page because new page owner increased > - * refcounter. > + * refcounter. As well, if it is LRU folio, add the folio to LRU > + * list in here. Use the old state of the isolated source folio to > + * determine if we migrated a LRU folio. dst was already unlocked > + * and possibly modified by its owner - don't rely on the folio > + * state. > */ > - folio_put(dst); > + if (unlikely(!is_lru)) > + folio_put(dst); > + else > + folio_putback_lru(dst); > > /* > * A folio that has been migrated has all references removed
On Sun, Jun 18, 2023 at 7:00 PM Huang, Ying <ying.huang@intel.com> wrote: > > Hi, Yosry, > > Yosry Ahmed <yosryahmed@google.com> writes: > > > This reverts commit c3096e6782b733158bf34f6bbb4567808d4e0740. > > > > That commit made sure we immediately add the new page to the LRU before > > remove_migration_ptes() is called in migrate_move_folio() (used to be > > __unmap_and_move() back then), such that the rmap walk will rebuild the > > correct mlock_count for the page again. This was needed because the > > mlock_count was lost when the page is isolated. This is no longer the > > case since mlock_count no longer overlays page->lru. > > > > Revert the commit (the code was foliated afterward the commit, so the > > revert is updated as such). > > > > Signed-off-by: Yosry Ahmed <yosryahmed@google.com> > > --- > > mm/migrate.c | 24 +++++++++--------------- > > 1 file changed, 9 insertions(+), 15 deletions(-) > > > > diff --git a/mm/migrate.c b/mm/migrate.c > > index 01cac26a3127..68f693731865 100644 > > --- a/mm/migrate.c > > +++ b/mm/migrate.c > > @@ -1279,19 +1279,6 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, > > if (unlikely(!is_lru)) > > goto out_unlock_both; > > The patch itself looks good to me! Thanks! > > Reviewed-by: "Huang, Ying" <ying.huang@intel.com> Thanks for taking a look! > > And, it seems that we can remove the above 2 lines and "out_unlock_both" > label now. That can make the code simpler a little. Right? I am not familiar with this code. If we remove the above condition then pages that have is_lru == 0 (i.e __PageMovable(src) is true) and page_was_mapped == 1 will call remove_migration_ptes(). This wouldn't happen without removing the above 2 lines. If this combination is impossible (is_lru == 0 && page_was_mapped == 1), then yeah we can remove the above condition. It looks like __SetPageMovable() is only called by zsmalloc, z3fold, and balloon_page_insert(). The former 2 will never have those pages mapped into userspace. I am not familiar with balloon_page_insert(), but my gut feeling is that those are pages used by the driver and are also not mapped into userspace. So I guess we can just remove the condition, but a confirmation for the above would be reassuring :) > > Best Regards, > Huang, Ying > > > - /* > > - * When successful, push dst to LRU immediately: so that if it > > - * turns out to be an mlocked page, remove_migration_ptes() will > > - * automatically build up the correct dst->mlock_count for it. > > - * > > - * We would like to do something similar for the old page, when > > - * unsuccessful, and other cases when a page has been temporarily > > - * isolated from the unevictable LRU: but this case is the easiest. > > - */ > > - folio_add_lru(dst); > > - if (page_was_mapped) > > - lru_add_drain(); > > - > > if (page_was_mapped) > > remove_migration_ptes(src, dst, false); > > > > @@ -1301,9 +1288,16 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, > > /* > > * If migration is successful, decrease refcount of dst, > > * which will not free the page because new page owner increased > > - * refcounter. > > + * refcounter. As well, if it is LRU folio, add the folio to LRU > > + * list in here. Use the old state of the isolated source folio to > > + * determine if we migrated a LRU folio. dst was already unlocked > > + * and possibly modified by its owner - don't rely on the folio > > + * state. > > */ > > - folio_put(dst); > > + if (unlikely(!is_lru)) > > + folio_put(dst); > > + else > > + folio_putback_lru(dst); > > > > /* > > * A folio that has been migrated has all references removed >
Yosry Ahmed <yosryahmed@google.com> writes: > On Sun, Jun 18, 2023 at 7:00 PM Huang, Ying <ying.huang@intel.com> wrote: >> >> Hi, Yosry, >> >> Yosry Ahmed <yosryahmed@google.com> writes: >> >> > This reverts commit c3096e6782b733158bf34f6bbb4567808d4e0740. >> > >> > That commit made sure we immediately add the new page to the LRU before >> > remove_migration_ptes() is called in migrate_move_folio() (used to be >> > __unmap_and_move() back then), such that the rmap walk will rebuild the >> > correct mlock_count for the page again. This was needed because the >> > mlock_count was lost when the page is isolated. This is no longer the >> > case since mlock_count no longer overlays page->lru. >> > >> > Revert the commit (the code was foliated afterward the commit, so the >> > revert is updated as such). >> > >> > Signed-off-by: Yosry Ahmed <yosryahmed@google.com> >> > --- >> > mm/migrate.c | 24 +++++++++--------------- >> > 1 file changed, 9 insertions(+), 15 deletions(-) >> > >> > diff --git a/mm/migrate.c b/mm/migrate.c >> > index 01cac26a3127..68f693731865 100644 >> > --- a/mm/migrate.c >> > +++ b/mm/migrate.c >> > @@ -1279,19 +1279,6 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, >> > if (unlikely(!is_lru)) >> > goto out_unlock_both; >> >> The patch itself looks good to me! Thanks! >> >> Reviewed-by: "Huang, Ying" <ying.huang@intel.com> > > Thanks for taking a look! > >> >> And, it seems that we can remove the above 2 lines and "out_unlock_both" >> label now. That can make the code simpler a little. Right? > > I am not familiar with this code. If we remove the above condition > then pages that have is_lru == 0 (i.e __PageMovable(src) is true) and > page_was_mapped == 1 will call remove_migration_ptes(). This wouldn't > happen without removing the above 2 lines. If this combination is > impossible (is_lru == 0 && page_was_mapped == 1), then yeah we can > remove the above condition. > > It looks like __SetPageMovable() is only called by zsmalloc, z3fold, > and balloon_page_insert(). The former 2 will never have those pages > mapped into userspace. I am not familiar with balloon_page_insert(), > but my gut feeling is that those are pages used by the driver and are > also not mapped into userspace. You can take a look at migrate_folio_unmap(), where "page_was_mapped" will not be set to 1 if !is_lru. Best Regards, Huang, Ying > So I guess we can just remove the condition, but a confirmation for > the above would be reassuring :) > >> >> Best Regards, >> Huang, Ying >> >> > - /* >> > - * When successful, push dst to LRU immediately: so that if it >> > - * turns out to be an mlocked page, remove_migration_ptes() will >> > - * automatically build up the correct dst->mlock_count for it. >> > - * >> > - * We would like to do something similar for the old page, when >> > - * unsuccessful, and other cases when a page has been temporarily >> > - * isolated from the unevictable LRU: but this case is the easiest. >> > - */ >> > - folio_add_lru(dst); >> > - if (page_was_mapped) >> > - lru_add_drain(); >> > - >> > if (page_was_mapped) >> > remove_migration_ptes(src, dst, false); >> > >> > @@ -1301,9 +1288,16 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, >> > /* >> > * If migration is successful, decrease refcount of dst, >> > * which will not free the page because new page owner increased >> > - * refcounter. >> > + * refcounter. As well, if it is LRU folio, add the folio to LRU >> > + * list in here. Use the old state of the isolated source folio to >> > + * determine if we migrated a LRU folio. dst was already unlocked >> > + * and possibly modified by its owner - don't rely on the folio >> > + * state. >> > */ >> > - folio_put(dst); >> > + if (unlikely(!is_lru)) >> > + folio_put(dst); >> > + else >> > + folio_putback_lru(dst); >> > >> > /* >> > * A folio that has been migrated has all references removed >>
On Sun, Jun 18, 2023 at 9:29 PM Huang, Ying <ying.huang@intel.com> wrote: > > Yosry Ahmed <yosryahmed@google.com> writes: > > > On Sun, Jun 18, 2023 at 7:00 PM Huang, Ying <ying.huang@intel.com> wrote: > >> > >> Hi, Yosry, > >> > >> Yosry Ahmed <yosryahmed@google.com> writes: > >> > >> > This reverts commit c3096e6782b733158bf34f6bbb4567808d4e0740. > >> > > >> > That commit made sure we immediately add the new page to the LRU before > >> > remove_migration_ptes() is called in migrate_move_folio() (used to be > >> > __unmap_and_move() back then), such that the rmap walk will rebuild the > >> > correct mlock_count for the page again. This was needed because the > >> > mlock_count was lost when the page is isolated. This is no longer the > >> > case since mlock_count no longer overlays page->lru. > >> > > >> > Revert the commit (the code was foliated afterward the commit, so the > >> > revert is updated as such). > >> > > >> > Signed-off-by: Yosry Ahmed <yosryahmed@google.com> > >> > --- > >> > mm/migrate.c | 24 +++++++++--------------- > >> > 1 file changed, 9 insertions(+), 15 deletions(-) > >> > > >> > diff --git a/mm/migrate.c b/mm/migrate.c > >> > index 01cac26a3127..68f693731865 100644 > >> > --- a/mm/migrate.c > >> > +++ b/mm/migrate.c > >> > @@ -1279,19 +1279,6 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, > >> > if (unlikely(!is_lru)) > >> > goto out_unlock_both; > >> > >> The patch itself looks good to me! Thanks! > >> > >> Reviewed-by: "Huang, Ying" <ying.huang@intel.com> > > > > Thanks for taking a look! > > > >> > >> And, it seems that we can remove the above 2 lines and "out_unlock_both" > >> label now. That can make the code simpler a little. Right? > > > > I am not familiar with this code. If we remove the above condition > > then pages that have is_lru == 0 (i.e __PageMovable(src) is true) and > > page_was_mapped == 1 will call remove_migration_ptes(). This wouldn't > > happen without removing the above 2 lines. If this combination is > > impossible (is_lru == 0 && page_was_mapped == 1), then yeah we can > > remove the above condition. > > > > It looks like __SetPageMovable() is only called by zsmalloc, z3fold, > > and balloon_page_insert(). The former 2 will never have those pages > > mapped into userspace. I am not familiar with balloon_page_insert(), > > but my gut feeling is that those are pages used by the driver and are > > also not mapped into userspace. > > You can take a look at migrate_folio_unmap(), where "page_was_mapped" > will not be set to 1 if !is_lru. Oh I was looking in the wrong place huh, thanks. Will remove it when I respin! > > Best Regards, > Huang, Ying > > > So I guess we can just remove the condition, but a confirmation for > > the above would be reassuring :) > > > >> > >> Best Regards, > >> Huang, Ying > >> > >> > - /* > >> > - * When successful, push dst to LRU immediately: so that if it > >> > - * turns out to be an mlocked page, remove_migration_ptes() will > >> > - * automatically build up the correct dst->mlock_count for it. > >> > - * > >> > - * We would like to do something similar for the old page, when > >> > - * unsuccessful, and other cases when a page has been temporarily > >> > - * isolated from the unevictable LRU: but this case is the easiest. > >> > - */ > >> > - folio_add_lru(dst); > >> > - if (page_was_mapped) > >> > - lru_add_drain(); > >> > - > >> > if (page_was_mapped) > >> > remove_migration_ptes(src, dst, false); > >> > > >> > @@ -1301,9 +1288,16 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, > >> > /* > >> > * If migration is successful, decrease refcount of dst, > >> > * which will not free the page because new page owner increased > >> > - * refcounter. > >> > + * refcounter. As well, if it is LRU folio, add the folio to LRU > >> > + * list in here. Use the old state of the isolated source folio to > >> > + * determine if we migrated a LRU folio. dst was already unlocked > >> > + * and possibly modified by its owner - don't rely on the folio > >> > + * state. > >> > */ > >> > - folio_put(dst); > >> > + if (unlikely(!is_lru)) > >> > + folio_put(dst); > >> > + else > >> > + folio_putback_lru(dst); > >> > > >> > /* > >> > * A folio that has been migrated has all references removed > >> >
On 19.06.23 05:59, Yosry Ahmed wrote: > On Sun, Jun 18, 2023 at 7:00 PM Huang, Ying <ying.huang@intel.com> wrote: >> >> Hi, Yosry, >> >> Yosry Ahmed <yosryahmed@google.com> writes: >> >>> This reverts commit c3096e6782b733158bf34f6bbb4567808d4e0740. >>> >>> That commit made sure we immediately add the new page to the LRU before >>> remove_migration_ptes() is called in migrate_move_folio() (used to be >>> __unmap_and_move() back then), such that the rmap walk will rebuild the >>> correct mlock_count for the page again. This was needed because the >>> mlock_count was lost when the page is isolated. This is no longer the >>> case since mlock_count no longer overlays page->lru. >>> >>> Revert the commit (the code was foliated afterward the commit, so the >>> revert is updated as such). >>> >>> Signed-off-by: Yosry Ahmed <yosryahmed@google.com> >>> --- >>> mm/migrate.c | 24 +++++++++--------------- >>> 1 file changed, 9 insertions(+), 15 deletions(-) >>> >>> diff --git a/mm/migrate.c b/mm/migrate.c >>> index 01cac26a3127..68f693731865 100644 >>> --- a/mm/migrate.c >>> +++ b/mm/migrate.c >>> @@ -1279,19 +1279,6 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, >>> if (unlikely(!is_lru)) >>> goto out_unlock_both; >> >> The patch itself looks good to me! Thanks! >> >> Reviewed-by: "Huang, Ying" <ying.huang@intel.com> > > Thanks for taking a look! > >> >> And, it seems that we can remove the above 2 lines and "out_unlock_both" >> label now. That can make the code simpler a little. Right? > > I am not familiar with this code. If we remove the above condition > then pages that have is_lru == 0 (i.e __PageMovable(src) is true) and > page_was_mapped == 1 will call remove_migration_ptes(). This wouldn't > happen without removing the above 2 lines. If this combination is > impossible (is_lru == 0 && page_was_mapped == 1), then yeah we can > remove the above condition. > > It looks like __SetPageMovable() is only called by zsmalloc, z3fold, > and balloon_page_insert(). The former 2 will never have those pages > mapped into userspace. I am not familiar with balloon_page_insert(), > but my gut feeling is that those are pages used by the driver and are > also not mapped into userspace. On XEN, there is xen_alloc_ballooned_pages(), which ends up mapping balloon-inflated pages into user space (for something like MMIO IIRC). But the XEN balloon does not use the balloon compaction framework, so __SetPageMovable() does not apply. The other balloon_page_insert() users (VMware balloon, CMM, virtio-balloon) shouldn't be doing something like that.
On 19.06.23 09:56, David Hildenbrand wrote: > On 19.06.23 05:59, Yosry Ahmed wrote: >> On Sun, Jun 18, 2023 at 7:00 PM Huang, Ying <ying.huang@intel.com> wrote: >>> >>> Hi, Yosry, >>> >>> Yosry Ahmed <yosryahmed@google.com> writes: >>> >>>> This reverts commit c3096e6782b733158bf34f6bbb4567808d4e0740. >>>> >>>> That commit made sure we immediately add the new page to the LRU before >>>> remove_migration_ptes() is called in migrate_move_folio() (used to be >>>> __unmap_and_move() back then), such that the rmap walk will rebuild the >>>> correct mlock_count for the page again. This was needed because the >>>> mlock_count was lost when the page is isolated. This is no longer the >>>> case since mlock_count no longer overlays page->lru. >>>> >>>> Revert the commit (the code was foliated afterward the commit, so the >>>> revert is updated as such). >>>> >>>> Signed-off-by: Yosry Ahmed <yosryahmed@google.com> >>>> --- >>>> mm/migrate.c | 24 +++++++++--------------- >>>> 1 file changed, 9 insertions(+), 15 deletions(-) >>>> >>>> diff --git a/mm/migrate.c b/mm/migrate.c >>>> index 01cac26a3127..68f693731865 100644 >>>> --- a/mm/migrate.c >>>> +++ b/mm/migrate.c >>>> @@ -1279,19 +1279,6 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, >>>> if (unlikely(!is_lru)) >>>> goto out_unlock_both; >>> >>> The patch itself looks good to me! Thanks! >>> >>> Reviewed-by: "Huang, Ying" <ying.huang@intel.com> >> >> Thanks for taking a look! >> >>> >>> And, it seems that we can remove the above 2 lines and "out_unlock_both" >>> label now. That can make the code simpler a little. Right? >> >> I am not familiar with this code. If we remove the above condition >> then pages that have is_lru == 0 (i.e __PageMovable(src) is true) and >> page_was_mapped == 1 will call remove_migration_ptes(). This wouldn't >> happen without removing the above 2 lines. If this combination is >> impossible (is_lru == 0 && page_was_mapped == 1), then yeah we can >> remove the above condition. >> >> It looks like __SetPageMovable() is only called by zsmalloc, z3fold, >> and balloon_page_insert(). The former 2 will never have those pages >> mapped into userspace. I am not familiar with balloon_page_insert(), >> but my gut feeling is that those are pages used by the driver and are >> also not mapped into userspace. > > On XEN, there is xen_alloc_ballooned_pages(), which ends up mapping > balloon-inflated pages into user space (for something like MMIO IIRC). > But the XEN balloon does not use the balloon compaction framework, so > __SetPageMovable() does not apply. > > The other balloon_page_insert() users (VMware balloon, CMM, > virtio-balloon) shouldn't be doing something like that. Ah, and I remember they even can't, because in balloon_page_insert() we also do a __SetPageOffline(). And such typed pages cannot be mapped into user space (because the type overlays the mapcount).
On Mon, Jun 19, 2023 at 12:58 AM David Hildenbrand <david@redhat.com> wrote: > > On 19.06.23 09:56, David Hildenbrand wrote: > > On 19.06.23 05:59, Yosry Ahmed wrote: > >> On Sun, Jun 18, 2023 at 7:00 PM Huang, Ying <ying.huang@intel.com> wrote: > >>> > >>> Hi, Yosry, > >>> > >>> Yosry Ahmed <yosryahmed@google.com> writes: > >>> > >>>> This reverts commit c3096e6782b733158bf34f6bbb4567808d4e0740. > >>>> > >>>> That commit made sure we immediately add the new page to the LRU before > >>>> remove_migration_ptes() is called in migrate_move_folio() (used to be > >>>> __unmap_and_move() back then), such that the rmap walk will rebuild the > >>>> correct mlock_count for the page again. This was needed because the > >>>> mlock_count was lost when the page is isolated. This is no longer the > >>>> case since mlock_count no longer overlays page->lru. > >>>> > >>>> Revert the commit (the code was foliated afterward the commit, so the > >>>> revert is updated as such). > >>>> > >>>> Signed-off-by: Yosry Ahmed <yosryahmed@google.com> > >>>> --- > >>>> mm/migrate.c | 24 +++++++++--------------- > >>>> 1 file changed, 9 insertions(+), 15 deletions(-) > >>>> > >>>> diff --git a/mm/migrate.c b/mm/migrate.c > >>>> index 01cac26a3127..68f693731865 100644 > >>>> --- a/mm/migrate.c > >>>> +++ b/mm/migrate.c > >>>> @@ -1279,19 +1279,6 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, > >>>> if (unlikely(!is_lru)) > >>>> goto out_unlock_both; > >>> > >>> The patch itself looks good to me! Thanks! > >>> > >>> Reviewed-by: "Huang, Ying" <ying.huang@intel.com> > >> > >> Thanks for taking a look! > >> > >>> > >>> And, it seems that we can remove the above 2 lines and "out_unlock_both" > >>> label now. That can make the code simpler a little. Right? > >> > >> I am not familiar with this code. If we remove the above condition > >> then pages that have is_lru == 0 (i.e __PageMovable(src) is true) and > >> page_was_mapped == 1 will call remove_migration_ptes(). This wouldn't > >> happen without removing the above 2 lines. If this combination is > >> impossible (is_lru == 0 && page_was_mapped == 1), then yeah we can > >> remove the above condition. > >> > >> It looks like __SetPageMovable() is only called by zsmalloc, z3fold, > >> and balloon_page_insert(). The former 2 will never have those pages > >> mapped into userspace. I am not familiar with balloon_page_insert(), > >> but my gut feeling is that those are pages used by the driver and are > >> also not mapped into userspace. > > > > On XEN, there is xen_alloc_ballooned_pages(), which ends up mapping > > balloon-inflated pages into user space (for something like MMIO IIRC). > > But the XEN balloon does not use the balloon compaction framework, so > > __SetPageMovable() does not apply. > > > > The other balloon_page_insert() users (VMware balloon, CMM, > > virtio-balloon) shouldn't be doing something like that. > > Ah, and I remember they even can't, because in balloon_page_insert() we > also do a __SetPageOffline(). And such typed pages cannot be mapped into > user space (because the type overlays the mapcount). Thanks David, good to know! I will remove the condition as Ying suggested in the next version then! > > -- > Cheers, > > David / dhildenb >
diff --git a/mm/migrate.c b/mm/migrate.c index 01cac26a3127..68f693731865 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1279,19 +1279,6 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, if (unlikely(!is_lru)) goto out_unlock_both; - /* - * When successful, push dst to LRU immediately: so that if it - * turns out to be an mlocked page, remove_migration_ptes() will - * automatically build up the correct dst->mlock_count for it. - * - * We would like to do something similar for the old page, when - * unsuccessful, and other cases when a page has been temporarily - * isolated from the unevictable LRU: but this case is the easiest. - */ - folio_add_lru(dst); - if (page_was_mapped) - lru_add_drain(); - if (page_was_mapped) remove_migration_ptes(src, dst, false); @@ -1301,9 +1288,16 @@ static int migrate_folio_move(free_page_t put_new_page, unsigned long private, /* * If migration is successful, decrease refcount of dst, * which will not free the page because new page owner increased - * refcounter. + * refcounter. As well, if it is LRU folio, add the folio to LRU + * list in here. Use the old state of the isolated source folio to + * determine if we migrated a LRU folio. dst was already unlocked + * and possibly modified by its owner - don't rely on the folio + * state. */ - folio_put(dst); + if (unlikely(!is_lru)) + folio_put(dst); + else + folio_putback_lru(dst); /* * A folio that has been migrated has all references removed
This reverts commit c3096e6782b733158bf34f6bbb4567808d4e0740. That commit made sure we immediately add the new page to the LRU before remove_migration_ptes() is called in migrate_move_folio() (used to be __unmap_and_move() back then), such that the rmap walk will rebuild the correct mlock_count for the page again. This was needed because the mlock_count was lost when the page is isolated. This is no longer the case since mlock_count no longer overlays page->lru. Revert the commit (the code was foliated afterward the commit, so the revert is updated as such). Signed-off-by: Yosry Ahmed <yosryahmed@google.com> --- mm/migrate.c | 24 +++++++++--------------- 1 file changed, 9 insertions(+), 15 deletions(-)