| Message ID | 20200907163628.26495-1-vbabka@suse.cz (mailing list archive) |
|---|---|
| Headers | show |
| Series | disable pcplists during page isolation | expand |
On 07.09.20 18:36, Vlastimil Babka wrote: > As per the discussions [1] [2] this is an attempt to implement David's > suggestion that page isolation should disable pcplists to avoid races. This is > done without extra checks in fast paths, as I mentioned should be possible in > the discussion, and explained in patch 5. Patches 1-4 are preparatory cleanups. > > Note this is untested RFC for now. Based on v5.9-rc4 plus Pavel's patch [2] > (slated as a quick fix for mainline+stable). > > [1] https://lore.kernel.org/linux-mm/20200901124615.137200-1-pasha.tatashin@soleen.com/ > [2] https://lore.kernel.org/linux-mm/20200903140032.380431-1-pasha.tatashin@soleen.com/ > > Vlastimil Babka (5): > mm, page_alloc: clean up pageset high and batch update > mm, page_alloc: calculate pageset high and batch once per zone > mm, page_alloc(): remove setup_pageset() > mm, page_alloc: cache pageset high and batch in struct zone > mm, page_alloc: disable pcplists during page isolation > > include/linux/gfp.h | 1 + > include/linux/mmzone.h | 2 + > mm/internal.h | 4 ++ > mm/memory_hotplug.c | 24 +++---- > mm/page_alloc.c | 138 ++++++++++++++++++++++------------------- > mm/page_isolation.c | 45 +++++++++++--- > 6 files changed, 127 insertions(+), 87 deletions(-) > Thanks for looking into this! Just a heads-up that -mm and -next contain some changes to memory hotplug code, whereby new pageblocks start out in MIGRATE_ISOLATE when onlining, until we're done with the heavy lifting. Might require some tweaks, similar to when isolating pageblocks. Will dive into this in the following days. What's you're general perception of performance aspects?
On 9/8/20 8:29 PM, David Hildenbrand wrote: > On 07.09.20 18:36, Vlastimil Babka wrote: >> As per the discussions [1] [2] this is an attempt to implement David's >> suggestion that page isolation should disable pcplists to avoid races. This is >> done without extra checks in fast paths, as I mentioned should be possible in >> the discussion, and explained in patch 5. Patches 1-4 are preparatory cleanups. >> >> Note this is untested RFC for now. Based on v5.9-rc4 plus Pavel's patch [2] >> (slated as a quick fix for mainline+stable). >> >> [1] https://lore.kernel.org/linux-mm/20200901124615.137200-1-pasha.tatashin@soleen.com/ >> [2] https://lore.kernel.org/linux-mm/20200903140032.380431-1-pasha.tatashin@soleen.com/ >> >> Vlastimil Babka (5): >> mm, page_alloc: clean up pageset high and batch update >> mm, page_alloc: calculate pageset high and batch once per zone >> mm, page_alloc(): remove setup_pageset() >> mm, page_alloc: cache pageset high and batch in struct zone >> mm, page_alloc: disable pcplists during page isolation >> >> include/linux/gfp.h | 1 + >> include/linux/mmzone.h | 2 + >> mm/internal.h | 4 ++ >> mm/memory_hotplug.c | 24 +++---- >> mm/page_alloc.c | 138 ++++++++++++++++++++++------------------- >> mm/page_isolation.c | 45 +++++++++++--- >> 6 files changed, 127 insertions(+), 87 deletions(-) >> > > Thanks for looking into this! Just a heads-up that -mm and -next contain > some changes to memory hotplug code, whereby new pageblocks start out in > MIGRATE_ISOLATE when onlining, until we're done with the heavy lifting. > Might require some tweaks, similar to when isolating pageblocks. Thanks for the heads-up. I've posted updated patch 5/5 for the -next as a reply to the first one. It was a bit tricky to order everything correctly in online_pages(), hopefully I avoided any deadlock. > Will dive into this in the following days. What's you're general > perception of performance aspects? Thanks! I expect no performance change while no isolation is in progress, as there are no new tests added in alloc/free paths. During page isolation there's a single drain instead of once-per-pageblock, which is a benefit. But the pcplists are effectively disabled for the whole of online_pages(), offline_pages() or alloc_contig_range(), which will affect parallel page allocator users. It depends on how long these operations take and how heavy the parallel usage is, so I have no good answers. Might be similar to the current periodic drain.
On 2020-09-09 12:54, Vlastimil Babka wrote: > Thanks! I expect no performance change while no isolation is in > progress, as > there are no new tests added in alloc/free paths. During page isolation > there's > a single drain instead of once-per-pageblock, which is a benefit. But > the > pcplists are effectively disabled for the whole of online_pages(), > offline_pages() or alloc_contig_range(), which will affect parallel > page > allocator users. It depends on how long these operations take and how > heavy the > parallel usage is, so I have no good answers. Might be similar to the > current > periodic drain. I have seen some systems taking quite some time when offlining sections due to the migration of the respective pages not being that smooth and having do_migrate_range to do some spins. But to be fair, online_pages and offline_pages are not routines that get called that often, and we would be safe to assume that memory-hotplug operations are not constantly happening, but are rather one-offs operations. I am not sure about Xen and HV, IIRC Xen was using online_pages and offline_pages routines to do the ballooning? I will dive in this in the following days, thanks for the work Vlastimil
On 09.09.20 13:27, osalvador@suse.de wrote: > On 2020-09-09 12:54, Vlastimil Babka wrote: >> Thanks! I expect no performance change while no isolation is in >> progress, as >> there are no new tests added in alloc/free paths. During page isolation >> there's >> a single drain instead of once-per-pageblock, which is a benefit. But >> the >> pcplists are effectively disabled for the whole of online_pages(), >> offline_pages() or alloc_contig_range(), which will affect parallel >> page >> allocator users. It depends on how long these operations take and how >> heavy the >> parallel usage is, so I have no good answers. Might be similar to the >> current >> periodic drain. > > I have seen some systems taking quite some time when offlining sections > due to the migration of > the respective pages not being that smooth and having do_migrate_range > to do some spins. > But to be fair, online_pages and offline_pages are not routines that get > called that often, and we would be safe to assume that memory-hotplug > operations are not constantly happening, but are rather one-offs > operations. > > I am not sure about Xen and HV, IIRC Xen was using online_pages and > offline_pages routines to do the ballooning? No, they only add more memory blocks and online them. Memory hot(un)plug is an expensive operation already, I don't think that's an issue.