Message ID | 1557822213-19058-1-git-send-email-rppt@linux.ibm.com (mailing list archive) |
---|---|
State | New, archived |
Headers | show |
Series | docs: reorder memory-hotplug documentation | expand |
On 14.05.19 10:23, Mike Rapoport wrote: > The "Locking Internals" section of the memory-hotplug documentation is > duplicated in admin-guide and core-api. Drop the admin-guide copy as > locking internals does not belong there. > > While on it, move the "Future Work" section to the core-api part. Looks sane, but the future work part is really outdated, can we remove this completely? > > Signed-off-by: Mike Rapoport <rppt@linux.ibm.com> > --- > Documentation/admin-guide/mm/memory-hotplug.rst | 51 ------------------------- > Documentation/core-api/memory-hotplug.rst | 11 ++++++ > 2 files changed, 11 insertions(+), 51 deletions(-) > > diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst b/Documentation/admin-guide/mm/memory-hotplug.rst > index 5c4432c..72090ba 100644 > --- a/Documentation/admin-guide/mm/memory-hotplug.rst > +++ b/Documentation/admin-guide/mm/memory-hotplug.rst > @@ -391,54 +391,3 @@ Physical memory remove > Need more implementation yet.... > - Notification completion of remove works by OS to firmware. > - Guard from remove if not yet. > - > - > -Locking Internals > -================= > - > -When adding/removing memory that uses memory block devices (i.e. ordinary RAM), > -the device_hotplug_lock should be held to: > - > -- synchronize against online/offline requests (e.g. via sysfs). This way, memory > - block devices can only be accessed (.online/.state attributes) by user > - space once memory has been fully added. And when removing memory, we > - know nobody is in critical sections. > -- synchronize against CPU hotplug and similar (e.g. relevant for ACPI and PPC) > - > -Especially, there is a possible lock inversion that is avoided using > -device_hotplug_lock when adding memory and user space tries to online that > -memory faster than expected: > - > -- device_online() will first take the device_lock(), followed by > - mem_hotplug_lock > -- add_memory_resource() will first take the mem_hotplug_lock, followed by > - the device_lock() (while creating the devices, during bus_add_device()). > - > -As the device is visible to user space before taking the device_lock(), this > -can result in a lock inversion. > - > -onlining/offlining of memory should be done via device_online()/ > -device_offline() - to make sure it is properly synchronized to actions > -via sysfs. Holding device_hotplug_lock is advised (to e.g. protect online_type) > - > -When adding/removing/onlining/offlining memory or adding/removing > -heterogeneous/device memory, we should always hold the mem_hotplug_lock in > -write mode to serialise memory hotplug (e.g. access to global/zone > -variables). > - > -In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read > -mode allows for a quite efficient get_online_mems/put_online_mems > -implementation, so code accessing memory can protect from that memory > -vanishing. > - > - > -Future Work > -=========== > - > - - allowing memory hot-add to ZONE_MOVABLE. maybe we need some switch like > - sysctl or new control file. > - - showing memory block and physical device relationship. > - - test and make it better memory offlining. > - - support HugeTLB page migration and offlining. > - - memmap removing at memory offline. > - - physical remove memory. > diff --git a/Documentation/core-api/memory-hotplug.rst b/Documentation/core-api/memory-hotplug.rst > index de7467e..e08be1c 100644 > --- a/Documentation/core-api/memory-hotplug.rst > +++ b/Documentation/core-api/memory-hotplug.rst > @@ -123,3 +123,14 @@ In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read > mode allows for a quite efficient get_online_mems/put_online_mems > implementation, so code accessing memory can protect from that memory > vanishing. > + > +Future Work > +=========== > + > + - allowing memory hot-add to ZONE_MOVABLE. maybe we need some switch like > + sysctl or new control file. ... that already works if I am not completely missing the point here > + - showing memory block and physical device relationship. ... that is available for s390x only AFAIK > + - test and make it better memory offlining. ... no big news ;) > + - support HugeTLB page migration and offlining. ... I remember that Oscar was doing something in that area, Oscar? > + - memmap removing at memory offline. ... no, we don't want this. However, we should properly clean up zone information when offlining > + - physical remove memory. ... I don't even understand what that means. I'd vote for removing the future work part, this is pretty outdated.
On Tue, May 21, 2019 at 12:41:50PM +0200, David Hildenbrand wrote: > On 14.05.19 10:23, Mike Rapoport wrote: > > The "Locking Internals" section of the memory-hotplug documentation is > > duplicated in admin-guide and core-api. Drop the admin-guide copy as > > locking internals does not belong there. > > > > While on it, move the "Future Work" section to the core-api part. > > Looks sane, but the future work part is really outdated, can we remove > this completely? > > > > > Signed-off-by: Mike Rapoport <rppt@linux.ibm.com> > > --- > > + > > +Future Work > > +=========== > > + > > + - allowing memory hot-add to ZONE_MOVABLE. maybe we need some switch like > > + sysctl or new control file. > > ... that already works if I am not completely missing the point here > > > + - showing memory block and physical device relationship. > > ... that is available for s390x only AFAIK > > > + - test and make it better memory offlining. > > ... no big news ;) > > > + - support HugeTLB page migration and offlining. > > ... I remember that Oscar was doing something in that area, Oscar? > > > + - memmap removing at memory offline. > > ... no, we don't want this. However, we should properly clean up zone > information when offlining > > > + - physical remove memory. > > ... I don't even understand what that means. > > > I'd vote for removing the future work part, this is pretty outdated. Frankly, I haven't looked at the details, just simply moved the text over. I don't mind sending another mechanical patch that removes the future work part. But it would be far better if somebody who's actively working on memory hotplug would replace it with a description how this actually works ;-) > -- > > Thanks, > > David / dhildenb >
On Tue, May 21, 2019 at 12:41:50PM +0200, David Hildenbrand wrote: > > +Future Work > > +=========== > > + > > + - allowing memory hot-add to ZONE_MOVABLE. maybe we need some switch like > > + sysctl or new control file. > > ... that already works if I am not completely missing the point here It does. > > + - support HugeTLB page migration and offlining. > > ... I remember that Oscar was doing something in that area, Oscar? Yes, in general offlinining on hugetlb pages was already working, but we did not allow to offline 1GB-hugetlb pages on x86_64. I removed that limitation with ("commit: 10eeadf3045c mm,memory_hotplug: unlock 1GB-hugetlb on x86_64") , so now offlining on hugetlb pages should be fully operative. > I'd vote for removing the future work part, this is pretty outdated. Instead of removing it, I would rather make it consistent with the present. E.g: - Move page handling from memory-hotremove to offline stage - Enable a way to allocate vmemmap pages from hot-added memory etc.
On 21.05.19 18:11, Oscar Salvador wrote: > On Tue, May 21, 2019 at 12:41:50PM +0200, David Hildenbrand wrote: >>> +Future Work >>> +=========== >>> + >>> + - allowing memory hot-add to ZONE_MOVABLE. maybe we need some switch like >>> + sysctl or new control file. >> >> ... that already works if I am not completely missing the point here > > It does. > >>> + - support HugeTLB page migration and offlining. >> >> ... I remember that Oscar was doing something in that area, Oscar? > > Yes, in general offlinining on hugetlb pages was already working, but we did not > allow to offline 1GB-hugetlb pages on x86_64. > I removed that limitation with > ("commit: 10eeadf3045c mm,memory_hotplug: unlock 1GB-hugetlb on x86_64") , so now > offlining on hugetlb pages should be fully operative. > >> I'd vote for removing the future work part, this is pretty outdated. > > Instead of removing it, I would rather make it consistent with the present. > E.g: > > - Move page handling from memory-hotremove to offline stage > - Enable a way to allocate vmemmap pages from hot-added memory > etc. > > Fair enough, but at least the current content is absolutely useless and confusing/misleading.
diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst b/Documentation/admin-guide/mm/memory-hotplug.rst index 5c4432c..72090ba 100644 --- a/Documentation/admin-guide/mm/memory-hotplug.rst +++ b/Documentation/admin-guide/mm/memory-hotplug.rst @@ -391,54 +391,3 @@ Physical memory remove Need more implementation yet.... - Notification completion of remove works by OS to firmware. - Guard from remove if not yet. - - -Locking Internals -================= - -When adding/removing memory that uses memory block devices (i.e. ordinary RAM), -the device_hotplug_lock should be held to: - -- synchronize against online/offline requests (e.g. via sysfs). This way, memory - block devices can only be accessed (.online/.state attributes) by user - space once memory has been fully added. And when removing memory, we - know nobody is in critical sections. -- synchronize against CPU hotplug and similar (e.g. relevant for ACPI and PPC) - -Especially, there is a possible lock inversion that is avoided using -device_hotplug_lock when adding memory and user space tries to online that -memory faster than expected: - -- device_online() will first take the device_lock(), followed by - mem_hotplug_lock -- add_memory_resource() will first take the mem_hotplug_lock, followed by - the device_lock() (while creating the devices, during bus_add_device()). - -As the device is visible to user space before taking the device_lock(), this -can result in a lock inversion. - -onlining/offlining of memory should be done via device_online()/ -device_offline() - to make sure it is properly synchronized to actions -via sysfs. Holding device_hotplug_lock is advised (to e.g. protect online_type) - -When adding/removing/onlining/offlining memory or adding/removing -heterogeneous/device memory, we should always hold the mem_hotplug_lock in -write mode to serialise memory hotplug (e.g. access to global/zone -variables). - -In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read -mode allows for a quite efficient get_online_mems/put_online_mems -implementation, so code accessing memory can protect from that memory -vanishing. - - -Future Work -=========== - - - allowing memory hot-add to ZONE_MOVABLE. maybe we need some switch like - sysctl or new control file. - - showing memory block and physical device relationship. - - test and make it better memory offlining. - - support HugeTLB page migration and offlining. - - memmap removing at memory offline. - - physical remove memory. diff --git a/Documentation/core-api/memory-hotplug.rst b/Documentation/core-api/memory-hotplug.rst index de7467e..e08be1c 100644 --- a/Documentation/core-api/memory-hotplug.rst +++ b/Documentation/core-api/memory-hotplug.rst @@ -123,3 +123,14 @@ In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read mode allows for a quite efficient get_online_mems/put_online_mems implementation, so code accessing memory can protect from that memory vanishing. + +Future Work +=========== + + - allowing memory hot-add to ZONE_MOVABLE. maybe we need some switch like + sysctl or new control file. + - showing memory block and physical device relationship. + - test and make it better memory offlining. + - support HugeTLB page migration and offlining. + - memmap removing at memory offline. + - physical remove memory.
The "Locking Internals" section of the memory-hotplug documentation is duplicated in admin-guide and core-api. Drop the admin-guide copy as locking internals does not belong there. While on it, move the "Future Work" section to the core-api part. Signed-off-by: Mike Rapoport <rppt@linux.ibm.com> --- Documentation/admin-guide/mm/memory-hotplug.rst | 51 ------------------------- Documentation/core-api/memory-hotplug.rst | 11 ++++++ 2 files changed, 11 insertions(+), 51 deletions(-)