| Message ID | 20240617170545.3820912-4-jiaqiyan@google.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | Userspace controls soft-offline pages | expand |
On Mon, Jun 17, 2024 at 05:05:45PM +0000, Jiaqi Yan wrote: > Add the documentation for soft offline behaviors / costs, and what > the new enable_soft_offline sysctl is for. > > Signed-off-by: Jiaqi Yan <jiaqiyan@google.com> > --- > Documentation/admin-guide/sysctl/vm.rst | 33 +++++++++++++++++++++++++ > 1 file changed, 33 insertions(+) > > diff --git a/Documentation/admin-guide/sysctl/vm.rst b/Documentation/admin-guide/sysctl/vm.rst > index e86c968a7a0e..fc62fc272fc5 100644 > --- a/Documentation/admin-guide/sysctl/vm.rst > +++ b/Documentation/admin-guide/sysctl/vm.rst > @@ -36,6 +36,7 @@ Currently, these files are in /proc/sys/vm: > - dirtytime_expire_seconds > - dirty_writeback_centisecs > - drop_caches > +- enable_soft_offline > - extfrag_threshold > - highmem_is_dirtyable > - hugetlb_shm_group > @@ -267,6 +268,38 @@ used:: > These are informational only. They do not mean that anything is wrong > with your system. To disable them, echo 4 (bit 2) into drop_caches. > > +enable_soft_offline > +=================== > +Correctable memory errors are very common on servers. Soft-offline is kernel's > +solution for memory pages having (excessive) corrected memory errors. > + > +For different types of page, soft-offline has different behaviors / costs. > +- For a raw error page, soft-offline migrates the in-use page's content to > + a new raw page. > +- For a page that is part of a transparent hugepage, soft-offline splits the > + transparent hugepage into raw pages, then migrates only the raw error page. > + As a result, user is transparently backed by 1 less hugepage, impacting > + memory access performance. > +- For a page that is part of a HugeTLB hugepage, soft-offline first migrates > + the entire HugeTLB hugepage, during which a free hugepage will be consumed > + as migration target. Then the original hugepage is dissolved into raw > + pages without compensation, reducing the capacity of the HugeTLB pool by 1. > + > +It is user's call to choose between reliability (staying away from fragile > +physical memory) vs performance / capacity implications in transparent and > +HugeTLB cases. > + > +For all architectures, enable_soft_offline controls whether to soft offline > +memory pages. When setting to 1, kernel attempts to soft offline the pages > +whenever it thinks needed. When setting to 0, kernel returns EOPNOTSUPP to > +the request to soft offline the pages. Its default value is 1. > + > +It is worth mentioning that after setting enable_soft_offline to 0: > +- If RAS Correctable Errors Collector is running, its request to soft offline > + pages will fail. > +- On ARM, the request to soft offline pages from GHES driver will fail. > +- On PARISC, the request to soft offline pages from Page Deallocation Table > + will fail. I do not know about others but the 'fail' word feels wrong here. I would reword that as "... the request to soft offline pages from xxxx will not be performed". Other than that: Acked-by: Oscar Salvador <osalvador@suse.de>
On Tue, Jun 18, 2024 at 10:20 PM Oscar Salvador <osalvador@suse.de> wrote: > > On Mon, Jun 17, 2024 at 05:05:45PM +0000, Jiaqi Yan wrote: > > Add the documentation for soft offline behaviors / costs, and what > > the new enable_soft_offline sysctl is for. > > > > Signed-off-by: Jiaqi Yan <jiaqiyan@google.com> > > --- > > Documentation/admin-guide/sysctl/vm.rst | 33 +++++++++++++++++++++++++ > > 1 file changed, 33 insertions(+) > > > > diff --git a/Documentation/admin-guide/sysctl/vm.rst b/Documentation/admin-guide/sysctl/vm.rst > > index e86c968a7a0e..fc62fc272fc5 100644 > > --- a/Documentation/admin-guide/sysctl/vm.rst > > +++ b/Documentation/admin-guide/sysctl/vm.rst > > @@ -36,6 +36,7 @@ Currently, these files are in /proc/sys/vm: > > - dirtytime_expire_seconds > > - dirty_writeback_centisecs > > - drop_caches > > +- enable_soft_offline > > - extfrag_threshold > > - highmem_is_dirtyable > > - hugetlb_shm_group > > @@ -267,6 +268,38 @@ used:: > > These are informational only. They do not mean that anything is wrong > > with your system. To disable them, echo 4 (bit 2) into drop_caches. > > > > +enable_soft_offline > > +=================== > > +Correctable memory errors are very common on servers. Soft-offline is kernel's > > +solution for memory pages having (excessive) corrected memory errors. > > + > > +For different types of page, soft-offline has different behaviors / costs. > > +- For a raw error page, soft-offline migrates the in-use page's content to > > + a new raw page. > > +- For a page that is part of a transparent hugepage, soft-offline splits the > > + transparent hugepage into raw pages, then migrates only the raw error page. > > + As a result, user is transparently backed by 1 less hugepage, impacting > > + memory access performance. > > +- For a page that is part of a HugeTLB hugepage, soft-offline first migrates > > + the entire HugeTLB hugepage, during which a free hugepage will be consumed > > + as migration target. Then the original hugepage is dissolved into raw > > + pages without compensation, reducing the capacity of the HugeTLB pool by 1. > > + > > +It is user's call to choose between reliability (staying away from fragile > > +physical memory) vs performance / capacity implications in transparent and > > +HugeTLB cases. > > + > > +For all architectures, enable_soft_offline controls whether to soft offline > > +memory pages. When setting to 1, kernel attempts to soft offline the pages > > +whenever it thinks needed. When setting to 0, kernel returns EOPNOTSUPP to > > +the request to soft offline the pages. Its default value is 1. > > + > > +It is worth mentioning that after setting enable_soft_offline to 0: > > +- If RAS Correctable Errors Collector is running, its request to soft offline > > + pages will fail. > > +- On ARM, the request to soft offline pages from GHES driver will fail. > > +- On PARISC, the request to soft offline pages from Page Deallocation Table > > + will fail. > > I do not know about others but the 'fail' word feels wrong here. > I would reword that as "... the request to soft offline pages from > xxxx will not be performed". Will reword in v4. > > > Other than that: > > Acked-by: Oscar Salvador <osalvador@suse.de> Thanks Oscar! > > -- > Oscar Salvador > SUSE Labs
diff --git a/Documentation/admin-guide/sysctl/vm.rst b/Documentation/admin-guide/sysctl/vm.rst index e86c968a7a0e..fc62fc272fc5 100644 --- a/Documentation/admin-guide/sysctl/vm.rst +++ b/Documentation/admin-guide/sysctl/vm.rst @@ -36,6 +36,7 @@ Currently, these files are in /proc/sys/vm: - dirtytime_expire_seconds - dirty_writeback_centisecs - drop_caches +- enable_soft_offline - extfrag_threshold - highmem_is_dirtyable - hugetlb_shm_group @@ -267,6 +268,38 @@ used:: These are informational only. They do not mean that anything is wrong with your system. To disable them, echo 4 (bit 2) into drop_caches. +enable_soft_offline +=================== +Correctable memory errors are very common on servers. Soft-offline is kernel's +solution for memory pages having (excessive) corrected memory errors. + +For different types of page, soft-offline has different behaviors / costs. +- For a raw error page, soft-offline migrates the in-use page's content to + a new raw page. +- For a page that is part of a transparent hugepage, soft-offline splits the + transparent hugepage into raw pages, then migrates only the raw error page. + As a result, user is transparently backed by 1 less hugepage, impacting + memory access performance. +- For a page that is part of a HugeTLB hugepage, soft-offline first migrates + the entire HugeTLB hugepage, during which a free hugepage will be consumed + as migration target. Then the original hugepage is dissolved into raw + pages without compensation, reducing the capacity of the HugeTLB pool by 1. + +It is user's call to choose between reliability (staying away from fragile +physical memory) vs performance / capacity implications in transparent and +HugeTLB cases. + +For all architectures, enable_soft_offline controls whether to soft offline +memory pages. When setting to 1, kernel attempts to soft offline the pages +whenever it thinks needed. When setting to 0, kernel returns EOPNOTSUPP to +the request to soft offline the pages. Its default value is 1. + +It is worth mentioning that after setting enable_soft_offline to 0: +- If RAS Correctable Errors Collector is running, its request to soft offline + pages will fail. +- On ARM, the request to soft offline pages from GHES driver will fail. +- On PARISC, the request to soft offline pages from Page Deallocation Table + will fail. extfrag_threshold =================
Add the documentation for soft offline behaviors / costs, and what the new enable_soft_offline sysctl is for. Signed-off-by: Jiaqi Yan <jiaqiyan@google.com> --- Documentation/admin-guide/sysctl/vm.rst | 33 +++++++++++++++++++++++++ 1 file changed, 33 insertions(+)