| Message ID | 20210728225140.248408-2-jarkko@kernel.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Swap and fault all the pages of the test enclave binary | expand |
On 7/28/21 3:51 PM, Jarkko Sakkinen wrote: > Create /sys/kernel/debug/x86/sgx_total_mem, after the driver is > initialized, describing the total amount of SGX reserved memory (also known > as Enclave Page Cache (EPC)) available in the system. > > This can be used to implement stress that trigger the page reclaimer, and > it is probably useful information when admistering a system running > enclaves anyhow. How about this as a rewrite? Just like normal memory, SGX memory can be overcommitted. SGX has its own reclaim mechanism which kicks in when physical SGX memory (Enclave Page Cache / EPC) is exhausted. That reclaim mechanism is relatively rarely exercised and needs selftests to poke at it. The amount of EPC on the system is determined by the BIOS and it varies wildly between systems. It can be dozens of MB on desktops, or many GB on servers. To run in a reasonable amount of time, the selftest needs to know how much EPC there is in the system. Introduce a new debugfs file to export that information. > diff --git a/Documentation/x86/sgx.rst b/Documentation/x86/sgx.rst > index dd0ac96ff9ef..15bb8f4e42af 100644 > --- a/Documentation/x86/sgx.rst > +++ b/Documentation/x86/sgx.rst > @@ -250,3 +250,9 @@ user wants to deploy SGX applications both on the host and in guests > on the same machine, the user should reserve enough EPC (by taking out > total virtual EPC size of all SGX VMs from the physical EPC size) for > host SGX applications so they can run with acceptable performance. > + > +SGX debugging > +============= > + > +*/sys/kernel/debug/x86/sgx_total_mem* describes the total available > +SGX mwemory available in the system. ^ memory It would also be nice to say what the units are here, even if they are bytes. > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c > index 63d3de02bbcc..b65da19a53ee 100644 > --- a/arch/x86/kernel/cpu/sgx/main.c > +++ b/arch/x86/kernel/cpu/sgx/main.c > @@ -1,6 +1,7 @@ > // SPDX-License-Identifier: GPL-2.0 > /* Copyright(c) 2016-20 Intel Corporation. */ > > +#include <linux/debugfs.h> > #include <linux/file.h> > #include <linux/freezer.h> > #include <linux/highmem.h> > @@ -28,7 +29,10 @@ static DECLARE_WAIT_QUEUE_HEAD(ksgxd_waitq); > static LIST_HEAD(sgx_active_page_list); > static DEFINE_SPINLOCK(sgx_reclaimer_lock); > > -/* The free page list lock protected variables prepend the lock. */ > +/* Total EPC memory available in bytes. */ > +static unsigned long sgx_total_mem; > + > +/* The number of free EPC pages in all nodes. */ > static unsigned long sgx_nr_free_pages; > > /* Nodes with one or more EPC sections. */ > @@ -656,6 +660,8 @@ static bool __init sgx_setup_epc_section(u64 phys_addr, u64 size, > list_add_tail(§ion->pages[i].list, &sgx_dirty_page_list); > } > > + sgx_total_mem += nr_pages * PAGE_SIZE; > + > return true; > } > > @@ -823,6 +829,8 @@ static int __init sgx_init(void) > if (sgx_vepc_init() && ret) > goto err_provision; > > + debugfs_create_ulong("sgx_total_mem", 0444, arch_debugfs_dir, &sgx_total_mem); > + > return 0; > > err_provision: > Assuming the new changelog is adopted and typos fixed: Acked-by: Dave Hansen <dave.hansen@linux.intel.com>
On Thu, Jul 29, 2021 at 09:27:41AM -0700, Dave Hansen wrote: > On 7/28/21 3:51 PM, Jarkko Sakkinen wrote: > > Create /sys/kernel/debug/x86/sgx_total_mem, after the driver is > > initialized, describing the total amount of SGX reserved memory (also known > > as Enclave Page Cache (EPC)) available in the system. > > > > This can be used to implement stress that trigger the page reclaimer, and > > it is probably useful information when admistering a system running > > enclaves anyhow. > > How about this as a rewrite? > > Just like normal memory, SGX memory can be overcommitted. SGX has its > own reclaim mechanism which kicks in when physical SGX memory (Enclave > Page Cache / EPC) is exhausted. That reclaim mechanism is relatively > rarely exercised and needs selftests to poke at it. > > The amount of EPC on the system is determined by the BIOS and it varies > wildly between systems. It can be dozens of MB on desktops, or many GB > on servers. > > To run in a reasonable amount of time, the selftest needs to know how > much EPC there is in the system. > > Introduce a new debugfs file to export that information. > > > diff --git a/Documentation/x86/sgx.rst b/Documentation/x86/sgx.rst > > index dd0ac96ff9ef..15bb8f4e42af 100644 > > --- a/Documentation/x86/sgx.rst > > +++ b/Documentation/x86/sgx.rst > > @@ -250,3 +250,9 @@ user wants to deploy SGX applications both on the host and in guests > > on the same machine, the user should reserve enough EPC (by taking out > > total virtual EPC size of all SGX VMs from the physical EPC size) for > > host SGX applications so they can run with acceptable performance. > > + > > +SGX debugging > > +============= > > + > > +*/sys/kernel/debug/x86/sgx_total_mem* describes the total available > > +SGX mwemory available in the system. > > ^ memory > > It would also be nice to say what the units are here, even if they are > bytes. > > > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c > > index 63d3de02bbcc..b65da19a53ee 100644 > > --- a/arch/x86/kernel/cpu/sgx/main.c > > +++ b/arch/x86/kernel/cpu/sgx/main.c > > @@ -1,6 +1,7 @@ > > // SPDX-License-Identifier: GPL-2.0 > > /* Copyright(c) 2016-20 Intel Corporation. */ > > > > +#include <linux/debugfs.h> > > #include <linux/file.h> > > #include <linux/freezer.h> > > #include <linux/highmem.h> > > @@ -28,7 +29,10 @@ static DECLARE_WAIT_QUEUE_HEAD(ksgxd_waitq); > > static LIST_HEAD(sgx_active_page_list); > > static DEFINE_SPINLOCK(sgx_reclaimer_lock); > > > > -/* The free page list lock protected variables prepend the lock. */ > > +/* Total EPC memory available in bytes. */ > > +static unsigned long sgx_total_mem; > > + > > +/* The number of free EPC pages in all nodes. */ > > static unsigned long sgx_nr_free_pages; > > > > /* Nodes with one or more EPC sections. */ > > @@ -656,6 +660,8 @@ static bool __init sgx_setup_epc_section(u64 phys_addr, u64 size, > > list_add_tail(§ion->pages[i].list, &sgx_dirty_page_list); > > } > > > > + sgx_total_mem += nr_pages * PAGE_SIZE; > > + > > return true; > > } > > > > @@ -823,6 +829,8 @@ static int __init sgx_init(void) > > if (sgx_vepc_init() && ret) > > goto err_provision; > > > > + debugfs_create_ulong("sgx_total_mem", 0444, arch_debugfs_dir, &sgx_total_mem); > > + > > return 0; > > > > err_provision: > > > > Assuming the new changelog is adopted and typos fixed: > > Acked-by: Dave Hansen <dave.hansen@linux.intel.com> Thanks, I'll commit these changes and include to the next version. /Jarkko
diff --git a/Documentation/x86/sgx.rst b/Documentation/x86/sgx.rst index dd0ac96ff9ef..15bb8f4e42af 100644 --- a/Documentation/x86/sgx.rst +++ b/Documentation/x86/sgx.rst @@ -250,3 +250,9 @@ user wants to deploy SGX applications both on the host and in guests on the same machine, the user should reserve enough EPC (by taking out total virtual EPC size of all SGX VMs from the physical EPC size) for host SGX applications so they can run with acceptable performance. + +SGX debugging +============= + +*/sys/kernel/debug/x86/sgx_total_mem* describes the total available +SGX mwemory available in the system. diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c index 63d3de02bbcc..b65da19a53ee 100644 --- a/arch/x86/kernel/cpu/sgx/main.c +++ b/arch/x86/kernel/cpu/sgx/main.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright(c) 2016-20 Intel Corporation. */ +#include <linux/debugfs.h> #include <linux/file.h> #include <linux/freezer.h> #include <linux/highmem.h> @@ -28,7 +29,10 @@ static DECLARE_WAIT_QUEUE_HEAD(ksgxd_waitq); static LIST_HEAD(sgx_active_page_list); static DEFINE_SPINLOCK(sgx_reclaimer_lock); -/* The free page list lock protected variables prepend the lock. */ +/* Total EPC memory available in bytes. */ +static unsigned long sgx_total_mem; + +/* The number of free EPC pages in all nodes. */ static unsigned long sgx_nr_free_pages; /* Nodes with one or more EPC sections. */ @@ -656,6 +660,8 @@ static bool __init sgx_setup_epc_section(u64 phys_addr, u64 size, list_add_tail(§ion->pages[i].list, &sgx_dirty_page_list); } + sgx_total_mem += nr_pages * PAGE_SIZE; + return true; } @@ -823,6 +829,8 @@ static int __init sgx_init(void) if (sgx_vepc_init() && ret) goto err_provision; + debugfs_create_ulong("sgx_total_mem", 0444, arch_debugfs_dir, &sgx_total_mem); + return 0; err_provision:
Create /sys/kernel/debug/x86/sgx_total_mem, after the driver is initialized, describing the total amount of SGX reserved memory (also known as Enclave Page Cache (EPC)) available in the system. This can be used to implement stress that trigger the page reclaimer, and it is probably useful information when admistering a system running enclaves anyhow. Signed-off-by: Jarkko Sakkinen <jarkko@kernel.org> --- v2: * sgx_nr_all_pages -> sgx_total_mem Documentation/x86/sgx.rst | 6 ++++++ arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++- 2 files changed, 15 insertions(+), 1 deletion(-)