| Message ID | 2293bdca025ce54bd2996783e187e209f9c0f5f2.1649878359.git.reinette.chatre@intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | x86/sgx and selftests/sgx: Support SGX2 | expand |
On Wed, 2022-04-13 at 14:10 -0700, Reinette Chatre wrote: > With SGX1 an enclave needs to be created with its maximum memory demands > allocated. Pages cannot be added to an enclave after it is initialized. > SGX2 introduces a new function, ENCLS[EAUG], that can be used to add > pages to an initialized enclave. With SGX2 the enclave still needs to > set aside address space for its maximum memory demands during enclave > creation, but all pages need not be added before enclave initialization. > Pages can be added during enclave runtime. > > Add support for dynamically adding pages to an initialized enclave, > architecturally limited to RW permission at creation but allowed to > obtain RWX permissions after trusted enclave runs EMODPE. Add pages > via the page fault handler at the time an enclave address without a > backing enclave page is accessed, potentially directly reclaiming > pages if no free pages are available. > > The enclave is still required to run ENCLU[EACCEPT] on the page before > it can be used. A useful flow is for the enclave to run ENCLU[EACCEPT] > on an uninitialized address. This will trigger the page fault handler > that will add the enclave page and return execution to the enclave to > repeat the ENCLU[EACCEPT] instruction, this time successful. > > If the enclave accesses an uninitialized address in another way, for > example by expanding the enclave stack to a page that has not yet been > added, then the page fault handler would add the page on the first > write but upon returning to the enclave the instruction that triggered > the page fault would be repeated and since ENCLU[EACCEPT] was not run > yet it would trigger a second page fault, this time with the SGX flag > set in the page fault error code. This can only be recovered by entering > the enclave again and directly running the ENCLU[EACCEPT] instruction on > the now initialized address. > > Accessing an uninitialized address from outside the enclave also > triggers this flow but the page will remain inaccessible (access will > result in #PF) until accepted from within the enclave via > ENCLU[EACCEPT]. > > Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org> > Tested-by: Jarkko Sakkinen <jarkko@kernel.org> > Signed-off-by: Reinette Chatre <reinette.chatre@intel.com> I'm presuming that Haitao tested with this applied, right? BR, Jarkko
Hi Jarkko, On 4/14/2022 4:20 AM, Jarkko Sakkinen wrote: > On Wed, 2022-04-13 at 14:10 -0700, Reinette Chatre wrote: >> With SGX1 an enclave needs to be created with its maximum memory demands >> allocated. Pages cannot be added to an enclave after it is initialized. >> SGX2 introduces a new function, ENCLS[EAUG], that can be used to add >> pages to an initialized enclave. With SGX2 the enclave still needs to >> set aside address space for its maximum memory demands during enclave >> creation, but all pages need not be added before enclave initialization. >> Pages can be added during enclave runtime. >> >> Add support for dynamically adding pages to an initialized enclave, >> architecturally limited to RW permission at creation but allowed to >> obtain RWX permissions after trusted enclave runs EMODPE. Add pages >> via the page fault handler at the time an enclave address without a >> backing enclave page is accessed, potentially directly reclaiming >> pages if no free pages are available. >> >> The enclave is still required to run ENCLU[EACCEPT] on the page before >> it can be used. A useful flow is for the enclave to run ENCLU[EACCEPT] >> on an uninitialized address. This will trigger the page fault handler >> that will add the enclave page and return execution to the enclave to >> repeat the ENCLU[EACCEPT] instruction, this time successful. >> >> If the enclave accesses an uninitialized address in another way, for >> example by expanding the enclave stack to a page that has not yet been >> added, then the page fault handler would add the page on the first >> write but upon returning to the enclave the instruction that triggered >> the page fault would be repeated and since ENCLU[EACCEPT] was not run >> yet it would trigger a second page fault, this time with the SGX flag >> set in the page fault error code. This can only be recovered by entering >> the enclave again and directly running the ENCLU[EACCEPT] instruction on >> the now initialized address. >> >> Accessing an uninitialized address from outside the enclave also >> triggers this flow but the page will remain inaccessible (access will >> result in #PF) until accepted from within the enclave via >> ENCLU[EACCEPT]. >> >> Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org> >> Tested-by: Jarkko Sakkinen <jarkko@kernel.org> >> Signed-off-by: Reinette Chatre <reinette.chatre@intel.com> > > I'm presuming that Haitao tested with this applied, right? Yes, I will move his Tested-by to this patch. Reinette
diff --git a/arch/x86/kernel/cpu/sgx/encl.c b/arch/x86/kernel/cpu/sgx/encl.c index 92516aeca405..7ccda6fe1f8f 100644 --- a/arch/x86/kernel/cpu/sgx/encl.c +++ b/arch/x86/kernel/cpu/sgx/encl.c @@ -194,6 +194,112 @@ struct sgx_encl_page *sgx_encl_load_page(struct sgx_encl *encl, return __sgx_encl_load_page(encl, entry); } +/** + * sgx_encl_eaug_page() - Dynamically add page to initialized enclave + * @vma: VMA obtained from fault info from where page is accessed + * @encl: enclave accessing the page + * @addr: address that triggered the page fault + * + * When an initialized enclave accesses a page with no backing EPC page + * on a SGX2 system then the EPC can be added dynamically via the SGX2 + * ENCLS[EAUG] instruction. + * + * Returns: Appropriate vm_fault_t: VM_FAULT_NOPAGE when PTE was installed + * successfully, VM_FAULT_SIGBUS or VM_FAULT_OOM as error otherwise. + */ +static vm_fault_t sgx_encl_eaug_page(struct vm_area_struct *vma, + struct sgx_encl *encl, unsigned long addr) +{ + struct sgx_pageinfo pginfo = {0}; + struct sgx_encl_page *encl_page; + struct sgx_epc_page *epc_page; + struct sgx_va_page *va_page; + unsigned long phys_addr; + u64 secinfo_flags; + vm_fault_t vmret; + int ret; + + if (!test_bit(SGX_ENCL_INITIALIZED, &encl->flags)) + return VM_FAULT_SIGBUS; + + /* + * Ignore internal permission checking for dynamically added pages. + * They matter only for data added during the pre-initialization + * phase. The enclave decides the permissions by the means of + * EACCEPT, EACCEPTCOPY and EMODPE. + */ + secinfo_flags = SGX_SECINFO_R | SGX_SECINFO_W | SGX_SECINFO_X; + encl_page = sgx_encl_page_alloc(encl, addr - encl->base, secinfo_flags); + if (IS_ERR(encl_page)) + return VM_FAULT_OOM; + + epc_page = sgx_alloc_epc_page(encl_page, true); + if (IS_ERR(epc_page)) { + kfree(encl_page); + return VM_FAULT_SIGBUS; + } + + mutex_lock(&encl->lock); + + va_page = sgx_encl_grow(encl, false); + if (IS_ERR(va_page)) { + ret = PTR_ERR(va_page); + goto err_out_unlock; + } + + if (va_page) + list_add(&va_page->list, &encl->va_pages); + + ret = xa_insert(&encl->page_array, PFN_DOWN(encl_page->desc), + encl_page, GFP_KERNEL); + /* + * If ret == -EBUSY then page was created in another flow while + * running without encl->lock + */ + if (ret) + goto err_out_shrink; + + pginfo.secs = (unsigned long)sgx_get_epc_virt_addr(encl->secs.epc_page); + pginfo.addr = encl_page->desc & PAGE_MASK; + pginfo.metadata = 0; + + ret = __eaug(&pginfo, sgx_get_epc_virt_addr(epc_page)); + if (ret) + goto err_out; + + encl_page->encl = encl; + encl_page->epc_page = epc_page; + encl_page->type = SGX_PAGE_TYPE_REG; + encl->secs_child_cnt++; + + sgx_mark_page_reclaimable(encl_page->epc_page); + + phys_addr = sgx_get_epc_phys_addr(epc_page); + /* + * Do not undo everything when creating PTE entry fails - next #PF + * would find page ready for a PTE. + */ + vmret = vmf_insert_pfn(vma, addr, PFN_DOWN(phys_addr)); + if (vmret != VM_FAULT_NOPAGE) { + mutex_unlock(&encl->lock); + return VM_FAULT_SIGBUS; + } + mutex_unlock(&encl->lock); + return VM_FAULT_NOPAGE; + +err_out: + xa_erase(&encl->page_array, PFN_DOWN(encl_page->desc)); + +err_out_shrink: + sgx_encl_shrink(encl, va_page); +err_out_unlock: + mutex_unlock(&encl->lock); + sgx_encl_free_epc_page(epc_page); + kfree(encl_page); + + return VM_FAULT_SIGBUS; +} + static vm_fault_t sgx_vma_fault(struct vm_fault *vmf) { unsigned long addr = (unsigned long)vmf->address; @@ -213,6 +319,17 @@ static vm_fault_t sgx_vma_fault(struct vm_fault *vmf) if (unlikely(!encl)) return VM_FAULT_SIGBUS; + /* + * The page_array keeps track of all enclave pages, whether they + * are swapped out or not. If there is no entry for this page and + * the system supports SGX2 then it is possible to dynamically add + * a new enclave page. This is only possible for an initialized + * enclave that will be checked for right away. + */ + if (cpu_feature_enabled(X86_FEATURE_SGX2) && + (!xa_load(&encl->page_array, PFN_DOWN(addr)))) + return sgx_encl_eaug_page(vma, encl, addr); + mutex_lock(&encl->lock); entry = sgx_encl_load_page_in_vma(encl, addr, vma->vm_flags);