From patchwork Thu Oct 10 21:42:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11184519 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A7C9A112B for ; Thu, 10 Oct 2019 21:43:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 923E121835 for ; Thu, 10 Oct 2019 21:43:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727438AbfJJVnE (ORCPT ); Thu, 10 Oct 2019 17:43:04 -0400 Received: from mga18.intel.com ([134.134.136.126]:32755 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727437AbfJJVnD (ORCPT ); Thu, 10 Oct 2019 17:43:03 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Oct 2019 14:43:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,281,1566889200"; d="scan'208";a="194145477" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.41]) by fmsmga007.fm.intel.com with ESMTP; 10 Oct 2019 14:43:02 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: linux-sgx@vger.kernel.org Subject: [PATCH for_v23 1/9] x86/sgx: WARN once if an enclave is released with unfreed EPC pages Date: Thu, 10 Oct 2019 14:42:53 -0700 Message-Id: <20191010214301.25669-2-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20191010214301.25669-1-sean.j.christopherson@intel.com> References: <20191010214301.25669-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Add a WARN to detect EPC page leaks when releasing an enclave. The release flow uses the common sgx_encl_destroy() helper, which is allowed to be called while the reclaimer holds references to the enclave's EPC pages and so can't WARN in the scenario where the SECS is leaked because it has active child pages. Signed-off-by: Sean Christopherson --- arch/x86/kernel/cpu/sgx/encl.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kernel/cpu/sgx/encl.c b/arch/x86/kernel/cpu/sgx/encl.c index c13c3ba3430a..b4d7b2f9609f 100644 --- a/arch/x86/kernel/cpu/sgx/encl.c +++ b/arch/x86/kernel/cpu/sgx/encl.c @@ -511,6 +511,7 @@ void sgx_encl_release(struct kref *ref) fput(encl->backing); WARN_ONCE(!list_empty(&encl->mm_list), "mm_list non-empty"); + WARN_ON_ONCE(encl->secs_child_cnt || encl->secs.epc_page); kfree(encl); } From patchwork Thu Oct 10 21:42:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11184521 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5D1B714ED for ; Thu, 10 Oct 2019 21:43:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 47A9D218AC for ; Thu, 10 Oct 2019 21:43:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727459AbfJJVnE (ORCPT ); Thu, 10 Oct 2019 17:43:04 -0400 Received: from mga18.intel.com ([134.134.136.126]:32755 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726983AbfJJVnE (ORCPT ); Thu, 10 Oct 2019 17:43:04 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Oct 2019 14:43:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,281,1566889200"; d="scan'208";a="194145480" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.41]) by fmsmga007.fm.intel.com with ESMTP; 10 Oct 2019 14:43:03 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: linux-sgx@vger.kernel.org Subject: [PATCH for_v23 2/9] x86/sgx: Do not EWB SECS if the enclave is dead Date: Thu, 10 Oct 2019 14:42:54 -0700 Message-Id: <20191010214301.25669-3-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20191010214301.25669-1-sean.j.christopherson@intel.com> References: <20191010214301.25669-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Don't EWB if the enclave is dead when opportunistically zapping the SECS during reclaim as VA pages are freed by sgx_encl_destroy(), i.e. sgx_encl_ewb() will consume a bad encl->va_pages if the enclave has been destroyed. Add a comment in sgx_encl_destroy() to explicit call out that it's ok to free VA pages. Signed-off-by: Sean Christopherson --- arch/x86/kernel/cpu/sgx/encl.c | 6 +++++- arch/x86/kernel/cpu/sgx/reclaim.c | 19 +++++++++++-------- 2 files changed, 16 insertions(+), 9 deletions(-) diff --git a/arch/x86/kernel/cpu/sgx/encl.c b/arch/x86/kernel/cpu/sgx/encl.c index b4d7b2f9609f..ea21d3737a32 100644 --- a/arch/x86/kernel/cpu/sgx/encl.c +++ b/arch/x86/kernel/cpu/sgx/encl.c @@ -484,7 +484,11 @@ void sgx_encl_destroy(struct sgx_encl *encl) encl->secs.epc_page = NULL; } - + /* + * The reclaimer is responsible for checking SGX_ENCL_DEAD before doing + * EWB, thus it's safe to free VA pages even if the reclaimer holds a + * reference to the enclave. + */ while (!list_empty(&encl->va_pages)) { va_page = list_first_entry(&encl->va_pages, struct sgx_va_page, list); diff --git a/arch/x86/kernel/cpu/sgx/reclaim.c b/arch/x86/kernel/cpu/sgx/reclaim.c index 391fbc3e7e98..8143c9a20894 100644 --- a/arch/x86/kernel/cpu/sgx/reclaim.c +++ b/arch/x86/kernel/cpu/sgx/reclaim.c @@ -321,16 +321,19 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page, encl_page->epc_page = NULL; encl->secs_child_cnt--; - if (!encl->secs_child_cnt && - (atomic_read(&encl->flags) & - (SGX_ENCL_DEAD | SGX_ENCL_INITIALIZED))) { - ret = sgx_encl_get_backing(encl, PFN_DOWN(encl->size), - &secs_backing); - if (!ret) { - sgx_encl_ewb(encl->secs.epc_page, &secs_backing); + if (!encl->secs_child_cnt) { + if (atomic_read(&encl->flags) & SGX_ENCL_DEAD) { sgx_free_page(encl->secs.epc_page); - encl->secs.epc_page = NULL; + } else if (atomic_read(&encl->flags) & SGX_ENCL_INITIALIZED) { + ret = sgx_encl_get_backing(encl, PFN_DOWN(encl->size), + &secs_backing); + if (!ret) { + sgx_encl_ewb(encl->secs.epc_page, &secs_backing); + + sgx_free_page(encl->secs.epc_page); + encl->secs.epc_page = NULL; + } sgx_encl_put_backing(&secs_backing, true); } From patchwork Thu Oct 10 21:42:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11184525 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DF04818B7 for ; Thu, 10 Oct 2019 21:43:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BCE8021D7D for ; Thu, 10 Oct 2019 21:43:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727437AbfJJVnF (ORCPT ); Thu, 10 Oct 2019 17:43:05 -0400 Received: from mga18.intel.com ([134.134.136.126]:32755 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727440AbfJJVnE (ORCPT ); Thu, 10 Oct 2019 17:43:04 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Oct 2019 14:43:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,281,1566889200"; d="scan'208";a="194145483" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.41]) by fmsmga007.fm.intel.com with ESMTP; 10 Oct 2019 14:43:03 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: linux-sgx@vger.kernel.org Subject: [PATCH for_v23 3/9] x86/sgx: Fix a memory leak in sgx_encl_destroy() Date: Thu, 10 Oct 2019 14:42:55 -0700 Message-Id: <20191010214301.25669-4-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20191010214301.25669-1-sean.j.christopherson@intel.com> References: <20191010214301.25669-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Delete an enclave page's entry in the radix tree regardless of whether or not it has an associated EPC page. Note, the reclaimer doesn't need the tree to reclaim EPC pages, i.e. it's safe to delete the tree entry even if sgx_free_page() fails. Signed-off-by: Sean Christopherson --- arch/x86/kernel/cpu/sgx/encl.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/arch/x86/kernel/cpu/sgx/encl.c b/arch/x86/kernel/cpu/sgx/encl.c index ea21d3737a32..68ec8944692a 100644 --- a/arch/x86/kernel/cpu/sgx/encl.c +++ b/arch/x86/kernel/cpu/sgx/encl.c @@ -468,15 +468,14 @@ void sgx_encl_destroy(struct sgx_encl *encl) radix_tree_for_each_slot(slot, &encl->page_tree, &iter, 0) { entry = *slot; - if (entry->epc_page) { - if (!sgx_free_page(entry->epc_page)) { - encl->secs_child_cnt--; - entry->epc_page = NULL; - } - - radix_tree_delete(&entry->encl->page_tree, - PFN_DOWN(entry->desc)); + if (entry->epc_page && + !sgx_free_page(entry->epc_page)) { + encl->secs_child_cnt--; + entry->epc_page = NULL; } + + radix_tree_delete(&entry->encl->page_tree, + PFN_DOWN(entry->desc)); } if (!encl->secs_child_cnt && encl->secs.epc_page) { From patchwork Thu Oct 10 21:42:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11184537 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 882271920 for ; Thu, 10 Oct 2019 21:43:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 67DB720B7C for ; Thu, 10 Oct 2019 21:43:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726983AbfJJVnG (ORCPT ); Thu, 10 Oct 2019 17:43:06 -0400 Received: from mga18.intel.com ([134.134.136.126]:32755 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727460AbfJJVnE (ORCPT ); Thu, 10 Oct 2019 17:43:04 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Oct 2019 14:43:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,281,1566889200"; d="scan'208";a="194145486" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.41]) by fmsmga007.fm.intel.com with ESMTP; 10 Oct 2019 14:43:03 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: linux-sgx@vger.kernel.org Subject: [PATCH for_v23 4/9] x86/sgx: WARN on any non-zero return from __eremove() Date: Thu, 10 Oct 2019 14:42:56 -0700 Message-Id: <20191010214301.25669-5-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20191010214301.25669-1-sean.j.christopherson@intel.com> References: <20191010214301.25669-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org WARN on any non-zero return from __eremove() to make it clear that any kind of failure is unexpected. Technically, warning on negative values is unnecessary as the ENCLS helpers return SGX error codes, which are currently all postive. But, the more precise check might be misinterpreted as implying the negative values are expected/ok. Note, prior to a recent change, warning only on positive values was necessary to avoid a redundant double-WARN as the WARN resided outside of what is now sgx_free_page() and so could consume -EBUSY. Signed-off-by: Sean Christopherson --- arch/x86/kernel/cpu/sgx/main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c index 2b540abbb61f..5170d4ba1096 100644 --- a/arch/x86/kernel/cpu/sgx/main.c +++ b/arch/x86/kernel/cpu/sgx/main.c @@ -137,7 +137,7 @@ int sgx_free_page(struct sgx_epc_page *page) spin_unlock(&sgx_active_page_list_lock); ret = __eremove(sgx_epc_addr(page)); - WARN(ret > 0, "EREMOVE returned %d (0x%x)", ret, ret); + WARN(ret, "EREMOVE returned %d (0x%x)", ret, ret); spin_lock(§ion->lock); list_add_tail(&page->list, §ion->page_list); From patchwork Thu Oct 10 21:42:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11184531 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 23AA8112B for ; Thu, 10 Oct 2019 21:43:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0DA3C20B7C for ; Thu, 10 Oct 2019 21:43:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727460AbfJJVnG (ORCPT ); Thu, 10 Oct 2019 17:43:06 -0400 Received: from mga18.intel.com ([134.134.136.126]:32763 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727462AbfJJVnE (ORCPT ); Thu, 10 Oct 2019 17:43:04 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Oct 2019 14:43:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,281,1566889200"; d="scan'208";a="194145489" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.41]) by fmsmga007.fm.intel.com with ESMTP; 10 Oct 2019 14:43:03 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: linux-sgx@vger.kernel.org Subject: [PATCH for_v23 5/9] x86/sgx: WARN only once if EREMOVE fails Date: Thu, 10 Oct 2019 14:42:57 -0700 Message-Id: <20191010214301.25669-6-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20191010214301.25669-1-sean.j.christopherson@intel.com> References: <20191010214301.25669-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org WARN only once if EREMOVE fails to avoid spamming the kernel log if a catastrophic failure occurs. Warning on every failure is helpful for development, but is a bad idea for production code as EREMOVE rarely fails just once... Signed-off-by: Sean Christopherson --- arch/x86/kernel/cpu/sgx/main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c index 5170d4ba1096..84a44251387b 100644 --- a/arch/x86/kernel/cpu/sgx/main.c +++ b/arch/x86/kernel/cpu/sgx/main.c @@ -137,7 +137,7 @@ int sgx_free_page(struct sgx_epc_page *page) spin_unlock(&sgx_active_page_list_lock); ret = __eremove(sgx_epc_addr(page)); - WARN(ret, "EREMOVE returned %d (0x%x)", ret, ret); + WARN_ONCE(ret, "EREMOVE returned %d (0x%x)", ret, ret); spin_lock(§ion->lock); list_add_tail(&page->list, §ion->page_list); From patchwork Thu Oct 10 21:42:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11184535 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5F6F817EE for ; Thu, 10 Oct 2019 21:43:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 49EA8214E0 for ; Thu, 10 Oct 2019 21:43:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726986AbfJJVnG (ORCPT ); Thu, 10 Oct 2019 17:43:06 -0400 Received: from mga18.intel.com ([134.134.136.126]:32763 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727464AbfJJVnG (ORCPT ); Thu, 10 Oct 2019 17:43:06 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Oct 2019 14:43:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,281,1566889200"; d="scan'208";a="194145492" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.41]) by fmsmga007.fm.intel.com with ESMTP; 10 Oct 2019 14:43:03 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: linux-sgx@vger.kernel.org Subject: [PATCH for_v23 6/9] x86/sgx: Split second half of sgx_free_page() to a separate helper Date: Thu, 10 Oct 2019 14:42:58 -0700 Message-Id: <20191010214301.25669-7-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20191010214301.25669-1-sean.j.christopherson@intel.com> References: <20191010214301.25669-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Move the post-reclaim half of sgx_free_page() to a standalone helper so that it can be used in flows where the page is known to be non-reclaimable. Signed-off-by: Sean Christopherson --- arch/x86/kernel/cpu/sgx/main.c | 44 ++++++++++++++++++++++++++-------- arch/x86/kernel/cpu/sgx/sgx.h | 1 + 2 files changed, 35 insertions(+), 10 deletions(-) diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c index 84a44251387b..e22cdbb431a3 100644 --- a/arch/x86/kernel/cpu/sgx/main.c +++ b/arch/x86/kernel/cpu/sgx/main.c @@ -103,6 +103,39 @@ struct sgx_epc_page *sgx_alloc_page(void *owner, bool reclaim) return entry; } + +/** + * __sgx_free_page() - Free an EPC page + * @page: pointer a previously allocated EPC page + * + * EREMOVE an EPC page and insert it back to the list of free pages. The page + * must not be reclaimable. + */ +void __sgx_free_page(struct sgx_epc_page *page) +{ + struct sgx_epc_section *section; + int ret; + + /* + * Don't take sgx_active_page_list_lock when asserting the page isn't + * reclaimable, missing a WARN in the very rare case is preferable to + * unnecessarily taking a global lock in the common case. + */ + WARN_ON_ONCE(page->desc & SGX_EPC_PAGE_RECLAIMABLE); + + ret = __eremove(sgx_epc_addr(page)); + if (WARN_ONCE(ret, "EREMOVE returned %d (0x%x)", ret, ret)) + return; + + section = sgx_epc_section(page); + + spin_lock(§ion->lock); + list_add_tail(&page->list, §ion->page_list); + sgx_nr_free_pages++; + spin_unlock(§ion->lock); + +} + /** * sgx_free_page() - Free an EPC page * @page: pointer a previously allocated EPC page @@ -116,9 +149,6 @@ struct sgx_epc_page *sgx_alloc_page(void *owner, bool reclaim) */ int sgx_free_page(struct sgx_epc_page *page) { - struct sgx_epc_section *section = sgx_epc_section(page); - int ret; - /* * Remove the page from the active list if necessary. If the page * is actively being reclaimed, i.e. RECLAIMABLE is set but the @@ -136,13 +166,7 @@ int sgx_free_page(struct sgx_epc_page *page) } spin_unlock(&sgx_active_page_list_lock); - ret = __eremove(sgx_epc_addr(page)); - WARN_ONCE(ret, "EREMOVE returned %d (0x%x)", ret, ret); - - spin_lock(§ion->lock); - list_add_tail(&page->list, §ion->page_list); - sgx_nr_free_pages++; - spin_unlock(§ion->lock); + __sgx_free_page(page); return 0; } diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h index 160a3c996ef6..87e375e8c25e 100644 --- a/arch/x86/kernel/cpu/sgx/sgx.h +++ b/arch/x86/kernel/cpu/sgx/sgx.h @@ -85,6 +85,7 @@ void sgx_reclaim_pages(void); struct sgx_epc_page *sgx_try_alloc_page(void); struct sgx_epc_page *sgx_alloc_page(void *owner, bool reclaim); +void __sgx_free_page(struct sgx_epc_page *page); int sgx_free_page(struct sgx_epc_page *page); #endif /* _X86_SGX_H */ From patchwork Thu Oct 10 21:42:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11184529 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9AD1617EE for ; Thu, 10 Oct 2019 21:43:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8554020B7C for ; Thu, 10 Oct 2019 21:43:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727465AbfJJVnG (ORCPT ); Thu, 10 Oct 2019 17:43:06 -0400 Received: from mga18.intel.com ([134.134.136.126]:32755 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726986AbfJJVnG (ORCPT ); Thu, 10 Oct 2019 17:43:06 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Oct 2019 14:43:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,281,1566889200"; d="scan'208";a="194145496" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.41]) by fmsmga007.fm.intel.com with ESMTP; 10 Oct 2019 14:43:03 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: linux-sgx@vger.kernel.org Subject: [PATCH for_v23 7/9] x86/sgx: Use the post-reclaim variant of __sgx_free_page() Date: Thu, 10 Oct 2019 14:42:59 -0700 Message-Id: <20191010214301.25669-8-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20191010214301.25669-1-sean.j.christopherson@intel.com> References: <20191010214301.25669-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Use __sgx_free_page() in all locations where the EPC page is supposed to be unreclaimable so that a WARN fires if the page is unexpectedly marked for reclaim. Signed-off-by: Sean Christopherson --- arch/x86/kernel/cpu/sgx/encl.c | 8 ++++---- arch/x86/kernel/cpu/sgx/ioctl.c | 6 +++--- arch/x86/kernel/cpu/sgx/reclaim.c | 4 ++-- 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/arch/x86/kernel/cpu/sgx/encl.c b/arch/x86/kernel/cpu/sgx/encl.c index 68ec8944692a..d2349ca8975c 100644 --- a/arch/x86/kernel/cpu/sgx/encl.c +++ b/arch/x86/kernel/cpu/sgx/encl.c @@ -73,7 +73,7 @@ static struct sgx_epc_page *sgx_encl_eldu(struct sgx_encl_page *encl_page, ret = __sgx_encl_eldu(encl_page, epc_page, secs_page); if (ret) { - sgx_free_page(epc_page); + __sgx_free_page(epc_page); return ERR_PTR(ret); } @@ -479,7 +479,7 @@ void sgx_encl_destroy(struct sgx_encl *encl) } if (!encl->secs_child_cnt && encl->secs.epc_page) { - sgx_free_page(encl->secs.epc_page); + __sgx_free_page(encl->secs.epc_page); encl->secs.epc_page = NULL; } @@ -492,7 +492,7 @@ void sgx_encl_destroy(struct sgx_encl *encl) va_page = list_first_entry(&encl->va_pages, struct sgx_va_page, list); list_del(&va_page->list); - sgx_free_page(va_page->epc_page); + __sgx_free_page(va_page->epc_page); kfree(va_page); } } @@ -687,7 +687,7 @@ struct sgx_epc_page *sgx_alloc_va_page(void) ret = __epa(sgx_epc_addr(epc_page)); if (ret) { WARN_ONCE(1, "EPA returned %d (0x%x)", ret, ret); - sgx_free_page(epc_page); + __sgx_free_page(epc_page); return ERR_PTR(-EFAULT); } diff --git a/arch/x86/kernel/cpu/sgx/ioctl.c b/arch/x86/kernel/cpu/sgx/ioctl.c index eeed919ed3e2..7a6bc3bf8f8c 100644 --- a/arch/x86/kernel/cpu/sgx/ioctl.c +++ b/arch/x86/kernel/cpu/sgx/ioctl.c @@ -47,7 +47,7 @@ static void sgx_encl_shrink(struct sgx_encl *encl, struct sgx_va_page *va_page) encl->page_cnt--; if (va_page) { - sgx_free_page(va_page->epc_page); + __sgx_free_page(va_page->epc_page); list_del(&va_page->list); kfree(va_page); } @@ -220,7 +220,7 @@ static int sgx_encl_create(struct sgx_encl *encl, struct sgx_secs *secs) return 0; err_out: - sgx_free_page(encl->secs.epc_page); + __sgx_free_page(encl->secs.epc_page); encl->secs.epc_page = NULL; err_out_backing: @@ -441,7 +441,7 @@ static int sgx_encl_add_page(struct sgx_encl *encl, up_read(¤t->mm->mmap_sem); err_out_free: - sgx_free_page(epc_page); + __sgx_free_page(epc_page); kfree(encl_page); return ret; diff --git a/arch/x86/kernel/cpu/sgx/reclaim.c b/arch/x86/kernel/cpu/sgx/reclaim.c index 8143c9a20894..3f183dd0e653 100644 --- a/arch/x86/kernel/cpu/sgx/reclaim.c +++ b/arch/x86/kernel/cpu/sgx/reclaim.c @@ -323,7 +323,7 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page, if (!encl->secs_child_cnt) { if (atomic_read(&encl->flags) & SGX_ENCL_DEAD) { - sgx_free_page(encl->secs.epc_page); + __sgx_free_page(encl->secs.epc_page); encl->secs.epc_page = NULL; } else if (atomic_read(&encl->flags) & SGX_ENCL_INITIALIZED) { ret = sgx_encl_get_backing(encl, PFN_DOWN(encl->size), @@ -331,7 +331,7 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page, if (!ret) { sgx_encl_ewb(encl->secs.epc_page, &secs_backing); - sgx_free_page(encl->secs.epc_page); + __sgx_free_page(encl->secs.epc_page); encl->secs.epc_page = NULL; } From patchwork Thu Oct 10 21:43:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11184523 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B744112B for ; Thu, 10 Oct 2019 21:43:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 6615A214E0 for ; Thu, 10 Oct 2019 21:43:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727463AbfJJVnE (ORCPT ); Thu, 10 Oct 2019 17:43:04 -0400 Received: from mga17.intel.com ([192.55.52.151]:35146 "EHLO mga17.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727437AbfJJVnE (ORCPT ); Thu, 10 Oct 2019 17:43:04 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Oct 2019 14:43:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,281,1566889200"; d="scan'208";a="194145500" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.41]) by fmsmga007.fm.intel.com with ESMTP; 10 Oct 2019 14:43:03 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: linux-sgx@vger.kernel.org Subject: [PATCH for_v23 8/9] x86/sgx: Don't update free page count if EPC section allocation fails Date: Thu, 10 Oct 2019 14:43:00 -0700 Message-Id: <20191010214301.25669-9-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20191010214301.25669-1-sean.j.christopherson@intel.com> References: <20191010214301.25669-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Update the number of free pages only after an EPC section is fully initialized, else the free page count will be left in a bogus state if allocation fails. Signed-off-by: Sean Christopherson --- arch/x86/kernel/cpu/sgx/main.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c index e22cdbb431a3..48a4f37b5b3c 100644 --- a/arch/x86/kernel/cpu/sgx/main.c +++ b/arch/x86/kernel/cpu/sgx/main.c @@ -216,9 +216,10 @@ static bool __init sgx_alloc_epc_section(u64 addr, u64 size, page->desc = (addr + (i << PAGE_SHIFT)) | index; list_add_tail(&page->list, §ion->unsanitized_page_list); - sgx_nr_free_pages++; } + sgx_nr_free_pages += nr_pages; + return true; err_out: From patchwork Thu Oct 10 21:43:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 11184533 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 417DF14ED for ; Thu, 10 Oct 2019 21:43:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2C1B4218AC for ; Thu, 10 Oct 2019 21:43:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727464AbfJJVnG (ORCPT ); Thu, 10 Oct 2019 17:43:06 -0400 Received: from mga17.intel.com ([192.55.52.151]:35146 "EHLO mga17.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726983AbfJJVnE (ORCPT ); Thu, 10 Oct 2019 17:43:04 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Oct 2019 14:43:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.67,281,1566889200"; d="scan'208";a="194145503" Received: from sjchrist-coffee.jf.intel.com ([10.54.74.41]) by fmsmga007.fm.intel.com with ESMTP; 10 Oct 2019 14:43:04 -0700 From: Sean Christopherson To: Jarkko Sakkinen Cc: linux-sgx@vger.kernel.org Subject: [PATCH for_v23 9/9] x86/sgx: Reinstate per EPC section free page counts Date: Thu, 10 Oct 2019 14:43:01 -0700 Message-Id: <20191010214301.25669-10-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20191010214301.25669-1-sean.j.christopherson@intel.com> References: <20191010214301.25669-1-sean.j.christopherson@intel.com> MIME-Version: 1.0 Sender: linux-sgx-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Track the free page count on a per EPC section basis so that the value is properly protected by the section's spinlock. As was pointed out when the change was proposed[*], using a global non-atomic counter to track the number of free EPC pages is not safe. The order of non-atomic reads and writes are not guaranteed, i.e. concurrent RMW operats can write stale data. This causes a variety of bad behavior, e.g. livelocks because the free page count wraps and causes the swap thread to stop reclaiming. Signed-off-by: Sean Christopherson --- arch/x86/kernel/cpu/sgx/main.c | 11 +++++------ arch/x86/kernel/cpu/sgx/reclaim.c | 4 ++-- arch/x86/kernel/cpu/sgx/sgx.h | 18 +++++++++++++++++- 3 files changed, 24 insertions(+), 9 deletions(-) diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c index 48a4f37b5b3c..cc87690fa1ec 100644 --- a/arch/x86/kernel/cpu/sgx/main.c +++ b/arch/x86/kernel/cpu/sgx/main.c @@ -13,18 +13,17 @@ struct sgx_epc_section sgx_epc_sections[SGX_MAX_EPC_SECTIONS]; int sgx_nr_epc_sections; -unsigned long sgx_nr_free_pages; static struct sgx_epc_page *__sgx_try_alloc_page(struct sgx_epc_section *section) { struct sgx_epc_page *page; - if (list_empty(§ion->page_list)) + if (!section->free_cnt) return NULL; page = list_first_entry(§ion->page_list, struct sgx_epc_page, list); list_del_init(&page->list); - sgx_nr_free_pages--; + section->free_cnt--; return page; } @@ -97,7 +96,7 @@ struct sgx_epc_page *sgx_alloc_page(void *owner, bool reclaim) schedule(); } - if (sgx_nr_free_pages < SGX_NR_LOW_PAGES) + if (!sgx_at_least_N_free_pages(SGX_NR_LOW_PAGES)) wake_up(&ksgxswapd_waitq); return entry; @@ -131,7 +130,7 @@ void __sgx_free_page(struct sgx_epc_page *page) spin_lock(§ion->lock); list_add_tail(&page->list, §ion->page_list); - sgx_nr_free_pages++; + section->free_cnt++; spin_unlock(§ion->lock); } @@ -218,7 +217,7 @@ static bool __init sgx_alloc_epc_section(u64 addr, u64 size, list_add_tail(&page->list, §ion->unsanitized_page_list); } - sgx_nr_free_pages += nr_pages; + section->free_cnt = nr_pages; return true; diff --git a/arch/x86/kernel/cpu/sgx/reclaim.c b/arch/x86/kernel/cpu/sgx/reclaim.c index 3f183dd0e653..8619141f4bed 100644 --- a/arch/x86/kernel/cpu/sgx/reclaim.c +++ b/arch/x86/kernel/cpu/sgx/reclaim.c @@ -68,7 +68,7 @@ static void sgx_sanitize_section(struct sgx_epc_section *section) static inline bool sgx_should_reclaim(void) { - return sgx_nr_free_pages < SGX_NR_HIGH_PAGES && + return !sgx_at_least_N_free_pages(SGX_NR_HIGH_PAGES) && !list_empty(&sgx_active_page_list); } @@ -430,7 +430,7 @@ void sgx_reclaim_pages(void) section = sgx_epc_section(epc_page); spin_lock(§ion->lock); list_add_tail(&epc_page->list, §ion->page_list); - sgx_nr_free_pages++; + section->free_cnt++; spin_unlock(§ion->lock); } } diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h index 87e375e8c25e..c7f0277299f6 100644 --- a/arch/x86/kernel/cpu/sgx/sgx.h +++ b/arch/x86/kernel/cpu/sgx/sgx.h @@ -30,6 +30,7 @@ struct sgx_epc_page { struct sgx_epc_section { unsigned long pa; void *va; + unsigned long free_cnt; struct list_head page_list; struct list_head unsanitized_page_list; spinlock_t lock; @@ -73,12 +74,27 @@ static inline void *sgx_epc_addr(struct sgx_epc_page *page) #define SGX_NR_HIGH_PAGES 64 extern int sgx_nr_epc_sections; -extern unsigned long sgx_nr_free_pages; extern struct task_struct *ksgxswapd_tsk; extern struct wait_queue_head(ksgxswapd_waitq); extern struct list_head sgx_active_page_list; extern spinlock_t sgx_active_page_list_lock; +static inline bool sgx_at_least_N_free_pages(unsigned long threshold) +{ + struct sgx_epc_section *section; + unsigned long free_cnt = 0; + int i; + + for (i = 0; i < sgx_nr_epc_sections; i++) { + section = &sgx_epc_sections[i]; + free_cnt += section->free_cnt; + if (free_cnt >= threshold) + return true; + } + + return false; +} + bool __init sgx_page_reclaimer_init(void); void sgx_mark_page_reclaimable(struct sgx_epc_page *page); void sgx_reclaim_pages(void);