From patchwork Fri Dec 2 20:00:14 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jarkko Sakkinen X-Patchwork-Id: 9459109 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D6CBC6074E for ; Fri, 2 Dec 2016 20:00:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C3D9D285AE for ; Fri, 2 Dec 2016 20:00:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B8E00285B2; Fri, 2 Dec 2016 20:00:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 60B57285AE for ; Fri, 2 Dec 2016 20:00:37 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 7F93881C77 for ; Fri, 2 Dec 2016 12:00:37 -0800 (PST) X-Original-To: intel-sgx-kernel-dev@lists.01.org Delivered-To: intel-sgx-kernel-dev@lists.01.org Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 3CFA981C77 for ; Fri, 2 Dec 2016 12:00:36 -0800 (PST) Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga105.jf.intel.com with ESMTP; 02 Dec 2016 12:00:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,288,1477983600"; d="scan'208";a="38391316" Received: from rmullen-mobl.amr.corp.intel.com (HELO localhost) ([10.252.2.199]) by fmsmga006.fm.intel.com with ESMTP; 02 Dec 2016 12:00:34 -0800 From: Jarkko Sakkinen To: intel-sgx-kernel-dev@lists.01.org Date: Fri, 2 Dec 2016 22:00:14 +0200 Message-Id: <20161202200018.25552-5-jarkko.sakkinen@linux.intel.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20161202200018.25552-1-jarkko.sakkinen@linux.intel.com> References: <20161202200018.25552-1-jarkko.sakkinen@linux.intel.com> Subject: [intel-sgx-kernel-dev] [PATCH v5 4/8] intel_sgx: lock the enclave for the entire EPC eviction flow X-BeenThere: intel-sgx-kernel-dev@lists.01.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Project: Intel® Software Guard Extensions for Linux*: https://01.org/intel-software-guard-extensions" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-sgx-kernel-dev-bounces@lists.01.org Sender: "intel-sgx-kernel-dev" X-Virus-Scanned: ClamAV using ClamSMTP From: Sean Christopherson The SGX architecture does not allow multiple ETRACK sequences to be in flight at the same time if the enclave being tracked has one or more active threads in the enclave. A second ETRACK will fail due to PREV_TRK_INCMPL if it is attempted before EWB is executed on all EBLOCKed pages (in the presence of active enclave threads). Currently the driver releases the enclave's lock after ETRACK and then reacquires the lock prior to EWB. Releasing the lock from the thread that performed ETRACK, T1, allows a different thread, T2, to acquire the lock for its own ETRACK. T2's ETRACK will fail due to the architectural restrictions if a third thread, T3, is executing inside the enclave whose pages are being swapped. T3: T1: lock_mutex(E1) T1: ETRACK(E1) T1: unlock_mutex(E1) T2: lock_mutex(E1) T2: ETRACK(E1) fails ETRACK failing is not fatal in and of itself, but the driver is not designed to handle an ETRACK failure (BUG_ON non-zero return code). Rather than modifying the driver to handle the failing path, e.g by kicking T3 out of the enclave with an IPI, simply lock the enclave for the entire ETRACK->EWB sequence to prevent multiple concurrent ETRACKs on a single enclave. Holding the lock for ETRACK->EWB does not negatively impact performance in any way. Signed-off-by: Sean Christopherson Reviewed-by: Jarkko Sakkinen --- drivers/platform/x86/intel_sgx_page_cache.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/drivers/platform/x86/intel_sgx_page_cache.c b/drivers/platform/x86/intel_sgx_page_cache.c index 8b1cc82..c4865d2 100644 --- a/drivers/platform/x86/intel_sgx_page_cache.c +++ b/drivers/platform/x86/intel_sgx_page_cache.c @@ -261,15 +261,14 @@ static void sgx_write_pages(struct sgx_encl *encl, struct list_head *src) return; } - /* EBLOCK */ + mutex_lock(&encl->lock); + /* EBLOCK */ list_for_each_entry_safe(entry, tmp, src, load_list) { - mutex_lock(&encl->lock); evma = sgx_find_vma(encl, entry->addr); if (!evma) { list_del(&entry->load_list); sgx_free_encl_page(entry, encl, 0); - mutex_unlock(&encl->lock); continue; } @@ -278,25 +277,18 @@ static void sgx_write_pages(struct sgx_encl *encl, struct list_head *src) list_del(&entry->load_list); list_add_tail(&entry->load_list, &encl->load_list); entry->flags &= ~SGX_ENCL_PAGE_RESERVED; - mutex_unlock(&encl->lock); continue; } zap_vma_ptes(evma, entry->addr, PAGE_SIZE); sgx_eblock(entry->epc_page); cnt++; - mutex_unlock(&encl->lock); } /* ETRACK */ - - mutex_lock(&encl->lock); sgx_etrack(encl->secs_page.epc_page); - mutex_unlock(&encl->lock); /* EWB */ - - mutex_lock(&encl->lock); i = 0; while (!list_empty(src)) {