From patchwork Wed Nov 30 14:23:23 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jarkko Sakkinen X-Patchwork-Id: 9454465 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 5CC456071E for ; Wed, 30 Nov 2016 14:24:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5B97228443 for ; Wed, 30 Nov 2016 14:24:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5071E28447; Wed, 30 Nov 2016 14:24:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D566D28443 for ; Wed, 30 Nov 2016 14:24:03 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id EADE781D97 for ; Wed, 30 Nov 2016 06:24:03 -0800 (PST) X-Original-To: intel-sgx-kernel-dev@lists.01.org Delivered-To: intel-sgx-kernel-dev@lists.01.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 56D6581D97 for ; Wed, 30 Nov 2016 06:24:03 -0800 (PST) Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga105.fm.intel.com with ESMTP; 30 Nov 2016 06:24:02 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.31,574,1473145200"; d="scan'208"; a="1066160219" Received: from pstacey-mobl2.ger.corp.intel.com (HELO localhost) ([10.252.3.89]) by orsmga001.jf.intel.com with ESMTP; 30 Nov 2016 06:24:00 -0800 From: Jarkko Sakkinen To: intel-sgx-kernel-dev@lists.01.org Date: Wed, 30 Nov 2016 16:23:23 +0200 Message-Id: <20161130142323.5324-8-jarkko.sakkinen@linux.intel.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20161130142323.5324-1-jarkko.sakkinen@linux.intel.com> References: <20161130142323.5324-1-jarkko.sakkinen@linux.intel.com> Subject: [intel-sgx-kernel-dev] [PATCH v3 7/7] intel_sgx: lock the enclave for the entire EPC eviction flow X-BeenThere: intel-sgx-kernel-dev@lists.01.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Project: Intel® Software Guard Extensions for Linux*: https://01.org/intel-software-guard-extensions" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-sgx-kernel-dev-bounces@lists.01.org Sender: "intel-sgx-kernel-dev" X-Virus-Scanned: ClamAV using ClamSMTP From: Sean Christopherson The SGX architecture does not allow multiple ETRACK sequences to be in flight at the same time if the enclave being tracked has one or more active threads in the enclave. A second ETRACK will fail due to PREV_TRK_INCMPL if it is attempted before EWB is executed on all EBLOCKed pages (in the presence of active enclave threads). Currently the driver releases the enclave's lock after ETRACK and then reacquires the lock prior to EWB. Releasing the lock from the thread that performed ETRACK, T1, allows a different thread, T2, to acquire the lock for its own ETRACK. T2's ETRACK will fail due to the architectural restrictions if a third thread, T3, is executing inside the enclave whose pages are being swapped. T3: T1: lock_mutex(E1) T1: ETRACK(E1) T1: unlock_mutex(E1) T2: lock_mutex(E1) T2: ETRACK(E1) fails ETRACK failing is not fatal in and of itself, but the driver is not designed to handle an ETRACK failure (BUG_ON non-zero return code). Rather than modifying the driver to handle the failing path, e.g by kicking T3 out of the enclave with an IPI, simply lock the enclave for the entire ETRACK->EWB sequence to prevent multiple concurrent ETRACKs on a single enclave. Holding the lock for ETRACK->EWB does not negatively impact performance in any way. Signed-off-by: Sean Christopherson --- drivers/platform/x86/intel_sgx_page_cache.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/drivers/platform/x86/intel_sgx_page_cache.c b/drivers/platform/x86/intel_sgx_page_cache.c index 9492e91..091d0d3 100644 --- a/drivers/platform/x86/intel_sgx_page_cache.c +++ b/drivers/platform/x86/intel_sgx_page_cache.c @@ -293,15 +293,14 @@ static void sgx_write_pages(struct sgx_encl *encl, struct list_head *src) return; } - /* EBLOCK */ + mutex_lock(&encl->lock); + /* EBLOCK */ list_for_each_entry_safe(entry, tmp, src, load_list) { - mutex_lock(&encl->lock); evma = sgx_find_vma(encl, entry->addr); if (!evma) { list_del(&entry->load_list); sgx_free_encl_page(entry, encl, 0); - mutex_unlock(&encl->lock); continue; } @@ -310,25 +309,18 @@ static void sgx_write_pages(struct sgx_encl *encl, struct list_head *src) list_del(&entry->load_list); list_add_tail(&entry->load_list, &encl->load_list); entry->flags &= ~SGX_ENCL_PAGE_RESERVED; - mutex_unlock(&encl->lock); continue; } zap_vma_ptes(evma, entry->addr, PAGE_SIZE); sgx_eblock(entry->epc_page); cnt++; - mutex_unlock(&encl->lock); } /* ETRACK */ - - mutex_lock(&encl->lock); sgx_etrack(encl->secs_page.epc_page); - mutex_unlock(&encl->lock); /* EWB */ - - mutex_lock(&encl->lock); i = 0; while (!list_empty(src)) {