From patchwork Thu Dec 1 20:56:28 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jarkko Sakkinen X-Patchwork-Id: 9456897 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 3180D60585 for ; Thu, 1 Dec 2016 20:56:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 233312841E for ; Thu, 1 Dec 2016 20:56:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 182162851B; Thu, 1 Dec 2016 20:56:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C9F642841E for ; Thu, 1 Dec 2016 20:56:58 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id E564181F1C for ; Thu, 1 Dec 2016 12:56:58 -0800 (PST) X-Original-To: intel-sgx-kernel-dev@lists.01.org Delivered-To: intel-sgx-kernel-dev@lists.01.org Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id D842F81F1C for ; Thu, 1 Dec 2016 12:56:57 -0800 (PST) Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga104.jf.intel.com with ESMTP; 01 Dec 2016 12:56:57 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.33,726,1477983600"; d="scan'208";a="907744984" Received: from wmcsween-mobl.ger.corp.intel.com (HELO localhost) ([10.252.12.30]) by orsmga003.jf.intel.com with ESMTP; 01 Dec 2016 12:56:55 -0800 From: Jarkko Sakkinen To: intel-sgx-kernel-dev@lists.01.org Date: Thu, 1 Dec 2016 22:56:28 +0200 Message-Id: <20161201205632.8593-5-jarkko.sakkinen@linux.intel.com> X-Mailer: git-send-email 2.9.3 In-Reply-To: <20161201205632.8593-1-jarkko.sakkinen@linux.intel.com> References: <20161201205632.8593-1-jarkko.sakkinen@linux.intel.com> Subject: [intel-sgx-kernel-dev] [PATCH v4 4/8] intel_sgx: lock the enclave for the entire EPC eviction flow X-BeenThere: intel-sgx-kernel-dev@lists.01.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Project: Intel® Software Guard Extensions for Linux*: https://01.org/intel-software-guard-extensions" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-sgx-kernel-dev-bounces@lists.01.org Sender: "intel-sgx-kernel-dev" X-Virus-Scanned: ClamAV using ClamSMTP From: Sean Christopherson The SGX architecture does not allow multiple ETRACK sequences to be in flight at the same time if the enclave being tracked has one or more active threads in the enclave. A second ETRACK will fail due to PREV_TRK_INCMPL if it is attempted before EWB is executed on all EBLOCKed pages (in the presence of active enclave threads). Currently the driver releases the enclave's lock after ETRACK and then reacquires the lock prior to EWB. Releasing the lock from the thread that performed ETRACK, T1, allows a different thread, T2, to acquire the lock for its own ETRACK. T2's ETRACK will fail due to the architectural restrictions if a third thread, T3, is executing inside the enclave whose pages are being swapped. T3: T1: lock_mutex(E1) T1: ETRACK(E1) T1: unlock_mutex(E1) T2: lock_mutex(E1) T2: ETRACK(E1) fails ETRACK failing is not fatal in and of itself, but the driver is not designed to handle an ETRACK failure (BUG_ON non-zero return code). Rather than modifying the driver to handle the failing path, e.g by kicking T3 out of the enclave with an IPI, simply lock the enclave for the entire ETRACK->EWB sequence to prevent multiple concurrent ETRACKs on a single enclave. Holding the lock for ETRACK->EWB does not negatively impact performance in any way. Signed-off-by: Sean Christopherson Reviewed-by: Jarkko Sakkinen Tested-by: Jarkko Sakkinen --- drivers/platform/x86/intel_sgx_page_cache.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/drivers/platform/x86/intel_sgx_page_cache.c b/drivers/platform/x86/intel_sgx_page_cache.c index 8b1cc82..c4865d2 100644 --- a/drivers/platform/x86/intel_sgx_page_cache.c +++ b/drivers/platform/x86/intel_sgx_page_cache.c @@ -261,15 +261,14 @@ static void sgx_write_pages(struct sgx_encl *encl, struct list_head *src) return; } - /* EBLOCK */ + mutex_lock(&encl->lock); + /* EBLOCK */ list_for_each_entry_safe(entry, tmp, src, load_list) { - mutex_lock(&encl->lock); evma = sgx_find_vma(encl, entry->addr); if (!evma) { list_del(&entry->load_list); sgx_free_encl_page(entry, encl, 0); - mutex_unlock(&encl->lock); continue; } @@ -278,25 +277,18 @@ static void sgx_write_pages(struct sgx_encl *encl, struct list_head *src) list_del(&entry->load_list); list_add_tail(&entry->load_list, &encl->load_list); entry->flags &= ~SGX_ENCL_PAGE_RESERVED; - mutex_unlock(&encl->lock); continue; } zap_vma_ptes(evma, entry->addr, PAGE_SIZE); sgx_eblock(entry->epc_page); cnt++; - mutex_unlock(&encl->lock); } /* ETRACK */ - - mutex_lock(&encl->lock); sgx_etrack(encl->secs_page.epc_page); - mutex_unlock(&encl->lock); /* EWB */ - - mutex_lock(&encl->lock); i = 0; while (!list_empty(src)) {