From patchwork Tue Aug 1 22:36:16 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sean Christopherson X-Patchwork-Id: 9875633 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id E256260390 for ; Tue, 1 Aug 2017 22:36:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D502228735 for ; Tue, 1 Aug 2017 22:36:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CA28F28742; Tue, 1 Aug 2017 22:36:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4E01E2873A for ; Tue, 1 Aug 2017 22:36:31 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id CD19221E1DAEB; Tue, 1 Aug 2017 15:34:20 -0700 (PDT) X-Original-To: intel-sgx-kernel-dev@lists.01.org Delivered-To: intel-sgx-kernel-dev@lists.01.org Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id A176121D2DCEE for ; Tue, 1 Aug 2017 15:34:19 -0700 (PDT) Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga104.jf.intel.com with ESMTP; 01 Aug 2017 15:36:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.41,308,1498546800"; d="scan'208";a="134743809" Received: from sjchrist-ts.jf.intel.com ([10.54.74.20]) by fmsmga006.fm.intel.com with ESMTP; 01 Aug 2017 15:36:27 -0700 From: Sean Christopherson To: intel-sgx-kernel-dev@lists.01.org Date: Tue, 1 Aug 2017 15:36:16 -0700 Message-Id: <1501626977-6983-2-git-send-email-sean.j.christopherson@intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1501626977-6983-1-git-send-email-sean.j.christopherson@intel.com> References: <1501626977-6983-1-git-send-email-sean.j.christopherson@intel.com> Subject: [intel-sgx-kernel-dev] [PATCH v2 1/2] intel_sgx: freeze the swap thread during suspend/hibernate X-BeenThere: intel-sgx-kernel-dev@lists.01.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Project: Intel® Software Guard Extensions for Linux*: https://01.org/intel-software-guard-extensions" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: intel-sgx-kernel-dev-bounces@lists.01.org Sender: "intel-sgx-kernel-dev" X-Virus-Scanned: ClamAV using ClamSMTP Freeze the swap thread, ksgxswapd, during suspend/hibernate instead of stopping and re-starting the thread on suspend/resume. Freezing, as opposed to stopping/running, eliminates the possibility of kthread_run failing during resume, which in turn eliminates the need to handle the scenario where there is no swap thread running when an enclave needs to page in EPC memory. Previously, sgx_pm_resume didn't check the return of kthread_run, which can fail with -ENOMEM. Upon failure, SGX applications would hang due to the lack of a kernel thread to perform EPC swapping, and the kernel would crash on the next system suspend due to passing an error pointer to kthread_stop. Signed-off-by: Sean Christopherson --- drivers/platform/x86/intel_sgx/sgx.h | 2 -- drivers/platform/x86/intel_sgx/sgx_main.c | 11 +---------- drivers/platform/x86/intel_sgx/sgx_page_cache.c | 15 ++++++++++----- 3 files changed, 11 insertions(+), 17 deletions(-) diff --git a/drivers/platform/x86/intel_sgx/sgx.h b/drivers/platform/x86/intel_sgx/sgx.h index b5b21d4..b1d0ece 100644 --- a/drivers/platform/x86/intel_sgx/sgx.h +++ b/drivers/platform/x86/intel_sgx/sgx.h @@ -215,9 +215,7 @@ void sgx_tgid_ctx_release(struct kref *ref); extern struct mutex sgx_tgid_ctx_mutex; extern struct list_head sgx_tgid_ctx_list; -extern struct task_struct *ksgxswapd_tsk; -int ksgxswapd(void *p); int sgx_add_epc_bank(resource_size_t start, unsigned long size); int sgx_page_cache_init(void); void sgx_page_cache_teardown(void); diff --git a/drivers/platform/x86/intel_sgx/sgx_main.c b/drivers/platform/x86/intel_sgx/sgx_main.c index 32214cd..8469633 100644 --- a/drivers/platform/x86/intel_sgx/sgx_main.c +++ b/drivers/platform/x86/intel_sgx/sgx_main.c @@ -164,9 +164,6 @@ static int sgx_pm_suspend(struct device *dev) struct sgx_tgid_ctx *ctx; struct sgx_encl *encl; - kthread_stop(ksgxswapd_tsk); - ksgxswapd_tsk = NULL; - list_for_each_entry(ctx, &sgx_tgid_ctx_list, list) { list_for_each_entry(encl, &ctx->encl_list, encl_list) { sgx_invalidate(encl, false); @@ -178,13 +175,7 @@ static int sgx_pm_suspend(struct device *dev) return 0; } -static int sgx_pm_resume(struct device *dev) -{ - ksgxswapd_tsk = kthread_run(ksgxswapd, NULL, "kswapd"); - return 0; -} - -static SIMPLE_DEV_PM_OPS(sgx_drv_pm, sgx_pm_suspend, sgx_pm_resume); +static SIMPLE_DEV_PM_OPS(sgx_drv_pm, sgx_pm_suspend, NULL); static int sgx_dev_init(struct device *dev) { diff --git a/drivers/platform/x86/intel_sgx/sgx_page_cache.c b/drivers/platform/x86/intel_sgx/sgx_page_cache.c index 356f2c2..f1553324 100644 --- a/drivers/platform/x86/intel_sgx/sgx_page_cache.c +++ b/drivers/platform/x86/intel_sgx/sgx_page_cache.c @@ -78,7 +78,7 @@ static unsigned int sgx_nr_total_epc_pages; static unsigned int sgx_nr_free_pages; static unsigned int sgx_nr_low_pages = SGX_NR_LOW_EPC_PAGES_DEFAULT; static unsigned int sgx_nr_high_pages; -struct task_struct *ksgxswapd_tsk; +static struct task_struct *ksgxswapd_tsk; static DECLARE_WAIT_QUEUE_HEAD(ksgxswapd_waitq); @@ -368,12 +368,17 @@ static void sgx_swap_pages(unsigned long nr_to_scan) kref_put(&ctx->refcount, sgx_tgid_ctx_release); } -int ksgxswapd(void *p) +static int ksgxswapd(void *p) { + set_freezable(); + while (!kthread_should_stop()) { - wait_event_interruptible(ksgxswapd_waitq, - kthread_should_stop() || - sgx_nr_free_pages < sgx_nr_high_pages); + if (try_to_freeze()) + continue; + + wait_event_freezable(ksgxswapd_waitq, + kthread_should_stop() || + sgx_nr_free_pages < sgx_nr_high_pages); if (sgx_nr_free_pages < sgx_nr_high_pages) sgx_swap_pages(SGX_NR_SWAP_CLUSTER_MAX);