From patchwork Mon Jul 5 14:36:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jarkko Sakkinen X-Patchwork-Id: 12359221 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0924DC07E9C for ; Mon, 5 Jul 2021 14:37:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E49BD6195E for ; Mon, 5 Jul 2021 14:37:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231467AbhGEOjh (ORCPT ); Mon, 5 Jul 2021 10:39:37 -0400 Received: from mail.kernel.org ([198.145.29.99]:43794 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230285AbhGEOjg (ORCPT ); Mon, 5 Jul 2021 10:39:36 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 5E68A61283; Mon, 5 Jul 2021 14:36:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1625495819; bh=y52tpphSpDoVZ3rjuGEqTXUglt2049AtGx/h6xY+gf8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BnUyWaN7RVfSHU1mzMOfFIqvHaVehl52zd+zvY0vq1dXqKoC1twU5ozt7twXw0W8T ROqual2tbcjAz0ieNmNnAD9fNBZYzv9+sqAvdL1NJHi71g7FFdskjMNKv7UW6YlQb3 ouXuyY70bDLpdxELAODISC5/E0MPPbEZWB/Ajceo26q/sb/I2aiUFfvlo73uwq7SJh kKxLK9xS7/LmTUQ4Fduc/fK60ZV92Ny6hMyvO7ygXYGNIv4W6aq5wX97sqXGoFbaRi pqsznarbt2jrzXY+i8n1HC8CwjlEK5dimtx/RgEWKAYn6S0P7ZLXZ7Ms8lON6Dn3gE flL52tsYRvnBg== From: Jarkko Sakkinen To: Shuah Khan Cc: linux-kselftest@vger.kernel.org, linux-sgx@vger.kernel.org, Reinette Chatre , Borislav Petkov , Jarkko Sakkinen , Dave Hansen , Thomas Gleixner , Ingo Molnar , x86@kernel.org, "H. Peter Anvin" , Jonathan Corbet , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org Subject: [PATCH 1/4] x86/sgx: Add sgx_nr_all_pages to the debugfs Date: Mon, 5 Jul 2021 17:36:49 +0300 Message-Id: <20210705143652.116125-2-jarkko@kernel.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210705143652.116125-1-jarkko@kernel.org> References: <20210705143652.116125-1-jarkko@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Create /sys/kernel/debug/x86/sgx_nr_all_pages, which reports total number of EPC pages available in the system. Signed-off-by: Jarkko Sakkinen --- Documentation/x86/sgx.rst | 9 +++++++++ arch/x86/kernel/cpu/sgx/main.c | 10 +++++++++- 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/Documentation/x86/sgx.rst b/Documentation/x86/sgx.rst index dd0ac96ff9ef..3a8fefdebee0 100644 --- a/Documentation/x86/sgx.rst +++ b/Documentation/x86/sgx.rst @@ -250,3 +250,12 @@ user wants to deploy SGX applications both on the host and in guests on the same machine, the user should reserve enough EPC (by taking out total virtual EPC size of all SGX VMs from the physical EPC size) for host SGX applications so they can run with acceptable performance. + +SGX debugging +============= + +The total number of available EPC pages can observed by: + + # mount -t debugfs debugfs /sys/kernel/debug + # cat /sys/kernel/debug/x86/sgx_nr_all_pages + diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c index 63d3de02bbcc..43e4b6215e62 100644 --- a/arch/x86/kernel/cpu/sgx/main.c +++ b/arch/x86/kernel/cpu/sgx/main.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* Copyright(c) 2016-20 Intel Corporation. */ +#include #include #include #include @@ -28,7 +29,10 @@ static DECLARE_WAIT_QUEUE_HEAD(ksgxd_waitq); static LIST_HEAD(sgx_active_page_list); static DEFINE_SPINLOCK(sgx_reclaimer_lock); -/* The free page list lock protected variables prepend the lock. */ +/* The number of EPC pages in total in all nodes. */ +static unsigned long sgx_nr_all_pages; + +/* The number of free EPC pages in all nodes. */ static unsigned long sgx_nr_free_pages; /* Nodes with one or more EPC sections. */ @@ -656,6 +660,8 @@ static bool __init sgx_setup_epc_section(u64 phys_addr, u64 size, list_add_tail(§ion->pages[i].list, &sgx_dirty_page_list); } + sgx_nr_all_pages += nr_pages; + return true; } @@ -823,6 +829,8 @@ static int __init sgx_init(void) if (sgx_vepc_init() && ret) goto err_provision; + debugfs_create_ulong("sgx_nr_all_pages", 0444, arch_debugfs_dir, &sgx_nr_all_pages); + return 0; err_provision: From patchwork Mon Jul 5 14:36:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jarkko Sakkinen X-Patchwork-Id: 12359223 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04969C11F66 for ; Mon, 5 Jul 2021 14:37:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E55006195F for ; Mon, 5 Jul 2021 14:37:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231517AbhGEOjk (ORCPT ); Mon, 5 Jul 2021 10:39:40 -0400 Received: from mail.kernel.org ([198.145.29.99]:43860 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231511AbhGEOjj (ORCPT ); Mon, 5 Jul 2021 10:39:39 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id E817C61955; Mon, 5 Jul 2021 14:37:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1625495822; bh=cRp9ny7eyKyuD0MO2TdfPzp9DciYKWy2hpAippP5l1M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MOvW33LflC0rIYgHmOKdap42/n3ieZr2zx1FGV6tPK4wS1ToslmmoOSRAeAiLYTqj s61uoiZIoM6fV7JOOZqWp86ArFS5j4cjh/07Pc8qMPlZIWlk0JRiLOde8vAs9b7Ykn T1I/NHvENc89Kh7lrIcJ2T8DffQrWo29D5VLelHWR/Q2+O0J/9ZM1NTScectOaPmjr i2mXZ7kx2l/EtFJ2e+vkI0getpG5qGH+M7ah4g4cGooaD/JvPwjxZXvkVmXafA0MyH 5aculBgEFThJM+h/eow70W1cALwszuKJHSFjnN2+sUKtRu8aMqX9QE3k04PZESQfUd 6Tu3vtkn+J8sw== From: Jarkko Sakkinen To: Shuah Khan Cc: linux-kselftest@vger.kernel.org, linux-sgx@vger.kernel.org, Reinette Chatre , Borislav Petkov , Tianjia Zhang , Jarkko Sakkinen , Dave Hansen , Jethro Beekman , Borislav Petkov , linux-kernel@vger.kernel.org Subject: [PATCH 2/4] selftests/sgx: Fix Q1 and Q2 calculation in sigstruct.c Date: Mon, 5 Jul 2021 17:36:50 +0300 Message-Id: <20210705143652.116125-3-jarkko@kernel.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210705143652.116125-1-jarkko@kernel.org> References: <20210705143652.116125-1-jarkko@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org From: Tianjia Zhang Q1 and Q2 are numbers with *maximum* length of 384 bytes. If the calculated length of Q1 and Q2 is less than 384 bytes, things will go wrong. E.g. if Q2 is 383 bytes, then 1. The bytes of q2 are copied to sigstruct->q2 in calc_q1q2(). 2. The entire sigstruct->q2 is reversed, which results it being 256 * Q2, given that the last byte of sigstruct->q2 is added to before the bytes given by calc_q1q2(). Either change in key or measurement can trigger the bug. E.g. an unmeasured heap could cause a devastating change in Q1 or Q2. Reverse exactly the bytes of Q1 and Q2 in calc_q1q2() before returning to the caller. Fixes: 2adcba79e69d ("selftests/x86: Add a selftest for SGX") Link: https://lore.kernel.org/linux-sgx/20210301051836.30738-1-tianjia.zhang@linux.alibaba.com/ Signed-off-by: Tianjia Zhang Signed-off-by: Jarkko Sakkinen Reviewed-by: Dave Hansen --- The original patch did a bad job explaining the code change but it turned out making sense. I wrote a new description. v3: - Adjusted the fixes tag to refer to the correct commit v2: - Added a fixes tag. tools/testing/selftests/sgx/sigstruct.c | 41 +++++++++++++------------ 1 file changed, 21 insertions(+), 20 deletions(-) diff --git a/tools/testing/selftests/sgx/sigstruct.c b/tools/testing/selftests/sgx/sigstruct.c index dee7a3d6c5a5..92bbc5a15c39 100644 --- a/tools/testing/selftests/sgx/sigstruct.c +++ b/tools/testing/selftests/sgx/sigstruct.c @@ -55,10 +55,27 @@ static bool alloc_q1q2_ctx(const uint8_t *s, const uint8_t *m, return true; } +static void reverse_bytes(void *data, int length) +{ + int i = 0; + int j = length - 1; + uint8_t temp; + uint8_t *ptr = data; + + while (i < j) { + temp = ptr[i]; + ptr[i] = ptr[j]; + ptr[j] = temp; + i++; + j--; + } +} + static bool calc_q1q2(const uint8_t *s, const uint8_t *m, uint8_t *q1, uint8_t *q2) { struct q1q2_ctx ctx; + int len; if (!alloc_q1q2_ctx(s, m, &ctx)) { fprintf(stderr, "Not enough memory for Q1Q2 calculation\n"); @@ -89,8 +106,10 @@ static bool calc_q1q2(const uint8_t *s, const uint8_t *m, uint8_t *q1, goto out; } - BN_bn2bin(ctx.q1, q1); - BN_bn2bin(ctx.q2, q2); + len = BN_bn2bin(ctx.q1, q1); + reverse_bytes(q1, len); + len = BN_bn2bin(ctx.q2, q2); + reverse_bytes(q2, len); free_q1q2_ctx(&ctx); return true; @@ -152,22 +171,6 @@ static RSA *gen_sign_key(void) return key; } -static void reverse_bytes(void *data, int length) -{ - int i = 0; - int j = length - 1; - uint8_t temp; - uint8_t *ptr = data; - - while (i < j) { - temp = ptr[i]; - ptr[i] = ptr[j]; - ptr[j] = temp; - i++; - j--; - } -} - enum mrtags { MRECREATE = 0x0045544145524345, MREADD = 0x0000000044444145, @@ -367,8 +370,6 @@ bool encl_measure(struct encl *encl) /* BE -> LE */ reverse_bytes(sigstruct->signature, SGX_MODULUS_SIZE); reverse_bytes(sigstruct->modulus, SGX_MODULUS_SIZE); - reverse_bytes(sigstruct->q1, SGX_MODULUS_SIZE); - reverse_bytes(sigstruct->q2, SGX_MODULUS_SIZE); EVP_MD_CTX_destroy(ctx); RSA_free(key); From patchwork Mon Jul 5 14:36:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jarkko Sakkinen X-Patchwork-Id: 12359225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80807C07E9C for ; Mon, 5 Jul 2021 14:37:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 69B0A61966 for ; Mon, 5 Jul 2021 14:37:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231511AbhGEOjm (ORCPT ); Mon, 5 Jul 2021 10:39:42 -0400 Received: from mail.kernel.org ([198.145.29.99]:43972 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231539AbhGEOjm (ORCPT ); Mon, 5 Jul 2021 10:39:42 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 8FCBD6195A; Mon, 5 Jul 2021 14:37:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1625495825; bh=4dKc/HZAppIt8IkwQuN2ogmAZCk+GY5EwArJF9/nIh4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tQiBodzzT8uc6DFESjHyHpRlZ0UwxQLH+GyBaxMCT97GQGiEXMRBrT008nNlX+pqt OtepsbvOGaa8LUu9b/9t5aer9IBgNV6C7MiHNYnXSNb+ZXS2MrpzOI3BV96X6aT0+L usKDl62RNuZo1j6XgPmwk14ipFIA0Q0O4v9ErLhGl4IBDbbMBd8kAQyaih2NTUQqnc 3jKkJMSfTLwfrTqCWonNCWog+3KyBGDZtLFFCT+HHhlxl26xZwdOVoO7+LxzR2npIU +0rl4+ksS6BIC/TPzgFMnFrALCLEDk/wigx7j2RaLwxKh5RecmRHy7e1CWvdx05sdK b8kRfr9Tgejqw== From: Jarkko Sakkinen To: Shuah Khan Cc: linux-kselftest@vger.kernel.org, linux-sgx@vger.kernel.org, Reinette Chatre , Borislav Petkov , Jarkko Sakkinen , Dave Hansen , linux-kernel@vger.kernel.org Subject: [PATCH 3/4] selftests/sgx: Assign source for each segment Date: Mon, 5 Jul 2021 17:36:51 +0300 Message-Id: <20210705143652.116125-4-jarkko@kernel.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210705143652.116125-1-jarkko@kernel.org> References: <20210705143652.116125-1-jarkko@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Define source per segment so that enclave pages can be added from different sources, e.g. anonymous VMA for zero pages. In other words, add 'src' field to struct encl_segment, and assign it to 'encl->src' for pages inherited from the enclave binary. Signed-off-by: Jarkko Sakkinen --- tools/testing/selftests/sgx/load.c | 5 +++-- tools/testing/selftests/sgx/main.h | 1 + tools/testing/selftests/sgx/sigstruct.c | 8 ++++---- 3 files changed, 8 insertions(+), 6 deletions(-) diff --git a/tools/testing/selftests/sgx/load.c b/tools/testing/selftests/sgx/load.c index 00928be57fc4..9946fab2a3d6 100644 --- a/tools/testing/selftests/sgx/load.c +++ b/tools/testing/selftests/sgx/load.c @@ -107,7 +107,7 @@ static bool encl_ioc_add_pages(struct encl *encl, struct encl_segment *seg) memset(&secinfo, 0, sizeof(secinfo)); secinfo.flags = seg->flags; - ioc.src = (uint64_t)encl->src + seg->offset; + ioc.src = (uint64_t)seg->src; ioc.offset = seg->offset; ioc.length = seg->size; ioc.secinfo = (unsigned long)&secinfo; @@ -226,6 +226,7 @@ bool encl_load(const char *path, struct encl *encl) if (j == 0) { src_offset = phdr->p_offset & PAGE_MASK; + encl->src = encl->bin + src_offset; seg->prot = PROT_READ | PROT_WRITE; seg->flags = SGX_PAGE_TYPE_TCS << 8; @@ -238,13 +239,13 @@ bool encl_load(const char *path, struct encl *encl) seg->offset = (phdr->p_offset & PAGE_MASK) - src_offset; seg->size = (phdr->p_filesz + PAGE_SIZE - 1) & PAGE_MASK; + seg->src = encl->src + seg->offset; j++; } assert(j == encl->nr_segments); - encl->src = encl->bin + src_offset; encl->src_size = encl->segment_tbl[j - 1].offset + encl->segment_tbl[j - 1].size; diff --git a/tools/testing/selftests/sgx/main.h b/tools/testing/selftests/sgx/main.h index 68672fd86cf9..452d11dc4889 100644 --- a/tools/testing/selftests/sgx/main.h +++ b/tools/testing/selftests/sgx/main.h @@ -7,6 +7,7 @@ #define MAIN_H struct encl_segment { + void *src; off_t offset; size_t size; unsigned int prot; diff --git a/tools/testing/selftests/sgx/sigstruct.c b/tools/testing/selftests/sgx/sigstruct.c index 92bbc5a15c39..202a96fd81bf 100644 --- a/tools/testing/selftests/sgx/sigstruct.c +++ b/tools/testing/selftests/sgx/sigstruct.c @@ -289,14 +289,14 @@ static bool mrenclave_eextend(EVP_MD_CTX *ctx, uint64_t offset, static bool mrenclave_segment(EVP_MD_CTX *ctx, struct encl *encl, struct encl_segment *seg) { - uint64_t end = seg->offset + seg->size; + uint64_t end = seg->size; uint64_t offset; - for (offset = seg->offset; offset < end; offset += PAGE_SIZE) { - if (!mrenclave_eadd(ctx, offset, seg->flags)) + for (offset = 0; offset < end; offset += PAGE_SIZE) { + if (!mrenclave_eadd(ctx, seg->offset + offset, seg->flags)) return false; - if (!mrenclave_eextend(ctx, offset, encl->src + offset)) + if (!mrenclave_eextend(ctx, seg->offset + offset, seg->src + offset)) return false; } From patchwork Mon Jul 5 14:36:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jarkko Sakkinen X-Patchwork-Id: 12359227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-20.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5AE7C07E99 for ; Mon, 5 Jul 2021 14:37:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A7F8961966 for ; Mon, 5 Jul 2021 14:37:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231557AbhGEOjp (ORCPT ); Mon, 5 Jul 2021 10:39:45 -0400 Received: from mail.kernel.org ([198.145.29.99]:44068 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231539AbhGEOjo (ORCPT ); Mon, 5 Jul 2021 10:39:44 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 483256195C; Mon, 5 Jul 2021 14:37:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1625495827; bh=JuPCvcyURhAudU9gIfifiJz70vErFSX5KY1xpZ69oQE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hXMIokCtWBjuB6GeT17osWO4o+M4WTVprmbX6HmyFX2iwLY6fGid3IqphxI+aUr+G SXN80pO6R1rR0pbW3tYvsxHOgQzssXxfjsZTPTjLx7mBM0XwgZyzpgfdakZPOOqrOa TxR35ba/XrYqDc+HGrU7LiVf0313zhk3sNsXr34H7hQLpOX1h0SUph8JZjogBVcSPI OdBpBhkd4+ShcGD1p8658+qyeJR7rxikwdLswlvLmkVnsH8KjPaEWc7tdjQ04LIr0u VOCqQJ0rVrD5wn5R5tOcLgKaQ1RaPQuT0QCLhPPhknV7XMrHiz00wkyfrvtZWwVShZ Yy13ixOum3DmQ== From: Jarkko Sakkinen To: Shuah Khan Cc: linux-kselftest@vger.kernel.org, linux-sgx@vger.kernel.org, Reinette Chatre , Borislav Petkov , Jarkko Sakkinen , Dave Hansen , linux-kernel@vger.kernel.org Subject: [PATCH 4/4] selftests/sgx: Trigger the reclaimer and #PF handler Date: Mon, 5 Jul 2021 17:36:52 +0300 Message-Id: <20210705143652.116125-5-jarkko@kernel.org> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210705143652.116125-1-jarkko@kernel.org> References: <20210705143652.116125-1-jarkko@kernel.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-sgx@vger.kernel.org Create a heap for the test enclave, which has the same size as all available Enclave Page Cache (EPC) pages in the system. This will guarantee that all test_encl.elf pages *and* SGX Enclave Control Structure (SECS) have been swapped out by the page reclaimer during the load time. Actually, this adds a bit more stress than that since part of the EPC gets reserved for the Version Array (VA) pages. For each test, the page fault handler gets triggered in two occasions: - When SGX_IOC_ENCLAVE_INIT is performed, SECS gets swapped in by the page fault handler. - During the execution, each page that is referenced gets swapped in by the page fault handler. Signed-off-by: Jarkko Sakkinen --- tools/testing/selftests/sgx/load.c | 33 +++++++++++++++---- tools/testing/selftests/sgx/main.c | 42 ++++++++++++++++++++++++- tools/testing/selftests/sgx/main.h | 3 +- tools/testing/selftests/sgx/sigstruct.c | 6 ++-- 4 files changed, 74 insertions(+), 10 deletions(-) diff --git a/tools/testing/selftests/sgx/load.c b/tools/testing/selftests/sgx/load.c index 9946fab2a3d6..a312a132ac33 100644 --- a/tools/testing/selftests/sgx/load.c +++ b/tools/testing/selftests/sgx/load.c @@ -21,6 +21,8 @@ void encl_delete(struct encl *encl) { + struct encl_segment *heap_seg = &encl->segment_tbl[encl->nr_segments - 1]; + if (encl->encl_base) munmap((void *)encl->encl_base, encl->encl_size); @@ -30,6 +32,8 @@ void encl_delete(struct encl *encl) if (encl->fd) close(encl->fd); + munmap(heap_seg->src, heap_seg->size); + if (encl->segment_tbl) free(encl->segment_tbl); @@ -111,7 +115,10 @@ static bool encl_ioc_add_pages(struct encl *encl, struct encl_segment *seg) ioc.offset = seg->offset; ioc.length = seg->size; ioc.secinfo = (unsigned long)&secinfo; - ioc.flags = SGX_PAGE_MEASURE; + if (seg->measure) + ioc.flags = SGX_PAGE_MEASURE; + else + ioc.flags = 0; rc = ioctl(encl->fd, SGX_IOC_ENCLAVE_ADD_PAGES, &ioc); if (rc < 0) { @@ -124,9 +131,10 @@ static bool encl_ioc_add_pages(struct encl *encl, struct encl_segment *seg) -bool encl_load(const char *path, struct encl *encl) +bool encl_load(const char *path, struct encl *encl, unsigned long heap_size) { const char device_path[] = "/dev/sgx_enclave"; + struct encl_segment *seg; Elf64_Phdr *phdr_tbl; off_t src_offset; Elf64_Ehdr *ehdr; @@ -188,6 +196,8 @@ bool encl_load(const char *path, struct encl *encl) ehdr = encl->bin; phdr_tbl = encl->bin + ehdr->e_phoff; + encl->nr_segments = 1; /* one for the heap */ + for (i = 0; i < ehdr->e_phnum; i++) { Elf64_Phdr *phdr = &phdr_tbl[i]; @@ -203,7 +213,6 @@ bool encl_load(const char *path, struct encl *encl) for (i = 0, j = 0; i < ehdr->e_phnum; i++) { Elf64_Phdr *phdr = &phdr_tbl[i]; unsigned int flags = phdr->p_flags; - struct encl_segment *seg; if (phdr->p_type != PT_LOAD) continue; @@ -240,14 +249,26 @@ bool encl_load(const char *path, struct encl *encl) seg->offset = (phdr->p_offset & PAGE_MASK) - src_offset; seg->size = (phdr->p_filesz + PAGE_SIZE - 1) & PAGE_MASK; seg->src = encl->src + seg->offset; + seg->measure = true; j++; } - assert(j == encl->nr_segments); + assert(j == encl->nr_segments - 1); + + seg = &encl->segment_tbl[j]; + seg->offset = encl->segment_tbl[j - 1].offset + encl->segment_tbl[j - 1].size; + seg->size = heap_size; + seg->src = mmap(NULL, heap_size, PROT_READ | PROT_WRITE, + MAP_ANONYMOUS | MAP_PRIVATE, -1, 0); + seg->prot = PROT_READ | PROT_WRITE; + seg->flags = (SGX_PAGE_TYPE_REG << 8) | seg->prot; + seg->measure = false; + + if (seg->src == MAP_FAILED) + goto err; - encl->src_size = encl->segment_tbl[j - 1].offset + - encl->segment_tbl[j - 1].size; + encl->src_size = encl->segment_tbl[j].offset + encl->segment_tbl[j].size; for (encl->encl_size = 4096; encl->encl_size < encl->src_size; ) encl->encl_size <<= 1; diff --git a/tools/testing/selftests/sgx/main.c b/tools/testing/selftests/sgx/main.c index e252015e0c15..772ba1d72619 100644 --- a/tools/testing/selftests/sgx/main.c +++ b/tools/testing/selftests/sgx/main.c @@ -112,17 +112,57 @@ FIXTURE(enclave) { struct sgx_enclave_run run; }; +#define SGX_NR_ALL_PAGES_PATH "/sys/kernel/debug/x86/sgx_nr_all_pages" + +static int sysfs_get_ulong(const char *path, unsigned long *value) +{ + struct stat sbuf; + ssize_t ret = 0; + char buf[128]; + int fd; + + ret = stat(path, &sbuf); + if (ret) + return ret; + + fd = open(path, O_RDONLY); + if (fd < 0) + return fd; + + ret = read(fd, buf, sizeof(buf)); + if (ret < 0) + goto out; + + /* Clear the read bytes count. */ + ret = 0; + + errno = 0; + *value = strtoul(buf, NULL, 0); + if (errno) + ret = -1; + +out: + close(fd); + return ret; +} + FIXTURE_SETUP(enclave) { Elf64_Sym *sgx_enter_enclave_sym = NULL; + unsigned long nr_all_pages; struct vdso_symtab symtab; struct encl_segment *seg; char maps_line[256]; FILE *maps_file; unsigned int i; void *addr; + int ret; + + ret = sysfs_get_ulong(SGX_NR_ALL_PAGES_PATH, &nr_all_pages); + if (ret) + ksft_exit_skip("Failed to read " SGX_NR_ALL_PAGES_PATH "\n"); - if (!encl_load("test_encl.elf", &self->encl)) { + if (!encl_load("test_encl.elf", &self->encl, nr_all_pages * 4096)) { encl_delete(&self->encl); ksft_exit_skip("cannot load enclaves\n"); } diff --git a/tools/testing/selftests/sgx/main.h b/tools/testing/selftests/sgx/main.h index 452d11dc4889..a286861dc289 100644 --- a/tools/testing/selftests/sgx/main.h +++ b/tools/testing/selftests/sgx/main.h @@ -12,6 +12,7 @@ struct encl_segment { size_t size; unsigned int prot; unsigned int flags; + bool measure; }; struct encl { @@ -32,7 +33,7 @@ extern unsigned char sign_key[]; extern unsigned char sign_key_end[]; void encl_delete(struct encl *ctx); -bool encl_load(const char *path, struct encl *encl); +bool encl_load(const char *path, struct encl *encl, unsigned long heap_size); bool encl_measure(struct encl *encl); bool encl_build(struct encl *encl); diff --git a/tools/testing/selftests/sgx/sigstruct.c b/tools/testing/selftests/sgx/sigstruct.c index 202a96fd81bf..50c5ab1aa6fa 100644 --- a/tools/testing/selftests/sgx/sigstruct.c +++ b/tools/testing/selftests/sgx/sigstruct.c @@ -296,8 +296,10 @@ static bool mrenclave_segment(EVP_MD_CTX *ctx, struct encl *encl, if (!mrenclave_eadd(ctx, seg->offset + offset, seg->flags)) return false; - if (!mrenclave_eextend(ctx, seg->offset + offset, seg->src + offset)) - return false; + if (seg->measure) { + if (!mrenclave_eextend(ctx, seg->offset + offset, seg->src + offset)) + return false; + } } return true;