From patchwork Tue Mar 8 11:10:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jarkko Sakkinen X-Patchwork-Id: 12773522 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9889BC433F5 for ; Tue, 8 Mar 2022 11:10:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 054588D0003; Tue, 8 Mar 2022 06:10:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 006A18D0001; Tue, 8 Mar 2022 06:10:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E0DC38D0003; Tue, 8 Mar 2022 06:10:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.25]) by kanga.kvack.org (Postfix) with ESMTP id D473F8D0001 for ; Tue, 8 Mar 2022 06:10:54 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 9D1C8243F1 for ; Tue, 8 Mar 2022 11:10:54 +0000 (UTC) X-FDA: 79220951628.05.23FDDD5 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf25.hostedemail.com (Postfix) with ESMTP id 0ADA2A0002 for ; Tue, 8 Mar 2022 11:10:53 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7D8E0615FF; Tue, 8 Mar 2022 11:10:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E8DB9C340EC; Tue, 8 Mar 2022 11:10:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1646737853; bh=WFgarrj7YVpHc+BKJTPJxYeEYbi76FtZ0eeCRFRqG1Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oxjmeSyvoQwzOY1Cm834gXsI/z3vB2GRWDWv4fICOaZRvtKV1YcIlNe1emEzAdkhS E649NWxLq3ZqXaiRBSUunEUzFZNlIPck6S2ntQlYQUoslsZnbpfztS6SrQPgtu8VfY YrRJZGd6V/leH8pTIaRSCJ+3G7xN+KtL1ciD4VUVHBFP8ZU8J8jvxfw5X0IZyuiSaD gfj6bp2Y2HvnlPF4SuY0TmAxY3gsn/Ep09VWfy/BsOBV1ETJoXB/P3CyMJaym+HDSQ 4KTARfatOkIOiUQFeWszFgi79eFpVOaVyKRKaAsctGU01O0iqqbhs31hhHfLrxAOoR 0Wm6FffB1QTRQ== From: Jarkko Sakkinen To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: Dave Hansen , Nathaniel McCallum , Reinette Chatre , Alexander Viro , linux-sgx@vger.kernel.org, linux-kernel@vger.kernel.org, Andrew Morton , Jarkko Sakkinen Subject: [PATCH RFC v2 1/3] mm: Add f_op->populate() for populating memory outside of core mm Date: Tue, 8 Mar 2022 13:10:01 +0200 Message-Id: <20220308111003.257351-2-jarkko@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220308111003.257351-1-jarkko@kernel.org> References: <20220308111003.257351-1-jarkko@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 0ADA2A0002 X-Stat-Signature: e3r8awgwdsx71xiw5n6558gghu35me4y X-Rspam-User: Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=oxjmeSyv; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf25.hostedemail.com: domain of jarkko@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=jarkko@kernel.org X-Rspamd-Server: rspam03 X-HE-Tag: 1646737853-302262 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: SGX memory is managed outside the core mm. It doesn't have a 'struct page' and get_user_pages() doesn't work on it. Its VMAs are marked with VM_IO. So, none of the existing methods for avoiding page faults work on SGX memory. Add f_op->populate() to overcome this issue: int (*populate)(struct file *, unsigned long start, unsigned long end); Then in populate_vma_page_range(), allow it to be used in the place of get_user_pages() for memory that falls outside of its scope. Suggested-by: Dave Hansen Signed-off-by: Jarkko Sakkinen --- v4: * Reimplement based on Dave's suggestion: https://lore.kernel.org/linux-sgx/c3083144-bfc1-3260-164c-e59b2d110df8@intel.com/ * Copy the text from the suggestion as part of the commit message (and cover letter). v3: - if (!ret && do_populate && file->f_op->populate) + if (!ret && do_populate && file->f_op->populate && + !!(vma->vm_flags & (VM_IO | VM_PFNMAP))) (reported by Matthew Wilcox) v2: - if (!ret && do_populate) + if (!ret && do_populate && file->f_op->populate) (reported by Jan Harkes) include/linux/fs.h | 1 + mm/gup.c | 9 +++++---- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/include/linux/fs.h b/include/linux/fs.h index e2d892b201b0..54151af88ee0 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1993,6 +1993,7 @@ struct file_operations { long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long); long (*compat_ioctl) (struct file *, unsigned int, unsigned long); int (*mmap) (struct file *, struct vm_area_struct *); + int (*populate)(struct file *, unsigned long start, unsigned long end); unsigned long mmap_supported_flags; int (*open) (struct inode *, struct file *); int (*flush) (struct file *, fl_owner_t id); diff --git a/mm/gup.c b/mm/gup.c index a9d4d724aef7..66736a188a9c 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1519,8 +1519,11 @@ long populate_vma_page_range(struct vm_area_struct *vma, * We made sure addr is within a VMA, so the following will * not result in a stack expansion that recurses back here. */ - return __get_user_pages(mm, start, nr_pages, gup_flags, - NULL, NULL, locked); + if ((vma->vm_flags & (VM_IO | VM_PFNMAP)) && vma->vm_file->f_op->populate) + return vma->vm_file->f_op->populate(vma->vm_file, start, end); + else + return __get_user_pages(mm, start, nr_pages, gup_flags, + NULL, NULL, locked); } /* @@ -1619,8 +1622,6 @@ int __mm_populate(unsigned long start, unsigned long len, int ignore_errors) * range with the first VMA. Also, skip undesirable VMA types. */ nend = min(end, vma->vm_end); - if (vma->vm_flags & (VM_IO | VM_PFNMAP)) - continue; if (nstart < vma->vm_start) nstart = vma->vm_start; /* From patchwork Tue Mar 8 11:10:02 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jarkko Sakkinen X-Patchwork-Id: 12773523 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD189C433F5 for ; Tue, 8 Mar 2022 11:10:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6206D8D0005; Tue, 8 Mar 2022 06:10:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D0088D0001; Tue, 8 Mar 2022 06:10:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4C0948D0005; Tue, 8 Mar 2022 06:10:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0170.hostedemail.com [216.40.44.170]) by kanga.kvack.org (Postfix) with ESMTP id 3D5288D0001 for ; Tue, 8 Mar 2022 06:10:59 -0500 (EST) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id EEAECA867F for ; Tue, 8 Mar 2022 11:10:58 +0000 (UTC) X-FDA: 79220951796.27.AD9676B Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf29.hostedemail.com (Postfix) with ESMTP id 6846C120005 for ; Tue, 8 Mar 2022 11:10:58 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id CACBE615FE; Tue, 8 Mar 2022 11:10:57 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6D959C36AE3; Tue, 8 Mar 2022 11:10:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1646737856; bh=/ZE7KoA/CEZ9Cv7O8oC7vdpXC4AHqtlt9/5UjASb5C4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rqXAeGfgtHehh/O++trSzt+l6ncC1IiQmQu0xmci6iUm3XBdyzAZOVLYDKqAacvaz 3NJq3a8t3Hv2lfTBAoxiL2ngCQkG0+H7aM6kKEySXj/i0NQd3W3VkbqE7uTUt6+TF0 RKPU9XN65F+rNAvreKORb9HHnl5UZYbKBbhRdxJQGlgmWICIwesMOleIVkHK7r44pL Rvw4AVbR/MJtFylEFyYGxicn/LakH/xjAa08l1dTMpmYA3Znp+Zf+o62n4ezoXPlwB fYhGv47xyTYuQiCG+uT3tgbIZR7YQuHIga1AxLJY+DPy3TgOLB+YK3/TddQdzg7h6j XlasGRTyfwO8A== From: Jarkko Sakkinen To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: Dave Hansen , Nathaniel McCallum , Reinette Chatre , Alexander Viro , linux-sgx@vger.kernel.org, linux-kernel@vger.kernel.org, Andrew Morton , Jarkko Sakkinen Subject: [PATCH RFC v2 2/3] x86/sgx: Export sgx_encl_page_alloc() Date: Tue, 8 Mar 2022 13:10:02 +0200 Message-Id: <20220308111003.257351-3-jarkko@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220308111003.257351-1-jarkko@kernel.org> References: <20220308111003.257351-1-jarkko@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6846C120005 X-Stat-Signature: fnxb1z3fgz8tumyfz9czyn963ssji4gd X-Rspam-User: Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=rqXAeGfg; spf=pass (imf29.hostedemail.com: domain of jarkko@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=jarkko@kernel.org; dmarc=pass (policy=none) header.from=kernel.org X-Rspamd-Server: rspam07 X-HE-Tag: 1646737858-516400 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Move sgx_encl_page_alloc() to encl.c and export it so that it can be used in the implementation for MAP_POPULATE, which requires to allocate new enclave pages. Signed-off-by: Jarkko Sakkinen --- arch/x86/kernel/cpu/sgx/encl.c | 38 +++++++++++++++++++++++++++++++++ arch/x86/kernel/cpu/sgx/encl.h | 3 +++ arch/x86/kernel/cpu/sgx/ioctl.c | 38 --------------------------------- 3 files changed, 41 insertions(+), 38 deletions(-) diff --git a/arch/x86/kernel/cpu/sgx/encl.c b/arch/x86/kernel/cpu/sgx/encl.c index f24a41d3ec70..0256918b2c2f 100644 --- a/arch/x86/kernel/cpu/sgx/encl.c +++ b/arch/x86/kernel/cpu/sgx/encl.c @@ -913,6 +913,44 @@ int sgx_encl_test_and_clear_young(struct mm_struct *mm, return ret; } +struct sgx_encl_page *sgx_encl_page_alloc(struct sgx_encl *encl, + unsigned long offset, + u64 secinfo_flags) +{ + struct sgx_encl_page *encl_page; + unsigned long prot; + + encl_page = kzalloc(sizeof(*encl_page), GFP_KERNEL); + if (!encl_page) + return ERR_PTR(-ENOMEM); + + encl_page->desc = encl->base + offset; + encl_page->encl = encl; + + prot = _calc_vm_trans(secinfo_flags, SGX_SECINFO_R, PROT_READ) | + _calc_vm_trans(secinfo_flags, SGX_SECINFO_W, PROT_WRITE) | + _calc_vm_trans(secinfo_flags, SGX_SECINFO_X, PROT_EXEC); + + /* + * TCS pages must always RW set for CPU access while the SECINFO + * permissions are *always* zero - the CPU ignores the user provided + * values and silently overwrites them with zero permissions. + */ + if ((secinfo_flags & SGX_SECINFO_PAGE_TYPE_MASK) == SGX_SECINFO_TCS) + prot |= PROT_READ | PROT_WRITE; + + /* Calculate maximum of the VM flags for the page. */ + encl_page->vm_max_prot_bits = calc_vm_prot_bits(prot, 0); + + /* + * At time of allocation, the runtime protection bits are the same + * as the maximum protection bits. + */ + encl_page->vm_run_prot_bits = encl_page->vm_max_prot_bits; + + return encl_page; +} + /** * sgx_zap_enclave_ptes() - remove PTEs mapping the address from enclave * @encl: the enclave diff --git a/arch/x86/kernel/cpu/sgx/encl.h b/arch/x86/kernel/cpu/sgx/encl.h index 1b6ce1da7c92..3df0d3faf3a1 100644 --- a/arch/x86/kernel/cpu/sgx/encl.h +++ b/arch/x86/kernel/cpu/sgx/encl.h @@ -113,6 +113,9 @@ int sgx_encl_get_backing(struct sgx_encl *encl, unsigned long page_index, void sgx_encl_put_backing(struct sgx_backing *backing, bool do_write); int sgx_encl_test_and_clear_young(struct mm_struct *mm, struct sgx_encl_page *page); +struct sgx_encl_page *sgx_encl_page_alloc(struct sgx_encl *encl, + unsigned long offset, + u64 secinfo_flags); void sgx_zap_enclave_ptes(struct sgx_encl *encl, unsigned long addr); struct sgx_epc_page *sgx_alloc_va_page(void); unsigned int sgx_alloc_va_slot(struct sgx_va_page *va_page); diff --git a/arch/x86/kernel/cpu/sgx/ioctl.c b/arch/x86/kernel/cpu/sgx/ioctl.c index d8c3c07badb3..3e3ca27a6f72 100644 --- a/arch/x86/kernel/cpu/sgx/ioctl.c +++ b/arch/x86/kernel/cpu/sgx/ioctl.c @@ -169,44 +169,6 @@ static long sgx_ioc_enclave_create(struct sgx_encl *encl, void __user *arg) return ret; } -static struct sgx_encl_page *sgx_encl_page_alloc(struct sgx_encl *encl, - unsigned long offset, - u64 secinfo_flags) -{ - struct sgx_encl_page *encl_page; - unsigned long prot; - - encl_page = kzalloc(sizeof(*encl_page), GFP_KERNEL); - if (!encl_page) - return ERR_PTR(-ENOMEM); - - encl_page->desc = encl->base + offset; - encl_page->encl = encl; - - prot = _calc_vm_trans(secinfo_flags, SGX_SECINFO_R, PROT_READ) | - _calc_vm_trans(secinfo_flags, SGX_SECINFO_W, PROT_WRITE) | - _calc_vm_trans(secinfo_flags, SGX_SECINFO_X, PROT_EXEC); - - /* - * TCS pages must always RW set for CPU access while the SECINFO - * permissions are *always* zero - the CPU ignores the user provided - * values and silently overwrites them with zero permissions. - */ - if ((secinfo_flags & SGX_SECINFO_PAGE_TYPE_MASK) == SGX_SECINFO_TCS) - prot |= PROT_READ | PROT_WRITE; - - /* Calculate maximum of the VM flags for the page. */ - encl_page->vm_max_prot_bits = calc_vm_prot_bits(prot, 0); - - /* - * At time of allocation, the runtime protection bits are the same - * as the maximum protection bits. - */ - encl_page->vm_run_prot_bits = encl_page->vm_max_prot_bits; - - return encl_page; -} - static int sgx_validate_secinfo(struct sgx_secinfo *secinfo) { u64 perm = secinfo->flags & SGX_SECINFO_PERMISSION_MASK; From patchwork Tue Mar 8 11:10:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jarkko Sakkinen X-Patchwork-Id: 12773524 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1A92C433FE for ; Tue, 8 Mar 2022 11:11:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5103A8D0006; Tue, 8 Mar 2022 06:11:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4BE258D0001; Tue, 8 Mar 2022 06:11:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3ADB68D0006; Tue, 8 Mar 2022 06:11:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.27]) by kanga.kvack.org (Postfix) with ESMTP id 2C92D8D0001 for ; Tue, 8 Mar 2022 06:11:03 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0846420522 for ; Tue, 8 Mar 2022 11:11:03 +0000 (UTC) X-FDA: 79220952006.03.CBD2AF8 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf30.hostedemail.com (Postfix) with ESMTP id 6F22880007 for ; Tue, 8 Mar 2022 11:11:02 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C5C2F615F4; Tue, 8 Mar 2022 11:11:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D486DC340EC; Tue, 8 Mar 2022 11:11:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1646737861; bh=Cgl0qA66PzKkXnlxny3eTwo2XWjpLBhZQNxtn6qkKoU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CVtFPe97vYFj1FySCYQWFItXlCjdHBBevaQR2H2MDTh3XK+vDmVt//BRtA8eHgtSQ sjX3dV+dGnVsmQYynsHa/6CG+cH0iJ2JSz3zSBflrcMfirdI7SyWgZhv0+b1zK8Qtx LQYHNrL7tJC94dmVVVh2CnD+25qW+oEDV+a+ghj9X2HmsDMXGQ90hDhpJZ0b6QOBT6 lp7hj/hCt2UF8R+q8mi8232uYww0mHcsALmH/I0HEEQR9/JhaihoUeYb4yqZzavdE7 vcolydOfAj/JirilBmcmgMWxom4Fj+/urpSMPIBbrl/KqfGN9mquzr1bqZP6F5GxKm 93WsFFOMUijxg== From: Jarkko Sakkinen To: linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: Dave Hansen , Nathaniel McCallum , Reinette Chatre , Alexander Viro , linux-sgx@vger.kernel.org, linux-kernel@vger.kernel.org, Andrew Morton , Jarkko Sakkinen Subject: [PATCH RFC v2 3/3] x86/sgx: Implement EAUG population with MAP_POPULATE Date: Tue, 8 Mar 2022 13:10:03 +0200 Message-Id: <20220308111003.257351-4-jarkko@kernel.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220308111003.257351-1-jarkko@kernel.org> References: <20220308111003.257351-1-jarkko@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6F22880007 X-Stat-Signature: 9ery7fyzdpjei4tt6egxgcz4ef8epnkp X-Rspam-User: Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=CVtFPe97; spf=pass (imf30.hostedemail.com: domain of jarkko@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=jarkko@kernel.org; dmarc=pass (policy=none) header.from=kernel.org X-Rspamd-Server: rspam07 X-HE-Tag: 1646737862-829457 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: With SGX1 an enclave needs to be created with its maximum memory demands pre-allocated. Pages cannot be added to an enclave after it is initialized. SGX2 introduces a new function, ENCLS[EAUG] for adding pages to an initialized enclave. Add support for dynamically adding pages to an initialized enclave with mmap() by populating pages with EAUG. Use f_ops->populate() callback to achieve this behaviour. Signed-off-by: Jarkko Sakkinen --- arch/x86/kernel/cpu/sgx/driver.c | 128 +++++++++++++++++++++++++++++++ 1 file changed, 128 insertions(+) diff --git a/arch/x86/kernel/cpu/sgx/driver.c b/arch/x86/kernel/cpu/sgx/driver.c index aa9b8b868867..848938334e8a 100644 --- a/arch/x86/kernel/cpu/sgx/driver.c +++ b/arch/x86/kernel/cpu/sgx/driver.c @@ -9,6 +9,7 @@ #include #include "driver.h" #include "encl.h" +#include "encls.h" u64 sgx_attributes_reserved_mask; u64 sgx_xfrm_reserved_mask = ~0x3; @@ -101,6 +102,132 @@ static int sgx_mmap(struct file *file, struct vm_area_struct *vma) return 0; } +static int sgx_encl_augment_page(struct sgx_encl *encl, unsigned long offset) +{ + struct sgx_pageinfo pginfo = {0}; + struct sgx_encl_page *encl_page; + struct sgx_epc_page *epc_page; + struct sgx_va_page *va_page; + u64 secinfo_flags; + int ret; + + /* + * Ignore internal permission checking for dynamically added pages. + * They matter only for data added during the pre-initialization phase. + * The enclave decides the permissions by the means of EACCEPT, + * EACCEPTCOPY and EMODPE. + */ + secinfo_flags = SGX_SECINFO_R | SGX_SECINFO_W | SGX_SECINFO_X; + encl_page = sgx_encl_page_alloc(encl, offset, secinfo_flags); + if (IS_ERR(encl_page)) + return PTR_ERR(encl_page); + + epc_page = sgx_alloc_epc_page(encl_page, true); + if (IS_ERR(epc_page)) { + ret = PTR_ERR(epc_page); + goto err_alloc_epc_page; + } + + va_page = sgx_encl_grow(encl); + if (IS_ERR(va_page)) { + ret = PTR_ERR(va_page); + goto err_grow; + } + + mutex_lock(&encl->lock); + + /* + * Adding to encl->va_pages must be done under encl->lock. Ditto for + * deleting (via sgx_encl_shrink()) in the error path. + */ + if (va_page) + list_add(&va_page->list, &encl->va_pages); + + /* + * Insert prior to EADD in case of OOM. EADD modifies MRENCLAVE, i.e. + * can't be gracefully unwound, while failure on EADD/EXTEND is limited + * to userspace errors (or kernel/hardware bugs). + */ + ret = xa_insert(&encl->page_array, PFN_DOWN(encl_page->desc), + encl_page, GFP_KERNEL); + + /* + * If ret == -EBUSY then page was created in another flow while + * running without encl->lock + */ + if (ret) + goto err_xa_insert; + + pginfo.secs = (unsigned long)sgx_get_epc_virt_addr(encl->secs.epc_page); + pginfo.addr = encl_page->desc & PAGE_MASK; + pginfo.metadata = 0; + + ret = __eaug(&pginfo, sgx_get_epc_virt_addr(epc_page)); + if (ret) + goto err_eaug; + + encl_page->encl = encl; + encl_page->epc_page = epc_page; + encl_page->type = SGX_PAGE_TYPE_REG; + encl->secs_child_cnt++; + + sgx_mark_page_reclaimable(encl_page->epc_page); + + mutex_unlock(&encl->lock); + + return 0; + +err_eaug: + xa_erase(&encl->page_array, PFN_DOWN(encl_page->desc)); + +err_xa_insert: + sgx_encl_shrink(encl, va_page); + mutex_unlock(&encl->lock); + +err_grow: + sgx_encl_free_epc_page(epc_page); + +err_alloc_epc_page: + kfree(encl_page); + + return VM_FAULT_SIGBUS; +} + +/* + * Add new pages to the enclave sequentially with ENCLS[EAUG]. Note that + * sgx_mmap() validates that the given VMA is within the enclave range. Calling + * here sgx_encl_may_map() second time would too time consuming. + */ +static int sgx_populate(struct file *file, unsigned long start, unsigned long end) +{ + struct sgx_encl *encl = file->private_data; + unsigned long length = end - start; + unsigned long pos; + int ret; + + /* EAUG works only for initialized enclaves. */ + if (!test_bit(SGX_ENCL_INITIALIZED, &encl->flags)) + return -EINVAL; + + for (pos = 0 ; pos < length; pos += PAGE_SIZE) { + if (signal_pending(current)) { + if (!pos) + ret = -ERESTARTSYS; + + break; + } + + if (need_resched()) + cond_resched(); + + ret = sgx_encl_augment_page(encl, start + pos); + if (ret) + break; + } + + return ret; +} + static unsigned long sgx_get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, @@ -133,6 +260,7 @@ static const struct file_operations sgx_encl_fops = { .compat_ioctl = sgx_compat_ioctl, #endif .mmap = sgx_mmap, + .populate = sgx_populate, .get_unmapped_area = sgx_get_unmapped_area, };