From patchwork Thu Feb 21 23:44:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824851 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3E3741399 for ; Thu, 21 Feb 2019 23:50:57 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2DA5F31D2C for ; Thu, 21 Feb 2019 23:50:57 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 217AB31D30; Thu, 21 Feb 2019 23:50:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A238831D2C for ; Thu, 21 Feb 2019 23:50:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE13E8E00C3; Thu, 21 Feb 2019 18:50:55 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C9FC08E00C4; Thu, 21 Feb 2019 18:50:55 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B811A8E00C3; Thu, 21 Feb 2019 18:50:55 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id 74C078E00B5 for ; Thu, 21 Feb 2019 18:50:55 -0500 (EST) Received: by mail-pf1-f198.google.com with SMTP id z1so347023pfz.8 for ; Thu, 21 Feb 2019 15:50:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=Y4ax2hOd4Buhijx/oNSuXPhWpslJbmlh2LJYha9eHFk=; b=NRpWBYWj+7fzbNd/Li1Ky8OQzPc/PoZeF/Gq+FVGb3Yk3ICDSqYjvsmtX8UdR6SUVY hQx8vewlhFZdQ+xXAPQwQeNVxnpkpJt3pC/HqU7msNqWJhN0J8oqELIvYGU/p8MvB2/e nhnVYVxNo3W5iNqtOT5qskcdCz/MoSDXcWGrs3+3Os00fDnz2Xx5O4Oyl+wK91Amk2Mk CtVFH9EPgpou2q59IcFPawFzEOK4/GirMWwwPRJwiHAoGYGhjp/kKMfaXrR4Rj6QGGZ2 ANBXQDPXlenbNmjsUTWuObYjF7Le1bwWpkjfM+yunTsnBDJ5wu3eX7umlvWgSonLXHEC jgdA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAuZaKGXl8ITvGd9XryiHSGm9GAlLKtXSJ8NmgR/l6TW0ICquvp6V RSoCLa8rcVoq/smGMF3jVevtOGotO7/Zo32LI9jsnjIdAkihEih6vsFQ0xYAYEi7m5yhrnkMfON D5TQwl4S5UdlwuiuoeoXRohm06PhyyC4qs7CaCRKOfUsDzoyBzzykFV1xtGAKvy+6sA== X-Received: by 2002:a63:ce41:: with SMTP id r1mr1061771pgi.119.1550793055147; Thu, 21 Feb 2019 15:50:55 -0800 (PST) X-Google-Smtp-Source: AHgI3IYuYDyh+NP9Bf8ZO3K/Ukso1rbVRZDJ2lznF8zglhIWm+WymnwNLDb7TDsTXcvyuWWmUtas X-Received: by 2002:a63:ce41:: with SMTP id r1mr1061738pgi.119.1550793054416; Thu, 21 Feb 2019 15:50:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793054; cv=none; d=google.com; s=arc-20160816; b=XuwSxpfh10LDlQ0j5z/cvyLmi0StsjlU9jhRem+NhBxGL/nA33aK5/lqZS7Q9RSvn6 Du57HSlYB/+YN8xqZ8LIUHPidM95kXKZ8kON7SjzSfznDquWuL0Rr1GY4AimAUQ+jvn/ uqUfir4EdZ1iam2Psxzj7GCdgpTv621vpzYmPXGMMaeLAjh3QUNaOeDCws/TgVQ0pG/3 72BVGfKtGvTG2ita2okqoy6HG+F8qpxeySbw77GUlpGEjQBA/Y5P7WhjGMk74wqzbC7q CFUIrgkHtL6UtOXoxb3rvuFHQkO38Pl44+c3ffMZe52idW2tvTuWivm9uKxn7Y6MmKAE HhCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=Y4ax2hOd4Buhijx/oNSuXPhWpslJbmlh2LJYha9eHFk=; b=AuAdcbRhaLkAzgkcahIbwkixq+ltV9F2Ps9Wgzu1ROx5EErjjt1pxlzjljEg3xBK8U 6lBMLVUSVQXxavDu4kbexO8sGisobVW8kdjZGvj6j22yMGM6VqVMKQnO3kg+4zIMd89n 2fjWupuyf2frJLPx0VQw365V4uhb2yyac54RPr/73xalsUvsskZY6bsQ6rkDJxfxrlV7 WxP+vXla04FQtsy3maGWeduUYKN+6dYlWxtoiuAr82kKzCuN800sD8jB8zUQyxEBtRx2 sxnojrWUU3i7p9Ut4BscxIohYdWedu1fEjatt9NibhVTW25q0SR43fdC8reuYqSBA+fQ XkhQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.50.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:50:54 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:50:54 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394801" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:50:52 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Kees Cook , Dave Hansen , Masami Hiramatsu , Rick Edgecombe Subject: [PATCH v3 01/20] x86/jump_label: Use text_poke_early() during early init Date: Thu, 21 Feb 2019 15:44:32 -0800 Message-Id: <20190221234451.17632-2-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit There is no apparent reason not to use text_poke_early() during early-init, since no patching of code that might be on the stack is done and only a single core is running. This is required for the next patches that would set a temporary mm for text poking, and this mm is only initialized after some static-keys are enabled/disabled. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Cc: Masami Hiramatsu Acked-by: Peter Zijlstra (Intel) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/jump_label.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/jump_label.c b/arch/x86/kernel/jump_label.c index f99bd26bd3f1..e7d8c636b228 100644 --- a/arch/x86/kernel/jump_label.c +++ b/arch/x86/kernel/jump_label.c @@ -50,7 +50,12 @@ static void __ref __jump_label_transform(struct jump_entry *entry, jmp.offset = jump_entry_target(entry) - (jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE); - if (early_boot_irqs_disabled) + /* + * As long as only a single processor is running and the code is still + * not marked as RO, text_poke_early() can be used; Checking that + * system_state is SYSTEM_BOOTING guarantees it. + */ + if (system_state == SYSTEM_BOOTING) poker = text_poke_early; if (type == JUMP_LABEL_JMP) { From patchwork Thu Feb 21 23:44:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824855 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C9DBF922 for ; Thu, 21 Feb 2019 23:51:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B916231D2C for ; Thu, 21 Feb 2019 23:51:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AC6F331D30; Thu, 21 Feb 2019 23:51:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 31CF531D2C for ; Thu, 21 Feb 2019 23:51:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C6B658E00C5; Thu, 21 Feb 2019 18:50:56 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C21728E00B5; Thu, 21 Feb 2019 18:50:56 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A6F108E00C5; Thu, 21 Feb 2019 18:50:56 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by kanga.kvack.org (Postfix) with ESMTP id 67D048E00B5 for ; Thu, 21 Feb 2019 18:50:56 -0500 (EST) Received: by mail-pl1-f199.google.com with SMTP id w18so313791plq.10 for ; Thu, 21 Feb 2019 15:50:56 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=TDi+ZaxaGFwMJnbi/KBZqpgNpblOvtV7DEanhYCdmjY=; b=K8N24PormxV+dEH+1jewt9asa2ry96tJ7PU9o7auf5pZC1puP/AwNsUYhd322rsTsP YhI+jCWA62EnpqKBuM9PeFMJXy0nGJvJmSaAuycP3DMRo93mozpPk5fQWGGx8Sc7qtnG 3x4IyYlGa17MufiE95Fdg93Pn4XBHrNdjog7mvYIE60oH4UPVMsnQaV0HzANhHQhBlDc v7r3G8YhKosivGYkvsrRzbMuFRKAeOmiJuN+VEyDmGdKIKuCShX3LkgLlL5YjqhGdq79 QOTqiuKWDiQFrTawWkPtPl/+7qXDf10PQuZKRylTr+de5UcLuBLHlOa3qNjxgZ0WhRX6 gA2Q== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAubld2AqrKVyrFP514Kk6EKtwjm0n1SJPvS3WyDjP/w5sCG9z/a6 GCMyjbUlcyrY4yLnAPuJGJokUOZfxSAWaDEo6a52qGgHWegtI1CUsUaRU9gUCBsxtqvClP487TO fkGr+AFnCcES56on+4qfzBMW+X2KuYwN592CFIgZ+WfFiqcM5AgqxQY4CQ59juLkhHA== X-Received: by 2002:a63:d442:: with SMTP id i2mr1085375pgj.246.1550793056064; Thu, 21 Feb 2019 15:50:56 -0800 (PST) X-Google-Smtp-Source: AHgI3IY/iv4hBEd8A4TCyrQvuxBfTX9ltVqKMV4Eh9nI0BKqU5r2wPWBqhrxeICsBovhqkeMr23e X-Received: by 2002:a63:d442:: with SMTP id i2mr1085334pgj.246.1550793055303; Thu, 21 Feb 2019 15:50:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793055; cv=none; d=google.com; s=arc-20160816; b=I27LP6yjhQId7oAdU6YXL6I2aODuMdlDUFDVys+TLu6DBbI5COTyIiyHQM58lA78o6 OejAAC5Mxydc36cKV0tfaAvEXZuBlxrmvwlmLPaUuXYI4IOJsCHqhpbT+qHvQFLuD1hp dabu3oDv64Hkmmvi+jP0AP1Y6l/dH6V8OC8oSMASYZkH5bsmuFEngydYgIcpXdOkk7Yr dkvAxPO0aD6f+Uf6V/KWOS3nfsReKoXOIiqsBWSLDi8JxjkxZfZNk1gURthKWwHI3TLQ MLPfS2zLdFyGTAU4p9ZvIzeIB63k0sDHTo0EF24ah+wDZ4WLqeLAQpZfDRX4ZxfYu34u xakA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=TDi+ZaxaGFwMJnbi/KBZqpgNpblOvtV7DEanhYCdmjY=; b=xuBXzVmyKmdDOxT67hglBqAakbjSZ7paXlcx5hDFd2JSafMfo6Ex9BAF6CizxRe0HQ mhTMcWbB+IMBoAEsTpLXh6P9fN/fvS0PxHxbkyfs1qCMOD1CsE+69robDS8dHa2YE3kW 8Mbf4zszisDlmA95Xz10Zei/gQ16mQhfHxP2A4Sn8s9ZPAAiz5wl8RecXdFALwawwPaD K9A21DH9bOFOPRt4vtno5yrkZLAnXB/qqFSRFC2mX75IxRVjPXO/SWh5Rh3T87QmG9TE vNfOG94XPJZx3tBVwq/+Of8FXOn8ZTXJZBuA1sUN7eFT93kmLRzZSNCTOeCZe4YquYFT M4mQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.50.55 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:50:55 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:50:54 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394811" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:50:53 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Kees Cook , Dave Hansen , Nadav Amit , Rick Edgecombe Subject: [PATCH v3 02/20] x86/mm: Introduce temporary mm structs Date: Thu, 21 Feb 2019 15:44:33 -0800 Message-Id: <20190221234451.17632-3-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Andy Lutomirski Using a dedicated page-table for temporary PTEs prevents other cores from using - even speculatively - these PTEs, thereby providing two benefits: (1) Security hardening: an attacker that gains kernel memory writing abilities cannot easily overwrite sensitive data. (2) Avoiding TLB shootdowns: the PTEs do not need to be flushed in remote page-tables. To do so a temporary mm_struct can be used. Mappings which are private for this mm can be set in the userspace part of the address-space. During the whole time in which the temporary mm is loaded, interrupts must be disabled. The first use-case for temporary mm struct, which will follow, is for poking the kernel text. [ Commit message was written by Nadav Amit ] Cc: Kees Cook Cc: Dave Hansen Acked-by: Peter Zijlstra (Intel) Reviewed-by: Masami Hiramatsu Tested-by: Masami Hiramatsu Signed-off-by: Andy Lutomirski Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/include/asm/mmu_context.h | 33 ++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h index 19d18fae6ec6..d684b954f3c0 100644 --- a/arch/x86/include/asm/mmu_context.h +++ b/arch/x86/include/asm/mmu_context.h @@ -356,4 +356,37 @@ static inline unsigned long __get_current_cr3_fast(void) return cr3; } +typedef struct { + struct mm_struct *prev; +} temp_mm_state_t; + +/* + * Using a temporary mm allows to set temporary mappings that are not accessible + * by other cores. Such mappings are needed to perform sensitive memory writes + * that override the kernel memory protections (e.g., W^X), without exposing the + * temporary page-table mappings that are required for these write operations to + * other cores. Using temporary mm also allows to avoid TLB shootdowns when the + * mapping is torn down. + * + * Context: The temporary mm needs to be used exclusively by a single core. To + * harden security IRQs must be disabled while the temporary mm is + * loaded, thereby preventing interrupt handler bugs from overriding + * the kernel memory protection. + */ +static inline temp_mm_state_t use_temporary_mm(struct mm_struct *mm) +{ + temp_mm_state_t state; + + lockdep_assert_irqs_disabled(); + state.prev = this_cpu_read(cpu_tlbstate.loaded_mm); + switch_mm_irqs_off(NULL, mm, current); + return state; +} + +static inline void unuse_temporary_mm(temp_mm_state_t prev) +{ + lockdep_assert_irqs_disabled(); + switch_mm_irqs_off(NULL, prev.prev, current); +} + #endif /* _ASM_X86_MMU_CONTEXT_H */ From patchwork Thu Feb 21 23:44:34 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824857 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EFE251399 for ; Thu, 21 Feb 2019 23:51:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DEEC731D2C for ; Thu, 21 Feb 2019 23:51:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D2DA831D30; Thu, 21 Feb 2019 23:51:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 683F531D2C for ; Thu, 21 Feb 2019 23:51:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D56E98E00C6; Thu, 21 Feb 2019 18:50:57 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D2D7B8E00B5; Thu, 21 Feb 2019 18:50:57 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF8EA8E00C6; Thu, 21 Feb 2019 18:50:57 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) by kanga.kvack.org (Postfix) with ESMTP id 6FF638E00B5 for ; Thu, 21 Feb 2019 18:50:57 -0500 (EST) Received: by mail-pg1-f198.google.com with SMTP id 23so332865pgr.11 for ; Thu, 21 Feb 2019 15:50:57 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=ai+qJljkOSM+BNa8+5qAy5ubO6YA1LWKpTMBiUyjPN0=; b=nnBPVKcpup5Dmq+AEQ3FF7U22p+kSnntprT1fjSETVki7zv9P4B5pf6qrD5f13N8wf 5fh2hBIGERlTIJxpi5SaML8OYFYvbNOV1c6IeXRDXjGvBeNIBo0nSFCIpqhRxgEFxfaD InpLqPiFaGAc4IjqfmYrSbiz5kf+m605yizp9KYaciEWKJiJZTfecfSt0NrQa3pvv7FU gcOGYkCBrV8b06OhRRKiFlSk0tLUqlg8kmbVFOJZ1vdDyL5g+Xq5jZREHPLg3q/Qf243 +gdW3YQ8/V8KwNmiR7I/sq0h+TsBXUU0WhGvTl2OhQu3l9Z2kedxYkNJdXAnyXijahxB nV1A== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAuY8vlE2VipS0y6bcc2oO9C14Dv3lBAHwnblr5vK+hnxxh5KQnoq etkyaA0oVQBJesLyPdIlU07FxCxWM1Xl90Wb2CoQnqEKP4GwII90GN+2PNirCz1BW8xu/mDpZrv UnJJD2Iou7QVSrrz2nsINob3geu31Wdn7Oix7uyEH5FA66pKqT1/RSAwMLApLoJ2szw== X-Received: by 2002:a63:6f09:: with SMTP id k9mr1085943pgc.326.1550793057068; Thu, 21 Feb 2019 15:50:57 -0800 (PST) X-Google-Smtp-Source: AHgI3IaY0DzRqnxCzPxf3K+FhRY7NGJ4uzZ3CmBnKqS144T0ZmdpCxyvonxN4hpRpnO6OrB4+CJO X-Received: by 2002:a63:6f09:: with SMTP id k9mr1085901pgc.326.1550793056272; Thu, 21 Feb 2019 15:50:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793056; cv=none; d=google.com; s=arc-20160816; b=s7H+5iC0H60rLhoyfVmOW2Zl+dI6udaKfMhDbr3dajFoGKUHCigk/LdA2NRTqvHOxP cVe0aC6ioZR3ePGrxhBYwJFYpfX9bpUtNBmJcP1sGei1MxFA0CHnboS62RqzUeYY2kPT W7HJt4D7372fYVfRk7us9sHCldglRZV3xN74pPsV3pK8cYl6esf99butV4db3diM0ClC YXZmQH+s8OEK/8fYd+Wl0/SiZGn7NCzpjwV2btLXpce5zRYrxz5a03iHhN334S1ljn7Z cBjJILw21SwBoqu4U9OZl3oEx2y3960vYKk4jSdDETk+AA/t8LLP2ni9IX/8nJlJlcXw 31gw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=ai+qJljkOSM+BNa8+5qAy5ubO6YA1LWKpTMBiUyjPN0=; b=nLBqaB14XduiRlxAw58Y4wfiIsnSQSRfGnFx4eM0+13cnneal9vwdcI9FVzKnBDA7w 407a37MimoFuL9au8Eeb+D7TBK8vbhlmCoMBLVqz/VpgaLt0s+8Zezk7A7PrQWf5HWtD 8QPvxZpzHW9++PKd+Pf2HT+9zKX7E/0o8BvnfYLPX0I7GTlqoSavZmVgcFrqf0B/Ipkj obs7NpXIrBLHaTD/4ZsyHFiD8V7RX75GMn11HFUoVVVYC+UqEy8UDHONsOqI+2sIq1BE acpFpWJXQuHFJvSm1E9y416NONHOAF+cTE3fKHyHQhOUbVu6uByOwddJgkCrv81pHisn yB2Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.50.56 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:50:56 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:50:55 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394815" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:50:54 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit Subject: [PATCH v3 03/20] x86/mm: Save DRs when loading a temporary mm Date: Thu, 21 Feb 2019 15:44:34 -0800 Message-Id: <20190221234451.17632-4-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit Prevent user watchpoints from mistakenly firing while the temporary mm is being used. As the addresses that of the temporary mm might overlap those of the user-process, this is necessary to prevent wrong signals or worse things from happening. Cc: Andy Lutomirski Signed-off-by: Nadav Amit --- arch/x86/include/asm/mmu_context.h | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h index d684b954f3c0..0d6c72ece750 100644 --- a/arch/x86/include/asm/mmu_context.h +++ b/arch/x86/include/asm/mmu_context.h @@ -13,6 +13,7 @@ #include #include #include +#include extern atomic64_t last_mm_ctx_id; @@ -358,6 +359,7 @@ static inline unsigned long __get_current_cr3_fast(void) typedef struct { struct mm_struct *prev; + unsigned short bp_enabled : 1; } temp_mm_state_t; /* @@ -380,6 +382,22 @@ static inline temp_mm_state_t use_temporary_mm(struct mm_struct *mm) lockdep_assert_irqs_disabled(); state.prev = this_cpu_read(cpu_tlbstate.loaded_mm); switch_mm_irqs_off(NULL, mm, current); + + /* + * If breakpoints are enabled, disable them while the temporary mm is + * used. Userspace might set up watchpoints on addresses that are used + * in the temporary mm, which would lead to wrong signals being sent or + * crashes. + * + * Note that breakpoints are not disabled selectively, which also causes + * kernel breakpoints (e.g., perf's) to be disabled. This might be + * undesirable, but still seems reasonable as the code that runs in the + * temporary mm should be short. + */ + state.bp_enabled = hw_breakpoint_active(); + if (state.bp_enabled) + hw_breakpoint_disable(); + return state; } @@ -387,6 +405,13 @@ static inline void unuse_temporary_mm(temp_mm_state_t prev) { lockdep_assert_irqs_disabled(); switch_mm_irqs_off(NULL, prev.prev, current); + + /* + * Restore the breakpoints if they were disabled before the temporary mm + * was loaded. + */ + if (prev.bp_enabled) + hw_breakpoint_restore(); } #endif /* _ASM_X86_MMU_CONTEXT_H */ From patchwork Thu Feb 21 23:44:35 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824861 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9A0A4922 for ; Thu, 21 Feb 2019 23:51:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8800631D2C for ; Thu, 21 Feb 2019 23:51:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7C42B31D30; Thu, 21 Feb 2019 23:51:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E5B9B31D2C for ; Thu, 21 Feb 2019 23:51:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE2618E00C7; Thu, 21 Feb 2019 18:50:58 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C92E08E00B5; Thu, 21 Feb 2019 18:50:58 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B81538E00C7; Thu, 21 Feb 2019 18:50:58 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) by kanga.kvack.org (Postfix) with ESMTP id 742F28E00B5 for ; Thu, 21 Feb 2019 18:50:58 -0500 (EST) Received: by mail-pg1-f198.google.com with SMTP id f5so324914pgh.14 for ; Thu, 21 Feb 2019 15:50:58 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=c2nnTu8a9kXCHgnsdlQMzxixfmI/LTHdM4n5jWyrqqo=; b=jdbzs8BKEfPoRyqvekmzjKQiFeHYjMsYepSCaRAzI47Voo6cWcjCLGnUL4gQ+/VDmZ 2cE9Un0x9kXi+cxf9B5Z9j7++LNScurIS0v8T20NtHF7dDAZMS/84XhIhwGNQBhEjUZh qbGukro7t2vbKyHkKtr+mZNkbhBVgEArU5HwVIfp3QRXNzLZRZGJDp12YIQYC9yt22/9 4sturO1d3EmzvQeNCngUxkywwTPYgxWoiT6wTIfo7JMKdvVFE5FrWEu43bjYM6O+m40M nuVMTw7Gg5OgiqpyIeusBlHSwBQj5Iv75u7lAcPoQMX5wo+BowYjP20oH5cEnXMNFoZH ZItg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAuZ8sZPYpQc+pozHQxRnqBRE+q6qA266Gk4yn5MQJBaDigkMMzkD nwZ77l47icig7vvLWp75RkFvQPlj0QOm+3lQwZBlGMgaIgnbzugJDrN5JtpZYJuu3ZO/qHkaajU SHE5iAVffii+we5eUD/U8OwUfduqu8tqDN24DJywNVjp/WBwYiyBCArwwGacHyICREw== X-Received: by 2002:a17:902:7405:: with SMTP id g5mr1205987pll.230.1550793058141; Thu, 21 Feb 2019 15:50:58 -0800 (PST) X-Google-Smtp-Source: AHgI3IYt/tzgxcU52gLy3kO1RHqa0WWOOOavtEwD5IohD1lrUeFyDpm7QJt4xoaXlF7pwMdyvaKk X-Received: by 2002:a17:902:7405:: with SMTP id g5mr1205944pll.230.1550793057202; Thu, 21 Feb 2019 15:50:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793057; cv=none; d=google.com; s=arc-20160816; b=aI2i1hjXZkEQKrCXAbUg4bLdrRxs88OJ5w+Ifj3NZXSs8Lx3xmEhYl9RZqWn2klaxt Zrkf+Wv9xI8+jTZhrr/7zLHbS9DVNMrJENz+f1TJ2zCP9Kmfp3tAjNPy+mAz+nu42B/t OdP3u1zgTgxYpdZ5jH35pw6hs3ufyDdm40ErxMBxOCrAsz/2NX6lvkegwxPZpxp1zvYs A83e/BEVhB+1/7rqywBHoisUQ7PjVOD/aMjD5y8lts4cOnMHxbmJjN2X5aGB2ZMuM0k8 cUtkULsYzlKuOmsgkOXdMvC2NqVSIR838HQcujoPPltfxHgIeu37XJwafJTG5/oiazoq Ec5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=c2nnTu8a9kXCHgnsdlQMzxixfmI/LTHdM4n5jWyrqqo=; b=iuwxZVq5LI4J0nvdX421qQmIUhPUe9QSGm10uP3a4HjuVrAhKNnT3iI9MF5H5CF1Zz ELuro+gmU8rq4zV5Qsr/4GPyQb2NxY24UL0+Wv6h6bkXLuv+c8QpHMXEi6E6nUbD0Y2y 0TQTeb+2PzVqrDV0QJbBByB3hFqXlM/tCk53Zy+e3ix3bOYOidlm1zOj3Ajnn9RwNx5a DeSV67f74rJwQ/YEr7r6RKHIh84gY0arxoNHBLveS3bPjt4zDCvEH0QiRS9eN7XYwVCs IFOpdadF+l16KiSa5UoOD1tp6e1KQDqhtYWkLHoZwWTaQcF3rb7vv8mf8g5RpGv/qH7l QNQA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.50.57 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:50:57 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:50:56 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394821" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:50:55 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Kees Cook , Dave Hansen , Rick Edgecombe Subject: [PATCH v3 04/20] fork: Provide a function for copying init_mm Date: Thu, 21 Feb 2019 15:44:35 -0800 Message-Id: <20190221234451.17632-5-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit Provide a function for copying init_mm. This function will be later used for setting a temporary mm. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Acked-by: Peter Zijlstra (Intel) Reviewed-by: Masami Hiramatsu Tested-by: Masami Hiramatsu Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- include/linux/sched/task.h | 1 + kernel/fork.c | 24 ++++++++++++++++++------ 2 files changed, 19 insertions(+), 6 deletions(-) diff --git a/include/linux/sched/task.h b/include/linux/sched/task.h index 44c6f15800ff..c5a00a7b3beb 100644 --- a/include/linux/sched/task.h +++ b/include/linux/sched/task.h @@ -76,6 +76,7 @@ extern void exit_itimers(struct signal_struct *); extern long _do_fork(unsigned long, unsigned long, unsigned long, int __user *, int __user *, unsigned long); extern long do_fork(unsigned long, unsigned long, unsigned long, int __user *, int __user *); struct task_struct *fork_idle(int); +struct mm_struct *copy_init_mm(void); extern pid_t kernel_thread(int (*fn)(void *), void *arg, unsigned long flags); extern long kernel_wait4(pid_t, int __user *, int, struct rusage *); diff --git a/kernel/fork.c b/kernel/fork.c index b69248e6f0e0..1b43753c1884 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -1299,13 +1299,20 @@ void mm_release(struct task_struct *tsk, struct mm_struct *mm) complete_vfork_done(tsk); } -/* - * Allocate a new mm structure and copy contents from the - * mm structure of the passed in task structure. +/** + * dup_mm() - duplicates an existing mm structure + * @tsk: the task_struct with which the new mm will be associated. + * @oldmm: the mm to duplicate. + * + * Allocates a new mm structure and duplicates the provided @oldmm structure + * content into it. + * + * Return: the duplicated mm or NULL on failure. */ -static struct mm_struct *dup_mm(struct task_struct *tsk) +static struct mm_struct *dup_mm(struct task_struct *tsk, + struct mm_struct *oldmm) { - struct mm_struct *mm, *oldmm = current->mm; + struct mm_struct *mm; int err; mm = allocate_mm(); @@ -1372,7 +1379,7 @@ static int copy_mm(unsigned long clone_flags, struct task_struct *tsk) } retval = -ENOMEM; - mm = dup_mm(tsk); + mm = dup_mm(tsk, current->mm); if (!mm) goto fail_nomem; @@ -2187,6 +2194,11 @@ struct task_struct *fork_idle(int cpu) return task; } +struct mm_struct *copy_init_mm(void) +{ + return dup_mm(NULL, &init_mm); +} + /* * Ok, this is the main fork-routine. * From patchwork Thu Feb 21 23:44:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824867 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0D1CA14E1 for ; Thu, 21 Feb 2019 23:51:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EE4D531D2C for ; Thu, 21 Feb 2019 23:51:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E22AD31D30; Thu, 21 Feb 2019 23:51:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4286931D2C for ; Thu, 21 Feb 2019 23:51:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C94C78E00C9; Thu, 21 Feb 2019 18:50:59 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C20658E00B5; Thu, 21 Feb 2019 18:50:59 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A93F08E00C9; Thu, 21 Feb 2019 18:50:59 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) by kanga.kvack.org (Postfix) with ESMTP id 65F3C8E00B5 for ; Thu, 21 Feb 2019 18:50:59 -0500 (EST) Received: by mail-pg1-f199.google.com with SMTP id o24so341999pgh.5 for ; Thu, 21 Feb 2019 15:50:59 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=n3xL8tglsp1p5h94RExQEs/it39xNrHdfjkbRBXvvQk=; b=a8qP9JNkxW6mE1TkD97X9oaPgFseWbt1CqlkyOIkIMEGKBXmmXDtNRfaCbeNAyqnCS 16VZBpc3SgbDpw6dEq92js7WdJN+4GwCOegynfEQUvQtdjNKIR4BQ0hKYRSw4seOw9ij kvre3dbV1e2ZRGOVs3szKlisPkr847wYsCvuGU6W2i1bDybczuaEmD9H2EuNS8oA/+8I ZrVm65cty5I6yfA3lagCntzd6brC2ecJwlu/pYsEkngUpr7ol8Iye8vhAaSH824IWAS6 Juy8r+VnYnlXyVTuVqxRjs1cZeZahX7psltUAnNuuSLcHsECf2yfedC2dW4xr53u0FdD OhbQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAubQW0fAGeoUOu44KVqOc8g7t30Nl88St+25CW023Xw7E+GtJu2h mhBnoKpzXn9IXhHUsUT42/Myf8JZwIdQqpehiit+0ye1suLA3bPLC3zMZu6R+0dpfzQWZVA9ldS 5TR0RHMd/wJPdM4NBPIItq8H8RkMmKQBbcGYtHqsk0LXxJ7xh+N/2bRbXYw0l4ks2vA== X-Received: by 2002:a65:64d9:: with SMTP id t25mr1106496pgv.244.1550793059067; Thu, 21 Feb 2019 15:50:59 -0800 (PST) X-Google-Smtp-Source: AHgI3IbaKAPGGw6sB/35DUn1+d5SdF5gAuKmAWisRKNsL7808JH/c1VgbbwbiKVjfGkxw7j04PqZ X-Received: by 2002:a65:64d9:: with SMTP id t25mr1106450pgv.244.1550793058260; Thu, 21 Feb 2019 15:50:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793058; cv=none; d=google.com; s=arc-20160816; b=yPwcHBeU57cd2gL+64uQg3udGhlotSVjMtotzWls9o1COdh+fTZ+VyBAkVD/k0JzBX ZV+ZFmUTorr+Gzv6NZCtboklMNLCl4knSvANWAe6jM1GeQVsGwCHKv3gI63lQHNwc7Jt P1EK4JJNvfooujOBl6el8W1ZyAWpGPn6rlTTy1uCmpJlWgHGSlLwbKOhiBDvdV5qtbI4 AoY+vMR56b6aGBb0ok6kr98ILDV75suWU1NygUeybraQRlSgnCmBeu+NmpIEqtVEKfzh 1HQ+w/KgU3BJRtMLkAqGcy+ah8eefaQgY6xpMaelUfOCwIg2lqwIldL+bcY6leiepz3+ HUQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=n3xL8tglsp1p5h94RExQEs/it39xNrHdfjkbRBXvvQk=; b=jCErq1psP2RrUjjkhqcBj67TONXjuzzv7fddgMHfSYpTUdoAAJ551bcmux6RuE002Q 4RPcPveqtHg9cEapXrJBZ8iu3MbNxfZNkfmtbewJHnmQRD97MT0gT0XGvITE5Smpaeg2 vCJeS8UKnBbobnUAP3/MeW9+LUegESCyUf3sIsZB1JgmZcQm+Q4L/YHLv+S7DnFjtamG ksU5tL64lHc9srI+31CnQKfVphiCnN3YBhRsVcLHlSp9KpjkeIaRXiPMrrkWYRYYLgMf MXwIH0W5uHSGdkIGBqUirWylEFNyArv5FQ37sVrWfY9CaRYNyokX16VcFhd2V6oFmbDM QtmQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.50.58 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:50:58 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:50:57 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394827" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:50:56 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Kees Cook , Dave Hansen , Rick Edgecombe Subject: [PATCH v3 05/20] x86/alternative: Initialize temporary mm for patching Date: Thu, 21 Feb 2019 15:44:36 -0800 Message-Id: <20190221234451.17632-6-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit To prevent improper use of the PTEs that are used for text patching, the next patches will use a temporary mm struct. Initailize it by copying the init mm. The address that will be used for patching is taken from the lower area that is usually used for the task memory. Doing so prevents the need to frequently synchronize the temporary-mm (e.g., when BPF programs are installed), since different PGDs are used for the task memory. Finally, randomize the address of the PTEs to harden against exploits that use these PTEs. Cc: Kees Cook Cc: Dave Hansen Acked-by: Peter Zijlstra (Intel) Reviewed-by: Masami Hiramatsu Tested-by: Masami Hiramatsu Suggested-by: Andy Lutomirski Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/include/asm/pgtable.h | 3 +++ arch/x86/include/asm/text-patching.h | 2 ++ arch/x86/kernel/alternative.c | 3 +++ arch/x86/mm/init_64.c | 36 ++++++++++++++++++++++++++++ init/main.c | 3 +++ 5 files changed, 47 insertions(+) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 40616e805292..e8f630d9a2ed 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -1021,6 +1021,9 @@ static inline void __meminit init_trampoline_default(void) /* Default trampoline pgd value */ trampoline_pgd_entry = init_top_pgt[pgd_index(__PAGE_OFFSET)]; } + +void __init poking_init(void); + # ifdef CONFIG_RANDOMIZE_MEMORY void __meminit init_trampoline(void); # else diff --git a/arch/x86/include/asm/text-patching.h b/arch/x86/include/asm/text-patching.h index f8fc8e86cf01..a75eed841eed 100644 --- a/arch/x86/include/asm/text-patching.h +++ b/arch/x86/include/asm/text-patching.h @@ -39,5 +39,7 @@ extern void *text_poke_kgdb(void *addr, const void *opcode, size_t len); extern int poke_int3_handler(struct pt_regs *regs); extern void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler); extern int after_bootmem; +extern __ro_after_init struct mm_struct *poking_mm; +extern __ro_after_init unsigned long poking_addr; #endif /* _ASM_X86_TEXT_PATCHING_H */ diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 12fddbc8c55b..ae05fbb50171 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -678,6 +678,9 @@ void *__init_or_module text_poke_early(void *addr, const void *opcode, return addr; } +__ro_after_init struct mm_struct *poking_mm; +__ro_after_init unsigned long poking_addr; + static void *__text_poke(void *addr, const void *opcode, size_t len) { unsigned long flags; diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index bccff68e3267..125c8c48aa24 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -53,6 +53,7 @@ #include #include #include +#include #include "mm_internal.h" @@ -1383,6 +1384,41 @@ unsigned long memory_block_size_bytes(void) return memory_block_size_probed; } +/* + * Initialize an mm_struct to be used during poking and a pointer to be used + * during patching. + */ +void __init poking_init(void) +{ + spinlock_t *ptl; + pte_t *ptep; + + poking_mm = copy_init_mm(); + BUG_ON(!poking_mm); + + /* + * Randomize the poking address, but make sure that the following page + * will be mapped at the same PMD. We need 2 pages, so find space for 3, + * and adjust the address if the PMD ends after the first one. + */ + poking_addr = TASK_UNMAPPED_BASE; + if (IS_ENABLED(CONFIG_RANDOMIZE_BASE)) + poking_addr += (kaslr_get_random_long("Poking") & PAGE_MASK) % + (TASK_SIZE - TASK_UNMAPPED_BASE - 3 * PAGE_SIZE); + + if (((poking_addr + PAGE_SIZE) & ~PMD_MASK) == 0) + poking_addr += PAGE_SIZE; + + /* + * We need to trigger the allocation of the page-tables that will be + * needed for poking now. Later, poking may be performed in an atomic + * section, which might cause allocation to fail. + */ + ptep = get_locked_pte(poking_mm, poking_addr, &ptl); + BUG_ON(!ptep); + pte_unmap_unlock(ptep, ptl); +} + #ifdef CONFIG_SPARSEMEM_VMEMMAP /* * Initialise the sparsemem vmemmap using huge-pages at the PMD level. diff --git a/init/main.c b/init/main.c index e2e80ca3165a..f5947ba53bb4 100644 --- a/init/main.c +++ b/init/main.c @@ -496,6 +496,8 @@ void __init __weak thread_stack_cache_init(void) void __init __weak mem_encrypt_init(void) { } +void __init __weak poking_init(void) { } + bool initcall_debug; core_param(initcall_debug, initcall_debug, bool, 0644); @@ -730,6 +732,7 @@ asmlinkage __visible void __init start_kernel(void) taskstats_init_early(); delayacct_init(); + poking_init(); check_bugs(); acpi_subsystem_init(); From patchwork Thu Feb 21 23:44:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824869 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 40316922 for ; Thu, 21 Feb 2019 23:51:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2EA7731D2C for ; Thu, 21 Feb 2019 23:51:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 22CD831D30; Thu, 21 Feb 2019 23:51:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 507C831D2C for ; Thu, 21 Feb 2019 23:51:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E60D8E00CA; Thu, 21 Feb 2019 18:51:01 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 196668E00B5; Thu, 21 Feb 2019 18:51:01 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EB8A38E00CA; Thu, 21 Feb 2019 18:51:00 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) by kanga.kvack.org (Postfix) with ESMTP id 9EBE48E00B5 for ; Thu, 21 Feb 2019 18:51:00 -0500 (EST) Received: by mail-pl1-f197.google.com with SMTP id w17so275180plp.23 for ; Thu, 21 Feb 2019 15:51:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=O07umIFKOz6W0sIx5craaaoizFYzAP28pSU0DhhenOc=; b=s/JFMiPuGuttCeHK23zYPp/I3XYGBJnLBM6ZOJge3tT+ACiDpnpdou6wEB8F0qLfMD HvVudWsVACz7YHM/scOWrTAEgJCZZXmtXOls62qDW2RbQ5I0OsoippU3HNspn19iKNRR +D8cNXpMuWSjEF3/pG0q3V7eVD2S1891w6dir30dXM12Re4xAw6vugwMokKJBZBBfCbE aopsApij0yv2dPqCHlrCNxUbUei4uFJk96P0DxaVJBAouyCjADC+sUqOHBmXF3EbfKQw 9tk+NWA83MmZf23K5Sj1eh1XCfOjqVI+JZhUB0uOwEMjlMXmOfr+0VjMbpeZidD6HkFf 9FyQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAuaNz9F+u+0PbrcDKEZr7Z10kwgYToIhD66tfN3aIkpXjrj09hVB SXh6T0AAoYGGUzP1vp2/kmGiPilSKYkhbtCL1m6gZ2w2OmADRoyvVOTwicQApZZ/k5WZt1/SDYw HtYTlAkpKGY4eeezjfrxcIOUOQdf5/sjVrO6pM/m4ek3cr8j6T/KT+Hl+eeQzk/0DOQ== X-Received: by 2002:a17:902:147:: with SMTP id 65mr1204263plb.116.1550793060287; Thu, 21 Feb 2019 15:51:00 -0800 (PST) X-Google-Smtp-Source: AHgI3IbM0GVCwzA+o1hG/p+gTmrk93c6qhHYYnlcIeRoPmO5s7BDbOJlrIGXBPjgMyRal8w5ulYP X-Received: by 2002:a17:902:147:: with SMTP id 65mr1204206plb.116.1550793059362; Thu, 21 Feb 2019 15:50:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793059; cv=none; d=google.com; s=arc-20160816; b=mb+G4VVdpIDTa1b8kvYO6+Axtgm+iP28/wHKLnHAwuP45F0ginG4BEpnUy095mmHdo 8HSFK/FfI6G7TPfiAYAHh58RV7LgG6GZVDHEBeHdrXZ2HRAnpFroV2M87r0DoEq3fWik P1BZ4BZ+wxL/+NwIQjRMLu53hE/+TZ1jhZj4VMBXHiC+6u6BlgE15cGOGBBM8xRm3pxm q+dAmNei9KswcXGc1+zkSYizMT19HYn5typHAhhRPQJFGoTVXGX6gv9SPj5qFeZBFbj3 xwCwPTJafigJRvPAGbvBU6vWxqMmGaeFmc8LFoSSVctlplmVDxSfLBvsYRhCdoGWcZoK jOmQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=O07umIFKOz6W0sIx5craaaoizFYzAP28pSU0DhhenOc=; b=xWTIgFoGJZqpmcBdDJfNm9Nw7IJ6pceg8zHdqBBClNhVwXBGiESAC9WQClgHks4pol 51YLxcHENhiXWhE6AQENMUJN+NDgQwJIJgWFTbB/qnZQlT4V3ee+WbPSGkZNXSuCaNKT sh6I7lNtm/UNnOxOMDkgn9tdUeGAWgknLkKi9NuKH6Z9yfaG2kSbWqNrAra7YKxrBtfv gh1AY6ApT4PGo5LIUHzFCaNb91ByN76L4h7BJrS42vfsTQLknZqlKGODWJHYs/UP+Mk2 29wy84Fo8ZwSu53lQL77Yb50UBBPhGj/fIpB+kD9GasP69upnetjmfzzEuSd3Zlx4WVE dgkQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.50.59 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:50:59 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:50:58 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394832" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:50:57 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Kees Cook , Dave Hansen , Masami Hiramatsu , Rick Edgecombe Subject: [PATCH v3 06/20] x86/alternative: Use temporary mm for text poking Date: Thu, 21 Feb 2019 15:44:37 -0800 Message-Id: <20190221234451.17632-7-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit text_poke() can potentially compromise security as it sets temporary PTEs in the fixmap. These PTEs might be used to rewrite the kernel code from other cores accidentally or maliciously, if an attacker gains the ability to write onto kernel memory. Moreover, since remote TLBs are not flushed after the temporary PTEs are removed, the time-window in which the code is writable is not limited if the fixmap PTEs - maliciously or accidentally - are cached in the TLB. To address these potential security hazards, use a temporary mm for patching the code. Finally, text_poke() is also not conservative enough when mapping pages, as it always tries to map 2 pages, even when a single one is sufficient. So try to be more conservative, and do not map more than needed. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Cc: Masami Hiramatsu Acked-by: Peter Zijlstra (Intel) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/include/asm/fixmap.h | 2 - arch/x86/kernel/alternative.c | 108 +++++++++++++++++++++++++++------- arch/x86/xen/mmu_pv.c | 2 - 3 files changed, 86 insertions(+), 26 deletions(-) diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h index 50ba74a34a37..9da8cccdf3fb 100644 --- a/arch/x86/include/asm/fixmap.h +++ b/arch/x86/include/asm/fixmap.h @@ -103,8 +103,6 @@ enum fixed_addresses { #ifdef CONFIG_PARAVIRT FIX_PARAVIRT_BOOTMAP, #endif - FIX_TEXT_POKE1, /* reserve 2 pages for text_poke() */ - FIX_TEXT_POKE0, /* first page is last, because allocation is backward */ #ifdef CONFIG_X86_INTEL_MID FIX_LNW_VRTC, #endif diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index ae05fbb50171..cfe5bfe06f9d 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -683,41 +684,104 @@ __ro_after_init unsigned long poking_addr; static void *__text_poke(void *addr, const void *opcode, size_t len) { + bool cross_page_boundary = offset_in_page(addr) + len > PAGE_SIZE; + struct page *pages[2] = {NULL}; + temp_mm_state_t prev; unsigned long flags; - char *vaddr; - struct page *pages[2]; - int i; + pte_t pte, *ptep; + spinlock_t *ptl; + pgprot_t pgprot; /* - * While boot memory allocator is runnig we cannot use struct - * pages as they are not yet initialized. + * While boot memory allocator is running we cannot use struct pages as + * they are not yet initialized. There is no way to recover. */ BUG_ON(!after_bootmem); if (!core_kernel_text((unsigned long)addr)) { pages[0] = vmalloc_to_page(addr); - pages[1] = vmalloc_to_page(addr + PAGE_SIZE); + if (cross_page_boundary) + pages[1] = vmalloc_to_page(addr + PAGE_SIZE); } else { pages[0] = virt_to_page(addr); WARN_ON(!PageReserved(pages[0])); - pages[1] = virt_to_page(addr + PAGE_SIZE); + if (cross_page_boundary) + pages[1] = virt_to_page(addr + PAGE_SIZE); } - BUG_ON(!pages[0]); + /* + * If something went wrong, crash and burn since recovery paths are not + * implemented. + */ + BUG_ON(!pages[0] || (cross_page_boundary && !pages[1])); + local_irq_save(flags); - set_fixmap(FIX_TEXT_POKE0, page_to_phys(pages[0])); - if (pages[1]) - set_fixmap(FIX_TEXT_POKE1, page_to_phys(pages[1])); - vaddr = (char *)fix_to_virt(FIX_TEXT_POKE0); - memcpy(&vaddr[(unsigned long)addr & ~PAGE_MASK], opcode, len); - clear_fixmap(FIX_TEXT_POKE0); - if (pages[1]) - clear_fixmap(FIX_TEXT_POKE1); - local_flush_tlb(); - sync_core(); - /* Could also do a CLFLUSH here to speed up CPU recovery; but - that causes hangs on some VIA CPUs. */ - for (i = 0; i < len; i++) - BUG_ON(((char *)addr)[i] != ((char *)opcode)[i]); + + /* + * Map the page without the global bit, as TLB flushing is done with + * flush_tlb_mm_range(), which is intended for non-global PTEs. + */ + pgprot = __pgprot(pgprot_val(PAGE_KERNEL) & ~_PAGE_GLOBAL); + + /* + * The lock is not really needed, but this allows to avoid open-coding. + */ + ptep = get_locked_pte(poking_mm, poking_addr, &ptl); + + /* + * This must not fail; preallocated in poking_init(). + */ + VM_BUG_ON(!ptep); + + pte = mk_pte(pages[0], pgprot); + set_pte_at(poking_mm, poking_addr, ptep, pte); + + if (cross_page_boundary) { + pte = mk_pte(pages[1], pgprot); + set_pte_at(poking_mm, poking_addr + PAGE_SIZE, ptep + 1, pte); + } + + /* + * Loading the temporary mm behaves as a compiler barrier, which + * guarantees that the PTE will be set at the time memcpy() is done. + */ + prev = use_temporary_mm(poking_mm); + + kasan_disable_current(); + memcpy((u8 *)poking_addr + offset_in_page(addr), opcode, len); + kasan_enable_current(); + + /* + * Ensure that the PTE is only cleared after the instructions of memcpy + * were issued by using a compiler barrier. + */ + barrier(); + + pte_clear(poking_mm, poking_addr, ptep); + if (cross_page_boundary) + pte_clear(poking_mm, poking_addr + PAGE_SIZE, ptep + 1); + + /* + * Loading the previous page-table hierarchy requires a serializing + * instruction that already allows the core to see the updated version. + * Xen-PV is assumed to serialize execution in a similar manner. + */ + unuse_temporary_mm(prev); + + /* + * Flushing the TLB might involve IPIs, which would require enabled + * IRQs, but not if the mm is not used, as it is in this point. + */ + flush_tlb_mm_range(poking_mm, poking_addr, poking_addr + + (cross_page_boundary ? 2 : 1) * PAGE_SIZE, + PAGE_SHIFT, false); + + /* + * If the text does not match what we just wrote then something is + * fundamentally screwy; there's nothing we can really do about that. + */ + BUG_ON(memcmp(addr, opcode, len)); + + pte_unmap_unlock(ptep, ptl); local_irq_restore(flags); return addr; } diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c index 0f4fe206dcc2..82b181fcefe5 100644 --- a/arch/x86/xen/mmu_pv.c +++ b/arch/x86/xen/mmu_pv.c @@ -2319,8 +2319,6 @@ static void xen_set_fixmap(unsigned idx, phys_addr_t phys, pgprot_t prot) #elif defined(CONFIG_X86_VSYSCALL_EMULATION) case VSYSCALL_PAGE: #endif - case FIX_TEXT_POKE0: - case FIX_TEXT_POKE1: /* All local page mappings */ pte = pfn_pte(phys, prot); break; From patchwork Thu Feb 21 23:44:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824871 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 436C5922 for ; Thu, 21 Feb 2019 23:51:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3162631D2C for ; Thu, 21 Feb 2019 23:51:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2554E31D30; Thu, 21 Feb 2019 23:51:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B112E31D2C for ; Thu, 21 Feb 2019 23:51:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C716B8E00CB; Thu, 21 Feb 2019 18:51:01 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BCCEB8E00B5; Thu, 21 Feb 2019 18:51:01 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A72A48E00CB; Thu, 21 Feb 2019 18:51:01 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) by kanga.kvack.org (Postfix) with ESMTP id 5D36F8E00B5 for ; Thu, 21 Feb 2019 18:51:01 -0500 (EST) Received: by mail-pf1-f199.google.com with SMTP id r9so334862pfb.13 for ; Thu, 21 Feb 2019 15:51:01 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=H4uqmF5HLlBo9r9rzOTTevq+cJIiPJBHTeYeqiaw1PU=; b=SmYNvdWr1ypJ0cO/ISo6EIS+s5IDUZY4/jv1M/UOfMlFnZ5mRJ3sxRwT91eicP0b74 V3wstIJ1JhVfEisQWMMo5xypIe+do11qOrtO8jS4eX3hTAhaZZ8MKzeaoe4cVYls01vC 2lf+d6SP+a4xZgpj9X8HswJus/x3RkbuEbbbsZdz+XIlIIrIQd8NL7rDe9fnSCb88Gqh DsGGaBhSOF0+za4RS1eGofGzipwC93vDLFVXk9EPLtDsqP6MQXoW1hCcMaXD0FbS1SfV AiPrm8HEbQU+vNWHJTEYtE0Tcg+wExE97QD/F2zjLPCn/glw6ryylBOD5Pv+1jTV9AQU tCJg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAubznRvXhtud0xIisnVRxCOMP+nXUPuaRk2A4vyX5yHPVFV+4RUL cT3GW15T6Eir+sf69FrdlpT7OoFU0TMAsfhFWR9EZa4F4UFZga1fWSw3F1S9i09D6VwlH1mKTlX CqPiC+fivz4YpMPPQaHQZ87X/IN1IYTanGG87ASN7YgKobAE7G99ZLKxR159064/oAg== X-Received: by 2002:a17:902:6a3:: with SMTP id 32mr1144026plh.319.1550793061060; Thu, 21 Feb 2019 15:51:01 -0800 (PST) X-Google-Smtp-Source: AHgI3IaLeqcjnmLxBn7QOcuM7v5cm93ESGmTR0hauSU+DXTgCmTl4Sw5sg6553rX8+FYxXFV8nVn X-Received: by 2002:a17:902:6a3:: with SMTP id 32mr1143991plh.319.1550793060269; Thu, 21 Feb 2019 15:51:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793060; cv=none; d=google.com; s=arc-20160816; b=ayb/kVb8u2X1RTjKFdbJ02bjwGYIDahAV0IH19eTUONXJbVhfoCbVESW1BYcVxfujo 2n8U1vXNtutogRRUHZo5p+EnUiUQNu3mL9wbyptoSCdhAJwACqw2dKj7K8qSDa2Hthbr +Nb0NKIsnnO6TkVyzQKH5dsry2RQKtdRiRHmqB5W1e3AhK5NueaQYA4TMpytwMYRztsY Mkl9TuoTocP+IBF+0sOkF8rvjrz9EpVsydEUWId3C/Af2APlqTzAtOzQzVZZILpt77is cBfJcWyIMikZNJiuoEkd+YiByPB5fXGuugb4ATaQ+XlNMCX+K/WG6CsPT/rzBwGNEWSO TYGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=H4uqmF5HLlBo9r9rzOTTevq+cJIiPJBHTeYeqiaw1PU=; b=XlHzdbDrM7mHjWh9imRiWN+oyG030aEWxdkX0yP4ehZxVt0bYl6Kzst18jgSpToh6C Fu7pw1GmlxnzOvs/yYfjXPIjbfsLnP6sFxllWQFYi4LBpLUWK1gBfjK/SA9YGfEHVd5x HMre8kjKGL9aaPPSw7VL7mRngCJlNnV2sRSyuMr6vjZZ2wIYQ3Fl2j8cuB71lfKFDSeQ PduYaVGjRnQiQmwQC1RT31jrdyb2e66gXs0cum9oGanSLACqC7DtGIVW8EOfwve9Syh6 tq9xQluc1LIwQ4vmdrW1tpFHkepBtlXLklSvrinky/InAwumSdRn6KGDfdqWoS62zyn4 TrBw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.51.00 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:51:00 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:50:59 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394837" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:50:58 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Rick Edgecombe Subject: [PATCH v3 07/20] x86/kgdb: Avoid redundant comparison of patched code Date: Thu, 21 Feb 2019 15:44:38 -0800 Message-Id: <20190221234451.17632-8-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit text_poke() already ensures that the written value is the correct one and fails if that is not the case. There is no need for an additional comparison. Remove it. Acked-by: Peter Zijlstra (Intel) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/kgdb.c | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-) diff --git a/arch/x86/kernel/kgdb.c b/arch/x86/kernel/kgdb.c index 1461544cba8b..057af9187a04 100644 --- a/arch/x86/kernel/kgdb.c +++ b/arch/x86/kernel/kgdb.c @@ -746,7 +746,6 @@ void kgdb_arch_set_pc(struct pt_regs *regs, unsigned long ip) int kgdb_arch_set_breakpoint(struct kgdb_bkpt *bpt) { int err; - char opc[BREAK_INSTR_SIZE]; bpt->type = BP_BREAKPOINT; err = probe_kernel_read(bpt->saved_instr, (char *)bpt->bpt_addr, @@ -765,11 +764,6 @@ int kgdb_arch_set_breakpoint(struct kgdb_bkpt *bpt) return -EBUSY; text_poke_kgdb((void *)bpt->bpt_addr, arch_kgdb_ops.gdb_bpt_instr, BREAK_INSTR_SIZE); - err = probe_kernel_read(opc, (char *)bpt->bpt_addr, BREAK_INSTR_SIZE); - if (err) - return err; - if (memcmp(opc, arch_kgdb_ops.gdb_bpt_instr, BREAK_INSTR_SIZE)) - return -EINVAL; bpt->type = BP_POKE_BREAKPOINT; return err; @@ -777,9 +771,6 @@ int kgdb_arch_set_breakpoint(struct kgdb_bkpt *bpt) int kgdb_arch_remove_breakpoint(struct kgdb_bkpt *bpt) { - int err; - char opc[BREAK_INSTR_SIZE]; - if (bpt->type != BP_POKE_BREAKPOINT) goto knl_write; /* @@ -790,10 +781,7 @@ int kgdb_arch_remove_breakpoint(struct kgdb_bkpt *bpt) goto knl_write; text_poke_kgdb((void *)bpt->bpt_addr, bpt->saved_instr, BREAK_INSTR_SIZE); - err = probe_kernel_read(opc, (char *)bpt->bpt_addr, BREAK_INSTR_SIZE); - if (err || memcmp(opc, bpt->saved_instr, BREAK_INSTR_SIZE)) - goto knl_write; - return err; + return 0; knl_write: return probe_kernel_write((char *)bpt->bpt_addr, From patchwork Thu Feb 21 23:44:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824877 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D29AB1399 for ; Thu, 21 Feb 2019 23:51:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BD3A631D2C for ; Thu, 21 Feb 2019 23:51:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B192131D30; Thu, 21 Feb 2019 23:51:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 992D831D2C for ; Thu, 21 Feb 2019 23:51:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0D9958E00CC; Thu, 21 Feb 2019 18:51:03 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 00D7B8E00B5; Thu, 21 Feb 2019 18:51:02 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E66E58E00CC; Thu, 21 Feb 2019 18:51:02 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) by kanga.kvack.org (Postfix) with ESMTP id 9F29B8E00B5 for ; Thu, 21 Feb 2019 18:51:02 -0500 (EST) Received: by mail-pf1-f200.google.com with SMTP id r9so334898pfb.13 for ; Thu, 21 Feb 2019 15:51:02 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=hQkTPFjoHK3661xQlHVp+31XY04uuglTGuhGFhn0F6E=; b=mR6TEWM2MVJXRRDczJqHDvFIhbGXFJ3l7l1kjC5cJTtR0GmxqhbOMilTKTRLwtLdyz BLpWsQ+5apSXej1VtU9fmgxnEYXLQp0FkYFK4RTZFnij63oHidE4z58ARKYH4rej6bbh k+acyq3nyVqXU29xsH3LrrqOdAhjeR4f8gRfo8FAUMl2956XxK9UVCwzcA4wcAolwzRx tn7ZppuuCYMaqHo2s/Mn914L7tq5bIL4yzJBbwAn9dc1pM3oa4mjjlvXtfsg3Qif1jUC ckrQgbzPqFELRLrg8VUUyhyYXC5TUzt1MJnClIZIavaJF3AqCSDbgJ4w/l3GQ2mlKxaD yV8Q== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAuZns1GfzuS/N5wVbNWbqf9SXBg1RjPbFdQzZrXYfzyGhajUfQCZ a2EMHdwfi1XAfLMNvp4XZgjpe7Qsd8/AJmOYS6FwpWQN2IPLIROwwfh4H8Ja0YrOD6cGNIl+Oll AOl4Cuc3cfg5/iBkKYFAr0qAn19ubbEAsz6l5JhZHAoTI4eQXIYtmENY2E3BGAPM+Iw== X-Received: by 2002:a63:5b1f:: with SMTP id p31mr1107463pgb.56.1550793062309; Thu, 21 Feb 2019 15:51:02 -0800 (PST) X-Google-Smtp-Source: AHgI3IbsQu4L3cnlUlXL60gc/3X48ZYGQfx5SOQnlm4DR41RHTOIC3RswgNfDZ/t9DiRBB5IvH8J X-Received: by 2002:a63:5b1f:: with SMTP id p31mr1107407pgb.56.1550793061190; Thu, 21 Feb 2019 15:51:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793061; cv=none; d=google.com; s=arc-20160816; b=h8K/h1cQ3wDqh2meH5Fg3G65y1WiUmCXmWDCCW3SrTGrIhhAVzaSc1iqzubgMJBjVq J5PnilR0Xk4OdaS6k6deZr+jUFt3XUk5rB35eyV8dr5N/c1TmuMKEe8ILgBNVIiEEkQZ UTxhM9D4+mysTxO9L9iYesLVXKlEudRVLptOGETcBqchS+kdil5CZ6Lgd/0obvLNKgJ8 ExG8qHc0Mamap/MoXNmjFjSsBtbWQXQJSMke4m4RugcRER7WMYjkcXKFH32V+vs0e/YR As/yMNUFvAl3cRfdrKhL2rYSlNwEPXyEIqsmXWn/VsnneytdUxLwnvnPVw7W0+ANotlm Bi3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=hQkTPFjoHK3661xQlHVp+31XY04uuglTGuhGFhn0F6E=; b=WbnsL/gUPcS0zFReJDo//9ffiw+Uxz4OX0kJnRybnbze9WR0RdNq8d53zPy5TnurB/ Rq31OuLjatRRutyJjHOJc5JpTxpxEEYmZlasF1VvH093g/A4GBbtiuYCgjOw2iw8X/BC K5XURfo8Kwi3gPPBrBJVI623FI6bwkEIr5XXliuv+HSd2EskIf6e6h/Rp/wkZk2ii+05 M58/evWxZv/puL6D+NpDhriS58N5uNkTBEMZ1XV1oEvHkIuEVmWZtXptry75r1mEr9u5 JPqKYJ80S8MNuspvyg2h2SU2TjGQVjGZCc1bBLVJwuGRYKO8DsPOIqrO+IyAZjdb8hmV wo1w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.51.01 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:51:01 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:51:00 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394844" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:50:59 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Rick Edgecombe Subject: [PATCH v3 08/20] x86/ftrace: Set trampoline pages as executable Date: Thu, 21 Feb 2019 15:44:39 -0800 Message-Id: <20190221234451.17632-9-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit Since alloc_module() will not set the pages as executable soon, set ftrace trampoline pages as executable after they are allocated. For the time being, do not change ftrace to use the text_poke() interface. As a result, ftrace still breaks W^X. Reviewed-by: Steven Rostedt (VMware) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/ftrace.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c index 8257a59704ae..13c8249b197f 100644 --- a/arch/x86/kernel/ftrace.c +++ b/arch/x86/kernel/ftrace.c @@ -742,6 +742,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size) unsigned long end_offset; unsigned long op_offset; unsigned long offset; + unsigned long npages; unsigned long size; unsigned long retq; unsigned long *ptr; @@ -774,6 +775,7 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size) return 0; *tramp_size = size + RET_SIZE + sizeof(void *); + npages = DIV_ROUND_UP(*tramp_size, PAGE_SIZE); /* Copy ftrace_caller onto the trampoline memory */ ret = probe_kernel_read(trampoline, (void *)start_offset, size); @@ -818,6 +820,12 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size) /* ALLOC_TRAMP flags lets us know we created it */ ops->flags |= FTRACE_OPS_FL_ALLOC_TRAMP; + /* + * Module allocation needs to be completed by making the page + * executable. The page is still writable, which is a security hazard, + * but anyhow ftrace breaks W^X completely. + */ + set_memory_x((unsigned long)trampoline, npages); return (unsigned long)trampoline; fail: tramp_free(trampoline, *tramp_size); From patchwork Thu Feb 21 23:44:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824879 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 624C21399 for ; Thu, 21 Feb 2019 23:51:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 54F4E31D2C for ; Thu, 21 Feb 2019 23:51:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4850D31D30; Thu, 21 Feb 2019 23:51:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CB20031D2C for ; Thu, 21 Feb 2019 23:51:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8EE1F8E00B5; Thu, 21 Feb 2019 18:51:03 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 872388E00CD; Thu, 21 Feb 2019 18:51:03 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 78C238E00B5; Thu, 21 Feb 2019 18:51:03 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id 371998E00CD for ; Thu, 21 Feb 2019 18:51:03 -0500 (EST) Received: by mail-pf1-f198.google.com with SMTP id o67so313706pfa.20 for ; Thu, 21 Feb 2019 15:51:03 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=3G8hdtdnz6A4RGh5p5LJ2XCAn2wTvpUmINy+UI0phGA=; b=TWdRv2dLceYFqmzfPIpAAjvt0yjTrujvvJceCM/B5ZYvbgjG+suaWVTuTB/bJ6vrYN EXt0AFNqNKcUR7j2JlckmAJd6+n2xcvLMvXmjM5HZcVcpCrYEVegcsEs2FEUZUxgIenl ha3vfhpn9GHzM+1O1qu8t4WlUU90YEHNSatA7Q/PxuwLOw9rnDh10eLWDqGxmg5GcepT ep7JMiCHDzcBdjaUFfpM9N4aYq55Dz1wgxpDc10dfoxnFUCe30LnhO6CyBq3HuN/bpDN OCr34NWUXQwsgbbd3HfAsCg4PieX2zBVZscFOKrPHw9k9eGXRxdu8cLpbbcB3eESj7Bz cM6g== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAuaoELtUW8a/Fp/9cCLfpEkSF8y+/kCN+N9Sf32oRq+lREQ/0LG9 xuDf6IjyRZmNvK7p+cxKk9YsSlfrCBesH+mSbwj2dlEBSM2ydAxaz5sKiRBJ7vD4VXXSoIaTZZQ ulFbtDqZsKEX1NUt4bHz+Z8uVKbbM72UKztpihZ26Ca3Os+JbuJzX5viVIAMmNOgLBA== X-Received: by 2002:a63:cf01:: with SMTP id j1mr1076303pgg.342.1550793062887; Thu, 21 Feb 2019 15:51:02 -0800 (PST) X-Google-Smtp-Source: AHgI3IYrwJja9IeGg3jIRfbsQEviRqy255Ilib6t5yAWInYpd7qvnpxA3tYQmb1E9SosmdfjR8WD X-Received: by 2002:a63:cf01:: with SMTP id j1mr1076279pgg.342.1550793062212; Thu, 21 Feb 2019 15:51:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793062; cv=none; d=google.com; s=arc-20160816; b=lUg9IIzzcVI6NafLALPr2Y1BlLVTjoA+Ftcv0GVVAnARtt15MASuo3s1Wm/U2e9c/0 pr/MkE/dGHrJqMkca94QLMMSbB2J2cXAjbHStKxdkpAEktXlHDzbN/qM4jBgXLt3j6kN 6ZlScDNh6nEyLYTiDOUyYupmUvS0Bu2yKmc70ajXxXPZQCmBlcVWnXcZM60e7+xeXD8u ZOq6/W9UhUQJUoNqLuvIigwZOuHv5dIYyRs/beKZ8j5CsSPzbfT28uRkYx3KqxQkBdqC 1PkKlp0gQ/b7IFl6bK0nZ8Wy7H/MwpEiaeAa5577s+uCuRHpSO+IHfmJMpd9JI3rVZ3f 6O8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=3G8hdtdnz6A4RGh5p5LJ2XCAn2wTvpUmINy+UI0phGA=; b=gTOIlaioUm4rOFSU+TaKs35O4tJtmBfjgsWvqb1PD8RoRr/icyTC0UGZFCrpe+jqkT orr3C1GIUcrRLe1tUyd+iT4nURxI3T6rYO1n3qdTgA2aWkl+XGE5huIximirEtV0irZl k2f0WjDpkTyAw+UfcXXSRwkYWlx4JjgeooiOt/txSKfQdtYVeLk2aMXlyJ0cDdAl/gmM UurVlOsy0hFA/HVl7tRrpCLvhGBx/0qdPUYgHjlRLVWbZGnPuSBv0IiHBBQJ0dGLJeWG v1GOkWGRZrx4eueVtiwyVUG0Cs6r9n6RnZhmGMxxOnb3lBcIUdbJ1OJEToH/cLH2FbJ7 Hyqw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.51.02 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:51:02 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:51:01 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394901" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:51:00 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Rick Edgecombe Subject: [PATCH v3 09/20] x86/kprobes: Set instruction page as executable Date: Thu, 21 Feb 2019 15:44:40 -0800 Message-Id: <20190221234451.17632-10-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit This patch is a preparatory patch for a following patch that makes module allocated pages non-executable. The patch sets the page as executable after allocation. While at it, do some small cleanup of what appears to be unnecessary masking. Acked-by: Masami Hiramatsu Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/kprobes/core.c | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c index 4ba75afba527..98c671e89889 100644 --- a/arch/x86/kernel/kprobes/core.c +++ b/arch/x86/kernel/kprobes/core.c @@ -431,8 +431,20 @@ void *alloc_insn_page(void) void *page; page = module_alloc(PAGE_SIZE); - if (page) - set_memory_ro((unsigned long)page & PAGE_MASK, 1); + if (!page) + return NULL; + + /* + * First make the page read-only, and only then make it executable to + * prevent it from being W+X in between. + */ + set_memory_ro((unsigned long)page, 1); + + /* + * TODO: Once additional kernel code protection mechanisms are set, ensure + * that the page was not maliciously altered and it is still zeroed. + */ + set_memory_x((unsigned long)page, 1); return page; } @@ -440,8 +452,12 @@ void *alloc_insn_page(void) /* Recover page to RW mode before releasing it */ void free_insn_page(void *page) { - set_memory_nx((unsigned long)page & PAGE_MASK, 1); - set_memory_rw((unsigned long)page & PAGE_MASK, 1); + /* + * First make the page non-executable, and only then make it writable to + * prevent it from being W+X in between. + */ + set_memory_nx((unsigned long)page, 1); + set_memory_rw((unsigned long)page, 1); module_memfree(page); } From patchwork Thu Feb 21 23:44:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824881 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F2F84922 for ; Thu, 21 Feb 2019 23:51:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DFFB731D2C for ; Thu, 21 Feb 2019 23:51:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D36A231D30; Thu, 21 Feb 2019 23:51:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 35DBC31D2C for ; Thu, 21 Feb 2019 23:51:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 086CD8E00CE; Thu, 21 Feb 2019 18:51:05 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 00F698E00CD; Thu, 21 Feb 2019 18:51:04 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E1C9F8E00CE; Thu, 21 Feb 2019 18:51:04 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) by kanga.kvack.org (Postfix) with ESMTP id 986438E00CD for ; Thu, 21 Feb 2019 18:51:04 -0500 (EST) Received: by mail-pg1-f198.google.com with SMTP id f5so325065pgh.14 for ; Thu, 21 Feb 2019 15:51:04 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=PGQf8pkpnNe3t3IPgvET7mlJMbEiHMZ/T85JdFzL/O4=; b=pTS8J/oZ1jAsZ+pLc02hYzEHwg0dinEv5c2K/518NigOsrT5YycEkQrJ4mqZlWPAhV q5VrqHnOSNW5YGlWFrPVT+7X0KtdvjVxEuMGneUDtidrOEKUacV+yJMAXhTK6XZGFtd8 guUykyLMEQZS8g/utKE7DealomAFEmg2pbBpVbgF5K23luk6U8d7fOp/PhxNrhzT9Cgp uJpzLC3e602t8ACQgBC5IoKD++MzJWoCGUjN03rJRiUW2q7vbQJczA+VTZUv/fYTxibE XBpxTAvdCm+lRxy/GAOUA/nBbcyfvqTj2lFcHEm6syKMHqar2ddhNPIirzW1VHQyx8vA qP6g== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAub7cMpHZJkqidyKsFF2/klklwKtn3qPVUQrRhtCJ1/lejvhR8So hrFWfi24kHratfKsUxdrB0dvVq8BeXA1RgKv7V3tQsatR/OiQYzyD2UXxi9p6iwohIn3gRBltji NVwnmYq8XIKxm4PAbAhG5LcTpHtfpKsaizkmX/tG6cCTHdBfRko8/fSwnI2CR5Lpvrg== X-Received: by 2002:a17:902:9307:: with SMTP id bc7mr1176530plb.234.1550793064269; Thu, 21 Feb 2019 15:51:04 -0800 (PST) X-Google-Smtp-Source: AHgI3IYP74AcN7Op67i66gEMSRRXSZkZoE36A/ZgIIIIW1uMdNx+Do2/FZevRvVmFc0QRc7o2srl X-Received: by 2002:a17:902:9307:: with SMTP id bc7mr1176484plb.234.1550793063357; Thu, 21 Feb 2019 15:51:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793063; cv=none; d=google.com; s=arc-20160816; b=TVDPNJD5tN58OVutl2htWP1QYamRXkYufzC5yimsVn09GKaGBDbleRXIyuiwHPLtSl aBs8lp2Nvm3i87P6w8meRvbgR+UHtwG5qOO4lea9UzyNrJcP1eHyhHJCLevybXDYmCbY ptU09V7IKcfLFMZKkg5Egn5H0i0rPTTCBidlMNYkwOdkcNG8vbD7osfFXRVk9pCCnOIg oeg7Ph2DZFZLe1a301gveYkzTWKJWk32hcLL8NaBsBes7lmVmtN5mTdbLRmQ/RlUHzfR g+W38F0PBkMZdmODV/7ZGEQGQWAdWtgJ3VgVBJv0qIN+ZE9K1FO8/TTV4Ho8KZfeDrl1 J9Ew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=PGQf8pkpnNe3t3IPgvET7mlJMbEiHMZ/T85JdFzL/O4=; b=uyEfw7pDGh/wEW336zMVxeg13ZFfQ0fn21L7ShGFo741OIaSHkUPwkFLzd0q8bd47y TrbRzMgw1GkiKfPubS3OzN4aJbzK0Q/vZsi0wzrNNOicp/GA+OiKLjtbFWkqNW1AJ5d4 UUg08YIenkmSlCwQJWKcXrY5Usu5AHNGsSU9ppBRzrEAVx81Hl4rf/I+8Fgsg7kh5bpv LC3tASfnXRQdYzkCXBiq05eEuvTCRD8gjgXJD8h693aeTYY+HPVAH4FC1egdIEuotPsy vpop33Lrr8y7Qolu72GDisOrP2IEcgVsTVvKNuxzuhGffJ1KgG4Z/JnkqYfm9FvrRgP2 jlCA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.51.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:51:03 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:51:02 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394912" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:51:01 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Kees Cook , Dave Hansen , Masami Hiramatsu , Jessica Yu , Rick Edgecombe Subject: [PATCH v3 10/20] x86/module: Avoid breaking W^X while loading modules Date: Thu, 21 Feb 2019 15:44:41 -0800 Message-Id: <20190221234451.17632-11-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit When modules and BPF filters are loaded, there is a time window in which some memory is both writable and executable. An attacker that has already found another vulnerability (e.g., a dangling pointer) might be able to exploit this behavior to overwrite kernel code. This patch prevents having writable executable PTEs in this stage. In addition, avoiding having W+X mappings can also slightly simplify the patching of modules code on initialization (e.g., by alternatives and static-key), as would be done in the next patch. This was actually the main motivation for this patch. To avoid having W+X mappings, set them initially as RW (NX) and after they are set as RO set them as X as well. Setting them as executable is done as a separate step to avoid one core in which the old PTE is cached (hence writable), and another which sees the updated PTE (executable), which would break the W^X protection. Cc: Kees Cook Cc: Peter Zijlstra Cc: Dave Hansen Cc: Masami Hiramatsu Cc: Jessica Yu Suggested-by: Thomas Gleixner Suggested-by: Andy Lutomirski Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/alternative.c | 28 +++++++++++++++++++++------- arch/x86/kernel/module.c | 2 +- include/linux/filter.h | 1 + kernel/module.c | 5 +++++ 4 files changed, 28 insertions(+), 8 deletions(-) diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index cfe5bfe06f9d..b75bfeda021e 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -667,15 +667,29 @@ void __init alternative_instructions(void) * handlers seeing an inconsistent instruction while you patch. */ void *__init_or_module text_poke_early(void *addr, const void *opcode, - size_t len) + size_t len) { unsigned long flags; - local_irq_save(flags); - memcpy(addr, opcode, len); - local_irq_restore(flags); - sync_core(); - /* Could also do a CLFLUSH here to speed up CPU recovery; but - that causes hangs on some VIA CPUs. */ + + if (boot_cpu_has(X86_FEATURE_NX) && + is_module_text_address((unsigned long)addr)) { + /* + * Modules text is marked initially as non-executable, so the + * code cannot be running and speculative code-fetches are + * prevented. Just change the code. + */ + memcpy(addr, opcode, len); + } else { + local_irq_save(flags); + memcpy(addr, opcode, len); + local_irq_restore(flags); + sync_core(); + + /* + * Could also do a CLFLUSH here to speed up CPU recovery; but + * that causes hangs on some VIA CPUs. + */ + } return addr; } diff --git a/arch/x86/kernel/module.c b/arch/x86/kernel/module.c index b052e883dd8c..cfa3106faee4 100644 --- a/arch/x86/kernel/module.c +++ b/arch/x86/kernel/module.c @@ -87,7 +87,7 @@ void *module_alloc(unsigned long size) p = __vmalloc_node_range(size, MODULE_ALIGN, MODULES_VADDR + get_module_load_offset(), MODULES_END, GFP_KERNEL, - PAGE_KERNEL_EXEC, 0, NUMA_NO_NODE, + PAGE_KERNEL, 0, NUMA_NO_NODE, __builtin_return_address(0)); if (p && (kasan_module_alloc(p, size) < 0)) { vfree(p); diff --git a/include/linux/filter.h b/include/linux/filter.h index d531d4250bff..b9f93e62db96 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -694,6 +694,7 @@ static inline void bpf_prog_unlock_ro(struct bpf_prog *fp) static inline void bpf_jit_binary_lock_ro(struct bpf_binary_header *hdr) { set_memory_ro((unsigned long)hdr, hdr->pages); + set_memory_x((unsigned long)hdr, hdr->pages); } static inline void bpf_jit_binary_unlock_ro(struct bpf_binary_header *hdr) diff --git a/kernel/module.c b/kernel/module.c index 2ad1b5239910..ae1b77da6a20 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -1950,8 +1950,13 @@ void module_enable_ro(const struct module *mod, bool after_init) return; frob_text(&mod->core_layout, set_memory_ro); + frob_text(&mod->core_layout, set_memory_x); + frob_rodata(&mod->core_layout, set_memory_ro); + frob_text(&mod->init_layout, set_memory_ro); + frob_text(&mod->init_layout, set_memory_x); + frob_rodata(&mod->init_layout, set_memory_ro); if (after_init) From patchwork Thu Feb 21 23:44:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824891 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 43293922 for ; Thu, 21 Feb 2019 23:51:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 324BC31D2C for ; Thu, 21 Feb 2019 23:51:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2650631D5B; Thu, 21 Feb 2019 23:51:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9406C31D3C for ; Thu, 21 Feb 2019 23:51:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DA47B8E00CF; Thu, 21 Feb 2019 18:51:05 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D7FCE8E00CD; Thu, 21 Feb 2019 18:51:05 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF4FF8E00CF; Thu, 21 Feb 2019 18:51:05 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) by kanga.kvack.org (Postfix) with ESMTP id 7EC3E8E00CD for ; Thu, 21 Feb 2019 18:51:05 -0500 (EST) Received: by mail-pl1-f197.google.com with SMTP id z1so310456pln.11 for ; Thu, 21 Feb 2019 15:51:05 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=g3nO4xFM3yb0X+OIqTEtEp6LbffMj61RqStKCNeSSiM=; b=qc74dj2/rpjaBZkFInegxLpCxx0xwQUBjOsZPGs+vreArroybHmRQl2MChKYU2RHnG Y8I8lHtawJAFkKbC/A0hoOdinKgjaiTEHNG1XeDpE0MWxuWUFiuxXInWbahIrrG4tGYQ 11KZPlQ19tXVoan33VI4zOHYeRjEnC0whfxkflER9RWp8SaIC1yDbXLDZW4UGO/I7Zv5 I7t5F6NfGwZkQHygCflZtZTxjFBdKouXj481httBqD9UYg8gFU+jaue/obrWPAf/Y5xI UJs1XJmvUGG2CPvPrxop86kaa7q05N5zkLzSjo8Xuw4Mo6j/7xMZYaZIGOzYs9jnfAey LPLg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAubKru3RhpeIf28FfrYGmmoQffGZTazXFyB4qLjBW0fJQb4e+re5 hyZiBMtUHQe05waD3LxMjTau4nWVAY4HoqTK5orx0gZg+tCoJJlhRuWCvSeQDzYFXVB+jk+IUfL xlftb/ThVhbztWVQMu2iyAdiaQQMUKE72RCsOUl2Y4rA1gVpqYkU8lva+wVQ08L5QAA== X-Received: by 2002:a62:1706:: with SMTP id 6mr1159670pfx.28.1550793065191; Thu, 21 Feb 2019 15:51:05 -0800 (PST) X-Google-Smtp-Source: AHgI3IZCAJe8wv9av38zzUAyLBCZF+ONqaV9cES9443A74+6g0x6M3sLe6DiIDyx/3gL2H00lzwK X-Received: by 2002:a62:1706:: with SMTP id 6mr1159630pfx.28.1550793064314; Thu, 21 Feb 2019 15:51:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793064; cv=none; d=google.com; s=arc-20160816; b=H41EPtIrkcZnimSVk+Yh+PI/kn0Eo1QuDDN+9oiZYm5SNLtqUDRWoe3XRpc1vSHoAK /I4WqCROyiGpZR/rAUohJOIXGyj8mMxQFj34+UpE5FjVPyeGe49vYNSZXU/TIiXjbn8I LUFF75ainLzeZ2OFuaCNzPppQCrOHcChM9HGy4Wlg3YD2nvDKSaaTo6oGNIWAe7Ida0m ajpv5l/RAprBKkLbFuPhmO60Tm8Kf8s+sZ72oOgLCFeFO32nT/+mmgs17UZ5bU14K6n9 MD7VltI9lP1y2L3pKYC66tFYpNU1wuB3zY36uDoIv9kKFkMVs/6ApXUScSOjWey7u4i1 L2VQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=g3nO4xFM3yb0X+OIqTEtEp6LbffMj61RqStKCNeSSiM=; b=y+XoCBuIevFusM2JCkc0dIM0akiKMaIvTFjVXp6BaXNsishs8IvX1xC80SNNZBo2un 4jIIdIAVhJrJWGVfdOsM4SCoDMPvfP2/GEjfPUwq1+EYLH0ay3czCs61Aqnyzy1uMgvY cRYIEiv7G5ewCC6zOKnOrIFsOyoPa2fv7e4h583tKqvJ4JXgrkU1x/HPwymxKTbjXqlv uv0Q8Lj+9dfpBS+zHF53t1RyYEChI1oT/xQdU25sLzsDmNXTy9PPKdL0SlLgF4MK+lKc qKNYpvLpu3O394LJE/HC+YtjJWJ5OH5YlUBSFPT/jbzisry6j0o6Czyr4G232CNZ0KRF OTSA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.51.04 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:51:04 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:51:03 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394918" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:51:02 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Kees Cook , Dave Hansen , Masami Hiramatsu , Rick Edgecombe Subject: [PATCH v3 11/20] x86/jump-label: Remove support for custom poker Date: Thu, 21 Feb 2019 15:44:42 -0800 Message-Id: <20190221234451.17632-12-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit There are only two types of poking: early and breakpoint based. The use of a function pointer to perform poking complicates the code and is probably inefficient due to the use of indirect branches. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Cc: Masami Hiramatsu Acked-by: Peter Zijlstra (Intel) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/jump_label.c | 26 ++++++++++---------------- 1 file changed, 10 insertions(+), 16 deletions(-) diff --git a/arch/x86/kernel/jump_label.c b/arch/x86/kernel/jump_label.c index e7d8c636b228..e631c358f7f4 100644 --- a/arch/x86/kernel/jump_label.c +++ b/arch/x86/kernel/jump_label.c @@ -37,7 +37,6 @@ static void bug_at(unsigned char *ip, int line) static void __ref __jump_label_transform(struct jump_entry *entry, enum jump_label_type type, - void *(*poker)(void *, const void *, size_t), int init) { union jump_code_union jmp; @@ -50,14 +49,6 @@ static void __ref __jump_label_transform(struct jump_entry *entry, jmp.offset = jump_entry_target(entry) - (jump_entry_code(entry) + JUMP_LABEL_NOP_SIZE); - /* - * As long as only a single processor is running and the code is still - * not marked as RO, text_poke_early() can be used; Checking that - * system_state is SYSTEM_BOOTING guarantees it. - */ - if (system_state == SYSTEM_BOOTING) - poker = text_poke_early; - if (type == JUMP_LABEL_JMP) { if (init) { expect = default_nop; line = __LINE__; @@ -80,16 +71,19 @@ static void __ref __jump_label_transform(struct jump_entry *entry, bug_at((void *)jump_entry_code(entry), line); /* - * Make text_poke_bp() a default fallback poker. + * As long as only a single processor is running and the code is still + * not marked as RO, text_poke_early() can be used; Checking that + * system_state is SYSTEM_BOOTING guarantees it. It will be set to + * SYSTEM_SCHEDULING before other cores are awaken and before the + * code is write-protected. * * At the time the change is being done, just ignore whether we * are doing nop -> jump or jump -> nop transition, and assume * always nop being the 'currently valid' instruction - * */ - if (poker) { - (*poker)((void *)jump_entry_code(entry), code, - JUMP_LABEL_NOP_SIZE); + if (init || system_state == SYSTEM_BOOTING) { + text_poke_early((void *)jump_entry_code(entry), code, + JUMP_LABEL_NOP_SIZE); return; } @@ -101,7 +95,7 @@ void arch_jump_label_transform(struct jump_entry *entry, enum jump_label_type type) { mutex_lock(&text_mutex); - __jump_label_transform(entry, type, NULL, 0); + __jump_label_transform(entry, type, 0); mutex_unlock(&text_mutex); } @@ -131,5 +125,5 @@ __init_or_module void arch_jump_label_transform_static(struct jump_entry *entry, jlstate = JL_STATE_NO_UPDATE; } if (jlstate == JL_STATE_UPDATE) - __jump_label_transform(entry, type, text_poke_early, 1); + __jump_label_transform(entry, type, 1); } From patchwork Thu Feb 21 23:44:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824895 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CBBDA1399 for ; Thu, 21 Feb 2019 23:51:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BA45031D2C for ; Thu, 21 Feb 2019 23:51:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id ADDAC31D30; Thu, 21 Feb 2019 23:51:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2598031D2C for ; Thu, 21 Feb 2019 23:51:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0384B8E00D0; Thu, 21 Feb 2019 18:51:07 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 011968E00CD; Thu, 21 Feb 2019 18:51:06 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CE0CF8E00D0; Thu, 21 Feb 2019 18:51:06 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) by kanga.kvack.org (Postfix) with ESMTP id 8E67C8E00CD for ; Thu, 21 Feb 2019 18:51:06 -0500 (EST) Received: by mail-pf1-f197.google.com with SMTP id 134so310674pfx.21 for ; Thu, 21 Feb 2019 15:51:06 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=IakGIipbE34Dq5lbcoYo7VqYScdEo+qlyfphRgvUc/Y=; b=pgpgf8+Xt0PSvRqWGUHwCjEuWBby3naKUPnUD8DIUs0PK4qC9JDZGHC+5lI23N6JGD 9/fp4yWTlejC9q0nJAEaUG7DwunLYbG1NcZkA6SPBS1TYWPOpgJxj5QWY9TjRsHvKOBd XH/dPY49yOF7Y5jJ5oGRjuU8p6WrJjKYzEOJif2Xurz/nUkeYgnBkMptP7gP8xJhBD2o H0V1A+ESbAynHlB0bNH44js9sS+kzpeX1FLELYI24N3UxE4zFU5VSrnBcArvT45mE+aG eF/uji8gDotIgWNQxiDSATB4VjTpQxCC890s55JL4NKppLb9QR2qccQ3ZqKvPUB0xowc d2EA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAuaE7mPucmh7mgGxmZQmiAAc+6fqD/3UZGqvPmU+HIMHvhJGDfJC HbMXvWb+Ad7GopADSVeTKg2LfSwfFqAKVYEEYLPrHCV0+PFsl54umJibyvzotVn3dOOwhcC4aOM Zk0igkvmh5YtCzMFg3XQsPnM8xdZLenqTJo+XhkN1tpl19ul5PaISCiRPtx0LaUxARQ== X-Received: by 2002:a17:902:bd43:: with SMTP id b3mr1208840plx.186.1550793066127; Thu, 21 Feb 2019 15:51:06 -0800 (PST) X-Google-Smtp-Source: AHgI3IZQBgw5v+QGB6Wdf3A8wLZeijmmQhZGRmC1nIKcfesKwL97VAFIMBa4zn3wwh4QuPh9tLzO X-Received: by 2002:a17:902:bd43:: with SMTP id b3mr1208802plx.186.1550793065408; Thu, 21 Feb 2019 15:51:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793065; cv=none; d=google.com; s=arc-20160816; b=EQDYDUWeeytH40O4EyraQQMz4Y21R3Rd/n09Uo77Gha0LBEG6k9vm4EOLEowmtdHHy fbA75wHRj8DytczFzBFY9/P7goHpubpN69gHX5eh8qEm9ALKlceyCm6j/fDLpOX19T28 JglrBluZ4ab9O+7ukbFRG4lVjV/eRdHhGJ5xJDQkIaVaz5cZDJbPgC0fKjYzPMFb9wNZ d8Po7xjqzYX9/YBvNBpxgRYxt/lQzjUwIXmNCNr7ieaKNbRux4wr6ubWBKeKZWKhue/o aUkPYqEoUiN0mwUG2FVP2jznwJf/t6ZgCYu7U7bGQuVAOoPnpgYnfisV2dY4az/jSjnv VQjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=IakGIipbE34Dq5lbcoYo7VqYScdEo+qlyfphRgvUc/Y=; b=SEn1cgHHFLy6L8l5OGHwCYhS2aWM7D2cs1XOY4A1bgLQLQIw1h0eQTafcizcdRqbxl YIPtJauWVy8+7ZBa0pIGK4VC6xrhel/BO4t+QgP6v0isyb4Jv1LSC9Tiw4QYs4aJiTZu Fu8XKyCzVc5FQjo2+S6tABRUlUdVw+0TKDV9WM+hZpLxxLr2dk1LYKZ7sqq6tuCnHPxJ yijIR+FSeo94iC49Y3mUXx2xI2SIw68yI9FMokiOPzTyD2uw4rlfnZu6YDhhcI56i8Vi S7JPgBzQU2uX6gpV15dfsNrhE0bIvQeGvBaJaTKOQuRySLO7m9vxzxYraIFgdAvfP8DO 18gQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.51.05 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:51:05 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:51:05 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394921" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:51:03 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Kees Cook , Dave Hansen , Masami Hiramatsu , Rick Edgecombe Subject: [PATCH v3 12/20] x86/alternative: Remove the return value of text_poke_*() Date: Thu, 21 Feb 2019 15:44:43 -0800 Message-Id: <20190221234451.17632-13-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit The return value of text_poke_early() and text_poke_bp() is useless. Remove it. Cc: Andy Lutomirski Cc: Kees Cook Cc: Dave Hansen Cc: Masami Hiramatsu Acked-by: Peter Zijlstra (Intel) Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/include/asm/text-patching.h | 4 ++-- arch/x86/kernel/alternative.c | 11 ++++------- 2 files changed, 6 insertions(+), 9 deletions(-) diff --git a/arch/x86/include/asm/text-patching.h b/arch/x86/include/asm/text-patching.h index a75eed841eed..c90678fd391a 100644 --- a/arch/x86/include/asm/text-patching.h +++ b/arch/x86/include/asm/text-patching.h @@ -18,7 +18,7 @@ static inline void apply_paravirt(struct paravirt_patch_site *start, #define __parainstructions_end NULL #endif -extern void *text_poke_early(void *addr, const void *opcode, size_t len); +extern void text_poke_early(void *addr, const void *opcode, size_t len); /* * Clear and restore the kernel write-protection flag on the local CPU. @@ -37,7 +37,7 @@ extern void *text_poke_early(void *addr, const void *opcode, size_t len); extern void *text_poke(void *addr, const void *opcode, size_t len); extern void *text_poke_kgdb(void *addr, const void *opcode, size_t len); extern int poke_int3_handler(struct pt_regs *regs); -extern void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler); +extern void text_poke_bp(void *addr, const void *opcode, size_t len, void *handler); extern int after_bootmem; extern __ro_after_init struct mm_struct *poking_mm; extern __ro_after_init unsigned long poking_addr; diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index b75bfeda021e..c63707e7ed3d 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -264,7 +264,7 @@ static void __init_or_module add_nops(void *insns, unsigned int len) extern struct alt_instr __alt_instructions[], __alt_instructions_end[]; extern s32 __smp_locks[], __smp_locks_end[]; -void *text_poke_early(void *addr, const void *opcode, size_t len); +void text_poke_early(void *addr, const void *opcode, size_t len); /* * Are we looking at a near JMP with a 1 or 4-byte displacement. @@ -666,8 +666,8 @@ void __init alternative_instructions(void) * instructions. And on the local CPU you need to be protected again NMI or MCE * handlers seeing an inconsistent instruction while you patch. */ -void *__init_or_module text_poke_early(void *addr, const void *opcode, - size_t len) +void __init_or_module text_poke_early(void *addr, const void *opcode, + size_t len) { unsigned long flags; @@ -690,7 +690,6 @@ void *__init_or_module text_poke_early(void *addr, const void *opcode, * that causes hangs on some VIA CPUs. */ } - return addr; } __ro_after_init struct mm_struct *poking_mm; @@ -892,7 +891,7 @@ int poke_int3_handler(struct pt_regs *regs) * replacing opcode * - sync cores */ -void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler) +void text_poke_bp(void *addr, const void *opcode, size_t len, void *handler) { unsigned char int3 = 0xcc; @@ -934,7 +933,5 @@ void *text_poke_bp(void *addr, const void *opcode, size_t len, void *handler) * the writing of the new instruction. */ bp_patching_in_progress = false; - - return addr; } From patchwork Thu Feb 21 23:44:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824903 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C3BF61399 for ; Thu, 21 Feb 2019 23:51:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B051531D2C for ; Thu, 21 Feb 2019 23:51:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A49F331D2E; Thu, 21 Feb 2019 23:51:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6344031D5A for ; Thu, 21 Feb 2019 23:51:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE2098E00D1; Thu, 21 Feb 2019 18:51:07 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C91888E00CD; Thu, 21 Feb 2019 18:51:07 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BA80D8E00D1; Thu, 21 Feb 2019 18:51:07 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) by kanga.kvack.org (Postfix) with ESMTP id 75FFA8E00CD for ; Thu, 21 Feb 2019 18:51:07 -0500 (EST) Received: by mail-pl1-f197.google.com with SMTP id 71so288861plf.19 for ; Thu, 21 Feb 2019 15:51:07 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=vnm0CR07Tfw1GDwsiV5/vDSm+bo6LLBKcYWCZj/kFdA=; b=LYgo6yrPWj8QKgdLzpY3WuV17bZmEjoMajvlWGB7URp2ylu+mI+Z82HNkBnEVyathR s3VPxzWNz+Y8Hn5MS19HJLPDSN6gRR4hJyb5bxJXiaQrQ6/zXrGdd6tTDGMc+Be7LcbK sM/SVbxMHNGcY+qx473EJmJOd2C4WdU1ZNe0pKp804fmfz/3il0CyvlPKi75OFW/lH8z OB1MzxyHfkFrL8PLUInSczxfzJAbFhBFvPhP/MwNP5pSV+04dhvGNt3kDNzTSqHD6koB kO8yzoWlI4BWvQT4x75HQEyzbHzOSf9so1aPcE7f9Xh/zqVbv7f87h627qWUUefPXn+/ NQzA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAuavLhU0rNKACrghMgppwog1tsSml7z0cy3oeVaERbDDvYjjVDjs zii6iW4GNkNpkfLuW3z7BB6h1W6NuYBqrW30kkxtUfP4h+I0OyG2oRiQ0W2yzNpyPBdXkxbziT/ mQuW05Fg6w4fwkef4ZNlbxfjzRSURnfnQbB7IdCQUKHkMep4+RcPwuFWNRFRq3Qpttw== X-Received: by 2002:a63:1061:: with SMTP id 33mr1080586pgq.226.1550793067145; Thu, 21 Feb 2019 15:51:07 -0800 (PST) X-Google-Smtp-Source: AHgI3Ibl/1xtYG/hbpzNFXT5P+tY412bDqklpIoR4/gD6rEhtq+sCc01WkFh+dBYQU+j0SY5EqWX X-Received: by 2002:a63:1061:: with SMTP id 33mr1080543pgq.226.1550793066351; Thu, 21 Feb 2019 15:51:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793066; cv=none; d=google.com; s=arc-20160816; b=ReMGZ0v+tP5coROaTk8Uvp4tVvRSfrA8c3EYz4YGnVsPdWH2y+8c6sD+3s1PhxNlPD lEm60rey2r4RQuHFyTNJYILWsPnk8BV0xyTV+IUCH72XnEY6vg9u3+YSxbjzBOCO2WHT rQxPHV7sSby73jj7nId+B86QbLYP9Q45PMiG1E2Em6+2GddC5VqLfe+FZVLlUd0lgIuQ rJGCRRbs6R5uFekwugBEj/gFtlFtxANEf3vF4qoToHzVsgHXwp0Bh5IFvDBHCcTEqigY 5VNl0MUdm2aN6GNscskeC/fOY/Gh/YB0VpxZ/1hlSVbAJoktdXJ8xpq+r7ce4WWOL0y+ 8CrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=vnm0CR07Tfw1GDwsiV5/vDSm+bo6LLBKcYWCZj/kFdA=; b=AYnz1cuz0nSWPv0id/4ShPz5bBYtUv/YzwoF89hKRzaNwMnAETJ0by5howa/imY5+k u8ULBwxRjnoeGcbMYXfsl1HVCKNQOOHfHeTxwXyHg9I9XdJTjrNtEGJmmrQ6Js2U7XfM BzgNVsP28nADcv7SQSbesARLrbhG1jTBQ0QAy9QysyueeYIUV02s05gJA9F0jNiUndIU H5x2uQJKeG9PBciN2B9LtTHAHN2mUFIp40jJ/CoQBhNNyLr/tf0yWl93o4QVnUDOOp0l 0WL4e0WuDMzWGl9NZrGBFIH/U9MyfQO2nYgjyc5oCir/D18qza3Fu8yZCrVGXtRB5ZyH 3voA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.51.06 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:51:06 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:51:05 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394927" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:51:05 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Rick Edgecombe Subject: [PATCH v3 13/20] x86/mm/cpa: Add set_direct_map_ functions Date: Thu, 21 Feb 2019 15:44:44 -0800 Message-Id: <20190221234451.17632-14-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Add two new functions set_direct_map_default_noflush() and set_direct_map_invalid_noflush() for setting the direct map alias for the page to its default valid permissions and to an invalid state that cannot be cached in a TLB, respectively. These functions do not flush the TLB. Note, __kernel_map_pages() does something similar but flushes the TLB and doesn't reset the permission bits to default on all architectures. Also add an ARCH config ARCH_HAS_SET_DIRECT_MAP for specifying whether these have an actual implementation or a default empty one. Cc: Dave Hansen Cc: Andy Lutomirski Cc: Peter Zijlstra Signed-off-by: Rick Edgecombe --- arch/Kconfig | 4 ++++ arch/x86/Kconfig | 1 + arch/x86/include/asm/set_memory.h | 3 +++ arch/x86/mm/pageattr.c | 14 +++++++++++--- include/linux/set_memory.h | 10 ++++++++++ 5 files changed, 29 insertions(+), 3 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index 4cfb6de48f79..79a9ec371964 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -249,6 +249,10 @@ config ARCH_HAS_FORTIFY_SOURCE config ARCH_HAS_SET_MEMORY bool +# Select if arch has all set_direct_map_invalid/default() functions +config ARCH_HAS_SET_DIRECT_MAP + bool + # Select if arch init_task must go in the __init_task_data section config ARCH_TASK_STRUCT_ON_STACK bool diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 26387c7bf305..291c6566cf88 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -66,6 +66,7 @@ config X86 select ARCH_HAS_UACCESS_FLUSHCACHE if X86_64 select ARCH_HAS_UACCESS_MCSAFE if X86_64 && X86_MCE select ARCH_HAS_SET_MEMORY + select ARCH_HAS_SET_DIRECT_MAP select ARCH_HAS_STRICT_KERNEL_RWX select ARCH_HAS_STRICT_MODULE_RWX select ARCH_HAS_SYNC_CORE_BEFORE_USERMODE diff --git a/arch/x86/include/asm/set_memory.h b/arch/x86/include/asm/set_memory.h index 07a25753e85c..ae7b909dc242 100644 --- a/arch/x86/include/asm/set_memory.h +++ b/arch/x86/include/asm/set_memory.h @@ -85,6 +85,9 @@ int set_pages_nx(struct page *page, int numpages); int set_pages_ro(struct page *page, int numpages); int set_pages_rw(struct page *page, int numpages); +int set_direct_map_invalid_noflush(struct page *page); +int set_direct_map_default_noflush(struct page *page); + extern int kernel_set_to_readonly; void set_kernel_text_rw(void); void set_kernel_text_ro(void); diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index 4f8972311a77..fff9c91ad177 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -2209,8 +2209,6 @@ int set_pages_rw(struct page *page, int numpages) return set_memory_rw(addr, numpages); } -#ifdef CONFIG_DEBUG_PAGEALLOC - static int __set_pages_p(struct page *page, int numpages) { unsigned long tempaddr = (unsigned long) page_address(page); @@ -2249,6 +2247,17 @@ static int __set_pages_np(struct page *page, int numpages) return __change_page_attr_set_clr(&cpa, 0); } +int set_direct_map_invalid_noflush(struct page *page) +{ + return __set_pages_np(page, 1); +} + +int set_direct_map_default_noflush(struct page *page) +{ + return __set_pages_p(page, 1); +} + +#ifdef CONFIG_DEBUG_PAGEALLOC void __kernel_map_pages(struct page *page, int numpages, int enable) { if (PageHighMem(page)) @@ -2282,7 +2291,6 @@ void __kernel_map_pages(struct page *page, int numpages, int enable) } #ifdef CONFIG_HIBERNATION - bool kernel_page_present(struct page *page) { unsigned int level; diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h index 2a986d282a97..82477e934b1a 100644 --- a/include/linux/set_memory.h +++ b/include/linux/set_memory.h @@ -10,6 +10,16 @@ #ifdef CONFIG_ARCH_HAS_SET_MEMORY #include +#ifndef CONFIG_ARCH_HAS_SET_DIRECT_MAP +static inline int set_direct_map_invalid_noflush(struct page *page) +{ + return 0; +} +static inline int set_direct_map_default_noflush(struct page *page) +{ + return 0; +} +#endif #else static inline int set_memory_ro(unsigned long addr, int numpages) { return 0; } static inline int set_memory_rw(unsigned long addr, int numpages) { return 0; } From patchwork Thu Feb 21 23:44:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824905 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 57CA5922 for ; Thu, 21 Feb 2019 23:51:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 487CB31D2C for ; Thu, 21 Feb 2019 23:51:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3A85C31D3C; Thu, 21 Feb 2019 23:51:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5066B31D2C for ; Thu, 21 Feb 2019 23:51:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F2BA78E00D2; Thu, 21 Feb 2019 18:51:08 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id F01498E00CD; Thu, 21 Feb 2019 18:51:08 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF4668E00D2; Thu, 21 Feb 2019 18:51:08 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) by kanga.kvack.org (Postfix) with ESMTP id 8DAD18E00CD for ; Thu, 21 Feb 2019 18:51:08 -0500 (EST) Received: by mail-pg1-f198.google.com with SMTP id 11so309882pgd.19 for ; Thu, 21 Feb 2019 15:51:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=iG1R2RBktKeFpHfZ1a/wtU6qsgc12QlvnPW3fBUPBtk=; b=E5Lbzr2GVkdbP8yTpxBxMXx3sSQRav1+z/a08dAiMRCxxOnDAVUg9j+qlMfVQ3WQ4Y Ch02hl+dRJ3MqEv9cFrRfXQPO7jL+lASWPY67Z3Jmt58TOd9I4AZPEpSqMVJBMmMacxs RxT0t/BHUHVDNgCMamsNVbsNZj0mPgD+66tfMQNTie4Ty2+UmAKQV3PKaRjrje60iwkL pfQb++Ia/6JpI1jQQntx1Aqiff9jWGJRGKYW2dJogpvjsj5qAQ3aTjRCo+/7VPbew/2D AW6tCP4LiyDCADQ/v9JEtu1cBDJQxXY7KsIsAdeIm9Zw2FEBubGrPt4DQYCR/lpsWhuZ 1jvA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAuZ0ywv1AVPOzFf6I8ooNCu443OXsrIeTzeFmTxXJ8m6PQwge9B1 xRRtArZ5yXsKp9L2L58LVDxshUuDXfnGAvG/G0lKLYoRyH6Y8eMI/OagKw8PLru82+4NSYorOam 3nlAbVbg4W3LV4MmV4+XH6NnxgAMOgmhxvU52FPYqw8P99P7FlczGSqtbQqG3OegdNw== X-Received: by 2002:a63:534c:: with SMTP id t12mr1040688pgl.205.1550793068235; Thu, 21 Feb 2019 15:51:08 -0800 (PST) X-Google-Smtp-Source: AHgI3IayCtXafJgp3eUkVgHgR8aM6TrKnDz0Ocpu8/t1tLeTYrOWKywk6owwaq+/XiA+EPFGb/LX X-Received: by 2002:a63:534c:: with SMTP id t12mr1040634pgl.205.1550793067310; Thu, 21 Feb 2019 15:51:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793067; cv=none; d=google.com; s=arc-20160816; b=PjBsjKGLiwrzfQ8eBuFYaDUruI/p12YBPQvez/9+vQeeG7UNvq13DKn9qkQhHRCpBR dDMGccm3Uqz8byhqHEwBFWH9J2Had+nDuRhwnumgLWxS5cQCEeEKIASMNHyvNqXNGrEb 0TxPzMr82JW76KyoWIJDhUVFQPq+znRl1LGAfZc78D1Tt0kDj8XOUtNM6qfYg17aJhwN IcGgqZccFrR9Io5IP+yoHNNIAwWL8AML1scRkRe8OKWT3UzYxigYU74fOB8/SVv89G+9 XDvW+dstdOXYmZ6rw/9yCg3QAYZU9SAM/rw5PzBwmehrEENlnLSh5cX4VxGMgPhV5jEe QKog== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=iG1R2RBktKeFpHfZ1a/wtU6qsgc12QlvnPW3fBUPBtk=; b=ZEZ10NEbafMtlYbiNROT0iU592aVRM+G3J4X3434LaNbGRwi1GzDlA9xb1iqItmXTZ gfwzBSpV83bHizYL9dlTQMReZK4rzyicL07Iof2noctMQRwpOHNxLHAqPqbtpiJJ0tC4 MG4mQhLW7Gq0M6l3irsS82N6+kY9x3SpJALW/d+6fpGV33e9BO/EjDneKUUhwTI9ClYE TIBLteiqwB4nuRCI9q6lPbt8QLtAJGc83fmJRBPFaf0HR04uvhc83ImgjFuaK0bsdhqm x3o2LWnNH8KBShl7kf0Q0pAYUmCilOaMSe7izr6BAb4tdu3PgO/lW3+rqOZBad6S54QH HcjQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.51.07 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:51:07 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:51:06 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394932" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:51:05 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Rick Edgecombe , "Rafael J. Wysocki" , Pavel Machek Subject: [PATCH v3 14/20] mm: Make hibernate handle unmapped pages Date: Thu, 21 Feb 2019 15:44:45 -0800 Message-Id: <20190221234451.17632-15-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Make hibernate handle unmapped pages on the direct map when CONFIG_ARCH_HAS_SET_ALIAS is set. These functions allow for setting pages to invalid configurations, so now hibernate should check if the pages have valid mappings and handle if they are unmapped when doing a hibernate save operation. Previously this checking was already done when CONFIG_DEBUG_PAGEALLOC was configured. It does not appear to have a big hibernating performance impact. The speed of the saving operation before this change was measured as 819.02 MB/s, and after was measured at 813.32 MB/s. Before: [ 4.670938] PM: Wrote 171996 kbytes in 0.21 seconds (819.02 MB/s) After: [ 4.504714] PM: Wrote 178932 kbytes in 0.22 seconds (813.32 MB/s) Cc: Dave Hansen Cc: Andy Lutomirski Cc: Peter Zijlstra Cc: "Rafael J. Wysocki" Cc: Pavel Machek Cc: Borislav Petkov Acked-by: Pavel Machek Signed-off-by: Rick Edgecombe --- arch/x86/mm/pageattr.c | 4 ---- include/linux/mm.h | 18 ++++++------------ kernel/power/snapshot.c | 5 +++-- mm/page_alloc.c | 7 +++++-- 4 files changed, 14 insertions(+), 20 deletions(-) diff --git a/arch/x86/mm/pageattr.c b/arch/x86/mm/pageattr.c index fff9c91ad177..1cffee05f987 100644 --- a/arch/x86/mm/pageattr.c +++ b/arch/x86/mm/pageattr.c @@ -2257,7 +2257,6 @@ int set_direct_map_default_noflush(struct page *page) return __set_pages_p(page, 1); } -#ifdef CONFIG_DEBUG_PAGEALLOC void __kernel_map_pages(struct page *page, int numpages, int enable) { if (PageHighMem(page)) @@ -2302,11 +2301,8 @@ bool kernel_page_present(struct page *page) pte = lookup_address((unsigned long)page_address(page), &level); return (pte_val(*pte) & _PAGE_PRESENT); } - #endif /* CONFIG_HIBERNATION */ -#endif /* CONFIG_DEBUG_PAGEALLOC */ - int __init kernel_map_pages_in_pgd(pgd_t *pgd, u64 pfn, unsigned long address, unsigned numpages, unsigned long page_flags) { diff --git a/include/linux/mm.h b/include/linux/mm.h index 80bb6408fe73..5748e9ce133e 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2642,37 +2642,31 @@ static inline void kernel_poison_pages(struct page *page, int numpages, int enable) { } #endif -#ifdef CONFIG_DEBUG_PAGEALLOC extern bool _debug_pagealloc_enabled; -extern void __kernel_map_pages(struct page *page, int numpages, int enable); static inline bool debug_pagealloc_enabled(void) { - return _debug_pagealloc_enabled; + return IS_ENABLED(CONFIG_DEBUG_PAGEALLOC) && _debug_pagealloc_enabled; } +#if defined(CONFIG_DEBUG_PAGEALLOC) || defined(CONFIG_ARCH_HAS_SET_DIRECT_MAP) +extern void __kernel_map_pages(struct page *page, int numpages, int enable); + static inline void kernel_map_pages(struct page *page, int numpages, int enable) { - if (!debug_pagealloc_enabled()) - return; - __kernel_map_pages(page, numpages, enable); } #ifdef CONFIG_HIBERNATION extern bool kernel_page_present(struct page *page); #endif /* CONFIG_HIBERNATION */ -#else /* CONFIG_DEBUG_PAGEALLOC */ +#else /* CONFIG_DEBUG_PAGEALLOC || CONFIG_ARCH_HAS_SET_DIRECT_MAP */ static inline void kernel_map_pages(struct page *page, int numpages, int enable) {} #ifdef CONFIG_HIBERNATION static inline bool kernel_page_present(struct page *page) { return true; } #endif /* CONFIG_HIBERNATION */ -static inline bool debug_pagealloc_enabled(void) -{ - return false; -} -#endif /* CONFIG_DEBUG_PAGEALLOC */ +#endif /* CONFIG_DEBUG_PAGEALLOC || CONFIG_ARCH_HAS_SET_DIRECT_MAP */ #ifdef __HAVE_ARCH_GATE_AREA extern struct vm_area_struct *get_gate_vma(struct mm_struct *mm); diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c index 640b2034edd6..f69b2920d4f4 100644 --- a/kernel/power/snapshot.c +++ b/kernel/power/snapshot.c @@ -1334,8 +1334,9 @@ static inline void do_copy_page(long *dst, long *src) * safe_copy_page - Copy a page in a safe way. * * Check if the page we are going to copy is marked as present in the kernel - * page tables (this always is the case if CONFIG_DEBUG_PAGEALLOC is not set - * and in that case kernel_page_present() always returns 'true'). + * page tables. This always is the case if CONFIG_DEBUG_PAGEALLOC or + * CONFIG_ARCH_HAS_SET_DIRECT_MAP is not set. In that case kernel_page_present() + * always returns 'true'. */ static void safe_copy_page(void *dst, struct page *s_page) { diff --git a/mm/page_alloc.c b/mm/page_alloc.c index d295c9bc01a8..92d0a0934274 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1074,7 +1074,9 @@ static __always_inline bool free_pages_prepare(struct page *page, } arch_free_page(page, order); kernel_poison_pages(page, 1 << order, 0); - kernel_map_pages(page, 1 << order, 0); + if (debug_pagealloc_enabled()) + kernel_map_pages(page, 1 << order, 0); + kasan_free_nondeferred_pages(page, order); return true; @@ -1944,7 +1946,8 @@ inline void post_alloc_hook(struct page *page, unsigned int order, set_page_refcounted(page); arch_alloc_page(page, order); - kernel_map_pages(page, 1 << order, 1); + if (debug_pagealloc_enabled()) + kernel_map_pages(page, 1 << order, 1); kernel_poison_pages(page, 1 << order, 1); kasan_alloc_pages(page, order); set_page_owner(page, order, gfp_flags); From patchwork Thu Feb 21 23:44:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824909 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D8BDD922 for ; Thu, 21 Feb 2019 23:51:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CA5A831D2C for ; Thu, 21 Feb 2019 23:51:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BE42531D3C; Thu, 21 Feb 2019 23:51:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0E30631D2C for ; Thu, 21 Feb 2019 23:51:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3BA018E00D3; Thu, 21 Feb 2019 18:51:10 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 390958E00CD; Thu, 21 Feb 2019 18:51:10 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 25F578E00D3; Thu, 21 Feb 2019 18:51:10 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) by kanga.kvack.org (Postfix) with ESMTP id C14578E00CD for ; Thu, 21 Feb 2019 18:51:09 -0500 (EST) Received: by mail-pg1-f197.google.com with SMTP id 11so309909pgd.19 for ; Thu, 21 Feb 2019 15:51:09 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=oSQbwij47yA17Mv3fSz7kQf6by0ffGswaqjnStQPtTI=; b=SZwkOlYtHR5V/N3VVmG5DOkuzdnmb1t7W6XqNa/JL06hadtOdS03deJQiCCOWG0WTa z5YuQhjpMh5ESTN+2LKA7OP2feDirJ9TYnkqx4XoYVgVhDXIxspmSDUJ8iB11r/Jc4Tg yoFevHEXsYH1MMYajkLQ3jro0Ih6RNfimf1RVIxGr/nf8Mkym//w1O1dD2OOG29DKBHk cP4LLTg+27vDpmfw5PL0kEbmOH92Lp+dm1wEQFitEvW1ZDjkolOh2xVqLEfDVr4NdTYg PS5hGTilL5W+eJ0mIQSV2GfPad1RlvvEtO9eSeBVCW/oTAH32lPlCabQZdx4XRcNUTWr bSuQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAubxo0oL8Ebj8MpVp4NRyhdwdw6UL1kdV6cXVHWfaQiGW8C6Sxh/ PAiMWDRzcAgRxdSR1+Lb3N9/BZZPnRs9K3hiXrcIpfNayZ/GKRH/u6w4m1FsgsVCmdOvIoUnFNZ ip+WdJW9lwB18gbrNa4pAff7k1gya8x8K43XlFE/xmZKw0GpUCNrZ4PsdtdnvZbXO2Q== X-Received: by 2002:a17:902:684:: with SMTP id 4mr1167453plh.3.1550793069419; Thu, 21 Feb 2019 15:51:09 -0800 (PST) X-Google-Smtp-Source: AHgI3IaZGb41Wec+8dVE1L1VxzuUM3ONecKG5YsTixouxNQH/LXEXG6lTX+GlPv+t1nX2lkyitbK X-Received: by 2002:a17:902:684:: with SMTP id 4mr1167387plh.3.1550793068274; Thu, 21 Feb 2019 15:51:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793068; cv=none; d=google.com; s=arc-20160816; b=rc1U0U86n9gW/i7hHOh8ReKquvCedEARpQQALOpScm8JGNWcIs6QlfFmz5zruc101W hl4zJ1hkwtloa4GFASMEyiOdKXsUBD5iMlrHhYtnlnuTgEvPoazjE+tz7TLQulS/7DOJ JFtrzVwmteTE+5ZNX4ehTLjlyqznyVi18gV4lu+DYW8Y0uyyKOwrJ59pIaJlM2Ugt7dB VaA3MyUg0jcYt0eIvvScrp9ooPZG6Ajs6AGRsc/iLQ0HuTy4e4frxP3RAcp7C/8xkbKc ofmPi/D85IhykTwIwa1EdzLEVVXLoJVr6Uwx5Ym52zrsFjRxmEX0FOWEL0fHUNJZMAMF xgMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=oSQbwij47yA17Mv3fSz7kQf6by0ffGswaqjnStQPtTI=; b=H7guowkBPrAZDBXY/P7Gun5GXBtltAuwXtWFa19teQQkKAuH7Y/6VWof9Ibdnb/o7P ji33AoASQDh1oPKRihjvwVizFQX8JmJj5n2i3FB4fnq4rXR8sYwkkA1uBrwI/1FtXCjc Y9On/QJ6RRhoUuDh1ee9mxng932sAtiKB8ASQWOwVKPcja48UpIsLcWLZ7piO1AZiguu lRehOpYEPJrOiD+Blh+T9dz9s0oeYwoYaqpzMHDXl+DWKuNRxFyFvL08BK+PydHCePd6 IIOwLfaj0geegnonhaktZCYTD8/7tpeI3b/TPIpOL6wuG0xjzj4xVmpY2k0fludnku2Q Eyiw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.51.08 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:51:08 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:51:07 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394938" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:51:06 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Rick Edgecombe Subject: [PATCH v3 15/20] vmalloc: Add flag for free of special permsissions Date: Thu, 21 Feb 2019 15:44:46 -0800 Message-Id: <20190221234451.17632-16-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Add a new flag VM_FLUSH_RESET_PERMS, for enabling vfree operations to immediately clear executable TLB entries before freeing pages, and handle resetting permissions on the directmap. This flag is useful for any kind of memory with elevated permissions, or where there can be related permissions changes on the directmap. Today this is RO+X and RO memory. Although this enables directly vfreeing non-writeable memory now, non-writable memory cannot be freed in an interrupt because the allocation itself is used as a node on deferred free list. So when RO memory needs to be freed in an interrupt the code doing the vfree needs to have its own work queue, as was the case before the deferred vfree list was added to vmalloc. For architectures with set_direct_map_ implementations this whole operation can be done with one TLB flush when centralized like this. For others with directmap permissions, currently only arm64, a backup method using set_memory functions is used to reset the directmap. When arm64 adds set_direct_map_ functions, this backup can be removed. When the TLB is flushed to both remove TLB entries for the vmalloc range mapping and the direct map permissions, the lazy purge operation could be done to try to save a TLB flush later. However today vm_unmap_aliases could flush a TLB range that does not include the directmap. So a helper is added with extra parameters that can allow both the vmalloc address and the direct mapping to be flushed during this operation. The behavior of the normal vm_unmap_aliases function is unchanged. Cc: Borislav Petkov Suggested-by: Dave Hansen Suggested-by: Andy Lutomirski Suggested-by: Will Deacon Signed-off-by: Rick Edgecombe --- include/linux/vmalloc.h | 13 +++++ mm/vmalloc.c | 113 +++++++++++++++++++++++++++++++++------- 2 files changed, 107 insertions(+), 19 deletions(-) diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 398e9c95cd61..345bb9d2f578 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -21,6 +21,11 @@ struct notifier_block; /* in notifier.h */ #define VM_UNINITIALIZED 0x00000020 /* vm_struct is not fully initialized */ #define VM_NO_GUARD 0x00000040 /* don't add guard page */ #define VM_KASAN 0x00000080 /* has allocated kasan shadow memory */ +/* + * Memory with VM_FLUSH_RESET_PERMS cannot be freed in an interrupt or with + * vfree_atomic(). + */ +#define VM_FLUSH_RESET_PERMS 0x00000100 /* Reset direct map and flush TLB on unmap */ /* bits [20..32] reserved for arch specific ioremap internals */ /* @@ -135,6 +140,14 @@ extern struct vm_struct *__get_vm_area_caller(unsigned long size, extern struct vm_struct *remove_vm_area(const void *addr); extern struct vm_struct *find_vm_area(const void *addr); +static inline void set_vm_flush_reset_perms(void *addr) +{ + struct vm_struct *vm = find_vm_area(addr); + + if (vm) + vm->flags |= VM_FLUSH_RESET_PERMS; +} + extern int map_vm_area(struct vm_struct *area, pgprot_t prot, struct page **pages); #ifdef CONFIG_MMU diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 871e41c55e23..0e341581832e 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -18,6 +18,7 @@ #include #include #include +#include #include #include #include @@ -1055,24 +1056,9 @@ static void vb_free(const void *addr, unsigned long size) spin_unlock(&vb->lock); } -/** - * vm_unmap_aliases - unmap outstanding lazy aliases in the vmap layer - * - * The vmap/vmalloc layer lazily flushes kernel virtual mappings primarily - * to amortize TLB flushing overheads. What this means is that any page you - * have now, may, in a former life, have been mapped into kernel virtual - * address by the vmap layer and so there might be some CPUs with TLB entries - * still referencing that page (additional to the regular 1:1 kernel mapping). - * - * vm_unmap_aliases flushes all such lazy mappings. After it returns, we can - * be sure that none of the pages we have control over will have any aliases - * from the vmap layer. - */ -void vm_unmap_aliases(void) +static void _vm_unmap_aliases(unsigned long start, unsigned long end, int flush) { - unsigned long start = ULONG_MAX, end = 0; int cpu; - int flush = 0; if (unlikely(!vmap_initialized)) return; @@ -1109,6 +1095,27 @@ void vm_unmap_aliases(void) flush_tlb_kernel_range(start, end); mutex_unlock(&vmap_purge_lock); } + +/** + * vm_unmap_aliases - unmap outstanding lazy aliases in the vmap layer + * + * The vmap/vmalloc layer lazily flushes kernel virtual mappings primarily + * to amortize TLB flushing overheads. What this means is that any page you + * have now, may, in a former life, have been mapped into kernel virtual + * address by the vmap layer and so there might be some CPUs with TLB entries + * still referencing that page (additional to the regular 1:1 kernel mapping). + * + * vm_unmap_aliases flushes all such lazy mappings. After it returns, we can + * be sure that none of the pages we have control over will have any aliases + * from the vmap layer. + */ +void vm_unmap_aliases(void) +{ + unsigned long start = ULONG_MAX, end = 0; + int flush = 0; + + _vm_unmap_aliases(start, end, flush); +} EXPORT_SYMBOL_GPL(vm_unmap_aliases); /** @@ -1494,6 +1501,72 @@ struct vm_struct *remove_vm_area(const void *addr) return NULL; } +static inline void set_area_direct_map(const struct vm_struct *area, + int (*set_direct_map)(struct page *page)) +{ + int i; + + for (i = 0; i < area->nr_pages; i++) + if (page_address(area->pages[i])) + set_direct_map(area->pages[i]); +} + +/* Handle removing and resetting vm mappings related to the vm_struct. */ +static void vm_remove_mappings(struct vm_struct *area, int deallocate_pages) +{ + unsigned long addr = (unsigned long)area->addr; + unsigned long start = ULONG_MAX, end = 0; + int flush_reset = area->flags & VM_FLUSH_RESET_PERMS; + int i; + + /* + * The below block can be removed when all architectures that have + * direct map permissions also have set_direct_map_() implementations. + * This is concerned with resetting the direct map any an vm alias with + * execute permissions, without leaving a RW+X window. + */ + if (flush_reset && !IS_ENABLED(CONFIG_ARCH_HAS_SET_DIRECT_MAP)) { + set_memory_nx(addr, area->nr_pages); + set_memory_rw(addr, area->nr_pages); + } + + remove_vm_area(area->addr); + + /* If this is not VM_FLUSH_RESET_PERMS memory, no need for the below. */ + if (!flush_reset) + return; + + /* + * If not deallocating pages, just do the flush of the VM area and + * return. + */ + if (!deallocate_pages) { + vm_unmap_aliases(); + return; + } + + /* + * If execution gets here, flush the vm mapping and reset the direct + * map. Find the start and end range of the direct mappings to make sure + * the vm_unmap_aliases() flush includes the direct map. + */ + for (i = 0; i < area->nr_pages; i++) { + if (page_address(area->pages[i])) { + start = min(addr, start); + end = max(addr, end); + } + } + + /* + * Set direct map to something invalid so that it won't be cached if + * there are any accesses after the TLB flush, then flush the TLB and + * reset the direct map permissions to the default. + */ + set_area_direct_map(area, set_direct_map_invalid_noflush); + _vm_unmap_aliases(start, end, 1); + set_area_direct_map(area, set_direct_map_default_noflush); +} + static void __vunmap(const void *addr, int deallocate_pages) { struct vm_struct *area; @@ -1515,7 +1588,8 @@ static void __vunmap(const void *addr, int deallocate_pages) debug_check_no_locks_freed(area->addr, get_vm_area_size(area)); debug_check_no_obj_freed(area->addr, get_vm_area_size(area)); - remove_vm_area(addr); + vm_remove_mappings(area, deallocate_pages); + if (deallocate_pages) { int i; @@ -1925,8 +1999,9 @@ EXPORT_SYMBOL(vzalloc_node); void *vmalloc_exec(unsigned long size) { - return __vmalloc_node(size, 1, GFP_KERNEL, PAGE_KERNEL_EXEC, - NUMA_NO_NODE, __builtin_return_address(0)); + return __vmalloc_node_range(size, 1, VMALLOC_START, VMALLOC_END, + GFP_KERNEL, PAGE_KERNEL_EXEC, VM_FLUSH_RESET_PERMS, + NUMA_NO_NODE, __builtin_return_address(0)); } #if defined(CONFIG_64BIT) && defined(CONFIG_ZONE_DMA32) From patchwork Thu Feb 21 23:44:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824913 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6CB2814E1 for ; Thu, 21 Feb 2019 23:51:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5F81331D2C for ; Thu, 21 Feb 2019 23:51:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 52BE231D5D; Thu, 21 Feb 2019 23:51:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B707C31D2C for ; Thu, 21 Feb 2019 23:51:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B34B88E00D5; Thu, 21 Feb 2019 18:51:11 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id B0B7F8E00CD; Thu, 21 Feb 2019 18:51:11 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 986598E00D5; Thu, 21 Feb 2019 18:51:11 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) by kanga.kvack.org (Postfix) with ESMTP id 510E38E00CD for ; Thu, 21 Feb 2019 18:51:11 -0500 (EST) Received: by mail-pf1-f200.google.com with SMTP id i5so365794pfi.1 for ; Thu, 21 Feb 2019 15:51:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=spn5iYpcMMQcbIoRMSyV5W8LqHdI1+8JkLDgWezzEzY=; b=DQ9zY7OjIL2ZBw5BO0LAINE/X8fMIQ8dzYyhOm0ymUmFMskiwq0EAEP6v6oE2L3IcW IU78b5t51ZEa8IIxJVqumeSNtwSepsiEAYbxGISPBNFL2L/Gia7o158jY9iN5cH8ggEl v558CW+4IoXL4Ra65oOZMvYQBf/SBZA78dQJakDVdk+pX8/5MEcvuJKrkR1qhvOhDjGF CFikuXoH/tlmwQ7YH4aqIuap4rlYbyqX9iGebeoALykWO+iRbUeeFw+L9/cWmn6fAq+W M1MG0t7ILL+Zm7TwabeLPOMNBcm64Jbs8hM0g8ZqxeWgHXC2Pt4QV/9aaPeu6fb0Jgph ybEA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAuY9WidCEeW24sF5pzMkjUNTydc74ziyuQMPwnF5aRx8ubAeOIlM 7XWz1ifQKoXDIdoesiLZT8TUOhaIRVyjQlPDa8x3lk56muuRzW//HCgwORUEk6pY6Fn9HMJB7NO D4ZHbCb41bu0Y20NtrEq+7Fs+biG6Xn3UjB+E5gN37/En/fdTqfTvZ/N3h6OoZEOr4Q== X-Received: by 2002:a17:902:765:: with SMTP id 92mr18755pli.95.1550793070859; Thu, 21 Feb 2019 15:51:10 -0800 (PST) X-Google-Smtp-Source: AHgI3IYPUQU+tCVgqcG6JLVxBGy2G8Z6K0YEq2o2u7+5hbTC/9AqfCRhm+76QILmo4lL9egC4Ac0 X-Received: by 2002:a17:902:765:: with SMTP id 92mr18679pli.95.1550793069202; Thu, 21 Feb 2019 15:51:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793069; cv=none; d=google.com; s=arc-20160816; b=HiKjyXUWkWGTtTM8/wOKsRWNjtjm+dSRzAMlV7/l5A+RLERklukB+B+h1A9IXyO9Ce FNos5TUXneuv63ZivExU4s4dyIam0+PaLfZLdUabLA7eV064kCg9tE4fSTYAAI+o60Kd +qPVgXmLsWuQzc7JJFPpyEI+ND9C94eOPx/Mhuf8qxE3IaaZdrqqxU9iRPfuzY27EVbM MZQUUGeJf9MN2h4+QNcYFm94KMNxjagbIE7G348ukTTqd5HEco7o92LHIaav0PSAxEwz dNKb36D8MUaBp3LCQ7sSKJMs7ZZSQ3Y+0LMFMxYTTg+nEpPxVxWiMOVs7lRGU1URp5Yb fKVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=spn5iYpcMMQcbIoRMSyV5W8LqHdI1+8JkLDgWezzEzY=; b=oKpyMka73aDBxvZRPzC2H4d5+hc7Nq9Zh3mKSHVFxw3aqqLu3zpMPIceQlgMBqN4xM x5a5PVba/O4ME2CsKHIyToE3tmrEIKwLA311R3nCDzLi6DPl7Vp7AIIffksByUWy9hQj yYRR96RUtFnnlxhEck1k6vqBN8q0EsTa4ZctO/D6wSNjMksyFIFwF+O/a0wIiUUjAQVc V8b5Gq28XSKZIPS0NQjFn7F25iebNwd8FfV1KLZ+gfEQAyaBfe+PEJIjDUSc9iXlZDAx 8iVnf9TJsNHd4MlENZeD/8jJ0yZpj28c0Cn0Gq4z+Qww4BZLaW1z4o6gYKHq1hW0BOTk hWkA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.51.09 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:51:09 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:51:08 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394943" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:51:07 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Rick Edgecombe , Jessica Yu , Steven Rostedt Subject: [PATCH v3 16/20] modules: Use vmalloc special flag Date: Thu, 21 Feb 2019 15:44:47 -0800 Message-Id: <20190221234451.17632-17-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Use new flag for handling freeing of special permissioned memory in vmalloc and remove places where memory was set RW before freeing which is no longer needed. Since freeing of VM_FLUSH_RESET_PERMS memory is not supported in an interrupt by vmalloc, the freeing of init sections is moved to a work queue. Instead of call_rcu it now uses synchronize_rcu() in the work queue. Lastly, there is now a WARN_ON in module_memfree since it should not be called in an interrupt with special memory as is required for VM_FLUSH_RESET_PERMS. Cc: Jessica Yu Cc: Steven Rostedt Signed-off-by: Rick Edgecombe --- kernel/module.c | 77 +++++++++++++++++++++++++------------------------ 1 file changed, 39 insertions(+), 38 deletions(-) diff --git a/kernel/module.c b/kernel/module.c index ae1b77da6a20..3b97dfb47afb 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -98,6 +98,10 @@ DEFINE_MUTEX(module_mutex); EXPORT_SYMBOL_GPL(module_mutex); static LIST_HEAD(modules); +/* Work queue for freeing init sections in success case */ +static struct work_struct init_free_wq; +static struct llist_head init_free_list; + #ifdef CONFIG_MODULES_TREE_LOOKUP /* @@ -1949,6 +1953,8 @@ void module_enable_ro(const struct module *mod, bool after_init) if (!rodata_enabled) return; + set_vm_flush_reset_perms(mod->core_layout.base); + set_vm_flush_reset_perms(mod->init_layout.base); frob_text(&mod->core_layout, set_memory_ro); frob_text(&mod->core_layout, set_memory_x); @@ -1972,15 +1978,6 @@ static void module_enable_nx(const struct module *mod) frob_writable_data(&mod->init_layout, set_memory_nx); } -static void module_disable_nx(const struct module *mod) -{ - frob_rodata(&mod->core_layout, set_memory_x); - frob_ro_after_init(&mod->core_layout, set_memory_x); - frob_writable_data(&mod->core_layout, set_memory_x); - frob_rodata(&mod->init_layout, set_memory_x); - frob_writable_data(&mod->init_layout, set_memory_x); -} - /* Iterate through all modules and set each module's text as RW */ void set_all_modules_text_rw(void) { @@ -2024,23 +2021,8 @@ void set_all_modules_text_ro(void) } mutex_unlock(&module_mutex); } - -static void disable_ro_nx(const struct module_layout *layout) -{ - if (rodata_enabled) { - frob_text(layout, set_memory_rw); - frob_rodata(layout, set_memory_rw); - frob_ro_after_init(layout, set_memory_rw); - } - frob_rodata(layout, set_memory_x); - frob_ro_after_init(layout, set_memory_x); - frob_writable_data(layout, set_memory_x); -} - #else -static void disable_ro_nx(const struct module_layout *layout) { } static void module_enable_nx(const struct module *mod) { } -static void module_disable_nx(const struct module *mod) { } #endif #ifdef CONFIG_LIVEPATCH @@ -2120,6 +2102,11 @@ static void free_module_elf(struct module *mod) void __weak module_memfree(void *module_region) { + /* + * This memory may be RO, and freeing RO memory in an interrupt is not + * supported by vmalloc. + */ + WARN_ON(in_interrupt()); vfree(module_region); } @@ -2171,7 +2158,6 @@ static void free_module(struct module *mod) mutex_unlock(&module_mutex); /* This may be empty, but that's OK */ - disable_ro_nx(&mod->init_layout); module_arch_freeing_init(mod); module_memfree(mod->init_layout.base); kfree(mod->args); @@ -2181,7 +2167,6 @@ static void free_module(struct module *mod) lockdep_free_key_range(mod->core_layout.base, mod->core_layout.size); /* Finally, free the core (containing the module structure) */ - disable_ro_nx(&mod->core_layout); module_memfree(mod->core_layout.base); } @@ -3424,17 +3409,34 @@ static void do_mod_ctors(struct module *mod) /* For freeing module_init on success, in case kallsyms traversing */ struct mod_initfree { - struct rcu_head rcu; + struct llist_node node; void *module_init; }; -static void do_free_init(struct rcu_head *head) +static void do_free_init(struct work_struct *w) { - struct mod_initfree *m = container_of(head, struct mod_initfree, rcu); - module_memfree(m->module_init); - kfree(m); + struct llist_node *pos, *n, *list; + struct mod_initfree *initfree; + + list = llist_del_all(&init_free_list); + + synchronize_rcu(); + + llist_for_each_safe(pos, n, list) { + initfree = container_of(pos, struct mod_initfree, node); + module_memfree(initfree->module_init); + kfree(initfree); + } } +static int __init modules_wq_init(void) +{ + INIT_WORK(&init_free_wq, do_free_init); + init_llist_head(&init_free_list); + return 0; +} +module_init(modules_wq_init); + /* * This is where the real work happens. * @@ -3511,7 +3513,6 @@ static noinline int do_init_module(struct module *mod) #endif module_enable_ro(mod, true); mod_tree_remove_init(mod); - disable_ro_nx(&mod->init_layout); module_arch_freeing_init(mod); mod->init_layout.base = NULL; mod->init_layout.size = 0; @@ -3522,14 +3523,18 @@ static noinline int do_init_module(struct module *mod) * We want to free module_init, but be aware that kallsyms may be * walking this with preempt disabled. In all the failure paths, we * call synchronize_rcu(), but we don't want to slow down the success - * path, so use actual RCU here. + * path. module_memfree() cannot be called in an interrupt, so do the + * work and call synchronize_rcu() in a work queue. + * * Note that module_alloc() on most architectures creates W+X page * mappings which won't be cleaned up until do_free_init() runs. Any * code such as mark_rodata_ro() which depends on those mappings to * be cleaned up needs to sync with the queued work - ie * rcu_barrier() */ - call_rcu(&freeinit->rcu, do_free_init); + if (llist_add(&freeinit->node, &init_free_list)) + schedule_work(&init_free_wq); + mutex_unlock(&module_mutex); wake_up_all(&module_wq); @@ -3826,10 +3831,6 @@ static int load_module(struct load_info *info, const char __user *uargs, module_bug_cleanup(mod); mutex_unlock(&module_mutex); - /* we can't deallocate the module until we clear memory protection */ - module_disable_ro(mod); - module_disable_nx(mod); - ddebug_cleanup: ftrace_release_mod(mod); dynamic_debug_remove(mod, info->debug); From patchwork Thu Feb 21 23:44:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824917 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C8DCE1399 for ; Thu, 21 Feb 2019 23:51:48 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BB87831D2C for ; Thu, 21 Feb 2019 23:51:48 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AF38531D30; Thu, 21 Feb 2019 23:51:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 464C131D2C for ; Thu, 21 Feb 2019 23:51:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D1B228E00CD; Thu, 21 Feb 2019 18:51:11 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CBE9E8E00D6; Thu, 21 Feb 2019 18:51:11 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B30EA8E00D4; Thu, 21 Feb 2019 18:51:11 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f197.google.com (mail-pl1-f197.google.com [209.85.214.197]) by kanga.kvack.org (Postfix) with ESMTP id 6AAA98E00D4 for ; Thu, 21 Feb 2019 18:51:11 -0500 (EST) Received: by mail-pl1-f197.google.com with SMTP id s22so318456plq.7 for ; Thu, 21 Feb 2019 15:51:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=wlyB6G1b+Tm41nLBBmWshnK0KkaA3tTinkVpxpprDWU=; b=jiyNeEoLJPbe3ysqJieZtUAoHY8a6NxL+9hiNSGQSo3WbCHAOO4kna3471Pr2afrMN h3J+4+7BtulZJ2g8yZBtGkcOdPdeBTVSi2XWnf9iPuRjaGfD6kc/EVnViM154BvEZnHY mtXZVuyRcliqBxuCprdKsqX8K1qQWNW3zjdKbcBVxgAE2+KFVPz391X3BlT2B4EqE+5j 9se8nEkYUcarbz5XLXVZk8SBDrLAKsVgZWzCzyJh9wfkPGAUHA+SzGc9OYZ5kCBsrdyn lfc2FGSM+X6/4VAKCO+x3R8tMgJvpLKfHTxVmHR8La6RoDgJg6hJOq598AafjdQ+K/Xd ETDg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAubay3TIpuWEldMoewTpIvLPkHhnMWSePttEtgMyq7c0oBcnQwgF Ccw73w87EytK+mKFuV+OGj6UaGYJbXVn1vPdUwMJCW1JfNlWhRCBOjnjEQBfFnRKXGVVeRlM2sI gdFIwegfTmSUcCbg/oKCagIVmQHhGuCzNwRmPYbYCSpgP7g/AJD7vqxtgD+94GyGZkA== X-Received: by 2002:a17:902:6bc7:: with SMTP id m7mr1223989plt.106.1550793071029; Thu, 21 Feb 2019 15:51:11 -0800 (PST) X-Google-Smtp-Source: AHgI3IYsrEx/PaShl8Jeua71g0BpS+zpIalcy31KXVdoiuPt3WQuxAjE0y9qPkenQwVoP0md63rl X-Received: by 2002:a17:902:6bc7:: with SMTP id m7mr1223947plt.106.1550793070071; Thu, 21 Feb 2019 15:51:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793070; cv=none; d=google.com; s=arc-20160816; b=e7cLln3MfGDsSKUG3n+yWCUfvHESpRS0kTlEPHkA5670ir3yk6Epm0ZEVGrnrZ5IIl AE/kG53cMWBvsh4osw/uvC1edSqYa9wHSxxSuOJUAONNBOVQrxS9awLtpa5xWI3JeQSs wc1aHgvdrxCK4lGvuzlBj4lMqjFs2X65apoPScK+0vRfPc5MtcuqHOodXc8IrgE3JtsF vzCClpBhYlK5xd/G2Bwp/OEYs2G0iLRuXNz0vLjUvBTttBN81rBxo5Qu7vZaiqkQhW5I gbZCJc7/DzlVhlvjSOWqE3u1lbcM/fpWbyspS6dnViaUjHCKo0nbt0TJjiAW6o4itj0V x7sQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=wlyB6G1b+Tm41nLBBmWshnK0KkaA3tTinkVpxpprDWU=; b=Hi/Ialk4okQ+e+rtnIO/k05+5/LHufen2Ah6cTUvgaek8MyCdLztEZh9ZH5YRTKEkM 5aYJKLB7gh/7US7bJ+OYe9gKc/ZgrqEAEtj+u5IxUvD5/jfWwY5wjeo41Z6qHbHabVeB 9nyqSjhFw8qWohkU2XlFqV5O5XDTaoAR2LRIbZJFPc6dSi+FP0BGoeeGZbLF4ASGaR+5 wOJR/+vCbkXuwlVf3k25cqXeQ7FqBHdELhpSQlLGJKRvP55AZQAQY4ZPvScYgJDZGLls jCBPa3dw817fHtZqWKbb1kqAGGOxZ8NuQa135DLPdYpYCwWxVk1RrYSCpBVpTF3JEsDm CoGA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.51.09 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:51:10 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:51:09 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394951" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:51:08 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Rick Edgecombe , Daniel Borkmann , Alexei Starovoitov Subject: [PATCH v3 17/20] bpf: Use vmalloc special flag Date: Thu, 21 Feb 2019 15:44:48 -0800 Message-Id: <20190221234451.17632-18-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Use new flag VM_FLUSH_RESET_PERMS for handling freeing of special permissioned memory in vmalloc and remove places where memory was set RW before freeing which is no longer needed. Don't track if the memory is RO anymore because it is now tracked in vmalloc. Cc: Daniel Borkmann Cc: Alexei Starovoitov Signed-off-by: Rick Edgecombe --- include/linux/filter.h | 17 +++-------------- kernel/bpf/core.c | 1 - 2 files changed, 3 insertions(+), 15 deletions(-) diff --git a/include/linux/filter.h b/include/linux/filter.h index b9f93e62db96..f7b6c8a2e591 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -20,6 +20,7 @@ #include #include #include +#include #include @@ -483,7 +484,6 @@ struct bpf_prog { u16 pages; /* Number of allocated pages */ u16 jited:1, /* Is our filter JIT'ed? */ jit_requested:1,/* archs need to JIT the prog */ - undo_set_mem:1, /* Passed set_memory_ro() checkpoint */ gpl_compatible:1, /* Is filter GPL compatible? */ cb_access:1, /* Is control block accessed? */ dst_needed:1, /* Do we need dst entry? */ @@ -681,27 +681,17 @@ bpf_ctx_narrow_access_ok(u32 off, u32 size, u32 size_default) static inline void bpf_prog_lock_ro(struct bpf_prog *fp) { - fp->undo_set_mem = 1; + set_vm_flush_reset_perms(fp); set_memory_ro((unsigned long)fp, fp->pages); } -static inline void bpf_prog_unlock_ro(struct bpf_prog *fp) -{ - if (fp->undo_set_mem) - set_memory_rw((unsigned long)fp, fp->pages); -} - static inline void bpf_jit_binary_lock_ro(struct bpf_binary_header *hdr) { + set_vm_flush_reset_perms(hdr); set_memory_ro((unsigned long)hdr, hdr->pages); set_memory_x((unsigned long)hdr, hdr->pages); } -static inline void bpf_jit_binary_unlock_ro(struct bpf_binary_header *hdr) -{ - set_memory_rw((unsigned long)hdr, hdr->pages); -} - static inline struct bpf_binary_header * bpf_jit_binary_hdr(const struct bpf_prog *fp) { @@ -736,7 +726,6 @@ void __bpf_prog_free(struct bpf_prog *fp); static inline void bpf_prog_unlock_free(struct bpf_prog *fp) { - bpf_prog_unlock_ro(fp); __bpf_prog_free(fp); } diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 19c49313c709..465c1c3623e8 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -804,7 +804,6 @@ void __weak bpf_jit_free(struct bpf_prog *fp) if (fp->jited) { struct bpf_binary_header *hdr = bpf_jit_binary_hdr(fp); - bpf_jit_binary_unlock_ro(hdr); bpf_jit_binary_free(hdr); WARN_ON_ONCE(!bpf_prog_kallsyms_verify_off(fp)); From patchwork Thu Feb 21 23:44:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824919 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 21696922 for ; Thu, 21 Feb 2019 23:51:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1398A31D2C for ; Thu, 21 Feb 2019 23:51:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 07D0931D3C; Thu, 21 Feb 2019 23:51:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9954331D2C for ; Thu, 21 Feb 2019 23:51:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 037FC8E00D6; Thu, 21 Feb 2019 18:51:13 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id EDBD98E00D4; Thu, 21 Feb 2019 18:51:12 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA3B38E00D6; Thu, 21 Feb 2019 18:51:12 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) by kanga.kvack.org (Postfix) with ESMTP id 9A6198E00D4 for ; Thu, 21 Feb 2019 18:51:12 -0500 (EST) Received: by mail-pl1-f200.google.com with SMTP id b4so316269plb.9 for ; Thu, 21 Feb 2019 15:51:12 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=QccAqP9ksTYPDqnzYjlcAcLDZYYNNkUf8Gwz8DkOtZc=; b=nHwLgKvX6c4h38m98FK14jAJuZgRL8xJmVJkAExjeRNsOiJdGKBeiYRIJciqnMl3WH 2fbz3JhI/4WYtnwNoIOiETf0fenc6+mUQDkYWNudybVbftwBBOqh/MtnIGCm0+CE6bfa pYV4w4dtJAWYrybBqEIuYXLBvWsHozQXLMsVk6t71D0me+CzZRYclGXrRWM2ecugPfgD B03bV4i4SrRzSOu1eyvumz012uw8jgW8CiInM/3lUpzgf7lmJ+ZgoB/2GJUlDdYd5Ulv kWQuj3btXtSmGEnS/TqntpvbhCKv2ed2mn94Lkc82BwR7e5wZMQYGTGNRdw/koCFwODa 5x0A== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAuZcF0GG2hF4K5uy2+PSAYaQb7lEc/lWk6AthMyjGKcO0pQAfNlc +kDhVviAozEJvfzZuYJbZnZysAfPSZhkXEfDpVYfLwChGlBJJICbUUYZjhbWXP9J8962Jg7iWSW Kl1UjXFVkzsEI/qYN+BEEz+FESPXrCaC7gc0gTlxvKDVCQ1f79BP99q+t5qe2iogWsQ== X-Received: by 2002:a62:f51d:: with SMTP id n29mr1133364pfh.21.1550793072296; Thu, 21 Feb 2019 15:51:12 -0800 (PST) X-Google-Smtp-Source: AHgI3IZU+N4zo+AyvPZY4MPxsLe9ed/LWrubWi8OmtUHyCwINETYAhJ4Tfi62Avb8ieM7W9MbN1y X-Received: by 2002:a62:f51d:: with SMTP id n29mr1133312pfh.21.1550793071080; Thu, 21 Feb 2019 15:51:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793071; cv=none; d=google.com; s=arc-20160816; b=x6r/N5Iq9SFLQ3KR8UxH9HErwbT/XreLDA3ty4gYaXFitOr4w5w8BxHkKPevdd84zE w7TIS2zarGibR9UxsDN+rm/A00cW8F8O5jApVIZVDyd9ki7QAjSAbhAHCG9HtzWQOYAQ 9XoYzCsWq29j5wOWz67I089ZOF/0Xm9dfE12dS0zWhVYHA6nLzGekBakNZRp2D3kEEnK 9JRJVaw38uMNgnrFsf/NIqmeU5LngncLSYIMSZ8kNwg9frGMMbP75dw4qb/tZNXwm6vQ JyL+xcFhx1w2qRf6hsm9OEVSv0W+55aIprf89ca+i95+F3zJGhjgzg8IqRfSGANcg3AV xaYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=QccAqP9ksTYPDqnzYjlcAcLDZYYNNkUf8Gwz8DkOtZc=; b=aXYd8KRyUZ9j97ZYafMca/vBzXS3R74BEkdhBwgY2NbYDOVkSTeOswnyR8H3Tomm+j 5Ke7gRBulkowv5Qnq59eS3C7dZW+9HUNIG4w1TQxsmQeir+78zUaRSyRJivT6jZ6U4La EWCbOLWT+96gJGR8Wfe+Ja+naooU2+xcgvAQnlfbA7pU5otafmkpuq7uWgQK85UQ+kOj 39FGI4wEnY5j5pv1dp8UWxhqFzJRvDfGnEB0K4Q00pm7UnhD+dQIALoNGAACbMZ3LmPq 77Wzpt4LkCErttb8KXGSfXcIu2VO7JVQSkuS/DMmQk04MdN9kar4uM7opw44AdKFGhMj qbfw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.51.10 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:51:11 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:51:10 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394958" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:51:09 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Rick Edgecombe , Steven Rostedt Subject: [PATCH v3 18/20] x86/ftrace: Use vmalloc special flag Date: Thu, 21 Feb 2019 15:44:49 -0800 Message-Id: <20190221234451.17632-19-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Use new flag VM_FLUSH_RESET_PERMS for handling freeing of special permissioned memory in vmalloc and remove places where memory was set NX and RW before freeing which is no longer needed. Cc: Steven Rostedt Acked-by: Steven Rostedt (VMware) Signed-off-by: Rick Edgecombe --- arch/x86/kernel/ftrace.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/arch/x86/kernel/ftrace.c b/arch/x86/kernel/ftrace.c index 13c8249b197f..93efe3955333 100644 --- a/arch/x86/kernel/ftrace.c +++ b/arch/x86/kernel/ftrace.c @@ -692,10 +692,6 @@ static inline void *alloc_tramp(unsigned long size) } static inline void tramp_free(void *tramp, int size) { - int npages = PAGE_ALIGN(size) >> PAGE_SHIFT; - - set_memory_nx((unsigned long)tramp, npages); - set_memory_rw((unsigned long)tramp, npages); module_memfree(tramp); } #else @@ -820,6 +816,8 @@ create_trampoline(struct ftrace_ops *ops, unsigned int *tramp_size) /* ALLOC_TRAMP flags lets us know we created it */ ops->flags |= FTRACE_OPS_FL_ALLOC_TRAMP; + set_vm_flush_reset_perms(trampoline); + /* * Module allocation needs to be completed by making the page * executable. The page is still writable, which is a security hazard, From patchwork Thu Feb 21 23:44:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824923 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4C5791399 for ; Thu, 21 Feb 2019 23:51:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3EB8131D2C for ; Thu, 21 Feb 2019 23:51:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 32AF531D30; Thu, 21 Feb 2019 23:51:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D83E331D2C for ; Thu, 21 Feb 2019 23:51:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6C5558E00D7; Thu, 21 Feb 2019 18:51:13 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 686E58E00D4; Thu, 21 Feb 2019 18:51:13 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 517F18E00D4; Thu, 21 Feb 2019 18:51:13 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f198.google.com (mail-pg1-f198.google.com [209.85.215.198]) by kanga.kvack.org (Postfix) with ESMTP id 1380E8E00D7 for ; Thu, 21 Feb 2019 18:51:13 -0500 (EST) Received: by mail-pg1-f198.google.com with SMTP id e5so317773pgc.16 for ; Thu, 21 Feb 2019 15:51:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=twme+qjq33jZ13XlnvG7AH24qCU88X9Kbd4vP+8UtMM=; b=jkjOW7PuA7kEOZIyp1Gp5zgjQvaJdpFUXnf7M3tUVt7zpXc/2YrN5IEWR8bKW5ARUd pAAYpd37fNyh5Z5qBn8NAUUG01uwI8hi1F4eCBqc9Kw2d1aOoko+HaVziTDrz1LOo3Wm ceU8I1L25q6k1Et6n0m0bxsmOcxzLRKSZS/Gh3cJZa1cj79LRfoVe80bcOsHcbmf2mYv whS8Pxnwsx2ALMEGWHX+u2mvWl0qTckFEzPi1awVH7bT6xoE2J3+YiRlHLq9UtCzG9if 84o7ra7JmV4p8tfuFzLQ1JcdEhujuSos7bJTOHkdeI/8aOgEImPzidkt0TaTJkJ0RZS8 iZmQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAuYky4wvv96Ovr+m35lLYgaviKMSSt/6wMaMOgbXTULBcIvjlYWb OIX6u+lIxlroX7W23XlCXnyGmL6xbL9DuNI3DFa8Ah+lLV1QjlujYJZrU+gI/ZeKdGlrq5AuR2T MAqhPoRtMekf2oGQ0Dd5rH7/bYAWjfdoc/TcTWPaLIi9dmgcMkUhg+DI+Z7wJrzFRdA== X-Received: by 2002:a17:902:22a:: with SMTP id 39mr1165244plc.153.1550793072747; Thu, 21 Feb 2019 15:51:12 -0800 (PST) X-Google-Smtp-Source: AHgI3Ibpo/e/BOv/b9u2I/sWFfsKnNdZ2w17bAMSK6+hSXNn6ZU9OBTFIuf1fKRK31r4wfU1g4Ok X-Received: by 2002:a17:902:22a:: with SMTP id 39mr1165213plc.153.1550793072043; Thu, 21 Feb 2019 15:51:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793072; cv=none; d=google.com; s=arc-20160816; b=s68qqi6et1GOSTdhMVXRtbJLClJJ8gcA9xFoOZ5HK88A7gzhn/kxt7PctpudzW9Scq 7cJj3FAhhddJvXKJpwkwQ7eRF47+5s2P6Yfg3hPWfpFo12yK3Fj8bfL9H+XEtSNaYTqh 9kc4UyX4rJW30LB3kLJ3cDAsSlXiKS0TYtNSViF6oMMJ/61WxGU6YPUYOxdUDPo6a8T9 MSU/FIiDjjNJK7P3lpg26C5US8FZV2oeLtK5BcR9CimEkgnBUIwCo7M/EBWyNuch6CPi sWn1Nju02m00IyGzpYukPGgSSXpjJLjC0aKiIKfAfwIpWx4C4m3f3unUdNClq2+y/nQ5 h8mw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=twme+qjq33jZ13XlnvG7AH24qCU88X9Kbd4vP+8UtMM=; b=IC/sokXqB6InHv2ZQWuMKi7tPcmCAHGmc1wmrq4knvxsZc7EhgzOY9fFGslquhGf8C 3+KUYjl7nte1ooaz9/IBfwFS6OhIWvo3VuPwphBYqKc4OsuCwcoKifIFmdtlUARjT4Ib 3R+yxPf33mdV+1I4sQhNpYmaTGWrvImKjohFyjov1Bsa/q423lpgGm+Dq1yr8fmngrCA I9l+S6Kdswl1GTKaJKlsiQyI+bLznA0GlSf1okI6J8cQGCsudfjlnL5Q3675lqzf1HM3 3eqnGr1S7B5RROw7qaOIWv9jNBbiT36vASOLYHZ9+JBPBHkQ35+pzikZmIj9LSgbevQJ EPaw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.51.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:51:12 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:51:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394963" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:51:10 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Rick Edgecombe , Masami Hiramatsu Subject: [PATCH v3 19/20] x86/kprobes: Use vmalloc special flag Date: Thu, 21 Feb 2019 15:44:50 -0800 Message-Id: <20190221234451.17632-20-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Use new flag VM_FLUSH_RESET_PERMS for handling freeing of special permissioned memory in vmalloc and remove places where memory was set NX and RW before freeing which is no longer needed. Cc: Masami Hiramatsu Signed-off-by: Rick Edgecombe --- arch/x86/kernel/kprobes/core.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c index 98c671e89889..8b56935d7b53 100644 --- a/arch/x86/kernel/kprobes/core.c +++ b/arch/x86/kernel/kprobes/core.c @@ -434,6 +434,7 @@ void *alloc_insn_page(void) if (!page) return NULL; + set_vm_flush_reset_perms(page); /* * First make the page read-only, and only then make it executable to * prevent it from being W+X in between. @@ -452,12 +453,6 @@ void *alloc_insn_page(void) /* Recover page to RW mode before releasing it */ void free_insn_page(void *page) { - /* - * First make the page non-executable, and only then make it writable to - * prevent it from being W+X in between. - */ - set_memory_nx((unsigned long)page, 1); - set_memory_rw((unsigned long)page, 1); module_memfree(page); } From patchwork Thu Feb 21 23:44:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgecombe, Rick P" X-Patchwork-Id: 10824935 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 943A8922 for ; Thu, 21 Feb 2019 23:51:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 868B831D2C for ; Thu, 21 Feb 2019 23:51:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7AE7831D2E; Thu, 21 Feb 2019 23:51:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1EE4B31D3C for ; Thu, 21 Feb 2019 23:51:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6F0508E00D8; Thu, 21 Feb 2019 18:51:14 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 69FB18E00D4; Thu, 21 Feb 2019 18:51:14 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4F08D8E00D8; Thu, 21 Feb 2019 18:51:14 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f199.google.com (mail-pf1-f199.google.com [209.85.210.199]) by kanga.kvack.org (Postfix) with ESMTP id 0B5708E00D4 for ; Thu, 21 Feb 2019 18:51:14 -0500 (EST) Received: by mail-pf1-f199.google.com with SMTP id i5so365869pfi.1 for ; Thu, 21 Feb 2019 15:51:14 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=5kZP+xlzcd2ctLk3fWzs8E8qpboYLOQ3hu5nUlPFhXg=; b=Rk7mzY5dhDixxuVJtJ3okppgsAEBClKUd/WCzUaumRgb9NjSb1Nq/BgMc+1zecyLps LqyCu8+abQhqOjCQ5z+NmPeTLDWdo4Uihxt0xUQxYvJ0H8pbWCCgl84Crr9WjoAMXXac jVYsK2LFA40AdySwx9uUE0rhlCc2a/GuKtn7iPQPSFOPdmvF81gSH1Cix0t4rQVwkO8j AsBL01rB79GdQogCw2YE8ilzTm+LmKZjYVWBwwoiqePryNQpHaPIzQruSbPToCmEvrBm tnUepfczgkh+ynqqP0OWaiDoBFIyc47vkEFC9JFNMkdAuOfZgeyR8eOiAtxXm3nRD1wx i9Uw== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AHQUAubmSLACWh4GBfaYDjaG1VDu5Mfitquqv2BtOf7xCc5vQ2Zb57qP 23UUgSIZCeW1yyTbJ8CTHXt65LxLs+lAB60KmrEyABrWZjd5SgFrBWrcsYQNqDK6od2sKcXYgi7 X2JyeZ/8ev6gKdD7SlvK1nDuBK+xpyRfloTuYYKseoEqY5uRMx6CfI/lFjxpQ/9vYYw== X-Received: by 2002:a62:1706:: with SMTP id 6mr1160085pfx.28.1550793073726; Thu, 21 Feb 2019 15:51:13 -0800 (PST) X-Google-Smtp-Source: AHgI3IaDRrRRuMDaYgrQdoghqF6ZHULz6c7YiyQOLIBpti3E2m02F8V5+UWbvEIvLQhyjSwMPCzR X-Received: by 2002:a62:1706:: with SMTP id 6mr1160046pfx.28.1550793073025; Thu, 21 Feb 2019 15:51:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550793073; cv=none; d=google.com; s=arc-20160816; b=jUV+lHtqcPJERQnxMZOmd/5zqQEE9Gw9TWcGA/ZzB0Hl90bKrQqDzjZy+PFjvZSoAS 08KlCdy5Fsh7mqqShYmhbq4u9e1pHWVcUnC2uRPDhcPIi65OtDzZ6923PdpiwpttIbv6 SOhQQ5n4g3deYhWrk58TLkPGJB5tah+APXnWG4cmd4CPDmoyHTflOnVG6MTuQ8kSaFYP kN6udCwwRO/JW39G4V5IHVKBTQUFGzlm08XJoKPEPUtZotuu9TdniRLu6gjm+mXLEOGC dmSv8/ijKElcvbLJTUIH03V53cASuUjlXC2bYFNrItf1tyQzJqbVe4veu1IjmL8ukg3k E0GQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=5kZP+xlzcd2ctLk3fWzs8E8qpboYLOQ3hu5nUlPFhXg=; b=ikywDyiYXw/4KpVp2vgn7sYblVRfoS9guQ0jGPJy1xL6R2DrNxKoOMq7Yzq0fPoWro TZXjEOwsESyg3oNuCxZLen6ngPPC74Y86XamEYan0W8L5YU4/98RCPGi2afLSZSDbSsl jAIMjx+0A0al+s/vCcXNbWYAgqOdXe+tIWrsyEHS95ykWKUhwL/laS7YxiVFqkSwQh3P 7sgUfeprQmWwiLy39Vlo6BnmsEsbqbk1DR3GU9tgK9S3PZjCnaqd+Fbj5Txi5E+v/t2o vOoqd085dFFLqNwq5ROVFAjy+kfdRDgHuDwMk5/7eRhJK+Y8921YIaY6xBTXN2eECiTJ kUvg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga03.intel.com (mga03.intel.com. [134.134.136.65]) by mx.google.com with ESMTPS id c4si238494pfn.83.2019.02.21.15.51.12 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Feb 2019 15:51:13 -0800 (PST) Received-SPF: pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) client-ip=134.134.136.65; Authentication-Results: mx.google.com; spf=pass (google.com: domain of rick.p.edgecombe@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Feb 2019 15:51:12 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,397,1544515200"; d="scan'208";a="322394969" Received: from linksys13920.jf.intel.com (HELO rpedgeco-DESK5.jf.intel.com) ([10.54.75.11]) by fmsmga005.fm.intel.com with ESMTP; 21 Feb 2019 15:51:11 -0800 From: Rick Edgecombe To: Andy Lutomirski , Ingo Molnar Cc: linux-kernel@vger.kernel.org, x86@kernel.org, hpa@zytor.com, Thomas Gleixner , Borislav Petkov , Nadav Amit , Dave Hansen , Peter Zijlstra , linux_dti@icloud.com, linux-integrity@vger.kernel.org, linux-security-module@vger.kernel.org, akpm@linux-foundation.org, kernel-hardening@lists.openwall.com, linux-mm@kvack.org, will.deacon@arm.com, ard.biesheuvel@linaro.org, kristen@linux.intel.com, deneen.t.dock@intel.com, Nadav Amit , Masami Hiramatsu , Rick Edgecombe Subject: [PATCH v3 20/20] x86/alternative: Comment about module removal races Date: Thu, 21 Feb 2019 15:44:51 -0800 Message-Id: <20190221234451.17632-21-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190221234451.17632-1-rick.p.edgecombe@intel.com> References: <20190221234451.17632-1-rick.p.edgecombe@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Nadav Amit Add a comment to clarify that users of text_poke() must ensure that no races with module removal take place. Cc: Masami Hiramatsu Signed-off-by: Nadav Amit Signed-off-by: Rick Edgecombe --- arch/x86/kernel/alternative.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index c63707e7ed3d..a1335b9486bf 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -809,6 +809,11 @@ static void *__text_poke(void *addr, const void *opcode, size_t len) * It means the size must be writable atomically and the address must be aligned * in a way that permits an atomic write. It also makes sure we fit on a single * page. + * + * Note that the caller must ensure that if the modified code is part of a + * module, the module would not be removed during poking. This can be achieved + * by registering a module notifier, and ordering module removal and patching + * trough a mutex. */ void *text_poke(void *addr, const void *opcode, size_t len) {