From patchwork Fri Oct 18 16:10:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199027 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F145F1951 for ; Fri, 18 Oct 2019 16:11:21 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CDF79222C6 for ; Fri, 18 Oct 2019 16:11:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="fQxG2Z0W"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="gMmfpLUm" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CDF79222C6 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=luCzNxDO82sWMFEn0Gs+MBJbhsDGsvd8vNrsp7AsHf4=; b=fQxG2Z0W9ZPUb8 Vgr9VTEyGiSYq2S7lSnH7UNxfvyocPqo7caokYMGPi5n7DRa6jpQt10gEepG3RreLFzp1Wb1KQcH6 sH7kA8lB1BSbt/w2aE+4HzKkicLCWfBAg3BNxL+H7z3PMt7PodQuCVdYvj2rUy96TLeksgdjuF9IZ HgHEBLM/w40Dejq2gtA2t3N0bnsKdEhS9JXTyfiwUiywj/xeURkOrK0IBFFtTO6ry/ILEqLbnITjf ANLXOk7pEXiMGcTsN1tgNOnK66qlGygqa1ip67514AA9VH+Ebsuy5kQ7UK0UqPp6lmxnJkPRffUAh oE2RV+zL+gcsO/8Zz5UQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUqW-0001xV-Sc; Fri, 18 Oct 2019 16:11:20 +0000 Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUq1-0001TB-C9 for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:10:50 +0000 Received: by mail-pl1-x649.google.com with SMTP id 99so4015047plc.18 for ; Fri, 18 Oct 2019 09:10:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=fOGiRQkTcafFw3kdPLJfkScWMPKQ55OPiVpVgBq1L2Y=; b=gMmfpLUmWSms+DWxdPHIfAD6vs9RBHnPXI+iTSb/gCG+R7cNXs2b5uBPJCYTEo7izz /Raf56KKS95suuvqh1yzOQjpOZYsH/4vRvMlKN0ZIyyBfnU2AeT4YoERdtJm3yrC4+vY 0OxEL7940UbAaXzNMVID1dP6HRdAhIQ6wFjNYG57zbpFHwZ0WwfTPVCFFZhJpOF/t3A9 9CLRQ0xV3QFyyGRY1rJZV9hFziEOlin2nV27aZ33hx/xGULb4Cz9V3pJJJy/KJcOVnua JRuCV/9ChbbioD0f/pMbO9vPUsu8raQ2egyKkU/VBLQwhJoGxgstGPYnuLnJSbGzElKZ XzGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=fOGiRQkTcafFw3kdPLJfkScWMPKQ55OPiVpVgBq1L2Y=; b=XkgsXViKQeZsY+lxmDKhmewHS4m3TuI0xjjoGBWtjXAoDDiffzN8GlOQFsy0xZ6Utz HUKbpTFyRxtjYs8meH4TJDk7BmLZ5iHySgUjjuB8zcUBYO22waWpsfWnvfryjl1wUkSr ZSmeCRc6PV22X+MJrD0qqknna29QZuutvZ8O0HOEB0q0hrvr56dYm+EyDXxbrd7BrzGZ 1IjUdJeX4+wK46bNW/H/EE6UUMhyB/YVGdkb0GhAXf9aynkeZTraIE9mzjKE4DpNR1ue LRys8hS2mhUZb9AhC9uq9l8+a+EeBth0gSLno51nyEbvcK3Kd3wnu8gBhAsM7FfQTKE/ 9C3g== X-Gm-Message-State: APjAAAWg0nweXBhDsyJmUdTRpII65XLrvoT0gd1NNjK/HlM/7Rb3MWN2 TP+vCz9OIY3AxCax0GJ0OP3XSGH41YdIstRbwTE= X-Google-Smtp-Source: APXvYqxdcdFBRFb0HNyGzQvnCMLB0keAhMf4yh0jcUv/6ytScnNhV6MYVkz//a3X1pTXSJd9ILJJXg39d/ovVZN0Qvw= X-Received: by 2002:a63:3c5a:: with SMTP id i26mr10727727pgn.207.1571415047168; Fri, 18 Oct 2019 09:10:47 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:16 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-2-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 01/18] arm64: mm: don't use x18 in idmap_kpti_install_ng_mappings From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_091049_441622_9933EBF0 X-CRM114-Status: GOOD ( 10.20 ) X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:649 listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org idmap_kpti_install_ng_mappings uses x18 as a temporary register, which will result in a conflict when x18 is reserved. Use x16 and x17 instead where needed. Signed-off-by: Sami Tolvanen Reviewed-by: Nick Desaulniers --- arch/arm64/mm/proc.S | 63 ++++++++++++++++++++++---------------------- 1 file changed, 32 insertions(+), 31 deletions(-) diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index a1e0592d1fbc..fdabf40a83c8 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -250,15 +250,15 @@ ENTRY(idmap_kpti_install_ng_mappings) /* We're the boot CPU. Wait for the others to catch up */ sevl 1: wfe - ldaxr w18, [flag_ptr] - eor w18, w18, num_cpus - cbnz w18, 1b + ldaxr w17, [flag_ptr] + eor w17, w17, num_cpus + cbnz w17, 1b /* We need to walk swapper, so turn off the MMU. */ pre_disable_mmu_workaround - mrs x18, sctlr_el1 - bic x18, x18, #SCTLR_ELx_M - msr sctlr_el1, x18 + mrs x17, sctlr_el1 + bic x17, x17, #SCTLR_ELx_M + msr sctlr_el1, x17 isb /* Everybody is enjoying the idmap, so we can rewrite swapper. */ @@ -281,9 +281,9 @@ skip_pgd: isb /* We're done: fire up the MMU again */ - mrs x18, sctlr_el1 - orr x18, x18, #SCTLR_ELx_M - msr sctlr_el1, x18 + mrs x17, sctlr_el1 + orr x17, x17, #SCTLR_ELx_M + msr sctlr_el1, x17 isb /* @@ -353,46 +353,47 @@ skip_pte: b.ne do_pte b next_pmd + .unreq cpu + .unreq num_cpus + .unreq swapper_pa + .unreq cur_pgdp + .unreq end_pgdp + .unreq pgd + .unreq cur_pudp + .unreq end_pudp + .unreq pud + .unreq cur_pmdp + .unreq end_pmdp + .unreq pmd + .unreq cur_ptep + .unreq end_ptep + .unreq pte + /* Secondary CPUs end up here */ __idmap_kpti_secondary: /* Uninstall swapper before surgery begins */ - __idmap_cpu_set_reserved_ttbr1 x18, x17 + __idmap_cpu_set_reserved_ttbr1 x16, x17 /* Increment the flag to let the boot CPU we're ready */ -1: ldxr w18, [flag_ptr] - add w18, w18, #1 - stxr w17, w18, [flag_ptr] +1: ldxr w16, [flag_ptr] + add w16, w16, #1 + stxr w17, w16, [flag_ptr] cbnz w17, 1b /* Wait for the boot CPU to finish messing around with swapper */ sevl 1: wfe - ldxr w18, [flag_ptr] - cbnz w18, 1b + ldxr w16, [flag_ptr] + cbnz w16, 1b /* All done, act like nothing happened */ - offset_ttbr1 swapper_ttb, x18 + offset_ttbr1 swapper_ttb, x16 msr ttbr1_el1, swapper_ttb isb ret - .unreq cpu - .unreq num_cpus - .unreq swapper_pa .unreq swapper_ttb .unreq flag_ptr - .unreq cur_pgdp - .unreq end_pgdp - .unreq pgd - .unreq cur_pudp - .unreq end_pudp - .unreq pud - .unreq cur_pmdp - .unreq end_pmdp - .unreq pmd - .unreq cur_ptep - .unreq end_ptep - .unreq pte ENDPROC(idmap_kpti_install_ng_mappings) .popsection #endif From patchwork Fri Oct 18 16:10:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199029 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 348E617EE for ; Fri, 18 Oct 2019 16:11:46 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 03904222C2 for ; Fri, 18 Oct 2019 16:11:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="BkBEcMbh"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="i/jDfxdk" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 03904222C2 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=FyUeX+zmZNBEZRdsACaDphKQQDmq2u9NoZS04ck3a0s=; b=BkBEcMbh2beThj deavCqMgzMEqGslF/Cm872vdbUnynaONDZ8U+sYTJCezdm0E9EHQHV5yrp6W/iktSOzHDzIzRzNYQ XwoXGuDezO3/UxfyLqCVURvtQbxFLfxlHtg7vuTREEw8ajQJq9NW513hzNwrXG9Iew6tR8QGqDDRG 4lloew6YHwOq7CrRYU5UtIy/44ZLbPPOWK+Sr5m0v5lo20NZZLEXDmJ9Z72n4C6IDLQDUmUL5kdAR RktAMpPMqXV/3AXyelYPlI6p3BfIfJtkXGsC/1uBD+qW4S3uav6RhvsLnYMnMz8MUBAy74yEkrNrK rHO3f7UUhIQ6PqU/QVSQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUqu-0002Ih-It; Fri, 18 Oct 2019 16:11:44 +0000 Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUq5-0001Uc-3x for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:10:54 +0000 Received: by mail-pl1-x649.google.com with SMTP id g7so4044024plo.5 for ; Fri, 18 Oct 2019 09:10:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=jUS0dhIzgmFNCiOAuEUXvcuYNkuGL56ryAS+MUv7C/I=; b=i/jDfxdkcHMKcb5mbJursCxkiE0tLbcAT50ryu4atkNnzE9tZW2TQb4DunSXt4JOkt hyqhR6jkusYNpX0FQ9xmE9MPM+OkoO9XULEt54Q1eChfRkMXo6zH0LLx5BfDV9XxTViq ZY4f/W9+YSMQLmVbr5Hvv7U5xrLsx6ePGTXBQJ2GfNYSLJa/kqRf7pp4MN5xmr1Pm9vV U0cyLViFkAWt3zk7EDLd8J6L/oTDFeTktzFlIC0dgmBDCYk0vOIQhRFpRS9l6HZ8lGvH d40Qs4Z3TDODrADDi1vpWtmXBnOPWrst+YFdZPhbhVcTVQ3PCuWW86qVwWdHqVcy9DgV E2Sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=jUS0dhIzgmFNCiOAuEUXvcuYNkuGL56ryAS+MUv7C/I=; b=E2bFxqUBBnuSwNsdpE8eZVCFVyaCnwXaqYMogv8YRsNKxHNt0ZIbn20ygj302Iyb43 4rs55Qw52sbcx1n3uzla1jj92RAhAahWeNlR+sX3nqQgVp7zfbP4odNf2Y2VfoyWRDHP H6vrTcrojXV/GRViwFbAG0ICNFhEyBEqPUjYf5wH+R3ehHefc6qIPbBW9uJpHNLX3pAw EqQjt4dr+WfxvyriyHAMFXm97u8cGPDXL3rklSzvMyY7qCY2DhKyhiHQ/5u/TXwhZ/tc 4C3UE+3TNyc9RyyzjkGqcpcPlMSgbrRKsWEntrG68ne1jfViaiKu9JkXGzUh1f4AkFWI o5Ew== X-Gm-Message-State: APjAAAUxu7lKUwwtOE0FuB/R2xJm/GYDxb+qgGgnYra0MhUZeuXc/iyq m1N7Bw/xVm4oKhofb+pMw3FHKD4ijvQkHnTpmBg= X-Google-Smtp-Source: APXvYqz/HNu+DPymnhir5Vk43GMqjgbnDJ4wWnLMl8QOvY1mBOlys6EjB+4bJm7Ch3Zb+wSpV6DGjPmmablFROD07lw= X-Received: by 2002:a63:3201:: with SMTP id y1mr10741272pgy.174.1571415050913; Fri, 18 Oct 2019 09:10:50 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:17 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-3-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 02/18] arm64/lib: copy_page: avoid x18 register in assembler code From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_091053_206796_76378040 X-CRM114-Status: UNSURE ( 8.98 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:649 listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org From: Ard Biesheuvel Register x18 will no longer be used as a caller save register in the future, so stop using it in the copy_page() code. Link: https://patchwork.kernel.org/patch/9836869/ Signed-off-by: Ard Biesheuvel Signed-off-by: Sami Tolvanen --- arch/arm64/lib/copy_page.S | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/arch/arm64/lib/copy_page.S b/arch/arm64/lib/copy_page.S index bbb8562396af..8b562264c165 100644 --- a/arch/arm64/lib/copy_page.S +++ b/arch/arm64/lib/copy_page.S @@ -34,45 +34,45 @@ alternative_else_nop_endif ldp x14, x15, [x1, #96] ldp x16, x17, [x1, #112] - mov x18, #(PAGE_SIZE - 128) + add x0, x0, #256 add x1, x1, #128 1: - subs x18, x18, #128 + tst x0, #(PAGE_SIZE - 1) alternative_if ARM64_HAS_NO_HW_PREFETCH prfm pldl1strm, [x1, #384] alternative_else_nop_endif - stnp x2, x3, [x0] + stnp x2, x3, [x0, #-256] ldp x2, x3, [x1] - stnp x4, x5, [x0, #16] + stnp x4, x5, [x0, #-240] ldp x4, x5, [x1, #16] - stnp x6, x7, [x0, #32] + stnp x6, x7, [x0, #-224] ldp x6, x7, [x1, #32] - stnp x8, x9, [x0, #48] + stnp x8, x9, [x0, #-208] ldp x8, x9, [x1, #48] - stnp x10, x11, [x0, #64] + stnp x10, x11, [x0, #-192] ldp x10, x11, [x1, #64] - stnp x12, x13, [x0, #80] + stnp x12, x13, [x0, #-176] ldp x12, x13, [x1, #80] - stnp x14, x15, [x0, #96] + stnp x14, x15, [x0, #-160] ldp x14, x15, [x1, #96] - stnp x16, x17, [x0, #112] + stnp x16, x17, [x0, #-144] ldp x16, x17, [x1, #112] add x0, x0, #128 add x1, x1, #128 - b.gt 1b + b.ne 1b - stnp x2, x3, [x0] - stnp x4, x5, [x0, #16] - stnp x6, x7, [x0, #32] - stnp x8, x9, [x0, #48] - stnp x10, x11, [x0, #64] - stnp x12, x13, [x0, #80] - stnp x14, x15, [x0, #96] - stnp x16, x17, [x0, #112] + stnp x2, x3, [x0, #-256] + stnp x4, x5, [x0, #-240] + stnp x6, x7, [x0, #-224] + stnp x8, x9, [x0, #-208] + stnp x10, x11, [x0, #-192] + stnp x12, x13, [x0, #-176] + stnp x14, x15, [x0, #-160] + stnp x16, x17, [x0, #-144] ret ENDPROC(copy_page) From patchwork Fri Oct 18 16:10:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199031 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 981991951 for ; Fri, 18 Oct 2019 16:12:17 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6D67E222BD for ; Fri, 18 Oct 2019 16:12:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="ZVZYxW+0"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="ZvzQdFSz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6D67E222BD Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=7IMJt5X46oYRI/DJCzkyO+xEdPUlvNHd8v3xADNBqtE=; b=ZVZYxW+0zOUbLU Yw9fOTcZgMbeIbMxbEpTbanzuwwCV+/aFnEHDs1JyHnF3SHXTRZZU4sZTkJ/n4gySZfxZPzaW+LMp TOfw962/gYJz4jYFurF8PVKhK5dwLH5LPwRvDj4Ap1lvwZtN3EnrGBm04K/l7e7GsZsaGL6fbgsAb +gZ3dktFG2SM2r5OL0SKXHCPGKFk2bl2oMN1a1DDzTfktqR873Y94FMzLFVS+rpO/qaxQWeSKvWMY fiHJzwViAv9OxvztCfjQoUbWAniVT4u8zrKieOvyveB0sbeEaCrHEqjZ6QOzGba7Q0Nt6IKRP1UDG iKEFFC/67DoBotmb7XhQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUrH-0002bn-LK; Fri, 18 Oct 2019 16:12:07 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUq9-0001Y1-LO for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:10:59 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id m18so4791047ybf.20 for ; Fri, 18 Oct 2019 09:10:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=72rGRuWFiviz942NA6zVkn9gcWlAdBMK2y71uV80e5M=; b=ZvzQdFSzTLbEjP4zxdusVf3ReL3ljnpZzJfPdrbpnE9rMeefuhQHID/ee1FAoqvYLH O+/PdDLy+eoWqhA9UIEIUFRCMC5xqhNowPZWx2ww/HoS+63BhQ5I1HOf+y5t9GV4AlzB 4wvvMPqaOkuUSsDFHTVS6xJHhNUp3s1c9Y1ghonvzQD2ZPiXv5LmNp53Biqgoh9GKiCJ afEukw6/WMLv2boh6kNWUds4TgOnVVcCplvVwI02NOYpPnoOVZxpUv2JWmCC5FcJjdP9 qyuCgJgGG2jATFBKXIAZK/fXaf2KYXXK+X2rgvP6lspQZZSv0gH/6LOEhAEKe42LtQC+ cKeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=72rGRuWFiviz942NA6zVkn9gcWlAdBMK2y71uV80e5M=; b=Dm/Lan5oCY3KLzYQw/x9mq6a0I6sCXogFKBCvFY0rhVAKyz2nEGyl3xuS6/lNVEzk3 Xhdn08IrI4lYQBHC2/A1rG1dN4iJlHPo7bXs5oEKnXlWtNluBOt3Lsm1IzYwvTzggnsV 8Ve9q3bEVBS07fL4bTc5BTBgC7Pn4gw7FoCD6kw7aPTSuQtII+XbPJCoI+RGvOoBpBCh 6gIa3+JgukQKOi+Cqeqmt5QrBSpFJz2VhjtMG33EHFOrekSFz8NhFl1Y+Iqeb09p1Mx7 47GCZA+ZZML7ywzCR9tq8NqRzGFcFpM625E8ljJA6fw3H1oqTPx5cx+0Si0exaTgOrZQ 4K8g== X-Gm-Message-State: APjAAAUwlCCzaqCmnwUplxb90UBEQBzQrSC39PUDiHAjE7f7sLettPTz ZvYur8VOCIG15bKpHweDrKAgN1lPYOdFeQRA7Ao= X-Google-Smtp-Source: APXvYqxINkQeafZ9JYtVaWg+qUXlNkxSB5ctm2wREg/j/aMT/J4ez6ebkhDJlYmKEc3YyNUDF8Zxh7j2jKVxFaFkVRU= X-Received: by 2002:a0d:d804:: with SMTP id a4mr7899178ywe.454.1571415054615; Fri, 18 Oct 2019 09:10:54 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:18 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-4-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 03/18] arm64: kvm: stop treating register x18 as caller save From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_091057_766835_AA348EA5 X-CRM114-Status: GOOD ( 10.36 ) X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:b4a listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org From: Ard Biesheuvel In preparation of using x18 as a task struct pointer register when running in the kernel, stop treating it as caller save in the KVM guest entry/exit code. Currently, the code assumes there is no need to preserve it for the host, given that it would have been assumed clobbered anyway by the function call to __guest_enter(). Instead, preserve its value and restore it upon return. Link: https://patchwork.kernel.org/patch/9836891/ Signed-off-by: Ard Biesheuvel Signed-off-by: Sami Tolvanen --- arch/arm64/kvm/hyp/entry.S | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index e5cc8d66bf53..20bd9a20ea27 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -23,6 +23,7 @@ .pushsection .hyp.text, "ax" .macro save_callee_saved_regs ctxt + str x18, [\ctxt, #CPU_XREG_OFFSET(18)] stp x19, x20, [\ctxt, #CPU_XREG_OFFSET(19)] stp x21, x22, [\ctxt, #CPU_XREG_OFFSET(21)] stp x23, x24, [\ctxt, #CPU_XREG_OFFSET(23)] @@ -38,6 +39,7 @@ ldp x25, x26, [\ctxt, #CPU_XREG_OFFSET(25)] ldp x27, x28, [\ctxt, #CPU_XREG_OFFSET(27)] ldp x29, lr, [\ctxt, #CPU_XREG_OFFSET(29)] + ldr x18, [\ctxt, #CPU_XREG_OFFSET(18)] .endm /* @@ -87,12 +89,9 @@ alternative_else_nop_endif ldp x14, x15, [x18, #CPU_XREG_OFFSET(14)] ldp x16, x17, [x18, #CPU_XREG_OFFSET(16)] - // Restore guest regs x19-x29, lr + // Restore guest regs x18-x29, lr restore_callee_saved_regs x18 - // Restore guest reg x18 - ldr x18, [x18, #CPU_XREG_OFFSET(18)] - // Do not touch any register after this! eret sb @@ -114,7 +113,7 @@ ENTRY(__guest_exit) // Retrieve the guest regs x0-x1 from the stack ldp x2, x3, [sp], #16 // x0, x1 - // Store the guest regs x0-x1 and x4-x18 + // Store the guest regs x0-x1 and x4-x17 stp x2, x3, [x1, #CPU_XREG_OFFSET(0)] stp x4, x5, [x1, #CPU_XREG_OFFSET(4)] stp x6, x7, [x1, #CPU_XREG_OFFSET(6)] @@ -123,9 +122,8 @@ ENTRY(__guest_exit) stp x12, x13, [x1, #CPU_XREG_OFFSET(12)] stp x14, x15, [x1, #CPU_XREG_OFFSET(14)] stp x16, x17, [x1, #CPU_XREG_OFFSET(16)] - str x18, [x1, #CPU_XREG_OFFSET(18)] - // Store the guest regs x19-x29, lr + // Store the guest regs x18-x29, lr save_callee_saved_regs x1 get_host_ctxt x2, x3 From patchwork Fri Oct 18 16:10:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199033 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5BFE417EE for ; Fri, 18 Oct 2019 16:12:36 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3940E21D7C for ; Fri, 18 Oct 2019 16:12:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="MwhcinvN"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="TZOVdcpB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3940E21D7C Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=eidiNV5JQXMGqkjWsKDBj+n7kFwkXj9f+4G5UQmtG8c=; b=MwhcinvNBh/K8U i4JlOm8jimlK2k6Y0P3DFTDvr7QlMfyCOx5Zw3GWPcTMm8nFg9c9NJODRK0TuNWueoUc5/lZlWXz6 F/ax5ni4yH+EbFDwcrUX1Wi8JxNUrjdaufy2POqVd7L9M+3FfEJ6eNLoJ8Jhc6P4DecXmy9dolWgI CqbK3A5ZTuQmd50pU5v7nsNOreDSFIDhlLTkniWC4BQRRrJk4UeJhSHXMUPtKJsdmyn7NE/we/fsz z490y7hKfQ1i6iBnT4/arVmQ2CfXa6lQ84WWH0116jN8Pg+JNjGoGcZwLck3KoJBm89ZlvL+1c9eo fdYBGicicF3dNBFy9jmA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUrd-0002tT-M9; Fri, 18 Oct 2019 16:12:29 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUqD-0001cU-Ti for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:11:03 +0000 Received: by mail-pf1-x44a.google.com with SMTP id s139so4953202pfc.21 for ; Fri, 18 Oct 2019 09:10:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=SzRVhVCQSEs4UpxvfljRvYSxQu5VFVx82TZExeQC7I8=; b=TZOVdcpBJiow5T4VOPKVfmYmovhUmd2ZBG9xflMRKW8S5BQ2L7yu5g6JY0/TCCHH+I XufZXerCCZfchS8x9jnvOoLMze/QWKGKm/0eGk215XJdEA1kME5jv8jnjb92Pb1eVWkw yiTgUjP8Rao1lkPZ3wsgn20YfPu5vdw09AcOGQOKuwSW35OqEXFlPzTQhP7pZvaCNMAW MWtaplAUSr+50SFN34KRGzFF9OlQtJcKYXSWZanQMwKs4H+DZ9A3MX4sVyK4cRYJFJ+w rlR21WxyyCXlD1Tc/Mn6bTnrGGsiH8nauEVx60lyfrl/C39gDgUhzHfYMSQ6uZ9ljLwO QH/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=SzRVhVCQSEs4UpxvfljRvYSxQu5VFVx82TZExeQC7I8=; b=jLIXIk2dMk0I4YsFRDyr8hGpWDToqdxrZ/LcyEmhB14Kl5n0oyR+ktgOnYVfyD19JL 6UgMGnKuzwv3dPsXZweLXUKqqEJtlZGalBCwQjond7VNqVe3MnYkyE9WE/ney8gLt5QE dMa3QAQ/+hzhuk7QQBZK6OKogwJR1TjfNWVxO3Zq+JRsDgKcBE+cbbMz+vcD7XoZyF6I IA/aWe+h/lLtSHaDyLOjxT0wstPF+QTguz4SgV8Ak657m8aF4ID7thPaSjRTLUzKY8B5 bzttyuhX5Ra+9CFYRtdU6USmOzI6QCIw2675BDf9Wc7ZMqdtzFuX20S8w7juxrgfRy1Z DpnQ== X-Gm-Message-State: APjAAAVlaOQptXKBKWU6VVG025IfLUXohD073WjAv8lfv1ce6rcNHkYh l3UB39Ppu9lRMMaquQzSqlzx6F7O0in7mP/kZQM= X-Google-Smtp-Source: APXvYqymenYk+Nj4KHMBe+/txGcAk8emmf58ZAJAVtmISgexrXgZK+srN7qpXD3xJ732mL5rywUz419mv8CrbkTon24= X-Received: by 2002:a65:68c2:: with SMTP id k2mr10843389pgt.241.1571415058696; Fri, 18 Oct 2019 09:10:58 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:19 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-5-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 04/18] arm64: kernel: avoid x18 as an arbitrary temp register From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_091102_036067_C7C6684D X-CRM114-Status: UNSURE ( 8.64 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:44a listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org From: Ard Biesheuvel The code in __cpu_soft_restart() uses x18 as an arbitrary temp register, which will shortly be disallowed. So use x8 instead. Link: https://patchwork.kernel.org/patch/9836877/ Signed-off-by: Ard Biesheuvel Signed-off-by: Sami Tolvanen --- arch/arm64/kernel/cpu-reset.S | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/cpu-reset.S b/arch/arm64/kernel/cpu-reset.S index 6ea337d464c4..32c7bf858dd9 100644 --- a/arch/arm64/kernel/cpu-reset.S +++ b/arch/arm64/kernel/cpu-reset.S @@ -42,11 +42,11 @@ ENTRY(__cpu_soft_restart) mov x0, #HVC_SOFT_RESTART hvc #0 // no return -1: mov x18, x1 // entry +1: mov x8, x1 // entry mov x0, x2 // arg0 mov x1, x3 // arg1 mov x2, x4 // arg2 - br x18 + br x8 ENDPROC(__cpu_soft_restart) .popsection From patchwork Fri Oct 18 16:10:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199037 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6792917EE for ; Fri, 18 Oct 2019 16:12:47 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3FE8B21D7C for ; Fri, 18 Oct 2019 16:12:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="YdTmsmg8"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="RTWmB+YR" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3FE8B21D7C Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wtjMzzc59R2wxZq223KzbMKzGgoRoQAumIN1bbUiSTQ=; b=YdTmsmg8ha9tQ3 cq3Q2YIEy2Cl24iUKENCNjJJznDgl3KKAOMWwsyMHyAZVgsytCByxEKGq6Bzkp6hKpA0jfpkBYvsp iFIDHwXZFdv/SIXXKU5xXFSB4yyfG0O/rWr/U6ncJqEI0P2ABOdrWjxiES4lknxxu+jYsdN/ZqOig nQmoHQ70ssl27ehR6LqehPIs25qDrLX2ooPBlXq/2M5Zvuzljb6pJuCihaaV53nJL2493wiiMr46C Mc+HyCM72swc4lJ6blwAf0XRFpXuohzLvRDZJGV+Cav1V4nKiSAonc9Ik1VkezWoNgn4EIza3tISY eNEbNz9H/Jr+wpmactzA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUru-0003H2-CU; Fri, 18 Oct 2019 16:12:46 +0000 Received: from mail-vs1-xe49.google.com ([2607:f8b0:4864:20::e49]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUqI-0001fj-5j for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:11:08 +0000 Received: by mail-vs1-xe49.google.com with SMTP id j2so1571140vso.0 for ; Fri, 18 Oct 2019 09:11:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=soVi6argGh8CkyJ/tZS/XQoN8aAv6F/7ncnrrIAGrbI=; b=RTWmB+YRhbyTrupTegTePDqh83wahNiLo24V/vkd3XNh2/VjRfOzsP8l9UA07iBYVh cNCOrmz1KApedjuZfTuf+oEAy7uJ9vOjlPpxsg7OkJReg5LlWaNnJSadKKQxT6uV8oo9 DYHGRvWBz3/GUdtlA0wIAJuRmYFSW5vXFhnZ39LEFxcVI9DqpO8eIN9kDWvp9NjS8BGy xN5pkbK5dusYfEE++LF7k5cNY+Cu1x/v4ps1Ft2zcALY7q2NKOD2KsdADykKFGjX/ek9 pgnN2RfC3QCLk3Iaa8HyaKxHiBf6citZAP2zBGFrT6GXMyCWAMsRsDGnPTlW+RbabIhg vExA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=soVi6argGh8CkyJ/tZS/XQoN8aAv6F/7ncnrrIAGrbI=; b=jqNSF7wRFfPPu3zvLfXFGNaozmB+0zyVeCoioNg+mn7OINoTLXNqmdNFofxJCc5tnY dGdN6ykW0tnY3Ix7g1fPwwYy+2kdndr0d1v1awC533obdyPm2U84VUmaSn104mLf/mFn avLFsA2Xzn1457Hul1jr+o057FfeqosZfEdmGLnDLSzTGy5pMe7ygJdKP4x4VqSVXwx8 pyBK64lNSvLbOnIZ5ilyCe4oefdt2H04ujSIjNYHoUlTK26cu40lzbCXKQp85TdOJ06v /r4UxchobC9HXkwZ2IVsVDzuryJWOYGhoYx9cqLi8e+ddrxQCLf9RZiCgPp72XLGMuoH sdvQ== X-Gm-Message-State: APjAAAUZUWvBTKfZ9I+lnq5DE9O17mlnjN0regE/GTIr70CYWK3vVA42 sIzrggA4PH/+kVVzVSksWnw//YkSVeYDGPrwBoI= X-Google-Smtp-Source: APXvYqzkAvflPUA4hxOT5c7IqsE0Ybkf3fYa0ytmTSeu2hOJz51kZPeU6sdMsbUTLU2Tls8gLg9QkcQsftcMtRZzrhY= X-Received: by 2002:a67:e34b:: with SMTP id s11mr5965401vsm.195.1571415062790; Fri, 18 Oct 2019 09:11:02 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:20 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-6-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 05/18] arm64: kbuild: reserve reg x18 from general allocation by the compiler From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_091106_339791_9026F713 X-CRM114-Status: UNSURE ( 9.99 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:e49 listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org From: Ard Biesheuvel Before we can start using register x18 for a special purpose (as permitted by the AAPCS64 ABI), we need to tell the compiler that it is off limits for general allocation. So tag it as 'fixed', and remove the mention from the LL/SC compiler flag override. Link: https://patchwork.kernel.org/patch/9836881/ Signed-off-by: Ard Biesheuvel Signed-off-by: Sami Tolvanen Reviewed-by: Nick Desaulniers --- arch/arm64/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 2c0238ce0551..1c7b276bc7c5 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -55,7 +55,7 @@ endif KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) \ $(compat_vdso) $(cc_has_k_constraint) -KBUILD_CFLAGS += -fno-asynchronous-unwind-tables +KBUILD_CFLAGS += -fno-asynchronous-unwind-tables -ffixed-x18 KBUILD_CFLAGS += $(call cc-disable-warning, psabi) KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso) From patchwork Fri Oct 18 16:10:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199041 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2E7511951 for ; Fri, 18 Oct 2019 16:13:01 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EF18A21D7C for ; Fri, 18 Oct 2019 16:13:00 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="ZDx+iTcM"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="V/G2C0eH" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EF18A21D7C Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=HocY0Gno6rpATTkNjMOw2BVNFy2xrTurE64aaKllcqs=; b=ZDx+iTcMJSdhEg I+WlZ6TdsfLDNmtcRRkp3dSGJJq4hlWk5zK/3tmKLO6vEffeOysTWwFuEvKsEP2CWytRSrXNHJc89 8RaTaPELPnnlT/58qncrBMocKP29ruCprJkvVYzC42nPDNCsh0JHuRMvVnlfkVl3TazWv8WkhA8+a ylDwXGj5VxS7iezNGQM79S2+kAh7QZNluT041XWhsE+D+zpM2LqM4SWnISvSSBA2w2GZ534CavYT+ 8J+2YZSLje23oz5hJrJ0njNntH3wkDn/AYMN29MbmID3DPGkGNbZ6XmkHhrzH4r0RVYe3VWOYbTZb Y1bbPG2+4f7T7ByrjMew==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUs8-0003VL-42; Fri, 18 Oct 2019 16:13:00 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUqK-0001iK-BE for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:11:11 +0000 Received: by mail-pg1-x54a.google.com with SMTP id w13so4578138pge.15 for ; Fri, 18 Oct 2019 09:11:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=nb/Sj1uUqCudVY1GzIbe2UX9PioIzl4YsjEJIj4hCD8=; b=V/G2C0eHJuPFOwRh3Dly/jwzcNCfbqXZV4MkzwN2SNOr80mirycaX1gRYAdHm2nLaX L3s1Hdym5MJqVVC7EbxIONG5E9OrsLDGm/g9rxl9q6VkEYbJizPFWhrE2SoYG1EEa5bd kmq3AFfoHkK/lcfP+YMHK2P6rD4c7/BqU34GihwEPvGXsvWycorPC3nB1T5iQJLYZ0wp HthLQe47pJySnkhLNgFPMCMXC5bz7pk/D1ENZQ/6v4J+34an+emFSzLC4n5CC8ZphyVn M8eWlCdd8FlM2IxWOnFsNm/Gruld+HPi/gc5fIJl8YF1FQBw3r+c3wE58gfz3/IRXdN9 WEUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=nb/Sj1uUqCudVY1GzIbe2UX9PioIzl4YsjEJIj4hCD8=; b=I0pHyxMAQyyBQOQ5+/Ynh9dC6vNzBU0+95srnI3w6Ot90suW3zeHa/TKgUg6LJs7Ub 2KnaVvJINSGZqfcg1l6l+PyPcL6NerUt6T6vyHTpPZZTS2dICffpLDjDtawJVeFryjqE OWZJHL3nvJKdLG805DVV2yBIhIMj4sjJ2INee0hMfbCx2lHT+XGMgWJkoF4as3Is9bjI m1bv4UXOAED0vXa1KBFe78FqsPu5JDBtT7CwrtYIs9m3SSWC7aOWi/b3KUsVHdYx+9yq dH7T69dLgryiZ4m8i/VV8HFQLQAqxx2u6JiP5dmvqXtPLiI240dKe3fxp2CWu3+EuebF 1Iqw== X-Gm-Message-State: APjAAAVBtj9T1jCvxGX2xbyvSjm2WxhIdKSxBV1RaQmtLg4cbPmzo0Sd P9K4esWPyHe49xE5Z7+FAcZWA1kT1ujbiTSb430= X-Google-Smtp-Source: APXvYqw0i1ys5Qza3Rohx9re+KGxZAQHbujZiTFqNgG2/0VGDxLHssWCSip4KyplPyALaC1/Qrg6JgVmOYQ2+WCmJ30= X-Received: by 2002:a63:ce07:: with SMTP id y7mr10981743pgf.234.1571415065039; Fri, 18 Oct 2019 09:11:05 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:21 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-7-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 06/18] add support for Clang's Shadow Call Stack (SCS) From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_091108_473670_D6498881 X-CRM114-Status: GOOD ( 21.07 ) X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:54a listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org This change adds generic support for Clang's Shadow Call Stack, which uses a shadow stack to protect return addresses from being overwritten by an attacker. Details are available here: https://clang.llvm.org/docs/ShadowCallStack.html Signed-off-by: Sami Tolvanen --- Makefile | 6 ++ arch/Kconfig | 39 ++++++++ include/linux/compiler-clang.h | 2 + include/linux/compiler_types.h | 4 + include/linux/scs.h | 88 ++++++++++++++++++ init/init_task.c | 6 ++ init/main.c | 3 + kernel/Makefile | 1 + kernel/fork.c | 9 ++ kernel/sched/core.c | 2 + kernel/sched/sched.h | 1 + kernel/scs.c | 162 +++++++++++++++++++++++++++++++++ 12 files changed, 323 insertions(+) create mode 100644 include/linux/scs.h create mode 100644 kernel/scs.c diff --git a/Makefile b/Makefile index ffd7a912fc46..e401fa500f62 100644 --- a/Makefile +++ b/Makefile @@ -846,6 +846,12 @@ ifdef CONFIG_LIVEPATCH KBUILD_CFLAGS += $(call cc-option, -flive-patching=inline-clone) endif +ifdef CONFIG_SHADOW_CALL_STACK +KBUILD_CFLAGS += -fsanitize=shadow-call-stack +DISABLE_SCS := -fno-sanitize=shadow-call-stack +export DISABLE_SCS +endif + # arch Makefile may override CC so keep this after arch Makefile is included NOSTDINC_FLAGS += -nostdinc -isystem $(shell $(CC) -print-file-name=include) diff --git a/arch/Kconfig b/arch/Kconfig index 5f8a5d84dbbe..a222adda8130 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -521,6 +521,45 @@ config STACKPROTECTOR_STRONG about 20% of all kernel functions, which increases the kernel code size by about 2%. +config ARCH_SUPPORTS_SHADOW_CALL_STACK + bool + help + An architecture should select this if it supports Clang's Shadow + Call Stack, has asm/scs.h, and implements runtime support for shadow + stack switching. + +config SHADOW_CALL_STACK_VMAP + def_bool n + depends on SHADOW_CALL_STACK + help + Use virtually mapped shadow call stacks. Selecting this option + provides better stack exhaustion protection, but increases per-thread + memory consumption as a full page is allocated for each shadow stack. + +choice + prompt "Return-oriented programming (ROP) protection" + default ROP_PROTECTION_NONE + help + This option controls kernel protections against return-oriented + programming (ROP) attacks. + +config ROP_PROTECTION_NONE + bool "None" + +config SHADOW_CALL_STACK + bool "Clang Shadow Call Stack" + depends on ARCH_SUPPORTS_SHADOW_CALL_STACK + depends on CC_IS_CLANG && CLANG_VERSION >= 70000 + help + This option enables Clang's Shadow Call Stack, which uses a shadow + stack to protect function return addresses from being overwritten by + an attacker. More information can be found from Clang's + documentation: + + https://clang.llvm.org/docs/ShadowCallStack.html + +endchoice + config HAVE_ARCH_WITHIN_STACK_FRAMES bool help diff --git a/include/linux/compiler-clang.h b/include/linux/compiler-clang.h index 333a6695a918..9af08391f205 100644 --- a/include/linux/compiler-clang.h +++ b/include/linux/compiler-clang.h @@ -42,3 +42,5 @@ * compilers, like ICC. */ #define barrier() __asm__ __volatile__("" : : : "memory") + +#define __noscs __attribute__((no_sanitize("shadow-call-stack"))) diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h index 72393a8c1a6c..be5d5be4b1ae 100644 --- a/include/linux/compiler_types.h +++ b/include/linux/compiler_types.h @@ -202,6 +202,10 @@ struct ftrace_likely_data { # define randomized_struct_fields_end #endif +#ifndef __noscs +# define __noscs +#endif + #ifndef asm_volatile_goto #define asm_volatile_goto(x...) asm goto(x) #endif diff --git a/include/linux/scs.h b/include/linux/scs.h new file mode 100644 index 000000000000..dfbd80faa528 --- /dev/null +++ b/include/linux/scs.h @@ -0,0 +1,88 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Shadow Call Stack support. + * + * Copyright (C) 2018 Google LLC + */ + +#ifndef _LINUX_SCS_H +#define _LINUX_SCS_H + +#include +#include +#include + +#ifdef CONFIG_SHADOW_CALL_STACK + +#ifdef CONFIG_SHADOW_CALL_STACK_VMAP +# define SCS_SIZE PAGE_SIZE +#else +# define SCS_SIZE 1024 +#endif + +#define SCS_GFP (GFP_KERNEL | __GFP_ZERO) + +extern unsigned long init_shadow_call_stack[]; + +static inline void *task_scs(struct task_struct *tsk) +{ + return task_thread_info(tsk)->shadow_call_stack; +} + +static inline void task_set_scs(struct task_struct *tsk, void *s) +{ + task_thread_info(tsk)->shadow_call_stack = s; +} + +extern void scs_init(void); +extern void scs_set_init_magic(struct task_struct *tsk); +extern void scs_task_init(struct task_struct *tsk); +extern void scs_task_reset(struct task_struct *tsk); +extern int scs_prepare(struct task_struct *tsk, int node); +extern bool scs_corrupted(struct task_struct *tsk); +extern void scs_release(struct task_struct *tsk); + +#else /* CONFIG_SHADOW_CALL_STACK */ + +static inline void *task_scs(struct task_struct *tsk) +{ + return 0; +} + +static inline void task_set_scs(struct task_struct *tsk, void *s) +{ +} + +static inline void scs_init(void) +{ +} + +static inline void scs_set_init_magic(struct task_struct *tsk) +{ +} + +static inline void scs_task_init(struct task_struct *tsk) +{ +} + +static inline void scs_task_reset(struct task_struct *tsk) +{ +} + +static inline int scs_prepare(struct task_struct *tsk, int node) +{ + return 0; +} + +static inline bool scs_corrupted(struct task_struct *tsk) +{ + return false; +} + +static inline void scs_release(struct task_struct *tsk) +{ +} + +#endif /* CONFIG_SHADOW_CALL_STACK */ + +#endif /* _LINUX_SCS_H */ diff --git a/init/init_task.c b/init/init_task.c index 9e5cbe5eab7b..5e55ff45bbbf 100644 --- a/init/init_task.c +++ b/init/init_task.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include @@ -184,6 +185,11 @@ struct task_struct init_task }; EXPORT_SYMBOL(init_task); +#ifdef CONFIG_SHADOW_CALL_STACK +unsigned long init_shadow_call_stack[SCS_SIZE / sizeof(long)] + __init_task_data __aligned(SCS_SIZE); +#endif + /* * Initial thread structure. Alignment of this is handled by a special * linker map entry. diff --git a/init/main.c b/init/main.c index 91f6ebb30ef0..fb8bcdd729b9 100644 --- a/init/main.c +++ b/init/main.c @@ -93,6 +93,7 @@ #include #include #include +#include #include #include @@ -578,6 +579,8 @@ asmlinkage __visible void __init start_kernel(void) char *after_dashes; set_task_stack_end_magic(&init_task); + scs_set_init_magic(&init_task); + smp_setup_processor_id(); debug_objects_early_init(); diff --git a/kernel/Makefile b/kernel/Makefile index daad787fb795..313dbd44d576 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -102,6 +102,7 @@ obj-$(CONFIG_TRACEPOINTS) += trace/ obj-$(CONFIG_IRQ_WORK) += irq_work.o obj-$(CONFIG_CPU_PM) += cpu_pm.o obj-$(CONFIG_BPF) += bpf/ +obj-$(CONFIG_SHADOW_CALL_STACK) += scs.o obj-$(CONFIG_PERF_EVENTS) += events/ diff --git a/kernel/fork.c b/kernel/fork.c index bcdf53125210..ae7ebe9f0586 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -94,6 +94,7 @@ #include #include #include +#include #include #include @@ -451,6 +452,8 @@ void put_task_stack(struct task_struct *tsk) void free_task(struct task_struct *tsk) { + scs_release(tsk); + #ifndef CONFIG_THREAD_INFO_IN_TASK /* * The task is finally done with both the stack and thread_info, @@ -834,6 +837,8 @@ void __init fork_init(void) NULL, free_vm_stack_cache); #endif + scs_init(); + lockdep_init_task(&init_task); uprobes_init(); } @@ -907,6 +912,7 @@ static struct task_struct *dup_task_struct(struct task_struct *orig, int node) clear_user_return_notifier(tsk); clear_tsk_need_resched(tsk); set_task_stack_end_magic(tsk); + scs_task_init(tsk); #ifdef CONFIG_STACKPROTECTOR tsk->stack_canary = get_random_canary(); @@ -2022,6 +2028,9 @@ static __latent_entropy struct task_struct *copy_process( args->tls); if (retval) goto bad_fork_cleanup_io; + retval = scs_prepare(p, node); + if (retval) + goto bad_fork_cleanup_thread; stackleak_task_init(p); diff --git a/kernel/sched/core.c b/kernel/sched/core.c index dd05a378631a..e7faeb383008 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -6013,6 +6013,8 @@ void init_idle(struct task_struct *idle, int cpu) raw_spin_lock_irqsave(&idle->pi_lock, flags); raw_spin_lock(&rq->lock); + scs_task_reset(idle); + __sched_fork(0, idle); idle->state = TASK_RUNNING; idle->se.exec_start = sched_clock(); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 0db2c1b3361e..c153003a011c 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -58,6 +58,7 @@ #include #include #include +#include #include #include #include diff --git a/kernel/scs.c b/kernel/scs.c new file mode 100644 index 000000000000..47324e8d313b --- /dev/null +++ b/kernel/scs.c @@ -0,0 +1,162 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Shadow Call Stack support. + * + * Copyright (C) 2019 Google LLC + */ + +#include +#include +#include +#include +#include +#include + +#define SCS_END_MAGIC 0xaf0194819b1635f6UL + +static inline void *__scs_base(struct task_struct *tsk) +{ + return (void *)((uintptr_t)task_scs(tsk) & ~(SCS_SIZE - 1)); +} + +#ifdef CONFIG_SHADOW_CALL_STACK_VMAP + +/* Keep a cache of shadow stacks */ +#define SCS_CACHE_SIZE 2 +static DEFINE_PER_CPU(void *, scs_cache[SCS_CACHE_SIZE]); + +static void *scs_alloc(int node) +{ + int i; + + for (i = 0; i < SCS_CACHE_SIZE; i++) { + void *s; + + s = this_cpu_xchg(scs_cache[i], NULL); + if (s) { + memset(s, 0, SCS_SIZE); + return s; + } + } + + return __vmalloc_node_range(SCS_SIZE, SCS_SIZE, + VMALLOC_START, VMALLOC_END, + SCS_GFP, PAGE_KERNEL, 0, + node, __builtin_return_address(0)); +} + +static void scs_free(void *s) +{ + int i; + + for (i = 0; i < SCS_CACHE_SIZE; i++) { + if (this_cpu_cmpxchg(scs_cache[i], 0, s) != 0) + continue; + + return; + } + + vfree_atomic(s); +} + +static int scs_cleanup(unsigned int cpu) +{ + int i; + void **cache = per_cpu_ptr(scs_cache, cpu); + + for (i = 0; i < SCS_CACHE_SIZE; i++) { + vfree(cache[i]); + cache[i] = NULL; + } + + return 0; +} + +void __init scs_init(void) +{ + cpuhp_setup_state(CPUHP_BP_PREPARE_DYN, "scs:scs_cache", NULL, + scs_cleanup); +} + +#else /* !CONFIG_SHADOW_CALL_STACK_VMAP */ + +static struct kmem_cache *scs_cache; + +static inline void *scs_alloc(int node) +{ + return kmem_cache_alloc_node(scs_cache, SCS_GFP, node); +} + +static inline void scs_free(void *s) +{ + kmem_cache_free(scs_cache, s); +} + +void __init scs_init(void) +{ + scs_cache = kmem_cache_create("scs_cache", SCS_SIZE, SCS_SIZE, + 0, NULL); + WARN_ON(!scs_cache); +} + +#endif /* CONFIG_SHADOW_CALL_STACK_VMAP */ + +static inline unsigned long *scs_magic(struct task_struct *tsk) +{ + return (unsigned long *)(__scs_base(tsk) + SCS_SIZE - sizeof(long)); +} + +static inline void scs_set_magic(struct task_struct *tsk) +{ + *scs_magic(tsk) = SCS_END_MAGIC; +} + +void scs_task_init(struct task_struct *tsk) +{ + task_set_scs(tsk, NULL); +} + +void scs_task_reset(struct task_struct *tsk) +{ + task_set_scs(tsk, __scs_base(tsk)); +} + +void scs_set_init_magic(struct task_struct *tsk) +{ + scs_save(tsk); + scs_set_magic(tsk); + scs_load(tsk); +} + +int scs_prepare(struct task_struct *tsk, int node) +{ + void *s; + + s = scs_alloc(node); + if (!s) + return -ENOMEM; + + task_set_scs(tsk, s); + scs_set_magic(tsk); + + return 0; +} + +bool scs_corrupted(struct task_struct *tsk) +{ + return *scs_magic(tsk) != SCS_END_MAGIC; +} + +void scs_release(struct task_struct *tsk) +{ + void *s; + + s = __scs_base(tsk); + if (!s) + return; + + WARN_ON(scs_corrupted(tsk)); + + scs_task_init(tsk); + scs_free(s); +} From patchwork Fri Oct 18 16:10:22 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199043 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B1F8D17EE for ; Fri, 18 Oct 2019 16:13:20 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8FF9D222C2 for ; Fri, 18 Oct 2019 16:13:20 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="npChJyHd"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="KV/ow4iF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8FF9D222C2 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=aSls6LqNIL6lllKYTxdnKtTYE8YJer5lO4Qiq9sScHg=; b=npChJyHdw1FRCi UjVrgxIXiOoGKS+/xo4MRT8ai+DYZBkCkL2g9PXN0Y3WFLwA67Hny28Ak0cC94mydT0dtMyjUY+iS F/Ar0O9Z5hxGMIxsq35VYsLJMwNs0UsKywWTUi2chyTsAtU1zLRRWP3UEY/SaxA+SURnaainb+UPn OaQ4qfc33Izvh7p4ewJRYNZi0vOqWbpBYsUOrGIcopBNrnYS9MenqZhja/LhPSVZtDz7AI4yR4fWZ fycLRojGvpAEglGLTYvA59Gz7RQ4JYQ/KgJ9iYr4v3MGuzToZ9+og1IkGoDsRekx/TEHfdfqZVtBe y8vPZX2I1wNXDFbID0KA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUsR-0003qN-CX; Fri, 18 Oct 2019 16:13:19 +0000 Received: from mail-pf1-f201.google.com ([209.85.210.201]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUqN-0001lM-75 for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:11:14 +0000 Received: by mail-pf1-f201.google.com with SMTP id z4so4989097pfn.0 for ; Fri, 18 Oct 2019 09:11:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=W3WDjYH/VAvmVL4+53CGlFooJvBVNnR4Xo1gyIsB1cA=; b=KV/ow4iF9rNc040TkpTO6MVpIairBWehEzBlCF7oKFaCOnaCnM5zX4GvWthj1gqY0d PX9Y3DI/oK29Df6zneqURkzRbTBLJqkuaLffnq7A4El6cHZNlrUbQpzFcfL4ESnRkS+0 kFhJ8Srmh16l8pEUmTU/JhFulu4JDEH9dAL61TECT+ox0IG5tl+4w5amGJdnwnMlbBfZ fvGk8hbTsGc8lLqAgbYtD+1aodLStTRQ0VjzQsrgM1JFg2d+31jVYMY7X6BIqzpdsBol R753jt71O16ZF2aSp6FFKGjB7W8gPpZIrKVa5w89N/ZSJgIv+lGCKKJdnaWyQl7QvbhY plHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=W3WDjYH/VAvmVL4+53CGlFooJvBVNnR4Xo1gyIsB1cA=; b=i5MV0SjOdX4lZMEWTuKchecHwhMD3O2LsVBpGoQkMi1DwhB/OZX4SDL2Cw/AmeQUqO Uj5YVXz1FJHjur3hxoswhyRZ7iAuZUtMW0SAByLSWMiemxadd8b/Qj5oqouvC5ZKB6lz rMuElZIB9ft6kOZ4VFDrQlgIcEW0B7JuGlIK4PWkXCdhINH/7BI4vvLSa3x6H7qas6FT 0Hxz+o73KLhU508TMzJz+LVnCZOebwq2tAxPKKatdnSkVHYpg+mZ9fm6ahPch1j7xjoC dalKLD7R6ykoIMqRMbe0rNQZjrzPEsfhfh15NkhGb9tbrxSiNauFOQHcJzuktouBu7v5 Y9Ww== X-Gm-Message-State: APjAAAXe/iCS2eSZ7MvTK0xkraiNdShD7lhplpjoabFZ/GzIn8kHrj+g VNCwdOtinFb5/JBwMuVSZW5uo6x5CLpE2FNNeOE= X-Google-Smtp-Source: APXvYqwgwFmbX6ZGPldjZxfBYIvVprCTpRWpmeoJv1AAdr3RY4ifrVx/55EnGx/lsXIR5pOg65l4zkBFehMDe52czU4= X-Received: by 2002:a63:cf46:: with SMTP id b6mr10822679pgj.90.1571415067795; Fri, 18 Oct 2019 09:11:07 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:22 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-8-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 07/18] scs: add accounting From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_091112_317350_AAA23B6D X-CRM114-Status: GOOD ( 12.59 ) X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [209.85.210.201 listed in list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org This change adds accounting for the memory allocated for shadow stacks. Signed-off-by: Sami Tolvanen --- drivers/base/node.c | 6 ++++++ fs/proc/meminfo.c | 4 ++++ include/linux/mmzone.h | 3 +++ kernel/scs.c | 20 ++++++++++++++++++++ mm/page_alloc.c | 6 ++++++ mm/vmstat.c | 3 +++ 6 files changed, 42 insertions(+) diff --git a/drivers/base/node.c b/drivers/base/node.c index 296546ffed6c..111e58ec231e 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -415,6 +415,9 @@ static ssize_t node_read_meminfo(struct device *dev, "Node %d AnonPages: %8lu kB\n" "Node %d Shmem: %8lu kB\n" "Node %d KernelStack: %8lu kB\n" +#ifdef CONFIG_SHADOW_CALL_STACK + "Node %d ShadowCallStack:%8lu kB\n" +#endif "Node %d PageTables: %8lu kB\n" "Node %d NFS_Unstable: %8lu kB\n" "Node %d Bounce: %8lu kB\n" @@ -438,6 +441,9 @@ static ssize_t node_read_meminfo(struct device *dev, nid, K(node_page_state(pgdat, NR_ANON_MAPPED)), nid, K(i.sharedram), nid, sum_zone_node_page_state(nid, NR_KERNEL_STACK_KB), +#ifdef CONFIG_SHADOW_CALL_STACK + nid, sum_zone_node_page_state(nid, NR_KERNEL_SCS_BYTES) / 1024, +#endif nid, K(sum_zone_node_page_state(nid, NR_PAGETABLE)), nid, K(node_page_state(pgdat, NR_UNSTABLE_NFS)), nid, K(sum_zone_node_page_state(nid, NR_BOUNCE)), diff --git a/fs/proc/meminfo.c b/fs/proc/meminfo.c index ac9247371871..df352e4bab90 100644 --- a/fs/proc/meminfo.c +++ b/fs/proc/meminfo.c @@ -103,6 +103,10 @@ static int meminfo_proc_show(struct seq_file *m, void *v) show_val_kb(m, "SUnreclaim: ", sunreclaim); seq_printf(m, "KernelStack: %8lu kB\n", global_zone_page_state(NR_KERNEL_STACK_KB)); +#ifdef CONFIG_SHADOW_CALL_STACK + seq_printf(m, "ShadowCallStack:%8lu kB\n", + global_zone_page_state(NR_KERNEL_SCS_BYTES) / 1024); +#endif show_val_kb(m, "PageTables: ", global_zone_page_state(NR_PAGETABLE)); diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index bda20282746b..fcb8c1708f9e 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -200,6 +200,9 @@ enum zone_stat_item { NR_MLOCK, /* mlock()ed pages found and moved off LRU */ NR_PAGETABLE, /* used for pagetables */ NR_KERNEL_STACK_KB, /* measured in KiB */ +#if IS_ENABLED(CONFIG_SHADOW_CALL_STACK) + NR_KERNEL_SCS_BYTES, /* measured in bytes */ +#endif /* Second 128 byte cacheline */ NR_BOUNCE, #if IS_ENABLED(CONFIG_ZSMALLOC) diff --git a/kernel/scs.c b/kernel/scs.c index 47324e8d313b..0e3cba49ea1a 100644 --- a/kernel/scs.c +++ b/kernel/scs.c @@ -7,9 +7,11 @@ #include #include +#include #include #include #include +#include #include #define SCS_END_MAGIC 0xaf0194819b1635f6UL @@ -59,6 +61,11 @@ static void scs_free(void *s) vfree_atomic(s); } +static struct page *__scs_page(struct task_struct *tsk) +{ + return vmalloc_to_page(__scs_base(tsk)); +} + static int scs_cleanup(unsigned int cpu) { int i; @@ -92,6 +99,11 @@ static inline void scs_free(void *s) kmem_cache_free(scs_cache, s); } +static struct page *__scs_page(struct task_struct *tsk) +{ + return virt_to_page(__scs_base(tsk)); +} + void __init scs_init(void) { scs_cache = kmem_cache_create("scs_cache", SCS_SIZE, SCS_SIZE, @@ -128,6 +140,12 @@ void scs_set_init_magic(struct task_struct *tsk) scs_load(tsk); } +static void scs_account(struct task_struct *tsk, int account) +{ + mod_zone_page_state(page_zone(__scs_page(tsk)), NR_KERNEL_SCS_BYTES, + account * SCS_SIZE); +} + int scs_prepare(struct task_struct *tsk, int node) { void *s; @@ -138,6 +156,7 @@ int scs_prepare(struct task_struct *tsk, int node) task_set_scs(tsk, s); scs_set_magic(tsk); + scs_account(tsk, 1); return 0; } @@ -157,6 +176,7 @@ void scs_release(struct task_struct *tsk) WARN_ON(scs_corrupted(tsk)); + scs_account(tsk, -1); scs_task_init(tsk); scs_free(s); } diff --git a/mm/page_alloc.c b/mm/page_alloc.c index ecc3dbad606b..fe17d69d98a7 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5361,6 +5361,9 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) " managed:%lukB" " mlocked:%lukB" " kernel_stack:%lukB" +#ifdef CONFIG_SHADOW_CALL_STACK + " shadow_call_stack:%lukB" +#endif " pagetables:%lukB" " bounce:%lukB" " free_pcp:%lukB" @@ -5382,6 +5385,9 @@ void show_free_areas(unsigned int filter, nodemask_t *nodemask) K(zone_managed_pages(zone)), K(zone_page_state(zone, NR_MLOCK)), zone_page_state(zone, NR_KERNEL_STACK_KB), +#ifdef CONFIG_SHADOW_CALL_STACK + zone_page_state(zone, NR_KERNEL_SCS_BYTES) / 1024, +#endif K(zone_page_state(zone, NR_PAGETABLE)), K(zone_page_state(zone, NR_BOUNCE)), K(free_pcp), diff --git a/mm/vmstat.c b/mm/vmstat.c index 6afc892a148a..9fe4afe670fe 100644 --- a/mm/vmstat.c +++ b/mm/vmstat.c @@ -1118,6 +1118,9 @@ const char * const vmstat_text[] = { "nr_mlock", "nr_page_table_pages", "nr_kernel_stack", +#if IS_ENABLED(CONFIG_SHADOW_CALL_STACK) + "nr_shadow_call_stack_bytes", +#endif "nr_bounce", #if IS_ENABLED(CONFIG_ZSMALLOC) "nr_zspages", From patchwork Fri Oct 18 16:10:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199047 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4B4071951 for ; Fri, 18 Oct 2019 16:13:37 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1F629222C5 for ; Fri, 18 Oct 2019 16:13:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="D/OWUZc7"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="omhYD5xW"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="uErhWC3l" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1F629222C5 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=F3VjykywLg8FkA69773ai9PGawr78jZxdadDrDdtfaU=; b=D/OWUZc7l8f8T4 rfTr9yJ/viP3BCXGQxFceSkKxDxKohZo5nQPcwxzN+PCFpqDhbAJwCVPT9oDksw+eyc/maRd5rC1g 4l5RPzle3RmOOnc/0mE+MKTJMm7gh2JSQ3cM+H+BiLfYYPWFXghRL65q9aI8hbCoHoN97ZulcTnRk lveOplhojcV/IkP4u1qOaYRxkdZg41WuDYFEjehpB+YV4wWpS5CQwjkmjlSNMAFP57CnudW/6jTUC KcGxtSNzCWyV19oXtLkVpO2rpZuLawjkWrauOn4sXMZGhlmvk+WP4hyJNaUhzZlpzeQyaUGiYYBYr sIogO2DavPcvm5KxDlfA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUsc-00042p-Cs; Fri, 18 Oct 2019 16:13:30 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUqR-0001s1-Ub for linux-arm-kernel@bombadil.infradead.org; Fri, 18 Oct 2019 16:11:15 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:Cc:To:From:Subject: References:Mime-Version:Message-Id:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=+oBtFp6IQZ5946CFExp5BXllfwypP5AUtOYhvu+6eDg=; b=omhYD5xW/ih8KBnWMjhg2WkWy hE/U8JsWj+jLPOCV+zPTBCr6NzDp2A67mVB4zs6dFKFKsADwRbQKSzVHIXV6hyIH6HsMOBmDK9QXu 29cTzfUQV+8VcG/TugF9EdXDSWLmpEhU6j4fjYDQhZh3zQZeI7lduqTTuQacpZCedvl05qaXLI5c2 OHp2exMt4gEWeCVfVEBmDpcH7aN52a+REAB/DJKC+paCdC7HTzKD5Z7j5UsTB+/o053kplPcfgkI/ xJyy/A8ubqU6uKlSW175hfAG5x0c816qE6fL6TOYfUbf5ZpzwcshSc/aIekUc9B21n8IHAYySEHGD bibYmCHng==; Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by casper.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUqq-0004Lp-UL for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:11:42 +0000 Received: by mail-pf1-x44a.google.com with SMTP id a2so4971116pfo.12 for ; Fri, 18 Oct 2019 09:11:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=+oBtFp6IQZ5946CFExp5BXllfwypP5AUtOYhvu+6eDg=; b=uErhWC3lFP8THQ9YjycpXkAl85COCd6lh+RqxQAfdCNUSVTiEfspc2alvbCgArV2fu ls4p2vGwJ3M8+8dC+pxUbBcocxnmbIOx1oZI7haycpR/idqzr0yRfNB9QU0r3L0Ad+LB AODIdu+bBRBco2jvuX999sV5JPTfKSeK6DQgqagm1V48qwgIQ9QQo3kPVBNgSxrwUzDc J7z1g1ZIddJjuwZ9NXT+KTuZuEnEru2YIfSdVRPO6N1MNFiWgJDAJs9A6AicwQY5z5mf Ozj1xQ9RCTG/l6zakh/4DEHsBfg5zP3zC8lFWduaZXJzlKOfJV8siLAxWkKPzGoNNYpY BIXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=+oBtFp6IQZ5946CFExp5BXllfwypP5AUtOYhvu+6eDg=; b=dYhVWLcjv4MNulWPZOc+lL9z4cfFXSq72rc4oli5v9KvW/U3M6B4ZhQGfZ3O835TxW C38IiLS65v3Jc1GfcToS1VNxd5++SB/zjO9dq0+C9WTHwhIWBS8CQNJtP3FQ4lRyWaQ3 MKZkb1gTbDcVaPpV2kL6tgCGhC6AxwbSCr/JWyJd5otvWWaK6brA3OZnzzQHLNEGRkZ9 CxkP4nCwIigUeaGyCw77sxKYEelHcLgTrcxpMjTO5EH8+eMPpQAMvCUp8oIK6+efEA0m 8Byh8yNy7x41VI8PU9AhNvZsaSuYQ3oRjzddcrISwOjrq/tY3IXiPMHnutFs2qFVqFa4 6k0Q== X-Gm-Message-State: APjAAAUgMolb2IdqcE4KxfS6B3Eby/bG0IzF8L2Do0jtGNT+Js/yNo1n vZ2Y9b8TY73ojekaYDl6opQcur4xW9qYuNLohcM= X-Google-Smtp-Source: APXvYqxHqmmpD11MJRO6yuoyBvltYKFWJkQq7UxJVT4qvc6swUR9mDIPTvziXaDvcDGhe+mECq8Qr7PX0sGP/3daZE0= X-Received: by 2002:a63:7845:: with SMTP id t66mr10836733pgc.31.1571415070584; Fri, 18 Oct 2019 09:11:10 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:23 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-9-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 08/18] scs: add support for stack usage debugging From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_171141_014005_B8ED8463 X-CRM114-Status: GOOD ( 11.94 ) X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on casper.infradead.org summary: Content analysis details: (-7.7 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:44a listed in] [list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list -0.0 SPF_PASS SPF: sender matches SPF record 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Implements CONFIG_DEBUG_STACK_USAGE for shadow stacks. Signed-off-by: Sami Tolvanen --- kernel/scs.c | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/kernel/scs.c b/kernel/scs.c index 0e3cba49ea1a..1ec5c5a8dfae 100644 --- a/kernel/scs.c +++ b/kernel/scs.c @@ -161,6 +161,44 @@ int scs_prepare(struct task_struct *tsk, int node) return 0; } +#ifdef CONFIG_DEBUG_STACK_USAGE +static inline unsigned long scs_used(struct task_struct *tsk) +{ + unsigned long *p = __scs_base(tsk); + unsigned long *end = scs_magic(tsk); + uintptr_t s = (uintptr_t)p; + + while (p < end && *p) + p++; + + return (uintptr_t)p - s; +} + +static void scs_check_usage(struct task_struct *tsk) +{ + static DEFINE_SPINLOCK(lock); + static unsigned long highest; + unsigned long used = scs_used(tsk); + + if (used <= highest) + return; + + spin_lock(&lock); + + if (used > highest) { + pr_info("%s: highest shadow stack usage %lu bytes\n", + __func__, used); + highest = used; + } + + spin_unlock(&lock); +} +#else +static inline void scs_check_usage(struct task_struct *tsk) +{ +} +#endif + bool scs_corrupted(struct task_struct *tsk) { return *scs_magic(tsk) != SCS_END_MAGIC; @@ -175,6 +213,7 @@ void scs_release(struct task_struct *tsk) return; WARN_ON(scs_corrupted(tsk)); + scs_check_usage(tsk); scs_account(tsk, -1); scs_task_init(tsk); From patchwork Fri Oct 18 16:10:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199053 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A21F51951 for ; Fri, 18 Oct 2019 16:13:47 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 804E6222BD for ; Fri, 18 Oct 2019 16:13:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="b8vRADGc"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="cCT9hs91" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 804E6222BD Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=XW8UPCp6M127uwbHVzrAy86afaEooYchPbulTAn0aBk=; b=b8vRADGcMPkNaA xMTgSS45Rxpz7t+95vv3tVIk72BIAzkUBHTx7V8MbrHF0QGHnLKKuj8yCWziCtgtcJQIAUF0hS7Bu N21bQwLx8lBn6VgxTw9c4zoAJ5KtoMUhNF4pZYj8CRo2sDKgGrJYWZImXqOpWUCoDYBR5DnvfAktn /1d8zJqO0WGPnSjX+YuAnknuuzVSdD/hGWXfiJ3Ql3pNTaBe2z1y3phcqak43n5hmbMLL1W1lCEUj 01iLzDW+YYwOYZuLZuvPzKuT8J9cmtwSug3sFl+IHzKnDHDR/IZAPasRrAsNvxQPhY0KiFajhVSj4 6+wtymR/PWY4u1oKeXLg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUsr-0004JM-Pa; Fri, 18 Oct 2019 16:13:45 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUqR-0001pr-3W for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:11:16 +0000 Received: by mail-pf1-x44a.google.com with SMTP id 194so4984215pfu.3 for ; Fri, 18 Oct 2019 09:11:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=xX5gUTsoboVAmCrU5x0kxTyEAN1I1znyOrkoj+71V3E=; b=cCT9hs91spgek13wGsP9y0ttC9ve4IQ6YalGlMpwduvwu42Woqshf/GsDukku4Rd7z LzngBaQqis4pXbheJCLuLc/BuznaLuuhhZ0eQYxuqnnHcZIUFUvXDs57h723E1aVsqCN hDJ9NJVmzOzxHezv4iMvDqkeOoKw8JtHeSXKma4kAFNaEABY3mU6axzTP9w4Lq+UPCd3 3Ot78e0aaFfEd1x8s4TQXDNAkM6/+cID/LMqGDex4NuLmCmij2J8d1teYJAiH7hir7mz bJcHXcpZNWl2UPJrt20n+Cj5d8KKW3//pqk3cC5O9Pz9D8EOb1EO2oSOjKTQQUWNt+am FQTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=xX5gUTsoboVAmCrU5x0kxTyEAN1I1znyOrkoj+71V3E=; b=qZBWz+91wC8CcbaSvv24JQZHzym3GtHvUPHuII+tqVdx7vNJFGxeTyXQbf6+pd3sNR 1i+SpIOvd06zwj0K2yOcFD+cgH5cLXAOm5K44mz+7s3Y+wOVoHJoRXdDhf/Ujq1CamPg NtXDSnrnaV3pYuPi2Ts9LNKfaZk14fvpv6WhuP+YHZrqkQogGzct+5KGTr6RLqVSTNXP M6ljp6qp6ZKbgFN1j4gsp/Y1a9LTJxw1O6jjm7+vEPXWH9PNMVIN9xLp2tHnuyuIgiXG NT35JQIoxnmvy9Pn7XBFXrsXHxNr6D1JNchDGmuG2RRVcLzGU9QTzzQMJrTjC76zBrnB 32Ng== X-Gm-Message-State: APjAAAXKwPf4y62UwhDRQE+VHGuH30IZsnqdrG+AcpMVnc51WItnpeKJ +8zMWnsh9DkJaecgED04y+sQXL96cQ3xhR96v0w= X-Google-Smtp-Source: APXvYqxwiguloHdhaLpo4FBp0WjObBKG43Zxu/Qf9Jmum2rfBGkEmvxrlieIAShHEtrTW/4mJFtXISqp4bSzlzbpPKY= X-Received: by 2002:a63:e148:: with SMTP id h8mr10684150pgk.297.1571415072880; Fri, 18 Oct 2019 09:11:12 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:24 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-10-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 09/18] trace: disable function graph tracing with SCS From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_091115_156712_DD0232F6 X-CRM114-Status: UNSURE ( 9.38 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:44a listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org With CONFIG_FUNCTION_GRAPH_TRACER, function return addresses are modified in ftrace_graph_caller and prepare_ftrace_return to redirect control flow to ftrace_return_to_handler. This is incompatible with return address protection. Signed-off-by: Sami Tolvanen --- kernel/trace/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig index e08527f50d2a..b7e5e3bfa0f4 100644 --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig @@ -161,6 +161,7 @@ config FUNCTION_GRAPH_TRACER depends on HAVE_FUNCTION_GRAPH_TRACER depends on FUNCTION_TRACER depends on !X86_32 || !CC_OPTIMIZE_FOR_SIZE + depends on ROP_PROTECTION_NONE default y help Enable the kernel to trace a function at both its return From patchwork Fri Oct 18 16:10:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199059 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6B4CF1951 for ; Fri, 18 Oct 2019 16:14:08 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 48EE421D7C for ; Fri, 18 Oct 2019 16:14:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Tg5kABe7"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="JytSfSIK" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 48EE421D7C Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=fIY9TmQyn09kUDn8ZM1sg6JoCGyXNHFTOEBBpA98IXY=; b=Tg5kABe7h664zw Tzuae0WkMXyAn9uOrtqSpgpZjyzGNRUj7T9BABrTSq/rc+iMMcpdn4QfgtXv5DXixvqB04eOZh3Dl DFMxbksl9cfTL3aqWOEC/l9OFrO7sRNzK1qhX+qVnem0AlaNEKtqELRhc1xuZI7MztMmOZ5HCTfJc eRTVXuhg+CA3FljUJfL4uKZHnDjE/UKo2WNv+OX6n3+SUvLhTlEjHlSf4/143W7RiwSNy247OjU+y 5B13Rv+BBFaTgt+ZC/HUtTfS0WTAAKgg/Im271SIvCHRUKdLbFzRULR43icebnj1ziKqi5LICf1fO UXVIkrzVdy2Xc3zatBVg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUtD-0004aY-Ae; Fri, 18 Oct 2019 16:14:07 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUqV-0001tK-Mz for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:11:21 +0000 Received: by mail-pf1-x449.google.com with SMTP id z13so4972552pfr.15 for ; Fri, 18 Oct 2019 09:11:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=O8IkwU6sBVUF8FqvV1I5skAqoYbhLBF9oLtPiANlCGQ=; b=JytSfSIKC9plsUuL7Anvr6QOhk8r1uFIhvXpVGgpoIt+Pceeda3VIOzbcA7+0/BsI2 zIOFDZDlBlALPUkKw3QeckVXjlO15PINjv6We8JOgEFa99U5KrlSx7XXOz6Nx0YB48ik JUIvMysSkaZBaFFzF9YCNL6v7dsTQWajujUMHj1WzB/oICg2lLsquw3Vw81wZ9YrRh+I eqZjR6qEmEq1xmlqn4gPhCaDAzL6Ck5jxkdyZpOLHnnw7A9jN+Xtb/99WqHJGyBIz0eH RHpYhCrhAeYHu1mm3FYIj4Y1yIWOuV3/oL+MaGiwsAiiSjet6tZhPh7Y20s53JxWp6Uo RIpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=O8IkwU6sBVUF8FqvV1I5skAqoYbhLBF9oLtPiANlCGQ=; b=pxfXA0HUIn57jG8+T5k63GKOq2Or6r91QPCzvqNaZb2RZv1Cdm4xqMSrC+A+EYAyeT HK0KuElcn2wgA8P6y7cOLz3BLQSGZPBKlo6ydRG7felsTuA8KLRIF1vJK0GnwJDHGCVz f+PQWThzKNP9YUQT6gXIif4kRQM1jfBL/AJNX44LFTJsdq+kDtwi/UHcVBfyFmj1E/uy UNGjQB/O72J/DfMsRQYb5jzIYPwcEF4xAPV1Io0mfS2qkhIS7kK8cDeErpFqitN5ovCT UCXMHbc+tjLptFFksXLwo6pgNt+vUD+gFNurElyqfdjHIrhWZi0pxHUiyrgz/EJkm844 /8uA== X-Gm-Message-State: APjAAAV4ud4pJuAPJ1MPazMeb1WPpcI1odFBYjSLOsGbpgskIWKDQaBd 8wLwbkkY2CTgOQcpmszoLLBjximDXa8lLZ4RYII= X-Google-Smtp-Source: APXvYqxCc4imQBOuO4slIZxb1lJUVZnQFeO3XHf/hzNVQOpLbC+EVQgHSML0nT8J+/sJkR2vVeisW1XnmtFkqXhxt3k= X-Received: by 2002:a65:68c1:: with SMTP id k1mr11253965pgt.286.1571415075496; Fri, 18 Oct 2019 09:11:15 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:25 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-11-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 10/18] kprobes: fix compilation without CONFIG_KRETPROBES From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_091119_776468_43367C9A X-CRM114-Status: GOOD ( 10.62 ) X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:449 listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org kprobe_on_func_entry and arch_kprobe_on_func_entry need to be available even if CONFIG_KRETPROBES is not selected. Signed-off-by: Sami Tolvanen Acked-by: Masami Hiramatsu --- kernel/kprobes.c | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/kernel/kprobes.c b/kernel/kprobes.c index 53534aa258a6..b5e20a4669b8 100644 --- a/kernel/kprobes.c +++ b/kernel/kprobes.c @@ -1829,6 +1829,25 @@ unsigned long __weak arch_deref_entry_point(void *entry) return (unsigned long)entry; } +bool __weak arch_kprobe_on_func_entry(unsigned long offset) +{ + return !offset; +} + +bool kprobe_on_func_entry(kprobe_opcode_t *addr, const char *sym, unsigned long offset) +{ + kprobe_opcode_t *kp_addr = _kprobe_addr(addr, sym, offset); + + if (IS_ERR(kp_addr)) + return false; + + if (!kallsyms_lookup_size_offset((unsigned long)kp_addr, NULL, &offset) || + !arch_kprobe_on_func_entry(offset)) + return false; + + return true; +} + #ifdef CONFIG_KRETPROBES /* * This kprobe pre_handler is registered with every kretprobe. When probe @@ -1885,25 +1904,6 @@ static int pre_handler_kretprobe(struct kprobe *p, struct pt_regs *regs) } NOKPROBE_SYMBOL(pre_handler_kretprobe); -bool __weak arch_kprobe_on_func_entry(unsigned long offset) -{ - return !offset; -} - -bool kprobe_on_func_entry(kprobe_opcode_t *addr, const char *sym, unsigned long offset) -{ - kprobe_opcode_t *kp_addr = _kprobe_addr(addr, sym, offset); - - if (IS_ERR(kp_addr)) - return false; - - if (!kallsyms_lookup_size_offset((unsigned long)kp_addr, NULL, &offset) || - !arch_kprobe_on_func_entry(offset)) - return false; - - return true; -} - int register_kretprobe(struct kretprobe *rp) { int ret = 0; From patchwork Fri Oct 18 16:10:26 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199065 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D991F1951 for ; Fri, 18 Oct 2019 16:14:22 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 73A3021D7C for ; Fri, 18 Oct 2019 16:14:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="qEnitIim"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="NSG0XzUq" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 73A3021D7C Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=grBW8UtL6BiZc60ZIJYwcBieh79CawU91ml5JCxglZU=; b=qEnitIimmKzvCr dmadqipaqeaiastwlAn7zrSOEzo4b8+APHzCFCm6PzOLd0ynAq+GLto/ossSBziLraPlUwuoGsITk m+DdLGszyCxPKW3obbFpVifjvtbz/noVrtl5y2yk5wO2vZq3LvNvTs4x+0yVd4y4QPebnHAsrWUmH Pf241Txk70fDiLwdukkkC+kHHqjF/RPRzJzCeMEgrdT4pPoGGPGTLyqbaWGJclyJQGR4daR9/kSyE UnYw8s6qBMnVtdkgNylmKq8AAOM69+gw3dWM/yAVEmyQWWpClwfd1+osDMEzx1XIIgJHK5rsUwQ2y 5hKMloz7HgBHwjHhCG5w==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUtO-0004pc-Q2; Fri, 18 Oct 2019 16:14:18 +0000 Received: from mail-pl1-x64a.google.com ([2607:f8b0:4864:20::64a]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUqW-0001wW-UG for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:11:22 +0000 Received: by mail-pl1-x64a.google.com with SMTP id 99so4016035plc.18 for ; Fri, 18 Oct 2019 09:11:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=fICIDxFYsM9a9Ff5jMCA2HNdKL4X15DyG8HPfHVUdX4=; b=NSG0XzUqxk2+63gPHHlONBw5Q4nS2y744Eo1rGIDmimGan5ZvchrFconEmIXiVv79S 0SiQYCOcKQncsgO+OjD18O6E+2SSMSAk43T8F4SPII5CwYW5Z2DpjcXEvu90lABe0T7d 54pDxZVczkH9+W6UKw5BGz/1SAtjmaCTeJP9IigFW7/8iSrzVXwx+vaItOIgEL2NFpn8 PCwsy3taMFVH2Is9tZDbWuOIcT/YBmZIupi5pgO/dSJwAnrHQ+9LRhx/mZ3aQl1rS+Hn PYSAfr/54k98LWSAkxuS0/kkDyQ70rjvHFcfVRO0CwmtYXN/ED4JgjMIrSHGnZhfeFY/ nwUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=fICIDxFYsM9a9Ff5jMCA2HNdKL4X15DyG8HPfHVUdX4=; b=XDZtpcvZG0cw13kGQLJWW69Rf+ZcpWfmqt9fTkdRQcz9uGFKKIT2CskkAjmnCf+Km2 iMaWVh+SsbxSDyupgnBBTmoxmun/qqD8MM/45Eh0JkxOtgU8qpMpq8lor1IA5o5pz2q7 Dc63pM+E0edbVHUPprcEMgtT6mM/jwkH7ARQI/xDr3leaYXvCHKyq1+3vlf4chfkdS+U dVf2wg89j3Y6QkRQEHfs/EgiL2Hy3dndSoryCacSRnVwtJvfQDjgv/zpa7wrXb5KYTmG 60OmA6HEZ2Y7qa3N+JOSU2DjGtHLjo+C146Sb93lH+Ij2L2GTWuN/Hd2mBboz3MkdkI4 j95w== X-Gm-Message-State: APjAAAVDziuiR13ARnFEdDyn97urS5EAxXqCskX0ZK7hg7X/AFAGJqX3 5tSzLsAkJbql8LXy6fb8oWSO7lLCx+S2tEGhivQ= X-Google-Smtp-Source: APXvYqy2IPmHgVzSQFLWH/l27Qtv31FYTpjwXUbB8shECHuqpnU1nIwwQdz+u4oVI6rxTnp6d2Le2ZcdmFuior/slJo= X-Received: by 2002:a65:6092:: with SMTP id t18mr11012516pgu.418.1571415077997; Fri, 18 Oct 2019 09:11:17 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:26 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-12-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 11/18] kprobes: disable kretprobes with SCS From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_091121_004556_1A7179A3 X-CRM114-Status: GOOD ( 10.64 ) X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:64a listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org With CONFIG_KRETPROBES, function return addresses are modified to redirect control flow to kretprobe_trampoline. This is incompatible with return address protection. Signed-off-by: Sami Tolvanen --- arch/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/Kconfig b/arch/Kconfig index a222adda8130..4646e3b34925 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -171,7 +171,7 @@ config ARCH_USE_BUILTIN_BSWAP config KRETPROBES def_bool y - depends on KPROBES && HAVE_KRETPROBES + depends on KPROBES && HAVE_KRETPROBES && ROP_PROTECTION_NONE config USER_RETURN_NOTIFIER bool From patchwork Fri Oct 18 16:10:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199077 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D51B019A1 for ; Fri, 18 Oct 2019 16:14:55 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A7E0721D7C for ; Fri, 18 Oct 2019 16:14:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="ChmER3Z6"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="sClupJ7s" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A7E0721D7C Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=jT2/2WSb3U7Gfo6QnvZYcMfQoJz2WKBG4hQNu/4Xh7o=; b=ChmER3Z6fcpp/0 4FaVDwCKHH3pvXClaKVGBIDOQamVyk7noYUp0FmJb8VSM8+8GwrQx8/6CfR/LMKnf2kXzKy8b5+8U UbWXlXzx6yugBFpotrHXLaMXinkCryleWZFdue1p8NOo+w6KOT9R87W1FfB2bfx8IZv+ItcRtLWyA tcaCEJaNF/PFYPNoxqsAK0PzYetBqcg95N3Y4ttjjX7YJxlRRc4OVOhK8L9FOdk8Ge3SPz4S9KfB2 EeCoBXg8kQEwPMxchKtukFQ1zpvpPpeYrwT6puTXY5uCAeG/7J5Y9mQsUawpd2UQDkUQIAI1+MUAN 4CWPCGw91pOXaRZrK/3A==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUty-0005Mb-LH; Fri, 18 Oct 2019 16:14:54 +0000 Received: from mail-pf1-x449.google.com ([2607:f8b0:4864:20::449]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUqY-0001zt-FP for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:11:24 +0000 Received: by mail-pf1-x449.google.com with SMTP id x10so4958641pfr.20 for ; Fri, 18 Oct 2019 09:11:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=K1ZhNofnBR/XSzepuNm2GXj6CjTITzvlb/GIVw7T7D4=; b=sClupJ7s2vyir/7TQw4u1wskJ40f2afok9L62g3bdETWOSrvPIoJbt2TUVhCDZl2Ll jSD1Pe1pU7jtmC4+A/khfQCqIGy7o+x3Glnf1WXjzHqx9iAelj7PV0FH9T6XgXbpgXPr QmyFwYa7yB+qCqYypJwWf511MkYFKNAU+aHSUjTgdgKc/LGoN7I1EdcPjslTcJn6Nsnh zpS9sYd2YqSiWR32qZg1jO6XyvMv14Qmysa4U0MKWEQOtGZOZsv14zS5QNAzrCYTZwhv xDZy49ncrITpJUlYkIIuqsGVZMQ4LMcDek2uuDnjzpOLCvDTsilhXvrJN1MvHxPElje/ wDsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=K1ZhNofnBR/XSzepuNm2GXj6CjTITzvlb/GIVw7T7D4=; b=E7l2qSS3fqFMgd9A9uya4RzlIkytiRX7EZ3VPdjig5zbds6QFB+8681JBIweXuBKS7 JXtdP96/elWR/S4oPwJdmU9R8oeOjbYDQD2Vgw1bbKSymM7JIpaIdPoWCkK1Vi9lFFby tugYw/eXZV0McBvurlSZie5FAeOurfX+yxJZrpV/0hrHQBNGe3BWZhyBdyBXCjE85sud 2ezowe0l8Pswn53HA34xTmsafdGcGXxBSNpa67rb/ZT6iGh4LKs0XGzs6gPiksQCoX9A nZ1XPAbM3/8cznG+W11XEVugVdLrYy6TI5hz3+NQKB6BFGMNsUFEVFbl5yMeIvxYhFgk 7u2Q== X-Gm-Message-State: APjAAAUvF2KBGDx3GUGu552REbyDa9TlpLrjckp9FtcmC+UVenblYgfw I9IR+cpTUVCG+7nqB6hrngelT9p+t7h7u2Y4lbs= X-Google-Smtp-Source: APXvYqxQZgJ8+dUeSHjL/X4r9AY12JB0O/dDMacaQgZEGqyCDds26CgWDH7ZPg+xHY8lSTXkA3/VdzbOvvEqn3IOjDM= X-Received: by 2002:a63:eb52:: with SMTP id b18mr10634742pgk.205.1571415080458; Fri, 18 Oct 2019 09:11:20 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:27 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-13-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 12/18] arm64: reserve x18 only with Shadow Call Stack From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_091122_560548_DFDFA589 X-CRM114-Status: UNSURE ( 8.91 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:449 listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Only reserve x18 with CONFIG_SHADOW_CALL_STACK. Note that all external kernel modules must also have x18 reserved if the kernel uses SCS. Signed-off-by: Sami Tolvanen Acked-by: Nick Desaulniers --- arch/arm64/Makefile | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile index 1c7b276bc7c5..ef76101201b2 100644 --- a/arch/arm64/Makefile +++ b/arch/arm64/Makefile @@ -55,7 +55,7 @@ endif KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) \ $(compat_vdso) $(cc_has_k_constraint) -KBUILD_CFLAGS += -fno-asynchronous-unwind-tables -ffixed-x18 +KBUILD_CFLAGS += -fno-asynchronous-unwind-tables KBUILD_CFLAGS += $(call cc-disable-warning, psabi) KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso) @@ -72,6 +72,10 @@ stack_protector_prepare: prepare0 include/generated/asm-offsets.h)) endif +ifeq ($(CONFIG_SHADOW_CALL_STACK), y) +KBUILD_CFLAGS += -ffixed-x18 +endif + ifeq ($(CONFIG_CPU_BIG_ENDIAN), y) KBUILD_CPPFLAGS += -mbig-endian CHECKFLAGS += -D__AARCH64EB__ From patchwork Fri Oct 18 16:10:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199073 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 323361951 for ; Fri, 18 Oct 2019 16:14:47 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1011E21D7C for ; Fri, 18 Oct 2019 16:14:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="WfeOHHB5"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="XB6QWz3N" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1011E21D7C Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Sgec1mvFUL6cZ3iszoxGQ7quBUe8MZkiTDzO7qok+bA=; b=WfeOHHB5aSDl2o wsXJfzYmBZerBKrZBGxrRZgzx2OxdSk29Bspxxt+Sk+ShCAUImWuY9BZz6zODmzGejYR7qahoXXA/ 6G5KgrfWRzMH+jsDGsZKqHi0hJnjZONhdLz2Mq7ZpjHhJ89GHXO9v4xLhXyz9veR2CEEDDFRdcR1J G1SpNnY5kiMxLqV7hcZa3o8gH8qYxXdzuSbXs1P3MQAZhDBtFPpQSFGONbq0W1i1/imEVlIzsm/Pz iwO2qDvidyqU2C1kDwedoxZg0OloLF0+FpdAT9bTsQ17dm5iFzKYbtKL1ZAwDRO05SaEqMSTxKw8y jkk1mSbx1wvLC0CSPEpA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUtj-00055p-Kn; Fri, 18 Oct 2019 16:14:39 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUqa-000224-Qx for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:11:26 +0000 Received: by mail-pg1-x54a.google.com with SMTP id u20so667798pga.4 for ; Fri, 18 Oct 2019 09:11:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=J6SOPI1rQkBRz/oAhlc59Qnl/pEg0yYrtpuJkIBOBkk=; b=XB6QWz3NWtjFWDb+Z+2Cr8pmFgRAQYSnL3Uid2Hg1L3X4a6VceEhm2xwPy1aK+VoSw 9aDveAT40bEva7mRzlDierznfNSbLqDMAgSgjhFynz+GencRC+yEXI9GlTtBT22sGU3p BgssCzMTgiVVKQ7dMBIH2LGWSYP103Svydpv/Qk8+Xt+dKrQ9nOnszXcbv17KY+4wwg4 NXeKJ9olZaMc5VRteVjdV9T7bI7kUuufX9b3S6ryE8d0aJM7RC3oqJ3jQ0cSx2x8TLsX ddVquFcsQoFiGmYXXscZ5m21z4LgANVA4sAoVnGTE5uNuWZVS07mkE/xs/dMfJWy0ygM tC4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=J6SOPI1rQkBRz/oAhlc59Qnl/pEg0yYrtpuJkIBOBkk=; b=TRKgTMtsZAv/Q0tpN4xUDHOCo+6Y+jd7SGyRz3UO1tF9i5Ajukd9rPYNOCPwSZvXyc eWPEMPwg1St9KqA7lASzkEaiFbATbHenmwpoAY33gYPidv7C2YTAXRwJdCT5jqolWD/5 wmz3eN1nSK33VsqLaS+slPYdgfZhPHSV+9d7oIo5rEHvDv32Uz5qFb9Z4HSmxCad+Kup EeUn20HA6aBLDG30if9+9UdIFR/mI1aXmrLV8uRccVz4f53pPfM4Odta8xyfJm2WKO6h kvmpPNVkT79cxHI0Vos58B4FTrkF6FCcqvD1KsLK/mahPyjBrRMPdnCh+IaUvGJAxHxS XFRg== X-Gm-Message-State: APjAAAWIKYgHjC51sv214rtRziJEhRKjlOvEuDE69XJnytcRD7VbPjq/ dyiK5BeJaaflrs1pnmljbdorcATtA+RhDsW++Ho= X-Google-Smtp-Source: APXvYqx8PtU7/rtmLemGGFK39JXzIOz2FvaCxezwKfcFnavxjhOconPajgJzRwBjZVVbgrCp+Uo7fSkc9snzmg9De7Q= X-Received: by 2002:a63:1e59:: with SMTP id p25mr10856086pgm.361.1571415082807; Fri, 18 Oct 2019 09:11:22 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:28 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-14-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 13/18] arm64: preserve x18 when CPU is suspended From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_091125_024578_23B1C6AB X-CRM114-Status: UNSURE ( 7.83 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:54a listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Don't lose the current task's shadow stack when the CPU is suspended. Signed-off-by: Sami Tolvanen --- arch/arm64/mm/proc.S | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index fdabf40a83c8..9a8bd4bc8549 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -73,6 +73,9 @@ alternative_endif stp x8, x9, [x0, #48] stp x10, x11, [x0, #64] stp x12, x13, [x0, #80] +#ifdef CONFIG_SHADOW_CALL_STACK + stp x18, xzr, [x0, #96] +#endif ret ENDPROC(cpu_do_suspend) @@ -89,6 +92,9 @@ ENTRY(cpu_do_resume) ldp x9, x10, [x0, #48] ldp x11, x12, [x0, #64] ldp x13, x14, [x0, #80] +#ifdef CONFIG_SHADOW_CALL_STACK + ldp x18, x19, [x0, #96] +#endif msr tpidr_el0, x2 msr tpidrro_el0, x3 msr contextidr_el1, x4 From patchwork Fri Oct 18 16:10:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199083 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8311717EE for ; Fri, 18 Oct 2019 16:15:19 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6086121D7C for ; Fri, 18 Oct 2019 16:15:19 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="WvppbJs9"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="FGaZKiG+" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6086121D7C Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=0D8sYqaRVDHdrNYZA7OiqoCbimXyuShesYMzcZ5mz64=; b=WvppbJs9y0FQqS HQjdUIeY3rM7slxKGQDfjPcrDH7asiTeHuN8PWZ01NfY8DU7LQXhaWGJNUrX89DzK6VR8gaQargJF PZou3dfRDyu1l1LroViPmYaTJ0+Wg6IoU0F8nw1X2Sd3CXpjo6vK1xsGeNgxCqxlgCHkER+U5r99d jZCsShkT+B/X7/K+998htvLWAqhHXv6ZfMs+EAkXsJD5epEdoQ7oPIcVt9fNOLOM/uUleLW7j1Npx QV/JFG+voNak87MJBcP2ZtetqZ/GuiSLBhRDdy+jBgGMfzPbfbmlv7RdSzayi+HA1zoFlTLahEybH 96sojV0OyNZim+pEe92g==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUuM-0006Vy-Gv; Fri, 18 Oct 2019 16:15:18 +0000 Received: from mail-pf1-x44a.google.com ([2607:f8b0:4864:20::44a]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUqd-00023j-Ob for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:11:29 +0000 Received: by mail-pf1-x44a.google.com with SMTP id x10so4958833pfr.20 for ; Fri, 18 Oct 2019 09:11:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=OMV7GU+CS755gzThihulnjNw1pxqVtHrVsdKKTavPQY=; b=FGaZKiG+Ux25snyrOJrBiRd7RwQxjjLhUDr+StQE9m6QImfqUDq6yp6j+8Toagonr0 9TtbMLb3ay20pI9YbGG7nJzzM0vR4AyRpchI4t/ZYgICefWGJVsG/b/aJxIHIw5w9UV3 EIJDiPWiyVwP1F234VZ4D2+fA00WZK8lZRp8IvN7MdZJh8XNzU3kgbsv5o54oZ+iiZsI KghuQJd7vpAsaqImv6J86r3heyO3yrp6I8mcuRhzR5+TbYZJ39FxxZRefwS+iHZszAKS nursS3e3ycHoRmOFQHS1v/YDrqGB9JFjdDc8idnF+2J0IDA0s0+5SNkcUWwuinwq2OFT V+bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=OMV7GU+CS755gzThihulnjNw1pxqVtHrVsdKKTavPQY=; b=k0hQK/nx/D/ShVRRS5cnzOAY1tolJMn4YWDmiCmaQcQ56JLAY5Cw4bYrVeee7q7dsm NHrWo3KOiLr/Zs+/Xm6QscP6lSfngYiouEXnZ4mZHiGFjACYZ2Dd3jqqES/dqrZE/ee6 ANQ+Mg4PDkpedbGPgE5zmrHVhjKwSF3kbRAwEkqiATKgNTeWqjI+uuuHK1BeFv1lKNzq LHEkeiOYX9oWsoxXg1krvceR+XEUDtrM66Kiakb88hNPzXHOrp+HXdvFCGnGSwvBYQwZ wgKKQNTcW47eCBUQKBPMYtgBpVh8xxZiEjTWyDKqOqAscxuHNplUny5tNoQ/5Awl1CdD IcLA== X-Gm-Message-State: APjAAAX8Bzfo+iGm21Z1qqmHF4gvY5Z0+nQky1H8TA7sHwLK5XJiLuxM VtTxRbTcLDt+Tb3e7M+8+CgmFnnsPG2s0BGzZCY= X-Google-Smtp-Source: APXvYqycxw3zlWpqg/6yGEhb7CgOxW1rmJc0Y43aPOFAV6KRrpZHK92ZxBUvHsBpIOsv4SiMThIv0qzjvIsHF4JRnyI= X-Received: by 2002:a63:1904:: with SMTP id z4mr11066720pgl.413.1571415085386; Fri, 18 Oct 2019 09:11:25 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:29 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-15-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 14/18] arm64: efi: restore x18 if it was corrupted From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_091127_815578_E331679C X-CRM114-Status: UNSURE ( 8.81 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:44a listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org If we detect a corrupted x18 and SCS is enabled, restore the register before jumping back to instrumented code. Signed-off-by: Sami Tolvanen --- arch/arm64/kernel/efi-rt-wrapper.S | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kernel/efi-rt-wrapper.S b/arch/arm64/kernel/efi-rt-wrapper.S index 3fc71106cb2b..945744f16086 100644 --- a/arch/arm64/kernel/efi-rt-wrapper.S +++ b/arch/arm64/kernel/efi-rt-wrapper.S @@ -34,5 +34,10 @@ ENTRY(__efi_rt_asm_wrapper) ldp x29, x30, [sp], #32 b.ne 0f ret -0: b efi_handle_corrupted_x18 // tail call +0: +#ifdef CONFIG_SHADOW_CALL_STACK + /* Restore x18 before returning to instrumented code. */ + mov x18, x2 +#endif + b efi_handle_corrupted_x18 // tail call ENDPROC(__efi_rt_asm_wrapper) From patchwork Fri Oct 18 16:10:30 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199087 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B639514E5 for ; Fri, 18 Oct 2019 16:15:32 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 943FE21D7C for ; Fri, 18 Oct 2019 16:15:32 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="gWTw1ej4"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="MU3xv8yS" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 943FE21D7C Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Bcu04wbcdd7y38jDq1fbMzRd0GHWEsGWwKxvEnxyoAk=; b=gWTw1ej49pSg6H dBCreHW5aWkT3/eBC38ZbpBBiLp/HXQpaWnGAHT+6wxsY8AwFZ3N1OR5N+xQrpRwGI5T5fatcTu+8 s425GIfhb1LcacwbVf8ZGBgdk2tTObWFBlb16Clk+q4tT1yeKaKEe06EPrv6GXQdrYcaLDTir9V05 /9sJGzzz0UOxbZ1UKe3Q6mGv1k4QCWPC6ESYzNhsPu62fMnOs5tBWRLjNVKoYftxEqGG5UMOX5lCl hgrsCKSSnIrAvDKaOwPMqVAE/uZfjwhx7eEDCbLSFOKs2K1jIgjzhIypS4Y1S5LTRba1efx7Gw8aR I0UbDVm6681IiaIuR1Rg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUuZ-00077G-Pe; Fri, 18 Oct 2019 16:15:31 +0000 Received: from mail-pg1-x54a.google.com ([2607:f8b0:4864:20::54a]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUqe-000265-Of for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:11:30 +0000 Received: by mail-pg1-x54a.google.com with SMTP id u4so4565401pgp.23 for ; Fri, 18 Oct 2019 09:11:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=kAZH5b29Q7ARe7ApEbNubNt6T6v1C763yQTiGjc3PQk=; b=MU3xv8ySUWa8q2gpjX+qFLOwCKhVBxOWHnEJ3R9sBKpRssnEcrJMlv2k41DcBabefp YGOIRyOHtuLqhQjP/H6UeJPJ31HbWh9EpsVg0P4HVVdbNapzd4whA6CQO0jcE6HFqrZI pgFlyp8aklTxhdiWcaDBLc5CpmFKJZswMRP/d6udWasrZrb61oeIxbFInJG3ZOTF4sVk 8JrRWRNQEQR/efYLvQF070upFjEKFVFMcnO8HN9mWGmqJ5+afIdNL+vr/Zp/pbDcSsCQ e4yhuM2R8kT/imgzLMvhUd7v/AWzMuvSo0KChcal0UxqcYxBSXGTsqlDEDweUlIG3qti 24Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=kAZH5b29Q7ARe7ApEbNubNt6T6v1C763yQTiGjc3PQk=; b=LnK+xEzkskfann2/ObzCqfpCG+jzyaVd6/ZQtB/UYdQGcE6UGOr0jy/3h7mXW722oR yWtIgeSAdwzR7M68Hs3ljPcr6Gkj3BD7v5zT8HzKOjS1QUBDy0pV+OueXY70GsVkLNNr AMZ0y3CoFTjBW0o8a2KGpCEWvphlDbEL4FSN1wK5jludjgNXaPAjXcpHFNuUpPnlx3MH veng/v9m5PRrrlCpuTnZNa7u8AdSZpQOk2ngEvujgql4nIcb41+izJVHuzYByBzzJDwz 80yjbYGsi3LO7slI7XsbmrGdOquuBZ87RlSjXkmw43AeLjhRwPDCbdQ5KMYmWbx9/J24 Xpug== X-Gm-Message-State: APjAAAV9fTlzPmqQw/De6sGWjTZxPZJQBZogcxd/xrVUAhKmzLdw0Q5/ w4FZhM10vW7+tvaa/x6gl78Q3vXnxPvKrwivQCU= X-Google-Smtp-Source: APXvYqxrZ7wEgagooCPTAQmdCRYdis4uNkIIYm64oppMcYVDpcyhHuDu43I31sdZRydHODqHUlaqbmkndVluNi8asAA= X-Received: by 2002:a63:ff08:: with SMTP id k8mr10900425pgi.8.1571415087691; Fri, 18 Oct 2019 09:11:27 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:30 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-16-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 15/18] arm64: vdso: disable Shadow Call Stack From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_091129_018429_A8C3E7D7 X-CRM114-Status: UNSURE ( 9.79 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:54a listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Signed-off-by: Sami Tolvanen --- arch/arm64/kernel/vdso/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile index dd2514bb1511..b23c963dd8df 100644 --- a/arch/arm64/kernel/vdso/Makefile +++ b/arch/arm64/kernel/vdso/Makefile @@ -19,7 +19,7 @@ obj-vdso := $(addprefix $(obj)/, $(obj-vdso)) ldflags-y := -shared -nostdlib -soname=linux-vdso.so.1 --hash-style=sysv \ --build-id -n -T - +ccflags-y += $(DISABLE_SCS) ccflags-y := -fno-common -fno-builtin -fno-stack-protector -ffixed-x18 ccflags-y += -DDISABLE_BRANCH_PROFILING From patchwork Fri Oct 18 16:10:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199093 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CB30714E5 for ; Fri, 18 Oct 2019 16:15:53 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A87C621852 for ; Fri, 18 Oct 2019 16:15:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="C5I/aXw/"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="XTGR5ysV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A87C621852 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=W4SwKqouFNjDbFRG3ABAZkRoRANO8aDephTqE3Szufo=; b=C5I/aXw/VAfTFQ 3aC0A/MHjKhnJ5uRYbBYItaErw6VIbFj+UZurS4n7voqGZin9O4bofQiWlQ3jCa/AlMl/x/wRlDIB cxa0KwSVgClv8WWXg4640rdt7fiqkWroNzM9hCAfRlXE8IjbmlnM3T3u11Eal2Ya6wxEFh3TnH7j1 aHdPx2z3LIvTKro3WFVUNq7k2UcjfgHUqD99cfct1OxlN+LSE/xV6s/Efu4zndd0vWqjVN67tYjbM lullrNqmv8/qRVELoNtNG82U2VWL+Eg0jIAhf5JU8XezJf3rt3vj6iPoukHH05P+POWUWpxkC3DJx nkz7PxYDmnPb/a5OyfcA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUuu-0007Qn-EX; Fri, 18 Oct 2019 16:15:52 +0000 Received: from mail-pg1-x549.google.com ([2607:f8b0:4864:20::549]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUqj-000288-Mj for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:11:35 +0000 Received: by mail-pg1-x549.google.com with SMTP id e15so4576918pgh.19 for ; Fri, 18 Oct 2019 09:11:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=21PQYFGMSyR0vWwlJoWUR0F8CwzgoNMVM1L0Pe6MFrQ=; b=XTGR5ysVCvFcDPfDZ2uRa4LSCFJbZwlyniF9TrCYGMBPbkIf0K085EjiQt4ffVYZbT yaiGK3y6faisAh6U9/CxmFlLU76VW2yJBLJXrJA65l2MTuNELcBcEAKKoaa0/ytobYt1 fHBOBarOVqcb9kanEUVa8Ddgrfl9iIQEWkgSVtgsKdKE18WzVjzt/JiQzP1DlMT6n4jU kAXZNvTi8r9G3/D4MyaIAco80dTDl8K/TpV2nIA1UUazOM41jXMV5YIAb/W//1YPMnKI Y+DGVHPZ9eY1T49HNVfdlWN6taf0ZPGpl8IGYtJKJPC8pDOBJ6EMgatwO8Hvvq48VXqN DmCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=21PQYFGMSyR0vWwlJoWUR0F8CwzgoNMVM1L0Pe6MFrQ=; b=I2l7qGOzY+SHRzYZzpU/G0FF5SA8EMW6iGSmi8L62Zdk5Vxycsrp46XDSWgmjjQ2Ek k/hB4en2z+HuTUilL1Mh7RK6vwPhLlgkXXsEUdhBnKmQEMjrV6/eyTBUoLzF+rt70WP6 E5Z96HXSyOyepowPK6DRJzu5uLsSy2SxhCchhkb1JthuJbF3oG2kcrjh3EnfNApRiQyM iS4YIDfaSWcady/VG2TTgETNk7FFP0/vstgwyVjP4ZmX49OJa+2gOgLhYOSHSfc1/Waj LkXFWKg8e7l0WyiIHfsiZAm0oJXPAgYZXDOuf0KR49fhAexj5cJjkXCUV/4GscPU30R0 HfYg== X-Gm-Message-State: APjAAAXP44BGWEMIqSw2QELqTNxH1FzTHL/F+7AyCiv5m6QWZBYV/Lfb KsU3/XzZcThnScnHaRDGvW1kcoN8Ow8+NU1RvAY= X-Google-Smtp-Source: APXvYqz2h3FK6fOLNjeif4Cg24fCITnU5ZQBquMxea8azTfQWEvOTaNE9xdQyiziN7s60FAxuGyTh4t3prrtUVRlSk8= X-Received: by 2002:a63:1250:: with SMTP id 16mr10784356pgs.331.1571415090313; Fri, 18 Oct 2019 09:11:30 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:31 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-17-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 16/18] arm64: kprobes: fix kprobes without CONFIG_KRETPROBES From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_091133_793490_D5E8550F X-CRM114-Status: UNSURE ( 8.83 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:549 listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org This allows CONFIG_KRETPROBES to be disabled without disabling kprobes entirely. Signed-off-by: Sami Tolvanen --- arch/arm64/kernel/probes/kprobes.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c index c4452827419b..98230ae979ca 100644 --- a/arch/arm64/kernel/probes/kprobes.c +++ b/arch/arm64/kernel/probes/kprobes.c @@ -551,6 +551,7 @@ void __kprobes __used *trampoline_probe_handler(struct pt_regs *regs) return (void *)orig_ret_address; } +#ifdef CONFIG_KRETPROBES void __kprobes arch_prepare_kretprobe(struct kretprobe_instance *ri, struct pt_regs *regs) { @@ -564,6 +565,7 @@ int __kprobes arch_trampoline_kprobe(struct kprobe *p) { return 0; } +#endif int __init arch_init_kprobes(void) { From patchwork Fri Oct 18 16:10:32 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199105 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5DE4917EE for ; Fri, 18 Oct 2019 16:16:22 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 38360205C9 for ; Fri, 18 Oct 2019 16:16:22 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="YfVTGW2w"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="B7xenBDH" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 38360205C9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=dC9Kn66prJOr9OyXfDweOfGNpHblTXpGcqzWQaywG5Y=; b=YfVTGW2wILreOA Ex2I/NYiS4XIJuR9DsoZ7CsQnAN/0Fm8DtieclhXQ/v+BOSBAlBVGkudt2xKHa+W1T+a+uTF3QGDP HM1yhXqvGOKxPte6RKOmjaY9Lueq5TSdjnh8G0G1IvO/B0T1nc16ohEj27QRGNwIR3ZSbvBxbclYN yoqqwsjMpmYIypDVINVCnl4Ft5pxNxxr7fuzV+u3uYenivnIB1CNt30+BieWkpQxWiwR/m93Z8ZDm WKEBONAp7TDti1uqQ+8ZMsQJWqVm3zIVh6G8jjdovSOOo79gyN3Ju7hZQ90Q1BXHGCVS2Ncl9Rh8N DeTnZyG/GDv9FSwvNu2g==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUvF-0007jN-J2; Fri, 18 Oct 2019 16:16:13 +0000 Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUql-00029y-JA for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:11:37 +0000 Received: by mail-pl1-x649.google.com with SMTP id j9so4008788plk.21 for ; Fri, 18 Oct 2019 09:11:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=5u5MugUg2eXeQck5Unb3LYQo29vaVROeFk+dSkSebHA=; b=B7xenBDHHI7ayNn+BP1Q6X8wnTR9uTlyid2Lxqej61gj2k9iN/wuKOJNKBobLxaFjR hT53He9gZ81WBRA7k1srlE5y9pRls5zgdJRc8CkufohNb7pDylEtWxFwrE2uoJHPqVMe SXZ+LI2gVL3V+ZFsZVGAp19D4zWSpAGia2X5kfETCOjg6mYUy8e3oO5INhcW9tdCJs3Z R+qaaFeh57hjiYMhdf3zjE/30gJ8GCvzwq1j+lgdEDaXxK/SXne8U2YrUzyBDK25HLGv sScwvilKx4tvHpfuXY9/27TdPb7wP+y5Fbv94C3Elzkmapg0qVwLdFfipvrL/n4U5zdf mmCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=5u5MugUg2eXeQck5Unb3LYQo29vaVROeFk+dSkSebHA=; b=i95bzeekCTLiiLENnvTxAnlFMLm3VDj1g4UYr2FuYRUTH66JrhoLL+DjWzuCQkr+ew VPJBBkuLckG4XeRFHDrktFDTF0eqkgYOiB3K71dhV3830Mmy4dCe89E7otTmxQssOfiZ syD3f/gfCTiGl2H1ekVjL9NO3J01ee/d3PxY+trAj1zFIdygOqT423uew7bheU+S6FjE wtupDKUmh6q4eNHEHOOOhJ6CU38TIJ5YGbW7RCarduVOQCvQ9T4apegbskDzfQjY+PfK YxtRlaqj4r2bmE5umBQKBrXXfREeFPyYShmKse8SMZimBuXq+7wf3K3brrlFGaeoVmTx DPpw== X-Gm-Message-State: APjAAAUi3CHvyVAeO/6tXqAV2luhDRpcDExGIG2Ht6s6RD3Ywq58j8ob ZCfsOLHhHOsKILWdmbmhHjhw/ZekEzr5xoYEM4o= X-Google-Smtp-Source: APXvYqzL1rNkHTnWVT3YklNHjYJF0z1ethJHe5MAlJ9H7HUMSZwkLdmWhiBEeQet4XLN4HbPHGGrf5qLcex/d/OTBLw= X-Received: by 2002:a63:541e:: with SMTP id i30mr10990238pgb.130.1571415092950; Fri, 18 Oct 2019 09:11:32 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:32 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-18-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 17/18] arm64: disable SCS for hypervisor code From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_091135_654090_79343143 X-CRM114-Status: UNSURE ( 9.55 ) X-CRM114-Notice: Please train this message. X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:649 listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Signed-off-by: Sami Tolvanen --- arch/arm64/kvm/hyp/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile index ea710f674cb6..96073d81cb3b 100644 --- a/arch/arm64/kvm/hyp/Makefile +++ b/arch/arm64/kvm/hyp/Makefile @@ -4,7 +4,8 @@ # ccflags-y += -fno-stack-protector -DDISABLE_BRANCH_PROFILING \ - $(DISABLE_STACKLEAK_PLUGIN) + $(DISABLE_STACKLEAK_PLUGIN) \ + $(DISABLE_SCS) KVM=../../../../virt/kvm From patchwork Fri Oct 18 16:10:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 11199109 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D4A7714E5 for ; Fri, 18 Oct 2019 16:16:37 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AB740205C9 for ; Fri, 18 Oct 2019 16:16:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="tDJfPF2Z"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="VFUqOFiJ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AB740205C9 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:References:Mime-Version :Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=lyzhzfZS5RkBHKkPPpBJzoqVxlY1rO3HC0DSZFAhZ+Y=; b=tDJfPF2Z2whIU/ dAH8/5i0BQi515uDnQO4jRh6FnDm2R4TMjajc+SBr9GvyeMJ1UhKA0fkIKJFOgItpqznL7sVVO8r+ AXtkDLaWPl9HIMP6/Yjtd459sdNm1IZsFgUDIVyzmzls2YbNBn5yembXOUEXl1BHXFqXy7P5tbUeU geIglAliNQxqyOmh4OTShRhbq5UvldFVPSb/hb81O6nrf3hql/qcF9jwW+EDTUlaVWgszupPzIAib Xdy7L38StdAL9tG/vcPEdGyZ2gftv2ebIOJk7hOwxWki5i/o3pkJqs1494gB2Eh/Xsnh12RgIZX82 zNp4PafS7wPJhMqE4m1A==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUvc-00082Z-PU; Fri, 18 Oct 2019 16:16:36 +0000 Received: from mail-pl1-x649.google.com ([2607:f8b0:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1iLUqn-0002CQ-Rk for linux-arm-kernel@lists.infradead.org; Fri, 18 Oct 2019 16:11:40 +0000 Received: by mail-pl1-x649.google.com with SMTP id 70so4042412ple.1 for ; Fri, 18 Oct 2019 09:11:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=UZX+R+9PK3foZtlSwTu2VkSnILWNlGOO043crMQhF3g=; b=VFUqOFiJsw8yc592kxs4Pzo8P9U3Yx2XXWMNC7PhZfttZQ8Ysek8zr7CFrwKpxtycK ZmZk4WQwp1Ff3sMikiVcyerUQ7jLUUQHZZkhPeKvRie7cy5ipT4kvIsI8wO7Arzad4mG kQQJa3PoyhF1+jm9kkeVnk8yAj+xrtkYOG0tJLjfvzZpEj/hPf4D+JaWRoWhIztHQSu+ cCCspxnPdj3yKndmUXGahGhEjgw/s6GFXutDpjW30JdSsMACtJBEJ+im3u65G8KxK8lE tACfPxFPIIlv+s5hLPV/kM6CGeFj67hWREsy2WvQdaBFdsDuFIj9+E/B6iiJdgplf9tE PfBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=UZX+R+9PK3foZtlSwTu2VkSnILWNlGOO043crMQhF3g=; b=Ua1EZt/3HrBvxKAcZ+mobDjjZuq6eNbu1YNK62iOaRN+Y7xTSF2QieGrv87E7lu+zM Qv5lOBCAUWoiLL6j5ngcdUFFOXDCBVcUe85Ncep0a/Bc5FjM2kzEK6shgVkkWt3Gj77u +h0RV/iAxIPYYTPX/v4KCXh3I/hj1M6FRhltCTPsC00+vmsvdTMQ9//KiFsLpWQDsTeE h9hmYpWejxU90gjV0bnKfPvZsd4S05aMkmi7g2Ix0msp4Tar55AP+4zczane37bF+LrJ cSCg1wj9r6y5EJgKPWDqTlV5Y75FLaNKaSQctGf0bHsa1lXdT3R2g95eH5bBXaxiWbpG ON2g== X-Gm-Message-State: APjAAAXKLVapha0Xor9OceYgt3PMfEJS51AtPfi8A0fDcyqTEPcxSfML eI037klP/ryYT6zwlyaBttZYoLCkQ/TFBn9jwK4= X-Google-Smtp-Source: APXvYqzU5WSssPnVWIErUHwAgG3tKsgH9pwSKJwO1rvzO9stV/umbZ2JJfPFKvz1OfXlaF98CrqrDDplhH1eiJnUwBE= X-Received: by 2002:a63:78cc:: with SMTP id t195mr10900935pgc.304.1571415095479; Fri, 18 Oct 2019 09:11:35 -0700 (PDT) Date: Fri, 18 Oct 2019 09:10:33 -0700 In-Reply-To: <20191018161033.261971-1-samitolvanen@google.com> Message-Id: <20191018161033.261971-19-samitolvanen@google.com> Mime-Version: 1.0 References: <20191018161033.261971-1-samitolvanen@google.com> X-Mailer: git-send-email 2.23.0.866.gb869b98d4c-goog Subject: [PATCH 18/18] arm64: implement Shadow Call Stack From: Sami Tolvanen To: Will Deacon , Catalin Marinas , Steven Rostedt , Ard Biesheuvel X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20191018_091138_014254_A4C1B435 X-CRM114-Status: GOOD ( 17.91 ) X-Spam-Score: -7.7 (-------) X-Spam-Report: SpamAssassin version 3.4.2 on bombadil.infradead.org summary: Content analysis details: (-7.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:649 listed in] [list.dnswl.org] -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Medium sender X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Kees Cook , kernel-hardening@lists.openwall.com, Nick Desaulniers , linux-kernel@vger.kernel.org, clang-built-linux@googlegroups.com, Sami Tolvanen , Laura Abbott , Dave Martin , linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org This change implements shadow stack switching, initial SCS set-up, and interrupt shadow stacks for arm64. Signed-off-by: Sami Tolvanen --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/scs.h | 60 ++++++++++++++++++++++++++++ arch/arm64/include/asm/stacktrace.h | 4 ++ arch/arm64/include/asm/thread_info.h | 3 ++ arch/arm64/kernel/Makefile | 1 + arch/arm64/kernel/asm-offsets.c | 3 ++ arch/arm64/kernel/entry.S | 23 +++++++++++ arch/arm64/kernel/head.S | 9 +++++ arch/arm64/kernel/irq.c | 2 + arch/arm64/kernel/process.c | 3 ++ arch/arm64/kernel/scs.c | 39 ++++++++++++++++++ arch/arm64/kernel/smp.c | 4 ++ 12 files changed, 152 insertions(+) create mode 100644 arch/arm64/include/asm/scs.h create mode 100644 arch/arm64/kernel/scs.c diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 3f047afb982c..9bf179db5da9 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -66,6 +66,7 @@ config ARM64 select ARCH_USE_QUEUED_RWLOCKS select ARCH_USE_QUEUED_SPINLOCKS select ARCH_SUPPORTS_MEMORY_FAILURE + select ARCH_SUPPORTS_SHADOW_CALL_STACK select ARCH_SUPPORTS_ATOMIC_RMW select ARCH_SUPPORTS_INT128 if GCC_VERSION >= 50000 || CC_IS_CLANG select ARCH_SUPPORTS_NUMA_BALANCING diff --git a/arch/arm64/include/asm/scs.h b/arch/arm64/include/asm/scs.h new file mode 100644 index 000000000000..14ba192dc6f0 --- /dev/null +++ b/arch/arm64/include/asm/scs.h @@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_SCS_H +#define _ASM_SCS_H + +#ifndef __ASSEMBLY__ + +#include + +#ifdef CONFIG_SHADOW_CALL_STACK + +extern void scs_init_irq(void); + +static inline void scs_save(struct task_struct *tsk) +{ + void *s; + + asm volatile("mov %0, x18" : "=r" (s)); + task_set_scs(tsk, s); +} + +static inline void scs_load(struct task_struct *tsk) +{ + asm volatile("mov x18, %0" : : "r" (task_scs(tsk))); + task_set_scs(tsk, NULL); +} + +static inline void scs_thread_switch(struct task_struct *prev, + struct task_struct *next) +{ + scs_save(prev); + scs_load(next); + + if (unlikely(scs_corrupted(prev))) + panic("corrupted shadow stack detected inside scheduler\n"); +} + +#else /* CONFIG_SHADOW_CALL_STACK */ + +static inline void scs_init_irq(void) +{ +} + +static inline void scs_save(struct task_struct *tsk) +{ +} + +static inline void scs_load(struct task_struct *tsk) +{ +} + +static inline void scs_thread_switch(struct task_struct *prev, + struct task_struct *next) +{ +} + +#endif /* CONFIG_SHADOW_CALL_STACK */ + +#endif /* __ASSEMBLY __ */ + +#endif /* _ASM_SCS_H */ diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h index 4d9b1f48dc39..b6cf32fb4efe 100644 --- a/arch/arm64/include/asm/stacktrace.h +++ b/arch/arm64/include/asm/stacktrace.h @@ -68,6 +68,10 @@ extern void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk); DECLARE_PER_CPU(unsigned long *, irq_stack_ptr); +#ifdef CONFIG_SHADOW_CALL_STACK +DECLARE_PER_CPU(unsigned long *, irq_shadow_call_stack_ptr); +#endif + static inline bool on_irq_stack(unsigned long sp, struct stack_info *info) { diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h index f0cec4160136..8c73764b9ed2 100644 --- a/arch/arm64/include/asm/thread_info.h +++ b/arch/arm64/include/asm/thread_info.h @@ -41,6 +41,9 @@ struct thread_info { #endif } preempt; }; +#ifdef CONFIG_SHADOW_CALL_STACK + void *shadow_call_stack; +#endif }; #define thread_saved_pc(tsk) \ diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile index 478491f07b4f..b3995329d9e5 100644 --- a/arch/arm64/kernel/Makefile +++ b/arch/arm64/kernel/Makefile @@ -63,6 +63,7 @@ obj-$(CONFIG_CRASH_CORE) += crash_core.o obj-$(CONFIG_ARM_SDE_INTERFACE) += sdei.o obj-$(CONFIG_ARM64_SSBD) += ssbd.o obj-$(CONFIG_ARM64_PTR_AUTH) += pointer_auth.o +obj-$(CONFIG_SHADOW_CALL_STACK) += scs.o obj-y += vdso/ probes/ obj-$(CONFIG_COMPAT_VDSO) += vdso32/ diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c index 214685760e1c..f6762b9ae1e1 100644 --- a/arch/arm64/kernel/asm-offsets.c +++ b/arch/arm64/kernel/asm-offsets.c @@ -33,6 +33,9 @@ int main(void) DEFINE(TSK_TI_ADDR_LIMIT, offsetof(struct task_struct, thread_info.addr_limit)); #ifdef CONFIG_ARM64_SW_TTBR0_PAN DEFINE(TSK_TI_TTBR0, offsetof(struct task_struct, thread_info.ttbr0)); +#endif +#ifdef CONFIG_SHADOW_CALL_STACK + DEFINE(TSK_TI_SCS, offsetof(struct task_struct, thread_info.shadow_call_stack)); #endif DEFINE(TSK_STACK, offsetof(struct task_struct, stack)); #ifdef CONFIG_STACKPROTECTOR diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index cf3bd2976e57..ca49938b99d0 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -172,6 +172,10 @@ alternative_cb_end apply_ssbd 1, x22, x23 +#ifdef CONFIG_SHADOW_CALL_STACK + ldr x18, [tsk, #TSK_TI_SCS] // Restore shadow call stack + str xzr, [tsk, #TSK_TI_SCS] +#endif .else add x21, sp, #S_FRAME_SIZE get_current_task tsk @@ -278,6 +282,12 @@ alternative_else_nop_endif ct_user_enter .endif +#ifdef CONFIG_SHADOW_CALL_STACK + .if \el == 0 + str x18, [tsk, #TSK_TI_SCS] // Save shadow call stack + .endif +#endif + #ifdef CONFIG_ARM64_SW_TTBR0_PAN /* * Restore access to TTBR0_EL1. If returning to EL0, no need for SPSR @@ -383,6 +393,9 @@ alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0 .macro irq_stack_entry mov x19, sp // preserve the original sp +#ifdef CONFIG_SHADOW_CALL_STACK + mov x20, x18 // preserve the original shadow stack +#endif /* * Compare sp with the base of the task stack. @@ -400,6 +413,12 @@ alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0 /* switch to the irq stack */ mov sp, x26 + +#ifdef CONFIG_SHADOW_CALL_STACK + /* also switch to the irq shadow stack */ + ldr_this_cpu x18, irq_shadow_call_stack_ptr, x26 +#endif + 9998: .endm @@ -409,6 +428,10 @@ alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0 */ .macro irq_stack_exit mov sp, x19 +#ifdef CONFIG_SHADOW_CALL_STACK + /* x20 is also preserved */ + mov x18, x20 +#endif .endm /* GPRs used by entry code */ diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 989b1944cb71..2be977c6496f 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -27,6 +27,7 @@ #include #include #include +#include #include #include #include @@ -424,6 +425,10 @@ __primary_switched: stp xzr, x30, [sp, #-16]! mov x29, sp +#ifdef CONFIG_SHADOW_CALL_STACK + adr_l x18, init_shadow_call_stack // Set shadow call stack +#endif + str_l x21, __fdt_pointer, x5 // Save FDT pointer ldr_l x4, kimage_vaddr // Save the offset between @@ -731,6 +736,10 @@ __secondary_switched: ldr x2, [x0, #CPU_BOOT_TASK] cbz x2, __secondary_too_slow msr sp_el0, x2 +#ifdef CONFIG_SHADOW_CALL_STACK + ldr x18, [x2, #TSK_TI_SCS] // Set shadow call stack + str xzr, [x2, #TSK_TI_SCS] +#endif mov x29, #0 mov x30, #0 b secondary_start_kernel diff --git a/arch/arm64/kernel/irq.c b/arch/arm64/kernel/irq.c index 04a327ccf84d..fe0ca522ff60 100644 --- a/arch/arm64/kernel/irq.c +++ b/arch/arm64/kernel/irq.c @@ -21,6 +21,7 @@ #include #include #include +#include unsigned long irq_err_count; @@ -63,6 +64,7 @@ static void init_irq_stacks(void) void __init init_IRQ(void) { init_irq_stacks(); + scs_init_irq(); irqchip_init(); if (!handle_arch_irq) panic("No interrupt controller found."); diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c index 71f788cd2b18..4490632047d6 100644 --- a/arch/arm64/kernel/process.c +++ b/arch/arm64/kernel/process.c @@ -52,6 +52,7 @@ #include #include #include +#include #include #if defined(CONFIG_STACKPROTECTOR) && !defined(CONFIG_STACKPROTECTOR_PER_TASK) @@ -508,6 +509,8 @@ __notrace_funcgraph struct task_struct *__switch_to(struct task_struct *prev, ptrauth_thread_switch(next); ssbs_thread_switch(next); + scs_thread_switch(prev, next); + /* * Complete any pending TLB or cache maintenance on this CPU in case * the thread migrates to a different CPU. diff --git a/arch/arm64/kernel/scs.c b/arch/arm64/kernel/scs.c new file mode 100644 index 000000000000..6f255072c9a9 --- /dev/null +++ b/arch/arm64/kernel/scs.c @@ -0,0 +1,39 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Shadow Call Stack support. + * + * Copyright (C) 2019 Google LLC + */ + +#include +#include +#include + +DEFINE_PER_CPU(unsigned long *, irq_shadow_call_stack_ptr); + +#ifndef CONFIG_SHADOW_CALL_STACK_VMAP +DEFINE_PER_CPU(unsigned long [SCS_SIZE/sizeof(long)], irq_shadow_call_stack) + __aligned(SCS_SIZE); +#endif + +void scs_init_irq(void) +{ + int cpu; + + for_each_possible_cpu(cpu) { +#ifdef CONFIG_SHADOW_CALL_STACK_VMAP + unsigned long *p; + + p = __vmalloc_node_range(SCS_SIZE, SCS_SIZE, + VMALLOC_START, VMALLOC_END, + SCS_GFP, PAGE_KERNEL, + 0, cpu_to_node(cpu), + __builtin_return_address(0)); + + per_cpu(irq_shadow_call_stack_ptr, cpu) = p; +#else + per_cpu(irq_shadow_call_stack_ptr, cpu) = + per_cpu(irq_shadow_call_stack, cpu); +#endif /* CONFIG_SHADOW_CALL_STACK_VMAP */ + } +} diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index dc9fe879c279..cc1938a585d2 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -44,6 +44,7 @@ #include #include #include +#include #include #include #include @@ -357,6 +358,9 @@ void cpu_die(void) { unsigned int cpu = smp_processor_id(); + /* Save the shadow stack pointer before exiting the idle task */ + scs_save(current); + idle_task_exit(); local_daif_mask();