From patchwork Thu Apr 1 23:23:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 12179833 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-21.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4EE61C433B4 for ; Thu, 1 Apr 2021 23:26:33 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ECDA961107 for ; Thu, 1 Apr 2021 23:26:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ECDA961107 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=LkgS3CY9Q4oEb/64XiSFlIzSgogvN/68nhTe8LDmGgM=; b=rKWy62Hp2lfq5M1Az8xqDCBTM UAlaANKHQla43fizyWbt6Rr5MfKEbdkTPU3JRG+7bZjaL4gDzjuQFMUw3EBstOc7UdunE5YMKzcf8 mecgcc5vb2rvm4WP0NZwUb2Ue+suX0Ub6dbvm/sg1W/ZaQiYWibF+IcftEKgI0fmrdPL46PM2WdC/ lq4mWME8lBOGNrByzhWF8E5cLoqfxxFSI5/DBjWbYVAIzV63U78tvCsQhwV4ZW3/d7Yp5oOzOOwvF ZRBH3Wih9L7wU+Eh9tkkYm2MmgFtUzg9QNMeH7PdZwW6SyDGkxelX7LVJgLBpg2f2G2DCMiVh/iEz HnWBStmBg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lS6fw-00BJzX-JS; Thu, 01 Apr 2021 23:24:32 +0000 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lS6fK-00BJnw-NJ for linux-arm-kernel@lists.infradead.org; Thu, 01 Apr 2021 23:24:01 +0000 Received: by mail-pf1-x429.google.com with SMTP id j25so2526850pfe.2 for ; Thu, 01 Apr 2021 16:23:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9ijlYcKqhmRnIjh3GfaZu6VzSGgGwccaBk76YZfh88U=; b=WKcAnIleO2ypEZLKdHh37aVTO4tnIydkC5OCyMTVr+JpikHZ2/ZMU3FugvxEly10iU XSYN4IoCMfmYa9AiUhoVcjinurr136nmz9lf7q3gbfnnfDCOVEDSAGVTeituYbklhN9Y yq+0k7ScK7oeFnBPFOBIMf0UvJV0M6vj4GtgI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9ijlYcKqhmRnIjh3GfaZu6VzSGgGwccaBk76YZfh88U=; b=J/CHIpCy9nhlyhgGd/UDdF6rj+bx5iJDgXFgUPOab1T+3nG6jmZQgtCjF+QrZFchvS hR23Xei6beAK3irfR82dgSXMYrIalAg0O8Bsl8iZSgrXyn7GxHxLrEevKH0egemFDf3O Vs5mtWwCPQkMGrfJKEW1tY7MwAoyftbRVqoc2n4nP1CDtYYKKTdQbT3E+ZinkFXemLWf o34QKDc1yQHfIIPKoLqliqhPeUUxi2J16wIIq1NntsxCYAgpOWSMxfLT+SpkoDq1Zecg OnmnNNX7ogKCLy9KaVJu67GPhogToTEjBWpJzoDrYuQREpdm/0ENpj5WZCuzByZcE/Ss U57g== X-Gm-Message-State: AOAM530sGhh/EHpg2m6mQVxKYxmP8svYjIbindilzgTj87mJidw0kP/2 nau4p+ofEy2Ft0kxUb2WLnwDpg== X-Google-Smtp-Source: ABdhPJwcYRMQryh7JJASN23+Q3HQF6uWuabr3WsRHjWCDIRpgK6DJhS0r2tl16IIlvy5t61K9IC1DA== X-Received: by 2002:a62:7b83:0:b029:1f1:5ef3:b4d9 with SMTP id w125-20020a627b830000b02901f15ef3b4d9mr9608204pfc.65.1617319432493; Thu, 01 Apr 2021 16:23:52 -0700 (PDT) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id k64sm6606207pgk.23.2021.04.01.16.23.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Apr 2021 16:23:51 -0700 (PDT) From: Kees Cook To: Thomas Gleixner Cc: Kees Cook , Peter Zijlstra , Elena Reshetova , x86@kernel.org, Andy Lutomirski , Catalin Marinas , Will Deacon , Mark Rutland , Alexander Potapenko , Alexander Popov , Ard Biesheuvel , Jann Horn , Vlastimil Babka , David Hildenbrand , Mike Rapoport , Andrew Morton , Jonathan Corbet , Randy Dunlap , kernel-hardening@lists.openwall.com, linux-hardening@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v10 1/6] jump_label: Provide CONFIG-driven build state defaults Date: Thu, 1 Apr 2021 16:23:42 -0700 Message-Id: <20210401232347.2791257-2-keescook@chromium.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210401232347.2791257-1-keescook@chromium.org> References: <20210401232347.2791257-1-keescook@chromium.org> MIME-Version: 1.0 X-Patch-Hashes: v=1; h=sha256; g=aa54c44d1d71b9550d6015efc734f667917094a1; i=Vishx6UyAXwYzcnoSyP+eBB3iQyx+/i5smsbQfc0cnA=; m=vc4sSYlf+uaSlLSFP5TpbQv56VaSRpBLpRuMltXaB4Q=; p=mNH2Bo/K9vrGz9sBtTDV8UFO0eJ8yv8BbR/DeIaO1es= X-Patch-Sig: m=pgp; i=keescook@chromium.org; s=0x0x8972F4DFDC6DC026; b=iQIzBAABCgAdFiEEpcP2jyKd1g9yPm4TiXL039xtwCYFAmBmVgEACgkQiXL039xtwCbiEBAArOm /EtpeLQvJQ16Bi0gBHOl/CblUQyhrscVrs8xNm/lKfSaeaGIDJaNfx0EZ4L7n2IVkpGYSodLxZlwj ve4GueMybGaAPJWYFy3L+jAe9eYTp3zyYwvQsSRg55iW5msvqK3ZUeJqC/CxPrDdwT05noLV9mDub lAzrVY/pLiXE0HTK9ogAdyiNFQY2yA44UmGTIkmgZ30jl0d4aSVtwsViZeMS8pCrK22xkiccq/bRY 72ASLPxyOBOL00UWKeUmbmdUfNW+BkNlK7IaN+w5QBfiKuQWTf/WdZ+8c5tQWjLE3kJ4DRx8mrBcy 1vYJ1m1ZKfAOe8uV8waSURiUMzfgB/uL5on3W1e6s8SkChf0KGAZMyDGuIflDnvFUy8OIfM8Ak/fg hoYQ2dUXkKa50EdvadntVka+XmxokcSERzN0e9tOQlYfeLiL5TINkkDQFAallIMnHy6ImGd9Yd23F zSvCmvv0683j96kJXsEcfl0wH3LYYCyRS0fIK1ssPxPAViuN0vY1jlIeDn734NiR4+itDg8pTepIM h8ozYc9GPgYBTzXQPG29KmXRHc4es7o4enMoBQ1Gj0PZ5f9c0fr6AUpoe7c1jVtsL4wJD6tUuC9do T40yZwF6fDGX1jsVc/mbC1ug6gurwbG+Xo1xiQ2I1YUyr1MimqrfE4LvelISHwZM= X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210402_002354_962528_F64C58BF X-CRM114-Status: GOOD ( 15.25 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org As shown in jump_label.h[1], choosing the initial state of static branches changes the assembly layout. If the condition is expected to be likely it's inline, and if unlikely it is out of line via a jump. A few places in the kernel use (or could be using) a CONFIG to choose the default state, which would give a small performance benefit to their compile-time declared default. Provide the infrastructure to do this. [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/linux/jump_label.h?h=v5.11#n398 Acked-by: Peter Zijlstra (Intel) Link: https://lore.kernel.org/lkml/20200324220641.GT2452@worktop.programming.kicks-ass.net/ Signed-off-by: Kees Cook --- include/linux/jump_label.h | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/include/linux/jump_label.h b/include/linux/jump_label.h index d92691262f51..05f5554d860f 100644 --- a/include/linux/jump_label.h +++ b/include/linux/jump_label.h @@ -382,6 +382,21 @@ struct static_key_false { [0 ... (count) - 1] = STATIC_KEY_FALSE_INIT, \ } +#define _DEFINE_STATIC_KEY_1(name) DEFINE_STATIC_KEY_TRUE(name) +#define _DEFINE_STATIC_KEY_0(name) DEFINE_STATIC_KEY_FALSE(name) +#define DEFINE_STATIC_KEY_MAYBE(cfg, name) \ + __PASTE(_DEFINE_STATIC_KEY_, IS_ENABLED(cfg))(name) + +#define _DEFINE_STATIC_KEY_RO_1(name) DEFINE_STATIC_KEY_TRUE_RO(name) +#define _DEFINE_STATIC_KEY_RO_0(name) DEFINE_STATIC_KEY_FALSE_RO(name) +#define DEFINE_STATIC_KEY_MAYBE_RO(cfg, name) \ + __PASTE(_DEFINE_STATIC_KEY_RO_, IS_ENABLED(cfg))(name) + +#define _DECLARE_STATIC_KEY_1(name) DECLARE_STATIC_KEY_TRUE(name) +#define _DECLARE_STATIC_KEY_0(name) DECLARE_STATIC_KEY_FALSE(name) +#define DECLARE_STATIC_KEY_MAYBE(cfg, name) \ + __PASTE(_DECLARE_STATIC_KEY_, IS_ENABLED(cfg))(name) + extern bool ____wrong_branch_error(void); #define static_key_enabled(x) \ @@ -482,6 +497,10 @@ extern bool ____wrong_branch_error(void); #endif /* CONFIG_JUMP_LABEL */ +#define static_branch_maybe(config, x) \ + (IS_ENABLED(config) ? static_branch_likely(x) \ + : static_branch_unlikely(x)) + /* * Advanced usage; refcount, branch is enabled when: count != 0 */ From patchwork Thu Apr 1 23:23:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 12179839 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C061C43461 for ; Thu, 1 Apr 2021 23:26:35 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F3E8A61105 for ; Thu, 1 Apr 2021 23:26:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F3E8A61105 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=LZEkEXMfB/Y8sKtYO68vHvaHZeKZFfscaamn1tgUX0E=; b=Vl1NkvXU9PUnwX/X1W/vnOIgF 0Rd8ydQQTSV5nH7F1T3Sk7o5hvhv0u1Cv4UqXYy36lZ2DZPB6qh0XDVLZRC4shifm7rO066Gk+s0J 8AkVw52/jHjwn8/krOo+WkKeXyORgGgMmLPa3CSsp7ClzBoBX0tY1OYNl0GnD8PjDCCeo66n/dRKG wwwIzWncLlRkXCWahy1JpOf2Y/nCgdiYD8G8XokAwTo8aVUuFuVKZ/pGMD/TzhF/+ZbiF7pWIYO2K W6hkMOSwMEII/Hf9Oq6vqcyhAkPmWAWByCBWTdReQtwVtkVom2Hc1lFD64d87oMOipaskoYr7f908 ews992CpA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lS6fn-00BJyr-1y; Thu, 01 Apr 2021 23:24:23 +0000 Received: from mail-pg1-x533.google.com ([2607:f8b0:4864:20::533]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lS6fK-00BJny-N4 for linux-arm-kernel@lists.infradead.org; Thu, 01 Apr 2021 23:23:59 +0000 Received: by mail-pg1-x533.google.com with SMTP id v186so2529746pgv.7 for ; Thu, 01 Apr 2021 16:23:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UG6lpZSAG8t+NfdJici9YkMGF34rxaYTMGPEmVDXXFs=; b=l8S/+ejU94GwXqXJSUyS/eNkcpIc7xdvequeiDmmOj+tYzryNYiSUA6JrVp487CYR2 tnSLdnlF6KzYbHc3HKnMPnO9xgUvAqIc/jkuucyWMc8TbQJbDc6UnjLbWhBVqiQGOha+ aOnrezt7IVhuXE6XI/3E0r3vDeG8jpQfJ2fhs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UG6lpZSAG8t+NfdJici9YkMGF34rxaYTMGPEmVDXXFs=; b=OUEODBn00MibWCC5eh1P9+Uox9LHi1hLidYKJecPvOLr9b0Me1DBornBX/XJTVYQaI 1ykfH5xHk1wSOINZlxRAhEF/jsltWsbOrRQY90u7hf+nbupUXaHPZ2Lzuaq/Xi/wVefl 2x0taD5Wrukl6mvG2l7pPaEyN40wK+VOMVy8ybBxzE9bLISvMi2WBvlHaPC4Q4cIiiOw 2Oeuc8OjTrwYQb0FSd7yQR1Tz00jc7456eHXuD3m8++d9Y/xPUHhzw5y7GJ0/qminKTx iTvsbx2M0ujmPuLprPtHRhAWYhs68Sq+OlK0Tvn5I+FJEV4BGcXtkcsRdkKRYLa1Fqx4 kufg== X-Gm-Message-State: AOAM530yMSLUxizuWrQuJYFjt2pysLZspEE1N3bMlcwuwwp7zcKcvH4y Swc5ClaoEGTbaHfgGszPE7UbjA== X-Google-Smtp-Source: ABdhPJzFCy8btNup6SOnGlmYqdVp8w6Awwl6ipqVL9OM241/47URJI/UqOO3mkuVCV70DR9cLI8gaw== X-Received: by 2002:a63:650:: with SMTP id 77mr9469602pgg.190.1617319433279; Thu, 01 Apr 2021 16:23:53 -0700 (PDT) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id n10sm6363653pjo.15.2021.04.01.16.23.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Apr 2021 16:23:51 -0700 (PDT) From: Kees Cook To: Thomas Gleixner Cc: Kees Cook , Alexander Potapenko , Vlastimil Babka , Elena Reshetova , x86@kernel.org, Andy Lutomirski , Peter Zijlstra , Catalin Marinas , Will Deacon , Mark Rutland , Alexander Popov , Ard Biesheuvel , Jann Horn , David Hildenbrand , Mike Rapoport , Andrew Morton , Jonathan Corbet , Randy Dunlap , kernel-hardening@lists.openwall.com, linux-hardening@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v10 2/6] init_on_alloc: Optimize static branches Date: Thu, 1 Apr 2021 16:23:43 -0700 Message-Id: <20210401232347.2791257-3-keescook@chromium.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210401232347.2791257-1-keescook@chromium.org> References: <20210401232347.2791257-1-keescook@chromium.org> MIME-Version: 1.0 X-Patch-Hashes: v=1; h=sha256; g=2f4b59272e09d0180c87e3d8378d95ea375990ec; i=7VrM8Pzr5MGi8vAaDOW4xeiDtVYbjHgJsdLlT/sxDXE=; m=OLLKLAHsfNT1d7+pflxMGJuAvSt8GZaetKIXzfk0ilc=; p=cKq4RSNZSRZ+ASTC07w37jURMQB1hxGsdgqU07adDH0= X-Patch-Sig: m=pgp; i=keescook@chromium.org; s=0x0x8972F4DFDC6DC026; b=iQIzBAABCgAdFiEEpcP2jyKd1g9yPm4TiXL039xtwCYFAmBmVgIACgkQiXL039xtwCaolw//ZCB Logxo6dDfC7433hgQdOfpHxCZxm/7rRC+f4joWq57/QBmOMjF3W073sD+R9vdDDCGsEiNo12ISjVa G+W/a9MmfBRhcDv4JpBDlh+Dw2dFA6Ix4+3Q8R9LCG2x8jUfAyp1yii+WOrGDhliFvL3Pd+RMa9W0 CnwK2yvrB108eSO8Ryikc91HMJnXqE5c4Lp1lW2CJhX2dqrqXZKMNp4o8bjv7UdsSEu/m9mJrpCho 6MeQreVX2J2AcCceALV2gngk2JdeOaO7oHnMm6Bi9NSFB2y6ZP5NK9PVE+28bRCgklWdAxJny2Biz Q+6qRteMMChf4RkN668cPz6+bAzfxX5UIxlU+SK3f0EnDFZFVBy62XJ+j2LoGUyMEU1m3ckvWOhxg Sh2NFhQF++EM5Dp6JmARfwgu1T0w/3Tr8AYh8sC31bdT2rLCwF24VQ+IMnewePUGopvjPDmww+t7X /I1dZST8HC21YoNAddrZtWfXnGe2BZr/j2QXHLUQRRBrw8ey4mj//A/iFrKjTOVftnP1PKOvIaEHP BjsAJO5EAzUSYilh+l8IxTyO/3ek103mtxJs+8/LlJwngI+OmtRSal6hnbv0deFcHZtYYWIi69MNG y3sfIi8Kj3wUjpeBziLMxM+cTUdsKW/aHPjRiOVT6W4tPmNgpwiG7OG3vLnPz98I= X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210402_002354_921803_FAF3E11C X-CRM114-Status: GOOD ( 16.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The state of CONFIG_INIT_ON_ALLOC_DEFAULT_ON (and ...ON_FREE...) did not change the assembly ordering of the static branches: they were always out of line. Use the new jump_label macros to check the CONFIG settings to default to the "expected" state, which slightly optimizes the resulting assembly code. Reviewed-by: Alexander Potapenko Link: https://lore.kernel.org/lkml/CAG_fn=X0DVwqLaHJTO6Jw7TGcMSm77GKHinrd0m_6y0SzWOrFA@mail.gmail.com/ Acked-by: Vlastimil Babka Link: https://lore.kernel.org/lkml/5d626b9b-5355-be94-e8e2-1be47f880f30@suse.cz Signed-off-by: Kees Cook --- include/linux/mm.h | 10 ++++++---- mm/page_alloc.c | 4 ++-- mm/slab.h | 6 ++++-- 3 files changed, 12 insertions(+), 8 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 77e64e3eac80..2ccd856ac0d1 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2871,18 +2871,20 @@ static inline void kernel_poison_pages(struct page *page, int numpages) { } static inline void kernel_unpoison_pages(struct page *page, int numpages) { } #endif -DECLARE_STATIC_KEY_FALSE(init_on_alloc); +DECLARE_STATIC_KEY_MAYBE(CONFIG_INIT_ON_ALLOC_DEFAULT_ON, init_on_alloc); static inline bool want_init_on_alloc(gfp_t flags) { - if (static_branch_unlikely(&init_on_alloc)) + if (static_branch_maybe(CONFIG_INIT_ON_ALLOC_DEFAULT_ON, + &init_on_alloc)) return true; return flags & __GFP_ZERO; } -DECLARE_STATIC_KEY_FALSE(init_on_free); +DECLARE_STATIC_KEY_MAYBE(CONFIG_INIT_ON_FREE_DEFAULT_ON, init_on_free); static inline bool want_init_on_free(void) { - return static_branch_unlikely(&init_on_free); + return static_branch_maybe(CONFIG_INIT_ON_FREE_DEFAULT_ON, + &init_on_free); } extern bool _debug_pagealloc_enabled_early; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 3e4b29ee2b1e..267c04b8911d 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -167,10 +167,10 @@ unsigned long totalcma_pages __read_mostly; int percpu_pagelist_fraction; gfp_t gfp_allowed_mask __read_mostly = GFP_BOOT_MASK; -DEFINE_STATIC_KEY_FALSE(init_on_alloc); +DEFINE_STATIC_KEY_MAYBE(CONFIG_INIT_ON_ALLOC_DEFAULT_ON, init_on_alloc); EXPORT_SYMBOL(init_on_alloc); -DEFINE_STATIC_KEY_FALSE(init_on_free); +DEFINE_STATIC_KEY_MAYBE(CONFIG_INIT_ON_FREE_DEFAULT_ON, init_on_free); EXPORT_SYMBOL(init_on_free); static bool _init_on_alloc_enabled_early __read_mostly diff --git a/mm/slab.h b/mm/slab.h index 076582f58f68..774c7221efdc 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -601,7 +601,8 @@ static inline void cache_random_seq_destroy(struct kmem_cache *cachep) { } static inline bool slab_want_init_on_alloc(gfp_t flags, struct kmem_cache *c) { - if (static_branch_unlikely(&init_on_alloc)) { + if (static_branch_maybe(CONFIG_INIT_ON_ALLOC_DEFAULT_ON, + &init_on_alloc)) { if (c->ctor) return false; if (c->flags & (SLAB_TYPESAFE_BY_RCU | SLAB_POISON)) @@ -613,7 +614,8 @@ static inline bool slab_want_init_on_alloc(gfp_t flags, struct kmem_cache *c) static inline bool slab_want_init_on_free(struct kmem_cache *c) { - if (static_branch_unlikely(&init_on_free)) + if (static_branch_maybe(CONFIG_INIT_ON_FREE_DEFAULT_ON, + &init_on_free)) return !(c->ctor || (c->flags & (SLAB_TYPESAFE_BY_RCU | SLAB_POISON))); return false; From patchwork Thu Apr 1 23:23:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 12179827 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E8C6BC433ED for ; Thu, 1 Apr 2021 23:26:20 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5624F61105 for ; Thu, 1 Apr 2021 23:26:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5624F61105 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=vT4CAsx0rUVQ3l2dhCAZUmJO62+YUrf16436Rt8QkHs=; b=Ko8d6LSkUN0S+Sn40cUbsTQlB XRYT3FxP47iuyE9dKviy3SfkGLiva6Wat8MRaRMvb3fAZQpvbHV+p9GdkH3/3x0/oLeqFGi+EJLqs DqjMq2C7IEYHPEET5gFNND+GM+TRUxXKzJiAvRh4+3UqgQ9MvDanmCJIipZuhZzZobJ5Ot5ajlk4g I2Whte//imxhOsbeDO76X6298BrjFoeH8zkZRPxf+ccrxt2A5ooPcNrHVl0NdaUDzSIj4PrzSLORO O4NhfYsW0a5E2QzkxBtYZvQtbQwbx9z4PHsGJ9ngprwCteFcjaJBMulIA+S1qwnTPvgtlPavtGuJQ QYcCy5rcA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lS6g3-00BK0a-ET; Thu, 01 Apr 2021 23:24:39 +0000 Received: from mail-pj1-x1034.google.com ([2607:f8b0:4864:20::1034]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lS6fL-00BJpo-GI for linux-arm-kernel@lists.infradead.org; Thu, 01 Apr 2021 23:24:01 +0000 Received: by mail-pj1-x1034.google.com with SMTP id ha17so1925744pjb.2 for ; Thu, 01 Apr 2021 16:23:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VrIFQNf41FN9f1NB7aAm3sQMswETAMmqXbyRjaq9+6s=; b=Q/Ul3aNjSpg/X2onwLu+M5YHAUHpsGWSTx884oFFZFLVTtso2FOV/u2BarBzPtXc/n 6cOEgFnIdc7fyzE42/b6PkBXSaQmPHwpidApASA0YGfVokEQxfYJDI+ythDt2IUi9TyI x8TTpXC02Gz2vIxER7aGvlCuo/Ak50h+hVWOQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VrIFQNf41FN9f1NB7aAm3sQMswETAMmqXbyRjaq9+6s=; b=pmx/MWZSexnJVzPrA4an3W7kWPjLZ1yt3Bxz5BknC12AKWCyTZ2BtObv2ARa747yZK Qz+S0HreLOliPXSpleFNG1VPD4fyD4TRavnNRG6Oc1gEl1ZZXtFK6TSmzNNLuOCb5362 B79a6FMfcJKGoiYHYCQEj+MV5CBoFB/QOMBNPqACWC/TrNzfC8Waym/+kCAJk0lmRhGu hBZyg3wTNXvX47bIuRn6s1R+WSg9AQJJnEMobpId/R8Q6oH89FnRlv4VWrfU2okHzUi3 TBKbVx203EzNh7Pd4+YbHFI5Qor/wZWI7RuO9mck4+nNOwR9uVKqScffTuWrT4/Yga98 JVmA== X-Gm-Message-State: AOAM530orVZ1u8dlOWCD8LHYk5KxNsRdfZrNqw2G2r+YcorGZdC1cD8T J7MCZPEw7MwCBwAsQGWGbnqKGA== X-Google-Smtp-Source: ABdhPJxoDHhzdLSWx7IrG2Ljv3DyEPlFEawximwY4/8rCt8VGmvuEoSEJtx7ywLw4gTTZEr4FAVh9Q== X-Received: by 2002:a17:902:7786:b029:e6:cc0f:4dff with SMTP id o6-20020a1709027786b02900e6cc0f4dffmr10184598pll.4.1617319433842; Thu, 01 Apr 2021 16:23:53 -0700 (PDT) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id a29sm6576630pfg.130.2021.04.01.16.23.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Apr 2021 16:23:51 -0700 (PDT) From: Kees Cook To: Thomas Gleixner Cc: Kees Cook , Elena Reshetova , x86@kernel.org, Andy Lutomirski , Peter Zijlstra , Catalin Marinas , Will Deacon , Mark Rutland , Alexander Potapenko , Alexander Popov , Ard Biesheuvel , Jann Horn , Vlastimil Babka , David Hildenbrand , Mike Rapoport , Andrew Morton , Jonathan Corbet , Randy Dunlap , kernel-hardening@lists.openwall.com, linux-hardening@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v10 3/6] stack: Optionally randomize kernel stack offset each syscall Date: Thu, 1 Apr 2021 16:23:44 -0700 Message-Id: <20210401232347.2791257-4-keescook@chromium.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210401232347.2791257-1-keescook@chromium.org> References: <20210401232347.2791257-1-keescook@chromium.org> MIME-Version: 1.0 X-Patch-Hashes: v=1; h=sha256; g=95dfd6ddc3558d43f34396ffe6ddabee0386809e; i=R0fcIKmEUh8zot7Dpk8JcJdIeXVfXkVsg5oQI7Df8iY=; m=DpAISBt1P8rBqdsbU5gorkSdKBigCKStgTSYV1Mo6JQ=; p=37Nu50r180SuP1Jf6ejmMiVy5AZ/lJziO8nzlUuXMn8= X-Patch-Sig: m=pgp; i=keescook@chromium.org; s=0x0x8972F4DFDC6DC026; b=iQIzBAABCgAdFiEEpcP2jyKd1g9yPm4TiXL039xtwCYFAmBmVgIACgkQiXL039xtwCb1wRAAqxm 7jEKKzsVpUqvSqYeDc5bKfukhlJq398lS/MxFEVVNyDfq2eqt0H6eajP/IirQTgi4APTK4QKKo+hq k4eAlLKfZuL/XOGFY9KhUilEgRfXJsi7OPQEm70UmYZACuTNdv6JxBFnkeeY07VqTG3L+DXzJNoaH 4iVCN6woRUCoeMgdESuZcC66utoJLyyd85UQf8guxVWZ/qCkVbVowgbCuNKwF65wneNK2OEPW34WE LJoPZ4jG2zvt3tcXwYztspyBcqwsXccT6uBDtjE6f9WVLPiochwiTOFthU7sDcAv+7MBxmBehXCVa pfdmrbbX9E/BPWqrXPo6lzk6/x0HjRycRW+YcS0ya+U67k3yqXXD1/adc0q5DM+snoSHQ+DuxOn9Q Z52EmTz6s8nCyCo1VPFTfnFExnGrN8nA6HYxZmx5PBgWaj0itN7hgPvLW48PRJyXPYbFTPMZcoSt5 n14guPfGtWobK5uNrU8foXTDKqe/Hm7E2DN0oQNF8w094S6BYIzeqwxSZZ/a/2b2qWTjCcHTixnsV cc7sDlKqrrRwmmXUXdmzLpNXWn166Q1aNTrp1L38eu/GLU0kFdALYkJZIa4GvpPrqcfLb0hU5TITR Tg3NLdCCvDZ36mudKjqEDKZk0REuAGHlwIlMv0rIGJOThE/RU3VmUWQ2/IvLL1OY= X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210402_002356_753524_76FA2761 X-CRM114-Status: GOOD ( 39.20 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This provides the ability for architectures to enable kernel stack base address offset randomization. This feature is controlled by the boot param "randomize_kstack_offset=on/off", with its default value set by CONFIG_RANDOMIZE_KSTACK_OFFSET_DEFAULT. This feature is based on the original idea from the last public release of PaX's RANDKSTACK feature: https://pax.grsecurity.net/docs/randkstack.txt All the credit for the original idea goes to the PaX team. Note that the design and implementation of this upstream randomize_kstack_offset feature differs greatly from the RANDKSTACK feature (see below). Reasoning for the feature: This feature aims to make harder the various stack-based attacks that rely on deterministic stack structure. We have had many such attacks in past (just to name few): https://jon.oberheide.org/files/infiltrate12-thestackisback.pdf https://jon.oberheide.org/files/stackjacking-infiltrate11.pdf https://googleprojectzero.blogspot.com/2016/06/exploiting-recursion-in-linux-kernel_20.html As Linux kernel stack protections have been constantly improving (vmap-based stack allocation with guard pages, removal of thread_info, STACKLEAK), attackers have had to find new ways for their exploits to work. They have done so, continuing to rely on the kernel's stack determinism, in situations where VMAP_STACK and THREAD_INFO_IN_TASK_STRUCT were not relevant. For example, the following recent attacks would have been hampered if the stack offset was non-deterministic between syscalls: https://repositorio-aberto.up.pt/bitstream/10216/125357/2/374717.pdf (page 70: targeting the pt_regs copy with linear stack overflow) https://a13xp0p0v.github.io/2020/02/15/CVE-2019-18683.html (leaked stack address from one syscall as a target during next syscall) The main idea is that since the stack offset is randomized on each system call, it is harder for an attack to reliably land in any particular place on the thread stack, even with address exposures, as the stack base will change on the next syscall. Also, since randomization is performed after placing pt_regs, the ptrace-based approach[1] to discover the randomized offset during a long-running syscall should not be possible. Design description: During most of the kernel's execution, it runs on the "thread stack", which is pretty deterministic in its structure: it is fixed in size, and on every entry from userspace to kernel on a syscall the thread stack starts construction from an address fetched from the per-cpu cpu_current_top_of_stack variable. The first element to be pushed to the thread stack is the pt_regs struct that stores all required CPU registers and syscall parameters. Finally the specific syscall function is called, with the stack being used as the kernel executes the resulting request. The goal of randomize_kstack_offset feature is to add a random offset after the pt_regs has been pushed to the stack and before the rest of the thread stack is used during the syscall processing, and to change it every time a process issues a syscall. The source of randomness is currently architecture-defined (but x86 is using the low byte of rdtsc()). Future improvements for different entropy sources is possible, but out of scope for this patch. Further more, to add more unpredictability, new offsets are chosen at the end of syscalls (the timing of which should be less easy to measure from userspace than at syscall entry time), and stored in a per-CPU variable, so that the life of the value does not stay explicitly tied to a single task. As suggested by Andy Lutomirski, the offset is added using alloca() and an empty asm() statement with an output constraint, since it avoids changes to assembly syscall entry code, to the unwinder, and provides correct stack alignment as defined by the compiler. In order to make this available by default with zero performance impact for those that don't want it, it is boot-time selectable with static branches. This way, if the overhead is not wanted, it can just be left turned off with no performance impact. The generated assembly for x86_64 with GCC looks like this: ... ffffffff81003977: 65 8b 05 02 ea 00 7f mov %gs:0x7f00ea02(%rip),%eax # 12380 ffffffff8100397e: 25 ff 03 00 00 and $0x3ff,%eax ffffffff81003983: 48 83 c0 0f add $0xf,%rax ffffffff81003987: 25 f8 07 00 00 and $0x7f8,%eax ffffffff8100398c: 48 29 c4 sub %rax,%rsp ffffffff8100398f: 48 8d 44 24 0f lea 0xf(%rsp),%rax ffffffff81003994: 48 83 e0 f0 and $0xfffffffffffffff0,%rax ... As a result of the above stack alignment, this patch introduces about 5 bits of randomness after pt_regs is spilled to the thread stack on x86_64, and 6 bits on x86_32 (since its has 1 fewer bit required for stack alignment). The amount of entropy could be adjusted based on how much of the stack space we wish to trade for security. My measure of syscall performance overhead (on x86_64): lmbench: /usr/lib/lmbench/bin/x86_64-linux-gnu/lat_syscall -N 10000 null randomize_kstack_offset=y Simple syscall: 0.7082 microseconds randomize_kstack_offset=n Simple syscall: 0.7016 microseconds So, roughly 0.9% overhead growth for a no-op syscall, which is very manageable. And for people that don't want this, it's off by default. There are two gotchas with using the alloca() trick. First, compilers that have Stack Clash protection (-fstack-clash-protection) enabled by default (e.g. Ubuntu[3]) add pagesize stack probes to any dynamic stack allocations. While the randomization offset is always less than a page, the resulting assembly would still contain (unreachable!) probing routines, bloating the resulting assembly. To avoid this, -fno-stack-clash-protection is unconditionally added to the kernel Makefile since this is the only dynamic stack allocation in the kernel (now that VLAs have been removed) and it is provably safe from Stack Clash style attacks. The second gotcha with alloca() is a negative interaction with -fstack-protector*, in that it sees the alloca() as an array allocation, which triggers the unconditional addition of the stack canary function pre/post-amble which slows down syscalls regardless of the static branch. In order to avoid adding this unneeded check and its associated performance impact, architectures need to carefully remove uses of -fstack-protector-strong (or -fstack-protector) in the compilation units that use the add_random_kstack() macro and to audit the resulting stack mitigation coverage (to make sure no desired coverage disappears). No change is visible for this on x86 because the stack protector is already unconditionally disabled for the compilation unit, but the change is required on arm64. There is, unfortunately, no attribute that can be used to disable stack protector for specific functions. Comparison to PaX RANDKSTACK feature: The RANDKSTACK feature randomizes the location of the stack start (cpu_current_top_of_stack), i.e. including the location of pt_regs structure itself on the stack. Initially this patch followed the same approach, but during the recent discussions[2], it has been determined to be of a little value since, if ptrace functionality is available for an attacker, they can use PTRACE_PEEKUSR/PTRACE_POKEUSR to read/write different offsets in the pt_regs struct, observe the cache behavior of the pt_regs accesses, and figure out the random stack offset. Another difference is that the random offset is stored in a per-cpu variable, rather than having it be per-thread. As a result, these implementations differ a fair bit in their implementation details and results, though obviously the intent is similar. [1] https://lore.kernel.org/kernel-hardening/2236FBA76BA1254E88B949DDB74E612BA4BC57C1@IRSMSX102.ger.corp.intel.com/ [2] https://lore.kernel.org/kernel-hardening/20190329081358.30497-1-elena.reshetova@intel.com/ [3] https://lists.ubuntu.com/archives/ubuntu-devel/2019-June/040741.html Co-developed-by: Elena Reshetova Signed-off-by: Elena Reshetova Link: https://lore.kernel.org/r/20190415060918.3766-1-elena.reshetova@intel.com Reviewed-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87im5769op.ffs@nanos.tec.linutronix.de/ Signed-off-by: Kees Cook --- .../admin-guide/kernel-parameters.txt | 11 ++++ Makefile | 4 ++ arch/Kconfig | 23 ++++++++ include/linux/randomize_kstack.h | 54 +++++++++++++++++++ init/main.c | 23 ++++++++ 5 files changed, 115 insertions(+) create mode 100644 include/linux/randomize_kstack.h diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 04545725f187..bee8644a192e 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -4061,6 +4061,17 @@ fully seed the kernel's CRNG. Default is controlled by CONFIG_RANDOM_TRUST_CPU. + randomize_kstack_offset= + [KNL] Enable or disable kernel stack offset + randomization, which provides roughly 5 bits of + entropy, frustrating memory corruption attacks + that depend on stack address determinism or + cross-syscall address exposures. This is only + available on architectures that have defined + CONFIG_HAVE_ARCH_RANDOMIZE_KSTACK_OFFSET. + Format: (1/Y/y=enable, 0/N/n=disable) + Default is CONFIG_RANDOMIZE_KSTACK_OFFSET_DEFAULT. + ras=option[,option,...] [KNL] RAS-specific options cec_disable [X86] diff --git a/Makefile b/Makefile index 31dcdb3d61fa..8a959a264588 100644 --- a/Makefile +++ b/Makefile @@ -811,6 +811,10 @@ KBUILD_CFLAGS += -ftrivial-auto-var-init=zero KBUILD_CFLAGS += -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang endif +# While VLAs have been removed, GCC produces unreachable stack probes +# for the randomize_kstack_offset feature. Disable it for all compilers. +KBUILD_CFLAGS += $(call cc-option, -fno-stack-clash-protection) + DEBUG_CFLAGS := # Workaround for GCC versions < 5.0 diff --git a/arch/Kconfig b/arch/Kconfig index 2bb30673d8e6..4fe6b047fcbc 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -1055,6 +1055,29 @@ config VMAP_STACK backing virtual mappings with real shadow memory, and KASAN_VMALLOC must be enabled. +config HAVE_ARCH_RANDOMIZE_KSTACK_OFFSET + def_bool n + help + An arch should select this symbol if it can support kernel stack + offset randomization with calls to add_random_kstack_offset() + during syscall entry and choose_random_kstack_offset() during + syscall exit. Careful removal of -fstack-protector-strong and + -fstack-protector should also be applied to the entry code and + closely examined, as the artificial stack bump looks like an array + to the compiler, so it will attempt to add canary checks regardless + of the static branch state. + +config RANDOMIZE_KSTACK_OFFSET_DEFAULT + bool "Randomize kernel stack offset on syscall entry" + depends on HAVE_ARCH_RANDOMIZE_KSTACK_OFFSET + help + The kernel stack offset can be randomized (after pt_regs) by + roughly 5 bits of entropy, frustrating memory corruption + attacks that depend on stack address determinism or + cross-syscall address exposures. This feature is controlled + by kernel boot param "randomize_kstack_offset=on/off", and this + config chooses the default boot state. + config ARCH_OPTIONAL_KERNEL_RWX def_bool n diff --git a/include/linux/randomize_kstack.h b/include/linux/randomize_kstack.h new file mode 100644 index 000000000000..fd80fab663a9 --- /dev/null +++ b/include/linux/randomize_kstack.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef _LINUX_RANDOMIZE_KSTACK_H +#define _LINUX_RANDOMIZE_KSTACK_H + +#include +#include +#include + +DECLARE_STATIC_KEY_MAYBE(CONFIG_RANDOMIZE_KSTACK_OFFSET_DEFAULT, + randomize_kstack_offset); +DECLARE_PER_CPU(u32, kstack_offset); + +/* + * Do not use this anywhere else in the kernel. This is used here because + * it provides an arch-agnostic way to grow the stack with correct + * alignment. Also, since this use is being explicitly masked to a max of + * 10 bits, stack-clash style attacks are unlikely. For more details see + * "VLAs" in Documentation/process/deprecated.rst + */ +void *__builtin_alloca(size_t size); +/* + * Use, at most, 10 bits of entropy. We explicitly cap this to keep the + * "VLA" from being unbounded (see above). 10 bits leaves enough room for + * per-arch offset masks to reduce entropy (by removing higher bits, since + * high entropy may overly constrain usable stack space), and for + * compiler/arch-specific stack alignment to remove the lower bits. + */ +#define KSTACK_OFFSET_MAX(x) ((x) & 0x3FF) + +/* + * These macros must be used during syscall entry when interrupts and + * preempt are disabled, and after user registers have been stored to + * the stack. + */ +#define add_random_kstack_offset() do { \ + if (static_branch_maybe(CONFIG_RANDOMIZE_KSTACK_OFFSET_DEFAULT, \ + &randomize_kstack_offset)) { \ + u32 offset = raw_cpu_read(kstack_offset); \ + u8 *ptr = __builtin_alloca(KSTACK_OFFSET_MAX(offset)); \ + /* Keep allocation even after "ptr" loses scope. */ \ + asm volatile("" : "=o"(*ptr) :: "memory"); \ + } \ +} while (0) + +#define choose_random_kstack_offset(rand) do { \ + if (static_branch_maybe(CONFIG_RANDOMIZE_KSTACK_OFFSET_DEFAULT, \ + &randomize_kstack_offset)) { \ + u32 offset = raw_cpu_read(kstack_offset); \ + offset ^= (rand); \ + raw_cpu_write(kstack_offset, offset); \ + } \ +} while (0) + +#endif diff --git a/init/main.c b/init/main.c index 53b278845b88..f498aac26e8c 100644 --- a/init/main.c +++ b/init/main.c @@ -844,6 +844,29 @@ static void __init mm_init(void) pti_init(); } +#ifdef CONFIG_HAVE_ARCH_RANDOMIZE_KSTACK_OFFSET +DEFINE_STATIC_KEY_MAYBE_RO(CONFIG_RANDOMIZE_KSTACK_OFFSET_DEFAULT, + randomize_kstack_offset); +DEFINE_PER_CPU(u32, kstack_offset); + +static int __init early_randomize_kstack_offset(char *buf) +{ + int ret; + bool bool_result; + + ret = kstrtobool(buf, &bool_result); + if (ret) + return ret; + + if (bool_result) + static_branch_enable(&randomize_kstack_offset); + else + static_branch_disable(&randomize_kstack_offset); + return 0; +} +early_param("randomize_kstack_offset", early_randomize_kstack_offset); +#endif + void __init __weak arch_call_rest_init(void) { rest_init(); From patchwork Thu Apr 1 23:23:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 12179829 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68C80C433ED for ; Thu, 1 Apr 2021 23:26:25 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E984A61103 for ; Thu, 1 Apr 2021 23:26:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E984A61103 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=RyEvf3SkY1LPBADhleEG8PWA2xFXP16i/TeFexol/H8=; b=EW8MVOnKMvmrtErHyp6SC67RF qu6BRDnlqWO87CTBuBxSBowhbStVkQZ/khOVvgbgaLRbZechS4A/jNU9NouKx/aVc3ZYtXC8GWn05 Lx81usLnQjUuh3yd4viRphfAhNC2jcIxFsoDU9WsD9I+q+ZZlJUTCat8jP2SmI3Bpy1UfD6GnfU8M Xyf3CeJyB3nHOjHQRhPVM/Dd7cKJroFLqCY9XOzXb+a8xmmV2Wso9bAdsRABiAfkH+aLhf6nLMsmY IB9CExtdXyXKXhQdkE2P098ubkFHmzxuG2GaOaD0Toc+DFyV6SPNwRosKzuJvH+9L3zZzd/5gSpLT EbwPL8dtA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lS6fT-00BJs9-RZ; Thu, 01 Apr 2021 23:24:04 +0000 Received: from mail-pf1-x435.google.com ([2607:f8b0:4864:20::435]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lS6fK-00BJnt-9E for linux-arm-kernel@lists.infradead.org; Thu, 01 Apr 2021 23:23:59 +0000 Received: by mail-pf1-x435.google.com with SMTP id a12so2512670pfc.7 for ; Thu, 01 Apr 2021 16:23:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0iK0aKBdGLAGxgJdxBWHAkXtURWP8RQAHBQ4WEZMvwk=; b=Z2CHDUVkMSklshFb3NxT2bGXR27w+GSF7tsZqr9rE37jCHuIY3AbMQ6qxvUEcX5ZQ1 1/gG1wOvJE0yHvcAYlTXZLJsaCgA7GWr2Hw8NKcZ1wFqXLQBLSWNe7bKEbfp4Kyy+15z Cd+u2AWoLCR2TLY6GXxxprij+RZMuXhv0eeN8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0iK0aKBdGLAGxgJdxBWHAkXtURWP8RQAHBQ4WEZMvwk=; b=qJpExWTneBLbYrex3+rUm0ssBHujMk9aGGArvJkgj+PMam2zHtPRD7ecfPNoskKkDg dqHSddjOlSKmo4e8xMghn5JoOXQwRJaw8KXT6m9NashVZBWuyjHiLeN80E4VAlHIAiNQ pMrsulIPCZivj9+e57Y4qY8QTvOKxASxPGZ6ip3AK3dZespekEW0w3KEelz8p9URSmT0 oh+8UlXUlkj8ygNA+AIhJO6Vf7vQDHoEZA78/oJNUbRw3bMnr4+POLvlZlVqnddwVyW3 RaL/i3iI7C8uyt95cATWsQNRkqxUDYonwWxZBJcKKf9IDQtnmOMhKjSNoj5jU02L8evQ UceQ== X-Gm-Message-State: AOAM5314H/0FHTkZGQ04MKDHzv/S6wNf4oZRwcViwybSngSemb3IIFKv ejnhWIW8W95e1IwibV/Y3UzxvA== X-Google-Smtp-Source: ABdhPJxvvMTlD4QwOl/dE2dQVrae6FQePxPse4C77UO8+oMVT91t15kw3UOGk9sRJC6/hxpNs1Q28g== X-Received: by 2002:a05:6a00:1350:b029:227:7a8b:99c9 with SMTP id k16-20020a056a001350b02902277a8b99c9mr9603503pfu.73.1617319432097; Thu, 01 Apr 2021 16:23:52 -0700 (PDT) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id g21sm6489969pfk.30.2021.04.01.16.23.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Apr 2021 16:23:51 -0700 (PDT) From: Kees Cook To: Thomas Gleixner Cc: Kees Cook , Elena Reshetova , x86@kernel.org, Andy Lutomirski , Peter Zijlstra , Catalin Marinas , Will Deacon , Mark Rutland , Alexander Potapenko , Alexander Popov , Ard Biesheuvel , Jann Horn , Vlastimil Babka , David Hildenbrand , Mike Rapoport , Andrew Morton , Jonathan Corbet , Randy Dunlap , kernel-hardening@lists.openwall.com, linux-hardening@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v10 4/6] x86/entry: Enable random_kstack_offset support Date: Thu, 1 Apr 2021 16:23:45 -0700 Message-Id: <20210401232347.2791257-5-keescook@chromium.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210401232347.2791257-1-keescook@chromium.org> References: <20210401232347.2791257-1-keescook@chromium.org> MIME-Version: 1.0 X-Patch-Hashes: v=1; h=sha256; g=5ebd0a02df09ef237ee07421c9f73e94deeaf170; i=np7yed3mY+gWIWkexmB7CyDLgwsIh0xV2RGaksJc7tI=; m=bWJ56lJBn9Ma2AQu8aultasTTfx/BX5M5azaoQbfD3I=; p=EuIQlileJ2PUD6W0Q1r2arSRAFiVT/zqDgDnM3BVsdE= X-Patch-Sig: m=pgp; i=keescook@chromium.org; s=0x0x8972F4DFDC6DC026; b=iQIzBAABCgAdFiEEpcP2jyKd1g9yPm4TiXL039xtwCYFAmBmVgIACgkQiXL039xtwCYJfQ//R01 +ruGpfNJgR38fkaOdtzHbDqDlJxmSNFRC8/dN3FGy/e8NTkweXZd+TPUWSRssw9mev1qqdZWr1DaQ GeFsLCwKMWeZLe5D1t34WvH09xuhIxcOIrFKYqbs5J76yT492D7yIk/MAkGnvXh1042U5k3At+Vcn rwHW9ynX6sTvAzEiiTQGJKcSDvQSfUCoi1oIFEugCNoesi5WgfCbdIqD8nOiCkb8Zfl6JKEUtyceE WT61pe3ZutEbpn7k78ypOmwqFBPhJm2F9VrOIXz+CcmYTBEjowzd0iErcIUMannJfXP05bfL550cP 0VYAEG0IFFissaoezJRmFqCgx3osXEkLkAm/LWNfXfW36ijHZZgJR69TuHYwPEdneGOrcU46McEt5 9+0lT1L9ajNTB01KrLzS2RhQIV6+uXjTIoS4Q0tHSMqpMMsOvbjvXANwMlMHg6vTPpqVrLFf2JScy K4yGVN6gS9pGqJ0c0iMK91nYmrlze3N2+eu5P1lcYVBfmvwpa2PNWkAtMkcrkabJCBWbao7w9yJTd C1EQD9gN1AbTlckl+EcM3Kqq4E5z9/zzJawIMLpHbg7au5q00TadCiR0huOkuR+e4hWTvWvzuSdgK EFXsFx+LXQjd3kXGaBtnkoNEFz3nPp7Ag7P7eYwvaHi26k5yW9Cy0dWZGC9FGAWM= X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210402_002354_438804_45014DB1 X-CRM114-Status: GOOD ( 18.28 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Allow for a randomized stack offset on a per-syscall basis, with roughly 5-6 bits of entropy, depending on compiler and word size. Since the method of offsetting uses macros, this cannot live in the common entry code (the stack offset needs to be retained for the life of the syscall, which means it needs to happen at the actual entry point). Reviewed-by: Thomas Gleixner Link: https://lore.kernel.org/lkml/87lfa369tv.ffs@nanos.tec.linutronix.de/ Signed-off-by: Kees Cook --- arch/x86/Kconfig | 1 + arch/x86/entry/common.c | 3 +++ arch/x86/include/asm/entry-common.h | 16 ++++++++++++++++ 3 files changed, 20 insertions(+) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 2792879d398e..4b4ad8ec10d2 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -165,6 +165,7 @@ config X86 select HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD if X86_64 select HAVE_ARCH_USERFAULTFD_WP if X86_64 && USERFAULTFD select HAVE_ARCH_VMAP_STACK if X86_64 + select HAVE_ARCH_RANDOMIZE_KSTACK_OFFSET select HAVE_ARCH_WITHIN_STACK_FRAMES select HAVE_ASM_MODVERSIONS select HAVE_CMPXCHG_DOUBLE diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c index a2433ae8a65e..810983d7c26f 100644 --- a/arch/x86/entry/common.c +++ b/arch/x86/entry/common.c @@ -38,6 +38,7 @@ #ifdef CONFIG_X86_64 __visible noinstr void do_syscall_64(unsigned long nr, struct pt_regs *regs) { + add_random_kstack_offset(); nr = syscall_enter_from_user_mode(regs, nr); instrumentation_begin(); @@ -83,6 +84,7 @@ __visible noinstr void do_int80_syscall_32(struct pt_regs *regs) { unsigned int nr = syscall_32_enter(regs); + add_random_kstack_offset(); /* * Subtlety here: if ptrace pokes something larger than 2^32-1 into * orig_ax, the unsigned int return value truncates it. This may @@ -102,6 +104,7 @@ static noinstr bool __do_fast_syscall_32(struct pt_regs *regs) unsigned int nr = syscall_32_enter(regs); int res; + add_random_kstack_offset(); /* * This cannot use syscall_enter_from_user_mode() as it has to * fetch EBP before invoking any of the syscall entry work diff --git a/arch/x86/include/asm/entry-common.h b/arch/x86/include/asm/entry-common.h index 2b87b191b3b8..14ebd2196569 100644 --- a/arch/x86/include/asm/entry-common.h +++ b/arch/x86/include/asm/entry-common.h @@ -2,6 +2,7 @@ #ifndef _ASM_X86_ENTRY_COMMON_H #define _ASM_X86_ENTRY_COMMON_H +#include #include #include @@ -70,6 +71,21 @@ static inline void arch_exit_to_user_mode_prepare(struct pt_regs *regs, */ current_thread_info()->status &= ~(TS_COMPAT | TS_I386_REGS_POKED); #endif + + /* + * Ultimately, this value will get limited by KSTACK_OFFSET_MAX(), + * but not enough for x86 stack utilization comfort. To keep + * reasonable stack head room, reduce the maximum offset to 8 bits. + * + * The actual entropy will be further reduced by the compiler when + * applying stack alignment constraints (see cc_stack_align4/8 in + * arch/x86/Makefile), which will remove the 3 (x86_64) or 2 (ia32) + * low bits from any entropy chosen here. + * + * Therefore, final stack offset entropy will be 5 (x86_64) or + * 6 (ia32) bits. + */ + choose_random_kstack_offset(rdtsc() & 0xFF); } #define arch_exit_to_user_mode_prepare arch_exit_to_user_mode_prepare From patchwork Thu Apr 1 23:23:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 12179831 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A362BC433B4 for ; Thu, 1 Apr 2021 23:26:26 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0FF5561103 for ; Thu, 1 Apr 2021 23:26:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0FF5561103 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=NgOf3MljYkU3dCVV3030DKNRI5l7vxPIbjWZQc9TU1c=; b=fS8uUqEy2ykG3CTp3q9YS0pGm rp/7QLgNCYShMGNebqmsqGVzid2q5lYimewYiAMOCmBL4jdAoOhvbxHz7TDkw/XG9STvTXhJzdF+U OyGmSqafJCB2/3NgLbRgxT9pntCmUp4Kr5IohGpSOUE5JAtHp5kzlgTUFZj3IAaZymIhtf666Ojme VcxAzxuo7RqQA1i+IMLfRm9zLeQQdQXy//c2Tam/NuEKpJF2r7FMo7jcig/PtU4ndO1yTPGziC7nD B5KcZGwEDcTzG/NtqZ8t52Ia7uZByJ7IRnJhZlnyIhEak8uqoOlIrJ3RPxh//pVNoQzaPRLJuBm9k 6mG2cubBA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lS6gF-00BK2e-8d; Thu, 01 Apr 2021 23:24:51 +0000 Received: from mail-pg1-x533.google.com ([2607:f8b0:4864:20::533]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lS6fL-00BJqq-SZ for linux-arm-kernel@lists.infradead.org; Thu, 01 Apr 2021 23:24:03 +0000 Received: by mail-pg1-x533.google.com with SMTP id t140so2513947pgb.13 for ; Thu, 01 Apr 2021 16:23:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=dHwr6OVFfk7s7Yw8a90FyZmnLv02IEGSs6EhbG8UN8M=; b=apoCPv6eRxC+65o8Yfokym7TMqmTLua+kwS5uawMkAr+K8bzn0V6/+UNB+E8jjMTT9 kqBvLjd/Y8r/2j+FcF251klrukr4C2g4wEp9/G2iZwIGl2vQVJKEhV3vbmAeJ9arWHcM k8t+ERXme2Uowb4UveF5DKhHLDGzwVc27v6PY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dHwr6OVFfk7s7Yw8a90FyZmnLv02IEGSs6EhbG8UN8M=; b=f0E0Ft4Lgy5mUKQDK/5+a2nbHI5uTz41lKLcziH0hjyxTvkegAs1xbLNeXofkAg2qj r4rAUwW3Ndc6WNh2z9HkjTUQBHO37XmxI7qyZnj5aSOZWkMEEhC3uTijPth6NnBdFsGA u6vXs7u4HxcKn8uMEkKy2p3xF6nwxmC1y94AslJO0zmLOP4qODdB6OPiGDi44fYGr2nm uijmiV1AuwFo9ebJ6OVULx08Vf3mf/cpKzLFjLM7HObmDr1THgSjGdJkyWld1KgPTjDK Q1yIsxcVDCWpYR1KOsoM8FDJYHm6sle37bBRU9Yv0RI7+ONJvp4DOibETWBuUDYLnf7p 5/BQ== X-Gm-Message-State: AOAM533javdzaPkj34ZPemRovU46kfhdY0ku3T0RUKwKlxT0yReeFuR3 6nzo0dbAe1RMKbfBRlmuMU34fA== X-Google-Smtp-Source: ABdhPJy0JI+h7arv5jEq6paI9Bu5w08cDNSCg/uQ/t7RP6xwDr38VSpWxgu747yOW+trYuYmyvFaZA== X-Received: by 2002:a62:b412:0:b029:21f:6b06:7bdd with SMTP id h18-20020a62b4120000b029021f6b067bddmr9677058pfn.51.1617319434704; Thu, 01 Apr 2021 16:23:54 -0700 (PDT) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id z4sm6790276pgv.73.2021.04.01.16.23.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Apr 2021 16:23:54 -0700 (PDT) From: Kees Cook To: Thomas Gleixner Cc: Kees Cook , Elena Reshetova , x86@kernel.org, Andy Lutomirski , Peter Zijlstra , Catalin Marinas , Will Deacon , Mark Rutland , Alexander Potapenko , Alexander Popov , Ard Biesheuvel , Jann Horn , Vlastimil Babka , David Hildenbrand , Mike Rapoport , Andrew Morton , Jonathan Corbet , Randy Dunlap , kernel-hardening@lists.openwall.com, linux-hardening@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v10 5/6] arm64: entry: Enable random_kstack_offset support Date: Thu, 1 Apr 2021 16:23:46 -0700 Message-Id: <20210401232347.2791257-6-keescook@chromium.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210401232347.2791257-1-keescook@chromium.org> References: <20210401232347.2791257-1-keescook@chromium.org> MIME-Version: 1.0 X-Patch-Hashes: v=1; h=sha256; g=248270d6e87ad6ffcee20e448a3bbf6614f92c8d; i=6FjHrMJQ1a03WxuPiMt8aIK5MW/T0bayLG0SuUm7Wt4=; m=DW/OKOjAEidoxl6ODEV6zN0c7U/uCvK1t/uZsIkKPaM=; p=nb+ZCQAESoMtgqRKEEjn/wowCv7zZGeRTSUAEFbjSEQ= X-Patch-Sig: m=pgp; i=keescook@chromium.org; s=0x0x8972F4DFDC6DC026; b=iQIzBAABCgAdFiEEpcP2jyKd1g9yPm4TiXL039xtwCYFAmBmVgIACgkQiXL039xtwCZDLw//WsN q3e+PfWfPV1g3MMiPrlRGvZeP5grL9TtxqK1tGNvEy6dp+ulJLVcVDUgjKq6d2+6/ZtpF/CXp1Djm YjJGAOaElG7IEw7D5rWkdaIDh/+dJB9i4OG7OQStbX2RdOTztew9FhzttIrvtz9CdY/62YtNZwfSM MMJuDLR6sDqH/hCS9vInn5lqd459ACg1MQxu5GoytTWRNX8WWvf5JQzZhQjfEc8krX0rpOsQzDr5t mO5emz5pHZsbxfKob6wQsl+exwaKhfwPphM0Tn5RCDnVvf9bTDJVhQnNwIT2+Q378sZ5hcpwvz8Yx 3mhh09xaCK/zQ3/kmN1gqG6pPlCtoxRBbEFTzbhj16pX65K+bS5BZ5XWM3ugP8EXfXyWrpo52dSr5 /t6uVKyB16iSTt7H9rEcp1wXS1076HCT6M8Wde/qn1X2QK1ZLA0cesEWm/X4sk/w1Ve1TWe6PmQpW Nk6T3IyLa/fkxj14KMiTNTz2sMFoeqV/i4d24nIL+3TQG5LqAqG/9iT56DYYHkYs11nmeEHf4qbLV bn/R7twlh65mDdXZ/L6iFluiC+fm1yLrDq4JIwtrmQJbUqE0m9f8KUCzMgXfYFSY8lb5Wo0OxNxOr uzdIl+VgJ/ySASy1+o5Q6GjAhge97rgSd/h8Isb33l3zzjkc6/vquWyNirJA8dIg= X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210402_002401_359758_8353343B X-CRM114-Status: GOOD ( 20.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Allow for a randomized stack offset on a per-syscall basis, with roughly 5 bits of entropy. (And include AAPCS rationale AAPCS thanks to Mark Rutland.) In order to avoid unconditional stack canaries on syscall entry (due to the use of alloca()), also disable stack protector to avoid triggering needless checks and slowing down the entry path. As there is no general way to control stack protector coverage with a function attribute[1], this must be disabled at the compilation unit level. This isn't a problem here, though, since stack protector was not triggered before: examining the resulting syscall.o, there are no changes in canary coverage (none before, none now). [1] a working __attribute__((no_stack_protector)) has been added to GCC and Clang but has not been released in any version yet: https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;h=346b302d09c1e6db56d9fe69048acb32fbb97845 https://reviews.llvm.org/rG4fbf84c1732fca596ad1d6e96015e19760eb8a9b Signed-off-by: Kees Cook Acked-by: Will Deacon --- arch/arm64/Kconfig | 1 + arch/arm64/kernel/Makefile | 5 +++++ arch/arm64/kernel/syscall.c | 16 ++++++++++++++++ 3 files changed, 22 insertions(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 1f212b47a48a..2d0e5f544429 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -146,6 +146,7 @@ config ARM64 select HAVE_ARCH_MMAP_RND_COMPAT_BITS if COMPAT select HAVE_ARCH_PFN_VALID select HAVE_ARCH_PREL32_RELOCATIONS + select HAVE_ARCH_RANDOMIZE_KSTACK_OFFSET select HAVE_ARCH_SECCOMP_FILTER select HAVE_ARCH_STACKLEAK select HAVE_ARCH_THREAD_STRUCT_WHITELIST diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile index ed65576ce710..6cc97730790e 100644 --- a/arch/arm64/kernel/Makefile +++ b/arch/arm64/kernel/Makefile @@ -9,6 +9,11 @@ CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_insn.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_return_address.o = $(CC_FLAGS_FTRACE) +# Remove stack protector to avoid triggering unneeded stack canary +# checks due to randomize_kstack_offset. +CFLAGS_REMOVE_syscall.o = -fstack-protector -fstack-protector-strong +CFLAGS_syscall.o += -fno-stack-protector + # Object file lists. obj-y := debug-monitors.o entry.o irq.o fpsimd.o \ entry-common.o entry-fpsimd.o process.o ptrace.o \ diff --git a/arch/arm64/kernel/syscall.c b/arch/arm64/kernel/syscall.c index b9cf12b271d7..263d6c1a525f 100644 --- a/arch/arm64/kernel/syscall.c +++ b/arch/arm64/kernel/syscall.c @@ -5,6 +5,7 @@ #include #include #include +#include #include #include @@ -43,6 +44,8 @@ static void invoke_syscall(struct pt_regs *regs, unsigned int scno, { long ret; + add_random_kstack_offset(); + if (scno < sc_nr) { syscall_fn_t syscall_fn; syscall_fn = syscall_table[array_index_nospec(scno, sc_nr)]; @@ -55,6 +58,19 @@ static void invoke_syscall(struct pt_regs *regs, unsigned int scno, ret = lower_32_bits(ret); regs->regs[0] = ret; + + /* + * Ultimately, this value will get limited by KSTACK_OFFSET_MAX(), + * but not enough for arm64 stack utilization comfort. To keep + * reasonable stack head room, reduce the maximum offset to 9 bits. + * + * The actual entropy will be further reduced by the compiler when + * applying stack alignment constraints: the AAPCS mandates a + * 16-byte (i.e. 4-bit) aligned SP at function boundaries. + * + * The resulting 5 bits of entropy is seen in SP[8:4]. + */ + choose_random_kstack_offset(get_random_int() & 0x1FF); } static inline bool has_syscall_work(unsigned long flags) From patchwork Thu Apr 1 23:23:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 12179835 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A634C433ED for ; Thu, 1 Apr 2021 23:26:33 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D91FE61105 for ; Thu, 1 Apr 2021 23:26:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D91FE61105 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=chromium.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=SSYAKonzAYJ2ATSsDHMO9+jSBtq7E0sa/oNiAkKNWz0=; b=V28ZgS9Ee0MYJoFjt9sNGR2vR iirj2gIQZn2LWHk2cePgoy46wCW2bjzgLNIjoNQdW8IXTchhInpMnU+yPPcoHMsNW2kan99pAb75k R9EkOmTVJNqWdX88WZIbudm7MZiIEJhKKqLPstm5S3L+SRfFSUzVd30yEoc69yEfK+dAPEOzozXKV pajizDyTAlZ5qcWgck2HJCZQ+PebSSqec2N9Bk6e3aQ7YdgDk9vCh5qbQMJ0CxuTclXg9zQyrv/4e M5NWZdLyQH0vQLhqUQCpPCIceSX2gYHlr6+OSnJy6CIk2prNEXLVZ30FUoWvbZ/LZL8m0tnNTCGTn X66LXvWYQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lS6gP-00BKAK-La; Thu, 01 Apr 2021 23:25:01 +0000 Received: from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lS6fM-00BJqr-LI for linux-arm-kernel@lists.infradead.org; Thu, 01 Apr 2021 23:24:03 +0000 Received: by mail-pl1-x629.google.com with SMTP id y2so1747503plg.5 for ; Thu, 01 Apr 2021 16:23:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LvZQ1bPDTn+jfy4ZjpxxRS3b5GxvPPmzyAjth6NaQ44=; b=F42wJOLiSpeN/4fzwW/N9mYN1ryaPRGoZdaLqlVuodcAvGQ/ABIjgVX+b+at7lWvdh jETxFZAabFX4f9lDzotk6Kj0Lg2Y/9s9xOtR4bTEFmgSRaHosFkbLoH8wmrTPZ0KycU5 e2szNTHZWHmNtWCzVWeIu3x9aVYmZd9gsKNMQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LvZQ1bPDTn+jfy4ZjpxxRS3b5GxvPPmzyAjth6NaQ44=; b=iCd6w9YZn90Qys/Sq2BljKFjROwYPOONbgOuUOfk1QsXVsbdnGnID8WvZnvn9siejz 3oLAkIclwCpKta1PpUpt++QaKh9Q5bBZVC8greUvpdBLWcrzjdbkx5NivFi762+F9DIk ojJm8qyB6DcR29hVpjSy5v/1sLKKoxetwr2hFCFiRJCdty2zkRM7ZvY2hb6Lm0R684VD fmvGbjtqcnkQpMf5NHJBK4Jt/HP7nb1JQ214KGaL3jVJhZnzC+qPHC8GNrpKm/iMef2c pbApFp2FOirP9ynQDzwcUzcUNOiPZG5CjNM4uXRMuGwQnxRvbDfQCuQ0EBS7QSupbyJP CsJw== X-Gm-Message-State: AOAM530MHTCWc+bdScBBwWbCi/6+pSkyscuw2Ri7WACR3OBHuSxx+Qx4 27UDcF88hgKX1QOUEx4a7tlVhQ== X-Google-Smtp-Source: ABdhPJyJvuTUIP8/7JclWdgrMtrP8e/Mxf3nuL8lBiVp7B+1ZeXl9ez94SqDJnG3QluQM6UrJWyP4Q== X-Received: by 2002:a17:90b:100e:: with SMTP id gm14mr11159194pjb.217.1617319435264; Thu, 01 Apr 2021 16:23:55 -0700 (PDT) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id co20sm6203964pjb.32.2021.04.01.16.23.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 Apr 2021 16:23:54 -0700 (PDT) From: Kees Cook To: Thomas Gleixner Cc: Kees Cook , Elena Reshetova , x86@kernel.org, Andy Lutomirski , Peter Zijlstra , Catalin Marinas , Will Deacon , Mark Rutland , Alexander Potapenko , Alexander Popov , Ard Biesheuvel , Jann Horn , Vlastimil Babka , David Hildenbrand , Mike Rapoport , Andrew Morton , Jonathan Corbet , Randy Dunlap , kernel-hardening@lists.openwall.com, linux-hardening@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v10 6/6] lkdtm: Add REPORT_STACK for checking stack offsets Date: Thu, 1 Apr 2021 16:23:47 -0700 Message-Id: <20210401232347.2791257-7-keescook@chromium.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210401232347.2791257-1-keescook@chromium.org> References: <20210401232347.2791257-1-keescook@chromium.org> MIME-Version: 1.0 X-Patch-Hashes: v=1; h=sha256; g=04ba1fd2cf94c15b6c425d21eadf26c7b5b595c9; i=b69wRsxT78r/3tM1mGa7N6ME6+rlXyFg15giRWRwPAQ=; m=XEkb9gtEVwIXoR3CRFWDIQGhFt30zOfIashAyp1d4uo=; p=d33GDHyvnRxlVkBNxjBj+O38zle2N7FWLUDCSfaX1O0= X-Patch-Sig: m=pgp; i=keescook@chromium.org; s=0x0x8972F4DFDC6DC026; b=iQIzBAABCgAdFiEEpcP2jyKd1g9yPm4TiXL039xtwCYFAmBmVgMACgkQiXL039xtwCaFWBAAnoV bf3FCj1tsOpxawVbSq+bVH2eLNWxW93P/OuD2SZViYnIltHwsMQb5O5JgjncW8xaOmaxRSKlAHAun c8QcJmj4OtK9gw6w9RGgvv/IFpIerKQ9rmBX1g3E9pYYL9w3c9VPWCCEy/n7JmUxYQRdueDWTFxbM DJcbI5C+2IcnzFrLA9H+wols1KqBnYq+b4dXGbk5komriRvkgReB+BDYk0Xk+CETCOSiHLluDFQBB o4m/PZXGsdWMD8Te1U+mf9I5VkhN/tQx3X4hchBhH1+Nmia+fhRki75vFi4V+nSg9uUwxBrnbvy0e wb+gePWPXj9YaEOUMwEy+OS6/ZvEFF4iXdsIokIGW7FylzIQnFltv1/iqj88+OPI+ptCESfD+jqyo h4VlZj/uVKKzJVktDMm8BrXCZTYeRD8ZvvxcTZRpdpSwPPH/p04bvjvs95CmBvn49r+7IpnVrUsH6 Q5EEB2mMj3/08pFtDtKkLRJi0b103YakkJghevfP1cQS6TlRL/XsKiM3v5UVjaH+OToPHUDfUCxSt ZzVucB9RlDb5QHXAW/55qYH2YKkEVAZt0B72UH+tH1XCxNucHUz84/Da5+eVTToByC7in/7nXdNU0 QHf1hWPssVLB1w4iS8kzT09u4F86a4AlFf/lxJmVXI1e4coBbEsRlccoJFqqRc7s= X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210402_002401_356368_BD4658EF X-CRM114-Status: GOOD ( 18.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org For validating the stack offset behavior, report the offset from a given process's first seen stack address. Add script to calculate the results to the LKDTM kselftests. Signed-off-by: Kees Cook --- drivers/misc/lkdtm/bugs.c | 17 +++++++++ drivers/misc/lkdtm/core.c | 1 + drivers/misc/lkdtm/lkdtm.h | 1 + tools/testing/selftests/lkdtm/.gitignore | 1 + tools/testing/selftests/lkdtm/Makefile | 1 + .../testing/selftests/lkdtm/stack-entropy.sh | 36 +++++++++++++++++++ 6 files changed, 57 insertions(+) create mode 100755 tools/testing/selftests/lkdtm/stack-entropy.sh diff --git a/drivers/misc/lkdtm/bugs.c b/drivers/misc/lkdtm/bugs.c index 110f5a8538e9..0e8254d0cf0b 100644 --- a/drivers/misc/lkdtm/bugs.c +++ b/drivers/misc/lkdtm/bugs.c @@ -134,6 +134,23 @@ noinline void lkdtm_CORRUPT_STACK_STRONG(void) __lkdtm_CORRUPT_STACK((void *)&data); } +static pid_t stack_pid; +static unsigned long stack_addr; + +void lkdtm_REPORT_STACK(void) +{ + volatile uintptr_t magic; + pid_t pid = task_pid_nr(current); + + if (pid != stack_pid) { + pr_info("Starting stack offset tracking for pid %d\n", pid); + stack_pid = pid; + stack_addr = (uintptr_t)&magic; + } + + pr_info("Stack offset: %d\n", (int)(stack_addr - (uintptr_t)&magic)); +} + void lkdtm_UNALIGNED_LOAD_STORE_WRITE(void) { static u8 data[5] __attribute__((aligned(4))) = {1, 2, 3, 4, 5}; diff --git a/drivers/misc/lkdtm/core.c b/drivers/misc/lkdtm/core.c index b2aff4d87c01..8024b6a5cc7f 100644 --- a/drivers/misc/lkdtm/core.c +++ b/drivers/misc/lkdtm/core.c @@ -110,6 +110,7 @@ static const struct crashtype crashtypes[] = { CRASHTYPE(EXHAUST_STACK), CRASHTYPE(CORRUPT_STACK), CRASHTYPE(CORRUPT_STACK_STRONG), + CRASHTYPE(REPORT_STACK), CRASHTYPE(CORRUPT_LIST_ADD), CRASHTYPE(CORRUPT_LIST_DEL), CRASHTYPE(STACK_GUARD_PAGE_LEADING), diff --git a/drivers/misc/lkdtm/lkdtm.h b/drivers/misc/lkdtm/lkdtm.h index 5ae48c64df24..99f90d3e5e9c 100644 --- a/drivers/misc/lkdtm/lkdtm.h +++ b/drivers/misc/lkdtm/lkdtm.h @@ -17,6 +17,7 @@ void lkdtm_LOOP(void); void lkdtm_EXHAUST_STACK(void); void lkdtm_CORRUPT_STACK(void); void lkdtm_CORRUPT_STACK_STRONG(void); +void lkdtm_REPORT_STACK(void); void lkdtm_UNALIGNED_LOAD_STORE_WRITE(void); void lkdtm_SOFTLOCKUP(void); void lkdtm_HARDLOCKUP(void); diff --git a/tools/testing/selftests/lkdtm/.gitignore b/tools/testing/selftests/lkdtm/.gitignore index f26212605b6b..d4b0be857deb 100644 --- a/tools/testing/selftests/lkdtm/.gitignore +++ b/tools/testing/selftests/lkdtm/.gitignore @@ -1,2 +1,3 @@ *.sh !run.sh +!stack-entropy.sh diff --git a/tools/testing/selftests/lkdtm/Makefile b/tools/testing/selftests/lkdtm/Makefile index 1bcc9ee990eb..c71109ceeb2d 100644 --- a/tools/testing/selftests/lkdtm/Makefile +++ b/tools/testing/selftests/lkdtm/Makefile @@ -5,6 +5,7 @@ include ../lib.mk # NOTE: $(OUTPUT) won't get default value if used before lib.mk TEST_FILES := tests.txt +TEST_PROGS := stack-entropy.sh TEST_GEN_PROGS = $(patsubst %,$(OUTPUT)/%.sh,$(shell awk '{print $$1}' tests.txt | sed -e 's/\#//')) all: $(TEST_GEN_PROGS) diff --git a/tools/testing/selftests/lkdtm/stack-entropy.sh b/tools/testing/selftests/lkdtm/stack-entropy.sh new file mode 100755 index 000000000000..b1b8a5097cbb --- /dev/null +++ b/tools/testing/selftests/lkdtm/stack-entropy.sh @@ -0,0 +1,36 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# +# Measure kernel stack entropy by sampling via LKDTM's REPORT_STACK test. +set -e +samples="${1:-1000}" + +# Capture dmesg continuously since it may fill up depending on sample size. +log=$(mktemp -t stack-entropy-XXXXXX) +dmesg --follow >"$log" & pid=$! +report=-1 +for i in $(seq 1 $samples); do + echo "REPORT_STACK" >/sys/kernel/debug/provoke-crash/DIRECT + if [ -t 1 ]; then + percent=$(( 100 * $i / $samples )) + if [ "$percent" -ne "$report" ]; then + /bin/echo -en "$percent%\r" + report="$percent" + fi + fi +done +kill "$pid" + +# Count unique offsets since last run. +seen=$(tac "$log" | grep -m1 -B"$samples"0 'Starting stack offset' | \ + grep 'Stack offset' | awk '{print $NF}' | sort | uniq -c | wc -l) +bits=$(echo "obase=2; $seen" | bc | wc -L) +echo "Bits of stack entropy: $bits" +rm -f "$log" + +# We would expect any functional stack randomization to be at least 5 bits. +if [ "$bits" -lt 5 ]; then + exit 1 +else + exit 0 +fi