From patchwork Sat Jan 27 07:55:24 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 10187347 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id BA6D160385 for ; Sat, 27 Jan 2018 08:04:49 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AB2D828CB5 for ; Sat, 27 Jan 2018 08:04:49 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9F99728D40; Sat, 27 Jan 2018 08:04:49 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id 4710628CB5 for ; Sat, 27 Jan 2018 08:04:48 +0000 (UTC) Received: (qmail 14179 invoked by uid 550); 27 Jan 2018 08:04:36 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 14157 invoked from network); 27 Jan 2018 08:04:35 -0000 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.46,421,1511856000"; d="scan'208";a="196642688" From: Dan Williams To: tglx@linutronix.de Cc: linux-arch@vger.kernel.org, Cyril Novikov , kernel-hardening@lists.openwall.com, Peter Zijlstra , Catalin Marinas , x86@kernel.org, Will Deacon , Russell King , Ingo Molnar , gregkh@linuxfoundation.org, "H. Peter Anvin" , torvalds@linux-foundation.org, alan@linux.intel.com Date: Fri, 26 Jan 2018 23:55:24 -0800 Message-ID: <151703972396.26578.7326612698912543866.stgit@dwillia2-desk3.amr.corp.intel.com> In-Reply-To: <151703971300.26578.1185595719337719486.stgit@dwillia2-desk3.amr.corp.intel.com> References: <151703971300.26578.1185595719337719486.stgit@dwillia2-desk3.amr.corp.intel.com> User-Agent: StGit/0.17.1-9-g687f MIME-Version: 1.0 Subject: [kernel-hardening] [PATCH v5 02/12] array_idx: sanitize speculative array de-references X-Virus-Scanned: ClamAV using ClamSMTP 'array_idx' is proposed as a generic mechanism to mitigate against Spectre-variant-1 attacks, i.e. an attack that bypasses boundary checks via speculative execution). The 'array_idx' implementation is expected to be safe for current generation cpus across multiple architectures (ARM, x86). Based on an original implementation by Linus Torvalds, tweaked to remove speculative flows by Alexei Starovoitov, and tweaked again by Linus to introduce an x86 assembly implementation for the mask generation. Co-developed-by: Linus Torvalds Co-developed-by: Alexei Starovoitov Suggested-by: Cyril Novikov Cc: Russell King Cc: Peter Zijlstra Cc: Catalin Marinas Cc: Will Deacon Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: x86@kernel.org Signed-off-by: Dan Williams --- include/linux/nospec.h | 64 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 include/linux/nospec.h diff --git a/include/linux/nospec.h b/include/linux/nospec.h new file mode 100644 index 000000000000..f59f81889ba3 --- /dev/null +++ b/include/linux/nospec.h @@ -0,0 +1,64 @@ +// SPDX-License-Identifier: GPL-2.0 +// Copyright(c) 2018 Intel Corporation. All rights reserved. + +#ifndef __NOSPEC_H__ +#define __NOSPEC_H__ + +/* + * When idx is out of bounds (idx >= sz), the sign bit will be set. + * Extend the sign bit to all bits and invert, giving a result of zero + * for an out of bounds idx, or ~0UL if within bounds [0, sz). + */ +#ifndef array_idx_mask +static inline unsigned long array_idx_mask(unsigned long idx, unsigned long sz) +{ + /* + * Warn developers about inappropriate array_idx usage. + * + * Even if the cpu speculates past the WARN_ONCE branch, the + * sign bit of idx is taken into account when generating the + * mask. + * + * This warning is compiled out when the compiler can infer that + * idx and sz are less than LONG_MAX. + */ + if (WARN_ONCE(idx > LONG_MAX || sz > LONG_MAX, + "array_idx limited to range of [0, LONG_MAX]\n")) + return 0; + + /* + * Always calculate and emit the mask even if the compiler + * thinks the mask is not needed. The compiler does not take + * into account the value of idx under speculation. + */ + OPTIMIZER_HIDE_VAR(idx); + return ~(long)(idx | (sz - 1UL - idx)) >> (BITS_PER_LONG - 1); +} +#endif + +/* + * array_idx - sanitize an array index after a bounds check + * + * For a code sequence like: + * + * if (idx < sz) { + * idx = array_idx(idx, sz); + * val = array[idx]; + * } + * + * ...if the cpu speculates past the bounds check then array_idx() will + * clamp the index within the range of [0, sz). + */ +#define array_idx(idx, sz) \ +({ \ + typeof(idx) _i = (idx); \ + typeof(sz) _s = (sz); \ + unsigned long _mask = array_idx_mask(_i, _s); \ + \ + BUILD_BUG_ON(sizeof(_i) > sizeof(long)); \ + BUILD_BUG_ON(sizeof(_s) > sizeof(long)); \ + \ + _i &= _mask; \ + _i; \ +}) +#endif /* __NOSPEC_H__ */