From patchwork Fri Mar 9 14:46:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kees Cook X-Patchwork-Id: 10270721 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2E2B56016D for ; Fri, 9 Mar 2018 14:46:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2200429E3C for ; Fri, 9 Mar 2018 14:46:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 148C829E40; Fri, 9 Mar 2018 14:46:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.3 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.wl.linuxfoundation.org (Postfix) with SMTP id 0FD0A29E3E for ; Fri, 9 Mar 2018 14:46:31 +0000 (UTC) Received: (qmail 9236 invoked by uid 550); 9 Mar 2018 14:46:29 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Delivered-To: mailing list kernel-hardening@lists.openwall.com Received: (qmail 8170 invoked from network); 9 Mar 2018 14:46:28 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=date:from:to:cc:subject:message-id:mime-version:content-disposition; bh=q0VzOF52Dq6HyWLumCgzjCAydt0BRNKrK6W+LyfTT+A=; b=lO/zf0/auQMPer4Jrw0lKZY4dkNKLWtxESUIBBMedlPbs+71UHe6vt+B6egL1vYb+R bPJQladc9WOSm1ORUJxMOa6O4JhX7lAgnxWkxErnpgSBMO0kySfAy0nbG2yB5WffNwu5 HImKpUrCy5/776m1OCOyTugZlRKFRHsHkmY+U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:mime-version :content-disposition; bh=q0VzOF52Dq6HyWLumCgzjCAydt0BRNKrK6W+LyfTT+A=; b=MkwGyEvra6dkr5OEoZyW6mJ30NNHFfCmWTEZ5UcCxOAlHOyG3SLLVD3253PWlgQjjY 7UgeXNo4kfWsNoGDx3lkFzOoTh/+P8+QNSLKHRZqMJWCg+v1ze1C/jDO4Fc8USHcv25r YHC+ft/HVPhT5k1i2DUdFN40388XleJNbmJCUk5b6NKHqEkII7V4C6ZNo6sleghecg0P DKNZ1VKFQsBEuMxeopBBwA5UlZTSklpImNByYdo763ijM91jl/4RCzo02PNb9DSXUsqP TCbleFBCBvky77I3U78bS5RAfeFTVutS2I0RDELBJmtP5uXmO3tfZHcfKv8CJx1gQZRI w/Gg== X-Gm-Message-State: APf1xPADp0jzTIr1YOnGBJ8dKWmCUzgQiLY9uUp3HzA8XiNvVFIz41Lr IsDuOVv9ih6LBxcZE7MNznxgKw== X-Google-Smtp-Source: AG47ELuP0YLvD9ibb5CyIC+uZMSn4CFz4IMA21wTP/zwZ5ja51Yr0FhfINBc5xBmJ1UlfiMSNCuzWA== X-Received: by 10.101.75.70 with SMTP id k6mr25267136pgt.335.1520606776456; Fri, 09 Mar 2018 06:46:16 -0800 (PST) Date: Fri, 9 Mar 2018 06:46:13 -0800 From: Kees Cook To: Thomas Gleixner Cc: linux-kernel@vger.kernel.org, Segher Boessenkool , kernel-hardening@lists.openwall.com Subject: [PATCH][RFC] rslib: Remove VLAs by setting upper bound on nroots Message-ID: <20180309144613.GA48965@beast> MIME-Version: 1.0 Content-Disposition: inline X-Virus-Scanned: ClamAV using ClamSMTP Avoid VLAs[1] by always allocating the upper bound of stack space needed. The existing users of rslib appear to max out at 32 roots, so use that as the upper bound. Alternative: make init_rs() a true caller-instance and pre-allocate the workspaces. Will this need locking or are the callers already single-threaded in their use of librs? Using kmalloc in this path doesn't look great, especially since at least one caller (pstore) is sensitive to allocations during rslib usage (it expects to run it during an Oops, for example). [1] https://lkml.org/lkml/2018/3/7/621 Signed-off-by: Kees Cook --- lib/reed_solomon/decode_rs.c | 7 ++++--- lib/reed_solomon/reed_solomon.c | 5 ++++- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/lib/reed_solomon/decode_rs.c b/lib/reed_solomon/decode_rs.c index 0ec3f257ffdf..3e3becb836a6 100644 --- a/lib/reed_solomon/decode_rs.c +++ b/lib/reed_solomon/decode_rs.c @@ -31,9 +31,10 @@ * of nroots is 8. So the necessary stack size will be about * 220 bytes max. */ - uint16_t lambda[nroots + 1], syn[nroots]; - uint16_t b[nroots + 1], t[nroots + 1], omega[nroots + 1]; - uint16_t root[nroots], reg[nroots + 1], loc[nroots]; + uint16_t lambda[RS_MAX_ROOTS + 1], syn[RS_MAX_ROOTS]; + uint16_t b[RS_MAX_ROOTS + 1], t[RS_MAX_ROOTS + 1]; + uint16_t omega[RS_MAX_ROOTS + 1], root[RS_MAX_ROOTS]; + uint16_t reg[RS_MAX_ROOTS + 1], loc[RS_MAX_ROOTS]; int count = 0; uint16_t msk = (uint16_t) rs->nn; diff --git a/lib/reed_solomon/reed_solomon.c b/lib/reed_solomon/reed_solomon.c index 06d04cfa9339..1ad9094ddf66 100644 --- a/lib/reed_solomon/reed_solomon.c +++ b/lib/reed_solomon/reed_solomon.c @@ -51,6 +51,9 @@ static LIST_HEAD (rslist); /* Protection for the list */ static DEFINE_MUTEX(rslistlock); +/* Ultimately controls the upper bounds of the on-stack buffers. */ +#define RS_MAX_ROOTS 32 + /** * rs_init - Initialize a Reed-Solomon codec * @symsize: symbol size, bits (1-8) @@ -210,7 +213,7 @@ static struct rs_control *init_rs_internal(int symsize, int gfpoly, return NULL; if (prim <= 0 || prim >= (1<= (1<= (1< RS_MAX_ROOTS) return NULL; mutex_lock(&rslistlock);