From patchwork Thu Dec 5 07:30:24 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11274297 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6D9CF930 for ; Thu, 5 Dec 2019 07:31:14 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 52FF2224F8 for ; Thu, 5 Dec 2019 07:31:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 52FF2224F8 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iclaW-0006Vj-8U; Thu, 05 Dec 2019 07:30:12 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iclaV-0006Vd-30 for xen-devel@lists.xenproject.org; Thu, 05 Dec 2019 07:30:11 +0000 X-Inumbo-ID: 1125e7b2-1731-11ea-a0d2-bc764e2007e4 Received: from mx1.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 1125e7b2-1731-11ea-a0d2-bc764e2007e4; Thu, 05 Dec 2019 07:30:10 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 5EC69B247; Thu, 5 Dec 2019 07:30:09 +0000 (UTC) From: Jan Beulich To: "xen-devel@lists.xenproject.org" References: <521c23e1-9d89-8f26-572c-1b6f6bc3cbaa@suse.com> Message-ID: <4d783acc-35be-1b6d-00c7-f1ce88abdbcc@suse.com> Date: Thu, 5 Dec 2019 08:30:24 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.9.1 MIME-Version: 1.0 In-Reply-To: <521c23e1-9d89-8f26-572c-1b6f6bc3cbaa@suse.com> Content-Language: en-US Subject: [Xen-devel] [PATCH 1/3] lz4: refine commit 9143a6c55ef7 for the 64-bit case X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Stefano Stabellini , Julien Grall , Wei Liu , Konrad Wilk , George Dunlap , Andrew Cooper , Jeremi Piotrowski , Mark Pryor , Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" I clearly went too far there: While the LZ4_WILDCOPY() instances indeed need prior guarding, LZ4_SECURECOPY() needs this only in the 32-bit case (where it simply aliases LZ4_WILDCOPY()). "cpy" can validly point (slightly) below "op" in these cases, due to cpy = op + length - (STEPSIZE - 4); where length can be as low as 0 and STEPSIZE is 8. However, instead of removing the check via "#if !LZ4_ARCH64", refine it such that it would also properly work in the 64-bit case, aborting decompression instead of continuing on bogus input. Reported-by: Mark Pryor Reported-by: Jeremi Piotrowski Signed-off-by: Jan Beulich Tested-by: Mark Pryor --- a/xen/common/lz4/decompress.c +++ b/xen/common/lz4/decompress.c @@ -147,7 +147,7 @@ static int INIT lz4_uncompress(const uns goto _output_error; continue; } - if (unlikely((unsigned long)cpy < (unsigned long)op)) + if (unlikely((unsigned long)cpy < (unsigned long)op - (STEPSIZE - 4))) goto _output_error; LZ4_SECURECOPY(ref, op, cpy); op = cpy; /* correction */ @@ -279,7 +279,7 @@ static int lz4_uncompress_unknownoutputs goto _output_error; continue; } - if (unlikely((unsigned long)cpy < (unsigned long)op)) + if (unlikely((unsigned long)cpy < (unsigned long)op - (STEPSIZE - 4))) goto _output_error; LZ4_SECURECOPY(ref, op, cpy); op = cpy; /* correction */ From patchwork Thu Dec 5 07:30:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11274299 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 37DC6930 for ; Thu, 5 Dec 2019 07:31:45 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1E14B206DB for ; Thu, 5 Dec 2019 07:31:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1E14B206DB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iclb2-0006Yv-IB; Thu, 05 Dec 2019 07:30:44 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iclb0-0006Yo-U8 for xen-devel@lists.xenproject.org; Thu, 05 Dec 2019 07:30:42 +0000 X-Inumbo-ID: 23b76d1d-1731-11ea-8214-12813bfff9fa Received: from mx1.suse.de (unknown [195.135.220.15]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 23b76d1d-1731-11ea-8214-12813bfff9fa; Thu, 05 Dec 2019 07:30:42 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 77CAAB25B; Thu, 5 Dec 2019 07:30:41 +0000 (UTC) From: Jan Beulich To: "xen-devel@lists.xenproject.org" References: <521c23e1-9d89-8f26-572c-1b6f6bc3cbaa@suse.com> Message-ID: Date: Thu, 5 Dec 2019 08:30:56 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.9.1 MIME-Version: 1.0 In-Reply-To: <521c23e1-9d89-8f26-572c-1b6f6bc3cbaa@suse.com> Content-Language: en-US Subject: [Xen-devel] [PATCH 2/3] lz4: pull out constant tables X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Stefano Stabellini , Julien Grall , Wei Liu , Konrad Wilk , George Dunlap , Andrew Cooper , Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Rasmus Villemoes There's no reason to allocate the dec{32,64}table on the stack; it just wastes a bunch of instructions setting them up and, of course, also consumes quite a bit of stack. Using size_t for such small integers is a little excessive. $ scripts/bloat-o-meter /tmp/built-in.o lib/built-in.o add/remove: 2/2 grow/shrink: 2/0 up/down: 1304/-1548 (-244) function old new delta lz4_decompress_unknownoutputsize 55 718 +663 lz4_decompress 55 632 +577 dec64table - 32 +32 dec32table - 32 +32 lz4_uncompress 747 - -747 lz4_uncompress_unknownoutputsize 801 - -801 The now inlined lz4_uncompress functions used to have a stack footprint of 176 bytes (according to -fstack-usage); their inlinees have increased their stack use from 32 bytes to 48 and 80 bytes, respectively. Signed-off-by: Rasmus Villemoes [Linux commit bea2b592fd18eb8ffa3fc4ad380610632d03a38f] Use {,u}int8_t instead of plain "int" for the tables. Signed-off-by: Jan Beulich --- a/xen/common/lz4/decompress.c +++ b/xen/common/lz4/decompress.c @@ -39,6 +39,11 @@ #include "defs.h" +static const uint8_t dec32table[] = {0, 3, 2, 3, 0, 0, 0, 0}; +#if LZ4_ARCH64 +static const int8_t dec64table[] = {0, 0, 0, -1, 0, 1, 2, 3}; +#endif + #if defined(__XEN__) || defined(__MINIOS__) static int INIT lz4_uncompress(const unsigned char *source, unsigned char *dest, @@ -51,10 +56,6 @@ static int INIT lz4_uncompress(const uns BYTE *cpy; unsigned token; size_t length; - size_t dec32table[] = {0, 3, 2, 3, 0, 0, 0, 0}; -#if LZ4_ARCH64 - size_t dec64table[] = {0, 0, 0, -1, 0, 1, 2, 3}; -#endif while (1) { @@ -109,7 +110,7 @@ static int INIT lz4_uncompress(const uns /* copy repeated sequence */ if (unlikely((op - ref) < STEPSIZE)) { #if LZ4_ARCH64 - size_t dec64 = dec64table[op - ref]; + int dec64 = dec64table[op - ref]; #else const int dec64 = 0; #endif @@ -175,11 +176,6 @@ static int lz4_uncompress_unknownoutputs BYTE * const oend = op + maxoutputsize; BYTE *cpy; - size_t dec32table[] = {0, 3, 2, 3, 0, 0, 0, 0}; -#if LZ4_ARCH64 - size_t dec64table[] = {0, 0, 0, -1, 0, 1, 2, 3}; -#endif - /* Main Loop */ while (ip < iend) { @@ -245,7 +241,7 @@ static int lz4_uncompress_unknownoutputs /* copy repeated sequence */ if (unlikely((op - ref) < STEPSIZE)) { #if LZ4_ARCH64 - size_t dec64 = dec64table[op - ref]; + int dec64 = dec64table[op - ref]; #else const int dec64 = 0; #endif From patchwork Thu Dec 5 07:31:29 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jan Beulich X-Patchwork-Id: 11274301 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B34B138D for ; Thu, 5 Dec 2019 07:32:27 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5FCC8206DB for ; Thu, 5 Dec 2019 07:32:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5FCC8206DB Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iclbY-0006cl-RR; Thu, 05 Dec 2019 07:31:16 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1iclbY-0006cd-B3 for xen-devel@lists.xenproject.org; Thu, 05 Dec 2019 07:31:16 +0000 X-Inumbo-ID: 380cb900-1731-11ea-a0d2-bc764e2007e4 Received: from mx1.suse.de (unknown [195.135.220.15]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 380cb900-1731-11ea-a0d2-bc764e2007e4; Thu, 05 Dec 2019 07:31:15 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id ABC77B22C; Thu, 5 Dec 2019 07:31:14 +0000 (UTC) From: Jan Beulich To: "xen-devel@lists.xenproject.org" References: <521c23e1-9d89-8f26-572c-1b6f6bc3cbaa@suse.com> Message-ID: Date: Thu, 5 Dec 2019 08:31:29 +0100 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.9.1 MIME-Version: 1.0 In-Reply-To: <521c23e1-9d89-8f26-572c-1b6f6bc3cbaa@suse.com> Content-Language: en-US Subject: [Xen-devel] [PATCH 3/3] lz4: fix system halt at boot kernel on x86_64 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Stefano Stabellini , Julien Grall , Wei Liu , Konrad Wilk , George Dunlap , Andrew Cooper , Ian Jackson Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" From: Krzysztof Kolasa Sometimes, on x86_64, decompression fails with the following error: Decompressing Linux... Decoding failed -- System halted This condition is not needed for a 64bit kernel(from commit d5e7caf): if( ... || (op + COPYLENGTH) > oend) goto _output_error macro LZ4_SECURE_COPY() tests op and does not copy any data when op exceeds the value. added by analogy to lz4_uncompress_unknownoutputsize(...) Signed-off-by: Krzysztof Kolasa [Linux commit 99b7e93c95c78952724a9783de6c78def8fbfc3f] The offending commit in our case is fcc17f96c277 ("LZ4 : fix the data abort issue"). Signed-off-by: Jan Beulich --- a/xen/common/lz4/decompress.c +++ b/xen/common/lz4/decompress.c @@ -133,8 +133,12 @@ static int INIT lz4_uncompress(const uns /* Error: request to write beyond destination buffer */ if (cpy > oend) goto _output_error; +#if LZ4_ARCH64 + if ((ref + COPYLENGTH) > oend) +#else if ((ref + COPYLENGTH) > oend || (op + COPYLENGTH) > oend) +#endif goto _output_error; LZ4_SECURECOPY(ref, op, (oend - COPYLENGTH)); while (op < cpy) @@ -262,7 +266,13 @@ static int lz4_uncompress_unknownoutputs if (cpy > oend - COPYLENGTH) { if (cpy > oend) goto _output_error; /* write outside of buf */ - +#if LZ4_ARCH64 + if ((ref + COPYLENGTH) > oend) +#else + if ((ref + COPYLENGTH) > oend || + (op + COPYLENGTH) > oend) +#endif + goto _output_error; LZ4_SECURECOPY(ref, op, (oend - COPYLENGTH)); while (op < cpy) *op++ = *ref++;